====== swapping when pleny of free of ram ======

[[OpenZFS #18893|https://github.com/openzfs/zfs/issues/18893]]

Host swapping burst observed when ZFS ARC cache grows.

Scenario:
  * ''spl_cache_grow_work'' → ''__vmalloc'' → ''alloc_pages'', and it asks for an order-10
    allocation, a 4 MiB contiguous block, with ''GFP_ATOMIC''. That is ZFS (the SPL layer) growing one of its slab caches.
  * Why it swaps with 75 GB free
    * The free memory is in small pieces. Normal has only 8 free blocks of order 10 (from /proc/buddyinfo), so kswapd is woken to build one.
    * To make a contiguous 4 MiB block it scans the LRU and swaps out cold VM pages. It also runs the ZFS shrinker,
      which is why the ARC keeps shrinking (8–9 GB, down from about 17 GB).
  * This also explains why lowering ''swappiness'' never stopped it, since this reclaim is driven by the failed contiguous allocation,
    not by general memory pressure.
 
Workarounds:
  * Raise ''vm.compaction_proactiveness'' from 20 to about 60, so ''kcompactd'' keeps ''order-10'' blocks available.
    * can make things worst when on Proxmox (CPU spikes due to ''kcompactd'' heavy work)
    * can cause swapping loops in low mem conditions (''kcompact'' wants to create continous 4MB block will throw to swap lots of small pages)
  * Protect the ARC. Set ''zfs_arc_min'' (about 16 GB). It stops the kswapd runs from trimming it, but doesn't stop the swapping.
  * ''zswap'': enabling it on more hosts makes these bursts cheap, since the swap-outs would be compressed in RAM.