ZFS Dedupliciation

For deduplication, it is recommended to have L2ARC cache size of 2-5GB per 1TB of disk.

Enable deduplication:

zfs set dedup=on tank/data

Once all deduped datasets are destroyed the dedup table will be removed and the performance impact is cleared.

List dedup flags from all volumes:

zfs get dedup | egrep '(on|off)'

zpool status -D rpool

dedup: DDT entries 3755572, size 501B on disk, 161B in core

bucket              allocated                       referenced          
______   ______________________________   ______________________________
refcnt   blocks   LSIZE   PSIZE   DSIZE   blocks   LSIZE   PSIZE   DSIZE
------   ------   -----   -----   -----   ------   -----   -----   -----
     1    3,06M    341G    324G    325G    3,06M    341G    324G    325G
     2     428K   44,2G   41,9G   42,1G     963K   98,7G   93,7G   94,0G
     4    84,9K   6,34G   5,40G   5,46G     384K   27,0G   22,7G   22,9G
     8    16,7K    245M    107M    137M     173K   2,52G   1,09G   1,40G
    16    7,61K    144M   79,0M   88,9M     146K   2,79G   1,51G   1,69G
    32      564   8,93M   3,35M   4,02M    22,8K    382M    146M    174M
    64      110   1,32M    588K    776K    9,46K    113M   51,6M   67,3M
   128       52   1,07M    670K    748K    8,78K    183M    114M    127M
   256       37    947K    576K    652K    13,7K    389M    239M    266M
   512        4   10,5K   10,5K     16K    3,17K   8,15M   8,15M   12,7M
    1K        6   43,5K     15K     28K    8,34K   60,6M   21,6M   38,8M
    2K        1   36,5K      8K      8K    2,08K   75,9M   16,6M   16,6M
 Total    3,58M    392G    372G    373G    4,75M    473G    444G    446G

Where DDT table memory usage can be calculated:

• echo '3755572 * 161 /1024/1024' | bc is 576 MB used memory
• echo '3755572 * 501 /1024/1024' | bc is 1794 MB used on disk

zdb -DD rpool