meta data for this page
  •  

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
vm:proxmox:ceph:performance [2024/05/17 19:05] – niziakvm:proxmox:ceph:performance [2025/01/08 19:05] (current) – niziak
Line 23: Line 23:
   * same number of primary PG per OSD = read operations spread evenly   * same number of primary PG per OSD = read operations spread evenly
     * primary PG - original/first PG - others are replicas. Primary PG is used for read.     * primary PG - original/first PG - others are replicas. Primary PG is used for read.
 +  * use relatively more PG than for big cluster - better balance, but handling PGs consumes resources (RAM)
 +    * i.e. for 7 OSD x 2TB PG autoscaler recommends 256 PG. After changing to 384 IOops drastivally increases and latency drops.
 +      Setting to 512 PG wasn't possible because limit of 250PG/OSD.
  
 === balancer === === balancer ===
Line 41: Line 44:
 === PG autoscaler === === PG autoscaler ===
  
 +Better to use in warn mode, to do not put unexpected load when PG number will change.
 <code bash> <code bash>
 ceph mgr module enable pg_autoscaler ceph mgr module enable pg_autoscaler
Line 46: Line 50:
 ceph osd pool set rbd pg_autoscale_mode warn ceph osd pool set rbd pg_autoscale_mode warn
 </code> </code>
 +
 +It is possible to set desired/target size of pool. This prevents autoscaler to move data every time new data are stored.
 +
 +==== check cluster balance ====
 +
 +<code bash>
 +ceph -s
 +ceph osd df # shows standard deviation
 +</code>
 +
 +no tools to show primary PG balancing. Tool on https://github.com/JoshSalomon/Cephalocon-2019/blob/master/pool_pgs_osd.sh