meta data for this page
  •  

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
vm:proxmox:ceph:performance [2024/05/17 19:01] – niziakvm:proxmox:ceph:performance [2025/01/08 19:05] (current) – niziak
Line 23: Line 23:
   * same number of primary PG per OSD = read operations spread evenly   * same number of primary PG per OSD = read operations spread evenly
     * primary PG - original/first PG - others are replicas. Primary PG is used for read.     * primary PG - original/first PG - others are replicas. Primary PG is used for read.
 +  * use relatively more PG than for big cluster - better balance, but handling PGs consumes resources (RAM)
 +    * i.e. for 7 OSD x 2TB PG autoscaler recommends 256 PG. After changing to 384 IOops drastivally increases and latency drops.
 +      Setting to 512 PG wasn't possible because limit of 250PG/OSD.
  
 === balancer === === balancer ===
Line 39: Line 42:
  
  
 +=== PG autoscaler ===
  
 +Better to use in warn mode, to do not put unexpected load when PG number will change.
 +<code bash>
 +ceph mgr module enable pg_autoscaler
 +#ceph osd pool set <pool> pg_autoscale_mode <mode>
 +ceph osd pool set rbd pg_autoscale_mode warn
 +</code>
 +
 +It is possible to set desired/target size of pool. This prevents autoscaler to move data every time new data are stored.
 +
 +==== check cluster balance ====
 +
 +<code bash>
 +ceph -s
 +ceph osd df # shows standard deviation
 +</code>
 +
 +no tools to show primary PG balancing. Tool on https://github.com/JoshSalomon/Cephalocon-2019/blob/master/pool_pgs_osd.sh