Placement Groups

PG

• https://old.ceph.com/pgcalc/
• Max PG per OSD is 300
• Result must be rounded up to the nearest power of 2.

Adjust reweight according to current OSD utilisation - to prevent filling one OSD

ceph osd reweight-by-utilization

In CRUSH hierarchies with a smaller number of OSDs, it’s possible for some OSDs to get more PGs than other OSDs, resulting in a higher load. You can reweight OSDs by PG distribution to address this situation by executing the following:

osd reweight-by-pg [<oload:int>] [<max_change:float>] [<max_osds:int>] [<pools>...]            reweight OSDs by PG distribution [overload-percentage-for-consideration, default 120]

ceph osd reweight-by-pg 101 0.05 999

NOTE:

• 101 is a oload parameter, which means overload percentage for consideration 101%. For example, an argument of 200 would reweight OSDs which are twice as utilized as the average OSD.
• 0.05 is a mach_change - maximum modification of weight
• max_osds - maximum number of OSD to be modifies (default is 4)

When the cluster is healthy, the balancer will incrementally move a small fraction of unbalanced PGs in order to improve distribution. This fraction will not exceed a certain threshold that defaults to 5%. Set it to 1%:

ceph config set mgr target_max_misplaced_ratio .01