http://192.168.180.206:8001/ 2026-05-15T12:00:03+00:00 http://192.168.180.206:8001/ http://192.168.180.206:8001/_media/wiki:dokuwiki.svg text/html 2025-10-29T09:47:07+00:00 Anonymous (anonymous@undisclosed.example.com) http://192.168.180.206:8001/vm:proxmox:ceph:compression?rev=1761731227&do=diff CEPH inline compression <https://docs.ceph.com/en/latest/rados/configuration/bluestore-config-ref/#inline-compression> BlueStore compression performance get compression ratio ceph df detail --- POOLS --- POOL ID PGS STORED (DATA) (OMAP) OBJECTS USED (DATA) (OMAP) %USED MAX AVAIL QUOTA OBJECTS QUOTA BYTES DIRTY USED COMPR UNDER COMPR rbd 1 512 2.3 TiB 2.3 TiB 15 KiB 638.96k 6.4 TiB 6.4 TiB 44 KiB 59.17 1.5 TiB N/A … text/html 2023-05-31T07:23:00+00:00 Anonymous (anonymous@undisclosed.example.com) http://192.168.180.206:8001/vm:proxmox:ceph:db?rev=1685517780&do=diff DB block.db and block.wal The DB stores BlueStore’s internal metadata and the WAL is BlueStore’s internal journal or write-ahead log. It is recommended to use a fast SSD or NVRAM for better performance.Important Since Ceph has to write all data to the journal (or WAL+DB) before it can ACK writes, having this metadata and OSD performance in balance is really important! text/html 2025-10-31T16:52:36+00:00 Anonymous (anonymous@undisclosed.example.com) http://192.168.180.206:8001/vm:proxmox:ceph:guest_performance?rev=1761929556&do=diff Guest performance Tips Guests: * writeback cache - it groups sequential small writes into one big pack, so CEPH will handle it as one transaction The best results for Windows guest: * HDD (no SSD) * probably CEPH better handles bunch of ordered requests (like for rotational drives) text/html 2024-05-17T07:21:40+00:00 Anonymous (anonymous@undisclosed.example.com) http://192.168.180.206:8001/vm:proxmox:ceph:issues?rev=1715930500&do=diff Issues auth: unable to find a keyring It is not possible to create ceph OSD neither from WebUI nor cmdline: pveceph osd create /dev/sdc Running command: /bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-2/activate.monmap stderr: 2021-01-28T10:21:24.996+0100 7fd1a848f700 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.bootstrap-osd.keyring: (2) No such file or directory 2021-01-28T10:2… text/html 2022-07-25T07:41:58+00:00 Anonymous (anonymous@undisclosed.example.com) http://192.168.180.206:8001/vm:proxmox:ceph:move_disks?rev=1658734918&do=diff Move OSDs disks between hosts reinstall PVE After reinstallation of PVE, CEPH automatically detects OSD but keeps it as down. To run OSD it is need to: ceph-volume lvm activate --all 1st info If you used ceph-deploy and/or ceph-disk to set up these OSDs (that is, if they are stored on labeled GPT partitions such that upstart is automagically starting up the ceph-osd daemons for you without you putting anythign in /etc/fstab to manually mount the volumes) then all of this should be plug … text/html 2021-12-01T07:15:40+00:00 Anonymous (anonymous@undisclosed.example.com) http://192.168.180.206:8001/vm:proxmox:ceph:osd_creation?rev=1638342940&do=diff Header Proxmox Virtual Environment 7.1-4 Node 'pve2' No OSD selected Logs () create OSD on /dev/sdc (bluestore) creating block.db on '/dev/nvme0n1p3' Physical volume "/dev/nvme0n1p3" successfully created. Volume group "ceph-50bbae14-dbdb-48c0-9fa9-8a50cb991285" successfully created Rounding up size to full physical extent 3.99 GiB Logical volume "osd-db-7d84ca53-a110-44ba-93f2-f06108526ed9" created. Warning: The kernel is still using the old partition table. The new table will be used at… text/html 2021-04-08T09:44:48+00:00 Anonymous (anonymous@undisclosed.example.com) http://192.168.180.206:8001/vm:proxmox:ceph:performance_monitoring?rev=1617875088&do=diff CEPH performance monitoring basic info ceph ceph -s ceph -w ceph df ceph osd tree ceph osd df tree rados rados df Where: * USED COMPR: amount of space allocated for compressed data (i.e. this includes comrpessed data plus all the allocation, replication and erasure coding overhead). text/html 2026-04-01T15:31:19+00:00 Anonymous (anonymous@undisclosed.example.com) http://192.168.180.206:8001/vm:proxmox:ceph:performance?rev=1775057479&do=diff CEPH performance * BlueStore Config Reference: Sizing * <https://yourcmc.ru/wiki/Ceph_performance> * Ceph Performance Tuning Checklist * New to Ceph, HDD pool is extremely slow * Ceph Storage Performance * Ceph: A Journey to 1 TiB/s * <https://www.boniface.me/posts/pvc-ceph-tuning-adventures/> Performance tips Ceph is build for scale and works great in large clusters. In small cluster every node will be heavily loaded. * adapt PG to number of OSDs to spread traffic evenly * … text/html 2025-10-31T16:51:10+00:00 Anonymous (anonymous@undisclosed.example.com) http://192.168.180.206:8001/vm:proxmox:ceph:pg?rev=1761929470&do=diff Placement Groups PG Calculations * <https://old.ceph.com/pgcalc/> * Max PG per OSD is 300 * Result must be rounded up to the nearest power of 2. reweight Adjust reweight according to current OSD utilisation - to prevent filling one OSD ceph osd reweight-by-utilization text/html 2023-07-28T09:22:44+00:00 Anonymous (anonymous@undisclosed.example.com) http://192.168.180.206:8001/vm:proxmox:ceph:rebalance_recovery?rev=1690536164&do=diff Rebalance / recovery speed Form <https://www.suse.com/support/kb/doc/?id=000019693>: * osd max backfills This is the maximum number of backfill operations allowed to/from OSD. The higher the number, the quicker the recovery, which might impact overall cluster performance until recovery finishes. text/html 2021-01-11T19:47:36+00:00 Anonymous (anonymous@undisclosed.example.com) http://192.168.180.206:8001/vm:proxmox:ceph:remove_node?rev=1610394456&do=diff Remove failed node pvecm status Highest expected: 4 pvecm delnode <hostname> Killing node 2 pvecm status Highest expected: 3 And reload WebUI to refresh cluster nodes. Remove CEPH monitor ceph mon remove <host> Remove CEPH OSD ceph osd out osd.2 ceph osd crush remove osd.2 ceph auth del osd.2 ceph osd rm osd.2 ceph osd crush rm <host> text/html 2021-05-19T18:54:35+00:00 Anonymous (anonymous@undisclosed.example.com) http://192.168.180.206:8001/vm:proxmox:ceph:replace_node?rev=1621450475&do=diff Replace node New Proxmox node under the same hostname pve3 was installed. * Remove old monitor ceph mon remove pve3 * On pve3 node: * Remove mon.pve3 entry from /etc/pve/ceph.conf * Remove mon_host = X.X.X.X in section global * From GUI