meta data for this page
  •  

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
vm:proxmox:ceph:issues [2022/04/22 07:52] – created niziakvm:proxmox:ceph:issues [2024/05/17 09:21] (current) – niziak
Line 1: Line 1:
 ====== Issues ====== ====== Issues ======
 +
 +=== auth: unable to find a keyring ===
 +
 +It is not possible to create ceph OSD neither from WebUI nor cmdline: <code bash>pveceph osd create /dev/sdc</code>
 +
 +<code>
 +Running command: /bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-2/activate.monmap
 + stderr: 2021-01-28T10:21:24.996+0100 7fd1a848f700 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.bootstrap-osd.keyring: (2) No such file or directory
 +2021-01-28T10:21:24.996+0100 7fd1a848f700 -1 AuthRegistry(0x7fd1a0059030) no keyring found at /etc/pve/priv/ceph.client.bootstrap-osd.keyring, disabling cephx
 +</code>
 +
 +<file init /etc/pve/ceph.conf>
 +[client]
 +         keyring = /etc/pve/priv/$cluster.$name.keyring
 +
 +[mds]
 +         keyring = /var/lib/ceph/mds/ceph-$id/keyring
 +</file>
 +
 +ceph.conf Variables
 +  * **$cluster** - cluster name. For proxmox it is ''ceph''
 +  * **$type** - daemon process ''mds'' ''osd'' ''mon''
 +  * **$id** - daemon or client indentifier. For ''osd.0'' it is ''0''
 +  * **$host** - hostname where the process is running
 +  * **$name** - Expands to $type.$id. I.e: ''osd.2'' or ''client.bootstrap''
 +  * **$pid** - Expands to daemon pid
 +
 +**SOLUTION:**
 +<code bash>cp /var/lib/ceph/bootstrap-osd/ceph.keyring /etc/pve/priv/ceph.client.bootstrap-osd.keyring</code>
 +alternative to try: change ceph.conf
 +
 +=== Unit -.mount is masked. ===
 +
 +<code>
 +Running command: /usr/bin/systemctl start ceph-osd@2
 + stderr: Failed to start ceph-osd@2.service: Unit -.mount is masked.
 +-->  RuntimeError: command returned non-zero exit status: 1
 +</code>
 +
 +It was caused by ''gparted'' which wasn't correctly shutdown.
 +  * [[https://askubuntu.com/questions/1191596/unit-mount-is-masked|Unit -.mount is masked]]
 +  * [[https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=948739|gparted should not mask .mount units]]
 +  * [[https://unix.stackexchange.com/questions/533933/systemd-cant-unmask-root-mount-mount/548996]]
 +
 +**Solution:**
 +
 +<code bash>systemctl --runtime unmask -- -.mount</code>
 +
 +To list runtime masked units:
 +<code bash>ls -l /var/run/systemd/system | grep mount | grep '/dev/null' | cut -d ' ' -f 11</code>
 +
 +To unescape systemd unit names: 
 +<code bash> systemd-escape -u 'rpool-data-basevol\x2d800\x2ddisk\x2d0.mount'</code>
 +
 +
 +
 +===== lock on rbd =====
 +
 +<code bash>
 +root@pve1:~# rbd lock ls vm-201-disk-0
 +
 +There is 1 exclusive lock on this image.
 +Locker            ID                         Address                    
 +client.310904979  auto 18446462598732841336  192.168.28.237:0/4057457529
 +
 +root@pve1:~# rbd lock remove vm-201-disk-0 "auto 18446462598732841336" client.310904979
 +</code>
  
 ===== mons are allowing insecure global_id reclaim ===== ===== mons are allowing insecure global_id reclaim =====