Nodes upgraded to Proxmox 9.2.1 and CEPH to 20.2.1
After unexpected power failure whole cluster was restarted. Ceph managers cannot start:
-- Boot a09ae7c1598944da8df13189610e4366 -- lip 05 11:45:34 pve3 systemd[1]: Started ceph-mgr@pve3.service - Ceph cluster manager daemon. lip 05 11:46:01 pve3 ceph-mgr[2425]: 2026-07-05T11:46:01.068+0200 7aa4a5f836c0 -1 client.0 error registering admin socket command: (17) File exists lip 05 11:46:01 pve3 ceph-mgr[2425]: 2026-07-05T11:46:01.068+0200 7aa4a879c6c0 -1 client.0 error registering admin socket command: (17) File exists lip 05 11:46:01 pve3 ceph-mgr[2425]: 2026-07-05T11:46:01.068+0200 7aa4a879c6c0 -1 client.0 error registering admin socket command: (17) File exists lip 05 11:46:01 pve3 ceph-mgr[2425]: 2026-07-05T11:46:01.068+0200 7aa4a879c6c0 -1 client.0 error registering admin socket command: (17) File exists lip 05 11:46:01 pve3 ceph-mgr[2425]: 2026-07-05T11:46:01.068+0200 7aa4a879c6c0 -1 client.0 error registering admin socket command: (17) File exists lip 05 11:46:01 pve3 ceph-mgr[2425]: 2026-07-05T11:46:01.068+0200 7aa4a879c6c0 -1 client.0 error registering admin socket command: (17) File exists lip 05 11:47:15 pve3 ceph-mgr[2425]: ./src/mgr/PyFormatter.h: In function 'virtual void PyFormatter::close_section()' thread 7aa4ae7b86c0 time 2026-07-05T11:47:15.135657+0200 lip 05 11:47:15 pve3 ceph-mgr[2425]: ./src/mgr/PyFormatter.h: 84: FAILED ceph_assert(cursor != root) lip 05 11:47:15 pve3 ceph-mgr[2425]: ceph version 20.2.1 (1846e8e84cd244e621f1395ea824e304691b5a58) tentacle (stable - None) lip 05 11:47:15 pve3 ceph-mgr[2425]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7aa4dab55aa6] lip 05 11:47:15 pve3 ceph-mgr[2425]: 2: (PyFormatter::close_section()+0x90) [0x617cf58d8fc0] lip 05 11:47:15 pve3 ceph-mgr[2425]: 3: (ActivePyModules::get_perf_schema_python(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::ch> lip 05 11:47:15 pve3 ceph-mgr[2425]: 4: /usr/bin/ceph-mgr(+0x18aff4) [0x617cf58f2ff4] lip 05 11:47:15 pve3 ceph-mgr[2425]: 5: /lib/x86_64-linux-gnu/libpython3.13.so.1.0(+0x11a75b) [0x7aa4db8e875b] lip 05 11:47:15 pve3 ceph-mgr[2425]: 6: PyObject_Vectorcall() lip 05 11:47:15 pve3 ceph-mgr[2425]: 7: _PyEval_EvalFrameDefault() lip 05 11:47:15 pve3 ceph-mgr[2425]: 8: /lib/x86_64-linux-gnu/libpython3.13.so.1.0(+0x110af3) [0x7aa4db8deaf3] lip 05 11:47:15 pve3 ceph-mgr[2425]: 9: /lib/x86_64-linux-gnu/libpython3.13.so.1.0(+0x10f769) [0x7aa4db8dd769] lip 05 11:47:15 pve3 ceph-mgr[2425]: 10: PyObject_CallMethod() lip 05 11:47:15 pve3 ceph-mgr[2425]: 11: (PyModuleRunner::serve()+0x66) [0x617cf59c2d46] lip 05 11:47:15 pve3 ceph-mgr[2425]: 12: (PyModuleRunner::PyModuleRunnerThread::entry()+0x130) [0x617cf59c34c0] lip 05 11:47:15 pve3 ceph-mgr[2425]: 13: /lib/x86_64-linux-gnu/libc.so.6(+0x92b7b) [0x7aa4da4d4b7b] lip 05 11:47:15 pve3 ceph-mgr[2425]: 14: /lib/x86_64-linux-gnu/libc.so.6(+0x1107f8) [0x7aa4da5527f8]
Restart helps:
systemctl restart ceph-mgr@pve3