- 1 Remove/Delete Ceph
- Warning: Removing/Deleting ceph will remove/delete all data stored on ceph as well!
- 1.1 Login to Proxmox Web GUI
- 1.2 Click on one of the PVE nodes
- 1.3 From right hand side panel, Navigate to Ceph -> Pools record items under Name
- 1.4 Navigate to Ceph -> CephFS, record existing cephFS names
- 1.5 From left hand side menu, Click on Datacenter
- 1.6 From right hand side, Click on Storage
- 1.7 Delete all items which we saw under CephFS and Pools from step 3 and step 4
- 1.8 From right hand side panel, Click on master node, Navigate to Ceph -> CephFS, Stop and Destroy all Metadata Services
- 1.9 Click on master node, from right hand side panel, Navigate to Ceph -> OSD, Mark all OSDs as Out
- 1.10 From right hand side menu, Right Click on one of the PVE nodes’ name, Click on >_ Shell to open terminal
- 1.11 Mark OSD down
ceph osd down 0 && ceph osd destroy 0 --force ceph osd down 1 && ceph osd destroy 1 --force ceph osd down 2 && ceph osd destroy 2 --force
1.13 Remove ceph configuration file by executing the following command from terminal (Refer to step 10)
rm /etc/ceph/ceph.conf
1.14 On each of the PVE node, execute the following command to stop ceph monitor service
systemctl stop ceph-mon@<hostname or monid>
# e.g.
systemctl stop ceph-mon@labnode1
1.16 Remove ceph configuration file from all nodes
rm -r /etc/pve/ceph.conf rm -r /etc/ceph rm -rf /var/lib/ceph
If we get the following error or similar
rm: cannot remove '/var/lib/ceph/osd/ceph-0': Device or resource busyWe can use this command to unmonut first then, try to remove it again
umount /var/lib/ceph/osd/ceph-0 rm -r /var/lib/ceph
1.17 Reboot all PVE nodes
1.18 Clear leftover ceph configuration files and services, execute the following command on each nodes
pveceph purge
1.20 Clear the OSD disks from each OSD nodes, so that we can use those disks later
# Remove the lvm signature from the disk # Note: Change the drive letter (sdx) accordingly fdisk /dev/sdx
Then Enter d, Press Enter key, Enter w, Press Enter key
rm -r /dev/ceph-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx
Note: To find the correct value, we can use this command “ls /dev | grep ceph” or just type “rm -r /dev/ceph-” then Press Tab key to auto complete the rest
# Restart the PVE host reboot
1.21 Remove all ceph packages if desired
apt purge ceph-mon ceph-osd ceph-mgr ceph-mds
More to remove if desired
apt purge ceph-base ceph-mgr-modules-core