We can convert RDM disks while they are online or offline. If you power off your VM, you can convert RDM disks by migrating them to another Datastore. When migrating you should enable "Configure Per Disk" option and select of the the Disk Format. It does not matter whether you select Thin,Thick Lazy Zeroed or Thick Eager Zeroed. The disadvantage of this method is that your vm stays powered of during the migration and downtime will be long depending on your disk size.
For a shorter downtime you can follow the procedure below:
Shutdown your VM. Right click on the VM and select Edit Settings.
Select delete files from datastore (Do not worry, it only deletes RDM mapping file. Actual data will not be deleted. If we dont select "delete files from datastore" option, this disk can not be added to the vm later on. We have to make sure mapping file is deleted.)
Make sure you take a note of current scsi controller ID for RDM disks. Click OK
Right click on the VM and select Edit Settings again. Add New Device > Choose RDM Disk
Select the RDM disk we just removed.
Select "Compatibility Mode" as Virtual
Select "Disk Mode" as Dependent
Set "Virtual Device Node" by using the same SCSI id that was set earlier. (SCSI 0:1 for this example)
Power On the Virtual Machine
Right click on VM > Migarate > Change Storage Only > Configure Per Disk > Change Disk Format to Thin or Thick> Complete the wizard.