I know that I can delete Contacts using the REST API. I've just tested this and this works. This is a two step process.
Firstly you need to retrieve the Adapter objects, then you can use these objects to delete Contacts from All Contacts or MobileConnect. Refer to the following documentation:
Step 1. Get Delete Options
Step 2. Delete Contact(s)
I have another question..is there a way to update ContactKey using REST API?