As Salesforce suggested These guest user site profiles are associated to guest user licenses which give the ability to only read/create on standard objects and not update.
Is there any workaround to update Contact standard object from Salesforce Site?