You need to write a sharing rule to open up the access rights. It can be criteria or ownership based as per your use case.
Provided that, the user's profile/permission set must have Edit access to Contact object.
Refer Define Sharing Rules
Here is a way to open up visibility & access rights