My site guest user needs access for all the contacts in the instance. Right now site guest users can access contacts created by guest user. They are not able to access contacts created by Salesforce users. OWD setting for Contact is Public Read/Write. I don't want to change the OWD at this moment.
It seems that you imply Default Internal Access OWD when you mention OWD setting for Contact is Public Read/Write. The Guest User is by definition an external type of user. So what you need basically is: (1) Enable External Sharing Model; (2) set Default External Access for Contact as Public Read/Write.
Refer to this official documentation External Organization-Wide Defaults Overview for more detail.