According to your requirement, you have to define OWD of the object to "Private". Then create role hierarchy or public groups to open up to access (either Read-Only or Read-Write) upon creating sharing rules to other users whom you want to share the records.
Along with the record level sharing, the object should have at-least Read access to the profile or in permission sets.
There is a nice trailhead Control Access to Records which will help you to clear the doubts on record level sharing.