I have two users who are identical in roles, profiles, and group memberships.  One can see a particular contact; the other cannot.

We are set up so that Accounts have default access of "Private" and Contacts have default access of "Controlled by Parent".  Both have "Grant access using hierarchies" checked.  There are no sharing rules that apply to the contact in question.  We don't use sales teams or territories.  The contact in question does have an account.

I know what causes the difference, but I don't know why.  The user who can see the contact is the owner of a case where that contact appears in the "contact name" field.  If I change the ownership of the case, the user no longer sees that contact.  

I thought I understood how record ownership and sharing work in Salesforce, but this one has me puzzled.  I can't find any explanation for this in the Salesforce documentation I've looked at.  Can anyone tell me what is happening here?