I don't think there's a way to do this with SOQL. I might be wrong but if you're not hitting issues with the number of records you could just let it grab as many as there are and just use the first result found for each opportunity ID.
You could do multiple queries, one for each opportunity, but then you're at risk of hitting query limits.
A better solution for this might be to use a trigger or similar to always have one OpportunityTeamMember flagged as the primary member for a given opportunity (you'd have to enforce the restriction yourself in code) and then you could just filter the query by that as well:
Select UserId From OpportunityTeamMember Where OpportunityId In :opptyIdSet and IsPrimary__c = true