Although this question is already really old here is what I found out today by chance:
A user needs to have the Manage External Users permissions and must be included in a sharing group of the partner user's manager role.
If you navigate to a partner community user and click on the role, you see what role it reports to and in what sharing group the role is. I think per default this is "Role" & "Role, Role, Internal and Portal Subordinates". This means the user you'd like to have access to has to be in the reporting role or in a subordinate role of the reporting role
As you can see, the partner role, reports to the role "Geschäftsführung" and is shared within "Geschäftsführung". Due to our role setup, which has "Kundenbetreuung" as a direct subordinate role to "Geschäftsführung", all users in "Kundenbetreuung" which have "Manage External Users" can access that functionality.
Sub roles of a direct subordinate role of the role a partner role reports to do not have access. I had to learn that the hard way ("child" role to "Kundenbetreuung" in my example).