You could create a formula field on the object that holds the `Owner.UserRole.Name` then use that in your validation rule like:

    AND(
       ISCHANGED(OwnerId),
       PRIORVALUE(Owner_Role__c) <> $UserRole.Name
    )  
Which would only allow users to change the owner if the existing owner has the same Role as the running user.