Since you didn't specify an access modifier for the inner class, the default one (private) has been used, as stated in the documentation:
The private access modifier declares that this class is only known locally, that is, only by this section of code. This is the default access for inner classes—that is, if you don't specify an access modifier for an inner ...
Relationship fields are not automatically populated in a trigger context. This means that:
if(SE.Activity__r.Id != null)
Will always be false. You need to check the lookup field directly:
if(SE.Activity__c != null)
Note that your code will also fail in some situations if too many records share one parent. You should use an AggregateResult query to avoid ...