I need to figure out if values in PermissionSet for current user are changed in trigger, i'm thinking about using oldMap but i can't figure it out.

Maybe this code will explain what i'm trying to do:

    for (User u : trigger.New)
    {
        List<PermissionSetAssignment> lstOldPermSet = Trigger.oldMap.get(u.id).PermissionSetAssignments;
        List<PermissionSetAssignment> lstNewPermSet = // get PermissionSetAssignments via soql query

        if(lstOldPermSet != lstNewPermSet)
        {

// code goes in here

        }

    }

Any suggestions?