I assume that modal is a child component.
You need to send component event from child to parent and in parent event handler close the modal.
In Parent
<aura:handler name="myComponentEvent" event="c:componentEvent" action="{!c.handleMyComponentEvent}"/>
Parent controller
({
handleMyComponentEvent : function(component, event, helper) {
component.find("overlayLib").notifyClose();
}
})
You can optionally stop propagation of the event here using
event.stopPropagation();
if you do not want your event to propagate further
In child
<aura:registerEvent name="myComponentEvent" type="c:componentEvent"/>
handleUploadFinished: function (component, event) {
var uploadedFiles = event.getParam("files");
console.log(uploadedFiles[0].documentId); // See? it works.
var myEvent = component.getEvent("myComponentEvent");
myEvent.fire();
}
And event itself
<aura:event type="COMPONENT">
</aura:event>
Obviously with desired event name, in example
componentEvent