Problem: I have a requirement which says, need to open a modal window which opens on click of related list button and this modal needs to be a lightning component,As we cant open a lightning component from related list button I achieved this by embedding this lightning component in a Visualforce page and then including this page in the related list button and then selected the option to open in existing window, so right now what i am facing is that the modal component isn't opening as a popup rather than its opening in a separate tab within the service console.
Possible solution: I skipped the popup part and added it as a lightning quick action and it works fine.
What suggestion i am looking for: Is there a way to open a lightning component from related list button? If i am opening a visualforce page which has the modal component, is there a way to restrict it from opening it in a new tab instead open as a popup on existing record page layout?