When viewing the connected app in
App Manager the
Visible in Lightning Experience checkbox is not checked. If I'm using Salesforce to SSO into another app, how can I surface this connected app in the App Launcher?
As per my understanding, we can't have connected app as part of App Launcher. Instead as a workaround, you may have a web-link tab, and that tab you can use in the particular Application.
So, if you want to see the connected app, you can click the tab which will redirect to the connected app page.
PS - It depends on the permission you're having to see the connected app.