The heart of the question is how to avoid having a dependency on the managed package - needing it to always be installed - yet still be able to invoke methods when it is installed. Unfortunately Apex lacks the language features (for example Java's reflection) that allow that.
While the Type class allows instances of a class to be created using just the string name of the class, to invoke any methods a cast to an explicit (compile-time know) interface or class is needed which gets you right back into having the dependency. (Using Type allows the extension of managed package code but does not break the dependency on that managed package code.)
If the subscriber org is not managed code then you will have to add/remove classes that depend on the managed package depending on whether it is present or not. You can insulate most of your code from that by using e.g. an interface and substituting the implementation: either have one that depends on the managed package or have one that defaults the behaviour or fails if called.
If the subscribing code is also a managed package then things get harder and introducing a third piece of code is necessary. See this Breaking managed package dependencies blog post for more explanation of that. It is so painful that it would only be worth doing for a small number of major features (unless tools were created to generate the code).
Calling the tooling API from Apex is possible. But it is a web service callout, a separate transaction, requires parameters to be passed in the Apex code string, doesn't allow results to be directly returned and is presumably slow. So not the semantics of a typical method call.