No, there is no filtering by App in the setup for custom objects, etc as Mr. Fox commented.
If it comes to the configuration of the app itself, custom app settings are not like settings of the core force.com features. They typically end up as custom settings ( http://help.salesforce.com/HTViewHelpDoc?id=cs_about.htm&language=en_US ) or are configured with the help of custom objects. Also there is no sophisticated way to extend the given setup pages. Therefore a common pattern is to provide a visualforce page for the configuration of a custom app. This page will be presented to users either as custom tab (bad, because tabs are a limited resource in most editions) or as configuration link of a package (that's what I prefer)
This said, the setup drilldown and usage is probably a bit different as you might expect - since the app related config usually resides somewhere else. So at the setup there is no filter to show things drilled down to a custom app.
Also generally consider an App as less dominant as you might assume. I see a salesforce app only as a collection of tabs.
Mind also the difference between packages and apps.