One way could be to sniff at the current page's URL and realize that the Id passed is the tab's id (you can use it to access the Custom Tab's detail page in setup).
You could then compare it with tab's URL from the describeTabs()
call.
Ctrl:
public class Hello
{
public String currentApp {get;private set;}
public String allTabsDebug {get; private set;}
public String currentTab {get; private set;}
public String currentTabDebug {get; private set;}
public Hello(){
String tabId = ApexPages.currentPage().getParameters().get('sfdc.tabName');
for(Schema.DescribeTabSetResult dtsr : Schema.describeTabs()) {
if(dtsr.isSelected()) {
currentApp = dtsr.label;
allTabsDebug = JSON.serializePretty(dtsr.tabs);
if(String.isNotBlank(tabId)){
for(Schema.DescribeTabResult dtr : dtsr.tabs){
if(String.isNotBlank(dtr.url) && dtr.url.contains(tabId)){
currentTab = dtr.label;
currentTabDebug = JSON.serializePretty(dtr);
break;
}
}
}
break;
}
}
}
}
Page
<apex:page controller="Hello">
<apex:pageBlock title="Input">
<apex:pageBlockSection title="Input" columns="1">
<apex:outputText label="URL" value="{!$CurrentPage.url}" />
<apex:outputText label="sfdc.tabName param" value="{!$CurrentPage.parameters.sfdc.tabName}" />
</apex:pageBlockSection>
<apex:pageBlockSection title="Output" columns="1">
<apex:outputText label="Selected App" value="{!currentApp}" />
<apex:outputText label="Selected Tab?" value="{!currentTab}" />
<apex:outputText label="Selected Tab's Debug" value="{!currentTabDebug}" />
</apex:pageBlockSection>
</apex:pageBlock>
<apex:pageBlock title="Debug (all tabs in that app)">
<pre>{!allTabsDebug}</pre>
</apex:pageBlock>
</apex:page>
Not sure about it to be honest, looks bit fragile. But probably will be better solution than doing some CSS sniffing which tab is highlighted and sending it back to Apex with <apex:page action>
:)