I found a simple if not somewhat hacky way to resolve this issue without needing to bring in the SLDS stylesheets at all (which carry their own problems with legacy VF pages). I simply added an extra empty column with 100% width to my dataTable like so:

<apex:dataTable value="{!foo}" var="f" cellspacing="4">
    <apex:column>
        <!--column 1 data-->
    </apex:column>
    <apex:column>
        <!--column 2 data-->
    </apex:column>
    <apex:column>
        <!--column 3 data-->
    </apex:column>
    <!-- BELOW IS THE MAGIC -->
    <apex:column width="100%" rendered="{!onLightning}"/>
</apex:dataTable>

The onLightning var simply looks at the current user's UITheme to know when to render this column:

public boolean onLightning {get {if (onLightning == null) onLightning = userInfo.getUiThemeDisplayed() == 'Theme4d'; return onLightning;} set;}

Apparently by adding in a blank column with 100% width, the lightning stylesheet's bias to force tables to fill 100% of their container is effectively alleviated, and every column is then able to dynamically stack one next to one another while respecting their specified width. However, the cellspacing attribute doesn't actually seem to take any effect... though I'm satisfied with the default value that seems to be getting applied by the stylesheets:

I found a simple if not somewhat hacky way to resolve this issue without needing to bring in the SLDS stylesheets at all (which carry their own problems with legacy VF pages). I simply added an extra empty column with 100% width to my dataTable like so:

<apex:dataTable value="{!foo}" var="f" cellspacing="4">
    <apex:column>
        <!--column 1 data-->
    </apex:column>
    <apex:column>
        <!--column 2 data-->
    </apex:column>
    <apex:column>
        <!--column 3 data-->
    </apex:column>
    <!-- BELOW IS THE MAGIC -->
    <apex:column width="100%" rendered="{!onLightning}"/>
</apex:dataTable>

The onLightning var simply looks at the current user's UITheme to know when to render this column:

public boolean onLightning {get {if (onLightning == null) onLightning = userInfo.getUiThemeDisplayed() == 'Theme4d'; return onLightning;} set;}

Apparently by adding in a blank column with 100% width, the lightning stylesheet's bias to force tables to fill 100% of their container is effectively alleviated, and every column is then able to dynamically stack one next to one another while respecting their specified width. However, the cellspacing attribute doesn't actually seem to take any effect... though I'm satisfied with the default value that seems to be getting applied by the stylesheets:

UPDATE: Looks like the core problem, as indicated by a closer examination of the browser's page inspect tool, is that tables in LEX are given a min-width of 100%. So an easy solution as it turns out is to set this to 0% in the table style:

<apex:dataTable value="{!foo}" var="f" style="min-width:0%">
    <apex:column>
        <!--column 1 data-->
    </apex:column>
    <apex:column>
        <!--column 2 data-->
    </apex:column>
    <apex:column>
        <!--column 3 data-->
    </apex:column>
</apex:dataTable>

I found a simple if not somewhat hacky way to resolve this issue without needing to bring in the SLDS stylesheets at all (which carry their own problems with legacy VF pages). I simply added an extra empty column with 100% width to my dataTable like so:

<apex:dataTable value="{!foo}" var="f" cellspacing="4">
    <apex:column>
        <!--column 1 data-->
    </apex:column>
    <apex:column>
        <!--column 2 data-->
    </apex:column>
    <apex:column>
        <!--column 3 data-->
    </apex:column>
    <!-- BELOW IS THE MAGIC -->
    <apex:column width="100%" rendered="{!onLightning}"/>
</apex:dataTable>

The onLightning var simply looks at the current user's UITheme to know when to render this column:

public boolean onLightning {get {if (onLightning == null) onLightning = userInfo.getUiThemeDisplayed() == 'Theme4d'; return onLightning;} set;}

Apparently by adding in a blank column with 100% width, the lightning stylesheet's bias to force tables to fill 100% of their container is effectively alleviated, and every column is then able to dynamically stack one next to one another. However, the cellspacing attribute doesn't actually seem to take any effect... though I'm satisfied with the default value that seems to be getting applied by the stylesheets:

I found a simple if not somewhat hacky way to resolve this issue without needing to bring in the SLDS stylesheets at all (which carry their own problems with legacy VF pages). I simply added an extra empty column with 100% width to my dataTable like so:

<apex:dataTable value="{!foo}" var="f" cellspacing="4">
    <apex:column>
        <!--column 1 data-->
    </apex:column>
    <apex:column>
        <!--column 2 data-->
    </apex:column>
    <apex:column>
        <!--column 3 data-->
    </apex:column>
    <!-- BELOW IS THE MAGIC -->
    <apex:column width="100%" rendered="{!onLightning}"/>
</apex:dataTable>

The onLightning var simply looks at the current user's UITheme to know when to render this column:

public boolean onLightning {get {if (onLightning == null) onLightning = userInfo.getUiThemeDisplayed() == 'Theme4d'; return onLightning;} set;}

Apparently by adding in a blank column with 100% width, the lightning stylesheet's bias to force tables to fill 100% of their container is effectively alleviated, and every column is then able to dynamically stack one next to one another while respecting their specified width. However, the cellspacing attribute doesn't actually seem to take any effect... though I'm satisfied with the default value that seems to be getting applied by the stylesheets: