Our issue was we had an middle level component wrapping the child component that missed the slots (not shown in the original question). Went down a rabbit hole thinking named slots and unnamed slots worked different with elements vs. text content, which was NOT the case. They both take in elements the same way.
The above code works as sdfcfox mentioned:
ParentComponent (.html)
<template>
<slot name="header"></slot>
<slot name="footer"></slot>
</template>
ChildComponent (.html)
<template>
<c-parent-component>
<div slot="header">
<lightning-button label="Header Button 1"></lightning-button>
<lightning-button label="Header Button 2"></lightning-button>
</div>
<div slot="footer">
<lightning-button label="Footer Button 1"></lightning-button>
<lightning-button label="Footer Button 2"></lightning-button>
</div>
</c-parent-component>
</template>