Similar, although not the same component, the lightning:pill supports adding icons within it:
<lightning:pill label="Pill Label">
<aura:set attribute="media">
<lightning:icon iconName="standard:account" alternativeText="Account"/>
</aura:set>
</lightning:pill>
some styling is needed to hide the remove option though:
.THIS button > lightning-primitive-icon{
display:none;
}
Otherwise, you will have to tweak the SLDS Badge Blueprint - similar to what @codeyinthecloud posted.