Similar, although not the same component, the lightning:pill supports adding icons within it:

<lightning:pill label="Pill Label">
    <aura:set attribute="media">
        <lightning:icon iconName="standard:account"  alternativeText="Account"/>
    </aura:set>
</lightning:pill>

some styling is needed to hide the remove option though:

.THIS  button > lightning-primitive-icon{
    display:none;
}

Otherwise, you will have to tweak the SLDS Badge Blueprint - similar to what @codeyinthecloud posted.

Similar, although not the same component, the lightning:pill supports adding icons within it:

<lightning:pill label="Pill Label">
    <aura:set attribute="media">
        <lightning:icon iconName="standard:account"  alternativeText="Account"/>
    </aura:set>
</lightning:pill>

some styling is needed to hide the remove option though:

.THIS .slds-pill__remove{
    display:none;
}

Otherwise, you will have to tweak the SLDS Badge Blueprint - similar to what @codeyinthecloud posted.