tl;dr - use "standard" icons
In case it is helpful for future readers, I was able to find a solution to this.
I was just building out an aura tab and encountered this same issue. While most icons appeared too large for the tab, I found that any icons selected from the "standard" category rendered correctly (in both size and color). The list of "standard" icons can be found here.
Code Example with Images
For example, in the setTabIcon method:
using an icon selection like this will render the icon incorrectly:
But using a standard icon will render the size and color appropriate for lightning tabs