For the Favicon to work, I had to upload the image to the orgs assets library:

[![enter image description here][1]][1]

Only then was I able to reference my icon, which I had to upload as a 16x16 png image.

Finally, I was able to reference it using the file-asset's relative path as follows:

[![enter image description here][2]][2]

Notice that the icon name when I reference it has a versioning appended to it, 

    <link rel="icon" href="/file-asset/coveofavicon?v=1" type="image/png" />

[![enter image description here][3]][3]

I did not find this documented anywhere =(, hope this helps other people.


  [1]: https://i.sstatic.net/Al0Rj.png
  [2]: https://i.sstatic.net/7NWQA.png
  [3]: https://i.sstatic.net/wfpi4.png