Are you sure the rich text field is visible to the site profile? It's not included in the regular list, you have to go to the site settings and access the profile from there where you can then check whether the field level security is good. I assume it is since you have the URL , but then I don't know why you can't access the image. Have you tried using straight tags instead of using the Visualforce image tag?
We use rich text fields for our website and that displays images from the same server without problems; the image below is accessed via this URL:
