This is SObject default sorting. As covered in that article, you can write a wrapper and implement Comparable to apply your own sorting (just using basic String comparison for the names will do what you want here).

IIRC, to change sorting in the UI without the need for Apex, add a formula field that takes the Name value and exposes it as a string, then sort using that formula field instead.

This is SObject default sorting. As covered in that article, you can write a wrapper and implement Comparable to apply your own sorting (just using basic String comparison for the names will do what you want here).

UPDATE:

According to Uwe's testing of using a formula field that wrapped the Name, and using the formula field for sorting in the UI, this did not resolve this issue for him.