Yes, this is the default behavior, and no there's no "direct" workaround. The reason why is because the Map returned by the call is actually a special map that I'll call CaseInsensitiveKeyMap. A normal map would more appropriately be called a CaseSensitiveKeyMap, because "Hello" and "hello" are two unique keys (because they use hashCode(), which will definitely return unique values for even a change in string case).
On the other hand, the describe calls, such as
fieldSets.getMap(), effectively call
String.toLowerCase() on each key before calling
hashCode(), such that you can ask for
name (or any other permutation) to get the correct field. Field names are case insensitive in Apex Code, and this map mimics that behavior.
If you absolutely wanted to have a case sensitive map, you'd have to construct your own Map, iterate through the values from the describe call, and set each key individually. This would be a large waste of CPU time for no real benefit, since there shouldn't be any specific reason why you need to have the correct casing.