I am trying to remove the web Email and Contact Name from Case layout. But, I am getting a message , "This field Must be displayed on the Page Layout and can be removed". Why there are "Always on page layout Field" in Salesforce. Does salesforce need it for backend tables maintenance or Normalization ?
This occurs when the field definition itself is marked as required, you should see a little blue dot next to the field when viewing the layout. As apposed to being marked as required on the layout, which shows a red asterix next to the field (which you can edit via the toolbox icon to the right of the field).
Custom Fields you can control the 'Required' attribute on the field (assuming its not a packaged field, in which case only the vendor can change it). For Standard Fields i am afraid as you've suspected this is down to the engineers at Salesforce who have marked it as such and it cannot be changed and as such you cannot effect the enforcement to retain the field on the layouts. I found this question about a similar problem on the Opportunity layout. Sorry its not better news.
If you don't use these fields at all, you can set them to not Visibile in the field accessibility, even for system administrator. You won't see them in page layout and that should fix your problem.