Normally, you can put debug monitoring on a user by name to see the details on the error they're getting (especially for the non-descript Error Loading a Visualforce Page).
But when you are trying to debug a site page, how do you get a debug log for the guest user?
Update
As of Spring '18 Salesforce has dropped the requirement to include the debug_logs cookie to get debug logs for the force.com site requests.
In Winter ’17, we introduced a change that required guest users to set browser cookies to activate debug logging. We had the best of intentions: All your public site visitors share one guest user license. Thus, when you enable logging for the guest user, the visitors’ collective activities can fill your debug logs quickly, hitting the log-size limit and causing log truncation. Sadly, people didn’t like the Winter ’17 change. Setting cookies is complicated, and the change made collecting debug logs for public users’ asynchronous activity impossible. We’ve seen the error of our ways. Your public users no longer need a debug_logs browser cookie to trigger logging. This change applies to both Lightning Experience and Salesforce Classic.