One important aspect, besides the CRSF limitation, is that when you use the string constructor, you're explicitly avoiding creating a metadata link between your page and the class.
What this means for you is that if someone decides that your page is no longer used, and they delete the page, it can break your code, as you'll now be redirecting to a non-existent page. If you use the factory variable instead, you get automatic metadata protection. Similarly, if you make a typo in a string, you won't be notified, while if you're using the Page factory, you'll avoid simple typos or missing files during deployment.
When you attempt to delete the page, you'll be notified that the page is used in that class. Also, you can see a page's usage by clicking on the "Where is this used" button in the UI. This only detects "strong" links to the page, not weak links, such as using plain URLs. Change sets and packages also provide dependency management, but they can't protect you from simple string references.
Similarly, when possible, you should prefer to use the $Page global variable to link pages together, and use the Visualforce URL type for custom buttons and links that reference the page. Creating strong links in your metadata will help prevent accidental deletions.