Is that safe or what could possible go wrong?
There have been a lot of "breaking changes" since 29. For example, versions 34 and 40 in particular were major changes in how JSON is handled, security changes, and so on. Your code could have hidden bugs waiting for you.
Am I safe to assume everything is right when it compiles and all tests run green?
Only if your unit tests were actual unit tests and not smoke tests. A smoke test is a test designed only to catch the crudest of errors, such as compilation errors or very obvious mistakes, but doesn't put the code through its paces. For example, a test involving JSON might use a proper, 100% correct JSON string, but not test a string with missing values (this is a change that occurred in serialization around v32 or so). You may need to manually review your code.
What else should I check?
You need to check the release notes for every version (just the Development > Apex sections, not all 1000+ pages), then check your code to see if they fall in that category of change.
Most release notes only add features, but you need to check for anything marked as a "change" in your code. Make sure you check your unit tests for classes that would be affected by the change.
In addition to unit tests, I strongly recommend that you also at least do a quick manual review of the code, especially the unit tests, as well as doing some manual UI testing when possible. Since you're taking the time to go up to a modern version, a review of your unit tests is in order.
Is there a place where I could read what changed for classes in all releases?
The release notes is the official channel, but it doesn't read like a "change log", unfortunately. I don't think anyone's built one of those.
In summary, changing your version from 29 to 50 is actually a moderate risk. Things can go wrong in subtle ways. If you have great tests, then that risk is lowered, but if possible, I would recommend not relying solely on the code. You also should spot-check your code for breaking changes from the release notes and perform some manual testing.