Have a read of Here Comes The Hammer, you may find it illuminating.
It is not clear how Salesforce determines which organization instances tests to run and in which release, is this just an arbitrary and completely random process?
The Hammer means taking every single Apex test that you or anyone else has created and running it twice.
They run every test.
What is the value of seeing that 58.53% is green on the chart? Does that mean that this is percent of tests run successfully on the previous or next release? Does that mean that this is percent of test run successfully in old release but failed in new release? Does that mean that Salesforce has fixed those failures in the next release?
You can help us fulfill this commitment by using data silo. Tests that utilize the new data silo paradigm (seeAllData=false) create their own data, and don’t actually need access to the production data stores. These tests are much faster for us to run, and much more reliable. We don’t need to worry about reading your debug statements, because they’re outputting fake data. We don’t need to worry about having a copy of your production data, because data silo tests do not rely on that data to succeed. Org-independent tests make The Hammer smile.
This number tells you how many of your tests no longer rely on
SeeAllData. You should strive for very nearly 100%.
So what's the benefit?
This complex juggling act happens three times a year. We make a commitment to you that your customizations will continue to run as you expect, no matter how much we do to change the system. In all my years of being in enterprise software, I have never heard of such a commitment. I’ve never heard of such a smooth upgrade path: you go to bed on Friday, you wake up Saturday, and your system has been upgraded. I still have flashbacks to lenghty (sic), painful upgrade projects for other companies I have worked for – those were never easy times. To save you from these flashbacks, we do the complex ballet described here (which still makes it sound easier than it is) three times a year.
For the 60 million tests we ran this past release, we uncovered 25 potential regressions in time to fix them prior to release. On the surface, that sounds like a lot of work for a tiny result. Each of those 25 issues we solved were going to impact someone in some way, potentially in some critical way, and they often span multiple customers. The result is quite large from the point of view of each customer who never had to know that they were saved from a big headache.