It is possible to programatically run tests in the platform by creating
ApexTestQueueItem records. They reference classes by their Id, and you can even optionally specify if you want the test to calculate coverage or not.
When inserting those records you'll automatically get an
AsyncApexJob Id assigned to the
ParentJobId on the previously mentioned queue record. We can use this Id to query for results, like:
SELECT Id, ClassesEnqueued, ClassesCompleted, EndTime, MethodsEnqueued, MethodsCompleted, MethodsFailed, Status, TestTime FROM ApexTestRunResult WHERE AsyncApexJobId = '7071I00002EvjEvQAJ'
Now, according to the documentation, this record holds the information for our tests alright. The amount of classes enqueued, passed, etc.
My problem is: some seconds after putting some test classes to run, this record is marked as
Complete (on the Status field) by the platform, but the classes and methods values do not match what has been specified on the test.
If I specify that I want, for example, to run 12 tests, then when the record is updated to
Complete I'll get:
ClassesEnqueued: 12 ClassesCompleted: 8
The same issue applies for the
So 4 classes and some methods are missing, but the test result record is still set to
Am I missing something here?
One of those 12 classes was not a test, thank you @sfdcfox for pointing that out. But all other 11 classes are.
When querying the
SELECT Id, ClassesEnqueued, ClassesCompleted, EndTime, MethodsEnqueued, MethodsCompleted, MethodsFailed, Status, TestTime FROM ApexTestRunResult WHERE AsyncApexJobId = '7071I00002Evpx8QAB'
...I get a result like this:
Id: 05m1I000000XsLjQAK ClassesEnqueued: 11 ClassesCompleted: 8 Status: Completed MethodsEnqueued: 35 MethodsCompleted: 32 MethodsFailed: 0 EndTime: 2018-08-17T03:52:55.000Z TestTime: 2417
And when querying the
ApexTestResult records related to the same job:
SELECT Id, ApexClass.Name, MethodName, ApexTestRunResult.Status, Outcome FROM ApexTestResult WHERE AsyncApexJobId = '7071I00002Evpx8QAB'
I get 35 rows of records, all passing. Which also doesn't match the result from ApexTestRunResult (35 vs 32).
Querying the specific tests and retrieving the summary from the relationship also shows the mismatch between the enqueued and completed tests (although all are passing).