.Net application to run Apex TC through tfs build system

Question

I am currently automating a salesforce application using selenium and apex TC. We use the salesforce Soap api and enterprise wsdl for our DML and SOQL updates, queries, etc. We are also using the soap api to run the apex tests. We run all of our selenium tests through tfs build using the microsoft build system. The reporting reports on each test method if it to fail or pass. I am trying to figure out how to run each apex test method instead of running the entire class.

The code below is one selenium method that runs the entire apex TC class with many methods. I want to be able to do a 1 to 1 and have a selenium method call each apex test method however I am not finding away to do this.

The Selenium test method:

[TestMethod]
    public void TestFromEventDetailsSprint181()
    {
        //This line of code logs into the dev org as defined in App.config and runs the specified apex test class.
        //   Use intellisense to see the available properties returned in testResults. This can be used to assert against for reporting back to TFS. 

        //CommonFunctionLibrary.SfApex.RunTestsResult testResults = Helper.RunSalesforceApexTestClass("TestForecastedRevWithoutECUpts_Test", null);
        CommonFunctionLibrary.SfApex.RunTestsResult testResults = Helper.RunSalesforceApexTestClass("somenamespace.someclass", null);

        //Some notable data available in the returned results to assert against:
        //   numTestRuns = # of testmethods executed in the apex test class
        //   numFailures = # of failed test methods
        //   failures = array of failure structures, each contains the methodName and message (see below)
        //   successes = array of success structures

        string failureMessage = "";
        if (testResults.numFailures > 0)
        {
            foreach (var failure in testResults.failures)
            {
                failureMessage += "Test failure encountered in method: " + failure.methodName + ", error: " + failure.message + ".\n";
            }
        }

        //For demo purposes this assert assumes at least 1 testmethod will fail, as designed in the example TestClassTemplate run above.
        Assert.AreEqual(0, testResults.numFailures, failureMessage);
    }

The soap api called method to run the apex test class:

public static RunTestsResult RunSalesforceApexTestClass(string testClassName, string testClassNamespace)
    {
        bool isSandbox = !AppSettings["EnvironmentType"].Equals("developer");

        ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;

        //Log into SFDC. Retrieve the Session ID and instance URL.
        SoapClient soapClient = new SoapClient((!isSandbox ? "Sfdeveloper" : "Sfsandbox"));

        SfPartner.LoginResult loginResult = soapClient.login(null, null, AppSettings["Username"], AppSettings["Password"]);

        SfApex.SessionHeader apexSessionHeader = new SfApex.SessionHeader();
        apexSessionHeader.sessionId = loginResult.sessionId;

        RunTestsRequest runTestsRequest = new RunTestsRequest();
        runTestsRequest.classes = new[] { testClassName };
        runTestsRequest.@namespace = (string.IsNullOrEmpty(testClassNamespace) ? AppSettings["OrgNameSpace"] : testClassNamespace);

        ApexService apexService = new ApexService();
        apexService.Url = loginResult.serverUrl.Replace("/u/", "/s/");   //The instance URL has a path change to call the SOAP service
        apexService.SessionHeaderValue = apexSessionHeader;

        RunTestsResult ret = apexService.runTests(runTestsRequest);

        return ret;

    }

Answer 1

The Apex SOAP API is a bit dated now and hasn't received significant updates for some time. All the interesting bits are now put in the Tooling API.

Unfortunately Salesforce have been lax in maintaining parity between the SOAP and REST versions of the Tooling API (as at Spring '17). With the SOAP Tooling API you are either limited to running an entire Test class synchronously, or running running asynchronously and being able to specify specific test suits. See Soap Calls.

Compare that to the REST Tooling API resources /runTestsSynchronous and /runTestsAsynchronous. With those you can specify the classId and then the test method names that you want to run.

For your scenario I'd go with /runTestsSynchronous. The POST JSON body will look something like:

[{
   "classId":
      "01pD0000000Fhy9IAC",
   "testMethods": [
      "testMethod1",
      "testMethod2",
      "testMethod3"
   ]
}, {
   "maxFailedTests": "1"
}]

The fun part of that of course is determining what the test method names are. You could either use the SymbolTable or pull apart the calls that the Developer Console uses to identify the test method names. I suspect the latter isn't using an official public API.