14

This Limits Documentation mentions:

Total number of test classes that can be queued per a 24-hour period (5) - The greater of 500 or 10 multiplied by the number of test classes in the organization

with the caveat that

This limit applies to tests running asynchronously. This includes tests started through the Salesforce user interface including the Developer Console or by inserting ApexTestQueueItem objects using SOAP API.

  1. What qualifies as running a test synchronously?
  2. How can I ensure that my team members working concurrently in a sandbox do not exceed the AsyncApexTests Limit while building, refining and executing unit tests? (This quickly brings test development to a grinding halt.)
  3. Does running tests via the Eclipse "run tests" menu option kick off an async or synchronous test run?

An update that is a bit like an answer:

If you need to run a unit test synchronously, you can do so in the developer console's run tests UI by selecting just a single class to run. Selecting two or more classes will execute them asynchronously.

This makes receiving the AsyncApexTests Limit exceeded exception a little bit more bearable and less blocking.

The Developer Console runs tests asynchronously in the background, unless your test run includes only one class and you’ve not chosen Always Run Asynchronously in the Test menu.

Source

  • I imagine the tests run by the package upload process might quality? – ca_peterson Dec 12 '13 at 0:35
  • I was told by a SalesForce support person that running tests via the Developer Console doesn't count towards that limit. I have yet to hit that error in any orgs. I even tried setting our Continuous Integration server to run every 10 minutes for several hours but the error didn't appear. – BarCotter Dec 12 '13 at 1:09
  • @BarCotter I personally have run into the error, but it took many "run all tests" to get there. We have over 200 tests in our project's test suite. – sfdcfox Dec 12 '13 at 4:14
  • We've hit it twice in as many days. Trying to get a handle on how to manage it effectively. (aside from telling our developers to stop running so many tests) – Mark Pond Dec 12 '13 at 7:43
7

Ive a few references for you Mark.

From here

Tests started using Run All Tests or Run Test no longer run synchronously. When you run tests by clicking Your Name > Setup > Develop > Apex Classes, and then either click Run All Tests from the Apex Classes page or Run Test from a class page, your test results are displayed in the Apex Test Execution page and the Developer Console This means that you won’t be blocked by the test run; you can do other work in the Salesforce user interface and you can access the Apex Test Execution page at a later time to verify if the tests have completed execution.

All tests started through the Salesforce user interface now run asynchronously. Tests that are run as part of a deployment, a package install, or a package upload still run synchronously.

From here:

You can use the runTests() call from the SOAP API to run tests synchronously:

RunTestsResult[] runTests(RunTestsRequest ri)

This call allows you to run all tests in all classes, all tests in a specific namespace, or all tests in a subset of classes in a specific namespace, as specified in the RunTestsRequest object.

Conclusion:

Based on the links above, tests can be run from the following places:

  • The Salesforce user interface - asynchronously
  • The Force.com IDE - synchronously
  • The Force.com Developer Console - asynchronously
  • The API - synchronously
  • As part of a deployment, a package install, or a package upload - synchronously
  • By inserting records into the ApexTestQueueItem table - asynchronously
  • Im still unsure about tests being run via the Force.com IDE. I would of assumed that they would be using the API and therefore run synchronously. – BarCotter Dec 14 '13 at 21:45
  • You can use the API to run tests asynchronously by inserting records into the ApexTestQueueItem table. – Phil Rymek Dec 14 '13 at 22:21
  • "The Force.com Developer Console - asynchronously" - Actually the developer console has an option whether to execute synchronously or asynchronously. Unless otherwise the developer checks that option, the tests there are executed synchronously by default. – Lightning Evangelist Dec 16 '16 at 16:19
5

Originally unit tests in Salesforce only ran synchronously. In summer '11 asynchronous unit tests were released with the primary benefit being that they don't block development; you don't need to wait for them to run, and they dont result in the org administration locked error.

There are many ways to run unit tests, and over time it seems Salesforce is trying to eliminate the synchronous method of running unit tests entirely.

What qualifies as running a test synchronously?

There are API calls that run unit tests both synchronously and asynchronously. This documentation explains how to do it both ways.

Unit tests that run as a result of the metadata api (e.g. deploys from Force.com IDE & Ant Migration Toolkit) run synchronously. This can be proven because the results do not appear in the Apex Test Execution page within Salesforce. Compare this with unit tests run via the developer console, where the Apex Test Execution screen does display these results. Furthermore, all asynchronous unit tests results appear on this screen, as it appears to be baked into the API itself.

How can I ensure that my team members working concurrently in a sandbox do not exceed the AsyncApexTests Limit while building, refining and executing unit tests? (This quickly brings test development to a grinding halt.)

The limit seems fairly high, personally I have never hit it. I would imagine that you could avoid hitting it by avoiding use of functionality that runs all unit tests, and instead focus on running only the relevent unit tests. While obviously a stupid solution, this is the type of limit you can hack -- just add 100 dummy classes marked as @isTest.

Alternatively you can programatically determine how many unit tests have ran in a day and build some type of monitor to warn the developers when they are close to hitting the limits. Query the Status (Completed, Failed) and CreatedDate fields of the ApexTestQueueItem sObject to get insight into how many unit tests have ran in a day. You could even use the CreatedBy field to build functionality that blames developers who are abusing the unit test execution functionality.

Does running tests via the Eclipse "run tests" menu option kick off an async or synchronous test run?

It does a synchronous run. This can be determined because the results do not appear in the Apex Test Execution screen and the Force.com IDE returns code coverage results line by line telling you what is covered, and what is not covered. The latter information is currently not available when unit tests are ran asynchronously; at least not via publicly facing API as far as I know.

4

When packaging you sit and wait for the tests to run, and it shows progress as they go—that could be an example of them running synchronously, and perhaps when they're run on install as that process can take a long time with no output which would indicate a blocking action.

Apart from those scenarios it seems to me that by their very nature (with respect to data visibility etc.) they'd always be run in a separate thread, and therefore, asynchronously.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.