How do governing limits work with @testSetup from Spring 15' release

Question

The new @testSetup method is covered pretty well by Jesse Altman in his: New Spring ’15 Feature: @testSetup article but as mentioned in the comments, there is a question about how this new feature works with the governing limits.

Does DML and SOQL statements count against the governing limits of my test methods? If so, how do they impact the governing limits of my overall test class?

Answer 1

As I have just rewritten our internal unit test design patterns I've tested this quiet thoroughly and believe I have a proper grasp on how @testSetup currently handles governing limits within the test class

In short, any DML statement within the context of @testSetup will not count against your overall test class limits but SOQL statements will.

For example:

@testSetup public static void virtualSandbox(){
    Account[] accts = new List<Account>();
    for(Integer i=0;i<3;i++) {
        Account a = new Account(Name='Acme' + i,BillingCity='San Francisco');
        accts.add(a);
    }
    INSERT accts;
    System.debug('Total Number of SOQL Queries allowed in this apex code context: ' +  Limits.getQueries());

    List<Account> account_test = [SELECT Id, Name, QB_Company_File__c FROM Account];
    System.debug('Total Number of SOQL Queries allowed in this apex code context: ' +  Limits.getQueries());
}

The above INSERT on Accounts will have no influence on your governing limit in of itself but the SOQL statement will count as 1.

NOTE: If your org has events that are triggered from that INSERT (which you likely do), THEN any DML or SOQL statement that proceeds from those are considered outside the context of @testSetup and will impact your overall governing limits for your test class.

As always these limits are reset within any test method when you invoke Test.startTest(); and Test.stopTest(); but this could affect how much data you can setup within your data setup method (for example when cascading dependencies like Contacts, which need Accounts).

Workflow Rules (WFR):

If you have a WFR on Leads that will INSERT an Activity based on certain criteria, then the INSERT of that Activity will not affect your limits but again if it triggers any DML/SOQL then expect it to impacting your limits

Workaround:

Because I have foundation test classes that cover a majority of the essential trigger context for every object I wanted to include as much data as necessary for certain test classes, but unfortunately I would run into limit issues quickly because of triggers. So thankfully, since we use Kevin O'Hara's TriggerHandler Framework, we were able to implement his handy bypass() method which temporarily pauses all trigger activity within your TriggerHandler class

For example:

@testSetup public static void virtualSandbox(){
    TriggerHandler.bypass('AccountTriggerHandler'); // This will ignore the AccountTriggerHandler.cls that extends the TriggerHandler.cls framework

    Account[] accts = new List<Account>();
    for(Integer i=0;i<3;i++) {
        Account a = new Account(Name='Acme' + i,BillingCity='San Francisco');
        accts.add(a);
    }
    INSERT accts;

    TriggerHandler.clearBypass('AccountTriggerHandler'); // This will turn it back on as needed for the remainder of the test.
}

Hope this proves to be useful in someone, feel free to comment with any adjustments you believe are necessary to this explanation.

Answer 2

I believe that the accepted answer here is incorrect. The accepted answer states that

any DML statement within the context of @testSetup will not count against your overall test class limits but SOQL statements will.

I tested this with something simple (in a scratch org, no other automation going on)

@IsTest
private class LimitTest {

    @TestSetup
    static void setup() {
        insert new Account(Name = 'ACME');
        Account a = [SELECT Id FROM Account];
        insert new Contact(FirstName = 'Daffy', LastName = 'Duck', AccountId = a.Id);
        Contact c = [SELECT Id FROM Contact];
        System.debug('CPU Used: ' + Limits.getCpuTime());
        System.debug('SOQL Used: ' + Limits.getQueries());
        System.debug('DML Used: ' + Limits.getDmlStatements());
    }

    @IsTest
    static void testBehaviour() {
        System.debug('CPU Used: ' + Limits.getCpuTime());
        System.debug('SOQL Used: ' + Limits.getQueries());
        System.debug('DML Used: ' + Limits.getDmlStatements());
    }
}

Here, both the SOQL and DML usage are reported as 2 at the end of test setup, and at the start of the test method. In my case CPU usage was 56 at the end of test setup, and 0 at the start of the test method.

Obviously, I get a fresh set of limits once I call Test.startTest(), but if I have test-specific setup to do before the Test.startTest(), then using @TestSetup has not given me any extra capacity to do so.

It seems that of the key limits: DML, SOQL, and CPU; only the CPU time is reset at the beginning of each test method.

Answer 3

Revisiting this in Winter `24, I concur with the answer that Aidan posted. The SOQL Query and DML limit consumption from the @testSetup will carry over into the subsequent test methods.

However, we can work around that using Test.startTest() and Test.stopTest() within the @testSetup method itself.

E.g. Wrapping the DML and SOQL operations of the @testSetup method with a start/stop test resets DML and SOQL limits before passing execution into the test method.

@IsTest
private class LimitTest {

    @TestSetup
    static void setup() {
        Test.startTest();
        insert new Account(Name = 'ACME');
        Account a = [SELECT Id FROM Account];
        insert new Contact(FirstName = 'Daffy', LastName = 'Duck', AccountId = a.Id);
        Contact c = [SELECT Id FROM Contact];
        
        System.debug('CPU Used: ' + Limits.getCpuTime());
        System.debug('SOQL Used: ' + Limits.getQueries());
        System.debug('DML Used: ' + Limits.getDmlStatements());
        
        Test.stopTest();
        
        System.debug('End CPU Used: ' + Limits.getCpuTime());
        System.debug('End SOQL Used: ' + Limits.getQueries());
        System.debug('End DML Used: ' + Limits.getDmlStatements());
    }

    @IsTest
    static void testBehaviour1() {
        System.debug('CPU Used: ' + Limits.getCpuTime());
        System.debug('SOQL Used: ' + Limits.getQueries());
        System.debug('DML Used: ' + Limits.getDmlStatements());
        Test.startTest();
        List<Account> testAccounts = [Select Id from Account];
        Assert.areNotEqual(0, testAccounts.size());
        Test.stopTest();
    }
    
    @IsTest
    static void testBehaviour2() {
        System.debug('CPU Used: ' + Limits.getCpuTime());
        System.debug('SOQL Used: ' + Limits.getQueries());
        System.debug('DML Used: ' + Limits.getDmlStatements());
    }
    
    @IsTest
    static void testBehaviour3() {
        System.debug('CPU Used: ' + Limits.getCpuTime());
        System.debug('SOQL Used: ' + Limits.getQueries());
        System.debug('DML Used: ' + Limits.getDmlStatements());
    }
}