What are the advantage of the @testSetUp annotation

Question

Before Salesforce introduced @testSetup annotation in 2015, I used to create common data in a method and call it in every test method.

Ex:

@isTest
private class clsAccountTest {

    private static Account testAccount;

    private static void createCommonData(){
        testAccount = new Account(Name= 'Avinash', BillingPostalCode = '97214');
        insert testAccount;
    }

    @isTest static void itShouldAskForZipCode(){
        createCommonData();
        //testAccount is accessible.
        testAccount.BillingPostalCode = '12345';
        update testAccount;
    }
}

Now, with @testSetUp, it would look like:

@isTest
private class clsAccountTest {

    private static Account testAccount;

    @testSetup static void commonData() {
        testAccount = new Account(Name= 'Avinash', BillingPostalCode = '97214');
        insert testAccount;
        System.Debug('****SOQL Queries issued****'+limits.getQueries()); //returns 5
        System.Debug('****DML Statements issued****'+limits.getDMLStatements()); // returns 4. Issued 1 in testSetUp and it caused 3 more in Account trigger.
    }

    @isTest static void itShouldAskForZipCode(){
        System.Debug('****SOQL Queries issued****'+limits.getQueries()); //return 5
        System.Debug('****DML Statements issued****'+limits.getDMLStatements());    //return 4  
        createCommonData();
        //testAccount is not avaialble, so you have to query for it.
        testAccount = [SELECT Id, Name FROM Account WHERE Name = 'Avinash' LIMIT 1];

        testAccount.BillingPostalCode = '12345';
        update testAccount;
    }
}

What do we gain by using this other than less execution time? The Number of SOQL statements, DML statements etc., generated in the testSetUp method are assigned to each and every test method. In fact, it forces us to use more queries as the static variables are not accessible in test methods.

In this answer, the user said that DML statements issued within the testSetUp method are not counted against governor limits, but the DML statements caused by it in the triggers will be counted. I tested this by inserting a record of an object that doesn't have any trigger code and that statement is not correct. Even the DML statements issued within the testSetUp method are counted against governor limits of each and every test method.

I'm wondering if there are any advantages other than less execution time.

Answer 1

To be honest, the increase in speed is about the only tangible benefit to us (as developers).

Salesforce themselves arguably sees more benefit from people using this annotation than we developers (directly) do. If it's faster for us, that means that it also places less load on Salesforce's pods.

As others (including yourself) have noted, it doesn't save on DML or SOQL compared to having a non-annotated static setup method. As another negative (noted by @KeithC), using an @testSetup annotated method means that you don't even have access to the IDs of the records that you're creating. This ends up causing us to use more SOQL queries for each test.

The other benefits that @SebastianKessel pointed out are good points (separating test setup from the test proper, re-usability), but we'd get the same benefit from just having a separate static setup method (without the annotation).

I continue to use @testSetup because, while the benefits to me may not be very noticeable at times, it doesn't really negatively impact me to use it.

I'll end with the pattern that I use to work around the fact that we can't get at the ids of records that are created in an @testSetup method without querying.

+edit: To be clear, this pattern does still perform extra query/queries that wouldn't be needed if we weren't using @testSetup. There isn't any way around that. The point is to provide useful information (record Ids) to individual tests in a central method using as few queries as possible.

@isTest
public class TestSomeClass{
    // My current preference is to have one map per object that I want the Ids of.
    // You really only need one map to hold ids though, even if you have multiple objects.
    private static Map<String, Id> myObjectIdsMap;

    @testSetup
    public static setupEnvironment(){
        // Your standard setup method, aggregating things in collections to keep
        //   various limits (DML and SOQL, mostly) under control
        List<MyObject__c> testRecs = new List<MyObject__c>();

        testRecs.add(new MyObject__c(/*initial state set in here*/));

        insert testRecs;
    }

    private static void setup(){
        // Initializing my map(s) here is mostly just a habit.
        // Should be fine to initialize on the same line it was initially declared on.
        myObjectIdsMap = new Map<String, Id>();

        // While zero-indexed arrays have been burned into my mind as a programmer, 
        //   one-indexed labeling feels better to me for what we're about to do
        Integer i = 1;

        // For every object that we want to grab the Ids from, we need to run a loop.
        // The integer 'i' needs to be reset after each loop.
        // Ordering by Id ASC doesn't guarantee that we'll get the records back in
        //   the same order that they were inserted in, but it seems to do well enough
        //   for unit tests (in practice).
        // If you have another field you can use for ordering, use it (also, don't rely on 
        //   Id ordering to be correct in non-test code).
        for(MyObject__c myObj :[SELECT Id FROM MyObject__c ORDER BY Id ASC]){

            // Using <object label> + <integer> + 'Id' as a convention here
            //   because it's simple and easy to remember.
            // Prefixing the key with the object name is why we only really need one map.
            myObjectIdsMap.put('myObject' + i + 'Id', myObj.Id);
            i++;
        }
    }

    public testMethod void myTest(){
        // The non-annotated setup() method needs to be called in every test method
        // Yes, this removes one of the minor benefits (not needing to explicitly call
        //   a setup method) of using @testSetup, but keeping the common
        //   'auxiliary' setup contained to its own method is better practice
        //   than repeating queries in each test method.
        setup();

        // If you know the value that you want to update for your test, you don't 
        // need to use a query to get the record to update.
        // Just use the sObject constructor, which allows you to set the Id
        MyObject__c myRec = new MyObject__c(
          Id = myObjectIdsMap.get('myObject1Id'),
          some_field__c = 'updated value'
        );

        Test.startTest();
        update myRec;
        Test.stopTest();

        // assertions as appropriate
        // You'll probably need to query the record at this point, but that would
        //   likely need to be done even if you weren't using an @testSetup method
    }
}

Answer 2

The product manager (I think) at the time this feature was introduced explains it like this:

If you want information that is common to all tests, it can be inserted in the test setup method and queried in each test method. The idea here is not to reduce the number of SOQL queries, it is to reduce the amount of data being inserted into the system. If you insert 1000 records in test setup, run fifteen test methods, and you run a query 15 times to get the 1000 records each time, that's still less expensive (and faster) than inserting 1000 records 15 times.

Answer 3

From my humble point of view, the biggest advantage is to not have to duplicate code and have a consistent set of data that you can use for all the tests in a given class (or classes).

However, there are a couple of other good use-cases.

1. Your code is a lot easier to read, having one place where all the data is created
2. If you have other test classes that need the data you're creating here, you can always call this method and have it do it for you, saving cycles and making your code more maintainable.

==UPDATE==

In my haste to write, I forgot to point out that the idea of the above points is to not insert the same rows x times. Since SFDC rolls back changes to those rows before the beginning of each method, you get one set of DML for all tests. @Keith C, in another answer, put it more eloquently.

Answer 4

The Apex Developer Guide does indicate speed is the primary motivation, especially in the face of data-volume:

Using Test Setup Methods

Use test setup methods (methods that are annotated with @testSetup) to create test records once and then access them in every test method in the test class. Test setup methods can be time-saving when you need to create reference or prerequisite data for all test methods, or a common set of records that all test methods operate on.

Test setup methods can reduce test execution times especially when you’re working with many records. Test setup methods enable you to create common test data easily and efficiently. By setting up records once for the class, you don’t need to re-create records for each test method. Also, because the rollback of records that are created during test setup happens at the end of the execution of the entire class, the number of records that are rolled back is reduced. As a result, system resources are used more efficiently compared to creating those records and having them rolled back for each test method.

If a test class contains a test setup method, the testing framework executes the test setup method first, before any test method in the class. Records that are created in a test setup method are available to all test methods in the test class and are rolled back at the end of test class execution. If a test method changes those records, such as record field updates or record deletions, those changes are rolled back after each test method finishes execution. The next executing test method gets access to the original unmodified state of those records.

In addition to improving your own deployment times, this should reduce the amount of work required by the development team to run its hammer testing.

Answer 5

I have a design pattern that I use so frequently for unit tests that I made an Eclipse template for it. You can also find it here.

@isTest
private class ${class_name} {
    //setup testClass variables here. For example:
    //RecordType Ids
    /*private static final Id Customer_RECORD_TYPE_ID = Schema.SObjectType.Account.getRecordTypeInfosByName().get('Customer').getRecordTypeId();*/

    //A common Loop Counter for bulikification purposes
    /*private static final integer MAX_LOOP_COUNTER = 200;*/

    @testSetup static void setupTestData(){
        //setup common test data here
        ${cursor}
    }

    static testMethod void testSetupData(){
        //use this method to validate that you have test data setup correctly
        //this will protect you from changes to workflow, process, and validation that break your test code!
    }

    static testMethod void myUnitTest() {
        // TO DO: implement unit test AFTER you have your test data all setup and verified as OK
    }
}

I always use an @testSetup, and then to defend myself from declarative changes, I use a testMethod to ensure that my test data was setup correctly! After those two pieces are working correctly, I'll start implmenting the actual test code.