Does Test.stopTest() ensure a System.Schedule/Database.Batchable completes in a test?

Question

I am struggling to successfully test some code that is essentially the same as the code here Apex test class for schedulable class that checks for 5 batches running already. I can get the System.Schedule to be called but the scheduled code appears to never be run in the test.

The Apex Scheduler documentation (referenced in Does Test.stopTest() cause ALL waiting asynchronous functions to fire?) says:

Use the Test methods startTest and stopTest around the System.schedule method to ensure it finishes before continuing your test. All asynchronous calls made after the startTest method are collected by the system. When stopTest is executed, all asynchronous processes are run synchronously.

I created the separate test case shown below and can reproduce the problem there; test1 passes but test2 and test3 fail. (Using API 30.)

What am I missing here?

Test:

@isTest
private class ScheduleBatchTest {
    private enum TestCase {
        DatabaseExecuteBatch,
        ScheduleBatch,
        Schedule
    }
    @isTest
    static void test1() {
        execute(TestCase.DatabaseExecuteBatch);
    }
    @isTest
    static void test2() {
        execute(TestCase.ScheduleBatch);
    }
    @isTest
    static void test3() {
        execute(TestCase.Schedule);
    }
    private static void execute(TestCase tc) {
        Account[] accounts = new Account[] {
                new Account(Name = 'Acme'),
                new Account(Name = 'Nike')
                };
        insert accounts;
        Test.startTest();
        if (tc == TestCase.DatabaseExecuteBatch) {
            Database.executeBatch(new BatchableExecutorTestBatchable());
        } else if (tc == TestCase.ScheduleBatch) {
            System.scheduleBatch(new BatchableExecutorTestBatchable(),
                    'my description', 5, 100);
        } else if (tc == TestCase.Schedule) {
            System.schedule('my job', '0 0 13 * * ?', new ScheduledBatchable());
        }
        Test.stopTest();
        Account[] actuals = [select Site from Account where Id in :accounts];
        System.assertEquals(accounts.size(), actuals.size());
        for (Account actual : actuals) {
            System.assertEquals('executed', actual.Site);
        }
    }
    private class ScheduledBatchable implements Schedulable {
        public void execute(SchedulableContext sc) {
            Database.executeBatch(new BatchableExecutorTestBatchable(), 100);
        }
    }
}

Test batchable:

public class BatchableExecutorTestBatchable implements
        Database.Batchable<SObject>, Database.Stateful { 
    public Database.QueryLocator start(Database.BatchableContext bc) {
        Database.QueryLocator ql = Database.getQueryLocator([
                select Name from Account order by Name
                ]);
        insert new Account(Name = 'start');
        return ql;
    }
    public void execute(Database.BatchableContext bc, List<SObject> scope) {
        for (SObject sob : scope) {
            Account a = (Account) sob;
            a.Site = 'executed';
        }
        update scope;
        insert new Account(Name = 'execute');
    }
    public void finish(Database.BatchableContext bc) {
        insert new Account(Name = 'finish');
    }
}

PS

Replacing the asserts after the Test.stopTest() with this:

System.debug(tc + ' ' + [select ApexClassId, JobType, Status from AsyncApexJob]);

resulted in this debug output showing two flavours of not completed AsyncApexJob:

• DatabaseExecuteBatch (AsyncApexJob:{JobType=BatchApex, Status=Completed, Id=7075000000acpcpAAA, ApexClassId=01p500000005gPGAAY}, AsyncApexJob:{JobType=BatchApexWorker, Status=Completed, Id=7075000000acpcqAAA, ApexClassId=01p500000005gPGAAY})
• ScheduleBatch (AsyncApexJob:{JobType=BatchApex, Status=Queued, Id=7075000000acpcsAAA, ApexClassId=01p500000005gPGAAY})
• Schedule (AsyncApexJob:{JobType=ScheduledApex, Status=Queued, Id=7075000000acpcuAAA, ApexClassId=01p500000005wM3AAI})

Answer 1

Here's the problem from the docs:

When testing your batch Apex, you can test only one execution of the execute method. You can use the scope parameter of the executeBatch method to limit the number of records passed into the execute method to ensure that you aren't running into governor limits.

Additionally I don't think batch classes are executed from schedulable classes in test classes but I couldn't find that supporting documentation. From my own experience, testing schedulable batch methods only queues the batches while testing the batch method execution actually runs the batch method. So the asserts after your schedulable tests will always fail if they expect a batch job to complete.

Basically, this means you should test the batch and schedulable classes separately. Once you remove the asserts from the schedulable tests cases everything should work fine and it looks like you'll have full coverage.

Also, you only need one '@isTest' line at the top of your test class. I'm also assuming that the schedulable method at the botton of the test class is not a part of that test class but just a mockup of a standalone class used during production. Just in case, here are how the separate classes should be written:

Test Class

@isTest
private class ScheduleBatchTest {
private enum TestCase {
    DatabaseExecuteBatch,
    ScheduleBatch,
    Schedule
}
static void test1() {
    execute(TestCase.DatabaseExecuteBatch);
}
static void test2() {
    execute(TestCase.ScheduleBatch);
}
static void test3() {
    execute(TestCase.Schedule);
}
private static void execute(TestCase tc) {
    Account[] accounts = new Account[] {
            new Account(Name = 'Acme'),
            new Account(Name = 'Nike')
            };
    insert accounts;
    Test.startTest();
    if (tc == TestCase.DatabaseExecuteBatch) {
        Database.executeBatch(new BatchableExecutorTestBatchable());
        Account[] actuals = [select Site from Account where Id in :accounts];
        System.assertEquals(accounts.size(), actuals.size());
        for (Account actual : actuals) {
            System.assertEquals('executed', actual.Site);
        }
    } else if (tc == TestCase.ScheduleBatch) {
        System.scheduleBatch(new BatchableExecutorTestBatchable(),
                'my description', 5, 100);
        //assert batch  job queued here
    } else if (tc == TestCase.Schedule) {
        System.schedule('my job', '0 0 13 * * ?', new ScheduledBatchable());
        //assert scheduled job queued here
    }
    Test.stopTest();        
}

Batch Class:

public class BatchableExecutorTestBatchable implements Database.Batchable<SObject>, Database.Stateful { 
    public Database.QueryLocator start(Database.BatchableContext bc) {
        Database.QueryLocator ql = Database.getQueryLocator([
            select Name from Account order by Name
            ]);
        insert new Account(Name = 'start');
        return ql;
    }
    public void execute(Database.BatchableContext bc, List<SObject> scope) {
        for (SObject sob : scope) {
            Account a = (Account) sob;
            a.Site = 'executed';
        }
        update scope;
        insert new Account(Name = 'execute');
    }
    public void finish(Database.BatchableContext bc) {
        insert new Account(Name = 'finish');
    }
}

Schedulable Class:

private class ScheduledBatchable implements Schedulable {
    public void execute(SchedulableContext sc) {
        Database.executeBatch(new BatchableExecutorTestBatchable(), 100);
    }
}

Let me know if this works.

Answer 2

I thought I'd add to Jomtung's answer.

Additionally I don't think batch classes are executed from schedulable classes in test classes

Methods annotated with @testSetup seem to end up running the whole flow.

I'm using this pattern for my batch apex - http://advancedapex.com/dreamforce13

I have an Apex trigger that schedules a scheduleable 1 second in the future; the scheduleable enqueues my batch job. In my testSetup method I insert an account and receive the expected results. However when I try the same inside of a regular testMethod, the execution makes it to the point of calling Database.executeBatch. After that though, the Test.stopTest() sees that the Scheduleable is done executing and the testmethod resumes.

The same thing happens in an @testSetup annotated method, Test.stopTest() only causes it to wait for the Scheduleable to finish up. The setup method itself finishes executing as well. However, after the setup method finishes, the batch job runs!