Batch performance impact of Database.Stateful

Question

I know that a Batch will generally run faster when the Database.Stateful marker interface is not used.

What I don't know is, how badly the serializing and deserializing of state between each execute affects performance.

How much would it help to reduce the state to a minimum? Or are the costs nearly always the same? No matter how much data needs to maintained?

I am asking because I am running into heap limits when batching over a large collection. I could reduce the heap consumption by moving parts of the data passed into execute by putting it into the state.

Answer 1

Obviously Growing data in your stateful collections may lead to hit heap limits. A batch class as a whole can process 50 million records, 10,000 records per batch and only 10,000 batches per 24 hrs with a state heap limit of 6 MB.

BUT don't even try to go any close to these limits as they may slow it down to a slowest mode.

Some opinions:

1. Don't carry more than 10,000 records in each collection (list or map) during transition between states for "Heap size limits". Often use "Clear" to clear out data after being used.
2. You can also use finish method to call another batch for "CPU time limits"
3. You can use relation build data. for ex: Fetch one record from queryLocator and then fetch other related data on the basis of that record in the execute method. As each batch behaves as a separate program with heap being used as database for them.
4. Keep track of heap with every ending execute and maintain accordingly using this code: Integer startingHeap = Limits.getHeapSize(); Integer addedHeap = Limits.getHeapSize() - startingHeap; and keep removing from collection or saving it temporarily to some temporary Object when you are near to hit limits.