18

Background

We're trying to do a type of pseudo-parallel processing mechanism, where a record will be processed twice. The thought was to make an overlapping series of values that look like this:

|------- PHASE 1 -------|------- PHASE 2 --------|
null null null null null|0    1    2    3    4    // First Batch
0    1    2    3    4   |5    6    7    8    9    // Second Batch
5    6    7    8    9   |null null null null null // Third Batch

I'm accomplishing this pattern with a custom iterator that looks like this:

public class OverlapBatch implements Database.Batchable<Integer>, Iterable<Integer>, Iterator<Integer> {
    Integer[] values;

    public Iterator<Integer> iterator() {
        return this;
    }

    public Boolean hasNext() {
        return !values.isEmpty();
    }

    public Integer next() {
        return values.remove(0);
    }

    public Iterable<Integer> start(Database.BatchableContext bc) {
        values = new Integer[0];

        Integer[] temp = new Integer[0];

        for(Integer index = 0; index < 5; index++) {
            values.add(null);
        }

        for(Integer index = 0; index < 10; index++) {
            temp.add(index);
            if(temp.size() == 5) {
                values.addAll(temp);
                values.addAll(temp);
                temp.clear();
            }
        }

        for(Integer index = 0; index < 5; index++) {
            values.add(null);
        }
        return this;
    }

    public void execute(Database.BatchableContext bc, Integer[] scope) {
        System.debug(LoggingLevel.ERROR, scope.size());
        System.debug(LoggingLevel.ERROR, scope);
    }

    public void finish(Database.BatchableContext bc) {

    }
}

The only difference is that we're querying records and trying to overlay Id values instead, but even this code shows the problem we're having. The code is called like this:

Database.executeBatch(new OverlayBatch(), 10);

Executing the code, one would expect there to be three batches, but instead I only get two. Checking the logs for both of the batches, the logs show that null values are ignored:

// Batch 1
16:21:59.040 (40836245)|USER_DEBUG|[41]|ERROR|10
16:21:59.040 (40881446)|USER_DEBUG|[42]|ERROR|(0, 1, 2, 3, 4, 0, 1, 2, 3, 4)
// Batch 2
16:21:59.037 (37841546)|USER_DEBUG|[41]|ERROR|10
16:21:59.037 (37894244)|USER_DEBUG|[42]|ERROR|(5, 6, 7, 8, 9, 5, 6, 7, 8, 9)

The nulls disappear, and I'm left only with non-null values. This behavior seems undocumented, and is actually undesirable, since we're trying to process a two-phase system. Of course, I could use other techniques, but is there a clean way to just use nulls in this scenario?

  • 1
    Pity its not documented. I guess you'll have to resort to some non-null sentinel value e.g. -1. – Keith C Aug 26 '14 at 22:37
  • 2
    Yeah you can't have null values through the batch set. I would work a bit differently and use String instead of Integer, then cast them into the execute. At least with string you have a bit more control and can use a special tag like '-', '', 'mynullvalue' – Cloud Ninja Aug 26 '14 at 23:05
  • I tried this out in my development org. It was pretty strange. The only way I was able to maintain a null value is to basically make a Decorator for the Integer. I imagine you either tried that or would have wanted to avoid that. – Programmable Medley May 15 '15 at 1:53
  • Is this to avoid mixed dml operation or such? – bigassforce Aug 9 '15 at 16:37
  • 2
    @bigassforce No, it was to provide the ability to show some records as "in progress", basically by having two overlapping phases. I ultimately ended up scrapping that design and went for something that was more stable over time, since one of our goals was to not overwhelm the systems we were integrating with. – sfdcfox Aug 9 '15 at 21:55
4

The only way I was able to get a null Iterable was to decorate it with an actual class.

I made a simple custom Decorator class like this one.

global class BatchableInteger
{
    Integer BatachIntegerValue { get; private set; }
    public BatchableInteger(Integer integerInBatch) { BatachIntegerValue = integerInBatch; }
}

Then I used it like your example.

public class BatchOverlay implements Database.Batchable<BatchableInteger>, Iterable<BatchableInteger>, Iterator<BatchableInteger> 
{
    List<BatchableInteger> BatchIterables { get; private set; }

    public Iterator<BatchableInteger> iterator() { return this; }

    public Boolean hasNext() { return !BatchIterables.isEmpty(); }

    public BatchableInteger next() { return BatchIterables.remove(0); }

    public Iterable<BatchableInteger> start(Database.BatchableContext batchContext) 
    {
        BatchIterables = new List<BatchableInteger>();

        List<BatchableInteger> temporaryIntegerList = new List<BatchableInteger>();

        for(Integer index = 0; index < 5; index++) 
            BatchIterables.add(new BatchableInteger(null));

        for(Integer index = 0; index < 10; index++) 
        {
            temporaryIntegerList.add(new BatchableInteger(index));
            if(temporaryIntegerList.size() == 5) 
            {
                BatchIterables.addAll(temporaryIntegerList);
                BatchIterables.addAll(temporaryIntegerList);
                temporaryIntegerList.clear();
            }
        }

        for(Integer index = 0; index < 5; index++)
            BatchIterables.add(new BatchableInteger(null));

        return this;
    }

    public void execute(Database.BatchableContext batchConext, List<BatchableInteger> scope) 
    {
        System.debug(LoggingLevel.ERROR, scope.size());
        System.debug(LoggingLevel.ERROR, scope);
    }

    public void finish(Database.BatchableContext batchContext) { }
}

I imagine the loss of the null value has to do with the serialization process. It happens with JSON serialization too, so I imagine there is something similar going on.

1

If you plan on only ever utilizing positive integers in your code, you could replace null with -1. This would allow you to do what you are looking to do without resorting to a changing the values to strings.

If you need to use negative numbers, you will need to use a string and cast them to integers when you need them to be integers.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.