Whilst parsing text files using Batch Apex, we retrieved an Heap Size error. The Heap Size limit happens before our custom Iterator. Because we parse an attachment, change it into a Blob and then retrieve 1 large String value. This transformation from Blob to String is providing the issue. The Blob itself has a very small Heap size but when we transform it to a String, the heap size easily quadruples. Is there a way to create an Iterator on a Blob object? I'm assuming that this is not possible since we cannot use any 'split' functions on a Blob file.

Any ideas?

  • 1
    Could you share the relevant part of your code and tell if even a single blob/string-conversion is exceeding the limit, or the ongoing conversion by the iteration? You say "before our custom Iterator" which might lead me to the assumtion that even the first conversion blows the limit. But I want to avoid blind guessing here. – Uwe Heim Feb 6 '15 at 10:41
  • Hi Uwe, correct. In the constructor of the Batch class, we perform the conversion from Blob to String. This conversion is blowing up the Heap size. – Robin Wijnen Feb 6 '15 at 11:07

If the limit would be blown by a repetitive conversion, it's important to make sure that big string falls out of scope before the next iteration to prevent a growing heap consumption.

If even a single conversion blows the heap-size limit, there is not much you can do programmatically.

You could try one feature what I remember but never used called "Future Methods with Higher Limits (Pilot)".

It allows you to do stuff like

public static void myFutureMethod() {
    // Your code here

With that feature you might be able to move the conversion into a future call. But without deep look into you code, I can't tell if it's really possible in your situation.

The feature seems to be still in pilot. If that is true, you need to apply (via support, you AE or evangelist) to get it activated. Not guaranteed that you get it.

| improve this answer | |
  • Hi Uwe, great tip! But I believe you cannot call an @ future method from a batch class? Maybe a solution would be to first (not use the batch class) use an @ future method which converses the Blob to a String. And parse that string (using a .split with a regex), retrieving a list<String> and store that towards a custom Table. Some kind of 'ODS' table. And then using the ODS table to fetch the records, according to the batch size and do our logic. However, not sure if the first @future method will be able to overcome the limit. – Robin Wijnen Feb 6 '15 at 12:40
  • n.b. regex using splits on large Strings frequently run into 'regex too complicated error' – cropredy Feb 6 '15 at 20:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.