4

i just made a test, to see how much data i can put in a string before i reach the the heapsize limit and i get an unexpected behaviour

string  csvFile = '"XXXXXXX","XXPX XXRXXXX","XXPX BP","XLXXXXR","XXXXVXXX PRXXXDXXXX","XXPX XXXXVXXX","XDRXXXDXX LXGXLX","XXXXRX XXLX","LXXXX XX","DXXX XXXZXX XXXXVXXX","DXXX XXXX XXXXVXXX","XXDXXX BP","XXDXXX XXXXXLX","PXRXXXX XVX","DXXXXXXXZXXXX","XXXX XXRXXXXRX","XXGXXXX XXRXXXXRX","DXXX DX XXXXXXX","LXXGX DX XXXXXXX","PXD","XXXXX XXXXRXXXXXLX","XXXX RXXXPXXX","XXDXRXZZX RXXXPXXX","PRXVXXXXX RXXXPXXX","XXXXXX RXXXPXXX","XXP RXXXPXXX","XXXXRX XXLXXXXX 1","XXXXRX XXLXXXXX 2","XXXXRX XXX","XXDXRXZZX XXXXL","XXDXRXZZX XXRXXXXRX","XXXXXX XXRXXXXRX","PRXVXXXXX XXRXXXXRX","XXP XXRXXXXRX","XXXXX XLXXXXX","XXDXXX PRXXXXX","DXXX XXXXXZXXXX","XXPX DXXXXXXXX","XHXXVX XXXXXRX","DXXX XXXXXXXXX XXXXXRX","DXXX XXXDXXZX XXXXXRX","XXPXRXX XXXXLX","XXPXRXX RXXXDXX XXXXDXXX","PRXXXXXXXXXXXXXXZXXXX PRXXXXXXXXXX","DXXX PRXXXXXXXXXXXXXXZXXXX PRXXXXXXXXXX"';

string  csvFile2='',csvFile3;
system.debug(limits.getLimitHeapSize());
system.debug(limits.getHeapSize());
for(integer i=0;i<3618;i++){
csvFile2+='\n'+csvFile;
}


system.debug(1.0*limits.getHeapSize());
system.debug((1.0*limits.getHeapSize())/limits.getLimitHeapSize()*100);

with 3618 iteraction i get

  • Heap size 3001245.0 Which correspond to ~50% of the heap size limit in the Synchronous context

but if i go to 3619 iteractions in the for, i get the error "Apex heap size too large: 6001368".... which surprise me a lot...

I'm not getting out why this behaviour is occurring. Do you have any similar experience? There is a reason for this?

Seems more about limits on strings, in effect the string csvFile2 overcome 3M characters... but i dont know

3

Consider the following code:

Integer repeatSize = 6000000;
string  csvFile1 = '*'.repeat(repeatSize);
string  csvFile2 = '*'.repeat(repeatSize);
string  csvFile3 = '*'.repeat(repeatSize);
string  csvFile4 = '*'.repeat(repeatSize);
string  csvFile5 = '*'.repeat(repeatSize);
string  csvFile6 = '*'.repeat(repeatSize);
string  csvFile7 = '*'.repeat(repeatSize);
string  csvFile8 = '*'.repeat(repeatSize);
System.debug(Limits.getHeapSize());

Here, we allocate 48,001,066 bytes without an exception (yes, in Execute Anonymous).

You see, checking heap limits every time something is constructed would be too expensive, so it only happens during garbage collection, which occurs only periodically. If I add a ninth line at the top, you'll see this in the logs:

17:46:53.1 (51330206)|HEAP_ALLOCATE|[10]|Bytes:6000000
17:46:53.1 (51639211)|EXCEPTION_THROWN|[EXTERNAL]|System.LimitException: Apex heap size too large: 54001056
17:46:53.1 (51933182)|SYSTEM_METHOD_EXIT|[10]|String.repeat(Integer)
17:46:53.1 (52044530)|FATAL_ERROR|System.LimitException: Apex heap size too large: 54001056

You'll see that the system finally noticed we violated the limits and did something about it. Also notice that the error is generated after the HEAP_ALLOCATE, but before the VARIABLE_ASSIGNMENT. In other words, if you're over the limit after the allocation phase, you may get the exception. The limit check always occurs after the last allocation and everything is garbage collected.

You can easily construct a string all the way up to near the maximum size by managing how your allocations are performed. The following code runs just fine:

string  csvFile = '"XXXXXXX","XXPX XXRXXXX","XXPX BP","XLXXXXR","XXXXVXXX PRXXXDXXXX","XXPX XXXXVXXX","XDRXXXDXX LXGXLX","XXXXRX XXLX","LXXXX XX","DXXX XXXZXX XXXXVXXX","DXXX XXXX XXXXVXXX","XXDXXX BP","XXDXXX XXXXXLX","PXRXXXX XVX","DXXXXXXXZXXXX","XXXX XXRXXXXRX","XXGXXXX XXRXXXXRX","DXXX DX XXXXXXX","LXXGX DX XXXXXXX","PXD","XXXXX XXXXRXXXXXLX","XXXX RXXXPXXX","XXDXRXZZX RXXXPXXX","PRXVXXXXX RXXXPXXX","XXXXXX RXXXPXXX","XXP RXXXPXXX","XXXXRX XXLXXXXX 1","XXXXRX XXLXXXXX 2","XXXXRX XXX","XXDXRXZZX XXXXL","XXDXRXZZX XXRXXXXRX","XXXXXX XXRXXXXRX","PRXVXXXXX XXRXXXXRX","XXP XXRXXXXRX","XXXXX XLXXXXX","XXDXXX PRXXXXX","DXXX XXXXXZXXXX","XXPX DXXXXXXXX","XHXXVX XXXXXRX","DXXX XXXXXXXXX XXXXXRX","DXXX XXXDXXZX XXXXXRX","XXPXRXX XXXXLX","XXPXRXX RXXXDXX XXXXDXXX","PRXXXXXXXXXXXXXXZXXXX PRXXXXXXXXXX","DXXX PRXXXXXXXXXXXXXXZXXXX PRXXXXXXXXXX"';

string[]  csvFile2= new string[0];
String csvFile3;
system.debug(limits.getLimitHeapSize());
system.debug(limits.getHeapSize());
for(integer i=0;i<6144;i++){
csvFile2.add(csvFile+i);
}
csvFile3 = String.join(csvFile2, '\n');
csvFile2 = null;

system.debug(1.0*limits.getHeapSize());
system.debug((1.0*limits.getHeapSize())/limits.getLimitHeapSize()*100);

Here, we allocate only small chunks of memory right until the end, therefore reducing the probability that we'll get a heap limit exception, despite using 85% of the available heap.

The problem you're having is that when you use the + operator on a string, you have to allocate a temporary string the size of both the left and right sides; if you're doing addition on strings that are over 3,000,000 characters, you run the risk of a heap allocation causing an exception.

Conversely, if you use an array to join pieces together, you avoid the large allocation until the end, so you're much more likely not to exceed the heap limit as long as you're careful. Notice how I immediately dump the contents of the array once I perform the join. This is to reduce the likelihood that I'll hit the heap limit exception.

1
  • thanks for the explanation... i was not able to figure out that.
    – Klodj_Meta
    Jul 15 '17 at 16:56
0

Speculation, but a += perhaps compiles down to these steps:

  1. csvFile2 holds reference to a 3M string
  2. create new string (the right hand side of the expression) of csvFile2 + '\n' + csvFile which is also a 3M (plus a bit) string; this forces the garbage collector to run to obtain the space (as it's a big allocation) but cvFile2 is still holding a reference to the 3M string so that can't be garbage collected
  3. assign the new 3M (plus a bit) string to csvfile2 which means the reference to the original 3M string has now been dropped so that string is eligible to be garbage collected

So at the end of step 2, 6M (plus a bit) of heap is needed so the limit exception is thrown; it isn't until step 3 that the original 3M string could be garbage collected but that is never reached.

Don't think there is a way round this; just live with say a 2.5M limit (to leave headroom for other objects).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.