Consider the following code:
Integer repeatSize = 6000000;
string csvFile1 = '*'.repeat(repeatSize);
string csvFile2 = '*'.repeat(repeatSize);
string csvFile3 = '*'.repeat(repeatSize);
string csvFile4 = '*'.repeat(repeatSize);
string csvFile5 = '*'.repeat(repeatSize);
string csvFile6 = '*'.repeat(repeatSize);
string csvFile7 = '*'.repeat(repeatSize);
string csvFile8 = '*'.repeat(repeatSize);
System.debug(Limits.getHeapSize());
Here, we allocate 48,001,066 bytes without an exception (yes, in Execute Anonymous).
You see, checking heap limits every time something is constructed would be too expensive, so it only happens during garbage collection, which occurs only periodically. If I add a ninth line at the top, you'll see this in the logs:
17:46:53.1 (51330206)|HEAP_ALLOCATE|[10]|Bytes:6000000
17:46:53.1 (51639211)|EXCEPTION_THROWN|[EXTERNAL]|System.LimitException: Apex heap size too large: 54001056
17:46:53.1 (51933182)|SYSTEM_METHOD_EXIT|[10]|String.repeat(Integer)
17:46:53.1 (52044530)|FATAL_ERROR|System.LimitException: Apex heap size too large: 54001056
You'll see that the system finally noticed we violated the limits and did something about it. Also notice that the error is generated after the HEAP_ALLOCATE, but before the VARIABLE_ASSIGNMENT. In other words, if you're over the limit after the allocation phase, you may get the exception. The limit check always occurs after the last allocation and everything is garbage collected.
You can easily construct a string all the way up to near the maximum size by managing how your allocations are performed. The following code runs just fine:
string csvFile = '"XXXXXXX","XXPX XXRXXXX","XXPX BP","XLXXXXR","XXXXVXXX PRXXXDXXXX","XXPX XXXXVXXX","XDRXXXDXX LXGXLX","XXXXRX XXLX","LXXXX XX","DXXX XXXZXX XXXXVXXX","DXXX XXXX XXXXVXXX","XXDXXX BP","XXDXXX XXXXXLX","PXRXXXX XVX","DXXXXXXXZXXXX","XXXX XXRXXXXRX","XXGXXXX XXRXXXXRX","DXXX DX XXXXXXX","LXXGX DX XXXXXXX","PXD","XXXXX XXXXRXXXXXLX","XXXX RXXXPXXX","XXDXRXZZX RXXXPXXX","PRXVXXXXX RXXXPXXX","XXXXXX RXXXPXXX","XXP RXXXPXXX","XXXXRX XXLXXXXX 1","XXXXRX XXLXXXXX 2","XXXXRX XXX","XXDXRXZZX XXXXL","XXDXRXZZX XXRXXXXRX","XXXXXX XXRXXXXRX","PRXVXXXXX XXRXXXXRX","XXP XXRXXXXRX","XXXXX XLXXXXX","XXDXXX PRXXXXX","DXXX XXXXXZXXXX","XXPX DXXXXXXXX","XHXXVX XXXXXRX","DXXX XXXXXXXXX XXXXXRX","DXXX XXXDXXZX XXXXXRX","XXPXRXX XXXXLX","XXPXRXX RXXXDXX XXXXDXXX","PRXXXXXXXXXXXXXXZXXXX PRXXXXXXXXXX","DXXX PRXXXXXXXXXXXXXXZXXXX PRXXXXXXXXXX"';
string[] csvFile2= new string[0];
String csvFile3;
system.debug(limits.getLimitHeapSize());
system.debug(limits.getHeapSize());
for(integer i=0;i<6144;i++){
csvFile2.add(csvFile+i);
}
csvFile3 = String.join(csvFile2, '\n');
csvFile2 = null;
system.debug(1.0*limits.getHeapSize());
system.debug((1.0*limits.getHeapSize())/limits.getLimitHeapSize()*100);
Here, we allocate only small chunks of memory right until the end, therefore reducing the probability that we'll get a heap limit exception, despite using 85% of the available heap.
The problem you're having is that when you use the + operator on a string, you have to allocate a temporary string the size of both the left and right sides; if you're doing addition on strings that are over 3,000,000 characters, you run the risk of a heap allocation causing an exception.
Conversely, if you use an array to join pieces together, you avoid the large allocation until the end, so you're much more likely not to exceed the heap limit as long as you're careful. Notice how I immediately dump the contents of the array once I perform the join. This is to reduce the likelihood that I'll hit the heap limit exception.
