9

I ran the below code to understand where this code breaks for heap size. Our use case is we have quote lines and we need to send it to external system for printing in documents. we can have 1000's of quote lines with sales text.

public static string testheap(integer loopruntimes){
        Apttus_Proposal__Proposal_Line_Item__c heapobj= [select id,APTS_Full_Description_Sales__c from
                                                         Apttus_Proposal__Proposal_Line_Item__c 
                                                         where id = 'aHG6s0000001Kf9GAE'];
         string finalstring = '';
        string heapinfo = '';
        heapinfo = heapinfo + 'CurrentMemory:'+ Limits.getHeapSize() + 'Allocated memory:' + Limits.getLimitHeapSize() + ' \n';
        for (integer i=0;i<loopruntimes;i++){
            
            finalstring = finalstring +  heapobj.APTS_Full_Description_Sales__c;
            heapinfo = heapinfo + 'Currentiteration:'+i + ' CurrentMemory:'+ Limits.getHeapSize() + 'Allocated memory:' + Limits.getLimitHeapSize() + ' \n';
        } 

The APTS_Full_Description_Sales__c is a text area field can contain upto 131072. in this case when i save its contents to text file it contains 375 kb.

When i trace heap below is what i get:

heapinfoCurrentMemory:18742Allocated memory:6000000 
Currentiteration:0 CurrentMemory:36222Allocated memory:6000000 
Currentiteration:1 CurrentMemory:53690Allocated memory:6000000 
Currentiteration:2 CurrentMemory:71158Allocated memory:6000000 
Currentiteration:3 CurrentMemory:88626Allocated memory:6000000 
Currentiteration:4 CurrentMemory:106094Allocated memory:6000000 
Currentiteration:5 CurrentMemory:123563Allocated memory:6000000 
Currentiteration:6 CurrentMemory:141032Allocated memory:6000000 
Currentiteration:7 CurrentMemory:158501Allocated memory:6000000 
Currentiteration:8 CurrentMemory:175970Allocated memory:6000000 
Currentiteration:9 CurrentMemory:193439Allocated memory:6000000 
Currentiteration:10 CurrentMemory:210909Allocated memory:6000000 
Currentiteration:11 CurrentMemory:228379Allocated memory:6000000 
Currentiteration:12 CurrentMemory:245849Allocated memory:6000000 
Currentiteration:13 CurrentMemory:263319Allocated memory:6000000 
Currentiteration:14 CurrentMemory:280789Allocated memory:6000000 
Currentiteration:15 CurrentMemory:298259Allocated memory:6000000 
Currentiteration:16 CurrentMemory:315729Allocated memory:6000000 
Currentiteration:17 CurrentMemory:333199Allocated memory:6000000 
Currentiteration:18 CurrentMemory:350669Allocated memory:6000000 
Currentiteration:19 CurrentMemory:368139Allocated memory:6000000 
Currentiteration:20 CurrentMemory:385609Allocated memory:6000000 
Currentiteration:21 CurrentMemory:403079Allocated memory:6000000 
Currentiteration:22 CurrentMemory:420549Allocated memory:6000000 
Currentiteration:23 CurrentMemory:438019Allocated memory:6000000 
Currentiteration:24 CurrentMemory:455489Allocated memory:6000000 
Currentiteration:25 CurrentMemory:472959Allocated memory:6000000 
Currentiteration:26 CurrentMemory:490429Allocated memory:6000000 
Currentiteration:27 CurrentMemory:507899Allocated memory:6000000 
Currentiteration:28 CurrentMemory:525369Allocated memory:6000000 
Currentiteration:29 CurrentMemory:542839Allocated memory:6000000 
Currentiteration:30 CurrentMemory:560309Allocated memory:6000000 
Currentiteration:31 CurrentMemory:577779Allocated memory:6000000 
Currentiteration:32 CurrentMemory:595249Allocated memory:6000000 
Currentiteration:33 CurrentMemory:612719Allocated memory:6000000 
Currentiteration:34 CurrentMemory:630189Allocated memory:6000000 
Currentiteration:35 CurrentMemory:647659Allocated memory:6000000 
Currentiteration:36 CurrentMemory:665129Allocated memory:6000000 
Currentiteration:37 CurrentMemory:682599Allocated memory:6000000 
Currentiteration:38 CurrentMemory:700069Allocated memory:6000000 
Currentiteration:39 CurrentMemory:717539Allocated memory:6000000 
Currentiteration:40 CurrentMemory:735009Allocated memory:6000000 
Currentiteration:41 CurrentMemory:752479Allocated memory:6000000 
Currentiteration:42 CurrentMemory:769949Allocated memory:6000000 
Currentiteration:43 CurrentMemory:787419Allocated memory:6000000 
Currentiteration:44 CurrentMemory:804889Allocated memory:6000000 
Currentiteration:45 CurrentMemory:822359Allocated memory:6000000 
Currentiteration:46 CurrentMemory:839829Allocated memory:6000000 
Currentiteration:47 CurrentMemory:857299Allocated memory:6000000 
Currentiteration:48 CurrentMemory:874769Allocated memory:6000000 
Currentiteration:49 CurrentMemory:892239Allocated memory:6000000 
Currentiteration:50 CurrentMemory:909709Allocated memory:6000000 
Currentiteration:51 CurrentMemory:927179Allocated memory:6000000 
Currentiteration:52 CurrentMemory:944649Allocated memory:6000000 
Currentiteration:53 CurrentMemory:962119Allocated memory:6000000 
Currentiteration:54 CurrentMemory:979589Allocated memory:6000000 
Currentiteration:55 CurrentMemory:997059Allocated memory:6000000 
Currentiteration:56 CurrentMemory:1014529Allocated memory:6000000 
Currentiteration:57 CurrentMemory:1032000Allocated memory:6000000 
Currentiteration:58 CurrentMemory:1049471Allocated memory:6000000 
Currentiteration:59 CurrentMemory:1066942Allocated memory:6000000 
Currentiteration:60 CurrentMemory:1084413Allocated memory:6000000 
Currentiteration:61 CurrentMemory:1101884Allocated memory:6000000 
Currentiteration:62 CurrentMemory:1119355Allocated memory:6000000 
Currentiteration:63 CurrentMemory:1136826Allocated memory:6000000 
Currentiteration:64 CurrentMemory:1154297Allocated memory:6000000 
Currentiteration:65 CurrentMemory:1171768Allocated memory:6000000 
Currentiteration:66 CurrentMemory:1189239Allocated memory:6000000 
Currentiteration:67 CurrentMemory:1206710Allocated memory:6000000 
Currentiteration:68 CurrentMemory:1224181Allocated memory:6000000 
Currentiteration:69 CurrentMemory:1241652Allocated memory:6000000 
Currentiteration:70 CurrentMemory:1259123Allocated memory:6000000 
Currentiteration:71 CurrentMemory:1276594Allocated memory:6000000 
Currentiteration:72 CurrentMemory:1294065Allocated memory:6000000 
Currentiteration:73 CurrentMemory:1311536Allocated memory:6000000 
Currentiteration:74 CurrentMemory:1329007Allocated memory:6000000 
Currentiteration:75 CurrentMemory:1346478Allocated memory:6000000 
Currentiteration:76 CurrentMemory:1363949Allocated memory:6000000 
Currentiteration:77 CurrentMemory:1381420Allocated memory:6000000 
Currentiteration:78 CurrentMemory:1398891Allocated memory:6000000 
Currentiteration:79 CurrentMemory:1416362Allocated memory:6000000 
Currentiteration:80 CurrentMemory:1433833Allocated memory:6000000 
Currentiteration:81 CurrentMemory:1451304Allocated memory:6000000 
Currentiteration:82 CurrentMemory:1468775Allocated memory:6000000 
Currentiteration:83 CurrentMemory:1486246Allocated memory:6000000 
Currentiteration:84 CurrentMemory:1503717Allocated memory:6000000 
Currentiteration:85 CurrentMemory:1521188Allocated memory:6000000 
Currentiteration:86 CurrentMemory:1538659Allocated memory:6000000 
Currentiteration:87 CurrentMemory:1556130Allocated memory:6000000 
Currentiteration:88 CurrentMemory:1573601Allocated memory:6000000 
Currentiteration:89 CurrentMemory:1591072Allocated memory:6000000 
Currentiteration:90 CurrentMemory:1608543Allocated memory:6000000 
Currentiteration:91 CurrentMemory:1626014Allocated memory:6000000 
Currentiteration:92 CurrentMemory:1643485Allocated memory:6000000 
Currentiteration:93 CurrentMemory:1660956Allocated memory:6000000 
Currentiteration:94 CurrentMemory:1678427Allocated memory:6000000 
Currentiteration:95 CurrentMemory:1695898Allocated memory:6000000 
Currentiteration:96 CurrentMemory:1713369Allocated memory:6000000 
Currentiteration:97 CurrentMemory:1730840Allocated memory:6000000 
Currentiteration:98 CurrentMemory:1748311Allocated memory:6000000 
Currentiteration:99 CurrentMemory:1765782Allocated memory:6000000 
Currentiteration:100 CurrentMemory:1783254Allocated memory:6000000 
Currentiteration:101 CurrentMemory:1800726Allocated memory:6000000 
Currentiteration:102 CurrentMemory:1818198Allocated memory:6000000 
Currentiteration:103 CurrentMemory:1835670Allocated memory:6000000 
Currentiteration:104 CurrentMemory:1853142Allocated memory:6000000 
Currentiteration:105 CurrentMemory:1870614Allocated memory:6000000 
Currentiteration:106 CurrentMemory:1888086Allocated memory:6000000 
Currentiteration:107 CurrentMemory:1905558Allocated memory:6000000 
Currentiteration:108 CurrentMemory:1923030Allocated memory:6000000 
Currentiteration:109 CurrentMemory:1940502Allocated memory:6000000 
Currentiteration:110 CurrentMemory:1957974Allocated memory:6000000 
Currentiteration:111 CurrentMemory:1975446Allocated memory:6000000 
Currentiteration:112 CurrentMemory:1992918Allocated memory:6000000 
Currentiteration:113 CurrentMemory:2010390Allocated memory:6000000 
Currentiteration:114 CurrentMemory:2027862Allocated memory:6000000 
Currentiteration:115 CurrentMemory:2045334Allocated memory:6000000 
Currentiteration:116 CurrentMemory:2062806Allocated memory:6000000 
Currentiteration:117 CurrentMemory:2080278Allocated memory:6000000 
Currentiteration:118 CurrentMemory:2097750Allocated memory:6000000 
Currentiteration:119 CurrentMemory:2115222Allocated memory:6000000 
Currentiteration:120 CurrentMemory:2132694Allocated memory:6000000 
Currentiteration:121 CurrentMemory:2150166Allocated memory:6000000 
Currentiteration:122 CurrentMemory:2167638Allocated memory:6000000 
Currentiteration:123 CurrentMemory:2185110Allocated memory:6000000 
Currentiteration:124 CurrentMemory:2202582Allocated memory:6000000 
Currentiteration:125 CurrentMemory:2220054Allocated memory:6000000 
Currentiteration:126 CurrentMemory:2237526Allocated memory:6000000 
Currentiteration:127 CurrentMemory:2254998Allocated memory:6000000 
Currentiteration:128 CurrentMemory:2272470Allocated memory:6000000 
Currentiteration:129 CurrentMemory:2289942Allocated memory:6000000 
Currentiteration:130 CurrentMemory:2307414Allocated memory:6000000 
Currentiteration:131 CurrentMemory:2324886Allocated memory:6000000 
Currentiteration:132 CurrentMemory:2342358Allocated memory:6000000 
Currentiteration:133 CurrentMemory:2359830Allocated memory:6000000 
Currentiteration:134 CurrentMemory:2377302Allocated memory:6000000 
Currentiteration:135 CurrentMemory:2394774Allocated memory:6000000 
Currentiteration:136 CurrentMemory:2412246Allocated memory:6000000 
Currentiteration:137 CurrentMemory:2429718Allocated memory:6000000 
Currentiteration:138 CurrentMemory:2447190Allocated memory:6000000 
Currentiteration:139 CurrentMemory:2464662Allocated memory:6000000 
Currentiteration:140 CurrentMemory:2482134Allocated memory:6000000 
Currentiteration:141 CurrentMemory:2499606Allocated memory:6000000 
Currentiteration:142 CurrentMemory:2517078Allocated memory:6000000 
Currentiteration:143 CurrentMemory:2534550Allocated memory:6000000 
Currentiteration:144 CurrentMemory:2552022Allocated memory:6000000 
Currentiteration:145 CurrentMemory:2569494Allocated memory:6000000 
Currentiteration:146 CurrentMemory:2586966Allocated memory:6000000 
Currentiteration:147 CurrentMemory:2604438Allocated memory:6000000 
Currentiteration:148 CurrentMemory:2621910Allocated memory:6000000 
Currentiteration:149 CurrentMemory:2639382Allocated memory:6000000 
Currentiteration:150 CurrentMemory:2656854Allocated memory:6000000 
Currentiteration:151 CurrentMemory:2674326Allocated memory:6000000 
Currentiteration:152 CurrentMemory:2691798Allocated memory:6000000 
Currentiteration:153 CurrentMemory:2709270Allocated memory:6000000 
Currentiteration:154 CurrentMemory:2726742Allocated memory:6000000 
Currentiteration:155 CurrentMemory:2744214Allocated memory:6000000 
Currentiteration:156 CurrentMemory:2761686Allocated memory:6000000 
Currentiteration:157 CurrentMemory:2779158Allocated memory:6000000 
Currentiteration:158 CurrentMemory:2796630Allocated memory:6000000 
Currentiteration:159 CurrentMemory:2814102Allocated memory:6000000 
Currentiteration:160 CurrentMemory:2831574Allocated memory:6000000 
Currentiteration:161 CurrentMemory:2849046Allocated memory:6000000 
Currentiteration:162 CurrentMemory:2866518Allocated memory:6000000 
Currentiteration:163 CurrentMemory:2883990Allocated memory:6000000 
Currentiteration:164 CurrentMemory:2901462Allocated memory:6000000 
Currentiteration:165 CurrentMemory:2918934Allocated memory:6000000 
Currentiteration:166 CurrentMemory:2936406Allocated memory:6000000 
Currentiteration:167 CurrentMemory:2953878Allocated memory:6000000 
Currentiteration:168 CurrentMemory:2971350Allocated memory:6000000 
Currentiteration:169 CurrentMemory:2988822Allocated memory:6000000 
Currentiteration:170 CurrentMemory:3006294Allocated memory:6000000 
Currentiteration:171 CurrentMemory:3023766Allocated memory:6000000 
Currentiteration:172 CurrentMemory:3041238Allocated memory:6000000 
Currentiteration:173 CurrentMemory:3058710Allocated memory:6000000 
Currentiteration:174 CurrentMemory:3076182Allocated memory:6000000 
Currentiteration:175 CurrentMemory:3093654Allocated memory:6000000 
Currentiteration:176 CurrentMemory:3111126Allocated memory:6000000 
Currentiteration:177 CurrentMemory:3128598Allocated memory:6000000 
Currentiteration:178 CurrentMemory:3146070Allocated memory:6000000 
Currentiteration:179 CurrentMemory:3163542Allocated memory:6000000 

09:18:33.1 (188576648)|SYSTEM_METHOD_EXIT|[33]|System.debug(ANY)
09:18:33.1 (188591730)|METHOD_EXIT|[1]|01p6s0000005JDY|VatsanExstreamDAOTestPOC1.testheap(Integer)
09:18:33.1 (188611000)|SYSTEM_MODE_EXIT|false
09:18:33.188 (188689643)|CUMULATIVE_LIMIT_USAGE
09:18:33.188 (188689643)|LIMIT_USAGE_FOR_NS|(default)|
  Number of SOQL queries: 1 out of 100
  Number of query rows: 1 out of 50000
  Number of SOSL queries: 0 out of 20
  Number of DML statements: 0 out of 150
  Number of Publish Immediate DML: 0 out of 150
  Number of DML rows: 0 out of 10000
  Maximum CPU time: 0 out of 25000
  Maximum heap size: 5895187 out of 6000000 ******* CLOSE TO LIMIT

My Question is eventhough heap size at 180th time is only 3163542 why does the overall heap near 6 MB. if i increase the loop variable to 185 the code breaks with heap size limit?

  1. Any insights on how heap is calcuated will be very helpful?
  2. We need to send this data to external system for document creation. Any insights on any best practice that i need to follow to handle these kind of scenarios would help? we are already looking at doing it async but i fear even there we will hit 12 MB easily
5
  • Question #2 - sync or async, you'll need to break your payload into 6mb or 12 mb max chunks, stuff each chunk into a transaction that pushes them out to a target system, then reassemble the chunks on the target system.
    – identigral
    Feb 5 at 18:42
  • @identigral - Thank you, But how do i identify when its going to hit 6 MB. see the last iteration, the current heap is nowhere near allocated 6 MB heap but it hit heap exception.
    – RedDevil
    Feb 6 at 16:31
  • Good question. Best we've been able to do is to come up with a heuristic that given a payload of N objects that (in our estimate) weighs in at X bytes, assume a 30% safety buffer to avoid hitting the heap limit. That works most of the time but sometimes it fails...which requires retries with a smaller chunk size.
    – identigral
    Feb 7 at 17:06
  • Does this help you clearing some of the things salesforce.stackexchange.com/questions/127424/…
    – manjit5190
    Feb 8 at 14:42
  • Not really. Doesnt explain how to write heap aware code.
    – RedDevil
    Feb 9 at 19:08
10
+50

I believe you are encountering the result of strings being immutable in Apex. Essentially, when you have a statement like the following in a loop:

heapinfo = heapinfo + 'Currentiteration:' + i + ' CurrentMemory:'+ Limits.getHeapSize() +
          'Allocated memory:' + Limits.getLimitHeapSize() + ' \n';

You are temporary allocating a number of additional temporary strings to build up heapinfo. Each of which uses up heap space in each loop iteration. However, you aren't seeing the contribution these temporary strings are making in Limits.getHeapSize(). Likely due to the order of execution of the components.

If you want to see what is going on in detail set the APEX_CODE logging level to FINER. You will then see the HEAP_ALLOCATE events in the debug log.

E.g. For the following:

string accumulator = '';
for(integer i = 0; i < 1000; i++) {
  accumulator = accumulator + 'Currentiteration:' + i + ' CurrentMemory:' + Limits.getHeapSize() + 'Allocated memory:' + Limits.getLimitHeapSize() + ' \n';
}
System.debug(accumulator);

I get the following entries repeating for the loop iteration.

11:50:56.17 (20101685)|HEAP_ALLOCATE|2|Bytes:4
11:50:56.17 (20110321)|VARIABLE_ASSIGNMENT|2|i|6
11:50:56.17 (20117859)|STATEMENT_EXECUTE|2
11:50:56.17 (20121624)|STATEMENT_EXECUTE|[3]
11:50:56.17 (20131069)|HEAP_ALLOCATE|[3]|Bytes:1
11:50:56.17 (20151987)|HEAP_ALLOCATE|[3]|Bytes:4
11:50:56.17 (20170238)|HEAP_ALLOCATE|[3]|Bytes:4
11:50:56.17 (20183268)|HEAP_ALLOCATE|[3]|Bytes:432
11:50:56.17 (20201422)|HEAP_ALLOCATE|[3]|Bytes:7
11:50:56.17 (20215330)|HEAP_ALLOCATE|[3]|Bytes:9
11:50:56.17 (20224795)|HEAP_ALLOCATE|[3]|Bytes:441
11:50:56.17 (20242235)|VARIABLE_ASSIGNMENT|[3]|AB2.accumulator|"Currentiteration:0 C (421 more) ..."

There are some fixed bytes allocated with every loop, but a couple of them (Bytes:432 and Bytes:441) will grow with each loop iteration as the accumulator string grows in length. Each successive loop iteration becomes more and more expensive in terms of heap usage as each temporary string needs to be allocated.

Without the presence of a StringBuilder class in Apex (idea) appending strings becomes an expensive operation. Especially in a loop and when dealing with concatenating multiple strings.

Any workarounds to get around this string concatenation issue to avoid running into heap size?

The simplest immediate change would be to avoid building up a massive heapinfo string. This could instead be handled by System.debug() statements. Ideally you would remove these statements once the code is working.

The bigger problem is how do you build up a very large string in finalstring. In the absence of a StringBuilder or StringBuffer type class your options are limited. You either need to greatly reduce the number of records you need to process or try and free up heap as you are processing. Or maybe a combination of both.

As you mentioned in the comments, using a List<String> to build up the contents with String.join(listOfStrings, '') within the for loop will avoid allocating unnecessary temporary strings. It's kind of a work around StringBuilder. You could probably wrap it in a class and add some helper methods to convert integers etc... into strings as well.

E.g. Basic

List<string> accumulator = new List<string>();
for(integer i = 0; i < 1000; i++) {
  accumulator.add('Currentiteration:');
  accumulator.add(String.valueOf(i));
  accumulator.add(' CurrentMemory:');
  accumulator.add(String.valueOf(Limits.getHeapSize()));
  accumulator.add('Allocated memory:');
  accumulator.add(String.valueOf(Limits.getLimitHeapSize()));
  accumulator.add(' \n');
}
string result = String.Join(accumulator, '');
System.debug(result);

E.g. With a class

public class StringBuilder {
    private List<string> accumulator = new List<string>();
    public void add(String s) { 
        accumulator.add(s);
    }
    public void add(integer i) {
        accumulator.add(String.valueOf(i));
    }
    public override string toString() {
        return String.Join(accumulator, '');
    }
}

StringBuilder sb = new StringBuilder();
for(integer i = 0; i < 1000; i++) {
  sb.add('Currentiteration:');
  sb.add(i);
  sb.add(' CurrentMemory:');
  sb.add(Limits.getHeapSize());
  sb.add('Allocated memory:');
  sb.add(Limits.getLimitHeapSize());
  sb.add(' \n');
}
string result = sb.toString();
System.debug(result);

Switching to asynchronous processing will give you more head room and a better mechanism that is already suited to working with batches of records.

In reality you will be dealing with a collection of Apttus_Proposal__Proposal_Line_Item__c records. There are different for loop syntaxes you can use to iterate over a SOQL result. for (variable : [soql_query]) and for (variable_list : [soql_query]) are good options here.

Depending on how you are working you could also start assigning null to anything you are keeping in memory to free up heap space.

Also how do I write heap aware code?

The debug log is probably the best place to monitor heap usage. Debugging out the Limits.getHeapSize() will also provide you some indication of heap usage.

4
  • Thanks, Any workarounds to get around this string concatenation issue to avoid running into heap size. Also how do i write heap aware code?
    – RedDevil
    Feb 10 at 4:44
  • Thank you Daniel. Instead of building up string manually,i used a list of string to store them and concatenated them at the end. that seemed to do the trick and work. Even if the heap cross 6 MB its only for a little while and it works.
    – RedDevil
    Feb 11 at 5:08
  • Further, there is a big memory difference between doing string concatenation between how its done in my question and using string.join of a list. the latter is very heap effective. Just putting it here so that its useful to someone.
    – RedDevil
    Feb 11 at 5:09
  • 1
    I'd originally looked at using String.format(stringToFormat, formattingArguments) to try and reduce the string allocation within the loop, and had sort of discounted String.join(). But you are right. If the List of strings is built up and only joined once at the end of the process it will greatly reduce the working heap size. Feb 11 at 19:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.