3

We have a scenario where we need to query 2 large text area fields and add them as a response to an external system.

The source object on which these fields are present, we have used SOQL for loop to iterate through them and get required data, but even with SOQL for loop just querying data for 200 records results in us hitting more than 90% heap which fails the transactions afterwards. In total we may need to send these fields for 1000-2000 records of that object.

As a workaround what we have implemented is to loop through the records in limits of 40 and check for heap and do the concatenation. if we cross 80% heap we ask the external system to call again.

Basically custom SOQL for loop implementation. Is there a better way to do this and how should this be approached?

4
  • Does this need to be asynchronous, or could you switch to an async context (queueable, batchable) where you get twice the heap limits and ways to split the data into smaller chunks to avoid loading a lot of data in one go?
    – Phil W
    Commented Jun 17, 2021 at 14:23
  • Basically, could you push the data out to the external system, instead of responding to a direct request?
    – Phil W
    Commented Jun 17, 2021 at 14:24
  • 1
    BTW, your workaround seems OK as an actual solution; we do similar in some cases though what we specifically do is return an encoded "continuation" value that tells the server where it has gotten to in its previous processing. If there's a "continuation" value returned by the service, the client calls again. Once there's no continuation value the client knows it has all the data.
    – Phil W
    Commented Jun 17, 2021 at 14:26
  • @PhilW, we are sending this data in response to initial callout we made. the external systems pulls these record ids and they call salesforce. if we have to do it through queueable we will lose context of the thread and they will be waiting for the response.
    – RedDevil
    Commented Jun 17, 2021 at 18:50

1 Answer 1

3

Keep in mind that there's a maximum response size even if you didn't hit the heap limits (in fact, the response size is limited by heap size). As such, what you've got going on is probably the best way to approach your problem. That's presuming, of course, the external system can't just call a Salesforce API directly, which would be more ideal.

5
  • can you please elaborate on the point of using salesforce API. the external system can call salesforce API directly?
    – RedDevil
    Commented Jun 17, 2021 at 18:51
  • Salesforce provides standard REST APIs for querying data (as well as CRUD on records). This would not be able to concatenate long text areas though.
    – Phil W
    Commented Jun 17, 2021 at 18:57
  • @PhilW, do you know the response size for a rest api ?
    – RedDevil
    Commented Jun 17, 2021 at 18:58
  • Have a look at the documentation.
    – Phil W
    Commented Jun 17, 2021 at 19:00
  • 1
    @RedDevil There's a batch retrieve that does what you want (at least, mostly). You can retrieve 2000 records by Id. You can also do a plain query to get the results, but you may have to deal with pagination (nextUrl will be set to the cursor for the remaining items, called in a loop). There's no need to get Apex involved here.
    – sfdcfox
    Commented Jun 17, 2021 at 19:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .