We're calling an external REST service from Salesforce and are receiving a JSON response of more than 45 MB. Has anyone attempted anything like this?

  • Just to clarify, you are calling a Salesforce REST service from an external client or you are calling an external REST service from within Salesforce (apex)? – Jayant Das Mar 25 '19 at 18:05
  • @JayantDas I have added clarification to my question. – Sander de Jong Mar 25 '19 at 18:13
  • Thanks. Had thought that you are calling from Salesforce but good to get that clarified. Derek's answer is what I would think addresses this scenario. – Jayant Das Mar 25 '19 at 18:14

There's just no way to cram that much data into Salesforce as-is.

I'd imagine you'd need to introduce a layer in the middle to break that JSON up into more manageable pieces, and also make use of async processing (probably using Queueable).

As I'm sure you know, the transaction limit on heap size is 6MB (12MB Async), and that space needs to hold the incoming response, store it again (plus overhead) when you deserialize it, and have space left over to do whatever processing you need to do.

Another option would be to have the middle layer (Heroku, some VM on some cloud provider, etc...) do the processing for you, and then create/update/delete whatever it is that you need to do via accessing Salesforce's APIs

| improve this answer | |
  • I like your suggestion of having the middle layer call Salesforce's APIs. I don't know if the middle layer will be able to handle that kind of load. An alternative would be to have the middle layer implement a paging mechanism, but this would also mean more work on the Salesforce side. – Sander de Jong Mar 25 '19 at 18:11
  • I also think that doing tens of thousands of REST calls to Salesforce will not be as efficient as processing 45 times 1 MB JSON files, for example. – Sander de Jong Mar 25 '19 at 18:14
  • 1
    @SanderdeJong Depending on what precisely you're doing, you could create multiple records in Salesforce with a single request, or use the composite resource to package up multiple API calls into a single call. – Derek F Mar 25 '19 at 21:06

In Apex, no chance. The maximum response size is limited to 6/12/36MB (depending on context). You'd have to do client-side processing (e.g. Visualforce via the AJAX Proxy) in order to handle this amount of data.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.