7

I have the below code where in we are accepting the blob data from the REST Apex class. We are inserting the document (blob) that we are receiving as a feed item. I was able to send file sizes upto 54 MB after which it started erroring out.

The governor limit states that the heap size for synchronous transactions is 6MB then how is it that this REST class is able to accept file sizes 9 times the heap size limit? and when we print the limits in the debug even that is not showing a high number at all. Where is this file stored when the class receives it ? How is the heap size getting calculated?

@RestResource(urlMapping='/uploadFile/*')
global class REST_StoreAttachment {

    @httpPost
    global static String postMethod(){

        Blob requestBody=RestContext.request.requestBody;

        system.debug('Heap size before'+ limits.getHeapSize() + ' ==== '+ Limits.getLimitHeapSize());

        ContentVersion content = new ContentVersion();
        content.Title = 'test';
        content.PathOnClient = 'fileName.txt';
        content.VersionData = requestBody;
        content.Origin = 'H';

        insert content;

        FeedItem post = new FeedItem();
        post.ParentId = '00000000000000';         //hard coded to a Id
        post.Body = 'New attachment added';
        post.RelatedRecordId = content.Id; 
        post.Type = 'ContentPost';
        insert post;

        system.debug('Heap size after'+ limits.getHeapSize() + ' ==== '+ Limits.getLimitHeapSize());
        return 'success';
    }  
}

System Debug

Heap size before2189 ==== 6000000

Heap size after2514 ==== 6000000

  • Heap size is 6MegaByte(6MB) , which is greater than 50MegaBit(Mb), notice the capital and small B. 50Mb == 6.25MB. – Pranay Jaiswal Sep 13 '18 at 11:00
  • My bad, I put that incorrectly. Its 50 MegaByte file which is getting loaded. – Sam Sep 13 '18 at 11:11
  • 2
    Interesting.. I sometime struggle with file of size 4MB, I wonder how its working for you for 50MB. I use file upload standard endpoint for bigger file upload. :/ Too strange for me – Pranay Jaiswal Sep 13 '18 at 11:16
  • If I see the governor limits I see this -- [Email services heap size is 36 MB]. Looks like this is something which exists today same as for Email services, but is not documented, may be? – Jayant Das Sep 18 '18 at 20:20
  • Would you like to see the following question which may can help. salesforce.stackexchange.com/questions/162050/… – Naval Sharma Sep 21 '18 at 7:03
4

Interesting. I suspect that what might be going on is, certain objects retrieved from certain methods point directly to underlying Java representations without being processed as part of Apex, and remain so until manipulated. Such objects sometimes possess weird behaviours compared to objects created directly in Apex.

One example I've seen is after deserialization of JSON. If a JSON attribute maps to an Integer attribute in the Apex class I deserialize it to, but a number like 1.00 was in the JSON, Apex somehow jams a Decimal into an Integer container instead of converting to the integer 1. Only when you try and actually do Integer math on it does it complain that it's not an Integer.

Just a thought: suppose you redundantly convert the blob to Base64 and back, and assign that to VersionData. Does it start enforcing heap limits now?

  • Converting the doc to base64 string blows the code with Apex heap size too large error. So clearly only on string conversion the document is getting loaded into the heap.. Where is that document stored until that point? – Sam Sep 14 '18 at 10:46
  • Surely it's still stored on the Java heap somewhere. I think it's just that Apex's methods of accounting for heap size are vague and undisclosed and often inconsistent. My theory is that RestContext.request.requestBody is a direct reference to a Java Blob object created outside the Apex transaction. The memory was allocated by Java not Apex so it doesn't get calculated onto the heap. The moment Apex processes it, the Apex heap counter sees it. – Charles T Sep 14 '18 at 16:01
  • Let's assume that is the case, but would it be safe to go ahead with an implementation to accept files sizes greater than 6MB? – Sam Sep 17 '18 at 5:18
  • Well, you could try it out but there is certainly no guarantee it won't break. The "safest" option is to accept large files using the standard Salesforce REST APIs that can accept files up to 2GB. – Charles T Sep 17 '18 at 14:51
0

There seem to be following limits on standard REST SObject Basic Information or SObject Rows APIs. If you are uploading multi part message you should be able to upload following sizes as per documentation:

https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_sobject_insert_update_blob.htm

Using the SObject Basic Information or SObject Rows APIs, the maximum file size for uploads is 2 GB for ContentVersion objects and 500 MB for all other eligible standard objects. Using the SObject Collections API, the maximum total size of all files in a single request is 500 MB.

For non-multipart messages which seems to be a case of the custom endpoint the limits are as below:

You can insert or update blob data using a non-multipart message, but you are limited to 50 MB of text data or 37.5 MB of base64–encoded data.

Which corresponds to the error limit you are getting now. Might be related to your issue potentially.

  • 4
    Not sure if OP is asking for alternatives or seeking the guidance about weird behavior of Apex rest. – Pranay Jaiswal Sep 13 '18 at 11:19
  • While this may not be the exact answer, but this does look like to point towards a scenario why 50MB of file is being accepted in the first place. Not completely sure, but it does look like that the limits available here on standard REST API calls supposedly are available in custom REST implementations as well. – Jayant Das Sep 13 '18 at 12:31
  • Updated the answer that this is just a potential alternative solution and does not explain the heap size calculation. – Michal V Sep 13 '18 at 12:43
  • Actually OP is not looking for an alternative solution as @PranayJaiswal has mentioned. OP is seeking guidance as why this was allowed. And what you have provided here may be a direction towards that. You may like to actually edit your answer towards that direction. I find this information useful as it makes me think that if this behavior what you highlighted is somehow applicable for custom APIs as well, and that's why OP is actually able to upload a file of 50MB. – Jayant Das Sep 13 '18 at 12:56
  • @vernemi2: As others have pointed out, this is not what I am looking for. But instead trying to understand how is it that that an apex class is able to process a file 9 times the heap size? – Sam Sep 14 '18 at 1:35
0

Heap Limits ----- The only context that can even store 20 MB of data in RAM is Email Services. That means you'd have to kick off an email to Salesforce just to get your code to have enough memory to load your file in memory, much less have a chance of doing anything. Batch Apex, future methods, and Queueable all only have a 12 MB limit.

You have more in the heap than just file data. Heap can be tricky to measure anyway because of garbage collection but you can see how much data you added by taking a diff: you can calculate the increase in heap by below

Integer startingHeap = Limits.getHeapSize(); // add data to heap Integer addedHeap = Limits.getHeapSize() - startingHeap;

Is this sync call or async call? for async call heap limit is high (10Mb) Below is the salesfoce limit for file. Heap limit is different than the file limit Content: maximum file size • 2 GB • 2 GB (including headers) when uploaded via Chatter REST API • 2 GB (including headers) when uploaded via REST API • 38 MB when uploaded via SOAP API • 10 MB when uploaded via Bulk API • 10 MB for Google Docs • 10 MB when uploaded via Visualforce

  • hope it was helpful – Ashwini Dengi Sep 19 '18 at 7:47
  • This is considered as a synchronous call. And it is clearly documented that email services have a heap size of 36MB, likewise I wanted to understand if there was authoritative answer on why the Apex Rest Services is able to accept files upto 50MB when the documentation clearly states that for a synchronous call the heap size is 6MB? – Sam Sep 19 '18 at 14:39
  • you mean to say heap size should be greater than the acceptable file size? What i feel is both are different limits independent. – Ashwini Dengi Sep 21 '18 at 4:37
  • There is another documentation about the governor limit which states that the file request or response body cannot be more than 6MB (Sync apex) or 12MB(Async Apex).. developer.salesforce.com/docs/atlas.en-us.apexcode.meta/… – Sam Sep 21 '18 at 6:05
  • But yes, to answer your question, if the request body is not stored in Heap then where is it getting stored and this is in contradiction to the governor limit. – Sam Sep 21 '18 at 6:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.