16

I have a Lightning Aura Component with an Apex Controller selecting a (kind of) huge amount of data. It is querying a custom object elfBCProject__c with a sub select on a child object mueGPMOrderCard__c.

The total JSON payload is about ~140kB. The component is used in a Community, but I'm pretty sure my issue would happen exactly the same without community just in Lightning Experience.

In Apex there is a method like this:

@AuraEnabled public static Map<String,Object> LoadProjects(Map<String,Object> params) { // ...

It queries the records and is returning the records and some more stuff put together in a Map

The (simplified!) SOQL-query looks like this

            +' select * '
            +' ( '
                +' select * '
                + 'from mueGPMOrderCards__r '
            +' )'
            +' from elfBCProject__c '
            +' where Id!=null '
                +' and  ' // ... some filter come here

(please imagine the * as ALL fields)

The SOQL-query returns this in APEX (serialized and send by email). It is 1 elfBCProject__c having 79 mueGPMOrderCard__c inside in the list mueBCOrderCard__r

  "attributes" : {
     "type" : "elfBCProject__c",
     "url" : "/services/data/v47.0/sobjects/elfBCProject__c/a081r00001WYrkOAAT"
  },
  "Id" : "a081r00001WYrkOAAT",
  "OwnerId" : "0051r000008kTRoAAM",
  "IsDeleted" : false,
  // ... lots of fields here .... reduced! 
  "mueGPMOrderCards__r" : {
     "totalSize" : 79,
     "done" : true,
     "records" : [ {
        "attributes" : {
           "type" : "mueGPMOrderCard__c",
           "url" : "/services/data/v47.0/sobjects/mueGPMOrderCard__c/a0R1r00000TApKkEAL"
        },
        "mueGPMProject__c" : "a081r00001WYrkOAAT",
        "Id" : "a0R1r00000TApKkEAL",
        "IsDeleted" : false,
        // ... more fields come here ... 
        // ... more records come here ... 

So far so good.

Now returning the stuff all to my Lightning Aura Component:

        return new Map<String,Object>{
            'success'                           =>  true,
            'projects'                          =>  projects, 
            'projectCount'                      =>  projects.size(),
            // ... more stuff to return comes here ...
        });

The important part is, that I know there are exactly 79 OrderCards for the project in the database. This is a fact. APEX gets 79, Reports show 79, Listviews show 79.

But in JavaScript, only 61 are present in the result. See console in screenshot below.

The console.log() happens on the FULL result a first command in the JS-callback method. What could cause that loss of records?

enter image description here

Update 1

The loss seems to happen at the end. The LAST records in the list just get cut. I can't identify anything special on the lost records. I can rule out permission, access or security. The user I am testing with can see all 79 records everywhere.

Update 2

Now it gets even more interesting, when I additionally add a serialized version of my database-result to my return statement:

    return new Map<String,Object>{
        'projects'                          => projects, 
        'jsonProjects'                      => system.JSON.serializePretty(o),
        // ... more stuff to return comes here ...
    });

this blows the overall payload above 300kb (no error, all good) and now it comes: Looking into the JSON, again we have

... "mueGPMOrderCards__r" : { "totalSize" : 79, ...

So the deserialization (done by Salesforce likely the Aura Framework) seems to loose the records.

Update 3

After reading sfdcfox hint to check if "totalSize" : 79 really matches the body of the JSON I've inspected the content of the JSON structures a little bit more. It is very hard to avoid mistakes and the best way to inspect it was to copy huge JSON strings in the browsers JavaScript console an assign it to a temporary variable. This way, I was able to see that:

  • in APEX the result indicate totalSize:79, but actually contains only 61 records
  • the received result in the JavaScript callback accordingly has only 61 records
  • my own JSON-string jsonProjects added during Update 2 has only 61 records in Apex and 61 records in JavaScript.

So my temporary conclusion, that that the sub-query-limit mentioned by @sfdcfox is the reason

  • From Update 2: Did you scroll to the end of the JSON-ified values to see if all 79 records were actually present? – sfdcfox Jan 24 at 16:13
  • @sfdcfox you are a genius! They are NOT all present! it says 79, but in the JSON there are only the 61 really there. What the heck could cause that strangeness? – Uwe Heim Jan 24 at 16:20
  • See the edit in my answer. That should help. – sfdcfox Jan 24 at 16:31
  • Hmm... That's definitely curious. Still, I'm pretty sure that it's not a payload size problem, there's something else going on. Whatever it is, it's not in the code already in your question, though. – sfdcfox Jan 24 at 16:51
  • @sfdcfox very likely my review last Friday was wrong and it seems, that you are right and we fall into the limit and get a queryLocator... my data is so big, that I can't easily see what I got and I need to carefully inspect the result. So if we really get the queryLocator-Limit-Issue as you mentioned (this is what I currently believe): is there any example how to process it and get ALL the child-records with additional apex? I can't find something matching to our Sub-Query-Scenario only stuff for batch apex. And unfortunately, we really need ALL the fields, so no way to reduce the payload... – Uwe Heim Jan 27 at 13:28
16

I tested three basic boundaries: total payload size, size of an element, and number of items.

Total Payload Size, Element Size

First, I created a map with 9 keys, each key being a 3,000,000 character string. This weighs in at just over 27 MB of data. Aura rendered the page in a couple of seconds, all elements present and accounted for.

Edit

If you use @AuraEnabled(cacheable=true), there's a 10 MB limit. In most cases, caching is desirable, but if you need a large amount of data, disabling caching can allow you to exceed this limit.

Large Number of Items

I then created a map consisting of 100,000 elements, but just smaller strings, weighing in at about 1.2 MB of JSON-ified data. This took much longer to render (aura:iteration doesn't like 100,000 items), but all the elements were present.


No, there doesn't appear to be a maximum size aside from whatever your bandwidth and memory can handle.


Edit: I just realized that this may be an issue with the volume of data you're querying--in Apex. See, if you query too many fields, you can cause the sub-query to be broken up into chunks with a QueryLocator. The threshold for this behavior depends on the number of fields you query.

Try using .size() on the value before returning the value, and see if you get an exception. If so, you'll need to write a wrapper class, then use a for-each loop to extract the children:

sObject[] children = new sObject[0];
for(sObject record: queryResult.childRelationship__r) {
    children.add(record);
}
|improve this answer|||||
  • 1
    @ChristianSzandorKnapp The limit appears to be on the request size, and it's 4 MB. I successfully returned a 27+ MB response in testing. – sfdcfox Jan 27 at 15:14
  • 1
    @ChristianSzandorKnapp Any single string may not exceed Limits.getLimitHeapSize(), even if you go over the limit with a number of objects. There's hardly any situation where you'd need to use JSON, because the platform does the conversion for you automatically. – sfdcfox Jan 31 at 11:40
  • 1
    @sfdcfox we ditched the JSON serialisation on apex side. We are trying to load 10k PricebookEntries. We're now hitting an internal Aura Error: imgur.com/a/JRdMmWL. It reads: Error: AuraStorage.set() cannot store apex:action$ : of size 13771153b because it's over the max size of 10485760b So we are hitting a limit with 10 MB data - still wondering about your 27+ MB data. – Christian Szandor Knapp Feb 4 at 8:18
  • 1
    so after removing chacheable=true on the apex method, this error went away. – Christian Szandor Knapp Feb 4 at 9:06
  • 2
    @PranayJaiswal That's the limit the client can send to the server. There is no apparent limit for the maximum size the server can send the client (aside from the cache limit, if using cache). – sfdcfox Feb 4 at 13:47
4

I created a very simple component:

js controller:

({
    doInit : function (component, event, helper){
        var action = component.get("c.getAccountWithContacts");
        action.setCallback(this, function(response){
            console.log(response.getReturnValue());
        });
        $A.enqueueAction(action);
    }
})

Apex Controller

public class test {
    @AuraEnabled
    public static Map<String, Object> getAccountWithContacts() {
        List<Account> accounts = [
            SELECT Id, (
                SELECT Id
                FROM Contacts
            )
            FROM Account
            WHERE Name = 'Bulk Aura Test'
        ];

        return new Map<String, Object>{
            'accounts' => accounts,
            'accountsSize' => accounts.size(),
            'status' => 'success'
        };
    }
}

Component

<aura:component access="global" controller="test">
   <aura:handler name="init" action="{!c.doInit}" value="{!this}"/>
</aura:component>

I used this script to set up the data:

Account newAccount = new Account(Name = 'Bulk Aura Test');

insert newAccount;

List<Contact> newContacts = new List<Contact>();

for (Integer i = 0; i < 100; i++) {
    newContacts.add(new Contact(
        FirstName = 'test',
        LastName = string.valueOf(i) + 'Contact',
        AccountId = newAccount.Id
    ));
}

insert newContacts;

My debug

Debug

Conclusion

My instinct is that there is no hard limit on the size of the results. I'd put my money on something in the code causing the problem with such a limited sample I can't guarantee that either. Use the sample I posted but adapt it to pull in your record as that would give you a minimum viable code sample that you can reproduce the issue with and hopefully give us more information to help you.

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.