1

I have a component that is running extremely slow. I'm seeing it at times take upwards of 30 seconds to pull back and process data (roughly 3k records) received via an imperative Apex call. I've attempted to use the lightning inspector and the chrome dev tools and can't seem to isolate the specific bottleneck. The profile recording of this via dev tools looks like this and doesn't give (at least to me) much information: chrome dev tools profile

The data pulled back is processed in JS to pull out certain values that can be used by the users to filter the data, and to convert a couple of checkboxes to display as Yes/No instead of True/False. The records are passed back via a wrapper object that contains:

  • A list of wrapper objects around Opportunity (to handle displaying lookup fields etc.)
  • Total count of opportunities
  • Sum of field on all opportunities

The data is displayed using a lightning-datatable like so:

<div if:false={isLoading}>
    <lightning-datatable
            key-field="Id"
            data={allOpportunities}
            columns={columns}
            onrowselection={updateSelectedRows}
            onsort={onClickColumnToSort}
            sorted-by={sortedBy}
            sorted-direction={sortedDirection}>
    </lightning-datatable>
</div>

Here's the imperative Apex and the processing that occurs:

getPipelineRecordsByUserId() {
    getPipelineRecordsByUser({userIds: this.searchUserId}).then(result => {
        if (result) {
            //parse wrapper objects to get possible filter criteria
            this.setFilterOptions(result.pipelineRecords);
            // simple math that is done if on mobile
            this.setOptionsSize();
            //aggregate fields
            this.opportunitiesCount = result.numberOfOpps;
            this.calculatedEstimatedPremium = result.totalEstimatedPremium;

            //list of wrapper objects around opportunity
            let records = result.pipelineRecords;
            
            //method to process return to get possible filter criteria
            this.processPipelineRecords(records);
        }
    })
    .catch(error => {
        this.error = error;
    });
}

// Method to set filter criteria
setFilterOptions(opportunities) {
    let mapper = {
        mapping(x) {
            let tmpObj = {};
            tmpObj.label = x;
            tmpObj.value = x;
            return tmpObj;
        },
        brokerMapping(x) {
            let tmpObj = {};
            tmpObj.label = x.split("|")[1];
            tmpObj.value = x.split("|")[0];
            return tmpObj;
        },
        monthMapping(x) {
            let tmpObj = {};
            tmpObj.label = x.split('. ')[1];
            tmpObj.value = x.split('. ')[1];
            return tmpObj;
        }
    };

    /**
     * Get all values for filters, create a set to get rid of duplicates, and map into objects.
     * Concatenate with the defaults (All, None, etc.)
     */
    let stateVals = opportunities.map(a => a.BrokerState).filter(a => a !== undefined);
    let stageVals = opportunities.map(a => a.Stage).filter(a => a !== undefined);
    let cityVals = opportunities.map(a => a.BrokerCity).filter(a => a !== undefined);
    let productVals = opportunities.map(a => a.PipelineProduct).filter(a => a !== undefined);
    let monthVals = opportunities.map(a => a.EffectiveMonth).filter(a => a !== undefined && a !== "None");
    let brokerMap = opportunities.filter(a => a.BrokerId !== undefined).map(a => a.BrokerId + '|' + a.BrokerName);

    let stateArr = Array.from(new Set(stateVals), mapper.mapping).sort(objNameSort);
    let stageArr = Array.from(new Set(stageVals), mapper.mapping).sort(objNameSort);
    let cityArr = Array.from(new Set(cityVals), mapper.mapping).sort(objNameSort);
    let productArr = Array.from(new Set(productVals), mapper.mapping).sort(objNameSort);
    let monthArr = Array.from(new Set(monthVals), mapper.monthMapping).sort(monthSort);
    let brokerArr = Array.from(new Set(brokerMap), mapper.brokerMapping).sort(objNameSort);

    this.brokerStateOptions = allFilterValue.concat(stateArr);
    this.stageOptions = allFilterValue.concat(stageArr);
    this.brokerCityOptions = allFilterValue.concat(cityArr);
    this.pipelineProdOptions = allFilterValue.concat(productArr);
    this.effectiveMonthOptions = allNoneFilterValue.concat(monthArr);
    this.brokerNameOptions = allFilterValue.concat(brokerArr);
}

//method to handle modifying to setup links to records etc.
processPipelineRecords(opportunities) {
    opportunities.forEach(a => {
        a.brokerNameLink = '/' + a.BrokerId;
        a.opportunityNameLink = '/' + a.OpportunityId;
        if (a.NotExpected) {
            a.NotEx = 'Yes';
        } else {
            a.NotEx = 'No';
        }
        if (a.InBusiness) {
            a.InBusiness = 'Yes';
        } else {
            a.InBusiness = 'No';
        }
        if (a.ReviewCompleted) {
            a.RevCompl = 'Yes';
        } else {
            a.RevCompl = 'No';
        }
        if (a.ManuallyManaged) {
            a.ManualManage = 'Yes';
        } else {
            a.ManualManage = 'No';
        }
    });
    this.allOpportunities = opportunities;
    this.isLoading = false;
}

And finally here is the Apex that returns the data:

@AuraEnabled
public static MyPipelineSearchCriteria getPipelineRecordsByUser(String userIds) {
    // wrapper object containing List<MyPipelineRecord>, Integer NumberOfOpps, Integer TotalEstimatedPremium
    MyPipelineSearchCriteria searchCriteria = new MyPipelineSearchCriteria();
    // wrapper object around opportunity to help display relationship fields
    searchCriteria.pipelineRecords = new List<MyPipelineRecord>();

    String ownerFilter = '';
    String queryString = 'SELECT  Id,' +
                        'Name,' +
                        'StageName,' +
                        'Effective_Month__c,' +
                        'Effective_date__c,' +
                        'Submission_Not_Expected__c,' +
                        'Universe_Review_Completed__c,' +
                        'Estimated_Premium__c,' +
                        'Previous_Stage__c,' +
                        'Broker_City__c,' +
                        'Broker_state__c,' +
                        'Broker_Lookup__r.name,' +
                        'Account.sic,' +
                        'Account.Industry_Tier__c,' +
                        'Account.Appetite_Class__c,' +
                        'Account.annualRevenue,' +
                        'Account.name,' +
                        'AccountId,' +
                        'Prospecting_Producer__r.Name,' +
                        'Account.TIV_Range__c,' +
                        'Account.Buyer_Estimated_Premium__c,' +
                        'Inforce_Business__c,' +
                        'Strategy_To_Win__c,' +
                        'Pipeline_Product__c,' +
                        'Historical_Pricing__c, ' +
                        'Prevent_Underwriter_Auto_Reassign__c, ' +
                        'Owner.Name ' +
                        'FROM Opportunity ' +
                        'WHERE RecordTypeId = \'' + String.escapeSingleQuotes(CommonConstants.PIPELINE_RECORD_RECORD_TYPE_ID) + '\' AND ' +
                            'StageName != \'Closed\'';

    if (userIds == '') {
        queryString += ' and OwnerId =:currentUserId ORDER BY Broker_Lookup__r.Name, StageName, Effective_Month__c ASC ';
    }
    else {
        ownerFilter = getFilterByOwner(userIds);
        queryString += ownerFilter + ' ORDER BY Broker_Lookup__r.Name, StageName, Effective_month__c ASC ';
    }
    queryString += ' LIMIT 49999';

    List<Opportunity> searchCriteriaRecords = Database.query(queryString);
    getAggregates(searchCriteriaRecords, searchCriteria);

    for (Opportunity record : searchCriteriaRecords) {
        searchCriteria.pipelineRecords.add(new MyPipelineRecord(record));
    }

    return searchCriteria;
}

private static String getFilterByOwner(String userIds) {
    List<PermissionSetAssignment> psaList = [
            SELECT  Id,
                    PermissionSet.Name
            FROM    PermissionSetAssignment
            WHERE   (
                        PermissionSet.Name = :SMPSName OR
                        PermissionSet.Name = :AEPSName
                    ) AND
                    AssigneeId = :userIds
            LIMIT 1
    ];
    String permSetName = psaList.isEmpty() ? null : psaList[0].PermissionSet.Name;
    String filterValue = '';
    if (permSetName == null) {
        filterValue = ' AND OwnerId =:userIds';
    }
    else {
        if (permSetName.equals(AEPSName)) {
            filterValue = ' AND Account.Account_Executive__c =:userIds';
        }
        if (permSetName.equals(SMPSName)) {
            filterValue = ' AND Account.Service_Manager__c =:userIds';
        }
    }
    return filterValue;
}

private static void getAggregates(List<Opportunity> searchCriteriaRecords, MyPipelineSearchCriteria searchCriteria) {
    String aggregateQuery = 'SELECT SUM(Estimated_Premium__c)totalEstPrem,' +
                            'COUNT(Id)totalOpps ' +
                            'FROM Opportunity ' +
                            'WHERE Id IN :searchCriteriaRecords';

    AggregateResult[] grouped = Database.query(aggregateQuery);
    Object totalOpps = grouped[0].get('totalOpps');
    Object totalEstPrem = grouped[0].get('totalEstPrem');
    searchCriteria.NumberOfOpps = Integer.valueOf(totalOpps);
    searchCriteria.TotalEstimatedPremium = Integer.valueOf(totalEstPrem);
}

I know this is quite a dump of code, but its a quite complex process. From what I can tell the Apex executes pretty quickly, but its hanging somewhere in the JS. Any suggestions on how to speed up this process would be great.

Edit: Here's a picture when using the Query Plan feature: query plan

  • 1
    Not an ideal solution, but != is not a selective operation. You may want to use = or use IsClosed = false to take advantage of indexing – Sebastian Kessel Jul 9 at 22:16
  • You can rule out the apex processing by seeing how much time it takes the apex request to complete in the Network tab of browser. If the call is getting bundled with others, it can take time. – manjit5190 Jul 9 at 22:20
  • Likely to be the query, so use this tooling help.salesforce.com/… to confirm that. – Keith C Jul 9 at 22:24
  • I'm not sure it's the query, I've added a picture of the query plan to the question. @KeithC – lpbearden Jul 9 at 22:57
5

My educated guess would be that the bottleneck is during rendering. Given ~25 fields times ~3,000 rows, that's going to result in at least ~75k DOM elements minimum to render (probably closer to ~225k elements, from what I know of SLDS), which is going to take some time. Notice how about from about 12 seconds to 63 seconds, all of that is just rendering time, and then after that comes the after-render handlers, about another 10 seconds. There's just too many DOM elements. Consider using pagination to keep it under about ~100 rows (~3k total rendered components) in order to avoid this performance issue. LWC is exceptionally fast, but rendering hundreds of thousands of DOM elements is not a good idea.

| improve this answer | |
  • So, the solution would be to still query everything, but instead of displaying all 3k rows to add pagination in and that should cut down on most of the rendering time? I'll try that first thing in the morning and see if that helps. – lpbearden Jul 9 at 22:56
  • @lpbearden Definitely. You'll want to experiment to find where the balance is, but probably somewhere between a maximum of 200 to 500 rows. Once you reach a certain number of elements, the rendering time rises close to exponentially. See my playground example for tree-grid, it has similar performance characteristics. – sfdcfox Jul 9 at 23:01
  • @lpbearden I also just built a version for datatable that goes to 1000x10000 to give you an idea of rendering times that you can see. I suggest using the arrow keys to bump the sliders slowly, or you might be in for a surprise. – sfdcfox Jul 9 at 23:27
  • 1
    This was definitely the issue, after implementing the paginator from the lwc-recipes it is much more performant. Thanks for the tip! – lpbearden Jul 10 at 0:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.