Lightning Web Component extremely poor performance

Question

I have a component that is running extremely slow. I'm seeing it at times take upwards of 30 seconds to pull back and process data (roughly 3k records) received via an imperative Apex call. I've attempted to use the lightning inspector and the chrome dev tools and can't seem to isolate the specific bottleneck. The profile recording of this via dev tools looks like this and doesn't give (at least to me) much information:

The data pulled back is processed in JS to pull out certain values that can be used by the users to filter the data, and to convert a couple of checkboxes to display as Yes/No instead of True/False. The records are passed back via a wrapper object that contains:

• A list of wrapper objects around Opportunity (to handle displaying lookup fields etc.)
• Total count of opportunities
• Sum of field on all opportunities

The data is displayed using a lightning-datatable like so:

<div if:false={isLoading}>
    <lightning-datatable
            key-field="Id"
            data={allOpportunities}
            columns={columns}
            onrowselection={updateSelectedRows}
            onsort={onClickColumnToSort}
            sorted-by={sortedBy}
            sorted-direction={sortedDirection}>
    </lightning-datatable>
</div>

Here's the imperative Apex and the processing that occurs:

getPipelineRecordsByUserId() {
    getPipelineRecordsByUser({userIds: this.searchUserId}).then(result => {
        if (result) {
            //parse wrapper objects to get possible filter criteria
            this.setFilterOptions(result.pipelineRecords);
            // simple math that is done if on mobile
            this.setOptionsSize();
            //aggregate fields
            this.opportunitiesCount = result.numberOfOpps;
            this.calculatedEstimatedPremium = result.totalEstimatedPremium;

            //list of wrapper objects around opportunity
            let records = result.pipelineRecords;
            
            //method to process return to get possible filter criteria
            this.processPipelineRecords(records);
        }
    })
    .catch(error => {
        this.error = error;
    });
}

// Method to set filter criteria
setFilterOptions(opportunities) {
    let mapper = {
        mapping(x) {
            let tmpObj = {};
            tmpObj.label = x;
            tmpObj.value = x;
            return tmpObj;
        },
        brokerMapping(x) {
            let tmpObj = {};
            tmpObj.label = x.split("|")[1];
            tmpObj.value = x.split("|")[0];
            return tmpObj;
        },
        monthMapping(x) {
            let tmpObj = {};
            tmpObj.label = x.split('. ')[1];
            tmpObj.value = x.split('. ')[1];
            return tmpObj;
        }
    };

    /**
     * Get all values for filters, create a set to get rid of duplicates, and map into objects.
     * Concatenate with the defaults (All, None, etc.)
     */
    let stateVals = opportunities.map(a => a.BrokerState).filter(a => a !== undefined);
    let stageVals = opportunities.map(a => a.Stage).filter(a => a !== undefined);
    let cityVals = opportunities.map(a => a.BrokerCity).filter(a => a !== undefined);
    let productVals = opportunities.map(a => a.PipelineProduct).filter(a => a !== undefined);
    let monthVals = opportunities.map(a => a.EffectiveMonth).filter(a => a !== undefined && a !== "None");
    let brokerMap = opportunities.filter(a => a.BrokerId !== undefined).map(a => a.BrokerId + '|' + a.BrokerName);

    let stateArr = Array.from(new Set(stateVals), mapper.mapping).sort(objNameSort);
    let stageArr = Array.from(new Set(stageVals), mapper.mapping).sort(objNameSort);
    let cityArr = Array.from(new Set(cityVals), mapper.mapping).sort(objNameSort);
    let productArr = Array.from(new Set(productVals), mapper.mapping).sort(objNameSort);
    let monthArr = Array.from(new Set(monthVals), mapper.monthMapping).sort(monthSort);
    let brokerArr = Array.from(new Set(brokerMap), mapper.brokerMapping).sort(objNameSort);

    this.brokerStateOptions = allFilterValue.concat(stateArr);
    this.stageOptions = allFilterValue.concat(stageArr);
    this.brokerCityOptions = allFilterValue.concat(cityArr);
    this.pipelineProdOptions = allFilterValue.concat(productArr);
    this.effectiveMonthOptions = allNoneFilterValue.concat(monthArr);
    this.brokerNameOptions = allFilterValue.concat(brokerArr);
}

//method to handle modifying to setup links to records etc.
processPipelineRecords(opportunities) {
    opportunities.forEach(a => {
        a.brokerNameLink = '/' + a.BrokerId;
        a.opportunityNameLink = '/' + a.OpportunityId;
        if (a.NotExpected) {
            a.NotEx = 'Yes';
        } else {
            a.NotEx = 'No';
        }
        if (a.InBusiness) {
            a.InBusiness = 'Yes';
        } else {
            a.InBusiness = 'No';
        }
        if (a.ReviewCompleted) {
            a.RevCompl = 'Yes';
        } else {
            a.RevCompl = 'No';
        }
        if (a.ManuallyManaged) {
            a.ManualManage = 'Yes';
        } else {
            a.ManualManage = 'No';
        }
    });
    this.allOpportunities = opportunities;
    this.isLoading = false;
}

And finally here is the Apex that returns the data:

@AuraEnabled
public static MyPipelineSearchCriteria getPipelineRecordsByUser(String userIds) {
    // wrapper object containing List<MyPipelineRecord>, Integer NumberOfOpps, Integer TotalEstimatedPremium
    MyPipelineSearchCriteria searchCriteria = new MyPipelineSearchCriteria();
    // wrapper object around opportunity to help display relationship fields
    searchCriteria.pipelineRecords = new List<MyPipelineRecord>();

    String ownerFilter = '';
    String queryString = 'SELECT  Id,' +
                        'Name,' +
                        'StageName,' +
                        'Effective_Month__c,' +
                        'Effective_date__c,' +
                        'Submission_Not_Expected__c,' +
                        'Universe_Review_Completed__c,' +
                        'Estimated_Premium__c,' +
                        'Previous_Stage__c,' +
                        'Broker_City__c,' +
                        'Broker_state__c,' +
                        'Broker_Lookup__r.name,' +
                        'Account.sic,' +
                        'Account.Industry_Tier__c,' +
                        'Account.Appetite_Class__c,' +
                        'Account.annualRevenue,' +
                        'Account.name,' +
                        'AccountId,' +
                        'Prospecting_Producer__r.Name,' +
                        'Account.TIV_Range__c,' +
                        'Account.Buyer_Estimated_Premium__c,' +
                        'Inforce_Business__c,' +
                        'Strategy_To_Win__c,' +
                        'Pipeline_Product__c,' +
                        'Historical_Pricing__c, ' +
                        'Prevent_Underwriter_Auto_Reassign__c, ' +
                        'Owner.Name ' +
                        'FROM Opportunity ' +
                        'WHERE RecordTypeId = \'' + String.escapeSingleQuotes(CommonConstants.PIPELINE_RECORD_RECORD_TYPE_ID) + '\' AND ' +
                            'StageName != \'Closed\'';

    if (userIds == '') {
        queryString += ' and OwnerId =:currentUserId ORDER BY Broker_Lookup__r.Name, StageName, Effective_Month__c ASC ';
    }
    else {
        ownerFilter = getFilterByOwner(userIds);
        queryString += ownerFilter + ' ORDER BY Broker_Lookup__r.Name, StageName, Effective_month__c ASC ';
    }
    queryString += ' LIMIT 49999';

    List<Opportunity> searchCriteriaRecords = Database.query(queryString);
    getAggregates(searchCriteriaRecords, searchCriteria);

    for (Opportunity record : searchCriteriaRecords) {
        searchCriteria.pipelineRecords.add(new MyPipelineRecord(record));
    }

    return searchCriteria;
}

private static String getFilterByOwner(String userIds) {
    List<PermissionSetAssignment> psaList = [
            SELECT  Id,
                    PermissionSet.Name
            FROM    PermissionSetAssignment
            WHERE   (
                        PermissionSet.Name = :SMPSName OR
                        PermissionSet.Name = :AEPSName
                    ) AND
                    AssigneeId = :userIds
            LIMIT 1
    ];
    String permSetName = psaList.isEmpty() ? null : psaList[0].PermissionSet.Name;
    String filterValue = '';
    if (permSetName == null) {
        filterValue = ' AND OwnerId =:userIds';
    }
    else {
        if (permSetName.equals(AEPSName)) {
            filterValue = ' AND Account.Account_Executive__c =:userIds';
        }
        if (permSetName.equals(SMPSName)) {
            filterValue = ' AND Account.Service_Manager__c =:userIds';
        }
    }
    return filterValue;
}

private static void getAggregates(List<Opportunity> searchCriteriaRecords, MyPipelineSearchCriteria searchCriteria) {
    String aggregateQuery = 'SELECT SUM(Estimated_Premium__c)totalEstPrem,' +
                            'COUNT(Id)totalOpps ' +
                            'FROM Opportunity ' +
                            'WHERE Id IN :searchCriteriaRecords';

    AggregateResult[] grouped = Database.query(aggregateQuery);
    Object totalOpps = grouped[0].get('totalOpps');
    Object totalEstPrem = grouped[0].get('totalEstPrem');
    searchCriteria.NumberOfOpps = Integer.valueOf(totalOpps);
    searchCriteria.TotalEstimatedPremium = Integer.valueOf(totalEstPrem);
}

I know this is quite a dump of code, but its a quite complex process. From what I can tell the Apex executes pretty quickly, but its hanging somewhere in the JS. Any suggestions on how to speed up this process would be great.

Edit: Here's a picture when using the Query Plan feature:

Answer 1

My educated guess would be that the bottleneck is during rendering. Given ~25 fields times ~3,000 rows, that's going to result in at least ~75k DOM elements minimum to render (probably closer to ~225k elements, from what I know of SLDS), which is going to take some time. Notice how about from about 12 seconds to 63 seconds, all of that is just rendering time, and then after that comes the after-render handlers, about another 10 seconds. There's just too many DOM elements. Consider using pagination to keep it under about ~100 rows (~3k total rendered components) in order to avoid this performance issue. LWC is exceptionally fast, but rendering hundreds of thousands of DOM elements is not a good idea.