0

I am trying to implement a solution that will allow users to create 10k+ records via UI. Basically, there is a custom component that allows multiple record creation. There could be 20-50k records, so I`d not use Bulk API. I cannot use Batch job as there are many jobs stuck in a queue all the time. Also, it has to take as less time as possible as the user would like to see the data in CRM after DML (up to 1-2 mins).

I came up with two solutions:

  1. Call Apex REST API from Apex class for every 10k records and do DML in Apex Webservice.

  2. Implement solution on middleware (Node, Java, Python) and send data there for calculation. Middleware will convert and prepare data accordingly to Salesforce import format and will call standard/custom REST API for every 10k records.

I can go with the first one and save time & money for implementation as no need for middleware, server, etc. But I am not sure it is a good solution.

Any help would be appreciated.

2 Answers 2

4

Since you mentioned UI - you can chunk the records in the javascript(LWC/Aura) and call the apex method with 10K(or smaller based on your processing needs) records in one go.

As the limit is for the transaction - calling one method from UI to Apex considers a transaction and allows 10K limits

We had similar requirements with multiple levels of dependencies. But we found a creative way to use JavaScript effectively, thus allowing us to create more than 10K records and show proper status to the user.

Sample psuedo code for vanilla record creation may look like -

function processRecords() {
    let records = this.records;
    let chunkedRecords = chunkRecords(this.records, this.chunk_count);
    let statuses=[];
    for(let i=0;i<chunkedRecords.length;i++){
        statuses[i] = await insertRecordInApex(chunkedRecords[i]);
        //check success/failure operations here and handle as per business need
    }

}
4
  • hmmm...that is interesting. But on js I have a different data representation, it is some kind of template based on which records will be generated. So, I`ll have to generate the structure on js and create an array with around 40k items and then call apex 4 times. Well, it could be an option. But array with 40k items would not be too heavy?
    – m_konyk
    Feb 1, 2023 at 15:01
  • 2
    @m_konyk There is a 4MB payload limit when calling Apex, so you might not be able to upload 10k per batch anyways. However, you can also parallelize the calls by calling up to 5 at once (more than that will start a boxcar effect, which will dramatically slow progress).
    – sfdcfox
    Feb 1, 2023 at 15:26
  • @sfdcfox clear, thanks. But what about those 40-60k items on the client side? Would it be sufficient for the performance? I mean when the tab/page "freezes", so the user will have to click "Wait" or "Close the tab"? If not, I would better go with REST API solution and extract data calculation to middleware.
    – m_konyk
    Feb 1, 2023 at 15:37
  • 2
    @m_konyk calls to Apex are always asynchronous. Even though the example code appears to be stuck in a loop, await will actually allow the browser's UI to continually update, and no "freeze" will occur. In fact, you could even display a progress bar as the records are processed to give the user an indication that the system hasn't frozen.
    – sfdcfox
    Feb 1, 2023 at 15:39
0

This is not possible, the 10k records per transaction limit is a governor limit that cannot be bypassed.

For your solutions:

  1. You described a batch in which you control the process instead of the platform.
  2. Middleware would use either the Batch API or would send multiple requests (chunks), which would be equivalent of running a batch yourself.
2
  • thanks for the answer. It is clear that I cannot bypass it. I meant I need a solution to implement such requirements. #2 - Yes, that is described. I was wondering will #1 would be a good solution in general according to SFDC best practices or if I should go with the 2nd? Because, I`ll spend less time, effort, and budget for #1 implementation.
    – m_konyk
    Feb 1, 2023 at 14:56
  • 1
    The first approach is not ideal because of the UX. If the user closes the window/tab then your process halts. On the second approach at least the user is free to do whatever they want and you can mimic the batch process internally in your middleware (and hopefully post a message back to Salesforce if something goes wrong). Feb 1, 2023 at 14:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .