0

I am developing a Lightning Web Component that uses the Bulk API 2.0 to insert a large amount of data to Salesforce, exposing the functionality to non-technical users without access to the Setup.

So far, when the user uploads a CSV using a <lightning-input type="file">, I managed to use the Fetch API from JavaScript, and the Trusted URLs and Remote Site Settings from Salesforce, to successfuly create a job, add data to it and close it, resulting in a successful insert.

Although all of this was achieved by hardcoding the bearer token I obtained by requesting a POST call to a Connected App that granted the API Access, since in the user context of Lightning Experience, API Access is not enabled as far as I know.

So my question would be, how can I best request an access token from the front-end of my LWC so I can call the API? Without having to hardcode sensitive information such as client secrets, passwords, etc. and securing the received Bearer Token

Lastly, a few things I would like to make clear:

  1. Using Apex Batch is not an option. Sending the information from the LWC to Apex is costy. You need to separate it in a lot of chunks because the maximum payload you can pass as a parameter to an Apex function is around 4 MB, after it gets converted to base64. We are working with ~130 MB of data, so creating one Async Apex Job for each chunk would block the Flex Queue with our jobs, and considering our production environment is shared with other applications, and the max number of Jobs you can have in the Flex Queue is 100, we can't go with this approach.

  2. Calling the API from Apex is not ideal because we would still need to pass the data to Apex with the 4 MB limit, and create a Bulk Job for every 4 MB since you can't add data with multiple calls using the Bulk API 2.0, it only allows you to do one PUT request for each job. The good thing about JavaScript is that you can handle large CSVs easily with libraries like Papa Parse and send larger chunks of data in the PUT request.

  3. Using tools such as dataloader.io is neither an option, since these external tools are not allowed by our client.

I hope I made myself clear, please add a comment if you need more information, thanks.

1
  • Your question is a XY Problem. One typical LWC-based solution for processing a CSV is to ingest it client-side, break it up into chunks and then send each chunk over to controller for processing. You've got multiple levels of XY here. The controller can store each/all chunks in a staging object (custom or standard), then one batch job could be kicked off. 130 MB in a CSV would prompt a Wat from us.
    – identigral
    Commented Nov 24, 2023 at 17:03

0

You must log in to answer this question.

Browse other questions tagged .