1

I need to create a client application where users can upload CSV files, which will then be used to populate Custom Big Objects.

My problem is I'm not sure which API to use. The Salesforce documentation for "Populate a Custom Big Object" says "You can use a CSV file to load data into a custom big object via SOAP API Bulk AP". I'm assuming this is a typo and it means we can use SOAP or Bulk API.

I'll be uploading around 300,000 records at once. With that being said, would it be better for the application to use Bulk API or SOAP API?

Are there any examples of this being done? I tried Googling this topic and couldn't find much relevant information. Even the Salesforce documentation is lacking when it comes to explaining how to populate Big Objects.

Also, before it's mentioned, we're not able to use the Data Loader, hence the need for this application

Improve this question

2 Answers 2

Reset to default
1

You can use Data Loader to populate Big Objects, you just need to enable 'Use Bulk API' in settings.

Warning: Big Objects have a number of caveats, the first one being that you have to pay for storage. This isn't well-documented, unfortunately. First 1 million records are free, then you have to pay (in 50M increments). If the size of your projected dataset is less than 1M, I would use regular objects.

Improve this answer
0

I've used the Bulk API successfully. It support csvs. You need to set your headers as Content-Type : text/csv

Loading the data is pretty straightforwrd once you have your CSV setup. Here's a good trailhead module for it

For your mappings, you need to build a transformation spec That's covered here. From the docs:

The transformation spec (spec.csv) provides the instructions for how to map the data in your import file to Salesforce data fields.

This is a second csv that you need to build out and POST to SF.

If you need assistance with the Bulk API lmk

Improve this answer
4
  • Ohana, thanks for the answer. I looked at the Trailhead module for Bulk API but it only describes how to do it from Workbench and not programmatically. Would you be able to share your code or at least some snippets of it?
    – Dymas
    May 22, 2019 at 15:58
  • Sure thing.... what are you trying to do?
    – PartOfTheOhana
    May 22, 2019 at 16:04
  • I'd like to post a CSV file to the bulk API endpoint in order to populate a big object with that data. Do I need to use the Soap API to login? I don't see a way to login/authenticate on the Bulk API
    – Dymas
    May 22, 2019 at 17:14
  • @Dymas the Bulk API uses the same framework that the REST API. That means the Bulk API supports OAuth. Post your question in a seperate question so I can give you oAuth instructions and the request instructions. If the scope of your initial question has been addressed, please mark it as closed.
    – PartOfTheOhana
    May 22, 2019 at 22:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .