1

I am using the simple-salesforce Python package to upsert records into our Salesforce instance. I am finding that each bulk API call counts as 6 API requests. Is this correct? Is there a way to reduce this whilst performing this operation.

The function I am using to perform the upsert:

def bulk_upsert(self, object_name, data_set, identity_field):


    upsert_response = SFBulkType(object_name, self.bulk_con.connect.bulk_url,
           self.bulk_con.connect.headers, self.bulk_con.connect.session).upsert(data_set, identity_field)

    return upsert_response

The object name is the custom object created in Salesforce for our instance, the dataset is a list of dictionaries containing 200 records, the identity field is the external id for the Salesforce object.

I establish the Salesforce connection (sf) and then loop through requested_chunks which is a list of the data to upsert broken into 200 records each.

for request in request_chunks:

    upsert_response = sf.bulk_upsert(sf_object, request, sf_identity) 

I am finding that each time through this loop it is costing 6 credits. I have done a comparison with the integration recommended by simple-salesforce and that also costs 6 calls per bulk upsert. Any thoughts?

1

Is your python lib uses Bulk API or simple record insert/update? If it uses Salesforce Bulk API it needs 1 request to create Bulk Job, 1 request to upload records, 1 request to mark job as posted. And needs to poll request to track job status if records are processed

1

The Bulk API is designed for bulk uploads of data. If you're importing less than 1,000 records, you're definitely wasting API calls. Six calls sounds about right; it definitely requires at least three just to initialize, upload, and start the Bulk API call, plus more calls to wait for completion. Use the normal upsert API call if you want to upload smaller batches of data (e.g. 200 records at a time).

  • Will this cost 1 API call per record? So a batch of 400 records which was split into two API requests to the bulk API previously, would then be 400 separate upsert calls? – James Wellington Apr 15 at 15:42
  • I can increase the batch size from 200 to circa 2000 without coming into problems with total characters. I went with 200 as assumed it wouldn't change the API number of calls. The documentation states that the API splits batches into 200 records behind the scenes. – James Wellington Apr 15 at 15:49
  • @JamesWellington The synchronous upsert call can do 200 records API call. The Bulk API can process up to 10,000 records per file, and millions of records per day. The Bulk API will indeed split the file in to batches of 200. If you use the bulk API, there's really no reason to send less than 10,000 records per file. You'll want to read the documentation for more information. – sfdcfox Apr 15 at 16:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.