1

We are looking for a scalable & performant way to download Salesforce Objects which can support objects with tens of millions records. I have two approaches in mind:

  1. Use Bulk Query API 2.0: It's simple and easy-to-use API, however it's streaming API which doesn't support concurrent downloading (you only get the 'locator' only from the previous batch read: https://developer.salesforce.com/docs/atlas.en-us.api_bulk_v2.meta/api_bulk_v2/query_get_job_results.htm). This means the download job can only be single-threaded.

  2. Use Bulk Query API 1.0: It support parallel downloading with 'PKChunking' enabled, however this feature is not available for all Salesforce objects, for example, the Order, OrderItem objects doesn't support PkChunking.

Please help me understand which one is better and if there is any better way to do the same.

Thanks.

1
  • I'm not sure that the proposed duplicate answers this question. This question is specifically about potential performance differences between Bulk API 1.0 with PK chunking and Bulk API 2.0.
    – David Reed
    Jun 19 '20 at 4:52

Browse other questions tagged or ask your own question.