0

I need literally all the records. Documentation says that if I hit 120 sec. limit then I should refactor my query to retrieve less amount of data. The only option not to skip data but collect it part by part in multiple requests is LIMIT+OFFSET clause. But OFFSET doesn't support values greater than 2000, and even if did, the option is poor due to data race. Among remaining options there is getUpdated request, which is limited to last 30 days, and Async SOQL, which requires a custom object to store the result. The latter seems helpful only when the query itself takes a magnitude more time than the non-async query to the result object, i.e. some GROUP BY aggregations, filters etc. In my case the result object will have as many records as the source.

Is there any reliable approach to get such a big listing? I'm dealing with tens of millions records in organization. Nested SELECT queries are eventually impossible due to timeout. Even a trivial "SELECT Id" fails if it's CaseHistory. Haven't found any satisfactory answer yet.

Improve this question
2
  • 1
    Where do you need those data? For processing of large amounts of data, things like bulk API and batch jobs would be the best approaches
    – kurunve
    Feb 7, 2022 at 14:33
  • @kurunve almost all bulk stuff is about uploading data, not downloading. There are query jobs but there's exactly same query timeout bottleneck. As far as I see by doc.
    – Alexander
    Feb 7, 2022 at 14:50

1 Answer 1

Reset to default
0

Since, it looks like you need to query entire Case table, PK Chunking might work best for this. https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/async_api_headers_enable_pk_chunking.htm.

There are many techniques for this, out of which you have also mentioned a few. I would suggest to try the different options, check this trail (https://trailhead.salesforce.com/content/learn/modules/large-data-volumes/conduct-data-queries-and-searches) and see what works best for you.

Improve this answer
2
  • Thank you! Haven't heard of PK Chunking before. The implementation and testing is going to take time but I think the answer satisfies the topic curiosity enough to be accepted before helpfulness proof in practice.
    – Alexander
    Feb 7, 2022 at 15:08
  • Thanks my friend :) Looking forward to the outcome. Cheers!.
    – user111032
    Feb 7, 2022 at 15:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .