6

I am working on a Apex REST framework that acts as an abstraction layer to make it easier for all our other systems to consume data from salesforce (i.e. consistent naming scheme for field names, abstraction for complex filters, pre-filtered lists, different representations for the same SObjects under differend endpoints, etc) compared to the standard REST Api.

However, I struggle with implementing proper pagination. The usual approach to use limit & offset as query parameters does not seem to work, since we have way over 50k records in certain SObjects, but the SOQL OFFSET only allows a maximum value of 2000.

I read about SOQL in for-Loops, which internally "utilizes SOAP query() and queryMore()", but I have not been able to implement this functionality with Apex directly.

How to implement pagination? I don't necessarily need limit/offset, if there are better patterns to utilize standard platform functionality, I would be very happy to use them.

3

Why not design your own querry cursor in Apex?

Theory: Salesforce ID fields are autoincremented when created and behave like a number so you can use Less than and Greater than operators.

So if you can send the lastRecordID to your rest endpoint you can create your own querry cursor.

eg.

Select id from Opportunity where  Id >= '00658000003rJFwAAM' order by Id Limit 2000

is a perfectly valid SOQL and will return all records having ID greater than 00658000003rJFwAAM. Thus this can be used for the Pagination logic in your stateless rest endpoint

Always store the last recordId and use that ID to get next set of records, No need to use offset anymore. Given that you design stateful client and stateless server.

This method is used for querying large data volume when the normal SOQL with where clause on other fields timeout.

Source: https://developer.salesforce.com/blogs/developer-relations/2015/11/pk-chunking-techniques-massive-orgs.html

  • 1
    I like the idea. SInce I already have a pagination wrapper that contains the next_batch-URL, I should be able to simply replace the offset=xx I am currently using with some sort of last_id=xxx. If implementation works, I will definitely let you know. – J. Lietzau Dec 12 '18 at 9:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.