2

I have a set of 4 Queueable apex classes that perform an ordered series of tasks on a large number of records. It queries a limited number of records, modifies them so that they no longer meet the WHERE requirements, then re-queues the initializing class until there are no more records called.

We have run ~600k records through these classes in the past without problems, but needed to increase the speed of processing so we changed the batch size from 500 records to 2500 records. On the first run through the process looped about five times then threw a "Too many SOQL queries: 201" error.

I checked the obvious places (for loops, recursive triggers, nested queries) and can't figure out why we would suddenly be maxing out SOQL queries. I can't find information that explains how governor limits affect queueable apex: in our initial rollout there were concerns about cumulative SOQL queries but this loop has run thousands of times in one go without issue so I assumed we were safe.

How are SOQL queries counted in queueable apex? Are there other places I should be checking to catch stray queries? In case it's relevant, the records that are actually manipulated are pulled with Database.query() and updated with Database.update().

  • Tbh I feel you should be using batch apex instead. As you don't have to enqueue jobs again and again I feel it would be faster – Pranay Jaiswal Aug 5 '19 at 21:25
  • The first solution I built was based on batch, but there is some branching logic (if they don't need Class2, skip to Class3 and the like) with completion dependencies that made queueable better suited. – NNPPOINTER Aug 5 '19 at 22:44
3

Queueable Apex is an asynchronous context as far as limits go, which does affect a variety of limits, including the total SOQL count (200 instead of 100). Each Queueable job enqueued and executed represents a separate transaction, which is the unit of limits consumption.

What is happening here appears to be your process automation downstream of the Queueable. If your Queueable job makes one and only one query in its execute() method, it's only being charged for one query - but if it then performs updates on 2,500 records, the 200 other SOQL queries being attempted are coming from your triggers or other automation running downstream of those updates. Those operations, so long as they execute synchronously (i.e. are not run in further asynchronous operations) fall within the transaction context of the Queueable job and are counted against its limits.

It's important to remember that each batch of 200 records is a separate trigger invocation. This can result in limits consumption looking other than you might expect.

In many customized orgs it will not be possible to process 2,500 records in a transaction, full stop, because of the limits consumption imposed by triggers, processes, flows, and so forth.

| improve this answer | |
  • Thanks @David, I had only been checking within each transaction and hadn't pursued other downstream automation. After combing through the debug log it appears as though the Database.update call was running in batches of 200, so the expanded batch size was indeed putting me over the edge.I was able to add a recursion stop that got rid of those and we were off to the races! – NNPPOINTER Aug 5 '19 at 22:47
  • @NNPPOINTER Be careful here. Your downstream automation is part of the transaction. And all trigger events run in batches of 200. Many simple implementations of recursion control (I don't see any sign that you have recursion, by the way) result in you not processing records you meant to process. – David Reed Aug 5 '19 at 22:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.