2

I have a set of 4 Queueable apex classes that perform an ordered series of tasks on a large number of records. It queries a limited number of records, modifies them so that they no longer meet the WHERE requirements, then re-queues the initializing class until there are no more records called.

We have run ~600k records through these classes in the past without problems, but needed to increase the speed of processing so we changed the batch size from 500 records to 2500 records. On the first run through the process looped about five times then threw a "Too many SOQL queries: 201" error.

I checked the obvious places (for loops, recursive triggers, nested queries) and can't figure out why we would suddenly be maxing out SOQL queries. I can't find information that explains how governor limits affect queueable apex: in our initial rollout there were concerns about cumulative SOQL queries but this loop has run thousands of times in one go without issue so I assumed we were safe.

How are SOQL queries counted in queueable apex? Are there other places I should be checking to catch stray queries? In case it's relevant, the records that are actually manipulated are pulled with Database.query() and updated with Database.update().

Improve this question
2
  • Tbh I feel you should be using batch apex instead. As you don't have to enqueue jobs again and again I feel it would be faster
    – Pranay Jaiswal
    Aug 5, 2019 at 21:25
  • The first solution I built was based on batch, but there is some branching logic (if they don't need Class2, skip to Class3 and the like) with completion dependencies that made queueable better suited.
    – NNPPOINTER
    Aug 5, 2019 at 22:44

1 Answer 1

Reset to default
5

Queueable Apex is an asynchronous context as far as limits go, which does affect a variety of limits, including the total SOQL count (200 instead of 100). Each Queueable job enqueued and executed represents a separate transaction, which is the unit of limits consumption.

What is happening here appears to be your process automation downstream of the Queueable. If your Queueable job makes one and only one query in its execute() method, it's only being charged for one query - but if it then performs updates on 2,500 records, the 200 other SOQL queries being attempted are coming from your triggers or other automation running downstream of those updates. Those operations, so long as they execute synchronously (i.e. are not run in further asynchronous operations) fall within the transaction context of the Queueable job and are counted against its limits.

It's important to remember that each batch of 200 records is a separate trigger invocation. This can result in limits consumption looking other than you might expect.

In many customized orgs it will not be possible to process 2,500 records in a transaction, full stop, because of the limits consumption imposed by triggers, processes, flows, and so forth.

Improve this answer
3
  • Thanks @David, I had only been checking within each transaction and hadn't pursued other downstream automation. After combing through the debug log it appears as though the Database.update call was running in batches of 200, so the expanded batch size was indeed putting me over the edge.I was able to add a recursion stop that got rid of those and we were off to the races!
    – NNPPOINTER
    Aug 5, 2019 at 22:47
  • @NNPPOINTER Be careful here. Your downstream automation is part of the transaction. And all trigger events run in batches of 200. Many simple implementations of recursion control (I don't see any sign that you have recursion, by the way) result in you not processing records you meant to process.
    – David Reed ♦
    Aug 5, 2019 at 22:52
  • @DavidReed Are the limits the same if SOSL is used instead of SOQL? I'm trying to do this for the first time and I'm not sure about the gotchas. Thanks for helping!
    – Bahman.A
    Jun 7, 2021 at 19:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .