Maximum throughput for Batch Apex?

Question

I am working on a system that currently relies on batch apex to process ten of thousands of records by hitting an external API to pass data to another system for processing.

We get a "success" or "error" response from the HTTP request, and then later, we receive a post back to see the true processing state of the API call.

If each of these API calls takes 1 sec to process, and I am running 50k of these, then what would be my maximum throughput in external API calls/second?

Right now, it appears that since I am using a call to Database.executeBatch(batchToStart, TOTAL_BATCH_SIZE);, then everything is running sequentially, and I am getting 1 call API / per second.

I believe I could split the job up manually into batches and get a maximum of 5 calls / per second ( if no other jobs are running). Is this correct?

I can only have 5 jobs running in parallel, and there is no way to run the API calls in parallel within the job.

Additionally ( thanks to @identigral )

This post : Throttling http callouts across queueables executed in parallel

Indicates that if we move to Queuable, then we could get a max of 50 calls per second. Is this the correct conclusion?

I would like 1000 calls / second , do I need to move off of platform?

Answer 1

There's a variety of factors at play, but we can go through them. First, there's the start method itself. Only one start method can be running at once, so we would want optimally fast start methods to get up to speed as fast as possible. Second, there is typically a small delay between execute calls (I've seen it be as bad as about four seconds), so we want to minimize the number of execute methods, which means maximizing the number of callouts per transaction, ideally 100.

Assuming you can get that kind of situation, you could reach an average of 288 callouts per minute, assuming each callout lasts 1 second and a there is a 4 second delay between execute methods. 50k records at this speed takes about 2 hours, 53 minutes, and 36 seconds. This time would be significantly reduced during evening and night hours when the systems are less active; if we assume a 1/2 second delay instead, we get very close to 300 callouts per minute, or about 2 hours, 47 minutes, and 24 seconds. Not very much difference, since the callout times dominate the transaction speed.

On the other hand, if you use only one callout per execute, you're looking at 1 callout per second on average (across 5 batches), it would take close to 2 days, 21 hours, 26 minutes, and 40 seconds for 50,000 records given a 4 second delay between execute calls, or 13 hours, 53 minutes, and 20 seconds with a 0.5 second delay between each call. The actual value is going to be somewhere between those two extremes, perhaps around 26-28 hours.

Overall, the more you can do per transaction, the faster you'll get done. Keep in mind that execution units also have to deal with other transactions like Queueable, future, and scheduled methods, so keep those elements in mind when you design your Batchable process. Even a single Batchable that runs expediently should be sufficient to handle the volume you're talking about.