Platform Event batching

Question

I have a simple Platform Event called Timesheet_Approval__e.

As a test, I publish a batch of 1000 events with this anonymous apex:

List<Timesheet_Approval__e> approvals = new 
List<Timesheet_Approval__e>();
for (Integer n=0; n<1000; n++) {
    approvals.add(new Timesheet_Approval__e()); 
}
EventBus.publish(approvals);

In the subscriber trigger, there will be some fairly heavy-duty processing to be done, but for the moment I just have the following:

trigger Timesheet_Approval on Timesheet_Approval__e (after insert) {

  System.debug('*** APPROVALS '+Trigger.new.size());

}

so I can see how many events have been received.

I note that the initial batch of 1000 events is split into smaller batches of variable size, generally around 100, and the subscriber was triggered 11 times. The total number of events received is 1000, as expected.

Is there any way of controlling how many events are received in a single trigger invocation? I want to limit it because of the processing I need to to, to avoid hitting governor limits.

Any advice would be appreciated.

Answer 1

Well we do not have complete control over the batch size of the platform event. It can process upto 2000 events at once. You can try the below mechanisms to see you are able to split them.

Idea A :

Declare a dummy platform event which would be identical to the event you are supposed to fire. Dummy_Event__e.

Now when you are firing the actual event , fire the dummy event instead.

Write a trigger on Dummy Event and then create split them into multiple buckets and then for each bucket fire your orignal event.

Thus you would have to use multiple EventBus.publish(approvals); in your dummy event trigger , but if it fires and plays them separately in a different transaction, its fine.

Idea B :

Determine whats the max number of events you can process before breaking any limit. Then in your Timesheet_Approval split the events in multiple buckets you can handle and then process 1st bucket. For remaining bucket clone the events and fire them using multiple publish.

Answer 2

We can now configure the batch size that our Platform Event triggers process. The default size for this is 2000. This same configuration also allows us to modify who the trigger runs as (instead of the default Automated Process user).

Details of this capability and how to configure the PlaformEventSubscriberConfig can be found here.

This can be configured via either the Tooling API or the Metadata API.