Many platform events per transaction and affect on Heap

Question

Given

• 100 platform events published for Foo__e in the same second
• Each platform event represents 100K of data (10MB data altogether across the 100 events)

When

• SFDC presents the events to the subscribing trigger Foo.trigger

Then

• Does the SFDC event "batcher" consider whether the heap size of Foo.trigger would be breached when deciding on the batch size?

In my example, would the SFDC event batcher present only < 60 events (< 6MB) as that is less than the 6MB Heap avail?

Rationale for question: I have a 3rd party system that is publishing events, each with payloads of 256K. If the upstream publisher publishes many events all at once, the trigger's heap could be breached.

I know that summer 19 gives me more control over the size of the Trigger batch for processing (CPU/heap limit management) but not sure this will solve the trigger blowing up in the first place before it ever gets out of the starting block.

Answer 1

This is an example of a question caused by overthinking

Virtually no heap is consumed by a trigger upon entry (< 10,000 bytes). As such, even if you had 100 platform events per transaction and each were 100K bytes of data each, then the trigger's heap consumption upon entry is miniscule.

When you think about it (after overthinking it), it has to be this way. Here's a thought experiment:

Imagine you had Opportunity with 500 custom fields (not unusual for sure). During Data Loader, you update 200 Opportunities. Opportunity.trigger would almost certainly fail immediately if Trigger.new was included in heap. And the Lightning Platform would be near useless.

The coding solution for Platform Events can be found in the Platform Events Developer Guide

It would look something like this (assuming the platform events need to be converted into sobjects)

trigger FooTrigger on Foo__e (after insert) {
  Bar__c[] bars = new List<Bar__c>();
  for (Foo__e f: Trigger.new) {
    bars.add(new Bar__c (
         Field1__c = f.someFieldA__c,
         ...));

    // set the checkpoint
    EventBus.TriggerContext.currentContext()
                           .setResumeCheckpoint(f.ReplayId);
    if (Limits.getHeapSize() >= Limits.getLimitHeapSize()) {
       break;  // we don't want to rollback work-in-progress 
    }
  }
  insert bars;  // database commit   
}