6

Use case:

  1. Apex Foo.trigger subscribes to Foo__e platform event
  2. Hundreds of events happily processed
  3. On event n, Apex trigger executes and, for application-defined reason, decides it has to throw EventBus.RetryableException. Before throwing exception, trigger checks EventBus.TriggerContext.currentContext().retries to be sure value is < 9 as per doc
  4. SFDC then decides to stop any new Foo__e event from being consumed until hours (and I mean 10-30 hours) later.

Observed in both PROD na30 v43.0 and sandbox V44.0

Code fragment is here:

try {
   ...do DML
}
catch (Exception e) {
   if (e.getMessage().contains('UNABLE_TO_LOCK_ROW') ) {
     if (EventBus.TriggerContext.currentContext().retries < 9) {
       throw new EventBus.RetryableException(...msg...);
     }
     else {
       ... fallback alerting
     }
   else {
     ... non-retryable exception handling
   }
}

How I verified:

  • I had timestamp of each consumed Foo__e event
  • I had persisted a Log__c (via a Platform Event so it wouldn't be rolled back) when I threw the RetryableException
  • The timestamp of the RetryableException aligned to the minute of when I last successfully consumed a Foo__e. 30 hours later, SFDC magically started allowing Foo__e to be consumed.

Support case 20248182 filed

  • Can you add the support case number here? – Daniel Ballinger Oct 2 '18 at 18:52
  • 1
    @DanielBallinger - DONE – cropredy Oct 2 '18 at 21:06
  • 1
    The support case definitely seems like the way to go on this. I've pinged two applicable PM's to see if they have any input to share. I wonder if there is something about the retry window for events going on here. Only once the problem event drops out the retry window does it get going again. That's only a random guess on my part though. – Daniel Ballinger Oct 2 '18 at 21:20
  • @DanielBallinger - Thanks - I saw your tweet to PM; I had contacted him earlier when I observed same event in V44 which I "blew off" as a pre-release issue; but when it happened in PROD v43, case was filed – cropredy Oct 3 '18 at 0:08
  • Let us know if a known issue comes out of it. Definitely seems like something where the resolution will come from withing Salesforce. – Daniel Ballinger Oct 3 '18 at 0:10
2

@cropredy The engineering team has identified the issue -- the message passed in to RetryableException is too long (more than 255 characters) and that caused an error when we were persisting the state to the DB. Because the state wasn't persisted correctly, the Trigger subscription became idled until the server was restarted (causing the subscription to be re-initiated). We have a bug on our end which needs to handle truncating the message to < 255 characters, and we will be fixing that. In the mean-time, can you can manually truncate the error message in your code which should resolve this issue.

  • Wee hah! Nice piece of diagnostics and explains why no one else has seen this. – cropredy Oct 5 '18 at 19:37
2

https://twitter.com/_johnbrock/status/1047236441548279808

Thanks Daniel -- I am aware of the issue and tracking the case internally. We are escalating to our engineering team to take a look.

  • John Brock (Salesforce PM)

Sounds like they are on to it.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.