2

The system has batch call out APEX classes to sync Salesforce information with another Email sending provider, Active Campaign. Every time we insert or update a custom lead or custom opportunity, the system will call out the batch classes to communicate with Active Campaign so we can send proper emails to our customers.

Everyday we have a plenty of records to change on Lead and Opportunity. Occasionally, the system reaches the 100 limit for Apex Flex Queue. Therefore, I modified the code when the 100 limit for Apex Flex Queue has been reached, new tasks will be scheduled in Scheduled Jobs. However, today both the Apex Flex Queue and the Scheduled jobs limit have been reached, which means at the backend Salesforce queued 200 tasks in the system.

I would like to know whether there are people who have experienced the same situation and how they solved this problem. I believe the code has been optimised to its maximum capacity, but if you want to see my code, please tell me. One of the reasons I do not put my code here is that there are four batch classes and three triggers to execute the whole process. There would be too much to put on the forum.

Thanks,

1
  • Check out the appleman advanced apex 3rd edition book chapter 7 for an async apex pattern that won't run into your issue.
    – cropredy
    Feb 28, 2016 at 18:23

3 Answers 3

5

In situations like this I've added an intermediate queue custom object to represent the records that need to be processed by the batch.

Then, rather than immediately starting a batch job every time a record changes, it gets added as a reference in the custom object. The batch jobs now run over the records in the queue object.

This has a number of advantages:

  1. You only need a single batch job running on a schedule
  2. The individual batch job can process all the outstanding records in one go. This might reduce the number of outbound API calls you need to make. Bulkification!
  3. You can flag individual records in the queue that have problems while continuing to process other records.
  4. If a record is already in the queue to be processed you don't need to add it again. This improves the situation where someone makes several changes to a record in rapid succession.

A more elaborate example of this approach was presented at Dreamforce in the Message Queuing - Building a Messaging Framework on the Salesforce1 Platform talk by @anup.

10
  • Thank you for your reply. The business requirement needs the records to be sync in real-time. If we sync, say, once in 24 hours, the changes might be too late to make. Some other people suggested me to use future method. I actually am not quite sure if future method can work with our business requirements. Do you have some explanations or do you know where I can find the resources to look at?
    – Yifei Pei
    Feb 18, 2016 at 23:26
  • With a scheduled batch job you can get the run time down to fairly frequent. I've had jobs running every 5 minutes. Others every 15 minutes. There was a Dreamforce 2014 presentation called "Building a Messaging Framework on the Salesforce1 Platform". If you can find the video it outlines one end-to-end approach. Feb 19, 2016 at 0:26
  • Thank you Daniel. I think this solution is good for my current situation. Are there more articles to explain how to do it? Is there any example to show the steps I need to follow?
    – Yifei Pei
    Feb 23, 2016 at 1:29
  • @YifeiPei I've linked to a Dreamforce Presentation that covered an advanced version. You should be able to simplify it based on your requirements. Feb 23, 2016 at 1:51
  • You did reply quite promptly! I haven't seen the video but I have seen the slides. The content is quite high level to focus on the architecture. Are there instructions on some detailed examples?
    – Yifei Pei
    Feb 23, 2016 at 3:16
0

As a beginner of Stack Exchange, I am not sure whether I need to open a new question or make it a sub topic of this question. Anyway, if you think I should make a new question, later I will make one.

For this callout, there is a class to delete existing "tags" in the external system and then later add the new "tags". Therefore, an update will use two classes to callout. The first class to delete "tags" and the second class as the finish method of the first class to add "tags".

The delete class make an array to contain the full options of "tags" and use a loop to callout the delete API one by one. This is really not efficient. However, after checking the API, I found out that there isn't an API code to "check whether a tag is existing and delete it". That is why the system is deleting the whole array one by one now.

The API document is here http://www.activecampaign.com/api/overview.php

Our classes has only used the "Contact" part of the API. contact_sync and tag_remove are the major commands to use.

0

Perhaps we are going to use @future method as a temporary measure to see how it performs. Later if still have problems, I think this object reference method is a good long term solution.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .