1

We are working on a package where the data from salesforce needs to be exported to external database instantaneously on DML. The export can be configured such that the user can select the Standard/Custom objects and the fields to export. Our pain point is Triggers which makes the future callouts needs to be implemented on each object that needs to be synced. So, if a new custom object is created which needs to be synced, trigger needs to be created with a specific lines of code.

Question is, can Streaming API be used here? Based on the configuration, we can create PushTopics dynamically from Apex. But where do we place the code to make the callouts? If I understand Streaming API correctly, subscriber could be either be the VF Page or Java code. Java program needs to be hosted on Heroku or likewise. But can VF pages be used to make callouts as soon as it gets a notification. Also, does the VF page needs to be active all the time, as in does someone need to baby sit to make sure session doesnt expire?

Improve this question

1 Answer 1

Reset to default
0

As it stands currently at Winter '16, the Streaming API is not a good fit for constantly and reliably syncing data to an external system.

Yes, you could have an external Java program setup and subscribe to a PushTopic. However, as per Message Reliability:

Streaming API doesn’t guarantee durability and reliable delivery of notifications. Streaming servers don’t maintain any client state and don’t keep track of what’s delivered.

I've heard rumors that this may change in the future, but as of now it is more useful for integrations where you don't need every message delivered.

I haven't seen a solution that 'instantaneously' syncs large volumes of records out to an external system. I have seen various combinations of outbound messages and custom queue records with scheduled batch apex provide reliable syncing of data to external systems.

From comments:

can [we] avoid hosting the Java program on Heroku and rather, subscribe the push topics within salesforce by any other means but Visualforce.

No, you won't be able to subscribe to the push topics from Salesforce outside of a user having a Visualforce page open in their browser. It isn't actually Salesforce that has subscribed to the stream. It is the CometD client in the users web browser. The long polling that is used requires keeping a port open to listen for updates. You can't do that from Apex. Salesforce won't let you sit indefinitely in Apex with a port open.

Improve this answer
4
  • Yes, I read about Message reliability. That's a chance we are willing to take. However, the question here more is if we can avoid hosting the Java program on Heroku and rather, subscribe the push topics within salesforce by any other means but Visualforce.
    – Kunal Shah
    Commented Oct 27, 2015 at 20:47
  • @KunalShah No, you won't be able to subscribe to the push topics from Salesforce outside of a user having a Visualforce page open in their browser. It isn't actually Salesforce that has subscribed to the stream. It is the CometD client in the users web browser. The long polling that is used requires keeping a port open to listen for updates. You can't do that from Apex. Salesforce won't let you sit indefinitely in Apex with a port open.
    – Daniel Ballinger
    Commented Oct 27, 2015 at 22:23
  • Thanks for the information. I just wanted to be 100% sure if there are any other ways. Appreciate your help.
    – Kunal Shah
    Commented Oct 28, 2015 at 22:43
  • My experience with Streaming API is that it is not 100% reliable, at least on the NA26 cluster where we have it running. We use it to generate notifications on new Leads and of the 200-300 we get daily, we see about 99% reliability, so we have an outside batch job that runs hourly to pick up any that were missed.
    – AlwaysThinkin
    Commented Nov 10, 2015 at 15:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .