Salesforce call Heroku Web Service

Question

We have a salesforce application, and the application triggers a callout to a heroku service that gathers some data from a third party database. After gathering and processing the data from the third party database, Heroku service need to updated records back in Salesforce.

Challenges: The data process time maybe long and data set is large, the callback from Heroku to Salesforce cannot happens in 2min.

Solution: We are thinking to bring this data retrieving and updating salesforce part to an asynchronize process. So that after data is ready, Heroku triggers a bulk update to the salesforce account who send the request.

Question: Is my solution doable? Since the requests from Heroku to salesforce need be authenticated, and our service need to update data within SF for different SF users who send the request, how would the Heroku talk back to salesforce be done?

Thank you in advance.

Answer 1

Salesforce Connected Apps

I would recommend you consider whats known as a Connected App in Salesforce. This uses the oAuth authentication system on Force.com, which follows a UI flow (provided by Salesforce) to allow the Heroku app to obtain whats known as a "oAuth token" which it can store. Using this approach none of the user/password credentials are ever known and or stored by the Heroku app. Furthermore the Salesforce administrator can review connected app connections and revoke access at anytime if needed.

Proposed Approach.

• Doing work on the users behalf. To ensure the Heroku service makes updates on behalf of the user, each user would register with the service via an oAuth UI flow. Which can be started by giving them a link to your Connected App, this is a one time only action, if they are already logged in, will simply display a prompt to confirm they permit the service to perform operations on their behalf.
• Worker Queue Custom Object. Next create a Custom Object as a kind of Work Queue, records here are populated via Apex Triggers associated with events in Salesforce that instigate the work.
• Heroku Worker. Finally a Heroku scheduled process reads from the Work Queue, using the OwnerId field (User Id) to map to the oAuth tokens previously registered with the service and call the Salesforce REST API with the appropriate token. Deleting items from the Work Queue once processes successfully.

Heroku Application Architecture

You essentially need to consider a Heroku architecture with both a web worker to allow your users to connect via the browser and register with it and also a scheduled worker to poll the work queue. This article from the Heroku site is a good starting point.

Background jobs are key to building truly scalable web apps as they transfer both time and computationally intensive tasks from the web layer to a background process outside the user request/response lifecycle. This ensures that web requests can always return immediately and reduces compounding performance issues that occur when requests become backlogged.

• Building your Web Worker. Build a page which when visited will respond by asking the user to authenticate themselves via the oAuth flow, when complete will store the oAuth token in your applications storage layer. Salesforce provide a number of API toolkits, my personal preference (given I'm a Java guy), is the poorly named Salesforce Database.com SDK (don't worry it works with regular orgs!). This contains an implementation of the oAuth API, so its simply a matter of write a Java controller to capture the token.

• Building your Background Worker. The big reveal here, is a background Worker is nothing more (in Java terms) than a simple Java class with a 'main' function. Thats it. Basically when it runs it connects via the Salesforce Web Service Connector (WSC) to query the Work Queue, grouping by OwnerId for better performance. Then establishing new WSC connections for each group of Work Items, by mapping the OwnerId to the oAuth tokens in the apps storage layer.

One possible other consideration is the Salesforce Streaming API to monitor your Work Queue real time. In this case the Herok worker would not be scheduled, but would run all the time, effectively waking up when records are inserted on the Work Item custom object. If you want to consider this keep in mind the message reliability statement here.

Answer 2

I think this sounds perfectly possible, you can have Salesforce make the requests to Heroku passing along the ID of the user performing the operation. You can use Heroku's scheduler to look for completed jobs on the Heroku side and have it send the data back to Salesforce. For that you'd want to use an integration user with a single set of credentials, either specifically set the OwnerId fields on the new records to the user who originated each job.

If data ownership was better left on the Salesforce side you could either use a trigger or a custom Apex REST webservice (probably the better option).