0

I am creating a salesforce deduping application using python. The issue I am running into is that deduping applications literally take hours to run through large databases. How do others execute programs that take hours to complete when salesforce has built-in governor limits?

Is there a way to bypass these limits?

Basically I want to pass in the entire database of accounts into a python application that will run a deduping match based on some percentage similarity. Then have a human verify the records with a basic UI that says "Yes" or "No" - Writing this out so everybody has some general context around what I am trying to go for here.

I am trying to keep the app in Salesforce, but handle all the processing in Python to avoid the governor limits.

  • The governor limits apply, but apply differently, if you're doing most processing off-platform. For example, you'll probably want to pull your Accounts down with the Bulk API and post-process them in Python to identify duplicates. I wrote code to do this in Raiser's Edge (!) many years ago using Python and a simple Flask UI, and it worked very well. – David Reed Nov 25 '19 at 20:10
  • @DavidReed Are you allowed to share such code? Interested to understand your overall foundation. I feel bad asking, but worth a shot! – Matthew Metros Nov 25 '19 at 20:12
  • It's old, nasty code, but sure. trapeza and trapeza-import. – David Reed Nov 25 '19 at 20:23
  • @DavidReed I think my question might have been misinterpreted - I am looking to keep the app in Salesforce, but have the actual code live in a python app via flask. I am thinking this way, because deduping is a long-running operation and I think Salesforce governor limits would not be able to hand this. How are these other deduping apps handling these long running operations? Does their code live in APEX or are they calling a script outside of the platform? – Matthew Metros Nov 25 '19 at 20:26
0

Salesforce limits apply, when you executing operations in SF.

Your python app will first retrieve all data from SF(Probably using BULK API) and then process it in python code, probably then upload back into SF.

The only limit you have to worry about is API limit and Bulk Job limits

  • Well to be specific not only, because during load of the data, even using Bulk API all automation processes are running and those can hit the limits within one transaction. – user1974566 Nov 25 '19 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.