I am creating a salesforce deduping application using python. The issue I am running into is that deduping applications literally take hours to run through large databases. How do others execute programs that take hours to complete when salesforce has built-in governor limits?
Is there a way to bypass these limits?
Basically I want to pass in the entire database of accounts into a python application that will run a deduping match based on some percentage similarity. Then have a human verify the records with a basic UI that says "Yes" or "No" - Writing this out so everybody has some general context around what I am trying to go for here.
I am trying to keep the app in Salesforce, but handle all the processing in Python to avoid the governor limits.