5

I need some help

I have looked at conventional methods of using De-Duping tools however because i work for an organisation which is multi alliance the de-duping tool looks at the entire database, when i actually need to look at segments of the database

SO i have decided to try and create my own de-duping tool.

So far i have created the following apex . The apex currently looks at the company name on the lead and if there is an exact match with another company name in the database it provide the user the error message ‘another new lead has the same company name”

This is great if the company name is exact, however i need it to be more flexible .

For example if “Burger King Limited” was a lead created in 2012, and a seller has decided to create a lead called “Burger King LTD” in 2013 which is the same company as the lead which was created in 2012. I want to build a fuzzy logic, that looks at the new lead and if there is Slight resemblance then disregard the new lead

Trigger DuplicateLeadPreventer on Lead
                               (before insert, before update) {

//Get map of record types we care about from Custom Setting
 Map<String, Manage_Lead_Dupes_C__c> leadrtmap = Manage_Lead_Dupes_C__c.getAll();




 //Since only certain leads will match, put them in a separate list
 List<Lead> LeadstoProcess = new List<Lead> ();

 //Company to Lead Map
 Map<String, Lead> leadMap = new Map<String, Lead>();

    for (Lead lead : Trigger.new) {

     //Only process for Leads in our RecordTypeMap
         if (leadrtmap.keyset().contains(lead.RecordTypeId) ) {

        // Make sure we don't treat an Company name that 
       // isn't changing during an update as a duplicate. 

              if (
                 (lead.company != null) &&
                 (Trigger.isInsert ||
                 (lead.company != Trigger.oldMap.get(lead.Id).company))
                 ) 
                 {

                    // Make sure another new lead isn't also a duplicate 

                        if (leadMap.containsKey(lead.company)) {
                            lead.company.addError('Another new lead has the '
                                            + 'same company name.');
                        } else {
                            leadMap.put(lead.company , lead);
                            LeadstoProcess.add(lead);
                        }
                }
    } //end RT If Check
    } //End Loop

    /*
     Using a single database query, find all the leads in 
     the database that have the same company address as any 
     of the leads being inserted or updated. 

   */

    Set<String> ExistingCompanies = new Set<String> ();

            for (Lead l: [Select Id, Company from Lead WHERE Company IN :leadMap.keyset()
                             AND RecordTypeId IN :leadrtmap.keyset()]) {
                          ExistingCompanies.add(l.Company);
                }

    //Now loop through leads to process, since we should only loop if matches
    for (Lead l : LeadstoProcess) {
        if (ExistingCompanies.contains(l.company) ) {
             l.company.addError('A lead with this company '
                               + 'name already exists.');
        }
    }
}
  • 2
    Welcome to SFSE, Masond3. While de-duping is definitely a Salesforce topic, I wonder if a question regarding fuzzy logic might be better served on Stack Overflow since it is more of a general programming concept. – Mike Chale Apr 16 '13 at 10:52
  • You should look at an AppExchange App like CRMFusion DupeBlocker or DataScout for eg, I've used DupeBlocker which you can set to run in the user's context, rather than overriding security, I've used it to dedupe only against the silo of data visible to the user. The complexity of something homegrown will grow rather rapidly as more business needs surface. – techtrekker Apr 16 '13 at 19:39
  • 3
    Food for thought: getLevenshtein as pinched from apex code string methods reference, and +1 on techtrekker and WesNolte's comments – bigassforce Apr 17 '13 at 0:06
  • I came across this website fuzzymatchcompanynames.com. This one claims to fuzzy match company names. It claims to take care of acronyms and even mergers! I tried with small contact list, and it did match some those were not exact match. Can some one try with a bigger list and share experience. – user22492 Jul 28 '15 at 12:58
7

Fuzzy matching on data is an entire industry, and due to its programmatic complexity is not well-suited - in it's most advanced forms - to the Force.com platform. Most companies that offer comprehensive data de-duplication and other master data management services have their core engine outside of Salesforce.

What you're trying to build is admirable and definitely very fun, but in practical terms not possible unless you create very hard limits on the scope of what you can dedupe i.e. exact, or near-exact matches. I do know a particular company that has built fuzzy logic on the platform for this purpose but it was very difficult and quite limited, and it was their entire company's purpose!

If you do want to try out some of the concepts and algorithms that are being used elsewhere have a look at the background and academics behind record linkage.

  • Hi thank you for your feedback / answers however current de duping tools on the market place are not fit for our organisation . I have one instance but 5 different banks , the de duping tools are applied to the whole date base so looking at 5 banks. However I need to apply to only one bank. That's why attempting to build my own de/duping tool. I have found the following link which looks like it solves my issue salesforce.stackexchange.com/questions/9619/… – Masond3 Apr 16 '13 at 19:48
  • 1
    @Masond3 check out this outstanding writeup on the use of Levenshtein distance algorithm – bigassforce Apr 17 '13 at 0:08
  • Hi user320 well it looks like I am on the correct track I just need more thorough research thanks for the pointers in the right direction :) – Masond3 Apr 17 '13 at 0:16
0

If you just pass the name and address of your new prospective lead to my AccountsByAddress and it will return the list of (fuzzy) matching Accounts. If you want Leads instead you can adapt (or copy) the code and change all the stuff like BillingCity to City etc.

  • Hi Marc Thank you for your swift response. In regards to your post, do you mean use your logic in the section where you have" Find Accounts Matching Address". Or do i need to use the whole logic including the Phonetic Algorithm – Masond3 Apr 17 '13 at 13:13
  • Well AccountsByAddress needs the phonetic and helper (ops) functions so copy it all into YourClass. Make AccountsByAddress public static and you can call them via YourClass.AccountsByAddress. – Marc Apr 19 '13 at 8:32
  • Hey Marc . Thank you for your answer. I may need some help implementing this into my sf org. Are you contactable by any other methods than via this ? – Masond3 Apr 21 '13 at 15:13
0

I recently have had a need to implement some sort of basic fuzzy matching. I am wondering if anyone has considered implementing a "trie" as a solution to approximate searching needs. A form of search tree seems like it would work well if the script limits can be avoided. Does anyone have any experience with this? I have not been able to find anyone else having done this in my (not thorough) research so far.

0

Fuzzy logic matching is a difficult problem to solve "casually." Try an existing algorithm like FuzzBox: http://sullivansoftwaresystems.com/cgi-bin/fuzzy-logic-match-algorithm.cgi

  • you can share reference links in comments below the question rather than separate answer.. or if you can add details around how the reference can be applied in the context of the current question, it will be useful. – Vamsi Krishna Gosu Sep 30 '14 at 1:23
  • @VamsiKrishna he doesn't have sufficient reputation to be able to make comments yet. – crmprogdev Sep 30 '14 at 1:43
  • my bad, i didn't notice that.. thanks @crmprogdev – Vamsi Krishna Gosu Sep 30 '14 at 1:46
0

LeanData makes a solution that fuzzy matches leads to accounts in Salesforce. They also have auto-routing of leads and campaign reporting tools.

0

Fuzzy matching is a complex method to develop and time-consuming as well. Few companies like full circle insight and Vyakar commit that they have developed advanced fuzzy match algorithm but I think it’s all about software output, credibility and how accurate the tool performs.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.