Duplicate Lead creation from EmailService

Question

I have an EmailService that takes the data from an email and creates a Lead if no match is found for the supplied email address. If a Lead is found then the Lead is updated with the new details supplied.

This works fine when emails come in with a suitable gap. If 2 emails come in quick succession (ie. simultaneously?) then duplicate Leads are created for the same email address. Is there a way to stop this happening?

The code is essentially

global class leadCreationEmailService implements Messaging.InboundEmailHandler {

    global Messaging.InboundEmailResult handleInboundEmail(Messaging.inboundEmail email,
                                                                Messaging.InboundEnvelope env) {

    //parse email and get email, phone, etc...

    list<Lead> leads = [select id from Lead where email=:emailAddress];

        if(leads.size() > 0){
            l = leads[0];
            l.MobilePhone = phone;
            l.firstName = firstName;
            l.lastName = lastName;
        }
        else{
            l = new Lead();
            l.firstName = firstName;
            l.lastName = lastName;
            l.Company = 'my co.';
            l.email = emailAddress;
            l.MobilePhone = phone;
            l.LeadSource = 'Website';

            insert l;
        }

        result.success = true;
        return result;
    }
}

I expect its because there are 2 threads running concurrently and so the 2nd thread is doing the search before the 1st thread has finished.

I have tried moving the code to a future method to try and force a 'time gap' but this also fails.

UPDATE - putting code in a future method is more reliable than first thought; with the email processing done in the service and the DML statements in the async method.

Any ideas?

Answer 1

You can use "FOR UPDATE" to make sure that the "in-flight" lead will be found.

The moment you insert the record, it is saved to the database (see the Triggers and Order of Execution topic), but not committed until the end. FOR UPDATE allows you to select records that have reached this "temporary save" state, even if it's an insert and they don't "really" exist yet.

This means that your two handlers can interact with each other, as long as one handler or the other is a few milliseconds out from the other one. Odds are, this will happen. Of course, it's still theoretically possible that you'd get duplicates, but using locking statements greatly reduces that probability.

Answer 2

The first thing that comes to mind for me is running a batch every 15 minutes to look for recently created duplicates. I know there are other asynchronous approaches that will work well here (possibly better), but I have not had much luck for them. The system.scheduleBatch method worked very reliably for me when I did try to run a frequent process. Something like the below:

public with sharing class LeadDedupeBatch implements Database.Batchable<Lead>, Schedulable
{
    @TestVisible static Boolean reschedule = true;
    public void execute(SchedulableContext context) { Database.executeBatch(this, 10000); }
    public List<Lead> start(Database.BatchableContext context)
    {
        Datetime threshold = Datetime.now().addMinutes(-15);
        return [SELECT Email FROM Lead WHERE CreatedDate >= :threshold];
    }
    public void execute(Database.BatchableContext context, List<Lead> records)
    {
        List<Id> duplicateIds = new List<Id>();
        Set<String> observedEmails = new Set<String>();
        for (Lead record : records)
        {
            if (!observedEmails.contains(record.Email))
                observedEmails.add(record.Email);
            else
                duplicateIds.add(record.Id);
        }
        Database.delete(duplicateIds);
    }
    public void finish(Database.BatchableContext context)
    {
        if (reschedule) system.scheduleBatch(this, 15);
    }
}

You could use merge to perform your de-duplication, but here it sounds like you may just want to delete the duplicate records that shouldn't have been created. It wouldn't be too much more complicated, though. It just gets a little tricky because if there are more than two records, you have to do multiple merges, and you need at least one merge per email address, so you consume your limit on DML Statements quickly.

The above is just an idea to get you started. I know there are probably better ways to chain jobs, but this approach was pretty easy to get rolling, pretty easy to tear down, and pretty easy to test.

If deleting does not meet your needs (for example if there are side-effects for insertion you'd rather avoid), your best bet is to create a separate object to track these inbound emails, and use a similar batch structure to iterate over this shadow object and de-duplicate, insert, and update based on that.