2

I'm implementing Database.Merge to merge duplicate Account records in bulk, but it allows only 3 records to merge. If I perform this dml in loop I'll get Too many DML statements: 151 error, so I have created batch apex and limiting the data, I'm wondering is this the best way to do this

global class DuplicateMergeBatch implements Database.Batchable<sObject>, Database.Stateful {

public String queryString;

global Database.QueryLocator start(Database.BatchableContext BC) {
  String ph = '814606';
  queryString = 'SELECT Id, Name, Phone, Type FROM Account WHERE Phone =:ph ORDER BY CreatedDate ASC';
    return Database.getQueryLocator(queryString);
}
  global void execute(Database.BatchableContext BC, List<Account> scope) {

Map<String,List<Account>> accountMap = new Map<String,List<Account>>();
  Set<String> accountName = new Set<String>();
  Set<String> accountPhone = new Set<String>();
  Set<String> accountType = new Set<String>();

  for(Account a :scope){
     accountName.add(a.Name.toLowerCase());
     accountPhone.add(a.Phone.toLowerCase());
     accountType.add(a.Type.toLowerCase());
  }

  //Get All non formula fields of account
  String sobjectApiName = 'Account';
  Map<String, Schema.SObjectType> schemaMap = Schema.getGlobalDescribe();
  Map<String, Schema.SObjectField> fieldMap = schemaMap.get(SobjectApiName).getDescribe().fields.getMap();
  List<String> fieleList = new List<String>();
  for(String fieldName : fieldMap.keySet()){
     Schema.DescribeFieldResult fieldResult = fieldMap.get(fieldName).getDescribe();
     if(!fieldResult.isCalculated() && fieldResult.isCreateable() && !fieldResult.isExternalId()){
        fieleList.add(fieldName);
     }
     /*if(fieldName != 'isPartner' && fieldName != 'IsCustomerPortal'){
        fieleList.add(fieldName);
     }*/
  }

  String fetchFields = 'SELECT '+String.join(fieleList,',')+' FROM '+sobjectApiName+' WHERE Name IN :accountName AND Phone IN :accountPhone AND Type IN :accountType ORDER BY CreatedDate ASC limit 5';
  integer i = 1;
  for(Account a : Database.query(fetchFields)){
     String key = a.Name+a.Phone+a.Type;
     key = key.toLowerCase();
     if(accountMap.containsKey(key)){
        accountMap.get(key).add(a);
     }else {
        accountMap.put(key,new List<Account>{a});
     }
     i++;
  }
  System.debug('i >'+i);
System.debug('accountMap >> '+accountMap);
  List<Account> mergeList;
for(String key : accountMap.keySet()){
     System.debug('Size >>'+ accountMap.get(key).size());
    if(accountMap.get(key).size()>1 && accountMap.get(key).size() <6){
        mergeList = new List<Account>();
        Account masterAccount = accountMap.get(key)[0];
        mergeList.addAll(accountMap.get(key));
        mergeList.remove(0);
        System.debug('mergeList >> '+mergeList);
        for(Account a : mergeList){
            Database.MergeResult mergeResult = Database.merge(masterAccount, a, true);
        }
    }
}

//if(mergeList.size()>0)
//  update mergeList;

}

global void finish(Database.BatchableContext BC) {

}}

to execute this batch :

Database.executeBatch(new DuplicateMergeBatch(), 5);

2 Answers 2

7

This is a rather old-school method of merging duplicates. We built a better system in our org that simply queries the DuplicateRecordSet table for duplicates and merges them. Actual deduplication logic is instead made a Duplicate Rule; the system's built-in matching logic reduces the need to do anything complicated, just query and merge. It also allows us to adjust the rules if need to without a new deployment.


You have a lot of minor things wrong with your implementation, and a few big things wrong. One of the biggest errors you have here is:

 String key = a.Name+a.Phone+a.Type;

This could have some implications. Without some sort of separator, it's entirely possible (mostly through bad data entry) that you might mismatch records. For example:

Account Name:  Five5     Five
Account Phone: 55-1234   555-1234

While this is kind of silly, it can, and does happen. It may be the same account, but maybe not. I'm sure there's real-world examples where this can get you in trouble. Instead, you can use logic like I demonstrate in this answer.

Account key = new Account(
  Name=a.Name.toLowerCase(), 
  Phone=a.Phone==null? null: a.Phone.toLowerCase(), 
  Type=a.Type==null? null: a.Type.toLowerCase());

While it's a bit more verbose, it's guaranteed to keep the fields separate, regardless of the data in those fields.


You don't need to query every single field on the Account. Salesforce will only update values that are specified in the master record. Just query for the fields you're actually using.


Don't use Database.Stateful unless you have to. It's a performance hit. And no, you're not storing any state, so you don't need to use it (even queryString isn't being modified during the execute method, so does not need to be stateful).


What's the purpose of i? Maybe consider documenting this with comments...


if(accountMap.get(key).size()>1 && accountMap.get(key).size() <6){

Why 6? Your code will fail if there are more than 3 accounts to process in the same list. Also, you don't use the key in that loop. And, you copy a list for no good reason. Here's how this could be revised:

for(Account[] accList: accountMap.values()) {
    Account master = accList.remove(0);
    Account[] dupes = new Account[0];
    while(accList.size()>1) {
        dupes.add(accList.remove(0));
        if(dupes.size() == 2) {
            Database.merge(master, dupes);
            dupes.clear();
        }
    }
    if(!dupes.isEmpty()) {
        Database.merge(master, dupes);
    }
}

3
  • Hey, can you show me the non-"oldschool" way of doing it? I'm trying to implement a Database.merge functionality on the DuplicateRecordSet object, but I cant get it to work..
    – erikvm
    Oct 16, 2019 at 12:11
  • @erikvm Please feel free to ask a new question, including your attempts, and I'd be glad to take a look at it after I drop the kids off at school. Adding DuplicateRecordSet to this answer wouldn't be appropriate.
    – sfdcfox
    Oct 16, 2019 at 12:29
  • @erikvm I just saw your question pop up. I'll take a look.
    – sfdcfox
    Oct 16, 2019 at 12:30
0

Performing merge operation on bulk records in production would be really risk. I would suggest you should take this operation out of salesforce. For example i would export all accounts data in question and process csv to records to be deleted and records to be updates. after doing sufficient spot checking i would apply those changes in production. Note that you will have to process the child objects separably.

5
  • I'm agree it's risky to merge data in production, but database.merge fails to merge any single record it will roll back automatically and from csv we can't merge records of related contacts and opportunicaties Jun 25, 2018 at 6:13
  • Yes, you will have to do re-parenting of all child objects as i mentioned in my answer. But it will show you data at every step and you will have better control over merge process. Jun 25, 2018 at 6:24
  • The another important point is look at query "fetchFields". This will either fail or super slow on any large size org. Jun 25, 2018 at 6:25
  • 2
    @AtulRajguru9 Reparenting is automatic. That's the point of the merge function. All children will automatically move to the new parent.
    – sfdcfox
    Jun 25, 2018 at 6:29
  • @sfdcfox, I was talking about my approach. In my approach we will have to do Reparenting. Jun 25, 2018 at 6:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .