Batch avoid inserting duplicate records

Question

In a batch that deserializes a large json in the start method, iterates over its rows and creates new records in execute. How to check if there are already records with that external id (field on the Account object) in the system and skip inserting those rows?

Answer 1

Can you set that external Id as Unique on the Account object ? If yes, then you can use in the execute part a Database.insert(recordsToInsert, allOrNone). Set the boolean to false, that way if you try to insert an already existing record it will fail but the other records will nevertheless be successfully inserted.

Answer 2

Instead of using insert operation use upsert operation with external Id.

E.g. if your list name is accountsList and External Id field name is ExternalId__c

replace your insert DML with below dml statement. It will update existing record by external Id instead of creating new.

upsert accountsList ExternalId__c;

If you don't want to update existing record also then you have to filter out accounts to be inserted based on External Id of existing account.

Code snippet:

List<Account> accountsList = AccountsListTOBeUpdated;// passed from iterator to execute method.
Set<String> newExternalIdSet = new Set<Id>();
Set<String> existingExternalIdSet = new Set<Id>();
for(Account account : accountsList) {
    newExternalIdSet.add(account.ExternalId__c);
}

for(Account account : [SELECT Id, ExternalId__c FROM Account WHERE ExternalId__c IN :newExternalIdSet]) {
    existingExternalIdSet.add(account.ExternalId__c);
}

List<Account> filteredAccountsList = new List<Account>();
for(Account account : accountsList) {
    if(!existingExternalIdSet.contains(account.ExternalId__c)) {
        filteredAccountsList.add(account);
    }
}

insert filteredAccountsList;