0

Ok the situation i have is I have a nightly batch job that reads all picklist items and then puts them into an object so we can offload them to a data warehouse

If the item already exists (combo of object + field + code) i want this to be ignored and to continue in the batch job

i attempted creating a matching rule and a duplicate rule to prevent duplicates, but what is occurring is the moment it finds one duplicate, the job fatally errors. I want the job to say ok, dupe here, ignore and keep running. How do i prevent a fatal error in this situation? Im also not married to matching and duplicate rules if theres a more reasonable solution

1 Answer 1

1

To ignore duplicates, just set the optional allOrNone flag to false:

Database.insert(records, false);

Alternatively, you could use an external Id field, set that value to a unique hash value, then upsert against that. This has an advantage of not needing to worry about duplicates, because it is both an insert and update operation in one (the updates would be harmless, because it would always be the same computed value).

PicklistEntry__c[] entries = new PicklistEntry__c[0];
for(sObjectType sType: Schema.getGlobalDescribe().values()) {
    for(sObject field: sType.getDescribe().fields.getMap().values()) {
        for(PicklistEntry entry: field.getDescribe().getPicklistValues()) {
            String objectName = ''+sType;
            String fieldName = ''+field;
            String value = entry.getValue();
            String computedHash = 
                EncodingUtil.convertToHex(
                    Crypto.generateDigest('SHA3-512', Blob.valueOf(objectName+'.'+fieldName+'.'+value))
                );
            entries.add(
                new PicklistEntry__c(
                    ObjectName__c=,
                    FieldName__c=''+field,
                    Value__c=entry.getValue(),
                    ExternalId__c=computedHash
                )
            );
        }
    }
}
Database.upsert(entries, PicklistEntry__c.ExternalId__c);

You could always query the database beforehand to see if the values exist, and if not, then insert them. This is slightly more work, but would be more efficient overall. If you design this right, you could also have a way to delete entries that no longer exist. You can use a Set or Map to determine which values to exist, and which you need to create and delete.


Keep in mind that, it sounds like your current code doesn't deal with deletions. If you delete/purge all the records before inserting, that should resolve all your duplicates, and even eliminate the need for a duplicate rule.


So, depending on your needs, there's at least four possible solutions you could use.

1
  • the delete purge method is how i wanted to do it, but the warehose db says the new timestamp will trigger an update on his end. so ended up going with the query beforehand on a concatenated primary key, seems to be working out great so far, will do some further tests then accept answer
    – Jay Rizzi
    Apr 26, 2022 at 15:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .