0

Let's say I have a generic "container" record that is used to store JSON responses from third-party API's. These Container records have a single field 'data', which is a JSON string identifying various values

I want to take this data and use Apex to parse it into an Apex map and use the data to create new records with it. And I want to do this for potentially large data sets. 

for(containerRecord__c record : listOfContainerRecordsToParse) {
   List<Application__c> applicationsToCreate = new List<Applications__c>();
   Map<String, String> data = ...parse record JSON data field into an Apex map...

   Application__c app = new Application__c (
      Name = data.get('applicant_last_name') + ' - Application',
      ...Assign a bunch of other fields...
   )
   //Add new app to list
   applicationsToCreate.add(app);
};

//Insert the list of records
insert applicationsToCreate;

Ok, easy enough. But now what if I want to try and associate this new record with a Contact record (using some common matching, for example, same first and last name). My initial thought was I could do a SOQL query in the for loop trying to find a matching contact...but for large numbers that would hit the governance limit, would it not?

But, because each contact will be unique, I can't think of a way of doing this that doesn't involve looping over each record and running a SOQL query inside of it to find the possible matching contact.

Has anyone done something like this? What would be the correct way to do this and not hit governance limits?

Improve this question

1 Answer 1

Reset to default
0

Your first instinct (no soql in for loop) is correct.

At a high level ...

You need two passes - one to collect all the first/last names in the parsed container and build a skeleton Application__c; then query Contacts into a map, then loop back through applications and apply the lookup Id by searching on the map keys.

More details

You'll need a map of applicationsByContactName so pass 2 can find the right Application(s) matching your query

Improve this answer
5
  • Interesting. If you have a large amount of contacts and you try to query them all into a map, would that also not hit the total number of records retrieved limit (50,000)?
    – SCVandal
    Sep 23, 2019 at 21:42
  • @SCVandal you're only querying those contacts that match the incoming container's first/last. Highly unlikely a single apex transaction can handle 50,000+ incoming json containers without blowing up on heap or cpu time so you may have other, more difficult to resolve governor issues depending on the incoming transaction. Plus, you are limited to 10,000 DML rows per transaction
    – cropredy
    Sep 23, 2019 at 21:45
  • Ah ok, so when you build the list of contacts, the limit is on the number you retrieve, not on the number you query? For example if I expect [SELECT Id FROM Contact WHERE FirstName =:firstName AND LastName =:lastName] to return, say, 2500 contacts, that will be the limit. Even if it took me having to query every contact in my org (let's throw out a huge number; 100,000 for example), correct?
    – SCVandal
    Sep 23, 2019 at 21:55
  • @SCVandal - correct-a-mundo (that is, your assertion is correct, the limit is on the number of rows retrieved, not # of rows in database )
    – cropredy
    Sep 23, 2019 at 21:56
  • Fantastic. Thank you so much for your answer and the follow-up! Very insightful!
    – SCVandal
    Sep 23, 2019 at 22:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .