1

We have two custom objects - obj1 and obj2. These are related by a custom Id. We need to iterate through obj1, get its relevant records from obj2 based on Id and further query for permission of each object in the user permission table and make external call outs for them. Obj1 has around 5 M records and each record can have around 10K records in obj2. I will be executing this through a batch job. Is there a possibility of hitting any governor limits in this scenario? Following is pseudo-code for execute method of the batch job.

  global Database.Querylocator start (Database.BatchableContext BC) {
        return Database.getQueryLocator('select id from obj1'); 
    }

global void execute (object1Scope) {
   For every object in object1Scope
      Fetch all relevant objects in object2
         For each object retrieved from object2 
                  fetch permissions from user permission table
      Multiple external callouts
 update 
}
  • 1
    Yes there is a near certainty of hitting governor limits... – Adrian Larson Nov 27 '19 at 13:52
  • @AdrianLarson Could you please tell me what limits I can be hitting and how I can avoid them in first place? – Neha Ummareddy Nov 27 '19 at 14:25
  • @AdrianLarson Is there any better way of solving this? – Neha Ummareddy Nov 27 '19 at 14:32
  • I might sound a bit fuzzy here , but i think that you should query your "obj1" records in start method and then based on that fetch "obj2" records in execute method.Also , you can do callouts in queuable which will be scheduled from your execute method of batch. Refer this link - ipfs-sec.stackexchange.cloudflare-ipfs.com/salesforce/A/…. Adjust batch size. – Rohit C Nov 27 '19 at 14:41
  • @RohitC Yes, I will be querying obj1 in start method which will further trigger execute. I have updated the question. Is this what you were suggesting? – Neha Ummareddy Nov 27 '19 at 14:57
3

Yes, you have a high likelihood - more like a certainty, really - of hitting a variety of limits in this scenario.

We need to iterate through obj1, get its relevant records from obj2 based on Id and further query for permission of each object in the user permission table and make [multiple] external call outs for them. Obj1 has around 5 M records and each record can have around 10K records in obj2

This certainly implicates

  • The callout limit, which is 100 per transaction.
  • The DML rows limit (10,000), if you make any updates on Obj2 and process more than one Obj1's related records per execute().
  • The SOQL rows limit (50,000), if you process more than five Obj1's related records per execute().
  • Possibly the SOQL queries limit (200 in batch) depending on how you structure your execute().
  • The heap size limit.
  • The CPU time limit.

Solutions involving spawning other asynchronous processes from your batch, in execute() or in finish(), risk a variety of other limits depending on the architecture, including the Apex flex queue limit, heap size, Asynchronous Apex invocations, and so on.

This process will require very careful architecting and is definitely not solvable in the scope of an SFSE answer. Consider consulting with an experienced Salesforce architect, and/or moving this process off-platform to a middleware solution.

If you're in a situation where you have fifty billion records (is that actually correct?) in Obj2, you need to be working with LDV (large data volume) and architecture experts.

| improve this answer | |
  • N.B. LDV = "Large Data Volume" (I had to look this up) – cropredy Nov 27 '19 at 18:30
  • Sorry about that! Added for clarity. – David Reed Nov 27 '19 at 18:34
  • @DavidReed Thank you for the elaborate explanation. We got an insight into the kind of limits we can hit. – Neha Ummareddy Nov 28 '19 at 8:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.