0
  1. We've requirement to 'PERIODICALLY' fetch description of all object and their custom fields and store in NEW custom object(FieldsAndObjectDescription)
  2. We've huge Org and have more than 20K custom fields and around 1000 Custom objects.
  3. Any time any field/object description is changed or new field added in future, details should be updated in New object(FieldsAndObjectDescription) (synchronous or asynchronous, whatever is easy)
  4. I'm thinking to use Tooling API for this(Tried and it is working fine).

Problem : How to write a job which can be scheduled periodically to do this.

Since, 20K records needs to be created(and potentially updated) ,I was thinking to write a batch job but Stuck how to start?

  1. Could I run Rest(Tooling) API in Start method? I'm not sure, if that is even possible . But If that is , what should be passed to 'Database.queryLocator?
  2. If not in 'Start' method, Do I need to do call tooling API from 'Execute' method, then what should be in 'Start' Method. This, if possible, seems to be somehow preferable as I know I shouldn't cause Tooling API to return 20K record in single request(which might crash, although not tested). I prefer to make multiple requests.
  3. If Batch job itself is right way to work with tooling API, what is the best way to handle Tooling API response, which might be in thousands ?
3

Could I run Rest(Tooling) API in Start method? I'm not sure, if that is even possible . But If that is , what should be passed to 'Database.queryLocator?

Yes, you're allowed a callout. But you wouldn't return a query locator, but instead a list of values. It doesn't matter what you return as long as they're non-null values. You'd get the totalSize from the initial response, and divide it by the size of each page. Assuming you use a scope size of 1 (recommended), that means you'd do this:

public class BatchClassName implements Database.Batchable<Integer>, Database.Stateful, Database.AllowsCallouts {
  ToolingApiResult result;
  public Integer[] start(Database.BatchableContext context) {
   result = ToolingApi.query('select id from customfield');
    Integer[] scopes = new Integer[0];
    Integer counter = 0, totalScopes = Math.floor(result.totalSize / (decimal)result.records.size()).intValue();
    while(counter < totalScopes) {
      scopes.add(counter++);
    }
    saveToDatabase(result.records);
    return scopes;
  }
}

If not in 'Start' method, Do I need to do call tooling API from 'Execute' method, then what should be in 'Start' Method. This, if possible, seems to be somehow preferable as I know I shouldn't cause Tooling API to return 20K record in single request(which might crash, although not tested). I prefer to make multiple requests.

In the execute method, call the nextRecordsUrl, then save all of the records you just collected to the database.

If Batch job itself is right way to work with tooling API, what is the best way to handle Tooling API response, which might be in thousands ?

It's one possible method. I would personally use Queueable instead, which skips the complexity having to figure out where to start/stop.

Also, I should mention that you're probably going to need a Named Credential to get this to work. In asynchronous code, UserInfo.getSessionId() is null, so you need a way to get a valid access token.


Note: This is intentionally abstracted; it is up to you to determine how to expose the ToolingApi to your code, perhaps as a utility class or some other method. Similarly, the saveToDatabase method is just an abstract method you'll write to process the results and commit the records to the database.

  • Thanks sfdcfox, as always, your suggestions/ideas are up-to-the-mark. thanks again ! – sfdc_Qrius Jul 8 '18 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.