what would be the best way to insert bulk data to a custom object

Question

I need to make 2 calls per day to API Rest and process the data. I have used “apex scheduler” to automate the 2 calls a day, then call to my apex class where I have all the logic to call the webservice and the object field mapping I store in a list, then I validate if there are records, if there is, I use the “apex queueable” in which it calls a batch apex to do the upsert with an external id. Sometimes I get these 2 errors:

1. Apex CPU Time Limit Exceeded
2. Apex heap size too large

The webservice returns around 100K records per day, that is, 50k records per call, and the custom object has approximately 30 fields.

The code will be somethig like that:

//execute Apex scheduler

//call future method.

 @future(callout=true)
    public static void testCallout(){
        //logic callout and validation
        List<customObject__c>cobj = new List<customObject__c>();
        for(Object res : response ){
            //more logic..,.
            obj.add(
            new customObject__c(
            field1__c = mappingfield...,
            field2__c = mappingfield...
            )
            );
        }
    }
    if(obj.size()>0){
        //call queueable apex and after that, call batch apex inside queueable apex to upsert records
    }

Answer 1

The best way to insert bulk data into a custom object is to use Batch Apex.

To use batch Apex, write an Apex class that implements the Salesforce-provided interface Database. Batchable and then invoke the class programmatically.

To monitor or stop the execution of the batch Apex job, from Setup, enter Apex Jobs in the Quick Find box, then select Apex Jobs.

The following blog would be really helpful to implement Batch Apex.

https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_batch_interface.htm

Thanks