how can aggregate the response returned by multiple calls in the list

Question

I am making callout and getting the response. I am fetching the records in the batch of 200 so if there is 400 records there would be 2 callouts and getting data but the issue is second callout response overriding the first one.

And I can see only last 100 records. While in the debug logs I can see 100 and 100 records fetched in both the calls.

Here my class:

integer value = 0;
    do{
       HttpRequest req = new HttpRequest();
       HttpResponse res = new HttpResponse();
       req.setEndpoint( 'https://pi.pardot.com/api/prospect/version/3/do/query?user_key='+userKey+'&'+'api_key='+apiKey+'&'+'output=bulk&format=json&updated_after='+DayValue+'&'+'sort_by=updated_at&sort_order=ascending'+'&'+'offset='+value);                                  
       req.setMethod( 'GET' );
       req.setBody( 'user_key='+userKey+'&'+'api_key='+apiKey);
       req.setHeader('Accept', 'application/json ');
       req.setHeader('Content-Type','application/json');
       res = new Http().send( req );
       System.debug('***Response****'+res.getBody());
       wrapper = (PardotProspectWrapper)JSON.deserialize(res.getBody(), PardotProspectWrapper.class);

        //this I tried to add to store the data in Pardot_Prospect__c 
        ppobjlst = new List<Pardot_Prospect__c>();

       prospectIdlist = new List<String>();

       Pardot_Prospect__c ppobj = new Pardot_Prospect__c();
        for (PardotProspectWrapper.prospect p : wrapper.result.prospect) {
            ppobj = new Pardot_Prospect__c();
            ppobj.Prospect_Id__c = string.valueOf(p.id);
            ppobj.First_name__c = p.first_name;
            ppobj.Campaign_Name__c = p.campaign.name;
            prospectIdlist.add(string.valueOf(p.id));
            ppobjlst.add(ppobj);  
          } 
            value = value + 200;
       }while(value<=400);

        insertRecords(ppobjlst);      
    }

public static void insertRecords(List<Pardot_Prospect__c> ppobjlst){
    system.debug('**INSIDE the insertRecord method*****'+ppobjlst);
    upsert ppobjlst Prospect_Id__c;
}

Answer 1

Just move the list instantiation outside your loop structure:

List<MyObject__c> records = new List<MyObject__c>
do
{
    ...
}
while (...)

When you instantiate the list within your loop, you empty it out each time you iterate, clearing your results from the previous iteration.