2

I'm using SSJS WSProxy object to create a paginated API request. I'm trying to limit my batches to 50 results per iteration to use that data in future call, which is SFMC remove contacts from a journey. It requires to limit the request to 50 contacts as described here. I followed the WSProxy documentation on advanced retrieves and successfully made it work, but the issue I'm facing is that first retrieve returns requested batch size, while next one returns all data. Options I tried are:

var allContactsToBeDeletedKey = 'my-de-key';
var allContactsToBeDeletedFields = ['SubscriberKey', 'Field1', 'Field2'];
var allContactsToBeDeletedFilter = {
    Property : 'Field2',
    SimpleOperator : 'equals',
    Value: false
};
var standardBatchSize = {BatchSize:50};
var deleteProxy = new Script.Util.WSProxy();
var hasMoreData = true;
while(hasMoreData) {
    hasMoreData = false;
    /*Option 1 - no batch size parameter in getNextBatch()*/
    var deleteData = deleteReqId == null ?
        deleteProxy.retrieve('DataExtensionObject['+allContactsToBeDeletedKey+']', allContactsToBeDeletedFields, allContactsToBeDeletedFilter, standardBatchSize, null) :
        deleteProxy.getNextBatch('DataExtensionObject['+allContactsToBeDeletedKey+']', deleteReqId);
    /*Option 2 - additional batch size parameter in getNextBatch()*/
    var deleteData = deleteReqId == null ?
        deleteProxy.retrieve('DataExtensionObject['+allContactsToBeDeletedKey+']', allContactsToBeDeletedFields, allContactsToBeDeletedFilter, standardBatchSize, null) :
        deleteProxy.getNextBatch('DataExtensionObject['+allContactsToBeDeletedKey+']', deleteReqId, standardBatchSize);
}
  • Option 1 - first request returns 50 rows, next returns all remaning data without batch size
  • Option 2 - first request returns 50 rows, next errors out
6
+50

Currently it is not possible to set the batch size on the getNextBatch() method of WSProxy, and for now, it will always return the max # of records up to 2500. I did talk to the product owner, and it's on his radar to add it to an upcoming release.

That said, you don't actually need getNextBatch() to continue your retrieve request. It's just a helper function. You can make another retrieve() call, but just set ContinueRequest to the RequestID of your previous call. Below is an example I wrote retrieving Content Areas (I know, they're Classic Email Objects :P) from a folder in batches of 2.

<script runat="server">
function Write(str) {
   Platform.Response.Write(str);
}

/*------------------------------------------*/

function Stringify(obj) {
   return Platform.Function.Stringify(obj);
}

var clientID = Platform.Function.AuthenticatedMemberID();
var api = new Script.Util.WSProxy(); // Create WSProxy object

api.setClientId({ "ID": clientID}); // Set ClientID if you need to

var retrieveObject = 'ContentArea'; // Set your object

var cols = ['Name','Content','CustomerKey','Layout','ModifiedDate','ObjectID','CategoryID','ID','RowObjectID','Client.ID','CreatedDate','IsDynamicContent','IsSurvey','IsBlank','Key']; // Choose the properties you'd like the call to return

var filter = { 
    Property: "CategoryID", 
    SimpleOperator: "equals", 
    Value: '443311'
};

var opts   = {
    BatchSize: 2
};
var props  = {
    QueryAllAccounts: false
};

var data;
var moreData = true;
var counter = 1;
var reqID = null;

do {
    moreData = false; //in case we don't make it in to the data conditional
    Write('<b>Iteration '+counter+'</b><br />');
    if (reqID) {props['ContinueRequest'] = reqID;} //set ContinueRequest ID if we have requestID from a previous run
    data = api.retrieve(retrieveObject, cols, filter, opts, props);

    if(data != null) {
        moreData = data.HasMoreRows;
        reqID = data.RequestID;
        if(data && data.Results) {
            for (var i = 0; i < data['Results'].length; i++){
                Write(data['Results'][i]['Name']+' : MID '+data['Results'][i]['Client']['ID']+' : '+data['Results'][i]['ModifiedDate']+"<br />\n");
            }//for
        }//if
    }//if
    counter++;
} while(moreData);


//Write('<br /><b>RAW RESULTS</b><br /><xmp>'+Stringify(data).replace(/([\[\{\]\},:])/g, function(str, selector){ return(selector+"\n");})+"</xmp>\n");
Write('<br /><b>RAW RESULTS</b><br /><xmp>'+Stringify(data).replace(/,"/g, ",\n\"")+"\n");})+"</xmp>\n");
</script>

The important piece is this line:

if (reqID) {props['ContinueRequest'] = reqID;} 

I'm setting ContinueRequest for the next time I'm going to make a retrieve() call.

So, in your call, instead of passing null as the last parameter of your retrieve, pass an object. On the first iteration, it should be an empty object (or you could set the QueryAllAccounts option to false for fun). Then as you loop through, set the ContinueRequest key to your latest requestID value, and do the retrieve again. Stop using getNextBatch() altogether. Hopefully that makes sense!

  • Thank you for the well thought out response and including an alternative option. – Gortonington Aug 3 '18 at 13:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.