Retrieve more than 2500 records from a DE using SSJS

Question

Trying to retrieve more than 2500 records from a DE. My thought is to have a for loop (to process the recordset) within a while loop (to retrieve records).

var data;
var recordCount = 0;

do {    
    data = rDE.Rows.Retrieve(complexfilter);
    recordCount = data.length;

    for( i = 0; i < recordCount; i++) {
        var rLeadId = data[i].LeadId;
        Write(i + " : " + rLeadId + "<br/>")
    }
} while (recordCount > 0) 

The error is:

throw "There was an error retrieving the rows."

Answer 1

This process does work. I was forgetting to update the data so the complexfilter would return a new dataset.