Retrieving more than 2500 Records using ExactTarget Java Fuel SDK

Question

We've got the the following code that is supposed to retrieve ALL unsubscribed entries from a list. With our data, this is hitting the 2500 limit.

I had a look at the ruby sdk, where they have a "continue" method if there are more results. I couldn't find the equivalent in the Java sdk though...

Can anybody point me in the right direction on how to do this?

ETListSubscriberService listSubscriberService = new ETListSubscriberServiceImpl();

... //create filter with list id and unsubscribed status

ETResponse<ETListSubscriber> response = listSubscriberService.get(client, filterByListIdAndUnsubscribedStatus);

response.getResults(); //does not return more than the 2500 limit

if(response.hasMoreResults()){

   //what to do from here???

}

Thanks for any pointers.

Answer 1

if you receive status='OK' it means less than 2500 records, you don't need to send more API calls.

If you receive status='MoreDataAvailable', it means there are more records, you need to get the requestID value returned, then make another API call with RetrieveRequest.ContinueRequest with that requestID value.Put it into a while loop, until you see status='OK' then exit the loop

Answer 2

one of the things that comes back from your request is the requestid.

set the property ContinueRequest on your retrieval to that, and do another retrieve.

Keep doing this until you stop getting the .hasMoreResults

(Not a java guy, so that is as close as i can get you)

Answer 3

try like this:

while (response.hasMoreResults()) { String continueRequest = response.getRequestId(); response = ETDataExtension.select(client, v_OBJ_FILTER, continueRequest, filter); lineNumberHdr = lineNumberHdr + response.getResults().size(); logger.info("Processati " + response.getResults().size() + " records di " + lineNumberHdr + " records");
check_result(response, dataList, con, v_OUTPUT_TABLE, v_STRSQL_INSERT, "EXPORT", BatchSize); }

Answer 4

The FuelSDK-Java 1.1.0 has got so many issues that my team has given up using that library. Apparently they have a retrieve method in ETSoapObject but has got no implementations in the entire library.

What you need to do is to create a new class and expose that retrieve method.

public class MyETDataExtension extends ETDataExtension {

    public static <T extends ETSoapObject> ETResponse<T> retrieve(ETClient client, String continueRequest, Class<T> type) throws ETSdkException {

        return retrieve(client, null, new ETFilter(), continueRequest, type);
    }
}

Now use your new class like this:-

public class ETDataExtensionTest {

    public static void main(String[] args){

        ETClient client;
        try {
            client = new ETClient("/et2.properties");     
            ETResponse<ETDataExtensionRow> response = MyETDataExtension.select(client,"key=9C75F619-E4DA-423F-826E-89A59F69D9B9");

            int i=1;
            boolean end = false;
            String continueRequest = response.getRequestId();

            while(!end){
                for (ETDataExtensionRow row : response.getObjects()){
                    System.out.println(i + "------ " + row.getColumn("Email address"));
                    i++;
                }

                if (!response.hasMoreResults()){
                    break;
                }

                response = MyETDataExtension.retrieve(client, continueRequest, ETDataExtensionRow.class);
            }


        } catch (ETSdkException e) {                
            e.printStackTrace();
        }
    }
}