I am using REST API call to get history data. For this I am writing SSJS in Cloud page and am trying to store the data in a data extension. The result of REST API call is more than 100 records. But by default it is giving me page 1 , page size = 100. How to set page size to maximum? I am using a for loop, if condition in the body to get specific activity details for all Id's


In general,but not always the case, you can use the $page and $pageSize parameters to control this.

For example:

/fake/myendPoint?$page=1&$pageSize=500 would return a payload of the first 500 items in the return.

Do be careful though as not every endpoint will do this. Also, sometimes in order for $pageSize to work, you need to indicate the page first (like my sample) or it will not affect the return.

EDIT: for anything being retrieved in bulk, I would look to either have multiple calls that are looped through and concatenated (will likely be a high resource draw and risk of timeout issues) or I would look to do a data extract of the full load, push this to an FTP and then utilize your service to grab this data and properly ingest it.

| improve this answer | |
  • Thanks for the reply. I am able to retrieve the data with increased page size now. But there is a limit for 10000 records only. Is there any other way to get the complete data set at a time ? I have around 800000 records to get updated in a d.e. – Harry Jun 2 at 16:47
  • @Harry I can almost 100% guarantee any call trying to pull that many records at once to SFMC will time out, even if you could. Your best bet to get those is to either do a data extract to a CSV and then import that CSV to your other system or to do a for loop around the retrieve calls and concat the results together to form one massive super array of data. – Gortonington Jun 2 at 20:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.