0

I am using REST API call to get history data. For this I am writing SSJS in Cloud page and am trying to store the data in a data extension. The result of REST API call is more than 100 records. But by default it is giving me page 1 , page size = 100. How to set page size to maximum? I am using a for loop, if condition in the body to get specific activity details for all Id's

1 Answer 1

2

In general,but not always the case, you can use the $page and $pageSize parameters to control this.

For example:

/fake/myendPoint?$page=1&$pageSize=500 would return a payload of the first 500 items in the return.

Do be careful though as not every endpoint will do this. Also, sometimes in order for $pageSize to work, you need to indicate the page first (like my sample) or it will not affect the return.

EDIT: for anything being retrieved in bulk, I would look to either have multiple calls that are looped through and concatenated (will likely be a high resource draw and risk of timeout issues) or I would look to do a data extract of the full load, push this to an FTP and then utilize your service to grab this data and properly ingest it.

2
  • Thanks for the reply. I am able to retrieve the data with increased page size now. But there is a limit for 10000 records only. Is there any other way to get the complete data set at a time ? I have around 800000 records to get updated in a d.e.
    – Harry
    Commented Jun 2, 2020 at 16:47
  • @Harry I can almost 100% guarantee any call trying to pull that many records at once to SFMC will time out, even if you could. Your best bet to get those is to either do a data extract to a CSV and then import that CSV to your other system or to do a for loop around the retrieve calls and concat the results together to form one massive super array of data. Commented Jun 2, 2020 at 20:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .