3
This is actually easier to do then you might think. Like @EazyE stated, you would extract the file, use a file transfer to push it to the FTP then use another File Trasfer to unzip it. Once you have the csv files that were in the zip file onto the FTP server you can then import them.
Step 1: Create Data Extract - You already seem to know this, so will skip ...
2
Script activities do have timeouts.
Large data volume, especially stuff like refreshing data extensions, is generally something that you want to process with SQL, not ssjs.
The problem is in the general approach, looking into WSProxy will probably just change details (a few hundred / thousand records more or less until it times out).
==
With that disclaimer ...
1
I compared your code with a working example from a recent implementation -
From that I see a couple small things off in your code:
contentType header is set differently.
You needlessly stringify the payload
resp.content needs to be stringified and parsed.
All these things are "formal" errors, which don't apply to POSTMan, so that would be ...
1
Here's the sargable method that I typically use to get yesterday's records:
select
o.subscriberkey
from _open o
where
o.eventdate >= convert(date, getDate()-1)
and o.eventdate < convert(date, getDate())
and o.isunique = 1
Converting to date effectively resets time portion to midnight and using a range captures all times between midnights.
Appears ...
1
as per your last comment / screenshot -
EazyE's original reply is correct, but your screenshot shows a misinterpretation of what was suggested. I'll try and elaborate.
ID_MKT_Cloud__c apparently is a field in your target DE.
The error tells you that in your target DE, it is in fact a mandatory field, so nothing can write data into the target without ...
Only top voted, non community-wiki answers of a minimum length are eligible