We get files placed on our FTP to use as data extensions for email campaigns. I have a custom automation with 4 steps (using a combination of query activities, ampscript and SSJS):
- Import File to Media_Master
- Add a record of this happening to Media_Log (using the name of the file as a job reference (Job_Ref) and a datestamp)
- Create a Data Extension by finding the latest Job_Ref on the Media_Log and using this as the name
- Updating a query's target data extension to this data extension and updating the definition to 'SELECT * FROM Media_Master WHERE Job_Ref = 'Job_Ref_Here' and then running it
We recently noticed that the import is not importing the full amount of records on the data extension. When we do a manual import with the same file, it works and we get the full amount. However, when we run it via this automation, we always lose records, and I'm pretty sure it's 'new subscribers' that it's not recognising. Any ideas why or how my automation could be ignoring new subscribers, or at least not adding them to the data extension?