-1

I could not find anywhere the number of files that can be queued in a fileDrop automation. I searched here: https://help.salesforce.com/s/articleView?id=sf.mc_as_define_a_triggered_automation.htm&type=5 and here: https://help.salesforce.com/s/articleView?id=sf.mc_overview_limits.htm&type=5

Can anyone help, please?

2
  • 1
    This sounds to me as an XY problem. Can you explain why you expect to be in a situation where more than 2-3 files would be queued up? Commented Apr 22 at 16:14
  • Thank you for the promt response Lukas, the ones sending the files to the FTP are an external company, they say that in their system the maximum rows they can process are 10K per file, and there can be an audience of about 20M which means 2000 files in total. I wanted to tell them that's not viable but could not find any documentation about it
    – SebOB
    Commented Apr 23 at 6:52

1 Answer 1

1

Firstly, I will really recommend to see if you by any chance have the external company to increase the file size in their end. The overall recommendation is to keep the files below 2GB, and the fewer files you need to process the better. Keep in mind, that a file import automation is often performing other activities than just importing the file. You might have other steps, e.g. SQL Queries, which process the data which is being imported.

Also, please observe, that automation runs are a cost driver in your license. If you need to process 2000 files each time, while you could have done the same with 3-4 files, you will quickly hit your limit. As per this doc, you have following number of annual automation runs included in your license, depending on the edition:

  • Pro: 15K
  • Corporate: 45K
  • Enterprise: 100K

As you can see, running the same automation 2000 times will quickly make you reach your limits.

That said, there aren't any limits around how many files could be queued, but only one automation can run per file. You can have multiple automations running on a single folder, but they cannot be triggered off the same file.

File Queueing is set up to keep the file ready for triggering the automation if it is dropped before the first run is completed. You can also turn this off if you do not want the triggers to queue. If you queue your runs, even if the previous run fails, it will still go through the queue until it is finished - so here, you don't have the same dependency as between individual automation steps.

Queue will run through the files in the order they were dropped, but it will use the most recent files. So if you overwrite a file in a file drop for a later queued trigger (here it is important to keep unique names for all the dropped files), it will not use the original data, but the new data.

If you turn off queueing, any triggers that happen during a currently running instance will cause that instance to stop and restart based on the new file trigger.

File drop automations can be triggered upon the placement of a file onto the FTP -- but it's typically assumed to be a single file, rather than a potentially large number of them. Trying to trigger File Drop Automations using a large number of files could make the system heavy, due to the way it queues files. If you are dropping a significant volume of files at a high frequency, there is a high chance that the automation may not work as expected on these files and potentially fail or process them at a very slow rate.

So, just to confirm - no hard limit on queue size, but I will really push for merging the files into less than 10 for the reasons listed above.

1
  • On top of it I found this: help.salesforce.com/s/… "FTP file upload limit is now 2,000 files per minute, per session"
    – SebOB
    Commented Apr 23 at 15:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .