Salesforce Stack Exchange is a question and answer site for Salesforce administrators, implementation experts, developers and anybody in-between. It only takes a minute to sign up.
Sign up to join this community
I'm looking for the API call (or set of) that will allow me to upload a roughly 20mb file to somewhere in Salesforce that would later let me process this file via APEX to update various objects based on data in the file.
I'm currently using the Force.com PHP Toolkit, and am looking at the CRM content option. Though I'm unsure how to proceed or if this is the place I want to be uploading too.
Realistically, you're not going to be able to do this, for a variety of reasons:
The only context that can even store 20 MB of data in RAM is Email Services. That means you'd have to kick off an email to Salesforce just to get your code to have enough memory to load your file in memory, much less have a chance of doing anything. Batch Apex, future methods, and Queueable all only have a 12 MB limit.
Depending on the complexity of your logic, it's highly unlikely you'll get enough execution time in Apex Code to do what you're trying to do, unless you use Batch Apex, which only has 12 MB of heap (see above). Apex Code is a fairly powerful language, but it's not exactly fast. In some optimal cases, you can get about 18 million instructions, but realistically it'll probably be closer to just 2-4 million instructions. It's hard to process that much data with the limits on CPU time.
Email Services, the only context that has more than 12 MB of heap, has limits as well. Each attachment can only be 5 MB, and there's a 10 MB limit on the entire email. That means you'd have to upload the file, and then send an email, and hope that the code finishes processing within the 60 second CPU limit. This is just going to complicate your code even further.
Many of the cooler string processing techniques are tied to regular expressions, which in turn has a 1 MB limit. For example, you probably can't use String.split, String.replaceAll, and so on. That'll make it even harder to keep inside the CPU limits.
Processing raw files in Apex Code simply isn't practical. You'll want to run your heavy processing in your PHP script, Heroku, Amazon cloud services, or pretty much anywhere else. The limits we have in Apex Code are perfect for our most usual business cases, but raw file processing is not one of them.
