The product I'm developing is going to act like acorporate backup solution at first but there are some data processing features that will be developed later that are the real focus.
Let's say I wanted to get a full copy of every single record in, say, an Account table (followed by every other table in the instance later on). I've dealt with clients that easily have over 1.5 million records in their Account history. What is the preferred way to do this without causing governor or other limit problems with as few api calls as necessary? Speed won't be a blg deal at first, I just want to do right for the customer's instance.
This could be done either way: externally from API requests from a python app on an external server or from a script written in Apex, but the end result is the same-- every single record in my SQL database ready to be queried in bulk with a happy customer whose instance hasn't been inconvenienced.
I believe I might have a few options. Batch apex might be one of them, but then there's also the ability to create batch export jobs from the API that I believe will export a full csv that you just have to build a parcer for. There might be other directions I'm not aware of.
Since this is going to be an appexchange listing for many clients at once, NO SALESFORCE CONNECT. It needs to be done through code, not through expensive packages.
Thank you very much for your thoughts!