We use the salesforce dataloader to do occasional exports of our entire database so we can use our local database tools to run analysis and data cleaning that we can't manage via reporting and selective extracts (we also don't have any in-house apex devs with any more bandwidth).
It's painfully slow at the moment, does anyone have any experience of using the Bulk data API to try and achieve the same thing (or perhaps that's what most dataloaders use anyway?).
I'd like to try and get an idea of if that's likely to be any faster, if there are any other implications to using the bulk API (speed, running out of requests etc?).
In terms of what we're downloading it's around 29 objects, totaling to many hundreds of thousands of records, our most recent extract runs to about 1.7 gig of .csv files...
I guess the way I'd go about doing it is by building queries using Sobject.describe global and then describe on each object to build a list of all headers then do some kind of SQOL request for where not null?
On reviewing with colleagues it is not the data loader that is causing the hold up in our current workflow - it is the Weekly Export Service, which we now have to schedule for mid Sat in order to get it for Monday, we believe it is because of queues on our instance (NA8). I am exploring the Bulk API to see if it would be a superior alternative.