I'm querying a large (1+ million) number of records from the SFDX CLI:
sfdx force:data:soql:query \
--targetusername '[email protected]' \
--soqlqueryfile "my_query.soql" \
--resultformat csv \
--bulk
This gives me a job status ID, whose progress I can monitor both in the Salesforce UI and with sfdx force:data:bulk:status
. The job takes about 1 minute to complete, and the UI indicates that it was executed as 6 batches in parallel:
How do I get the results from all batches? The documentation says that sfdx force:data:soql:bulk:report -i $jobid
should be used, but that only gives me a partial result: Apparently this downloads records from only one of the batches, as determined by running wc -l file.csv
on the output and comparing the number of CSV lines to the number of records in the batches. Running the same command again returns the same batch results.
The CLI documentation does not mention how to get more than one sub-batch from sfdx force:data:soql:bulk:report
- only an argument for the job ID is available, and that only gets part of the result. There is no obvious way to download the batch results from the UI either.
How do I get the results from the other batches?