Salesforce Stack Exchange is a question and answer site for Salesforce administrators, implementation experts, developers and anybody in-between. It only takes a minute to sign up.
Sign up to join this community
I have to insert 6.5 million records from Oracle into SFDC Contacts.
I know I can do it by Apex batch but not sure how can I connect Oracle DB from Apex batch or shall I consume REST service exposed by external system having / connected that Oracle DB.
Is there any other better option like Mule soft / ETL / heroku etc. Please guide me.
Thanks in advance.
Apex batch processing is not a good solution for a one-time operation inserting 6.5 million records from an external source.
You want the Bulk API, which is specifically designed for manipulating data via ETL in very large volumes. There are many Bulk API clients available, including the Salesforce Data Loader, Workbench, Amaxa (which I wrote), and various middleware platforms.
Most of the available tools expose very similar capabilities because they are all built on the same API mechanics. Note that if you choose to use the Salesforce Data Loader you do need to configure it to use the Bulk API rather than the SOAP API in its preferences.
If you have ongoing bulk data import needs, I would strongly recommend you evaluate middleware ETL platforms that can connect to the Bulk API.
It depends on whether this is a one-time load, or frequent sync process.
For one-time loads, the standard approach for up to 5 million records is to use the Data Loader. To speed up the processing, load a case with SF to request deferred sharing calculation. Also see if you can disable triggers and workflows.
Since you have more than the 5 million limit for data loader, and I'm assuming you need to translate record owners and field values, I'd recommend something like Mulesoft or Cast Iron. But those tools are built for ongoing ETLs.
This could be a non-trivial project, as you'll have to join the contacts to accounts. You'll also need to get past validation, de-duplication, etc. I'd recommend engaging a Salesforce architect with some experience in large data volumes. Good luck!
