12
From our comment thread on the OP:
You need to configure an external Id field on your object; this will generally by the primary key used by your external system with which you are syncing. See http://na1.salesforce.com/help/doc/en/faq_import_general_what_is_an_external.htm and http://na1.salesforce.com/help/doc/en/custom_field_attributes.htm
Change your ...
12
Inserting null values is an option that you need to manually set for the api call. Once you set that option you just leave the cells empty and it will null out any fields where the cell is empty.
In dataloader.io you set this option here:
That option is greyed out for me and from the dataloader.io discussion boards it looks like they haven't implemented ...
11
The batch size thats specified in the document is for a single batch.You can process as many batches as you like limiting to 2000 per 24 hour rolling period.
10k is the maximum batch size that you can set per batch.So say you have 50K records then 5 batches will be the minimum no of batches required .
Bulk API can process any no of records .Only limit you ...
10
Precision is the total number of digits allowed both left and right of the decimal, and scale is the number of digits to the right of the decimal. The number of digits just to the left of the decimal is precision-scale, or in other words, 2 significant digits. As configured, the field's maximum value is 99.99999999. You'd want to edit the field to allow 3 ...
9
Starting in a relativley recent release (I believe APIv29) the SOAP API (and possibly others) returns a header describing your org's API limits.
This LimitInfoHeader should be returned on all SOAP calls made from the APIv29 WSDL, and shows you the current use and 24hr rolling limit.
9
1) If I need to load 10,000,000 (10 million) records into salesforce, Can I use Dataloader and select the 'Use Bulk API' option on settings screen?
The data loader supports any size file that your system can handle and Java can read in memory. In theory, it could support hundreds of millions of records. The bulk API has smaller limits, but the data loader ...
9
I would suggest you to go with data loader using SOAP API, which is by default (not Bulk API).
Why?
You do not need to write any code to use Data loader
It is tested solution, so no bugs
With SOAP API you will need around 500+ API (100000 records / 200 batch size = 500 API required) which is not too much (we have 100k API limit per 24 HRS for ...
9
A great resource for understanding row locking situations is the Record Locking Cheat Sheet (PDF link). Note at the top of page 2 that insert, update, and delete of Opportunities locks the parent Account.
Since your batch job is running in parallel across its batches, the batches can independently attempt to gain locks on the same Account, resulting in ...
8
Has anyone else experienced this?
I have and to say it was quite annoying was an understatement. I had to call an update trigger either after it was inserted or after it's second update. I got it to work fine, but as soon as the records were inserted through DataLoader, it bombed because of this issue.
I asked a similar question and the response reflected ...
7
The bulk API is ideally for much larger than 1500 records.
Like you state, in cases where there are 100k of records a typical use of the SOAP API would quickly begin to eat away at the daily API limit for an org. And this is really what you should be aiming for.
When I used to teach this in DEV502, we would state (somewhat arbitrarily) for over 50k ...
6
You can use External ID fields to track the primary key (PK) of the external system in salesforce.com. Then, each night, you simply scan your backend for any changes, and use the upsert function to update the related records.
Assuming this is intended as a one-way synchronization with the backend as the final authority on what data is "correct", this is a ...
6
There is a clue in this help page on compound fields:
Compound fields can't be included in Bulk queries. Use the individual field components to access or update field values.
It turns out that you can use the component fields like so:
SELECT BillingStreet, BillingCity,
BillingState, BillingCountry, BillingPostalCode,
BillingLatitude, ...
5
As far as I know it's not possible to meet all your requirements. Specifically the one about keeping the same Id / URL. I'd love to be proved wrong :)
You'll have to insert a new Account with right record type and link/reparent all related data, then eventually delete the old PA. Maybe you could defer the delete and create a new field "click here to go to ...
5
Upserts require the use of an external id field on the object. You could probably concatenate your composite key, assign that value to the external id in the Salesforce record, and then use that for upserting.
5
Since this hasn't been responded to after having been posted for several weeks, it would appear that no one has a timely suggestion for you.
My suggestion would be pre-process your CSV files using a script that scans them for field entries containing blank values and inserts a "#N/A" if it finds one. You could do that separately or as part of some kind of "...
5
Proceeding with apex, describe and SOQL will make it slower and in my experience it will rather add new bottlenecks and problems than remove existing.
On thing you try is the weely export feautre to export everything. But depending on the size of all your data, this can also be very slow. Also it splits up the export in many zip files, which are hard to ...
5
My understanding is that Report object in salesforce store information about the report like name , developername, Description. however reports object and columns are stored in metadata . We can't use the BuLK API to get the data from reports. However, you can query the data using the bulk API.
5
It sounds like there was a discussion with someone in Salesforce that the Bulk API wasn't the right tool for monitoring deleted records.
There is always the Data Export functionality. It's kind of async, but unfortunately doesn't have an API.
Other than the REST API there is also the SOAP API, which is synchronous. See queryAll()
5
Call the Limits API with a GET request:
GET /services/data/v45.0/limits
Your response map will include
"DailyBulkApiRequests" : {
"Max" : 10000,
"Remaining" : 10000,
}
as well as a breakdown by application.
4
I've seen this issue before, if you copy the CSV file's text into notepad, and then save it using the ASCII format, it will ensure that you're using character code 39 and not 96:
Check out this link to view the different character codes:
http://www.asciitable.com/
4
Yes, you can use the REST API to insert tasks for account. The REST api only supports actions on individual records, if you want to use one call for multiple records you'll need to use the BULK API. Whether being inefficient and doing one call per record matters has to do with how many API calls your organizations has available. Large organizations with ...
4
Problem is that the email field is NOT necessarily unique. So even the VLOOKUP may end up in matching many ids for one email.
If you would have the email in an external unique id field, you could utilize the smart-lookup feature of workbench https://workbench.developerforce.com/login.php - from all those dataloader-like tools I allways found workbench as ...
4
Daily limit, not per job:
You can submit up to 5,000 batches per rolling 24 hour period
Source: http://sforce.co/1avfW6t
A job can be active for 24 hours, so the amount of batches created for a job is at most 5000.
4
Upserts are slower than the relevant operations, but the actual time used depends on your strategy. Inserts are faster than upserts on all new records, updates are faster than upserts on all existing records, and mixed upserts are slower than inserts and updates combined.
The difference can indeed be about 100% in time (double). The difference in time for ...
4
The raw SOQL limit is 20,000 characters, so you can't make a raw dynamic statement that exceeds this value. You'll need to chunk your query into multiple parts. In theory, you'll get around 1,000 rows, maximum, depending on how many fields you use (and assuming you 15-character length ID values). If possible, consider using the retrieve call, which only ...
4
I suggest that you create a specific User for this bulk load process that is not used for any other purpose. Then your trigger can base its conditional logic on that:
if (UserInfo.getUserName() == 'bulkloadprocess@abc.com')) ...
4
All Records in Salesforce must have an Owner which is either a User or a Queue, if you try and set OwnerId to Null or blank "", this will always fail as the field is mandatory therefore Null is not an acceptable value.
As the Salesforce security model for data is built around record ownership this is correct behaviour as it will mean that any relevant ...
4
What is the maximum time limit whithin which we can add batches to the Bulk data load using Bulk API
The Bulk API Limits states:
You can submit up to 5,000 batches per rolling 24 hour period. You can't create new batches associated with a job that is more than 24 hours old.
Additionally, the job itself has a time limit:
Batches and jobs that are older than seven days are removed from the queue regardless of job status. The seven days are measured ...
4
The sObject tree is only for the REST API, not the Bulk API. While they both use a REST-ful nature, only the REST API supports the sObject tree. This generally means you have to upload records in batches of 200, with a tree no more than 5 levels deep.
You can still use JSON structures with the bulk API, but it follows a slightly different format, outlined ...
4
Your suggestions are all valid in there own right, but they do become dependent on your data.
I would lean towards #3 the most, as that is what I most often do (because it works with most of my data sets). The only time #3 would be bad is if you have 10's of thousands of records with the same lookup. This also doesn't pair well with number 1.
Consider 6000 ...
Only top voted, non community-wiki answers of a minimum length are eligible
