Questions tagged [bulk-api]

Bulk API is based on REST principles and is optimized for loading or deleting large sets of data. You can use it to query, insert, update, upsert, or delete a large number of records asynchronously by submitting batches which are processed in the background by Salesforce.

Filter by
Sorted by
Tagged with
11
votes
4answers
1k views

Bulk API and Static Variable State

Our trigger logic has some recursion detection built in. Essentially, we have some static variables that act as counters for trigger executions. When our set "max" gets hit, our triggers quit to avoid ...
10
votes
2answers
15k views

How to perform upsert in Salesforce Bulk API

I have created a class with which I can insert, update and delete records from Salesforce using their bulk API. I have searched the web and have come across various resources; but I cannot seem to ...
10
votes
1answer
2k views

How to convert Person (consumer) Account to Business Account and keep the same URL

The backstory: The org I am working in allows Person (consumer) and Business records to be created by users and a host data push process in Account. As a result, today we have a large number of each ...
9
votes
1answer
7k views

How to handle NULL values when inserting with the Dataloader(.io)

I'm uploading records with the amazing dataloader.io and have a hard time to upload some records with NO value for optional fields. What do I have to put in those fields? #N/A 'null' NULL '' "" 0 Is ...
8
votes
4answers
14k views

Bulk api query batch size limit is 10,000 records

When running bulk api query, batch size limit is 10,000 records (from documentation: A batch can contain a maximum of 10,000 records.) What can I do if my query result is bigger than 10000, how can I ...
8
votes
1answer
897 views

Can't I use ALL ROWS in asynchronous Salesforce queries?

I would like to perform asynchronous queries that could retrieve deleted rows as well. However, if I submit a batch with SOQL like select Id, IsDeleted, Name from Account where IsDeleted = true and ...
8
votes
3answers
2k views

Bulk API Query - Splitting large query into batches

I'm currently using the bulk-api to return all records for a given sObject. I can make it work via one very large batch, but I think it would be more efficient if I could split it into multiple ...
7
votes
1answer
1k views

Query API Limits

Is there a way to programmatically query how many API calls an organization has made for the 24-hour period? We have a program that imports data from external databases to SFDC in a fancy way, and we ...
7
votes
2answers
2k views

How to retrieve compound fields like address using Bulk API?

Attempting to retrieve address fields (such as BillingAddress on Account) via Bulk API gives an error: Failed to process query: FUNCTIONALITY_NOT_ENABLED: Selecting compound data not supported in ...
7
votes
1answer
358 views

Recommended architecture for bulk-refreshing Salesforce?

We would like to keep Salesforce synced with data from our organization's back-end. The organizational data gets updated by nightly batch processes, so "real-time" syncing to Salesforce isn't in view. ...
7
votes
4answers
3k views

Bulk API & Foreign Key Resolution on Lookup Fields

We are designing an Integration leveraging Bulk API & Enterprise WSDL (The client code is handled by the source system). The target Force.com Object has few Lookup Fields. Wanted to understand how ...
7
votes
1answer
396 views

Providing multiple batches to the Bulk API 2.0

We're attempting to create multi-batch jobs in the Bulk API 2.0. Our previous single-batch workflow was to do: POST /services/data/v43.0/jobs/ingest PUT /services/data/v43.0/jobs/ingest/<job-id>...
7
votes
1answer
2k views

Unable to upload an Attachment - Bulk API

I'm using Bulk API to upload attachments and getting an error: InvalidBatch : Failed to parse CSV header. Number of Records Processed: 0 While I'm getting following error at the Salesforce ...
7
votes
1answer
2k views

Problem creating Job in Bulk API

I have been successfully able to use REST API for my application but I found that Bulk API will be better for my application. So, now I am trying to switch to Bulk API but I am stuck at the first step....
6
votes
1answer
850 views

Uploading 100,000 binary files as attachments to Case record - best approach

I want to upload files (.doc, .pdf) from external system to the attachments of Case record. There are two approaches I have found from Salesforce Help. 1) Uploading Attachments by data loader ...
6
votes
1answer
282 views

Is there a way to know if a bulk query result was truncated due to size limit?

The documentation says that bulk queries' results are limited in size. http://www.salesforce.com/us/developer/docs/api_asynch/Content/asynch_api_concepts_limits.htm However I can't find any ...
5
votes
1answer
220 views

A Few API Related Doubts

My first is related to Enterprise WSDL vs Partner WSDL. I know the basic difference between two: tightly coupled vs loosely coupled, one is tied to specific organization while other is used to build ...
5
votes
3answers
531 views

How can I *automatically* “restore” a sandbox environment with exactly the same data set?

I would like to be able to automatically "restore" a sandbox (actually developer org) with the same set of data. The org will be used for in-house training purposes, and the idea is that at the end of ...
5
votes
2answers
3k views

Will Trigger and Workflow get fired while using Bulk API

I am trying to upsert records in Account using Bulk API. After upserting records I need to update some of the custom field values to standard field values using trigger or workflow. Is it possible ??? ...
5
votes
1answer
609 views

How many batches can be created in a JOB via Bulk API?

I went through limitations about Bulk API but could not get explanation on the limit of Batches created per job in Salesforce. I can read Batch limitation but no limitation in Job. Your help is ...
5
votes
1answer
788 views

Bulk data API: How to Avoid Upserts

I'm working on an daily integration of about 5 million records (file delta's is not an option). Best practices for large data volumes recommends avoiding upserts. The only alternative solution I can ...
5
votes
2answers
3k views

How to query archived Tasks using Bulk Api

I'm trying to retrieve all Tasks in our Salesforce instance, including those that were archived more than 12 months ago. Since we have over 3 million records, I decided to try the bulk api. It ...
5
votes
1answer
1k views

Disable Data Loader by Profile

Is it possible to disable the data loader based on profile (or role)? We have some non-tech users who like to use the data loader for bulk updates, unfortunately, they are also making mistakes when ...
5
votes
3answers
6k views

Setting field value to Null when using Bulk API

I am trying to create a Data Load Tool similar to Data Loader which will upload the data from a CSV file to Salesforce. This tool is built using java and using "Bulk API". While this tool works fine, ...
5
votes
2answers
86 views

Transformation Spec limitation in Bulk API v1.0

We're using Transformation Spec feature of the bulk api when uploading records: https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/asynch_api_mapping.htm This allows us to ...
5
votes
1answer
149 views

Bulk API Row Lock Root

How to get confirmation that Inserts/Deletes of Contacts using the Bulk API will cause Row Lock errors on the parent Accounts, especially if the # of Contacts is large and there are few parent ...
5
votes
2answers
975 views

Rollback/Fail Entire Data Load on Error Instead of Partial Load

we have a process where we are loading events in via the Data Loader and have a before insert trigger on the event/activity to handle these records as they are being processed. We'd like that if one ...
5
votes
1answer
1k views

How to reliably detect whether BULK API is enabled for an org

We have a managed package which loads data into customer Salesforce orgs. We attempt to use the BULK API first and fall back to the REST API if necessary. So we need to be able to detect if the BULK ...
5
votes
1answer
1k views

SalesForce Batch Compression Unexpected Error

In SalesForce' Bulk API, on page 60 of their documentation: Compression The only valid compression value is gzip. Compression is optional, but strongly recommended. Note that compression doesn'...
4
votes
2answers
4k views

BulkAPI vs Dataloader?

I have never worked on handling large datasets and reading through the documentation raised the below questions in my mind: 1) If I need to load 10,000,000 (10 million) records into salesforce, Can I ...
4
votes
2answers
400 views

What is the maximum time limit whithin which we can add batches to the Bulk data load using Bulk API

I am wondering will there be any time limit for the batches to added to a Bulk API job once after the job has been initiated and before it can be closed? To be short, how long will a Bulk API job ...
4
votes
2answers
564 views

Specify large set of IDs in SOQL query

I have an object with around 2 million records and list of IDs that I need to extract. The list is around 20k entries. Surely I can't just chug them into query parameter? What is the best way to do ...
4
votes
1answer
89 views

Are salesforce reports persistent objects on the database?

Are salesforce.com reports actual objects that can be accessed on the database (e.g. using the BULK API)? Or are they really only metadata containing the queries required to return the data 'on the ...
4
votes
2answers
268 views

Updating record from Ruby on Rails to Salesforce

I'm using a Ruby gem that utilizes the Salesforce Bulk API and a few sidekiq workers to update records within Salesforce and am running into a UNABLE_TO_LOCK_ROW error. The issue stems from the ...
4
votes
1answer
862 views

AllOrNoneHeader for Bulk API

I am trying to switch off Autocommit feature by using AllOrNoneHeader in Bulk API implementation through JAVA. I have searched over internet but see only example of implementing it for SOAP based ...
4
votes
3answers
2k views

REST OAuth connection for Bulk API queries

Is it possible to authenticate an OAuth app for the user and then make Bulk API queries on their behalf? The results are easily going to be more than 20k records each authenticated user. When I have ...
4
votes
1answer
727 views

Bulk API - Records in different parallel batches not visible in trigger

We're loading thousands of records into a custom object (Address__c) over the Bulk API, in parallel mode. It has a Master-Detail relationship with Contact There's a checkbox field on the object (...
4
votes
1answer
1k views

SOQL query to load all documents in a content library

I am trying to query for all the documents in a library using the BULK API. The SOQL query I use is SELECT ContentDocument.Id,ContentDocument.CreatedDate,ContentDocument.LastModifiedDate,...
4
votes
0answers
62 views

SOQL query results differ on selecting from varying columns

I'm querying on Contact object where the results are varying on Bulk API vs querying on developer console. Total records on Contact is 997820. Query on developer console: select count(id) from ...
4
votes
0answers
158 views

Upserting objects with many to many relationship using api

I am writing a script to run on a schedule to automatically import data from our accounting software into salesforce using their API. I cannot find a good way to upsert objects with many to many ...
4
votes
0answers
519 views

Unsupported Content Type while using curl

We are currently working on with bulk api to load json data's into salesforce.But unfortunately we are not able to load it and getting error as unsupported content type does salesforce currently ...
4
votes
1answer
517 views

BULK API -PK Chunking Header Query - job content type : application/xml

Is it possible to use PK Chunking Header for a Query job of type XML. It works with CSV and when i pass it for xml jobs it fails and gives this error InternalServerError : PKChunking failed. ...
4
votes
2answers
398 views

Retrieve ActivityHistory via Bulk API?

I'm trying to download all of the ActivityHistories from our SFDC account. I would like to use the Bulk API to avoid hitting the API rate limit for the day. I'm using the following query to do so: ...
3
votes
1answer
2k views

NUMBER_OUTSIDE_VALID_RANGE: value outside of valid range on numeric field: 100.0

I get this error in the results of a Bulk API upsert: 'NUMBER_OUTSIDE_VALID_RANGE:Normalized Score: value outside of valid range on numeric field: 100.0:normalized_score__c --' The metadata for that ...
3
votes
1answer
266 views

Bulk API Error: UNABLE_TO_LOCK_ROW:unable to obtain exclusive access to this record or 124 records

During the process of loading 90,000 opportunities into Salesforce using the Bulk API, I get this error: UNABLE_TO_LOCK_ROW:unable to obtain exclusive access to this record or 124 records... After ...
3
votes
4answers
442 views

Salesforce large query conditions: query 20000 contacts by emails and update fields

I have a csv file which has 20000 email addresses in it. I need to update some fields on these contacts. the dataloader is not suitable as I don't have the ID field of these email addresses. query in ...
3
votes
2answers
1k views

How to retrieve multiple contacts with single REST call

I am getting 20 contact ID from getUpdated.Now I need information for each of these 20 contacts.It seems costly to request each contact through a REST url.Is there any way how I can query multiple ...
3
votes
1answer
959 views

How to perform upsert in SalesForce Bulk API with Multiple Update Key?

I created some classes in Java, which integrate my existing system with SalesForce via Bulk API. The component runs smoothly when inserting new objects. I was wondering how it would be possible to ...
3
votes
1answer
1k views

Inserting 'Log a Call' task using REST API

I am currently trying to insert a new 'Log a Call' task for accounts / contacts using the REST API. I have the information I require from the calls in an SQL database table including the accountId/...
3
votes
1answer
471 views

Does Salesforce Bulk API 2.0 accept JSON as the contentType?

I see that Bulk API 1.0 supports JSON payload as the content type for insert operations. However, I cannot ascertain if Bulk API 2.0 also supports JSON. In my Job Create body I've tried sending: { "...