Questions tagged [bulk-api]
Bulk API is based on REST principles and is optimized for loading or deleting large sets of data. You can use it to query, insert, update, upsert, or delete a large number of records asynchronously by submitting batches which are processed in the background by Salesforce.
408
questions
0
votes
1
answer
967
views
Bulk API upsert with both External Id and SFID in JSON
I'm using the Bulk API to upsert records into Salesforce. The Request contains both an External ID and also a Salesforce Id.
Even though in the job detail references using the external Id, will ...
- 411
1
vote
0
answers
64
views
Bulk Query with PK Chunking via WSC -- error reading extracted records with CSVReader
I'm having an issue with the CSVReader class in the Salesforce WSC.
The error I get when trying to read the contents of a batch is "com.sforce.async.CSVReader$CSVParseException: Found unescaped quote. ...
- 31
4
votes
1
answer
2k
views
Is it impossible to export more than 2000 records from an aggregate SOQL query?
It seems odd to me that this would be simply impossible to do, but the aggregate query below returns more than 2000 records, and it appears that there is no possible way to even run the query, let ...
- 948
0
votes
1
answer
1k
views
Create Account and Contact at the same time with Bulk API
I'd like to create an account and then a contact in the same call using bulk api. I've used composite api calls to create an account and contact in the same API request.
I would now like to do the ...
- 3
1
vote
2
answers
2k
views
Is it possible to change the size of the Auto-Chunked Batches in bulk API
We are implementing a Bulk API V2 client in order to insert hundreds of thousands of records.
Our current implementation is the following :
first we create a job
we add a batch containing all the ...
- 21
1
vote
1
answer
4k
views
Workbench: Bulk API to delete 1M records
I am looking for an official resource or step-by-step instructions to use Bulk API to delete more than a Million of records from a custom object. I couldn't find a proper documentation online and ...
- 721
4
votes
3
answers
2k
views
Execution order of multiple Duplicates rules on an Object
I have two active duplicate rules on Account object.
Is there a particular order in which they are executed? or is it random?
If a record is marked as "duplicated" by a Duplicate Rule, this record ...
- 149
0
votes
1
answer
1k
views
Bulk API fails to insert w/ null RecordTypeId, but Rest API succeeds
I've come across an error where the Bulk API and the Rest API differ in their behavior when inserting records with record types. To sum up the problem:
Suppose one or more custom record types are ...
- 121
0
votes
0
answers
432
views
Error 'Java Heap Space' in Data Loader
With Data Loader (Version 44.0.0),I am trying to insert Attachments in Salesforce. There are 12k PDF files. Each file is around 190 kb. So total attachment are 2.28 GB
This is the structure of my ...
- 845
0
votes
1
answer
278
views
Importing 40k PDF as attchments
I have 40k PDF's in windows shared drive. Each pdf should be attached to parent account. The name of each pdf is an external ID in salesforce.
e.g AA123.pdf corresponds to AccountRecordId '...
- 845
2
votes
1
answer
485
views
Bulk API vs Bulk API 2.0 Query Length
We've been using the BulkAPI (XML/SOAP) successfully for several months now and were looking to migrate over to the BulkAPI v2.0.
However, when submitting a job (queryAll) to the 2.0 API we're ...
- 121
0
votes
1
answer
329
views
Bulk API library for Java
I searched on the internet for a bulk API library for Java, but I could not find anything relevant. Do you know any bulk API library for Java?
- 1,554
0
votes
1
answer
395
views
how we know which version to pass in the bulk API
I am integrating the SF via connected APP and I'm trying to do bulk query and i followed the document and got to know how to achieve.
My question is, how can i know which version should i pass in the ...
- 101
0
votes
1
answer
6k
views
Query "Bulk Data Load Jobs" IDs
Is there a way to use WorkBench/Apex/Rest-Explorer to query all "Bulk Data Load Jobs" IDs for last X days?
Use Case - We want to set up some logs and monitoring on different users consuming our org ...
1
vote
1
answer
415
views
WhoId, OwnerId and WhatId CSV Headers?
Polymorphic & Relationship Fields
I'm trying to insert a task via Bulk API and providing the below csv headers, but getting Malformed_ID error.
Owner:User.FederationIdentifier
Contact:Who.Id
...
- 175
0
votes
1
answer
2k
views
how to query job batches info bulk api 2.0
Before, using bulk api 1.0 endpoint. I could make request to get all job's batches like so: `
req.setEndpoint('callout:BulkApi' + '/' + jobId + '/batch');
req.setMethod('GET');
req.setHeader('...
2
votes
0
answers
91
views
REST API Calls Consumption in Marketing Cloud [duplicate]
If 1 million records are pushed into a data extension via REST API, does this count as 1 call or would it consume many calls?
- 31
4
votes
1
answer
1k
views
Is there a way to delete >1 Million records from a custom Big Object?
First thing that came to my mind was writing a Batch Apex to do this but I get this error.
System.UnexpectedException: The requested operation is not supported by this SObject
Deleting via Bulk API ...
- 1,352
0
votes
1
answer
829
views
BULK API Status Not Found
A simple bulk API which I am trying to create a JOB, and I am getting the status "NOT FOUND" and code: 400.
The same request working in the Workbench. Not sure what I am missing.
bulkApiController....
- 1,589
0
votes
1
answer
1k
views
simple_salesforce - Cannot bulk insert more than 10 records at a time [closed]
I am using the simple_salesforce Python package. I tried bulk inserting 100 records using bulk.insert(), and did so without any errors. However, when I checked how many records were present after, ...
0
votes
1
answer
551
views
Optimal Data Load Speeds
We are currently attempting to load a large amount of data, and are currently using the BulkAPI, with chunk size of 200 and batches of 2000. Is there documentation anywhere that would outline the ...
- 121
0
votes
1
answer
277
views
Exporting of and capturing changes on a large set of records
We have to load the entire Salesforce Lead data(this data might be millions records)to external system using Bulk API.
Once all the data loaded and we thought to initiated to catch the added/modified/...
- 15
0
votes
1
answer
1k
views
SalesForce Bulk API response to Pandas DataFrame [closed]
I'm a beginner with the Salesforce API and recently, I've been trying make bulk API requests and convert the response into a pandas DataFrame, however, the results I'm having a tough time with are ...
0
votes
1
answer
1k
views
Upsert using relationships
I have a custom object Relation__c which has two fields a__c and b__c both are lookups to the standard Account Object.
The Account object has an external Id field called z__c
If I have to use Bulk ...
- 175
1
vote
0
answers
947
views
Salesforce Bulkv2.0 job failed with error message: Failure writing to File store
I executed a bulk job using the Salesforce Bulkv2 API.
Operation: Upsert
Status: Failed
State Message: Failure writing to File store
I can't find information on this error and am wondering what ...
- 175
2
votes
1
answer
1k
views
Will Bulk API be affected by System.isBatch()?
I have a trigger that needs to be run during a Bulk API insert/update of records but that needs to not run during an Batch Apex Job like:
global class myBatchClass implements Database.Batchable<...
- 501
0
votes
0
answers
1k
views
Got "java.io.IOException: Premature EOF" error randomly when retrieve large data from Salesforce
When we use salesforce bulk query to retrieve data from salesforce module.
There are more than 10 million records in the target module.
we got this kind of error randomly:
[FATAL]: local_project711....
- 1
1
vote
1
answer
914
views
InvalidUrl Bulk API creating a bulk job using postman
URI i'm using is: https://yourInstance.salesforce.com/services/async/46.0/job so in my case example is https://cs89.salesforce.com/services/async/46.0/job but it still says InvalidUrl.
X-Session-Id ...
2
votes
2
answers
617
views
using SalesforceBulk API, batch_status() return 'state': 'Queued'
While searching the StackExchange for 'Queued' I found that most answers are related to apex code.
In my case I'm using the SalesforceBulk API (through python).
My scenario:
When calling the ...
- 21
1
vote
1
answer
786
views
Bulk API V1 Query - ResultId missing when running concurrent Bulk Jobs
When querying for the result ids of a batch on a closed Bulk (v1) query job, sometimes the result id is not returned; instead the full returned xml is:
<result-list xmlns="http://www.force.com/...
- 146
2
votes
1
answer
2k
views
Uploading multiple batches in Bulk API 2.0
I am currently using Bulk API 2.0 to insert, update and delete data in Salesforce.
What I am confused about is uploading more than 1 batch of CSV records to a job before it is closed?
I create a ...
4
votes
0
answers
784
views
Bulk data load jobs missing batches and stuck in Open state
I am having a problem with Bulk Data Load Jobs where they seem to be submitted without Batches and as such appear to get stuck in Open state until they are manually Aborted.
This is similar to a ...
- 41
1
vote
0
answers
487
views
Bulk Job Status - Failed vs Not Processed?
What are the use cases for setting a Bulk (version 1) job to a status of Failed vs Not Processed?
Salesforce documentation gives the same description for both
The batch failed to process the full ...
- 146
1
vote
2
answers
1k
views
Populating Big Objects from CSV
I need to create a client application where users can upload CSV files, which will then be used to populate Custom Big Objects.
My problem is I'm not sure which API to use. The Salesforce ...
- 11
3
votes
2
answers
4k
views
How can I monitor the bulk API limit?
Salesforce limits you to 10,000 batches per 24-hour period.
For reference: https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/...
- 440
1
vote
1
answer
1k
views
How to enable Bulk API v2.0
We are loading data into Salesforce from Informatica and have enabled Bulk API option. But in the Salesforce Batch status pages, we see that v1 of the Bulk API is being used.
We are using 41.0 ...
- 111
6
votes
1
answer
8k
views
Bulk API Error: UNABLE_TO_LOCK_ROW:unable to obtain exclusive access to this record or 124 records
During the process of loading 90,000 opportunities into Salesforce using the Bulk API, I get this error:
UNABLE_TO_LOCK_ROW:unable to obtain exclusive access to this record or 124 records...
After ...
- 9,336
0
votes
1
answer
1k
views
Bulk API 2.0 Close Request Failing - Cannot specify query parameters in the resource
I am doing a POC on the BULK API and have run through this trailhead: https://trailhead.salesforce.com/en/content/learn/modules/api_basics/api_basics_bulk where you call the BULK API through WorkBench....
- 93
1
vote
2
answers
579
views
API requests calculation on a bulk upsert costing 6 calls
I am using the simple-salesforce Python package to upsert records into our Salesforce instance. I am finding that each bulk API call counts as 6 API requests. Is this correct? Is there a way to reduce ...
1
vote
0
answers
796
views
Python-driven Bulk API 2.0 hardDelete yields "errorMessage": "InvalidBatch : InvalidBatch : The 'delete' batch must contain only ids"
I am trying to write a bit of python code to permanently delete some records using the Salesforce.com Bulk API v2.
Using curl, I have success. I:
create an Bulk API 2 hardDelete job
PUT the file to ....
0
votes
1
answer
498
views
Bulk API 2.0 trying to post new job, but request is redirected
I am trying to create bulk job using Bulk API 2.0, but my post request is redirected by salesforce to GET request. As a result, job is not created and I get /jobs/ingest GET response instead. I have ...
- 3
0
votes
1
answer
561
views
Bulk API 2.0 Daily CSV upload, need new job every day?
Does a daily CSV upload using the Bulk API 2.0 require a new job created each day? Or can one job handle a new upload daily.
Having a hard time finding this on the docs. I see the sample process ...
- 103
0
votes
1
answer
856
views
Apex: An error has occured reading the CSV fileList index out of bounds: 2
I recieve this error when testing the bulk uploading that we have implemented on our SF environment
Apex Class:
public class CarrierUpdateController {
public transient Blob fileContent {get;set;} ...
- 73
1
vote
1
answer
2k
views
How to upload ContentVersion records using BulkAPI?
So I've been trying to follow the sample java app provided in the documentation to write some Java code to upload ContentVersion records (files) into Salesforce using the Bulk API. It should be so ...
- 63
2
votes
1
answer
1k
views
I am querying more than 2000 records. Is it a good idea to use the BULK API?
I'm new to the Salesforce community, so I apologize if this question is asked wrong.
I was running into essentially the exact same problem mentioned in this post. Basically, I have a query that ...
- 147
2
votes
2
answers
2k
views
Bulk API 2.0 Upload for files bigger than 150Mb
I'm trying to use the Bulk API 2.0 to upload big files. I need to know how I have to upload the data in case the file converted to base64 exceeds the 150Mb allowed. After reading the documentation I'...
- 61
1
vote
1
answer
2k
views
Salesforce Rest API create account
We are using the Salesforce Rest API to integrate our external website with Salesforce Sales Cloud. When a user registers on our website, we create a contact record in Salesforce. The response to ...
- 61
0
votes
1
answer
501
views
External Website integration with Salesforce
We have an external website that is integrated with Salesforce via REST API. On the website, when a user registers, a contact record is created and also a welcome email is sent right away. Users ...
- 61
0
votes
1
answer
2k
views
Can the Bulk API solve the CPU Time Limit Issue I am having
As it stands currently, I have created a scheduled apex job that is supposed to run nightly and compare records being imported into a 'staging' object against records already in custom objects, and ...
- 187
1
vote
1
answer
2k
views
DateTime values in Bulk Query results - difference between CSV and JSON formats
I'm running Bulk API queries in Python, using both the simple_salesforce and salesforce_bulk libraries. I've encountered a confusing result when querying DateTime fields.
Given a simple_salesforce ...
- 92.8k