Exporting Data From Salesforce - REST or SOAP?

Question

We have a requirement where we need to export records from salesforce tables into our application programmatically. I am reading on the various API's for data integration from salesforce mainly SOAP vs REST API's.

From what I understand, REST should be used when the web service requests are from mobile devices or browser based clients (since they understand JSON). Is that true? If that's the case how do Bulk REST API's fit the bill (since they are meant for bulk data transfers)?

I would like to get your thoughts on what are pros and cons in using SOAP or REST API's to export data records. Note that the requests are being made a java ee server application programmatically. After retrieving the records (probably in batches) from salesforce records we process (validate, filter etc) them and load them in our internal tables.

Thanks.

Answer 1

As a starting point, have a read of Which API Should I Use?.

When to Use REST API

REST API provides a powerful, convenient, and simple REST-based Web services interface for interacting with Salesforce. Its advantages include ease of integration and development, and it’s an excellent choice of technology for use with mobile applications and Web projects. However, if you have a large number of records to process, you may wish to useBulk API.

When to Use SOAP API

SOAP API provides a powerful, convenient, and simple SOAP-based Web services interface for interacting with Salesforce.You can use SOAP API to create, retrieve, update, or delete records. You can also use SOAP API to perform searches and much more. Use SOAP API in any language that supports Web services. For example, you can use SOAP API to integrate Salesforce with your organization’s ERP and finance systems, deliver real-time sales and support information to company portals, and populate critical business systems with customer information.

When to Use Bulk API

Bulk API is based on REST principles and is optimized for loading or deleting large sets of data. You can use it to query, insert, update, upsert, or delete a large number of records asynchronously by submitting batches which are processed in the background by Salesforce. SOAP API, in contrast, is optimized for real-time client applications that update small numbers of records at a time. Although SOAP API can also be used for processing large numbers of records, when the data sets contain hundreds of thousands of records, it becomes less practical. Bulk API is designed to make it simple to process data from a few thousand to millions of records.

The correct API for the job will depend on where you are calling it from, how many records you are working with and how long you are prepared to wait for the result.

Your use case

From your description of your server applications I'd suggest starting with the Bulk API. It sounds like your code is running without a user waiting for immediate output, so the asynchronous nature of the calls shouldn't be an issue. Moving more records in fewer API calls will also help keep your API call count down.

The SOAP API is another option if the number of records isn't particularly large and you are more comfortable using XML SOAP Messages after importing the WSDL.

At what number of records you should switch from the SOAP API to the Bulk API is a bit of a gray area. The SOAP API would work well from 1 to several thousand records. Note that it batchs up to 2000 records per query result. The Bulk API works better when dealing with thousands of records (or more). The JSON response data format makes it easy to process the data with JavaScript in the browser.

There is also a discussion style blog post Salesforce APIs – What They Are & When to Use Them.

---

UPDATE for comments:

I do not see much benefit using REST over SOAP. Hence the question.

With SOAP you can import a WSDL into your tooling of choice and have native methods generated for working with the API. This makes calling the API and accessing the data relatively straight forward (E.g. strongly typed classes in .NET and Java).

There is some overhead for all the SOAP marshaling code that is generated. REST strips away a lot of this and sends more minimal messages that are closer to how HTTP operates.

Consider this diagram I've borrowed from the popular StackOverflow question: REST and SOAP

- Image by Nakkeeran

Any thoughts on the (non-bulk) REST API for exporting records?

In general, the REST API (non-bulk) isn't the first choice for a server to Salesforce integration where the focus is more on moving large volumes of records rather than responding to the user quickly and integrating with software running in a browser.

You could use the REST API if you want, but the SOAP and BULK APIs are generally better suited for your use case.

Answer 2

IMO deciding which API to use generally comes down to how well-supported the chosen API is, both within your technical environment and within your team's expertise. Both work just fine, and the overhead of SOAP is not enough of a drawback for a decision really to come down to it.

If I'm doing an app-to-app integration where I need to access the imported objects from within something like Java, I find SOAP to be a good choice because you get the strong typing on the import side and a set of classes that define the object model you'll be dealing with.

The Bulk API is a good choice if you're dealing with large data volumes where the time taken to export in batches would be too high.

REST may be more appropriate if you're doing a browser-based or mobile app integration, or have an environment that already "speaks" JSON and OAuth. It has pretty similar characteristics to the SOAP API, so the technical endpoint environment is the strongest factor to consider between those two.

Based on the limited info you've given about your use case, I think you'll find the Bulk API is a good way to go if you're doing huge data dumps. The Streaming API is also worth a look if your "export records" requirement is a synching requirement where you'd want to be notified for every new/changed record that you are synching.

If you give some more info on exactly what you're trying to do, you may get better answers about pros and cons. The good news is that there is usually multiple correct decisions and you are unlikely to regret using one API over the other if you do a quick proof of concept.

Answer 3

http://www.salesforce.com/us/developer/docs/api_rest/

The REST API's that are standard are really helpful if the data volumes are less .If the data volumes are large bulk API will be suitable .

http://www.salesforce.com/us/developer/docs/api_rest/index_Left.htm#StartTopic=Content/dome_query.htm

The above explanation describes the Query REST API and this is helpful as it auto chunks the record and any no of data can be pulled in chunks .