3

I am using Partner WSDL to get data from salesforce,

To do it I am using those steps: 1. Getting the object list:

 sforceService.describeGlobal()
  1. Getting the table field

    sforceService.describeSObjects(..)

  2. Build a query from the fields and table name and use

    sforceService.query(..)

  3. Getting more data

    sforceService.queryMore(..)

for small object it takes around 5~20 second, for big object it can take over a minute.

There is any way to improve the performance?

Update:

I do my test on contact object, if I want to get all his data, it take me 01:06 minutes.

based on this link I tried to change the batch size to 2000 but that's only slow the select query (1:30 minutes)

Any ideas?

3

The first thing you should probably do is add code to record timestamps at each major step in your code and establish which part(s) are slow and use that to guide the changes you make.

In general performance is going to depend on:

  • The speed of the API
  • How big each table row is i.e. how many fields you query and how big each of them is
  • How many table rows i.e. which rows you query

On the first point, the Salesforce bulk API is "optimized for loading or deleting large sets of data" and so may help. See "Chapter 9 of the Bulk API Developer's Guide for how to use that API - you can return CSV or XML data.

On the second point, do you need all the fields? Halving the number of fields will improve the speed by up to x2.

Perhaps the third point offers the most opportunity. If your requirement is to extract every row once only then you are stuck. But it is common to extract data say once a day, and in that case you can choose to only extract e.g. rows that have changed (LastModifiedDate after a day ago) which will typically be only a small percentage of the total rows and so be much faster. Or perhaps there is some other "relevant subset" that you can pick that is also only a small percentage of the total rows.

| improve this answer | |
  • Hi Keith, I am now testing on object Contact that contains around 90~100 columns, I have to display all the data to the user... there is around 2500 rows there, each query/queryMore takes around 6 second that mean that I do it 10 times * 6 second = 1 minutes. that's around 3mb data that normal connection should handel in 3~5 second. NOTE: I ignore base64 columns – shai Jun 1 '14 at 14:01
  • 1
    @shai I take your point that it is a lot slower than the theoretical maximum speed. But is trying to get 2500x100=250,000 fields displayed to a user quickly achieving anything useful? What business requirement are you addressing? – Keith C Jun 1 '14 at 14:35
  • My main target is to create an Microsoft olap cube from it, to do it I have to use an DataProvider/ODBC driver. there is no free DataProdiver/ODBC driver to connect to salesforce so I want to create a text file with the needed tables and then to use the Microsoft Ace driver to take data from that text file. I agree that 250,000 fields sound a lot, but at the end it is only a 3MB text file... that's doesn't sound so much (guess most of the fields are empty) – shai Jun 1 '14 at 14:40
  • @shai Thanks for explaining. If you can't just tell your users to wait (and cache the data outside of Salesforce for the next time they come back) I'd give the Bulk API a go. – Keith C Jun 1 '14 at 14:54
  • Thanks for trying to help, The problem is that's 1~2 minutes for that object, but the CampaignMember object for example is 20 times bigger, and that's take me 20~40 minutes (that's start to be a problem) – shai Jun 1 '14 at 14:59
0

In your query try ordering the query by created date .Ordering may make the query efficient .

If you can afford to use filter ,use that in your query .If more no of data has to retrieved like 5 million or so use bulk API to retrieve.

Here your overall perfomance cost is time for desctibe call + query time

If your requirement can afford storing field names at client end then describe call can be avoided to improve performance

| improve this answer | |
  • For example, I want to get the object "Contact", that's around 3MB (if you create text file from it)... It takes me few minutes to get this object, that's very slow and not something that I can work with... – shai Jun 1 '14 at 11:19
  • whats your query currently? – Mohith Shrivastava Jun 1 '14 at 14:20
  • I do Select (104 fields of the table) from Contact – shai Jun 1 '14 at 14:26
  • can you order By created date and see whats the improvement ? – Mohith Shrivastava Jun 1 '14 at 14:29
  • That column exists on every salesforce object? (My code should be generic for every object in salesforce ) – shai Jun 1 '14 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.