Regarding web services

Question

Though I’m working on Salesforce platform for some time but I’m still new to webservices and reading a lot about it recently.

I’m working independently and recently my customer( little tech Savvy) asked me to use web services to fetch approx. 4-5 K(thousands) from one of their other system ,daily into his SFDC org. He said , he can provide me WSDL that I should use in some Batch process running once per day.

This left me confused. As far my understanding from POCs I’ve done, Call outs doesn't seem like ideal choice.. when you've to pick thousands of records in response.

If they want to insert/update data in SFDC, then better choice would be , they should consume SFDC enterprise WSDL in their system to make changes in SFDC. Isn’t it ?

SO in nutshell , I’ve these questions :

1. Would it be fine choice to to use WSDL to call that external system and in response get that many records and make changes in SFDC ?

2. If above is fine, I’m thinking, I won’t seem logical to get all data in one call. What kind of issues I could face if volume in response is large?

As far what I got, I should be recommending using bulk API(in future, volume could be more) to get such large amount of data. But what I know, I could use data loader or some Middleware (informatica etc) with Bulk API. But customer doesn’t have budget for middleware.

3. Is data loader ideal choice in that scenario? Customer doesn’t want that any manual step. I know Data loader CLI is also used to automate loading of data. But what I information do I need to ask from customer? would that be direct database connection ?(JDBC /ODBC , etc.)

4. What If customer doesn’t agree to Data loader, what is the next best choice?

5. Lastly Customer is particular about security. What kind of security/authentication mechanism should I recommend to securely pick data from that system?

I know , it is getting long, Please provide any suggestions or some directions atleast, as I’m not getting much help just by reading articles/forums.

Thanks for all the help. It would be greatly appreciated.

Answer 1

1. Yes you can "pull" the data from the other system. But note that the Apex WSDL tooling (WSDL2Apex) has a number of limitations so you would need to experiment and test before being sure that will work. If the other system also supports JSON/REST that may be a better way to go.
2. Salesforce limits the maximum size of the XML and also has limits such as heap size and CPU time. It would be wise to agree an API where you can ask for a block of the data at once so you can stay well within those limits.
3. Web services are generally a better choice than creating and consuming files and firing up the process to do that.
4. Web services.
5. Communicate over HTTPS. There are many authentication option so it is a matter of picking something that the other system supports and Salesforce also supports.

Alternatively the other system could "push" the data to Salesforce by using one of the several web service APIs that Salesforce supports, assuming that system can make web service calls. But that is moving the development burden from you to them.