6

I need to export data to XML/Excel or CSV from Salesforce.com Checking the current limits I just see those options and limitation:

  1. Render file via Visualforce page with special Mime-Type <- LIMITS 15 MB max page size and maximum records to iterate through in read-only mode
  2. Render file via Blob class in Apex Batch mode <-- LIMITS 12 MB heap size in batch mode.

That's way to few MB for my purpose. Did anyone came up with a better yet not documented solution? Please post it here!

I thought of appending to an existing file during an Apex Batch job, but the Blob class doesn't have any append like methods.

  • Even if Blob would have an append method and you'd use Database.stateful or whatever - you'll hit heap size limits ;) – eyescream Oct 30 '12 at 20:53
  • I forgot to mention that I want/need to export this big CSV directly from a Force.com app. That' why I tagged it Apex. – Robert Sösemann Nov 3 '12 at 11:47
  • How does the end-product have to be delivered? Must it be delivered real-time? Describe the complete desired lifecycle --- for instance, a user clicks "Export Gigantic Amount of Data" and 5 seconds later the user has a file downloaded to their computer. Or a user clicks "Export" and they receive an email with their download. Or every day at 3 AM the user receives an email with the data. What's the scenario? – zachelrath Nov 19 '12 at 16:53
8

If your sole objective is to get the Salesforce data out into a CSV then there are several options outside of apex. I see in your question you tagged the question as apex, so these options may not suit your requirements.

  1. You could use the Data Loader to export the data. As @eyescream points out in the comments, this can be further automated if required - Using Data Loader from the command line
  2. You could create a Data Export (Setup | Data Management | Data Export). This can be scheduled to run on an interval (weekly or monthly) and creates zip files that you can download with the required CSV data.
  3. You could use the Soap API to run an Apex query and then save the results to CSV yourself. There are existing tools that can do this for you if required.

If you need a purely native apex solution you could try and implement some form of server side pagination combined with the @ReadOnly annotation (see Working with Large Sets of Data).

For pagination, you can use a combination of Limit and Offset. That said, Offset still appears to be in developer preview (according to the documentation) and there are notes about efficiency with offsets into a large result sets. The maximum offset of 2000 rows will limit the usefulness of this approach.

Using a Standard Set Controller as covered in How to query more than 50000 records in apex and bind it in vf page in jqgrid and paging? seems like a good option for paging through large amounts of data, but I'm not sure how you would utilise it to create a CSV.

If neither of those options work for pagination you will most likely need to devise your own method to create pages based on the data being exported.

Beyond that I don't think it will be possible to create a CSV of arbitrary size. One way or another the limits will stop you.

|improve this answer|||||
  • 2
    mention the fact that Data Loader can be scripted and even scheduled with cron / Windows Task Scheduler and "you have my axe" (Gimli style). – eyescream Oct 30 '12 at 20:45
  • @eyescream I was obligated as a New Zealander to update the answer due to the LOTR reference. :) I used to live on the hill above Weta workshops and worked in a store that sold products to the production crew. – Daniel Ballinger Oct 30 '12 at 20:58
  • Oh man, I envy you so much! Would love to visit NZ one day (my best friend did, only because I've left the company too soon and he was second best SF expert around :P) And I've read Trilogy only 9 times so far ;) – eyescream Oct 30 '12 at 21:05
  • I forgot to mention that I want/need to export this big CSV directly from a Force.com app. That' why I tagged it Apex. – Robert Sösemann Nov 3 '12 at 11:45
2

try out email services. 1. Create an email service

  1. Use inbound email handler and send an email to the unique id

  2. and in the email service process handling class, code it in such a way that you create a blob and store the document in a contentversion (library) and send the link to that document as an email to the specified email address.

Heap size Limit in salesforce for Email services is 36MB and you can send 2000000 inbound emails per day.

|improve this answer|||||
  • Sounds very good. I will try this out. – Robert Sösemann Nov 16 '12 at 13:30
  • sure @RobertS try it out and let me know – Sathya Nov 17 '12 at 7:49
  • 1
    did you ever get this to work @RobertSösemann? – Shmuel Kamensky Feb 28 '17 at 22:46
1

You can use the Bulk API and do it all from the command line with curl (well, almost all)

http://www.salesforce.com/us/developer/docs/api_asynch/Content/asynch_api_code_curl_walkthrough.htm

This would be recommended over using the SOAP API as it'll more efficiently handle a lot more data.

It'll also export as CSV for you.

|improve this answer|||||
  • I forgot to mention that I want/need to export this big CSV directly from a Force.com app. That' why I tagged it Apex. – Robert Sösemann Nov 3 '12 at 11:46
  • You can't do it then. – Steven Herod Nov 4 '12 at 10:21
1

I think you can do it by using JSRemoting & FileApi in html5 but that will work in chrome only. Also can try https://github.com/eligrey/FileSaver.js

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.