5

I'm building a CSV exporter for use within a Customer Portal that allows them to export in bulk some of their data (basically usage logs, which there are a ton of). For some super users there are more than 50K records available to them. Is there anyway to go about exporting 50K+ records as a CSV for this use case?

My understanding is that if you've got a query returning more than 50K rows your SOL if you want to build that into a CSV in a native context. I've already got a better solution (requiring a date filter to export) for longterm, but would be curious if the community has tackled this requirement before.

Restrictions

  • Can't use reports in customer portal
  • Can't use data loader as customer portal user
0

2 Answers 2

4

The 50k records returned by SOQL is a hard limit. I haven't heard of this particular limit being "black tab increased", but if the customer does not need much more than that it may be worth a try to contact their sales rep and ask.

Failing that, if the export is native Apex, you'll have to break it up into 50k row chunks.

It's possible that with HTML5/JS or Flash, and a clever paginated Apex exporter, you could simulate a larger-than-50k data load even though you're still only chunking out 50k records at a time. Also could look into a browser-based vanilla API integration which would also be able to get at more than 50k records at a time, although as you point out, customer portal users can't use the API.

If the exports aren't ad hoc, you could also do something like generating the export using an API integration running on a timed batch, but upload the large extract into Salesforce as a Document or Attachment. Then give your users the link to that uploaded document.

2
  • 2
    Confirming that query row governor limits aren't editable via black tab: it would actually require a change in salesforce's source code to enable something like that. Commented Apr 29, 2013 at 23:41
  • excellent tip, a scheduled export and upload to attachment would get the job done Commented Apr 30, 2013 at 23:09
3

Take a look at the @ReadOnly annotation. It greatly increases the query row limit, however you'll still have to cope with the 200k statement limit, which would hinder creating a CSV of any reasonably large size.

My suggestion would be to have a visualforce page with a @ReadOnly annotation and then use the JSON.serialize method to return your result set to the client as a large JSON string. Then you can use client-side javascript to build a csv and download it via a data URI.

Another option would be repeated calls to javascript remoting methods to chunk-by-chunk build the result set in memory on the client.

3
  • 1
    Excellent suggestion, that does a lot to increase the max amount of data I can get out. That said, I'm struggling to think of how that would end up in a spreadsheet ready download format, i.e. how to send the browser javascript and then have it end up as a text/csv? Any thoughts? Feel free to tell me I need to go to stackoverflow on this one, but figured i'd check with you first Commented Apr 30, 2013 at 23:08
  • Pretty sure this is possible but would take some fancy HTML5/JS or Flash. And yes, SO would be a good bet :)
    – jkraybill
    Commented Apr 30, 2013 at 23:31
  • My experience with this is limited, but you can look at composing the CSV in memory and then using window.open with a data URI (developer.mozilla.org/en-US/docs/data_URIs) but it's very browser dependent. I'd expect a better response on SO on how to actually generate it properly. Commented Apr 30, 2013 at 23:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .