0

I have a requirement to parse CSV and store data into SFDC table. I know , it can be doable by using dataloader / import wizard but there are lot of validation logic will be in place.

So we have decided to use Apex (to parse CSV). Now my question is where shall I place that CSV in SFDC org.

Can I keep it as SFDC File ?

Improve this question

1 Answer 1

Reset to default
4

If you search Salesforce Stack Exchange, you'll find at least dozens of examples of lousy Apex CSV parsers: parsers that fail on files with embedded quotes, embedded commas, embedded newlines; parsers that can't handle anything but UTF-8; parsers that blow up if a row doesn't match the columns headers; on and on.

CSV is a format defined by RFC with a variety of dialects in active use, particularly that of Microsoft Excel. It is not parseable correctly with simple methods like Apex's String#split(). A CSV parser must correctly handle cells with embedded commas, rows with embedded newlines, and quoted cells, including internal escaped quotes.

Apex is not well suited to parsing CSV at all, still less parsing it in a way that is both performant and respectful of governor limits. Even if you write the world's best Apex CSV parser, you're still limited to operating on strings of up to 6 million characters (or you get a StringException), or a maximum of 12 megabytes of heap covering both your file and your data (if and only if you are in an asynchronous context).

In the vast majority of implementation contexts, one of the other possible architectures is better in every way:

  • Parse a CSV uploaded by a user in a Lightning component on the front end, using an existing, specialized CSV library like PapaParse. Pass data from the CSV into Apex to be persisted in the database. Write your validation logic in either JavaScript or Apex.
  • Use an actual enterprise middleware or ETL platform that can parse CSV data and manage the integration of the data into Salesforce for you without heap limitations.

In the future, Evergreen functions may be another option.

But adopting Apex is likely to hamstring you and produce significant limitation and unreliability in your implementation.

Improve this answer
7
  • I already have a CSV parser. My question was , where to place that CSV in salesforce org
    – Arijit Nag Chowdhury
    Commented May 12, 2020 at 14:41
  • @ArijitNagChowdhury Why are you retaining processed CSVs at all? If you want to maintain any file in Salesforce, however, Lightning Files are the correct place.
    – David Reed
    Commented May 12, 2020 at 14:53
  • Thanks .. I think I got it I will keep it in Document .. List<Document> doc = [SELECT Id, name, Body,createdDate from Document where Name = 'FL_insurance_sample.csv']; if (doc.size() > 0){ System.debug('doc '+doc[0].Body.toString()); }
    – Arijit Nag Chowdhury
    Commented May 12, 2020 at 15:54
  • 1
    Why? Documents are limited to 5MB, so you're just adding another limit.
    – David Reed
    Commented May 12, 2020 at 16:09
  • 1
    It's the same thing.
    – David Reed
    Commented May 12, 2020 at 16:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .