0

I got some code which is used to import csv, parse csv then save it in a custom object. My problem is when I try to save it to a custom object (using visual force) I finally reached an error of Apex CPU time limit exceeded after trying to save 800+ csv rows with 20 fields. I tried minimizing the csv data to test how much the code can save without getting that error and I was able to save just 200 rows (still with 20 fields). Can anyone pls help me optimize this code to overcome such limit and somehow prevent the error.

(the bold ones are the lines that the lines specified after multiple attempts ) THANKS !

public Map<String,List<String>> cName {
    get {
        if(objectFieldMap == null) {
            cName = new Map<String,List<String>>();
        } else {
            cName = new Map<String,List<String>>();
            **for(String var : objectFieldMap.keyset()) {              
                List<String> smfield = objectFieldMap.get(var);**
                    if(smfield[1] != '--Select--'){
                        String originalName = csvMap.get(Integer.valueOf(smfield[1]));
                        cName.put(var,new List<String>{smfield[0],smfield[1],originalName,smfield[2]});
                    }
            }
        }
        //          return cName;
    }
    set;
}


for(Integer i=1;i<allFields.size();i++){
    Report__c indiReportList = new Report__c();

    **for(String theField : cName.keyset()) {
        List<String> finalMapField = cName.get(theField);**
            String fielddataType = finalMapField[3];
        try {
            if(fielddataType == 'String') {
                String finalData = allFields[i][Integer.valueOf(finalMapField[1])];
                if(finalData.substring(0,1) == '"' && finalData.substring(finalData.length()-1) == '"'){
                    finalData = finalData.substring(1,finalData.length()-1);
                }
                indiReportList.put(theField,finalData);
            } else {
                String rawData = allFields[i][Integer.valueOf(finalMapField[1])];
                rawData = rawData.replace(',', '');
                rawData = rawData.replace('$','');
                if(rawData.substring(0,1) == '"' && rawData.substring(rawData.length()-1) == '"'){
                    rawData = rawData.substring(1,rawData.length()-1);
                }

                double finalData = Double.valueOf(rawData);

                indiReportList.put(theField,finalData);
            }
        }
        catch (Exception e){
            hasError = true;
            String typeString = (fielddataType == 'String')?'String':'Number';

            theErrorMessage += '<br/>'+'Row '+i+' Column '+ finalMapField[2]+ '. The '+finalMapField[0]+' field only accepts '+typeString+' value.';
        }

    }
    if(hasError){
    }else {
        indiReportList.put('Sample_Object__c',Id.valueOf(platformSelected));
        indiReportList.put('Status__c','New');
        reportList.add(indiReportList);
    }

}

--------------------------debug log(after saving 200 csv with rows)------------------------------------

Number of SOQL queries: 1 out of 100
Number of query rows: 999 out of 50000
Number of SOSL queries: 0 out of 20
Number of DML statements: 1 out of 150
Number of DML rows: 199 out of 10000 Maximum CPU time: 11866 out of 10000 ******* CLOSE TO LIMIT
Maximum heap size: 258424 out of 6000000
Number of callouts: 0 out of 100
Number of Email Invocations: 0 out of 10
Number of future calls: 0 out of 50
Number of queueable jobs added to the queue: 0 out of 50
Number of Mobile Apex push calls: 0 out of 10
2
  • 2
    Your code is a little hard to follow but at first sight seems overly complex. Code like this Parse a CSV with APEX keeps things simple which helps performance; lots of maps and sets should not be needed. But there will always be a size limit given the CPU limit that is applied. An alternative approach, if the data you are parsing goes through the browser, is to do the parsing in JavaScript using e.g. PapaParse and then insert the objects over one of the JavaScript APIs.
    – Keith C
    Jun 17 '15 at 22:23
  • per @sfdcfox, your batchclass will need to implement Iterables and Iterators to return CSV rows for each batch execute() scope
    – cropredy
    Jun 17 '15 at 23:28
1

To properly process CSV files of arbitrary size, you need to process the data asynchronously. Apex Code is slow, and you're not likely going to get more much more than perhaps 400-500 rows within the 10 second time limit (approximately 8000-1000 total "cells" or fields), even with heavy optimization. Instead, you're going to want to consider a batchable interface, and just read the data at a leisurely pace, creating the records as time permits during each execution cycle, or perhaps batching them up near the end, using a two-phase parse/create cycle. Batchable classes have increased memory, longer CPU timeouts, and, of course, can chain together multiple cycles for even longer processing.

3
  • my problem is about saving the data because it takes too long because my csv data is not automatically passed to its corresponding field ( like account csv to account name from object) but it is dynamically set using a dynamic selectlist which lists the rows from csv and is manually chosen by user via visualforce...(like object field Account Site can be any csv field as long as it has the same data type like csv Address) Jun 18 '15 at 1:45
  • The problem is simply parsing the data. Trust me. I've written a CSV parser and "data loader". You're not the first one to roll your own. I speak from experience.
    – sfdcfox
    Jun 18 '15 at 1:50
  • thx for the explanation @sfdcfox...I'll try to research a bit more about what you said..I'm totally a newbie at CSV parsing(just started days ago)..may I ask an example on how to change data processing from synchronous to asynchronous...It'll help me a lot.. Jun 18 '15 at 2:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.