7

What

I am trying to use Dataloader CLI to export Contacts created or modified since the process ran the last time.

The process runs properly, but it always retrieves the same records, and I want it to extract only the ones that have been modified (or created) since this process ran the last time.

Examples

This is how I run the process:

C:\Program Files (x86)\salesforce.com\Data Loader\bin>process.bat "C:\dataloadercli" contact-extract

This is part of the output message shown when I run the process:

Rate: 3.600.000 records per hour. Estimated time to complete: 0 minutes and 0 seconds. There are 9 successes and 0 errors. 2015-05-29 08:13:14,434 INFO [contact-extract] progress.NihilistProgressAdapter doneSuccess (NihilistProgressAdapter.java:63) - The operation has fully complet ed. There were 9 successful extractions and 0 errors.

My process-conf.xml is like this:

<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd">
<beans>

<bean id="contact-extract"
          class="com.salesforce.dataloader.process.ProcessRunner"
          singleton="false">
      <description>Descarga los contactos</description>
        <property name="name" value="contact-extract"/>
        <property name="configOverrideMap">
            <map>
                <entry key="sfdc.debugMessages" value="false"/>
                <entry key="sfdc.debugMessagesFile" value="c:\\salesforce\\logs\\sfdcSoapTrace.log"/>
                <entry key="sfdc.endpoint" value="https://test.salesforce.com"/>
                <entry key="sfdc.username" value="xxxxx@xxxx"/>
                <entry key="sfdc.password" value="xxxxxxx"/>
                <entry key="process.encryptionKeyFile" value="c:\\dataloadercli\\key.txt"/>
                <entry key="sfdc.timeoutSecs" value="600"/>
                <entry key="sfdc.loadBatchSize" value="200"/>
                <entry key="sfdc.extractionRequestSize" value="500"/>
                <entry key="sfdc.extractionSOQL" value="select ID__c, MobilePhone, Birthdate FROM Contact WHERE ID__c != NULL"/>
                <entry key="process.operation" value="extract"/>
                <entry key="process.mappingFile" value="c:\\dataloadercli\\mappings\\contact-extract-map.sdl"/>
                <entry key="sfdc.entity" value="Contact"/>
                <entry key="dataAccess.type" value="csvWrite"/>
                <entry key="dataAccess.name" value="c:\\dataloadercli\\desdeSalesforce\contact-extract.csv"/>
                <entry key="process.useEuropeanDates" value="true"/>
                <entry key="dataAccess.readUTF8" value="true"/>
                <entry key="dataAccess.writeUTF8" value="true"/>
                <entry key="process.initialLastRunDate" value="2005-12-01T00:00:00.000-0800"/>
            </map>
        </property>
    </bean>

</beans>

Questions

Any idea about how can I do this??

Thanks!

5
  • Did you ever figure this out? I have a similar question. Hopeful that you found the answer! – Caleb Sidel Dec 27 '16 at 15:56
  • Sorry, I didn't.\n – Victor_JF May 30 '17 at 13:48
  • I found that you can filter using @process.lastRunDate@ in the WHERE clause if you use "dataAccess.type=databaseRead" in the process-conf.xml bean if your data come from a database instead of a CSV file. – Victor_JF May 30 '17 at 13:54
  • 1
    Thank you Victor_JF for the reply! I ended up modifying the file via Windows batch scripts to put the date into the file before the data loader runs. Not very clean, but it works. – Caleb Sidel May 30 '17 at 17:44
  • Hi Caleb, what changes did you make to the windows batch file to achieve this ? I've a similar requirement. – Ajinkya Phadnis May 19 '20 at 9:47
1

There is no automatic solution for Apex CLI out of the box where you can use the process.lastRunDate from the properties file. However you can set a date filter like below which will limit the rows returned. The first example is where you hardcode a set datetime and the second example you can use a datetime function that is part of the SOQL.

  1. Example (Pull everythng modified after midnight on August 1, 2015) select ID__c, MobilePhone, Birthdate FROM Contact WHERE ID__c != NULL AND LastModifiedDate > 2015-08-01T00:00:00.000Z

  2. Example: (Pull Everything modified in the past 3 days) select ID__c, MobilePhone, Birthdate FROM Contact WHERE ID__c != NULL AND LastModifiedDate = LAST_N_DAYS:3

1

You can use @process.lastRunDate@ in the query. An example can be found on Data Loader Guide, page 34.

0

I have found that Spring Placeholder and SpEL work. While Dataloader allows for use of @process.lastRunDate@ in SQL queries, queries against some outside data source for IMPORT into Salesforce, it does not allow for @process.lastRunDate@ in SOQL, to limit data to be EXTRACTED form salesforce. However, it is noted in the Dataloader documentation that it is based on SPRING. From Dataloader documentation: The Data Loader configuration files are based on the Spring Framework, which is an open-source,full-stack Java/J2EE application framework. The Spring Framework allows you to use XML files to configure beans. Each bean represents an instance of an object; the parameters correspond to each object's setter methods. See the bean LASTRUN using PropertyPlaceholderConfigurer and the use of #{} wrapping around ${} below. ${} is placeholder replacement syntax to replace with named property from some property source (in this case a property file) and #{} is SpEL syntax, allowing for doing SUBSTR on the returned property value.

<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd">
<beans>

    
  <!--Have not yet found a way to use LAST RUN file on a process by process basis when job is running multiple extracts-->
  <bean id="lastRun" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer " lazy-init="default">
      <property name="location" value="../../LastRun/Export Opportunities_lastRun.properties"/>        
      <property name="ignoreResourceNotFound" value="true" />          
      <property name="ignoreUnresolvablePlaceholders" value="true" />  
      <property name="valueSeparator" value="?" />     
                                                          
  </bean> 
  
  
  <bean id="A-ExportOpportunitiesCsv" class="com.salesforce.dataloader.process.ProcessRunner" singleton="false">
    <description> *** Export(Extract) Opportunities ***"</description>
    <property name="name" value="Export Opportunities"/>
    <property name="configOverrideMap">
      <!-- Overrides and additions to values provided in config.properties -->
      <map>
        <entry key="process.operation"          value="extract"/>
        <entry key="process.outputError"        value="OpportunitiesERROR.csv"/>
        <entry key="process.outputSuccess"      value="OpportunitiesSUCCESS.csv"/>
        <entry key="dataAccess.type"            value="csvWrite"/>
        <entry key="dataAccess.name"            value="..\..\DATA\Opportunities.csv"/>
        <entry key="dataAccess.writeUTF8"       value="false"/>
        <entry key="sfdc.entity"                value="Opportunity"/>
        <entry key="sfdc.extractionRequestSize" value="2000"/>
        <entry key="sfdc.extractionSOQL" 
               value="SELECT id, CreatedDate, LastModifiedDate, SystemModstamp
                      FROM Opportunity 
                      WHERE SystemModstamp >= 
    #{ '${process.lastRunDate}'.substring(0,10)+'T00:00:00.000-0400'}
                      and  IsDeleted = FALSE
                     "/>
      </map>
    </property>
   
  </bean>

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.