Salesforce Command line Dataloader - Passing Parameters to Process-conf.xml ?

Question

My currrent situation is I have process-conf.xml, with multiple environments and multiple orgs eg. dev and QA Then an entry for each Object(Account, Opportunity) that I may be inserting for Automation.

<bean id="insert-account-dev3" class="com.salesforce.dataloader.process.ProcessRunner"  singleton="false">
    <description>Insert Account Records used by automated regression
        tests.</description>
    <property name="name" value="insert-account-dev3" />
    <property name="configOverrideMap">
        <map>
            <entry key="sfdc.endpoint" value="https://test.salesforce.com" />
            <entry key="sfdc.username" value="<b>xxxxxxxxxxxxxx@xxx.dev</b>" />
            <entry key="sfdc.password"
                value="<b>XXXXXXXXXXXXXXXXXX</b>" />
            <entry key="dataAccess.type" value="csvRead" />
        </map>
    </property>
</bean>

<bean id="delete-account-dev3" class="com.salesforce.dataloader.process.ProcessRunner"  singleton="false">
    <description>Delete Account records used by automated regression
        tests.</description>
    <property name="name" value="delete-account-dev3" />
    <property name="configOverrideMap">
        <map>
            <entry key="sfdc.endpoint" value="https://test.salesforce.com" />
            <entry key="sfdc.username" value="<b>xxxxxxxxxxxxxx@xxx.dev</b>" />
            <entry key="sfdc.password"
                value="<b>XXXXXXXXXXXXXXXXXX</b>" />
            <entry key="dataAccess.type" value="csvRead" />
        </map>
    </property>
</bean>

    <bean id="insert-account-QA" class="com.salesforce.dataloader.process.ProcessRunner"    singleton="false">
    <description>Insert Account Records used by automated regression
        tests.</description>
    <property name="name" value="insert-account-QA" />
    <property name="configOverrideMap">
        <map>
            <entry key="sfdc.endpoint" value="https://test.salesforce.com" />
            <entry key="sfdc.username" value="<b>xxxxxxxxxxxxxx@xxx.QA</b>" />
            <entry key="sfdc.password"
                value="XXXXXXXXXXXXXXXXXXXXX" />
            <entry key="dataAccess.type" value="csvRead" />
        </map>
    </property>
</bean>

<bean id="delete-account-QA" class="com.salesforce.dataloader.process.ProcessRunner"    singleton="false">
    <description>Delete Account records used by automated regression
        tests.</description>
    <property name="name" value="delete-account-QA" />
    <property name="configOverrideMap">
        <map>
            <entry key="sfdc.endpoint" value="https://test.salesforce.com" />
            <entry key="sfdc.username" value="<b>xxxxxxxxxxxxxx@xxx.QA</b>" />
            <entry key="sfdc.password"
                value="<b>XXXXXXXXXXXXXXXXX</b>" />
            <entry key="dataAccess.type" value="csvRead" />
        </map>
    </property>
</bean>


etc....

currently what I find at the moment to be a problem is if passwords expire or change, I will then have to change my process-conf in multiple places.

Is there anyway to Parameterize, it perhaps having a seperate file like so

devUserName=xxxxxxxxxxx@xxxx.dev
devToken=xxxxxxxxxxxxxxxxxx
qaUserName=xxxxxxxxxxx@xxxx.QA
qaToken=xxxxxxxxxxxxxx
etc.(Other Orgs)

Then having beans

<bean id="insert-account-dev" class="com.salesforce.dataloader.process.ProcessRunner"   singleton="false">
    <description>Insert Account Records used by automated regression
        tests.</description>
    <property name="name" value="insert-account-dev" />
    <property name="configOverrideMap">
        <map>
            <entry key="sfdc.endpoint" value="https://test.salesforce.com" />
            <entry key="sfdc.username" value="**devUserName**" />
            <entry key="sfdc.password"
                value="**devToken**" />
            <entry key="dataAccess.type" value="csvRead" />
        </map>
    </property>
</bean>

Answer 1

Yes, you should absolutely be making use of the config.properties file - it's best practice. You can then add all of your UN and PW to it and adjust the key's accordingly.

project/config/config.properties

sfdc.endpoint=https://login.salesforce.com
sfdc.username=................
process.encryptionKeyFile=C:\\Users\\gazaino\\Projects\\DL\\key.txt
sfdc.password.int=...................
sfdc.password.test=..................
sfdc.password=.....................
sfdc.debugMessages=true
process.enableLastRunOutput=false
sfdc.loadBatchSize=200
sfdc.timeoutSecs=600
sfdc.extractionRequestSize=500

There should be a single config.properties file located in the same path as your process-conf.xml file. You can see it includes more than just UN and PW. You can also see I'm storing my DEV and UAT server information as well. Depending on the server you go to, adjust the parameter as necessary.

---

project/config/process-conf.xml

...
<!-- ENROLLMENT CODE EXTRACT -->
<bean id="enrollmentCodeExtract" class="com.salesforce.dataloader.process.ProcessRunner" singleton="false">
  <description>Extract enrollment code records that don't have their requirement fields set.</description>
    <property name="name" value="enrollmentCodeExtract"/>
    <property name="configOverrideMap">
        <map>
            <entry key="sfdc.entity" value="Enrollment_Code__c"/>
            <entry key="sfdc.debugMessagesFile" value="C:\Users\gazaino\Projects\DL\enrollmentCode\log\enrollmentCodeExtract.log"/>
            <entry key="sfdc.extractionSOQL" value="SELECT Id FROM Enrollment_Code__c WHERE Background_Check_Drug_Screen_Required__c=null OR Client_On_Boarding_Docs_Required__c=null"/>
            <entry key="process.operation" value="extract"/>
            <entry key="process.mappingFile" value="C:\Users\gazaino\Projects\DL\enrollmentCode\enrollmentCodeExtract.sdl"/>
            <entry key="dataAccess.type" value="csvWrite"/>
            <entry key="dataAccess.name" value="C:\Users\gazaino\Projects\DL\enrollmentCode\enrollmentCodeExtract.csv"/>
        </map>
    </property>
</bean>
...

The process-conf.xml file still contains key parameters that are required for that specific bean, but it pulls parameters which are generalized and can be used across multiple beans.

---

CLI Execution

cd dataloader/bin
process.bat project/config enrollmentCodeExtract

1. Navigate to the bin directory of your dataloader
2. execute process.bat with argument 1 being the path to your process-conf.xml file and argument 2 being the name of the bean you wish to run.

---

This KB Article has a great explanation of where you should store specific parameters:

Property value interactions

There are three places where you can define property-value pairs -

The command line

The command line requires two parameters - the configuration directory and the name of the process. You can also include any other property value pairs that could be listed in the process-conf.xml file. Any property-value pairs included as part of the command line are used in place of any other specifications in the other files.

config.properties

File located in the configuration folder, which can contain any property-value pairs. Any property-value pairs listed in this file are used in place of their specification in the process-conf.xml file.

process-conf.xml

File that contains properties for all processes. Each process has it's own section in the file. To invoke a particular process defined in process-conf.xml pass the name of the process in the command via the process.name parameter. For example; jar -cp DataLoader.jar -Dsalesforce.config.dir=conf com.salesforce.lexiloader.process.ProcessRunner process.name=csvUpsertAccount. Otherwise, the process-conf.xml file will not be referenced and all required Data Loader values will need to be entered in the config.properties file.