12

We are using Jenkins and Github to automate deployments from dev to test to production and currently do a full deploy (the whole package) based on what is stored in GitHub.

We would like to improve this by doing incremental builds => deploying only the changes commited to GitHub since the last deploy. (preferably using Jenkins)

I have found a couple of things (found, not tested)

What I want to know now is how I can decide between those or if there are other (maybe better) plugins that I can use. For example the first 2 are older than 3 years and don't support Lightning. The third has received some recent developments but has some issues according to GitHub.

Any help and tips are very welcome.

6

There are a bunch of third party solutions offering native CI for Salesforce. Full disclosure that I lead one of the companies building one but I'll provide you with all options so you can investigate.

  1. Gearset - 30 day trial and no org install so easy to start with.

  2. Copado

  3. AutoRabit

  4. Flosum

SalesforceDX will also bring standardised tooling to make Salesforce CI easier to plug into ANT. The tooling will still be built ontop of existing standards so any investment in a third party today shouldn't hamper you migrating later. Certainly I can speak for Gearset and all our stuff will be completely interopable

  • 1
    Haha, you'll provide all the options but only link to the one you are affiliated with. Might as well just link to all of them! – Adrian Larson Oct 13 '16 at 22:39
  • 1
    That actually wasn't intentional! I was just in a hurry and didn't have time to grab links. Good point though and I'll update later today – Kevin Boyle Oct 15 '16 at 15:31
  • Unless I'm missing something those are just other Continious integration tools? However I like your answer, it does not answer to my question. Thank you for taking the time though. – Nielsm Oct 17 '16 at 8:13
  • If it doesn't answer then I think I should delete it or edit it :) – Kevin Boyle Oct 17 '16 at 11:48
  • With Gearset, you would be able to compare your repo with your org and build a package of just the changes. I suspect you can do the same with the others but I haven't tested them, I just didn't want to promote our own solution in isolation. Happy to dig into your use-case in more detail but Gearset will definitely build a package that encompasses just the changes – Kevin Boyle Oct 17 '16 at 11:49
2

Its very late to answer but here is my setup which works:

Code Merged -> Create a deploy folder -> checkout the target branch in workspace - > Merge the current branch to target branch (no commit)-> Run git diff -> Create changes (This will basically create txt files for the next steps) - > Start creating package.xml using the txt files generated in the previous steps –> add these to deploy folder –> Deploy via ant migration tool.

Please find a high level explanation for each step . I have modified all the bash scripts to work for Windows for now.

Code Merged -> Developer does the code merge Run build.xml via Jenkins Ant trigger and specify the target as “sf_build”

Create a deploy folder -> As “sf_build” depends on “sf_prepare_deployment”, the make directory gets fired.

checkout the target branch in workspace - > A batch script to checkout the target branch. And merge the hotfix or feature branch to it. And fire diff command and output it to diff.txt.

Create changes (This will basically create txt files for the next steps) - > This is again a bash script which splits the changes based on destructive or metadata file types.

Start creating package.xml using the txt files generated in the previous steps –> A bash file to generate package.xml by reading the txt files created above and based on the files extension create the package.xml file. for this use: generate dynamic package.xml, my file name is "generate_package.sh" which is also used in build.xml. Start reading final.txt file like this :

if [ -e $1 ]
then   
        SCRIPTFILE=$3'/'$2'.xml'
        echo Scriptfile is: $SCRIPTFILE
        echo ===PKGXML===
        echo Creating new package.xml
        echo '<?xml version="1.0" encoding="UTF-8"?><Package></Package>' > $SCRIPTFILE
        ARRAY=()
        while read CFILE
        do
         // run logic from site above
        done < $1
    if [[ "$2" != *"destructive"* ]]
    then
            xmlstarlet/xml.exe ed -L -s Package -t elem -n version -v "40.0" $SCRIPTFILE
    fi
    xmlstarlet/xml.exe ed -L -i Package -t attr -n xmlns -v "http://soap.sforce.com/2006/04/metadata" $SCRIPTFILE
    echo ====FINAL PACKAGE.XML=====
    cat $SCRIPTFILE

• Deploy via ant migration tool. -> The target named “sf_build” gets finally triggered and “sf:deploy” runs and deploys(checkOnly= false) or validates(checkOnly=true) the package created .

Snippet of build.xml

<target name="sf_prepare_deployment">
    <echo message="START: Create diff dir" />
    <echo message="Checking against targer: ${target}" />
    <mkdir dir="../${diffDir}" />
    <mkdir dir="../${diffDir}/src" />
    <exec executable="C:\Program Files\Git\bin\bash.exe" osfamily="windows" failonerror="true">
        <arg value="${scriptName}.sh" />
        <arg value="${target}" />
        <arg value="${targetName}" />
    </exec>
    <exec executable="/bin/bash" osfamily="unix" failonerror="true">
        <arg value="${scriptName}.sh" />
        <arg value="${target}" />
    </exec>
    <exec executable="C:\Program Files\Git\bin\bash.exe" osfamily="windows">
        <arg value="create_changes.sh" />
        <arg value="${diffDir}" />
    </exec>

    <exec executable="/bin/bash" osfamily="unix">
        <arg value="create_changes.sh" />
        <arg value="${diffDir}" />
    </exec>
    <echo message="Generating package.xml" />

    <exec executable="C:\Program Files\Git\bin\bash.exe" osfamily="windows">
        <arg value="generate_package.sh" />
        <arg value="../final.txt" />
        <arg value="package" />
        <arg value="C:/Jenkins/workspace/deploy/src" />
    </exec>

    <exec executable="/bin/bash" osfamily="unix">
        <arg value="generate_package.sh" />
        <arg value="../final.txt" />
        <arg value="package" />
        <arg value="../${diffDir}/src" />
    </exec>
    <echo message="Package generated." />
    <echo message="Adding properties if applicable" />

    <exec executable="C:\Program Files\Git\bin\bash.exe" osfamily="windows">
        <arg value="properties_helper.sh" />
        <arg value="${propsEnv}" />
        <arg value="../${diffDir}/src" />
    </exec>

    <exec executable="/bin/bash" osfamily="unix">
        <arg value="properties_helper.sh" />
        <arg value="${propsEnv}" />
        <arg value="../${diffDir}/src" />
    </exec>
    <echo message="Properties added. Finished." />
</target>

<target name="sf_build" depends="sf_prepare_deployment">
    <echo message="START: SFDC Deployment" />
    <echo message="../${diffDir}/src" />
    <sf:deploy  username="${sf.deploy.username}" password="${sf.deploy.password}"
                serverurl="${sf.deploy.serverurl}" sessionId="${sf.deploy.sessionId}"
                deployRoot="../${diffDir}/src" maxPoll="${sf.maxPoll}"
                pollWaitMillis="${sf.pollWaitMillis}" checkOnly="${sf.checkOnly}"
                runAllTests="${sf.runAllTests}" logType="${sf.logType}"
    />
</target>

Jenkins setup as follows:

Jenkins here has two stages/Jobs:

  1. Validate – Listens to the PR on a particular branch. This only validates the package (checkOnly = true in Ant Migration tool) and run automated code reviews. This runs when Developer creates a PR.
  2. Merge – Polls the SCM for changes. Only gets activated when PR is validated successfully in the above step. When Scrum Masters clicks merge, this merges the code in the target branch and deploys the package (checkOnly = false in Ant Migration tool).
  • thank you for your reaction. I finally ended up doing something similar using nodejs. – Nielsm Sep 17 '18 at 9:24
  • @Nielsm Have you figured out how to create destructive changes? Only that piece is missing in my setup. – Nagendra K S Sep 18 '18 at 9:55
  • destructive changes is in git and copied along to the target directory. This was probably the easiest part :p – Nielsm Sep 18 '18 at 14:04
  • For destructive changes we need to create destructiveChanges.xml, how did you manage to create that automatically. – Nagendra K S Sep 18 '18 at 14:31
  • Also @Nielsm did you manage to deploy aura components via the above process, I have a only controller js changes in aura, but as git diff only returns js file name we are unable to deploy that as the .cmp file should also be included in the aura folder. – Nagendra K S Sep 18 '18 at 16:56
2

Process that we finally implemented in Jenkins

  1. Source Code Management

Connect to your git repository

  1. Execute shell Script will get the previous commit (last one that is deployed) from a file and use it to generate a diff. Finally the script will initialize a nodejs package that I have added to my git repo. (which is convenient as I can now manage it together with my deployments)

    prevcommit=$(<lastcommit.txt)
    echo Previous commit is $prevcommit
    git diff --name-status $prevcommit..HEAD --diff-filter=ACMR > diff.txt
    cd packageXMLGenerator
    npm install
    cd ..

  2. Execute nodejs This part will call on the script I wrote. The tools.js script will go over all files in the diff.txt file and:

    • copy the file to the target directory (including *.meta-xml file if needed)
    • decide if it should create an entry in package.xml or not. An entry is always * for the metadata type of the file. In the end the target directory will contain ALL files from the diff + a package.xml file containing only mentions of metadata in the package. The timeout part of the script will make sure that the next build step only executes if this one is finished.

the script:

//include script file to generate package xml
console.log('workspace' + process.env['WORKSPACE']);
var tools = require(process.env['WORKSPACE'] + '/packageXMLGenerator/tools.js');

//set some params to initialize
var owner = 'USERNAME', project = 'PRODJECTNAME', branch = 'BRANCHNAME', targetdirectory = 'TARGETDIR/';

//execute script
var flag = false;
tools.startDynamicPackageXMLGeneration(owner,project,branch,targetdirectory).then(function(){
    flag = true;    
});

function checkFlag() {
    if(flag == false) {
       setTimeout(checkFlag, 100); /* this checks the flag every 100 milliseconds*/
    } else {
          console.log('done');
    }
}
checkFlag();
  1. Invoke ant - will deploy the target directory to the target org
  • That is correct, but what if in an aura component only *.js files have been changed, the git diff will give you the file names like abc.js, but how will you deploy only js files in AuraBundleComponent. – Nagendra K S Sep 19 '18 at 6:58
  • so far we did not have any problems with this, it just works. Can't say for sure that we have had this scenario already though. I will have to follow up on that. – Nielsm Sep 20 '18 at 6:23
1

You can try force-dev-tool

Command line tool supporting the Force.com development lifecycle

It supports generation of "packages" for deployment based on git diff. Here is the command:

$ git diff master feature/vat | force-dev-tool changeset create vat

After that you can deploy this "package" using following command

$ force-dev-tool deploy -d config/deployments/vat

There is a limitations which components can be deployed by this tool (mostly due to the limitation of Tooling API), but it works fine with the most popular types (ApexClasses, VF Pages, Lightning Components, etc.)

  • could you please specify the limitations / list the components that cannot be deployed by force-dev-tool? And right now, i am also having an issue generating the package.xml , when the git diff return around 600 files. Have you encountered this? – OK999 Jul 19 '17 at 4:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.