Speed up/optimise build time (ant deployment)

Question

I have a build/ant deployment related question for which I did not find any reasonable answers with a google search, hence posting the question here.

I have around 9 individual repositories where one is dependent on the other (no cyclic dependency). Whenever a dev raises a PR we have a mechanism where a scratch org is created and all of the 9 repositories are deployed to the SF scratch org and if the deployment fails, it also means that the build fails. This process takes too long (about 70-80 minutes).

The issue is that the build takes 70-80 minutes even when there is a small change (1-2 class files).

Is there a way to speed this up? When I save a class in the developer console, it is pretty fast, is there a way to emulate the same with an ant deployment?

Edit: I do not want to run any unit tests as part of the faster build.

Answer 1

In this case, as a first step, you will need to ascertain what is taking the time.

1. 9 deployments with no test running?
2. Does Jenkins job have some pre-deployment activities?
3. Are All local tests are running with each deployment which are causing issues.

---

Possible solutions:

9 deployments with No test running?

If the lag is due to deployments running then you can do two things:

1. Combine the repos and then deploy.
2. Deploy only delta changes instead of whole repos. In this, you will need a script to create a delta deploy to check the changes between two commits or merges.

Does Jenkins job have some pre-deployment activities?

The only solution will be to reduce activities.

Are All local tests are running with each deployment which are causing issues.

Options:

1. Deploy without running tests and run all tests with the last deployment.
2. Combine everything and then deploy.
3. Deploy delta with specific test running. More maintenance and overhead.

You can pick and choose what is your scenario.

Answer 2

When I save a class in the developer console, it is pretty fast, is there a way to emulate the same with an ant deployment?

When you save a class in the Developer Console, the remainder of your metadata is already present in the org. Scratch orgs always start from an empty slate, so you must perform all of the required deployments to populate the org with metadata.

I see a couple of potential routes to achieve speedups here, but nothing that's going to be a silver bullet. One key question for me is what exactly the relationship is between these repos, and what stage you're at in the development process when a PR is made against one of them. You definitely want to do a full test pass across your entire application before you move to production, and it is beneficial to get that pass done as early in the development lifecycle as possible to catch bugs early.

• Flatten your dependency tree/increase the modularity of your code. If you can reduce the scope of metadata that needs to be deployed just to effectively test changes in one module, you may be able to dramatically reduce your overall build times.

  For example, if you have nine modules, and eight of them simply depend upon a single base module (rather than depending upon one other as well), you may be able to reduce the scope of deployment required for feature tests, and reserve deploying the whole application for an integration test pass. Each PR would just test the module changed with that base module, rather than deploying all nine modules.

• Ensure that you are not repeatedly running the same tests. For example, if your deploys are all running with RunAllTests test level, you're burning excess time running the same tests over and over again.
• Prebuild and cache scratch orgs with everything save one module deployed so that you can quickly deploy just one module and test it. It's a little harder for me to see how this would work in your situation, but it's something worth considering depending on the overall shape of your development process - particularly if these repos actually do build packages rather than just unmanaged deployments. Having nine modules would definitely increase the complexity of building this out and cost you more scratch orgs.

Incidentally, there is really no reason to use Ant deployments with scratch orgs. Salesforce DX itself is perfectly capable of deploying both Metadata API source and Salesforce DX format source, and that would mean that you need only one tool.

70-80 minutes is not an outrageous time for a very large, very complex project to do a complete org build. I think the key here is going to be understanding what you actually have to deploy to create a viable test org for each change that's made.