SFDX development 'without ScratchOrg and Unlocked Packages' with Git [closed]

Question

I think that the first reaction that anyone will have after reading my question will be 'Why do you want to move to DX without using ScratchOrgs and Unlocked Packages'? Well, we are planning to use ScratchOrgs and Unlocked Packages but our Salesforce Organization size does not allow us to migrate all our work overnight at this point of time. So, the problems that we are trying to solve are:

1) Add a source control - git.

2) Start using DX for deployment and other useful commands for development.

3) Use DX mdapi for deployment.

Our current plan for git:

1) Developers will always create a branch from master.

2) Developers will test their changes on their sandbox from their feature/devBranch.

3) Create a Pull Request to merge code with deploymentReadyBranch.

4) After the merge happens, a diff will run between deploymentReadyBranch and master

git diff --name-only HEAD^

All updated files will be archived and a package.xml will be created using

sfdx force:source:convert --rootdir <dir> --outputdir <dir>

This will give us a directory with changed files and supporting package.xml and then, the code will be deployed to an environment of choice.

5) After successful deployment deploymenrReadyBranch will be merged with master.

Does this strategy look good? Suggestions on branching strategy will be very helpful.

Challenge: 1) As of now, SFDX does not provide any way to create a deployment package.xml based on a diff (in our case diff between master and deploymentReadyBranch). The process that I mentioned in Step 4 above seems a little risky because there will be some manipulation involved with scripts.

With manipulation, I mean that if an apex class .cls file is modified, diff will only show me the .cls file and not the supporting cls-meta.xml file. So, I will have to take the name to modified .cls and copy the respective cls-meta.xml file and so on. For objects in DX, I will have to copy the whole folder of the changed object.

2) A workaround can be to ask all the developers to copy the changed files to a common folder which will then be used to convert to package.xml deployment package.

3) In this post-Development Life Cycle of using GIT in salesforce. sfdxfox mentioned not to have a strategy to create a feature branch from master.

Any suggestions on this will be useful. Also, a direction to properly use DX without ScrathOrgs with a vision to smooth transition to ScratchOrgs and Unlocked Packages will be really helpful.

Thanks.

Answer 1

First, as I said before, don't merge directly in to master. Also, don't do crazy diffs and stuff to try and make it work this way. You're right to be concerned about automatic package.xml generation, it's a non-trivial task at best. I recommend reading A successful Git branching model (not my blog/post, just really useful!) for a better branching strategy. We use a variant of this, and we've had lots of success with it.

One common misconception (I did it, too!) is to think that you need to migrate overnight. That's simply not practical, and will waste many precious work-hours on a fruitless project. Having personally wasted about 160 hours worth of work on it over a period of ... well, longer than I care to admit, the entire project was just a failure.

Our strategy that we intend to start using later this month uses unlocked packages. It does not, however, use Scratch Orgs, as we are a supersized org, and it will take a number of unlocked packages and some architectural changes to fix the problem.

All you need to do is start converting your code in units. Create dependencies within your packages. For example, we have a common set of core utilities (TestingFramework, TriggerFramework, CurrentUserInfo) that will all end up in a core package. We have a few additional packages that will be used for core object and field definitions. Then, we'll have a number of small packages that depend on any other packages, and contain specific features.

This won't be done overnight. It won't likely even be done in a month. What we'll be asking our developers to do is to create and update packages as we work on existing code, slowly but surely migrating the majority of our work in to packages. 2GP (Second Generation Packaging) is not a technical debt that needs to be resolved overnight, but you can pay it down in increments. The longer you take to get started with it, however, the longer it will take when you finally get started.

Once you have your 2GP mostly in order, Scratch Orgs can come next. 2GP means you'll spend less time uploading metadata and instead just installing packages. And you'll want to create some "data plans" to import sample data for your developers to test with. Scratch Orgs are a larger undertaking, and I strongly recommend you already have your packages created before you get too far down that road.

Before you commit to anything, though, make sure that you do more research first. Also, create a Developer Org, and experiment with Dev Hub before you start working with versions in your org. You can use this to experiment with 2GP and your git branching strategy using a small portion of your metadata before you commit to something larger. The best path to migration is to not try and do it all at once.

Once your developers feel comfortable with the idea of 2GP, they can start converting core functions first, then build dependent packages afterwards. In most cases, core functions are rarely more than a small slice of the overall code base, so converting this first should provide some immediate feedback and confidence in the process.

To ease the process until then, consider getting a service like GearSet or AutoRABIT (note: please do appropriate research before choosing a service) that can automate your deployments for you. GearSet is what we use in-house, and it has the ability to connect with Git, and can merge between any two orgs or an org and a branch. Find something that's in your budget and has the features you need, they're definitely out there.