Disclosure: I am on the CumulusCI team at Salesforce.org and am one of the release engineers for the Nonprofit Success Pack.
General NPSP Guidance
Because the Nonprofit Success Pack is a complex suite of six managed packages and additional unpackaged metadata, we strongly recommend using CumulusCI, our free and open source build orchestration tool that works with Salesforce DX, to build scratch orgs with NPSP.
We recently released a six-module Trailhead trail, Build Applications with CumulusCI, that teaches how to use CumulusCI to build applications with this tool. The final module, CumulusCI for Post-Install Customizations, walks through the process of creating a project using CumulusCI and Salesforce DX that extends the Nonprofit Success Pack.
Your Specific Situation
The pathway you laid out looks pretty solid - it sounds like you've digested the thesis of source-driven development thoroughly. I'll sketch out a little more detail around how you'd achieve this with CumulusCI.
The first step, as you already know, is to locate the metadata you want from your production org (other than NPSP itself - your dependencies will handle installing that for you with CumulusCI), and get this metadata into a source repository. One route to do that is to create an unmanaged package in your production org and add the metadata making up your application to that package. Salesforce will try to help you out by spidering dependencies and including them too, but this is nonetheless often an iterative and error-prone process. You'll want to include, for example, custom fields you've created on NPSP objects, as well as your own custom objects and Apex code and other work, but not the NPSP's own components (you can generally identify them by having a namespace prefix, such as npsp__
or npe04__
).
Once you identify your metadata fully, you can extract the content of that unmanaged package with CumulusCI via
cci org connect production
cci task run retrieve_packaged --org production -o package <name of unmanaged package>
You'll then get down a package of metadata, by default in the folder packaged
, that you can convert into Salesforce DX format if desired with sfdx force:mdapi:convert
, and that metadata can form the basis of your project.
You can capture a sample data set using CumulusCI's bulk data tooling and make it part of your repository to add into each scratch org. We walk through that process in Data Management with CumulusCI on Trailhead.
Once you've got your metadata and data squared away, you'd build a scratch org using
cci flow run dev_org --org dev
which would automatically contain NPSP, your custom metadata, and your sample data set.
In the scratch org, you'd build out your work, and capture it locally with a task like
cci task run retrieve_changes --org dev
Committing that code to a feature branch (CumulusCI Flow) helps you manage review and ensure you're merging the changes you want to merge into your mainline.
As a note, Communities are one of the more challenging features of Salesforce to work with in source control. Consider exploring the new ExperienceBundle
feature, which was GA in Spring '20 and needs to be explicitly turned on in your org settings (part of the SFDX scratch org definition file, or in Salesforce Setup). That feature promises to make retrieval, source management, and redeployment of Community metadata somewhat easier, and gives you nicer, textual-format metadata to work with.
When you're ready to install into production, you can either deploy directly from your source tree, or build an unlocked package as you describe. I'll stop there as I've never actually build at NPSP-dependent unlocked package - hopefully other contributors can talk more about that end stage of the process.
Resources