Salesforce DX - Installing NPSP

Question

I've started switching over to SFDX from MavensMate, and very quickly have run in to a show stopper. When using MM, I'd create a sandbox, re-attach Git repos and be going again.

SFDX expects me to use scratch orgs, which sounds awesome in since I won't have to deal with the Git shenanigans that MM required me to do when I would refresh a sandbox.

Problem is, I can't manage to get NPSP to install in my scratch org, so none of my code can be imported since it all relies heavily on NPSP.

Using the 'installedPackages' folder in my SFDX project, I have NPSP and all prerequisites

installedPackages
    - npe01.installedPackage-meta.xml
    - npe03.installedPackage-meta.xml
    - npe4.installedPackage-meta.xml
    - npe5.installedPackage-meta.xml
    - npe02.installedPackage-meta.xml
    - npsp.installedPackage-meta.xml

After doing a push, I'll begin receiving emails that the packages have been installed successfully. Strangely, if i look in the 'Installed Packages' UI of the scratch org, none of these packages ever appear. Eventually, I get to the NPSP package and will recieve an email that installation failed

Organization: ACFP-SCRATCH (00D0v0000000nOd)
User: User User (0050v000000nuHM)
Package: Nonprofit Success Pack (04t1Y000001I8yU)
Error Number: 1829127348-16128 (1735776485)
Problem:

1. (Account.BDE_Entry_FS) In field: RecordTypeId - no CustomField named Account.RecordTypeId found
Account.BDE_Entry_FS: In field: RecordTypeId - no CustomField named Account.RecordTypeId found

2. (Account.BDE_List_FS) In field: RecordTypeId - no CustomField named Account.RecordTypeId found
Account.BDE_List_FS: In field: RecordTypeId - no CustomField named Account.RecordTypeId found

3. (Account.Manage_Household_Custom) In field: RecordTypeId - no CustomField named Account.RecordTypeId found
Account.Manage_Household_Custom: In field: RecordTypeId - no CustomField named Account.RecordTypeId found

4. (Opportunity.BDE_Entry_FS_v2) In field: RecordTypeId - no CustomField named Opportunity.RecordTypeId found
Opportunity.BDE_Entry_FS_v2: In field: RecordTypeId - no CustomField named Opportunity.RecordTypeId found
.... There are hundreds more entries that all stem from the lack of RecordTypeId

I have seen other folks talk about using the sfdx force:mdapi to install packages using old package.xml's, but these seems counter-intuitive since the whole point of SFDX was to stop using package.xml and metadata apis.

Has anyone found a way to install NPSP in to a scratch org using the SFDX toolset?? At this point I feel like I'm better off going back to MM, at least my workflows could be automated fully.

edit: I'm also aware of using

sfdx force:package:install -i xxxxxxxx -u scratch-org -w 5

to install packages, but this feels very unintuitive. If the installedPackages directory can be leveraged to install packages, why must I manually run installation commands?

Answer 1

Disclosure: I am on the CumulusCI team at Salesforce.org and am one of the release engineers working on the Nonprofit Success Pack.

The accepted answer on this question is/was correct but is now somewhat outdated.

The recommended route to install the Nonprofit Success Pack in a scratch org is via CumulusCI, which is the SFDX orchestration build tool we use to build the package. CumulusCI is free and open source.

CumulusCI automatically manages NPSP's install dependencies:

• The five underlying managed packages:
  • Contacts & Organizations
  • Recurring Donations
  • Relationships
  • Households
  • Affiliations
• Creation of a default Opportunity Record Type
• Deployment of unpackaged Account Record Types

It selects the latest version of each managed package by inspecting the GitHub releases found in our open source repositories.

CumulusCI is designed to make it easy to build new projects that extend existing managed packages like NPSP. When you start a new project with cci project init, the ensuing questionnaire will ask if you'd like to extend NPSP or EDA and automatically configure dependencies for you. This process is covered in the Trailhead module CumulusCI for Post-Install Customizations, which I co-wrote.

Manual Installation

If you choose not to use CumulusCI, you can still install NPSP in a scratch org by executing the above steps manually or in your own scripting.

Be aware that each biweekly release of NPSP is dependent upon the latest release of the five underlying packages, which are themselves released irregularly (but in SFDO's biweekly release cadence). Ensure that your scripting both

• Pins a specific set of managed package versions.
• Is regularly updated to the latest versions of the managed packages

or

• Automatically resolves the latest versions by inspecting the GitHub releases in our open source repositories.

Otherwise, you risk building your scratch orgs against increasingly outdated versions of NPSP that will not reflect what you see in production through our biweekly push upgrades.

To appropriately configure Record Types, script the creation of any Opportunity Record Type prior to installing NPSP, and deploy the content of the directory unpackaged/pre/account_record_types from NPSP's repository.

Be aware that the repository structure is subject to change. NPSP's CumulusCI configuration is the source of truth for how to script an installation.

Answer 2

EDIT: I have switched to using CumulusCI to manage installing organization dependencies. Pairing it with Github actions yields a good CI/CD workflow. While it does require a bit of additional setup, it can be completed in a few hours and starts bearing fruit almost immediately. I have left this answer here for posterity, or if someone just refuses to use CumulusCI for some reason

Okay, it looks like this is an issue with the way that the NPSP installs and not SFDX itself.

First, the NPSP has a bunch of prerequisite packages, see the NPSP installer page for more info:

https://install.salesforce.org/products/npsp

The prerequisite packages will install just fine, but the NPSP package itself fails because it is missing Account Record Types, Opportunity Record Types, and Opportunity Sales Processes.

These are defined by package.xml, and installed using the Metadata API, not managed packages.

• Opportunity Package.xml
• Account package.xml

This is why the SFDX installation fails.

You have two options here:

• Manually create the record types and opportunity sales process before deploying in your Scratch org
• Recreate the package.xml structure locally and use force:mdapi:convert to make it the proper SFDX format

The first option is straight-forward, but is slow and error prone. The second option only needs to be done once, and it can be deployed using the standard SFDX deploy commands for ever after.

Recreating the package.xml structure locally is easy enough, it looks like this:

NPSP_Setup
 - objects
   - Account.object
   - Opportunity.object
 Package.xml

Copy the contents of the NPSP installer files in to your local file structure, then run:

sfdx force:mdapi:convert -r .\NPSP_Setup\ -d ./npsp_metadata

This command will convert the old package.xml format in to the new SFDX format.

Copy the objects folder out of the npsp_metadata folder and in to your force-app/main/default folder. The 'default' folder should now contain installedPackages, objects, and the Account and Opportunity folders within objects.

Now do a push.... and the install will still fail. It seems that SFDX installs components in some arbitrary order, so the RecordTypes still don't exist before the NPSP package attempts to install.

So the solution I've ended upon is to use the following batch file to prepare a scratch org. You'll still need the package.xml files i mentioned above, in the structure I described.

echo off
set SCRATCH_ORG=%1
IF NOT DEFINED SCRATCH_ORG (exit 1)

echo "Creating Default NPSP Record Type and Opportunity Sales Processes"
sfdx force:mdapi:deploy -d .\NPSP_Setup\ -w 5 -u %SCRATCH_ORG%

echo "Installing Contacts & Organizations 3.7.05"
sfdx force:package:install -i 04t80000000gYcfAAE -w 5 -u %SCRATCH_ORG%

echo "Installing Household 3.9.0.8"
sfdx force:package:install -i 04t80000000jYrOAAU -w 5 -u %SCRATCH_ORG%

echo "Installing Affiliations 3.6.0.5"
sfdx force:package:install -i 04t80000001AVBMAA4 -w 5 -u %SCRATCH_ORG%

echo "Installing Relationships 3.6.0.5"
sfdx force:package:install -i 04t80000000tpCGAAY -w 5 -u %SCRATCH_ORG%

echo "Installing Recurring Donations 3.10.0.4"
sfdx force:package:install -i 04t80000000tpCBAAY -w 5 -u %SCRATCH_ORG%

echo "Installing Nonprofit Success Pack 3.116.0.5"
sfdx force:package:install -i 04t1Y000001I8yUQAS -w 5 -u %SCRATCH_ORG%

Hopefully someone can find a better solution than this, I really wish I could just deploy this stuff using SFDX and not a batch file.

It's also worth noting that once the Account and Opportunity metadata has been upload, doing a push via SFDX works correctly. It is also interesting that if you push just the Account and Opportunity settings, they apply correctly. It only seems to be when pushing them at the same time as installedPackages, everything fails.

So the primary problem seems to be that Salesforce DX has no way of knowing if objects are dependent on installedPackages, or vice-versa. It always installs packages first, so if you need metadata in the org before that, you'll need to rely on some other process to put it there first.