sfdx Scratch Org creation extremely unstable, unusable

Question

Salesforce's Edge network (and infrastructure in general) is well known for being low quality and having lots of outages. sfdx's buggyness is also well known. However, there seem to be some specifically terrible issues with creating scratch orgs.

I am trying to set up automatic scratch org creation, from an org shape, and installing required packages, to support a CI/CD flow. However, scratch org creation almost never works, and installing packages almost never works. The errors are non-deterministic.

I'm creating a scratch org with a command like:

sfdx force:org:create -v DEV_HUB -f org-shape.json --durationdays 1 -a MyScratchOrg

With the contents of org-shape.json:

{
    "orgName": "MyScratchOrg",
    "features": [
        "Communities",
        "ContactsToMultipleAccounts",
        "PersonAccounts",
        "AddCustomObjects:30",
        "MaxCustomLabels:5",
        "PlatformEncryption",
        "PlatformConnect:1",
        "StateAndCountryPicklist"
    ],
    "sourceOrg": "dev hub org ID"
}

I'm trying to run these commands in CircleCI. The base image is salesforce/salesforcedx:7.172.0-full so sfdx is up to date.

Org creation errors

ERROR running force:org:create: A fatal signup error occurred. Please try again. If you still see this error, contact Salesforce Support for assistance

Try this: See https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/sforce_api_objects_signuprequest.htm for information on error code C-9999.

and

ERROR running force:org:create: Socket timeout occurred while listening for results.

Try this: Use command "sfdx force:data:soql:query -q "SELECT Status FROM ScratchOrgInfo WHERE Id='3SR4w000000aP4SGBZ'"" to retry. You may consider increasing --wait parameter value to increase timeout.

and

ERROR running force:org:create: Socket timeout occurred while listening for results.

and

ERROR running force:org:create: We encountered a JSON web token error, which is likely not an issue with Salesforce CLI. Here’s the error:

(note no error was provided after this)

and

ERROR running force:org:create: ERROR_HTTP_420

and

ERROR running force:org:create: The client has timed out.

Package install errors:

If my setup script is ever unlucky enough to make it to the package install step, it never works. I'm running install commands like:

sfdx force:package:install -u MyScratchOrg --package 04t4W000002g20RQAQ -w=99

Which fails with:

ERROR running force:package:install: ERROR_HTTP_420

and

ERROR running force:package:install: No authorization information found for MyTestOrg.

Question

Are there any special hacks required to create scratch orgs? Is it possible to work with scratch orgs, or are they too low quality to reliably use in CI/CD systems?

Answer 1

That won't solve all your issues, but Salesforce has created a new (and still in beta) new command, sf env create scratch that takes care of network issues and also let you resume a failed Scratch Org creation. It won't solve a fatal signup issue though, I'm afraid you'll have to write a catch in your script to launch creation again if it fails with this kind of error.

If these errors are happening very often, you can try to create a Case but I'm not sure it will be easy to fix as it's random, but maybe you can get the root cause and a Known Issue created.

There is also one of your error which ask to add a --wait flag, adding it would let the command wait for your Scratch Org for longer and avoid your script to fail.

Regarding Packaging, there are currently new commands that are in beta and that you can try. Salesforce is looking for feedbacks here so it's a good time for you to try and give your issues so that the new commands will take care of them.

Lastly, there is a Salesforce DX group in the Trailhead Community where you can try to ask and see if others have the same issues.