Sample Data in Developer Sandbox

Question

I've seen a few threads on this subject, but they're pretty old. Thought I'd reach out and see if there's there were any developments on this subject.

We have a highly complex data structure (500+ objects, multiple dependencies), and trying to move away from developing in a full/partial sandbox, to developing in personal developer sandboxes. However, we need a small subset of data to work with. That also includes many custom settings which of course are being created empty in dev sandbox after refresh, and our code highly relies on their data (too many to try to move them to MDT, would be a long project).

We don't want to have to move every piece of half baked code to full sandbox just for testing during development, that misses the point of developing in personal sandboxes in my opinion.

Has anyone ever tried to tackle this challenge? What did you do? Any thoughts and recommendations would be highly appreciated!

Thanks

Answer 1

Disclosure: I am the author of this free and open source project.

I had exactly the same problem, so I wrote Amaxa. It is an open source, command line data loader built in Python, whose raison d'etre is extracting self-consistent data subsets from a production Salesforce org and loading them into sandboxes.

Features that specifically target this use case:

• Select a set of data from Production by specifying top-level objects, such as one or several Accounts, and let Amaxa follow relationships to extract a complete, self-consistent set of related records.
• Load data set in a single operation, rebuilding all relationships including reference cycles and hierarchies.
• Store data in version control as readable, diffable CSV files.
• Emphasis on correctness and validation (never load bad data).

The one thing that I'm not sure about is your Custom Settings. Amaxa doesn't have functionality that specifically targets Custom Settings, and I'm not entirely sure off the cuff how they'll behave if you include them in your operation definition. However, sObject data is well supported, regardless of the size or complexity of your data model.

Also worth a look is CumulusCI (I am on the CumulusCI team at Salesforce.org). CumulusCI is primarily targeted at scratch orgs, not sandboxes, but also has sophisticated tools for data seeding, and explicitly supports Custom Settings with a special task. It doesn't have the same data subsetting features as Amaxa, however, due to its focus on scratch orgs.

Answer 2

Salesforce DX is tailor-suited for this method of development, except you would work in Scratch Orgs instead of Developer Edition orgs. This lets you set up temporary orgs, deploy whatever subsets of metadata you want to, and even deploy a limited set of related data (force:data:tree:import) drawn from data you've built (force:data:tree:export).

Once you've set up the scripts you'd need, you can spin up a new org in just a few minutes with minimal fuss. No waiting in long server queues for a new Sandbox to appear, no need to move full sets of metadata just to add a feature to one specific functionality you've built, etc. Other solutions are indeed out there, but if you haven't looked at DX yet, you may want to take a look. It also comes with a Trailhead to get started quickly.

Answer 3

I have created Forceea, an open-source data factory, which can easily generate data of very complex data structures for both real data (e.g. in a sandbox or scratch org) or testing data. It works synchronously and asynchronously, if you want to create thousands (or even millions) of records.

Forceea uses Dadela data generation language, which makes it very flexible and powerful. Have a look at the User Guide (you can download it from https://nmitrakis.com/Forceea-UserGuide).