0

Is there a Salesforce standard (or open source) approach to importing hierarchical data into a "custom" object?

Suppose all UserRole records are exported to .csv (assume the org has/uses a UserRole hierarchy). What would the steps be to import that data into a custom object and maintain relationships?

How would we take hierarchical .csv data and load all its relationships in "one" transaction that is painless for an every day admin? I am trying to avoid manual labor of loading roots, then 1st childs, 2nd childs, 3rd childs, etc. Is there a scripted pattern people use for this?

1 Answer 1

0

There are a number of products that offer this kind of relational import across an object network. Many of them are commercially available and offer nice GUIs or web interfaces.

I'm going to offer two specific solutions because I am the author or coauthor of both of them. Both are free and open source. Both are written in Python and used at the command line.

CumulusCI

CumulusCI is Salesforce.org's CI build tool/scratch org automation engine. It includes several predefined tasks for working with relational data that is persisted from a Salesforce org into a local SQLite database and then reloaded into other orgs.

CumulusCI's biggest strength is scratch org data and large data volume testing. It typically operates on whole-org data sets: you extract everything for a particular set of sObjects and fields, and then load that whole data set into another org. It scales well up to 10s of millions of records.

There's a few things it doesn't handle well (yet, we're working on it all the time, safe harbor etc!), including polymorphic lookups and long text elements, but it's a thoroughly tested production solution used at Salesforce.org to build and test major managed packages and implementations. CumulusCI can be installed using Homebrew or Python's Pip package manager.

Amaxa

Amaxa is a personal project of mine. It is purely a relational ETL tool for Salesforce data (it does not provide the vast automation capabilities that CumulusCI has), and uses CSV files as the local store rather than a SQLite database or SQL script.

Amaxa is weaker than CumulusCI in some respects - it doesn't automatically map Record Types, for example, and has never been tested at million-record scale - but stronger for a couple of specific use cases. It particularly excels with extracting a complete, consistent subset of a production org's data and then loading it into other orgs. It also handles polymorphic lookups and long text data well. Amaxa can be installed with Python's Pip package manager or as a binary download for Mac, Linux, or Windows.

Your Specific Use Case

Your use case is a little unusual because you're not just interested in extracting and loading the data, but also in transforming it to a different object.

This use case is possible in both Amaxa and CumulusCI. You'd write an mapping file (CCI) or operation definition (Amaxa) in YAML to extract your data from the origin sObject, which would get you a data source containing all of the records and relationships in that object.

You'd then write a second operation definition that would remap the data from the stored sObject to the target sObject. At that point, the fact that the stored Ids refer to the original object is irrelevant; they're just unique identifiers to the ETL engine. The load would first push all of the records up with the relationship fields blank, and then perform a second update pass to populate all of the relationships.

0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .