To migrate "trees of related data" from an external system to Salesforce I am using dynamic chains of Apex Queueables. Imagine I am "synchronizing" Accounts, Opportunities and Contacts from an external CRM into Salesforce.
There is a separate Queueable class for each object type and to stay within limits each Queueable is just migrating a certain amount of objects. This is how a typical flow could look like:
AccountQueueable: Get 10 Accounts
ContactQueueable: Get 1000 Contacts of that 10 Accounts
OpportunityQueueable: Get 200 Opptys from Contacts and Accounts in 1./2.
AccountQueueable: Rerun for next 10 accounts
ContactQueueable: Get related Contacts ...you get the scheme
This works with 3 related object types but if I get more I am unable to run even the smallest scenario in my dev org because I hit the limit document here https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_queueing_jobs.htm
For Developer Edition and Trial organizations, the maximum stack depth for chained jobs is 5, which means that you can chain jobs four times and the maximum number of jobs in the chain is 5, including the initial parent queueable job.
I can't use Batch as I am not working on a single object. I also can't increase the limit as Salesforce told me that's a hard limit of dev orgs.
Maybe my overall approach is wrong?! What would you do here?