Deployment In Chunks Using Metadata API Deploy Method

Question

We have a big project depending on Metadata API and we are using deploying of all components at once.

But now size of components have become too large (80MB) and our server can't send files with this size. If we send files by dividing zip into 20 MB chunks then deployment get failed due to component dependencies.

Is there any way Metadata API like Tooling API where we can create Metadata container on Salesforce side and when we have enough required components then we process the deployment.

The major issue we are facing is dependencies of components if we deploy by multiple zip files. File A can have dependencies with File B and vice-versa. We are doing file based deployment.

Any help will be appreciated.

Answer 1

You may be able to get some progress by considering at a more general component level what dependencies exist and breaking it up by component type. For example try deploying selectively in this order from the applicable /src subfolders and within those see if you can filter further, e.g. test classes.

1. All Custom Objects and layouts
2. All Custom Labels (if you have any)
3. All Apex classes (without tests)
4. All Apex test classes
5. All Apex triggers
6. All Visualforce pages

This approach makes use of the fact that you Setup related information cannot directly reference Apex code, so you can try to deploy as much of that ahead of the code. Then leverage knowledge that Apex code does not reference private Apex test methods, so these can be later deployed. You may also have further outer dependencies in your code base you can identify generally. Once this is all done, issue a runAll tests request.

NOTE: The above assumes your not deploying to production, if so you will have to combine the Apex classes and tests together.

Answer 2

I think you could use unmanaged packages as containers to solve this. First, a distinction: I'm not referring to using unmanaged packages to distribute the code between orgs. What I mean is you can create multiple unmanaged packages in the source org, add metadata to them, and then retrieve from the packages via the metadata api.

The nice thing about packages is that they automatically include any metadata required by metadata you've included in the package. There are a few exceptions such as test classes which have to be manually selected.

To test it out, create an unmanaged package in the org (Setup -> Create -> Package) and add some metadata to it. Any metadata referenced by the metadata you include will be added to the package. You can create multiple packages and add metadata from different functions to them.

Then, retrieve from the metadata api with an ant call like:

<sf:retrieve 
  username="${sf.username}" 
  password="${sf.password}" 
  serverurl="${sf.serverurl}" 
  retrieveTarget="PackageNameHere" 
  packageNames="PackageNameHere"/>

You'll get all the metadata and dependencies in a directory with a package.xml file designed to deploy the code into an unmanaged package in another org. Then create the package in the target org manually (no api for this step) with the same name as from the source and then run a normal deploy from the retrieveTarget. It's not distributing via an unmanaged package, it's just using them as a container for metadata.

The way it deploys into the unmanaged package in the target org is by including the following line in the package.xml:

  <fullName>YourPackageHere</fullName>

In the target org, the unmanaged packages will contain all the metadata you deployed without needing to manually select it.

The other nice thing about this approach is that the retrieved metadata only includes whatever was in the package. This means if the package has 2 fields on Contact, your objects/Contact.object file will only have those two fields and nothing else.