How to exclude invalid fields that can't be deployed by the Migration Tool from .object files when automating deployment?

Question

I'm trying to add a level of automation to our development processes. My goal at the moment is to be able to automatically retrieve check-ins to our development feature release branch and then automatically deploy them into the QA Org for further testing. I've managed to get this all working with svn, the Migration Tool, and cruise control, but my main issue happens when including object definition files (e.g. Account.object) in the package when they have fields that can't be deployed.

I want to get away from new fields having to be setup manually or managed through change sets. It would be great to be able to include the Account.object file in the deployment package when there's been a new field added or modified so there doesn't have to be manually editing of the object file each time.

The problem with this approach is that there are multiple fields included in the .object definition that the Migration Tool can't/won't deploy. I get these error trying to deploy just our Account.object file to another Org:

BUILD FAILED
FAILURES:
Error: objects/Account.object(10881,15):Cannot modify managed object: entity=Web
Link, component=00b30000001pBPs, field=Url, state=MANAGED_INSTALLED: newValue='h
ttps://{URL A}
?sessionId={!API.Session_ID}
&serverUrl={!API.Partner_Server_URL_80}
&id={!Account.Id}', oldValue='{URL B}
?sessionId={!API.Session_ID}
&serverUrl={!API.Partner_Server_URL_80}
&id={!Account.Id}'
Error: objects/Account.object(10906,15):encodingKey must be specified
Error: objects/Account.object(10926,15):encodingKey must be specified
Error: objects/Account.object(10944,15):encodingKey must be specified
Error: objects/Account.object(10962,15):encodingKey must be specified
Error: objects/Account.object(10981,15):encodingKey must be specified
Error: objects/Account.object(11001,15):encodingKey must be specified
Error: objects/Account.object(11012,15):encodingKey must be specified
Error: objects/Account.object(11022,15):encodingKey must be specified
Error: objects/Account.object(11032,15):encodingKey must be specified
Error: objects/Account.object(11042,15):encodingKey must be specified
Error: objects/Account.object(11052,15):encodingKey must be specified
Error: objects/Account.object(11070,15):encodingKey must be specified
Error: objects/Account.object(11088,15):encodingKey must be specified
Error: objects/Account.object(11106,15):encodingKey must be specified
Error: objects/Account.object(11124,15):encodingKey must be specified
Error: objects/Account.object(11142,15):encodingKey must be specified
Error: objects/Account.object(11166,15):encodingKey must be specified
Error: objects/Account.object(11200,15):encodingKey must be specified
Error: objects/Account.object(11231,15):encodingKey must be specified
Error: objects/Account.object(11255,15):encodingKey must be specified
Error: objects/Account.object(11287,15):encodingKey must be specified
Error: objects/Account.object(11354,15):encodingKey must be specified
Error: objects/Account.object(11417,15):encodingKey must be specified
Error: objects/Account.object(3984,13):Field Segment__c is a picklist field. Pic
klist fields are only supported in certain functions. <a href="javascript:openPo
pupFocusEscapePounds('https://help.salesforce.com/apex/htdoor?loc=help&target=ti
ps_on_building_formulas.htm%23picklists_and_msps&section=Customizing&language=en
_US&release=182.8.1&instance=CS3', 'Help', 1024, 768, 'width=1024,height=768,res
izable=yes,toolbar=yes,status=yes,scrollbars=yes,menubar=yes,directories=no,loca
tion=yes,dependant=no', false, false);">Tell me more</a>

        at com.salesforce.ant.DeployTask.handleResponse(DeployTask.java:116)
        at com.salesforce.ant.SFDCMDAPIAntTaskRunner.runTask(SFDCMDAPIAntTaskRun
ner.java:94)
        at com.salesforce.ant.DeployTask.execute(DeployTask.java:59)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.jav
a:105)
        at org.apache.tools.ant.Task.perform(Task.java:348)
        at org.apache.tools.ant.Target.execute(Target.java:357)
        at org.apache.tools.ant.Target.performTasks(Target.java:385)
        at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1329)
        at org.apache.tools.ant.Project.executeTarget(Project.java:1298)
        at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExe
cutor.java:41)
        at org.apache.tools.ant.Project.executeTargets(Project.java:1181)
        at org.apache.tools.ant.Main.runBuild(Main.java:698)
        at org.apache.tools.ant.Main.startAnt(Main.java:199)
        at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
        at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)

Total time: 25 seconds

So the first error appears to be because a managed field is different between the Development and QA Orgs (that's fine but I'd like to exclude it), then there's a shed load of "encoding key must be specified" errors, and finally a complaint about deploying a picklist definition.

So I'm wondering:

1. How do I exclude invalid fields (e.g. managed / picklist) from the .object file to make the deployment work? (Unless there's a way to deploy picklists?)
2. What does the "encoding key must be specified" error mean?

Thanks for your help

Ray

Answer 1

Sense Check. If I have understood correctly you have reached a point where all your metadata and code for your solution is in SVN, nice! Then I assume you have developers deploying from that into orgs to perform development. Such as adding custom fields to Account and then downloading the Account.object file to commit back into SVN. Here you are finding additional information and metadata which is causing problems. If so maybe this will help to ensure only custom fields are committed in your Account.object files.

Downloading only Account Custom Fields via Eclipse. When your developing in your DE orgs you can open up the Project settings and under Force.com, use Project Contents to select only the custom fields for the Account object. This will result in a .object file with only custom fields in it. If your developers ensure they follow this practice, your deployment scripts should have less issues around this object.

As you can also see, only your custom WebLinks and ListViews can also be downloaded in the same way.

NOTE: The package.xml generated by Eclipse uses the CustomField metadata type to list the individual fields being added to account. This is not needed and would cause additional overhead for your developers to keep this in sync. Simply ensuring that the package.xml that gets used by your build scripts includes the Account.object file will be enough.

Hope this helps!

Answer 2

This error occurs when the encoding has not been specified for an element in the .object file. For example, if the encoding was missing (I'm including it) for the xml like the one below in Opportunity.object:

<webLinks>
    <fullName>DeliveryStatus</fullName>
    <availability>online</availability>
    <displayType>link</displayType>
    <encodingKey>UTF-8</encodingKey>
    <hasMenubar>true</hasMenubar>
    <hasScrollbars>true</hasScrollbars>
    <hasToolbar>true</hasToolbar>
    <height>600</height>
    <isResizable>true</isResizable>
    <linkType>url</linkType>
    <masterLabel>Delivery Status</masterLabel>
    <openType>newWindow</openType>
    <position>none</position>
    <protected>false</protected>
    <showsLocation>true</showsLocation>
    <showsStatus>true</showsStatus>
    <url>http://www.fedex.com/cgi-bin/tracking?tracknumbers={!Opportunity_Tracking_Number}&amp;action=track&amp;language=english&amp;cntry_code=us</url>
</webLinks>

That is, if the line <encodingKey>UTF-8</encodingKey> was missing.

You can manually edit the xml and just add that line for whatever element Salesforce is complaining about.

Answer 3

The solution I've used for this in the past is to not include whole object definitions in the package.xml. Instead, the package.xml contains all the piecemeal bits you want to deploy for each object: explicit lists of custom fields, workflows, etc.

It's manual maintenance overhead, and seems like it should be unnecessary, but I couldn't find a simpler solution.

An alternative would be to use the metadata API to dynamically query for fields that meet some "definitely should be and able to be deployed" criteria, then deploy those. That would maybe be worth it on very large / long projects, but seems unlikely to save time otherwise.

Answer 4

Answer to question 2:

I've just hit this same "encodingKey must be specified" error when trying to deploy a .object file that includes a webLink. It appears to be a versioning problem where in API 27 this field is mandatory but in earlier API versions it wasn't.

I used the Force IDE to pull the metadata from an org. That code dates from API 25 or earlier, and although the package.xml had version 27 set in it I am guessing that the Force IDE code was effectively using API 25. I used the Ant deploy task to push the metadata to an org, and that seemed to respect the package.xml version number and so result in the error. The fix for me for now was to change the version in package.xml to 26 and that works. We will be upgrading to the recently released Force IDE soon, but that requires the whole team to change at once because of the gruesome "Upgrade Project" process so takes a bit of co-ordination. But I hope that will eliminate this problem.

Answer to question 1:

I've never really experienced "invalid fields" and you certainly can deploy picklist fields including picklist values. (But there are gotchas: you can't remove picklist values once they are there in an org and you need to include any record types in your package.xml if you are using them.) But you often do need to work at the field level and be selective in what you reference (as jkraybill answered). Read http://www.salesforce.com/us/developer/docs/api_meta/api_meta.pdf very carefully and take a look at what the Force IDE "Add/Remove Metadata Components…" wizard produces as you make various selections.

For example, this package.xml fragment result in a Contact.object metadata file that just contains the two local custom fields:

<types>
    <members>Contact.FullTime__c</members>
    <members>Contact.HoursWorkedPerWeek__c</members>
    <name>CustomField</name>
</types>

Manual editing of the metadata files is just not a sustainable way to develop and at present we do not do that at all. (Figuring out what package.xml can and cannot do is an art in itself though.) Personally I would log a bug (AKA case) with salesforce if I ever found I had to.

Perhaps change sets are the way to go if you are not interested in version control and continuous integration. One gotcha with change sets is that they presently don't seem to play well with managed packages.

Answer 5

The best I've come up with so far is to perform an XSLT transformation of the .object file to strip the troublesome fields out before deploying.

I can identify the managed package fields because the field name has the package name + double underscore prepending the field name. Picklists can be identified because of the <picklist> element inside the <fields> element. This info can be used to match the elements in the XSLT and not copy them into the result file.

(I need the helpdesk to explain the "encodingKey must be specified" error)