5

Let's say I have two classes
- AccountController - contains logic
- TestCoverage - contains tests for AccountController

With API v32.0 it was (and still is) possible to deploy .zip which contains nothing but package.xml, request a test run, and get coverage result (covered/uncovered lines, and coverage warnings).

With API v34.0 Metadata deploy (with test run) does not return coverage for classes/triggers not included in the package.zip

This can be easily reproduced with workbench and a developer Org.

I will be using two zip files for tests below.
zip files structure is as follows
components-has-class.zip:
- unpackaged\
--- classes\
------ AccountController.cls
------ AccountController.cls-meta.xml
------ TestCoverage.cls
------ TestCoverage.cls-meta.xml
--- package.xml

components-no-class.zip:
- unpackaged\
--- classes\
--- package.xml

Step 1. Test data Preparation
Take components-has-class.zip and deploy it into your test Org how you do that does not matter. Important part is that these two classes must be in the Org for the next step.

Step 2. API 32.0 - where everything works as expected
login to workbench with you dev Org credential and select API 32.0 (version is important)

deploy components-no-class.zip with following settings:

  • allowMissingFiles: true
  • checkOnly: true
  • runAllTests: false
  • runTests: TestCoverage
    Leave other settings as in their default values

When deployment completes - observe that workbench returned coverage details for AccountController.cls as well as code coverage warnings.

Step 3. API 34.0 - coverage is not returned
login to workbench with you dev Org credential and select API 34.0 (version is important)

deploy components-no-class.zip with following settings:

  • allowMissingFiles: true
  • checkOnly: true
  • testLevel: RunSpecifiedTests
  • runTests: TestCoverage
    Leave other settings as in their default values

When deployment completes - observe that workbench did NOT return coverage details for AccountController.cls

Note - this is NOT a workbench issue - the same behaviour can be reproduced if you write your own deployment test e.g. in java using example from the documentation.

Workarounds
- either include all Class/Trigger files you want to get the coverage for in package.zip
- or use tooling API to run tests

Unfortunately both workarounds involve different degree of inconvenience, compared to old deploy/test method.

Does anyone have any suggestions of what else can be done to make good old deploy/test/get-coverage method working again with API v34.0? I can not get back to v32 because my tools have already been updated to v34.0 and metadata deployment no longer works with v32.0 because v34.0 changed signature of some API calls.
Or how to report it to SFDC? (I do not have premier support access)

  • Even without premier support, you can always file a support case with Salesforce. You just don't go to the front of the line like a premier support customer does. Click on Help > Support to get to where you contact support to file a case. – crmprogdev Jul 25 '15 at 20:44
  • Hello @crmprogdev. Thanks for the suggestion. Unfortunately I only have access to Developer Org(s) and these days it does not seem to be possible to open a case from a Development Org account. Confirmation: success.salesforce.com/answers?id=90630000000h3ImAAI – gaiser Jul 25 '15 at 21:48
  • A guess: did you try to play with testLevel param? It is new. – Ashwani Jul 26 '15 at 7:13
  • Hello @Ashwani. Not sure what you mean by "play with testLevel param". The only value of testLevel which allows setting specific test(s) to run is RunSpecifiedTests. All other settings of testLevel result in: INVALID_OPERATION: runTests can only be used with a testLevel of RunSpecifiedTests. This is in line with the documentation. – gaiser Jul 26 '15 at 9:52
  • @gaiser. I have tweeted to SF Support asking them to look at this post. That will bring their attention to this issue. Am confident someone from Salesforce will post if they have a response for you. – crmprogdev Jul 26 '15 at 15:20
5

Gaiser - Thank you for posting this and the repro steps. We talked through this internally and agree that we need to make some modifications to allow the behavior seen in v32

Your use case is something that I missed in design for the GA of test levels. We are already talking about a solution.

  • This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post - you can always comment on your own posts, and once you have sufficient reputation you will be able to comment on any post. – Boris Bachovski Jul 27 '15 at 22:24
  • To answer the first question, we do not have a way to support your use case in V34. The best alternative is to use the Tooling API for use cases related to test execution and code coverage. That said, we will be looking into ways to update the MD API deploy to allow for the V32 behavior. To answer the question on how to get this through to SFDC - posting and sharing through stack exchange worked – John Vogt Jul 27 '15 at 23:18
  • @JohnVogt considering this feature hasn't worked since API v32 (over a year now), I would think the documentation related to codeCoverage in the Metadata API should be updated to reflect this: developer.salesforce.com/docs/atlas.en-us.api_meta.meta/… – user2221343 Apr 4 '16 at 12:59
0

I believe this is by design. From the release notes:

When deploying to production, all tests, except those that originate from managed packages, are executed if your deployment package contains Apex classes or triggers. If your package doesn’t contain Apex components, no tests are run by default.

Since you aren't including any Apex components, the tests won't be run.

  • But he is specifying a test level and a test thus the docs say that it should be run - At least thats how I read it before deleting my answer – Eric Jul 26 '15 at 15:48
  • Hello @Daniel Hoechst . Unfortunately the excerpt from the doc you quoted is not applicable to the situation described in my question. The problem manifests itself when deploying (with test run) in Sandbox or Developer Org, not Production. As Eric has already mentioned - I am explicitly specifying tests to run. Tests are actually being run, but coverage is NOT reported. – gaiser Jul 26 '15 at 19:33
  • Ah, I see. I thought you were saying the tests weren't running. That is odd. I know two developers on the metadata API team. Just pinged them both. – Daniel Hoechst Jul 27 '15 at 3:22
  • Hello @Daniel Hoechst. Thank you for the ping. Yes, I have no problems with tests being run. The problem is with coverage report not being returned at all (in cases I described). This only happens in API v34.0 (did not test 33.0, but 32.0 and below return coverage report as expected/documented). – gaiser Jul 27 '15 at 7:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.