5

Let's say I have two classes
- AccountController - contains logic
- TestCoverage - contains tests for AccountController

With API v32.0 it was (and still is) possible to deploy .zip which contains nothing but package.xml, request a test run, and get coverage result (covered/uncovered lines, and coverage warnings).

With API v34.0 Metadata deploy (with test run) does not return coverage for classes/triggers not included in the package.zip

This can be easily reproduced with workbench and a developer Org.

I will be using two zip files for tests below.
zip files structure is as follows
components-has-class.zip:
- unpackaged\
--- classes\
------ AccountController.cls
------ AccountController.cls-meta.xml
------ TestCoverage.cls
------ TestCoverage.cls-meta.xml
--- package.xml

components-no-class.zip:
- unpackaged\
--- classes\
--- package.xml

Step 1. Test data Preparation
Take components-has-class.zip and deploy it into your test Org how you do that does not matter. Important part is that these two classes must be in the Org for the next step.

Step 2. API 32.0 - where everything works as expected
login to workbench with you dev Org credential and select API 32.0 (version is important)

deploy components-no-class.zip with following settings:

  • allowMissingFiles: true
  • checkOnly: true
  • runAllTests: false
  • runTests: TestCoverage
    Leave other settings as in their default values

When deployment completes - observe that workbench returned coverage details for AccountController.cls as well as code coverage warnings.

Step 3. API 34.0 - coverage is not returned
login to workbench with you dev Org credential and select API 34.0 (version is important)

deploy components-no-class.zip with following settings:

  • allowMissingFiles: true
  • checkOnly: true
  • testLevel: RunSpecifiedTests
  • runTests: TestCoverage
    Leave other settings as in their default values

When deployment completes - observe that workbench did NOT return coverage details for AccountController.cls

Note - this is NOT a workbench issue - the same behaviour can be reproduced if you write your own deployment test e.g. in java using example from the documentation.

Workarounds
- either include all Class/Trigger files you want to get the coverage for in package.zip
- or use tooling API to run tests

Unfortunately both workarounds involve different degree of inconvenience, compared to old deploy/test method.

Does anyone have any suggestions of what else can be done to make good old deploy/test/get-coverage method working again with API v34.0? I can not get back to v32 because my tools have already been updated to v34.0 and metadata deployment no longer works with v32.0 because v34.0 changed signature of some API calls.
Or how to report it to SFDC? (I do not have premier support access)

Improve this question
7
  • Even without premier support, you can always file a support case with Salesforce. You just don't go to the front of the line like a premier support customer does. Click on Help > Support to get to where you contact support to file a case.
    – crmprogdev
    Jul 25, 2015 at 20:44
  • Hello @crmprogdev. Thanks for the suggestion. Unfortunately I only have access to Developer Org(s) and these days it does not seem to be possible to open a case from a Development Org account. Confirmation: success.salesforce.com/answers?id=90630000000h3ImAAI
    – gaiser
    Jul 25, 2015 at 21:48
  • A guess: did you try to play with testLevel param? It is new.
    – Ashwani
    Jul 26, 2015 at 7:13
  • Hello @Ashwani. Not sure what you mean by "play with testLevel param". The only value of testLevel which allows setting specific test(s) to run is RunSpecifiedTests. All other settings of testLevel result in: INVALID_OPERATION: runTests can only be used with a testLevel of RunSpecifiedTests. This is in line with the documentation.
    – gaiser
    Jul 26, 2015 at 9:52
  • @gaiser. I have tweeted to SF Support asking them to look at this post. That will bring their attention to this issue. Am confident someone from Salesforce will post if they have a response for you.
    – crmprogdev
    Jul 26, 2015 at 15:20

2 Answers 2

Reset to default
5

Gaiser - Thank you for posting this and the repro steps. We talked through this internally and agree that we need to make some modifications to allow the behavior seen in v32

Your use case is something that I missed in design for the GA of test levels. We are already talking about a solution.

Improve this answer
3
  • This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post - you can always comment on your own posts, and once you have sufficient reputation you will be able to comment on any post.
    – Boris Bachovski
    Jul 27, 2015 at 22:24
  • To answer the first question, we do not have a way to support your use case in V34. The best alternative is to use the Tooling API for use cases related to test execution and code coverage. That said, we will be looking into ways to update the MD API deploy to allow for the V32 behavior. To answer the question on how to get this through to SFDC - posting and sharing through stack exchange worked
    – John Vogt
    Jul 27, 2015 at 23:18
  • @JohnVogt considering this feature hasn't worked since API v32 (over a year now), I would think the documentation related to codeCoverage in the Metadata API should be updated to reflect this: developer.salesforce.com/docs/atlas.en-us.api_meta.meta/…
    – user2221343
    Apr 4, 2016 at 12:59
0

I believe this is by design. From the release notes:

When deploying to production, all tests, except those that originate from managed packages, are executed if your deployment package contains Apex classes or triggers. If your package doesn’t contain Apex components, no tests are run by default.

Since you aren't including any Apex components, the tests won't be run.

Improve this answer
4
  • But he is specifying a test level and a test thus the docs say that it should be run - At least thats how I read it before deleting my answer
    – Eric
    Jul 26, 2015 at 15:48
  • Hello @Daniel Hoechst . Unfortunately the excerpt from the doc you quoted is not applicable to the situation described in my question. The problem manifests itself when deploying (with test run) in Sandbox or Developer Org, not Production. As Eric has already mentioned - I am explicitly specifying tests to run. Tests are actually being run, but coverage is NOT reported.
    – gaiser
    Jul 26, 2015 at 19:33
  • Ah, I see. I thought you were saying the tests weren't running. That is odd. I know two developers on the metadata API team. Just pinged them both.
    – Daniel Hoechst
    Jul 27, 2015 at 3:22
  • Hello @Daniel Hoechst. Thank you for the ping. Yes, I have no problems with tests being run. The problem is with coverage report not being returned at all (in cases I described). This only happens in API v34.0 (did not test 33.0, but 32.0 and below return coverage report as expected/documented).
    – gaiser
    Jul 27, 2015 at 7:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .