7

According to the documentation:

Test classes (classes that are annotated with @isTest) are excluded from the code coverage calculation. This exclusion applies to all test classes regardless of what they contain—test methods or utility methods used for testing

However I am currently finding that to be false (not the case).

As you can see in the included image (not all classes included in image) all my non test classes have coverage of 100% with a few stragglers dus to overzealous catch blocks. In total, for non test methods I have a grand total of 27 lines uncovered.

enter image description here

The coverage indicates that 324 lines are uncovered.

In my classes annotated with @isTest there are 297 lines across all @isTest annotated classes where the methods are not marked as test methods (Utility methods).

I have checked the code coverage from the Dev console, IntelliJ, the Setup UI and they all say the same thing 84.xx% total coverage.

I have cleared the test history, recompiled all classes, all the usual stuff.

Is the documentation wrong?

An example method in an @isTest annotated class

private static PageReference setTestPageReference(){
    PageReference pr = Page.Example_Page;
    pr.getHeaders().put('Host','https://login.salesforce.com');

    test.setCurrentPageReference(pr);
    return pr;
}

The class that contains the above method has 0/5 line covered. When I add the testmethod to the method definition, the class changes to have 0 uncovered lines and the code coverage for the entire org goes up by .01%

Basically it seems that code in a @isTest annotated class IS being included in test coverage which is a problem since I have a large test Utility class annotated with @isTest and none of the methods are marked as test methods.

So the Questions:

  1. Am I the only one observing this behavior?
  2. Should I worry about it?
  3. Any harm in adding the testmethod designation to the utility methods? (It feels wrong)
  • 1
    (1) I have observed this (2) I dont worry (3) seems "wrong" – cropredy Dec 27 '16 at 20:10
  • 4
    If you want it to report correctly just delete and recompile your test classes. Provided by Avinash on salesforce.stackexchange.com/questions/139238/… – Jesse Milburn Dec 27 '16 at 20:39
  • @JesseMilburn - Yea, already packaged though so not an option for some but will keep that in mind moving forward. It would also completely destroy the repo history lol...... – Eric Dec 27 '16 at 20:44
  • I wonder how this is not considered to be a duplicate while a similar old question has been marked so salesforce.stackexchange.com/questions/142375/… – highfive Dec 28 '16 at 4:07
  • @highfive - Probably should have been. Good find... – Eric Dec 28 '16 at 4:09
7

I've had similar conversations about this before with salesforce, because this was actually causing us issues in deployment. What we came away with was as follows.

The Documentation is Correct

According to Support, any class that has @isTest specified should be completely exempt from code coverage. It should display as 0/0 coverage.

The System is Wrong

They confirmed with R&D that someone internally has already logged a bug regarding this. There is a Known Issue that's tracking this bug.

We Should Test (For Now)

In our case, we had nearly 500 lines of code that were showing no coverage, and it was actually causing deployments to fail with a net total of 74% coverage. If you're getting deployment errors, then you need to unit test your unit tests. Otherwise, don't worry about it.

  • Thats great....Looks like I am going to have to "test my tests" as I will be adding a test library the customer can use to setup org, make calls, generate responses, etc during their test methods....Doing this will drop the coverage below 75% on packaging I fear.... – Eric Dec 27 '16 at 20:27
  • @Eric Depends on how big it is, and how close you are to the limit, of course. You can cover just as much as you need to get the coverage to upload (installation doesn't require coverage anymore), and if the security team asks about it, just say that you're working within the confines of the Known Issue, and they should be okay with that. – sfdcfox Dec 27 '16 at 20:35
  • Also, changed my templates to include the annotation when creating so hopefully that will abate the problem. – Eric Dec 27 '16 at 20:45
  • I think you can get in pretty major trouble with Salesforce if you do that sort of a++ coverage smoke test. I would advise against that. @Eric – Adrian Larson Dec 27 '16 at 21:31
  • 1
    I figured as much! Just don't want someone wandering by a few years down the road and thinking they can follow your lead there. :P – Adrian Larson Dec 27 '16 at 21:33
6

Am I the only one observing this behavior?

You are not. There is at least one relevant Known Issue: Non-test methods in a test class are counted as part of Apex code Coverage. It seems that if you create a class, then add the @IsTest definition after the fact, that can contribute to the issue.

Should I not worry about it?

I believe this bug does not affect deployments to production, which is generally the most critical time to have accurate code coverage numbers. I believe it would be unlikely to follow the reproduction steps listed in the Known Issue I linked to, since you would have to deploy a class as live and then convert it to the @IsTest annotation.

Any harm in adding the testmethod designation to the utility methods?

Security scanners will certainly frown upon test methods with no assertions. I'd avoid it.

  • You are correct, isTest added after class creation. Wonder if it will be an issue with packaging since tests are run before packaging or if it will follow the same pattern as if deploying and ignore the test classes. – Eric Dec 27 '16 at 20:25
  • 2
    Really appreciate your answer but can only accept one. Since sfdcfox directly addressed the issue around deployment and having to test your tests I am inclined to accept that answer. I wish I could accept both but alas all I can do is up vote. Again, thank you! – Eric Dec 27 '16 at 20:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.