How to handle installed Managed Packages failing tests?

Question

I have several AppExchange apps installed that are failing tests. What is the recommended approach to resolving this? Are tests run when a new package is installed? Is there a way to determine what is actually failing besides the name of the method and something like "expected true, actually false"? The test results rarely show enough information to be useful.

What I'm doing atm, is checking for updates for the packages, and then writing the company with the failing test results.

Any feedback would be appreciated.

Answer 1

Managed package tests are only run in the following scenarios:

• On package upload (only applicable to the package author)
• If explicitly selected to run (changing the namespace on the run dialog for Setup > Develop > Apex Test Execution, dev console, specific API calls to run them, like setting runTests in the metadata API deploy call).
• On metadata API deployments that have the runAllTests flag set to true (see the runAllTests description).

There have been situations in the past where managed tests run outside of these and all were deemed bugs and fixed. They do not ever run on change set deployments, although if there are managed triggers on objects your tests invoke those managed triggers will be run.

Answer 2

If you want to run all tests but the managed packages tests, you can use something like that:

public with sharing class AdministrationTools {
public static PageReference startLocalTests() {
List<List<ApexClass>> searchList = [FIND '@isTest' IN ALL FIELDS RETURNING ApexClass (Id WHERE NamespacePrefix = NULL AND Status = 'Active')];
ApexTestQueueItem[] queueItems = new List<ApexTestQueueItem>();
for(ApexClass myResults : searchList[0]) {
   queueItems.add(new ApexTestQueueItem(ApexClassId=myResults.Id));
}
insert queueItems;
PageReference newocp = new PageReference('/ui/setup/apex/ApexTestQueuePage');
       newocp.setRedirect(true);
       return newocp;
}
}

Then just you can call it via Anonymous or using an administrator Apex page like:

<apex:page controller="AdministrationTools">
    <apex:form>
        <apex:commandButton action="{!startLocalTests}" value="Start local tests" />
    </apex:form>
</apex:page>

Answer 3

Couple of things to point out regarding managed packages:

1. There is no way to get details on the code that is failing in a managed package. This is done presumably to protect the IP of the creator

2. If the tests are failing in your development org but not in production make sure that you have copied the necessary custom settings from production. It's common that managed packages depend on custom settings for their configuration. (I believe Salesforce was going to make the change that custom settings get copied from production to sandboxes when creating the sandbox, but I'm not sure in which release that was going to happen)

Answer 4

I had this problem in one of my projects. The problem is that when you test your code managed code is triggerd. Even if you don't want to. Like, you insert/update/delete smth that has trigger in managed package. That triggers it's code and for some reason it fails. Actually in my case there were governor limits in their package. I refactor my code a bit so their code don't hit the limit. Also they have nullpointer. We contacted them and they fixed that problem.