What to Javascript-test in Lightning components and what not

Question

Let's be honest. As an Apex developer used to provide near-100% test coverage for Apex the same should be true for Lightning components. As we move more code from the server to the client more test need to be written in Javascript. Especially as JS is more error-prone than Apex.

When I listened to Keir Bowden's Dreamforce 2016 talk about Lightning testing with Jasmine and read the docs for the current Lightning Testing Service (Pilot) (usable by everybody) many questions arise:

1. Ignore Server actions: As (according to the docs) I am not supposed to invoke Apex actions from JS test, what should I then test?
2. Test plumbing or only business logic: Lightning controllers have a lot of code even if they do near to nothing. How do I test all this error prone action calling, attribute reading and setting? And should I really do it with even more test code?
3. What to mock?: How much should I use stubbing/mocking/spying to simulate server responses?
4. Coverage: When can I be sure to have semantically covered my code well?
5. UI testing: Should I use UI-testing (Selenium, Provar) as a third type for real end-to-end testing?
6. Integration aspects: If not how can I be sure that JS indeed is triggering the right things in Apex?

and finally...

7. and finally what happened to the Lightning Component Builder - Test UI promised for Summer '17

So the overarching question is:

What to test in Apex and what in Jasmine and how to ensure to cover everything well?

Answer 1

Some great questions here, and while I can't answer all of them, let me (a) answer what I can, and (b) make it clear that we're listening. :-)

First, though, keep in mind that the Lightning Testing Service as it's available today is a PILOT release. It's incomplete, we know there are known and unknown unknowns, etc. Feedback is key to shaping the direction that the LTS takes going forward. So keep it up!

A couple of answers.

(1.) We think that if your tests depend on actual responses to full Apex requests, they'll be "flappy", that is, bounce between success and failure, because you're testing too much in one test. Mock your Apex requests/responses, and test for the various possible responses. Test your Apex on the Apex side. (You have to do that anyway, for test coverage purposes, right?)

(4.) Coverage. Really good question. We don't have an answer for you today, beyond what's provided in e.g. Jasmine itself. It's going to be a while before we give you Apex-style coverage reports. Safe harbor, etc.

(7.) Lightning Component Builder wasn't a test UI, but a replacement for building components in the Developer Console. It was going to include new UI for tests, but the actual testing was going to be a separate feature. (Or so I've heard.)

In any event, the Lightning Component Builder as a specific tool is currently on hold. (If you're on the partner community, a few details here: https://partners.salesforce.com/0D53A000038fWzJ) We've got a huge overall tooling initiative going in the form of Salesforce DX, and will have a lot more to say about all of that throughout the year, especially at TrailheaDX and Dreamforce.

TrailheaDX is just two weeks away. If you're joining us there, please say hello to all the team members in attendance. We’re looking forward to meeting you, telling you more, and hearing what more we can do to make testing productive. (We know the list is long!)

Answer 2

As per comments here are some examples to your queries:

I think the key thing to test is that your js behaves correctly on the client side as opposed to if a call to apex works. For example, if you are developing a component that drives some related component behaviour such as dependent visibility, write tests that evaluate that the dependency condition gets set to true/false correctly.

Markup //Sample component

    <aura:handler name="change" value="{!v.value}" action="{!c.handleValueChange}"/> 

    <c:controllingComponent value="{!v.value}" />

    <aura:if isTrue="{!v.showDepdendent}">
         <c:myDependentComponent />
    </aura:if />
</aura:component>

Js

handleValueChange: function (component, event, helper) {
    //evaluate your showDependency condition
    var condition = ....

    component.set("v.showDepdendent", condition);
}

The benefit I am gaining through doing the extra work (instead of evaluating my condition in the aura:if isTrue condition) is that I can now easily test the behaviour by setting a value to "v.value" in my test and then assert that the "v.showDepdendent" attribute evaluates to my expected result.

With the testing framework I spoke about I would just create the below test:

Markup

<aura:application extends="c:greased_TestCommon">

    <aura:handler name="init" value="{!this}" action="{!c.doInit}"/>
    <c:SampleComponent aura:id="sampleCmp" />

</aura:application>

Js

doInit: function (component, event, helper) {

    var sampleCmp = component.find("SampleComponent");

    var startTests = test.start({
        focused: sampleCmp
    })

    //////////// START OF TESTS ////////////
    startTests

        .then(test.wait(function () {
            sampleCmp.set("v.value", "TestValue");
        }))
        .then(test.assertEquals(true, "v.showDepdendent", "The dependent visibility should be true"))
    //////////// END OF TESTS ////////////
        // always include these fns to handle the end of the test
        .then(test.pass).catch(test.fail);

Sorry for the long example, but basically this gives you the ability to test UI behaviour alongside testing that your JS behaves correctly.

I know I didn't cover all scenarios but thought this was worth sharing.

Answer 3

With Lightning, we have transitioned to a rich-client model. Lightning components are able to execute business logic on the client, contact the server if needed and transform the UI being displayed using code authored in javascript.

When thinking about what/how to test, I personally start by referring to an approach which has become fairly well-known as 'Test Automation Pyramid' ( https://martinfowler.com/bliki/TestPyramid.html ).

In this context, apex tests are great to test the functionality (e.g. server side controllers, helper utils etc.) authored in apex. Apex tests range from being in the unit (focussed on validating business logic in a particular method) to the service (focussed on invoking a service and validating integration between various classes, database etc.) bucket of the pyramid.

Similarly, javascript tests written using a framework like jasmine are great to test client-side functionality. Just like apex-tests, javascript test frameworks can be used to unit-test logic in a javascript library, and also to validate integration between various client-side features/services and the browser.

Samples provided with the pilot package of "Lightning Testing Service" are focussed on showcasing how javascript test frameworks can be used to functionally tests the behavior and integration of Lightning Components with each other (event handling, attribute access, method invocation etc.) and with the framework (Aura, LockerService, LDS etc.). In other words, you can call such tests, "Lightning Component Tests" which validate functionality of the components.

A smaller set of end-to-end UI tests have their place on the pyramid. Internally at Salesforce, such tests leverage web-driver and Page Object Model to interact with Lightning Experience.

Wholistic approach to development on the platform with SFDX may tweak (as compared to 'Lightning Component Builder' slide you included) interface to how Lightning Components are developed/tested, but the goal is to ensure capabilities of LTS (e.g. Testing patterns, Continuous integration etc.) remain the same (or be enhanced by embracing industry standards and open-source technologies).