Pragmatic Front-End Testing Strategies (2)

This article is the second of a series of three articles.

Part 1: Necessity of Pragmatic Front-End Testing Strategies
Part 2: Testing Visual Components Using Storybook
Part 3: Testing Logic in State Management Using Cypress

In the first part of this series I discussed the importance of test automation and testing strategies, and why it is difficult to automate visual component testing. While it is not impossible to automate the visual quality assurance process, with the tools that are available to us today, the effort-to-return ratio is not particularly recommendable. Since this series deals with “pragmatic” testing strategies, let’s take a step back and try something else. Meaning, we’ll visually verify the results with our “eyes,” while automating the process of preparing the results to be verified as much as possible. Storybook, the tool I briefly introduced last week, is the most effective of the kind.

Like I mentioned in part one, Storybook is more of a UI development environment and playground for UI components than a testing tool. The main goal of Storybook is to enable developers to develop the UI component independently outside of the application. If you think about it, since the purpose of most of our testing tools is to help verify that “the module or a function can run appropriately in an independent environment,” Storybook can be considered to be a test as well.

(Every code I wrote for this series can be found in my Github Repository. While the article will include all of codes that I think are critical for understanding, you can check out the repository if you are curious about the entire code.)

Initializing Storybook

Initially, Storybook was designed only to test React codes, but now it supports multiple frameworks including React Native, Vue, Angular, Ember, and Riot. It can, of course, be used to manipulate the DOM directly without the framework, and the list of frameworks supported can be found on the app folder in the repository.

For each supporting framework, Storybook provides a different npm module, so in order to use Storybook correctly, you first have to download the appropriate npm module for the framework. For example, since this series uses React to build the Todo application, I had to install the React version of the Storybook. In order to facilitate this process, Storybook provides the CLI tool, and you can use the CLI to type in the following npx command to start your Storybook.

(Storybook v5.0.5 was used to write this article.)

npx -p @storybook/cli sb init

Running the command in the Command Line will read through the package.json for dependencies and determine which framework you are using to automatically install the appropriate version of the Storybook. Not only that, the command also installs number of boilerplates with the Storybook, so you can start using Storybook without much configuration.

If you open your project folder, you will now be able to see that two new folders .storybook and stories have been created. The .storybook contains config files to run the Storybook, and the stories folder is where you will actually write your code to register the components. Also, the package.json has been automatically updated to include storybook and build-storybook scripts. I’ll explain build-storybook later, and for now, let’s kick things off by running the storybook script.

npm run storybook

Running the command will create a local web server at the port number 9009, and a browser will automatically launch and display the current page(localhost:9009). That’s it! We took care of the entire configuration just with two lines of Command Line codes. If you wish to install and configure the Storybook without the help of CLI, help yourself with the official documentation.

Writing Your Story

Storybook refers to its test cases as “stories.” Like how each test case validates one input value for a single module, story is also one particular state of a single component. We’ll get to chunking stories later, but for now, let’s talk about registering the most basic component: <Header>

First, if you open the stories/index.js file, you will notice that the CLI tool has already registered some examples for the Button component. Let’s get rid of that, and write our own code like the following.

import React from 'react';

import {storiesOf} from '@storybook/react';
import {Header} from '../components/Header';
import '../components/App.css';

const stories = storiesOf('TodoApp', module);

stories.add('Header', () => (
  <div className="todoapp">
    <Header addTodo={() => {}} />
  </div>
));

The storiesOf function returns an object that can register and manage multiple stories. The first input acts like a category name, and can be used to group multiples stories with the same category. module, the second input, is required in order for the Storybook to use Hot Module Replacement to internally refresh the page without actually having to refresh, so it should always be passed.

You can add stories using the add method of the object returned by storiesOf function. The first input is the name of the story, and the second input is the function that returns a React element to render the component. In this example, the Header component must be below the DOM element that has the todoapp class in order for the CSS to function appropriately, so I added the <div className = “todoapp”> to the top node. Also, since this is not testing some action, I cheat by adding an empty addTodo function so no error occurs.

Saving this code, you should be able to see something like this.

Writing Story According to the Single Component State

As for the Header component, the story can be written and registered simply because the state does not change with respect to the props. However, for components that change states with respect to the props, it is better to register different stories for each states of the component. For example, the TodoItem component has three distinct states of “Normal”, “Completed”, and “Editing” aside from the content, so you have to register the three states as shown below.

stories.add('TodoItem - Normal', () => (
  <div className="todoapp">
    <ul className="todo-list">
      <TodoItem
        id={1}
        text="Have Breakfast"
        completed={false}
        editing={false}
      />
    </ul>
  </div>
));

stories.add('TodoItem - Completed', () => (
  <div className="todoapp">
    <ul className="todo-list">
      <TodoItem
        id={1}
        text="Have Breakfast"
        completed={true}
        editing={false}
      />
    </ul>
  </div>
));

stories.add('TodoItem - Editing', () => (
  <div className="todoapp">
    <ul className="todo-list">
      <TodoItem
        id={1}
        text="Have Breakfast"
        completed={false}
        editing={true}
      />
    </ul>
  </div>
));

Thing to keep in mind here is that, in order to properly display the TodoItem, you need the parent element <ul className = “todo-list”>. When this is all done, you will be able to see that each state of the TodoItem has been added as stories.

The Issues with Single Component Stories

So far, the stories we’ve written have been single component stories, and did not have any child component. Generally, when people think about a component, they think of small single components like “list-item”, “button”, and “input box.” However, real applications are combination of such smaller components, and there are complex components that encompass multiple components. Writing stories only about the single components and not writing about complex components is basically hoping the application works fine just by running unit tests without running a single integration test.

Considering the qualities of a good test discussed in the first part of this series, this raises the following issues.

1. Cannot Validate the Combination of Components of the Actual Application

To state the obvious, just because a single component is displayed without visually noticeable flaws, does not mean that the entire application will also be displayed without visually noticeable flaws. Especially, UI based on HTML/CSS is affected by numerous variables including the parent-child relationship between the DOM elements, CSS selectors, and z-index. In order to truly validate that the entire program works as intended, you need to check that all components are in the right order and that they are not unintentionally affecting other components.

Furthermore, stories written with too narrow a scope are difficult to compare visually. Comparing each button individually to a product design that shows an array of buttons would not only be tedious but also difficult.

2. Manipulating the Implementation Detail of the Parent Component Can Break the Application

Let’s recall back to how we had to add <div className = “todoapp”> and <ul className = “todo-list”> to the TodoItem so that the single component story can be displayed appropriately. To be accurate, we were acting in the scope of the TodoItem’s parent component, and we simply created a mock of the parent’s implementation detail. In this case, even if there is no change in the design, if the implementation detail of the parent is changed due to refactoring, the story will not be displayed as intended. In other words, every time the implementation detail of the parent changes, the story must be changed as well.

Furthermore, since we are directly injecting the component’s prop with the value ourselves, if the corresponding component’s prop interface is changed, we have to change the stories as well. In actuality, no matter which props interface of whatever component is changed, it basically means that, from its parent component’s perspective, change to the implementation detail. Considering the fact that the smaller you chunk down your units, the more affected parent component you end up with, chunking down units smaller and smaller translates to more maintenance cost for your stories.

The Issue with Complex Component Stories

Then, what about the complete opposite of writing the story only using the root component that encompasses the entire application? In this case, following issues can occur.

1. Difficulty of Analyzing Individual Units of Each Components

Say there are three components with three states each. In the completely integrated form, it can be said that there are maximum of twenty-seven (333) states. In this case, nine stories would suffice to register and test for a single component, but the integrated form requires much more cases. Also, since the entire UI will be displayed even if you’re testing a miniscule portion of the code, it can be difficult to visually distinguish the testing target.

2. Difficulty of Providing Input Values for Components

As the complexity of the component goes up, so does the complexity of the inputs. For a single component, you can get away with providing the test with three to four inputs, but once the number of components hit five, you would have to provide the test with around twenty input values. In this case, even if the test only requires two to three inputs, you still have to provide the rest just for the UI. Especially if you’re working with something like the Redux store, where you have to use a separate state management tool, you would have to create the state object like the store just to inject the input values.

3. Increased Dependency on the External Environment

Components not only display visual elements of the application but also create side effects by interacting with the external elements of the environment. Handling a reroute of the changed URL in the browser and retrieving data from the server by making a request to the API when the component is mounted are such examples. When testing a single component, the parts of the application that create the side effects can be disregarded to emphasize the components that have to do with visual aspects of the application. However, for complex components, it is difficult to distinguish a single child component during the test, so it allows for the side effects of the child components to run free, without restrictions.

Deciding on Story Units

As you can see, polarized testing methods have their own problems. Therefore, it is pivotal to slice up the application into appropriate units of stories. I, personally, prefer to use page-level units, while using a separate group of components to register components that fall in content categorization in layouts. It can also be a good idea to distinguish and group components that have nothing to do with the page’s content (i.e. layers) as well. Grouping the components like so can help ameliorate the issues with registering single component stories.

Complex stories, on the other hand, require a different approach. As for the first issue, difficulty of analyzing individual units of each components, can be resolved with the Knobs Addon, which validates multiple states from a single story. The second issue, the increased number of inputs, is rather tricky to deal with. However, the problem can be assuaged by creating a common form of input that represents multiple inputs at once. Also, creating custom addons with mocks of systems like redux store can simplify the store's input injection process.

Finally, the last issue can be solved by simply structuring the code well. By separating the codes in charge of visual elements from externally dependent codes, and managing them at the top of the component tree, the increased dependency can be relatively managed. Furthermore, it is also a good practice to maintain the pure component by delegating the side effect management to layers like redux-thunk and redux-saga.

Back to the Todo App

Most of what I said so far probably sounds like gibberish. Let’s get back our Todo App and apply what we just discussed to a real project.

Separating the Component’s Visual Element

To be honest, the Todo application is a small project compared to other projects, so I could theoretically register the root component as a story. However, since the root component is also in charge of routing, creation and injection of stores, and initial data load, it is best to distinguish these codes into separate groups. In the example code, the externally dependent codes were handled in src/index.js, and the App component only deals with the visual element.

// components/App.js

import React from 'react';
import Main from './Main';
import Header from './Header';
import Footer from './Footer';
import './App.css';

export default class App extends React.Component {
  render() {
    return (
      <div className="todoapp">
        <Header />
        <Main />
        <Footer />
      </div>
    );
  }
}

Store Mocking

Now, only thing that’s left for us to do is to write the story about App component. But wait! How are we going to feed the components inputs? The App component does not have a separate prop, and only knows how to render the children components. To make things worse, all of the children components require the inputs to be injected with the Redux store. To put it simply, we need the store to provide the component with inputs. However, it can be difficult to forge the inputs into forms that we want because store can only be changed through actions with Reducers.

The most realistic thing we could do is creating a mock object instead of using the real store. Luckily, the store’s API is structured very intuitively, so we can just create the mock object like the following.

function createMockStore(initialState) {
  return {
    dispatch() {},
    subscribe() {},
    getState() {
      return initialState;
    },
  };
}

Currently, this store only needs to know how to provide the component with the initial input values, and does not need to dynamically change states, so we do not need to implement methods like dispatch and subscribe. We just need to make sure that the getState method can return the initial value correctly.

Writing the Story

Now that we can provide it input values, we can finally start writing the story. Just one thing stands in our way (again,) and it is that one of the children components require an input value that is not a store. The black duck in the family is the data from react-router. Components like Footer and Main all retrieve the current page’s parameter information through the withRouter. Therefore, every time the root component is rendered, it must be wrapped with a component that provides information about the router. However, if you decide to use the BrowserRouter embedded in the application itself, it becomes bound to the browser’s URL. In order to control the input value, you would have to use a different kind of router or create a mock router yourself. For this example, we will use the StaticRouter, which is frequently used in the Back-End development environment.

Now, let’s actually start writing the story. First, establish the mock store with createMockStore and render Provider component and StaticRouter component together. For now, let’s keep things simple and provide it with one Todo task.

stories.add('App', () => {
  const store = createMockStore({
    todos: [
      {
        id: 1,
        text: 'Have Breakfast',
        completed: false
      }
    ]
  });

  return (
    <Provider store={store}>
      <StaticRouter location="/" context={{}}>
        <Route path="/:nowShowing?" component={App} />
      </StaticRouter>
    </Provider>
  );
});

You should be able to see that the entire application is displayed.

Establishing Input Values

Since we registered the entire application in one story, we can easily see multiple states of the applications at once. For example, when we were using single component, we had to register different stories for each state of the application, but now, we can display multiple states of Todo-lists at once.

const store = createMockStore({
  todos: [
    {
      id: 1,
      text: 'Have Breakfast',
      completed: false
    },
    {
      id: 2,
      text: 'Have Lunch',
      completed: true
    },
    {
      id: 3,
      text: 'Have Dinner',
      completed: false
    }
  ],
  editing: 3
});

Passing the input values like this should display Normal (1), Completed (2), and Editing (3), respectively.

Using the Knobs Addon

While we can see multiple states of the Todo-list in one glance, there is still much to do. We now have to consider when the “All”, “Active”, and “Completed” buttons at the bottom are activated, and when the “Select All” button at the top is activated. Registering multiple stories for such independent component states can be difficult to distinguish the change and can be incredibly costly. Luckily for us, there’s an addon for that.

Addons are iconic features of the Storybook. They are mainly used to interact with components registered onto the stories, and can be accessed from the “Panel” outside of the “Preview” area where the stories are presented. More information regarding addons and using addons can be found on the official website, and the Addon Gallery has a great collection of useful addons you can use.

The Knobs addon helps developers by allowing you to dynamically change the input values you feed the components by adding the input control panel, and it can be used to verify different states of the of a single story. Since this article will be example-based, refer to the Github Repository for in depth information on using Knobs Addon.

Installing and Configuring Knobs Addon

Installing and configuring the Knobs addon is sufficiently easy. First install the addon using the npm command.

npm install @storybook/addon-knobs --save-dev

Next, create the addon.js file in the ./stories/ folder, and enter the following.

import '@storybook/addon-knobs/register';

Then, open the src/stories/index.js file where your story should be registered, and add the following piece of code at the top of the code.

import { withKnobs, radios } from '@storybook/addon-knobs';

Finally, when you have added your decorators to the objects created with storiesOf, you’re all good to go.

const stories = storiesOf('Todo-App', module)
  .addDecorator(withKnobs);

Controlling the Router

Now, I’m going to control the router’s input value to change the state of the buttons at the bottom. Here, I used the radios function in order to use the radio buttons. The first input should be the label, and the second option should be the list of options for the button. The last input is the default.

stories.add(App, () => {
  // ... Identical to the previous code

  const location = radios('Filter', {
    All: '/All',
    Active: '/Active',
    Completed: '/Completed'
  }, '/All');

  return (
    <Provider store={store}>
      <StaticRouter location={location}>
        <Route path="/:nowShowing" component={App} />
      </StaticRouter>
    </Provider>
  );
});

For StaticRouter, you can randomly define the router’s URL using the location value. Instead of giving it the string value to define the location, if you feed it the value the radios function is going to return, you have successfully connected to the Knobs addon. Now, when you save the code, the Filter item will have appeared in the Knobs panel. Each push of the radio button will be able to change the state of the button at the Footer.

Controlling the Store

Now, let’s check that our “Select All” checkbox works as we intended it. The “Select All” checkbox can only be checked when the states of every item have been changed to completed, so in order to change the state of “Select All,” we have to change the state of the store. However, the Provider component does not allow for the reference of the component’s initially assigned store object to be dynamically changed, so we cannot call the createMockStore to change the state of the store. Also, since the state change with stores generally depend on the dispatch, to declare the state of a store out of no-where, we would need to write a separate method like setState.

In other words, in order to connect the store and the Knobs addon, we need to manipulate only the store’s state, not create a new store, every time the registered function executes. In this case, if you personally create an addon to use as a decorator, you can simplify the series of laborious tasks. While this task is not difficult, it may be difficult to explain the process of creating a custom addon in this article without divulging too much from the main topic of the article. If you are interested, I encourage you to check out Storybook’s tutorial documents and store addon’s source code.

Here, I will use the addon I previously created to connect the Knobs addon to the store. The following is the final product.

import React from 'react';
import {storiesOf} from '@storybook/react';
import {withKnobs, radios, boolean} from '@storybook/addon-knobs';
import {StaticRouter, Route} from 'react-router-dom';
import {withStore} from './addons/store';
import App from '../components/App';

const stories = storiesOf('Todo-App', module)
  .addDecorator(withKnobs)
  .addDecorator(withStore);

stories.add(
  'App',
  () => {
    const options = {
      All: '/All',
      Active: '/Active',
      Completed: '/Completed'
    };

    const location = radios('Filter', options, options.All);

    return (
      <StaticRouter location={location} context={{}}>
        <Route path="/:nowShowing" component={App} />
      </StaticRouter>
    );
  },
  {
    state: () => {
      const isAllCompleted = boolean('Complete All', false);
      const editing = boolean('Editing', false) ? 3 : null;

      return {
        todos: [
          {
            id: 1,
            text: 'Have Breakfast',
            completed: isAllCompleted || false
          },
          {
            id: 2,
            text: 'Have Lunch',
            completed: isAllCompleted || true
          },
          {
            id: 3,
            text: 'Have Dinner',
            completed: isAllCompleted || false
          }
        ],
        editing
      };
    }
  }
);

In the code, you can see that the part where we originally created a mock store to provide the store through the Provider is now gone, and the state has been inserted as the third input of the add function. The third input of the add function is used to define the value the decorator registered using addDecorator retrieves. The withStore decorator takes the value of the state key, and internally updates the store’s state. Also, since every time the value is changed at the Knobs panel, the third input value must be updated as well, the value of state is passed on as a function.

Toggle feature can also easily be added using the boolean function provided by the addon-knobs module. In the code above, I have added a feature to toggle the items currently being edited, along with the “Select All” state. By running the code, you will be able to see that Complete All and Editing labels have been added to the Knobs panel.

Sharing Your Storybook

Stories like the one we wrote can be made into a static file and be published on the web server. Let’s run the build-storybook npm script I glossed over at the beginning. You will be able to see that the storybook-static folder has been created at the root of your project folder. If you publish this folder through static servers like Github Pages, not only developers, but designers, producers, and coworkers in other departments can also view all stories related to this project. Sharing the page through the server can be extremely useful when used as tools for design QA or documentation content.

(Using Storybooks as communication tools is explained in great depth in The delightful storybook workflow)

Stories written for the purpose of this article are also published on my Github, so feel free to check it out.

End of Part 2

As you can see, all visual scenarios for the Todo application can be visually tested using a single story. While it is true that compared to the single component story where you only had to enter the props, it is much more complicated, judging by the final code, the intent of the code is clear without unnecessary content. Also, given the fact that the story for this code only ranges 40 lines, it is much shorter than registering multiple single components.

True, you probably will not be able to validate every visual scenarios of a more complex application using one story. For such cases, it would require the tester to be intentional in deciding the story unit chunks given the application’s size and characteristics. I hope that you can refer to the earlier parts of the article to successfully strategize your testing plans with consideration of pros and cons of single components versus complex components.

Going back to the Storybook page we created, you probably noticed that while it displays the visual elements of the application, parts that handle the user inputs do not function. Lest you forget that Storybook is a testing tool used only for visual elements. Attempting to test functional features of the application would be out of purpose.

As I mentioned in Part 1, the reason we separate visual testing from the rest is that visual testing is difficult to automate. The remaining functionalities can be automated using different tools. In Part 3, we will discuss using Cypress to test functional elements.

Appendix: Visual Regression Test

While I have not included much information about visual regression testing in this article for the sake of length, Storybook offers most of the modern tools for automated visual regression tests. The creators of Storybook also explains the automating visual tests in a separate document. Such tools include Perci, Applitools, and Chromatic, so for those who are interested, I have attached corresponding links below.

• Percy: Visual testing for Storybook for React
• Applitools: Visual Testing With Storybook
• Chromatic: Storybook and Chromatic tutorial