Testing

In the recent years, JavaScript has been widening the scope of use through rapid development, and the application standards put forth by the front-end environment are growing in complexity every day. Furthermore, testing environment also went through spectacular changes in disproportionate amount of time, and with the rise of Node.js, numerous testing tools have innundated the market and continue to undergo massive evolutions. Front end tests must consider different environments of users (browser, device, operating system, and etc,) so there are plenty of variables to take into account. For this reason, it is critical that developers understand the different tools and environments related to JavaScript testing, and a veteran expertise is required to design strategical tests customized to different projects.

This document will present the variety of tools used for JavaScript tests, and will explain which tools should be used for different situations.

(Designing test strategies and writing effective test codes for different projects will be discussed in another guide in greater detail.)

Table of Contents

• Defining Test and Test Types

  • What is a test?
  • Unit Testing
  • Integration Testing
  • E2E Testing

• JavaScript Testing Tools

  • Test Runner
  • Testing Framework
  • Assertion Library
  • Test Double Library

• JavaScript Testing Environments

  • Browser
  • Node.js
  • Browser vs. Node.js

• Jasmine

  • Defining Test Specs
  • Grouping Test Specs
  • Resetting Tests
  • Spy Class
  • Clock Class
  • Asynchronous Test

• Karma

  • Install and Usage
  • Measuring Code Coverage
  • Cross Browsing Test

• Jest

  • Easy Install and Usage
  • Easy Code Coverage Measurement
  • Built-In jsdom
  • Snapshot Testing
  • Test File Filtering
  • Sandbox Parallel Testing

• E2E Testing Tools

  • Selenium Webdriver
  • Cypress

• Installing and Using Cypress

  • Install and Set Up
  • Running on Interactive Mode
  • Running in the Background

• Writing Cypress Codes

  • Creating and Running Test Files
  • Writing Test Codes
• Afterword

Defining Test and Test Types

What is a test?

The word ‘test’ has different meanings in different fields due to the its universality. To define the word test from the software perspective, it is “an act to verify that the application functions appropriately under given requirements.” It is considered to be an independent field of development separate from programming, and in many large corporations, specialized testers from quality assurance (QA) organizations are hired to carry out such tasks.

However, as Agile Methodology and Test Driven Development , emphasized in Extreme programming, gained public traction after the turn of the millennium, testing grew to be accepted more and more as a part of the development cycle. In this document, test will refer only to automated tests written in codes. If the developer designs and writes the test code himself, it allows for more active improvement of the quality of the code by refactoring, and with this improvement in quality in the development stage, resources can be saved from unnecessarily communicating with the outside testers.

(This guide deals only with “automated tests that are written by the developer,” and does not cover anything beyond this scope)

There are many known types of tests, and developers tend to mainly categorize them based on the coverage. According to scope, the tests can be broken down into unit test, integration test, and E2E (End to End) test. Many often mistakenly think that any test written by the author for the program is a unit test, but since different tests have different pros and cons, it is important to differentiate various tests and be able to choose the appropriate test for each situation. In order to highlight this importance, the following sections will explain each test methods in detail.

Unit Testing

Unit tests refer to tests done on sections of the application (usually modules) independently in a separate environment. Because the test is conducted independently from the application, it is capable of testing very intricate details of a particular module or a class, and can be executed significantly faster than testing over a wider coverage. However, because unit tests require a mock object in order to control the dependency of modules, unit tests cannot verify whether the code functions harmoniously within the application as a whole. Also, since the application can be extremely sensitive even to subtlest changes to the API, unit tests are prone to error even with tiny refactoring.

Integration Testing

Integration test covers a slightly larger scope than the unit test, and it tests for the connectivity within aggregates of two or more modules. Because the purpose of integration tests is to examine the mutual functionality within a collection of modules, mock objects are not required as much, and can detect intermodular errors. Also, due to the fact that integration test is only affected by a relatively large change in the API, it is less prone to errors during refactoring of codes than the unit test. However, if a single module contains a heavy algorithm or complicated conditional statement, integration testing becomes more complicated than the unit testing, and also becomes more prone to repetitive testing.

E2E Testing

Both unit testing and integration testing are done from the perspective of developers who know the entire application inside and out to take a selection of the application to be tested. E2E test, on the other hand, is done from the perspective of the user, and for that reason, it is sometimes referred to as a Functional test or a User Interface (UI) test. E2E test can prevent errors that could happen with real users by running the test in an environment that mimics that of the users. Also, by directly manipulating the browser, E2E tests enable developers to test for conditions that cannot be simulated by the JavaScript API (such as manipulating the browser size and inputting items with real keyboards.) Furthermore, because E2E test codes are not directly affected by the internal structure of the project code, test does not fail even in the presence of wide coverage refactoring, and therefore, allows developers to take more blunt attempts at improvement.

Despite E2E test’s obvious benefits, because E2E tests are significantly slower than unit tests and integration tests, it is difficult to get timely feedback during the development cycle, and are extremely complicated to write due to the fact that E2E tests must consider the intricate combinations of the aggregated modules. Also, since the larger features cannot be tested in smaller segments, repetitive results are almost inevitable. Furthermore, because the test is not hosted in a sandbox environment, unexpected environmental errors (network error, timeout caused by process delay, etc.) may occur, which prevents developers from trusting the test wholeheartedly.

JavaScript Testing Tools

(This chapter will only cover tools regarding unit/integration tests. Since E2E testing tools are characteristically different, E2E testing tools will be covered in a separate chapter.)

Latest JavaScript testing tools provide a wide range of functionalities for testing, and different tool supports different features. In order to appropriately compare and select the tools, it can be helpful to first understand which features are needed to test JavaScript codes. Although there are no specific standards categorizing these features, features can mainly be categorized into a Test Runner, a feature that provides an environment in which to conduct tests in, and a Test Framework, a feature that provides a template for the test codes to be written. Other features include assertion library and test double library which facilitate efficient drafting of test codes.

Test Runner

Test runners first read the test files to execute the wirtten code, and prints the results in a specific format. The test results can be returned in whatever format the developer desires by designating a reporter. Additionally, test runners provide a watcher feature that automatically tests the parts that are affected by changed test code or source code.

Because, previously, JavaScript friendly environments were scarce, the test results could only be obtained through the webpage or the console after running the test code directly onto a browser. However, with the emergence of Node.js, it became possible to easily execute JavaScript codes, and consequently, it also became possible to use tools like test runners to automate these procedures.

Test runners can be categorized mainly into runners that run directly on the browser like Karma and those that run on the Node.js environment like Jest. Since in using test runners based on Node.js do not need to separate the environment for the runner and the environment for the code execution, they are usually included in the testing framework.

Testing Framework

Testing framework, as mentioned above, is a tool that provides a template for developers to use when writing test codes. The tests, created using the functions provided in the template, are executed automatically by the template, and the results are returned to the user according to the success or failure of the test. Most iconic testing frameworks include Mocha, Jasmine, AVA, and recently, Jest has been growing in reputation rapidly.

Below is a test code created using the Jasmine template. Functions like describe, beforeEach, it, and expect are global functions provided by Jasmine, and will be explained in greater detail in later chapters.

describe('calculations', () => {
  let a, b;
  
  beforeEach(() => {
    a = 10;
    b = 20;
  });
  
  it('sum two number', () => {
    expect(a + b).toBe(30);
  });
  
  it('multiply two number', () => {
    expect(a * b).toBe(200);
  });
});

Assertion Library

Tests are usually composed of reset and assertion statements, and assertions are used to specifically restrict the passing requirements for tests. Most testing frameworks provide various assertion APIs, but only when using Mocha, it is recommended to use an external assertion library like Chai.

Primitive assertion libraries were similar to JUnit, but Chai and Jasmine, libraries that are gaining popularity recently, are based on BDD(Behavior-driven development APIs, which bare close resemblance to the natural language constructs. Also, most assertion libraries provide plugins and extensions that allow users to add custom assertions.

Presented below is a test code created using the Jasmine template. Multiple assertions like expect(), toBeNull(), toEqual(), and toHaveBeenCalled() have been used.

expect(obj).not.toBeNull();
expect(obj).toEqual({
  name: 'Kim',
  age: 30
});
expect(result).toBe(true);
expect(result).toBeTruthy();
expect(spy).toHaveBeenCalled();

Test Double Library

Test double is an object that is tested in place of the original object, and is often used to arbitrarily inject external dependencies during isolated unit tests. Test double libraries help developers to create dummy objects, and according to the characteristics of a given test, various objects like spy, stub, and mock can be created. Just like in the case of assertion libraries, most testing frameworks provide various functions to support test doubles, and only when using Mocha, it is recommended to use an external test double library like Sinon.JS.

Test doubles are generally used in tests where it directly changes or adds objects and functions, and in Jest, a feature that supports modular test doubles are also included. Furthermore by using Clock object in Jasmine or Lolex object in SinonJS, developers can override control of the JavaScript’s built-in timer API for tests. Also, for famous libraries like Axios, it is possible that there already exist premade Mock libraries, so it is worth searching.

The following is an example of a test double created by the spyOn function from Jasmine. Spy is used to monitor if a certain method within an object has been called on, and if so, which inputs were passed on to it.

const person = {
  name: 'Kim',
  getName() {
    return this.name;
  },
  setName(name) {
    this.name = name;
  }
}

it('test spy', () => {
  spyOn(person, 'setName');
  spyOn(person, 'getName').and.callThrough();
  
  person.setName('Lee');
  const name = person.getName();
  
  expect(person.setName).toHaveBeenCalledWith('Lee');
  expect(person.getName).toHaveBeenCalled();
  expect(name).toBe('Kim');
});

JavaScript Testing Environments

As mentioned above, JavaScript tests can be hosted both in a browser environment and in a Node.js environment. However, it is necessary to acknowledge that both environments have different pros and cons; therefore, for each environment, runners must be selected with careful consideration. This chapter will include information regarding the strengths and weaknesses of each environment, and which tests are better suited.

Browser

Browser environment means running test codes in an actual browser, and currently, excluding the E2E testing tools, using Karma is the only way to achieve this goal. However, since Karma only acts as a test runner, Karma requires a separate testing framework, and it is generally recommended to use Jasmine.

When Karma is executed on the command line, it first builds its web server, creates an HTML page to run the test on, and proceeds to load the test codes and the entire source code on the webpage. Later, when the webpage is accessed directly through the browser, the loaded code will execute, and the results will be printed on the console. Karma, then, takes the resulting information and by using the designated reporter, presents the organized results on the command line.

The biggest benefit of this method is that because the test is ran in a real browser environment, all of functionality provided by the browser (network IO, rendering engine, etc.) can be used to test the application. Also, by using tools like Selenium, the same test can be conducted on different environments (operating system, browser), so developers can carry out compatibility tests and other tests regarding device environments.

However, the tests ran on an actual browser are inevitably slower than that of Node.js because a browser process is much heavier than a Node.js process. Also, in order to run tests in a browser environment, browser launchers have to be additionally installed, and the cost of creating and maintaining different browser environments cannot be overlooked.

In order to address these issues, a headless browser is often used during the development stages for the sake of timely feedback, and only when the product is ready to be distributed or is at the end of the development stage, can cross browsing test be recommended to be integrated with a CI server. Also by using external services like Browser Stack and Sauce Lab, developers can use Karma without having to design the cross browsing environment manually.

Node.js

Node.js environment means running test codes on Node.js, and recently Mocha and Jest have been used most widely. As mentioned above, in Node.js environments, test runners and frameworks are integrated into a single item, so it is relatively easy to install and execute. Also, it is still difficult to run modular tests in browser, so browser environments are restricted to using bundlers like webpack, but in Node.js environment, desired modules can be directly imported from individual processes to be tested, which makes it much simpler and safer.

However, the key disadvantage to using the Node.js environment is that it cannot aptly apply all of browser’s APIs because browsers maintained by Node.js lack APIs like DOM(Document Object Model) and BOM(Browser Object Model). Developers are using libraries like jsdom to create a virtual browser and address this issue, but it comes with numerous restrictions, since it cannot fully mimic a real browser. For example, a layout of an UI element cannot be tested due to the lack of the rendering engine, and navigation related modules cannot be tested. Furthermore, because the test cannot be ran on a browser, cross browsing tests cannot be done.

Browser vs. Node.js

If the reader is still confused as to which environment should be used, consider the following suggestions.

1. Use the browser environment if a cross browsing test is mandatory.
2. Use the browser environment if the project requires to be on actual browser features (rendering, network IO, navigation, etc.)
3. Otherwise, use the Node.js environment.

The reason “mandatory” is emphasized in the first item is that the necessity of cross browsing tests has decreased significantly in the recent years. Among the latest browsers, there is not a significant difference on how browsers handle standard representation of codes compared to older browsers. Also, since Babel takes care of syntactical issues regarding compatibility, and frameworks like React(https://reactjs.org) and Vue(https://vuejs.org) handle DOM manipulation, it is recommended to carefully consider whether the cross browsing test is truly necessary or not.

(From the stand point of [QA(Quality assurance)](https://en.wikipedia.org/wiki/Qualityassurance), it is only natural to ensure that the application will run flawlessly in any given environment. This guide discusses testing not from a QA’s perspective, but from a developer’s perspective, and in such case, it is paramount to weigh the benefits of writing and managing a cross browsing test.)_

The following tools are not only recommended by the guide but are also currently widely used.

• Browser : Karma + Jasmine
• Node.js : Jest

Each tool will be explained in greater detail in the following chapter.

Jasmine

Jasmine is an integrated testing framework that uses BDD style assertion API, and works well with both Node.js and browser environments. While using assertion libraries and test doubles in Mocha require Chai and Sinon, respectively, Jasmine provides an integrated product, so it is unnecessary to install additional libraries and is easy to use.

Defining Test Specs

Every test specs in Jasmine must be written using the it() function. it() function takes two parameters--the title of the spec as its first parameter and a function that will run the spec as its second parameter. Inside of the function that will run the specs, expect() statement can be used to verify the assertions. Let’s create a simple example that adds two numbers as an example.

function sum(a, b) {
  return a + b;
}

Below is an example of a spec to test the example function.

it('The sum() function returns the sum of two inputs.', () => {
  expect(sum(3, 5)).toBe(8);
});

Grouping Test Specs

Grouping related tests according to the target or purpose makes it easier to manage the test when the number of specs increase drastically, and test results can be presented in an organized and grouped format. In Jasmine, describe() can be used to group tests, and can be nested to form multiple lower level groups.

describe('Simple Arithmetics', () => {
    describe('sum()', () => {
        it('If there is only one input, raise an error.', () => {
          // ...
        });
        
        it('If there are two inputs, return the sum.', () => {
          expect(sum(3, 5)).toBe(8);
        };
    });
    
    describe('multiply()', () => {
      // ...
    });
});

Resetting Tests

Sometimes, when writing tests, repetitive resets are necessary. In this case, beforeEach() and afterEach() functions allow developers to separately define logics that are necessary before and after each specification is executed.

let uploader;

beforeEach(() => {
  uploader = new Uploader({
    url: 'http://test.url'
  });
});

afterEach(() => {
  uploader.destroy();
});

describe('Uploader', () => {
    it('Request File Upload', () => {
      // ...
    });

    it('Import upload queue', () => {
      // ...
    };
});

Spy Class

When using mock objects in JavaScript, the spy class is one of the most useful test doubles. Spy can not only act as a test double, but also stores information like how many times a function was called or which inputs were passed. This stored information can be extremely useful in validating the test. Spies can be created with relative ease by using functions like createSpy() and spyOn(), included in Jasmine, and can be used to validate different results by using assertions like toHaveBeenCalledWith().

let counter;

beforeEach(() => {
  counter = new Counter();
  spyOn(counter, 'inc');
  
  counter.inc(10);
  counter.inc(20);
});

it('inc() Check number of calls', () => {
  expect(counter.inc).toHaveBeenCalled();
});

it('inc() Check number of calls', () => {
  expect(counter.inc).toHaveBeenCalledTimes(2);
});

it('inc() Check number of calls', () => {
  expect(counter.inc).toHaveBeenCalledWith(10);
  expect(counter.inc).toHaveBeenCalledWith(20);
});

Clock Class

Jasmine allows developers to directly manipulate the JavaScript’s built-in timer API. However, when using timer, global functions like setTimeout and Date are affected internally, so it is critical to remember to exit the clock class after it has served its purpose.

beforeEach(() => {
    jasmine.clock().install(); // Clock starts
});

afterEach(() => {
    jasmine.clock().uninstall(); // Clock exits
});

it('setTimeout() The function will execute the callback function after given miliseconds', () => {
  const callback = jasmine.createSpy('callback');
 
  setTimeout(callback, 100);
  
  expect(callback).not.toHaveBeenCalled(); // Callback not yet executed
  jasmine.clock().tick(100); // after 100 ms
  expect(callback).toHaveBeenCalled(); // Callback executed
});

Asynchronous Test

Using Callback Functions

If the target test code is executed asynchronously, in order for test specs to finish, they have to wait until all asynchronous code is done executing. In this case, a separate parameter (usually named done) is used in the callback function describing it. When done parameter is declared in this callback function, the spec will be put on hold until done parameter is executed.

it('fetchData: After calling the API, executes the callback function asynchronously.', (done) => {
  api.fetchData((response => {
    expect(response).toEqual({
      success: true
    });
    done(); // Hold until this function executes
  });
});

Using a promise

If the callback function describing it returns a promise, Jasmine intrinsically knows to wait until the promise is resolved. Had the previous example returned a promise instead of using a callback, the spec would have been able to be written in a more concise format. In this case, the promise, result of the fetchData() function, is passed onto the it to be returned, the test will not exit until the promise is resolved.

it('fetchData: Returns a promise', () => {
  return api.fetchData().then(response => {
    expect(response).toEqual({
      success: true
    });
  });
});

Using asnyc/await

In environments that support the async/await, newly added in ES2017, async function can be passed on to it as a callback function directly. Since async function automatically returns a promise at the end, it does not need any additional return statements to operate identically as the example above, and the code becomes much more intuitive.

it('fetchData: Returns a promise.', async () => {
  const response = await api.fetchData();
  expect(response).toEqual({
    success: true
  });
});

Karma

Previously to run tests written in Jasmine in the browser environment, it was necessary to create a page and to load all of the source codes and test codes. Also, to check the test results, UI had to be added or the console from the browser developer tool was required. Karma is a tool that automates such series of operations when running tests in the browser environment, and also does the following.

• It builds a local web server, and creates an HTML page to load all of the codes and resources needed for tests.
• Automatically executes the defined browser’s process, and connects to the previously created page’s URL.
• Takes the results from the browser, and uses the designated reporter to present the data in various formats.

Install and Usage

Following sections will briefly introduce the process from installing and configuring Karma to actually using it.

Installing Karma

Using npm, the binary file can easily be downloaded.

$ npm install --save-dev karma

Installing Jasmine Plugin

To use Jasmine with Karma, Jasmine’s source code has to be loaded manually through the Karma configuration file. If using Jasmine as a plugin, Jasmine can be used immediately without having to go through this step (assuming Jasmine is already installed.)

$ npm install --save-dev karma-jasmine

Installing the Chrome Launcher

To automatically launch browsers, corresponding launchers must be installed. For example, to use the Chrome browser, karma-chrome-launcher must be installed. Karma offers a wide range of options besides Chrome, and the list of compatible browsers can be found on the official webpage.

$ npm install --save-dev karma-chrome-launcher

Configuration File

Karma can be configured using the karma.config.js file at the root folder of the project. Below is an example of a simple config file that only includes the bare minimum configuration, and detailed explanations on each configuration can be found on the official website.

module.exports = (config) => {
  config.set({
    frameworks: ['jasmine'],   // Using the Jasmine testing framework
    files: [
      'src/**/*.js',      // Source file path
      'test/**/*.spec.js'  // Test file path
    ],
    reporters: ['dots'],    // Designating a reporter (will be printed in dots)
    browsers: ['Chrome'],  // Designating a launcher to automatically execute the Chrome browser
    singleRun: true         // Exit Karma after a single test
  });
};

Testing

First, register the npm script in the package.json.

{
  // ...
  "scripts": {
    "test": "karma start"
  }
}

Now, the test can be ran conveniently through the command line.

$ npm test

After the Karma executes, the browser will open, and when the test is over, the results will be printed to the console as shown below. When the test is over, the browser will automatically close.

Measuring Code Coverage

The coverage of the written code can be measured using the Istanbul library. The Istanbul library operates by analyzing the source code and inserting a piece of code that measures the number of execution line by line. After the code finishes executing, the results can be printed in various formats including HTML, LCOV, and Cobertura, and can be integrated with the CI server to be used.

Istanbul can be ran over the command line, but is generally recommended to use the plugin format provided by the test runner. The coverage can easily be monitored with Karma by using karma-coverage plugin provided by Istanbul.

Installing karma-coverage plugin

$ npm install --save-dev karma-coverage

Editing the Config File

When it finishes installing, the config file must be customized. To the example config file, add coverage to reporters, and also in preprocessors, add coverage to the target source file.

module.exports = (config) => {
  config.set({
    frameworks: ['jasmine'],
    files: [
      'src/**/*.js',
      'test/**/*.spec.js'
    ],
    reporters: ['dots', 'coverage'], // Add coverage reporter
    coverageReporter: {
      type: 'html', // Define the coverage printing format
      dir: 'coverage' // Define where the coverage measurement report will be stored
    },
    browsers: ['Chrome'], 
    singleRun: true,
    preprocessors: {   
      'src/**/*.js': ['coverage'] // Define the preprocessors for the coverage measurement of the entire code
    }    
  });
};

Running and Checking the Results

Now, when Karma is executed, a folder named with the tested browser and the operating system Chrome 69.0.3497 (Mac OX X 10.14.0) can be found in the coverage folder. (If multiple launchers were used, a separate folder will be created for each environment.)

In the index.html file located in src folder, coverage measurements for each file will be listed.

Individual results can be viewed by clicking the corresponding file. The following image is a line by line coverage result of a calc.js file. The 1x to the left of the file means that the corresponding line was executed once throughout the entire test.

Cross Browsing Test

This document has so far dealt mainly with hosting tests on the local PC. By using the browser launcher plugin provided by Karma, developers can conduct a variety of browser tests on a local PC. However, it may be impossible to test solely on the developer’s local PC if a project requires to be tested on more devices. For example, Internet Explorer can only be installed once on a Windows operating system, and using a virtual machine or multiple PCs are the only way to test for multiple versions of Internet Explorer.

In such cases, by connecting Karma and Selenium WebDriver, tests can be conducted on remote PCs, and the results can be printed in a single file. To put it simply, instead of the Chrome launcher from the previous example, karma-webdriver-launcher is used. Then, the test can be executed by using a Hub like device to allow remote PCs to connect to the local Karma server.

Although this guide does introduce Selenium WebDriver later in the “E2E testing tools” section, actual installation and test environment configuration are not discussed. Refer to Creating an environment for multi-browser testing (Korean) for detailed information on installation and usage.

Jest

Jest is an opensource test framework created by Facebook, and is a test tool that is currently most hotly used. Although it has been around for quite some time, developers have only recently started to use it due to a recent performance and durability enhancement. Unlike Karma, Jest is executed on Node.js environment, and internally uses Jasmine style assertion API, so users who are acclimated to Jasmine can easily make the change.

Although this guide does not introduce specific usage, it will list out some of the benefits and useful features.

Easy Install and Usage

The biggest advantage of Jest is that it is very easy to install and to use. Not only does Jest provide test runner functionality, but also provides assertion, test double, code coverage, and basically every feature necessary for efficient testing; no additional installation is necessary. Also, since it can be executed without much configuration, or in default settings, novice users who are new to the tool can easily write tests to be applied to actual projects.

Below is an example of a Jest code to demonstrate its simplicity. First, it can be installed using an npm command.

$ npm install --save-dev jest

To make sure that the test runs smoothly, register the test to package.json.

{
  //...
  "test": "jest"
}

If the test is of *.spec.js format, test can be ran without any additional setup.

$ npm test

Then, the following result will be printed to the terminal.

Jest is not only useful for the novices, but also extremely effective with the experts. A separate config file jest.conf.js can be used to manipulate different settings, and jest property can be used within the package.json file to configure the tool as well. For example, to determine the path of the test file, use the testMatch option in package.json file as presented below.

{
  "name": "my-project",
  "jest": {
    "testMatch": ["<rootDir>/test/**/*.spec.js"]
  }
}

Detailed description about each option can be found on the official website.

Easy Code Coverage Measurement

Jest, much like Karma, uses Istanbul to measure code coverage. However, in Jest, this feature is also provided as an integrated functionality, users do not have to install additional files or change the setting file. To check for coverage for the previous example, simply add the --coverage option to the command line.

To execute an npm script, add -- as presented below.

$ npm test -- --coverage

After it finishes execution, the coverage measurement result will be printed in the command line, and a coverage folder will have been created in the project folder.

It can be seen that the index.html file in the lcov-report folder is identical to the one from the previous example using Karma.

Built-In jsdom

As mentioned above, DOM and window APIs provided by browsers cannot be used in Node.js environments. Therefore, when running front-end code tests, such APIs have to be created virtually, and jsdom is one of the libraries that is most widely used and most successfully created. However, since jsdom comes in a library format, a piece of reset code has to be executed each time a test is ran, and in the browser environment, it is a tedious task to say the least.

However, in Jest, jsdom is built-in and is automatically provided with appropriate environment settings every time a test is ran, so it can be used as if it is being used on a browser environment without much additional work.

Snapshot Testing

Snapshot testing is Jest’s symbolic feature. In simplest terms, snapshot test stores the internal status of an object exactly as a file, and compares how the file has changed after a test is ran. It is sort of a regression test, and it allows developers to visually compare the effects of a test without having to assign expected values to an extremely complex object with equally complicated internal structure.

Snapshot test is usually done to compare virtual DOM structures of React, and it only requires toMatchSnapshot() function to be added.

import React from 'react';
import Link from './Link';
import renderer from 'react-test-renderer';

it('Confirm rendering', () => {
  const tree = renderer
    .create(<Link page="http://ui.toast.com">TOAST UI</Link>)
    .toJSON();
  expect(tree).toMatchSnapshot();
});

Snapshot file will be created automatically after running the test, and the file will contain the identical information as the rendered DOM structure as shown below.

exports[`Check rendering 1`] = `
<a
  className="normal"
  href="http://ui.toast.com"
  onMouseEnter={[Function]}
  onMouseLeave={[Function]}
>
  TOAST UI
</a>
`;

Once the snapshot file is created, the test will fail if the component returns a DOM that is different from the one that is saved. Developers can inspect the changes, and if the changes are intentional, renew the snapshot, and if not, developers can attempt to fix the issue.

Such testing method can be used successfully with complex data structures even if the file is not a React file. However, snapshot testing does have a flaw that the purpose of the test may not be clear. Especially, if the developer composes or renews the snapshot without careful consideration of the resulting details, it may contribute to building inefficient testing habits.

Test File Filtering

Jest provides a feature that allows developers to specifically point out which file to run the test on. Jest, as default, synchronizes with a version control tool, like Git, and only includes files that changed since the last commit. This way, Jest prevents running redundant tests on already verified data.

Using the command line interface, developers can set the target with even more precision. Unlike test runners that cannot be interrupted with additional commands, Jest provides an interactive command line interface so developers can change the target files in the middle of the testing process. With the CLI, developers can even pause a running test or inject a test filter so that only files that match the criteria are tested. Also, if the snapshot test fails, it is possible to immediately renew the snapshot after checking the results..

The following video clip shows the process of using the a key to run the test on all files, q key to quit, and p key to run test only on desired patterns.

Sandbox Parallel Testing

The biggest advantage of running tests in the Node.js environment is the “speed.” Node.js processes are much lighter than that of the browsers, so initial running speed has to be faster than the browsers. Jest exploits this characteristic to run test files from independent processes. In this case, the global scope defined in each test cannot influence others and executes as if it were running in a sandbox, so the test becomes much safer.

However, if tests create children tests while being executed serially, it becomes slower than running the test in a single process. In order to address this issue, Jest parallelly executes multiple processes, and optimizes the processing speed by aptly monitoring the internal processes and the number of CPU cores. Also, by using the, previously explained, “test filtering” feature, Jest saves even more time by not running redundant tests.

As such, Jest maintains the high processing speed as well as a much safer testing environment.

E2E Testing Tools

So far, the tests mentioned in this guide like Karma and Jest are all tools best suited for unit and integration testing. E2E tests have been unfavored by developers because they are difficult to write, executes extremely slowly, and cannot be conducted in a controlled environment. However, since front end development and UI/UX cannot be completely isolated from the real users’ environment, E2E tests, tests held from the users’ perspective, have always been required.

In such trend, recently, tools like Cypress and TestCafe emerged, and they minimize the disadvantages the original E2E tests had while maximizing the benefits. This chapter will compare Selenium Webdriver, previously most famous tool, and Cypress, currently most famous tool, and will attempt to observe the growth of E2E testing tools.

Selenium WebDriver

Selenium WebDriver, often referred to as Selenium or WebDriver, is a name that collectively refers to the WebDriver API, newly introduced in Selenium 2.0, and the Selenium itself. While Selenium 1.0 operated by injecting JavaScript inside of the browsers, from 2.0, Selenium operates by controlling the browser from the outside, therefore allowing more intricate controls. Also, Selenium Grid provides developers a way to use the same test code to execute the tests on multiple devices.

WebDriver is an HTTP based protocol created to control the browser based on uniform APIs, and although it started as an individual specification, is now a standard maintained by W3C. In WebDriver, the browser acts as the server and the device requesting control (developer’s PC or CI server) acts as the client, forming a server-client architecture. It is used by installing browser drivers and developer client.

For example, in order for the browser to process the client’s request, a separate driver must be installed, and Selenium’s official webpage offers multiple drivers for different browsers like Chrome, Internet Explorer, Firefox, and Safari. Furthermore, since WebDriver is a protocol based on JSON supported by HTTP, it does not restrict languages when writing test codes, and so far, provides clients that support not only JavaScript, but also Java, C#, Ruby, Python, and much more.

Especially with JavaScript, there already is a variety of testing frameworks based on Node.js that uses WebDriver API, and here are some examples.

• Protractor : Testing framework for Angular projects
• WebdriverJS : Official Node.js representation by Selenium WebDriver; supports low-level API
• NightWatch : Provides wide range of features like Mocha based test runner, intuitive API, and integrated CI server
• WebdriverIO : Provides wide range of features like test runner, static web server, integrated CI server, REPL interface, etc, and has the most active community.

WebDriver can be said to be the most widely used E2E test as of current. It can be used by developers and specialized testers, and is used to perform numerous automatizations using the browser. However, WebDriver is still subjected to all of the disadvantages mentioned above, so is expensive to maintain and is not very plausible to be used in the development stages.

Cypress

Cypress, along with TestCafe, is the E2E test tool that stands in the spotlight today, and unlike WebDriver, operates by processing the actual application and the test code on the same browser. By doing so, Cypress eliminates the unnecessary communication between processes using protocols like HTTP and runs the tests in the same internal process, which makes testing faster and safer. Cypress also provides useful functionalities like allowing developers to check the current state of the test and to debug through a GUI based on the browser.

For example, Cypress stores all of the test commands and corresponding snapshots of the UI, so developers can visually inspect the UI at a certain point in the test. Also, the entire testing process can be stored as a recoding or the failed states can be stored as screenshots, so developers can easily track down the problem. Furthermore, because the tests run on the browser, if necessary, developers can use the Chrome developer tools to debug.

However, the disadvantage that it runs in the browser still exists, and such issue is explained in greater detail on the Cypress official documentation. For example, developers cannot open a new tab or a new page, and cannot navigate to a page that violates the same-origin policy. Also, because only JavaScript can run on browsers, testing code is restricted to be only written in JavaScript.

However, these disadvantages hold true for all E2E tests, and do not cause much trouble when testing JavaScript code. Cypress is built with a completely different purpose compared to Selenium WebDriver, and is optimized to be used by front end developers during the development stage. Especially, since Cypress eliminates the main disadvantage of E2E tests, the high maintenance cost and long runtime, developers can now use E2E tests as simply as using unit tests or integration tests. Given the benefits E2E tests provide, as tools like Cypress continue to be developed, more and more developers will compose and use E2E tests.

Installing and Using Cypress

Now, let’s actually install Cypress and use it to test a simple ToDoMVC application.

Install and Set Up

Cypress can easily be installed using npm.

$ npm install cypress --save-dev

Although Cypress can be ran without additional configuration, it is useful to register the npm script. Enter the following script in package.json.

{
  "scripts": {
    "cypress:open": "cypress open",
  }
}

Running on Interactive Mode

Entering the following command in the command line will start Cypress.

$ npm run cypress:open

A cypress folder will be created in the project folder when Cypress is first ran. This folder contains useful sample codes for developers who are new to Cypress. The integration/examples folder contains multiple test specs for different scenarios, so can be extremely useful when starting out with Cypress.

When Cypress is executed, a GUI application created using Electron will appear as presented below. The current test specs will be shown in the Tests tab, and even shows sample files in integration/examples folder. Let’s pick actions.specs.js file and actually run a test on it.

Test runner and individual tests will run sequentially when a test is selected as in the screenshot above. Listed below are brief introductions on each view of the test runner, and more detailed information can be found on the official website.

(1) Test Status Menu - Number of currently running and failed tests will be presented here along with the time-spent.
(2) URL Preview - The URL of the currently running application will be presented here. (3) Command Logs - All commands executed within the test specs will be presented here. - Developers can visually inspect the state of the corresponding command by hovering or clicking on the command. (4) App Preview -The application will be presented here as it executes.

Highlighted in purple in the screenshot above (A), is the currently running command, and in this case, the app preview shows the effect on the DOM element the current command has on it (B). Also, the layer (C) at the bottom of the app preview allows testers to compare the before and after of the command running.

Running in the Background

Cypress will execute with the GUI as presented above when executed with the open command. While this feature is useful during the development stages, it is unnecessary in the CI server environment, and in this case, Cypress can be executed in background mode, without the GUI, by the run command. Add the following script to package.json and run the following npm commands.

{
  "cypress:run": "cypress run",
}

$ npm run cypress:run

Executing the run command without specifying different options will take some time since it runs every single test. As tests run, each test result will be printed on the terminal as presented below.

When the last test finishes running, Cypress will print out the collective test results.

The recording for the entire testing process is saved automatically, and will be created in cypress/videos/examples folder as mp3 files.

Writing Cypress Codes

Now, let’s experience the Cypress API by writing a simple test using Todo MVC application. The ToDo MVC example can easily be obtained from the website, and since the installation process does not vary with frameworks, will not be discussed. This guide assumes that ToDo MVC is hosted on http://localhost:8888.

Creating and Running Test Files

The first thing in order is to add the test file. Insert the todo.spec.js file to the cypress/integration folder, and enter the following.

describe('Todo MVC', () => {
  beforeEach(() => {
    cy.visit('http://localhost:8888');
  });
});

Cypress uses Mocha API internally, so it can be observed that the functions like describe and beforeEach are similar to the previously explained Jasmine API. The only notable difference is the global object cy, and in Cypress, every command goes through the cy object. The cy.visit in the code above, is a command to load the page using the URL that was passed in as an input.

Writing Test Codes

Now, let’s execute the test file by entering npm run cypress:open in the command line to run Cypress and clicking the todo.spec.js in the test runner. “No tests found in file” message will appear in the command log since no tests have been written. Now let’s actually write the tests.

describe('Todo MVC', () => {
  beforeEach(() => {
    cy.visit('http://localhost:8888');
  });
  
  it('Enter Todo', () => {
    // Press enter after typing "Cypress Exercise" in a new-todo element
    cy.get('.new-todo').type('Cypress Exercise{enter}'); 

    // The new-todo element's value should be an empty string.
    cy.get('.new-todo').should('have.value', '');
    
    // The first child of todo-list element will include the "Cypress Exercise" text. 
    cy.get('.todo-list li').eq(0).should('contain', 'Cypress Exercise');
    
    // The todo-count element will include "1 item left" text. 
    cy.get('.todo-count').should('contain', '1 item left');
  });
});

As the code above demonstrate, most of the functions used within the cy object returns the cy object again, so can be called by chaining. This guide does not discuss the cy object any more than absolutely necessary, and recommends that readers refer to the official API documentation. However, since APIs like get, type, and should are extremely intuitive and simply built, it should not be difficult to understand what the tests are trying to prove without much explanation.

When the code above is saved, Cypress will notice that there have been changes in the code, and it can be seen that the test runner has started again. At the end of the tests, Cypress will present the following screens. In the command log area left of the screen, is every single command executed in this test, and each command can be hovered or clicked on to show the state of the application that corresponds to the command.

So far, it has been a brief introduction to Cypress. Cypress offers much more than discussed above. The official Cypress documentation presents tutorials, test writing guide, and API documentation, which are all incredibly well written, and the author recommends the readers to take the time to read each document carefully.

Afterword

So far, this document explored variety of testing tools used in the front-end environment like Jasmine, Karma, Jest, Selenium WebDriver, and Cypress. The numerous testing tools stand to reason that JavaScript is being used in wide range of areas, and that along with JavaScript, the testing tools are evolving as well. Through this document, author hopes that readers may understand the pros and cons of each tool, and to use them according to the characteristics of the project in order to better the quality of the product and the code.

The employee training provided at FE Development Lab related to this document are as listed below. It is recommended to take these courses as well.

• Testing
• Real World JavaScript Development using Webpacks

This document is an official Web Front-End development guide written and maintained by NHN Cloud FE Development Lab. Any errors, questions, and points of improvement pertaining to this document should be addressed to the official support channel (dl_javascript@nhn.com).