Building a 0.7KB Reactivity System Similar to Vue

Currently, the TOAST UI Grid team is working diligently for the major update that is the release of the version 4. The purpose of this update is to rewrite all of the previous code base written with Backbone and jQuery from scratch. The team is expecting to decrease unnecessary dependency, making the new Grid to be leaner and faster than before.

As a way of celebrating the alpha release of v4, I would like to elaborate on what differences between event based state management system like Backbone and reactivity systems like Vue and MobX are, why we created the reactivity system for ourselves, and what you need to consider to fully implement the reactivity systems using the actual source code.

What Is A Reactivity System?

Given the ubiquity of the word "reactive" in the field of programming (functional reactive programming etc.) the system will be described to be a reactivity system. For the duration of this article, "reactivity" will refer to the way systems like Vue and MobX functions, and more specifically, to systems that automatically detect the state change of an object to modify the state of other objects that use the modified object, or automatically refreshing the related view of the object. In other words, it is a system that automatically does everything that previous event based systems had to do, including emitting events to signal stage change and registering listeners to detect those changes.

Most of the frameworks released after Backbone actually support such method, and when AngularJS became popular, the expression “data binding” was used frequently. However, with Vue officially using the word “reactivity,” the word reactivity has become a symbolic description of Vue, and is often used in relation to implementation methods seen in Vue. In order to implement aforementioned reactivity, Vue uses getter/setter to register Proxies, and in Vue 3, which is currently under development, the reactivity will be implemented using ES2015’s Proxy.

MobX uses a reactivity system that is very similar to Vue, and since it uses getter/setter up until version 4, and Proxy starting from version 5, you can use the appropriate version that suits the necessary browser requirements. However, since MobX is just a state management library, in order to represent the UI, you would need to use it with other frameworks like React.

Event Driven vs. Reactivity

Now, let’s look at some simple example codes to better compare the benefits reactivity system has over the traditional event driven methods. The Backbone’s Model will be used to demonstrate the event driven system, and MobX’s observable will be used to demonstrate the reactivity system. Since the two will be used only to explain the basic concepts, even if you are not familiar with the libraries, you should have no problem following along.

Let’s say that there are two players in a game: A and B, and there is a Board that displays the sum of the two players’ scores. As each player’s score is updated, the sum displayed in the Board has to update itself as well. If you were to implement this feature using Backbone, it would look something like the following.

import {Model} from 'BackBone';

const playerA = new Model({
  name: 'A',
  score: 10
});

const playerB = new Model({
  name: 'B',
  score: 20
});

const Board = Model.extend({
  initialize(attrs, options) {
    this.playerA = options.playerA;
    this.playerB = options.playerB;
    
    this.listenTo(playerA, 'change:score', this._updateTotalScore);
    this.listenTo(playerB, 'change:score', this._updateTotalScore);

    this._updateTotalScore();
  },
  
  _updateTotalScore() {
    this.set('totalScore', this.playerA.get('score') + this.playerB.get('score'));
  }
});

const board = new Board(null, {playerA, playerB});

console.log(board.get('totalScore')); // 30

playerA.set('score', 20);
console.log(board.get('totalScore')); // 40

playerB.set('score', 30);
console.log(board.get('totalScore')); // 50

In the code to define the Board class, in order to detect changes in the score property of playerA and playerB, we are listening for the change:score event. If we were to detect changes in other properties, we would have to create new listeners.

Let’s compare this to the reactivity system. What would the code look like if the same feature were implemented in MobX?

const {observable} = require('mobx');

const playerA = observable({
  name: 'A',
  score: 10
});

const playerB = observable({
  name: 'B',
  score: 20
});

const board = observable({
  get totalScore() {
    return playerA.score + playerB.score;
  }
})

console.log(board.totalScore); // 30

playerA.score = 20;
console.log(board.totalScore); // 40

playerB.score = 30;
console.log(board.totalScore); // 50

From the code above, we can clearly see that the totalScore property is defined within the board object as a getter function. This property detects any changes to the observable values that is referenced inside of the getter function, and if any change is detected, the function calls the getter function again to update its value. Such properties are also known as computed or derived properties, and this is the defining characteristic of a reactivity system. In the MobX’s Official documentation, the architectural philosophy of the library is clearly stated in the following sentence.

Anything that can be derived from the application state, should be derived. Automatically.

In traditional event based systems, functions that update the property values like _updateTotalScore() decide the final value of the property, and if such functions are spread out throughout the code, it makes it difficult for you to clearly summarize the dependencies among different data sets. Contrarily, computed properties depends only on one getter function, and since it usually does not require additional setter function, the getter block provides a clear summary of how the value changes throughout the program.

Therefore, if the previous event driven systems were imperative, the reactivity system allows users to define the structures in a more declarative manner. As it should be clear from the example code, the codes that were declared are shorter and more intuitive compared to the imperative programming. Furthermore, it represents a clear relationship for all of related data.

Why We Built Our Own Reactivity System

TOAST UI Grid is an independent library that does not rely on other frameworks. Since it should be able to cooperate with any of the existing UI frameworks, Vue was not even considered to begin with. However, since MobX can only manage the states of the system, MobX provides much more diverse features while being much smaller than Vue. For example, MobX provides not only every day objects, but also provides features to translate variety types of objects like arrays and maps to be reactivity objects, multifarious observer functions, intercept & observe functions, asynchronous action. So, we thought, what if instead of creating a separate state manager, we just use MobX?

To be fair, MobX is already known to be great for creating general web applications. However, when creating a UI library like TOAST UI Grid, there are more aspects to consider, such as external library dependency, bundle size, and performance. Following are some of the reasons why we decided against using MobX.

1. External Library Dependency and Bundle Size

One of the main purpose of the TOAST UI Grid version 4 update was eliminating the previous dependency to external libraries (Backbone, jQuery.) Using external libraries puts extra pressure on the file size and performance, so it is best to minimize external dependency of any library. However, if we eliminate the previous dependency by introducing a new dependency, it defeats the purpose of this update.

Furthermore, the minified bundle size of MobX v4.9.4 is around 56KB (16KB if compressed using Gzip.) Considering that minified file size of Backbone is around 25KB (8KB if compressed using Gzip,) MobX is almost double that of Backbone. It would be a different story if we needed every single feature included in MobX, but since we were only interested in using a portion of the library, it certainly was an uncomfortable size. Therefore, it would be analogous to trying to lose weight by eating nothing but burgers every day.

2. Performance Issues with Processing Large Data

Like all technology, reactivity system is not a panacea. Especially for a system that has to handle large arrays of data like the Grid, the reactivity system can have numerous performance downsides. Let’s consider the following piece of code.

import { observable } from 'mobx';

const data = observable({
  rawData: [
    { firstName: 'Michael', lastName: 'Jackson' },
    { firstName: 'Michael', lastName: 'Johnson' }
  ],
  get viewData()  {
    return this.rawData.map(({firstName, lastName}) => ({
      fullName: `${firstName} ${lastName}`
    }));
  }
});

console.log(data.viewData[1].fullName); // Michael Jackson

data.rawData[1].lastName = 'Bolton';
console.log(data.viewData[1].fullName); // Michael Bolton

data.rawData.push({firstName: 'Michael', lastName: 'Jordan'})
console.log(data.viewData[2].fullName); // Michael Jordan

In the code above, the data.viewData is updated every time the data.rawData is changed. By studying the code above, we can see that the data.viewData is refreshed for every operation within the data.rawData array including any modification to property values and inserting new elements. However, the problem is that no matter how minuscule the change is, entire array is iterated to create a new array. Therefore, if the size of the array is considerably large, it could be extremely costly.

For example, if the rawData has 100K data points, the program would react to a single change in rawData by iterating hundred thousand times to create a new viewData array. To avoid such issues, one of the solutions is manually handling each change according to the type of the changed property using observe, but this method reduces the effectiveness of declarative definition of reactivity system. Furthermore, it could lead to having to write longer codes than simply editing each case separately.

Also, if you are trying to make the array to follow the reactivity model, the MobX (v4) uses getter for each index of the array to register the Proxy, and this process takes a significant toll on performance as well. According to some of the tests I’ve ran on the PC, it took the program around 150ms to process an array with hundred thousand elements, and if each element has more than thirty properties inside, it took the program more than 10 seconds to process it.

Since the goal for the TOAST UI Grid is to limit the processing time to under 500ms even if there are more than hundred thousand data points, we decided that it would be counter-effective to directly use MobX’s observable. Therefore, we frequently encountered a situation where we had to build reactivity portions of the entire array, or where we had to perform intricate operations according to the addition/deletion/modification of the element. For performance-sensitive applications, it was our impression that building our own reactivity system would allow a more flexible approach to the problem.

Understanding the Basics of a Reactivity System: getter/setter

Now, let’s dive straight into building our own reactivity state management system. As I explained previously, if the internal mechanism of the system were extremely complicated, we would not have even attempted to create it for ourselves. However, contrary to the general impression, the underlying philosophies of the system are incredibly simple, and you can create an effective state management system using very few lines of code.

As previously mentioned, there are two ways to create a reactivity system. However, Internet Explorer and other older browsers do not support ES2015’s Proxy. For the case of TOAST UI Grid, we used getter/setter for the sake of browser compatibility, and will explain the same in this article.

The simplest units forming the reactivity system are the observable object and the observe function. The system operates by calling the registered observer functions every time a property value within the observable object changes. Every property that can be accessed internally from the observable object has a registered observer function as a listener, and the observer functions are passed into the observe function as inputs.

While the names can vary from library to library, for this article, we will use observable and observe which is similar to the names mentioned in the MobX API. Note that they are not related to the RxJS Observable, so be mindful when trying it for yourself.

Before the actual implementation, let’s go over the usage of it, first.

const player = observable({
  name: 'A',
  score: 10
});

observe(() => {
  console.log(`${player.name} : ${player.score}`);
});
// A : 10

player.name = 'B';  // B : 10
player.score = 20;  // B : 20

If you look at the example code, you can see that observer function that we passed into the observe function as input runs once at the beginning, and continues to run every time time the property value of the player object changes.

In order to implement this functionality, we would need to be able to know before we access the object’s property whether the observe function is running. First, let’s write our observe function.

let currentObserver = null;

function observe(fn) {
  currentObserver = fn;
  fn();
  currentObserver = null;
}

The observe function saves the function that we received as input as currentObserver and runs the function. Now that we have the currentObserver, every time the getter function is called, we store this function in the observer array, and every time the setter function is called, we call all of the stored observer function. However, since it is possible for the observe function to reference a single property multiple times, exercise necessary precautions not to register an observer function redundantly.

function observable(obj) {
  Object.keys(obj).forEach((key) => {
    const propObservers = [];
    let _value = obj[key];

    Object.defineProperty(obj, key, {
      get() {
        if (currentObserver && !propObservers.includes(currentObserver)) {
          propObservers.push(currentObserver);
        }
        return _value;
      },
      set(value) {
        _value = value;
        propObservers.forEach(observer => observer());
      }
    });
  });

  return obj;
}

The key portion of the code is that we have defined a new variable _value in the function scope. The purpose of this variable is to prevent the setter function to fall into an infinite loop when the setter function register a value to the property by this[key] = value. Other than that, since the code is only around 20 lines, I will leave it up to the you to study and figure out what the observable function does for yourself.

Of course there are more features we need to implement, but we can still use what we have so far to address the example problem I previously mentioned without running into major errors. This thirty-some lines of code is the very core of any reactivity systems.

Implementing the Derived Properties

Now, let’s get started with the derived properties. There are multiple ways to implement the derived properties. We could use decorators like MobX’s @computed, or we could even separately define a computed object like Vue. For this article, we will look at using the getter to forge a derived property from the defined properties. As before, let’s look at the usage, first.

const board = observable({
  score1: 10,
  score2: 20,
  get totalScore() {
    return this.score1 + this.score2;
  }
});

console.log(board.totalScore); // 30;

board.score1 = 20;
console.log(board.totalScore); // 40;

We defined the value of board.totalScore using a getter, and every time board.score1 and board.score2 change, the board.totalScore is automatically recalculated and assigned.

It may seem like a sudden increase in difficulty. However, the underlying philosophy remains identical to the observe function we had just written. We simply need to call the observe function internally to update the respective property value every time. In order to do so, we first need to make certain that the property has a getter function assigned to it when we iterate through the object. Then, we just access the get property of what the Object.getOwnPropertyDescriptor returns.

const getter = Object.getOwnPropertyDescriptor(obj, key).get;

Now, once we have defined the getter, instead of configuring the setter, we modify the internal data by calling the observe function to modify the internal data with the resulting values from the getter function, and calling the registered observer functions. However, since this is used to access the object from inside of the getter, we have to use the call to provide the object as the context.

if (getter) {
  observe(() => {
    _value = getter.call(obj);
    propObservers.forEach(observer => observer());
  });
}

The final code should look something like the following.

function observable(obj) {
  Object.keys(obj).forEach((key) => {
    const getter = Object.getOwnPropertyDescriptor(obj, key).get;
    const propObservers = [];
    let _value = getter ? null : obj[key];

    Object.defineProperty(obj, key, {
      configurable: true,
      get() {
        if (currentObserver && !propObservers.includes(currentObserver)) {
          propObservers.push(currentObserver);
        }
        return _value;
      },
    });

    if (getter) {
      observe(() => {
        _value = getter.call(obj);
        propObservers.forEach(observer => observer());
      });
    } else {
      Object.defineProperty(obj, key, {
        set(value) {
          _value = value;
          propObservers.forEach(observer => observer());
        }
      });
    }
  });
  
  return obj;
}

Because we defined the getter and the setter separately, we have to set the configurable to true when we are registering the getter if we want to add a setter to it. Other than that, the code is practically identical to the original code. Also because the derived properties use the getter built for Proxies, not user-defined, it can handle serially derived properties as shown below.

const board = observable({
  score1: 10,
  score2: 40,
  get totalScore() {
    return this.score1 + this.score2;
  },
  get ratio1() {
    return this.score1 / this.totalScore;
  }
});

console.log(board.ratio1); // 0.2

board.score1 = 60;
console.log(board.ratio1); // 0.6

More Things to Consider

While the codes that we have written should suffice for a very general situation, there are still situations unaccounted for. Since it would be overly ambitious for me to explain both the entire context and all of the code in this single article, I will only explain only the most important aspects.

1. Excluded Code from the Initial Execution of observe Function

The observe function only saves the inputted function to the currentObserver when it is running for the first time. In other words, every observer can only by detected by the getter at the initial execution, so if the function has conditional statements inside itself, some of the code may not be included in the observable object’s property observer list.

const board = observable({
  score1: 10,
  score2: 20
});

observe(() => {
  if (board.score1 === 10) {
    console.log(`score1 : ${board.score1}`);
  } else {
    console.log(`score2 : ${board.score2}`);
  }
});
// score1 : 10

board.score1 = 20; // score2 : 20;

board.score2 = 30; // No Reaction

The observer function that we passed in to the observe accesses the board.score2 from the else block, but since this portion of the code is not executed when the code is first ran, any changes made to the board.score2 cannot be detected using what we have so far. One of the ways to address this problem is to configure the currentObserver every time the observer function is executed. Then, because the portion of the code that calls the includes with the intent of eliminating redundant observers can cause performance issues, it is better to use the Set instead of an array. If you cannot use the Set due to environment restrictions, you should assign a unique identifier to each observer, and use an object to manage the ids.

Secondly, because you have to check whether the observer exists every time, if you run into serially derived properties or if a chain of observer calls occur from inside of the observer, the current Observer could be set to null. To address this issue, you must use the array data type to manage the currentObserver as a stack.

2. unobserve Function

The observers that are assigned using the observe function remain until the targets are completely wiped from the memory. In this case, even if the target UI components are removed from the screen, it stays in the memory repeating unnecessary tasks. Therefore, the observe function must be able to return a unobserve (or dispose) function that can terminate the observation.

const board = observable({
  score: 10
});

const unobserve = observe(() => {
  console.log(board.score);
});
// 10

board.score = 20;  // 20

unobserve();
board.score = 30;  // No Reaction

As of now, we do not have a way to get rid of the observer from outside of the function because the propObservers array lives within the function scope. Furthermore, each observer function must be aware of all of the observer arrays that reference itself, so you have to create a management object that manages all of the observers within the module scope.

3. Reactivity on Arrays

I have already explained how the performance drops significantly when the reactivity systems are confronted with massive arrays. While we have tried different methods to account for it, we eventually decided not to use a reactivity system for the arrays in TOAST UI Grid. The cost of recreating an array with couple dozen elements is relatively insignificant, and most of the arrays have less than a hundred elements. To put it differently, it mostly does not matter that the arrays follow the reactivity if you update the object that has the array every time the array changes, the observer will react the same.

const data = obaservable({
  nums: [1, 2, 3],
  get squareNums() {
    return this.nums.map(num => num * num);
  }
});

console.log(squareNums); // [1, 4, 9]

data.nums = [...nums, 4];
console.log(squareNums); // [1, 4, 9, 16]

However, recreating a massive array still cannot be entirely ignored, since it could lead to detrimental effects on the overall performance. In order to address this problem, MobX and Vue use custom arrays that overrides built in methods like push and pop that internally calls the observers. In order to make sure that the program accesses the correct element of the array, Proxy getters are assigned to individual index identifiers. However, it is important to keep in mind that this method can be annoying to implement due to intricate relationships with numerous scenarios, and it could also negatively affect the performance.

For TOAST UI Grid, we decided to create a notify function, instead of structuring the entire array to adhere to the reactivity model, to forcefully call the observer functions to certain properties.

const data = observbable({
  nums: [1, 2, 3],
  get squareNums() {
    return this.nums.map(num => num * num);
  }
});

console.log(squareNums); // [1, 4, 9]

data.nums.push(4);
notify(data, 'num');

console.log(squareNums); // [1, 4, 9, 16]

It is true that since this notify function is manually called, it cannot be said to be in complete accordance with reactivity. However, this function is rarely used, and is only held out for those few cases where the program has to deal with a massive array. Therefore, in terms of exception handling, we believe it to be a reasonable choice.

Summary

What I have documented so far is, at its essence, what the TOAST UI Grid uses internally. While there are more features like cache handling and monolithic pure object return functions, since such features only take a little bit of the entire code, I have elected to ignore them.

The entire source code has been written in TypeScript, and is available on the Github repository. Excluding the type information, it is around 100 lines of code, 1.3KB when minified, and less than 0.7KB when compressed using Gzip. Compared to the 56KB of the minified MobX, it is a massive difference. Granted that it is less diverse in functionalities than MobX, for the purpose of TOAST UI Grid v4, these hundred lines of code have been working appropriately for us so far (not to mention that everyone on the team is content with the utility.)

In the world of programming, “you do not reinvent the wheel.” However, if the specific wheel that you are looking for is not in the pile, and you can create the wheel that you want at a reasonable cost, creating your own wheel is, in fact, the best option. It is still critical that you personally seek out new libraries and frameworks and learn how to use them, but nonetheless, I believe that understanding the underlying philosophies of those frameworks is more important. If you truly understand the core concept, you as a programmer can react more flexibly to wider range of conundrums, and would be able to implement more diverse functionalities.

We are still working diligently for the official release of TOAST UI Grid v4. Keep up to date with leaner and more customizeable Grid with Official Weekly and TOAST UI Twitter!