The hidden magic of Main Thread Scheduling
TL;DR:
Scheduling is a mechanism introduced lately by the React team to manage and prioritise tasks in the browser. It has become a case study for Google Chrome Dev team, to create a āMain Thread Schedulingā API.
This is groundbreaking work done by both of the teams. In this article Iāll talk about the APIs, how to use them, and how they will make a difference for our users!
If you were a part of the Frontend community for the past year and a half, the term āConcurrentā appears in almost every second tweet.
It all started with Dan Abramovās talk Beyond React 16 at JSConf Iceland 2018.
Dan showed how the React team built a generic way to ensure that high-priority updates donāt get blocked by a low-priority update. The React team called this mechanism āTime Slicingā and it was based on a Scheduler package they created.
This scheduler is called the āUserspace Schedulerā and was later used as a case study for Google Chrome developers for their work on a built-in scheduling mechanism.
The Problem
Letās take Lin Clarkās analogy from her talk in React Conf 2017 and compare our code to a Project Manager. Our project manager has 1 worker, the browser, but our worker is pretty busy, heās not 100% dedicated to our JavaScript code. It uses one thread to run our code, perform garbage collection, layout, paint, and more.
This issue buried the main problem: long-running JavaScript functions can block the thread and cause our worker to tip the balance and miss layout, paints, and more. This is a steep slope that immediately leads to an unresponsive page and a bad user experience.
The Solution
This problem is usually tackled by chunking and scheduling main thread work. In order to keep the browser responsive at all times, you break long tasks to smaller ones and yield back the control to the browser after an appropriate time. The time is calculated based on the current situation of the user and the browser.
But wait, how will I know to split the work based on time on my own? How do I even yield back control to the browser? š¤
To solve these problems we have Userspace Schedulers. So what are they?
Userspace Scheduler
A generic name for JS libraries built-in attempt to chunk up main thread work and schedule it at appropriate times. These libraries are responsible for doing the work and yielding the control back to the browser without blocking the main thread.
The main goal: Improve responsiveness and maintain high frame-rate.
The main examples are Reactās Scheduler package and Google Maps Scheduler.
These schedulers have been effective in improving responsiveness but they still have some issues, letās go over them:
- Determining when to yield to the browser ā Making intelligent decisions of when to yield is difficult with limited knowledge. As a Userspace scheduler, the scheduler is only aware of whatās happening in its own area.
The Reactās scheduler, for example, defaults to 30 FPS for every unit of work (which means around 1000ms/30=33.333ms) and adjusts it to higher a FPS rate if possible. Having said that, Reactās scheduler still checks between frames to see if thereās any user blocking task pending on the main thread and if there is, it yields back control to the browser. React does that by usingscheduling.isInputPending()
, we will talk about this function in the APIs section. - Regaining control after yielding ā When regaining control back from the browser, we will have to do the work of the same priority without returning back to the paused task until finishing the other work. That happens because we yield to the event loop and write a callback but there can already be callbacks waiting for that priority.
- Coordination between other tasks ā Since userspace schedulers donāt control all tasks on the page, their influence is limited. For example, the browser also has tasks to run on the main thread like garbage collection, layout, etc. and userspace schedulers canāt impact these tasks.
- Lack of API to schedule chunks of the script ā Developers can choose from
setTimeout
,postMessage
,requestAnimationFrame
, orrequestIdleCallback
, when choosing to schedule tasks. All of these have a different impact on the event loop and require a thorough knowledge of how it works.
The Reactās scheduler for example usessetTimeout
as shown here.
Main Thread Scheduling API
Since all current solutions have limitations, the Chrome team decided to create APIās for scheduling main thread work. These APIs are all gathered under the āMain-thread Scheduling APIā title and are currently an experimental feature not yet deployed to production nor beta version.
How can we try it?
To get the new Main Thread Scheduling APIs we need Chrome version 82.0.4084.0 and higher.
This version is available in Chromeās beta version or in Dev and Canary versions. I recommend downloading the Chrome Canary version since it can live alongside our current Chrome version. A download link can be found here.
Once downloaded, we need to turn on the feature-flag called Experimental web platform APIs
here: chrome://flags/#enable-experimental-web-platform-features
APIs
scheduler.yield
: When calling this function, we will yield to the event loop, such that a continuation task will run after the user agent services higher priority work, but before tasks of the same priority are allowed to run.
This function will return a Promise which will be resolved after the event loop services the higher priority work. We will also be able to specify a priority to the function scheduler.yield(āhighā)
stating we want control after tasks with this priority or higher were executed.
scheduling.isInputPending()
: This function will let us understand whether we have any pending input events waiting in the event loop and in that case we can yield back to the browser so it will handle these input events. This function is actually being used in Reactās Scheduler.
requestPostAnimationFrame
: This API isnāt implemented yet and is more of a draft API. This API will act as a bookend for the requestAnimationFrame
functionality, an āafter paintā callback.
To understand this ā The callback of requestAnimationFrame
runs just prior to the rendering and the callback of requestPostAnimationFrame
will run immediately after the rendering. This can be used to get a head-start on long running task and starting to create the next frame as soon as possible.
TaskController
: This API is the main API for controlling tasks, it contains a signal object with the following structure:
{ aborted: false, onabort: null, onprioritychange: null, priority: "user-visible" }
The TaskController
Object inherits its functionality from AbortController
and the signal inherits its functionality from AbortSignal
so when using it, we will be able to abort a task that wasnāt executed yet.
API looks like:
const controller = new TaskController(ābackgroundā)
and to get the signal we simply write controller.signal
.
scheduler.postTask
: This API can be used to post a task with a priority or a delay.
The postTask
function accepts a callback function and a signal. This signal can either be the one created from the TaskController
or just an object with priority property or delay priority containing a number.
The API shape is similar to other async APIās (fetch for example):
scheduler.postTask(callbackFunction, { priority: 'background' })
Itās important to note that creating a callback with background priority can also be done by using requestIdleCallback
. Having said that, posting and managing multiple priorities is much more complicated without these APIās.
There are more APIās that fall under the umbrella of Main Thread Scheduling but these are the ones I found important to note here.
Example
An important note is that Iām not using Reactās Concurrent Mode. Iām trying to show a solution based only on the new Scheduling API and not on Reactās Userspace scheduler (disclaimer: even the non Concurrent Mode React works with a scheduler but it doesnāt contain the time-slicing features).
Another small note, Iāve based my example project on Philipp Spiessās project for āScheduling in Reactā post.
Hereās a gif showing the app in action, try to look at all the details on the screen and what happens when I try to type:
On the screen, we see a header with an animation working with requestAnimationFrame
(rAF
), a search input, and a few pokemon (there are actually 200 rendered).
So why does it get stuck? š¤
What happens is as follows: on every keypress in the input, the whole pokemon list renders (I passed the search string to every pokemon so we will mark the search substring) and every pokemon has a synchronous timeout (a while loop of 2ms).
As we said, in my app I have 200 pokemon, leading each keypress to cause a render of about 400ms. To top it up, on the event handler I also simulated a synchronous heavy computation function of 25ms.
Letās look at a performance screenshot of whatās happening:
In the red box, you can see timings that I added or React added by itself.
The yellow box contains the call stack breakdown for each keypress.
Hereās a quick breakdown of whatās happening:
Every keypress leads to a long render (about 400ms), causing a Frame Drop(this can be inferred from the red triangle I wrapped with a blue circle).
Frame drop happens when the main thread is too busy with running our JavaScript code so it doesnāt get the chance to update the UI so the website freezes.
Before every render, in the timings section (the red box) we can see a small box I wrapped by green ovals, thatās our heavy computation function, it takes around 25ms as we can see in the tooltip. But sometimes, we donāt need that heavy computation to happen right away, maybe we can do it later. With Main Thread Scheduling API, we can do exactly that.
To do that, we need to change our event handler. At the moment it looks like this:
const onInputChange = (value) => { setInputValue(value); simulateHeavyComputation(value); };
Letās use postTask
and see the performance analysis:
const onInputChange = (value) => { setInputValue(value); scheduler.postTask(() => simulateHeavyComputation(value), { priority: "background", }); };
So what did we see?
Our heavy computation function now happens at the end (wrapped in the timings section with a green oval), after all the renders happen. The main difference is for our users, instead of waiting 3800ms for the UI to be interactive, they now wait 3600ms. Thatās still not so good but even 200ms is a difference.
Another amazing feature is to be able to cancel a task if it wasnāt executed yet.
In the previous example, we had a heavy computation function happening on every event, what if we would want it to happen only for the last event?
const onInputChange = (value) => { controller.abort(); controller = new TaskController("background"); const signal = controller.signal; setInputValue(value); scheduler.postTask( () => Promise.all([ fetch(`https://pokeapi.co/api/v2/pokemon/${value}`, { signal }), simularHeavyComputation(value), ]), { signal } ); };
So whatās happening here? hereās a quick explanation:
Weāre aborting the last signal we had and create a new TaskController every time we enter the event handler. That way we cancel all the tasks with the aborted signal attached to them. Below we can see the performance screenshot for this code, inside the timings section in a green oval we see that only the last task we created was executed.
Summing it up
We live in exciting times for the web development community. It looks like everyone involved truly aims for a better web and a better experience for our users.
I hope everyoneās feeling well and keeping themselves safe!
If you have any questions, Iām available on twitter, feel free to ask or comment, Iād love to hear your feedback!
Thanks for reading!
Matan.