Benedikt MeurerJavaScript Engine Hacker and Programming Language Enthusiast.2018-12-23T00:00:00-00:00Benedikt MeurerFrom Jekyll to eleventy2018-12-23T00:00:00-00:00https://benediktmeurer.de/2018/12/23/from-jekyll-to-eleventy/<p>I've been happily using the <a href="https://jekyllrb.com/">Jekyll</a> static site generator for <a href="https://benediktmeurer.de/">this website</a>
for almost 7½ years now, initially motivated by the excellent <a href="https://pages.github.com/">GitHub Pages</a>
service. Jekyll is awesome, since it gets you started easily without a lot of worrying about how to
setup lot's of different things.</p>
<p>But during the last month or so, <a href="https://mths.be/">Mathias</a> managed to convince me, that it makes
sense to dogfood our own software and the ecosystem around it - meaning it makes sense to look into
JavaScript based solutions. To be honest, I always felt like I should know more about how developers
use JavaScript first-hand. So I decided to give <a href="https://11ty.io/">eleventy</a> a try, which is also used
for the new <a href="https://v8.dev/">v8.dev</a> website.</p>
<p>I've been playing around with <a href="https://11ty.io/">eleventy</a> for about ten days now, and so far I'm quite
happy. It's not as well established as Jekyll, but it's fun to learn some basics of web development
and specifically JavaScript in the wild. The source code for my new website is available on
<a href="https://github.com/bmeurer/benediktmeurer.de">GitHub</a> like before, although it's no longer hosted on
GitHub Pages, but using <a href="https://netlify.com/">Netlify</a> right now. It's probably a bit overengineered
(i.e. using <a href="https://gulpjs.com/">gulp</a>, which is not strictly necessary), but as said, I'm using this
to learn more about JavaScript land. 😁</p>
<p>Source code: <a href="https://github.com/bmeurer/benediktmeurer.de">https://github.com/bmeurer/benediktmeurer.de</a></p>
Introducing jekyll-workbox-plugin2018-12-06T00:00:00-00:00https://benediktmeurer.de/2018/12/06/introducing-jekyll-workbox-plugin/<p>I spent some time today and hacked together a simple <a href="https://jekyllrb.com/">Jekyll</a> plugin to automatically
generate a service worker with <a href="https://developers.google.com/web/tools/workbox">Google Workbox</a>, with minimal
overhead / effort. I had previously used the <a href="https://github.com/lavas-project/jekyll-pwa">jekyll-pwa-plugin</a>,
which is awesome, but it's doing a bit too much for my taste:</p>
<ol>
<li>Local copy of the workbox distribution rather than fetching it from the Google CDN.</li>
<li>Generates both the actual service <code>sw.js</code> as well as a loader script <code>sw-register.js</code>, which also emits a
<code>sw.update</code> event, which I don't need.</li>
<li>Inserts a <code><script></code> snippet into each <code>.html</code> file generated that loads <code>sw-register.js</code>, prepending the
current timestamp to the URL to avoid caching.</li>
</ol>
<p>The <code>sw-register.js</code> itself also uses a timestamped URL to load the <code>sw.js</code> file. All of this is actually unnecessary
since I have configured my server to respond with <code>Cache-Control: no</code> to <code>sw.js</code> requests (and even that is no
longer necessary as <a href="https://developers.google.com/web/updates/2018/06/fresher-sw">I learned today</a>), plus I
already have a <code>main.js</code> that runs during page load, so I'd rather do the service worker registration there via
the common sequence:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token string">"serviceWorker"</span> <span class="token keyword">in</span> navigator<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> window<span class="token punctuation">.</span><span class="token function">addEventListener</span><span class="token punctuation">(</span><span class="token string">"load"</span><span class="token punctuation">,</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> navigator<span class="token punctuation">.</span>serviceWorker<span class="token punctuation">.</span><span class="token function">register</span><span class="token punctuation">(</span><span class="token string">"/sw.js"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>The only thing I actually need is the ability to do the precaching automatically and inject the right call to
<code>workbox.precaching.precacheAndRoute()</code> into the final <code>sw.js</code>, plus the convenience of adding the appropriate
<code>importScript()</code> call in the beginning.</p>
<p>So I created the <a href="https://github.com/bmeurer/jekyll-workbox-plugin">jekyll-workbox-plugin</a> (<a href="https://rubygems.org/gems/jekyll-workbox-plugin">ruby
gem</a>), which does that - and only that. I hope you'll find it
useful. You can find documentation regarding <a href="https://github.com/bmeurer/jekyll-workbox-plugin#installation">Installation</a>
and <a href="https://github.com/bmeurer/jekyll-workbox-plugin#getting-started">Getting Started</a> on the project page. In
a nutshell, you add <code>gem 'jekyll-workbox-plugin'</code> to the <code>jekyll_plugin</code> group in your <code>Gemfile</code></p>
<pre class="language-ruby"><code class="language-ruby"><span class="highlight-line">source <span class="token string">'https://rubygems.org'</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">gem <span class="token string">'jekyll'</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">group <span class="token symbol">:jekyll_plugins</span> <span class="token keyword">do</span></span><br><span class="highlight-line"> gem <span class="token string">'jekyll-workbox-plugin'</span></span><br><span class="highlight-line"><span class="token keyword">end</span></span></code></pre>
<p>and run <code>bundle</code> to install the gem. After the plugin has been installed successfully, add the following lines
to your <code>_config.yml</code> in order to tell Jekyll to use the plugin:</p>
<pre class="language-yaml"><code class="language-yaml"><span class="highlight-line"><span class="token key atrule">plugins</span><span class="token punctuation">:</span></span><br><span class="highlight-line"> <span class="token punctuation">-</span> jekyll<span class="token punctuation">-</span>workbox<span class="token punctuation">-</span>plugin</span></code></pre>
<p>Finally you need to create a template <code>sw.js</code> file, which looks something like this:</p>
<pre class="language-javascript"><code class="language-javascript"><span class="highlight-line"><span class="token comment">// sw.js</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token comment">// set names for both precache & runtime cache</span></span><br><span class="highlight-line">workbox<span class="token punctuation">.</span>core<span class="token punctuation">.</span><span class="token function">setCacheNameDetails</span><span class="token punctuation">(</span><span class="token punctuation">{</span></span><br><span class="highlight-line"> prefix<span class="token punctuation">:</span> <span class="token string">"my.site.tld"</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> suffix<span class="token punctuation">:</span> <span class="token string">"v1"</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> precache<span class="token punctuation">:</span> <span class="token string">"precache"</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> runtime<span class="token punctuation">:</span> <span class="token string">"runtime-cache"</span></span><br><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token comment">// let Workbox handle our precache list</span></span><br><span class="highlight-line"><span class="token comment">// NOTE: This will be populated by jekyll-workbox-plugin.</span></span><br><span class="highlight-line">workbox<span class="token punctuation">.</span>precaching<span class="token punctuation">.</span><span class="token function">precacheAndRoute</span><span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>It's important to have the <code>workbox.precaching.precacheAndRoute([])</code> in there, which <code>jekyll-workbox-plugin</code>
will automatically populate.</p>
<h2 id="update">Update <a class="bookmark" href="#update">#</a></h2>
<p>I've switched my <a href="https://benediktmeurer.de/">website</a> to use the <a href="https://11ty.io/">11ty</a> static site
generator instead of <a href="https://jekyllrb.com/">Jekyll</a>, mostly because I like to learn more about the
JavaScript ecosystem first-hand, while still using a static site generator. So I'm no longer using
this plugin myself.</p>
Faster async functions and promises2018-11-12T00:00:00-00:00https://benediktmeurer.de/2018/11/12/faster-async-functions-and-promises/<p>Asynchronous processing in JavaScript traditionally had a reputation for not being particularly fast. To make matters worse, debugging live JavaScript applications — in particular Node.js servers — is no easy task, <em>especially</em> when it comes to async programming. Luckily the times, they are a-changin’. This article explores how we optimized async functions and promises in V8 (and to some extent in other JavaScript engines as well), and describes how we improved the debugging experience for async code.</p>
<p><strong>Note:</strong> If you prefer watching a presentation over reading articles, then enjoy the video below! If not, skip the video and read on.</p>
<p><div class="embed-responsive embed-responsive-16by9"><iframe class="embed-responsive-item youtube-player" type="text/html" width="720" height="410" src="https://www.youtube.com/embed/DFP5DKDQfOc" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe></div></p>
<h2 id="a-new-approach-to-async-programming">A new approach to async programming <a class="bookmark" href="#a-new-approach-to-async-programming">#</a></h2>
<h3 id="from-callbacks-to-promises-to-async-functions">From callbacks to promises to async functions <a class="bookmark" href="#from-callbacks-to-promises-to-async-functions">#</a></h3>
<p>Before promises were part of the JavaScript language, callback-based APIs were commonly used for asynchronous code, especially in Node.js. Here’s an example:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">handler</span><span class="token punctuation">(</span><span class="token parameter">done</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">validateParams</span><span class="token punctuation">(</span><span class="token parameter">error</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span>error<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token function">done</span><span class="token punctuation">(</span>error<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token function">dbQuery</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">error<span class="token punctuation">,</span> dbResults</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span>error<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token function">done</span><span class="token punctuation">(</span>error<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token function">serviceCall</span><span class="token punctuation">(</span>dbResults<span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token parameter">error<span class="token punctuation">,</span> serviceResults</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>result<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token function">done</span><span class="token punctuation">(</span>error<span class="token punctuation">,</span> serviceResults<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>The specific pattern of using deeply-nested callbacks in this manner is commonly referred to as <em>“callback hell”</em>, because it makes the code less readable and hard to maintain.</p>
<p>Luckily, now that promises are part of the JavaScript language, the same code could be written in a more elegant and maintainable manner:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">handler</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token function">validateParams</span><span class="token punctuation">(</span><span class="token punctuation">)</span></span><br><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span>dbQuery<span class="token punctuation">)</span></span><br><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span>serviceCall<span class="token punctuation">)</span></span><br><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token parameter">result</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>result<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>Even more recently, JavaScript gained support for <a href="https://developers.google.com/web/fundamentals/primers/async-functions">async functions</a>. The above asynchronous code can now be written in a way that looks very similar to synchronous code:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">handler</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">await</span> <span class="token function">validateParams</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> dbResults <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">dbQuery</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> results <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">serviceCall</span><span class="token punctuation">(</span>dbResults<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>results<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> results<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>With async functions, the code becomes more succinct, and the control and data flow are a lot easier to follow, despite the fact that the execution is still asynchronous. (Note that the JavaScript execution still happens in a single thread, meaning async functions don’t end up creating physical threads themselves.)</p>
<h3 id="from-event-listener-callbacks-to-async-iteration">From event listener callbacks to async iteration <a class="bookmark" href="#from-event-listener-callbacks-to-async-iteration">#</a></h3>
<p>Another asynchronous paradigm that’s especially common in Node.js is that of <a href="https://nodejs.org/api/stream.html#stream_readable_streams"><code>ReadableStream</code>s</a>. Here’s an example:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> http <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">"http"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">http</span><br><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">createServer</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">req<span class="token punctuation">,</span> res</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">let</span> body <span class="token operator">=</span> <span class="token string">""</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> req<span class="token punctuation">.</span><span class="token function">setEncoding</span><span class="token punctuation">(</span><span class="token string">"utf8"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> req<span class="token punctuation">.</span><span class="token function">on</span><span class="token punctuation">(</span><span class="token string">"data"</span><span class="token punctuation">,</span> <span class="token parameter">chunk</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> body <span class="token operator">+=</span> chunk<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> req<span class="token punctuation">.</span><span class="token function">on</span><span class="token punctuation">(</span><span class="token string">"end"</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> res<span class="token punctuation">.</span><span class="token function">write</span><span class="token punctuation">(</span>body<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> res<span class="token punctuation">.</span><span class="token function">end</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span></span><br><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">listen</span><span class="token punctuation">(</span><span class="token number">1337</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>This code can be a little hard to follow: the incoming data is processed in chunks that are only accessible within callbacks, and the end-of-stream signaling happens inside a callback too. It’s easy to introduce bugs here when you don’t realize that the function terminates immediately and that the actual processing has to happen in the callbacks.</p>
<p>Fortunately, a cool new ES2018 feature called <a href="http://2ality.com/2016/10/asynchronous-iteration.html">async iteration</a> can simplify this code:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> http <span class="token operator">=</span> <span class="token function">require</span><span class="token punctuation">(</span><span class="token string">"http"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">http</span><br><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">createServer</span><span class="token punctuation">(</span><span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token parameter">req<span class="token punctuation">,</span> res</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">try</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">let</span> body <span class="token operator">=</span> <span class="token string">""</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> req<span class="token punctuation">.</span><span class="token function">setEncoding</span><span class="token punctuation">(</span><span class="token string">"utf8"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token keyword">await</span> <span class="token punctuation">(</span><span class="token keyword">const</span> chunk <span class="token keyword">of</span> req<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> body <span class="token operator">+=</span> chunk<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"> res<span class="token punctuation">.</span><span class="token function">write</span><span class="token punctuation">(</span>body<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> res<span class="token punctuation">.</span><span class="token function">end</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> res<span class="token punctuation">.</span>statusCode <span class="token operator">=</span> <span class="token number">500</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> res<span class="token punctuation">.</span><span class="token function">end</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span></span><br><span class="highlight-line"> <span class="token punctuation">.</span><span class="token function">listen</span><span class="token punctuation">(</span><span class="token number">1337</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Instead of putting the logic that deals with the actual request processing into two different callbacks — the <code>'data'</code> and the <code>'end'</code> callback — we can now put everything into a single async function instead, and use the new <code>for await…of</code> loop to iterate over the chunks asynchronously. We also added a <code>try-catch</code> block to avoid the <code>unhandledRejection</code> problem<sup class="footnote-ref"><a href="#fn1" id="fnref1">[1]</a></sup>.</p>
<p>You can already use these new features in production today! Async functions are <strong>fully supported starting with Node.js 8 (V8 v6.2 / Chrome 62)</strong>, and async iterators and generators are <strong>fully supported starting with Node.js 10 (V8 v6.8 / Chrome 68)</strong>!</p>
<h2 id="async-performance-improvements">Async performance improvements <a class="bookmark" href="#async-performance-improvements">#</a></h2>
<p>We’ve managed to improve the performance of asynchronous code significantly between V8 v5.5 (Chrome 55 & Node.js 7) and V8 v6.8 (Chrome 68 & Node.js 10). We reached a level of performance where developers can safely use these new programming paradigms without having to worry about speed.</p>
<p><img src="https://benediktmeurer.de/images/2018/doxbee-benchmark-20181112.svg" alt="doxbee benchmark results" title="doxbee benchmark results"></p>
<p>The above chart shows the <a href="https://github.com/v8/promise-performance-tests/blob/master/lib/doxbee-async.js">doxbee benchmark</a>, which measures performance of promise-heavy code. Note that the charts visualize execution time, meaning lower is better.</p>
<p>The results on the <a href="https://github.com/v8/promise-performance-tests/blob/master/lib/parallel-async.js">parallel benchmark</a>, which specifically stresses the performance of <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all"><code>Promise.all()</code></a>, are even more exciting:</p>
<p><img src="https://benediktmeurer.de/images/2018/parallel-benchmark-20181112.svg" alt="parallel benchmark results" title="parallel benchmark results"></p>
<p>We’ve managed to improve <code>Promise.all</code> performance by a factor of <strong>8×</strong>.</p>
<p>However, the above benchmarks are synthetic micro-benchmarks. The V8 team is more interested in how our optimizations affect <a href="https://v8.dev/blog/real-world-performance">real-world performance of actual user code</a>.</p>
<p><img src="https://benediktmeurer.de/images/2018/http-benchmarks-20181112.svg" alt="http benchmark results" title="http benchmark results"></p>
<p>The above chart visualizes the performance of some popular HTTP middleware frameworks that make heavy use of promises and <code>async</code> functions. Note that this graph shows the number of requests/second, so unlike the previous charts, higher is better. The performance of these frameworks improved significantly between Node.js 7 (V8 v5.5) and Node.js 10 (V8 v6.8).</p>
<p>These performance improvements are the result of three key achievements:</p>
<ul>
<li><a href="https://v8.dev/docs/turbofan">TurboFan</a>, the new optimizing compiler 🎉</li>
<li><a href="https://v8.dev/blog/orinoco">Orinoco</a>, the new garbage collector 🚛</li>
<li>a Node.js 8 bug causing <code>await</code> to skip microticks 🐛</li>
</ul>
<p>When we <a href="https://v8.dev/blog/launching-ignition-and-turbofan">launched TurboFan</a> in <a href="https://medium.com/the-node-js-collection/node-js-8-3-0-is-now-available-shipping-with-the-ignition-turbofan-execution-pipeline-aa5875ad3367">Node.js 8</a>, that gave a huge performance boost across the board.</p>
<p>We’ve also been working on a new garbage collector, called Orinoco, which moves garbage collection work off the main thread, and thus improves request processing significantly as well.</p>
<p>And last but not least, there was a handy bug in Node.js 8 that caused <code>await</code> to skip microticks in some cases, resulting in better performance. The bug started out as an unintended spec violation, but it later gave us the idea for an optimization. Let’s start by explaining the buggy behavior:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> p <span class="token operator">=</span> Promise<span class="token punctuation">.</span><span class="token function">resolve</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token punctuation">(</span><span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">await</span> p<span class="token punctuation">;</span></span><br><span class="highlight-line"> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">"after:await"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">p<span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">"tick:a"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token string">"tick:b"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>The above program creates a fulfilled promise <code>p</code>, and <code>await</code>s its result, but also chains two handlers onto it. In which order would you expect the <code>console.log</code> calls to execute?</p>
<p>Since <code>p</code> is fulfilled, you might expect it to print <code>'after:await'</code> first and then the <code>'tick'</code>s. In fact, that’s the behavior you’d get in Node.js 8:</p>
<p><img src="https://benediktmeurer.de/images/2018/await-bug-node-8-20181112.svg" alt="The await bug in Node.js 8" title="The await bug in Node.js 8"></p>
<p>Although this behavior seems intuitive, it’s not correct according to the specification. Node.js 10 implements the correct behavior, which is to first execute the chained handlers, and only afterwards continue with the async function.</p>
<p><img src="https://benediktmeurer.de/images/2018/await-bug-node-10-20181112.svg" alt="Node.js 10 no longer has the await bug" title="Node.js 10 no longer has the await bug"></p>
<p>This <em>“correct behavior”</em> is arguably not immediately obvious, and was actually surprising to JavaScript developers, so it deserves some explanation. Before we dive into the magical world of promises and async functions, let’s start with some of the foundations.</p>
<h3 id="tasks-vs.-microtasks">Tasks vs. microtasks <a class="bookmark" href="#tasks-vs.-microtasks">#</a></h3>
<p>On a high level there are <em>tasks</em> and <em>microtasks</em> in JavaScript. Tasks handle events like I/O and timers, and execute one at a time. Microtasks implement deferred execution for <code>async</code>/<code>await</code> and promises, and execute at the end of each task. The microtask queue is always emptied before execution returns to the event loop.</p>
<p><img src="https://benediktmeurer.de/images/2018/microtasks-vs-tasks-20181112.svg" alt="The difference between microtasks and tasks" title="The difference between microtasks and tasks"></p>
<p>For more details, check out Jake Archibald’s explanation of <a href="https://jakearchibald.com/2015/tasks-microtasks-queues-and-schedules/">tasks, microtasks, queues, and schedules in the browser</a>. The task model in Node.js is very similar.</p>
<h3 id="async-functions">Async functions <a class="bookmark" href="#async-functions">#</a></h3>
<p>According to MDN, an async function is a function which operates asynchronously using an implicit promise to return its result. Async functions are intended to make asynchronous code look like synchronous code, hiding some of the complexity of the asynchronous processing from the developer.</p>
<p>The simplest possible async function looks like this:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">computeAnswer</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token number">42</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>When called it returns a promise, and you can get to its value like with any other promise.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> p <span class="token operator">=</span> <span class="token function">computeAnswer</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// → Promise</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">p<span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span>console<span class="token punctuation">.</span>log<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// prints 42 on the next turn</span></span></code></pre>
<p>You only get to the value of this promise <code>p</code> the next time microtasks are run. In other words, the above program is semantically equivalent to using <code>Promise.resolve</code> with the value:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">computeAnswer</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> Promise<span class="token punctuation">.</span><span class="token function">resolve</span><span class="token punctuation">(</span><span class="token number">42</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>The real power of async functions comes from <code>await</code> expressions, which cause the function execution to pause until a promise is resolved, and resume after fulfillment. The value of <code>await</code> is that of the fulfilled promise. Here’s an example showing what that means:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">fetchStatus</span><span class="token punctuation">(</span><span class="token parameter">url</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> response <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token function">fetch</span><span class="token punctuation">(</span>url<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> response<span class="token punctuation">.</span>status<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>The execution of <code>fetchStatus</code> gets suspended on the <code>await</code>, and is later resumed when the <code>fetch</code> promise fulfills. This is more or less equivalent to chaining a handler onto the promise returned from <code>fetch</code>.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">fetchStatus</span><span class="token punctuation">(</span><span class="token parameter">url</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token function">fetch</span><span class="token punctuation">(</span>url<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token parameter">response</span> <span class="token operator">=></span> response<span class="token punctuation">.</span>status<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>That handler contains the code following the <code>await</code> in the async function.</p>
<p>Normally you’d pass a <code>Promise</code> to <code>await</code>, but you can actually wait on any arbitrary JavaScript value. If the value of the expression following the <code>await</code> is not a promise, it’s converted to a promise. That means you can <code>await 42</code> if you feel like doing that:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> v <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token number">42</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> v<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> p <span class="token operator">=</span> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// → Promise</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">p<span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span>console<span class="token punctuation">.</span>log<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// prints `42` eventually</span></span></code></pre>
<p>More interestingly, <code>await</code> works with any <a href="https://promisesaplus.com/">“thenable”</a>, i.e. any object with a <code>then</code> method, even if it’s not a real promise. So you can implement funny things like an asynchronous sleep that measures the actual time spent sleeping:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">Sleep</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token parameter">timeout</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">this</span><span class="token punctuation">.</span>timeout <span class="token operator">=</span> timeout<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"> <span class="token function">then</span><span class="token punctuation">(</span><span class="token parameter">resolve<span class="token punctuation">,</span> reject</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> startTime <span class="token operator">=</span> Date<span class="token punctuation">.</span><span class="token function">now</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token function">setTimeout</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token function">resolve</span><span class="token punctuation">(</span>Date<span class="token punctuation">.</span><span class="token function">now</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span> startTime<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token keyword">this</span><span class="token punctuation">.</span>timeout<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token punctuation">(</span><span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> actualTime <span class="token operator">=</span> <span class="token keyword">await</span> <span class="token keyword">new</span> <span class="token class-name">Sleep</span><span class="token punctuation">(</span><span class="token number">1000</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>actualTime<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Let’s see what V8 does for <code>await</code> under the hood, following the <a href="https://tc39.github.io/ecma262/#await">specification</a>. Here’s a simple async function <code>foo</code>:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token parameter">v</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> w <span class="token operator">=</span> <span class="token keyword">await</span> v<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> w<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>When called, it wraps the parameter <code>v</code> into a promise and suspends execution of the async function until that promise is resolved. Once that happens, execution of the function resumes and <code>w</code> gets assigned the value of the fulfilled promise. This value is then returned from the async function.</p>
<h3 id="await-under-the-hood"><code>await</code> under the hood <a class="bookmark" href="#await-under-the-hood">#</a></h3>
<p>First of all, V8 marks this function as <em>resumable</em>, which means that execution can be suspended and later resumed (at <code>await</code> points). Then it creates the so-called <code>implicit_promise</code>, which is the promise that is returned when you invoke the async function, and that eventually resolves to the value produced by the async function.</p>
<p><img src="https://benediktmeurer.de/images/2018/await-under-the-hood-20181112.svg" alt="Comparison between a simple async function and what the engine turns it into" title="Comparison between a simple async function and what the engine turns it into"></p>
<p>Then comes the interesting bit: the actual <code>await</code>. First the value passed to <code>await</code> is wrapped into a promise. Then, handlers are attached to this wrapped promise to resume the function once the promise is fulfilled, and execution of the async function is suspended, returning the <code>implicit_promise</code> to the caller. Once the <code>promise</code> is fulfilled, execution of the async function is resumed with the value <code>w</code> from the <code>promise</code>, and the <code>implicit_promise</code> is resolved with <code>w</code>.</p>
<p>In a nutshell, the initial steps for <code>await v</code> are:</p>
<ol>
<li>Wrap <code>v</code> — the value passed to <code>await</code> — into a promise.</li>
<li>Attach handlers for resuming the async function later.</li>
<li>Suspend the async function and return the <code>implicit_promise</code> to the caller.</li>
</ol>
<p>Let’s go through the individual operations step by step. Assume that the thing that is being <code>await</code>ed is already a promise, which was fulfilled with the value <code>42</code>. Then the engine creates a new <code>promise</code> and resolves that with whatever’s being <code>await</code>ed. This does deferred chaining of these promises on the next turn, expressed via what the specification calls a <a href="https://tc39.github.io/ecma262/#sec-promiseresolvethenablejob"><code>PromiseResolveThenableJob</code></a>.</p>
<p><img src="https://benediktmeurer.de/images/2018/await-step-1-20181112.svg" alt="await step 1" title="await step 1"></p>
<p>Then the engine creates another so-called <code>throwaway</code> promise. It’s called <em>throwaway</em> because nothing is ever chained to it — it’s completely internal to the engine. This <code>throwaway</code> promise is then chained onto the <code>promise</code>, with appropriate handlers to resume the async function. This <code>performPromiseThen</code> operation is essentially what <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/then"><code>Promise.prototype.then()</code></a> does, behind the scenes. Finally, execution of the async function is suspended, and control returns to the caller.</p>
<p><img src="https://benediktmeurer.de/images/2018/await-step-2-20181112.svg" alt="await step 2" title="await step 2"></p>
<p>Execution continues in the caller, and eventually the call stack becomes empty. Then the JavaScript engine starts running the microtasks: it runs the previously scheduled <a href="https://tc39.github.io/ecma262/#sec-promiseresolvethenablejob"><code>PromiseResolveThenableJob</code></a>, which schedules a new <a href="https://tc39.github.io/ecma262/#sec-promisereactionjob"><code>PromiseReactionJob</code></a> to chain the <code>promise</code> onto the value passed to <code>await</code>. Then, the engine returns to processing the microtask queue, since the microtask queue must be emptied before continuing with the main event loop.</p>
<p><img src="https://benediktmeurer.de/images/2018/await-step-3-20181112.svg" alt="await step 3" title="await step 3"></p>
<p>Next up is the <a href="https://tc39.github.io/ecma262/#sec-promisereactionjob"><code>PromiseReactionJob</code></a>, which fulfills the <code>promise</code> with the value from the promise we’re <code>await</code>ing — <code>42</code> in this case — and schedules the reaction onto the <code>throwaway</code> promise. The engine then returns to the microtask loop again, which contains a final microtask to be processed.</p>
<p><img src="https://benediktmeurer.de/images/2018/await-step-4-final-20181112.svg" alt="await step 4" title="await step 4"></p>
<p>Now this second <a href="https://tc39.github.io/ecma262/#sec-promisereactionjob"><code>PromiseReactionJob</code></a> propagates the resolution to the <code>throwaway</code> promise, and resumes the suspended execution of the async function, returning the value <code>42</code> from the <code>await</code>.</p>
<p><img src="https://benediktmeurer.de/images/2018/await-overhead-20181112.svg" alt="Summary of the overhead of await" title="Summary of the overhead of await"></p>
<p>Summarizing what we’ve learned, for each <code>await</code> the engine has to create <strong>two additional</strong> promises (even if the right hand side is already a promise) and it needs <strong>at least three</strong> microtask queue ticks. Who knew that a single <code>await</code> expression resulted in <em>that much overhead</em>?!</p>
<p><img src="https://benediktmeurer.de/images/2018/await-code-before-20181112.svg" alt="Original code for await" title="Original code for await"></p>
<p>Let’s have a look at where this overhead comes from. The first line is responsible for creating the wrapper promise. The second line immediately resolves that wrapper promise with the <code>await</code>ed value <code>v</code>. These two lines are responsible for one additional promise plus two out of the three microticks. That’s quite expensive if <code>v</code> is already a promise (which is the common case, since applications normally <code>await</code> on promises). In the unlikely case that a developer <code>await</code>s on e.g. <code>42</code>, the engine still needs to wrap it into a promise.</p>
<p>As it turns out, there’s already a <a href="https://tc39.github.io/ecma262/#sec-promise-resolve"><code>promiseResolve</code></a> operation in the specification that only performs the wrapping when needed:</p>
<p><img src="https://benediktmeurer.de/images/2018/await-code-comparison-20181112.svg" alt="Code improvements for await" title="Code improvements for await"></p>
<p>This operation returns promises unchanged, and only wraps other values into promises as necessary. This way you save one of the additional promises, plus two ticks on the microtask queue, for the common case that the value passed to <code>await</code> is already a promise. This new behavior is currently implemented behind the <code>--harmony-await-optimization</code> flag in V8 (starting with V8 v7.1). We’ve <a href="https://github.com/tc39/ecma262/pull/1250">proposed this change to the ECMAScript specification</a> as well; the patch is supposed to be merged once we are sure that it’s web-compatible.</p>
<p>Here’s how the new and improved <code>await</code> works behind the scenes, step by step:</p>
<p><img src="https://benediktmeurer.de/images/2018/await-new-step-1-20181112.svg" alt="New await step 1" title="New await step 1"></p>
<p>Let’s assume again that we <code>await</code> a promise that was fulfilled with <code>42</code>. Thanks to the magic of <a href="https://tc39.github.io/ecma262/#sec-promise-resolve"><code>promiseResolve</code></a> the <code>promise</code> now just refers to the same promise <code>v</code>, so there’s nothing to do in this step. Afterwards the engine continues exactly like before, creating the <code>throwaway</code> promise, scheduling a <a href="https://tc39.github.io/ecma262/#sec-promisereactionjob"><code>PromiseReactionJob</code></a> to resume the async function on the next tick on the microtask queue, suspending execution of the function, and returning to the caller.</p>
<p><img src="https://benediktmeurer.de/images/2018/await-new-step-2-20181112.svg" alt="New await step 2" title="New await step 2"></p>
<p>Then eventually when all JavaScript execution finishes, the engine starts running the microtasks, so it executes the <a href="https://tc39.github.io/ecma262/#sec-promisereactionjob"><code>PromiseReactionJob</code></a>. This job propagates the resolution of <code>promise</code> to <code>throwaway</code>, and resumes the execution of the async function, yielding <code>42</code> from the <code>await</code>.</p>
<p><img src="https://benediktmeurer.de/images/2018/await-overhead-removed-20181112.svg" alt="Summary of the reduction in await overhead" title="Summary of the reduction in await overhead"></p>
<p>This optimization avoids the need to create a wrapper promise if the value passed to <code>await</code> is already a promise, and in that case we go from a minimum of <strong>three</strong> microticks to just <strong>one</strong> microtick. This behavior is similar to what Node.js 8 does, except that now it’s no longer a bug — it’s now an optimization that is being standardized!</p>
<p>It still feels wrong that the engine has to create this <code>throwaway</code> promise, despite being completely internal to the engine. As it turns out, the <code>throwaway</code> promise was only there to satisfy the API constraints of the internal <code>performPromiseThen</code> operation in the spec.</p>
<p><img src="https://benediktmeurer.de/images/2018/await-optimized-20181112.svg" alt="await optimized" title="await optimized"></p>
<p>This was recently addressed in an <a href="https://github.com/tc39/ecma262/issues/694">editorial change</a> to the ECMAScript specification. Engines no longer need to create the <code>throwaway</code> promise for <code>await</code> — most of the time<sup class="footnote-ref"><a href="#fn2" id="fnref2">[2]</a></sup>.</p>
<p><img src="https://benediktmeurer.de/images/2018/node-10-vs-node-12-20181112.svg" alt="Comparison of await code before and after the optimizations" title="Comparison of await code before and after the optimizations"></p>
<p>Comparing <code>await</code> in Node.js 10 to the optimized <code>await</code> that’s likely going to be in Node.js 12 shows the performance impact of this change:</p>
<p><img src="https://benediktmeurer.de/images/2018/benchmark-optimization-20181112.svg" alt="Anticipated Node.js 12 performance improvements" title="Anticipated Node.js 12 performance improvements"></p>
<p><strong><code>async</code>/<code>await</code> outperforms hand-written promise code now</strong>. The key takeaway here is that we significantly reduced the overhead of async functions — not just in V8, but across all JavaScript engines, by patching the spec<sup class="footnote-ref"><a href="#fn3" id="fnref3">[3]</a></sup>.</p>
<h2 id="improved-developer-experience">Improved developer experience <a class="bookmark" href="#improved-developer-experience">#</a></h2>
<p>In addition to performance, JavaScript developers also care about the ability to diagnose and fix problems, which is not always easy when dealing with asynchronous code. <a href="https://developers.google.com/web/tools/chrome-devtools">Chrome DevTools</a> supports <em>async stack traces</em>, i.e. stack traces that not only include the current synchronous part of the stack, but also the asynchronous part:</p>
<figure>
<img src="https://benediktmeurer.de/images/2018/devtools-20181112.png" srcset="https://benediktmeurer.de/images/2018/devtools-20181112@2x.png 2x" title="Async stack traces in Chrome DevTools" alt="Async Stack Traces in Chome DevTools">
</figure>
<p>This is an incredibly useful feature during local development. However, this approach doesn’t really help you once the application is deployed. During post-mortem debugging, you’ll only see the <code>Error#stack</code> output in your log files, and that doesn’t tell you anything about the asynchronous parts.</p>
<p>We’ve recently been working on <a href="https://bit.ly/v8-zero-cost-async-stack-traces"><em>zero-cost async stack traces</em></a> which enrich the <code>Error#stack</code> property with async function calls. “Zero-cost” sounds exciting, doesn’t it? How can it be zero-cost, when the Chrome DevTools feature comes with major overhead? Consider this example where <code>foo</code> calls <code>bar</code> asynchronously, and <code>bar</code> throws an exception after <code>await</code>ing a promise:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">await</span> <span class="token function">bar</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token number">42</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">async</span> <span class="token keyword">function</span> <span class="token function">bar</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">await</span> Promise<span class="token punctuation">.</span><span class="token function">resolve</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">throw</span> <span class="token keyword">new</span> <span class="token class-name">Error</span><span class="token punctuation">(</span><span class="token string">"BEEP BEEP"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">catch</span><span class="token punctuation">(</span><span class="token parameter">error</span> <span class="token operator">=></span> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>error<span class="token punctuation">.</span>stack<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Running this code in Node.js 8 or Node.js 10 results in the following output:</p>
<pre class="language-text"><code class="language-text"><span class="highlight-line">$ node index.js</span><br><span class="highlight-line">Error: BEEP BEEP</span><br><mark class="highlight-line highlight-line-active"> at bar (index.js:8:9)</mark><br><span class="highlight-line"> at process._tickCallback (internal/process/next_tick.js:68:7)</span><br><span class="highlight-line"> at Function.Module.runMain (internal/modules/cjs/loader.js:745:11)</span><br><span class="highlight-line"> at startup (internal/bootstrap/node.js:266:19)</span><br><span class="highlight-line"> at bootstrapNodeJSCore (internal/bootstrap/node.js:595:3)</span></code></pre>
<p>Note that although the call to <code>foo()</code> causes the error, <code>foo</code> is not part of the stack trace at all. This makes it tricky for JavaScript developers to perform post-mortem debugging, independent of whether your code is deployed in a web application or inside of some cloud container.</p>
<p>The interesting bit here is that the engine knows where it has to continue when <code>bar</code> is done: right after the <code>await</code> in function <code>foo</code>. Coincidentally, that’s also the place where the function <code>foo</code> was suspended. The engine can use this information to reconstruct parts of the asynchronous stack trace, namely the <code>await</code> sites. With this change, the output becomes:</p>
<pre class="language-text"><code class="language-text"><span class="highlight-line">$ node --async-stack-traces index.js</span><br><span class="highlight-line">Error: BEEP BEEP</span><br><mark class="highlight-line highlight-line-active"> at bar (index.js:8:9)</mark><br><span class="highlight-line"> at process._tickCallback (internal/process/next_tick.js:68:7)</span><br><span class="highlight-line"> at Function.Module.runMain (internal/modules/cjs/loader.js:745:11)</span><br><span class="highlight-line"> at startup (internal/bootstrap/node.js:266:19)</span><br><span class="highlight-line"> at bootstrapNodeJSCore (internal/bootstrap/node.js:595:3)</span><br><mark class="highlight-line highlight-line-active"> at async foo (index.js:2:3)</mark></code></pre>
<p>In the stack trace, the topmost function comes first, followed by the rest of the synchronous stack trace, followed by the asynchronous call to <code>bar</code> in function <code>foo</code>. This change is implemented in V8 behind the new <code>--async-stack-traces</code> flag.</p>
<p>However, if you compare this to the async stack trace in Chrome DevTools above, you’ll notice that the actual call site to <code>foo</code> is missing from the asynchronous part of the stack trace. As mentioned before, this approach utilizes the fact that for <code>await</code> the resume and suspend locations are the same — but for regular <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/then"><code>Promise#then()</code></a> or <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/catch"><code>Promise#catch()</code></a> calls, this is not the case. For more background, see Mathias Bynens’s explanation on <a href="https://mathiasbynens.be/notes/async-stack-traces">why <code>await</code> beats <code>Promise#then()</code></a>.</p>
<h2 id="conclusion">Conclusion <a class="bookmark" href="#conclusion">#</a></h2>
<p>We made async functions faster thanks to two significant optimizations:</p>
<ul>
<li>the removal of two extra microticks, and</li>
<li>the removal of the <code>throwaway</code> promise.</li>
</ul>
<p>On top of that, we’ve improved the developer experience via <a href="https://bit.ly/v8-zero-cost-async-stack-traces"><em>zero-cost async stack traces</em></a>, which work with <code>await</code> in async functions and <code>Promise.all()</code>.</p>
<p>And we also have some nice performance advice for JavaScript developers:</p>
<ul>
<li>favor <code>async</code> functions and <code>await</code> over hand-written promise code, and</li>
<li>stick to the native promise implementation offered by the JavaScript engine to benefit from the shortcuts, i.e. avoiding two microticks for <code>await</code>.</li>
</ul>
<hr class="footnotes-sep">
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn1" class="footnote-item"><p>Thanks to <a href="https://twitter.com/matteocollina">Matteo Collina</a> for pointing us to <a href="https://github.com/mcollina/make-promises-safe/blob/master/README.md#the-unhandledrejection-problem">this issue</a>. <a href="#fnref1" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn2" class="footnote-item"><p>V8 still needs to create the <code>throwaway</code> promise if <a href="https://nodejs.org/api/async_hooks.html"><code>async_hooks</code></a> are being used in Node.js, since the <code>before</code> and <code>after</code> hooks are run within the <em>context</em> of the <code>throwaway</code> promise. <a href="#fnref2" class="footnote-backref">↩︎</a></p>
</li>
<li id="fn3" class="footnote-item"><p>As mentioned, <a href="https://github.com/tc39/ecma262/pull/1250">the patch</a> hasn’t been merged into the ECMAScript specification just yet. The plan is to do so once we’ve made sure that the change doesn’t break the web. <a href="#fnref3" class="footnote-backref">↩︎</a></p>
</li>
</ol>
</section>
Improving DataView performance in V82018-09-18T00:00:00-00:00https://benediktmeurer.de/2018/09/18/improving-dataview-performance-in-v8/<p><a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView"><code>DataView</code>s</a> are one of the two possible ways to do low-level memory accesses in JavaScript, the other one being <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray"><code>TypedArray</code>s</a>. Up until now, <code>DataView</code>s were much less optimized than <code>TypedArray</code>s in V8, resulting in lower performance on tasks such as graphics-intensive workloads or when decoding/encoding binary data. The reasons for this have been mostly historical choices, like the fact that <a href="http://asmjs.org/">asm.js</a> chose <code>TypedArray</code>s instead of <code>DataView</code>s, and so engines were incentivized to focus on performance of <code>TypedArray</code>s.</p>
<p>Because of the performance penalty, JavaScript developers such as the Google Maps team decided to avoid <code>DataView</code>s and rely on <code>TypedArray</code>s instead, at the cost of increased code complexity. This article explains how we brought <code>DataView</code> performance to match — and even surpass — equivalent <code>TypedArray</code> code in <a href="https://v8.dev/blog/v8-release-69">V8 v6.9</a>, effectively making <code>DataView</code> usable for performance-critical real-world applications.</p>
<h2 id="background">Background <a class="bookmark" href="#background">#</a></h2>
<p>Since the introduction of ES2015, JavaScript has supported reading and writing data in raw binary buffers called <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer"><code>ArrayBuffer</code>s</a>. <code>ArrayBuffer</code>s cannot be directly accessed; rather, programs must use a so-called <em>array buffer view</em> object that can be either a <code>DataView</code> or a <code>TypedArray</code>.</p>
<p><code>TypedArray</code>s allow programs to access the buffer as an array of uniformly typed values, such as an <code>Int16Array</code> or a <code>Float32Array</code>.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> buffer <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">ArrayBuffer</span><span class="token punctuation">(</span><span class="token number">32</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">const</span> array <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Int16Array</span><span class="token punctuation">(</span>buffer<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">let</span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> i <span class="token operator"><</span> array<span class="token punctuation">.</span>length<span class="token punctuation">;</span> i<span class="token operator">++</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> array<span class="token punctuation">[</span>i<span class="token punctuation">]</span> <span class="token operator">=</span> i <span class="token operator">*</span> i<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>array<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// → [0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121, 144, 169, 196, 225]</span></span></code></pre>
<p>On the other hand, <code>DataView</code>s allow for more fine-grained data access. They let the programmer choose the type of values read from and written to the buffer by providing specialized getters and setters for each number type, making them useful for serializing data structures.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> buffer <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">ArrayBuffer</span><span class="token punctuation">(</span><span class="token number">32</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">const</span> view <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">DataView</span><span class="token punctuation">(</span>buffer<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> person <span class="token operator">=</span> <span class="token punctuation">{</span> age<span class="token punctuation">:</span> <span class="token number">42</span><span class="token punctuation">,</span> height<span class="token punctuation">:</span> <span class="token number">1.76</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">view<span class="token punctuation">.</span><span class="token function">setUint8</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span> person<span class="token punctuation">.</span>age<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line">view<span class="token punctuation">.</span><span class="token function">setFloat64</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> person<span class="token punctuation">.</span>height<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>view<span class="token punctuation">.</span><span class="token function">getUint8</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// Expected output: 42</span></span><br><span class="highlight-line">console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>view<span class="token punctuation">.</span><span class="token function">getFloat64</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// Expected output: 1.76</span></span></code></pre>
<p>Moreover, <code>DataView</code>s also allow the choice of the endianness of the data storage, which can be useful when receiving data from external sources such as the network, a file, or a GPU.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> buffer <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">ArrayBuffer</span><span class="token punctuation">(</span><span class="token number">32</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">const</span> view <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">DataView</span><span class="token punctuation">(</span>buffer<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">view<span class="token punctuation">.</span><span class="token function">setInt32</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span> <span class="token number">0x8badf00d</span><span class="token punctuation">,</span> <span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// Little-endian write.</span></span><br><span class="highlight-line">console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>view<span class="token punctuation">.</span><span class="token function">getInt32</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span> <span class="token boolean">false</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// Big-endian read.</span></span><br><span class="highlight-line"><span class="token comment">// Expected output: 0x0DF0AD8B (233876875)</span></span></code></pre>
<p>An efficient <code>DataView</code> implementation has been a feature request for a long time (see <a href="https://crbug.com/225811">this bug report</a> from over 5 years ago), and we are happy to announce that DataView performance is now on par!</p>
<h2 id="legacy-runtime-implementation">Legacy runtime implementation <a class="bookmark" href="#legacy-runtime-implementation">#</a></h2>
<p>Until recently, the <code>DataView</code> methods used to be implemented as built-in C++ runtime functions in V8. This is very costly, because each call would require an expensive transition from JavaScript to C++ (and back).</p>
<p>In order to investigate the actual performance cost incurred by this implementation, we set up a performance benchmark that compares the native <code>DataView</code> getter implementation with a JavaScript wrapper simulating <code>DataView</code> behavior. This wrapper uses an <code>Uint8Array</code> to read data byte by byte from the underlying buffer, and then computes the return value from those bytes. Here is, for example, the function for reading little-endian 32-bit unsigned integer values:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">LittleEndian</span><span class="token punctuation">(</span><span class="token parameter">buffer</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token comment">// Simulate little-endian DataView reads.</span></span><br><span class="highlight-line"> <span class="token keyword">this</span><span class="token punctuation">.</span>uint8View_ <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Uint8Array</span><span class="token punctuation">(</span>buffer<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token class-name">LittleEndian</span><span class="token punctuation">.</span>prototype<span class="token punctuation">.</span><span class="token function-variable function">getUint32</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token parameter">byteOffset</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token punctuation">(</span></span><br><span class="highlight-line"> <span class="token keyword">this</span><span class="token punctuation">.</span>uint8View_<span class="token punctuation">[</span>byteOffset<span class="token punctuation">]</span> <span class="token operator">|</span></span><br><span class="highlight-line"> <span class="token punctuation">(</span><span class="token keyword">this</span><span class="token punctuation">.</span>uint8View_<span class="token punctuation">[</span>byteOffset <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">8</span><span class="token punctuation">)</span> <span class="token operator">|</span></span><br><span class="highlight-line"> <span class="token punctuation">(</span><span class="token keyword">this</span><span class="token punctuation">.</span>uint8View_<span class="token punctuation">[</span>byteOffset <span class="token operator">+</span> <span class="token number">2</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">16</span><span class="token punctuation">)</span> <span class="token operator">|</span></span><br><span class="highlight-line"> <span class="token punctuation">(</span><span class="token keyword">this</span><span class="token punctuation">.</span>uint8View_<span class="token punctuation">[</span>byteOffset <span class="token operator">+</span> <span class="token number">3</span><span class="token punctuation">]</span> <span class="token operator"><<</span> <span class="token number">24</span><span class="token punctuation">)</span></span><br><span class="highlight-line"> <span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<p><code>TypedArray</code>s are already heavily optimized in V8, so they represent the performance goal that we wanted to match.</p>
<figure>
<img src="https://benediktmeurer.de/images/2018/dataview-original-20180918.svg" alt="Original DataView performance" title="Original DataView performance">
<figcaption>Original <code>DataView</code> performance</figcaption>
</figure>
<p>Our benchmark shows that native <code>DataView</code> getter performance was as much as <strong>4 times</strong> slower than the <code>Uint8Array</code>-based wrapper, for both big-endian and little-endian reads.</p>
<h2 id="improving-baseline-performance">Improving baseline performance <a class="bookmark" href="#improving-baseline-performance">#</a></h2>
<p>Our first step in improving the performance of <code>DataView</code> objects was to move the implementation from the C++ runtime to <a href="https://v8.dev/blog/csa"><code>CodeStubAssembler</code> (also known as CSA)</a>. CSA is a portable assembly language that allows us to write code directly in TurboFan’s machine-level intermediate representation (IR), and we use it to implement optimized parts of V8’s JavaScript standard library. Rewriting code in CSA bypasses the call to C++ completely, and also generates efficient machine code by leveraging TurboFan’s backend.</p>
<p>However, writing CSA code by hand is cumbersome. Control flow in CSA is expressed much like in assembly, using explicit labels and <code>goto</code>s, which makes the code harder to read and understand at a glance.</p>
<p>In order to make it easier for developers to contribute to the optimized JavaScript standard library in V8, and to improve readability and maintainability, we started designing a new language called V8 <em>Torque</em>, that compiles down to CSA. The goal for <em>Torque</em> is to abstract away the low-level details that make CSA code harder to write and maintain, while retaining the same performance profile.</p>
<p>Rewriting the <code>DataView</code> code was an excellent opportunity to start using Torque for new code, and helped provide the Torque developers with a lot of feedback about the language. This is what the <code>DataView</code>’s <code>getUint32()</code> method looks like, written in Torque:</p>
<pre class="language-torque"><code class="language-torque"><span class="highlight-line"><span class="token keyword">macro</span> LoadDataViewUint32<span class="token punctuation">(</span>buffer<span class="token class-name">: JSArrayBuffer,</span> offset<span class="token class-name">: intptr,</span></span><br><span class="highlight-line"> requested_little_endian<span class="token class-name">: bool,</span></span><br><span class="highlight-line"> signed<span class="token class-name">: <span class="token keyword">constexpr</span> bool)</span><span class="token class-name">: Number {</span></span><br><span class="highlight-line"> <span class="token keyword">let</span> data_pointer<span class="token class-name">: RawPtr =</span> buffer<span class="token punctuation">.</span>backing_store<span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"> <span class="token keyword">let</span> b0<span class="token class-name">: uint32 =</span> LoadUint8<span class="token punctuation">(</span>data_pointer<span class="token punctuation">,</span> offset<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">let</span> b1<span class="token class-name">: uint32 =</span> LoadUint8<span class="token punctuation">(</span>data_pointer<span class="token punctuation">,</span> offset <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">let</span> b2<span class="token class-name">: uint32 =</span> LoadUint8<span class="token punctuation">(</span>data_pointer<span class="token punctuation">,</span> offset <span class="token operator">+</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">let</span> b3<span class="token class-name">: uint32 =</span> LoadUint8<span class="token punctuation">(</span>data_pointer<span class="token punctuation">,</span> offset <span class="token operator">+</span> <span class="token number">3</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">let</span> result<span class="token class-name">: uint32;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span>requested_little_endian<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> result <span class="token operator">=</span> <span class="token punctuation">(</span>b3 <span class="token operator"><</span><span class="token operator"><</span> <span class="token number">24</span><span class="token punctuation">)</span> <span class="token operator">|</span> <span class="token punctuation">(</span>b2 <span class="token operator"><</span><span class="token operator"><</span> <span class="token number">16</span><span class="token punctuation">)</span> <span class="token operator">|</span> <span class="token punctuation">(</span>b1 <span class="token operator"><</span><span class="token operator"><</span> <span class="token number">8</span><span class="token punctuation">)</span> <span class="token operator">|</span> b0<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> result <span class="token operator">=</span> <span class="token punctuation">(</span>b0 <span class="token operator"><</span><span class="token operator"><</span> <span class="token number">24</span><span class="token punctuation">)</span> <span class="token operator">|</span> <span class="token punctuation">(</span>b1 <span class="token operator"><</span><span class="token operator"><</span> <span class="token number">16</span><span class="token punctuation">)</span> <span class="token operator">|</span> <span class="token punctuation">(</span>b2 <span class="token operator"><</span><span class="token operator"><</span> <span class="token number">8</span><span class="token punctuation">)</span> <span class="token operator">|</span> b3<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"> <span class="token keyword">return</span> convert<span class="token operator"><</span>Number<span class="token operator">></span><span class="token punctuation">(</span>result<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>Moving the <code>DataView</code> methods to Torque already showed a <strong>3× improvement</strong> in performance, but did not quite match <code>Uint8Array</code>-based wrapper performance yet.</p>
<figure>
<img src="https://benediktmeurer.de/images/2018/dataview-torque-20180918.svg" alt="Torque DataView performance" title="Torque DataView performance">
<figcaption>Torque <code>DataView</code> performance</figcaption>
</figure>
<h2 id="optimizing-for-turbofan">Optimizing for TurboFan <a class="bookmark" href="#optimizing-for-turbofan">#</a></h2>
<p>When JavaScript code gets hot, we compile it using our TurboFan optimizing compiler, in order to generate highly-optimized machine code that runs more efficiently than interpreted bytecode.</p>
<p>TurboFan works by translating the incoming JavaScript code into an internal graph representation (more precisely, <a href="https://darksi.de/d.sea-of-nodes/">a “sea of nodes”</a>). It starts with high-level nodes that match the JavaScript operations and semantics, and gradually refines them into lower and lower level nodes, until it finally generates machine code.</p>
<p>In particular, a function call, such as calling one of the <code>DataView</code> methods, is internally represented as a <code>JSCall</code> node, which eventually boils down to an actual function call in the generated machine code.</p>
<p>However, TurboFan allows us to check whether the <code>JSCall</code> node is actually a call to a known function, for example one of the builtin functions, and inline this node in the IR. This means that the complicated <code>JSCall</code> gets replaced at compile-time by a subgraph that represents the function. This allows TurboFan to optimize the inside of the function in subsequent passes as part of a broader context, instead of on its own, and most importantly to get rid of the costly function call.</p>
<figure>
<img src="https://benediktmeurer.de/images/2018/dataview-turbofan-initial-20180918.svg" alt="Initial TurboFan DataView performance" title="Initial TurboFan DataView performance">
<figcaption>Initial TurboFan <code>DataView</code> performance</figcaption>
</figure>
<p>Implementing TurboFan inlining finally allowed us to match, and even exceed, the performance of our <code>Uint8Array</code> wrapper, and be <strong>8 times</strong> as fast as the former C++ implementation.</p>
<h2 id="further-turbofan-optimizations">Further TurboFan optimizations <a class="bookmark" href="#further-turbofan-optimizations">#</a></h2>
<p>Looking at the machine code generated by TurboFan after inlining the <code>DataView</code> methods, there was still room for some improvement. The first implementation of those methods tried to follow the standard pretty closely, and threw errors when the spec indicates so (for example, when trying to read or write out of the bounds of the underlying <code>ArrayBuffer</code>).</p>
<p>However, the code that we write in TurboFan is meant to be optimized to be as fast as possible for the common, hot cases — it doesn’t need to support every possible edge case. By removing all the intricate handling of those errors, and just deoptimizing back to the baseline Torque implementation when we need to throw, we were able to reduce the size of the generated code by around 35%, generating a quite noticeable speedup, as well as considerably simpler TurboFan code.</p>
<p>Following up on this idea of being as specialized as possible in TurboFan, we also removed support for indices or offsets that are too large (outside of Smi range) inside the TurboFan-optimized code. This allowed us to get rid of handling of the float64 arithmetic that is needed for offsets that do not fit into a 32-bit value, and to avoid storing large integers on the heap.</p>
<p>Compared to the initial TurboFan implementation, this more than doubled the <code>DataView</code> benchmark score. <code>DataView</code>s are now up to 3 times as fast as the <code>Uint8Array</code> wrapper, and around <strong>16 times as fast</strong> as our original <code>DataView</code> implementation!</p>
<figure>
<img src="https://benediktmeurer.de/images/2018/dataview-turbofan-final-20180918.svg" alt="Final TurboFan DataView performance" title="Final TurboFan DataView performance">
<figcaption>Final TurboFan <code>DataView</code> performance</figcaption>
</figure>
<h2 id="impact">Impact <a class="bookmark" href="#impact">#</a></h2>
<p>We’ve evaluated the performance impact of the new implementation on some real-world examples, on top of our own benchmark.</p>
<p><code>DataView</code>s are often used when decoding data encoded in binary formats from JavaScript. One such binary format is <a href="https://en.wikipedia.org/wiki/FBX">FBX</a>, a format that is used for exchanging 3D animations. We’ve instrumented the FBX loader of the popular <a href="https://threejs.org/">three.js</a> JavaScript 3D library, and measured a 10% (around 80 ms) reduction in its execution time.</p>
<p>We compared the overall performance of <code>DataView</code>s against <code>TypedArray</code>s. We found that our new <code>DataView</code> implementation provides almost the same performance as <code>TypedArray</code>s when accessing data aligned in the native endianness (little-endian on Intel processors), bridging much of the performance gap and making <code>DataView</code>s a practical choice in V8.</p>
<figure>
<img src="https://benediktmeurer.de/images/2018/dataview-vs-typedarray-20180918.svg" alt="DataView vs. TypedArray peak performance" title="DataView vs. TypedArray peak performance">
<figcaption><code>DataView</code> vs. <code>TypedArray</code> peak performance</figcaption>
</figure>
<p>We hope that you’re now able to start using <code>DataView</code>s where it makes sense, instead of relying on <code>TypedArray</code> shims. Please send us feedback on your <code>DataView</code> uses! You can reach us <a href="https://crbug.com/v8/new">via our bug tracker</a>, via mail to <a href="mailto:v8-users@googlegroups.com">v8-users@googlegroups.com</a>, or via <a href="https://twitter.com/v8js">@v8js on Twitter</a>.</p>
JavaScript engine fundamentals: optimizing prototypes2018-08-16T00:00:00-00:00https://benediktmeurer.de/2018/08/16/javascript-engine-fundamentals-optimizing-prototypes/<p>This article describes some key fundamentals that are common to all JavaScript engines — and not just <a href="https://twitter.com/v8js">V8</a>, the engine the authors (<a href="https://twitter.com/mathias">Mathias</a> and <a href="https://twitter.com/bmeurer">Benedikt</a>) work on. As a JavaScript developer, having a deeper understanding of how JavaScript engines work helps you reason about the performance characteristics of your code.</p>
<p><a href="https://benediktmeurer.de/2018/06/14/javascript-engine-fundamentals-shapes-and-inline-caches/">Previously</a>, we discussed how JavaScript engines optimize object and array access through the use of Shapes and Inline Caches. This article explains optimization pipeline trade-offs and describes how engines speed up accesses to prototype properties.</p>
<p><strong>Note:</strong> If you prefer watching a presentation over reading articles, then enjoy the video below! If not, skip the video and read on.</p>
<p><div class="embed-responsive embed-responsive-16by9"><iframe class="embed-responsive-item youtube-player" type="text/html" width="720" height="410" src="https://www.youtube.com/embed/IFWulQnM5E0" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe></div></p>
<h2 id="optimization-tiers-and-execution-trade-offs">Optimization tiers and execution trade-offs <a class="bookmark" href="#optimization-tiers-and-execution-trade-offs">#</a></h2>
<p><a href="https://benediktmeurer.de/2018/06/14/javascript-engine-fundamentals-shapes-and-inline-caches/">Our previous article</a> discussed how modern JavaScript engines all have the same overall pipeline:</p>
<p><img src="https://benediktmeurer.de/images/2018/js-engine-pipeline-20180816.svg" alt="To execute JavaScript, engines use an interpreter and one or more optimization tiers" title="To execute JavaScript, engines use an interpreter and one or more optimization tiers"></p>
<p>We also pointed out that although the high-level pipeline is similar between engines, there are often differences in the optimization pipeline. Why is that? <strong>Why do some engines have more optimization tiers than others?</strong> It turns out there is a trade-off between quickly getting code to run, or taking some more time but eventually running the code with optimal performance.</p>
<p><img src="https://benediktmeurer.de/images/2018/tradeoff-startup-speed-20180816.svg" alt="Tradeoff between startup and execution speed" title="Tradeoff between startup and execution speed"></p>
<p>An interpreter can produce bytecode quickly, but bytecode is generally not very efficient. An optimizing compiler on the other hand takes a little longer, but eventually produces much more efficient machine code.</p>
<p>This is exactly the model that V8 uses. V8's interpreter is called Ignition, and it's the fastest interpreter of all the engines (in terms of raw bytecode execution speed). V8's optimizing compiler is named TurboFan, and it eventually generates highly-optimized machine code.</p>
<p><img src="https://benediktmeurer.de/images/2018/tradeoff-startup-speed-v8-20180816.svg" alt="Tradeoff between startup and execution speed in V8" title="Tradeoff between startup and execution speed in V8"></p>
<p>This trade-off between startup latency and execution speed is the reason why some JavaScript engines choose to add optimization tiers in between. For example, SpiderMonkey adds a Baseline tier in between the interpreter and their full IonMonkey optimizing compiler:</p>
<p><img src="https://benediktmeurer.de/images/2018/tradeoff-startup-speed-spidermonkey-20180816.svg" alt="Tradeoff between startup and execution speed in SpiderMonkey" title="Tradeoff between startup and execution speed in SpiderMonkey"></p>
<p>nterpreter generates bytecode quickly, but the bytecode executes relatively slowly. Baseline takes a little longer to generate code, but it offers better run-time performance. And finally, the IonMonkey optimizing compiler takes the longest to produce machine code, but that code can run very efficiently.</p>
<p>Let's take a look at a concrete example and see how the pipelines in the different engines deal with it. Here’s some code that gets repeated often, in a hot loop.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">let</span> result <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">let</span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> i <span class="token operator"><</span> <span class="token number">4242424242</span><span class="token punctuation">;</span> <span class="token operator">++</span>i<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> result <span class="token operator">+=</span> i<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line">console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>result<span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>In V8 it starts running the bytecode in the Ignition interpreter. At some point the engine determines that the code is <em>hot</em> and starts up the TurboFan frontend, which is the part of TurboFan that deals with integrating profiling data and constructing a basic machine representation of the code. This is then sent to the TurboFan optimizer on a different thread for further improvements of the code.</p>
<p><img src="https://benediktmeurer.de/images/2018/pipeline-detail-v8-20180816.svg" alt="Overview of the V8 pipeline" title="Overview of the V8 pipeline"></p>
<p>While the optimizer is running, V8 continues executing the bytecode in Ignition. At some point the optimizer is done and we have executable machine code, and the execution can continue with that.</p>
<p>The SpiderMonkey engine also starts running the bytecode in the interpreter. But it has the additional Baseline tier, which means hot code is first sent to Baseline. The Baseline compiler generates Baseline code on the main thread and continues execution once ready.</p>
<p><img src="https://benediktmeurer.de/images/2018/pipeline-detail-spidermonkey-20180816.svg" alt="Overview of the SpiderMonkey pipeline" title="Overview of the SpiderMonkey pipeline"></p>
<p>If Baseline code is run for a while, SpiderMonkey eventually fires up the IonMonkey frontend, and kicks off the optimizer — very similarly to V8. It keeps running in Baseline while IonMonkey is optimizing. Finally, when the optimizer is done, the optimized code is executed instead of the Baseline code.</p>
<p>Chakra's architecture is very similar to SpiderMonkey's, but Chakra tries to run more things concurrently to avoid blocking the main thread. Instead of running any part of the compiler on the main thread, Chakra copies out the bytecode and the profiling data that the compiler likely needs and sends it to a dedicated compiler process.</p>
<p><img src="https://benediktmeurer.de/images/2018/pipeline-detail-chakra-20180816.svg" alt="Overview of the Chakra pipeline" title="Overview of the Chakra pipeline"></p>
<p>When the generated code is ready, the engine starts to run this SimpleJIT code instead of the bytecode. The same goes for the FullJIT. The benefit of this approach is that the pause times where the copy happens are usually much shorter compared to running the full compiler (frontend). But the downside of this approach is that the <em>copy heuristic</em> might miss some information that would be required for a certain optimization, so it's trading code quality for latency to some extent.</p>
<p>In JavaScriptCore, all optimizing compilers run <strong>fully concurrent</strong> with the main JavaScript execution; there's no copy phase! Instead, the main thread merely triggers compilation jobs on another thread. The compilers then use a complicated locking scheme to access profiling data from the main thread.</p>
<p><img src="https://benediktmeurer.de/images/2018/pipeline-detail-javascriptcore-20180816.svg" alt="Overview of the JavaScriptCore pipeline" title="Overview of the JavaScriptCore pipeline"></p>
<p>The advantage of this approach is that it reduces the jank caused by JavaScript optimization on the main thread. The downside is that it requires dealing with complex multithreading issues and paying some locking cost for various operations.</p>
<p>We've talked about the trade-off between generating code quickly like with an interpreter, or generating quick code with an optimizing compiler. But there is another trade-off: <strong>memory usage</strong>! To illustrate that, here's a simple JavaScript program that adds two numbers together.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">add</span><span class="token punctuation">(</span><span class="token parameter">x<span class="token punctuation">,</span> y</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> x <span class="token operator">+</span> y<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token function">add</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Let's take a look at the bytecode we generate for the <code>add</code> function using the Ignition interpreter in V8.</p>
<pre><code>StackCheck
Ldar a1
Add a0, [0]
Return
</code></pre>
<p>Don’t worry about the exact bytecode — you don’t really need to read it. The point is that it’s <strong>just four instructions</strong>!</p>
<p>When the code becomes hot, TurboFan generates the following highly-optimized machine code:</p>
<pre><code>leaq rcx,[rip+0x0]
movq rcx,[rcx-0x37]
testb [rcx+0xf],0x1
jnz CompileLazyDeoptimizedCode
push rbp
movq rbp,rsp
push rsi
push rdi
cmpq rsp,[r13+0xe88]
jna StackOverflow
movq rax,[rbp+0x18]
test al,0x1
jnz Deoptimize
movq rbx,[rbp+0x10]
testb rbx,0x1
jnz Deoptimize
movq rdx,rbx
shrq rdx, 32
movq rcx,rax
shrq rcx, 32
addl rdx,rcx
jo Deoptimize
shlq rdx, 32
movq rax,rdx
movq rsp,rbp
pop rbp
ret 0x18
</code></pre>
<p>That's a <strong>lot of code</strong>, especially when compared to the four instructions we had in the bytecode! In general, bytecode tends to be a lot more compact than machine code, especially optimized machine code. On the other hand, bytecode needs an interpreter to run, whereas the optimized code can be executed directly by the processor.</p>
<p>This is one of the main reasons why JavaScript engines don't just <em>"optimize everything"</em>. As we saw earlier, generating optimized machine code takes a long time, and on top of that, we just learned that <strong>optimized machine code also requires more memory</strong>.</p>
<p><img src="https://benediktmeurer.de/images/2018/tradeoff-memory-20180816.svg" alt="Optimization level vs memory usage" title="Optimization level vs memory usage"></p>
<p><strong>Summary:</strong> The reason JS engines have different optimization tiers is because of a fundamental trade-off between <em>generating code quickly</em> like with an interpreter, or <em>generating quick code</em> with an optimizing compiler. It's a scale, and adding more optimization tiers allows you to make more fine-grained decisions at the cost of additional complexity and overhead. In addition, there's a trade-off between the optimization level and the memory usage of the generated code. This is why JS engines try to optimize only <em>hot</em> functions.</p>
<h2 id="optimizing-prototype-property-access">Optimizing prototype property access <a class="bookmark" href="#optimizing-prototype-property-access">#</a></h2>
<p><a href="https://benediktmeurer.de/2018/06/14/javascript-engine-fundamentals-shapes-and-inline-caches/#optimizing-property-access">Our previous article</a> explained how JavaScript engines optimize object property loads using Shapes and Inline Caches. To recap, engines store the <code>Shape</code> of the object separately from the object's values.</p>
<p><img src="https://benediktmeurer.de/images/2018/shape-2-20180816.svg" alt="Object shapes" title="Object shapes"></p>
<p>Shapes enable an optimization called <em>Inline Caches</em> or <em>ICs</em> for short. Combined, Shapes and ICs can speed up repeated property accesses from the same place in your code.</p>
<p><img src="https://benediktmeurer.de/images/2018/ic-4-20180816.svg" alt="Shapes and Inline Caches" title="Shapes and Inline Caches"></p>
<h3 id="classes-and-prototype-based-programming">Classes and prototype-based programming <a class="bookmark" href="#classes-and-prototype-based-programming">#</a></h3>
<p>Now that we know how to make property access fast on JavaScript objects, let's look at one of the more recent additions to JavaScript: classes. Here's what the JavaScript class syntax looks like:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">Bar</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token parameter">x</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">this</span><span class="token punctuation">.</span>x <span class="token operator">=</span> x<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"> <span class="token function">getX</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">.</span>x<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>Although this appears to be a new concept in JavaScript, it's merely syntactic sugar for prototype-based programming that has been used in JavaScript forever:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">Bar</span><span class="token punctuation">(</span><span class="token parameter">x</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">this</span><span class="token punctuation">.</span>x <span class="token operator">=</span> x<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token class-name">Bar</span><span class="token punctuation">.</span>prototype<span class="token punctuation">.</span><span class="token function-variable function">getX</span> <span class="token operator">=</span> <span class="token keyword">function</span> <span class="token function">getX</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">.</span>x<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<p>Here, we assign a <code>getX</code> property on the <code>Bar.prototype</code> object. This works in exactly the same way as with any other object, because <strong>prototypes are just objects in JavaScript</strong>! In prototype-based programming languages like JavaScript, methods are shared via the prototypes, while fields are stored on the actual instances.</p>
<p>Let’s zoom in on what happens behind the scenes when we create a new instance of <code>Bar</code> called <code>foo</code>.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> foo <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Bar</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>The instance created from running this code has a shape with a single property <code>'x'</code>. The prototype of <code>foo</code> is the <code>Bar.prototype</code> that belongs to the class <code>Bar</code>.</p>
<p><img src="https://benediktmeurer.de/images/2018/class-shape-1-20180816.svg" alt="Classes and Shapes Part 1" title="Classes and Shapes Part 1"></p>
<p>This <code>Bar.prototype</code> has a shape of its own, containing a single property <code>'getX'</code> whose value is the function <code>getX</code> that just returns <code>this.x</code> when called. The prototype of <code>Bar.prototype</code> is the <code>Object.prototype</code> that's part of the JavaScript language. The <code>Object.prototype</code> is the root of the prototype tree and thus its prototype is <code>null</code>.</p>
<p><img src="https://benediktmeurer.de/images/2018/class-shape-2-20180816.svg" alt="Classes and Shapes Part 2" title="Classes and Shapes Part 2"></p>
<p>If you create another instance of the same class, both instances share the object shape, as we discussed earlier. Both instances point to the same <code>Bar.prototype</code> object.</p>
<h3 id="prototype-property-access">Prototype property access <a class="bookmark" href="#prototype-property-access">#</a></h3>
<p>Ok, so now we know what happens when we define a class and we create a new instance. But what happens if we call a method on an instance, like we're doing here?</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">Bar</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token parameter">x</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">this</span><span class="token punctuation">.</span>x <span class="token operator">=</span> x<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"> <span class="token function">getX</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">.</span>x<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> foo <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Bar</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">const</span> x <span class="token operator">=</span> foo<span class="token punctuation">.</span><span class="token function">getX</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// ^^^^^^^^^^</span></span></code></pre>
<p>You can think of any method call as two individual steps:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> x <span class="token operator">=</span> foo<span class="token punctuation">.</span><span class="token function">getX</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token comment">// is actually two steps:</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> $getX <span class="token operator">=</span> foo<span class="token punctuation">.</span>getX<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">const</span> x <span class="token operator">=</span> <span class="token function">$getX</span><span class="token punctuation">.</span><span class="token function">call</span><span class="token punctuation">(</span>foo<span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Step 1 is to load the method, which is just a property on the prototype (whose value happens to be a function). Step 2 is to call the function with the instance as the <code>this</code> value. Let's walk through that first step, which is loading the method <code>getX</code> from the instance <code>foo</code>.</p>
<p><img src="https://benediktmeurer.de/images/2018/method-load-20180816.svg" alt="Method load" title="Method load"></p>
<p>The engine starts at the <code>foo</code> instance and realizes there is no <code>'getX'</code> property on <code>foo</code>s shape, so it has to walk up the prototype chain for it. We get to <code>Bar.prototype</code>, look at its prototype shape, and see that it has the <code>'getX'</code> property at offset <code>0</code>. We look up the value at this offset in <code>Bar.prototype</code> and find the <code>JSFunction</code> <code>getX</code> that we were looking for. And that’s it!</p>
<p>JavaScript's flexibility makes it possible to mutate prototype chain links, for example:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> foo <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Bar</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line">foo<span class="token punctuation">.</span><span class="token function">getX</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// → true</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">Object<span class="token punctuation">.</span><span class="token function">setPrototypeOf</span><span class="token punctuation">(</span>foo<span class="token punctuation">,</span> <span class="token keyword">null</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line">foo<span class="token punctuation">.</span><span class="token function">getX</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// Uncaught TypeError: foo.getX is not a function</span></span></code></pre>
<p>In this example, we call <code>foo.getX()</code> twice, but each time it has a completely different meaning and result. This is why, although prototypes are just objects in JavaScript, speeding up prototype property access is even more challenging for JavaScript engines than speeding up <em>own</em> property access on regular objects.</p>
<p>Looking at programs in the wild, loading prototype properties is a very frequent operation: it happens every time you call a method!</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">Bar</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token parameter">x</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">this</span><span class="token punctuation">.</span>x <span class="token operator">=</span> x<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"> <span class="token function">getX</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">.</span>x<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> foo <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Bar</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">const</span> x <span class="token operator">=</span> foo<span class="token punctuation">.</span><span class="token function">getX</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Earlier, we discussed how engines optimize loading regular, <em>own</em> properties through the use of Shapes and Inline Caches. How can we optimize repeated loads of prototype properties on objects with similar shapes? We saw above how the property load happens.</p>
<p><img src="https://benediktmeurer.de/images/2018/prototype-load-checks-1-20180816.svg" alt="Prototype load checks" title="Prototype load checks"></p>
<p>n order to make that fast for repeated loads in this particular case, we need to know these three things:</p>
<p>The shape of <code>foo</code> does not contain <code>'getX'</code> and did not change. This means no one altered the object <code>foo</code> by adding or deleting a property, or by changing one of the property attributes.
The prototype of <code>foo</code> is still the initial <code>Bar.prototype</code>. This means no one changed <code>foo</code>s prototype by using <code>Object.setPrototypeOf()</code> or by assigning to the special <code>__proto__</code> property.
The shape of <code>Bar.prototype</code> contains <code>'getX'</code> and did not change. This means no one altered the <code>Bar.prototype</code> by adding or deleting a property, or by changing one of the property attributes.</p>
<p>In the general case, that means we have to perform 1 check on the instance itself, plus 2 checks for each prototype up to the prototype which holds the property we're looking for. <code>1+2N</code>checks (where <code>N</code> is the number of prototypes involved) may not sound too bad for this case, because the prototype chain is relatively shallow. But oftentimes engines have to deal with much longer prototype chains, like in the case of common DOM classes. Here’s an example of that:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> anchor <span class="token operator">=</span> document<span class="token punctuation">.</span><span class="token function">createElement</span><span class="token punctuation">(</span><span class="token string">"a"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// → HTMLAnchorElement</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> title <span class="token operator">=</span> anchor<span class="token punctuation">.</span><span class="token function">getAttribute</span><span class="token punctuation">(</span><span class="token string">"title"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>We have an <code>HTMLAnchorElement</code> and we call the <code>getAttribute()</code> method on it. Looking at the prototype chain of a simple anchor element, we can see that there are already 6 prototypes involved. And a lot of the interesting DOM methods are not on the direct prototypes, but even higher up in the chain.</p>
<p><img src="https://benediktmeurer.de/images/2018/anchor-prototype-chain-20180816.svg" alt="HTMLAnchorElement prototype chain" title="HTMLAnchorElement prototype chain"></p>
<p>The <code>getAttribute()</code> method is found on the <code>Element.prototype</code>. That means each time we call <code>anchor.getAttribute()</code>, the JavaScript engine needs to</p>
<ol>
<li>check that <code>'getAttribute'</code> is not on the <code>anchor</code> object itself,</li>
<li>check that the direct prototype is <code>HTMLAnchorElement.prototype</code>,</li>
<li>assert absence of <code>'getAttribute'</code> there,</li>
<li>check that the next prototype is <code>HTMLElement.prototype</code>,</li>
<li>assert absence of <code>'getAttribute'</code> there as well,</li>
<li>eventually check that the next prototype is <code>Element.prototype</code>,</li>
<li>and that <code>'getAttribute'</code> is present there.</li>
</ol>
<p>That’s a total of 7 checks! Since this kind of code is pretty common on the web, engines apply tricks to reduce the number of checks necessary for property loads on prototypes.</p>
<p>Going back to the earlier example, we perform a total of 3 checks when accessing <code>'getX'</code> on <code>foo</code>:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">Bar</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token parameter">x</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">this</span><span class="token punctuation">.</span>x <span class="token operator">=</span> x<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"> <span class="token function">getX</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">.</span>x<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> foo <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Bar</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">const</span> $getX <span class="token operator">=</span> foo<span class="token punctuation">.</span>getX<span class="token punctuation">;</span></span></code></pre>
<p>For each object involved up until the prototype that carries the property, we need to do shape checks for absence. It'd be nice if we could reduce the number of checks by folding the prototype check into the absence check. And that's essentially what engines do with a simple trick: <strong>instead of storing the prototype link on the instance itself, engines store it on the <code>Shape</code></strong>.</p>
<p><img src="https://benediktmeurer.de/images/2018/prototype-load-checks-2-20180816.svg" alt="Improved prototype load checks" title="Improved prototype load checks"></p>
<p>Each shape points to the prototype. This also means that every time the prototype of <code>foo</code> changes, the engine transitions to a new shape. Now we only need to check the shape of an object to both assert absence of certain properties and also guard the prototype link.</p>
<p>With this approach, we can reduce the number of checks required from <code>1+2N</code> to <code>1+N</code> for faster property access on prototypes. But that's still quite expensive, since it's still linear in the length of the prototype chain. Engines implement different tricks to further reduce this to a constant number of checks, especially for subsequent executions of the same property loads.</p>
<h3 id="validity-cells">Validity cells <a class="bookmark" href="#validity-cells">#</a></h3>
<p>V8 treats prototype shapes specially for this purpose. Each prototype has a unique shape, that is not shared with any other objects (specifically not with other prototypes) and each of these prototype shapes has a special <code>ValidityCell</code> associated with it.</p>
<p><img src="https://benediktmeurer.de/images/2018/validitycell-20180816.svg" alt="ValidityCell" title="ValidityCell"></p>
<p>This <code>ValidityCell</code> is invalidated whenever someone changes the associated prototype or any prototype above it. Let’s take a look at how this works exactly.</p>
<p>To speed up subsequent loads from prototypes, V8 puts an Inline Cache in place, with four fields:</p>
<p><img src="https://benediktmeurer.de/images/2018/ic-validitycell-20180816.svg" alt="ValidityCell in ICs" title="ValidityCell in ICs"></p>
<p>When warming up the inline cache during the first run of this code, V8 remembers the offset at which the property was found in the prototype, the prototype on which the property was found (<code>Bar.prototype</code> in this example), the shape of the instance (the shape of <code>foo</code> in this case), and also the link to the current ValidityCell of the <strong>immediate prototype</strong> that is linked to from the instance shape (which also happens to be <code>Bar.prototype</code> in this case).</p>
<p>The next time the Inline Cache is hit, the engine has to check the shape of the instance and the <code>ValidityCell</code>. If it's still valid, the engine can reach out directly to the <code>Offset</code> on the <code>Prototype</code>, skipping the additional lookups.</p>
<p><img src="https://benediktmeurer.de/images/2018/validitycell-invalid-20180816.svg" alt="ValidityCell invalidation" title="ValidityCell invalidation"></p>
<p>When the prototype is changed, a new shape is allocated and the previous <code>ValidityCell</code> is invalidated. So the Inline Cache misses the next time it's executed, resulting in worse performance.</p>
<p>Going back to the DOM element example from before, this means that any change to e.g. <code>Object.prototype</code> would not just invalidate Inline Caches for <code>Object.prototype</code> itself, but also for any prototype below including <code>EventTarget.prototype</code>, <code>Node.prototype</code>, <code>Element.prototype</code> and so on, all the way down until <code>HTMLAnchorElement.prototype</code>.</p>
<p><img src="https://benediktmeurer.de/images/2018/prototype-chain-validitycells-20180816.svg" alt="Prototype chain ValidityCells" title="Prototype chain ValidityCells"></p>
<p>Effectively, modifying <code>Object.prototype</code> while running your code means throwing performance out the window. <strong>Don't do it!</strong></p>
<p>Let's explore this a bit more with a concrete example: Say we have our class <code>Bar</code>, and we have a function <code>loadX</code> that calls a method on <code>Bar</code> objects. We call this <code>loadX</code> function a few times with instances of the same class.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">Bar</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token comment">/* … */</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">loadX</span><span class="token punctuation">(</span><span class="token parameter">bar</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> bar<span class="token punctuation">.</span><span class="token function">getX</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// IC for 'getX' on Bar instances</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token function">loadX</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">Bar</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">loadX</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">Bar</span><span class="token punctuation">(</span><span class="token boolean">false</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// IC in loadX now links ValidityCell for Bar.prototype.</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token class-name">Object</span><span class="token punctuation">.</span>prototype<span class="token punctuation">.</span><span class="token function-variable function">newMethod</span> <span class="token operator">=</span> <span class="token parameter">y</span> <span class="token operator">=></span> y<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// The ValidityCell in the loadX IC is invalid</span></span><br><span class="highlight-line"><span class="token comment">// now, because Object.prototype changed.</span></span></code></pre>
<p>The inline cache in <code>loadX</code> now points to the <code>ValidityCell</code> for <code>Bar.prototype</code>. If you then do something like mutate the <code>Object.prototype</code> — which is the root of all prototypes in JavaScript — the <code>ValidityCell</code> becomes invalid, and the existing Inline Caches miss the next time they're hit, resulting in worse performance.</p>
<p>Mutating <code>Object.prototype</code> is always a bad idea, as it invalidates any Inline Caches for prototype loads that the engine had put up until that point. Here's another example of what NOT to do:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token class-name">Object</span><span class="token punctuation">.</span>prototype<span class="token punctuation">.</span><span class="token function-variable function">foo</span> <span class="token operator">=</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token comment">/* … */</span></span><br><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token comment">// Run critical code:</span></span><br><span class="highlight-line">someObject<span class="token punctuation">.</span><span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// End critical code.</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">delete</span> <span class="token class-name">Object</span><span class="token punctuation">.</span>prototype<span class="token punctuation">.</span>foo<span class="token punctuation">;</span></span></code></pre>
<p>We extend <code>Object.prototype</code>, which invalidates any prototype Inline Caches the engine put in place up until that point. Then we run some code that uses the new prototype method. The engine has to start over from scratch and set up new Inline Caches for any prototype property accesses. And then finally, we <em>"clean up after ourselves"</em> and remove the prototype method we added earlier.</p>
<p>Cleaning up sounds like a good idea, right? Well actually, in this case it makes a bad situation even worse! Deleting the property modifies <code>Object.prototype</code>, so all the Inline Caches are invalidated all over again and the engine has to start from scratch once again.</p>
<p><strong>Summary:</strong> Although prototypes are just objects, they are treated specially by JavaScript engines to optimize the performance of method lookups on prototypes. Leave your prototypes alone! Or if you really need to touch prototypes, then do it before other code runs, so that you at least don't invalidate all the optimizations in the engine while your code is running.</p>
<h2 id="take-aways">Take-aways <a class="bookmark" href="#take-aways">#</a></h2>
<p>We’ve learned how JavaScript engines store objects and classes, and how <code>Shape</code>s, Inline Caches, and <code>ValidityCell</code>s help to optimize prototype operations. Based on this knowledge, we identified a practical JavaScript coding tip that can help boost performance: <strong>don't mess with prototypes</strong> (or if you really, really need to, then at least do it before other code runs).</p>
<h2 id="translations">Translations <a class="bookmark" href="#translations">#</a></h2>
<ul>
<li><a href="https://hijiangtao.github.io/2018/08/21/Prototypes/">Chinese</a></li>
<li><a href="https://shlrur.github.io/javascripts/javascript-engine-fundamentals-optimizing-prototypes/">Korean</a></li>
</ul>
JavaScript engine fundamentals: Shapes and Inline Caches2018-06-14T00:00:00-00:00https://benediktmeurer.de/2018/06/14/javascript-engine-fundamentals-shapes-and-inline-caches/<p>This article describes some key fundamentals that are common to all JavaScript engines — and not just <a href="https://twitter.com/v8js">V8</a>, the engine the authors (<a href="https://twitter.com/bmeurer">Benedikt</a> and <a href="https://twitter.com/mathias">Mathias</a>) work on. As a JavaScript developer, having a deeper understanding of how JavaScript engines work helps you reason about the performance characteristics of your code.</p>
<p><em>Note:</em> If you prefer watching a presentation over reading articles, then enjoy the video below! If not, skip the video and read on.</p>
<p><div class="embed-responsive embed-responsive-16by9"><iframe class="embed-responsive-item youtube-player" type="text/html" width="720" height="410" src="https://www.youtube.com/embed/5nmpokoRaZI" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe></div></p>
<h2 id="the-javascript-engine-pipeline">The JavaScript engine pipeline <a class="bookmark" href="#the-javascript-engine-pipeline">#</a></h2>
<p>It all starts with the JavaScript code you write. The JavaScript engine parses the source code and turns it into an Abstract Syntax Tree (AST). Based on that AST, the interpreter can start to do its thing and produce bytecode. Great! At that point the engine is actually running the JavaScript code.</p>
<p><img src="https://benediktmeurer.de/images/2018/js-engine-pipeline-20180614.svg" alt="JavaScript Engine Pipeline" title="JavaScript Engine Pipeline"></p>
<p>To make it run faster, the bytecode can be sent to the optimizing compiler along with profiling data. The optimizing compiler makes certain assumptions based on the profiling data it has, and then produces highly-optimized machine code.</p>
<p>If at some point one of the assumptions turns out to be incorrect, the optimizing compiler deoptimizes and goes back to the interpreter.</p>
<h3 id="interpreter%2Fcompiler-pipelines-in-javascript-engines">Interpreter/compiler pipelines in JavaScript engines <a class="bookmark" href="#interpreter%2Fcompiler-pipelines-in-javascript-engines">#</a></h3>
<p>Now, let’s zoom in on the parts of this pipeline that actually run your JavaScript code, i.e. where code gets interpreted and optimized, and go over some of the differences between major JavaScript engines.</p>
<p>Generally speaking, there’s a pipeline containing an interpreter and an optimizing compiler. The interpreter generates unoptimized bytecode quickly, and the optimizing compiler takes a little longer but eventually produces highly-optimized machine code.</p>
<p><img src="https://benediktmeurer.de/images/2018/interpreter-optimizing-compiler-20180614.svg" alt="JavaScript Engine Architecture" title="JavaScript Engine Architecture"></p>
<p>This generic pipeline is pretty much exactly how V8, the JavaScript engine used in Chrome and Node.js, works:</p>
<p><img src="https://benediktmeurer.de/images/2018/interpreter-optimizing-compiler-spidermonkey-20180614.svg" alt="V8 Architecture" title="V8 Architecture"></p>
<p>The interpreter in V8 is called Ignition, and is responsible for generating and executing bytecode. While it runs the bytecode, it collects profiling data, which can be used to speed up the execution later. When a function becomes <em>hot</em>, for example when it’s run often, the generated bytecode <em>and</em> the profiling data are passed on to TurboFan, our optimizing compiler, to generate highly-optimized machine code based on the profiling data.</p>
<p><img src="https://benediktmeurer.de/images/2018/interpreter-optimizing-compiler-spidermonkey-20180614.svg" alt="SpiderMonkey Architecture" title="SpiderMonkey Architecture"></p>
<p>SpiderMonkey, Mozilla’s JavaScript engine as used in Firefox and in <a href="https://github.com/mozilla/spidernode">SpiderNode</a>, does it a little differently. They have not one but two optimizing compilers. The interpreter optimizes into the Baseline compiler, which produces somewhat optimized code. Combined with profiling data gathered while running the code, the IonMonkey compiler can produce heavily-optimized code. If the speculative optimization fails, IonMonkey falls back to the Baseline code.</p>
<p><img src="https://benediktmeurer.de/images/2018/interpreter-optimizing-compiler-chakra-20180614.svg" alt="ChakraCore Architecture" title="ChakraCore Architecture"></p>
<p>Chakra, Microsoft’s JavaScript engine as used in Edge and <a href="https://github.com/nodejs/node-chakracore">Node-ChakraCore</a>, has a very similar setup with two optimizing compilers. The interpreter optimizes into SimpleJIT — where JIT stands for Just-In-Time compiler — which produces somewhat optimized code. Combined with profiling data, the FullJIT can produce more-heavily-optimized code.</p>
<p><img src="https://benediktmeurer.de/images/2018/interpreter-optimizing-compiler-jsc-20180614.svg" alt="JavaScriptCore Architecture" title="JavaScriptCore Architecture"></p>
<p>JavaScriptCore (abbreviated as JSC), Apple’s JavaScript engine as used in Safari and React Native, takes it to the extreme with three different optimizing compilers. LLInt, the Low-Level Interpreter, optimizes into the Baseline compiler, which can then optimize into the DFG (Data Flow Graph) compiler, which can in turn optimize into the FTL (Faster Than Light) compiler.</p>
<p>Why do some engines have more optimizing compilers than others? It’s all about trade-offs. An interpreter can produce bytecode quickly, but bytecode is generally not very efficient. An optimizing compiler on the other hand takes a little longer, but eventually produces much more efficient machine code. There is a trade-off between quickly getting code to run (interpreter) or taking some more time, but eventually running the code with optimal performance (optimizing compiler). Some engines choose to add multiple optimizing compilers with different time/efficiency characteristics, allowing for more fine-grained control over these trade-offs at the cost of additional complexity. Another trade-off relates to memory usage; see <a href="https://benediktmeurer.de/2018/08/16/javascript-engine-fundamentals-optimizing-prototypes/#optimization-tiers-and-execution-trade-offs">our follow-up article</a> for details on that.</p>
<p>We’ve just highlighted the main differences in the interpreter and optimizing compiler pipelines for each JavaScript engine. But besides these differences, at a high level, <em>all JavaScript engines have the same architecture</em>: there’s a parser and some kind of interpreter/compiler pipeline.</p>
<h2 id="javascript%E2%80%99s-object-model">JavaScript’s object model <a class="bookmark" href="#javascript%E2%80%99s-object-model">#</a></h2>
<p>Let’s look at what else JavaScript engines have in common by zooming in on how some aspects are implemented.</p>
<p>For example, how do JavaScript engines implement the JavaScript object model, and which tricks do they use to speed up accessing properties on JavaScript objects? As it turns out, all major engines implement this very similarly.</p>
<p>The ECMAScript specification essentially defines all objects as dictionaries, with string keys mapping to <em><a href="https://tc39.github.io/ecma262/#sec-property-attributes">property attributes</a></em>.</p>
<p><img src="https://benediktmeurer.de/images/2018/object-model-20180614.svg" alt="Object Model" title="Object Model"></p>
<p>Other than the <code>[[Value]]</code> itself, the spec defines these properties:</p>
<ul>
<li><code>[[Writable]]</code> which determines whether the property can be reassigned to,</li>
<li><code>[[Enumerable]]</code> which determines whether the property shows up in <code>for</code>-<code>in</code> loops,</li>
<li>and <code>[[Configurable]]</code> which determines whether the property can be deleted.</li>
</ul>
<p>The <code>[[double square brackets]]</code> notation looks funky, but that’s just how the spec represents properties that aren’t directly exposed to JavaScript. You can still get to these property attributes for any given object and property in JavaScript by using the <code>Object.getOwnPropertyDescriptor</code> API:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> object <span class="token operator">=</span> <span class="token punctuation">{</span> foo<span class="token punctuation">:</span> <span class="token number">42</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line">Object<span class="token punctuation">.</span><span class="token function">getOwnPropertyDescriptor</span><span class="token punctuation">(</span>object<span class="token punctuation">,</span> <span class="token string">"foo"</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// → { value: 42, writable: true, enumerable: true, configurable: true }</span></span></code></pre>
<p>Ok, so that’s how JavaScript defines objects. What about arrays?</p>
<p>You can think of arrays as a special case of objects. One difference is that arrays have special handling of array indices. Here <em>array index</em> is a special term in the ECMAScript specification. Arrays are limited to 2³²−1 items in JavaScript. An array index is any valid index within that limit, i.e. any integer number from 0 to 2³²−2.</p>
<p>Another difference is that arrays also have a magical <code>length</code> property.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> array <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token string">"a"</span><span class="token punctuation">,</span> <span class="token string">"b"</span><span class="token punctuation">]</span><span class="token punctuation">;</span></span><br><span class="highlight-line">array<span class="token punctuation">.</span>length<span class="token punctuation">;</span> <span class="token comment">// → 2</span></span><br><span class="highlight-line">array<span class="token punctuation">[</span><span class="token number">2</span><span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token string">"c"</span><span class="token punctuation">;</span></span><br><span class="highlight-line">array<span class="token punctuation">.</span>length<span class="token punctuation">;</span> <span class="token comment">// → 3</span></span></code></pre>
<p>In this example, the array has a <code>length</code> of <code>2</code> when it’s created. Then we assign another element to index <code>2</code>, and the <code>length</code> automatically updates.</p>
<p>JavaScript defines arrays similarly to objects. For example, all the keys including array indices are represented as strings explicitly. The first element in the array is stored under the key <code>'0'</code>.</p>
<p><img src="https://benediktmeurer.de/images/2018/array-1-20180614.svg" alt="Array with one element" title="Array with one element"></p>
<p>The <code>'length'</code> property is just another property that happens to be non-enumerable and non-configurable.</p>
<p>Once an element is added to the array, JavaScript automatically updates the <code>[[Value]]</code> property attribute of the <code>'length'</code> property.</p>
<p><img src="https://benediktmeurer.de/images/2018/array-2-20180614.svg" alt="Array with two elements" title="Array with two elements"></p>
<p>Generally speaking, arrays behave pretty similarly to objects.</p>
<h2 id="optimizing-property-access">Optimizing property access <a class="bookmark" href="#optimizing-property-access">#</a></h2>
<p>Now that we know how objects are defined in JavaScript, let’s dive into how JavaScript engines enable working with objects efficiently.</p>
<p>Looking at JavaScript programs in the wild, accessing properties is by far the most common operation. It’s crucial for JavaScript engines to make property access fast.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> object <span class="token operator">=</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> foo<span class="token punctuation">:</span> <span class="token string">"bar"</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> baz<span class="token punctuation">:</span> <span class="token string">"qux"</span></span><br><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token comment">// Here, we’re accessing the property `foo` on `object`:</span></span><br><span class="highlight-line"><span class="token function">doSomething</span><span class="token punctuation">(</span>object<span class="token punctuation">.</span>foo<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// ^^^^^^^^^^</span></span></code></pre>
<h3 id="shapes">Shapes <a class="bookmark" href="#shapes">#</a></h3>
<p>In JavaScript programs, it’s common to have multiple objects with the same property keys. Such objects have the same <em>shape</em>.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> object1 <span class="token operator">=</span> <span class="token punctuation">{</span> x<span class="token punctuation">:</span> <span class="token number">1</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">2</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">const</span> object2 <span class="token operator">=</span> <span class="token punctuation">{</span> x<span class="token punctuation">:</span> <span class="token number">3</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">4</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token comment">// `object1` and `object2` have the same shape.</span></span></code></pre>
<p>It’s also very common to access the same property on objects with the same shape:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">logX</span><span class="token punctuation">(</span><span class="token parameter">object</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>object<span class="token punctuation">.</span>x<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token comment">// ^^^^^^^^</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> object1 <span class="token operator">=</span> <span class="token punctuation">{</span> x<span class="token punctuation">:</span> <span class="token number">1</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">2</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">const</span> object2 <span class="token operator">=</span> <span class="token punctuation">{</span> x<span class="token punctuation">:</span> <span class="token number">3</span><span class="token punctuation">,</span> y<span class="token punctuation">:</span> <span class="token number">4</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token function">logX</span><span class="token punctuation">(</span>object1<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">logX</span><span class="token punctuation">(</span>object2<span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>With that in mind, JavaScript engines can optimize object property access based on the object’s shape. Here’s how that works.</p>
<p>Let’s assume we have an object with the properties <code>x</code> and <code>y</code>, and it uses the dictionary data structure we discussed earlier: it contains the keys as strings, and those point to their respective property attributes.</p>
<p><img src="https://benediktmeurer.de/images/2018/object-model-20180614.svg" alt="Object Model" title="Object Model"></p>
<p>If you access a property, e.g. <code>object.y</code>, the JavaScript engine looks in the <code>JSObject</code> for the key <code>'y'</code>, then loads the corresponding property attributes, and finally returns the <code>[[Value]]</code>.</p>
<p>But where are these property attributes stored in memory? Should we store them as part of the <code>JSObject</code>? If we assume that we’ll be seeing more objects with this shape later, then it’s wasteful to store the full dictionary containing the property names and attributes on the <code>JSObject</code> itself, as the property names are repeated for all objects with the same shape. That’s a lot of duplication and unnecessarily memory usage. As an optimization, engines store the <code>Shape</code> of the object separately.</p>
<p><img src="https://benediktmeurer.de/images/2018/shape-1-20180614.svg" alt="Object with shape" title="Object with shape"></p>
<p>This <code>Shape</code> contains all the property names and the attributes, except for their <code>[[Value]]</code>s. Instead the <code>Shape</code> contains the offset of the values inside of the <code>JSObject</code>, so that the JavaScript engine knows where to find the values. Every <code>JSObject</code> with this same shape points to exactly this <code>Shape</code> instance. Now every <code>JSObject</code> only has to store the values that are unique to this object.</p>
<p><img src="https://benediktmeurer.de/images/2018/shape-2-20180614.svg" alt="Objects with shape" title="Objects with shape"></p>
<p>The benefit becomes clear when we have multiple objects. No matter how many objects there are, as long as they have the same shape, we only have to store the shape and property information once!</p>
<p>All JavaScript engines use shapes as an optimization, but they don’t all call them shapes:</p>
<ul>
<li>Academic papers call them <em>Hidden Classes</em> (confusing w.r.t. JavaScript classes)</li>
<li>V8 calls them <em>Maps</em> (confusing w.r.t. JavaScript <code>Map</code>s)</li>
<li>Chakra calls them <em>Types</em> (confusing w.r.t. JavaScript’s dynamic types and <code>typeof</code>)</li>
<li>JavaScriptCore calls them <em>Structures</em></li>
<li>SpiderMonkey calls them <em>Shapes</em></li>
</ul>
<p>Throughout this article, we’ll continue to use the term <em>shapes</em>.</p>
<h3 id="transition-chains-and-trees">Transition chains and trees <a class="bookmark" href="#transition-chains-and-trees">#</a></h3>
<p>What happens if you have an object with a certain shape, but then you add a property to it? How does the JavaScript engine find the new shape?</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> object <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line">object<span class="token punctuation">.</span>x <span class="token operator">=</span> <span class="token number">5</span><span class="token punctuation">;</span></span><br><span class="highlight-line">object<span class="token punctuation">.</span>y <span class="token operator">=</span> <span class="token number">6</span><span class="token punctuation">;</span></span></code></pre>
<p>The shapes form so-called <em>transition chains</em> in the JavaScript engine. Here’s an example:</p>
<p><img src="https://benediktmeurer.de/images/2018/shape-chain-1-20180614.svg" alt="Transition chains" title="Transition chains"></p>
<p>The object starts out without any properties, so it points to the empty shape. The next statement adds a property <code>'x'</code> with a value <code>5</code> to this object, so the JavaScript engine transitions to a shape that contains the property <code>'x'</code> and a value <code>5</code> is added to the <code>JSObject</code> at the first offset <code>0</code>. The next line adds a property <code>'y'</code>, so the engine transitions to yet another shape that contains both <code>'x'</code> and <code>'y'</code>, and appends the value <code>6</code> to the <code>JSObject</code> (at offset <code>1</code>).</p>
<p><em>Note:</em> The order in which properties are added impacts the shape. For example, <code>{ x: 4, y: 5 }</code> results in a different shape than <code>{ y: 5, x: 4 }</code>.</p>
<p>We don’t even need to store the full table of properties for each <code>Shape</code>. Instead, every <code>Shape</code> only needs to know about the new property it introduces. For example, in this case we don’t have to store the information about <code>'x'</code> in that last shape, because it can be found earlier in the chain. To make this work, every <code>Shape</code> links back to its previous shape:</p>
<p><img src="https://benediktmeurer.de/images/2018/shape-chain-2-20180614.svg" alt="Transition chains simplified" title="Transition chains simplified"></p>
<p>If you write <code>o.x</code> in your JavaScript code, the JavaScript engine looks up the property <code>'x'</code> by walking up the transition chain until it finds the <code>Shape</code> that introduced property <code>'x'</code>.</p>
<p>But what happens if there’s no way to create a transition chain? For example, what if you have two empty objects, and you add a different property to each of them?</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> object1 <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line">object1<span class="token punctuation">.</span>x <span class="token operator">=</span> <span class="token number">5</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">const</span> object2 <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line">object2<span class="token punctuation">.</span>y <span class="token operator">=</span> <span class="token number">6</span><span class="token punctuation">;</span></span></code></pre>
<p>In that case we have to branch, and instead of a chain, we end up with a <em>transition tree</em>:</p>
<p><img src="https://benediktmeurer.de/images/2018/shape-tree-20180614.svg" alt="Transition tree" title="Transition tree"></p>
<p>Here, we create an empty object <code>a</code>, and then add a property <code>'x'</code> to it. We end up with a <code>JSObject</code> containing a single value, and two <code>Shapes</code>: the empty shape, and the shape with only a property <code>x</code>.</p>
<p>The second example starts with an empty object <code>b</code> too, but then adds a different property <code>'y'</code>. We end up with two shape chains, and a total of three shapes.</p>
<p>Does that mean we always start at the empty shape? Not necessarily. Engines apply some optimizations for object literals that already contain properties. Let’s say we either add <code>x</code> starting from the empty object literal, or have an object literal that already contains <code>x</code>:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> object1 <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line">object1<span class="token punctuation">.</span>x <span class="token operator">=</span> <span class="token number">5</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">const</span> object2 <span class="token operator">=</span> <span class="token punctuation">{</span> x<span class="token punctuation">:</span> <span class="token number">6</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></span></code></pre>
<p>In the first example, we start at the empty shape and transition to the shape that also contains <code>x</code>, just as we saw before.</p>
<p>In the case of <code>object2</code>, it makes sense to directly produce objects that already have <code>x</code> from the beginning instead of starting from an empty object and transitioning.</p>
<p><img src="https://benediktmeurer.de/images/2018/empty-shape-bypass-20180614.svg" alt="Bypassing empty shape" title="Bypassing empty shape"></p>
<p>The object literal that contains the property <code>'x'</code> starts at a shape that contains <code>'x'</code> from the beginning, effectively skipping the empty shape. This is what (at least) V8 and SpiderMonkey do. This optimization shortens the transition chains and makes it more efficient to construct objects from literals.</p>
<p>My previous blog post on <a href="https://medium.com/@bmeurer/surprising-polymorphism-in-react-applications-63015b50abc">surprising polymorphism in React applications</a> discusses how these subtleties can affect real-world performance.</p>
<p>Here’s an example of a 3D point object with properties <code>'x'</code>, <code>'y'</code>, and <code>'z'</code>.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> point <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line">point<span class="token punctuation">.</span>x <span class="token operator">=</span> <span class="token number">4</span><span class="token punctuation">;</span></span><br><span class="highlight-line">point<span class="token punctuation">.</span>y <span class="token operator">=</span> <span class="token number">5</span><span class="token punctuation">;</span></span><br><span class="highlight-line">point<span class="token punctuation">.</span>z <span class="token operator">=</span> <span class="token number">6</span><span class="token punctuation">;</span></span></code></pre>
<p>As we learned before, this creates an object with 3 shapes in memory (not counting the empty shape). To access the property <code>'x'</code> on that object, e.g. if you write <code>point.x</code> in your program, the JavaScript engine needs to follow the linked list: it starts at the <code>Shape</code> at the bottom, and then works its way up to the <code>Shape</code> that introduced <code>'x'</code> at the top.</p>
<p><img src="https://benediktmeurer.de/images/2018/shapetable-1-20180614.svg" alt="ShapeTable" title="ShapeTable"></p>
<p>That’s going to be really slow if we do this more often, especially when the objects have lots of properties. The time to find the property is <code>O(n)</code>, i.e. linear in the number of properties on the object. To speed up searching for properties, JavaScript engines add a <code>ShapeTable</code> data structure. This <code>ShapeTable</code> is a dictionary, mapping property keys to the respective <code>Shape</code>s that introduce the given property.</p>
<p><img src="https://benediktmeurer.de/images/2018/shapetable-2-20180614.svg" alt="ShapeTable" title="ShapeTable"></p>
<p>Wait a minute, now we’re back to dictionary lookups… That’s where we were before we started adding <code>Shape</code>s in the first place! So why do we bother with shapes at all?</p>
<p>The reason is that shapes enable another optimization called <em>Inline Caches</em>.</p>
<h3 id="inline-caches-(ics)">Inline Caches (ICs) <a class="bookmark" href="#inline-caches-(ics)">#</a></h3>
<p>The main motivation behind shapes is the concept of Inline Caches or ICs. ICs are the key ingredient to making JavaScript run fast! JavaScript engines use ICs to memorize information on where to find properties on objects, to reduce the number of expensive lookups.</p>
<p>Here’s a function <code>getX</code> that takes an object and loads the property <code>x</code> from it:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">getX</span><span class="token punctuation">(</span><span class="token parameter">o</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> o<span class="token punctuation">.</span>x<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>If we run this function in JSC, it generates the following bytecode:</p>
<p><img src="https://benediktmeurer.de/images/2018/ic-1-20180614.svg" alt="Generated bytecode" title="Generated bytecode"></p>
<p>The first <code>get_by_id</code> instruction loads the property <code>'x'</code> from the first argument (<code>arg1</code>) and stores the result into <code>loc0</code>. The second instruction returns what we stored to <code>loc0</code>.</p>
<p>JSC also embeds an Inline Cache into the <code>get_by_id</code> instruction, which consists of two uninitialized slots.</p>
<p><img src="https://benediktmeurer.de/images/2018/ic-2-20180614.svg" alt="Generated bytecode with IC" title="Generated bytecode with IC"></p>
<p>Now let’s assume we call <code>getX</code> with an object <code>{ x: 'a' }</code>. As we learned, this object has a shape with property <code>'x'</code> and the <code>Shape</code> stores the offset and attributes for that property <code>x</code>. When you execute the function for the first time, the <code>get_by_id</code> instruction looks up the property <code>'x'</code> and finds that the value is stored at offset <code>0</code>.</p>
<p><img src="https://benediktmeurer.de/images/2018/ic-3-20180614.svg" alt="Generated bytecode with IC" title="Generated bytecode with IC"></p>
<p>The IC embedded into the <code>get_by_id</code> instruction memorizes the shape and the offset at which the property was found:</p>
<p><img src="https://benediktmeurer.de/images/2018/ic-4-20180614.svg" alt="Generated bytecode with IC" title="Generated bytecode with IC"></p>
<p>For subsequent runs, the IC only needs to compare the shape, and if it’s the same as before, just load the value from the memorized offset. Specifically, if the JavaScript engine sees objects with a shape that the IC recorded before, it no longer needs to reach out to the property information at all — instead, the expensive property information lookup can be skipped entirely. That’s significantly faster than looking up the property each time.</p>
<h2 id="storing-arrays-efficiently">Storing arrays efficiently <a class="bookmark" href="#storing-arrays-efficiently">#</a></h2>
<p>For arrays, it’s common to store properties that are <em>array indices</em>. The values for such properties are called array elements. It would be wasteful memory-wise to store property attributes for each and every array element in every single array. Instead, JavaScript engines use the fact that array-indexed properties are writable, enumerable, and configurable by default, and store array elements separately from other named properties.</p>
<p>Consider this array:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> array <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token string">"#jsconfeu"</span><span class="token punctuation">]</span><span class="token punctuation">;</span></span></code></pre>
<p>The engine stores the array length (<code>1</code>), and points to the <code>Shape</code> that contains the offset and the attributes for the <code>'length'</code> property.</p>
<p><img src="https://benediktmeurer.de/images/2018/array-shape-20180614.svg" alt="Array shape" title="Array shape"></p>
<p>This is similar to what we’ve seen before… but where are the array values stored?</p>
<p><img src="https://benediktmeurer.de/images/2018/array-elements-20180614.svg" alt="Array elements" title="Array elements"></p>
<p>Every array has a separate <em>elements backing store</em> that contains all the array-indexed property values. The JavaScript engine doesn’t have to store any property attributes for array elements, because usually they are all writable, enumerable, and configurable.</p>
<p>What happens in the unusual case, though? What if you change the property attributes of array elements?</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token comment">// Please don’t ever do this!</span></span><br><span class="highlight-line"><span class="token keyword">const</span> array <span class="token operator">=</span> Object<span class="token punctuation">.</span><span class="token function">defineProperty</span><span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">,</span> <span class="token string">"0"</span><span class="token punctuation">,</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> value<span class="token punctuation">:</span> <span class="token string">"Oh noes!!1"</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> writable<span class="token punctuation">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> enumerable<span class="token punctuation">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> configurable<span class="token punctuation">:</span> <span class="token boolean">false</span></span><br><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>The above snippet defines a property named <code>'0'</code> (which happens to be an array index), but sets it attributes to non-default values.</p>
<p>In such edge cases, the JavaScript engine represents the <em>entire</em> elements backing store as a dictionary that maps array indices to property attributes.</p>
<p><img src="https://benediktmeurer.de/images/2018/array-dictionary-elements-20180614.svg" alt="Array dictionary elements" title="Array dictionary elements"></p>
<p>Even when just a single array element has non-default attributes, the entire array’s backing store goes into this slow and inefficient mode. <em>Avoid <code>Object.defineProperty</code> on array indices!</em> (I’m not sure why you would even want to do this. It seems like a weird, non-useful thing to do.)</p>
<h2 id="take-aways">Take-aways <a class="bookmark" href="#take-aways">#</a></h2>
<p>We’ve learned how JavaScript engines store objects and arrays, and how Shapes and ICs help to optimize common operations on them. Based on this knowledge, we identified some practical JavaScript coding tips that can help boost performance:</p>
<ul>
<li>Always initialize your objects in the same way, so they don’t end up having different shapes.</li>
<li>Don’t mess with property attributes of array elements, so they can be stored and operated on efficiently.</li>
</ul>
<p><em>Looking for more?</em> If you’re interested in how JavaScript engines speed up accesses to prototype properties, read the next article in this series, <a href="https://benediktmeurer.de/2018/08/16/javascript-engine-fundamentals-optimizing-prototypes/">“JavaScript engine fundamentals: optimizing prototypes”</a>.</p>
<h2 id="translations">Translations <a class="bookmark" href="#translations">#</a></h2>
<ul>
<li><a href="https://hijiangtao.github.io/2018/06/17/Shapes-ICs/">Chinese</a></li>
<li><a href="https://shlrur.github.io/javascripts/javascript-engine-fundamentals-shapes-and-Inline-caches/">Korean</a></li>
</ul>
Interview “Three Frontend Questions to Benedikt Meurer” for YGLF Kyiv 20182018-05-10T00:00:00-00:00https://benediktmeurer.de/2018/05/10/three-frontend-questions-to-benedikt-meurer/<p>Before the conference the <a href="http://yglf.com.ua/">YGLF crew</a> wanted to learn more about the speakers, so they had asked them <a href="https://twitter.com/hashtag/3FrontendQuestions">#3FrontendQuestions</a> related to their topics at YGLF and engineering experience. This is what I responded during the interview.</p>
<figure>
<img src="https://benediktmeurer.de/images/2018/yglf-interview-20180510.jpg" srcset="https://benediktmeurer.de/images/2018/yglf-interview-20180510.jpg, https://benediktmeurer.de/images/2018/yglf-interview-20180510@2x.jpg 2x" title="Interview with Benedikt Meurer" alt="Interview with Benedikt Meurer">
</figure>
<h2 id="benedikt-meurer">Benedikt Meurer <a class="bookmark" href="#benedikt-meurer">#</a></h2>
<p>Benedikt is a JavaScript compiler engineer who loves to tinker with different aspects of programming languages. He joined Google to work on the V8 JavaScript Engine, where he is working as tech lead for the Node.js performance team.</p>
<p>His talk at <a href="http://yglf.com.ua/">You Gotta Love Frontend Conference</a> is <a href="http://yglf.com.ua/schedule/#Benedikt-Meurer-25-14-00">“Brave New World: Moving on to ES201X”</a> as well as lightning talk <a href="http://yglf.com.ua/schedule/#Benedikt-Meurer-24-16-30">“JavaScript: The fairly odd parts”</a>.</p>
<h2 id="v8-has-market-dominance.-do-you-feel-the-lack-of-competition%3F">V8 has market dominance. Do you feel the lack of competition? <a class="bookmark" href="#v8-has-market-dominance.-do-you-feel-the-lack-of-competition%3F">#</a></h2>
<p>This is my very own opinion, and doesn’t reflect Google’s or Chrome’s point of view.</p>
<p>I personally do feel the lack of competition in this space. For the growing JavaScript platform it would be awesome to have more competition. Right now we do have some competition in the web space, but that’s only a small part of the JavaScript space.</p>
<p>For example, both Node and Electron are growing very fast, and there’s not really competition on the VM side.</p>
<p>Similarly React Native currently ships with JSC only, and that’s mostly because V8 doesn’t run on iOS, which is not a fundamental limitation.</p>
<h2 id="do-you-have-performance-regression-tests-for-each-pull-request%2Fcommit%3F-how-long-does-it-take-to-run-all-tests-on-v8%3F">Do you have performance regression tests for each pull request/commit? How long does it take to run all tests on V8? <a class="bookmark" href="#do-you-have-performance-regression-tests-for-each-pull-request%2Fcommit%3F-how-long-does-it-take-to-run-all-tests-on-v8%3F">#</a></h2>
<p>We try to have correctness tests for each commit, but even that is often challenging, due to the various heuristics and different paths a single line of JavaScript code can take through V8.</p>
<p>We have a set of agreed upon performance tests that run on each individual commit.</p>
<h2 id="funny-third-question.-you%E2%80%99re-have-this-quote-in-your-twitter-bio%3A-%E2%80%9Cprobably-broke-the-web-for-you-once-or-twice%E2%80%9D-%E2%80%94-please-tell-us-at-least-one-story%3F">Funny third question. You’re have this quote in your twitter bio: “Probably broke the web for you once or twice” — please tell us at least one story? <a class="bookmark" href="#funny-third-question.-you%E2%80%99re-have-this-quote-in-your-twitter-bio%3A-%E2%80%9Cprobably-broke-the-web-for-you-once-or-twice%E2%80%9D-%E2%80%94-please-tell-us-at-least-one-story%3F">#</a></h2>
<p>Oh yeah that’s a fun story (in retrospect). 😁</p>
<p>I once made YouTube super crashy in Chrome stable for like a week: one day we noticed a super high crash rate on YouTube in stable, it was around one renderer crash per 100k page views, but we were in the middle of the stable cycle, so chrome hadn’t changed for a while.</p>
<p>That was odd. Even worse we couldn’t reproduce any crashes on YouTube even with the full team clicking around YouTube for hours. Then a colleagues wife started to see very reproducible crashes, and we tried from home and voila we were able to repro.</p>
<p>By that time the YouTube team also realized that, and they figured out that it was some software on YouTube that was recently updated, which doesn’t run on the internal Google network. So they were able to roll back that update and roughly 5 days later, YouTube started to stabilize in chrome again.</p>
<p>Meanwhile we tried to figure out from some crash dumps what was happening under the hood. And all dumps showed that we somehow leak “the hole”, which is a special marker that is used to mark holes in <code>Array</code>s. By that time we were mostly running with the Crankshaft compiler and the array code hadn’t changed for several releases.</p>
<p>One of my colleagues, <a href="https://twitter.com/tverwaes">Toon Verwaest</a>, one of the V8 gurus spend about a week auditing every single line of code that deals with <code>Array</code>s and he found and fixed a couple of unrelated bugs along the lines.</p>
<p>Turned out that we had recently enabled TurboFan for some weird language constructs that Crankshaft couldn’t handle, and one of the functions in the YouTube update actually used TurboFan sometimes. And in that function there was a call to the <code>Array</code> constructor, which I recently added to TurboFan. And that was missing a single line which marked the resulting array as <em>holey</em> ( i.e. not packed). And so <a href="https://v8.dev/">V8</a> was using this array without doing proper hole checks on it.</p>
Impact of polymorphism on component-based frameworks like React2018-03-23T00:00:00-00:00https://benediktmeurer.de/2018/03/23/impact-of-polymorphism-on-component-based-frameworks-like-react/<p>I just realized that this is going to be the very first blog post of 2018 that I write myself - versus bugging
someone else to write a blog post. It's been quite a busy year for me already, plus I was sick a lot, and so
was my family. Anyways, here's something I've been meaning to send out for a while. And while the title mentions
<a href="https://reactjs.org/">React</a> explicitly, this is by no means limited to React, but probably affects a lot of
code out there, including a lot of <a href="https://nodejs.org/">Node.js</a> code bases, where this impact is even more severe.</p>
<p>I've written a blog post on <a href="https://medium.com/@bmeurer/surprising-polymorphism-in-react-applications-63015b50abc">Surprising polymorphism in React
applications</a> earlier
and this post goes in the same direction, although we'll explore a slightly different problem here. For more
background on the topic you may want to read <a href="https://mrale.ph/blog/2015/01/11/whats-up-with-monomorphism.html">What's up with
monomorphism?</a> and <a href="https://mrale.ph/blog/2012/06/03/explaining-js-vms-in-js-inline-caches.html">Explaining JavaScript
VMs in JavaScript - Inline Caches</a>
by my colleague <a href="https://twitter.com/mraleph">Vyacheslav Egorov</a>.</p>
<h2 id="a-motivating-example">A motivating example <a class="bookmark" href="#a-motivating-example">#</a></h2>
<p>So let's dive right in with a code example. Imagine you have a component-based framework like React and
you need to call certain methods on all components, like the <code>render</code> method in case of React. Here's a
simplified example of how this could look like:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token comment">// Base class for all components.</span></span><br><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">Component</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">render</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token string">""</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">HelloComponent</span> <span class="token keyword">extends</span> <span class="token class-name">Component</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">render</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token string">"<div>Hello</div>"</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">LinkComponent</span> <span class="token keyword">extends</span> <span class="token class-name">Component</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token parameter">target<span class="token punctuation">,</span> text</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">this</span><span class="token punctuation">.</span>target <span class="token operator">=</span> target<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">this</span><span class="token punctuation">.</span>text <span class="token operator">=</span> text<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"> <span class="token function">render</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token string">'<a href="'</span> <span class="token operator">+</span> <span class="token keyword">this</span><span class="token punctuation">.</span>target <span class="token operator">+</span> <span class="token string">'">'</span> <span class="token operator">+</span> <span class="token keyword">this</span><span class="token punctuation">.</span>text <span class="token operator">+</span> <span class="token string">"</a>"</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">DOM</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">static</span> <span class="token function">renderAll</span><span class="token punctuation">(</span><span class="token parameter">target<span class="token punctuation">,</span> components</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">let</span> html <span class="token operator">=</span> <span class="token string">""</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">const</span> component <span class="token keyword">of</span> components<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> html <span class="token operator">+=</span> component<span class="token punctuation">.</span><span class="token function">render</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"> target<span class="token punctuation">.</span>innerHTML <span class="token operator">=</span> html<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> components <span class="token operator">=</span> <span class="token punctuation">[</span></span><br><span class="highlight-line"> <span class="token keyword">new</span> <span class="token class-name">HelloComponent</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> <span class="token keyword">new</span> <span class="token class-name">LinkComponent</span><span class="token punctuation">(</span><span class="token string">"http://www.google.com"</span><span class="token punctuation">,</span> <span class="token string">"Search"</span><span class="token punctuation">)</span></span><br><span class="highlight-line"><span class="token punctuation">]</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token constant">DOM</span><span class="token punctuation">.</span><span class="token function">renderAll</span><span class="token punctuation">(</span>document<span class="token punctuation">.</span><span class="token function">getElementById</span><span class="token punctuation">(</span><span class="token string">"my-app"</span><span class="token punctuation">)</span><span class="token punctuation">,</span> components<span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>You call <code>DOM.renderAll</code> to render a collection of components to a part of the DOM. This is oversimplified
of course, but you get the concept. Also note that this code doesn't follow any of the security advice for dealing
with HTML/DOM from JavaScript, so please don't take any inspiration here.</p>
<p>The interesting code here is inside of <code>DOM.renderAll</code>, where we access the property <code>component.render</code> of different
<code>component</code> shapes (in V8 speak we don't say <em>shape</em> but we use the term <em>map</em> or sometimes <em>hidden class</em>).
In this simple example we only have two different shapes of <code>component</code>: Either an instance of
<code>HelloComponent</code> where <code>render</code> is found on the <code>HelloComponent.prototype</code>, or an instance of <code>LinkComponent</code> where
<code>render</code> is found on the <code>LinkComponent.prototype</code>. So the property access <code>component.render</code> in <code>DOM.renderAll</code>
will be 2-way polymorphic.</p>
<h2 id="inline-cache-states">Inline cache states <a class="bookmark" href="#inline-cache-states">#</a></h2>
<p>Let's use a simplified example <code>components1.js</code> here to illustrate this:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">Base</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">A</span> <span class="token keyword">extends</span> <span class="token class-name">Base</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">B</span> <span class="token keyword">extends</span> <span class="token class-name">Base</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">bar</span><span class="token punctuation">(</span><span class="token parameter">instance</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> instance<span class="token punctuation">.</span><span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">A</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">B</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">A</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">B</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Running this with <a href="https://v8.dev/docs/d8"><code>d8</code></a> and the <code>--trace-ic</code> flag we can see that the
<code>LoadIC</code> (inline cache) for the property access <code>instance.foo</code> goes polymorphic, indicated by the <code>P</code> state:</p>
<pre><code>$ out/Release/d8 --trace-ic components1.js
$ tools/ic-processor
LoadIC (.->1) at ~bar components1.js:6:19 foo (map 0x8f24d80cca1)
LoadIC (1->P) at ~bar components1.js:6:19 foo (map 0x8f24d80cc01)
=====================
Load: 2
Store: 0
KeyedLoad: 0
KeyedStore: 0
StoreInArrayLiteral: 0
</code></pre>
<p>In V8 we have these five different states for <code>LoadIC</code>s right now:</p>
<table>
<thead>
<tr>
<th>Marker</th>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>0</code></td>
<td><code>UNINITIALIZED</code></td>
<td>The property access was not executed so far.</td>
</tr>
<tr>
<td><code>.</code></td>
<td><code>PREMONOMORPHIC</code></td>
<td>The property access was executed once and we are likely going to go <code>MONOMORPHIC</code> on the next hit.</td>
</tr>
<tr>
<td><code>1</code></td>
<td><code>MONOMORPHIC</code></td>
<td>The property access was always executed with the same shape.</td>
</tr>
<tr>
<td><code>P</code></td>
<td><code>POLYMORPHIC</code></td>
<td>The property access was always executed with one of four different shapes.</td>
</tr>
<tr>
<td><code>N</code></td>
<td><code>MEGAMORPHIC</code></td>
<td>The property access has seen too many different shapes.</td>
</tr>
</tbody>
</table>
<p>The initial <code>0->.</code> transition is not shown with <code>--trace-ic</code> currently, because V8 does that internally on the fast
path where no logging is hooked up right now. In the example above we run through <code>instance.foo</code> with two different
shapes for <code>instance</code>, either an instance of <code>A</code> with <code>foo</code> on <code>A.prototype</code>, or an instance of <code>B</code> with <code>foo</code> on
<code>B.prototype</code>.
Let's beef up the example a bit and add more than four different shapes like shown in <code>components2.js</code> below:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">Base</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">A</span> <span class="token keyword">extends</span> <span class="token class-name">Base</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">B</span> <span class="token keyword">extends</span> <span class="token class-name">Base</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">C</span> <span class="token keyword">extends</span> <span class="token class-name">Base</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">D</span> <span class="token keyword">extends</span> <span class="token class-name">Base</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">E</span> <span class="token keyword">extends</span> <span class="token class-name">Base</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">bar</span><span class="token punctuation">(</span><span class="token parameter">instance</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> instance<span class="token punctuation">.</span><span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">A</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">B</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">C</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">D</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">E</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">A</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">B</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">C</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">D</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">bar</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">E</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Running this through <code>d8</code> again with <code>--trace-ic</code> we see that we stay <code>POLYMORPHIC</code> for the first four shapes,
but then we go <code>MEGAMORPHIC</code> (as indicated by the <code>N</code> state) eventually (on the second instance of <code>A</code> in this
case due to the initial <code>PREMONOMORPHIC</code> step):</p>
<pre><code>$ out/Release/d8 --trace-ic components2.js
$ tools/ic-processor
LoadIC (.->1) at ~bar components2.js:9:19 foo (map 0x15581a58ce81)
LoadIC (1->P) at ~bar components2.js:9:19 foo (map 0x15581a58cfc1)
LoadIC (P->P) at ~bar components2.js:9:19 foo (map 0x15581a58d0b1)
LoadIC (P->P) at ~bar components2.js:9:19 foo (map 0x15581a58d1a1)
LoadIC (P->N) at ~bar components2.js:9:19 foo (map 0x15581a58cde1)
=====================
Load: 5
Store: 0
KeyedLoad: 0
KeyedStore: 0
StoreInArrayLiteral: 0
</code></pre>
<p>The <code>MEGAMORPHIC</code> state is the <em>I don't know what to do about this</em> state of V8. Whenever a <code>LoadIC</code>
reaches the <code>MEGAMORPHIC</code> state, TurboFan will no longer be able to inline any fast-path for it (except for
some corner cases where TurboFan can find information about the object somewhere else), and will have to go
through the inline cache logic all the time instead. <code>MEGAMORPHIC</code> also indicates that the inline cache will
no longer try to cache information about how to access the property locally (i.e. on the property access site),
but instead fall back to a global cache (the so-called <em>megamorphic stub cache</em>).</p>
<h2 id="scalability-issues">Scalability issues <a class="bookmark" href="#scalability-issues">#</a></h2>
<p>This <em>megamorphic stub cache</em> is a global cache of fixed size where all <code>MEGAMORPHIC</code> sites will try to cache the
lookup information for properties. This is important to understand, as especially for big applications that means
you'll likely have a lot of contention on this resource. Highly polymorphic sites like <code>instance.foo</code> in the
example above are prime candidates for contenders, and you'll only be affected by this non-local effect once you
start integrating your applications or once you trigger a significant number of hash collisions on the global
cache with a single property access site.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> <span class="token constant">N</span> <span class="token operator">=</span> <span class="token number">10000</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">test</span><span class="token punctuation">(</span><span class="token parameter">fn</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">var</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> i <span class="token operator"><</span> <span class="token constant">N</span><span class="token punctuation">;</span> <span class="token operator">++</span>i<span class="token punctuation">)</span> result <span class="token operator">=</span> <span class="token function">fn</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token function">test</span><span class="token punctuation">(</span><span class="token parameter">x</span> <span class="token operator">=></span> x<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">makeNaive</span><span class="token punctuation">(</span><span class="token parameter">klasses</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> instances <span class="token operator">=</span> klasses<span class="token punctuation">.</span><span class="token function">map</span><span class="token punctuation">(</span><span class="token parameter">klass</span> <span class="token operator">=></span> <span class="token keyword">new</span> <span class="token class-name">klass</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">function</span> <span class="token function">naive</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">let</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">const</span> instance <span class="token keyword">of</span> instances<span class="token punctuation">)</span> result <span class="token operator">=</span> instance<span class="token punctuation">.</span><span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> <span class="token constant">DEGREES</span> <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token number">100</span><span class="token punctuation">,</span> <span class="token number">300</span><span class="token punctuation">,</span> <span class="token number">500</span><span class="token punctuation">,</span> <span class="token number">700</span><span class="token punctuation">,</span> <span class="token number">900</span><span class="token punctuation">]</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">const</span> degree <span class="token keyword">of</span> <span class="token constant">DEGREES</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> <span class="token constant">KLASSES</span> <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">let</span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> i <span class="token operator"><</span> degree<span class="token punctuation">;</span> <span class="token operator">++</span>i<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token constant">KLASSES</span><span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span><span class="token function">eval</span><span class="token punctuation">(</span><span class="token string">"(class C"</span> <span class="token operator">+</span> i <span class="token operator">+</span> <span class="token string">" { foo() { }})"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"> <span class="token keyword">const</span> <span class="token constant">TESTS</span> <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token function">makeNaive</span><span class="token punctuation">(</span><span class="token constant">KLASSES</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> j <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> j <span class="token operator"><</span> <span class="token constant">TESTS</span><span class="token punctuation">.</span>length<span class="token punctuation">;</span> <span class="token operator">++</span>j<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">test</span><span class="token punctuation">(</span><span class="token constant">TESTS</span><span class="token punctuation">[</span>j<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> j <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> j <span class="token operator"><</span> <span class="token constant">TESTS</span><span class="token punctuation">.</span>length<span class="token punctuation">;</span> <span class="token operator">++</span>j<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">var</span> startTime <span class="token operator">=</span> Date<span class="token punctuation">.</span><span class="token function">now</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token function">test</span><span class="token punctuation">(</span><span class="token constant">TESTS</span><span class="token punctuation">[</span>j<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>degree <span class="token operator">+</span> <span class="token string">":"</span><span class="token punctuation">,</span> Date<span class="token punctuation">.</span><span class="token function">now</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span> startTime<span class="token punctuation">,</span> <span class="token string">"ms."</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>The example above illustrates this problem, even with just a single access site <code>instance.foo</code> inside of the
<code>naive</code> function, and varying degrees of polymorphism. Running this through <code>d8</code> (in V8 6.6) you'll see that
the solution doesn't scale very well:</p>
<pre><code>$ out/Release/d8 components3.js
100: 43 ms.
300: 167 ms.
500: 1286 ms.
700: 1866 ms.
900: 2428 ms.
</code></pre>
<p>It actually scales very poorly since the VM spends more and more time missing on the global cache due to
collisions. Notice especially the cliff when going from 300 different shapes to 500 different shapes.</p>
<h2 id="a-potential-solution">A potential solution <a class="bookmark" href="#a-potential-solution">#</a></h2>
<p>Unfortunately big, component-based applications have naturally a high degree of polymorphism by design.
So what can you do to mitigate the negative impact? The main problem is having property accesses such
as <code>instance.foo</code> where <code>instance</code> can have a high number of different shapes. So if you try to not
have that many of these accesses in your application, you'll likely reduce the negative impact on the
global cache.</p>
<p>One way to do this is to simply load the method ahead of time, remember it together with the
object reference and then use
<a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/call"><code>Function.prototype.call</code></a>
in the hot loop. A convenient way to avoid having to carry around the pair of object and function
yourself is to use
<a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/bind"><code>Function.prototype.bind</code></a>
instead.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> <span class="token constant">N</span> <span class="token operator">=</span> <span class="token number">10000</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">test</span><span class="token punctuation">(</span><span class="token parameter">fn</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">var</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> i <span class="token operator"><</span> <span class="token constant">N</span><span class="token punctuation">;</span> <span class="token operator">++</span>i<span class="token punctuation">)</span> result <span class="token operator">=</span> <span class="token function">fn</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token function">test</span><span class="token punctuation">(</span><span class="token parameter">x</span> <span class="token operator">=></span> x<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">makeNaive</span><span class="token punctuation">(</span><span class="token parameter">klasses</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> instances <span class="token operator">=</span> klasses<span class="token punctuation">.</span><span class="token function">map</span><span class="token punctuation">(</span><span class="token parameter">klass</span> <span class="token operator">=></span> <span class="token keyword">new</span> <span class="token class-name">klass</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">function</span> <span class="token function">naive</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">let</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">const</span> instance <span class="token keyword">of</span> instances<span class="token punctuation">)</span> result <span class="token operator">=</span> instance<span class="token punctuation">.</span><span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">makeCall</span><span class="token punctuation">(</span><span class="token parameter">klasses</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> pairs <span class="token operator">=</span> klasses<span class="token punctuation">.</span><span class="token function">map</span><span class="token punctuation">(</span><span class="token parameter">klass</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> instance <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">klass</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> method <span class="token operator">=</span> instance<span class="token punctuation">.</span>foo<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token punctuation">{</span> instance<span class="token punctuation">,</span> method <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">function</span> <span class="token function">call</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">let</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">const</span> <span class="token punctuation">{</span> instance<span class="token punctuation">,</span> method <span class="token punctuation">}</span> <span class="token keyword">of</span> pairs<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> result <span class="token operator">=</span> <span class="token function">method</span><span class="token punctuation">.</span><span class="token function">call</span><span class="token punctuation">(</span>instance<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">makeBound</span><span class="token punctuation">(</span><span class="token parameter">klasses</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> fns <span class="token operator">=</span> klasses<span class="token punctuation">.</span><span class="token function">map</span><span class="token punctuation">(</span><span class="token parameter">klass</span> <span class="token operator">=></span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> instance <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">klass</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> instance<span class="token punctuation">.</span><span class="token function">foo</span><span class="token punctuation">.</span><span class="token function">bind</span><span class="token punctuation">(</span>instance<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">function</span> <span class="token function">bound</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">let</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">const</span> fn <span class="token keyword">of</span> fns<span class="token punctuation">)</span> result <span class="token operator">=</span> <span class="token function">fn</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> result<span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> <span class="token constant">DEGREES</span> <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token number">100</span><span class="token punctuation">,</span> <span class="token number">300</span><span class="token punctuation">,</span> <span class="token number">500</span><span class="token punctuation">,</span> <span class="token number">700</span><span class="token punctuation">,</span> <span class="token number">900</span><span class="token punctuation">]</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">const</span> degree <span class="token keyword">of</span> <span class="token constant">DEGREES</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">const</span> <span class="token constant">KLASSES</span> <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">let</span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> i <span class="token operator"><</span> degree<span class="token punctuation">;</span> <span class="token operator">++</span>i<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token constant">KLASSES</span><span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span><span class="token function">eval</span><span class="token punctuation">(</span><span class="token string">"(class C"</span> <span class="token operator">+</span> i <span class="token operator">+</span> <span class="token string">" { foo() { }})"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"> <span class="token keyword">const</span> <span class="token constant">TESTS</span> <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token function">makeBound</span><span class="token punctuation">(</span><span class="token constant">KLASSES</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">makeCall</span><span class="token punctuation">(</span><span class="token constant">KLASSES</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token function">makeNaive</span><span class="token punctuation">(</span><span class="token constant">KLASSES</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> j <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> j <span class="token operator"><</span> <span class="token constant">TESTS</span><span class="token punctuation">.</span>length<span class="token punctuation">;</span> <span class="token operator">++</span>j<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token function">test</span><span class="token punctuation">(</span><span class="token constant">TESTS</span><span class="token punctuation">[</span>j<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"> <span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> j <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> j <span class="token operator"><</span> <span class="token constant">TESTS</span><span class="token punctuation">.</span>length<span class="token punctuation">;</span> <span class="token operator">++</span>j<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">var</span> startTime <span class="token operator">=</span> Date<span class="token punctuation">.</span><span class="token function">now</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token function">test</span><span class="token punctuation">(</span><span class="token constant">TESTS</span><span class="token punctuation">[</span>j<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span></span><br><span class="highlight-line"> degree <span class="token operator">+</span> <span class="token string">"|"</span> <span class="token operator">+</span> <span class="token constant">TESTS</span><span class="token punctuation">[</span>j<span class="token punctuation">]</span><span class="token punctuation">.</span>name <span class="token operator">+</span> <span class="token string">":"</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> Date<span class="token punctuation">.</span><span class="token function">now</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">-</span> startTime<span class="token punctuation">,</span></span><br><span class="highlight-line"> <span class="token string">"ms."</span></span><br><span class="highlight-line"> <span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>Running this with V8 6.6, both approaches scale equally well and avoid the problem of hot <code>MEGAMORPHIC</code>
property accesses.</p>
<p><img src="https://benediktmeurer.de/images/2018/results66-20180323.svg" alt="Results for V8 6.6" title="Results for V8 6.6"></p>
<p>So the takeaway here is that a high degree of polymorphism is not bad per se, but if you have a
lot of <code>MEGAMORPHIC</code> property accesses on the critical path, then eventually your application is
going to spend a lot of time fighting for entries in the <em>megamorphic stub cache</em> (at least with
how V8 works currently). For component-based systems there are ways to avoid this situation by
preloading methods used in hot code later.</p>
<h2 id="update">Update <a class="bookmark" href="#update">#</a></h2>
<p>Just ran the same test again with V8 6.7.0 (candidate) as of today, and it seems that it does
scale better, altough I'm unsure what changed in this area recently (I have to admit I haven't
worked on this area of V8 a lot).</p>
<p><img src="https://benediktmeurer.de/images/2018/results67-20180323.svg" alt="Results for V8 6.7" title="Results for V8 6.7"></p>
<h2 id="disclaimer">Disclaimer <a class="bookmark" href="#disclaimer">#</a></h2>
<p>For the vast majority of JavaScript code that is written today, the heuristics in the JavaScript
VMs (i.e. in V8) are perfectly fine, even for high degrees of polymorphism. So unless you're
writing a framework like React, you may not need to worry about what's written in here. Also the
usual advice still applies, more than ever: Write idiomatic JavaScript, let the engine take care
of the performance, optimize only when necessary and after careful profiling.</p>
An Introduction to Speculative Optimization in V82017-12-13T00:00:00-00:00https://benediktmeurer.de/2017/12/13/an-introduction-to-speculative-optimization-in-v8/<p>Following up on my talk <a href="https://www.youtube.com/watch?v=cvybnv79Sek">"A Tale of TurboFan"</a> (<a href="https://docs.google.com/presentation/d/1UXR1H2elTdAYJJ0Eed7lUctCVUserav9sAYSidxp8YE">slides</a>) at <a href="http://2017.js-kongress.de/">JS Kongress</a>, I wanted to give some additional context on how TurboFan, V8's optimizing compiler, works and how V8 turns your JavaScript into highly-optimized machine code. For the talk I had to be brief and leave out several details. So I'll use this opportunity to fill the gaps, especially how V8 collects and uses the profiling information to perform speculative optimizations.</p>
<h2 id="overview">Overview <a class="bookmark" href="#overview">#</a></h2>
<p>Before we dive into the details of how TurboFan works, I'll briefly explain how V8 works on a high level. Let's have a look at this <em>simplified breakdown of how V8 works</em> (taken from the <a href="https://medium.com/reloading/javascript-start-up-performance-69200f43b201">"JavaScript Start-up Performance"</a> blog post by my colleague <a href="https://twitter.com/addyosmani">Addy Osmani</a>):</p>
<p><img src="https://benediktmeurer.de/images/2017/addy-20171213.png" alt="How V8 Works" title="addy.png"></p>
<p>Whenever Chrome or Node.js has to execute some piece of JavaScript, it passes the source code to V8. V8 takes that JavaScript source code and feeds it to the so-called <a href="https://en.wikipedia.org/wiki/Parsing#Computer_languages"><em>Parser</em></a>, which creates an <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree"><em>Abstract Syntax Tree (AST)</em></a> representation for your source code. The talk <a href="https://www.youtube.com/watch?v=Fg7niTmNNLg">"Parsing JavaScript — better lazy than eager?"</a> from my colleague <a href="https://twitter.com/marjakh">Marja Hölttä</a> contains some details of how this works in V8. The AST is then passed on to the recently introduced <a href="https://v8.dev/blog/ignition-interpreter">Ignition Interpreter</a>, where it is turned into a sequence of bytecodes. This sequence of bytecodes is then executed by Ignition.</p>
<p>During execution, Ignition collects <em>profiling information</em> or <em>feedback</em> about the inputs to certain operations. Some of this feedback is used by Ignition itself to speed up subsequent interpretation of the bytecode. For example, for property accesses such as <code>o.x</code>, where <code>o</code> has the same shape all the time (i.e. you always pass a value <code>{x:v}</code> for <code>o</code> where <code>v</code> is a String), we cache information on how to get to the value of <code>x</code>. Upon subsequent execution of the same bytecode we don't need to search for <code>x</code> in <code>o</code> again. The underlying machinery here is called <a href="https://en.wikipedia.org/wiki/Inline_caching"><em>inline cache (IC)</em></a>. You can find a lot of details about how this works for property accesses in the blog post <a href="http://mrale.ph/blog/2015/01/11/whats-up-with-monomorphism.html">"What's up with monomorphism?"</a> by my colleague <a href="https://twitter.com/mraleph">Vyacheslav Egorov</a>.</p>
<p>Probably even more important — depending on your workload — the <em>feedback</em> collected by the Ignition interpreter is consumed by the <a href="https://v8.dev/blog/launching-ignition-and-turbofan">TurboFan JavaScript compiler</a> to generate highly-optimized machine code using a technique called <em>Speculative Optimization</em>. Here the optimizing compiler looks at what kinds of values were seen in the past and assumes that in the future we're going to see the same kinds of values. This allows TurboFan to leave out a lot of cases that it doesn't need to handle, which is extremely important to execute JavaScript at peak performance.</p>
<h2 id="the-basic-execution-pipeline">The Basic Execution Pipeline <a class="bookmark" href="#the-basic-execution-pipeline">#</a></h2>
<p>Let's consider a reduced version of the example from my talk, focusing solely on the function <code>add</code>, and how this is executed by V8.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">add</span><span class="token punctuation">(</span><span class="token parameter">x<span class="token punctuation">,</span> y</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> x <span class="token operator">+</span> y<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token function">add</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>If you run this in the Chrome DevTools console, you'll see that it outputs the expected result <code>3</code>:</p>
<p><img src="https://benediktmeurer.de/images/2017/devtools-20171213.png" alt="Chrome DevTools" title="devtools.png"></p>
<p>Let's examine what happens under the hood in V8 to actually get to these results. We'll do this step by step for the function <code>add</code>. As mentioned before, we first need to parse the function source code and turn that into an Abstract Syntax Tree (AST). This is done by the <code>Parser</code>. You can see the AST that V8 generates internally using the <code>--print-ast</code> command line flag in a Debug build of the <a href="https://v8.dev/docs/d8"><code>d8 shell</code></a>.</p>
<pre><code>$ out/Debug/d8 --print-ast add.js
…
--- AST ---
FUNC at 12
. KIND 0
. SUSPEND COUNT 0
. NAME "add"
. PARAMS
. . VAR (0x7fbd5e818210) (mode = VAR) "x"
. . VAR (0x7fbd5e818240) (mode = VAR) "y"
. RETURN at 23
. . ADD at 32
. . . VAR PROXY parameter[0] (0x7fbd5e818210) (mode = VAR) "x"
. . . VAR PROXY parameter[1] (0x7fbd5e818240) (mode = VAR) "y"
</code></pre>
<p>This format is not very easy to consume, so let's visualize it.</p>
<p><img src="https://benediktmeurer.de/images/2017/ast-20171213.svg" alt="Abstract Syntax Tree" title="Abstract Syntax Tree"></p>
<p>Initially the function literal for <code>add</code> is parsed into a tree representation, with one subtree for the parameter declarations and one subtree for the actual function body. During parsing it is impossible to tell which names correspond to which variables in the program, mostly due to the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/var#var_hoisting"><em>funny <code>var</code> hoisting rules</em></a> and <code>eval</code> in JavaScript, but also for other reasons. So for every name the parser initially creates so-called <code>VAR PROXY</code> nodes. The subsequent scope resolution step connects these <code>VAR PROXY</code> nodes to the declaring <code>VAR</code> nodes or marks them as either <em>global</em> or <em>dynamic lookups</em>, depending on whether the parser has seen an <code>eval</code> expression in one of the surrounding scopes.</p>
<p>Once this is done we have a complete AST that contains all the necessary information to generate executable bytecode from it. The AST is then passed to the <code>BytecodeGenerator</code>, which is the part of the Ignition interpreter that generates bytecode on a per-function basis. You can also see the bytecode being generated by V8 using the flag <code>--print-bytecode</code> with the <code>d8</code> shell (or with <code>node</code>).</p>
<pre><code>$ out/Debug/d8 --print-bytecode add.js
…
[generated bytecode for function: add]
Parameter count 3
Frame size 0
12 E> 0x37738712a02a @ 0 : 94 StackCheck
23 S> 0x37738712a02b @ 1 : 1d 02 Ldar a1
32 E> 0x37738712a02d @ 3 : 29 03 00 Add a0, [0]
36 S> 0x37738712a030 @ 6 : 98 Return
Constant pool (size = 0)
Handler Table (size = 16)
</code></pre>
<p>This tells us that a new bytecode object was generated for the function <code>add</code>, which accepts three parameters: the implicit receiver <code>this</code>, and the explicit formal parameters <code>x</code> and <code>y</code>. The function doesn't need any local variables (the frame size is zero), and contains the sequence of four bytecodes:</p>
<pre><code>StackCheck
Ldar a1
Add a0, [0]
Return
</code></pre>
<p>To explain that, we first need to understand how the interpreter works on a high level. Ignition uses a so-called <a href="https://en.wikipedia.org/wiki/Register_machine"><em>register machine</em></a> (in contrast to the <em>stack machine</em> approach that was used by earlier V8 versions in the FullCodegen compiler). It holds its local state in interpreter registers, some of which map to real CPU registers, while others map to specific slots in the native machine stack memory.</p>
<p><img src="https://benediktmeurer.de/images/2017/interpreter-20171213.svg" alt="Interpreter overview" title="Interpreter Overview"></p>
<p>The special registers <code>a0</code> and <code>a1</code> correspond to the formal parameters for the function on the machine stack (in this case we have two formal parameters). Formal parameters are the parameters declared in the source code, which might be different from the actual number of parameters passed to the function at runtime. The last computed value of each bytecode is usually kept in a special register called the <code>accumulator</code>, the current <em>stack frame</em> or <em>activation record</em> is identified by the <code>stack pointer</code>, and the <code>program counter</code> points to the currently executed instruction in the bytecode. Let's check what the individual bytecodes do in this example:</p>
<ul>
<li><code>StackCheck</code> compares the <code>stack pointer</code> to some known upper limit (actually a lower limit since the stack grows downwards on all architectures supported by V8). If the stack grows above a certain threshold, we abort execution of the function and throw a <code>RangeError</code> saying that the stack was overflowed.</li>
<li><code>Ldar a1</code> loads the value of the register <code>a1</code> into the <code>accumulator</code> register (the name stands for <strong><em>LoaD Accumulator Register</em></strong>).</li>
<li><code>Add a0, [0]</code> loads the value from the <code>a0</code> register and adds it to the value in the <code>accumulator</code> register. The result is then placed into the <code>accumulator</code> register again. Note that <em>addition</em> here can also mean string concatenation, and that this operation can execute <strong>arbitrary JavaScript</strong> depending on the operands. The <a href="https://tc39.github.io/ecma262/#sec-addition-operator-plus"><code>+</code> operator</a> in JavaScript is really complex, and many people have tried to illustrate the complexity in talks. <a href="https://twitter.com/editingemily">Emily Freeman</a> recently gave a talk at JS Kongress titled <a href="https://www.youtube.com/watch?v=v8ToNvB-_Q8">"JavaScript's "+" Operator and Decision Fatigue"</a> on precisely this topic.
The <code>[0]</code> operand to the <code>Add</code> operator refers to a <em>feedback vector slot</em>, where Ignition stores the profiling information about the values we've seen during execution of the function. We'll get back to this later when we investigate how TurboFan optimizes the function.</li>
<li><code>Return</code> ends execution of the current function and transfers control back to the caller. The value returned is the current value in the <code>accumulator</code> register.</li>
</ul>
<p>My colleague <a href="https://twitter.com/fhinkel">Franziska Hinkelmann</a> wrote an article <a href="https://medium.com/dailyjs/understanding-v8s-bytecode-317d46c94775">"Understanding V8's Bytecode"</a> a while ago that gives some additional insight into how V8's bytecode works.</p>
<h2 id="speculative-optimization">Speculative Optimization <a class="bookmark" href="#speculative-optimization">#</a></h2>
<p>Now that you have a rough understanding of how V8 executes your JavaScript in the baseline case, it's time to start looking into how TurboFan fits into the picture, and how your JavaScript code can be turned into highly optimized machine code. The <a href="https://tc39.github.io/ecma262/#sec-addition-operator-plus"><code>+</code> operator</a> is already such a complex operation in JavaScript which has to do a lot of checks before it eventually does the number addition on the inputs.</p>
<p><img src="https://benediktmeurer.de/images/2017/add-operator-20171213.png" alt="Runtime Semantics of the + operator" title="add-operator.png"></p>
<p>It's not immediately obvious how this can be done in just a few machine instructions to reach peak performance (comparable to Java or C++ code). The magic keyword here is <em>Speculative Optimization</em>, which makes use of assumptions about possible inputs. For example, when we know that in the case of <code>x+y</code>, both <code>x</code> and <code>y</code> are numbers, we don't need to handle the cases where either of them is a string, or even worse — the case where the operands can be arbitrary JavaScript objects on which we need to run the abstract operation <a href="https://tc39.github.io/ecma262/#sec-toprimitive"><code>ToPrimitive</code></a> first.</p>
<p><img src="https://benediktmeurer.de/images/2017/to-primitive-20171213.png" alt="ToPrimitive operation" title="to-primitive.png"></p>
<p>Knowing that both <code>x</code> and <code>y</code> are numbers also means that we can rule out observable side effects — for example we know it cannot shut down the computer or write to a file or navigate to a different page. In addition we know that the operation won't throw an exception. Both of these are important for optimizations, because an optimizing compiler can only eliminate an expression if it knows for sure that this expression won't cause any observable side effects and doesn't raise exceptions.</p>
<p>Due to the dynamic nature of JavaScript we usually don't know the precise types of values until runtime, i.e. just by looking at the source code it's often impossible to tell the possible values of inputs to operations. That's why we need to speculate, based on previously collected <em>feedback</em> about the values we've seen so far, and then assume that we're going to always see similar values in the future. This might sound fairly limited, but it has proven to work well for dynamic languages like JavaScript.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">add</span><span class="token punctuation">(</span><span class="token parameter">x<span class="token punctuation">,</span> y</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> x <span class="token operator">+</span> y<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>In this particular case, we collect information about the input operands and the resulting value of the + operation (the <code>Add</code> bytecode). When we optimize this code with TurboFan and we've seen only numbers so far, we put checks in place to check that both <code>x</code> and <code>y</code> are numbers (in that case we know that the result is going to be a number as well). If either of these checks fail we go back to interpreting the bytecode instead — a process called <em>Deoptimization</em>. Thus TurboFan doesn't need to worry about all these other cases of the <code>+</code> operator and doesn't even need to emit machine code to handle those, but can focus on the case for numbers, which translates well to machine instructions.</p>
<p><img src="https://benediktmeurer.de/images/2017/closure-20171213.svg" alt="Closure structure" title="Closure Structure"></p>
<p>The feedback collected by Ignition is stored in the so-called <em>Feedback Vector</em> (previously named <em>Type Feedback Vector</em>). This special data structure is linked from the closure and contains slots to store different kinds of feedback, i.e. bitsets, closures or hidden classes, depending on the concrete <em>inline cache (IC)</em>. My colleague <a href="https://twitter.com/ripsawridge">Michael Stanton</a> gave a nice presentation at <a href="https://amsterdamjs.com/">AmsterdamJS</a> earlier this year titled <a href="https://www.youtube.com/watch?v=u7zRSm8jzvA">"V8 and How It Listens to You"</a>, which explains some of the concepts of the Feedback Vector in detail. The closure also links to the <em>SharedFunctionInfo</em>, which contains the general information about the function (like source position, bytecode, strict/sloppy mode, etc.), and there's a link to the <em>context</em> as well, which contains the values for the free variables of the function and provides access to the global object (i.e. the <code><iframe></code> specific data structures).</p>
<p>In the case of the add function, the Feedback Vector has exactly one interesting slot (in addition to the general slots like the call count slot), and this is a <code>BinaryOp</code> slot, where binary operations like <code>+</code>, <code>-</code>, <code>*</code>, etc. can record feedback about the inputs and outputs that were seen so far. You can check what's inside the feedback vector of a specific closure using the specialized <code>%DebugPrint()</code> intrinsic when running with the <code>--allow-natives-syntax</code> command line flag (in a Debug build of <code>d8</code>).</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">add</span><span class="token punctuation">(</span><span class="token parameter">x<span class="token punctuation">,</span> y</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> x <span class="token operator">+</span> y<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token function">add</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token operator">%</span><span class="token function">DebugPrint</span><span class="token punctuation">(</span>add<span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Running this with <code>--allow-natives-syntax</code> in <code>d8</code> we observe:</p>
<pre class="language-html"><code class="language-html"><span class="highlight-line">$ out/Debug/d8 --allow-natives-syntax add.js DebugPrint: 0xb5101ea9d89:</span><br><span class="highlight-line">[Function] in OldSpace … - feedback vector: 0xb5101eaa091: [FeedbackVector] in</span><br><span class="highlight-line">OldSpace - length: 1 SharedFunctionInfo: 0xb5101ea99c9</span><br><span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>SharedFunctionInfo</span> <span class="token attr-name">add</span><span class="token punctuation">></span></span></span><br><span class="highlight-line"> Optimized Code: 0 Invocation Count: 1 Profiler Ticks: 0 Slot #0 BinaryOp</span><br> BinaryOp:SignedSmall …<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>SharedFunctionInfo</span><br><span class="token punctuation">></span></span></code></pre>
<p>We can see the invocation count is 1, since we ran the function <code>add</code> exactly once. Also there's no optimized code yet (indicated by the arguably confusing <code>0</code> output). But there's exactly one slot in the Feedback Vector, which is a <code>BinaryOp</code> slot whose current feedback is <code>SignedSmall</code>. What does that mean? The bytecode <code>Add</code> that refers to the feedback slot 0 has only seen inputs of type <code>SignedSmall</code> so far and has also only produced outputs of type <code>SignedSmall</code> up until now.</p>
<p>But what is this <code>SignedSmall</code> type about? JavaScript doesn't have a type of that name. The name comes from an optimization that is done in V8 when representing small signed integer values that occur frequently enough in programs to deserve a special treatment (other JavaScript engines have similar optimizations).</p>
<h3 id="excurse%3A-value-representation">Excurse: Value Representation <a class="bookmark" href="#excurse%3A-value-representation">#</a></h3>
<p>Let's briefly explore how JavaScript values are represented in V8 to better understand the underlying concept. V8 uses a technique called <a href="https://en.wikipedia.org/wiki/Tagged_pointer">Pointer Tagging</a> to represent values in general. Most of the values we deal with live in the JavaScript heap, and have to be managed by the garbage collector (GC). But for some values it would be too expensive to always allocate them in memory. Especially for small integer values that are often used as indices to arrays and temporary computation results.</p>
<p><img src="https://benediktmeurer.de/images/2017/tagging-20171213.svg" alt="Tagging Scheme" title="Tagging Scheme"></p>
<p>In V8, we have two possible <em>tagged representations</em>: A <em>Smi</em> (short for <strong><em>Small Integer</em></strong>) and a <em>HeapObject</em>, which points to memory in the managed heap. We make use of the fact that all allocated objects are aligned on word boundaries (64-bit or 32-bit depending on the architecture), which means that the 2 or 3 least significant bits are always zero. We use the least significant bit to distinguish between a <em>HeapObject</em> (bit is 1) and a <em>Smi</em> (bit is 0). For <em>Smi</em> on 64-bit architectures the least significant 32 bits are actually all zero and the signed 32-bit value is stored in the upper half of the word. This is to allow efficient access to the 32-bit value in memory using a single machine instruction instead of having to load and shift the value, but also because 32-bit arithmetic is common for bitwise operations in JavaScript.</p>
<p>On 32-bit architectures, the <em>Smi</em> representation has the least significant bit set to 0 and a signed 31-bit value shifted to the left by one stored in the upper 31-bit of the word.</p>
<h3 id="feedback-lattice">Feedback Lattice <a class="bookmark" href="#feedback-lattice">#</a></h3>
<p>The <code>SignedSmall</code> feedback type refers to all values that have <em>Smi</em> representation. For the <code>Add</code> operation it means that it has only seen inputs represented as <em>Smi</em> so far and all outputs that were produced could also be represented as <em>Smi</em> (i.e. the values didn't overflow the range of possible 32-bit integer values). Let's check what happens if we also call add with other numbers that are not representable as <em>Smi</em>.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">add</span><span class="token punctuation">(</span><span class="token parameter">x<span class="token punctuation">,</span> y</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> x <span class="token operator">+</span> y<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line">console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token function">add</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line">console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span><span class="token function">add</span><span class="token punctuation">(</span><span class="token number">1.1</span><span class="token punctuation">,</span> <span class="token number">2.2</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token operator">%</span><span class="token function">DebugPrint</span><span class="token punctuation">(</span>add<span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>Running this again with <code>--allow-natives-syntax</code> in <code>d8</code> we observe:</p>
<pre class="language-html"><code class="language-html"><span class="highlight-line">$ out/Debug/d8 --allow-natives-syntax add.js DebugPrint: 0xb5101ea9d89:</span><br><span class="highlight-line">[Function] in OldSpace … - feedback vector: 0x3fd6ea9ef9: [FeedbackVector] in</span><br><span class="highlight-line">OldSpace - length: 1 SharedFunctionInfo: 0x3fd6ea9989</span><br><span class="highlight-line"><span class="token tag"><span class="token tag"><span class="token punctuation"><</span>SharedFunctionInfo</span> <span class="token attr-name">add</span><span class="token punctuation">></span></span></span><br><span class="highlight-line"> Optimized Code: 0 Invocation Count: 2 Profiler Ticks: 0 Slot #0 BinaryOp</span><br> BinaryOp:Number …<span class="token tag"><span class="token tag"><span class="token punctuation"></</span>SharedFunctionInfo</span><br><span class="token punctuation">></span></span></code></pre>
<p>First of all, we see that the invocation count is now 2, since we ran the function twice. And then we see that the <code>BinaryOp</code> slot value changed to <code>Number</code>, which indicates that we've seen arbitrary numbers for the addition (i.e. non-integer values). For addition there's a lattice of possible states for feedback, which roughly looks like this:</p>
<p><img src="https://benediktmeurer.de/images/2017/lattice-20171213.svg" alt="Feedback Lattice" title="Feedback lattice"></p>
<p>The feedback starts as <code>None</code>, which indicates that we haven't seen anything so far, so we don't know anything. The <code>Any</code> state indicates that we have seen a combination of incompatible inputs or outputs. The <code>Any</code> state thus indicates that the <code>Add</code> is considered <em>polymorphic</em>. In contrast, the remaining states indicate that the <code>Add</code> is <em>monomorphic</em>, because it has seen and produced only values that are somewhat the same.</p>
<ul>
<li><code>SignedSmall</code> means that all values have been small integers (signed 32-bit or 31-bit depending on the word size of the architecture), and all of them have been represented as <em>Smi</em>.</li>
<li><code>Number</code> indicates that all values have been regular numbers (this includes <code>SignedSmall</code>).</li>
<li><code>NumberOrOddball</code> includes all the values from <code>Number</code> plus <code>undefined</code>, <code>null</code>, <code>true</code> and <code>false</code>.</li>
<li><code>String</code> means that both inputs have been string values.</li>
<li><code>BigInt</code> means that both inputs have been BigInts, see the current <a href="https://tc39.github.io/proposal-bigint/">stage 2 proposal</a> for details.</li>
</ul>
<p>It's important to note that the feedback can only progress in this lattice. It's impossible to ever go back. If we'd ever go back then we risk entering a so-called <em>deoptimization loop</em> where the optimizing compiler consumes feedback and bails out from optimized code (back to the interpreter) whenever it sees values that don't agree with the feedback. The next time the function gets hot we will eventually optimize it again. So if we didn't progress in the lattice then TurboFan would generate the same code again, which effectively means it will bail out on the same kind of input again. Thus the engine would be busy just optimizing and deoptimizing code, instead of running your JavaScript code at high speed.</p>
<h2 id="the-optimization-pipeline">The Optimization Pipeline <a class="bookmark" href="#the-optimization-pipeline">#</a></h2>
<p>Now that we know how Ignition collects feedback for the <code>add</code> function, let's see how TurboFan makes use of that feedback to generate minimal code. I'll use the special intrinsic <code>%OptimizeFunctionOnNextCall()</code> to trigger optimization of a function in V8 at a very specific point in time. We often use these intrinsics to write tests that stress the engine in a very specific way.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">add</span><span class="token punctuation">(</span><span class="token parameter">x<span class="token punctuation">,</span> y</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> x <span class="token operator">+</span> y<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token function">add</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// Warm up with SignedSmall feedback.</span></span><br><span class="highlight-line"><span class="token operator">%</span><span class="token function">OptimizeFunctionOnNextCall</span><span class="token punctuation">(</span>add<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">add</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// Optimize and run generated code.</span></span></code></pre>
<p>Here we explicitly warm up the <code>x+y</code> site with <code>SignedSmall</code> feedback by passing in two integer values whose sum also fits into the small integer range. Then we tell V8 that it should optimize the function <code>add</code> (with TurboFan) when it's called the next time, and eventually we call <code>add</code>, which triggers TurboFan and then runs the generated machine code.</p>
<p><img src="https://benediktmeurer.de/images/2017/turbofan-20171213.svg" alt="TurboFan" title="TurboFan"></p>
<p>TurboFan takes the bytecode that was previously generated for <code>add</code> and extracts the relevant feedback from the <code>FeedbackVector</code> of <code>add</code>. It turns this into a graph representation and passes the graph through the various phases of the frontend, optimization and backend stages. I'm not going into the details of the passes here, that's a topic for a separate blog post (or a series of separate blog posts). Instead we're going to look at the generated machine code and see how the speculative optimization works. You can see the code generated by TurboFan by passing the <code>--print-opt-code</code> flag to <code>d8</code>.</p>
<p><img src="https://benediktmeurer.de/images/2017/overview-20171213.svg" alt="Generated assembly code" title="Generated assembly overview"></p>
<p>This is the x64 machine code that is generated by TurboFan, with annotations from me and leaving out some technical details that don't matter (i.e. the exact call sequence to the <code>Deoptimizer</code>). So let's see what the code does:</p>
<pre class="language-asm"><code class="language-asm"><span class="highlight-line"># Prologue</span><br><span class="highlight-line">leaq <span class="token register variable">rcx</span>,<span class="token operator">[</span>rip<span class="token operator">+</span><span class="token number">0x0</span><span class="token operator">]</span></span><br><span class="highlight-line">movq <span class="token register variable">rcx</span>,<span class="token operator">[</span><span class="token register variable">rcx</span><span class="token operator">-</span><span class="token number">0x37</span><span class="token operator">]</span></span><br><span class="highlight-line">testb <span class="token operator">[</span><span class="token register variable">rcx</span><span class="token operator">+</span><span class="token number">0xf</span><span class="token operator">]</span>,<span class="token number">0x1</span></span><br><span class="highlight-line">jnz CompileLazyDeoptimizedCode</span><br><span class="highlight-line">push <span class="token register variable">rbp</span></span><br><span class="highlight-line">movq <span class="token register variable">rbp</span>,<span class="token register variable">rsp</span></span><br><span class="highlight-line">push <span class="token register variable">rsi</span></span><br><span class="highlight-line">push <span class="token register variable">rdi</span></span><br><span class="highlight-line">cmpq <span class="token register variable">rsp</span>,<span class="token operator">[</span><span class="token register variable">r13</span><span class="token operator">+</span><span class="token number">0xdb0</span><span class="token operator">]</span></span><br><span class="highlight-line">jna StackCheck</span></code></pre>
<p>The prologue checks whether the code object is still valid or whether some condition changed which requires us to throw away the code object. This was recently introduced by my intern <a href="https://twitter.com/jupvfranco">Juliana Franco</a> as part of her <a href="https://v8.dev/blog/lazy-unlinking">"Internship on Laziness"</a>. Once we know that the code is still valid, we build the <em>stack frame</em> and check that there's enough space left on the stack to execute the code.</p>
<pre class="language-asm"><code class="language-asm"><span class="highlight-line"># Check x is a small integer</span><br><span class="highlight-line">movq <span class="token register variable">rax</span>,<span class="token operator">[</span><span class="token register variable">rbp</span><span class="token operator">+</span><span class="token number">0x18</span><span class="token operator">]</span></span><br><span class="highlight-line">test <span class="token register variable">al</span>,<span class="token number">0x1</span></span><br><span class="highlight-line">jnz Deoptimize</span><br><span class="highlight-line"># Check y is a small integer</span><br><span class="highlight-line">movq <span class="token register variable">rbx</span>,<span class="token operator">[</span><span class="token register variable">rbp</span><span class="token operator">+</span><span class="token number">0x10</span><span class="token operator">]</span></span><br><span class="highlight-line">testb <span class="token register variable">rbx</span>,<span class="token number">0x1</span></span><br><span class="highlight-line">jnz Deoptimize</span><br><span class="highlight-line"># Convert y from Smi to Word32</span><br><span class="highlight-line">movq <span class="token register variable">rdx</span>,<span class="token register variable">rbx</span></span><br><span class="highlight-line">shrq <span class="token register variable">rdx</span>, <span class="token number">32</span></span><br><span class="highlight-line"># Convert x from Smi to Word32</span><br><span class="highlight-line">movq <span class="token register variable">rcx</span>,<span class="token register variable">rax</span></span><br><span class="highlight-line">shrq <span class="token register variable">rcx</span>, <span class="token number">32</span></span></code></pre>
<p>Then we start with the body of the function. We load the values of the parameters <code>x</code> and <code>y</code> from the stack (relative to the frame pointer in <code>rbp</code>) and check if both values have <em>Smi</em> representation (since feedback for + says that both inputs have always been <em>Smi</em> so far). This is done by testing the least significant bit. Once we know that they are both represented as <em>Smi</em>, we need to convert them to 32-bit representation, which is done by shifting the value by 32 bit to the right.</p>
<p>If either <code>x</code> or <code>y</code> is not a <em>Smi</em> the execution of the optimized code aborts immediately and the <code>Deoptimizer</code> restores the state of the function in the interpreter right before the <code>Add</code>.</p>
<p>Side note: We could also perform the addition on the <em>Smi</em> representation here; that's what our previous optimizing compiler Crankshaft did. This would save us the shifting, but currently TurboFan doesn't have a good heuristic to decide whether it's beneficial to do the operation on <em>Smi</em> instead, which is not always the ideal choice and highly dependent on the context in which this operation is used.</p>
<pre class="language-asm"><code class="language-asm"><span class="highlight-line"># Add x and y (incl. overflow check)</span><br><span class="highlight-line">addl <span class="token register variable">rdx</span>,<span class="token register variable">rcx</span></span><br><span class="highlight-line">jo Deoptimize</span><br><span class="highlight-line"># Convert result to Smi</span><br><span class="highlight-line">shlq <span class="token register variable">rdx</span>, <span class="token number">32</span></span><br><span class="highlight-line">movq <span class="token register variable">rax</span>,<span class="token register variable">rdx</span></span><br><span class="highlight-line"># Epilogue</span><br><span class="highlight-line">movq <span class="token register variable">rsp</span>,<span class="token register variable">rbp</span></span><br><span class="highlight-line">pop <span class="token register variable">rbp</span></span><br><span class="highlight-line">ret <span class="token number">0x18</span></span></code></pre>
<p>Then we go on to perform the integer addition on the inputs. We need to test explicitly for overflow, since the result of the addition might be outside the range of 32-bit integers, in which case we'd need to go back to the interpreter, which will then learn <code>Number</code> feedback on the <code>Add</code>. Finally we convert the result back to <em>Smi</em> representation by shifting the signed 32-bit value up by 32 bit, and then we return the value in the accumulator register <code>rax</code>.</p>
<p>As said before, this is not yet the perfect code for this case, since here it would be beneficial to just perform the addition on <em>Smi</em> representation directly, instead of going to <em>Word32</em>, which would save us three shift instructions. But even putting aside this minor aspect, you can see that the generated code is highly optimized and specialized to the profiling feedback. It doesn't even try to deal with other numbers, strings, big ints or arbitrary JavaScript objects here, but focuses only on the kind of values we've seen so far. This is the <strong>key ingredient</strong> to peak performance for many JavaScript applications.</p>
<h3 id="making-progress">Making progress <a class="bookmark" href="#making-progress">#</a></h3>
<p>So what if you suddenly change your mind and want to add numbers instead? Let's change the example to something like this instead:</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">add</span><span class="token punctuation">(</span><span class="token parameter">x<span class="token punctuation">,</span> y</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> x <span class="token operator">+</span> y<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token function">add</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// Warm up with SignedSmall feedback.</span></span><br><span class="highlight-line"><span class="token operator">%</span><span class="token function">OptimizeFunctionOnNextCall</span><span class="token punctuation">(</span>add<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">add</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// Optimize and run generated code.</span></span><br><span class="highlight-line"><span class="token function">add</span><span class="token punctuation">(</span><span class="token number">1.1</span><span class="token punctuation">,</span> <span class="token number">2.2</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// Oops?!</span></span></code></pre>
<p>Running this with <code>--allow-natives-syntax</code> and <code>--trace-deopt</code> we observe the following:</p>
<p><img src="https://benediktmeurer.de/images/2017/deopt-20171213.png" alt="Deoptimization example" title="deopt.png"></p>
<p>That's a lot of confusing output. But let's extract the important bits. First of all, we print a reason why we had to deoptimize, and in this case it's <code>not a Smi</code>, which means we baked in the assumption somewhere that a value is a <em>Smi</em>, but now we saw a <em>HeapObject</em> instead. Indeed it's the value in <code>rax</code>, which is supposed to be a <em>Smi</em>, but it's the number 1.1 instead. So we fail on the first check for the <code>x</code> parameter and we need to deoptimize to go back to interpreting the bytecode. That is a topic for a separate article though.</p>
<h2 id="takeaway">Takeaway <a class="bookmark" href="#takeaway">#</a></h2>
<p>I hope you enjoyed this dive into how speculative optimization works in V8 and how it helps us to reach peak performance for JavaScript applications. Don't worry too much about these details though. When writing applications in JavaScript focus on the application design instead and make sure to use appropriate data structures and algorithms. Write idiomatic JavaScript, and let us worry about the low level bits of the JavaScript performance instead. If you find something that is too slow, and it shouldn't be slow, please <a href="http://crbug.com/v8/new">file a bug report</a>, so we get a chance to look into that.</p>
Connecting the dots2017-10-05T00:00:00-00:00https://benediktmeurer.de/2017/10/05/connecting-the-dots/<p>The last months have been a hectic time for me. I was hosting my first intern <a href="https://twitter.com/jupvfranco">Juliana Franco</a>
at Google working on the Deoptimizer during her <a href="https://twitter.com/v8js/status/915473224187760640"><em>internship on lazyiness</em></a>.
Then I was diagnosed with articular gout and almost couldn't walk for a week. And we finally moved into our new house with a lot
of help from my awesome colleagues on the V8 team. On the <a href="https://nodejs.org/">Node.js</a> front, I became the <a href="https://twitter.com/bmeurer/status/896996151731343361">tech lead of Node
performance</a> in the V8 team, joined the <a href="https://github.com/nodejs/benchmarking/commit/681b4e3570fdc3658db4fab7952ca2934d7d6f14">Node.js benchmarking working
group</a>, and I am now officially a
<a href="https://twitter.com/trott/status/913640867600019456">Node.js collaborator</a>. But I also had some time to close gaps in V8
performance now that <a href="https://v8project.blogspot.de/2017/05/launching-ignition-and-turbofan.html">Ignition and TurboFan</a>
finally launched <a href="https://medium.com/the-node-js-collection/node-js-8-3-0-is-now-available-shipping-with-the-ignition-turbofan-execution-pipeline-aa5875ad3367">everywhere</a>.</p>
<p>So today I'll give an overview of what I've done in V8 land in the last month, which is also a sneak preview of what's going
to change performance-wise in V8 6.3 (which will ship as part of Chrome 63 and probably Node 9 at some point). Fortunately
most of the changes were straight-forward at this point, because most of the infrastructure for the optimizations is in
place already and we only need to connect the dots now.</p>
<p>I'm going to use V8 version 5.8 (which is the last version before we switched to Ignition and TurboFan in Chrome),
6.1 (which is the current stable version in Chrome and also in Node 8) and 6.3 (which is the current development version)
for the performance comparisons.</p>
<h2 id="an-internship-on-laziness">An internship on laziness <a class="bookmark" href="#an-internship-on-laziness">#</a></h2>
<p>As mentioned above this was the first time I hosted an intern at Google. It sure comes with a lot of work and some
additional responsibilities, but it was super exciting. <a href="https://twitter.com/jupvfranco">Juliana</a> was an awesome
intern. I have to admit that the area that she was working on (<a href="https://goo.gl/N7hwEp">lazy deoptimization without code patching</a>
and <a href="https://v8.dev/blog/lazy-unlinking">lazy unlinking of deoptimized functions</a>) is not
my main area of expertise, so I had to rely a lot on my colleagues <a href="https://twitter.com/jarinsev">Jaroslav Sevcik</a>
and <a href="https://twitter.com/starzi">Michael Starzinger</a> to help her out. But the end result is just amazing, especially
getting rid of the weakly linked list of all closures in V8 is a major accomplishment.</p>
<p><img src="https://benediktmeurer.de/images/2017/jupvfranco-20171005.png" alt="Octane/Early Boyer"></p>
<p>So I'd like to use this opportunity here to say: <em>Thank you, <a href="https://twitter.com/jupvfranco">Juliana</a>, for spending
the summer with us, it was awesome to have you as an intern!</em></p>
<h2 id="object-constructor-calls-in-webpack-bundles">Object constructor calls in webpack bundles <a class="bookmark" href="#object-constructor-calls-in-webpack-bundles">#</a></h2>
<p>I already wrote a detailed <a href="https://benediktmeurer.de/2017/08/31/object-constructor-calls-in-webpack-bundles/">article</a> about this topic. To summarize,
<a href="https://webpack.js.org/">webpack</a> v3 generates code the following bundled code</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">var</span> m <span class="token operator">=</span> <span class="token function">__webpack_require__</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token function">Object</span><span class="token punctuation">(</span>m<span class="token punctuation">.</span>foo<span class="token punctuation">)</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// <- called without this (undefined/global object)</span></span></code></pre>
<p>for</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">import</span> foo <span class="token keyword">from</span> <span class="token string">"module"</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token function">foo</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// <- called without this (undefined/global object)</span></span></code></pre>
<p>essentially wrapping the target in the
<a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object"><code>Object</code> constructor</a>
to make sure the callee get's passed either <code>undefined</code> if
it's in strict mode or the global object if it's a sloppy mode function. By <a href="https://chromium-review.googlesource.com/c/v8/v8/+/643868">teaching TurboFan about the <code>Object</code>
constructor</a> we were able to close the performance gap
and make these calls as fast as direct calls or indirect calls via the
<a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/call"><code>Function.prototype.call</code></a>
builtin.</p>
<p><img src="https://benediktmeurer.de/images/2017/results-20170831.svg" alt="Object constructor calls performance results"></p>
<p>What was awesome about this change is that it went <a href="https://twitter.com/v8js/status/903150329973403648">viral</a> and soon
all major JavaScript engines, including <a href="https://twitter.com/SpiderMonkeyJS/status/903572265379520512">SpiderMonkey</a>,
<a href="https://twitter.com/Constellation/status/903841886367997952">JavaScriptCore</a> and
<a href="https://twitter.com/KuXMLP/status/904011957962915841">ChakraCore</a>, will support this optimization.</p>
<p><img src="https://benediktmeurer.de/images/2017/collaboration-20171005.png" alt="Collaboration"></p>
<h2 id="restoring-for..in-peak-performance">Restoring <code>for..in</code> peak performance <a class="bookmark" href="#restoring-for..in-peak-performance">#</a></h2>
<p>This was also already described in a <a href="https://benediktmeurer.de/2017/09/07/restoring-for-in-peak-performance/">detailed blog post</a>. The <strong>TL;DR</strong>
is that when we <a href="https://v8.dev/blog/launching-ignition-and-turbofan">launched Ignition and TurboFan</a>,
we did so with a couple of regressions that we'd need to address once the dust settles. One of the major performance hits
was a <strong>4-5x</strong> regression in <code>for..in</code> peak performance as noticed in <a href="https://www.nearform.com/blog/node-js-is-getting-a-new-v8-with-turbofan/">Get ready: A new V8 is coming, Node.js performance is
changing</a>.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">var</span> obj <span class="token operator">=</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> x<span class="token punctuation">:</span> <span class="token number">1</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> y<span class="token punctuation">:</span> <span class="token number">1</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> z<span class="token punctuation">:</span> <span class="token number">1</span></span><br><span class="highlight-line"><span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">var</span> total <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token keyword">var</span> prop <span class="token keyword">in</span> obj<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">if</span> <span class="token punctuation">(</span>obj<span class="token punctuation">.</span><span class="token function">hasOwnProperty</span><span class="token punctuation">(</span>prop<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> total <span class="token operator">+=</span> obj<span class="token punctuation">[</span>prop<span class="token punctuation">]</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>We managed to not only recover the regression on this micro-benchmark, but even performance compared to previous
Node versions. So expect a huge performance boost in Node 9 (might also end up in Node 8 if it turns out to be
fine to upgrade V8 during LTS cycles).</p>
<p><img src="https://benediktmeurer.de/images/2017/nearform-after-20170907.png" alt="for..in peak performance"></p>
<p>Also worth noting, that despite having regressed from Node 7 to Node 8, the <code>for..in</code> peak performance in the
upcoming LTS (Node 8) still improves compared to the previous LTS (Node 6).</p>
<h2 id="optimize-object-constructor-subclassing">Optimize Object constructor subclassing <a class="bookmark" href="#optimize-object-constructor-subclassing">#</a></h2>
<p>Subclassing <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object"><code>Object</code></a> explicitly
was about <strong>3-4x</strong> times slower (upon instance creation) than just skipping the <code>extends</code> clause completely. While it didn't
seem to be useful to write</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">A</span> <span class="token keyword">extends</span> <span class="token class-name">Object</span> <span class="token punctuation">{</span> <span class="token operator">...</span> <span class="token punctuation">}</span></span></code></pre>
<p>instead of just</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">A</span> <span class="token punctuation">{</span> <span class="token operator">...</span> <span class="token punctuation">}</span></span></code></pre>
<p>where the only observable difference will be the prototype chain of the constructor, there's the case of <em>class factories</em>.
When you use class factories to stamp out base classes, i.e. as mentioned
<a href="https://twitter.com/FremyCompany/status/905977048006402048">here</a> and
<a href="https://twitter.com/rauschma/status/905914341962252291">here</a>,</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> <span class="token function-variable function">Factory</span> <span class="token operator">=</span> <span class="token parameter">BaseClass</span> <span class="token operator">=></span></span><br><span class="highlight-line"> <span class="token keyword">class</span> <span class="token class-name">A</span> <span class="token keyword">extends</span> <span class="token class-name">BaseClass</span> <span class="token punctuation">{</span> <span class="token operator">...</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token keyword">const</span> MyObject <span class="token operator">=</span> <span class="token function">Factory</span><span class="token punctuation">(</span>Object<span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>where code will implicitly extend the
<a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object"><code>Object</code> constructor</a>.
Unfortunately V8 didn't do a good job at optimizing this use case, and in particular TurboFan didn't know about
it, so we had to <a href="https://chromium-review.googlesource.com/657019">introduce some magic</a> to recognize the case
where the <code>Object</code> constructor is being used as the base class, so that TurboFan can still fully inline the
object instantiation.</p>
<p><img src="https://benediktmeurer.de/images/2017/object-constructor-subclassing-20171005.svg" alt="Object constructor subclassing performance"></p>
<p>Compared to Chrome 58, which is latest version shipping with the old Crankshaft based optimization pipeline,
the performance of subclassing <code>Object</code> improved by <strong>5.4x</strong>.</p>
<h2 id="fast-path-for-typedarrays-in-function.prototype.apply">Fast-path for <code>TypedArray</code>s in <code>Function.prototype.apply</code> <a class="bookmark" href="#fast-path-for-typedarrays-in-function.prototype.apply">#</a></h2>
<p>I'm also actively trying to address long-standing issues. One of these was a <a href="http://crbug.com/v8/2435">report from 2012</a>
titled "<code>String.fromcharCode.apply(undefined, uint8Array)</code> is super-slow", which discovered that using
<a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/apply"><code>Function.prototype.apply</code></a>
with <code>TypedArray</code>s is embarrassingly slow in V8. This is specifically bad, since there's a nice use case in combination with
<a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/fromCharCode"><code>String.fromCharCode</code></a>
to construct Strings from character code sequences encoded in <code>Uint8Array</code>s or <code>Uint16Array</code>s.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token comment">// convert a typed array to a js string</span></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">ar2str</span><span class="token punctuation">(</span><span class="token parameter">uint16arr</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token comment">// break the computation into smaller arrays to avoid browser limits on array</span></span><br><span class="highlight-line"> <span class="token comment">// size for .apply() call</span></span><br><span class="highlight-line"> <span class="token keyword">var</span> res <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> len <span class="token operator">=</span> uint16arr<span class="token punctuation">.</span>length<span class="token punctuation">,</span></span><br><span class="highlight-line"> <span class="token constant">MAX_ELEMENTS_PER_CALL</span> <span class="token operator">=</span> <span class="token number">100000</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token keyword">while</span> <span class="token punctuation">(</span>i <span class="token operator"><</span> len<span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> res<span class="token punctuation">.</span><span class="token function">push</span><span class="token punctuation">(</span></span><br><span class="highlight-line"> String<span class="token punctuation">.</span><span class="token function">fromCharCode</span><span class="token punctuation">.</span><span class="token function">apply</span><span class="token punctuation">(</span></span><br><span class="highlight-line"> <span class="token keyword">null</span><span class="token punctuation">,</span></span><br><span class="highlight-line"> uint16arr<span class="token punctuation">.</span><span class="token function">subarray</span><span class="token punctuation">(</span>i<span class="token punctuation">,</span> <span class="token punctuation">(</span>i <span class="token operator">+=</span> <span class="token constant">MAX_ELEMENTS_PER_CALL</span><span class="token punctuation">)</span><span class="token punctuation">)</span></span><br><span class="highlight-line"> <span class="token punctuation">)</span></span><br><span class="highlight-line"> <span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> res<span class="token punctuation">.</span><span class="token function">join</span><span class="token punctuation">(</span><span class="token string">""</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>It turned out to be fairly straight-forward to just add a <a href="https://chromium-review.googlesource.com/657405">fast-path for <code>TypedArray</code>s to the <code>%CreateListFromArrayLike</code>
C++ runtime function</a>, which is used by <code>Function.prototype.apply</code> under the hood.</p>
<p><img src="https://benediktmeurer.de/images/2017/create-list-from-array-like-20171005.svg" alt="CreateListFromArrayLike performance"></p>
<p>So it's a <strong>2.2x</strong> to <strong>3.4x</strong> improvement compared to Chrome 58, and there's probably still some room for improvement
in the future. The same optimization was also later <a href="https://twitter.com/SpiderMonkeyJS/status/907950258973528064">ported to
SpiderMonkey</a>, where they observed similar speed-ups.</p>
<p><a href="https://twitter.com/SpiderMonkeyJS/status/907950258973528064"><img src="https://benediktmeurer.de/images/2017/spidermonkey-string-fromcharcode-20171005.png" alt="String.fromCharCode with Function.prototype.apply in SpiderMonkey"></a></p>
<h2 id="optimize-array.prototype.push-with-multiple-parameters">Optimize <code>Array.prototype.push</code> with multiple parameters <a class="bookmark" href="#optimize-array.prototype.push-with-multiple-parameters">#</a></h2>
<p>Earlier last month, SpiderMonkey's <a href="https://twitter.com/abargull">André Bargull</a>
<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1386001">discovered</a> that Firefox often missed opportunities to inline
calls to <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/push"><code>Array.prototype.push</code></a>
into optimized code, especially visible on the <a href="http://browserbench.org/Speedometer">Speedometer</a> Ember test and the
<a href="https://arewefastyet.com/#machine=29&view=single&suite=six-speed&subtest=spread-literal-es5"><code>six-speed-spread-literal-es5</code></a>
benchmark:</p>
<p><a href="https://twitter.com/SpiderMonkeyJS/status/906528938452832257"><img src="https://benediktmeurer.de/images/2017/spidermonkey-six-speed-spread-literal-es5-20171005.png" alt="SpiderMonkey six-speed-spread-literal-es5 finding"></a></p>
<p>A <a href="https://bugs.webkit.org/show_bug.cgi?id=175823">similar observation</a> had already been made by the JavaScriptCore folks.
And it turned out that we were also missing this optimization in V8. So I took the idea from SpiderMonkey and <a href="https://chromium-review.googlesource.com/657582">ported it
to TurboFan</a>.</p>
<p><img src="https://benediktmeurer.de/images/2017/array-push-20171005.svg" alt="Array.prototype.push performance"></p>
<p>This essentially removes the weird performance cliff when going from single argument to multiple arguments in a single
call to <code>Array.prototype.push</code>. And we also closed the gap on the
<a href="https://arewefastyet.com/#machine=29&view=single&suite=six-speed&subtest=spread-literal-es5"><code>six-speed-spread-literal-es5</code></a>
benchmark.</p>
<p><a href="https://twitter.com/bmeurer/status/907213466800414721"><img src="https://benediktmeurer.de/images/2017/v8-six-speed-spread-literal-es5-20171005.png" alt="V8 six-speed-spread-literal-es5 results"></a></p>
<h2 id="improved-constant-folding">Improved constant-folding <a class="bookmark" href="#improved-constant-folding">#</a></h2>
<p>We also realized that TurboFan was missing several opportunities for constant-folding, specifically the
<a href="https://arewefastyet.com/#machine=29&view=single&suite=six-speed&subtest=templatestring-es5"><code>six-speed-templatestring-es5</code></a> and
<a href="https://arewefastyet.com/#machine=29&view=single&suite=six-speed&subtest=templatestring-es6"><code>six-speed-templatestring-es6</code></a>
made it clear that we weren't doing a good job there (yes, those are micro-benchmarks and the iteration count chosen by
<a href="http://arewefastyet.com/">arewefastyet</a> is insane, so the impact on real-world will not be 20x unless your application
doesn't do anything else). So we connected a <a href="https://chromium-review.googlesource.com/662757">couple</a>
<a href="https://chromium-review.googlesource.com/663358">more</a> <a href="https://chromium-review.googlesource.com/663181">dots</a> and
observed some massive speed-ups on these benchmarks.</p>
<p><img src="https://benediktmeurer.de/images/2017/six-speed-templatestring-es5-20171005.png" alt="six-speed-templatestring-es5">
<img src="https://benediktmeurer.de/images/2017/six-speed-templatestring-es6-20171005.png" alt="six-speed-templatestring-es6"></p>
<h2 id="optimize-tagged-templates">Optimize tagged templates <a class="bookmark" href="#optimize-tagged-templates">#</a></h2>
<p><a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals">Tagged templates</a> are one of the coolest
features introduced with <a href="https://www.ecma-international.org/ecma-262/6.0/">ES2015</a> - if you ask me 😛. I specifically love
the <a href="http://2ality.com/2016/11/computing-tag-functions.html">Computing tag functions for ES6 template literals</a> idea, i.e.:</p>
<pre class="language-js"><code class="language-js"><span class="token comment">/**<br><span class="highlight-line"> * Tag function that returns a string that conforms</span><br><span class="highlight-line"> * to the normal (“cooked”) interpretation of</span><br><span class="highlight-line"> * template literals.</span><br><span class="highlight-line"> * `String.raw` is similar, but follows the “raw”</span><br><span class="highlight-line"> * interpretation.</span><br> */</span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">cook</span><span class="token punctuation">(</span><span class="token parameter">strs<span class="token punctuation">,</span> <span class="token operator">...</span>substs</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> substs<span class="token punctuation">.</span><span class="token function">reduce</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">prev<span class="token punctuation">,</span> cur<span class="token punctuation">,</span> i</span><span class="token punctuation">)</span> <span class="token operator">=></span> prev <span class="token operator">+</span> cur <span class="token operator">+</span> strs<span class="token punctuation">[</span>i <span class="token operator">+</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">,</span> strs<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">repeat</span><span class="token punctuation">(</span><span class="token parameter">times</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">function</span><span class="token punctuation">(</span><span class="token parameter"><span class="token operator">...</span>args</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token function">cook</span><span class="token punctuation">(</span><span class="token operator">...</span>args<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">repeat</span><span class="token punctuation">(</span>times<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token function">repeat</span><span class="token punctuation">(</span><span class="token number">3</span><span class="token punctuation">)</span><span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">abc</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span><span class="token number">3</span> <span class="token operator">+</span> <span class="token number">1</span><span class="token interpolation-punctuation punctuation">}</span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">;</span> <span class="token comment">// produces "abc4abc4abc4"</span></span></code></pre>
<p>Here the language specification even
<a href="https://tc39.github.io/ecma262/#sec-gettemplateobject">requires implementations to cache the so-called <em>TemplateObject</em></a> to
actively encourage constant-folding and avoid recomputation
when going to optimized code. Unfortunately we didn't really take advantage of that so far. So I <a href="https://chromium-review.googlesource.com/677462">started teaching both Ignition
and TurboFan about <em>template objects</em></a>, which brought the ES6 implementation
on par with the Babel transpiled code. Once that was done, we <a href="https://chromium-review.googlesource.com/677603">looked into constant-folding sealed
properties</a> consistently, i.e. own properties of objects that were either
frozen via <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/freeze"><code>Object.freeze</code></a>
or sealed via <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/seal"><code>Object.seal</code></a>.</p>
<p><img src="https://benediktmeurer.de/images/2017/template-string-tag-20171005.svg" alt="Tagged templates performance"></p>
<p>Together these changes yielded a massive performance improvement of up to <strong>23x</strong> and helped us to boost the
<a href="https://arewefastyet.com/#machine=29&view=single&suite=six-speed&subtest=templatestringtag-es6"><code>six-speed-templatestringtag-es6</code></a>
benchmark even further (keep in mind that this is a micro-benchmark).</p>
<p><a href="https://twitter.com/mathias/status/912223187005509632"><img src="https://benediktmeurer.de/images/2017/six-speed-templatestring-tag-es6-20171005.png" alt="Performance of tagged templates is now on par with transpiled code"></a></p>
<h2 id="properly-optimize-literals-in-inlined-functions">Properly optimize literals in inlined functions <a class="bookmark" href="#properly-optimize-literals-in-inlined-functions">#</a></h2>
<p>Then I stumbled upon a <a href="https://bugs.chromium.org/p/v8/issues/detail?id=6856">bug in TurboFan</a> where it would not always
optimize array, object or regular expression literals properly when they occur in inlined functions. So for example, let's
say you have code like this</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">function</span> <span class="token function">bar</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token punctuation">{</span> x<span class="token punctuation">:</span> <span class="token number">1</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token function">bar</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span>x<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>then <code>bar</code> will likely be inlined into <code>foo</code> during optimization, but the literal <code>{x:1}</code> will not be turned into an
inline allocation, and instead call out to a generic code stub - the <code>FastCloneShallowObject</code> builtin - which is <em>a lot
slower</em> than an inlined allocation, that can also be escape analyzed away. Interestingly, changing the code slightly to</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">bar</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token punctuation">{</span> x<span class="token punctuation">:</span> <span class="token number">1</span> <span class="token punctuation">}</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token function">bar</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span>x<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>so that TurboFan inlines based on the concrete closure, <code>JSFunction</code> in V8 speak, we don't emit any code for the literal,
but just return <code>1</code> as expected. Turns out that this was just a left over from a previous refactoring (when we moved the
literal boilerplates to the <a href="https://www.youtube.com/watch?v=u7zRSm8jzvA"><code>FeedbackVector</code></a>). So finishing the refactoring
and <a href="https://chromium-review.googlesource.com/680656">always specializing literals</a> like any other feedback fixes this
fancy performance cliff, yielding a roughly <strong>3x</strong> to <strong>4x</strong> performance improvement compared to Chrome 61, and almost an
<strong>8x</strong> improvement compared to Chrome 58 (which didn't get the inlining right for this case), on the micro-benchmarks from
the <a href="https://bugs.chromium.org/p/v8/issues/detail?id=6856#c1">tracking bug</a>.</p>
<p><img src="https://benediktmeurer.de/images/2017/inlined-literals-20171005.svg" alt="Inlined literals performance"></p>
<h2 id="optimize-arraybuffer-view-checks">Optimize <code>ArrayBuffer</code> view checks <a class="bookmark" href="#optimize-arraybuffer-view-checks">#</a></h2>
<p>Last week I was dragged into a conversion regarding <a href="https://github.com/nodejs/node/pull/15663">nodejs/node#15663</a> and
what could be done on the V8 side to help improve performance of these predicates. It turned out that all the relevant
builtins for either checking whether something is any <code>ArrayBuffer</code> view (i.e. either a <code>TypedArray</code> or a <code>DataView</code>),
a concrete <code>TypedArray</code> like <code>Uint8Array</code>, or just any <code>TypedArray</code>, weren't really optimized in V8 (neither the baseline
implementation nor the treatment inside TurboFan). Specifically to check for whether something is a <code>TypedArray</code>, the
best (and maybe only) way to do this now is to use the
<a href="https://tc39.github.io/ecma262/#sec-get-%25typedarray%25.prototype-@@tostringtag"><code>TypedArray.prototype[@@toStringTag]</code> getter</a>,
i.e. the pattern looks like this (as found in
<a href="https://github.com/nodejs/node/blob/f547db131f527528d60c8bcc60cb43462937a794/lib/internal/util/types.js"><code>lib/internal/util/types.js</code></a>):</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> ReflectApply <span class="token operator">=</span> Reflect<span class="token punctuation">.</span>apply<span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">uncurryThis</span><span class="token punctuation">(</span><span class="token parameter">func</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token punctuation">(</span><span class="token parameter">thisArg<span class="token punctuation">,</span> <span class="token operator">...</span>args</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token function">ReflectApply</span><span class="token punctuation">(</span>func<span class="token punctuation">,</span> thisArg<span class="token punctuation">,</span> args<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> TypedArrayPrototype <span class="token operator">=</span> Object<span class="token punctuation">.</span><span class="token function">getPrototypeOf</span><span class="token punctuation">(</span><span class="token class-name">Uint8Array</span><span class="token punctuation">.</span>prototype<span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> TypedArrayProto_toStringTag <span class="token operator">=</span> <span class="token function">uncurryThis</span><span class="token punctuation">(</span></span><br><span class="highlight-line"> Object<span class="token punctuation">.</span><span class="token function">getOwnPropertyDescriptor</span><span class="token punctuation">(</span>TypedArrayPrototype<span class="token punctuation">,</span> Symbol<span class="token punctuation">.</span>toStringTag<span class="token punctuation">)</span><span class="token punctuation">.</span>get</span><br><span class="highlight-line"><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">isTypedArray</span><span class="token punctuation">(</span><span class="token parameter">value</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token function">TypedArrayProto_toStringTag</span><span class="token punctuation">(</span>value<span class="token punctuation">)</span> <span class="token operator">!==</span> <span class="token keyword">undefined</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">isUint8Array</span><span class="token punctuation">(</span><span class="token parameter">value</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token function">TypedArrayProto_toStringTag</span><span class="token punctuation">(</span>value<span class="token punctuation">)</span> <span class="token operator">===</span> <span class="token string">"Uint8Array"</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">const</span> isArrayBufferView <span class="token operator">=</span> ArrayBuffer<span class="token punctuation">.</span>isView<span class="token punctuation">;</span></span></code></pre>
<p>Now you can use <code>isTypedArray(x)</code> to check whether <code>x</code> is any <code>TypedArray</code>, and <code>isUint8Array(x)</code> to check whether
<code>x</code> is an <code>Uint8Array</code>. TurboFan was already doing a good job at <code>Reflect.apply</code> and also dealing well with the
rest parameters in <code>uncurryThis</code>. So all that was left was to optimize
<a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer/isView"><code>ArrayBuffer.isView</code></a>
for the general check and
<a href="https://tc39.github.io/ecma262/#sec-get-%25typedarray%25.prototype-@@tostringtag"><code>TypedArray.prototype[@@toStringTag]</code> getter</a>
for the <code>TypedArray</code> checks.</p>
<p>The former was straight-forward, just <a href="https://chromium-review.googlesource.com/691660">adding a new <code>ObjectIsArrayBufferView</code>
predicate</a> to TurboFan and lowering calls to <code>ArrayBuffer.isView</code> to
this newly introduced predicate. The latter was a bit more involved, and required both changes to the <a href="https://chromium-review.googlesource.com/695021">baseline implementation
and the TurboFan treatment</a>. The fundamental idea was to use the fact
that each kind of <code>TypedArray</code> has a specific <a href="https://v8.dev/blog/elements-kinds">elements kind</a>,
i.e. <code>UINT8_ELEMENTS</code>, <code>FLOAT64_ELEMENTS</code>, and so on. So the implementation now simply switches on the elements kind
on the hidden class of the receiver and returns the proper String or <code>undefined</code> if it's not a <code>TypedArray</code>.</p>
<p><img src="https://benediktmeurer.de/images/2017/arraybuffer-view-checks-20171005.svg" alt="TypedArray predicate performance"></p>
<p>We observe up to <strong>14.5x</strong> performance improvements compared to Chrome 58.</p>
<h2 id="miserable-performance-when-storing-booleans-in-typed-arrays">Miserable performance when storing booleans in typed arrays <a class="bookmark" href="#miserable-performance-when-storing-booleans-in-typed-arrays">#</a></h2>
<p>This week's monday morning exercise. Every now and then some people ping some long-standing bugs hoping that someone would pick
them up and fix them. In this case it was the infamous <a href="http://crbug.com/287773">Chromium bug 287773</a>, which was originally
reported in late 2013, so more than 4 years ago. The reported problem is that storing booleans to typed arrays leads to
really bad performance in V8. I have to admit that I've been ignoring this bug for a while, since it wasn't really trivial
to fix in Crankshaft when I saw the bug for the first time, and then forgot about it. But thanks to TurboFan fixing this
bug was a <a href="https://chromium-review.googlesource.com/691731">no brainer</a>: We just need to update the <code>KEYED_STORE_IC</code> to
truncate <code>true</code>, <code>false</code>, <code>undefined</code> and <code>null</code> to numbers instead of sending the IC to <code>MEGAMORPHIC</code> state, and also
tell TurboFan to properly convert the right-hand side of the assignment to a number first (if it's not already a number).</p>
<p><img src="https://benediktmeurer.de/images/2017/typed-array-boolean-20171005.svg" alt="TypedArray boolean performance"></p>
<p>With this in place the performance of storing <code>true</code> or <code>false</code> to a <code>TypedArray</code> is now identical to storing integers,
compared to Chrome 61 that's a solid <strong>70x</strong> improvement.</p>
<h2 id="polymorphic-symbol-lookup-not-well-supported">Polymorphic symbol lookup not well supported <a class="bookmark" href="#polymorphic-symbol-lookup-not-well-supported">#</a></h2>
<p>Later that same day <a href="http://twitter.com/mraleph">Slava</a> popped up with an
<a href="https://twitter.com/mraleph/status/913818343429296128">issue</a> that the <a href="https://www.dartlang.org/">Dart</a> folks
ran into.</p>
<p><a href="https://twitter.com/mraleph/status/913818343429296128"><img src="https://benediktmeurer.de/images/2017/slava-20171005.png" alt="Slava complaining on Twitter"></a></p>
<p>This looked suspiciously similar to <a href="http://crbug.com/v8/6367">V8 issue 6367</a> that we had noticed before, but hadn't
had the time to dig into. The underlying issue is that for code like</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">const</span> sym <span class="token operator">=</span> <span class="token function">Symbol</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token parameter">o</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> o<span class="token punctuation">[</span>sym<span class="token punctuation">]</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>we don't deal well with the case where <code>o</code> has different hidden classes on the access to <code>o[sym]</code>. The responsible
<code>KEYED_LOAD_IC</code> would immediately go to <code>MEGAMORPHIC</code> state (read: <em>become generic</em>) when it sees more than one
hidden class for <code>o</code>. Doing the same with string names using the dot syntax, i.e.</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token parameter">o</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> o<span class="token punctuation">.</span>str<span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span></code></pre>
<p>is just fine and can handle up to 4 different hidden classes for <code>o</code> until it decides to go <code>MEGAMORPHIC</code>. Interestingly
I discovered that this was not a fundamental problem, in fact most of the relevant components in V8 could already
deal with multiple hidden classes even for the <code>KEYED_LOAD_IC</code>, so it was merely a matter of <a href="https://chromium-review.googlesource.com/695108">connecting the
dots</a> again (plus some <a href="https://chromium-review.googlesource.com/695307">yak
shaving</a> to repair a bug that I flushed out with the initial CL) to
fix the odd performance cliff with polymorphic Symbol lookups.</p>
<p><img src="https://benediktmeurer.de/images/2017/symbol-lookup-polymorphic-20171005.svg" alt="Polymorphic symbol lookup performance"></p>
<p><img src="https://benediktmeurer.de/images/2017/slava-fixed-20171005.png" alt="Slava saying it's fixed on Twitter"></p>
<p>But that still didn't fully cover the case for <a href="http://crbug.com/v8/6367">V8 issue 6367</a>, which was about the
<em>clone pattern</em> (as discovered in the ARES6 ML benchmark):</p>
<pre class="language-js"><code class="language-js"><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">A</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">static</span> <span class="token keyword">get</span> <span class="token punctuation">[</span>Symbol<span class="token punctuation">.</span>species<span class="token punctuation">]</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"> <span class="token function">clone</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">new</span> <span class="token class-name">this<span class="token punctuation">.</span>constructor</span><span class="token punctuation">[</span>Symbol<span class="token punctuation">.</span>species<span class="token punctuation">]</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">class</span> <span class="token class-name">B</span> <span class="token keyword">extends</span> <span class="token class-name">A</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">static</span> <span class="token keyword">get</span> <span class="token punctuation">[</span>Symbol<span class="token punctuation">.</span>species<span class="token punctuation">]</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> <span class="token keyword">this</span><span class="token punctuation">;</span></span><br><span class="highlight-line"> <span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"></span><br><span class="highlight-line"><span class="token keyword">function</span> <span class="token function">foo</span><span class="token punctuation">(</span><span class="token parameter">o</span><span class="token punctuation">)</span> <span class="token punctuation">{</span></span><br><span class="highlight-line"> <span class="token keyword">return</span> o<span class="token punctuation">.</span><span class="token function">clone</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token punctuation">}</span></span><br><span class="highlight-line"><span class="token function">foo</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">A</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span><br><span class="highlight-line"><span class="token function">foo</span><span class="token punctuation">(</span><span class="token keyword">new</span> <span class="token class-name">B</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span></span></code></pre>
<p>It turned out that while the polymorphic symbol lookup in <code>this.constructor[Symbol.species]</code> was addressed
by the <a href="https://chromium-review.googlesource.com/695108">above mentioned fix</a>, TurboFan would still refuse
to inline the polymorphic constructor call in <code>new this.constructor[Symbol.species]()</code>, constructing either
an instance of <code>A</code> or an instance of <code>B</code> depending on <code>this</code>. Again, it turned out that this was not something
fundamental, but just <a href="https://chromium-review.googlesource.com/700596">two trivial issues</a> where some parts
of TurboFan were blocking the optimization. Removing that we got an overall <strong>7.2x</strong> combined performance boost
for the <em>clone pattern</em> above.</p>
<h2 id="improve-performance-of-object.is">Improve performance of <code>Object.is</code> <a class="bookmark" href="#improve-performance-of-object.is">#</a></h2>
<p>And one last <a href="https://bugs.chromium.org/p/v8/issues/detail?id=6882">issue</a> that also <a href="https://github.com/nodejs/node/blob/306391c/lib/util.js#L619">originated in Node
land</a>. It turns out that <code>Object.is(x,-0)</code>
is a very elegant way to check whether an arbitrary value is minus zero, which is useful in several cases,
for example when you want to print <code>-0</code> as <code>"-0"</code> instead of <code>"0"</code> (which is what the <code>ToString</code> operation
yields otherwise).</p>
<p>Unfortunately <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/is"><code>Object.is</code></a>
was previously implemented as C++ builtin, despite having all the logic for a fast
<a href="https://v8.dev/docs/csa-builtins"><code>CodeStubAssembler</code></a> based version in place already. Also
TurboFan didn't really know anything about <code>Object.is</code>, not even for the fairly simple cases where one side is statically
known to be <code>-0</code> or <code>NaN</code> (the interesting cases), or where both inputs refer to the same SSA value, which can be
constant-folded to <code>true</code> easily since <code>Object.is</code> identifies <code>NaN</code>s (in contrast to strict equality).</p>
<p>As mentioned we had all the building blocks in place to handle <code>Object.is</code> in a fast way, so it was merely an
exercise in <a href="https://chromium-review.googlesource.com/700254">porting the existing implementation</a> and hooking
it up to TurboFan.</p>
<p><img src="https://benediktmeurer.de/images/2017/object-is-20171005.svg" alt="Object.is performance"></p>
<p>So performance of <code>Object.is</code> improved by up to <strong>14x</strong> since Chrome 58, and starting with Chrome 63 and/or Node 9,
it should be fine performance-wise to use <code>Object.is</code>, especially for the edge case checks, i.e. checking for <code>-0</code>
or checking for <code>NaN</code>.</p>
<h2 id="conclusion">Conclusion <a class="bookmark" href="#conclusion">#</a></h2>
<p>Congratulations, you made it through the whole blog post! 😉 No you won't get cookies, sorry...</p>
<p>But joking aside, these are exciting times. We had been busy with working on the long-term architectural
changes around <a href="https://benediktmeurer.de/2017/03/01/v8-behind-the-scenes-february-edition/">Ignition and TurboFan</a> - by the way
I'll be giving a talk at <a href="https://2017.js-kongress.de/">JS Kongress</a> this year titled <a href="https://2017.js-kongress.de/sessions/tale-turbofan-four-years-changed-v8-forever/">"A Tale of TurboFan:
Four years that changed V8 forever"</a> -
and now we're finally back in a state where we can move quickly and improve performance by just closing
the gaps. I feel like JavaScript has a bright future.</p>
<p><img src="https://benediktmeurer.de/images/2017/dream-20171005.png" alt="Dream"></p>