Benchmarks – Underscore.js vs Lodash.js vs Lazy.js

Update 10/10/2013 – A good point was made that doing the array creation isn’t really going to be different between the libraries. I’ve modified the find/map/lazy samples to reflect this, and updated the numbers appropriately.

Fast code is fun. And nothing is more fun than making your application faster by dropping in a new library, without spending time re-writing code or spending money on new hardware.

Luckily, there are 2 projects for your next node.js/web app that promise to do just this. lodash.js and lazy.js are both replacements for underscore.js that promise faster performance, as well as some new features.

Lodash is fairly well known for its excellent compatibility with underscore.js. Lazy, on the other hand, should potentially offer even better performance, at the cost of implementing a slightly different API.

Underscore = require('underscore')
Lodash = require('lodash')
Lazy = require('lazy.js')
exports.compare = {
  "underscore" : function () {
    var array = Underscore.range(1000)
  },
  "lodash" : function () {
    var array = Lodash.range(1000)
  },
  "lazy" : function () {
    var array = Lazy.range(1000).toArray()
  }
};
require("bench").runMain()

Running this comparison shows lodash as the winner, underscore close, and lazy way behind. That said, this item is too trivial to really be interesting, and it doesn’t really give lazy.js a fair chance to do any lazy evaluation, so lets keep going.

  • lodash – 110.98 operations / ms
  • underscore – 103.60 operations / ms
  • lazy – 28.85 operations /ms
Underscore = require('underscore')
Lodash = require('lodash')
Lazy = require('lazy.js')
var array = Underscore.range(1000)
exports.compare = {
  "underscore" : function () {
    Underscore.find(array, function(item) {
      return item == 500;
    })    
  },
  "lodash" : function () {
    Lodash.find(array, function(item) {
      return item == 500;
    })
  },
  "lazy" : function () {
    Lazy(array).find(function(item) {
      return item == 500;
    })
  }
};
require("bench").runMain()

And the results

  • WINNER -lazy – 175.65 operations /ms
  • lodash – 168.47 operations / ms
  • underscore – 36.98 operations / ms

Lazy.js is the clear winner here. Lets try another example to see if the setup changes with even more processing.

Underscore = require('underscore')
Lodash = require('lodash')
Lazy = require('lazy.js')

square = function(x) { return x * x; }
inc = function(x) { return x + 1; }
isEven = function(x) { return x % 2 === 0; }
var array = Underscore.range(1000)

exports.compare = {
  "underscore" : function () {
    Underscore.chain(array).map(square).map(inc).filter(isEven).take(5).value()
  },
  "lodash" : function () {
    Lodash.chain(array).map(square).map(inc).filter(isEven).take(5).value()
  },
  "lazy" : function () {
    Lazy(array).map(square).map(inc).filter(isEven).take(5)
  }
};
require("bench").runMain()
  • WINNER – lazy – 14375.12 operations /ms
  • lodash – 19.10 operations / ms
  • underscore – 7.17 operations / ms

Full source code is available on github

Advertisements

3 thoughts on “Benchmarks – Underscore.js vs Lodash.js vs Lazy.js

  1. Daniel says:

    Comparing the performance of Lazy.js and Lo-Dash is tough because it isn’t really apples to apples. In the Underscore/Lo-Dash paradigm, everything gives you back an array; so it’s tempting to call toArray() on everything in Lazy.js to get the same thing. But this is tying Lazy’s hands behind its back. The strong performance potential of Lazy comes from not having to create arrays in the first place.

    Think about it this way: why would you call range at all? Presumably because you want to iterate over it, right? So with Lazy.js, you can accomplish this with Lazy.range(1000).each(...), which creates no array and just iterates over the integers from 0 to 999.

    • adamnengland says:

      Daniel –

      Fair point regarding the range generation. I’ve updated the examples to discount the array generation time from the benchmarks. This really does push lazy.js ahead of the pack by a huge margin in the find examples.

      I appreciate your work on lazy.js, its a fantastic library, and a great contribution to the js community.

      • Daniel says:

        Haha, so… unfortunately now the tests are way too generous with Lazy!

        In the find example, since each library is doing a linear search I would be very surprised to see much of a difference at all assuming you were searching for the same value in each test. Since your Lazy test is just searching for 1 it returns almost immediately (!) whereas Underscore and Lo-Dash both scan to the middle of the array before returning a result.

        As for the other case (lazyExcel.js): this unfortunately isn’t a fair comparison either. With Lazy, when you create a sequence (e.g., Lazy(...).map(...).take(...)), all the library does is create a Lazy.Sequence object; i.e., it doesn’t iterate at all. Only when you call each does iteration actually occur.

        So a fair(er) test case would be to call value in the Underscore and Lo-Dash cases (as you’ve done), and to call each (and pass a no-op function) in the Lazy case. Granted, I think this one is my fault because you were basically copying the example I have in the project’s README. This blog post has shown me that I should put a bit more work into clarifying how Lazy should be used and how its performance can be compared to that of other libraries.

        (Incidentally, if you visit http://danieltao.com/lazy.js/ and go to the “Benchmark Results” section, the explanatory paragraph gives you the option of choosing each or toArray, along with a little “why?” link providing the same information I’ve just given here. To be fair, this is very hidden away! I should make this caveat more prominent.)

        Of course this all just highlights the fact that, again, it isn’t apples-to-apples. The fact is that there are cases where Lazy seriously outperforms even Lo-Dash—such as in the chained map().filter().take() case—but it’s highly dependent on context.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: