Saturday 2 March 2013

The only bad thing about Mason

At the moment, I'm writing a presentation about Mason2. The goal is to somehow convince my colleagues to consider using Mason. Instead of Template Toolkit.
As part of it, I thought I'd do a bit of performance benchmarking against Template Toolkit. So I put together a reasonably complex mini set of templates using both systems. Well, as complex as TT can take in reality. Which is not very much.

But anyway, let's jump straight to the bad news.

        Rate Mason    TT
Mason  901/s    --  -51%
TT    1852/s  106%    --


As you can see, with similar settings (code caching enabled) and over 10000 runs of two pages, Mason2 is twice as slow as Template Toolkit. Mason2 is still below the millisecond per page in my benchmark, but still that's quite a big difference.

As the common wisdom goes, CPU time is cheaper than programmer time, so it shouldn't be a big deal, given the huge gains of productivity you should get by using Mason2's very cool and powerful features.

Stay tuned for the full presentation to come!

PS: To follow up on the comments, feel free to have a look at the benchmark source https://bitbucket.org/jeteve/mason-pres/

J. 

9 comments:

  1. Please don't forget that your competition nowadays is not only TT, but also these:

    Text::Xslate
    HTML::Template:Compiled
    Template::Alloy

    ReplyDelete
    Replies
    1. Indeed, want to help implementing the benchmark in those other ones? Xslate site already have some figures BTW, but I don't know how reliable they are.

      Delete
    2. Reliable in what sense? If you're looking for "is this module used in the wild?" then yes. It powers some of the largest perl shops in Japan. Duckduckgo probably uses it too (haven't verified. please take with a grain of salt).

      I personally haven't used anything other than Xslate for well over 3 years.

      Delete
    3. Yeah Xslate looks very much like the king of performance, but I'm not sure if I should trust their benchmark figures to compare all the other slow template system between them. Plus they didn't considered Mason2 yet :)

      Delete
  2. You may want to have a look at https://metacpan.org/module/Template::Benchmark

    ReplyDelete
    Replies
    1. Very interesting, one day I'll write a Template::Benchmark plugin for Mason2 :)

      Delete
  3. A proper comparison has already been done by Sam Graham: template roundup

    ReplyDelete
  4. Many templating benchmarks are not very useful. The performance highly depends on the size of the template, how many commands they contained compared to plain text, what template options you use and if you can use a persistant environment oder have to use CGI.
    See also my bench.pl in the HTML::Template::Compiled distribution.

    So seeing just a benchmark output without any information about the points I mentioned above, this means nothing to me.

    ReplyDelete
    Replies
    1. There's now a link on the source in the post. Thanks!

      Delete