Home  /  RSS  /  RSS Comments  /  RSS for Benchmarks  /  Enter

Posts in category ‘Benchmarks’.

Serving All Israeli News Web Sites from a Single EC2 instance...

Thursday, November 13, 2014, by artyom ; Posted in: Progress, Benchmarks, Framework; 6 comments

For the last year the development of the CppCMS project was less active. The vast majority of the work hours were spent on a customer's project that used CppCMS to create an outstanding advertisement system.

The project is called Linicom.

Today, when Linicom is up, running and maintained by a larger team, so I can resume the activity on the core CppCMS project itself.

Few words about Linicom:

Linicom is an engine that provides content and visitor sensitive advertisements for almost all large Israeli news web sites: including Ynet, Haaretz, Jerusalem Post, Mako, Walla and other significant web sites in Israel and abroad.

Here some interesting facts:

The system runs behind Lighttpd and uses PostgreSQL for persistent data storage. Also PostgreSQL is used extensively, all real time data is stored in memory.

Almost every request needs data processing in order to provide highly customized advertisements. In technical terms it means that almost no request can be "outsourced" to a static files - every request for every customer should be processed explicitly.

This system is probably one of the classic applications of CppCMS technology - web based system that required to be fast and efficient, being able to handle outstanding and sometimes unexpectedly changing loads without problems and provide high QoS.

Use of in-memory data storage, caching and efficient handing of the data that can't be cached is were CppCMS shines. Having a big growth potential with a minimal required maintenance and high reliability allows the to handle the business-end safely without worrying about performance issues.

When the memory allocation matters

Wednesday, August 10, 2011, by artyom ; Posted in: Benchmarks; 3 comments

There are many different bottle necks in C++ networking applications, many of them related to the architecture, the way system calls are used, the way application generates the data.

However at certain point applications reach the point when memory allocation becomes the bottle neck, this is especially relevant for the cases where text strings are widely used. And this is usually the case for all networking and web applications that finally need to deal with strings.

The old and good std::string has one limitation - it requires memory allocation for the chunk it represents. This is actually correct for almost any kind of string around whether they reference counted, immutable - all they allocate chunks of memory.

Consider situation, where I need to search against path /foo/bar/123 in some hierarchical structure like std::map that uses std::string as the key. So I need to find a path using after fetching two keys foo and bar from the path.

So basically for the function:

void find_path(std::string const &str);

The call:

find_path("/foo/bar/123");

Would need to:

So I've got three memory allocations and this is not matter whether these strings are mutable or immutable, reference counted or not.

CppCMS should handle such code in multiple places so how it solves this problem?

  1. Create a special object cppcms::string_key it holds inside a full std::string but also can be created explicitly from a pair of two char const * pointers that define a text in range [begin,end).

    What is important that in in the second case object does not copy the data but only references it and I'm as a user responsible to ensure that the text range remains valid as long as I use this object.

  2. Now we add an overload for the function above:

    void find_path(char const *str);
void find_path(std::string const &str);

    And when we split the text into parts we use only "unowned-strings" such that creation of a strings /foo/bar/123, foo, bar would not require memory allocation at all.

It is a general line, but there are much more interesting tricks to deal with 0 or few allocation strings and streams, like creation of a small memory pools that release all strings in once, like using on stack storage for small text chunks and much more.

This technique is widely deployed in CppCMS 0.99.9 code and had allowed to improve performance of page generation by up to twice in some cases.

Now these tricks should be done careful as they rely on manual memory management so, unless you do something many-many times or you detect some bottle-neck in the application still stick with std::string as it is usually good enough. Memory allocation today is very fast, just don't abuse it.

Version 0.99.9 Released

Wednesday, August 10, 2011, by artyom ; Posted in: Progress, Benchmarks, Cache; 0 comments

New Features:

Breaking Changes:

Bugs:

CppCMS benchmarks vs Java, C#, PHP

Saturday, October 16, 2010, by artyom ; Posted in: Benchmarks, Cache; 25 comments

Long time ago I had posted a benchmarks comparing CppCMS based blog and PHP based one.

I wanted to compare real life applications with each other. For a long time I had been searching for similar applications in several technologies doing very similar jobs in leading technologies: PHP, Asp.Net and Java/JSP. The last two were particularly important as they use static type system and "compiled" languages as C# and Java that are known to be faster then other dynamic typed languages like PHP, Python, Ruby and Perl popular in web development.

Setup

Unfortunately I had failed to find such application, so finally I decided to write something representative and small on my own an application with following requirements:

  1. Uses simple time-out based page caching
  2. Uses MySQL and the database and keeps open connections in pool.
  3. For each request access to database (if page is not cached), fetches the page content and comments for "sample article" in blog.
  4. Converts text to HTML using a markdown filter and displays it on page.

I used following technologies:

I tested following parameters:

For each test the application was "warmed up" with 100 requests to fill the cache, and then 1000 request with max concurrency of 5 request are done, while certain percent of them is new pages and the other are taken from "warmed up" once.

Notes:

I used the fastest Markdown implementation I had found.

C# implementation is the same one that http://stackoverflow.com uses - it is actually heavily optimized implementation based on older C# implementation

The Java implementation is based on the above C# and the fastest one I had found.

Discount is the fastest C implementation of markdown that I had found.

Results:

Benchmarks Markdown

Memory

Summary

  1. C#, Java and PHP implementation behave very similarly and without significant differences.
  2. The memory usage of Java/Tomcat and Mono/Asp.Net was significantly higher - up one or two orders of magnitude in comparison to CppCMS and PHP
  3. Surprisingly PHP behaves very well, in comparison to "compiled" languages like Java and C#.

Revisiting

After doing some profiling it was clear that C implementation of Markdown was significantly faster then all other implementations. So I decided to create my own mini-markdown that make some basic handing of titles, lists, paragraphs and quotes at one level only. That is very simple syntax but implemented similarly in all 4 languages using same algorithm.

The results were following:

Benchmarks Markdown

The difference between CppCMS and other implementations was still significant but still much smaller then the difference between real markdown implementation. So the performance difference was less dramatic.

2nd Revision

And in the last revision I decided not to use any text filters by fetch ready HTML formatted content from DB and display it on the web as is.

Such comparison actually profile the most basic stuff:

  1. Caching
  2. SQL Connection
  3. Request/Response handling

And would ignore hundreds lines of code used in any web applications responsible for the actual business logic.

Benchmarks HTML

Conclusions

  1. Using C++ with CppCMS provides significant performance gains in developing web applications even in very basic case.
  2. The performance is effected not only by the framework itself but also by many other libraries that are in use. Using highly optimized C and C++ libraries may give significant performance gain in many cases.
  3. Such called "jit-compiled" languages as C# and Java and the frameworks based on the use significant amount of memory and still provide much lower performance then the one that can be achieved using real compiled languages like C++.
  4. It is good to remember that these benchmarks are still quite synthetic ones and in real life the actual performance depend on many factors - but using high quality and high performance libraries available for C++ have significant impact on performance.

Results Data

Markdown
---------
Miss %  CppCMS      Mono        PHP         JSP/Tomcat

0       3200.73     747.164     974.142     821.887
1       2891.2      427.727     724.173     337.736
2       2734.69     300.017     544.162     257.44
5       2285.95     162.686     301.507     130.023
10      1749.14     89.4447     174.724     68.5387
20      1247.86     47.7347     93.7919     25.7081
50      642.769     19.8311     38.979      15.1298
100     356.968     9.77116     20.1892     7.96328

Mini-Markdown
---------
Miss %  CppCMS      Mono        PHP         JSP/Tomcat

0       3103.14     763.222     1152.63     744.72
1       2933.97     728.971     1076.38     765.599
2       2944.42     726.338     1016.42     724.869
5       2804.44     661.613     866.32      822.927
10      2592.99     584.725     705.465     753.218
20      2239.03     471.576     507.021     674.488
50      1625.5      309.443     274.962     374.26
100     1156.09     197.123     159.974     164.515

HTML
-----
Miss %  CppCMS      Mono        PHP         JSP/Tomcat

0       3286.51     849.849     1147.21     808.038
1       3055.53     776.305     1137.35     748.829
2       2991.02     691.502     1122.88     693.439
5       2687.84     693.257     1074.22     756.618
10      2390.12     615.311     1016.27     604.452
20      1886.69     521.467     917.225     668.23
50      1947.93     346.672     669.693     289.656

System and Hardware

Related:

Code

The Code can be downloaded from there. note, to run it you will need to have some libraries installed and configure some hardcoded paths to make it run.

more...

Asp.Mono in Linux? Not Yet...

Thursday, August 7, 2008, by artyom ; Posted in: Progress, Benchmarks; one comment

During development of CppCMS I have always wanted to do some benchmarks against one of the most popular web development technologies: Asp.Net. I had found a blogging system: BlogEngine.Net that in its latest version had full support of mono under Linux, support of MySQL --- all I need to run benchmarks against CppCMS.

The beginning was promising. The instructions were simple, there was an actual blog running BE.Net under Apache mod_mono on Ubuntu. There were Linux specific instructions as well.

So, first of all I had installed mono 1.9.1 from Etch backports. The first problem I had to deal with was an installation problem --- the version of C# compiler and mono environment were different. This was solved quite simply. So, I could finally see BE.Net running under Mono using XML as data storage backend.

Then, I wanted to add MySQL storage backend. The problems had come very soon. The MySQL database script was written under assumption that table names are case insensitive. That was not true for MySQL under Linux. This problem was fixed.

more...

next page

Pages

Categories