CppCMS Blog :: FastCGI

Easy Comet: Server-Sent Events

HTML5 Comet Technologies

Today there are two major technologies for efficient implementation of Comet applications using HTML5:

The first ones - WS - provide full-duplex communication such that both client and server can send each other events without creating a new connection. The second ones - SSE - provide real time notification in a single direction, server to client.

It seems that WS is much more popular. Indeed, WS are much more powerful and their support is scheduled for an implementation in CppCMS. On the other hand, WS have very significant limitation: WS protocol is not HTTP. It may look like HTTP and its hand-shake is done at HTTP level, but afterwards, the communication is done using different protocol that is not compatible with HTTP.

As a result, there is no way to implement WS over existing web server API as FastCGI or SCGI. More than that, even if the application uses HTTP directly, not every web server would be able to proxy web sockets communication.

So, despite that WS is very promising technology, it is very hard to deploy it today in the real production environment.

On the other hand, SSE are implemented over plain HTTP without any changes. The protocol is very simple and supported by most browsers, Firefox, Opera, Safari, Chrome and expected to be supported by IE10.

There is no special changes required to underlying web server APIs, thus FastCGI or SCGI would work perfectly well. For example, SSE can be easily implemented using stable version of CppCMS without any special tools.

Additionally SSE support stream synchronization in case of disconnect. The fall-back to long-polling using XHR can be easily implemented.

An Example

‎Lets implement a simple page that receives stock price updates.

First, we create an EventSource object and then we attach a handler that would update the appropriate html filed upon notification from the server side:

function read_data() {
    var stream = new EventSource('/ticker');
    stream.onmessage = function(e){
        document.getElementById('price').innerHTML=e.data;
    };

    stream.onerror = function(e){
        console.log(e);
    };
}

read_data();

The code is very simple and trivial, disconnects are handled automatically and transparently. Now lets take a look on the server side that is little bit more challenging:

Upon request we prepare our content type as required and fetch the ID of the last known price that was sent to the client.

void main(std::string /*url*/)
{
    response().set_content_header("text/event-stream");
    response().set_header("Cache-Control", "no-cache");

    auto last_id = atoi(request().cgetenv("HTTP_LAST_EVENT_ID"));

After that we detach the HTTP context object from the application, such that we will be able to handle multiple connections simultaneously.

    auto context=release_context();

The idle connections would be stored in a special waiters_ set. We add a special callback that allows us to cleanup the clients that had disconnected:

    context->async_on_peer_reset([=](){
        this->waiters_.erase(context);
    });

Note, we use C++11 lambda expressions that make the code much more simple and clear.

Then we check if the last price id that is known to the client and if it differs we send the client an update asynchronously, otherwise we add the client to the waiting list:

    if(last_id != counter_) {
        async_send(context);
    }
    else
        waiters_.insert(context);
}

The code that sends the request to the client is quite simple, we send the last price id - simple counter that would use us for synchronization in case of disconnect and send an actual data.

void async_send(booster::shared_ptr<cppcms::http::context> waiter)
{
    waiter->response().out() <<
        "id:" <<  counter_ <<"\n"
        "data:" << price_ << "\n"
        "\n";

Then we setup a completion callback, if the operation fails (client had disconnected), we just exit, and the context would be automatically destroyed.

    waiter->async_flush_output([=,counter_](cppcms::http::context::completion_type status){
        if(status!=0)
            return;

Otherwise we check if there were an updates since the data was sent, and if yes we send the latest price once again, otherwise we add the client to the waiting list.

        if(counter_ != this->counter_) {
            this->async_send(waiter);
        }
        else {
            this->waiters_.insert(waiter);
        }
    });

This completes our function.

Note: that our lambda expression captures the waiter variable and keeps it alive till the handler is executed.

Now the last and the simplest thing - updating the price. Upon price update, we increase the identification counter and notify all objects in the waiting list.

void update_price(double new_one)
{
    counter_++;
    price_ = new_one;
    for(auto waiter : waiters_) {
        async_send(waiter);
    }

Afterwards we clear the list - now the callback object owns the context and would destroy it in case of an error.

    waiters_.clear();
}

The full code of the sample including simple timer based random price generation can be found there.

Falling Back to Long Polling

Is very simple. We would use the same protocol, but when the event is ready we would close the connection. In order to let the server to distinguish between EventSource and XHR long polling we would add a special header like X-Event-Source-Simulate: Long-Polling

Then we would change a little our async_send function by adding the following lines:

    if(waiter->request().getenv("HTTP_X_EVENT_SOURCE_SIMULATE")=="Long-Polling") {
       waiter->async_complete_response();
      return;
    }

Such that our function would look like:

    waiter->response().out() <<
        "id:" <<  counter_ <<"\n"
        "data:" << price_ << "\n"
        "\n";
    if(waiter->request().getenv("HTTP_X_EVENT_SOURCE_SIMULATE")=="Long-Polling") {
       waiter->async_complete_response();
       return;
    }
    waiter->async_flush_output(...)
    ...

Of course XHR simulation would have to send and manage Last-Event-Id header and parse the response, but the server side would look almost identically.

Connecting to the web server

When it is coming to configuring a web server you should make sure that it does not buffer out-coming request and sends them immediately to the client:

Lighttpd does this by default for FastCGI, SCGI and HTTP protocols.
Apache does this by default for SCGI and HTTP, but for FastCGI (mod_fastcgi) it requires -flush option to work correctly.

Now once again Nginx shows us its problems:

SCGI requires option scgi_buffering off
HTTP requires option http_buffering off

However, FastCGI does not implement such an option! See this ticket

So don't even try to use Nginx with FastCGI for Server-Sent Events.

Once again, give yourself a favor, use lighttpd

Thanks

Special thanks to Barbu Paul - Gheorghe,that had brought to my attention such a powerful and useful HTML5 feature.

IIS Supports FastCGI? Not really!

IIS officially supports FastCGI and even provides built-in module starting from version 7.0.

Does it mean that you can now develop generic applications that use the industry standard widely supported API and deploy them with IIS?

Not really.

Several things you should know about IIS and FastCGI:

It does not support multi-threaded FastCGI applications. That means your generic application would be able to process only a single request per process.

Even the standard example that comes with the most popular fastcgi library implementation would not work.

This basically means: you can't implement with IIS/FastCGI long polling techniques, or share some cache between multiple worker threads of the same process.
It does not allow you to use external FastCGI applications - something very common in web servers world. All popular web servers: Apache, Lighttpd, Nginx fully support it/

IIS must manage your application life cycle and does not allow you "to do what you think is good for you".

This basically means only one things: FastCGI support for IIS was designed to run PHP... Nothing more.

One again - Microsoft takes very good generic standard and... succeeds to implement it in the most horrible way.

What's Next?

The road map of the project includes two important milestones:

CppCMS core components refactoring including following:
- Removal of dependency on CgiCC -- today there is about 5% of CgiCC library is used, many features are not supported by it or are not supported well. For example: file upload handling in CgiCC is very primitive, limited and error prone, support of cookies buggy and so on.
- Using of Boost.Asio as internal event handler, because:
  1. It provides transparent synchronous and asynchronous event handling allowing future implementation of server push technologies.
  2. It provides efficient timer based event handling.
- Removal dependency of libfcgi and writing Boost.Asio friendly implementation of FastCGI/SCGI connectors. Implementation of HTTP connectors as well.
- Support of plug-in applications in CppCMS framework.
- Improving compilation speed by representing more pimpl idioms and removal of unnecessary classes.
Better support of i18n and and l10n:
- Transparent support of std::wstring with forms including automatic encoding testing and conversion.
- Support of std::locale for localization for outputs like numbers, dates, monetary, translation and so on.
- Optional support of ICU and icu::UnicodeString and icu::Locale that would add unsupported features by std::locale and allow replacement std::locale features with more correct implementations provided by ICU.

These changes will significantly break API backward compatibility, but it would be possible to adopt the code almost "mechanically" to the new API.

API Changes and mod-prefork

There have been lot of work in recent weeks in order to make deep internal changes in the framework. Now they include:

Transparent support of 3 web server APIs: fastcgi, cgi and scgi.
Support of new mod prefork that allows safer management of worker processes.
Implementation of a cache that is shared between forked processes.

more...

Components of CppCMS

There are several important components that CppCMS implements:

FastCGI Application Framework
Easy Berkeley DB -- Simple API to BDB
Templates System
Text Tools -- text to html conversion tools
Nice URLs support

I'll describe there all these in details there

more...

Benchmarks, First Results

Updated 28/12/07, 17:00

In order to show that all the time I had spend on this system wasn't wasted I had run a simple benchmark tests on following setup:

Server

Hardware: AMD Athlon XP 3000+ 64 bit, 1GB RAM.
OS: Debian Etch 64 bit.
Web server: Lighttpd 1.4.13
CMS: This blog software, configured with 5 working threads.
Data base of 85MB: 5,000 articles with 10 comments to each one.
Network devices
100Mbit network cards
Edimax $75 home router
Client
Hardware: Dell Latitude C400, PIII 750MHz, 256MB RAM
OS: Debian Etch 32 bit

more...

The Goals

When I started working on this project, many were (and still are) extremely skeptical about its future. Many questions were asked, like: "Are you masochist? Who you think will ever use it? The hardware is cheap, we do not need such things!" etc.

In this post I'll explain my point of view and I'll show why "C++ Web Development Framework" is more then just "yet another crazy idea of another geek"

more...

CppCMS Blog :: FastCGI

Easy Comet: Server-Sent Events

HTML5 Comet Technologies

An Example

Falling Back to Long Polling

Connecting to the web server

Thanks

IIS Supports FastCGI? Not really!

What's Next?

API Changes and mod-prefork

Components of CppCMS

Benchmarks, First Results

Server

Network devices

Client

The Goals