CppCMS meets Comet

8/27/09, by artyom ; Posted in: Progress, Framework, Comet; 8 comments

One of the major requirements for framework refactoring was support of Comet. Now, with introduction of asynchronous request handling and persistent application servers it becomes reality.

Client Side

There is a HTML source of simple chat client, that uses Dojo toolkit. It does following:

Submits new messages to the server application by posting form using XHR:

 function send_data() {
         var kw = {
                 url : "/chat/post",
                 form : "theform"
         };
         dojo.xhrPost(kw);
         dojo.byId("message").value="";
         return false;
 }

Receives new messages from the server using long poll via XHR:

 var message_count = 0;
 function read_data() {
         dojo.xhrGet( {
                 url: "/chat/get/" + message_count,
                 timeout: 120000,
                 handleAs: "text",
                 load: function(response, ioArgs) {
                         dojo.byId("messages").innerHTML =
                                 response
                                 + '<br/>'
                                 + dojo.byId("messages").innerHTML;
                         message_count++;
                         read_data();
                         return response;
                 },
                 error: function(response,ioArgs) {
                         read_data();
                         return response;
                 }

         });
 }
 dojo.addOnLoad(read_data);

So, the client side is quite simple (however error handling should be quite better).

Server Side

First we create our long running asynchronous application, that receives two kinds for requests: "/post" -- with new data, and "/get/NN" -- receive message nuber NN, we assign these calls to two member functions post and get.

class chat : public cppcms::application {
public:
    chat(cppcms::service &srv) : cppcms::application(srv)
    {
        dispatcher().assign("^/post$",&chat::post,this);
        dispatcher().assign("^/get/(\\d+)$",&chat::get,this,1);
    }

Now, this class includes two data members:

private:
    std::vector<std::string> messages_;
    std::vector<cppcms::intrusive_ptr<cppcms::http::context> > waiters_;

The history of all chat messages -- messages_ and all pending get requests that can't be satisfied, because the message still not exists -- waiters_

Each, "waiter" is actually pointer to request/response context that can be used for message transport.

Now, when new message arrives, post member function is called:

void post()
{
    if(request().request_method()=="POST") {
        if(request().post().find("message")!=request().post().end()) {
            messages_.push_back(request().post().find("message")->second);
            broadcast();
        }
    }
    release_context()->async_complete_response();
}

If the requested message was found, it is added to messages_ list and all waiters are notified using broadcast() member function.

At the end, the current request context is released and completed.

The broadcasting is done as following:

void broadcast()
{
    for(unsigned i=0;i<waiters_.size();i++) {
        waiters_[i]->response().set_plain_text_header();
        waiters_[i]->response().out() << messages_.back();
        waiters_[i]->async_complete_response();
        waiters_[i]=0;
    }
    waiters_.clear();
}

For each pending request the last message is written and the request closed. After that, all pending request are cleaned.

When get request arrives, it is handled by get(std::string no) member function, first of all we check if requested message exists, if so we just return it to user.

unsigned pos=atoi(no.c_str());
if(pos < messages_.size()) {
    response().set_plain_text_header();
    response().out()<<messages_[pos];
    release_context()->async_complete_response();
}

Otherwise, if the requested message is the last one, that does not exists, we add the request context to pending list waiters

else if(pos == messages_.size()) {
    waiters_.push_back(release_context());
}

If requested message it too late -- probably client error, we just set status to "404 Not Found" and return the response.

else {
    response().status(404);
    release_context()->async_complete_response();
}

No, all we need to do is to add application to the main running loop under script name "/char" and start the service.

cppcms::service service(argc,argv);
cppcms::intrusive_ptr<chat> app=new chat(service);
service.applications_pool().mount(app,"/chat");
service.run();

Summary

So, the simple chat service was written with about 50 lines of C++ code and about same amount of JavaScript code.

I must admit, that it is too simplistic and not efficient, for example: if new client connects it receives all messages one by one and not as bulk (can be easily fixed), I do not handle timeouts and disconnects. But the general idea is quite clear:

Asynchronous long running application that handles all request is created.
It manages all outstanding request and uses them for server side push.

This is actually a base for future development of tools like XML-RPC and JSON-RPC that allow client to call asynchronously server side objects, it can be used for implementation of any other Comet protocols.

Comments

Daniel Vallejos, at 9/4/09, 7:38 AM

Hello!

I'm start writing a qooxdoo server JSON-RPC that integrates in CppCms 0.0.4. I'm not expertize in web development...

You plan add JSON-RPC support in future?

http://qooxdoo.org/documentation/0.8/rpc_server_writer_guide

If you integrate qooxdo server RPC as add-on to your project, CppCms+qooxdo will support develop "GUI Like" web applications better than Wt.

Thanks you very much for your work.

I'll be back!

artyom, at 9/4/09, 7:53 AM

You plan add JSON-RPC support in future?

It would be the native type of RPC supported by CppCMS. Actually the next version of CppCMS (not 0.0.x series) uses JSON as its data format for communication and configuration.

JSON-RPC would be the major RPC for CppCMS.

I'm start writing a qooxdoo server JSON-RPC that integrates in CppCms 0.0.4. I'm not expertize in web development...

I would suggest to take a look on refactoring branch of framework project in SVN.

It supports new features like JSON and Comet that do not exist in CppCMS 0.0.x serices.

I still hand't written JSON-RPC API but it is the top priority feature that would be developed very soon.

Daniel Vallejos, at 9/16/09, 11:09 AM

Hello!

I read your Coding Standards guide lines.

I want to suggest you two points (with low importance):

1) Smart Pointers: Deprecation of auto_ptr. http://www.informit.com/guides/content.aspx?g=cplusplus&seqNum=403

2) Rename cppcms::application::cout because cout means "console" out AFAIK. And if I want use std::cout and "using namespace std;"... Do you like for example "wout"? (web out).

Thank you!

Carry ON!

(sorry if this place is not adequate for this suggestions)

artyom, at 9/16/09, 11:54 AM

Hi,

I read your Coding Standards guide lines.

I would suggest to read the updated one for next CppCMS version:

http://art-blog.no-ip.info/wikipp/en/page/cppcms_1x_coding_standards

Smart Pointers: Deprecation of auto_ptr

First of all I do not agree that auto_ptr should be deprecated. When C++0x comes probably move_ptr and shared_ptr should be used more but... Until C++0x would be supported by all major compilers this project would release two or three major versions ;).

Today, in modern C++ it is the only smart pointer that has clear move semantics that is very useful for factory pattern.

In any case... I'm beginning to support backward compatible ABI, and this means that I can't use any of boost::* classes including shared_ptr in API. Thus, auto_ptr is the only smart pointer I can use.

Rename cppcms::application::cout

It does not exist any more in CppCMS 1.x. Instead you call out() member function of http::response class. In any case, it is generally called by template rendering interface and not directly.

Daniel Vallejos, at 9/16/09, 7:26 PM

Hi!

In any case, it is generally called by > template rendering interface and not directly.

Yes. You are right!. But if I want to response for example a QOOXDOO-RPC (a variation of standard JSON-RPC) I have to use "out()" member function or another primitive method. Right?

Another question: Are you developing a new JSON parser/writer? If not, what library you are using?

Thanks!

artyom, at 9/16/09, 9:43 PM

I have to use "out()" member function or another primitive method. Right?

I think you whould not have to... Only create json-response.

Are you developing a new JSON parser/writer?

I had already written one. I didn't want to use existing because I have strict requirements of ABI compatibility.

Take a look on this code: json.h, json.cpp

LCN, at 8/21/10, 7:04 AM

Hi!

I am new to web development and I am trying to get the rationale of cppCMS. I tried the chat example that comes with version 0.99.2.1. Here are some questions I need help:

(1) My browser (Opera 10.11) echoes the messages I send. However, if I leave that web page alone for a while (about 2 minutes) and come back and type some characters and hit "Send" button, my browser no more echoes. What can be the cause of the inactivity? It would also be much helpful if this blog provides some debug tutorials.

(2)If I close the web page from the browser and reopen the same page, all the messages that were sent including the missing ones as mentioned in (1) show up. This looks to me that chat.cpp does not clean up the messages after it has responded to the chatting client. If I want the server to clear every message after it has responded to the client so that the succeeding users accessing to the same chat URL do not see the history messages, what should be done to chat.cpp?

(3) I borrowed the style of "Hello, World!" and made the following changes to chat.cpp:

<pre><code> int main(int argc,char **argv) { try { cppcms::service service(argc,argv); service.applications_pool().mount(cppcms::applications_factory<chat>()); //Why this is so slow? /* booster::intrusive_ptr<chat> c=new chat(service); service.applications_pool().mount(c); */ service.run(); } catch(std::exception const &e) { std::cerr<<"Catched exception: "<<e.what()<<std::endl; return 1; } return 0; }</code></pre>

The system now works extremely slowly. Any clarification would be much appreciated.

(4) This question does not belong to cppCMS but any help from here will be much appreciated!


function read_data() {
        dojo.xhrGet( {
            url: "/chat/get/" + message_count,
            timeout: 10000,
            handleAs: "text",
            load: function(response, ioArgs) {
                dojo.byId("messages").innerHTML = response + '
' + dojo.byId("messages").innerHTML;
                message_count++;
                read_data();
            },
            error: function(response,ioArgs) {
                read_data();
            }

        });
    }

read_data() appears to recursively call itself. Can this be a problem of memory usage?

(4) In Debian Lenny. Only root account can launch cppcms_run chat -c config.js

I get "Catched exception: system: Permission denied" error with non-root account. Any idea?


Web Server Host:127.0.0.1
Web Server Port:8080
Document Root:/usr/src/cppcms-0.99.2.1/examples/chat
Script:/chat
Api:http
Service IP:127.0.0.1
Service Port:80
Server:/usr/sbin/apache2 -f /usr/src/cppcms-0.99.2.1/examples/chat/cppcms_rundir/apache.conf
Starting Application Server:/chat
Application Server PID:2856
Starting Web Server
apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
Catched exception: system: Permission denied
Web Server PID:2858

------------------------------------
- Press Any Key To Stop The Server -
------------------------------------

cppcms_rundir/apache.log does not contain error messages. cppcms_rundir/apache.conf:

<pre><code> Listen 127.0.0.1:8080 PidFile /usr/src/cppcms-0.99.2.1/examples/chat/cppcms_rundir/srv.pid ErrorLog /usr/src/cppcms-0.99.2.1/examples/chat/cppcms_rundir/apache.log TypesConfig /usr/src/cppcms-0.99.2.1/examples/chat/cppcms_rundir/mime.types <VirtualHost *> DocumentRoot /usr/src/cppcms-0.99.2.1/examples/chat </VirtualHost> LoadModule alias_module /usr/lib/apache2/modules/mod_alias.so LoadModule mime_module /usr/lib/apache2/modules/mod_mime.so LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so ProxyPass /chat http://127.0.0.1:80/chat </code></pre>

Best Regards,

artyom, at 8/21/10, 11:42 AM

My browser (Opera 10.11) echoes the messages I send. However, if I leave that web page alone for a while (about 2 minutes) and come back and type some characters and hit "Send" button, my browser no more echoes

This example is very simple and both server side and client side implementations have very simple error handling. This example is just a general idea.

This looks to me that chat.cpp does not clean up the messages after it has responded to the chatting client

Yes. this is the way it behaves. You are welcome to change it for your own application. ;-)

read_data() appears to recursively call itself. Can this be a problem of memory usage?

I don't think so but I'm not a great expert with these toolkits. As I told before it is very simple client side implementation.

Add Comment:

Author
E-Mail	the email would not displayed
URL

You can write your messages using Markdown syntax.

You must enable JavaScript in order to post comments.

Project

Some rights reserved, the content of this blog is available under Creative Commons Attribution License 2.5 Israel.

Creative Commons