CppCMS Blog

How not to do Unicode...

All started from a small problem, how to print Unicode text to the Windows Console with option redirect to a file.

Let's say we have a program Hello that prints few words in several languages to the screen..

#include <stdio.h>

int main()
{
    printf("Мир Peace Ειρήνη\n");
    return 0;   
}

The program above is trivial and works fine under Windows if current console codepage is set to UTF-8. Also this can be fixed from the program by calling SetConsolseOutputCP(CP_UTF8).

Now simple tweak... Instead of that standard C printf we would use standard C++ std::cout... It works fine for GCC. But under Visual C++ it prints squares...

If I try redirection test.exe >test.txt - I get perfectly fine UTF-8 text...

I had started researching the issue and found the post of one of the Windows Unicode Gurus Michael Kaplan's.

I've tried to run _setmode(_fileno(stdout), _O_U8TEXT) as recommended by the Microsoft's Unicode guru and... By program crashed on attempt to write to the output stream.

Keeping searching for an answer I've got to this bug report...

Short summary:

User: Can't print UTF-8 to console with std::cout
MS: Closing - this is by design, see Michael Kaplan's article about writing to console
User: But if I do what suggested program crashes, and I still can't write Unicode to console
MS: Reactivate the ticket if it does not works
User: it does not!
MS: Use wide output...
User: I'd rather use fprintf in first place!?

To the summary...

If you use Visual C++ you can't use UTF-8 to print text to std::cout.

If you still want to, please read this amazingly long article about how to make wcout and cout working, but it does not really give a simple solution - finally falling to redefinition of the stream buffers...

So please, if you design API or Operating System, do not use kind of "Wide" API... This is is the wrong way to do Unicode.

Which reminds me... Spread around:

http://www.utf8everywhere.org/

Related Posts: http://blog.cppcms.com/post/62

RPM Repositories for CppCMS

Additionally to DEB Repositories, RPM repositories are ready as well.

The packages are build using Open Build Service:

Distributions: Fedora 15 and 16, CentOS 6, OpenSuse 11.4 and 12.1
Architectures: x86, x86_64
Packages: libcppcms, libcppcms-devel, cppcms-tools
Sources: cppcms

http://download.opensuse.org/repositories/home:/artyom-beilis/

CppCMS 1.0.1 - Bug Fix Release

CppCMS 1.0.1 Released

It is a bug fix release:

Fixed crash when using process-shared memory cache, when CppCMS compiled in Release mode with gcc 4.5 and 4.6.
Added support of CMake LIBDIR option to allow installation of libraries to lib64 instead of lib as required by Rad Had based distributions.

Additionally to the deb releases, now source-rpm is available at sourceforge.

In order to create RPMs for your system:

Install dependent packages:

yum install cmake gcc-c++ gcc make zlib-devel pcre-devel libicu-devel libgcrypt-devel

Download source rpm and rebuild it as:

rpmbuild --rebuild cppcms-1.0.1-1.src.rpm

You will get 3 binary rpm packages: libcppcms, libcppcms-devel and cppcms-tools in the ~/rpmbuild/RPMS/ directory. Install them and use them.

Special Thanks: to Sagi Ben-Akiva for initial spec file that made this process much simpler for me.

CppCMS 1.0.0 released and offered under dual license

CppCMS 1.0.0 was released today. Now it is offered under alternative commercial license. You can choose to use CppCMS under either open source LGPLv3 license or under a commercial license that allows your developing truly closed source applications.

For more details about licensing and pricing please visit our web site:
http://commercial.cppcms.com

The old-stable branch of CppCMS - 0.0.x is now deprecated and would not receive any updates. CppCMS users who still use CppCMS 0.0.8 are encouraged to upgrade to the CppCMS 1.

The new version includes:

Stable API and ABI
Comet and Ajax support
Powerful localization
Native Windows support
And much more

You can find more details about the differences between CppCMS 1.0.0 and CppCMS 0.0.8 here

Apt Repositories

New Apt repositories for Debian and Ubuntu are available.

Read: http://cppcms.com/wikipp/en/page/apt

CppCMS 0.999.1 - Release Candidate 2

All CppCMS 1.0.0 milestone tasks had completed and thus the second release candidate was released. It contains mostly bug fixes including several security bug fixes. It is expected to be the last RC before the stable version would be delivered.

All CppCMS users please download the RC2 and test it. If no critical problems would be found the stable version would be announced in a week or two.

Whats New?

Security Bug Fixes:

Fixed a bug when calling reset_session() would not reset it when session content itself was not changed.
Fixed a bug that could cause infinite waiting when POST data contained incomplete multi-part data potentially causing DOS.

Breaking Changes:

The protocol between cppcms_scale and cppcms clients had changed, you can't use old cppcms_scale with newer versions of cppcms and vise versa.

The protocol now uses absolute 64 bit time-stamp rather then relative one.

New Features:

Many new unit tests implemented giving good test coverage for the CppCMS framework.
Added support of nginx with SCGI protocol to cppcms_run
Many examples added: Asynchronous I/O, Serialization, Templates Inheritance. Some examples updated to use moder API.

Bug Fixes:

Fixed a bug that prevented "Keep-Alive" option to work with FastCGI protocol.
Session cookie was not delete on clear session call with server side session storage.
Fixed a bug that caused a session value to be unexposed what it was updated.
Fixed incorrect call of view_builder instead of unsafe_view_builder when unsefe option was given
Fixed status handling according to HTTP/CGI specifications in case of invalid POST input.
Fixed a bug when in certain situations sockets were not shutdown properly.
Added better error reporting when the content of invalid type was used during view rendering.
Fixed cppcms_run to setup nginx's PATH_INFO correctly according to the options available in the latest version.

Nginx? No thank you... Lighttpd

Nginx is 3rd (or 2nd depending on the statistics) most popular web server. It seems that it should be one of the best servers around however, every time I have to deal with it I understand that I really don't like it.

When you look deep down into the Nginx web server you realize that it not as good as you think it is. Even though, it is a good reverse proxy and very well suited for serving static files, when it comes to dynamic content it begins to feel sloppy.

It is not uncommon that you have a web site powered by nginx and in reality it works like a load balancer or reverse proxy in front of Apache or other services.

And this is not an accident, this popular webs sever has too many flaws:

First of all it is not capable to handle CGI scripts at all.
It does not support automatic control of a lifetime of FastCGI and SCGI processes.

Basically this means that you need to prepare a separate services to run your FastCGI application instead giving this simple job to a web server.
FastCGI support was broken for a long time and only recently we got an option to setup a CGI variable PATH_INFO correctly as required by CGI RFC.

Before that you need to do some tricks with rewriting and you could not get url-decoded PATH_INFO as required by RFC.
SCGI support is totally broken as:
- It requires from the SCGI server to send full HTTP headers instead of parsing Status header.
- PATH_INFO and SCRIPT_NAME do not work in the same way they were not working with fastcgi.
Basic security feature like Options -FollowSymLinks that is implemented in all normal web servers starting from a "fat" Apache to a small and tiny thttpd is not implemented!

Short explanation: consider Alice that creates a symbolic link such that it points to /var/users/passwords file. Alice does not have permissions to read this file but web sever may have such permissions. By creating an innocent symbolic link in the document root of her web server it escalates her abilities to read any file that the web server can read.

Every modern web server has an option to track symbolic links and prevent serving files outside of the scope of the web server document root, but nginx does not have one.
The documentation quality is overall lower then the quality of documents of Lighttpd or Apache web servers. Also the configuration is generally much more complicated and verbose even for simple tasks.

I've worked with nginx in different configurations and what I get most of the times is a feeling that the development of this web server is not performed according to protocol specifications but rather according what feature is needed as this specific point.

For example, it was mentioned in nginx source code that "Apache' specific PATH_INFO feature was not implemented. While PATH_INFO is a standardized environment variable defined by CGI 1.1 specifications and is not some "web server specific feature"

I know such note may look petty, but this is the general feeling of the level of implementation of many features.

So despite the popularity of this web server I would not choose it for serving dynamic content. If I need a high quality lightweight web server I'd always choose Lighttpd

It has a similar asynchronous design as nginx has.
It is very well suited for dynamic content and supports CGI, HTTP, FastCGI and SCGI protocols. The development version supports Tomcat's AJP13 protocol as well.
It is very well documented and has a rich set of modules.

Before you jump and say:

Buy lighttpd leaks!

I'll answer: "not true, it does not leak". It is one of the well know urban legends on the Internet.

Quoting:

there are no memory leaks just stupid ways to use lighttpd.

e.g. it is really stupid to stream large files via a fastcgi app, when x-lighttpd-send-file could do a much more efficient job.

For further reading:

So if you consider using Nginx on your web site, I'd strongly recommend to take a look on Lighttpd first.

CppBlog and Wiki++ are officially released

I had released the cppblog and wikipp latest versions based on CppCMS 1.x.x. Latest versions can be downloaded from the sourceforge. They are available under GPLv3 license.

The documentation on building these two web applications can be found there:

If you are want to use one of these applications in your language you are welcome to get the pot files from these project translate them and submit them to me. I'll happily add them to the trunk.

The project moved to the new web server and domain name

Please update your RSS feeds and bookmarks:

Project's main web site: http://cppcms.com
Project's blog: http://blog.cppcms.com
Project author's blog (Hebrew): http://artyom.cppcms.com

If you find any broken links, please report it to me.

CppCMS Project Receives Donations...

CppCMS project receives monetary donations using sourceforge donation system.

If you are using this project consider donating some money. Follow the donations button on the CppCMS projects web site or the button in this post.

The money would help to create a better product and provide better service to the CppCMS users community.