HipHop for PHP at Hyves

This is a cross-post from the Hyves Product Development blog.

HipHop is Facebook’s open source C++ compiler for PHP. HipHop (also known as HPHP) will compile your PHP code to highly optimized C++ code, which you can then compile (with g++) into a big binary that will run your web site (it includes a web server). If you want to know more about Hiphop and how it was created, you can check out the Facebook Engineering Blog article or the Github project.

After its introduction many bloggers have written background articles regarding Hiphop but few seem to have actually used it in a production environment. At Hyves, we use HipHop to run our web servers and our daemons, which are also written in in PHP. In this blog, I will detail some of our experiences and results.

Compiler and interpreter

Hiphop includes both and interpreter and a compiler. This is something that is not frequently mentioned in blog posts on Hiphop. The interpreter (called HPHPi) can be used for developing, thus saving you the hassle and the delays associated with compiling every time you change a single file. In general is has the same behavior as the compiled version of Hiphop. It is a bit slower than using PHP with an opcode cache, but it is not that bad and with a fast laptop it is still workable.

The compiler can be used to create an optimized binary for your code base. The Hyves code base is in the order of 3.5M LOC in PHP. The conversion from PHP to C++ has to be done by a single server, but we use a dedicated cluster of sixteen servers for compiling the C++ source code to an executable binary. This takes about seven minutes including the building (e.g. minimizing style sheets and javascript).

The resulting binary is approximately 500MB size. This cannot be deployed to our web servers in a serial fashion, as copying it would take too long. To deploy this to our web servers, we use a bittorent based p2p deploy system. In case of an emergency, we can roll out a fix in approximately ten minutes to our 300 web servers.

Update: Facebook just released a JIT compiling VM for Hiphop.

Behavioral differences

The HPHP interpreter, the compiled version and the official PHP binary all have slightly different behavior. HipHop is about 99% compatible with pure PHP. 99% times 3.5M LOC is a lot of problems. If you want to run HipHop, make sure that your unit tests are ran against all three versions automatically or you will drive yourself and your team members insane, especially in a mixed PHP / HPHP environment.

Some PHP 5.3 stuff is still not completely supported in Hiphop, such as namespaces, of which at the moment nearly every feature is broken. Extensions such as PHP’s SOAP extension have been ported but some behavior differences exist between the different runtimes.

Converting the Hyves code base to run on HipHop took a big effort, several of our software engineers worked on this projects for months.

Hiphop has some really cool features. There is a call_user_func_async() function (it does exactly what it says), and you can have two versions of you web site running at the same time, so that you can deploy a new version without downtime. Also, you can catch fatal (E_FATAL) errors with their stack traces with HPHP, a feature sorely lacking from PHP.

There is only sparse online documentation of these specific features so you will have to figure a lot of it our by reading the source code.

Performance

Performance of our Hiphop enabled web site has been very good. In particular our API has strongly benefited, with API calls being twice times as fast and requiring only a quarter of the cpu power compared to plain PHP. This has resulted in much faster mobile clients and better performance of the parts of our web site that use internal API calls.

In the top graph you can see our web servers 95% and 99% response times. We enabled HipHop for all our web servers in March 2011. The higher response times just before the switch are due to internal network congestions because of some issues with the first versions p2p deploy system we used.

The bottom graph shows the CPU load of our web servers, which has dropped by a factor of four after the switch to HipHop. This will enable us deprecate a big portion of our web servers in the future, resulting in high savings on hosting and power.

Open source

HipHop is an open source project, available on Github. If you file a bug, Facebook’s maintainers can be pretty responsive, well reported bugs can be fixed in a matter of days. Of course you can always fix it yourself on Github and try to get the fix pulled into the project.

Al though Facebook has reported that WordPress and Mediawiki are experimenting with HipHop, not many people actually seem to be using Hiphop in production. For now, the community is pretty small outside of Facebook.

Also Hyves contributed to the HipHop project, we contributed (among other things) bug reports, a number of bugfixes and the ability to use distcc so you can compile your web site on multiple machines.

Conclusion

All in all the switch to HipHop has been a big success for Hyves. Al though switching comes with a steep cost in terms of implementation and tooling, for a big web site such as Hyves it is definitely worth the effort. If you do not have at least a few dozen web servers, you should strongly consider buying more hardware in favor of spending a lot of effort on making Hiphop work for you.

Update

The original version of this article at the Hyves Product Development Blog made it to the front page of Hacker News.


About this entry