HipHop for PHP, the source code transformer that turns PHP code into C++ code that can then be compiled with g++, can also be used for static code analysis to find problems in PHP source code.

sb@vmware Money % hphp -t analyze --input-dir .
running hphp...
creating temporary directory /tmp/hphp_Zz7AXg ...
parsing inputs...
parsing inputs took 0'00" (20 ms) wall time
inferring types...
inferring types took 0'00" (10 ms) wall time
saving code errors, dependency graph and stats...
all files saved in /tmp/hphp_Zz7AXg ...
running hphp took 0'00" (208 ms) wall time

The script below takes a CodeErrors.js file (which is generated by hphp and in the example above is saved to /tmp/hphp_Zz7AXg) as its input and print an XML document in Checkstyle's format (the same XML format that is also used by PHP_CodeSniffer, for instance). This XML logfile can then be used with Hudson, for instance, in a continuous integration context.

<script src="http://gist.github.com/490858.js">

Tonight we’re hosting Scott MacVicar from Facebook as he presents their highly optimized version of PHP called HipHop. HipHop translates and compiles regular PHP source into a C++ binary that reduces CPU and memory usage and thus helps Facebook serve twice the content with a two-thirds the resources of a stock Apache and PHP [...]

Facebook was kind enough to open source the XHProf extension last year, but it flew under my radar until I saw a presentation including it earlier this year when they showed off HipHop. XHProf provides profiling information about your application, while being lightweight enough to run on a production server (against a percentage of requests). Once we got it installed we ran into a few limitations with the existing GUI. With the help of Graham Slater (one of our front end designers) I hacked on the Facebook code to come up with our own flavor. Let me stress the word “hack." I was given some really tough deadlines for this project so functionality was literally hacked into the existing codebase.

Key Features:

• Nice interface
• MySQL backend for data storage
• Stores key values such as Peak Memory Usage, Wall Time, and CPU Ticks in the database directly for easy lookup.
• Stores Get, Post, and Cookie data for easy comparison (so you can determine why a run hit that execution path)
• Display includes easy lookup for worst runs, most recent, etc.
• Support for profiling a percentage of requests, or on demand
• Support for not including a link to profile requests on certain files (like xml documents, images, etc).
• Uses Google Data Visualization API to graph runs over time
• Concept of “Similar” urls, to handle ?story=23 and ?story=24 having different URLs, but similar to identical code execution paths

Screenshots:

Download:

Please download the source code from GitHub

Warnings:

Seriously, we hacked this up! Please let me know if you encounter any bugs, there's probably a bunch.

Some recent attention in the PHP framework community has been focused on the recent publication of Symfony 2 Preview benchmarks showing that Symfony 2 outperforms Zend Framework by a factor of 3.5. It also outperforms every other benchmarked framework. This reminded me of the earlier Yii Framework benchmarks which allegedly proved that Yii outperforms Zend Framework by a factor of 7. At this point, the spirit of Glenn Beck would have me demonstrate that 3.5 multiplied by 2 (the number of eyes in Barrack Obama's skull) equals 7, thus proving the existence of a Liberal conspiracy led by a closet Muslim. That's probably bullshit though.

My fellow Zend Frameworkers, we cannot allow this to stand. We must put paid to these wild claims and prove, once and for all, that Zend Framework is the fastest. To present this undeniable truth rooted in solid statistics and Paul M. Jones (aka Benchmark God) inspired techniques, I have created the benchmark of benchmarks. Well, to be honest, I only really edited another benchmark. But still, it will prove Zend Framework is faster than everything else out there.

Before everyone gets too excited, the Symfony 2 Preview performance is quite impressive and it's consistent in my own rerun of the original benchmark. A lot of effort has obviously gone into speeding things up and it shows. My benchmark is not about busting some untruth, but about demonstrating how it is possible to outrace Symfony 2 and Yii with the Zend Framework. I'll dissect the methods after the results are reported .

To ensure the unchallenged fairness of my benchmark, I will publish its entire source code to Github. Unless there are any more sarcastic Irishmen out there, that should be relatively safe. I will also follow in the path of the Almighty Paul M. Jones, in whose shadow all PHP framework benchmarkers will find peace, flowing waters, milk and honey and, if not too much trouble, the extermination of all console noobs who insist that PC gaming is dying. Is that too much to ask, Paul? A little biblical gnashing of teeth would do them good. Paul (may he live forever) promotes using a standardised benchmarking platform, so I will be using an AWS instance reflecting Fabien Potencier's except it will be an m1.large instance instead (the other seeing a few issues in testing) and include some baseline HTML to make sure siege or ab are not misbehaving.

With all these preparations, the stage is set to demolish Symfony 2 and Yii and run them into the ground. Prior to running the benchmark, it's suggested to play the Battlestar Galactica theme tune in the background to set the right mood. Just think of Fabien Potencier dressed up as a Cylon (if only my Photoshop skills were better...sigh). It fits in well with the siege feedback. You can try some HipHop, but those Facebook guys must have been smoking something illegal because it had no impact during the Zend Framework runs. I also experimented with Metallica (surely that would work!) but no...nothing. Quite disappointing.

For those with attention deficit disorder, here are the results:

framework                |      rel |      avg |        1 |        2 |        3 |        4 |        5
------------------------ | -------- | -------- | -------- | -------- | -------- | -------- | --------
baseline-html            |   1.1972 |  5702.79 |  5527.29 |  5775.74 |  5779.59 |  5671.24 |  5760.10
baseline-php             |   1.0000 |  4763.50 |  4718.24 |  4751.51 |  4760.75 |  4763.68 |  4823.32
zend-1.10                |   0.1596 |   760.45 |   766.90 |   764.70 |   758.62 |   752.04 |   759.98
symfony-2.0.0alpha1      |   0.1366 |   650.61 |   641.98 |   655.41 |   653.86 |   656.68 |   645.12
solar-1.0.0beta3         |   0.1131 |   538.86 |   539.76 |   536.95 |   540.63 |   540.10 |   536.87
yii-1.1.1                |   0.0821 |   390.87 |   401.90 |   392.59 |   386.18 |   395.98 |   377.72
symfony-1.4.2            |   0.0441 |   210.22 |   211.20 |   209.78 |   210.72 |   210.49 |   208.92
cakephp-1.2.6            |   0.0406 |   193.56 |   193.84 |   193.35 |   193.27 |   192.57 |   194.75

As you can clearly see, Zend Framework is faster than all other frameworks. More enlightening, we beat both Symfony 2 and Yii. Lithium and Flow were omitted from the original benchmark because besides them being non-functional when forked from Fabien's benchmark (fo

Truncated by Planet PHP, read more at the original (another 6736 bytes)

Truncated by Planet PHP, read more at the original (another 6981 bytes)

No, I’m not talking about a break-dance move that will magically give you extra PHP performance. I’m referring, rather, to a new open source project announced by Facebook. HipHop is a “source code transformer” designed to reduce CPU and memory usage by transforming your PHP code into C++ code.

To understand why this helps, you need to know a little bit about how PHP works. PHP is a scripting language, like Ruby, Python, and Perl. This is great for developer productivity, as there’s no need for the  code to be compiled. The trade-off for this flexibility is in performance, with a scripting language requiring more CPU power and memory than a compiled language such as C++. What HipHop does is transform your PHP code into true C++ code, which can then be compiled into machine code. This is not the sort of thing you would do while developing, but when you are deploying your work you can compile it into the superior C++ code.

HipHop Performance Boosts:

• Web — 50% less CPU usage with the same traffic
• API — 30% less CPU usage with double the traffic

The performance boosts that Facebook has seen is based on its own real-world experience, with over 90% of Facebook servers being converted to HipHop code in the last six months. The benefits released were split into two categories, the Web tier and API tier, and the benefits witnessed for each. The web tier reduced 50% of server load with the same amount of connections, and the API tier saw a 30% decrease, even with a twofold increase in traffic. The difference is explained by the web layer requiring more presentation logic and more abstract information (related information) than the API tier. megaupload movies

What’s the catch?
You’ll have to avoid a few PHP functions, such as eval() and create_function(). Because of the dynamic nature of these functions, the HipHop team was unable to transform them into C++ code; however, these functions are rarely used in most PHP applications in general.

Currently your HipHop code has to run through a custom-written HTTP server bundled with HipHop; however, using HipHop code with Apache is high on the development road map.

While HipHop will increase the performance of your PHP code, it’s not necessarily going to be the silver bullet for your performance woes; for example, if your bottleneck is in your database performance, HipHop will be unable to help you. There’s no need to worry about it making your code slower though, as the entire PHP runtime has been emulated in a way that even the most complicated code will run at the same speed as pure PHP code.

Where can I find it?
HipHop will be released very soon to the open source community, but in the meantime you can read the full write-up on the Facebook blog, or view the Facebook presentation.

What do you think?
I’m interested in doing some benchmarks and seeing how HipHop performs with the SitePoint codebase. Do you think you’ll be checking it out when it’s released?

Related Posts

1. Free PHP Webinar: How to Increase Performance with Caching
2. OOP and Performance
3. Free Performance with MySQL Table Types

By now probably everybody that is connected to PHP world knows about Facebook’s HipHop.
So here are my thoughts about it.

1. This is a very cool and exciting technology. There were multiple attempts to do this in various ways, and from what it looks, HipHop is the most successful and appealing. I know doing these things is not easy – I tried something like that myself somewhere back in 2006 but was unable to go past very basic examples (I also made a mistake of choosing C instead of C++ as target, which I now think was not a smart decision). And the progress and support for almost all of the PHP magic is definitely great. There still are unsupported areas, but mostly are dark corners into which not many people venture (or should venture, at least

2. I do not see it as a replacement for php.net code or Zend Engine. The mainstream of PHP development still happens in the community around php.net and I do not think it is going to change anytime soon. I also see the fact that HipHop engine basically re-implements all the extensions as a serious disadvantage, for which the solution should be sought, even though I completely understand the technical reasons for doing so. I do not know which kind of the solution but while for the company like Facebook it might be OK to “freeze” certain set of extensions and features and maybe catch up periodically, I think it is obvious how such model can be a problem for the wider adoption.

3. If we talk about broader adoption of this technology in a wider PHP world, there are some serious challenges to consider:

• Plugins: right now, you must compile your whole application. However, many real-life applications have extensive plugin models, which consist of their main competitive advantage – take WordPress, Drupal, Magento, any Zend Framework app – all of these allow to write custom plugins and run them alongside the main code base. The model in which you have to rebuild whole application each time you add/change a line of code is not acceptable in these circumstances.
• Modularity: this is connected to the above, but is a bit different. PHP right now runs on Apache, IIS, Lighty, nginx, and tons of other servers in different environments. For the use of the technology in the wide variety of environments PHP is used in, one must be able to modularize  and separate the application PHP code from the runtime and the environment.
• Portability: I tried to run a bunch of my scripts through the demo that the HipHop team was very kind to give me – and most of them experiences unsupported functions or some function didn’t work exactly as their PHP counterparts (random example: PHP’s fopen() would error out given empty first argument, while HPHP implementation would open the current directory as file. There are more such things…). Now, I admit, it would be very easy to fix them, to use different functions and they would work, but again I think it’s obvious how it can be a problem for a PHP programmer, if you are not coding targeting HipHop specifically – and especially of you use 3rd-party code. Having different implementations always presents portability challenges.
• Thread-safety: A huge problem with PHP thread-safety is third-party libraries. Many of them are not thread safe either explicitly, or, even worse, implicitly – i.e. they just have either bugs or information leaks between threads (like changing some setting in one thread and feeling its effect in another – imagine chdir() leaking through the thread boundaries – see how it can be a problem?) HipHop team’s answer to this question was “we had fixed all those extensions” – which, without diminishing the capabilities of this excellent team, I think is not a satisfactory answer. It very well may be that they indeed fixed all the problems they could see in the code they run – but let us not forget they run one (albeit huge) application. I have hard time believing they have fixed every threading problem with every C library in existence that has interface to PHP and will continue to do so as long as C libraries exist. This of course goes back to modularity question, since here we have coupling between compiling and runtime model.
• Debugging: obviously, debugging C++ code is orders of magnitude harder than PHP code. So you either need ninja debugging skills, or a very good set of tools to allow you to identify a problem in a live server running non-source PHP.

4. I also see certain danger potential in the success of this project. The danger primary being that the performan

Truncated by Planet PHP, read more at the original (another 2916 bytes)

Well we all have read the announcement and chatted, twittered or whatever about it. I agree thats its cool this is being open sourced. I do fear however this means that there is no one less company looking after APC (mainly leaving just Yahoo then?). For now I think this is mainly interesting for "researchers" willing to work on improving the compatibility and more importantly the workflow around HipHop. I do not expect that anyone can really get a meaningful return on investment who makes this their business anytime soon (unless they can sell people on the hype), besides FB of course.

Overall I am not so impressed with the numbers. Even a 50% speed gain across the board .. what does that really mean? Lets say Facebook spends one third of the request (and I expect this to be even lower) on network and disk I/O, then this means that they managed a performance increase by a factor of 4 in the pure PHP processing parts. Note that I talked to some of the guys at the presentation and they said that the noted 50% speed up was not a result of switching the webserver (the binary generated by way of the C++ transformation includes a multi threaded webserver).

This is of course a nice speedup but wouldn't we expect at least a 2 fold .. or more increase if we could transform our PHP code into a C extension? This approach seems a lot more feasible for the masses. Like we could just drop in a ext/symfony or ext/ZF and thanks to autoloading magic our setup would continue to run unchanged. While it might not get that 4 fold speed up .. I would be willing to wager that it would get a 2-3 speed increase in the pure PHP processing.

So again I welcome any company open sourcing stuff that solved real world problems, especially if they are also willing to support the community in understanding their solution. But to me this solution will only matter for the ultra large companies where every single percent of saved CPU cycles matter. For the rest of us I think we need to rely on APC for a bit longer. But maybe with this transformator out there, other people will start working on the much more feasible approach of writing a PHP Code to C extension transformator.

A lot has already be speculated about what percentage of existing PHP users would benefit from using HipHop. The numbers seem to range somewhere between 0% and 100%. I will not try a guess myself, but rather try to shed some light on some the most important prerequisites I think there are for a company to benefit from HipHop.