Sebastian BergmannUsing HipHop for Static Analysis (27.7.2010, 09:30 UTC)

HipHop for PHP, the source code transformer that turns PHP code into C++ code that can then be compiled with g++, can also be used for static code analysis to find problems in PHP source code.

sb@vmware Money % hphp -t analyze --input-dir .
running hphp...
creating temporary directory /tmp/hphp_Zz7AXg ...
parsing inputs...
parsing inputs took 0'00" (20 ms) wall time
inferring types...
inferring types took 0'00" (10 ms) wall time
saving code errors, dependency graph and stats...
all files saved in /tmp/hphp_Zz7AXg ...
running hphp took 0'00" (208 ms) wall time

The script below takes a CodeErrors.js file (which is generated by hphp and in the example above is saved to /tmp/hphp_Zz7AXg) as its input and print an XML document in Checkstyle's format (the same XML format that is also used by PHP_CodeSniffer, for instance). This XML logfile can then be used with Hudson, for instance, in a continuous integration context.

<script src="http://gist.github.com/490858.js">
Link
Daniel KrookFacebook and HipHop at New York PHP (27.4.2010, 23:39 UTC)
Tonight we’re hosting Scott MacVicar from Facebook as he presents their highly optimized version of PHP called HipHop. HipHop translates and compiles regular PHP source into a C++ binary that reduces CPU and memory usage and thus helps Facebook serve twice the content with a two-thirds the resources of a stock Apache and PHP [...]
Link
Paul ReinheimerA GUI for XHProf (26.4.2010, 18:54 UTC)

Facebook was kind enough to open source the XHProf extension last year, but it flew under my radar until I saw a presentation including it earlier this year when they showed off HipHop. XHProf provides profiling information about your application, while being lightweight enough to run on a production server (against a percentage of requests). Once we got it installed we ran into a few limitations with the existing GUI. With the help of Graham Slater (one of our front end designers) I hacked on the Facebook code to come up with our own flavor. Let me stress the word “hack." I was given some really tough deadlines for this project so functionality was literally hacked into the existing codebase.

Key Features:

  • Nice interface
  • MySQL backend for data storage
  • Stores key values such as Peak Memory Usage, Wall Time, and CPU Ticks in the database directly for easy lookup.
  • Stores Get, Post, and Cookie data for easy comparison (so you can determine why a run hit that execution path)
  • Display includes easy lookup for worst runs, most recent, etc.
  • Support for profiling a percentage of requests, or on demand
  • Support for not including a link to profile requests on certain files (like xml documents, images, etc).
  • Uses Google Data Visualization API to graph runs over time
  • Concept of “Similar” urls, to handle ?story=23 and ?story=24 having different URLs, but similar to identical code execution paths

Screenshots:


Download:

Please download the source code from GitHub

Warnings:

Seriously, we hacked this up! Please let me know if you encounter any bugs, there's probably a bunch.

Link
Pádraic BradyPHP Framework Benchmarks: Entertaining But Ultimately Useless (23.2.2010, 16:33 UTC)
Some recent attention in the PHP framework community has been focused on the recent publication of Symfony 2 Preview benchmarks showing that Symfony 2 outperforms Zend Framework by a factor of 3.5. It also outperforms every other benchmarked framework. This reminded me of the earlier Yii Framework benchmarks which allegedly proved that Yii outperforms Zend Framework by a factor of 7. At this point, the spirit of Glenn Beck would have me demonstrate that 3.5 multiplied by 2 (the number of eyes in Barrack Obama's skull) equals 7, thus proving the existence of a Liberal conspiracy led by a closet Muslim. That's probably bullshit though.

My fellow Zend Frameworkers, we cannot allow this to stand. We must put paid to these wild claims and prove, once and for all, that Zend Framework is the fastest. To present this undeniable truth rooted in solid statistics and Paul M. Jones (aka Benchmark God) inspired techniques, I have created the benchmark of benchmarks. Well, to be honest, I only really edited another benchmark. But still, it will prove Zend Framework is faster than everything else out there.

Before everyone gets too excited, the Symfony 2 Preview performance is quite impressive and it's consistent in my own rerun of the original benchmark. A lot of effort has obviously gone into speeding things up and it shows. My benchmark is not about busting some untruth, but about demonstrating how it is possible to outrace Symfony 2 and Yii with the Zend Framework. I'll dissect the methods after the results are reported ;-).

To ensure the unchallenged fairness of my benchmark, I will publish its entire source code to Github. Unless there are any more sarcastic Irishmen out there, that should be relatively safe. I will also follow in the path of the Almighty Paul M. Jones, in whose shadow all PHP framework benchmarkers will find peace, flowing waters, milk and honey and, if not too much trouble, the extermination of all console noobs who insist that PC gaming is dying. Is that too much to ask, Paul? A little biblical gnashing of teeth would do them good. Paul (may he live forever) promotes using a standardised benchmarking platform, so I will be using an AWS instance reflecting Fabien Potencier's except it will be an m1.large instance instead (the other seeing a few issues in testing) and include some baseline HTML to make sure siege or ab are not misbehaving.

With all these preparations, the stage is set to demolish Symfony 2 and Yii and run them into the ground. Prior to running the benchmark, it's suggested to play the Battlestar Galactica theme tune in the background to set the right mood. Just think of Fabien Potencier dressed up as a Cylon (if only my Photoshop skills were better...sigh). It fits in well with the siege feedback. You can try some HipHop, but those Facebook guys must have been smoking something illegal because it had no impact during the Zend Framework runs. I also experimented with Metallica (surely that would work!) but no...nothing. Quite disappointing.

For those with attention deficit disorder, here are the results:

framework                |      rel |      avg |        1 |        2 |        3 |        4 |        5
------------------------ | -------- | -------- | -------- | -------- | -------- | -------- | --------
baseline-html | 1.1972 | 5702.79 | 5527.29 | 5775.74 | 5779.59 | 5671.24 | 5760.10
baseline-php | 1.0000 | 4763.50 | 4718.24 | 4751.51 | 4760.75 | 4763.68 | 4823.32
zend-1.10 | 0.1596 | 760.45 | 766.90 | 764.70 | 758.62 | 752.04 | 759.98
symfony-2.0.0alpha1 | 0.1366 | 650.61 | 641.98 | 655.41 | 653.86 | 656.68 | 645.12
solar-1.0.0beta3 | 0.1131 | 538.86 | 539.76 | 536.95 | 540.63 | 540.10 | 536.87
yii-1.1.1 | 0.0821 | 390.87 | 401.90 | 392.59 | 386.18 | 395.98 | 377.72
symfony-1.4.2 | 0.0441 | 210.22 | 211.20 | 209.78 | 210.72 | 210.49 | 208.92
cakephp-1.2.6 | 0.0406 | 193.56 | 193.84 | 193.35 | 193.27 | 192.57 | 194.75


As you can clearly see, Zend Framework is faster than all other frameworks. More enlightening, we beat both Symfony 2 and Yii. Lithium and Flow were omitted from the original benchmark because besides them being non-functional when forked from Fabien's benchmark (fo

Truncated by Planet PHP, read more at the original (another 6736 bytes)

Link
PHP ClassesPHP compiler performance (23.2.2010, 09:29 UTC)
By Manuel Lemos
With the release of Facebook HipHop PHP compiler, the matters of performance of PHP compilers and runtime engines gained special relevance.

This article presents a performance comparison of between different PHP compilers and runtime PHP execution engines, so we can evaluate what is the best solution that can be used in different PHP environments.
Link
Rafael DohmsHipHop for PHP: First look (6.2.2010, 14:50 UTC)

Just this tuesday Facebook announced a ambitious project called “HipHop for PHP”, if you missed it general opinion says you have been coding PHP in a cave. As I write this review no code has been posted yet, but Facebook has made a great move to open source the project so we can all get our hands on it, use it and contribute to it. So since the code is not out there yet, this is literally a first impression article based on the presentation made by Facebook and various posts from core PHP developers who got a first look at the technology before the release.

What is it?

To be blunt, its a PHP to C++ code transformer (compiler). But that does not make justice to it, so let’s look deeper. To those of you that know PHP intimately you understand the process behind running PHP, it is thus:

PHP Code –interpreter–> OP CODE –Zend Engine–> Machine Language

Generally caching solutions store OP Code and reuse it instead of running the interpreter for every request. What HipHop does is completely different and surprised quite a few people who decided to guess what they were doing. On a general view this is the process (simplified):

PHP Code –parser–> C++ Code –g++–> Compiled binary

Historically PHP has always been executed on the Zend Engine, heart of PHP that has been around since PHP3, but what this solution does is that the Zend Engine has been recoded into the HipHop Runtime Engine, which instead of OP Code takes in C++ code that has been generated based on the original PHP code.

Why HipHop?

Its a well known fact that running code in C is faster then running PHP code, for obvious reasons, its very common for large applications in PHP to port part of its codebase to C and package it into an extension, such as Yahoo and even PHP projects like Doctrine have done so, performance of simple operations can increase in as much as hundreds percent, depending on load and usage.

This is the premise for Facebook’s project, they have long contributed to APC and PHP to get more performance out of their code, but with the increased load of billions of pages served it was not enough, they decided then to solve the problem. One of the options on the table was move on to another language all together, but this is where PHP shines, Facebook declared that PHP is simply a great solution because they can easily and rapidly get new programers up to status and developing in PHP due to its simplicity, that and the fact that their code base consists of million+ lines of code made them decide that this was not a solution, thus HipHop started.

How does it work?

The idea is that PHP code can be divided into “mundane” and “magic” code. Being mundane code basic operations that are directly mapped to C++ functions. This code if converted to C++ can be executed with much higher performance, while the magic code, which is the really complex code to be converted would run at equal or slightly lower speeds. This is the point that determines if you application can benefit from this, is it more mundane then magic?

If your answer is yes, then you may want to look into it. The converter does a lot of processing identifying dependencies, doing static analysis and other operations to get the basic code, it then has to take care of the problematic issue, Typing. PHP is a weakly typed language, meaning variables can juggle their types to and from various types. In the backend of this Zend Engine implements the ZVAL type, which basically stores anything. For the C++ code the new variables are typed so the parser needs to do all this in its Type Interface. The project’s lead Engineer, Haiting Zhao, stated that one of the solutions was to map ZVALs to the C++ Variant type whenever its impossible to determine a specific type (failed type inference), or when typecasting occurs in the process of the script. After all this analysis code is finally generated.

Thus this code is compiled against the HipHop Runtime, which as I said works like the Zend Engine and works now with specialized types instead of the abstract types in the Zend Engine. Binary in hand this can now be run straight from the command line, or interfacing with a web server as its compatible with the libevent library. Currently Facebook also wrote a very simple web server to interface with its compiled code replacing its Apache on calls to this code (as far as information goes, they proxy PHP traffic to this server and leave resources going through Apache).

The good and th

Truncated by Planet PHP, read more at the original (another 6981 bytes)

Link
SitePoint » PHPBoost your PHP Performance 50% with HipHop (4.2.2010, 04:58 UTC)

HipHopNo, I’m not talking about a break-dance move that will magically give you extra PHP performance. I’m referring, rather, to a new open source project announced by Facebook. HipHop is a “source code transformer” designed to reduce CPU and memory usage by transforming your PHP code into C++ code.

To understand why this helps, you need to know a little bit about how PHP works. PHP is a scripting language, like Ruby, Python, and Perl. This is great for developer productivity, as there’s no need for the  code to be compiled. The trade-off for this flexibility is in performance, with a scripting language requiring more CPU power and memory than a compiled language such as C++. What HipHop does is transform your PHP code into true C++ code, which can then be compiled into machine code. This is not the sort of thing you would do while developing, but when you are deploying your work you can compile it into the superior C++ code.

HipHop Performance Boosts:

  • Web — 50% less CPU usage with the same traffic
  • API — 30% less CPU usage with double the traffic

The performance boosts that Facebook has seen is based on its own real-world experience, with over 90% of Facebook servers being converted to HipHop code in the last six months. The benefits released were split into two categories, the Web tier and API tier, and the benefits witnessed for each. The web tier reduced 50% of server load with the same amount of connections, and the API tier saw a 30% decrease, even with a twofold increase in traffic. The difference is explained by the web layer requiring more presentation logic and more abstract information (related information) than the API tier. megaupload movies

What’s the catch?
You’ll have to avoid a few PHP functions, such as eval() and create_function(). Because of the dynamic nature of these functions, the HipHop team was unable to transform them into C++ code; however, these functions are rarely used in most PHP applications in general.

Currently your HipHop code has to run through a custom-written HTTP server bundled with HipHop; however, using HipHop code with Apache is high on the development road map.

While HipHop will increase the performance of your PHP code, it’s not necessarily going to be the silver bullet for your performance woes; for example, if your bottleneck is in your database performance, HipHop will be unable to help you. There’s no need to worry about it making your code slower though, as the entire PHP runtime has been emulated in a way that even the most complicated code will run at the same speed as pure PHP code.

Where can I find it?
HipHop will be released very soon to the open source community, but in the meantime you can read the full write-up on the Facebook blog, or view the Facebook presentation.

What do you think?
I’m interested in doing some benchmarks and seeing how HipHop performs with the SitePoint codebase. Do you think you’ll be checking it out when it’s released?

Related Posts

  1. Free PHP Webinar: How to Increase Performance with Caching
  2. OOP and Performance
  3. Free Performance with MySQL Table Types


Link
PHP 10.0 Bloghiphop for php (4.2.2010, 00:45 UTC)

By now probably everybody that is connected to PHP world knows about Facebook’s HipHop.
So here are my thoughts about it.

1. This is a very cool and exciting technology. There were multiple attempts to do this in various ways, and from what it looks, HipHop is the most successful and appealing. I know doing these things is not easy – I tried something like that myself somewhere back in 2006 but was unable to go past very basic examples (I also made a mistake of choosing C instead of C++ as target, which I now think was not a smart decision). And the progress and support for almost all of the PHP magic is definitely great. There still are unsupported areas, but mostly are dark corners into which not many people venture (or should venture, at least :)

2. I do not see it as a replacement for php.net code or Zend Engine. The mainstream of PHP development still happens in the community around php.net and I do not think it is going to change anytime soon. I also see the fact that HipHop engine basically re-implements all the extensions as a serious disadvantage, for which the solution should be sought, even though I completely understand the technical reasons for doing so. I do not know which kind of the solution but while for the company like Facebook it might be OK to “freeze” certain set of extensions and features and maybe catch up periodically, I think it is obvious how such model can be a problem for the wider adoption.

3. If we talk about broader adoption of this technology in a wider PHP world, there are some serious challenges to consider:

  • Plugins: right now, you must compile your whole application. However, many real-life applications have extensive plugin models, which consist of their main competitive advantage – take WordPress, Drupal, Magento, any Zend Framework app – all of these allow to write custom plugins and run them alongside the main code base. The model in which you have to rebuild whole application each time you add/change a line of code is not acceptable in these circumstances.
  • Modularity: this is connected to the above, but is a bit different. PHP right now runs on Apache, IIS, Lighty, nginx, and tons of other servers in different environments. For the use of the technology in the wide variety of environments PHP is used in, one must be able to modularize  and separate the application PHP code from the runtime and the environment.
  • Portability: I tried to run a bunch of my scripts through the demo that the HipHop team was very kind to give me – and most of them experiences unsupported functions or some function didn’t work exactly as their PHP counterparts (random example: PHP’s fopen() would error out given empty first argument, while HPHP implementation would open the current directory as file. There are more such things…). Now, I admit, it would be very easy to fix them, to use different functions and they would work, but again I think it’s obvious how it can be a problem for a PHP programmer, if you are not coding targeting HipHop specifically – and especially of you use 3rd-party code. Having different implementations always presents portability challenges.
  • Thread-safety: A huge problem with PHP thread-safety is third-party libraries. Many of them are not thread safe either explicitly, or, even worse, implicitly – i.e. they just have either bugs or information leaks between threads (like changing some setting in one thread and feeling its effect in another – imagine chdir() leaking through the thread boundaries – see how it can be a problem?) HipHop team’s answer to this question was “we had fixed all those extensions” – which, without diminishing the capabilities of this excellent team, I think is not a satisfactory answer. It very well may be that they indeed fixed all the problems they could see in the code they run – but let us not forget they run one (albeit huge) application. I have hard time believing they have fixed every threading problem with every C library in existence that has interface to PHP and will continue to do so as long as C libraries exist. This of course goes back to modularity question, since here we have coupling between compiling and runtime model.
  • Debugging: obviously, debugging C++ code is orders of magnitude harder than PHP code. So you either need ninja debugging skills, or a very good set of tools to allow you to identify a problem in a live server running non-source PHP.

4. I also see certain danger potential in the success of this project. The danger primary being that the performan

Truncated by Planet PHP, read more at the original (another 2916 bytes)

Link
Lukas SmithShort HipHop blurp (3.2.2010, 21:46 UTC)

Well we all have read the announcement and chatted, twittered or whatever about it. I agree thats its cool this is being open sourced. I do fear however this means that there is no one less company looking after APC (mainly leaving just Yahoo then?). For now I think this is mainly interesting for "researchers" willing to work on improving the compatibility and more importantly the workflow around HipHop. I do not expect that anyone can really get a meaningful return on investment who makes this their business anytime soon (unless they can sell people on the hype), besides FB of course.

Overall I am not so impressed with the numbers. Even a 50% speed gain across the board .. what does that really mean? Lets say Facebook spends one third of the request (and I expect this to be even lower) on network and disk I/O, then this means that they managed a performance increase by a factor of 4 in the pure PHP processing parts. Note that I talked to some of the guys at the presentation and they said that the noted 50% speed up was not a result of switching the webserver (the binary generated by way of the C++ transformation includes a multi threaded webserver).

This is of course a nice speedup but wouldn't we expect at least a 2 fold .. or more increase if we could transform our PHP code into a C extension? This approach seems a lot more feasible for the masses. Like we could just drop in a ext/symfony or ext/ZF and thanks to autoloading magic our setup would continue to run unchanged. While it might not get that 4 fold speed up .. I would be willing to wager that it would get a 2-3 speed increase in the pure PHP processing.

So again I welcome any company open sourcing stuff that solved real world problems, especially if they are also willing to support the community in understanding their solution. But to me this solution will only matter for the ultra large companies where every single percent of saved CPU cycles matter. For the rest of us I think we need to rely on APC for a bit longer. But maybe with this transformator out there, other people will start working on the much more feasible approach of writing a PHP Code to C extension transformator.

Link
Stefan PriebschBefore you can dance (3.2.2010, 21:36 UTC)
A lot has already be speculated about what percentage of existing PHP users would benefit from using HipHop. The numbers seem to range somewhere between 0% and 100%. I will not try a guess myself, but rather try to shed some light on some the most important prerequisites I think there are for a company to benefit from HipHop.
Link
LinksRSS 0.92   RDF 1.
Atom Feed   100% Popoon
PHP5 powered   PEAR
ButtonsPlanet PHP   Planet PHP
Planet PHP