Planet PHP

CodeIgniter 2.0.2: Cross-Site Scripting (XSS) Fixes And Recommendations

2011-05-10T11:43:00Z

As many of my readers know, I have a keen dislike for regular expression based html sanitisation. Regular expressions simply do not understand html’s nested nature and the numerous possible html/CSS standards it must abide by. The result is that far too many developers try to program this understanding (and unfortunately their lack of comprehensive understanding) into home grown sanitisers using as little code and tests as possible. It’s a horrendous and reprehensible practice that has created a large field of so-called sanitisers and XSS cleaners which are riddled with obvious vulnerabilities despite all their sincere and utterly false claims to the contrary. The perception of safety they create is almost always a fantasy. As I’ve said before, this serves only one purpose – to lend support to claims that PHP is insecure. And why disagree given PHP’s prominence on the internet and this continuing refusal by developers to just do the right thing and use a secure solution that really does work?

Since I’ve completed my research into a broad set of these, for now, I’ll close with a final example given its widespread usage, confusing documentation and lack of a clear disclosure to date of security vulnerabilities.

On April 7, EllisLab released CodeIgniter 2.0.2 as a security maintenance release prompted by a report I sent to EllisLab shortly before St. Paddy’s Day (around mid-March). That report indicated the expected response and my own disclosure policy. This blog post is being published in accordance with those. The disclosure to date of the vulnerabilities afflicting previous CodeIgniter versions is mentioned only in the CodeIgniter 2.0.2 news release (from April 7) as follows:

An update to both CodeIgniter Reactor and CodeIgniter Core (v 2.0.1) was released today. This is a security maintenance release and is a recommended update for all sites. The security fix patches a small vulnerability in the cross site scripting filter.

EllisLab’s news release for CodeIgniter 2.0.2 makes mention of “a small vulnerability”. This small vulnerability is mentioned no where else (not even the actual changelog for 2.0.2). In reality, I reported seven distinct vulnerabilities across two classes. These vulnerabilities might allow an attacker to inject arbitrary html, CSS or Javascript, i.e. Cross-Site Scripting (XSS) into an application’s output. It would be nice if, in the future, EllisLab aim for more accuracy in their news releases and disclosed both the number and nature of the security vulnerabilities fixed in their release changelogs.

Users of CodeIgniter 2.0.x and 1.7.x are strongly urged to upgrade to CodeIgniter 2.0.2 (or later) as soon as possible to avail of these critical security fixes.

In addition, users are urged to follow some basic steps when writing or updating CodeIgniter applications:

Escape ALL data being injected into views using PHP’s htmlspecialchars() function, remembering to pass the character encoding being used as the third parameter. A helper function may be useful to keep the typing to a minimum.
Use htmlPurifier when you need to sanitise html data or user input such as html comments, html emails, or RSS/Atom content (basically any html you do not explicitly generate yourself!).
Ensure that all html pages are served with a valid Content-Type HTTP header and/or a meta tag equivalent which also declares the charset for that page. Note that html5 offers a separate charset element for this purpose. This helps prevent character encoding based XSS attacks by informing the browser of the correct character en

Truncated by Planet PHP, read more at the original (another 1696 bytes)

Regex html Sanitisation: Off With Its Head!

2011-03-17T16:58:00Z

Image via Wikipedia

A long time ago someone coined the phrase Cross-Site Scripting and it became popularly abbreviated as XSS (the X was suggested to avoid confusion with CSS). XSS is a family of vulnerabilities that allows an attacker to inject arbitrary content, often Javascript, into the output (not necessarily html) viewed by users of a web application. These injections tend to do bad things. It is a plague upon web applications and not just those written in PHP.

Defeating Cross-Site Scripting

The solutions which prevent and defend against XSS in html are commonly known:

If you inject data into html (e.g. a template), and cannot be 110% sure it never crossed paths with a malicious user, you escape it. In PHP this means passing such data through a function like PHP’s htmlspecialchars(), always remembering to pass the character encoding of your output as the third parameter. An alternative exists for cases where you do not determine the html markup of output, for example, when aggregating content from RSS or Atom feeds, from web service API responses, from html emails, from user comments where html is allowed, or even from the output of html transformers such as libraries which translate BBCode, Markdown or some other intermediate format into html. These alternatives are usually called html Sanitisers or XSS Cleaners.

The first case is simple, easy to execute, and very difficult to spoof. Its main problem is that it requires foreknowledge of the character encoding of the output since html special characters may differ between encodings. A simple example of this encoding difference is found by comparing UTF-8 and UTF-7. Whereas UTF-8 is US ASCII compatible, UTF-7 is not. Escaping UTF-7 markup as if it were UTF-8 would cause the escaping mechanism to fail in detecting the angular brackets that html tags are enclosed by since they occupy different points in UTF-7. Obviously, such a failure is a potential disaster – especially if your output supports a UTF-7 encoding, or if it never specifies a character encoding at all either via a header or a html meta tag since this may allow some browsers (cough…Internet Explorer) to guess the wrong encoding to use.

The second case is complex. There are no easy solutions or single paragraph pearls of wisdom you can rely on. Instead of a simple function, you need a library of code capable of parsing html and handling character encoding differences. Then you need a friendly API so programmers aren’t buried in the complexity of the task, a whitelist and whitelist limiter to defend against misconfiguration, knowledge of every html standard since the dawn of time and up to the minute advice on emerging html quirks across all browsers (even the ones you think are no longer used). After that, you are not done. You’re only beginning. You’re going to need a parser and lexer, a character encoding handler, a html tidier so you don’t break stuff, a possy of XSS wizards to tell you when you’re failing, and enough unit tests that if Sebastian Bergmann knew what you were doing, even he would jump

Truncated by Planet PHP, read more at the original (another 7585 bytes)

Using the Symfony2 console

2010-12-21T12:34:00Z

It is always almost the same story: "As a customer I want to import data from the the superseded system in to the newly build environment".
So whats the deal? Building the new system based on the database structure to make the import as easy as possible? No, way to many drawbacks and since we have our lessons learned we know that flexibility matters and we'd rather go for a better fitting database design.

Web-services for the reScue

As for the flexibility decoupling is the magic word. A web-service might be the right answer to this question. Why? Because there are a huge number of standards (HTTP, RSS, ATOM, XML-RPC, AJAX, …) supporting this approach, they are locally independend, and they really easy to implement. At least the old system had a RSS-feed in place. So I started investigating if there is a way in Symfony2 to call recieve data from a feed.

Do you know app/console?

Seems at least some people had the same idea as I had, retrieving data from a feed via HTTP GET. So Symfony2 comes up with a nice litte wrapper - the console. Unfortunately the documentation on the website only gives a very brief introduction to it.
Located in the app directory of your project it is ready to be called as soon your Symfony2 installation is ready to be used. Executing it on the command line without an argument it exposes its help page.

$ php app/console
Symfony version 2.0.0-DEV - app

Usage:
  [options] command [arguments]

Options:
  --help           -h Display this help message.
  --quiet          -q Do not output any message.
  --verbose        -v Increase verbosity of messages.
  --version        -V Display this program version.
  --ansi           -a Force ANSI output.
  --no-interaction -n Do not ask any interactive question.
  --shell          -s Launch the shell.

Available commands:
  help          Displays help for a command (?)
  list          Lists commands
assets
  :install    
init
  :bundle
router
  :debug        Displays current routes for an application
  :dump-apache  Dumps all routes as Apache rewrite rules

The console provides you with almost everything necessary to run your bundle from the command line. It is even capable of creating a base setup of a new bundle using the init:bundle command. For example:

$>php app/console init:bundle Application/MyBundle

Making you bundle accessible from the cli

Since the search in the online documentation directly passed me to the Doctrine integration of Symfony2 using the console to do some magic handling the cache or setting up the Doctrine environment I found this as a nice source of information how to do this. Also my college Jordi was very helpful getting a clue of all this.

Preparing your Bundle

Basically you just have to create a directory in your top level of your Bundle name Command and place every class file in to be available from the command line. Surely the filename and the name of the containing class have to end with (e.g. 'ImportCommand'). The nice thing with his is that Symfony2 finds your class on its own. So no further configuration has to be done.

Adding functionality

Before we could start implementing what your cli representation of your Bundle shall do we need to set some basics first.
For starters your command class has to extends \Symfony\Bundle\FrameworkBundle\Command\Command to integrate well into the Symfony2 environment.
Doing this you enables you to override the execute() and configure() functions. The names of he methods will let your guess their function.

Configuration

I earlier quoted that there is no further configuration is to be done. This is true in case of the registration of your command in Symfony2. Surely there is some configuration to be done in terms of name, arguments, description, and perhaps a little help text. The extended class (\Symfony\Bundle\FrameworkBundle\Command\Command) provides you with a set of setter methods to add the mentioned and more information to the configuration of your command. Since Symfony2 is still under development please checkout the API-documentation for a conclusive list of those setters.

The Name is used as the command in the console arguments to address your implementation. You should take care that this is all unique. One way to assure this is to add a prefix divided by colon from the real name (e.g 'myProjectName:impo

Truncated by Planet PHP, read more at the original (another 3270 bytes)

Behind The New VPR Homepage

2010-10-11T15:09:00Z

Found Line had the privilege of designing and developing Vermont Public Radio‘s (VPR) new homepage. VPR’s Online Manager, Jonathan Butler, wrote a blog post about how the redesigned homepage delivers more content to VPR.net visitors. Here I’ll talk about the technology behind this new homepage.

Atom Syndication Format

Most of VPR’s web content is currently stored in a proprietary content management system (CMS). The first step was publishing the needed content from the CMS out into an open format. VPR’s Dan Allen published from the CMS Atom feeds of the various content we would need for the new homepage.

PHP and Zend Framework

Once we had the needed Atom feeds, we created a Zend Framework web application to import these feeds. Components such as Zend_Feed_Reader made the job easier. The weather data comes from the National Oceanic and Atmospheric Administration’s (NOAA) National Weather Service’s National Digital Forecast Database XML/REST Service. The market data comes from the Google Finance API.

CouchDB

All of the data (except for weather, which is pulled directly with a bit of caching) is stored in CouchDB. Since we’re storing documents with varying metadata (i.e. Atom entries) choosing a document-oriented database made a lot of sense. CouchDB’s map/reduce views make for very efficient queries. Its RESTful HTTP API provides a lot of options for scaling and caching. We’re caching documents within the Zend Framework application and conditionally requesting these documents from CouchDB with ETags when a cache is available. The current cache is simply file-based but we’re considering using Memcached for even better performance. On a related note, the homepage itself also supports conditional HTTP requests.

Apache and nginx

We had the pleasure of working with another great Vermont company, ClearBearing, on this project. ClearBearing handles the infrastructure layer for VPR.net. The current website was (and still is) served up by Apache. ClearBearing already had nginx in place as a reverse proxy to handle caching. Since we weren’t replacing the entire CMS we needed a way for the existing CMS and this new application to coexist. We decided to host the new application separately (still administered by ClearBearing) and reverse proxy requests for certain URIs (primarily the homepage) to the new application. This provides a clean separation between the new application and the existing CMS while still allowing for them to be hosted on the same domain from an end-user perspective.

Licensing

You’ll notice that all of the standards used are open standards and all of the technologies used are free and open source software. Additionally, all of the code we have written for VPR is licensed to them under the permissive New BSD License. This means that VPR is free to use this code however they want, study and modify the code, and redistribute the code as-is or modified. They can do all of this without paying additional licensing fees.

Design

While this post is primarily about the technology behind VPR’s new homepage, I should mention that Found Line’s Jason Pelletier designed the new homepage and implemented all of its html and CSS. We worked closely with Jonathan Butler and John Van Hoesen, Vice President for News & Programming, to make sure the new design meets the needs of website visitors and the organization.

Next Steps

The launch of this new homepage was just one iteration with many more to come. As Jonathan Butler mentioned in his blog post, VPR has plans to further improve online services for their listeners. Look for a redesigned VPR Classical page soon. There are also plans for better mobile access to VPR and VPR Classical programming. Fortunately much of the work that has been done can be leveraged to create additional online services for VPR’s listeners.

A special thanks for the support of VPR’s listeners—you made this new homepage possible! If you have questions, comments, complaints, or suggestions about the new homepage then please contact VPR directly with your feedback (although you’re welcome to comment here as well).

Creating Zend_Tool Providers

2010-07-01T13:05:00Z

When I was at Symfony Live this past February, I assisted Stefan Koopmanschap in a full-day workshop on integrating Zend Framework in Symfony applications. During that workshop, Stefan demonstrated creating Symfony "tasks". These are classes that tie in to the Symfony command-line tooling -- basically allowing you to tie in to the CLI tool in order to create cronjobs, migration scripts, etc.

Of course, Zend Framework has an analogue to Symfony tasks in the Zend_Tool component's "providers". In this post, I'll demonstrate how you can create a simple provider that will return the most recent entry from an RSS or Atom feed.

Continue reading "Creating Zend_Tool Providers"

Switzerland, Microsoft and the JumpInCamp!

2010-04-15T16:13:00Z

As some of you may know from the tweets I’ve been posting for nearly 3 weeks now, I was invited to attend the very first edition of the JumpInCamp organized by Microsoft in April 2010.

The goal of this camp was to get the European PHP community leaders together and learn about the new products and new ideas Microsoft are working on. For those of you who read about the Microsoft web developer summit that took place in Redmond in December 2009 you might think it was the same thing however you would be utterly wrong.

While the camp in Redmond was very informative and we are learnt a great deal of new features coming up with Microsoft, it was vastly different than the JumpinCamp in Zurich where the focus of the camp was to get the developers to interact with the actual Microsoft developers instead of only learning about new features. The point of the JumpinCamp was to get your hands dirty in code so we all got a few hours of lectures, then sat down and worked on either implementing those solutions into our respective Open Source projects or even discussed and raised concerns we might have regarding some of their products.

I thought it might be nice to share some of the projects I’ve started working on while I was over there and what I had interests in:

OData

After meeting with Claudio Caldato, the program manager for the Interoperability team, we went over the OData project and after looking at the position of OData and it’s potential, I decided to join the team and start by developing a PEAR package that will allow producers to publish valid OData Atom Pub feeds and serve as a base driver for the PHP community (Which could be easily ported to Zend Framework, Symfony, Lithium, etc.)

Obviously one of the reason for and OData producer package is to be able to make all the Frapi users potential OData producers. Moreover, as some of you know, sometimes I get into rants about web semantics and microformat. When I saw OData I realized that we could potentially bring some microformat standards within OData Atom Pub feeds (Or JSON Feeds).

Another thing that lit me up was the JSON feed. As some of you may have read on this very blog a few months ago I wrote an article about having something called PJSF which basically is the concept or idea of defining a standard format for JSON feeds. When I saw that OData has the ability to generate JSON feeds, I obviously jumped on the occasion of making a difference in the semantics world

Azure

Azure basically is a platform that offers a flexible, familiar environment for developers to create cloud applications and services. With Windows Azure, you can shorten your time to market and adapt as demand for your service grows.

What does is really mean? Azure is an all-in-one cloud solution. Even though the concepts are a bit arduous to grasp, we can all thank Josh Holmes and Maarten Balliauw for their essential presence at this camp to help us with all the questions we had and their thorough understanding of their baby (Azure).

Azure was another thing that sprung to my mind for Frapi. What if we could get our Frapi customers to be deployed directly into the cloud? Obviously it’s possible to hack around all the possible Amazon web services and to get somewhat arranged so our customers woul

Truncated by Planet PHP, read more at the original (another 4879 bytes)

Using PHP DOM With XPath

2010-04-10T10:54:00Z

Often I hear people say "We use SimpleXML, because DOM is so noisy and complex". Well, I don't think so. This article explains how you can parse a XML (an Atom feed) using the PHP DOM extension. No other libraries are involved.

Load the feed

To load the feed, you need to create an new DOMDocument document using it's load() method. This works with the PHP stream wrappers, so you can load local files or urls. DOMdocument, has dedicated methos for XML strings and html files and strings, too.

$feed = new DOMDocument(); $feed->load('http://www.a-basketful-of-papayas.net/feeds/posts/default'); ...

If here is any problem with the resource, PHP will output error messages. You can use libxml_use_internal_errors() to block them. With libxml_clear_errors() the internal error list is cleared, libxml_get_errors() returns them so you could implemented you own error handling. Just ignore them for now:

$errorSetting = libxml_use_internal_errors(TRUE); $feed = new DOMDocument(); $feed->load('http://www.a-basketful-of-papayas.net/feeds/posts/default'); libxml_clear_errors(); libxml_use_internal_errors($errorSetting); ...

In the next step you should check if you got some content. I use the documentElement property for this. If it is not here, the feed has to be invalid because any XML needs at least one element node.

if (isset($feed->documentElement)) { ... } else { echo 'Invalid feed.'; }

Initialize XPath

Now a XPath object is needed to execute expressions. Atom feeds make use of namespaces, often declaring the atom namespace as default. But in XPath you have no default namespace, you need to register the namespace with an arbitrary prefix. It does not have to be the same prefix used in the XML file. It can't for the default namespace obviously because it has no prefix in the XML file.

... $xpath = new DOMXPath($feed); $xpath->registerNamespace('atom', 'http://www.w3.org/2005/Atom'); ...

If you load html into the DOMDocument using the special methods, all namespaces are ignored. You can skip the registration in this case.

Executing XPath Expressions

The DOMXPath object has two methods for executing xpath expressions. One is query(), it always returns a DOMNodelist. You should use the second one: evaluate(). It will return DOMNodelist objects by default, but depending on the expression it can return other types, too. With evaluate() you have direct access to the title text, it will return an empty string if the feed has no title.

The code selects the element nodes in the registered namespace and casts them to string.

... echo $xpath->evaluate('string(/atom:feed/atom:title)'), "\n"; echo $xpath->evaluate('string(/atom:feed/atom:subtitle)'), "\n"; ...

Next we will loop over all entries. A DOMNodelist works with foreach, the expression will return an empty list if it does not match, so no additional checking is needed. Inside the loop the entry node is used as a context argument for evaluate().

... foreach ($xpath->evaluate('//atom:entry') as $entryNode) { echo $xpath->evaluate('string(atom:title)', $entryNode), "\n"; echo $xpath->evaluate( 'string(atom:link[@rel="alternate" and @type="text/html"][1]/@href)', $entryNode ), "\n"; echo "\n"; } ...

Conditions

XPath expression can be conditions. It can be used to check if a entry has categories (tags). The return value of the following expression is a boolean value.

... if ($xpath->evaluate('count(atom:category) > 0', $entryNode)) { ... } ...

Loop over attributes

Each entry can have several categories. The title of the category is in it's attribute "term". You can select these attributes directly into a list.

echo 'Categories: '; foreach ($xpath->evaluate('atom:category/@term', $entryNode) as $index => $categoryAttribute) { if ($index > 0) { echo ', '; } echo $categoryAttribute->value; } echo "\n";

Complete Example

Here is the full script. Be aware that it outputs text. If you execute it using a webserver (and not the command line), you should add a header('Content-Type: text/plain') to the top.

Truncated by Planet PHP, read more at the original (another 2150 bytes)

Pirum, the Simple PEAR Channel Server Manager

2009-11-28T07:50:00Z

Some weeks ago during the Zend Conference, I quietly released Pirum, a simple PEAR channel server manager. As some people talk about it on "social networks", I thought I should write an official announcement on my blog to explain where I come from.

A PEAR channel server allows you to install PEAR packages with the PEAR command line. You are probably already familiar with it as it comes bundled with most PHP installations:

$ pear install ...

Pirum comes from my frustration with the current state of PEAR channel servers. Beside big Open-Source projects like symfony, I'm also responsible for smaller projects (like Twig, or Swift Mailer). But for all my software, I like to provide different way of installing them: directly from SVN via svn:externals, from my Git mirrors, from an archive to download, or with the PEAR command line.

Except for the PEAR package installation, the other ones are quite easy to setup and automate. But providing a PEAR channel proves to be more involving. The only serious PEAR channel server I'm aware of is Chiara_PEAR_Channel from Greg Beaver. This is the software I used for symfony for four years. It works really well, but is a bit cumbersome to setup for smaller projects (it needs a database, and you need to configure some categories, the contributors for the project, and more). That's fine for PEAR itself, but frankly, for the rest of us, that's just too much.

As a matter of fact, hosting a PEAR channel server is super simple. It's all about static files. That's right, even if you need some sort of PHP scripts to maintain a PEAR channel server, the frontend used by the PEAR command line tool can only be static files (mainly XML ones). And Chiara_PEAR_Server works exactly like this. The backend, done in PHP, is where you maintain your packages, and it generates the frontend, a bunch of static files for the frontend. That's a really great architecture as it allows to scale very easily. For instance, you should probably be able to host a PEAR channel server on a CDN line Amazon S3 in a matter of minutes.

Thanks to this decoupled and simple architecture, I wrote some PHP scripts to manage a PEAR channel server for the symfony plugins two years ago.

The idea for Pirum came when I started the Twig project. Obviously, I wanted a PEAR channel server for Twig, but I really did not want to use Chiara_PEAR_Server. So I started to hack my own and Pirum was born.

Pirum lets you setup PEAR channel servers in a matter of minutes. Pirum is best suited when you want to create small PEAR channels for a few packages written by a few developers.

Pirum consists of just one file, a command line tool, written in PHP. There is no external dependencies, no not need for a database, no need to setup credentials, and nothing need to be installed or configured.

Installing Pirum is as simple as downloading the pirum file and saving it where you see fit.

Of course, Pirum itself uses Pirum to provide a PEAR channel server for itself!

Even if Pirum only consists of just one file, it comes with a lot of great features:

Besides its size and simplicity, Pirum is packed with a lot of features:

It creates a full-featured PEAR channel server useable by any PEAR CLI;
Each channel has an html page describing the server and the packages it hosts;
New releases can be tracked by subscribing to an Atom feed.

Of course, it also comes with some limitations:

No support for fallback PEAR channel servers;
No category management (all packages are under a "default" category);
No web interface for managing the packages.

You can find more information about using Pirum on its official website.

Pirum is already used by the following Open-Source projects:

I hope that with Pirum, more Open-Source projects will start providing a PEAR channel as a mean to install their software. If you start using Pirum for your project, please send me an email so that I can update the list on the Pirum website.

The Mysteries Of Asynchronous Processing With PHP - Part 1: Asynchronous Benefits, Task Identification and Implementation Methods

2009-09-27T14:02:00Z

Imagine a world where clients will give up on receiving responses from your application in mere seconds, where failed emails will give rise to complaints and lost business, where there exist tasks that must be performed regularly regardless of how many requests your application receives. This is not a fantasy world, it's reality. In the real world your application must be responsive, reliable and capable of recovery from errors. These are obvious needs but all too often applications fail to realise them. Sometimes, developers even fail to realise they should even be concerned about them.

To offer an opening real-world example, I'll borrow from a recent discussion I had concerning the Pubsubhubbub Protocol. If you are unfamiliar with Pubsubhubbub (PuSH), it's a protocol which implements a publish-subscribe model where the publishers of RSS and Atom feeds can "push" updates to a group of Subscribers. The pushing is handled by an intermediary called a Hub which is pinged by the Publisher when they update a feed, and which then distributes the update to many Subscribers using a Callback URL they each have declared.

In that discussion, the original poster was having a problem. Whenever a Hub sent his Subscriber implementation an update, it seemed to do it repetitively for some mysterious reason. Eventually, the problem was identified. The Hub implements a five second timeout. If, after five seconds, the update request was not completed because the Subscriber failed to send a valid response, it was assumed to have failed. The Hub would then attempt it again, and again, until finally its configured number of retries was used up.

Why was the five second timeout being exceeded by the Subscriber? What was taking it so long in returning a response and finishing the request? You see, the Subscriber was not simply acknowledging the receipt of an update as demanded by the protocol, it was actually processing the entire update for its own use including a number of potentially expensive database operations before it completed the request. This was taking more than five seconds.

Here's the problem in a nutshell. The Subscriber was performing work that had absolutely nothing to do with returning a response to the Hub and it was having an impact on the time it took to complete the request. The Hub couldn't care less about the Subscriber's processing, it was expecting a quick confirmation that the update was received. Instead, the Subscriber was effectively making it wait while it did something completely unrelated to that response. Using Asynchronous Processing, the Subscriber should have offloaded the feed processing elsewhere leaving it free to quickly respond to the Hub.

What is Asynchronous Processing?

Asynchronous processing is a method of performing tasks outside the loop of the current request. Basically, you offload the task to another process, leaving the process serving the request free to respond quickly and without delay. Of course, not all tasks are caused by a request. Some can performed without a request trigger, like some forms of maintenance or log parsing.

Implementing asynchronous processing can take a few directions:

1. A parent process can spawn a child process to complete a task in the background allowing the parent process continue uninterrupted.
2. You could add tasks to a Job Queue (or even Message Queue) relying on a background daemon or scheduled process to perform batch processing of outstanding tasks in the queue.
3. You could simply have a scheduled standalone task without the queue, and which is performed regardless of what requests are received.

There are, I'm sure, many more variations. Most readers will recognise at least one of these (hint: cron ). Once you understand the nature of asynchronous processing you can find many uses for it in the most unlikely of places.

What Problems Does Asynchronous Processing Solve?

Our example demonstrates that resource intensive tasks can be detrimental to responsiveness, so much so that it can can become detrimental in turn to the client, whether it be a machine applying a configured timeout and being forced into retrying the same request over and over, or whether it be an actual person who has to stare at a blank page as the seconds tick by.

Resource intensive tasks are not the only ones worth applying asynchronous processing to, though they are likely the most obvious given their impact on clients. Most tasks worth offloading can be grouped into categories:

1. Tasks which are resource intensive, i.e. needing a lot of CPU cycles or memory to complete which will add to server load and delay client responses.
2. Tasks whi

Truncated by Planet PHP, read more at the original (another 8273 bytes)

What Do You Develop On?

2009-09-11T13:25:00Z

At work, we have quite a variety of kit that we use for development:

Cheap and cheerful desktop machines w/ multiple monitors and plenty of RAM, normally with AMD CPUs. These machines mostly run some form of Linux … Ubuntu and Debian are both popular.
Various laptops, a fair mix of MacBook Pros and other kit running Linux.
Virtual machines running on the desktops and laptops, used for cross-browser testing.
Virtual machines running on HP servers and blades, used for system testing, release testing and production.

It gives us a lot of flexibility, allows us to develop and test on standards-compliant environments (but still use Windows for testing IE), and most of the time the developer is the bottleneck not the equipment Recently, I’ve added both a netbook and an Atom-based mini-itx machine into the mix, and this blog post is my attempt to recommend that you consider doing the same.

Netbooks are incredibly popular in the wider computer-owning population. Over here in the UK, they come free with many mobile broadband packages, making them cheaper than many low-end laptops. They’re sold in the supermarket and the high street. Their small form factor and relative lightweight makes them appealing to people who would never willingly cart a traditional laptop around. And they run Windows, which most people are familiar with.

After an initial explosion of innovation, the specs have settled around a 1.6GHz Atom processor, 1 GB of RAM and a 10″ 1024×600 resolution screen. That’s not a lot of power, and it isn’t a lot of screen estate. How do your websites look on a netbook? Does your home page or your landing pages make an impact at that size, or is your site’s message partially or completely below the fold? How do the rest of the pages look? If you’re creating an app, does the user have enough of a working area to comfortably do their tasks? Try using Google Reader or Zimbra on a netbook to see examples of what to avoid.

And how do your websites run on a netbook? Too much Javascript, and the pages won’t be snappy. The CPU won’t keep up, and the different latencies and throughput of mobile broadband make round-trips back to the server much more noticeable. Javascript that fires at regular intervals (e.g. rotating marketing spotlight images) can force the CPU to switch execution speeds, and so drain the netbook’s battery much quicker.

Testing on a netbook is one way you can spot and deal with these problems before your customers do.

Stuart is running a course in Manchester in October immediately before the PHPNW09 conference on how to setup and organise your PHP developers to ensure things run smoothly for you and your customers. Learn more about the course, or sign-up now.