Wouldn’t it be nice if you had an idea of how people use the software that you write? I wanted to have an idea how the Apache HTTP Server is being used, and which features users consider important. So, I set up a short online survey of eight questions and sent a link to it to the HTTP Server project user and developer mailing lists. Over the next week and a half, I got 134 responses. Here are the survey results in shiny pie charts with witty interpretation.

On which operating system do you typically use Apache HTTP Server?

This question, actually number two in the survey, received quite a few write-in responses. I had only the top four operating systems in the answer list, but respondents wrote in several more. One respondent wrote: “I use Apache simply because it works transparently on Linux and Windows,” and another “FreeBSD on my hosting service, Linux at the office.” One respondent wrote: “Estou começando a programar em php!” (I’m starting to program in PHP!) The following graph has all of the operating systems that appeared in the survey responses:

Linux and Windows make up the vast majority of operating systems in use, accounting for over 80%. Sliced another way, 80% of respondents uses Apache on Unix-like platforms.

Why all those write-ins? My theory is that it has to do with the desire to make one’s platform of choice known. Usually when we are asked this question, the asking is done by marketeers trying to save some money by cutting support for less popular platforms. If we don’t speak up, our platform might get cut. Fortunately, Apache doesn’t work this way: support for various operating systems and platforms is provided by the developer community. Whether a specific operating system is popular does not matter: as long as there are developers interested in supporting a platform, the Apache HTTP Server will run on it.

How do you typically obtain your Apache HTTP Server Software

Technically, the httpd project only releases source code: once the source code archive is up on the download servers, the project is done with the release. Anyone can download and compile the software for his own use. However, a lot of users obtain httpd through some other means. They don’t have the expertise to compile their own, or find it unnecessary because their Linux distribution comes with httpd already installed. Some platforms, like Windows, ship without a compiler and building Apache on it is non-trivial. When asked how they typically obtained their Apache HTTP Server software, this is how the user community answered:

A large minority of respondents compiles their own httpd from source. This is a little surprising: I assumed that more users would use packages from their distributions or third party downloads. Of course, most respondents learned about the survey from the httpd developer and user mailing lists: one can expect a large proportion of sophisticated users that have the expertise and wish to exercise control over their deployments that requires custom-built HTTP Server software. What’s interesting is the same data cross-referenced by operating system:

Third party packages are popular on Windows, which ships without a compiler. Also, it is not trivial to build httpd and its supporting libraries on the platform. It’s much easier to grab an installer, especially since members of the httpd developer community usually make one available. To the credit of the port maintainer, all respondents who run FreeBSD install their httpd from the ports collection.

Which version of Apache HTTP Server do you mostly run?

There is very little surprise here: Apache HTTP Server 2.2.x is the most popular by far:

Which application API modules do you use?

Respondents could select multiple answers to this question. PHP remains the most popular module: this is unlikely to surprise anyone.

It’s interesting to see a strong showing for the good old CGI interface. The results may be slightly skewed by the fact that mod_python is no longer actively developed: as several people noted, its role has been taken over by mod_wsgi. I also did not include several integrations that might prove popular, like FastCGI.

How important are the following features of Apache HTTP Server to you?

This question allowed respondents to indicate how important various features of httpd are to them.

The features people feel most passionate about are clearly SSL support and URL rewriting/mapping: these each got about 50% “can’t live without.” Nobody does not care about URL rewriting. Scalability and extensibility are important, but aren’t “can’t live without” features. The easy configuration gets only a small amount of passion: I guess that if you are familiar enough with httpd to fill out the survey, you tend to not be intimidated by its configuration language.

Will you upgrade to Apache HTTP Server 2.4 when it is released?

At the time of the survey, version 2.4 of the server had not been released. Asked whether they would upgrade immediately upon release, most respondents chose the safe route of waiting a while. A few intrepid souls (fewer than 20%) planned to upgrade immediately.

In which environment do you mostly use Apache HTTP Server?

This was the only demographic question in the survey: an attempt to learn a little bit about the respondents.

Almost half of the respondents use httpd in a small or medium-sized business, and only one eight work in a Fortune 500 company. This discrepancy may be partially caused by the fact that there are millions of small and medium businesses, and the number of Fortune 500 companies is limited to… 500. Consultants and non-profit organizations are well represented; government respondents are a small minority.

If I could improve one thing about Apache HTTP Server, it would be…

This was a free-form write-in question: respondents could fill in whatever they wanted. Just under half of the respondents took advantage of this opportunity. The full list of responses will be shared with the HTTP Server development community, but I will quote a few.

Some respondents wanted to improve the feature set, such as:

• “A more advanced interface for adding or removing backend servers in reverse/balancer situations”
• “Proper cluster synchronized cache”
• “Easy cache control”

Quite a few addressed the configuration process:

• “Ease of configuration”
• “The overall configuration spaghetti jungle. While I do understand the–extreme–flexibility, when you investigate someone else’s server, you have to wade through 10s of files for perhaps one website”
• “Easier configuration of virtual hosts”
• “Configuration file syntax; I’d completely re-do it to be more modern and less confusing”
• “Config process could be simpler.”

One person asked for “a config GUI.” Another suggested a fixed interval release cycle. Documentation was another frequently raised topic. It is tempting to address or rebut many of the comments. I purposely refrained from editorializing here: a discussion about these comments should develop within the development community, on the developer mailing list.

Conclusion

The survey confirmed several things I suspected. For instance, Linux and PHP are popular. Apache HTTP Server 2.2.x is by far the most widely used version of the server. Other findings may be more unexpected: the importance of SSL and URL rewriting for so many, or the relatively strong showing of the CGI interface.

To my delight, one person offered in the write-in space: “It serves my needs completely” and another wanted to improve “nothing. The current product meets my needs. Thank you so much.”

You are very welcome.

Apache HTTP Server is a trademark of the Apache Software Foundation.

Now, this is only one example of the forty-thousand-odd unique malware infestations spotted in a depressingly short time, but my question is thus: why was a piece of malicious software running (inadvertently one assumes) on behalf of a user allowed to change a system-wide file like hosts? Shouldn’t there be a sandbox for code downloaded from the network that, if it needs to be run at all, prevents it from damaging the underlying operating system?

This situation paints for me the following picture: a tap is running, malware flowing like water into a sieve and onto the floor. The security industry is frantically mopping the floor, trying to stem the flow of malware. They are paid well for their trouble, but meanwhile the expensive rug that represents your business is getting awfully wet. It would be nice if someone could turn off the tap, or design an operating system that doesn’t leak like a sieve.

However, this post is not about Civilization III. It’s about the only other application I use that requires PowerPC compatibility: Quicken 2007. I have now used it for over ten years to manage my finances, track my investments, time and pay my bills, and forecast savings. A couple of weeks ago, Intuit sent out a notice to the effect that Quicken 2007 would not be compatible with Lion, and support for it (such as it was) would end. Customers were advised to migrate to Quicken for Windows (ha!) or Quicken Essentials, their long awaited ground-up rewrite that does take advantage of current SDKs and runs natively on Intel Macs.

Unfortunately, Quicken Essentials has significant feature discrepancies compared with the older product. It has no bill pay feature. It also can’t track investments: the web site suggests that you manually enter stock and fund prices which seems to me a slightly less fun proposition than drying untreated wooden plates and spoons with a tea towel. Finally, Intuit states that they “ we are evaluating options for Quicken Essentials for Mac”, which to me sounds like “It’s dead but we won’t tell you yet because we want to get some more revenue out of it” and is not a confidence builder.

Here’s what I would like my next financial management app to do:

• Run natively on my Mac, without having to run a VM
• Ingest bank statement data through OFX files from multiple financial institutions
• Ideally, pull said OFX files directly from the respective fiancial institutions’ websites (dream, dream)
• Track inter-account transfers. Ideally, instigate inter-account transfers but I’m not holding my breath
• Pay bills, with a settable future payment date. Quicken 2007 lost this capability when Wells Fargo dropped support for that version and WF’s interface is nice, but I now have to enter payments in two different places. This is not ideal
• Track loans: Balance, Interest and Impound
• Break down my Paycheck into various taxes and withholdings (a welcome new feature in Quicken 2007)
• Report on spending by category, tax table, comparison with previous years etc.
• Track investments, keeping track of security prices, cost basis, dividends, etc. for various investment accounts at multiple financial institutions

As far as I can see, I have the following alternatives:

• Drop $50 (or, temporarily, $25) on Quicken Essentials, see if I can live with the reduced feature set, and hope they don’t put me in the same position in the near future
• GNU Cash, an open source finance tracker which seems to have a fairly horrid user interface at first glance, but the major advantage is that there is no company that can unilaterally pull the plug on it
• Buy and Install Quicken for Windows on a VM and use that. Not a viable option as far as I’m concerned
• Buy iBank from the Apple App Store for $60 and see what it’s like. It’s getting some good recent reviews from people clearly in the same boat as I am
• Start using mint.com, which is now also owned by Intuit and has never struck me as the financial management app I need

Dear LazyWeb, what are your experiences with the above? Any alternatives I missed?

• Don’t put both webrat and capybara in your Gemfile. They both define a visit method with different behavior. I had to search long and hard until I happened upon a discussion that clued me in.
• Upgrading to the latest, greatest is very exciting, but if you pull in Rails 2.3.2 as an update, that’s what your installation will default to. This makes your app really unhappy if it’s based on Rails 3. Enforce your expectations by putting gem 'rails', '>= 3.0.0' in your Gemfile.
• Another way to get bitten by upgrades is finding out that the Rake developers took their Domain Specific Language definition private, leaving the rest of the world in the cold. The symptom is that Rake suddenly won’t do anything but emit rake aborted! undefined method `task' for #. Until the world catches up, you’ll need to put a small hack in your Rakefile.
• Ubuntu Linux’ package manager installs Ruby 1.8.7 as ‘ruby’, but Ruby 1.9.1 as ‘ruby1.9.1′. This is a pain, especially when you want to work on your code on several different systems. Fortunately, a post on an Ubuntu forum explains how to use their update-alternatives system to pick one over the other.
• When you use the Debian apt Ruby package, you’ll find that Ruby Gems get installed in /var/lib/gems which is not bad in and of itself, but to use the binaries like rake, cucumber and rails itself, you’ll want to put /var/lib/gems/1.9.1/bin on your PATH.
• You’ll find that this Debian/Ubuntu ruby1.9.1 package actually contains 1.9.2p0 (as of this writing).

Now that this is out of the way, I hope to have some fun with Behaviour Driven Development. Hope the development behaves better than the underlying tools did today.

Responses to the outage were mixed. As question-and-answer service Quora posted on their outage page: “we’d point fingers, but we wouldn’t be where we are today without EC2.” This is true: Amazon and its ilk provide relatively affordable and scalable hosting for applications, and relieve the current wave of startups of the burden of having to invest in and operate their own hosting. However, when you host your application on Amazon, you still have a single point of failure unless you very specifically engineer it to be resilient under failures. EC2 offers many features that can take you beyond a single host deployment. Customers who have adapted their deployment to take advantage of these features withstood last week’s outage with little or no customer-visible impact. Without such adaptations, your web application is no better off than if it were hosted on a conventional web hosting platform.

Amazon operates multiple Availability Zones that are supposed to isolate failures… which did not work too well last week because the issues cascaded across availability zones until the entire Eastern Region was affected. Resilience across geographic regions is not straightforward because the CAP Theorem kicks in: Consistency, Availability, Partition Tolerance, pick two. You can’t have all three at the same time. Engineering an application to withstand outage by distributing it across different availability zones, across regions, or even across different providers is a considerable and costly undertaking, which is not lightly embarked upon by a cash-strapped startup trying to get swiftly to market. Whether to spend this time and money, or whether to tolerate and respond to the occasional outage is a determination that every company will have to make for themselves.

HTTP Server 3.0: Who Needs It? Who Wants It? Who will Write It?
Whither httpd? Does our User Community need a quantum shift that would require a major new version number? Does our Developer have this need and would/could/are they in a position to start major new development on the project? Will 2.x serve us until the end of time?

This topic is partially inspired by the Keynote session Roy Fielding presented in Amsterdam in 2008 on Apache 3.0: two-and-a-half years later seems like a good time to take stock. If you want to talk about this, come to ApacheCon and join the Meetup. Did I mention that rates go up after Friday, October 8?

If you search for temme.net, you find that it is just east of Facebook and just north of Microsoft… and it’s certainly not the smallest possible size. Not bad for a little vanity website!

Learned a couple of interesting things:

• When you run gem outdated on a stock Snow Leopard system, it pulls information from an outdated source which makes it fail to run the next time. Only successfully updating RubyGems itself solves this issue.
• Nobody ever tells you that after sudo gem update rubygems-update, you have to run sudo /usr/bin/update_rubygems. Otherwise, it will keep using the old version and a) can’t update sqlite3-ruby which needs the newer RubyGems and b) will try to keep accessing the outdated source.
• When you want to use Aptana Studio with Eclipse 3.6 (Helios), make sure to install the plugin in the Eclipse installation itself, not under your own user account. This seems to be a bug in Eclipse itself that affects all plugins: if installed under a user account (for instance because the application installation directory is not writable by the user), the plugins don’t show up in the IDE and can’t be used.

There is no better way to procrastinate than to go learn something, and there is no better way to put off learning something than to mess around with tools.

Many software choices are available to professionals who need to run applications in their business. Some of these will be delivered by conventional vendors who have full control over the product and its development. However, over the past decade many Open Source applications have emerged as viable alternatives, developed using an open process by volunteers from many different companies.

Speaking from his experience as an Open Source Software developer, Sander will compare some security aspects of Open Source and Closed Source software, likely debunking some myths along the way. We will examine the security vulnerability mitigation process used by the Apache Software Foundation and discuss how an open development process can provide enhanced security.

See the meeting page for details. An RSVP link is at the bottom of the page.

Oh man an experienced sys admin told me to do it that way.
Please tell me what is wrong in this and where is this documented on Apache
docs.
I want to read.

Here is my response reproduced: read on.

The Apache HTTP Server needs read access to its configuration files and the files it serves. In and of itself, the server does not need write access anywhere on the system: even its log files are opened for write when the server is still root, and the open file descriptors passed to the child processes which change their user id to the lesser privileged user.

Read access only. The web server user should not own, or be able to write to, its configuration files or content.

Content, other than CGI scripts, generally does not need Execute permissions. Even PHP files that are interpreted by the server do not need to be Executable.

Certain applications, especially publishing platforms and Content Management Systems that you manage and populate through the web server itself using a browser, require that certain directories on the system be made writable by the web server user. You can do this by changing the owner of the directory to that user (usually www but ymmv), or by making the directory group-writable and changing the group to the group as which Apache runs.

Making directories writable by the web server should be done only with care and consideration. The usual threat model is that someone manages to upload (for instance) a PHP script of their own making into the document root, and simply executes that by accessing it through a browser. Now someone is executing code on your machine. Google for ‘r57′ for an example of what such code can do.

If a web app needs writable directories, it’s often better to have those outside the DocumentRoot: that way the uploads can’t be accessed from the outside through a direct URL. Some applications (WordPress for instance) support this, others do not.

In many cases, writable directories are not strictly necessary even though the web app might like them: rather than upload plugins (which contain code that gets executed or interpreted, yech!) through the web browser, upload them through ssh and manually unpack them on the server. The CMS Joomla! likes to write its configuration file to the Document Root on initial install (which promptly becomes a popular attack target) but if it can’t write to the Document Root, it will output the config to the browser to the user can manually upload it.

The Apache Documentation will merely tell you to make the server installation root-owned. The HTTP Server Documentation does not cover third party applications like WordPress or Joomla!, so it will not discuss their need to have some directories writable. I hope the above makes the picture a little more complete.