Linux vs. Windows Metrics -- Nothing Is Quite What It Seems To Be

The author compared the times of three tasks: boot up, loading Firefox, and shutting down. That's hardly a comprehensive set of tests. Some people commented to dismiss these metrics as "meaningless". They aren't meaningless but they certainly aren't the whole story. Others pointed out that IE on Windows was faster than Firefox on Linux and that MS Works was faster than OpenOffice. Some then responded that Works isn't the equivalent of OO and that MS Office would be a better comparison. It all got a little shrill with those who believe that Linux is faster than Windows and those who say it isn't so talking past each other and resolving nothing.

I'm going to try and sift through the morass and say what I think the numbers really mean and what they don't mean. Those with an agenda, either agenda, will, I'm sure, attack what I have to say. I think anyone who really tries to look at things objectively probably won't. I'm just not sure that very many people are truly objective.

In the interest of fairness let me disclose where I am coming from: Yes, I tend to have a pro-Linux bias. I also have a bias against hype and B.S. When it comes to my professional life I'm an IT mercenary. If someone wants me to support Windows systems along with Linux or UNIX systems I will gladly take their money and do the work. I also won't evangelize on behalf of Linux. Why not? In business everything comes down to a cost vs. benefits analysis and there are situations in the real world where a change of OS is far too costly to justify any perceived benefits. There are many situations where Windows or commercial, proprietary UNIX really and truly is the best fit. Back to those pesky metrics...

If you read the blog headline or the comments this is mostly portrayed as Linux vs. Windows. In reality that just isn't true. These numbers only compare Xandros preloaded on the Eee PC with Windows XP preloaded on the Eee PC. All Linux distributions are not created equal. On my five and a half year old Toshiba laptop, a box with specs very close to those of the Eee PC, some distributions are much faster than others. For example, Kubuntu is much slower than Vector Linux SOHO even through both have a KDE desktop. There are people running all sorts of Linux distributions on the Eee PC. I've also seen where some users have managed to strip down and shoehorn Windows Vista onto one. These metrics are about one specific laptop, one specific distribution and version of Linux, and Windows XP only, nothing more.

One of the Windows supporters tries to make the claim in the comments that Xandros must be well tweaked on the Eee PC and that XP must be poorly configured. Anyone who has tried XP on a system with similar specs knows that isn't true. Asus primarily sells Windows machines just like all the other major laptop vendors. They know how to configure XP. The fact is that both operating systems are well configured and optimized by Asus.

That brings us back to the numbers. There is no denying that on identical hardware running an identical application (Firefox) that Xandros is significantly faster than XP. Any comparison of dissimilar apps (IE vs FIrefox or Works vs. OpenOffice) is an apples and oranges comparison. It tells us nothing about the speed of the underlying OS. What if I compared AbiWord and Gnumeric instead of OO? Might Linux seem faster again? It's not a fair comparison, is it? Similarly, the time to shutdown or start up is a fair comparison and Xandros is clearly faster than XP again. All the arguments dismissing this as unimportant to most people doesn't change the fact that Xandros is faster at this given task.

What does this all prove? Not much, really. I think most people who have had significant experience with Linux know that in general, with identical apps, a well configured and optimized Linux system will outperform a well configured and optimized Windows system by some measurable amount. Whether that amount is significant or not depends on the application and what is important to the user. In the real world there are very few situations where people will chose Linux simply because it's faster. As the critics of the blog post correctly point out Asus delivers the Eee PC with applications that work very well on the laptop regardless of which OS is chosen. In the final analysis the blog post is little more than Linux hype. There are far more compelling reasons to buy an EeePC with Xandros rather than Windows.

Is there any value to the blog post? In fairness, yes, there is. It adds to an already large body of evidence that Linux scales down to low spec and legacy hardware easily. In general Linux is the better choice for such machines because it's modular. You can leave out bits you don't need. You can choose a lightweight desktop or lightweight applications, trading some features for faster performance. That faster performance is often the difference between comfortably usable and totally unusable on older or slower hardware. Linux distributions, good solid ones, built specifically for limited performance hardware are readily available.

In the Linux world lightweight applications and desktops are under current support and development which means that if a security vulnerability or a bug is found it gets fixed. Folks who choose to run Windows '98 instead on older hardware are stuck in place. Windows XP Lite is a much better alternative but it adds cost and people who are running legacy or low spec hardware are usually very cost conscious. Lightweight apps are less plentiful for Windows and in many if not most cases they will cost the user additional money they may not want to spend.

What usually makes Linux painful or impossible for the average Joe or Jane user is the task of installation and configuration. Asus took away that pain by offering a very attractive little box at a good price with an easy to use, ready to go Linux installation. Now the trick is getting Joe or Jane user to seriously consider that the Xandros Linux version, if they did their own cost vs. benefit analysis, really is the better choice and not just because it boots up faster.

Full Story