The now infamous Heartbleed bug found in the openSSL library, sent some shockwaves down the software industry. It also revealed some misconceptions about Open Source software that many people assume are common sense but are not necessarily true.
1) Linus’s Law
“Given enough eyeballs, all bugs are shallow” – Eric Raymond
Linus’s law claims that given enough users using it and developers reviewing its code, flaws in open-source code will be revealed and eventually fixed, resulting in a more correct and secure code compared to closed / proprietary software.
The book “Facts and fallacies of software engineering” refers to this as Fallacy #8. It cites research claiming that the rate at which additional bugs are uncovered does not scale linearly with the number of reviewers. I think that the fact the humans are not good at catching software bugs should be obvious to any developer. While obvious bugs such as syntax issues or use of anti-patterns can be caught during code review, many bugs can only be exhibited when actually using the program.
So that leaves the users part of the equation. Surely, if bugs are missed during code reviews, they will be caught during real-world use of the resulting program, right? again, this turned out to be false. Same as with code reviews, only obvious problems are easily caught. An issue that manifests itself only under very specific conditions or not in a way that is obvious to the end-user, can go undetected for a long time. The more complex a program is, and the more it does for the user, the harder it is to ascertain the correctness of the program from the user side. Since programs typically do things that would take a human being much more time, it’s very difficult to repeat the process manually to ensure correctness.
The Heartbleed issue exemplified this – OpenSSL is used and run by millions of companies around the world, many of which have dedicated software engineers working for them full-time, while reaching hundreds of millions of users. And yet, this issue was undiscovered for almost 2 years, and would have remained so if a security team hadn’t stepped up to the plate.
A comment I read on Hacker News a couple of days ago summarized it very well –
The “many eyes” canard isn’t just about formal reviews, though. It’s also about the idea that when something’s open source the reviews aren’t limited to maintainers or close colleagues who would be involved regardless of open vs. proprietary. What actually seems to happen in the great majority of cases is that the circle of people who actually understand any particular commit is only very slightly greater than it would be otherwise. The bugs are only a very tiny bit shallower, if at all. The fact that it was just one reviewer in this case, despite the code being widely available, only underscores that point.
In this way, the benefits of open-source regarding correctness and security compared to closed-source software are minimal, if any. The strength of open-source is actually in the ability to fix the bug yourself, even though in this case most people (and developers) were waiting for an official fix to the issue since patching security software and especially something like OpenSSL, which is known for the complexity of its code-base, is out of reach for most software developers.
2) Most Open Source is not professionally developed and maintained
As Heartbleed showed, even mission critical software used by a large portion of the Internet does not necessarily have the resources to be professionally maintained. The OpenSSL team receives only about $2000 yearly in donations. Many articles were written about this vulnerability in an accusatory tone, and yet none of them considered that those are people working voluntarily on their free time, performing what can only be considered as ungrateful public service.
The researcher who introduced the bug is not a professional software developer by occupation – he is a Ph.D student. While proficient enough to contribute to a very specialized library such as OpenSSL, he stumbled across a common pitfall with C programming dealing with memory management.
This is one of the core strengths and weaknesses of open-source – that anyone can contribute, regardless of background or proficiency. Many open-source projects have been created and maintained by amateurs for fun, intellectual curiosity or research purposes, which is part of its beauty and at the same time something which people using it should be aware of.
While some high profile projects get corporate sponsors that allow it to have professionals dedicated to and working on it full-time, most open-source projects barely scrape small amounts through donations. I believe that providing open-source with the same resources as commercial software is the best way to create sustainable projects of higher quality that everyone can benefit from.
For projects that do not get corporate sponsors (which is typically outside of your control), alternative business models such as Open-Core (for example, MySQL) and services and curation (for example, Red-Hat) are available. Relying on donations alone is simply not enough in most cases.
Hopefully, someone will pick up the ball with OpenSSL as it’s so important to the fabric of the Internet. Big companies using it as mission critical software, should be dedicating resources (i.e, developer time) to reviewing and improving the code. Everybody shares the responsibility of making the code better and correct, it should not fall on the shoulders of a few core maintainers.
3) Every software has bugs. Open Source typically has less resources to find it
Many people still feel entitled, even when receiving something for free. Many complaints and accusations were raised against the maintainers of OpenSSL, which have been doing an important public service for years. They have provided, for free, a library that allows people and businesses to encrypt their communications and protect their data. They did so on their free time and without significant recognition for their efforts.
Some have gone as far as to suggest the bug was introduced on purpose. Unless you are a conspiracy theorist, suggesting that people doing such thankless work as the OpenSSL devs would actually introduce a bug on purpose, is ridiculous.
The truth is that software will forever have bugs. This is something every software developer out there knows by experience from day 1. No matter how many tests you do, and how rigorous a process you have, some bugs will always slip through. You can minimize it with more resources, but you can never eliminate it entirely. Could the Heartbleed issue have been caught if all of its maintainers were working on it full-time and having more time to dedicate to a more rigorous testing and review process? it’s possible, but we’ll never know.
This is something that users of Open Source software should be aware of. Bugs will exist in all software, including Open Source software, while Open Source developers typically have much less resources to find and fix it.
So instead of feeling entitled and throwing accusations, be grateful for the free (in cost and in usage) tools that are available to you. When possible, give back to the maintainers to show your gratitude and to help support their efforts. If you are really dependent on an Open-Source solution, I would go as far as encourage the developers to work on it full-time, even if it means you no longer get a free (as in cost) ride.
Author: Eran Galperin