Tue 1 Feb 2005
Brian Jones writes on linux.com:
Red Hat has really been grating my nerves as of late. Undocumented features that are advertised but unusable, recommended updates that are thoroughly broken, failure to update packages with high-impact problems, and a poor software packaging and distribution policy have me testing other distributions with a goal of taking my business elsewhere.
This heightened level of aggravation with Red Hat started with their split from the community version. It should be noted that I think this was a smart business decision for Red Hat. There was no way they could continue to have a single distribution that was all things to all people. However (and some of this is coincidental), after that happened, I started having very large problems with the commercial version of Red Hat Linux (specifically, Red Hat Advanced Server). I wanted to standardize on the commercial version of Red Hat for all of the machines in my department that were running Linux. I also wanted that done so that we wouldn’t go down the road of maintaining more than one distribution as more and more production services were deployed on Linux.
Another part of my goal was to reduce the number of packages we build from source. Building packages from source means that every time the application is updated, we have to build it from source again, and hope it all works. One or two packages isn’t so bad, but we build our SSH server, SSL, print server, database server, web server, PHP, and plenty of other stuff from source. This adds up to a lot of overhead. Some of this is, I believe, historical in nature, so I decided to analyze the actual need for this, and I found some success in one service in particular: CUPS.
The CUPS Print Server Debacle
CUPS is a print server package. The first problem I had with the Red Hat version was that the version in the package is almost totally unrelated to the capabilities of the package. What exactly, then, is the purpose of having a version number on the package? That was an annoyance — luckily, the package, as it turns out, came with a copy of the ps2ps command that worked as it was supposed to, which is to say that it properly stripped out PJL code from PS files printed from Windows machines. Great news, since we could then retire an aging Perl script we used to do this. Great — we were ready to go into testing.
We tested printing against this machine for a couple of weeks. The week before we were to move it into production, we ran up2date on the machine, and ps2ps broke again. Wonderful. As if that weren’t enough, there’s another problem on that new server that could result in an unintentional DoS. The installed (and running by default) authd daemon’s config file (/etc/xinetd.d/authd) is configured by default with the key “instances” set to “UNLIMITED”. As a result, every time one of our guys who was running a fairly strict firewall tried to connect to the box via SSH, the authd server would spawn hundreds of processes trying to connect back to his machine. This caused the load to immediately spike, and the entire machine becomes completely unusable. Thanks, Red Hat.
Lackadaisical
In addition, other machines that were to move into production had also been updated, only to find that the aacraid driver was broken, and they couldn’t boot! The report in bugzilla for the RAID driver resulted in Red Hat saying they would release a fix for this in the next kernel package, which would be released the next time a security vulnerability was reported and fixed — they would release them in the same package. I couldn’t believe this, especially since I knew that the biggest server retailers used hardware that required that driver. As a result of this, my machines remained running with a locally-exploitable kernel vulnerability for the sake of a RAID driver. Thanks, Red Hat.
Up2Date Rollbacks, or not
This makes for a nice segue into the undocumented-but-advertised features area. After running up2date on these machines and noting their extreme brokenness, I very much wanted to do a rollback on certain packages. I had enabled rollbacks in my up2date configuration, only to find when I actually needed to do one that the process is completely undocumented by Red Hat. It would seem antithetical to the cause of releasing a product that is, if nothing else, stable and predictable, to release a package with features enabled that are not only undocumented, but to the best sources my searches could find, are not recommended for production use by Red Hat. Unfortunately, I didn’t learn of the issues with rollbacks until I needed to use the feature, at which time I found that the best solution was to scrap rollbacks, save my disk space, and just reinstall the package in question from scratch again. Rollbacks are not a part of a well-balanced administrative policy, as implemented by Red Hat.
The most recent update also broke the quota program. The feature that was broken is one that had been noted as being broken several months before the updated release. This was a recommended update from Red Hat, and it’s not like just a command line option is broken — the program, as a whole, is unusable. This was in Red Hat’s bugzilla in May 2004. How it wound up in an update in December of the same year is beyond me. How it hasn’t been updated since that time is also beyond me. Thanks, Red Hat.
New Face, Old Packages, Same Price
Finally, due to Red Hat being slow as molasses to implement new and highly recommended features of software packages, I am forced to completely discount them as a solution for some of the services I’d like to deploy without building from source. My prime example is OpenLDAP. I’ve been trying for way too long to migrate the department from NIS to LDAP for authentication. NIS won’t go away completely, because it’s useful for some things, but we like LDAP for several reasons, and for some things it’s far better than NIS, so we’re moving in this direction.
The current version of OpenLDAP is 2.2.23. The version the Red Hat package is based on is 2.0.x. That version of OpenLDAP, if I’m not mistaken, didn’t even fully support LDAPv2, let alone version 3, and used a backend that the OpenLDAP developers have long ago shunned. For some unknown reason that would seem like a whole lot of work, Red Hat appears to be applying patches to this old, decrepid version of OpenLDAP and distributing it, presumably with the expectation that someone, somewhere will use it. As far as I know, it is the oldest version of OpenLDAP distributed by default with any Linux distribution. If I use Red Hat for this service, I would be forced to build not only OpenLDAP, but the updated version of the Berkley DB backend that is (for the past ~2 years) the standard OpenLDAP backend recommended by the development team. I’d be doing this by myself, and applying updates by myself, in spite of paying Red Hat a yearly fee for “updates”. Thanks, Red Hat.
The Point
The point is, what exactly is my motivation for staying with Red Hat? I can’t seem to find one. I still use Fedora Core on things that don’t matter much (when I can get it to work — that distro has plenty of issues of its own), but outside of that, I’m evaluating other options. I’m finding some really nice tools. I’m finding far more up to date software packages. I’m finding much more admin-friendly tools and environments. And I’m finding that Red Hat, comparatively speaking, is not a very big deal in the server space. Overall, I’m seeing no benefit to using Red Hat’s enterprise line over, say, the Fedora line. Maybe they started charging for the wrong distribution.