Red Hat strikes a crushing blow against RHEL downstreams

Comment Red Hat has decided to stop making the source code of RHEL available to the public. From now on it will only be available to customers — who can't legally share it.

A superficially modest blog post from a senior Hatter announces that going forward, the company will only publish the source code of its CentOS Stream product to the world. In other words, only paying customers will be able to obtain the source code to Red Hat Enterprise Linux… And under the terms of their contracts with the Hat, that means that they can't publish it.

In the opinion of the Reg FOSS Desk, the blog post itself is so full of corporate language that it borders on obfuscatory. However, we've contacted the Red Hat press office, and the company confirmed that the release does say what we got out from reading between the lines. This is very bad news for downstream projects which rebuild the RHEL source code to produce compatible distributions, such as AlmaLinux, Rocky Linux, EuroLinux, and Oracle Unbreakable Linux.

The core difference is that CentOS Stream is upstream of RHEL: it's what will become the next point release of RHEL. At risk of sounding uncharitable, it's a sort of continuous rolling beta of the next version of RHEL. Alma, Rocky, and so on, and the former CentOS Linux, were downstream of RHEL: they were rebuilds from the same source code, guaranteeing perfect compatibility. So you could run one of the rebuilds, without paying Red Hat anything, while using the same drivers and getting perfect compatibility with RHEL apps.

You don't get that with CentOS Stream: It's a preview of the future of RHEL. Which is handy if you are a partner company developing products or drivers to run on RHEL, or you're a customer who wants to know what's going to come next. It's much less useful if you just want to run RHEL without paying. Or, of course, if you want to build your own copy of RHEL. We suspect that the wider RHEL user community doesn't care about Stream very much, and that may be a motivation behind the latest move.

In various forums online, there are outcries from users of downstream distros… just as there was when the Hat cancelled CentOS Linux a few years ago. Once again, people are talking about betrayal of trust, violating the GPL, and so on. However, as far as we can see, the Hat is acting perfectly in accordance with the terms of the GPL, which only requires them to make source code available to people using the binaries built from them: in other words, to its paying customers. The key point being is that to obtain those binaries, customers – as well as developers on free accounts – must agree to a license agreement and are under the terms of a contract, which overrides the GPL license of the code itself.

In a way, this could be interpreted as a logical continuation of the move made when the company brought CentOS in-house back in 2014. That move legitimized this particular one of the several extant RHEL rebuilds, which resulted in the rest of them essentially shutting down their efforts — except of course for Oracle, which has deep pockets to fund Oracle Linux, complete with cheaper enterprise support contracts, an enhanced Btrfs-compatible kernel and so on.

Having given the move time to successfully eliminate most of the clones, Red Hat then killed off its own official free version of its paid-for flagship product. Instead, it switched to offering a free test version, an announcement which it made accompanied with lots of positive-sounding language about community involvement, and so on. In fact, what it was really doing was cutting off those that might be seen, from its perspective, as a bunch of freeloaders. The move was accompanied with free production deployment of RHEL for developers – but only up to 16 machines.

Way forward

The door has not been completely closed. If we understand it correctly, in effect, Stream is periodically resynchronized with RHEL when there is a new major release. So, when RHEL 11.0 is released, Stream will briefly be in sync with it — which means that downstream distros could grab a copy of the code at that exact point in time, and build a new version compatible with that point-zero release of RHEL. The problem for the downstreams is that from that point on, they won't be able to get ahold of usable source code of each subsequent point release and the various ongoing updates.

Some commentators are pointing out that it's possible to sign up for a free Red Hat Developer account, and obtain the source code legitimately that way. This is perfectly true, but the problem is that the license agreement that you have to sign to get that account prevents you from redistributing the software.

So although the downstream distros could still get hold of the software source code, they can't actually use it. In principle, if they make substantial modifications, they can share those, but the whole raison d'être of RHEL-compatible distros is to avoid major changes and so retain "bug-for-bug compatibility."

Of course, they could take a "publish and be damned" attitude and do it anyway. At best, the likely result is immediate cancellation of their subscription and account. That could work but will result in a cat-and-mouse game: downstream distributors continually opening new free developer accounts, and the Hat potentially retaliating by blueprinting downloads and stomping on violators' accounts. It would not be a sustainable model.

At worst, though, they could face potentially getting sued into oblivion.

In summary, it will still be possible to obtain the source code, via several different routes — although some of these come with very serious restrictions attached. For now, the official reactions of both AlmaLinux and Rocky Linux are guardedly optimistic, although there are signs of concern in the discussion on the Rocky Linux forum.

Back in 2011, Red Hat changed the way that it distributed its source code packages in a way that certainly looked as if it was specifically intended to make life difficult for the rebuilds. We don't know the company's motivations, and it's certainly not going to tell us, but perhaps that move was not successful enough and thus led to actually bringing CentOS in house.

As FOSS desk said when CentOS Stream 9 came out, we feel that Red Hat's key mistake was adopting CentOS Linux in the first place. The move endorsed and legitimized a free competitor to the company's own paid-for, commercial product. (And one that was struggling at that point, which ought to have been no concern whatsoever to Red Hat.) If the plan was to inconvenience Oracle in some way, it failed, but it surely significantly cut into RHEL's sales.

Back then, the downstream distributors managed to find ways around that move, and it is perfectly possible that they will also be able to find ways around this one – but it will make continuing substantially more difficult. It's possible that the Hat is perturbed by the success of the new generation of rebuilds. While the bodies behind both Rocky and AlmaLinux are basically non-profits – Rocky is a public benefit corporation founded and owned by Greg Kurtzer – they are doing well.

For example, just last week, NASA licensed Rocky Linux for its internal use. CERN and Fermilab meanwhile use AlmaLinux.

The Big Purple Hat is presenting the move as not a particularly big deal – as if it were simply designed to boost takeup of Stream, while in fact it actually looks like a concerted attack upon the thriving new ecosystem of rebuilds that resulted from the cancellation of CentOS Linux.

TL;DR?

The timeline is long and involved, and if you are getting confused by now, we don't blame you in the least. The main events, and this jaded old vulture's interpretations of them, happened as follows.

• 1994: Red Hat Linux released.
• 2002: the first version of RHEL, version 2.1 – yes, really – released, based on RHL 7.2.
• 2003: Red Hat Linux 9 released, after which development stopped.
• 2006: First Oracle Linux released, based on RHEL 4.5 and with the same version number.
• 2011: Red Hat restructures its source releases, making life more difficult for the RHEL rebuilds.
• 2014: Red Hat adopts CentOS Linux, making it official.
• 2020: Red Hat ends development of CentOS Linux, switching to CentOS Stream.
• 2021: First release of AlmaLinux follows a few months later, followed by Rocky Linux a few months after that.
• 2023: Red Hat stops making RHEL source code available to non-customers.

Meanwhile…

The job cuts which struck both Kyndryl and Red Hat earlier this year in the United States are now starting here in central Europe, and several former colleagues and some personal friends of the Reg FOSS Desk were laid off this week, from the Hat as well as from other arms of Big Blue.

• Microsoft has made Azure Linux generally available. Repeat, Azure Linux
• Red Hat releases RHEL 9.2 to customers, with buffet of rebuilds for the rest of us
• CentOS Stream 9: Understanding the new Red Hat OS release for non-Red-Hat-type people
• CentOS project changes focus, no more rebuild of Red Hat Enterprise Linux – you'll have to flow with the Stream

If this move does in fact result in the demise of Alma, Rocky et al, companies and communities which hundreds of people have just spent several years building, the end result may be a boost to IBM's bottom line, but it will also mean public opinion turning even further against the company. Since it was founded some 30 years ago, Red Hat permitted clones and rebuilds of its operating systems, back to the early days of Red Hat Linux. That is where Mandrake Linux started out for example: as a rebuild of Red Hat Linux with the KDE desktop, at a point in time when the Hat felt that Qt's license precluded including it. Summarily killing them all off is not a move that is going to win the Big Purple Hat any friends… Except possibly among IBM shareholders.

What does this mean for Fedora?

Fedora users, and indeed contributors, need not fear – although some serious discontentment is showing on the Fedora-Devel mailing list.

Fedora is upstream of RHEL: Software developed and tested in Fedora flows into CentOS Stream, whence it flows into RHEL. RHEL is in fact what pays for much of the work that goes into Fedora. If a cynical but not entirely unfair summary of CentOS Stream is that it's a rolling beta of the next RHEL point release, Fedora is a sort of rolling alpha of the next RHEL major release.

So while RHEL depends on Fedora technologically, the reverse is not true; principally, Fedora only depends on RHEL financially.

There are server versions of Fedora, and Red Hat users wanting a free RHEL relative can use those however they wish. The differences are that they're based on newer code, so they are far from identical to RHEL and never will be… and of course there are no stable long-term support releases of Fedora. ®