Rocky Linux details the loopholes that will help its RHEL rebuild live on

Last week, the Rocky Linux project said it had found a way to continue delivering its RHEL-based distribution. Now we have some information on how it's doing it.

The decades-long battle between Red Hat and the various organizations cloning its enterprise Linux distro has taken a new turn. If you've been following the news recently, Red Hat fired a salvo against the RHEL rebuilds by stopping publication of its enterprise distro's source code to its Git repository. Soon afterwards, we covered the news that the Rocky Linux project announced that it had found a workaround.

In a blog post boldly entitled "Keeping Open Source Open", the project describes two different ways that it can obtain RHEL source code without contravening Red Hat's license agreements.

One of these is via the Hat's public UBI container images. These so-called Universal Base Images are discrete subsets of the RHEL user land, intended to be run under a container orchestration tool as a fully compatible container for running RHEL-specific applications. That's all well and good, but by its very definition, a container doesn't contain the operating system kernel or drivers – it's just user-space components. However, these UBIs are available to everyone, whether they are a RHEL customer or not, so they do provide a good start.

To build a complete standalone distro, though, you do also need to obtain the source code to the kernel and drivers. This source code is available to Red Hat customers – and nowadays, only to customers – in the form of SRPMs, or source RPMs. In pretty much all Linux distributions, for each binary package containing a compiled, ready-to-run program, there is also a corresponding package of source code in the distro's repositories.

In ordinary use, you never normally need these, but it's the nature of open source software that you can compile it yourself. So, for example, if you need a program that isn't in your distro's repositories, you can obtain the source code to that program and build it yourself. The problem is that most nontrivial programs incorporate bits of lots of other programs – in some cases, a ridiculous number of them. This is a problem that's occasionally pointed out by open souce luminaries; for example, Poul-Henning Kamp's essay "A Generation Lost in the Bazaar", which pointed out that even back in 2012 you needed 122 external packages in order to compile Firefox. (That was right at the start of Mozilla's rapid release cycle for Firefox.)

You may not need to rebuild bits of the distro itself, but you still need the source code in order to build things that aren't part of the distro. So distros include source packages, and that's what the RHEL rebuilders need.

The RHEL UBI containers also include the kernel headers – the source code that's needed to compile kernel modules. If you run a Linux distro in a VM and install the hypervisor's guest additions – for instance, to enable 3D acceleration – you may have seen a warning message that you need to install the kernel headers source package.

The Rocky project's other workaround uses RHEL VMs on pay-per-use public cloud providers. Those cloud VMs run full, complete RHEL instances. Rocky's continuous integration system can start a RHEL instance in the cloud – thereby making them Red Hat customers, at least briefly – using that instance to fetch all the new or changed source packages and post their contents to the Rocky Git servers, then shut down and delete the VM… at which point, the ownership – and the license agreement – are terminated.

That's the theory anyway. The blog post says:

The project seems confident that it's fully license-compliant, although it does note:

We suspect that this approach may be subject to change quite fast. While the RHEL ownership of the VM instance is indeed temporary, the cloud providers are effectively acting as resellers of Red Hat's products. In our original article, we noted that attempts to use Red Hat's free developer accounts to access the RHEL source code could well result in cat-and-mouse games, with the Hat terminating accounts it felt were infringing. We suspect this applies equally to accounts on cloud vendors: if those are used to obtain the source in ways that the Hat doesn't like, it seems very likely to us that the Hat will instruct the cloud providers to close those customer accounts… or risk losing the rights to resell RHEL.

• Rocky Linux claims to have found 'path forward' from CentOS source purge
• Linux 6.4 debuts after literally unremarkable development push
• Red Hat strikes a crushing blow against RHEL downstreams
• Red Hat to stop packaging LibreOffice for RHEL

RHEL remains an important product to a lot of big businesses, as is emphasised by the fact that "Big Purple" just extended the support lifetime for RHEL 7. This was due to go end-of-life next year, a decade after it was introduced, but it's just received a stay of execution: ELS until 2028… complete with kernel version 3.10.

As we reported previously, a lot of people in the wider Red Hat user community remain extremely upset over this source code policy shift, and many of them report that they are investigating moving to other distributions – notably Debian, possibly for its Long Term Support life cycle. The problem is that enterprise users do not choose distributions on the basis of technical merit, but rather in terms of third-party support. If you are using hardware whose vendor only offers drivers for RHEL, or the applications that you need are only supported on RHEL – or worse still, a specific (and quite possibly already obsolete) version of RHEL – then you may not be free to move to anything else.

As we recently described, Red Hat goes to extreme lengths to keep old kernels supported and secure. We suspect in part that may be why it's happy to give away container images – because they don't include that all-important, super-stable kernel.

Nobody chooses RHEL because it's state of the art. It isn't. In fact, it's about as far away from state of the art as it's possible to be, and in the rapidly changing world of open source, that's a very desirable attribute for a certain type of purchaser. They choose it because one specific version will be supported for a decade plus, and that in turn is why vendors support RHEL, and often nothing but RHEL. Back in the 1990s, there was a campaign whose website was named redhatisnotlinux.org – specifically because in the eyes of so many people and companies, Red Hat Linux really was the only distribution that mattered. For big businesses today, its descendant often still is. ®