TT Lab
Get started
Learn Learning paths Courses

Air-Gapped Mirrors and a Private CA

Trust it system-wide, or only for one repository?

Continue in TT Lab

In one line

On RHEL-family systems, you put a private CA in /etc/pki/ca-trust/source/anchors/ and spread it across the whole system with update-ca-trust extract. But making the entire system trust it is not always the right answer. If only one repository has to be verified with a different CA, sslcacert in the .repo narrows that scope.

Why this was needed

dnf on an air-gapped RHEL server looks at the internal repository. The moment you stand that repository up over HTTPS, you get errors of the Curl error (60) family along with Cannot download repomd.xml. The common patch is sslverify=0. Then anyone posing as the repository on the internal network can serve a package list. Repository metadata is the document that decides "what to install, with which hash", so not checking the origin of that document means entrusting the whole installation.

How it works

The system trust store. According to the RHEL 9 documentation, you place the certificate in /etc/pki/ca-trust/source/anchors/ (or the lower-priority /usr/share/pki/ca-trust-source/anchors/) and run update-ca-trust extract. Both PEM and DER are accepted. You can check with p11-kit's trust list, and add and remove with trust anchor <파일> (the placeholder is the file) and trust anchor --remove. In this lab image (Rocky 9.3), /etc/pki/tls/certs/ca-bundle.crt is a symbolic link pointing to /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem. What extract rewrites are the files under this extracted directory, and tools that look at the system bundle, such as curl and dnf, read that result.

Making only one repository trusted. The dnf configuration documentation describes the repository option sslcacert as "the path to the CA file used to verify the SSL certificate, the system default if empty", and gives sslverify a default of true. When you have to download from a repository a partner set up with its own CA but do not want every program on the server to trust that CA, you write sslcacert only in that repository's .repo. It is a way to narrow the scope of trust to exactly what is needed.

Creating a repository. createrepo_c <디렉터리> (the placeholder is the directory) reads the rpms and creates repodata/ (repomd.xml plus primary, filelists, and other). dnf starts by downloading repodata/repomd.xml under baseurl. The web server only has to serve files.

Signature checking is separate. TLS confirms "is this the right repository server", and the rpm GPG signature (gpgcheck) confirms "did the distribution make this package". An rpm copied from an official repository still has its original signature, so if you point gpgkey at the distribution's public key (/etc/pki/rpm-gpg/), you can install from the internal repository with signature checking on. The logic that "it is an internal repository, so I may turn off one of the two" does not hold.

What it looks like in the field

I measured this in this lab image. When I served an HTTPS repository with a private CA certificate and pointed to it with a .repo, dnf makecache went through Curl error (60): SSL peer certificate or SSH remote key was not OK ... [SSL certificate problem: unable to get local issuer certificate] and ended in 0.3 seconds with Cannot download repomd.xml ... All mirrors were tried (if you give -q, the earlier cause line disappears and only the last line remains, so leave it out when recording). When I put the root in anchors and ran update-ca-trust extract, label: Airgap Internal Root CA, trust: anchor, and category: authority appeared in trust list, and the same command passed. When I installed tree, the From repo of dnf info --installed tree was airgap — this line is how you confirm which repository something was installed from.

What you will do in the next lab

You build /srv/rpmrepo from the rpms of the offline repository and serve it over HTTPS at repo.airgap.internal:8443 with a private CA certificate. You write the .repo, record the failure, then make the system trust it with anchors and update-ca-trust and install tree. Finally, for a second repository set up with a different CA, you make only that repository trust it with sslcacert.

Reference documents: RHEL 9 — Using shared system certificates · trust(1) · DNF Configuration Reference (sslcacert, sslverify)