Go modules: air-gapped builds with a file proxy and vendor

Goal

Download Go modules on the connected side into a new module cache and move them to a file proxy, then on the air-gapped side build by two routes: GOPROXY=file:// and vendor. You confirm for yourself how to use go.sum as the import manifest, and the trap where one GOFLAGS line makes vendor ignored.

Why it matters

By default Go downloads from proxy.golang.org and asks sum.golang.org for hashes. In an air-gapped network neither is reachable. The cache/download of the module cache is itself in the shape of the proxy protocol, so simply moving it makes a proxy, and if go.sum is complete there is no need to ask the checksum database. This Pod can reach the internet (80/443), so it plays the download side, and the grader checks the air-gapped side with outbound HTTP(S) redirected to a closed proxy.

Steps

1. In /root/goair/app, create the module example.com/airgap-hello and write a main.go that prints the quote.Hello() of rsc.io/quote v1.5.2. Download it with a new module cache, GOMODCACHE=/root/goair/outside-cache, and also run go mod tidy.
2. Move the contents of /root/goair/outside-cache/cache/download to /srv/goproxy.
3. In the air-gapped-side configuration /root/goair/inside.env, write GOPROXY=file:///srv/goproxy and GOFLAGS=-mod=readonly.
4. Read that configuration and build with the outside blocked and a new module cache /root/goair/inside-cache to produce /root/goair/bin/hello-proxy.
5. Create a vendor directory in /root/goair/app.
6. Leaving the image's GOFLAGS=-mod=mod as it is, try a build with GOPROXY=off and an empty module cache. Save the failure message in /root/goair/gomod-trap.txt.
7. With GOPROXY=off, build with vendor alone to produce /root/goair/bin/hello-vendor.
8. Write an import record /root/goair/intake.txt, copying the module zip hash lines of go.sum (the lines without /go.mod attached) in the form 모듈 버전 해시 (the three fields are the module, the version, and the hash). Each hash must equal the .ziphash in /srv/goproxy.

Notes

• Current values: go env GOPROXY GOFLAGS GOMODCACHE GOSUMDB
• Reading the environment file: set -a; . /root/goair/inside.env; set +a
• Blocking the outside: HTTPS_PROXY=http://127.0.0.1:9 HTTP_PROXY=http://127.0.0.1:9 go build ... (Go honors these variables)
• Common mistake 1: building with the usual module cache, succeeding, and stopping there. The cache you had already filled helped, so you have not tested the proxy.
• Common mistake 2: creating vendor and not checking GOFLAGS. In this image vendor is ignored.

Download side: fetch dependencies into a new module cache

In /root/goair/app, create the module example.com/airgap-hello and a main.go, download rsc.io/quote v1.5.2 with GOMODCACHE=/root/goair/outside-cache, and also run go mod tidy.

Create the module with go mod init and download the exact version with go get. If you give a new cache directory as GOMODCACHE, only what you download this time piles up in it. tidy organizes the dependencies that are used into direct dependencies.

Move the module cache to a file proxy

Move the contents of /root/goair/outside-cache/cache/download to /srv/goproxy.

Under cache/download in the module cache, everything is already in the shape of the proxy protocol (@v/list, .info, .mod, .zip). Copy it keeping the directory structure as it is. The other directories of the cache, which hold unpacked source, are not needed.

Go environment file for the air-gapped side

In /root/goair/inside.env, write GOPROXY=file:///srv/goproxy and GOFLAGS=-mod=readonly.

These are name=value lines that the shell can read as they are. After the file scheme comes an absolute path, so there are three slashes. -mod=readonly keeps the build from quietly editing go.mod.

Build from the file proxy with the outside blocked

Read inside.env and build with the outside blocked and a new module cache /root/goair/inside-cache to produce /root/goair/bin/hello-proxy.

Read the environment file so that the variables are exported, give GOMODCACHE a new directory, and send outside addresses to the closed proxy. If you build with a cache you already filled, you have not tested the proxy.

The second route: vendor

Create a vendor directory in /root/goair/app.

Among the subcommands of go mod there is one that copies the dependency source into the repository. Check whether vendor/modules.txt appears as a result.

A build that fails even though vendor exists

Leaving the image's GOFLAGS=-mod=mod as it is, try a build with GOPROXY=off and an empty module cache, and save the failure message in /root/goair/gomod-trap.txt.

This step is for recording a failure. First look at the current value with go env GOFLAGS. If you recall from the reading how -mod=mod treats vendor, the message makes sense.

Build from vendor with GOPROXY=off

With GOPROXY=off, build with vendor alone to produce /root/goair/bin/hello-vendor.

There is a -mod value that makes Go use vendor. Override the environment's GOFLAGS or give it directly on the command line. It has to work even if the module cache is empty.

Write the import record from go.sum

Write /root/goair/intake.txt, copying the module zip hash lines of go.sum in the form 모듈 버전 해시 (the three fields are the module, the version, and the hash). Each hash must equal the .ziphash in /srv/goproxy.

A line of go.sum has three fields: module, version, and h1: hash. A line with /go.mod after the version is the hash of the go.mod file and differs from the zip. Compare line by line against the .ziphash files in the proxy directory.