The certificate was renewed, but the browser still showed the old one

Renewed, yet the old certificate is served

Goal

Reproduce a renewal incident against an HTTPS server that reads its certificate only once at startup, and write two monitoring scripts that watch the certificate the server serves rather than the one on disk.

Why it matters

The renewal tool changes the file while the server serves memory. Without a restart in between, the "renewal succeeded" log and the "expiring soon" warning are both true at the same time. The provided server /opt/app/tlsserve.py is a Python HTTPS server that calls load_cert_chain exactly once and, like most real servers, does not reread the file when it changes. The monitoring scripts in this lab are checked in both directions (same file FRESH, different file STALE) against a server that the grader starts separately.

Steps

Create /root/renew/ca.key and /root/renew/ca.crt (CN=Lab Root CA), /root/renew/server.key, and a 3-day /root/renew/v1.crt signed by the CA (SAN DNS:www.lab.internal).
Copy v1.crt to /root/renew/live.crt and server.key to /root/renew/live.key, then start python3 /opt/app/tlsserve.py --cert /root/renew/live.crt --key /root/renew/live.key --port 8443. Save the serial= line of the certificate retrieved with openssl s_client to /root/renew/02-served.txt.
Issue a 30-day /root/renew/v2.crt with the same key and overwrite live.crt with it (this is what a renewal tool does). Do not restart the server.
Use s_client again to save the serial= to /root/renew/04-served.txt, and save the body of curl -sk https://127.0.0.1:8443/ to /root/renew/04-body.txt. The disk must have v2 while the server has v1.
Create /root/renew/certdiff.sh <인증서파일> <host:port> (the first argument is the certificate file, the second the host:port). If the SHA-256 fingerprint of the file on disk and that of the certificate the server serves are the same, it must print FRESH and exit with 0; if they differ, it must print STALE and exit with 1. Save the output of running it against live.crt and 127.0.0.1:8443 to /root/renew/05-detect.txt.
Stop the server, start it again, and save the serial= from s_client to /root/renew/06-served.txt.
Create /root/renew/servedcheck.sh <host:port> <초> (the second argument is a number of seconds). If the certificate the server serves expires within that many seconds, it must print EXPIRING and exit with 1; otherwise it must print OK and exit with 0. Save the output of running it with 127.0.0.1:8443 604800 to /root/renew/07-monitor.txt.
In /root/renew/08-report.md, write the serial numbers of v1 and v2, the method by which you made the server read the new file, and why monitoring has to look at the certificate the server serves rather than the one on disk.

Notes

s_client waits for standard input, so use it like echo | openssl s_client -connect 127.0.0.1:8443 -servername www.lab.internal 2>/dev/null | openssl x509 -noout -serial.
Stopping the server: pkill -f tlsserve.py. When you start it again, run it in the background with nohup ... &.
To save the received certificate to a file temporarily inside a script, you can write it out with openssl x509 -out <임시파일> (replace the placeholder with a temporary file path).
Common mistake: if you restart the server before step 4, the incident disappears. If the monitoring script in step 7 looks at the file on disk, it fails against the grader's server.

CA and a 3-day v1

In /root/renew/, create ca.key, ca.crt (CN=Lab Root CA), server.key, and a 3-day v1.crt (SAN DNS:www.lab.internal).

The order is the same as in the previous module — self-sign the CA, a CSR with a SAN, sign with x509 -req (-copy_extensions copy). You will use the CSR again in step 3, so keep it as server.csr.

Read the certificate the server holds

Copy v1.crt to live.crt and server.key to live.key, start tlsserve.py on port 8443, and save the serial= line received with s_client to /root/renew/02-served.txt.

Run the server in the background with nohup and send its log to /root/renew/server.log. If you pass the whole s_client output to x509 -noout -serial, you get a single serial= line.

Renew — overwrite the file

Issue a 30-day /root/renew/v2.crt with the same key and overwrite live.crt with it. Do not restart the server.

Just sign the same CSR again (-days 30). A new serial is assigned and the expiry date moves later. cp v2.crt live.crt is all that a renewal tool does.

Disk has v2, server has v1

Save the serial= from s_client to /root/renew/04-served.txt and the body of curl -sk https://127.0.0.1:8443/ to /root/renew/04-body.txt.

The server keeps serving what it read at startup. Both files must show the serial of v1 for the incident to be reproduced. The -k in curl turns off verification because the CA is your own, and the body contains the serial the server reports about itself.

A script that compares disk and server

Create /root/renew/certdiff.sh <인증서파일> <host:port> (FRESH and exit 0 if they are the same, STALE and exit 1 if they differ), and save the output for live.crt and 127.0.0.1:8443 to /root/renew/05-detect.txt.

Get the two fingerprints — for the file, x509 -in FILE -fingerprint -sha256, and for the server, pass the s_client output to x509 -fingerprint -sha256. If you cannot get a certificate from the server, it is good to print ERROR and return 2. The grader runs two cases against its own server: the same file and a different file.

Restart — the server reads the new file

Stop the server, start it again, and save the serial= from s_client to /root/renew/06-served.txt.

Stop it with pkill -f tlsserve.py and start it again with the same command. Because it reads live.crt fresh at startup, this time the serial of v2 must appear.

Expiry monitoring of the served certificate

Create /root/renew/servedcheck.sh <host:port> <초> (EXPIRING and exit 1 if it expires within that many seconds, otherwise OK and exit 0), and save the output for 127.0.0.1:8443 604800 to /root/renew/07-monitor.txt.

Save the certificate received with s_client to a temporary file with x509 -out, and then apply -checkend to it. You must not look at the live.crt on disk — the grader gives its own server, which holds a 2-day certificate, a 7-day window and checks that EXPIRING comes out.

Incident report

In /root/renew/08-report.md, write the serial numbers of v1 and v2, the method by which you made the server read the new file, and why monitoring has to look at the certificate the server serves.

Extract the two serials from v1.crt and v2.crt with -serial and put them in as they are. If the words "restart" and "s_client" are included, the grader finds the conclusion.