TT Lab
Get started
Learn Learning paths Courses

CBA — Backstage Associate

The Portal Backend Started with Production Config Never Becomes Ready

Continue in TT Lab

Goal

Bring up a real Backstage backend the way production does. You layer two configuration files with --config, inject secrets as environment variables, open CORS so the browser UI can call it, and move the DB so data survives a restart. The verdict is based not on files but on the behavior of the running process (the listening port, readiness, 401/200, response headers, and data remaining after a restart).

Why it matters

A backend that worked locally, once deployed to production, listens on an unexpected port, is up but not ready, has API calls blocked only from the UI, or loses what you registered on every restart. The cause is mostly not the code but the order in which configuration layers, missing environment variables, a client and server with different origins, and an in-memory DB. Knowing the merge rules in your head is different from measuring their result on a running process.

The first VM start takes about 4 minutes, and the installation in step 1 takes about 30 seconds from the internet. This VM has no docker or podman, so you do not build a container image (command -v docker is empty). You do not build the frontend app either, and instead of a browser you check the CORS response with curl that adds an Origin header.

Steps

  1. Install the backend packages at pinned versions in /usr/local/cba-prod.
  2. Bring it up with the base and production configuration files layered via --config, and see what changes if you reverse the order.
  3. See that a backend started without ${PORTAL_API_TOKEN} does not become ready, and then inject the secret as an environment variable.
  4. Allow only the portal UI's origin with backend.cors.origin.
  5. Widen the allowed origins with an APP_CONFIG_ environment variable, without editing the file.
  6. Record that the registered location disappears when you restart on the :memory: DB.
  7. Move the DB to a directory so that it survives a restart.
  8. Write the production configuration checklist using values measured on the current process.

Notes

Install the backend for testing production configuration at pinned versions

In /usr/local/cba-prod, create an npm project (including package.json) and install three packages at exact versions with no range symbols. Keep the npm cache at npm_config_cache=/usr/local/cba-prod/.npmcache. @backstage/backend-defaults@0.17.8, @backstage/plugin-catalog-backend@3.9.1, better-sqlite3@12.4.1.

npm init -y and then npm install --save-exact .... package.json is not only a record of dependencies but also the reference by which the backend finds the project root when it starts (without it, startup dies with NoPkgJsonFound).

After layering the production file, it listens on a different port

Create /usr/local/cba-prod/app-config.yaml (base) and /usr/local/cba-prod/app-config.production.yaml (production). Base: app.baseUrl: http://localhost:3000, backend.baseUrl: http://localhost:7007, backend.listen.port: 7007, DB better-sqlite3 and ':memory:', catalog.rules: [{allow: [Component, Location]}], and a location file and ./catalog/portal.yaml (Component portal-web). Production: only backend.baseUrl: http://localhost:7300 and backend.listen.port: 7300. Create an index.js that contains only the catalog plugin, and /usr/local/cba-prod/start.sh, which starts the backend again with NODE_ENV=production as node index.js --config app-config.yaml --config app-config.production.yaml (appending output to /usr/local/cba-prod/backend.log), and run it. For comparison, start it once with the --config order reversed and look at the listening port, leave one line reversed_listen_port=<그때 포트> in /root/cba-prod/order.txt (the placeholder is the port at that time), and then go back to start.sh.

If you pass even one --config, automatic loading of the default file is turned off, and the files are layered in the order given, so the later one wins. The first log line Loading config from MergedConfigSource{...} shows the files actually read and their order. Check the listening port with ss -ltnp. Start start.sh with setsid nohup ... >> backend.log 2>&1 < /dev/null & so that it survives after the shell ends.

A backend started without its secret does not become ready

In the production file, add backend.auth.externalAccess with type static, token ${PORTAL_API_TOKEN}, and subject ops-cli. First start it without the environment variable, see what /.backstage/health/v1/readiness returns, and save the message containing Missing required config value from the log to /root/cba-prod/missing-env.txt. Then put PORTAL_API_TOKEN=<직접 만든 24자 이상 무작위 값> in /root/cba-prod/secrets.env with permission 600 (the placeholder is a random value of at least 24 characters that you generate yourself), change start.sh so that it reads that file and passes it on as an environment variable, and start again. Without the token, /api/catalog/entities must return 401, and with Authorization: Bearer <그 값> (the placeholder is that token value) it must return 200, and the token value must not be written in any YAML.

If the environment variable used in a ${VAR} substitution does not exist, that value is missing entirely, and the service that reads a required value fails while starting. The process does not die and the port is open, so it ends up "up but not working." NODE_ENV=production logs are JSON, so they are easy to read with jq -r .message. Generate the value with openssl rand -hex 16 or node -p "require('crypto').randomBytes(24).toString('base64')".

Only the browser's portal UI may call the backend

In the production file, add app.baseUrl: http://portal.example.test:3000 and backend.cors.origin: http://portal.example.test:3000, and start again with start.sh. A request with the header Origin: http://portal.example.test:3000 must get back Access-Control-Allow-Origin with the same value in the response, and with Origin: http://evil.example.test that header must be absent. Also send an OPTIONS preflight request (Access-Control-Request-Method: GET) once and look at the response code.

The frontend app is served in the browser at app.baseUrl and calls the API at backend.baseUrl. If their origins (scheme, host, port) differ, the browser decides whether to allow it by looking at the backend's CORS response headers. curl does not enforce CORS, so check by whether the header comes back: curl -s -D - -o /dev/null -H 'Origin: ...' URL.

Widen the origins only for this deployment, without editing the file

Let the local development UI http://localhost:3000 call this backend too, but do not edit the YAML. In start.sh, put the JSON array ["http://portal.example.test:3000","http://localhost:3000"] in the environment variable APP_CONFIG_backend_cors_origin and start again. Both origins must receive Access-Control-Allow-Origin, and http://evil.example.test must still not receive it.

In the name after APP_CONFIG_, _ becomes . to form the configuration key, and the value is interpreted as JSON first. Environment variables take priority over every configuration file. See what EnvConfigSource{count=...} in the first log line changes to. The JSON double quotes must survive inside the shell quoting.

After the restart, the registered location is gone

With the token, send {"type":"url","target":"https://git.example.test/portal/catalog-info.yaml"} to POST /api/catalog/locations to register it (201), count the entries of GET /api/catalog/locations, restart with start.sh, and count again. Leave three lines in /root/cba-prod/memory.txt: id=<등록 응답의 location.id>, before_restart=<개수>, and after_restart=<개수> (the placeholders are the location.id in the registration response and the counts).

Even if the registration response is 201, actually reading that url is a later processing stage (this host has no such address, so the read fails — here you only check that the registration record remains). Remember what the DB configuration was.

Move the DB to disk and it survives a restart

In the production file, add backend.database.connection.directory: /usr/local/cba-prod/db (the client is merged from the base file's better-sqlite3). Start again with start.sh, register the same url again, save that POST response JSON to /root/cba-prod/persist.json, then restart once more and confirm that GET /api/catalog/locations/<id> is 200.

Objects are merged deeply key by key, so you only need to write connection in the production file. If you give a directory, a SQLite file is created for each plugin — see with ls which files were created. In production you normally use PostgreSQL.

Fill in the production configuration checklist with the current process

In the first five lines of /root/cba-prod/report.md, write values measured from the backend that is up now — listen_port= (the port node listens on), base_port_open= (whether a connection to 7007 succeeds, yes/no), evil_origin_allowed= (whether http://evil.example.test receives ACAO, yes/no), env_config_count= (the EnvConfigSource count in the last start log), and locations_now= (the number of entries from GET /api/catalog/locations). Below that, write how the browser UI (app.baseUrl) and the backend (backend.baseUrl) communicate, the warning that appeared in the log under NODE_ENV=production, and the reason you did not build a Docker image on this VM.

The numbers and yes/no are values you measure now, not from memory. ss -ltnp, curl, grep 'Loading config from' backend.log | tail -1. For the warning, look for lines with "level":"warn" in the JSON log. Check whether the tool exists with command -v docker.