TT Lab
Get started
Learn Learning paths Courses

GitLab CI/CD

I inherited before_script, and one line disappeared

Continue in TT Lab

Goal

You strip duplication out of configuration with include, extends, !reference, YAML anchors, default, and spec:inputs, and confirm with the execution log and the merged configuration what each one's merge rule actually leaves in a job.

Why it matters

As a pipeline file grows, you split out the common parts for reuse, but each reuse mechanism has a different merge rule. Hashes are merged and arrays are replaced, anchors work only inside a file, and default is used only when nobody decided. If you don't know these rules, the incident "a setup command you believed was inherited goes missing from just one job" happens without any error. A template that takes inputs keeps you from copying the same job for each environment, but if you don't set an allowed range, even a typo in an environment name becomes a job as it is.

Steps

  1. Make /root/glci-reuse a git repository. In /root/glci-reuse/ci/templates.yml, put a hidden job .base (before_script is the one line echo base-setup, and the variable is LOG_LEVEL: info), and in .gitlab-ci.yml, bring in that file with include: - local: ci/templates.yml and then put stages [build, test] and a job build (stage build, extends: .base, script echo "LOG=$LOG_LEVEL"). Run it with gitlab-ci-local --shell-isolation --no-artifacts-to-source and see whether base-setup and LOG=info are printed in the build log.
  2. Add two more jobs. test (stage test) extends .base, writes the variables as LOG_LEVEL: debug and PYTEST: "1", and has script echo "LOG=$LOG_LEVEL PYTEST=$PYTEST". lint (stage test) extends .base and has its own before_script (echo lint-setup) and script echo lint. Run it and confirm that the test log has base-setup and LOG=debug PYTEST=1, and that the lint log has only lint-setup and no base-setup.
  3. Add a job package (stage build). Without using extends, write before_script as two items, !reference [.base, before_script] and echo package-setup, so that when run, base-setup is printed and then package-setup. The script is echo package.
  4. In /root/glci-reuse/broken-anchor.yml, include ci/templates.yml and put a job build that uses an anchor not defined in this file, *base_vars, in variables as <<: *base_vars. Save the output of gitlab-ci-local --file broken-anchor.yml --list to /root/glci-reuse/anchor-error.txt. And in .gitlab-ci.yml, define an anchor &docs_vars (DOCS_OUT: public) within the same file, and in the variables of a job docs (stage test), put <<: *docs_vars and DOCS_FMT: html, and with script echo "$DOCS_OUT/$DOCS_FMT" make public/html be printed.
  5. At the top level of .gitlab-ci.yml, under default:, put before_script echo default-setup. And a job report (stage test, script echo report) does not inherit the defaults, using inherit: default: false. Run it and confirm that the docs log has default-setup, the report log has no setup line at all, and the build and test logs still print base-setup.
  6. Make /root/glci-reuse/ci/deploy.yml a template with a spec:inputs header. The input env allows only one of staging and production, and replicas is a number with a default of 1. After the header (---), a job deploy-$[[ inputs.env ]] (stage deploy) runs echo "deploy <env> replicas=<replicas>". .gitlab-ci.yml adds deploy to stages and includes this file twice — staging (default replicas) and production (replicas 3).
  7. Save the output of gitlab-ci-local --preview to /root/glci-reuse/expanded.yml. This file must have the result with include, extends, !reference, anchors, default, and inputs all resolved. The grader runs the same command in a copy of the repository and checks that the content is the same and a few values.

Notes

Include a template file and inherit with extends

Make /root/glci-reuse a git repository. In /root/glci-reuse/ci/templates.yml, put a hidden job .base (before_script is the one line echo base-setup, and the variable is LOG_LEVEL: info), and in .gitlab-ci.yml, bring in that file with include: - local: ci/templates.yml and then put stages [build, test] and a job build (stage build, extends: .base, script echo "LOG=$LOG_LEVEL"). Run it with gitlab-ci-local --shell-isolation --no-artifacts-to-source and see whether base-setup and LOG=info are printed in the build log.

include merges several files into one configuration and then interprets it. A job whose name starts with a dot does not appear in the list and is used only as a fragment to hand down. gitlab-ci-local sees only files tracked by git, so git add a new file.

Hashes are merged and arrays are replaced whole

Add two more jobs. test (stage test) extends .base, writes the variables as LOG_LEVEL: debug and PYTEST: "1", and has script echo "LOG=$LOG_LEVEL PYTEST=$PYTEST". lint (stage test) extends .base and has its own before_script (echo lint-setup) and script echo lint. Run it and confirm that the test log has base-setup and LOG=debug PYTEST=1, and that the lint log has only lint-setup and no base-setup.

extends is a deep merge. A hash such as variables is merged key by key and overrides on top of the inherited keys, but an array such as before_script is not merged and is replaced whole by what the job wrote.

To merge arrays, use !reference

Add a job package (stage build). Without using extends, write before_script as two items, !reference [.base, before_script] and echo package-setup, so that when run, base-setup is printed and then package-setup. The script is echo package.

!reference is a tag that inserts the value of a specific key of another job (including a hidden job) in place. It can also point to a fragment of an included file, so it fills the limit of extends, where an array is replaced whole.

Anchors cannot cross a file boundary

In /root/glci-reuse/broken-anchor.yml, include ci/templates.yml and put a job build that uses an anchor not defined in this file, *base_vars, in variables as <<: *base_vars. Save the output of gitlab-ci-local --file broken-anchor.yml --list to /root/glci-reuse/anchor-error.txt. And in .gitlab-ci.yml, define an anchor &docs_vars (DOCS_OUT: public) within the same file, and in the variables of a job docs (stage test), put <<: *docs_vars and DOCS_FMT: html, and with script echo "$DOCS_OUT/$DOCS_FMT" make public/html be printed.

Anchors and aliases are handled when the YAML parser reads a single file. include is a stage where GitLab merges afterward, so another file's anchors are already gone. Reuse across files is done with extends or !reference.

default is a floor value — it gets pushed out by extends and inherit

At the top level of .gitlab-ci.yml, under default:, put before_script echo default-setup. And a job report (stage test, script echo report) does not inherit the defaults, using inherit: default: false. Run it and confirm that the docs log has default-setup, the report log has no setup line at all, and the build and test logs still print base-setup.

default is a global floor value filled into jobs that decided nothing, so it is not used in a job that already received before_script through extends. inherit is a switch that turns off, job by job, whether to receive the defaults.

Stamp out per-environment jobs with a template that takes inputs

Make /root/glci-reuse/ci/deploy.yml a template with a spec:inputs header. The input env allows only one of staging and production, and replicas is a number with a default of 1. After the header (---), a job deploy-$[[ inputs.env ]] (stage deploy) runs echo "deploy <env> replicas=<replicas>". .gitlab-ci.yml adds deploy to stages and includes this file twice — staging (default replicas) and production (replicas 3).

The spec header decides the shape and allowed range of the values that can be passed when you include. Inside the template, you use a value with $[[ inputs.이름 ]] (the placeholder stands for the input name). If you pass a value that is not in the allowed list, the pipeline is rejected before it is created.

Extract the merged configuration that GitLab will see

Save the output of gitlab-ci-local --preview to /root/glci-reuse/expanded.yml. This file must have the result with include, extends, !reference, anchors, default, and inputs all resolved. The grader runs the same command in a copy of the repository and checks that the content is the same and a few values.

The more the configuration is scattered over several files, the harder it is to answer "what does this job actually run" from the files alone. If you attach the merged result to a review, you do not have to work out the merge rules in your head. The pipeline editor on the GitLab screen has the same function too (viewing the full configuration).