Result: Merged (2026-10-28 01:03, kubeflow/trainer#4334), issue #4310 closed, milestone v2.4. Sourced from GitHub issues, PRs, reviews, timelines, inspection records, Trainer source code (the PR's base commit
8e076764, chart values for tags v2.1.0, v2.2.0, and v2.3.0), Kubeflow's AI policy document, and the contribution ledger draft. Times are in KST.
Table of Contents
- One-Line Summary
- 1. What Was the Problem: Values Ignored Without Error
- 2. Why Start with an Issue Instead of a PR
- 3. What Was Changed: Diff Summary
- 4. Response: Merged Overnight
- 5. Lessons Learned: Open Source Contribution Tips
- Links
One-Line Summary
| Issue | kubeflow/trainer#4310 Installation guide pins chart v2.1.0, so the runtimes and crds flags shown below it are silently ignored |
| PR | kubeflow/trainer#4334 fix(docs): use v2.3.0 in the installation guide |
| Changes | Updated installation and data cache documentation VERSION to v2.3.0 and added a version-pinned runtime installation command. 9 additions, 3 deletions |
| Response | Lead commented on the issue, "Let's bump to v2.3, want to try fixing it yourself?", and on the PR /ok-to-test·/lgtm·/approve all at once |
| Result | Issue opened 10-26 21:43 → Lead reply 10-27 00:28 → PR opened 14:28 → Approved 10-28 00:02 → Merged 01:03. PR took about 10 hours and 35 minutes; from issue to merge took about 27 hours and 20 minutes |
1. What Was the Problem: Values Ignored Without Error
TT Lab is in the process of migrating its table tennis scoreboard recognition model learning to Kubeflow, and Kubeflow Trainer is the component that runs those training jobs on Kubernetes.
While reading the installation documentation, I noticed a mismatch between the top and bottom sections. Line numbers are based on the PR's base commit 8e076764 .
docs/operator-guides/installation.mdline 29:export VERSION=v2.1.0, and for Helm installation,--version ${VERSION#v}.- the same page on lines 59–60and
lines 69–71: To install default runtimes together,
--set runtimes.defaultEnabled=true, or to selectively install them,--set runtimes.torchDistributed.enabled=trueinstructs adding flags like these. - line 48: If CRDs are managed separately,
--set crds.enabled=false.
chart's values.yaml opened across tags reveals the following.
| Chart Version | runtimes Key |
crds Key |
values.schema.json |
|---|---|---|---|
| v2.1.0 | None | None | None |
| v2.2.0 | line 181 | None | None |
| v2.3.0 | line 190 | line 25 | None |
Helm 은 값 검증 스키마가 없는 차트에 모르는 키를 넘겨도 오류를 내지 않습니다. 그래서 문서를 그대로 따라 하면 2.1.0 제어부가 깔리고, ClusterTrainingRuntime 은 하나도 생기지 않으며, 왜 그런지 알려 주는 메시지도 없습니다.
There were two more side issues. In the kustomize installation section, while the control plane had a version-pinned command (?ref=${VERSION}, line 87),
the runtime section only had a ?ref=master command (lines 96–99). The "Install with Release Version" path would mix release CRDs with master runtimes. Before documentation was migrated from kubeflow.org to this repository's doc site (#3308), both commands were present. Also, docs/user-guides/data-cache.md line 51
was also v2.1.0.
I note separately what was and was not verified. Chart values were checked by directly opening tag-specific files (table above, re-verified on 2026-10-29), and I verified that chart 2.3.0 is published in the OCI registry and that
manifests/overlays/runtimes·data-cache exists in v2.3.0. However, I did not actually run the Helm and kustomize commands on a live cluster to observe runtimes failing to generate under 2.1.0 (unmeasured). I explicitly stated this in the PR description as well.
2. Why Start with an Issue Instead of a PR
There was a precedent in this repository where a request to simply bump version numbers was closed. On 2026-07-27, issue #3807 "Bump helm install VERSION to v2.2.1" was closed on the same day by the lead, stating that "README versions are merely examples showing users how to select versions."
Submitting the same content directly as a PR could result in it being closed for the same reason. Therefore, three things were decided during the drafting stage (from the ledger's draft notes).
- Open an issue first instead of a PR.
- Frame it not as "the version is outdated" but as a functional bug where "commands on the same page are silently ignored in that version." Explicitly reference #3807, noting that this case differs because "commands on the same page rely on values that the example version lacks."
- Instead of proposing only version pinning, also propose keeping the example version while adding comments such as "
runtimes.*is supported since v2.2.0, andcrds.enabledsince v2.3.0." State that a PR will be submitted either way.
Issue #4310 was opened on 2026-10-26 at 21:43 in this format. It followed the bug report template's field order, and included an AI disclosure at the end of the body stating, "Claude assisted with research and drafting, including reading chart values across tags."
Two hours and 45 minutes later, the lead replied. Stating that the installation guide should be bumped to v2.3, they asked if I would like to fix it myself. In the same comment, they applied the /kind docs and /good-first-issue
labels. In effect, this meant choosing the first of the two options.
3. What Was Changed: Diff Summary
The PR was opened on the afternoon of the day the lead's reply was received.
# docs/operator-guides/installation.md
-export VERSION=v2.1.0
+export VERSION=v2.3.0
...
-Run the following command to deploy Kubeflow Trainer built-in runtimes
+Run the following command to deploy Kubeflow Trainer built-in runtimes:
+
+kubectl apply --server-side -k "https://github.com/kubeflow/trainer.git/manifests/overlays/runtimes?ref=${VERSION}"
+
+For the latest changes run:
kubectl apply --server-side -k "https://github.com/kubeflow/trainer.git/manifests/overlays/runtimes?ref=master"
# docs/user-guides/data-cache.md
-export VERSION=v2.1.0
+export VERSION=v2.3.0
The runtime commands strictly followed the pattern already used by control plane commands (version-pinned commands, with "latest changes" master commands underneath). Chart README, the exact file closed in #3807, was left untouched, and #3807 was linked in the PR description as the reason.
The "How was this tested" section of the PR body stated the following:
- Documentation was built in a
.readthedocs.yamlcontainer under the same conditions aspython:3.12-slim(Python 3.12, treating warnings as errors).make -C docs html SPHINXOPTS="-W --keep-going -q"finished with exit code 0 and no warnings in both master and this change, and the new commands were visible on the two rendered pages. git diff --checkwas clean..pre-commit-config.yamldoes not have a markdown linter.- Helm·kustomize 명령 자체는 클러스터에 돌리지 않았습니다.
AI usage followed Kubeflow's AI Policy. This policy requires building and testing AI-generated contributions locally before opening them, being able to explain them during review, disclosing them in the PR description or commit footers (Assisted-by·Co-authored-by), and ensuring AI does not automatically respond to maintainer feedback. The commit was given Assisted-by: Claude and DCO signatures, and an "AI assistance" paragraph was included in the body stating, "Claude helped find this and draft the changes, and assisted with running the verifications above." The document build mentioned above serves as the policy's "local verification." The commit appears on GitHub as an unsigned commit.
4. Response: Merged Overnight
| Time | Event |
|---|---|
| 10-26 21:43 | Issue #4310 opened. Welcome bot greets with "first issue" message. |
| 10-27 00:28 | Lead's reply and /kind docs·/good-first-issue |
| 10-27 14:28 | PR opened. Automatically added to project board, two reviewers assigned, size/S label attached. |
| 10-28 00:02 | Lead says "Thanks!" along with /ok-to-test·/lgtm·/approve in a single review. |
| 10-28 01:03 | Merged, issue #4310 closed, milestone v2.4. |
Review comments consisted of just a simple thank you. Since the direction was already decided in the issue, there appeared to be nothing left to question in the PR. All 13 checks, including those run after ok-to-test , passed successfully, with the longest test taking 58 minutes and 42 seconds.
We did not post any separate replies on the issue. The ledger noted: "User replies directly to issues," which effectively translated to replying via the PR the next day. This also aligns with the Kubeflow AI policy guideline that "AI must not automatically respond to maintainer feedback."
5. Lessons Learned: Open Source Contribution Tips
- 「Find closed precedents first and address their reasoning head-on.The lead's remark that "README versions are examples" is correct strictly in terms of version numbers. Rather than contradicting that statement, this issue highlighted a different problem: "Example versions do not match other commands on the same page."
- 「Proposing two fixes makes it easy for maintainers to choose.Whether to pin versions or add comments depends on the project's documentation policy. By presenting both and stating that a PR would be sent for either, the lead made a quick choice in a single line and handed over the work.
- 「"Silently ignored" becomes apparent through a single table.A table showing the line numbers where keys appeared across three tag
values.yamlversions demonstrated the problem without needing to reproduce it on a live cluster. Not reproducing it was explicitly noted as "not run on a cluster." - 「Run documentation PRs through the project's build conditions as well.Since Read the Docs treats warnings as errors, builds were run locally under the exact same conditions. This fulfilled the "local verification" required by the AI policy for documentation PRs.
Tt's Brief Note. The first line of the cookbook said, "Take out the version 2.1 pot," and a few lines down it said, "Press the steaming button on the pot." But the version 2.1 pot didn't have a steaming button! Even if you pretend to press it, the pot says nothing, and there are no steamed dumplings for dinner. When I asked, "Should we bump the pot version, or write 'the steaming button is available from version 2.2' instead?", the chef said, "Let's bump the version, would you like to fix it yourself?" The next morning, the fixed cookbook was placed on the bookshelf.
Links
All materials were accessed on 2026-10-29.
- Issue: kubeflow/trainer#4310
- PR: kubeflow/trainer#4334 (Merge commit
92f88b47) - Fixed documentation (base commit):
installation.md,data-cache.md - Chart values: v2.1.0, v2.2.0, v2.3.0
- Precedent: Issue #3807
- Kubeflow AI Policy: https://github.com/kubeflow/website/blob/master/content/en/docs/about/ai_policy.md