Resurrecting Flags and Seemingly Empty Buckets: The Story of How a SeaweedFS weed mini Help PR Led to a Behavior Change

Result: Merged (2026-10-16, seaweedfs/seaweedfs#11755). This was actually taken from GitHub PR and review originals, the SeaweedFS source (the PR's base commit 590a75b9), and TT Lab repository's commit and documentation records. Times are in Korea Standard Time (KST). As of 2026-10-17, no release includes this change yet (latest release 4.48, 2026-09-28).

Table of Contents

One-line Summary
1. What Was the Problem: Symptoms We Experienced
- Trap One: Deleted Flags Resurrect (2026-09-19)
- Trap Two: Restored Copy's Buckets Appear Empty (2026-09-24)
2. Cause: Which Lines in the Source?
- mini.options: Not "Did you pass it on the command line?" but "Is it equal to the default?"
- filer.toml: The image nails down the metadata location
3. What Was Changed: Diff Summary
4. How Did Bots and Maintainers React?
5. What We Learned: Open Source Contribution Tips
Links

One-line Summary


PR	seaweedfs/seaweedfs#11755 `mini: note mini.options and Docker filer.toml pitfalls in help`
Changes	`weed mini -h` 11 lines of Notes at the end of help. Code behavior unchanged
Reaction	Three review bots reviewed within 9 minutes, pointing out three issues. The maintainer opened a PR fixing the behavior #11758 and merged it first
Result	The maintainer added a Notes modification commit to our branch and merged it as a single commit (our account as author, maintainer as co-author)
Time Taken	PR published 2026-10-15 08:15 → Merged 2026-10-16 10:44, about 26.5 hours

Figure: Flow of a single contribution and what happened in #11755

1. What Was the Problem: Symptoms We Experienced

weed mini is a mode that runs SeaweedFS's master, volume server, filer, and S3 gateway as a single process. The TT Lab cluster uses this in two places: as a community/blog media storage, and as a production DB backup storage. The backup storage was originally MinIO, but we migrated to SeaweedFS in September 2026 when MinIO ended binary and image distributions.

Trap One: Deleted Flags Resurrect (2026-09-19)

Volume expansion stopped in the media storage, and the cause was the master's raft state carrying over across repeated restarts. Since it is a single master with no state to preserve, we attached -master.resumeState=false to make it start fresh every time.

However, when testing locally, even if we deleted this line from the command line and started it again, it still booted up with false . This was because mini wrote command-line flags to a file in the data directory and read them back on the next startup. We left a comment in the manifest saying, "Do not delete the line when rolling back." mini.options Nowhere did it mention this file. weed mini -h Nowhere did it mention this file.

Trap Two: Restored Copy's Buckets Appear Empty (2026-09-24)

A backup is only a backup if you can restore it. We tested restoration by copying the backup storage's data directory to a NAS and spinning up mini with the same image. When we attached the copy to a different path in the container and supplied -dir as that path, only two buckets were visible and zero objects existed. There were no errors. When we attached the copy to /data , all 10,004 objects migrated from MinIO were read with their keys, sizes, and SHA-256 hashes intact, exactly as they were when copied.

Both times, the issue was not "we configured something wrong" but "there were rules the tool didn't tell us." The next person is bound to walk into the exact same traps.

2. Cause: Which Lines in the Source?

All line numbers below are based on the PR's base commit 590a75b9 . TT Lab's used version 4.29 also had the same function contents (opened tag 4.29's mini.go to compare).

Figure: The two traps of weed mini and how they look after #11758

mini.options: Not "Did you pass it on the command line?" but "Is it equal to the default?"

Save: saveMiniConfiguration writes cmdMini.Flag.Visit to (except for all command-line given flags)(dir (lines 1166–1179). <dir>/mini.options (lines 1166–1179).
Apply: applyConfigFileOptions for each value in the file, if that flag's current value is equal to the default value(flag.Value.String() == flag.DefValue, line 1138), it overwrites it with the stored value.
Exception: Only the eight port flags checked whether they were "given on the command line" right after startup record them separately, and skip the provided ones (line 1130).

Therefore, if you give -master.resumeState=false once, it gets saved to the file, and upon starting after deleting the line, since the value is the default true , the file's false is reapplied. Even if you explicitly specify =true on the command line, it is the same as the default, so the result is the same—with the exception of port flags. The comment we left in the manifest, "Do not delete the line, change it to ", is also advice that won't work according to the source (this is based on reading the source, not actual measurement). Furthermore, since mini suppresses default logs, the message that "saved options were applied" ( =true ) is not visible in the default settings.V(2)).

filer.toml: The image fixes the metadata location

The official image copies the repository's docker/filer.toml to /etc/seaweedfs/filer.toml (lineDockerfile.go_build 82). The contents are three lines: turning on the [leveldb2] repository and dir = "/data/filerldb2" .
The filer, in filer_server.go lines 254–264 , if it finds filer.toml , it does not use the default repository location and leaves only a single warning line (skipping default store dir). The default location pulled by mini from -dir (line 1348) is discarded at this time.

As a result, volume files go to -dir , and filer metadata goes to /data/filerldb2 . If you move -dir elsewhere and leave only that location, the metadata disappears along with the container. The reason both buckets were visible in our restoration test is likely because the startup flag -bucket newly created the buckets on top of the empty metadata (an interpretation based on the source's bucket creation code and the flags in our manifest).

3. What was changed: Summary of diff

Without touching the code, we added Notes to the end of the help text for cmdMini of Long . This is the gist of the first submitted version.

+Notes:
+- Flags are saved to <dir>/mini.options and applied again on the next
+  start to every flag that is not given on the command line. Removing a
+  flag, or passing it with its default value, does not restore the
+  default on the next start. Edit or delete mini.options instead.
+- The official Docker image ships /etc/seaweedfs/filer.toml, which keeps
+  the filer metadata in /data/filerldb2 regardless of -dir. If you set
+  -dir to another path, keep /data on a persistent volume as well.
+  ...

There are two reasons we only fixed the documentation. Changing the behavior is a design decision for the maintainer to make, and an external contributor's first PR needs to be small and easy to review to have a good chance of being accepted. SeaweedFS's PR template includes a checkbox stating, "Any AI-generated PR has been reviewed line by line and kept to a minimum." We checked both boxes and added a sentence at the end of the body stating that we wrote it with AI assistance and reviewed it ourselves. We added a DCO signature (Signed-off-by) and an SSH signature to the commit. We stated directly in the body that testing consisted solely of comparing gofmt and the above functions in the source code.

4. How the Bot and Maintainer Responded

Three Bots Arrived in 9 Minutes

Right after opening the PR (10-15 08:16–08:24), three review bots—Devin, CodeRabbit, and Greptile—attached themselves. There were three public points raised, all of which were related to the first Note.

CodeRabbit: Our text stated it applies to "flags not provided on the command line," but the actual criterion is "does the current value equal the default?" Also, port flags skip saved values if the default is specified—mention the exception.
Greptile: Same port exception. Port flags can be reverted simply by specifying the default, so the instruction to "delete the file" is too broad.

Looking back at the source code in Section 2, the bots were right. Our phrasing was correct for general flags, but wrong for the eight port flags. Even after reading the list of exceptions (portFlagNames), we failed to include them in the text.

The Maintainer Acted Before We Responded

To be honest, we neither replied nor made fixes in response to these points for a full day. In the meantime (10-15 23:15), the maintainer referenced our PR and opened #11758 . The problem definition was precisely the two traps our PR had noted, and the solution was a change in behavior, not just the documentation.

mini.options Command-line flags are no longer saved at all. Only the values calculated by mini itself (currently the volume size limit) are pinned so that restarting keeps them the same. Old files are rewritten once upon application to leave only the calculated values.
Values in the file skip flags provided via the command line or WEED_* environment variables. The "does it match the default?" comparison is dropped in favor of checking "was it provided?"
Applying saved options prints a one-line notice upon startup (with passwords and token values masked).
filer.toml If the local storage location enabled by -dir is outsidewarnIfFilerStoreOutsideDir).

, a warning is issued at startup (-dir #11758 was merged at 10-16 10:36, and three minutes later, the maintainer pushed a commit to our branch updating the Notes to match the new behavior. The first Note was updated to "Command-line flags are never saved and always override saved values; only calculated values are saved," and "Warn at startup if outside

" was added to the second Note. In a comment, they summarized that thanks to this change, the bot's feedback regarding default comparisons and port exceptions was no longer applicable.

The Bot's Second Feedback and Sequence Explanation-dir Greptile left two new comments on the new commit: "The source code contains code that saves command-line flags," and "There is no code checking outside

." The bot was looking at code prior to #11758. To each inline comment, the maintainer replied, "This PR comes after #11758, which removes saving and adds the check," and the bot retracted both points in under a minute. Then it was merged at 10:44. Co-authored-by The merge was a squash combining the two commits, with our account listed as the commit author and the maintainer as

. Out of the 13 lines of final documentation, the text we originally wrote that remained intact was mostly the second Note, while the first Note was rewritten by the maintainer to fit the new behavior.

5. What We Learned: Tips for Open Source Contributions Documentation PRs can be the most polite bug reports.
An issue stating "this behavior is surprising" easily leads to debate, but a PR stating "let's document this behavior" forces the maintainer to confront the behavior directly. This time, the maintainer chose to fix it rather than just document it, and the result was all the better. When explaining behavior, read up to the list of exceptions. portFlagNames What the bot pointed out was precisely the
we had read but left out of the text. Documentation that is "mostly correct" is incorrect documentation for anyone caught by an exception. If it is correct, fix and re-upload; if it is incorrect, reply with the source line. This time the maintainer resolved it first, but leaving the review comment open for a day was our oversight. We now make it a rule to daily scan and respond to bot comments on uploaded PRs.
Bots do not know the order between PRs. If there is a dependent PR, writing "must be merged after X" in the body or comments ensures both humans and bots see the same picture. That is what the maintainer did.
Write symptoms and actual measurements with numbers. Instead of "the bucket looks empty," write "two buckets, 0 objects, no errors, /data attached to , 10,004 items are identical up to SHA-256." Sentences like this allow maintainers to gauge the scale of the problem without reproducing it.
Read the project's AI rules first. SeaweedFS accepts AI-generated PRs but verifies review and minimal scope as checklist items. Rules differ by project, and some places do not accept AI-written PRs at all. A single notice sentence and signature are basic across all projects.

Titi's word. I put up a sign in the help section saying "There is a hole here!", and the administrator came and plugged the hole! But they left the sign. When the bot says "The text on the sign is a bit wrong," we decided to rewrite it on the same day rather than the next day.