A Dashboard You Made by Clicking Is a Dashboard You Will Lose

In one line

The truth of a dashboard should live in a file. Grafana's database is merely a screen that has drawn that file.

Why this was needed

The path by which a dashboard created by clicking disappears is always the same.

During an outage you quickly create a few panels. They are useful.
You open that dashboard again next week. You share the link with the team.
Six months later you move the cluster or set Grafana up again.
Nobody can bring that dashboard back. The person who made it is already on another team.

There is one more quiet loss. When someone changes a threshold line or deletes a panel, no history remains. There is no way to answer the question "wasn't this panel here originally?"

How it works

Keeping a dashboard in a file takes two things. The dashboard JSON, and a provider file that tells Grafana to watch that directory. Confusing the two is the first hurdle.

# provisioning/dashboards/lab.yml — JSON 이 아니라 '어디를 보라' 는 지시다
apiVersion: 1
providers:
  - name: lab
    type: file
    updateIntervalSeconds: 10
    allowUiUpdates: false
    options:
      path: /root/graf/dashboards      # ← 여기 있는 *.json 이 대시보드가 된다

For a dashboard provided this way, meta.provisioned in the API response is true. If you try to overwrite the dashboard through the API in that state, Grafana refuses with Cannot save provisioned dashboard. This blocks the path by which the screen and the file could drift apart.

It is fine even if a dashboard with the same uid already exists in the database. Provisioning takes it over — which is why moving a dashboard you made by clicking into a file is possible. When building, create it conveniently in the UI, and when finished, pull out the JSON and solidify it into a file.

The uid is the address

Pinning the uid is the heart of this work. The dashboard link is /d/shop-api/..., so the uid is the address, and alerts, docs, bookmarks, and links from other dashboards all point to that value. If you delete the uid from exported JSON and put it back in, you get two of the same dashboard, and the links stay pointing to the old one.

Common misconceptions

"If I commit the exported JSON as is, I'm done." Inside that JSON, the data source is embedded by uid. If the uid differs per environment, it becomes a blank screen in other environments. You must pin the uid to the same value in every environment, or pull the data source itself out into a variable.

"It's fine to commit the id field too." id is a serial number that is meaningful only inside that Grafana. In the place you move it to, it can collide with another dashboard's id. In JSON that will travel around, it is safer to keep only uid and delete id.

What really matters in practice

The diff of dashboard JSON is hard for a person to read. Coordinates and fields move all over the place, so reviews easily drift into "LGTM."

So there is one rule worth agreeing on as a team — make people write, in one sentence in the PR description, the question the changed dashboard answers. If they cannot write that sentence, that change is usually just one more panel added, and that is the road to the wall of graphs that this course talked about at the start.