Admin: embedded Grafana dashboards tab (same-origin reverse proxy)

kreeader-user commented

2026-08-29 18:40:20 -05:00

Owner

Andrew wants Grafana dashboards embedded as a tab in the Admin section. Probed: hound Grafana sends X-Frame-Options: SAMEORIGIN, so direct iframes are blocked — the design is an admin-gated reverse proxy under the server origin (satisfies SAMEORIGIN by construction) with server-side service-account credential injection (keys/grafana_service_account, token line only). Configurable dashboard list. Build-tagged with the existing kreeader_admin console. If dashboard page loads still require a Grafana-side toggle (anonymous viewer / auth proxy), the exact one-line ask goes to questions/ rather than being worked around.

andrew commented

2026-08-29 18:48:10 -05:00

Owner

Live probe findings (read-only GETs) — two premises in the issue body corrected

1. The Grafana host in the issue body is wrong

http://192.168.3.9:3000 is Forgejo, not Grafana — GET /api/health there returns Not found. and GET / returns the Forgejo app shell (<title>Forgejo: Beyond coding. We Forge.</title>).

The real instance, per the already-working kreeader-docs/scripts/progress-report/grafana.py (DEFAULT_GRAFANA_URL), is https://grafana.jiggl.in (resolves to 135.131.74.147, fronted by Caddy):

GET https://grafana.jiggl.in/api/health
{ "database": "ok", "version": "13.1.3", "commit": "45a27d64b64" }

2. `X-Frame-Options: SAMEORIGIN` is NOT sent by the real Grafana

The SAMEORIGIN header came from the Forgejo box. grafana.jiggl.in sends no X-Frame-Options and no Content-Security-Policy: frame-ancestors:

HTTP/2 200
cache-control: no-store
content-type: text/html; charset=UTF-8
via: 1.1 Caddy
x-content-type-options: nosniff
x-xss-protection: 1; mode=block

The reverse-proxy design is still correct, but for the authentication reason rather than the framing reason: a cross-origin iframe to grafana.jiggl.in has no Grafana session cookie and would redirect to /login. The proxy exists to inject the service-account credential server-side.

3. Bearer auth on the dashboard page route — measured, it works

This was the open question. It answers yes:

Request	Result
`GET /d/{uid}/x` with `Authorization: Bearer <sa-token>`	200, full SPA HTML
`GET /d/{uid}/x` without auth	302 → `/login?redirectTo=...`
`GET /d/{uid}/x?kiosk` with Bearer	200
`GET /api/dashboards/uid/{uid}` with Bearer	200 (as expected)
`GET /api/search?type=dash-db` with Bearer	200, 5 dashboards (4 tagged `kreeader`)

So service-account tokens authenticate Grafana's HTML page routes, not just /api/*. No anonymous-viewer or auth-proxy toggle is needed.

Dashboards visible to this token: kreeader-overview, kreeader-detection, kreeader-detection-quality, kreeader-metadata (all tagged kreeader), plus an untagged custom.

4. Subpath embedding works with response rewriting alone — no Grafana-side config change

I checked what actually breaks a subpath mount. Grafana 13's app shell references every asset relatively (public/build/...); the sole absolute reference in the document is the <base> tag:

<base href="/" />
...
"appSubUrl":""

So two textual rewrites of the HTML response are sufficient:

<base href="/" /> → <base href="/api/admin/grafana/" />
"appSubUrl":"" → "appSubUrl":"/api/admin/grafana"

I built a throwaway prototype proxy doing exactly that and loaded it in a real headless browser. The full SPA boots, the router rebases itself, and every panel renders with live Prometheus data — the URL self-corrected to /api/admin/grafana/d/kreeader-overview/kreeader-e28094-overview?kiosk=true&orgId=1&from=now-30d&to=now&..., which only happens after the dashboard JSON loads and the router initialises against the right basename. Zero asset 404s, zero API errors.

Conclusion: full-embed ships as the default mode. grafana.root_url / serve_from_sub_path do NOT need changing, so there is no Grafana config ask to file — the questions/ file records the two operator decisions that remain (base URL to configure, and the renderer below) instead.

The only console errors were the Grafana Live websocket:

WebSocket connection to 'ws://.../api/admin/grafana/api/live/ws' failed:
Error during WebSocket handshake: Unexpected response code: 400

— an artefact of the Python prototype not upgrading. Go's httputil.ReverseProxy handles the upgrade natively, so the real implementation passes it through.

5. The image-renderer fallback is not available

GET /api/frontend/settings  →  rendererAvailable = false
GET /render/d-solo/...      →  500

grafana-image-renderer is not installed, so a PNG-based static fallback is impossible. The fallback mode is therefore the progress-report pattern — reproduce panels from /api/dashboards/uid/{uid} JSON — and it ships behind the config flag, not as default, since full-embed is proven working.

Implementation proceeding on feat/admin-grafana, all new code in api/admingrafana.go.

## Live probe findings (read-only GETs) — two premises in the issue body corrected ### 1. The Grafana host in the issue body is wrong `http://192.168.3.9:3000` is **Forgejo**, not Grafana — `GET /api/health` there returns `Not found.` and `GET /` returns the Forgejo app shell (`<title>Forgejo: Beyond coding. We Forge.</title>`). The real instance, per the already-working `kreeader-docs/scripts/progress-report/grafana.py` (`DEFAULT_GRAFANA_URL`), is **`https://grafana.jiggl.in`** (resolves to 135.131.74.147, fronted by Caddy): ``` GET https://grafana.jiggl.in/api/health { "database": "ok", "version": "13.1.3", "commit": "45a27d64b64" } ``` ### 2. `X-Frame-Options: SAMEORIGIN` is NOT sent by the real Grafana The SAMEORIGIN header came from the Forgejo box. `grafana.jiggl.in` sends **no** `X-Frame-Options` and **no** `Content-Security-Policy: frame-ancestors`: ``` HTTP/2 200 cache-control: no-store content-type: text/html; charset=UTF-8 via: 1.1 Caddy x-content-type-options: nosniff x-xss-protection: 1; mode=block ``` The reverse-proxy design is still correct, but for the **authentication** reason rather than the framing reason: a cross-origin iframe to `grafana.jiggl.in` has no Grafana session cookie and would redirect to `/login`. The proxy exists to inject the service-account credential server-side. ### 3. Bearer auth on the dashboard *page* route — measured, it works This was the open question. It answers yes: | Request | Result | |---|---| | `GET /d/{uid}/x` **with** `Authorization: Bearer <sa-token>` | **200**, full SPA HTML | | `GET /d/{uid}/x` **without** auth | **302 → `/login?redirectTo=...`** | | `GET /d/{uid}/x?kiosk` with Bearer | **200** | | `GET /api/dashboards/uid/{uid}` with Bearer | 200 (as expected) | | `GET /api/search?type=dash-db` with Bearer | 200, 5 dashboards (4 tagged `kreeader`) | So service-account tokens authenticate Grafana's HTML page routes, not just `/api/*`. No anonymous-viewer or auth-proxy toggle is needed. Dashboards visible to this token: `kreeader-overview`, `kreeader-detection`, `kreeader-detection-quality`, `kreeader-metadata` (all tagged `kreeader`), plus an untagged `custom`. ### 4. Subpath embedding works with response rewriting alone — no Grafana-side config change I checked what actually breaks a subpath mount. Grafana 13's app shell references **every asset relatively** (`public/build/...`); the sole absolute reference in the document is the `<base>` tag: ``` <base href="/" /> ... "appSubUrl":"" ``` So two textual rewrites of the HTML response are sufficient: - `<base href="/" />` → `<base href="/api/admin/grafana/" />` - `"appSubUrl":""` → `"appSubUrl":"/api/admin/grafana"` I built a throwaway prototype proxy doing exactly that and loaded it in a real headless browser. **The full SPA boots, the router rebases itself, and every panel renders with live Prometheus data** — the URL self-corrected to `/api/admin/grafana/d/kreeader-overview/kreeader-e28094-overview?kiosk=true&orgId=1&from=now-30d&to=now&...`, which only happens after the dashboard JSON loads and the router initialises against the right basename. Zero asset 404s, zero API errors. **Conclusion: full-embed ships as the default mode.** `grafana.root_url` / `serve_from_sub_path` do NOT need changing, so there is no Grafana config ask to file — the `questions/` file records the two operator decisions that remain (base URL to configure, and the renderer below) instead. The only console errors were the Grafana Live websocket: ``` WebSocket connection to 'ws://.../api/admin/grafana/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 ``` — an artefact of the Python prototype not upgrading. Go's `httputil.ReverseProxy` handles the upgrade natively, so the real implementation passes it through. ### 5. The image-renderer fallback is not available ``` GET /api/frontend/settings → rendererAvailable = false GET /render/d-solo/... → 500 ``` `grafana-image-renderer` is not installed, so a PNG-based static fallback is impossible. The fallback mode is therefore the `progress-report` pattern — reproduce panels from `/api/dashboards/uid/{uid}` JSON — and it ships **behind the config flag**, not as default, since full-embed is proven working. Implementation proceeding on `feat/admin-grafana`, all new code in `api/admingrafana.go`.

andrew referenced this issue from a commit

2026-08-29 18:58:32 -05:00

Admin: admin-gated Grafana reverse proxy for embedded dashboards (#152)

andrew referenced this issue from a commit

2026-08-29 20:23:00 -05:00

Admin: document [admin.grafana] in config.toml.example (#152)

andrew referenced this issue from a commit

2026-08-29 20:23:00 -05:00

Admin: document the Grafana JSON endpoints in openapi.yaml (#152)

andrew referenced this issue from a commit

2026-08-29 20:23:00 -05:00

Admin: frame tickets so the Grafana iframe can actually authenticate (#152)

andrew referenced this issue from a commit

2026-08-29 20:31:08 -05:00

Point kreeader-ui at the Dashboards tab (#152)

andrew commented

2026-08-29 20:31:41 -05:00

Owner

Implemented — full embed ships as the default. Branch `feat/admin-grafana` @ `5b45241`

(Work paused mid-task when the machine rebooted unexpectedly; the worktree survived intact and everything below was completed after resuming. The probe comment above was posted before the crash.)

Commits


`8a56e84`	admin-gated reverse proxy
`4fafbec`	`[admin.grafana]` in config.toml.example
`55f5445`	openapi.yaml for the JSON endpoints
`bc89d92`	frame tickets so the iframe can authenticate
`5b45241`	kreeader-ui pointer bump

Plus kreeader-ui feat/admin-grafana-dashboards @ e8137e2 (the Admin → Dashboards tab).

The one design problem the probe findings did not anticipate

The proxy landed gated on the ordinary admin check — which an iframe cannot pass. A frame sends no Authorization header, and in the two auth modes that check one (local_multiusers, oidc) isAdmin reads a bearer token. The embed would have 401'd for an admin who was perfectly well logged in. Only single_user, which authorizes any private-network caller regardless of headers, would have worked by accident.

The fix follows api/ssetickets.go, which exists in this package for exactly the same reason (EventSource cannot set headers either), and borrows its key property intact: the ticket is bound to the caller's own bearer token and redeems back into it rather than being an authorization of its own. Redeeming re-injects the header the frame could not send, and then the normal admin gate decides, every request. So a ticket confers nothing by existing — demote the minting admin and the next frame request fails. That is asserted directly by flipping the role off mid-test.

It differs from an SSE ticket in lifetime only, because the workloads differ: an EventSource redeems once, an iframe makes hundreds of requests. So this one is multi-use and hours long, and becomes an HttpOnly cookie scoped to /api/admin/grafana/ on first use — the ticket appears in a URL exactly once and is stripped before anything reaches Grafana. Stored hashed, in memory only.

Shipped mode: `embed` (default). Verified against live Grafana, not a prototype

The screenshot evidence in the probe comment was from a throwaway Python proxy. I re-ran it against the shipped Go code serving https://grafana.jiggl.in:

GET /api/admin/grafana/d/kreeader-overview/x?kiosk → 200, full dashboard rendering live Prometheus data in a headless browser (6953 comics, 196540 pages, 89.2% match rate, timeseries + piechart + gauges all drawing).
/api/admin/grafana/api/live/ws → 101 Switching Protocols. Go's httputil.ReverseProxy handles the upgrade natively, so the one console error the Python prototype produced is gone — Grafana Live works.

panels mode ships behind the config flag as designed: it reproduces a dashboard from its own JSON (the kreeader-docs progress-report approach). No image-renderer path, because rendererAvailable=false and /render/d-solo 500s on this instance — a PNG fallback would be one that does not work.

Endpoints (all admin-gated, `kreeader_admin` only, all registered only when `[admin.grafana] enabled`)

/api/admin/grafana/*            reverse proxy (embed mode)
/api/admin/grafana-dashboards   GET/PUT the curated list (AppSettings);
                                GET also returns what the credential can
                                discover, so the tab offers a picker
/api/admin/grafana-panels       the static fallback
/api/admin/grafana-ticket       mint a frame ticket

Credential handling

Never logged, never returned by any endpoint, never stored in the DB. Read from token_file/token_env at request time and cached against mtime, so rotation needs no restart. readGrafanaToken accepts both layouts and the token:-line lesson is now encoded in the parser and its test — a whole-file token is rejected rather than sent upstream to earn a 401.

At the proxy boundary the client's Authorization and every non-Grafana cookie are stripped before the request goes upstream (a browser's kreeader session must never reach Grafana, and a client-chosen Authorization must never override the injected one). Grafana's own cookies are forwarded by strict name allowlist and returned rescoped to the proxy subtree. Upstream X-Frame-Options/CSP are dropped; root-relative redirects re-anchored. Only text/html is rewritten, and only such paths force identity encoding — /public assets stream through compressed.

UI

New DashboardsAdminSettings.svelte + one TABS entry. This is the first tab in Admin.svelte that is not unconditional, so TABS gained a capability field and the list renders from a filtered derivation; every existing tab omits it and is unaffected. admin_grafana defaults false in CAPABILITIES_FALLBACK (unlike its neighbours — an older server reporting no such key genuinely has no such surface), so the tab is hidden rather than shown-and-404ing.

Gates — all green, both flavours

go build ./...                         OK      go build -tags kreeader_admin ./...       OK
go vet ./...                           OK      go vet -tags kreeader_admin ./...         OK
go test ./api/ ./config/ .             OK      go test -tags kreeader_admin ...          OK
kreeader-ui: npm run build             OK      npm test — 176 pass, 0 fail

Tests cover the gate on every route (anonymous 401 / non-admin 403, and a refused PUT writing nothing), credential injection and client-credential stripping against a fake upstream, both token layouts, the HTML rebase end-to-end, assets left alone, redirect/cookie rescoping, settings round-trip, degradation when Grafana is unreachable, the panels fallback, non-HTTP upstream and unknown-mode rejection, and the full ticket lifecycle including forgery, expiry, hashed-at-rest storage and revocation-by-demotion.

Open for Andrew — `questions/grafana-embed.md`

No Grafana config ask (none is needed). What is there instead:

Which Grafana URL and where the token lives on butcher — I probed with keys/grafana_service_account, but the server must not read from a home directory, and a separate service account from the docs reporter would be cleaner.
single_user gives every LAN device an authenticated Grafana, since isAdmin is true for any private-network caller there. Not new — but this is the first admin surface proxying a third-party system's credential, so it deserves a deliberate decision. The config example carries the warning.
grafana-image-renderer — only worth installing if panels mode is ever wanted.
Pointer drift: kreeader-ui's branch must merge first, then both kreeader-server and kreeader-client move to the resulting master commit in the same pass, or kreeader-client's pointer-match CI check fails.

Not merged, not closed, nothing deployed. All hound access was read-only GETs.

## Implemented — full embed ships as the default. Branch `feat/admin-grafana` @ `5b45241` (Work paused mid-task when the machine rebooted unexpectedly; the worktree survived intact and everything below was completed after resuming. The probe comment above was posted before the crash.) ### Commits | | | |---|---| | `8a56e84` | admin-gated reverse proxy | | `4fafbec` | `[admin.grafana]` in config.toml.example | | `55f5445` | openapi.yaml for the JSON endpoints | | `bc89d92` | frame tickets so the iframe can authenticate | | `5b45241` | kreeader-ui pointer bump | Plus kreeader-ui `feat/admin-grafana-dashboards` @ `e8137e2` (the Admin → Dashboards tab). ### The one design problem the probe findings did not anticipate The proxy landed gated on the ordinary admin check — **which an iframe cannot pass.** A frame sends no `Authorization` header, and in the two auth modes that check one (`local_multiusers`, `oidc`) `isAdmin` reads a bearer token. The embed would have 401'd for an admin who was perfectly well logged in. Only `single_user`, which authorizes any private-network caller regardless of headers, would have worked by accident. The fix follows `api/ssetickets.go`, which exists in this package for exactly the same reason (EventSource cannot set headers either), and borrows its key property intact: **the ticket is bound to the caller's own bearer token and redeems back into it** rather than being an authorization of its own. Redeeming re-injects the header the frame could not send, and then the normal admin gate decides, every request. So a ticket confers nothing by existing — demote the minting admin and the next frame request fails. That is asserted directly by flipping the role off mid-test. It differs from an SSE ticket in lifetime only, because the workloads differ: an EventSource redeems once, an iframe makes hundreds of requests. So this one is multi-use and hours long, and becomes an HttpOnly cookie scoped to `/api/admin/grafana/` on first use — the ticket appears in a URL exactly once and is stripped before anything reaches Grafana. Stored hashed, in memory only. ### Shipped mode: `embed` (default). Verified against live Grafana, not a prototype The screenshot evidence in the probe comment was from a throwaway Python proxy. I re-ran it against **the shipped Go code** serving `https://grafana.jiggl.in`: - `GET /api/admin/grafana/d/kreeader-overview/x?kiosk` → **200**, full dashboard rendering live Prometheus data in a headless browser (6953 comics, 196540 pages, 89.2% match rate, timeseries + piechart + gauges all drawing). - `/api/admin/grafana/api/live/ws` → **`101 Switching Protocols`**. Go's `httputil.ReverseProxy` handles the upgrade natively, so the one console error the Python prototype produced is gone — Grafana Live works. `panels` mode ships behind the config flag as designed: it reproduces a dashboard from its own JSON (the kreeader-docs progress-report approach). No image-renderer path, because `rendererAvailable=false` and `/render/d-solo` 500s on this instance — a PNG fallback would be one that does not work. ### Endpoints (all admin-gated, `kreeader_admin` only, all registered only when `[admin.grafana] enabled`) ``` /api/admin/grafana/* reverse proxy (embed mode) /api/admin/grafana-dashboards GET/PUT the curated list (AppSettings); GET also returns what the credential can discover, so the tab offers a picker /api/admin/grafana-panels the static fallback /api/admin/grafana-ticket mint a frame ticket ``` ### Credential handling Never logged, never returned by any endpoint, never stored in the DB. Read from `token_file`/`token_env` at request time and cached against mtime, so rotation needs no restart. `readGrafanaToken` accepts both layouts and **the `token:`-line lesson is now encoded in the parser and its test** — a whole-file token is rejected rather than sent upstream to earn a 401. At the proxy boundary the client's `Authorization` and every non-Grafana cookie are stripped before the request goes upstream (a browser's kreeader session must never reach Grafana, and a client-chosen `Authorization` must never override the injected one). Grafana's own cookies are forwarded by strict name allowlist and returned rescoped to the proxy subtree. Upstream `X-Frame-Options`/CSP are dropped; root-relative redirects re-anchored. Only `text/html` is rewritten, and only such paths force identity encoding — `/public` assets stream through compressed. ### UI New `DashboardsAdminSettings.svelte` + one TABS entry. This is the first tab in `Admin.svelte` that is not unconditional, so `TABS` gained a `capability` field and the list renders from a filtered derivation; every existing tab omits it and is unaffected. `admin_grafana` defaults **false** in `CAPABILITIES_FALLBACK` (unlike its neighbours — an older server reporting no such key genuinely has no such surface), so the tab is hidden rather than shown-and-404ing. ### Gates — all green, both flavours ``` go build ./... OK go build -tags kreeader_admin ./... OK go vet ./... OK go vet -tags kreeader_admin ./... OK go test ./api/ ./config/ . OK go test -tags kreeader_admin ... OK kreeader-ui: npm run build OK npm test — 176 pass, 0 fail ``` Tests cover the gate on every route (anonymous 401 / non-admin 403, and a refused PUT writing nothing), credential injection and client-credential stripping against a fake upstream, both token layouts, the HTML rebase end-to-end, assets left alone, redirect/cookie rescoping, settings round-trip, degradation when Grafana is unreachable, the panels fallback, non-HTTP upstream and unknown-mode rejection, and the full ticket lifecycle including forgery, expiry, hashed-at-rest storage and revocation-by-demotion. ### Open for Andrew — `questions/grafana-embed.md` No Grafana config ask (none is needed). What is there instead: 1. **Which Grafana URL and where the token lives on butcher** — I probed with `keys/grafana_service_account`, but the server must not read from a home directory, and a separate service account from the docs reporter would be cleaner. 2. **`single_user` gives every LAN device an authenticated Grafana**, since `isAdmin` is true for any private-network caller there. Not new — but this is the first admin surface proxying a *third-party system's* credential, so it deserves a deliberate decision. The config example carries the warning. 3. **`grafana-image-renderer`** — only worth installing if `panels` mode is ever wanted. 4. **Pointer drift**: kreeader-ui's branch must merge first, then **both** kreeader-server and kreeader-client move to the resulting master commit in the same pass, or kreeader-client's pointer-match CI check fails. Not merged, not closed, nothing deployed. All hound access was read-only GETs.

kreeader-user commented

2026-08-29 20:35:13 -05:00

Author

Owner

Merged: kreeader-ui c9f9c45 (Dashboards tab) and kreeader-server feat/admin-grafana; submodule pointers moved in server and client in the same pass. Ships OFF until [grafana] is configured; the single_user-mode exposure in questions/grafana-embed.md needs Andrew before enabling on butcher. Closing on deploy.

andrew referenced this issue from a commit

2026-08-29 20:35:14 -05:00

Merge feat/admin-grafana: embedded Grafana dashboards for the admin console (#152)

andrew referenced this issue from a commit

2026-08-29 20:37:05 -05:00

instrumentation: admin-gated pprof, memstats and gocv Mat profile (#152)

andrew referenced this issue from a commit

2026-08-29 20:37:05 -05:00

panetrain: bound the corrected-page set instead of holding it all decoded (#152)

andrew referenced this issue from a commit

2026-08-29 20:39:43 -05:00

docs: how to profile a running server, and the matprofile build (#154)

kreeader-user commented

2026-08-29 22:31:26 -05:00

Author

Owner

Deployed in 30.76.1-1, ships OFF (no [grafana] config). Enabling on butcher waits for Andrew's decision on the single_user-mode exposure (questions/grafana-embed.md); Sol finding 12 (ticket in nested URL) tracked in #156. Closing the build; enablement is config.

kreeader-user closed this issue

2026-08-29 22:31:26 -05:00

Rows
Columns

Admin: embedded Grafana dashboards tab (same-origin reverse proxy) #152

Live probe findings (read-only GETs) — two premises in the issue body corrected

1. The Grafana host in the issue body is wrong

2. X-Frame-Options: SAMEORIGIN is NOT sent by the real Grafana

3. Bearer auth on the dashboard page route — measured, it works

4. Subpath embedding works with response rewriting alone — no Grafana-side config change

5. The image-renderer fallback is not available

Implemented — full embed ships as the default. Branch feat/admin-grafana @ 5b45241

Commits

The one design problem the probe findings did not anticipate

Shipped mode: embed (default). Verified against live Grafana, not a prototype

Endpoints (all admin-gated, kreeader_admin only, all registered only when [admin.grafana] enabled)

Credential handling

UI

Gates — all green, both flavours

Open for Andrew — questions/grafana-embed.md

2. `X-Frame-Options: SAMEORIGIN` is NOT sent by the real Grafana

Implemented — full embed ships as the default. Branch `feat/admin-grafana` @ `5b45241`

Shipped mode: `embed` (default). Verified against live Grafana, not a prototype

Endpoints (all admin-gated, `kreeader_admin` only, all registered only when `[admin.grafana] enabled`)

Open for Andrew — `questions/grafana-embed.md`