kreeader/kreeader-server

Watch

Fork 1

Register the progress-report generator as a #123 task-registry entry #136

New issue

Open

opened 2026-08-27 14:30:12 -05:00 by kreeader-user · 3 comments

kreeader-user commented

2026-08-27 14:30:12 -05:00

Owner

Follow-up to the kreeader-docs progress-report CI job (v1 runs in Forgejo CI per Andrew's spec: additive workflow, read-only everywhere, CI secrets, generated-label output, weekly windows, MEASURED/ATTRIBUTED/FORECAST tiers, deterministic generator — no LLM). v2: register the generator in the Server Health task registry so it gains Run-now and task_runs history alongside the other scheduled tasks. Requires the generator to be invocable server-side (or via a task-registry exec seam) without weakening the read-only guarantees — the task entry must run with the same read-only role, not server credentials.

kreeader-user commented

2026-08-27 15:33:15 -05:00

Author

Owner

The generator this issue refers to has landed in kreeader-docs master (commits 7b14663 + 3dc3bbf) — noting it here rather than filing a duplicate issue.

What exists now:

.forgejo/workflows/progress-report.yml — new, isolated, workflow_dispatch only (no cron until a sample is approved). Nothing in build-and-release.yml or scripts/build.sh was touched.
scripts/progress-report/ — deterministic generator (psycopg + requests, pinned), the versioned rule set (rules.py, PR-001..PR-007), the digest seam (suggestions.py), a versioned prompt, and unit tests for window arithmetic / tier labelling / rule firing.
scripts/render-progress-report.sh — reuses docs/_shared/preamble.tex without entangling build.sh.
A live sample committed at docs/_generated/progress-report.md, with its data digest at docs/_generated/digest.json.

Access profile, for the registry entry: Forgejo REST GET only; Prometheus query_range + /api/v1/metadata at http://192.168.3.9:9099 (LAN-open, a plain variable, not a secret); Postgres SELECT only as the kreeader-docs role in a READ ONLY session. No writes, no deploys, no restarts. Secrets REPORT_FORGEJO_TOKEN and REPORT_PG_DSN reach the job through the environment only.

Every number is produced by deterministic code. The one place a language model is allowed is the labelled advisory subsection 4b, which is assembled from the report's own digest through an input-file seam and is non-blocking — absent or broken input degrades to a sentence and the report still succeeds.

The generator this issue refers to has landed in `kreeader-docs` master (commits `7b14663` + `3dc3bbf`) — noting it here rather than filing a duplicate issue. What exists now: - `.forgejo/workflows/progress-report.yml` — new, isolated, **`workflow_dispatch` only** (no cron until a sample is approved). Nothing in `build-and-release.yml` or `scripts/build.sh` was touched. - `scripts/progress-report/` — deterministic generator (`psycopg` + `requests`, pinned), the versioned rule set (`rules.py`, `PR-001`..`PR-007`), the digest seam (`suggestions.py`), a versioned prompt, and unit tests for window arithmetic / tier labelling / rule firing. - `scripts/render-progress-report.sh` — reuses `docs/_shared/preamble.tex` without entangling `build.sh`. - A **live sample** committed at `docs/_generated/progress-report.md`, with its data digest at `docs/_generated/digest.json`. Access profile, for the registry entry: Forgejo REST **GET only**; Prometheus `query_range` + `/api/v1/metadata` at `http://192.168.3.9:9099` (LAN-open, a plain variable, not a secret); Postgres **SELECT only** as the `kreeader-docs` role in a `READ ONLY` session. No writes, no deploys, no restarts. Secrets `REPORT_FORGEJO_TOKEN` and `REPORT_PG_DSN` reach the job through the environment only. Every number is produced by deterministic code. The one place a language model is allowed is the labelled advisory subsection 4b, which is assembled from the report's own digest through an input-file seam and is non-blocking — absent or broken input degrades to a sentence and the report still succeeds.

andrew referenced this issue from a commit

2026-08-29 21:10:57 -05:00

Server Health: an external-task seam, and the progress-report generator in the registry (#136)

kreeader-user commented

2026-08-29 21:11:57 -05:00

Author

Owner

Done on branch fix/openapi-tags-and-report-task (commit 4b9641e). Not merged, not closed. kreeader-docs is untouched — the docs-side wiring is specified below for the orchestrator to add there.

Shape: a seam, not a runner. #123's registry is explicitly a view over schedulers that already run in the server. The progress-report generator does not, and this issue's constraint is that registering it must not move it or weaken its read-only guarantees. So the server gains no ability to execute it; what it gains is a memory of what was reported and the ability to notice when nothing was.

New registry entry kind (api/healthtasks.go)

External — runs outside this process. Run-now answers 501 naming where the work happens (new ErrTaskExternal, distinct from ErrTaskNotRunnable: "runs elsewhere" and "runs on its own trigger" are different facts). A button that could only ever fail is worse than none.
CadenceSeconds — the declared expectation. Without it, "stopped" and "between runs" are the same observation, and silence stays undetectable.
GraceSeconds — how late is late. A weekly CI job slips by hours for ordinary reasons; raising at the first minute past due would train the operator to ignore the panel, which is the failure #123 exists to prevent.
Declared: progress-report — kreeader-docs CI, cadence 7 days, grace 36 hours.

The external-report contract

POST /api/admin/tasks/external/{name}/report
Authorization: Bearer <server admin token>
{ "started_at": RFC3339|unix, "finished_at": RFC3339|unix, "result": "ok"|"error"|"skipped",
  "url": "https://…", "summary": "one line" }

Admin-gated on the same seam as the rest of /api/admin/health/*.
{name} must be a declared external task. An unknown name — or an in-process task — is 404, never an auto-created entry: a POST must not be able to invent a scheduled task nobody scheduled, and a typo in a workflow has to fail loudly instead of accumulating history under a name no one reads.
Required: started_at, result. finished_at defaults to now and must not precede started_at. url must be absolute http(s). summary truncated at 500 chars.
The row is stored with the runner's own timestamps (new db.RecordTaskRun writes one already-finished row; StartTaskRun/FinishTaskRun remain for work this process watches). A run that took nine minutes elsewhere did not take zero seconds here.
skipped survives as itself — a generator that found nothing new ran correctly but published nothing, and flattening that to ok would lie about it.
Nothing is coerced: every rejection is a 400 with its reason. A weekly job gets one chance per run to be right.
Responses: 202 with the task view, 400 invalid_payload, 401/403, 404 not_found.

Raise-or-bump wiring

external_task_missed (warn, scope = task id) is raised by a derived emitter registered through health.RegisterRefresher, exactly like the Metron latch and the correction stall — "has not reported" is a state, so nothing can raise it at a call site and opening the Issues tab reconciles it.
- With history: missed at last_run + cadence + grace.
- Never reported: missed once the process has been up longer than cadence + grace. There is no other honest clock without a run to count from. Stated trade-off: a server restarted more often than the cadence will not raise the never-reported case, so an unwired workflow is caught by the first report that never comes rather than by the first restart. This is in the code comment and the doc, not hidden.
- Any report resolves it — a report is proof of life whatever the result.
external_task_failed (warn) covers a reported failure: it happened, so it is not "missed", but nobody should have to read the run history to notice. A later non-error run resolves it.

Also: TaskRun gains a URL column (additive, AutoMigrate) so a run that published something links to it; the task view gains external + expected_every_seconds so the UI renders "runs in kreeader-docs CI" rather than a dead button. openapi.yaml mirrors the endpoint (tagged Health & Ops per #137) plus the ExternalTaskReport schema and the new HealthTask/HealthTaskRun fields.

The exact call the kreeader-docs workflow must add

Full write-up in docs/external-tasks.md on the branch. First step of the progress-report job:

- name: Stamp the run's start
  run: echo "REPORT_STARTED_AT=$(date -u +%FT%TZ)" >> "$GITHUB_ENV"

Final step, if: always() so a failed generation still reports (a failure that reports is visible; a failure that stays quiet is indistinguishable from a run that never started):

- name: Report the run to the Kreeader server
  if: always()
  env:
    KREEADER_URL: ${{ secrets.KREEADER_URL }}
    KREEADER_ADMIN_TOKEN: ${{ secrets.KREEADER_ADMIN_TOKEN }}
    STARTED_AT: ${{ env.REPORT_STARTED_AT }}
    RELEASE_URL: ${{ env.REPORT_RELEASE_URL }}
    RESULT: ${{ job.status == 'success' && 'ok' || 'error' }}
  run: |
    curl -fsS -X POST \
      "$KREEADER_URL/api/admin/tasks/external/progress-report/report" \
      -H "Authorization: Bearer $KREEADER_ADMIN_TOKEN" \
      -H 'Content-Type: application/json' \
      -d "$(jq -nc \
            --arg started "$STARTED_AT" \
            --arg finished "$(date -u +%FT%TZ)" \
            --arg result "$RESULT" \
            --arg url "$RELEASE_URL" \
            '{started_at:$started, finished_at:$finished, result:$result}
             + (if $url == "" then {} else {url:$url} end)')"

Notes for whoever wires it:

KREEADER_ADMIN_TOKEN is a server admin credential and belongs in CI secrets. It is not one of the generator's read-only inputs and must not be used for anything the generator reads — appending one row to its own run history is the whole of its authority here.
Use result: skipped where the workflow deliberately publishes nothing (no changes in the window), not ok.
Send url only when a release was actually published; omit it rather than sending an empty or relative value (a relative url is a 400).

Tests — api/healthtasksexternal_test.go pins the four properties that would otherwise put the generator back where this issue found it: the runner's timestamps survive into the history; malformed reports and unknown names are refused without writing anything; Run-now is 501 and opens no run row; silence raises exactly one issue that bumps rather than duplicates and is resolved by the next report.

Gates: GOWORK=off CGO_LDFLAGS=-lprotobuf go build ./... clean, go vet ./... clean, go test ./api/ -count=1 ok (18.7s).

Done on branch `fix/openapi-tags-and-report-task` (commit `4b9641e`). Not merged, not closed. **kreeader-docs is untouched** — the docs-side wiring is specified below for the orchestrator to add there. **Shape: a seam, not a runner.** #123's registry is explicitly a view over schedulers that already run *in* the server. The progress-report generator does not, and this issue's constraint is that registering it must not move it or weaken its read-only guarantees. So the server gains no ability to execute it; what it gains is a memory of what was reported and the ability to notice when nothing was. **New registry entry kind** (`api/healthtasks.go`) - `External` — runs outside this process. Run-now answers **501** naming where the work happens (new `ErrTaskExternal`, distinct from `ErrTaskNotRunnable`: "runs elsewhere" and "runs on its own trigger" are different facts). A button that could only ever fail is worse than none. - `CadenceSeconds` — the declared expectation. Without it, "stopped" and "between runs" are the same observation, and silence stays undetectable. - `GraceSeconds` — how late is late. A weekly CI job slips by hours for ordinary reasons; raising at the first minute past due would train the operator to ignore the panel, which is the failure #123 exists to prevent. - Declared: **`progress-report`** — kreeader-docs CI, cadence **7 days**, grace **36 hours**. **The external-report contract** ``` POST /api/admin/tasks/external/{name}/report Authorization: Bearer <server admin token> { "started_at": RFC3339|unix, "finished_at": RFC3339|unix, "result": "ok"|"error"|"skipped", "url": "https://…", "summary": "one line" } ``` - Admin-gated on the same seam as the rest of `/api/admin/health/*`. - `{name}` must be a **declared external** task. An unknown name — or an in-process task — is **404**, never an auto-created entry: a POST must not be able to invent a scheduled task nobody scheduled, and a typo in a workflow has to fail loudly instead of accumulating history under a name no one reads. - Required: `started_at`, `result`. `finished_at` defaults to now and must not precede `started_at`. `url` must be absolute http(s). `summary` truncated at 500 chars. - The row is stored with **the runner's own timestamps** (new `db.RecordTaskRun` writes one already-finished row; `StartTaskRun`/`FinishTaskRun` remain for work this process watches). A run that took nine minutes elsewhere did not take zero seconds here. - `skipped` survives as itself — a generator that found nothing new ran correctly but published nothing, and flattening that to `ok` would lie about it. - Nothing is coerced: every rejection is a `400` with its reason. A weekly job gets one chance per run to be right. - Responses: `202` with the task view, `400 invalid_payload`, `401`/`403`, `404 not_found`. **Raise-or-bump wiring** - `external_task_missed` (warn, scope = task id) is raised by a derived emitter registered through `health.RegisterRefresher`, exactly like the Metron latch and the correction stall — "has not reported" is a *state*, so nothing can raise it at a call site and opening the Issues tab reconciles it. - With history: missed at `last_run + cadence + grace`. - Never reported: missed once the process has been up longer than `cadence + grace`. There is no other honest clock without a run to count from. **Stated trade-off:** a server restarted more often than the cadence will not raise the never-reported case, so an unwired workflow is caught by the first report that never comes rather than by the first restart. This is in the code comment and the doc, not hidden. - Any report resolves it — a report is proof of life whatever the result. - `external_task_failed` (warn) covers a reported failure: it happened, so it is not "missed", but nobody should have to read the run history to notice. A later non-error run resolves it. **Also:** `TaskRun` gains a `URL` column (additive, AutoMigrate) so a run that published something links to it; the task view gains `external` + `expected_every_seconds` so the UI renders "runs in kreeader-docs CI" rather than a dead button. `openapi.yaml` mirrors the endpoint (tagged `Health & Ops` per #137) plus the `ExternalTaskReport` schema and the new `HealthTask`/`HealthTaskRun` fields. --- ### The exact call the kreeader-docs workflow must add Full write-up in `docs/external-tasks.md` on the branch. First step of the progress-report job: ```yaml - name: Stamp the run's start run: echo "REPORT_STARTED_AT=$(date -u +%FT%TZ)" >> "$GITHUB_ENV" ``` Final step, `if: always()` so a failed generation still reports (a failure that reports is visible; a failure that stays quiet is indistinguishable from a run that never started): ```yaml - name: Report the run to the Kreeader server if: always() env: KREEADER_URL: ${{ secrets.KREEADER_URL }} KREEADER_ADMIN_TOKEN: ${{ secrets.KREEADER_ADMIN_TOKEN }} STARTED_AT: ${{ env.REPORT_STARTED_AT }} RELEASE_URL: ${{ env.REPORT_RELEASE_URL }} RESULT: ${{ job.status == 'success' && 'ok' || 'error' }} run: | curl -fsS -X POST \ "$KREEADER_URL/api/admin/tasks/external/progress-report/report" \ -H "Authorization: Bearer $KREEADER_ADMIN_TOKEN" \ -H 'Content-Type: application/json' \ -d "$(jq -nc \ --arg started "$STARTED_AT" \ --arg finished "$(date -u +%FT%TZ)" \ --arg result "$RESULT" \ --arg url "$RELEASE_URL" \ '{started_at:$started, finished_at:$finished, result:$result} + (if $url == "" then {} else {url:$url} end)')" ``` Notes for whoever wires it: - `KREEADER_ADMIN_TOKEN` is a **server** admin credential and belongs in CI secrets. It is not one of the generator's read-only inputs and must not be used for anything the generator reads — appending one row to its own run history is the whole of its authority here. - Use `result: skipped` where the workflow deliberately publishes nothing (no changes in the window), not `ok`. - Send `url` only when a release was actually published; omit it rather than sending an empty or relative value (a relative `url` is a 400). **Tests** — `api/healthtasksexternal_test.go` pins the four properties that would otherwise put the generator back where this issue found it: the runner's timestamps survive into the history; malformed reports and unknown names are refused without writing anything; Run-now is 501 and opens no run row; silence raises exactly one issue that bumps rather than duplicates and is resolved by the next report. Gates: `GOWORK=off CGO_LDFLAGS=-lprotobuf go build ./...` clean, `go vet ./...` clean, `go test ./api/ -count=1` ok (18.7s).

kreeader-user referenced this issue

2026-08-29 21:14:07 -05:00

openapi.yaml: tag operations by area so the generated API reference groups honestly #137

kreeader-user commented

2026-08-29 21:14:08 -05:00

Author

Owner

Merged to master (the tag test immediately caught five operations from branches merged after this one forked — tagged at merge). #137 closes when kreeader-docs retires its workaround on the next reference render; #136 closes once the docs workflow posts its first report (needs an admin credential in kreeader-docs CI secrets — queued for Andrew).

andrew referenced this issue from a commit

2026-08-29 21:14:09 -05:00

Merge fix/openapi-tags-and-report-task: openapi tagged by behaviour, external tasks in the registry (#137, #136)

kreeader-user added the

needs-andrew

label

2026-09-04 12:32:08 -05:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

kreeader/kreeader-server#136

No description provided.

Rows
Columns