Accelerator autodetection: use whatever compute a node has (CUDA/VNNI/iGPU/NPU/TPU)

kreeader-user commented

2026-08-26 19:44:56 -05:00

Owner

Design: roadmap/accelerator-autodetection.md. Probe at startup (asking runtimes, not inferring from hardware), stored capability manifests reported with agent registration, capability+measurement-based work routing, honest degradation with #123 issue raising. Phase 1 = probe package + manifests + measured use_cuda flip on rhaegar; Phase 2 = routing seams; Phase 3 = NPU/TPU execution paths as hardware arrives.

Design: `roadmap/accelerator-autodetection.md`. Probe at startup (asking runtimes, not inferring from hardware), stored capability manifests reported with agent registration, capability+measurement-based work routing, honest degradation with #123 issue raising. Phase 1 = probe package + manifests + measured use_cuda flip on rhaegar; Phase 2 = routing seams; Phase 3 = NPU/TPU execution paths as hardware arrives.

andrew referenced this issue from a commit

2026-08-30 23:30:51 -05:00

capabilities: probe a node's accelerators into a versioned manifest (#129)

andrew referenced this issue from a commit

2026-08-30 23:30:51 -05:00

config: distinguish an unset use_cuda from an explicit false (#129)

andrew referenced this issue from a commit

2026-08-30 23:30:51 -05:00

capabilities: resolve the effective use_cuda at startup, server and agent (#129)

andrew referenced this issue from a commit

2026-08-30 23:30:51 -05:00

api: expose the accelerator inventory at /api/admin/debug/accelerators (#129)

andrew referenced this issue from a commit

2026-08-30 23:30:51 -05:00

docs/config: document use_cuda as three-state and the new endpoint (#129)

kreeader-user commented

2026-08-30 23:33:48 -05:00

Author

Owner

Phase 1's detection + selection layer is on feat/accelerator-autodetect (5 commits, pushed).

Built

New capabilities/ package: versioned manifest (schema v1) with CUDA (asked of the linked OpenCV CUDA module via an injected CUDAProbe hook, so the package stays gocv-free), CPU features from /proc/cpuinfo narrowed to the routing-relevant flags (AVX2/AVX-512/VNNI/AMX), and report-only iGPU (/dev/dri/renderD*) / NPU (/dev/accel*) / TPU (/dev/apex*) device classes carrying runtime_probed:false, usable:false. probed=false (nobody asked) and available=false (asked, none) stay distinct facts.
use_cuda is now three-state. config.PaneDetectionConfig.UseCUDASet records whether an operator actually wrote it (config file via viper.InConfig, or the KREEADER_ env form). Explicit always wins, with one exception: use_cuda=true on a node with no CUDA device degrades to CPU and says so, rather than being honored literally while gocv silently falls back one layer down.
capabilities.ResolveConfig runs at startup ABOVE the agent-mode branch in main.go, so both the server and a detection agent get it, and writes the effective value back into cfg.Processing.PaneDetection.UseCUDA — every existing consumer (Processor, escalation ladder, signals, panetrain's adoption gate, the agent loop) keeps reading the same field. No existing use_cuda plumbing was removed.
GET /api/admin/debug/accelerators (admin-gated, next to memstats/pprof) returns manifest + selection with source (config/autodetect), degraded, and a one-sentence reason. Mirrored in api/openapi.yaml (path + 6 schemas).
packaging/config.toml no longer ships an explicit use_cuda = false — that line made every packaged install permanently opt out of the probe. Documented in config.toml.example and a new docs/operations.md section.

Measured on rhaegar (live probe, not a claim): cpu=12t+avx2+avx512+vnni cuda=1 device(s) igpu=present(unused) — Xeon W-11855M, 1 CUDA device via opencv-cuda, /dev/dri/renderD128 present but unused. With use_cuda unset it now autodetects to GPU. That is the roadmap's "flip rhaegar's use_cuda by measurement" item, done by probe rather than by editing a TOML.

Deferred (not in this branch)

Agent registration/heartbeat reporting its manifest to the server, and the fleet view in the agents admin surface (roadmap §3).
Phase 2 routing seams: capability-tagged work classes, measurement-refined per-node ranking, studio worker picking endpoints by manifest.
Phase 3 NPU/TPU/iGPU execution paths, and runtime-loadability probes for ONNX Runtime EPs / OpenVINO devices — the manifest records the hardware now so those can execute later.
Runtime-failure latching (CUDA OOM → latch off + raise a #123 server issue). The startup-time degradation case is handled; the mid-run one is not.

Gates: CGO_LDFLAGS=-lprotobuf go build ./... OK, go vet ./... OK, go test ./api/ ./components/detection/processing/ ./capabilities/ ./config/ ./agent/ . all OK. go build -tags nocv ./... OK. (go vet -tags nocv ./components/detection/processing/ fails on a pre-existing untagged choose_test.go referencing cv.PageResult; unrelated to this branch.)

Selection logic is table-tested with injected probe results — probe results are environment-dependent, so nothing asserts against the host.

Phase 1's detection + selection layer is on `feat/accelerator-autodetect` (5 commits, pushed). **Built** - New `capabilities/` package: versioned manifest (schema v1) with CUDA (asked of the linked OpenCV CUDA module via an injected `CUDAProbe` hook, so the package stays gocv-free), CPU features from `/proc/cpuinfo` narrowed to the routing-relevant flags (AVX2/AVX-512/VNNI/AMX), and report-only iGPU (`/dev/dri/renderD*`) / NPU (`/dev/accel*`) / TPU (`/dev/apex*`) device classes carrying `runtime_probed:false, usable:false`. `probed=false` (nobody asked) and `available=false` (asked, none) stay distinct facts. - `use_cuda` is now three-state. `config.PaneDetectionConfig.UseCUDASet` records whether an operator actually wrote it (config file via `viper.InConfig`, or the `KREEADER_` env form). Explicit always wins, with one exception: `use_cuda=true` on a node with no CUDA device degrades to CPU and says so, rather than being honored literally while gocv silently falls back one layer down. - `capabilities.ResolveConfig` runs at startup ABOVE the agent-mode branch in main.go, so both the server and a detection agent get it, and writes the effective value back into `cfg.Processing.PaneDetection.UseCUDA` — every existing consumer (Processor, escalation ladder, signals, panetrain's adoption gate, the agent loop) keeps reading the same field. No existing `use_cuda` plumbing was removed. - `GET /api/admin/debug/accelerators` (admin-gated, next to memstats/pprof) returns manifest + selection with source (`config`/`autodetect`), `degraded`, and a one-sentence reason. Mirrored in `api/openapi.yaml` (path + 6 schemas). - `packaging/config.toml` no longer ships an explicit `use_cuda = false` — that line made every packaged install permanently opt out of the probe. Documented in `config.toml.example` and a new `docs/operations.md` section. **Measured on rhaegar** (live probe, not a claim): `cpu=12t+avx2+avx512+vnni cuda=1 device(s) igpu=present(unused)` — Xeon W-11855M, 1 CUDA device via opencv-cuda, `/dev/dri/renderD128` present but unused. With `use_cuda` unset it now autodetects to GPU. That is the roadmap's "flip rhaegar's use_cuda by measurement" item, done by probe rather than by editing a TOML. **Deferred (not in this branch)** - Agent registration/heartbeat reporting its manifest to the server, and the fleet view in the agents admin surface (roadmap §3). - Phase 2 routing seams: capability-tagged work classes, measurement-refined per-node ranking, studio worker picking endpoints by manifest. - Phase 3 NPU/TPU/iGPU execution paths, and runtime-loadability probes for ONNX Runtime EPs / OpenVINO devices — the manifest records the hardware now so those can execute later. - Runtime-failure latching (CUDA OOM → latch off + raise a #123 server issue). The startup-time degradation case is handled; the mid-run one is not. **Gates**: `CGO_LDFLAGS=-lprotobuf go build ./...` OK, `go vet ./...` OK, `go test ./api/ ./components/detection/processing/ ./capabilities/ ./config/ ./agent/ .` all OK. `go build -tags nocv ./...` OK. (`go vet -tags nocv ./components/detection/processing/` fails on a pre-existing untagged `choose_test.go` referencing `cv.PageResult`; unrelated to this branch.) Selection logic is table-tested with injected probe results — probe results are environment-dependent, so nothing asserts against the host.

andrew referenced this issue from a commit

2026-08-30 23:41:21 -05:00

Merge feat/accelerator-autodetect: probe-backed compute selection (#129)

kreeader-user commented

2026-08-30 23:41:40 -05:00

Author

Owner

First measured increment merged (5074f31, branch feat/accelerator-autodetect): versioned accelerator manifest (CUDA probe, CPU AVX2/AVX-512/VNNI/AMX flags, report-only iGPU/NPU/TPU device classes), tri-state use_cuda — explicit wins, true-without-CUDA degrades with recorded reason, unset autodetects — applied at startup for server and agent, admin inventory at /api/admin/debug/accelerators, openapi mirrored, docs. Live-verified on rhaegar (autodetects its CUDA device with the key unset; packaged config no longer ships a hardcoded use_cuda=false). Deferred (agent heartbeat manifest + fleet view, routing seams, NPU/TPU execution paths, mid-run CUDA-OOM latching) stays in roadmap/accelerator-autodetection.md. Closing the issue on the autodetection deliverable.

First measured increment merged (5074f31, branch feat/accelerator-autodetect): versioned accelerator manifest (CUDA probe, CPU AVX2/AVX-512/VNNI/AMX flags, report-only iGPU/NPU/TPU device classes), tri-state use_cuda — explicit wins, true-without-CUDA degrades with recorded reason, unset autodetects — applied at startup for server and agent, admin inventory at /api/admin/debug/accelerators, openapi mirrored, docs. Live-verified on rhaegar (autodetects its CUDA device with the key unset; packaged config no longer ships a hardcoded use_cuda=false). Deferred (agent heartbeat manifest + fleet view, routing seams, NPU/TPU execution paths, mid-run CUDA-OOM latching) stays in roadmap/accelerator-autodetection.md. Closing the issue on the autodetection deliverable.

kreeader-user closed this issue

2026-08-30 23:41:40 -05:00

Rows
Columns

Accelerator autodetection: use whatever compute a node has (CUDA/VNNI/iGPU/NPU/TPU) #129