kreeader-server

kreeader-server is the Go daemon at the center of the Kreeader comic-reading platform. It scans and indexes a library of comic archives, detects panel ("pane") rectangles on every page with OpenCV for panel-by-panel reading, matches books against comic databases for metadata, embeds the Kreeader web client, and serves one HTTP API consumed by the web, desktop, mobile, and TV clients. Clients discover the server through mDNS service _kreeader._tcp, with the legacy _comictv._tcp service advertised alongside it.

Feature overview

• Library scanning and indexing. scanner/ walks the directories in library.paths, using fsnotify watches together with interval polling. It reads CBZ (ZIP, including zstd-compressed entries through klauspost/compress), CBR (rardecode), CB7 (7z through bodgit/sevenzip), CBT (tar), and image-based PDF (pdfcpu) files.

  Container detection uses magic-byte sniffing through archive.DetectFormat, not the filename extension, so a RAR archive misnamed with a .cbz extension is still readable; pages are read from the archive into memory and are never extracted to disk.

• Content-derived comic identity. archive.ComicID computes a comic ID as the first 16 hexadecimal characters of SHA-256 over a container-independent page signature: the page count plus the decoded bytes of the cover and one interior page. A book therefore keeps its ID across moves, renames, and CBR-to-CBZ conversion; when the scanner recognizes relocated content, it retargets the stored path.

  Unchanged files are skipped cheaply using path, size, and modification time.

• Layout detection. The geometric and learned detection pipeline lives in the extracted forgejo.jiggl.in/libraries/kreeader-cv module, providing an XY-cut profile ladder, a learned strip-classifier rung, and CPU and optional CUDA paths; the server connects that pipeline to the library and stores per-page results in the database.

  The web UI includes a corrections workbench at /?panes=1, with /panes redirecting there, while corrections are stored as ground truth in pane_corrections and are never overwritten by re-detection. Automatic retraining in components/detection/panetrain/auto.go gates candidate models against both the deployed model and the geometric baseline before adoption, records runs in pane_model_runs, and uses each page's model_gen to trigger re-detection after a newer model is adopted; configure the model file with processing.pane_detection.model_path, whose default is <cache_dir>/panemodel.json.

• Metadata pipeline. components/dataproviders/ integrates matching against a Grand Comics Database (GCD) SQLite dump, the ComicVine API, cover matching through perceptual hashes from forgejo.jiggl.in/libraries/perceptual-hash, and Metron. The API provides manual and bulk resolution endpoints; POST /api/comics/unmatch clears an incorrect manual match, including its field locks.

• Embedded web client. The Svelte client in the kreeader-ui git submodule is compiled into the binary with go:embed from kreeader-ui/dist and served at /. For frontend development, server.web_dir or -web-dir overrides the embedded files at runtime.

  Building with the nobrowser tag produces an API-only binary that does not require the submodule's dist directory.

• Authentication. All three authentication modes use the authUser seam in api/auth.go: single_user is the default and supplies one shared identity without sign-in; oidc is enabled by setting oidc.issuer and accepts Bearer ID tokens, with Kanidm covered by testing (optionally requiring membership in [oidc] user_group); and the opt-in local_multiusers mode provides username/password accounts, argon2id password hashes, server-side sessions, and the admin, user, and publisher roles.

  Local multi-user routes include /api/auth/login and /api/admin/users, and the first administrator is created with kreeader-server -create-admin or the [auth] bootstrap_admin_* settings. Optional WebAuthn/passkeys are enabled with [auth.webauthn] enabled, are off by default, work only with local_multiusers, and are an alternative to the password rather than a replacement for it.

• Embedded Dex identity provider. The optional IdP is off by default and enabled with [idp] enabled = true; it runs github.com/dexidp/dex in-process on the same mux, mounts at /idp by default, and stores its tables in the same database.

  A custom connector bridges Dex logins to the same local_multiusers accounts, while administrators can configure upstream Google, GitHub, and generic OIDC federation, as well as OAuth2 clients, at runtime through /api/admin/idp/connectors and /api/admin/idp/clients. Point oidc.issuer at the embedded instance for the server's own API to accept its tokens.

• Store and publisher system. Publisher accounts use the publisher role, and the Book Builder authoring pipeline supports page upload and ordering, metadata, layout detection and corrections, and server-side archive creation and encryption; per-device DRM content-key wrapping uses AES-256-GCM with ECDH/HKDF.

  Upload-time piracy detection uses perceptual hashes, places suspected content on hold for review without automatically rejecting it, and fails open. The unified support and DMCA system uses db.SupportCase and human-readable case numbers in the form CASE-YYYY-NNNNNN; the store also includes publisher freezing, ratings, and Lago billing in lago/, with details in docs/book-builder.md, docs/drm-design.md, docs/dmca-and-support.md, and docs/ratings-and-price-modifiers.md.

• Library-maintenance engine. librarian/ runs one background job at a time behind GET/POST /api/librarian[/run]. Its jobs are dedupe, which removes only byte-identical duplicates, re-validates the keeper before any deletion, and quarantines broken books instead of deleting them; identify, which tries ComicInfo.xml, then the GCD dump, then ComicVine; organize, which uses Sonarr-style [sorter] naming and places unidentified books under <Publisher>-Unsorted/ while preserving their relative structure; and chown. Headless organize writes a JSONL plan and requires -organize-apply for rename-only moves.

  Layout detection requires the book to be sorted first, as recorded by Comic.Sorted, and a manual detection request for an unsorted book returns HTTP 409 with {"error":"unsorted"}.

• Cover analysis. These runtime-optional features add no extra cgo dependency: printed-metadata OCR invokes tesseract as an external tool and is hidden when the binary is unavailable, while EAN/UPC barcode reading uses the pure-Go gozxing package.

  Trade-dress fingerprinting uses forgejo.jiggl.in/libraries/trade-dress-signature and exposes /api/dress/backfill and /api/comics/{id}/dressmatch. Scene-release tag-page detection and removal is available at /api/librarian/scenepages and quarantines content before rewriting it.

• Agent mode. With [agent] configured, the same binary runs as a remote pane-detection worker. Work is leased by library-relative path so the main server and agent can mount the library at different locations, while all decisions remain on the main server. See docs/agents.md.

• Distributed P2P tier. The optional forgejo.jiggl.in/libraries/kreeader-p2p tier is built on go-libp2p and is off by default. Nothing starts unless [distributed] enabled = true; panel-correction metrics, the database swarm, model weights, telemetry, and book metadata each also require separate, explicit user consent through the client's Settings → Distributed Data tab.

• Databases. Persistence uses GORM with SQLite by default; PostgreSQL is selected with database.driver = "postgres". The schema is managed with AutoMigrate at startup rather than migration files, and cmd/migrate-sqlite-postgres moves an existing installation between the two databases.

• Observability. The server exposes Prometheus metrics and health checks. Grafana setup is documented in docs/grafana.md.

Build

Initialize the web-client submodule for a default build, then use the build script:

git submodule update --init   # kreeader-ui (web client) — required for the default build
./build.sh                    # → ./kreeader-server
./build.sh nobrowser          # API-only binary, no embedded web client
./build.sh matprofile         # diagnostic build: countable gocv Mats (see below)

The matprofile build is for chasing memory: it compiles in gocv's Mat profile, so unclosed gocv.Mats (C memory the Go GC never frees) show up as live_mats in GET /api/admin/debug/memstats and as full allocation stacks at /api/admin/debug/pprof/gocv.io/x/gocv.Mat?debug=1. It is a diagnostic binary, never the packaged one — see docs/operations.md.

CGO is required because gocv needs OpenCV installed. The vendored fork under third_party/gocv carries OpenCV 5 API fixes; when only opencv5.pc exists, build.sh generates a pkg-config shim. The build also links -lprotobuf to work around a gocv link error.

tesseract is invoked as an optional external runtime tool and is not a build dependency.

Run the test suite with:

CGO_LDFLAGS="-lprotobuf" go test ./...

The detection packages require OpenCV in order to compile.

Run / quick start

1. Copy config.toml.example to config.toml.
2. Set library.paths to the directories containing the comic library.
3. Start the server with ./kreeader-server.

Configuration is searched for in /etc/kreeader, then the legacy /etc/comictv location, and finally falls back to ./config.toml. The server listens on server.port, which defaults to 8080. The web client is served at /, and Swagger UI is available at /docs; api/openapi.yaml is the hand-maintained API contract.

Production packaging is provided by PKGBUILD and packaging/, including the kreeader.service systemd unit.

Configuration

Configuration uses Viper. Values are resolved in this precedence order:

1. Command-line flags
2. KREEADER_ environment variables
3. Configuration file
4. Built-in defaults

Only command-line flags that were actually passed take precedence. Nested keys use underscores in environment variables; for example, library.paths becomes KREEADER_LIBRARY_PATHS.

Available flags include:

• -config
• -port
• -db
• -library a,b
• -cache
• -scan-interval
• -model
• -no-autotrain
• -no-scan
• -web-dir
• -create-admin

Run ./kreeader-server -h for the complete flag list.

Repository layout

Path	Description
main.go	Wiring for configuration, database setup, scanning, detection, the API, and optional subsystems.
scanner/	Library walking, fsnotify watches, change detection, and pruning.
archive/	CBZ, CBR, CB7, CBT, and PDF page readers; format sniffing; ComicID; and ZIP rewriting.
components/detection/	Server-side detection glue: the processing wrapper, OCR, and automatic training. The core CV pipeline is in the kreeader-cv module.
components/dataproviders/	GCD, ComicVine, the scraper queue, and Metron integrations.
db/	GORM models for comics and pages, metadata entities, users and sessions, and the store domain, plus SQLite and PostgreSQL support.
api/	Every HTTP route through RegisterRoutes, the authentication seam, and the embedded openapi.yaml and Swagger UI.
idp/	The embedded Dex identity provider, local-connector bridge, and hosted login.
librarian/	Background jobs for deduplication, identification, organization, and ownership changes.
agent/	Remote detection worker mode.
drm/, lago/, mail/, origins/, ratelimit/	Store DRM, billing, outbound mail, CORS/origin handling, and rate limiting.
libpath/	Library-relative path conversion; the database stores relative paths and hooks perform conversion.
config/	Viper configuration with defaults for every key.
cmd/experiments/	Detection tools including paneviz, scorepanes, rowprobe, trainpanes, and other one-off probes.
cmd/migrate-identity, cmd/migrate-sqlite-postgres, cmd/seedpub, cmd/p2ptest	One-time and operational tools.
kreeader-ui/	Git submodule containing the Svelte web client shared with kreeader-client.
third_party/gocv	Vendored gocv fork with OpenCV 5 support.
docs/	Factual documentation of current functionality, indexed by docs/README.md; FEATURES.md is the plain-language tour.
roadmap/	Forward-looking design documents for functionality that is not built, kept separate from docs/.
packaging/, PKGBUILD, kreeader.service	Arch packaging and systemd integration.

Extracted libraries

The codebase is being modularized. Five components now live as standalone Go modules under forgejo.jiggl.in/libraries/ and are consumed as ordinary dependencies:

Module	Purpose
kreeader-cv	Pane-detection computer-vision pipeline.
kreeader-p2p	Distributed data tier.
perceptual-hash	Cover and piracy matching.
trade-dress-signature	Cover trade-dress fingerprinting.
signed-seed-list	P2P bootstrap.

All five extracted modules are consumed as tagged Forgejo releases (kreeader-cv v0.1.0, kreeader-p2p v0.1.1, plus perceptual-hash, trade-dress-signature, signed-seed-list), so a standalone clone builds with no sibling checkouts. For co-developing a module with the server, create a go.work (gitignored) that uses . and the sibling ../libraries/<module> checkouts — and copy it into any worktree that must build against uncommitted module edits.

Documentation

Start with docs/README.md, the index for architecture, layout detection, metadata, the API, clients, operations, agents, authentication, DRM, and store design documentation. Documents under docs/ describe only implemented functionality; designs that have not been built are kept in roadmap/.