Watch

No description

Find a file

Repository files (latest commit first)
Filename	Latest commit message	Latest commit date
Andrew Tyler 72d76c5136 docs: replace attribution research with the rigorous, decision-forcing version Supersedes the earlier attribution sketch with a fully primary-sourced report that changes the build order: - Ship the closest-character + tail-direction heuristic FIRST — CoMix shows it beats the SOTA manga model (Magi) 3–4× on western comics; the manga numbers do not transfer. - Tail geometry is the highest-value signal (0.983 AP; drives diarisation 0.52→0.75); classical CV hits 81.2% tail detection on a western dataset. - Naive full-page VLM attribution fails catastrophically (0–1.11%); use VLMs for adjudication of ambiguous cases only, never primary, and never on raw pages. Refusal-classifier risk is real. - MILP constrained assignment vs the bank is the biggest single lever (+0.22–0.26); exemplar curation is +0.15. - Two licensing collisions with kreeader's store module: Magi models are non-commercial (reimplement the architecture, don't ship weights), and ComicVine is non-commercial + no-redistribution + one image per character. So build the exemplar bank from the user's OWN library — more accurate AND license-clean. - Realistic target 70–85% per-bubble with a curated bank, ~20–30% on off-panel speakers; human correction is the product. Includes the per-bubble BubbleText schema gap and kreeader-specific integration notes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-12 01:09:59 -05:00
docs	docs: replace attribution research with the rigorous, decision-forcing version	2026-08-12 01:09:59 -05:00

Andrew Tyler 72d76c5136 docs: replace attribution research with the rigorous, decision-forcing version

Supersedes the earlier attribution sketch with a fully primary-sourced
report that changes the build order:
- Ship the closest-character + tail-direction heuristic FIRST — CoMix
  shows it beats the SOTA manga model (Magi) 3–4× on western comics; the
  manga numbers do not transfer.
- Tail geometry is the highest-value signal (0.983 AP; drives diarisation
  0.52→0.75); classical CV hits 81.2% tail detection on a western dataset.
- Naive full-page VLM attribution fails catastrophically (0–1.11%); use
  VLMs for adjudication of ambiguous cases only, never primary, and never
  on raw pages. Refusal-classifier risk is real.
- MILP constrained assignment vs the bank is the biggest single lever
  (+0.22–0.26); exemplar curation is +0.15.
- Two licensing collisions with kreeader's store module: Magi models are
  non-commercial (reimplement the architecture, don't ship weights), and
  ComicVine is non-commercial + no-redistribution + one image per
  character. So build the exemplar bank from the user's OWN library —
  more accurate AND license-clean.
- Realistic target 70–85% per-bubble with a curated bank, ~20–30% on
  off-panel speakers; human correction is the product. Includes the
  per-bubble BubbleText schema gap and kreeader-specific integration
  notes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-08-12 01:09:59 -05:00

docs

docs: replace attribution research with the rigorous, decision-forcing version

2026-08-12 01:09:59 -05:00

docs/README.md

kreeader-talker — docs

Research and design for giving comic-book characters synthetic voices and turning a static comic into a voiced motion comic: each speech bubble spoken by its character while the camera pans across panels in time with the audio.

These docs are a feature scope-out, not shipped functionality — research findings + a design + an interactive mockup. Per the project's docs/roadmap convention, anything here that becomes real code graduates into factual docs at that point.

Start here

correction-system.md — the design: how the system discovers who says what, how a human corrects it (the full correction primitive set), casting, and the playback-timeline data model. Reuses the existing pane/transcript/character infrastructure.
mockup/index.html — open in a browser. An interactive mockup of the correction studio: click the numbered bubbles, assign speakers, cast per-character synthetic voices (audible via the browser's offline speech API), and hit Play motion comic to watch the camera pan panel-to-panel synced to the dialogue timeline. (attribute view · casting view)
engineering-notes.md — the build bridge: design-then-clone identity model, on-disk character record, the HTTP sidecar integration pattern, EU-AI-Act/right-of-publicity compliance, and comic-specific synthesis gotchas.

Research

research-voices.md — TTS + voice systems (2026): voice cloning vs attribute-based voice generation, engine survey, what fits the self-hosted GPU box, recommended stack + fallback.
research-attribution.md — speaker attribution: discovering who says each bubble (tail-direction CV, VLM attribution, bubble-type classification), and the exact correction primitives that feeds.
research-reference-images.md — character exemplar bank: where labeled per-character reference images come from (Marvel API dead; ComicVine 1/char; Fandom interior panels are the goldmine), the cover-vs-interior domain-shift trap, and a ship-embeddings-not-images legal posture.
research-prosody.md — intonation, pacing, and motion: mapping comic lettering cues to prosody, emotion inference, and the automated pane-panning ("Ken Burns") camera model timed to audio.
research-hardware-bench.md — TTS runtime feasibility on butcher's exact GPU (RTX 2070 Super, 8 GB, sm_75): VRAM, RTF, the Turing bf16/fp16 traps, and a pre-build validation checklist.
research-licenses.md — primary-source license audit of 10 open TTS engines (code + weights), for picking a commercial-safe, self-hostable engine.

Policy

ethics.md — the voice policy: attribute-designed synthetic voices by default; real-person cloning is opt-in, consent-bound, and never the default; generated audio is labeled synthetic.

What already exists to build on

Talker is a thin semantic + playback layer over shipped kreeader-server infrastructure: per-pane OCR transcripts (Page.Transcript), pane rectangles in reading order (Page.Panes), the per-book character roster (ComicCharacter), the pane-correction workbench patterns, and the pane-stepping reader. See correction-system.md for the mapping.