Commit Graph

22 Commits

Author SHA1 Message Date
felixfoertsch 05a1345750 rip subtitle manager → sibling project, keep extraction only
Build and Push Docker Image / build (push) Successful in 2m31s
subtitle management (list/detail pages, /api/subtitles, subtitle_files
table, SubtitleFile types, predictExtractedFiles, nav link) moved to a
new sibling project at ~/Developer/netfelix-subtitles-manager/ where
it'll be rebuilt standalone later. this project now owns audio fixing
+ subtitle extraction only.

extraction still runs end to end: analyzeItem still marks every subtitle
stream as "remove from container", buildExtractionOutputs still wires
the -map 0:s:N + sidecar outputs into the ffmpeg command, and execute.ts
still flips review_plans.subs_extracted so verify.ts can check desired
state — just derived from the streams directly instead of by writing a
row per file to the now-gone subtitle_files table.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 22:50:52 +02:00
felixfoertsch 91d8ed67b8 fix authoritative language sources getting demoted to needs_review
Build and Push Docker Image / build (push) Successful in 2m22s
rescan flagged every item where radarr/sonarr disagreed with jellyfin's
audio-track guess as needs_review=1, but the analyzer's authoritativeOg
check demands needs_review=0 — so the very items we had an authoritative
answer for were the ones dumped into the "Needs decision" bucket. Lost
(german dubs first on most files, sonarr authoritatively english) was
the perfect worst case: nearly every episode misclassified as manual.

trust the authoritative source unconditionally and reset needs_review=0
when it fires, mismatch or not.
2026-04-19 15:20:18 +02:00
felixfoertsch 495a40a6c6 restructure pipeline column headers with backward/skip/forward slots, rename ready → auto-approve
Build and Push Docker Image / build (push) Successful in 1m52s
- columnshell: three-row header (title, subtitle, backward · skip · forward grid)
- inbox: auto-processing checkbox moves into subtitle, page top-right toggle removed
- review: add ← back to inbox, rename approve-all-ready to approve auto
- queue: clear button relabeled ← back to inbox (matches existing behaviour)
- done: add ← back to review (reopen-all), clear moves to middle skip slot
- seriescard: action row mirrors movie card, now at the top
-  ready badge →  auto-approve with tooltip, review subtitle uses "auto · need decisions"
- new endpoints: /api/review/unsort-all, /api/review/reopen-all (+ tests)
2026-04-19 13:16:29 +02:00
felixfoertsch 0d560743f3 fix manual language overrides wiped on rescan, use series tvdb for sonarr, split seriescard controls
Build and Push Docker Image / build (push) Successful in 3m31s
- rescan: skip jellyfin/radarr/sonarr lookups when orig_lang_source='manual' so user pins survive webhook + full scans
- jellyfin: request SeriesProviderIds so episodes can resolve to the series-level tvdb id
- sonarr: drop the lookup[0] fallback that silently returned unrelated shows on tvdb misses
- seriescard: split badges+language and approve buttons onto separate rows; seasongroup header wraps with ml-auto so buttons don't overflow the narrow pipeline column
- tests: cover manual override preservation and episode → series tvdb resolution

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 07:41:08 +02:00
felixfoertsch 8e1deb39d5 analyzer tests: narrow orig_lang_source type to MediaItem union
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 10:20:22 +02:00
felixfoertsch 455796ebb6 analyzer: only count commentary-driven removes as heuristic
snapshot decisions after decideAction, before deduplicateAudioByLanguage;
diff to identify tracks that flipped keep→remove due to the commentary
regex — language-driven removes no longer falsely upgrade auto_class to
auto_heuristic even when the title coincidentally matches

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 10:19:22 +02:00
felixfoertsch c595ad3792 analyzer: classify each plan as auto/auto_heuristic/manual
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 10:15:01 +02:00
felixfoertsch cbf0025a81 drop review_plans.verified column and all its references
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 06:55:43 +02:00
felixfoertsch 1de5b8a89e address audit findings: subtitle rescan decisions, scan limit, parseId, setup gate
Build and Push Docker Image / build (push) Successful in 1m30s
worked through AUDIT.md. triage:
- finding 2 (subtitle rescan wipes decisions): confirmed. /:id/rescan now
  snapshots custom_titles and calls reanalyze() after the stream delete/
  insert, mirroring the review rescan flow. exported reanalyze + titleKey
  from review.ts so both routes share the logic.
- finding 3 (scan limit accepts NaN/negatives): confirmed. extracted
  parseScanLimit into a pure helper, added unit tests covering NaN,
  negatives, floats, infinity, numeric strings. invalid input 400s and
  releases the scan_running lock.
- finding 4 (parseId lenient): confirmed. tightened the regex to /^\d+$/
  so "42abc", "abc42", "+42", "42.0" all return null. rewrote the test
  that codified the old lossy behaviour.
- finding 5 (setup_complete set before jellyfin test passes): confirmed.
  the /jellyfin endpoint still persists url+key unconditionally, but now
  only flips setup_complete=1 on a successful connection test.
- finding 6 (swallowed errors): partial. the mqtt restart and version-
  fetch swallows are intentional best-effort with downstream surfaces
  (getMqttStatus, UI fallback). only the scan.ts db-update swallow was
  a real visibility gap — logs via logError now.
- finding 1 (auth): left as-is. redacting secrets on GET without auth
  on POST is security theater; real fix is an auth layer, which is a
  design decision not a bugfix. audit removed from the tree.
- lint fail on ffmpeg.test.ts: formatted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 17:41:36 +02:00
felixfoertsch d05e037bbc webhook: PascalCase payload + ItemAdded only, switch ✓✓ signal to ffprobe
Build and Push Docker Image / build (push) Successful in 1m56s
monitoring the mqtt broker revealed two bugs and one design dead-end:

1. the jellyfin-plugin-webhook publishes pascalcase fields
   (NotificationType, ItemId, ItemType) and we were reading camelcase
   (event, itemId, itemType). every real payload was rejected by the
   first guard — the mqtt path never ingested anything.

2. the plugin has no ItemUpdated / Library.* notifications. file
   rewrites on existing items produce zero broker traffic (observed:
   transcode + manual refresh metadata + 'recently added' appearance
   → no mqtt messages). ✓✓ via webhook is structurally impossible.

fix the webhook path so brand-new library items actually get ingested,
and narrow ACCEPTED_EVENTS to just 'ItemAdded' (the only library-side
event the plugin emits).

move the ✓✓ signal from webhook-corroboration to post-execute ffprobe
via the existing verifyDesiredState helper: after ffmpeg returns 0 we
probe the output file ourselves and flip verified=1 on match. the
preflight-skipped path sets verified=1 too. renamed the db column
webhook_verified → verified (via idempotent RENAME COLUMN migration)
since the signal is no longer webhook-sourced, and updated the Done
column tooltip to reflect that ffprobe is doing the verification.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 17:27:22 +02:00
felixfoertsch 9cdc054c4b audio titles: rewrite to canonical 'ENG - CODEC · CHANNELS', two-line pipeline card
Build and Push Docker Image / build (push) Successful in 59s
audio tracks now get a harmonized title on output (overriding any file
title like 'Audio Description' — review has already filtered out tracks
we don't want to keep). mono/stereo render numerically (1.0/2.0), matching
the .1-suffixed surround layouts. pipeline card rows become two-line so
long titles wrap instead of being clipped by the column.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 16:02:32 +02:00
felixfoertsch 027ea498c3 done column: double-checkmark when jellyfin webhook corroborates the plan
Build and Push Docker Image / build (push) Successful in 1m37s
adds review_plans.webhook_verified, set to 1 whenever a fresh analysis
(scan or post-execute webhook) sees is_noop=1, cleared if a webhook
later flips the plan off-noop. resurrected the try/catch alter table
migration pattern in server/db/index.ts for the new column.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 15:50:57 +02:00
felixfoertsch 4057b692ba audio: single EAC3 transcode target, prefer direct-play over lossless default
Build and Push Docker Image / build (push) Successful in 47s
two simplifications to how we pick and transcode the one-per-language
audio track, motivated by seeing inconsistent DTS → FLAC vs DTS →
EAC3 outputs in the wild:

transcode target:
- drop the FLAC path entirely. every incompatible source now targets
  EAC3 regardless of container or lossless/lossy status
- FLAC for movie audio is bad value: ~2-3× the file size vs EAC3, no
  Atmos spatial metadata (TrueHD Atmos → FLAC silently loses Atmos),
  no AVR passthrough on Apple TV
- one target = no more container-conditional surprises

winner within a language group (betterAudio):
- new priority: highest channels → Apple-compatible → default → index
- old order put 'default' on top which forced a DTS-HD MA transcode
  even when an AC3 track at equal channels was right next to it.
  flipping means AC3 beats DTS-HD MA at the same channel count — pure
  copy instead of a lossless-then-re-encode round trip
- channel count still dominates, so 7.1 TrueHD still beats 5.1 AC3
  (and gets transcoded, which is the right call for real surround)

tests: new case for DTS-HD MA default + AC3 non-default at 5.1 → AC3
wins, job_type=copy. new case for 7.1 TrueHD beats 5.1 AC3 default.
every other existing test still holds.
2026-04-14 10:23:49 +02:00
felixfoertsch 6698af020d analyzer: keep only one audio track per language, drop commentary/AD
Build and Push Docker Image / build (push) Successful in 42s
a release with 2× english (main + director's commentary, or a surround
track plus an audio-description track) was keeping both. the user only
wants one per language. rules, in priority order:

- always drop commentary / audio-description / visually-impaired /
  karaoke / sign-language tracks (matched by title regex + the
  is_hearing_impaired flag)
- within each kept-language group, pick one winner by:
  1. default disposition (main track the muxer chose)
  2. highest channel count
  3. apple-compatible codec (skip a transcode pass)
  4. lowest stream_index for stability

tests cover: commentary dropped even when it matches OG, AD flag
dropped, default beats non-default, higher channels beat default-less
candidates of equal type, Apple-compat tiebreak, per-language dedupe
runs independently, and single-stream files stay noop.
2026-04-14 10:07:02 +02:00
felixfoertsch a27e4f4025 close the jellyfin ping-pong via mqtt webhook subscriber
Build and Push Docker Image / build (push) Successful in 1m5s
after ffmpeg finishes we used to block the queue on a jellyfin refresh
+ re-analyze round-trip. now we just kick jellyfin and return. a new
mqtt subscriber listens for library events from jellyfin's webhook
plugin and re-runs upsertJellyfinItem — flipping plans back to pending
when the on-disk streams still don't match, otherwise confirming done.

- execute.ts: hand-off is fire-and-forget; no more sync re-analyze
- rescan.ts: upsertJellyfinItem takes source: 'scan' | 'webhook'.
  webhook-sourced rescans can reopen terminal 'done' plans when
  is_noop flips back to 0; scan-sourced rescans still treat done as
  terminal (keeps the dup-job fix from a06ab34 intact).
- mqtt.ts: long-lived client, auto-reconnect, status feed for UI badge
- webhook.ts: pure processWebhookEvent(db, deps) handler + 5s dedupe
  map to kill jellyfin's burst re-fires during library scans
- settings: /api/settings/mqtt{,/status,/test} + /api/settings/
  jellyfin/webhook-plugin (checks if the plugin is installed)
- ui: new Settings section with broker form, test button, copy-paste
  setup panel for the Jellyfin plugin template. MQTT status badge on
  the scan page.
2026-04-14 08:26:42 +02:00
felixfoertsch c5ea37aab9 address audit findings: schedule validation, settings json guard, pipeline types, a11y labels
Build and Push Docker Image / build (push) Successful in 58s
2026-04-13 15:48:55 +02:00
felixfoertsch 6d8a8fa6d6 drop the subtitle-languages setting, it never influenced extraction
Build and Push Docker Image / build (push) Successful in 53s
analyzer removes every subtitle unconditionally (see case 'Subtitle' in
decideAction) and the pipeline extracts all of them to sidecars — the config
was purely informational and only subtitles.ts echoed it back as
'keepLanguages' for a subtitle-manager ui that doesn't exist yet. we'll
revive language preferences inside that manager when it ships.

removes: the settings card + ui state, POST /api/settings/subtitle-languages,
the config default, the SUBTITLE_LANGUAGES env mapping, AnalyzerConfig's
subtitleLanguages field, RescanConfig's subtitleLanguages field, every
caller site (scan.ts / execute.ts / review.ts), and the keepLanguages
surface in subtitles.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 15:26:48 +02:00
felixfoertsch 6fcaeca82c write canonical iso3 language metadata, tighten is_noop, store full jellyfin data
Build and Push Docker Image / build (push) Failing after 16s
ffmpeg now writes -metadata:s:a:i language=<iso3> on every kept audio track so
files end up with canonical 3-letter tags (en → eng, ger → deu, null → und).
analyzer passes stream.profile (not title) to transcodeTarget so lossless
dts-hd ma in mkv correctly targets flac. is_noop also checks og-is-default and
canonical-language so pipeline-would-change-it cases stop showing as done.

normalizeLanguage gains 2→3 mapping, and mapStream no longer normalizes at
ingest so the raw jellyfin tag survives for the canonical check.

per-item scan work runs in a single db.transaction for large sqlite speedups,
extracted into server/services/rescan.ts so execute.ts can reuse it.

on successful job, execute calls jellyfin /Items/{id}/Refresh, waits for
DateLastRefreshed to change, refetches the item, and upserts it through the
same pipeline; plan flips to done iff the fresh streams satisfy is_noop.

schema wiped + rewritten to carry jellyfin_raw, external_raw, profile,
bit_depth, date_last_refreshed, runtime_ticks, original_title, last_executed_at
— so future scans aren't required to stay correct. user must drop data/*.db.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 13:56:19 +02:00
felixfoertsch 50d3e50280 fix '8 Mile is Turkish': jellyfin guesses never earn high confidence
Build and Push Docker Image / build (push) Successful in 28s
Two bugs compounded:

1. extractOriginalLanguage() in jellyfin.ts picked the FIRST audio stream's
   language and called it 'original'. Files sourced from non-English regions
   often have a local dub as track 0, so 8 Mile with a Turkish dub first
   got labelled Turkish.

2. scan.ts promoted any single-source answer to confidence='high' — even
   the pure Jellyfin guess, as long as no second source (Radarr/Sonarr)
   contradicted it. Jellyfin's dub-magnet guess should never be green.

Fixes:
- extractOriginalLanguage now prefers the IsDefault audio track and skips
  tracks whose title shouts 'dub' / 'commentary' / 'director'. Still a
  heuristic, but much less wrong. Fallback to the first track when every
  candidate looks like a dub so we have *something* to flag.
- scan.ts: high confidence requires an authoritative source (Radarr/Sonarr)
  with no conflict. A Jellyfin-only answer is always low confidence AND
  gets needs_review=1 so it surfaces in the pipeline for manual override.
- Data migration (idempotent): downgrade existing plans backed only by the
  Jellyfin heuristic to low confidence and mark needs_review=1, so users
  don't have to rescan to benefit.
- New server/services/__tests__/jellyfin.test.ts covers the default-track
  preference and dub-skip behavior.
2026-04-13 11:39:59 +02:00
felixfoertsch d12dd80209 fix: buildCommand now extracts subtitles to sidecars before stripping them
Build and Push Docker Image / build (push) Successful in 1m12s
Bug: every approve path (buildCommand used by review approve/approve-all/
series approve-all/season approve-all/retry/detail preview) was building
an ffmpeg command that -map'd only the 'keep' streams and dropped all
subtitles. For a file like Wuthering Heights with 37 embedded subs, the
run would delete every sub into the void — user expected extraction to
sidecar files per the pipeline contract.

buildPipelineCommand already did the right thing (extract every subtitle
with -map 0:s:N -c:s copy 'basename.lang.srt', then remux kept streams)
but it was only reached by tests. buildCommand now delegates to it — one
call site, subtitle extraction always runs, predictExtractedFiles records
the sidecar paths after job success (same logic, same basePath).

Added a regression test: buildCommand on a 2-subtitle file contains
-map 0:s:0, -map 0:s:1 and the expected 'basename.en.srt'/'.de.srt' paths.
2026-04-13 11:12:10 +02:00
felixfoertsch 874f04b7a5 wire scheduler into queue, add retry, dev-reset cleanup, biome 2.4 migrate
- execute: actually call isInScheduleWindow/waitForWindow/sleepBetweenJobs in runSequential (they were dead code); emit queue_status SSE events (running/paused/sleeping/idle) so the pipeline's existing QueueStatus listener lights up
- review: POST /:id/retry resets an errored plan to approved, wipes old done/error jobs, rebuilds command from current decisions, queues fresh job
- scan: dev-mode DELETE now also wipes jobs + subtitle_files (previously orphaned after every dev reset)
- biome: migrate config to 2.4 schema, autoformat 68 files (strings + indentation), relax opinionated a11y/hooks-deps/index-key rules that don't fit this codebase
- routeTree.gen.ts regenerated after /nodes removal
2026-04-13 07:41:19 +02:00
felixfoertsch f11861658e add bun:test coverage for analyzer + ffmpeg + validate, emit ffmpeg progress sse
- analyzer.test.ts: audio keep rules (OG + configured langs, unknown OG, undetermined lang, iso alias), ordering (OG first, reorder noop), subtitle forced-remove, transcode targets
- ffmpeg.test.ts: shellQuote, sortKeptStreams canonical order, buildCommand tmp+mv, type-relative maps (0:a:N), disposition, buildPipelineCommand sub extraction + transcode bitrate, predictExtractedFiles dedup
- validate.test.ts: parseId bounds + isOneOf narrowing
- execute: parse ffmpeg Duration + time, emit job_progress SSE events throttled at 500ms so ProcessingColumn progress bar fills in (it already listened)
- package: switch test script from placeholder echo to 'bun test'
2026-04-13 07:35:24 +02:00