subtitle management (list/detail pages, /api/subtitles, subtitle_files
table, SubtitleFile types, predictExtractedFiles, nav link) moved to a
new sibling project at ~/Developer/netfelix-subtitles-manager/ where
it'll be rebuilt standalone later. this project now owns audio fixing
+ subtitle extraction only.
extraction still runs end to end: analyzeItem still marks every subtitle
stream as "remove from container", buildExtractionOutputs still wires
the -map 0:s:N + sidecar outputs into the ffmpeg command, and execute.ts
still flips review_plans.subs_extracted so verify.ts can check desired
state — just derived from the streams directly instead of by writing a
row per file to the now-gone subtitle_files table.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rescan flagged every item where radarr/sonarr disagreed with jellyfin's
audio-track guess as needs_review=1, but the analyzer's authoritativeOg
check demands needs_review=0 — so the very items we had an authoritative
answer for were the ones dumped into the "Needs decision" bucket. Lost
(german dubs first on most files, sonarr authoritatively english) was
the perfect worst case: nearly every episode misclassified as manual.
trust the authoritative source unconditionally and reset needs_review=0
when it fires, mismatch or not.
- rescan: skip jellyfin/radarr/sonarr lookups when orig_lang_source='manual' so user pins survive webhook + full scans
- jellyfin: request SeriesProviderIds so episodes can resolve to the series-level tvdb id
- sonarr: drop the lookup[0] fallback that silently returned unrelated shows on tvdb misses
- seriescard: split badges+language and approve buttons onto separate rows; seasongroup header wraps with ml-auto so buttons don't overflow the narrow pipeline column
- tests: cover manual override preservation and episode → series tvdb resolution
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
snapshot decisions after decideAction, before deduplicateAudioByLanguage;
diff to identify tracks that flipped keep→remove due to the commentary
regex — language-driven removes no longer falsely upgrade auto_class to
auto_heuristic even when the title coincidentally matches
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Abraham Lincoln crashed with exit 234 because the file had 14 dvd_subtitle
streams: our extraction dict only keyed on the long form (dvd_subtitle)
while jellyfin stores the short form (dvdsub), so the lookup fell back
to .srt, ffmpeg picked the srt muxer, and srt can't encode image-based
subs. textbook silent dict miss.
replaced the extension dict with an EXTRACTABLE map that pairs codec →
{ext, codecArg} and explicitly enumerates every codec we can route to a
single-file sidecar. everything else (dvd_subtitle/dvdsub, dvb_subtitle/
dvbsub, unknown codecs) is now skipped at command-build time. the plan
picks up a note like '14 subtitle(s) dropped: dvdsub (eng, est, ind,
kor, jpn, lav, lit, may, chi, chi, tha, vie, rus, ukr) — not extractable
to sidecar' so the user sees exactly what didn't make it.
also added extractErrorSummary in execute.ts: when a job errors, scan
the last 60 stderr lines for fatal keywords (Error:, Conversion failed!,
Unsupported, Invalid argument, Permission denied, No space left, …),
dedupe, prepend the summary to the job's stored output. the review_plan
notes get the same summary — surfaces the real cause next to the plan
instead of burying it under ffmpeg's 200-line banner.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
user reported ad astra got the double checkmark instantly after
transcode — correct, and correct to flag: the post-execute
verifyDesiredState ran ffprobe on the file we had just written, so it
tautologically matched the plan every time. not a second opinion.
replaced the flow with the semantics we actually wanted:
1. refreshItem now returns { refreshed: boolean } — true when jellyfin's
DateLastRefreshed actually advanced within the timeout, false when it
didn't. callers can tell 'jellyfin really re-probed' apart from
'we timed out waiting'.
2. handOffToJellyfin post-job: refresh → (only if refreshed=true) fetch
fresh streams → upsertJellyfinItem(source='webhook'). the rescan SQL
sets verified=1 exactly when the fresh analysis sees is_noop=1, so
✓✓ now means 'jellyfin independently re-probed the file we wrote
and agrees it matches the plan'. if jellyfin sees a drifted layout
the plan flips back to pending so the user notices instead of the
job silently rubber-stamping a bad output.
3. dropped the post-execute ffprobe block. the preflight-skipped branch
no longer self-awards verified=1 either; it now does the same hand-
off so jellyfin's re-probe drives the ✓✓ in that branch too.
refreshItem's other two callers (review /rescan, subtitles /rescan)
ignore the return value — their semantics haven't changed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
worked through AUDIT.md. triage:
- finding 2 (subtitle rescan wipes decisions): confirmed. /:id/rescan now
snapshots custom_titles and calls reanalyze() after the stream delete/
insert, mirroring the review rescan flow. exported reanalyze + titleKey
from review.ts so both routes share the logic.
- finding 3 (scan limit accepts NaN/negatives): confirmed. extracted
parseScanLimit into a pure helper, added unit tests covering NaN,
negatives, floats, infinity, numeric strings. invalid input 400s and
releases the scan_running lock.
- finding 4 (parseId lenient): confirmed. tightened the regex to /^\d+$/
so "42abc", "abc42", "+42", "42.0" all return null. rewrote the test
that codified the old lossy behaviour.
- finding 5 (setup_complete set before jellyfin test passes): confirmed.
the /jellyfin endpoint still persists url+key unconditionally, but now
only flips setup_complete=1 on a successful connection test.
- finding 6 (swallowed errors): partial. the mqtt restart and version-
fetch swallows are intentional best-effort with downstream surfaces
(getMqttStatus, UI fallback). only the scan.ts db-update swallow was
a real visibility gap — logs via logError now.
- finding 1 (auth): left as-is. redacting secrets on GET without auth
on POST is security theater; real fix is an auth layer, which is a
design decision not a bugfix. audit removed from the tree.
- lint fail on ffmpeg.test.ts: formatted.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
monitoring the mqtt broker revealed two bugs and one design dead-end:
1. the jellyfin-plugin-webhook publishes pascalcase fields
(NotificationType, ItemId, ItemType) and we were reading camelcase
(event, itemId, itemType). every real payload was rejected by the
first guard — the mqtt path never ingested anything.
2. the plugin has no ItemUpdated / Library.* notifications. file
rewrites on existing items produce zero broker traffic (observed:
transcode + manual refresh metadata + 'recently added' appearance
→ no mqtt messages). ✓✓ via webhook is structurally impossible.
fix the webhook path so brand-new library items actually get ingested,
and narrow ACCEPTED_EVENTS to just 'ItemAdded' (the only library-side
event the plugin emits).
move the ✓✓ signal from webhook-corroboration to post-execute ffprobe
via the existing verifyDesiredState helper: after ffmpeg returns 0 we
probe the output file ourselves and flip verified=1 on match. the
preflight-skipped path sets verified=1 too. renamed the db column
webhook_verified → verified (via idempotent RENAME COLUMN migration)
since the signal is no longer webhook-sourced, and updated the Done
column tooltip to reflect that ffprobe is doing the verification.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
audio tracks now get a harmonized title on output (overriding any file
title like 'Audio Description' — review has already filtered out tracks
we don't want to keep). mono/stereo render numerically (1.0/2.0), matching
the .1-suffixed surround layouts. pipeline card rows become two-line so
long titles wrap instead of being clipped by the column.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
adds review_plans.webhook_verified, set to 1 whenever a fresh analysis
(scan or post-execute webhook) sees is_noop=1, cleared if a webhook
later flips the plan off-noop. resurrected the try/catch alter table
migration pattern in server/db/index.ts for the new column.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
two simplifications to how we pick and transcode the one-per-language
audio track, motivated by seeing inconsistent DTS → FLAC vs DTS →
EAC3 outputs in the wild:
transcode target:
- drop the FLAC path entirely. every incompatible source now targets
EAC3 regardless of container or lossless/lossy status
- FLAC for movie audio is bad value: ~2-3× the file size vs EAC3, no
Atmos spatial metadata (TrueHD Atmos → FLAC silently loses Atmos),
no AVR passthrough on Apple TV
- one target = no more container-conditional surprises
winner within a language group (betterAudio):
- new priority: highest channels → Apple-compatible → default → index
- old order put 'default' on top which forced a DTS-HD MA transcode
even when an AC3 track at equal channels was right next to it.
flipping means AC3 beats DTS-HD MA at the same channel count — pure
copy instead of a lossless-then-re-encode round trip
- channel count still dominates, so 7.1 TrueHD still beats 5.1 AC3
(and gets transcoded, which is the right call for real surround)
tests: new case for DTS-HD MA default + AC3 non-default at 5.1 → AC3
wins, job_type=copy. new case for 7.1 TrueHD beats 5.1 AC3 default.
every other existing test still holds.
a release with 2× english (main + director's commentary, or a surround
track plus an audio-description track) was keeping both. the user only
wants one per language. rules, in priority order:
- always drop commentary / audio-description / visually-impaired /
karaoke / sign-language tracks (matched by title regex + the
is_hearing_impaired flag)
- within each kept-language group, pick one winner by:
1. default disposition (main track the muxer chose)
2. highest channel count
3. apple-compatible codec (skip a transcode pass)
4. lowest stream_index for stability
tests cover: commentary dropped even when it matches OG, AD flag
dropped, default beats non-default, higher channels beat default-less
candidates of equal type, Apple-compat tiebreak, per-language dedupe
runs independently, and single-stream files stay noop.
two fixes based on actual behavior of the jellyfin webhook plugin:
- 'Webhook Url' setup value no longer re-serialized with mqtt://. show
the user's broker url verbatim so whatever protocol they use (ws://,
http://, etc.) survives the round trip
- dropped the server-side 'trigger a jellyfin rescan during the test'
machinery. a refresh that doesn't mutate metadata won't fire Item
Added, so relying on it produced false negatives. now we just wait
for any message on the topic; ui instructs the user to hit play on a
movie in jellyfin while the test runs — playback start is a
deterministic trigger, unlike library events
- setup panel now lists Notification Types as 'Item Added, Playback
Start'. playback start is for the test only; the production handler
still filters events down to item added / updated
- MqttSection now renders as a nested block inside the Jellyfin
ConnSection instead of its own card; ConnSection grew a children slot
- when the enable checkbox is off, broker/topic/credentials inputs and
the whole plugin setup panel are hidden; only the toggle + a small
save button remain
- 'Test Connection' became 'Test end-to-end': connects to the broker,
subscribes, picks a random scanned movie/episode, asks jellyfin to
refresh it, and waits for a matching webhook message. the UI walks
through all three steps (broker reachable → jellyfin rescan triggered
→ webhook received) with per-step success/failure so a broken
plugin config is obvious
- new mqtt_enabled config + toggle at top of the section; subscriber
only starts when the box is checked
- moved the whole MqttSection directly below the Jellyfin section so
all jellyfin-adjacent config lives together
- rewrote the plugin setup list to match the actual form order and
group it: 'Top of plugin page' (Server Url = jellyfin base URL),
'Generic destination', 'MQTT settings', 'Template'
- fields the user picks from a dropdown or toggles (Status,
Notification Type, Item Type, Use TLS, Use Credentials, QoS) now
render a 'select' hint instead of a broken Copy button
after ffmpeg finishes we used to block the queue on a jellyfin refresh
+ re-analyze round-trip. now we just kick jellyfin and return. a new
mqtt subscriber listens for library events from jellyfin's webhook
plugin and re-runs upsertJellyfinItem — flipping plans back to pending
when the on-disk streams still don't match, otherwise confirming done.
- execute.ts: hand-off is fire-and-forget; no more sync re-analyze
- rescan.ts: upsertJellyfinItem takes source: 'scan' | 'webhook'.
webhook-sourced rescans can reopen terminal 'done' plans when
is_noop flips back to 0; scan-sourced rescans still treat done as
terminal (keeps the dup-job fix from a06ab34 intact).
- mqtt.ts: long-lived client, auto-reconnect, status feed for UI badge
- webhook.ts: pure processWebhookEvent(db, deps) handler + 5s dedupe
map to kill jellyfin's burst re-fires during library scans
- settings: /api/settings/mqtt{,/status,/test} + /api/settings/
jellyfin/webhook-plugin (checks if the plugin is installed)
- ui: new Settings section with broker form, test button, copy-paste
setup panel for the Jellyfin plugin template. MQTT status badge on
the scan page.
root cause of duplicate pipeline entries: rescan.ts flipped done plans
back to pending whenever a post-job jellyfin refresh returned stale
metadata, putting the item back in review and letting a second jobs row
pile up in done. done is now sticky across rescans (error still
re-opens for retries).
second line of defense: before spawning ffmpeg, ffprobe the file and
compare audio count/language/codec order + embedded subtitle count
against the plan. if it already matches, mark the job done with the
reason in jobs.output and skip the spawn. prevents corrupting a
post-processed file with a stale stream-index command.
analyzer removes every subtitle unconditionally (see case 'Subtitle' in
decideAction) and the pipeline extracts all of them to sidecars — the config
was purely informational and only subtitles.ts echoed it back as
'keepLanguages' for a subtitle-manager ui that doesn't exist yet. we'll
revive language preferences inside that manager when it ships.
removes: the settings card + ui state, POST /api/settings/subtitle-languages,
the config default, the SUBTITLE_LANGUAGES env mapping, AnalyzerConfig's
subtitleLanguages field, RescanConfig's subtitleLanguages field, every
caller site (scan.ts / execute.ts / review.ts), and the keepLanguages
surface in subtitles.ts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
the old one-window scheduler gated only the job queue. now the scan loop and
the processing queue have independent windows — useful when the container
runs as an always-on service and we only want to hammer jellyfin + ffmpeg
at night.
config keys renamed from schedule_* to scan_schedule_* / process_schedule_*,
plus the existing job_sleep_seconds. scheduler.ts exposes parallel helpers
(isInScanWindow / isInProcessWindow, waitForScanWindow / waitForProcessWindow)
so each caller picks its window without cross-contamination.
scan.ts checks the scan window between items and emits paused/resumed sse.
execute.ts keeps its per-job pause + sleep-between-jobs but now on the
process window. /api/execute/scheduler moved to /api/settings/schedule.
frontend: ScheduleControls popup deleted from the pipeline header, replaced
with a plain Start queue button. settings page grows a Schedule section with
both windows and the job sleep input.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ffmpeg now writes -metadata:s:a:i language=<iso3> on every kept audio track so
files end up with canonical 3-letter tags (en → eng, ger → deu, null → und).
analyzer passes stream.profile (not title) to transcodeTarget so lossless
dts-hd ma in mkv correctly targets flac. is_noop also checks og-is-default and
canonical-language so pipeline-would-change-it cases stop showing as done.
normalizeLanguage gains 2→3 mapping, and mapStream no longer normalizes at
ingest so the raw jellyfin tag survives for the canonical check.
per-item scan work runs in a single db.transaction for large sqlite speedups,
extracted into server/services/rescan.ts so execute.ts can reuse it.
on successful job, execute calls jellyfin /Items/{id}/Refresh, waits for
DateLastRefreshed to change, refetches the item, and upserts it through the
same pipeline; plan flips to done iff the fresh streams satisfy is_noop.
schema wiped + rewritten to carry jellyfin_raw, external_raw, profile,
bit_depth, date_last_refreshed, runtime_ticks, original_title, last_executed_at
— so future scans aren't required to stay correct. user must drop data/*.db.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two regressions from the radarr/sonarr fix:
1. ERR_INVALID_URL spam — when radarr_enabled='1' but radarr_url is empty
or not http(s), every per-item fetch threw TypeError. We caught it but
still ate the cost (and the log noise) on every movie. New isUsable()
check on each service: enabled-in-config but URL doesn't parse →
warn ONCE and skip arr lookups for the whole scan.
2. Per-item HTTP storm — for movies not in Radarr's library we used to
hit /api/v3/movie (the WHOLE library) again per item, then two
metadata-lookup calls. With 2000 items that's thousands of extra
round-trips and the scan crawled. Now: pre-load the Radarr/Sonarr
library once into Map<tmdbId,..>+Map<imdbId,..>+Map<tvdbId,..>,
per-item lookups are O(1) memory hits, and only the genuinely-missing
items make a single lookup-endpoint HTTP call.
The startup line now reports the library size:
External language sources: radarr=enabled (https://..., 1287 movies in library), sonarr=...
so you can immediately see whether the cache loaded.
The real reason 8 Mile landed as Turkish: Radarr WAS being called, but the
call path had three silent failure modes that all looked identical from
outside.
1. try { … } catch { return null } swallowed every error. No log when
Radarr was unreachable, when the API key was wrong, when HTTP returned
404/500, or when JSON parsing failed. A miss and a crash looked the
same: null, fall back to Jellyfin's dub guess.
2. /api/v3/movie?tmdbId=X only queries Radarr's LIBRARY. If the movie is
on disk + in Jellyfin but not actively managed in Radarr, returns [].
We then gave up and used the Jellyfin guess.
3. iso6391To6392 fell back to normalizeLanguage(name.slice(0, 3)) for any
unknown language name — pretending 'Mandarin' → 'man' and 'Flemish' →
'fle' are valid ISO 639-2 codes.
Fixes:
- Both services: fetchJson helper logs HTTP errors with context and the
url (api key redacted), plus catches+logs thrown errors.
- Added a metadata-lookup fallback: /api/v3/movie/lookup/tmdb and
/lookup/imdb for Radarr, /api/v3/series/lookup?term=tvdb:X for Sonarr.
These hit TMDB/TVDB via the arr service for titles not in its library.
- Expanded NAME_TO_639_2: Mandarin/Cantonese → zho, Flemish → nld,
Farsi → fas, plus common European langs that were missing.
- Unknown name → return null (log a warning) instead of a made-up 3-char
code. scan.ts then marks needs_review.
- scan.ts: per-item warn when Radarr/Sonarr miss; per-scan summary line
showing hits/misses/no-provider-id tallies.
Run a scan — the logs will now tell you whether Radarr was called, what
it answered, and why it fell back if it did.
Two bugs compounded:
1. extractOriginalLanguage() in jellyfin.ts picked the FIRST audio stream's
language and called it 'original'. Files sourced from non-English regions
often have a local dub as track 0, so 8 Mile with a Turkish dub first
got labelled Turkish.
2. scan.ts promoted any single-source answer to confidence='high' — even
the pure Jellyfin guess, as long as no second source (Radarr/Sonarr)
contradicted it. Jellyfin's dub-magnet guess should never be green.
Fixes:
- extractOriginalLanguage now prefers the IsDefault audio track and skips
tracks whose title shouts 'dub' / 'commentary' / 'director'. Still a
heuristic, but much less wrong. Fallback to the first track when every
candidate looks like a dub so we have *something* to flag.
- scan.ts: high confidence requires an authoritative source (Radarr/Sonarr)
with no conflict. A Jellyfin-only answer is always low confidence AND
gets needs_review=1 so it surfaces in the pipeline for manual override.
- Data migration (idempotent): downgrade existing plans backed only by the
Jellyfin heuristic to low confidence and mark needs_review=1, so users
don't have to rescan to benefit.
- New server/services/__tests__/jellyfin.test.ts covers the default-track
preference and dub-skip behavior.
Bug: every approve path (buildCommand used by review approve/approve-all/
series approve-all/season approve-all/retry/detail preview) was building
an ffmpeg command that -map'd only the 'keep' streams and dropped all
subtitles. For a file like Wuthering Heights with 37 embedded subs, the
run would delete every sub into the void — user expected extraction to
sidecar files per the pipeline contract.
buildPipelineCommand already did the right thing (extract every subtitle
with -map 0:s:N -c:s copy 'basename.lang.srt', then remux kept streams)
but it was only reached by tests. buildCommand now delegates to it — one
call site, subtitle extraction always runs, predictExtractedFiles records
the sidecar paths after job success (same logic, same basePath).
Added a regression test: buildCommand on a 2-subtitle file contains
-map 0:s:0, -map 0:s:1 and the expected 'basename.en.srt'/'.de.srt' paths.
Subtitle extraction lives only in the pipeline now; a file is 'done' when it
matches the desired end state — no embedded subs AND audio matches the
language config. The separate Extract page was redundant.
- delete src/routes/review/subtitles/extract.tsx + SubtitleExtractPage
- delete /api/subtitles/extract-all + /:id/extract endpoints
- delete buildExtractOnlyCommand + unused buildExtractionOutputs from ffmpeg.ts
- detail page: drop Extract button + extractCommand textarea, replace with
'will be extracted via pipeline' note when embedded subs present
- pipeline endpoint: doneCount = is_noop OR status='done' (a file in the
desired state, however it got there); UI label 'N files in desired state'
- nav: drop the now-defunct 'Extract subs' link, default activeOptions.exact
to false so detail subpages (e.g. /review/audio/123) highlight their
parent ('Audio') in the menu — was the cause of the broken-feeling menu
- execute: actually call isInScheduleWindow/waitForWindow/sleepBetweenJobs in runSequential (they were dead code); emit queue_status SSE events (running/paused/sleeping/idle) so the pipeline's existing QueueStatus listener lights up
- review: POST /:id/retry resets an errored plan to approved, wipes old done/error jobs, rebuilds command from current decisions, queues fresh job
- scan: dev-mode DELETE now also wipes jobs + subtitle_files (previously orphaned after every dev reset)
- biome: migrate config to 2.4 schema, autoformat 68 files (strings + indentation), relax opinionated a11y/hooks-deps/index-key rules that don't fit this codebase
- routeTree.gen.ts regenerated after /nodes removal
- analyzer: rewrite checkAudioOrderChanged to compare actual output order, unify assignTargetOrder with a shared sortKeptStreams util in ffmpeg builder
- review: recompute is_noop via full audio removed/reordered/transcode/subs check on toggle, preserve custom_title across rescan by matching (type,lang,stream_index,title), batch pipeline transcode-reasons query to avoid N+1
- validate: add lib/validate.ts with parseId + isOneOf helpers; replace bare Number(c.req.param('id')) with 400 on invalid ids across review/subtitles
- scan: atomic CAS on scan_running config to prevent concurrent scans
- subtitles: path-traversal guard — only unlink sidecars within the media item's directory; log-and-orphan DB entries pointing outside
- schedule: include end minute in window (<= vs <)
- db: add indexes on review_plans(status,is_noop), stream_decisions(plan_id), media_items(series_jellyfin_id,series_name,type), media_streams(item_id,type), subtitle_files(item_id), jobs(status,item_id)
- remove nodes table, ssh service, nodes api, NodesPage route
- execute.ts: local-only spawn, atomic CAS job claim via UPDATE status
- wrap job done + subtitle_files insert + review_plans status in db transaction
- stream ffmpeg output per line with 500ms throttled flush
- bump version to 2026.04.13
- server-side filter + LIMIT 200 + totalCounts on GET /api/execute
- shared FilterTabs component with status-colored active tabs
- execute page: filter tabs, SSE live count updates, module-level cache
- replace inline tab pills in AudioListPage, SubtitleListPage with FilterTabs
- fix buildExtractOnlyCommand: skip -map 0:a when no audio streams exist
- bump version
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
rewrite from monolithic hono jsx to react 19 spa with tanstack router
+ hono json api backend. add scan, review, execute, nodes, and setup
pages. multi-stage dockerfile (node for vite build, bun for runtime).
previously, server/ and src/shared/lib/ were silently excluded by
global gitignore patterns (/server/ from emacs, lib/ from python).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
felixfoertsch