Commit Graph

29 Commits

Author SHA1 Message Date
a2bdecd298 rework scan page, add ingest-source browsing, bump version to 2026.04.15.8
All checks were successful
Build and Push Docker Image / build (push) Successful in 4m56s
2026-04-15 18:33:08 +02:00
c6698db51a improve running-job responsiveness, bump version to 2026.04.15.7
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m41s
2026-04-15 16:58:53 +02:00
e040c9a234 settings: mask API keys in GET /api/settings, add eye-icon reveal
GET /api/settings now returns jellyfin_api_key, radarr_api_key,
sonarr_api_key, mqtt_password as "***" when set (empty string when
unset). Real values only reach the client via an explicit
GET /api/settings/reveal?key=<key> call, wired to an eye icon on
each secret input in the Settings page.

Save endpoints treat an incoming "***" as a sentinel meaning "user
didn't touch this field, keep stored value", so saving without
revealing preserves the existing secret.

Addresses audit finding #3 (settings endpoint leaks secrets).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 08:15:08 +02:00
b0d06a1d8c execute: drain newly-approved jobs mid-run, use strict shared parseId
Queue previously processed a snapshot of pending jobs — anything approved
after Run-all clicked sat idle until the user clicked again. Now, when
the local queue drains, re-poll the DB once for newly-approved jobs
before exiting.

Also swap the looser local parseId (Number.parseInt accepted '42abc')
for the strict shared parseId in server/lib/validate.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 08:12:59 +02:00
0d6781973b rip out jellyfin handoff verification path and verify-unverified endpoint
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 06:58:55 +02:00
0e53640b94 done column: 'verify N' header button to backfill ✓ → ✓✓
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m5s
new POST /api/execute/verify-unverified that picks every plan with
status=done + verified=0 and runs handOffToJellyfin sequentially in
the background. each handoff fires the existing plan_update sse so
the done column promotes cards as jellyfin's verdict lands. exported
handOffToJellyfin so the route can reuse the same flow as a fresh job.

done column header shows a 'Verify N' action whenever there are
unverified done plans, alongside the existing 'Clear'. one click and
the user can backfill ✓✓ across every legacy done item without
re-transcoding.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 20:59:28 +02:00
51d56a4082 push verified=1 to the UI via a plan_update SSE event
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m5s
the ✓✓ write was landing in the db but never reaching the browser.
job_update fires once at job completion (card renders ✓, verified=0),
then handOffToJellyfin takes ~15s to refresh jellyfin + re-analyze +
UPDATE review_plans SET verified=1. no further sse, so the pipeline
page never re-polled and the card stayed at ✓ until the user
navigated away and back.

new plan_update event emitted at the end of handOffToJellyfin. the
pipeline page listens and triggers the same 1s-coalesced reload as
job_update, so the done column promotes ✓ → ✓✓ within a second of
jellyfin's verdict landing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 20:51:47 +02:00
d2983d5f38 skip non-extractable subs (dvdsub/dvbsub/unknown), summarise ffmpeg errors
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m30s
Abraham Lincoln crashed with exit 234 because the file had 14 dvd_subtitle
streams: our extraction dict only keyed on the long form (dvd_subtitle)
while jellyfin stores the short form (dvdsub), so the lookup fell back
to .srt, ffmpeg picked the srt muxer, and srt can't encode image-based
subs. textbook silent dict miss.

replaced the extension dict with an EXTRACTABLE map that pairs codec →
{ext, codecArg} and explicitly enumerates every codec we can route to a
single-file sidecar. everything else (dvd_subtitle/dvdsub, dvb_subtitle/
dvbsub, unknown codecs) is now skipped at command-build time. the plan
picks up a note like '14 subtitle(s) dropped: dvdsub (eng, est, ind,
kor, jpn, lav, lit, may, chi, chi, tha, vie, rus, ukr) — not extractable
to sidecar' so the user sees exactly what didn't make it.

also added extractErrorSummary in execute.ts: when a job errors, scan
the last 60 stderr lines for fatal keywords (Error:, Conversion failed!,
Unsupported, Invalid argument, Permission denied, No space left, …),
dedupe, prepend the summary to the job's stored output. the review_plan
notes get the same summary — surfaces the real cause next to the plan
instead of burying it under ffmpeg's 200-line banner.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 18:42:05 +02:00
afd95f06df ✓✓ is now jellyfin-corroborated, not a self-confirming ffprobe
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m11s
user reported ad astra got the double checkmark instantly after
transcode — correct, and correct to flag: the post-execute
verifyDesiredState ran ffprobe on the file we had just written, so it
tautologically matched the plan every time. not a second opinion.

replaced the flow with the semantics we actually wanted:

1. refreshItem now returns { refreshed: boolean } — true when jellyfin's
   DateLastRefreshed actually advanced within the timeout, false when it
   didn't. callers can tell 'jellyfin really re-probed' apart from
   'we timed out waiting'.

2. handOffToJellyfin post-job: refresh → (only if refreshed=true) fetch
   fresh streams → upsertJellyfinItem(source='webhook'). the rescan SQL
   sets verified=1 exactly when the fresh analysis sees is_noop=1, so
   ✓✓ now means 'jellyfin independently re-probed the file we wrote
   and agrees it matches the plan'. if jellyfin sees a drifted layout
   the plan flips back to pending so the user notices instead of the
   job silently rubber-stamping a bad output.

3. dropped the post-execute ffprobe block. the preflight-skipped branch
   no longer self-awards verified=1 either; it now does the same hand-
   off so jellyfin's re-probe drives the ✓✓ in that branch too.

refreshItem's other two callers (review /rescan, subtitles /rescan)
ignore the return value — their semantics haven't changed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 18:29:00 +02:00
d05e037bbc webhook: PascalCase payload + ItemAdded only, switch ✓✓ signal to ffprobe
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m56s
monitoring the mqtt broker revealed two bugs and one design dead-end:

1. the jellyfin-plugin-webhook publishes pascalcase fields
   (NotificationType, ItemId, ItemType) and we were reading camelcase
   (event, itemId, itemType). every real payload was rejected by the
   first guard — the mqtt path never ingested anything.

2. the plugin has no ItemUpdated / Library.* notifications. file
   rewrites on existing items produce zero broker traffic (observed:
   transcode + manual refresh metadata + 'recently added' appearance
   → no mqtt messages). ✓✓ via webhook is structurally impossible.

fix the webhook path so brand-new library items actually get ingested,
and narrow ACCEPTED_EVENTS to just 'ItemAdded' (the only library-side
event the plugin emits).

move the ✓✓ signal from webhook-corroboration to post-execute ffprobe
via the existing verifyDesiredState helper: after ffmpeg returns 0 we
probe the output file ourselves and flip verified=1 on match. the
preflight-skipped path sets verified=1 too. renamed the db column
webhook_verified → verified (via idempotent RENAME COLUMN migration)
since the signal is no longer webhook-sourced, and updated the Done
column tooltip to reflect that ffprobe is doing the verification.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 17:27:22 +02:00
a27e4f4025 close the jellyfin ping-pong via mqtt webhook subscriber
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m5s
after ffmpeg finishes we used to block the queue on a jellyfin refresh
+ re-analyze round-trip. now we just kick jellyfin and return. a new
mqtt subscriber listens for library events from jellyfin's webhook
plugin and re-runs upsertJellyfinItem — flipping plans back to pending
when the on-disk streams still don't match, otherwise confirming done.

- execute.ts: hand-off is fire-and-forget; no more sync re-analyze
- rescan.ts: upsertJellyfinItem takes source: 'scan' | 'webhook'.
  webhook-sourced rescans can reopen terminal 'done' plans when
  is_noop flips back to 0; scan-sourced rescans still treat done as
  terminal (keeps the dup-job fix from a06ab34 intact).
- mqtt.ts: long-lived client, auto-reconnect, status feed for UI badge
- webhook.ts: pure processWebhookEvent(db, deps) handler + 5s dedupe
  map to kill jellyfin's burst re-fires during library scans
- settings: /api/settings/mqtt{,/status,/test} + /api/settings/
  jellyfin/webhook-plugin (checks if the plugin is installed)
- ui: new Settings section with broker form, test button, copy-paste
  setup panel for the Jellyfin plugin template. MQTT status badge on
  the scan page.
2026-04-14 08:26:42 +02:00
a06ab34b98 make done plans terminal, add ffprobe preflight to skip already-processed files
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m34s
root cause of duplicate pipeline entries: rescan.ts flipped done plans
back to pending whenever a post-job jellyfin refresh returned stale
metadata, putting the item back in review and letting a second jobs row
pile up in done. done is now sticky across rescans (error still
re-opens for retries).

second line of defense: before spawning ffmpeg, ffprobe the file and
compare audio count/language/codec order + embedded subtitle count
against the plan. if it already matches, mark the job done with the
reason in jobs.output and skip the spawn. prevents corrupting a
post-processed file with a stale stream-index command.
2026-04-13 21:43:10 +02:00
6d8a8fa6d6 drop the subtitle-languages setting, it never influenced extraction
All checks were successful
Build and Push Docker Image / build (push) Successful in 53s
analyzer removes every subtitle unconditionally (see case 'Subtitle' in
decideAction) and the pipeline extracts all of them to sidecars — the config
was purely informational and only subtitles.ts echoed it back as
'keepLanguages' for a subtitle-manager ui that doesn't exist yet. we'll
revive language preferences inside that manager when it ships.

removes: the settings card + ui state, POST /api/settings/subtitle-languages,
the config default, the SUBTITLE_LANGUAGES env mapping, AnalyzerConfig's
subtitleLanguages field, RescanConfig's subtitleLanguages field, every
caller site (scan.ts / execute.ts / review.ts), and the keepLanguages
surface in subtitles.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 15:26:48 +02:00
23dca8bf0b split scheduling into scan + process windows, move controls to settings page
Some checks failed
Build and Push Docker Image / build (push) Failing after 8s
the old one-window scheduler gated only the job queue. now the scan loop and
the processing queue have independent windows — useful when the container
runs as an always-on service and we only want to hammer jellyfin + ffmpeg
at night.

config keys renamed from schedule_* to scan_schedule_* / process_schedule_*,
plus the existing job_sleep_seconds. scheduler.ts exposes parallel helpers
(isInScanWindow / isInProcessWindow, waitForScanWindow / waitForProcessWindow)
so each caller picks its window without cross-contamination.

scan.ts checks the scan window between items and emits paused/resumed sse.
execute.ts keeps its per-job pause + sleep-between-jobs but now on the
process window. /api/execute/scheduler moved to /api/settings/schedule.

frontend: ScheduleControls popup deleted from the pipeline header, replaced
with a plain Start queue button. settings page grows a Schedule section with
both windows and the job sleep input.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 14:50:25 +02:00
6fcaeca82c write canonical iso3 language metadata, tighten is_noop, store full jellyfin data
Some checks failed
Build and Push Docker Image / build (push) Failing after 16s
ffmpeg now writes -metadata:s:a:i language=<iso3> on every kept audio track so
files end up with canonical 3-letter tags (en → eng, ger → deu, null → und).
analyzer passes stream.profile (not title) to transcodeTarget so lossless
dts-hd ma in mkv correctly targets flac. is_noop also checks og-is-default and
canonical-language so pipeline-would-change-it cases stop showing as done.

normalizeLanguage gains 2→3 mapping, and mapStream no longer normalizes at
ingest so the raw jellyfin tag survives for the canonical check.

per-item scan work runs in a single db.transaction for large sqlite speedups,
extracted into server/services/rescan.ts so execute.ts can reuse it.

on successful job, execute calls jellyfin /Items/{id}/Refresh, waits for
DateLastRefreshed to change, refetches the item, and upserts it through the
same pipeline; plan flips to done iff the fresh streams satisfy is_noop.

schema wiped + rewritten to carry jellyfin_raw, external_raw, profile,
bit_depth, date_last_refreshed, runtime_ticks, original_title, last_executed_at
— so future scans aren't required to stay correct. user must drop data/*.db.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 13:56:19 +02:00
4a378eb833 pipeline: equal-width columns + per-column clear/stop button
All checks were successful
Build and Push Docker Image / build (push) Successful in 39s
Extract a ColumnShell component so all four columns share the same flex-1
basis-0 width (no more 24/16/18/16 rem mix) and the same header layout
(title + count + optional action button on the right).

Per-column actions:
- Review:     'Skip all' → POST /api/review/skip-all (new endpoint, sets all
              pending non-noop plans to skipped in one update)
- Queued:     'Clear'    → POST /api/execute/clear (existing; cancels pending jobs)
- Processing: 'Stop'     → POST /api/execute/stop (new; SIGTERMs the running
              ffmpeg via a tracked Bun.spawn handle, runJob's catch path
              marks the job error and cleans up)
- Done:       'Clear'    → POST /api/execute/clear-completed (existing)

All destructive actions confirm before firing.
2026-04-13 10:08:42 +02:00
874f04b7a5 wire scheduler into queue, add retry, dev-reset cleanup, biome 2.4 migrate
- execute: actually call isInScheduleWindow/waitForWindow/sleepBetweenJobs in runSequential (they were dead code); emit queue_status SSE events (running/paused/sleeping/idle) so the pipeline's existing QueueStatus listener lights up
- review: POST /:id/retry resets an errored plan to approved, wipes old done/error jobs, rebuilds command from current decisions, queues fresh job
- scan: dev-mode DELETE now also wipes jobs + subtitle_files (previously orphaned after every dev reset)
- biome: migrate config to 2.4 schema, autoformat 68 files (strings + indentation), relax opinionated a11y/hooks-deps/index-key rules that don't fit this codebase
- routeTree.gen.ts regenerated after /nodes removal
2026-04-13 07:41:19 +02:00
f11861658e add bun:test coverage for analyzer + ffmpeg + validate, emit ffmpeg progress sse
- analyzer.test.ts: audio keep rules (OG + configured langs, unknown OG, undetermined lang, iso alias), ordering (OG first, reorder noop), subtitle forced-remove, transcode targets
- ffmpeg.test.ts: shellQuote, sortKeptStreams canonical order, buildCommand tmp+mv, type-relative maps (0:a:N), disposition, buildPipelineCommand sub extraction + transcode bitrate, predictExtractedFiles dedup
- validate.test.ts: parseId bounds + isOneOf narrowing
- execute: parse ffmpeg Duration + time, emit job_progress SSE events throttled at 500ms so ProcessingColumn progress bar fills in (it already listened)
- package: switch test script from placeholder echo to 'bun test'
2026-04-13 07:35:24 +02:00
cdcb1ff706 drop multi-node ssh execution, unify job runner to local + fix job completion atomicity
- remove nodes table, ssh service, nodes api, NodesPage route
- execute.ts: local-only spawn, atomic CAS job claim via UPDATE status
- wrap job done + subtitle_files insert + review_plans status in db transaction
- stream ffmpeg output per line with 500ms throttled flush
- bump version to 2026.04.13
2026-04-13 07:25:19 +02:00
9a19350f7e add job scheduler: sleep between jobs, schedule window, FFmpeg progress parsing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 01:46:41 +01:00
99274d3ae8 add execute page filtering + colored FilterTabs component, fix ffmpeg audio-less files
- server-side filter + LIMIT 200 + totalCounts on GET /api/execute
- shared FilterTabs component with status-colored active tabs
- execute page: filter tabs, SSE live count updates, module-level cache
- replace inline tab pills in AudioListPage, SubtitleListPage with FilterTabs
- fix buildExtractOnlyCommand: skip -map 0:a when no audio streams exist
- bump version

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 12:01:27 +01:00
588a3d8f1f remove subtitle streams from container after extraction, remove job list limit, fix audio detail display
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m9s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 09:17:39 +01:00
39a924e6de log ffmpeg command to stdout for docker/unraid log viewer
All checks were successful
Build and Push Docker Image / build (push) Successful in 17s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 17:47:11 +01:00
41476a2f9b run jobs sequentially per target, parallel across local + nodes
All checks were successful
Build and Push Docker Image / build (push) Successful in 19s
"run all" now groups pending jobs by target (local or node), runs them one by
one within each group, but runs different targets in parallel. single "run"
button still fires immediately.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 17:44:42 +01:00
3682ee98e0 add structured logging with timestamps for docker/unraid log viewer
All checks were successful
Build and Push Docker Image / build (push) Successful in 15s
all server output now prefixed with ISO timestamp and level [INFO/WARN/ERROR].
logs requests, scan start/complete, job lifecycle, errors. skips noisy SSE
endpoints.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 17:29:22 +01:00
818b0d1396 add version badge in nav, apply path mappings at execution time, clear done/error jobs
All checks were successful
Build and Push Docker Image / build (push) Successful in 58s
- show version (from package.json) in nav bar, warn on frontend/server mismatch
- apply path_mappings to file access check and command string at execution time
  so existing scans with old jellyfin paths work without re-scanning
- add clear done/errors button on execute page
- bump version to 2026.03.04

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 17:22:14 +01:00
37fae33bbc add clear queue button on execute page
All checks were successful
Build and Push Docker Image / build (push) Successful in 20s
deletes all pending jobs, reverts their review plans back to pending so they
can be re-reviewed and re-approved.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 17:11:18 +01:00
d5f4afd26b split audio/subtitle concerns, remove docker-in-docker, add per-node path mapping
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m54s
- install ffmpeg in dockerfile (fixes exit code 127)
- buildCommand() now audio-only remux, no subtitle extraction
- add unapprove endpoint + ui button for approved items
- add batch extract-all subtitles endpoint + ui button
- audio detail page shows only video+audio streams
- remove global movies_path/series_path config, add per-node path mapping
- remove docker-in-docker command building (buildDockerCommand, buildDockerExtractOnlyCommand)
- ssh execution translates /movies/ and /series/ to node-specific paths
- remove media paths section from setup page
- add unraid-template.xml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 16:48:00 +01:00
5ac44b7551 restructure to react spa + hono api, fix missing server/ and lib/
rewrite from monolithic hono jsx to react 19 spa with tanstack router
+ hono json api backend. add scan, review, execute, nodes, and setup
pages. multi-stage dockerfile (node for vite build, bun for runtime).

previously, server/ and src/shared/lib/ were silently excluded by
global gitignore patterns (/server/ from emacs, lib/ from python).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 22:57:40 +01:00