Commit Graph

25 Commits

Author SHA1 Message Date
felixfoertsch 686434f5c3 remove jellyfin, mqtt, webhook services, fix tests, add schema migrations
- delete server/services/jellyfin.ts, webhook.ts, mqtt.ts and their tests
- strip jellyfin/mqtt imports and startup calls from index.tsx and settings.ts
- remove /jellyfin, /mqtt, /mqtt/status, /mqtt/test, /jellyfin/webhook-plugin endpoints from settings router
- clean ENV_MAP and isEnvConfigured of jellyfin/mqtt keys
- add db/index.ts migrations for series_key, duration_seconds, scan_status, scan_error, last_scanned_at (new columns absent on older dev DBs)
- move idx_media_items_series_key out of SCHEMA into migrate() so it runs after the column is added
- fix all test fixtures: drop jellyfin_id/series_jellyfin_id column refs, update MediaItem/MediaStream object literals to match current types

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 06:31:54 +02:00
felixfoertsch 9b04ed87d2 new schema: drop jellyfin columns, file_path as unique key
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 06:31:21 +02:00
felixfoertsch 8112bfeb65 per-track language override on audio detail page
Build and Push Docker Image / build (push) Successful in 3m3s
adds stream_decisions.custom_language (ISO 639-2 code or null) so the
user can correct a mislabeled audio track — e.g. a Spanish dub tagged
"und" in the container — without going through Jellyfin. the override
wins over stream.language everywhere it matters: the analyzer reads it
for keep/remove decisions and track ordering, the ffmpeg command builder
writes it as both the language metadata tag and the harmonized track
title, and reanalyze preserves it across reruns and rescans.

on the audio detail page, each pending audio row swaps its language
cell for an inline <select> populated from LANG_NAMES. picking the raw
file language clears the override; anything else sets it and triggers a
server-side reanalyze so keep/remove + target_index update immediately.
a small ✎ hint marks overridden tracks. rebuilt commands tag the output
accordingly so Jellyfin reads the corrected language.

PATCH /api/review/:id/stream/:streamId/language validates the code
against LANG_NAMES (accepts ISO 639-1/2/2B aliases, rejects garbage)
and runs reanalyze inside.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 00:05:31 +02:00
felixfoertsch 05a1345750 rip subtitle manager → sibling project, keep extraction only
Build and Push Docker Image / build (push) Successful in 2m31s
subtitle management (list/detail pages, /api/subtitles, subtitle_files
table, SubtitleFile types, predictExtractedFiles, nav link) moved to a
new sibling project at ~/Developer/netfelix-subtitles-manager/ where
it'll be rebuilt standalone later. this project now owns audio fixing
+ subtitle extraction only.

extraction still runs end to end: analyzeItem still marks every subtitle
stream as "remove from container", buildExtractionOutputs still wires
the -map 0:s:N + sidecar outputs into the ffmpeg command, and execute.ts
still flips review_plans.subs_extracted so verify.ts can check desired
state — just derived from the streams directly instead of by writing a
row per file to the now-gone subtitle_files table.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 22:50:52 +02:00
felixfoertsch 0fd3624d9f pipeline: uniform column headers, auto-process queue toggle, reopen → inbox
Build and Push Docker Image / build (push) Successful in 4m3s
column headers are now a fixed three-row layout (title / subtitle / button
row). every column always reserves all three rows so headers line up
regardless of contents; actions render disabled when their column is
empty instead of disappearing, which keeps the header height stable as
state changes.

the processing column gets a new "Auto-process Queue" checkbox that
mirrors the inbox's "Auto-process Inbox" toggle. backend adds an
auto_process_queue config, a maybeStartQueueProcessor() helper, a
POST /api/settings/auto-process-queue endpoint, and a hook in
enqueueAudioJob so approvals drain the queue hands-off when the toggle
is on.

reopen-all and per-item reopen now send items to the Inbox (sorted=0)
instead of back to Review. the done column's label and tooltip become
"← Back to inbox" to match, and the clear button moves to the right
slot so the header pattern (left=backward, right=forward) stays
consistent across columns.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 21:57:13 +02:00
felixfoertsch ef417bea09 schema: add auto_class + sorted to review_plans, drop confidence
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 10:10:40 +02:00
felixfoertsch a2bdecd298 rework scan page, add ingest-source browsing, bump version to 2026.04.15.8
Build and Push Docker Image / build (push) Successful in 4m56s
2026-04-15 18:33:08 +02:00
felixfoertsch cbf0025a81 drop review_plans.verified column and all its references
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 06:55:43 +02:00
felixfoertsch d05e037bbc webhook: PascalCase payload + ItemAdded only, switch ✓✓ signal to ffprobe
Build and Push Docker Image / build (push) Successful in 1m56s
monitoring the mqtt broker revealed two bugs and one design dead-end:

1. the jellyfin-plugin-webhook publishes pascalcase fields
   (NotificationType, ItemId, ItemType) and we were reading camelcase
   (event, itemId, itemType). every real payload was rejected by the
   first guard — the mqtt path never ingested anything.

2. the plugin has no ItemUpdated / Library.* notifications. file
   rewrites on existing items produce zero broker traffic (observed:
   transcode + manual refresh metadata + 'recently added' appearance
   → no mqtt messages). ✓✓ via webhook is structurally impossible.

fix the webhook path so brand-new library items actually get ingested,
and narrow ACCEPTED_EVENTS to just 'ItemAdded' (the only library-side
event the plugin emits).

move the ✓✓ signal from webhook-corroboration to post-execute ffprobe
via the existing verifyDesiredState helper: after ffmpeg returns 0 we
probe the output file ourselves and flip verified=1 on match. the
preflight-skipped path sets verified=1 too. renamed the db column
webhook_verified → verified (via idempotent RENAME COLUMN migration)
since the signal is no longer webhook-sourced, and updated the Done
column tooltip to reflect that ffprobe is doing the verification.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 17:27:22 +02:00
felixfoertsch 027ea498c3 done column: double-checkmark when jellyfin webhook corroborates the plan
Build and Push Docker Image / build (push) Successful in 1m37s
adds review_plans.webhook_verified, set to 1 whenever a fresh analysis
(scan or post-execute webhook) sees is_noop=1, cleared if a webhook
later flips the plan off-noop. resurrected the try/catch alter table
migration pattern in server/db/index.ts for the new column.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 15:50:57 +02:00
felixfoertsch 9bb46ae968 mqtt setup panel: gate on enable toggle, reorder, move next to jellyfin
Build and Push Docker Image / build (push) Successful in 52s
- new mqtt_enabled config + toggle at top of the section; subscriber
  only starts when the box is checked
- moved the whole MqttSection directly below the Jellyfin section so
  all jellyfin-adjacent config lives together
- rewrote the plugin setup list to match the actual form order and
  group it: 'Top of plugin page' (Server Url = jellyfin base URL),
  'Generic destination', 'MQTT settings', 'Template'
- fields the user picks from a dropdown or toggles (Status,
  Notification Type, Item Type, Use TLS, Use Credentials, QoS) now
  render a 'select' hint instead of a broken Copy button
2026-04-14 09:26:43 +02:00
felixfoertsch a27e4f4025 close the jellyfin ping-pong via mqtt webhook subscriber
Build and Push Docker Image / build (push) Successful in 1m5s
after ffmpeg finishes we used to block the queue on a jellyfin refresh
+ re-analyze round-trip. now we just kick jellyfin and return. a new
mqtt subscriber listens for library events from jellyfin's webhook
plugin and re-runs upsertJellyfinItem — flipping plans back to pending
when the on-disk streams still don't match, otherwise confirming done.

- execute.ts: hand-off is fire-and-forget; no more sync re-analyze
- rescan.ts: upsertJellyfinItem takes source: 'scan' | 'webhook'.
  webhook-sourced rescans can reopen terminal 'done' plans when
  is_noop flips back to 0; scan-sourced rescans still treat done as
  terminal (keeps the dup-job fix from a06ab34 intact).
- mqtt.ts: long-lived client, auto-reconnect, status feed for UI badge
- webhook.ts: pure processWebhookEvent(db, deps) handler + 5s dedupe
  map to kill jellyfin's burst re-fires during library scans
- settings: /api/settings/mqtt{,/status,/test} + /api/settings/
  jellyfin/webhook-plugin (checks if the plugin is installed)
- ui: new Settings section with broker form, test button, copy-paste
  setup panel for the Jellyfin plugin template. MQTT status badge on
  the scan page.
2026-04-14 08:26:42 +02:00
felixfoertsch 6d8a8fa6d6 drop the subtitle-languages setting, it never influenced extraction
Build and Push Docker Image / build (push) Successful in 53s
analyzer removes every subtitle unconditionally (see case 'Subtitle' in
decideAction) and the pipeline extracts all of them to sidecars — the config
was purely informational and only subtitles.ts echoed it back as
'keepLanguages' for a subtitle-manager ui that doesn't exist yet. we'll
revive language preferences inside that manager when it ships.

removes: the settings card + ui state, POST /api/settings/subtitle-languages,
the config default, the SUBTITLE_LANGUAGES env mapping, AnalyzerConfig's
subtitleLanguages field, RescanConfig's subtitleLanguages field, every
caller site (scan.ts / execute.ts / review.ts), and the keepLanguages
surface in subtitles.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 15:26:48 +02:00
felixfoertsch 23dca8bf0b split scheduling into scan + process windows, move controls to settings page
Build and Push Docker Image / build (push) Failing after 8s
the old one-window scheduler gated only the job queue. now the scan loop and
the processing queue have independent windows — useful when the container
runs as an always-on service and we only want to hammer jellyfin + ffmpeg
at night.

config keys renamed from schedule_* to scan_schedule_* / process_schedule_*,
plus the existing job_sleep_seconds. scheduler.ts exposes parallel helpers
(isInScanWindow / isInProcessWindow, waitForScanWindow / waitForProcessWindow)
so each caller picks its window without cross-contamination.

scan.ts checks the scan window between items and emits paused/resumed sse.
execute.ts keeps its per-job pause + sleep-between-jobs but now on the
process window. /api/execute/scheduler moved to /api/settings/schedule.

frontend: ScheduleControls popup deleted from the pipeline header, replaced
with a plain Start queue button. settings page grows a Schedule section with
both windows and the job sleep input.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 14:50:25 +02:00
felixfoertsch 6fcaeca82c write canonical iso3 language metadata, tighten is_noop, store full jellyfin data
Build and Push Docker Image / build (push) Failing after 16s
ffmpeg now writes -metadata:s:a:i language=<iso3> on every kept audio track so
files end up with canonical 3-letter tags (en → eng, ger → deu, null → und).
analyzer passes stream.profile (not title) to transcodeTarget so lossless
dts-hd ma in mkv correctly targets flac. is_noop also checks og-is-default and
canonical-language so pipeline-would-change-it cases stop showing as done.

normalizeLanguage gains 2→3 mapping, and mapStream no longer normalizes at
ingest so the raw jellyfin tag survives for the canonical check.

per-item scan work runs in a single db.transaction for large sqlite speedups,
extracted into server/services/rescan.ts so execute.ts can reuse it.

on successful job, execute calls jellyfin /Items/{id}/Refresh, waits for
DateLastRefreshed to change, refetches the item, and upserts it through the
same pipeline; plan flips to done iff the fresh streams satisfy is_noop.

schema wiped + rewritten to carry jellyfin_raw, external_raw, profile,
bit_depth, date_last_refreshed, runtime_ticks, original_title, last_executed_at
— so future scans aren't required to stay correct. user must drop data/*.db.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 13:56:19 +02:00
felixfoertsch 874f04b7a5 wire scheduler into queue, add retry, dev-reset cleanup, biome 2.4 migrate
- execute: actually call isInScheduleWindow/waitForWindow/sleepBetweenJobs in runSequential (they were dead code); emit queue_status SSE events (running/paused/sleeping/idle) so the pipeline's existing QueueStatus listener lights up
- review: POST /:id/retry resets an errored plan to approved, wipes old done/error jobs, rebuilds command from current decisions, queues fresh job
- scan: dev-mode DELETE now also wipes jobs + subtitle_files (previously orphaned after every dev reset)
- biome: migrate config to 2.4 schema, autoformat 68 files (strings + indentation), relax opinionated a11y/hooks-deps/index-key rules that don't fit this codebase
- routeTree.gen.ts regenerated after /nodes removal
2026-04-13 07:41:19 +02:00
felixfoertsch 93ed0ac33c fix analyzer + api boundary + perf + scheduler hardening
- analyzer: rewrite checkAudioOrderChanged to compare actual output order, unify assignTargetOrder with a shared sortKeptStreams util in ffmpeg builder
- review: recompute is_noop via full audio removed/reordered/transcode/subs check on toggle, preserve custom_title across rescan by matching (type,lang,stream_index,title), batch pipeline transcode-reasons query to avoid N+1
- validate: add lib/validate.ts with parseId + isOneOf helpers; replace bare Number(c.req.param('id')) with 400 on invalid ids across review/subtitles
- scan: atomic CAS on scan_running config to prevent concurrent scans
- subtitles: path-traversal guard — only unlink sidecars within the media item's directory; log-and-orphan DB entries pointing outside
- schedule: include end minute in window (<= vs <)
- db: add indexes on review_plans(status,is_noop), stream_decisions(plan_id), media_items(series_jellyfin_id,series_name,type), media_streams(item_id,type), subtitle_files(item_id), jobs(status,item_id)
2026-04-13 07:31:48 +02:00
felixfoertsch cdcb1ff706 drop multi-node ssh execution, unify job runner to local + fix job completion atomicity
- remove nodes table, ssh service, nodes api, NodesPage route
- execute.ts: local-only spawn, atomic CAS job claim via UPDATE status
- wrap job done + subtitle_files insert + review_plans status in db transaction
- stream ffmpeg output per line with 500ms throttled flush
- bump version to 2026.04.13
2026-04-13 07:25:19 +02:00
felixfoertsch c2e5b70b02 add schema migrations for unified pipeline: confidence, apple_compat, job_type, transcode_codec
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 01:42:18 +01:00
felixfoertsch 38b0faf55a add job_type column, simplify execute page: remove node/command columns, add type badge, make item title clickable
Build and Push Docker Image / build (push) Successful in 37s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 15:01:11 +01:00
felixfoertsch 76d3b1acfb remove path mappings, add subtitle summary endpoint, cache setup page, bump version
Build and Push Docker Image / build (push) Successful in 1m50s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 12:02:26 +01:00
felixfoertsch a4d5eb59e1 add configurable audio languages, sortable language lists in settings
Build and Push Docker Image / build (push) Successful in 1m9s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 09:51:03 +01:00
felixfoertsch ef785de955 add path mappings to translate jellyfin library paths to container mount paths
Build and Push Docker Image / build (push) Successful in 20s
jellyfin may use different internal paths (e.g. /tv/) than container mounts
(/series/). path_mappings config (or PATH_MAPPINGS env var) translates at scan
time. configurable via setup ui or env var format: /tv/=/series/,/data/=/movies/

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 16:57:22 +01:00
felixfoertsch d5f4afd26b split audio/subtitle concerns, remove docker-in-docker, add per-node path mapping
Build and Push Docker Image / build (push) Successful in 1m54s
- install ffmpeg in dockerfile (fixes exit code 127)
- buildCommand() now audio-only remux, no subtitle extraction
- add unapprove endpoint + ui button for approved items
- add batch extract-all subtitles endpoint + ui button
- audio detail page shows only video+audio streams
- remove global movies_path/series_path config, add per-node path mapping
- remove docker-in-docker command building (buildDockerCommand, buildDockerExtractOnlyCommand)
- ssh execution translates /movies/ and /series/ to node-specific paths
- remove media paths section from setup page
- add unraid-template.xml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 16:48:00 +01:00
felixfoertsch 5ac44b7551 restructure to react spa + hono api, fix missing server/ and lib/
rewrite from monolithic hono jsx to react 19 spa with tanstack router
+ hono json api backend. add scan, review, execute, nodes, and setup
pages. multi-stage dockerfile (node for vite build, bun for runtime).

previously, server/ and src/shared/lib/ were silently excluded by
global gitignore patterns (/server/ from emacs, lib/ from python).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 22:57:40 +01:00