@torrent-tv/proxy 2.85.1 → 2.87.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +2138 -1909
- package/CLAUDE.md +318 -197
- package/assets/calibration/NOTICE.md +2 -2
- package/bin/cli.js +684 -616
- package/biome.json +409 -233
- package/docs/container-architecture.md +218 -213
- package/docs/download-architecture.md +7 -7
- package/docs/encode-architecture.md +287 -11
- package/docs/encode-run-state.md +2 -2
- package/docs/logs.md +97 -97
- package/docs/{disk-architecture.md → storage-architecture.md} +107 -1
- package/openspec/changes/adaptive-bitrate/design.md +2 -2
- package/openspec/changes/adaptive-bitrate/proposal.md +1 -1
- package/openspec/changes/adaptive-bitrate/tasks.md +1 -1
- package/openspec/changes/chunked-request-bodies/design.md +1 -1
- package/openspec/changes/chunked-request-bodies/proposal.md +1 -1
- package/openspec/changes/chunked-request-bodies/tasks.md +1 -1
- package/openspec/changes/cold-start/design.md +1 -1
- package/openspec/changes/disk-cap/proposal.md +1 -1
- package/openspec/changes/proxy-observability/proposal.md +2 -2
- package/openspec/changes/track-selection/proposal.md +1 -1
- package/openspec/changes/transcode-quality/proposal.md +1 -1
- package/package.json +2 -2
- package/research/delivery-wedge-stand-ladder-2026-09-13.md +329 -329
- package/research/double-spawn-2026-09-10.md +171 -171
- package/routes/api/playback-plan/post.js +2 -2
- package/routes/api/sources/files/get.js +1 -1
- package/routes/api/sources/stats/get.js +1 -1
- package/routes/api/sources/warm/post.js +10 -3
- package/routes/api/subtitles/get.js +230 -230
- package/routes/api/transcode-sessions/fragment-far/post.js +3 -3
- package/routes/api/transcode-sessions/net-report/post.js +129 -94
- package/routes/api/transcode-sessions/post.js +160 -139
- package/routes/api/transcode-sessions/progress/get.js +3 -3
- package/routes/api/transcode-sessions/release/post.js +3 -3
- package/routes/api/transcode-sessions/seek/post.js +12 -3
- package/routes/stream/get.js +2 -2
- package/routes/transcode/audio-file/get.js +15 -6
- package/routes/transcode/audio-warm/get.js +8 -5
- package/routes/transcode/session-file/get.js +429 -268
- package/routes/transcode/variant-file/get.js +47 -7
- package/routes/transcode/variant-warm/get.js +18 -6
- package/scripts/render-run-graph.js +1 -1
- package/server.js +720 -465
- package/services/encode/CoverageMap.js +428 -428
- package/services/encode/CushionReport.js +345 -0
- package/services/encode/EncodeAdmission.js +332 -0
- package/services/{orchestrators → encode}/EncodeOrchestrator.js +285 -17
- package/services/encode/EncodeRun.js +16 -9
- package/services/encode/EncodeRuns.js +1219 -0
- package/services/encode/Encoder.js +18 -2
- package/services/encode/EncoderSelection.js +38 -0
- package/services/encode/LocalObservations.js +359 -0
- package/services/encode/NvencEncoder.js +5 -0
- package/services/encode/OutputOpening.js +745 -0
- package/services/encode/OutputTimes.js +579 -0
- package/services/encode/Renditions.js +2162 -0
- package/services/encode/RunProgress.js +84 -0
- package/services/encode/SoftwareEncoder.js +12 -7
- package/services/encode/args.js +402 -186
- package/services/encode/calibration.js +361 -0
- package/services/encode/contention.js +128 -128
- package/services/encode/encode-run-state.js +16 -20
- package/services/encode/fingerprint.js +322 -0
- package/services/{hwaccel.js → encode/hwaccel.js} +82 -42
- package/services/{learned-median.js → encode/learned-median.js} +11 -0
- package/services/encode/output/EncodedOutput.js +57 -0
- package/services/encode/output/Output.js +131 -0
- package/services/{output/LiveOutputs.js → encode/output/OutputCatalog.js} +141 -40
- package/services/encode/output/OutputSpec.js +390 -0
- package/services/{output → encode/output}/Timeline.js +12 -37
- package/services/encode/output/index.js +20 -0
- package/services/encode/output/ladder.js +32 -0
- package/services/{output → encode/output}/playlists.js +3 -0
- package/services/encode/output-key-format.js +21 -0
- package/services/encode/quality/EncodeCost.js +1083 -0
- package/services/encode/quality/HostLoad.js +525 -0
- package/services/encode/quality/HostTimings.js +249 -0
- package/services/encode/quality/OutputQualityState.js +28 -0
- package/services/encode/quality/QualityController.js +931 -0
- package/services/encode/quality/QualityOffer.js +416 -0
- package/services/encode/quality/drain-threat.js +47 -0
- package/services/encode/quality/link-budget.js +254 -0
- package/services/encode/quality/output-format.js +266 -0
- package/services/encode/quality/serving-output.js +216 -0
- package/services/encode/quality/visible-rung.js +57 -0
- package/services/encode/run-command.js +23 -20
- package/services/encode/run-inputs.js +42 -0
- package/services/{segment-formats → encode/segment-formats}/fmp4.js +63 -0
- package/services/{segment-formats → encode/segment-formats}/index.js +3 -3
- package/services/encode/segment-formats/init-compat.js +324 -0
- package/services/{segment-formats → encode/segment-formats}/mp4-boxes.js +8 -3
- package/services/encode/throughput.js +97 -0
- package/services/{orchestrators → media}/ContainerOrchestrator.js +45 -7
- package/services/media/KeyframeTables.js +316 -0
- package/services/{source → media}/SourceFile.js +7 -55
- package/services/{torrent-worker/subtitle-cues.js → media/SubtitleCues.js} +103 -132
- package/services/media/SubtitleOrchestrator.js +167 -0
- package/services/{audio-inventory.js → media/audio-inventory.js} +7 -1
- package/services/{container → media/container}/AviContainer.js +266 -266
- package/services/{container → media/container}/Container.js +596 -601
- package/services/media/container/ContainerFactory.js +56 -0
- package/services/media/container/KeyframeTable.js +272 -0
- package/services/{container → media/container}/MatroskaContainer.js +1485 -1485
- package/services/{container → media/container}/Mp4Container.js +1244 -1244
- package/services/{container → media/container}/SubtitleFileContainer.js +322 -322
- package/services/media/keyframe-probe.js +157 -0
- package/services/media/media-info-probe.js +93 -0
- package/services/{playback-planner.js → media/playback-planner.js} +82 -39
- package/services/{tracks → media/tracks}/AudioTrack.js +131 -131
- package/services/{tracks → media/tracks}/language-detect.js +174 -174
- package/services/server/OutputLifecycle.js +418 -0
- package/services/server/SegmentServing.js +1146 -0
- package/services/server/ViewerRequests.js +522 -0
- package/services/server/controllers/PlaybackController.js +36 -0
- package/services/{controllers → server/controllers}/SubtitleController.js +17 -33
- package/services/server/wire-outputs.js +958 -0
- package/services/storage/Diagnostics.js +202 -0
- package/services/storage/MachineBudget.js +235 -0
- package/services/{piece-store → storage}/allowance.js +107 -107
- package/services/{core-dumps.js → storage/core-dumps.js} +1 -1
- package/services/{files → storage/files}/CompletedFiles.js +116 -1
- package/services/{disk → storage}/free.js +33 -0
- package/services/storage/machine-memory.js +56 -0
- package/services/{memory-report.js → storage/memory-report.js} +575 -596
- package/services/{piece-store → storage/piece-store}/piece-disk-store.js +0 -7
- package/services/{piece-store → storage/piece-store}/shared-piece-store.js +28 -21
- package/services/{encode → storage/segment-store}/SegmentStore.js +395 -8
- package/services/storage/wire.js +175 -0
- package/services/{download → torrent/download}/SwarmSelection.js +2 -2
- package/services/{download → torrent/download}/registry.js +2 -2
- package/services/torrent/files.js +1 -1
- package/services/torrent/piece-store-of.js +29 -0
- package/services/{torrent-pool.js → torrent/torrent-pool.js} +26 -119
- package/services/{torrent-worker → torrent/worker}/background-fill.js +2 -2
- package/services/{torrent-worker → torrent/worker}/channel.js +271 -250
- package/services/{torrent-worker → torrent/worker}/client.js +77 -58
- package/services/torrent/worker/held-bytes.js +155 -0
- package/services/{torrent-worker → torrent/worker}/install-webrtc-shim.js +1 -1
- package/services/{torrent-worker → torrent/worker}/piece-reader.js +21 -30
- package/services/{torrent-worker → torrent/worker}/pool-adapter.js +101 -137
- package/services/{torrent-worker → torrent/worker}/protocol.js +26 -30
- package/services/torrent/worker/resume-warm.js +107 -0
- package/services/{torrent-worker → torrent/worker}/webrtc-shim.js +2 -2
- package/services/{torrent-worker → torrent/worker}/worker.js +68 -159
- package/services/{data-channel-handler.js → transport/data-channel-handler.js} +1356 -1356
- package/services/{delivery-probe.js → transport/delivery-probe.js} +842 -842
- package/services/{health-collector.js → transport/health-collector.js} +6 -35
- package/services/{packet-witness.js → transport/packet-witness.js} +808 -784
- package/services/{port-mapper.js → transport/port-mapper.js} +1 -1
- package/services/{usrsctp-state.js → transport/usrsctp-state.js} +1 -1
- package/services/{webrtc-manager.js → transport/webrtc-manager.js} +1 -1
- package/services/viewer/Assignments.js +340 -0
- package/services/{priority → viewer}/PriorityMap.js +390 -390
- package/services/{priority → viewer}/PriorityOrchestrator.js +325 -320
- package/services/viewer/Viewer.js +657 -489
- package/services/viewer/Viewers.js +368 -257
- package/services/viewer/active-output.js +37 -54
- package/services/viewer/audio-start.js +46 -0
- package/services/viewer/buffer-trend.js +81 -0
- package/services/viewer/choices.js +647 -0
- package/services/viewer/positions.js +126 -120
- package/services/viewer/report-intake.js +46 -44
- package/test/asset-paths.test.js +28 -0
- package/test/audio-inventory.test.js +3 -3
- package/test/audio-track-bitrate.test.js +40 -0
- package/test/auto-quality-step.test.js +707 -518
- package/test/available-share.test.js +1 -1
- package/test/background-fill.test.js +1 -1
- package/test/client-log-file.test.js +110 -110
- package/test/completed-files.test.js +1 -1
- package/test/concurrent-cost.test.js +13 -3
- package/test/container-media-info.test.js +3 -3
- package/test/core-dumps.test.js +1 -1
- package/test/coverage-follows-the-disk.test.js +40 -8
- package/test/cue-push-carries-what-the-walk-found.test.js +58 -0
- package/test/cut-times-timeline.test.js +2 -2
- package/test/cuts-follow-published-grid.test.js +9 -8
- package/test/data-channel-frame.test.js +1 -1
- package/test/dead-torrent-handle.test.js +1 -1
- package/test/declared-rates.test.js +2 -2
- package/test/declared-tracks.test.js +2 -2
- package/test/decode-cost-fit.test.js +1 -1
- package/test/decode-cost.test.js +34 -25
- package/test/delivery-probe.test.js +1 -1
- package/test/demand-register.test.js +4 -4
- package/test/dht-bootstrap.test.js +1 -1
- package/test/drain-threat.test.js +65 -0
- package/test/duplicate-add.test.js +1 -1
- package/test/encode-admission.test.js +146 -0
- package/test/encode-cost.test.js +199 -10
- package/test/encode-orchestrator.test.js +39 -1
- package/test/encode-plan-viewers.test.js +2 -2
- package/test/encode-run.test.js +2 -2
- package/test/encode-slope.test.js +1 -1
- package/test/encoded-output.test.js +39 -0
- package/test/encoder-readings.test.js +1 -1
- package/test/every-claimant-is-told-its-share.test.js +152 -0
- package/test/failed-encoder-still-serves-what-exists.test.js +354 -0
- package/test/failed-start-breaker.test.js +81 -30
- package/test/fastest-wires.test.js +1 -1
- package/test/ffmpeg-stream-counts.test.js +1 -1
- package/test/file-edges.test.js +3 -3
- package/test/fmp4-self-contained.test.js +1 -1
- package/test/fmp4-timestamps.test.js +1 -1
- package/test/fmp4-track-completeness.test.js +1 -1
- package/test/format-keyed-outputs.test.js +78 -0
- package/test/health-metrics.test.js +3 -2
- package/test/held-request-width.test.js +22 -23
- package/test/helpers/encode-run.js +136 -136
- package/test/helpers/held-file.js +29 -0
- package/test/helpers/manager.js +57 -0
- package/test/helpers/output-spec.js +63 -0
- package/test/host-load.test.js +1 -1
- package/test/index-accuracy-is-about-the-file.test.js +128 -0
- package/test/init-compat.test.js +219 -0
- package/test/input-lost-quiets-the-plan.test.js +1 -1
- package/test/input-unavailable.test.js +1 -1
- package/test/keeping-period.test.js +4 -4
- package/test/keyframe-index-accuracy.test.js +97 -45
- package/test/keyframe-table-budget.test.js +28 -33
- package/test/keyframes-belong-to-the-file.test.js +174 -57
- package/test/learned-median.test.js +1 -1
- package/test/letting-go-asks-who-is-watching.test.js +191 -0
- package/test/limit-row-by-area.test.js +60 -0
- package/test/link-budget.test.js +111 -0
- package/test/link-probe.test.js +56 -56
- package/test/local-encoding-data.test.js +273 -0
- package/test/look-ahead-coverage.test.js +1 -1
- package/test/manager-surface.test.js +294 -0
- package/test/matroska-blocks.test.js +0 -0
- package/test/matroska-cues-track.test.js +1 -1
- package/test/memory-budget.test.js +2 -2
- package/test/mp4-composition-times.test.js +0 -0
- package/test/mp4-subtitles.test.js +1 -1
- package/test/one-authority.test.js +45 -41
- package/test/one-budget.test.js +149 -0
- package/test/one-output-one-session.test.js +204 -0
- package/test/{disk-space.test.js → one-resource-divided.test.js} +30 -10
- package/test/one-serving-output-per-height.test.js +458 -0
- package/test/orchestrator-wired.test.js +46 -45
- package/test/output-catalog.test.js +39 -0
- package/test/output-shape.test.js +4 -51
- package/test/output-spec.test.js +251 -157
- package/test/output-speed.test.js +17 -15
- package/test/outputs-being-prepared.test.js +40 -0
- package/test/packet-witness-ring.test.js +2 -2
- package/test/packet-witness.test.js +1 -1
- package/test/piece-disk-store.test.js +2 -2
- package/test/piece-from-whole-file.test.js +1 -1
- package/test/piece-lru.test.js +1 -1
- package/test/piece-reader.test.js +2 -2
- package/test/piece-store-eviction.test.js +1 -1
- package/test/piece-store-never-refuses.test.js +1 -1
- package/test/piece-store-reservations.test.js +1 -1
- package/test/piece-store-slow-disk.test.js +1 -1
- package/test/piece-tail.test.js +1 -1
- package/test/plan-host-timings.test.js +1 -1
- package/test/plan-is-steady.test.js +225 -225
- package/test/priority-map-download.test.js +4 -4
- package/test/priority-map-emptied.test.js +204 -205
- package/test/priority-map-per-output.test.js +215 -209
- package/test/priority-map.test.js +1 -1
- package/test/probe-wedge-certainty.test.js +1 -1
- package/test/produced-copy-choice.test.js +15 -60
- package/test/quality-offer.test.js +162 -0
- package/test/quality-variants.test.js +1229 -1167
- package/test/rate-control-in-the-output.test.js +180 -0
- package/test/read-survives-withdrawal.test.js +2 -2
- package/test/read-window.test.js +10 -3
- package/test/resume-warm.test.js +1 -1
- package/test/run-inputs.test.js +80 -0
- package/test/run-position-follows-published-grid.test.js +13 -11
- package/test/run-progress.test.js +65 -0
- package/test/same-height-switch.test.js +664 -0
- package/test/seek-frees-the-viewers-requests.test.js +217 -0
- package/test/seek-landing.test.js +13 -13
- package/test/seek-target-not-superseded.test.js +28 -29
- package/test/segment-closed-on-both-branches.test.js +96 -0
- package/test/segment-cut-times.test.js +1 -1
- package/test/segment-hold-supersede.test.js +28 -35
- package/test/segment-judged-by-its-own-header.test.js +221 -0
- package/test/segment-serve-wiring.test.js +130 -39
- package/test/segment-store-eviction.test.js +22 -7
- package/test/segment-store.test.js +98 -14
- package/test/segment-true-start.test.js +1 -1
- package/test/segments-are-shared.test.js +17 -16
- package/test/send-chunk-bytes.test.js +1 -1
- package/test/serving-existing-output.test.js +102 -0
- package/test/serving-output.test.js +102 -0
- package/test/shared-piece-store.test.js +1 -1
- package/test/source-file.test.js +1 -1
- package/test/stale-request-after-seek.test.js +17 -21
- package/test/startup-readings.test.js +10 -5
- package/test/subtitle-cue-framing.test.js +5 -5
- package/test/subtitle-cue-source.test.js +36 -38
- package/test/subtitle-cue-walk.test.js +11 -10
- package/test/subtitle-defaults.test.js +1 -1
- package/test/subtitle-language.test.js +4 -4
- package/test/subtitle-track-numbering.test.js +5 -4
- package/test/supply-margin.test.js +1 -1
- package/test/swarm-follows-readers.test.js +3 -3
- package/test/swarm-reach.test.js +1 -1
- package/test/swarm-selection.test.js +3 -3
- package/test/tail-duplication.test.js +1 -1
- package/test/the-channel-has-both-ends.test.js +91 -0
- package/test/the-torrent-thread-serves-bytes.test.js +107 -0
- package/test/thread-pool.test.js +3 -3
- package/test/timeline.test.js +1 -12
- package/test/torrent-cost.test.js +1 -1
- package/test/tracks-begin-together.test.js +38 -24
- package/test/tunnel-renewal.test.js +1 -1
- package/test/two-viewers-one-picture.test.js +120 -87
- package/test/upload-hurry.test.js +3 -3
- package/test/usrsctp-state.test.js +1 -1
- package/test/video-facts.test.js +1 -1
- package/test/viewer-assignments.test.js +143 -0
- package/test/viewer-outputs.test.js +60 -52
- package/test/viewer-presence.test.js +22 -25
- package/test/viewer-requests-admission.test.js +89 -0
- package/test/viewer-states.test.js +189 -172
- package/test/viewer-subtitle-subscription.test.js +1 -1
- package/test/viewer.test.js +139 -136
- package/test/viewing-generations.test.js +390 -0
- package/test/visible-rung.test.js +37 -0
- package/test/wait-ledger.test.js +1 -1
- package/test/webrtc-shim.test.js +3 -3
- package/test/wedge-certainty.test.js +2 -2
- package/test/withdraw-piece-claim.test.js +2 -2
- package/test/worker-channel.test.js +1 -31
- package/utils/client-log-file.js +151 -151
- package/scripts/read-container-index.mjs +0 -108
- package/services/container/ContainerFactory.js +0 -122
- package/services/controllers/PlaybackController.js +0 -33
- package/services/disk/DiskSpace.js +0 -150
- package/services/disk/wire.js +0 -61
- package/services/hls-session-manager.js +0 -9689
- package/services/orchestrators/SubtitleOrchestrator.js +0 -160
- package/services/output/Output.js +0 -94
- package/services/output/OutputSpec.js +0 -202
- package/services/output/index.js +0 -1
- package/services/output/ladder.js +0 -26
- package/services/produced-index.js +0 -222
- package/services/quality/EncodeCost.js +0 -555
- package/services/torrent-worker/container-tracks.js +0 -414
- package/test/produced-index.test.js +0 -142
- /package/{services → bin}/thread-pool.js +0 -0
- /package/services/{available-share.js → encode/available-share.js} +0 -0
- /package/services/{decode-cost-fit.js → encode/decode-cost-fit.js} +0 -0
- /package/services/{encoder-readings.js → encode/encoder-readings.js} +0 -0
- /package/services/{host-load.js → encode/host-load.js} +0 -0
- /package/services/{output → encode/output}/cut-grid.js +0 -0
- /package/services/{output → encode/output}/rates.js +0 -0
- /package/services/{segment-formats → encode/segment-formats}/mpegts.js +0 -0
- /package/services/{torrent-cost.js → encode/torrent-cost.js} +0 -0
- /package/services/{container → media/container}/ebml-reader.js +0 -0
- /package/services/{ffmpeg-banner.js → media/ffmpeg-banner.js} +0 -0
- /package/services/{tracks → media/tracks}/ContainerTrack.js +0 -0
- /package/services/{tracks → media/tracks}/ImageSubtitleTrack.js +0 -0
- /package/services/{tracks → media/tracks}/SubtitleTrack.js +0 -0
- /package/services/{tracks → media/tracks}/TextSubtitleTrack.js +0 -0
- /package/services/{tracks → media/tracks}/VideoTrack.js +0 -0
- /package/services/{tracks → media/tracks}/index.js +0 -0
- /package/services/{files → storage/files}/piece-from-whole-file.js +0 -0
- /package/services/{disk → storage}/keep.js +0 -0
- /package/services/{piece-store → storage/piece-store}/piece-lru.js +0 -0
- /package/services/{disk → storage}/returns.js +0 -0
- /package/services/{demand → torrent/demand}/DemandRegister.js +0 -0
- /package/services/{demand → torrent/demand}/Urgency.js +0 -0
- /package/services/{demand → torrent/demand}/Window.js +0 -0
- /package/services/{demand → torrent/demand}/index.js +0 -0
- /package/services/{demand → torrent/demand}/pieces.js +0 -0
- /package/services/{download → torrent/download}/withdraw-claim.js +0 -0
- /package/services/{supply-margin.js → torrent/supply-margin.js} +0 -0
- /package/services/{torrent-source-key.js → torrent/torrent-source-key.js} +0 -0
- /package/services/{torrent-worker → torrent/worker}/fastest-wires.js +0 -0
- /package/services/{torrent-worker → torrent/worker}/handle-state.js +0 -0
- /package/services/{nat-classifier.js → transport/nat-classifier.js} +0 -0
- /package/services/{registry-api.js → transport/registry-api.js} +0 -0
- /package/services/{tunnel-client.js → transport/tunnel-client.js} +0 -0
- /package/services/{priority → viewer}/WaitLedger.js +0 -0
|
@@ -23,10 +23,90 @@ It decides from four things and no others:
|
|
|
23
23
|
| what is wanted | the priority map of that output |
|
|
24
24
|
| how many the machine can hold | `run-budget.js`, from measurements of this host |
|
|
25
25
|
|
|
26
|
-
**No viewer reaches
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
26
|
+
**No viewer reaches the decision.** `EncodePlan` and `EncodeOrchestrator` do not
|
|
27
|
+
import the viewer layer, name a consumer id or hold a person. What crosses is a
|
|
28
|
+
priority map: zones of segment numbers with a rank and a real time, and nobody's
|
|
29
|
+
name on it.
|
|
30
|
+
|
|
31
|
+
**The rest of `services/encode/` names people, and holds none.** A step and a
|
|
32
|
+
soundtrack are chosen, warmed and left per person, so `Renditions`,
|
|
33
|
+
`QualityController` and `CushionReport` say WHOSE — by consumer id, which is a
|
|
34
|
+
name and not an object. What they may ask about that name and what they may
|
|
35
|
+
tell the viewer layer about it is `services/viewer/choices.js`: the mode a size
|
|
36
|
+
was picked in, what a link measured, the soundtrack chosen, the step on screen,
|
|
37
|
+
what is being warmed, who is present, and the standing quality ask. Values in,
|
|
38
|
+
values out. No `Viewer` crosses the boundary and nothing outside the viewer
|
|
39
|
+
layer writes a viewer's record — which is the rule a position cost 77 encoder
|
|
40
|
+
starts for breaking (field 2026-09-13).
|
|
41
|
+
|
|
42
|
+
This paragraph used to claim the whole directory named nobody, which was never
|
|
43
|
+
true of `Renditions` and could not be: which soundtrack to make is a fact about
|
|
44
|
+
a person. What matters is that it is a NAME, so the viewer's record stays in
|
|
45
|
+
one place with one writer.
|
|
46
|
+
|
|
47
|
+
**What answered a request is the viewer's, per viewing** (roadmap item 97,
|
|
48
|
+
step 9). The page stamps every request with the GENERATION of its viewing — a
|
|
49
|
+
number it raises on the viewer's own seek, before it sends anything — and the
|
|
50
|
+
proxy reads it rather than counting one, so a request made before a seek and
|
|
51
|
+
delivered after it still says which viewing it belongs to. Three things follow,
|
|
52
|
+
and each is asked through `choices.js`, by name:
|
|
53
|
+
|
|
54
|
+
1. every file route asks FIRST whether that viewer still takes requests of that
|
|
55
|
+
generation, before a step or a soundtrack is resolved — resolving can create
|
|
56
|
+
an output and registers the viewer on it, which a request of a viewing
|
|
57
|
+
already left must not do;
|
|
58
|
+
2. a repeat of a request within one viewing is answered by the output that
|
|
59
|
+
answered it: the step route records the height asked for and the segment
|
|
60
|
+
(the init as −1), the picture's own route records under the height that
|
|
61
|
+
output is named after. A soundtrack records nothing of this kind;
|
|
62
|
+
3. a response holds its output from its first byte to its end, and no output is
|
|
63
|
+
disposed while an assignment holds it — except on a hardware encoder's
|
|
64
|
+
failure, where nothing can go on making what it made. The idle expiry asks
|
|
65
|
+
only the assignments, not whether anybody is registered, so a paused
|
|
66
|
+
viewer's claim on an output stays a separate decision.
|
|
67
|
+
|
|
68
|
+
**Which output serves a viewer is one rule, asked by every path, and the
|
|
69
|
+
choice is the viewer's own** (step 11). A request for a piece is answered, in
|
|
70
|
+
order, by what already answered that address in this viewing (`given`), by the
|
|
71
|
+
output chosen for this viewer at that height (`chosen`), and only then by the
|
|
72
|
+
rule — and a repeat whose output has gone is served from its stored piece or by
|
|
73
|
+
an output with a proven-compatible header, or answered `assignment-lost`. The
|
|
74
|
+
rule (`serving-output.js`, `outputSuits`) asks the picture's material, the size
|
|
75
|
+
a viewer who picked by hand requires, and what the viewer's OWN link does with
|
|
76
|
+
the output's whole load: the picture and the soundtrack they chose, each part
|
|
77
|
+
`known` (a limit), `estimated` (a stated average) or `unknown` (no bound), the
|
|
78
|
+
load as trustworthy as its worst part (`link-budget.js`). `unknown` is refused
|
|
79
|
+
against a measured link; `estimated` is admitted by estimate and reported to
|
|
80
|
+
the page as not confirmed; a bound is taken before an average. When nothing
|
|
81
|
+
suits, nothing is handed over: the answer is `output-unavailable`, with the
|
|
82
|
+
figures, and the page explains it. The shared `OutputCatalog.servedBy` record
|
|
83
|
+
and the worst link over all viewers are gone; the quality budget asks each
|
|
84
|
+
viewer's link for that viewer alone.
|
|
85
|
+
|
|
86
|
+
**A bitrate limit is part of an output, not a figure moved under it** (step
|
|
87
|
+
10). A software encode's `-maxrate`/`-bufsize` are named by its key
|
|
88
|
+
(`vbv=<maxrate>k-<bufsize>k@<level>`), so two limits are two outputs and a
|
|
89
|
+
slow viewer's limit can no longer land on every viewer of a picture — which is
|
|
90
|
+
what it did while it was a field of the output set by the worst of their links.
|
|
91
|
+
The level is pinned (`-level:v`) to the level of the NOMINAL output of that
|
|
92
|
+
size and frame rate, so every limit at one size writes the same header
|
|
93
|
+
and a viewer can be moved between them under the address they already play.
|
|
94
|
+
A limit above the nominal one is refused. Hardware encoders are given no limit
|
|
95
|
+
and state none (`vbv=-`). Whether the pinned level is the one x264 would choose
|
|
96
|
+
for the nominal output is checked before a release
|
|
97
|
+
(`stand/segment-compat/level-check.mjs`).
|
|
98
|
+
|
|
99
|
+
**Which limits a frame has is decided by its AREA** (step 14, decided with the
|
|
100
|
+
user 2026-09-24). The rows of limits are the ladder's frames at 16:9, sized by
|
|
101
|
+
the same function that sizes an encode; a frame takes the row nearest by the
|
|
102
|
+
ratio of areas, the larger row on the exact border, and a row with no nominal
|
|
103
|
+
of its own borrows the nearest row that has one (`limitRowFor`,
|
|
104
|
+
`nominalKbpsFor`). Every place that picks a row — opening an output, the move
|
|
105
|
+
between limits, the offer's price of a step and the link rule — is handed the
|
|
106
|
+
whole frame, so all of them name the same row. By height alone a 1920x800 film
|
|
107
|
+
fell in the 720 row although it is nearer the 1080 one. The proposed measured
|
|
108
|
+
table of lower limits was dropped on 2026-09-27. Step 14 now qualifies this
|
|
109
|
+
machine's encoder modes and does not set lower bitrate limits.
|
|
30
110
|
|
|
31
111
|
## Where a viewer is, and how it reaches the encoders
|
|
32
112
|
|
|
@@ -74,7 +154,7 @@ flowchart TB
|
|
|
74
154
|
ER[EncodeRun<br/>one process, one interval]
|
|
75
155
|
end
|
|
76
156
|
|
|
77
|
-
subgraph D["services/demand + download — what the swarm is told"]
|
|
157
|
+
subgraph D["services/torrent/demand + torrent/download — what the swarm is told"]
|
|
78
158
|
DR[DemandRegister]
|
|
79
159
|
SS[SwarmSelection]
|
|
80
160
|
end
|
|
@@ -127,17 +207,162 @@ That distinction is answered where the two facts meet, and neither layer is
|
|
|
127
207
|
handed the other:
|
|
128
208
|
|
|
129
209
|
- **which step is on their screen** is a fact about a PERSON, read off the
|
|
130
|
-
viewer as one field
|
|
210
|
+
viewer as one field. The page STATES it (`playingHeight` on the viewer's
|
|
211
|
+
report, sent the moment hls.js has switched), and `Renditions.viewerPlays`
|
|
212
|
+
records it. It used to be inferred from the first segment requested of a
|
|
213
|
+
rung, which is the player fetching rather than the picture having moved;
|
|
131
214
|
- **which output a step supersedes** is a fact about the FILM'S SHAPE, answered
|
|
132
|
-
by `
|
|
215
|
+
by `OutputCatalog.supersededBy(session, stepOnScreen)`, which takes a plain id
|
|
133
216
|
and has never seen a viewer.
|
|
134
217
|
|
|
135
218
|
A step, a soundtrack, and a step being warmed are consumed by whoever is
|
|
136
219
|
registered on them — everywhere but the picture, a person who stops watching is
|
|
137
|
-
let go of, so being known to an output is consuming it.
|
|
220
|
+
let go of, so being known to an output is consuming it. Whether an output is
|
|
221
|
+
wanted at all is asked of the viewer registry and nothing else: an output a
|
|
222
|
+
picture made on its own behalf once carried a made-up "family" claim that kept
|
|
223
|
+
it alive, which was a second answer to the same question and is gone. Every
|
|
224
|
+
viewer has a name; a request that names nobody makes no viewer. Through a warm-up both
|
|
138
225
|
the step on screen and the step being made ready are genuinely produced, which
|
|
139
226
|
is the price of the switch not being visible.
|
|
140
227
|
|
|
228
|
+
## A move between two limits of one height
|
|
229
|
+
|
|
230
|
+
A bitrate limit is part of an output, so a viewer whose link cannot carry the
|
|
231
|
+
limit on their screen is given ANOTHER output of the same height at a lower
|
|
232
|
+
limit — for them alone, and without their player being told (roadmap item 97,
|
|
233
|
+
step 12). The address `v/<height>/…` does not change; what changes is which
|
|
234
|
+
output this side answers it with.
|
|
235
|
+
|
|
236
|
+
1. **Prepared, not switched.** `Renditions.prepareSameHeightSwitch` opens (or
|
|
237
|
+
finds, by its key) the output of the same size at the next limit of
|
|
238
|
+
`limitsFor(height)` that the viewer's link admits, marks it a step of the
|
|
239
|
+
picture, and registers the viewer on it — that is what buys it an encoder.
|
|
240
|
+
The choice (`Assignments.choose`) is not touched: until the move is made they
|
|
241
|
+
are given what they were given. The move being prepared is a field of the
|
|
242
|
+
viewer, `Viewer.sameHeightSwitch`, and not `warmingVariantId`, which is a
|
|
243
|
+
step their player will switch to itself.
|
|
244
|
+
2. **Made on the store's own event.** `SegmentStore.onPublished` is filtered by
|
|
245
|
+
the prepared output's key. The move is made when the segment the viewer will
|
|
246
|
+
ask for NEXT is closed on it: one past the highest they were given in the
|
|
247
|
+
viewing they are in now, or — with nothing given in it, straight after a
|
|
248
|
+
seek — the one where they stand. It is read at the moment of the event, so a
|
|
249
|
+
segment closed behind that one, or an event that arrives after a seek, moves
|
|
250
|
+
nothing.
|
|
251
|
+
3. **One synchronous stretch.** The choice under each height it was chosen
|
|
252
|
+
under, the verdict, the step on screen, the registration on the new output
|
|
253
|
+
and off the old one change with no `await` between them.
|
|
254
|
+
4. **The output left is not disposed.** The picture is never left. A step that
|
|
255
|
+
is left loses the viewer, the plan stops its encoder by its empty map, and the
|
|
256
|
+
idle expiry removes it once no assignment holds it; a repeat of an address it
|
|
257
|
+
answered is still answered by it, because `given` is read before `chosen`.
|
|
258
|
+
5. **Cancelled when not wanted.** When the link carries the output on screen
|
|
259
|
+
again, when the prepared output no longer suits, when the player moves to
|
|
260
|
+
another rung, when encoding the prepared output has failed for good, or when
|
|
261
|
+
nothing of the film is left to give from it: the viewer stops watching the
|
|
262
|
+
prepared output and nothing else changes for anybody.
|
|
263
|
+
6. **Asked again on three events, never on a timer.** A segment closed on the
|
|
264
|
+
prepared output (`SegmentStore.onPublished`); the viewer's own report
|
|
265
|
+
(`Renditions.noteViewerReported`, from the net-report route), which is what
|
|
266
|
+
carries a change of their link; and encoding the prepared output failing for
|
|
267
|
+
good (`Renditions.noteProductionFailed`, told by `EncodeRuns`). Without the
|
|
268
|
+
last two, a move onto an output that closes nothing stood for ever, and the
|
|
269
|
+
quality budget — which leaves a viewer with a move pending alone — never
|
|
270
|
+
decided for them again. Each event may also make the move, on the same one
|
|
271
|
+
condition: the segment they will ask for next is closed. A move is not
|
|
272
|
+
prepared onto an output whose encoding has failed. The fourth way a move can
|
|
273
|
+
wait — no encoder placed on the output — does not arise until the admission
|
|
274
|
+
of step 13 can refuse, and that refusal will be its event.
|
|
275
|
+
|
|
276
|
+
The quality budget uses this lever first, in both directions: down to the
|
|
277
|
+
highest lower limit the link admits, up one limit at a time, and another height
|
|
278
|
+
only where the height on screen has no limit left to move to. A copy has no
|
|
279
|
+
limit and a hardware encode is given none, so for them the lever is still the
|
|
280
|
+
height. The measured table of lower bitrate limits was dropped on 2026-09-27;
|
|
281
|
+
the product currently supplies only the nominal limit through `limitsFor`.
|
|
282
|
+
Per-machine calibration (step 14) qualifies encoder modes and does not create
|
|
283
|
+
lower bitrate limits, so same-height bitrate moves have no target in the current
|
|
284
|
+
product configuration.
|
|
285
|
+
|
|
286
|
+
7. **A move up waits for the cushion; a move down does not** (roadmap item
|
|
287
|
+
98). Once the piece asked for next is closed, a move UP is made only when
|
|
288
|
+
the viewer holds the cushion this file needs (`minimumBufferSeconds`), and
|
|
289
|
+
their next report asks again until they do. A move DOWN was started because
|
|
290
|
+
their buffer would run dry, so it is made the moment the piece is ready.
|
|
291
|
+
|
|
292
|
+
## When the quality budget moves a viewer (roadmap item 98)
|
|
293
|
+
|
|
294
|
+
The page has no quality control: the quality is always automatic. What moves a
|
|
295
|
+
viewer is decided here, per viewer, and each part of it is a measurement.
|
|
296
|
+
|
|
297
|
+
1. **Judged on the viewer's report, not on a timer.** `QualityController.noteViewerReported`
|
|
298
|
+
runs from the net-report route. The chosen fifteen-second window of a slow
|
|
299
|
+
link, the ten-second buffer threshold and the thirty-second wait after every
|
|
300
|
+
action are gone.
|
|
301
|
+
2. **Down when the buffer would run dry first.** Their link does not carry the
|
|
302
|
+
stream they are given AND, on the buffer's trend, it ends sooner than another
|
|
303
|
+
output could close the piece they need, counting the time until their next
|
|
304
|
+
report (`quality/drain-threat.js`). The trend is a least-squares fit over the
|
|
305
|
+
shortest run of their reports spanning one segment
|
|
306
|
+
(`viewer/buffer-trend.js`), because the buffer rises a segment at a time and
|
|
307
|
+
two readings catch only the rise or the fall. The time to readiness is this
|
|
308
|
+
host's measured first-segment time. The levers: a lower limit of the same
|
|
309
|
+
height, then a lower height asked of the player as URGENT — their page
|
|
310
|
+
switches as soon as the rung is ready, without waiting for a cushion. Where
|
|
311
|
+
neither can be prepared they stay on what they are given, with no message.
|
|
312
|
+
3. **The picture the viewer sees bounds a re-encode.** The page sends the
|
|
313
|
+
frame it would show without enlarging, in physical pixels; it is kept on
|
|
314
|
+
`Viewer.visiblePicture`. The bound is the smallest rung of the source's
|
|
315
|
+
ladder whose frame is not smaller (`quality/visible-rung.js`). It sets the
|
|
316
|
+
box an output is opened at (`OutputOpening`), the ceiling of every step up,
|
|
317
|
+
and the rung a step down prefers. A copy of the source is never re-encoded
|
|
318
|
+
for being taller than that.
|
|
319
|
+
4. **A smaller picture moves the viewer only onto a READY rung**
|
|
320
|
+
(`Renditions.heightReadyFor`: the piece they ask for next closed on it);
|
|
321
|
+
with none ready, the next judgement of their link or of the machine applies
|
|
322
|
+
the bound. **A larger picture** asks one rung up when the machine and their
|
|
323
|
+
link have room.
|
|
324
|
+
5. **A step up is let go when its conditions go back**: a draining buffer, a
|
|
325
|
+
link that no longer carries the stream, or no room for the output. The page
|
|
326
|
+
drops the move it was preparing when the proxy stops asking for it.
|
|
327
|
+
6. **The page switches a variant when the rung is ready**, and for a request
|
|
328
|
+
that is not urgent only once it holds `minimumBufferSeconds`.
|
|
329
|
+
|
|
330
|
+
## A place on the machine
|
|
331
|
+
|
|
332
|
+
How many encoders ONE output may run is answered per output
|
|
333
|
+
(`EncodeRuns.maxRunsForOutput`) and is always at least one. What refuses the
|
|
334
|
+
encoder that would slow everybody is `EncodeAdmission`, across every output at
|
|
335
|
+
once (roadmap item 97, step 13).
|
|
336
|
+
|
|
337
|
+
1. **The unit is seconds of work per second of film**, the one the quality
|
|
338
|
+
offer judges steps in (`EncodeCost.loadOfOutput`). Costs add, and a set of
|
|
339
|
+
encoders is affordable when the machine, corrected for the share of it
|
|
340
|
+
nobody has priced, still makes a second of film per second. A soundtrack is
|
|
341
|
+
a small fraction of a picture; counting processes would refuse a viewer
|
|
342
|
+
their sound. On the addon host the sum reproduces the measurement: two 1080p
|
|
343
|
+
encodes at 1.96x alone cost 1.02 s/s and make 0.98x, measured 0.99x and
|
|
344
|
+
0.98x.
|
|
345
|
+
2. **A place is held by an OUTPUT**: by every live encoder at its output's
|
|
346
|
+
cost, and by every output a present viewer is being prepared onto — a step
|
|
347
|
+
being warmed or a move to another limit — once however many wait for it.
|
|
348
|
+
The second is read off the viewers' own records (`Viewers.outputsBeingPrepared`),
|
|
349
|
+
so a place ends with its record on whichever path that record is cleared,
|
|
350
|
+
and there is no register of its own to keep in step.
|
|
351
|
+
3. **Asked before a preparation is recorded**, in the same synchronous stretch
|
|
352
|
+
as the record: two preparations onto two different outputs with room for
|
|
353
|
+
one cannot both be admitted, because the second is asked after the first is
|
|
354
|
+
written. Refused, a move answers `noPlace` and the budget goes to a lower
|
|
355
|
+
height (never a higher one); a warm-up answers `output-unavailable`.
|
|
356
|
+
4. **The plan is bounded by it** (`EncodeOrchestrator.#withinMachine`): an
|
|
357
|
+
output may run no more encoders than still fit, and a new one on a full
|
|
358
|
+
machine gets none until one elsewhere ends.
|
|
359
|
+
5. **An admitted encoder is never taken away**, and a place promised to a
|
|
360
|
+
preparation is kept: the machine's answer is never below what already runs
|
|
361
|
+
on an output, nor below one for an output a preparation holds.
|
|
362
|
+
|
|
363
|
+
Not solved: places go to whoever asks first, so a second encoder catching up on
|
|
364
|
+
one output can hold the place a viewer opening another film waits for.
|
|
365
|
+
|
|
141
366
|
## What each of the eight other places used to do
|
|
142
367
|
|
|
143
368
|
Before 2026-09-08 the plan was one opinion among nine. Each of these placed or
|
|
@@ -153,7 +378,7 @@ killed encoders by a rule of its own; each now states the fact it knows.
|
|
|
153
378
|
| the hardware encoder failed | start a run at the dead one's start | what this host encodes with has changed |
|
|
154
379
|
| the input came back | start a run at the last requested segment | decide again |
|
|
155
380
|
| the cut table was corrected | restart the members at the measured instant | this run is producing in the wrong place; the instant is in the file's table |
|
|
156
|
-
| the bitrate cap changed | restart at where the encoder had got to |
|
|
381
|
+
| the bitrate cap changed | restart at where the encoder had got to | nothing: a limit is part of an output since 2.87.0, so another limit is another output and no running encoder is ever told a new one |
|
|
157
382
|
|
|
158
383
|
A **seek** is not in that list because it was already reduced to one thing: it
|
|
159
384
|
puts the viewer where they are. The settle timer behind it, its cooldown and its
|
|
@@ -194,7 +419,7 @@ comparable: `download-architecture.md`.
|
|
|
194
419
|
## What the master playlist declares, and why it is not cosmetic
|
|
195
420
|
|
|
196
421
|
`BANDWIDTH` and `AVERAGE-BANDWIDTH` per variant, both measured
|
|
197
|
-
(`services/output/rates.js`):
|
|
422
|
+
(`services/encode/output/rates.js`):
|
|
198
423
|
|
|
199
424
|
- the average is the file's own length over its duration, exact and known when
|
|
200
425
|
the session is created;
|
|
@@ -434,6 +659,21 @@ Three things follow, and each replaced a guess:
|
|
|
434
659
|
open-piece.js` did all three of those by guessing and is gone, along with the
|
|
435
660
|
session manager's copy of it.
|
|
436
661
|
|
|
662
|
+
## How a request for a segment ends
|
|
663
|
+
|
|
664
|
+
A request steers no encoder. The file is served if the store holds it whole;
|
|
665
|
+
otherwise the request waits for the store's own announcement that the segment
|
|
666
|
+
was published (`SegmentStore.waitFor`), for the waits of that output to be
|
|
667
|
+
invalidated (a viewer left the rung), or for the requester to go. How long it
|
|
668
|
+
may wait is the requester's: the page states it (`X-Hold-Ms`, its own deadline
|
|
669
|
+
less the round trip it has measured) and the route answers "retry" by then. A
|
|
670
|
+
requester that states nothing is held until it closes the connection. There is
|
|
671
|
+
no poll and no hold time chosen by the proxy.
|
|
672
|
+
|
|
673
|
+
Which output a request is answered by is decided in the encoding component
|
|
674
|
+
(`encode/OutputOpening.js`), without knowing who asked; the server operation
|
|
675
|
+
(`server/ViewerRequests.js`) places the requester on it.
|
|
676
|
+
|
|
437
677
|
## What is checked
|
|
438
678
|
|
|
439
679
|
`test/one-authority.test.js` holds the shape: one caller of `#startEncodeRun`,
|
|
@@ -442,7 +682,7 @@ reading its own map, and the soundtrack's start instant read off the table
|
|
|
442
682
|
rather than handed in.
|
|
443
683
|
|
|
444
684
|
`test/priority-map-per-output.test.js` holds the two scopes, over the real
|
|
445
|
-
viewer registry, the real `
|
|
685
|
+
viewer registry, the real `OutputCatalog` and the real `PriorityOrchestrator`.
|
|
446
686
|
|
|
447
687
|
`test/encode-plan.test.js` holds the arithmetic, including that every encoder
|
|
448
688
|
stops when nobody is watching the output.
|
|
@@ -451,3 +691,39 @@ stops when nobody is watching the output.
|
|
|
451
691
|
name is finished — the last one of a run included — one under a working name is
|
|
452
692
|
not and cannot be reached, closing it is one rename, and clearing up after one
|
|
453
693
|
run leaves every other run's work alone.
|
|
694
|
+
|
|
695
|
+
## Per-machine calibration and admission (roadmap item 97, step 14)
|
|
696
|
+
|
|
697
|
+
At startup, `encode/calibration.js` qualifies every mode the proxy may select,
|
|
698
|
+
including the software encoder kept for runtime fallback. Each mode uses the
|
|
699
|
+
production arguments on a generated test clip, and every segment must decode
|
|
700
|
+
independently. `encode/fingerprint.js` keys the results to the FFmpeg and x264
|
|
701
|
+
versions, CPU model and thread count for software, or FFmpeg version, device
|
|
702
|
+
model and driver for hardware. A changed key does not reuse measurements from
|
|
703
|
+
the previous configuration.
|
|
704
|
+
|
|
705
|
+
`encode/throughput.js` measures qualified modes at five frame sizes from
|
|
706
|
+
256×144 through 3840×2160. Between measured sizes it interpolates and subtracts
|
|
707
|
+
the measured interpolation error; it does not offer a size outside the measured
|
|
708
|
+
range. Calibration stops measuring slower modes and larger sizes once a faster
|
|
709
|
+
mode cannot sustain realtime. If a detected device has no qualified mode, the
|
|
710
|
+
proxy uses software encoding.
|
|
711
|
+
|
|
712
|
+
`encode/LocalObservations.js` stores observations from admitted encodes in the
|
|
713
|
+
proxy's `local-observations.json`. It keeps speed without competing encodes,
|
|
714
|
+
average and peak segment rates, output preparation time and the viewer buffer at
|
|
715
|
+
a transition. A matching fingerprint is required. These observations refine
|
|
716
|
+
output cost, the peak estimate of an unbounded hardware encoder and transition
|
|
717
|
+
preparation timing; an empty or stale file leaves the base policy in force.
|
|
718
|
+
|
|
719
|
+
`encode/EncodeAdmission.js` counts running encoders and outputs that viewers are
|
|
720
|
+
being placed on or prepared to use. Opening an output checks capacity and
|
|
721
|
+
records the viewer in one synchronous call through `OutputOpening` and
|
|
722
|
+
`ViewerRequests`; a second simultaneous opening sees the first reservation. If
|
|
723
|
+
no measured mode or remaining capacity is available, the proxy answers 409
|
|
724
|
+
`no-capacity` before playback. Unknown costs on occupied outputs count as
|
|
725
|
+
unknown capacity, not free capacity. The health report exposes safe encoding
|
|
726
|
+
headroom, and the pool's file-specific answer uses the same occupied-cost data.
|
|
727
|
+
If the opening is refused, the page asks the pool and prepares the file again
|
|
728
|
+
through an eligible proxy before showing video. Each refused proxy is excluded
|
|
729
|
+
for that opening attempt, so stale pool answers cannot make the page cycle.
|
package/docs/encode-run-state.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
# The encoder run — states and transitions
|
|
5
5
|
|
|
6
6
|
One run of one ffmpeg inside one transcode session. The table this is drawn
|
|
7
|
-
from is executed by `services/
|
|
7
|
+
from is executed by `services/encode/EncodeRuns.js`; every transition a real
|
|
8
8
|
run makes is logged as state, event and target, so a run that takes an edge
|
|
9
9
|
absent here is a violation the log names.
|
|
10
10
|
|
|
@@ -69,7 +69,7 @@ stateDiagram-v2
|
|
|
69
69
|
## What each state answers
|
|
70
70
|
|
|
71
71
|
Outputs depend on the state alone — computed here by calling the same
|
|
72
|
-
functions the
|
|
72
|
+
functions the encoding component (`EncodeRuns`) calls.
|
|
73
73
|
|
|
74
74
|
| state | reads its input | can be signalled | a missing segment | on the wire | may restart |
|
|
75
75
|
|---|---|---|---|---|---|
|
package/docs/logs.md
CHANGED
|
@@ -1,97 +1,97 @@
|
|
|
1
|
-
# Logs — where to look
|
|
2
|
-
|
|
3
|
-
No browser console copy-paste is needed: the page forwards its own log. But it
|
|
4
|
-
lands in TWO places depending on the phase of the session, and `docker logs` is
|
|
5
|
-
only one of them — see "Browser logs are in TWO places" below before concluding
|
|
6
|
-
anything from a half-session.
|
|
7
|
-
|
|
8
|
-
## HA proxy (aarch64, addon `b34a1737_torrent_tv_proxy`)
|
|
9
|
-
|
|
10
|
-
Image `b34a1737/aarch64-addon-torrent_tv_proxy:<version>` = `@torrent-tv/proxy` `<version>` (see `ha-addon/torrent_tv_proxy/config.yaml`).
|
|
11
|
-
|
|
12
|
-
- Console + file are the same: the proxy logs to both `stdout` and `/data/proxy.log` on start (`logging to /data/proxy.log as well as the console`).
|
|
13
|
-
- **Since 2.71.0 that is true of the torrent thread too, and it was not before.**
|
|
14
|
-
A worker thread loads its own instance of every module, so the logger's file
|
|
15
|
-
handle — set once, on the main thread — was null in the worker for the life of
|
|
16
|
-
the process. Measured over a whole 49 938-line file: zero lines from the piece
|
|
17
|
-
reader (`read "…"`, `supply "…"`, every wait and its cause) and zero from the
|
|
18
|
-
torrent pool, against 52 and 36 of them in `docker logs`, which every release
|
|
19
|
-
destroys. The worker now sends its lines to the main thread, which is the only
|
|
20
|
-
writer; a line from there reads `torrent-worker: …`.
|
|
21
|
-
- Via container:
|
|
22
|
-
```bash
|
|
23
|
-
ssh ha "sudo docker logs app_b34a1737_torrent_tv_proxy --tail 200"
|
|
24
|
-
ssh ha "sudo docker exec app_b34a1737_torrent_tv_proxy cat /data/proxy.log | tail -n 300"
|
|
25
|
-
```
|
|
26
|
-
- Filter by session: every transcode session logs its id, e.g. `003ed2fd-7c9b-4cd5-9d05-ff875ff2be23`, `hold segment-00068.mp4 failed`, `encode-run failed`, `EXITED_*`.
|
|
27
|
-
- Host file `/data/proxy.log` survives `docker logs` rotation; `core dumps: 1 present` line shows kept dump under `/data`.
|
|
28
|
-
|
|
29
|
-
SSH to HA requires `MACs hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,umac-128-etm@openssh.com` — server `OpenSSH_10.3` on `homeassistant.local` offers only `*-etm` (`Unable to negotiate` / `Corrupted MAC` otherwise). Already in `~/.ssh/config` `Host ha`.
|
|
30
|
-
|
|
31
|
-
## DO server (webauth.courses, `infra-server-1`)
|
|
32
|
-
|
|
33
|
-
- Server is `infra-server-1` (`ghcr.io/torrent-tv/server:latest`) on `do` (`206.189.97.152`).
|
|
34
|
-
- Frontend logs reach here only in the phases the proxy cannot be reached in — see "Browser logs are in TWO places" below. When they do, the server `console.log`s them with the prefix `[client <tag> <id> sig=<webrtcSessionId>]` (`server/routes/api/client-logs/post.js`), in the same container as its own lines.
|
|
35
|
-
- Via container:
|
|
36
|
-
```bash
|
|
37
|
-
ssh do "docker logs infra-server-1 --tail 200 | cat"
|
|
38
|
-
# filter a single viewing:
|
|
39
|
-
ssh do "docker logs infra-server-1 --tail 500 | grep 003ed2fd"
|
|
40
|
-
ssh do "docker logs infra-server-1 --tail 500 | grep '\[client'"
|
|
41
|
-
```
|
|
42
|
-
- No need to open eruda or copy the browser console on a phone — it is forwarded.
|
|
43
|
-
|
|
44
|
-
## Browser logs are in TWO places, split by phase — read both
|
|
45
|
-
|
|
46
|
-
Until 2026-09-13 the page's whole log went to the droplet's standard output,
|
|
47
|
-
which every release of the server destroys. A viewer reported a desync and a
|
|
48
|
-
frozen picture that day and the analysis ran on half the evidence, so the log
|
|
49
|
-
moved to the proxy — beside the proxy's own, on the same durable directory.
|
|
50
|
-
|
|
51
|
-
The split is by PHASE, and neither half is the whole session:
|
|
52
|
-
|
|
53
|
-
1. **Before a transport exists** — the page opening, the proxy being chosen, a
|
|
54
|
-
connection failing — there is nowhere else to send it, so it goes to the
|
|
55
|
-
SERVER (`infra-server-1`);
|
|
56
|
-
2. **From the moment the data channel is up**, the page's logger is given a
|
|
57
|
-
proxy sink (`loading.js`, `setProxySink`) and every batch goes to the PROXY
|
|
58
|
-
over that channel. This is the bulk of a viewing;
|
|
59
|
-
3. **At unload** it goes to the server again: `navigator.sendBeacon` is the only
|
|
60
|
-
thing a page that is going away can use, and a data channel is not;
|
|
61
|
-
4. **A batch that cannot reach the proxy** falls back to the server.
|
|
62
|
-
|
|
63
|
-
So a session that failed while connecting is on the droplet, a session that
|
|
64
|
-
failed while playing is on the addon host, and a complete reading of a long
|
|
65
|
-
failure needs both.
|
|
66
|
-
|
|
67
|
-
### The files on the proxy
|
|
68
|
-
|
|
69
|
-
One file per viewing session, in the proxy log's own directory (`/data` on the
|
|
70
|
-
addon host), named so the two halves join without guessing:
|
|
71
|
-
|
|
72
|
-
```
|
|
73
|
-
client-<YYYYMMDD-HHMMSS UTC of the session start>-<sessionId:8>-<torrent name:60>-<infoHash:8>.log
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
A session that has not chosen a torrent yet writes `no-torrent-yet` in that
|
|
77
|
-
last part — the file is keyed by the session id, not by the name, so choosing a
|
|
78
|
-
torrent later does NOT start a second file. Rotated at 16 MB to `<name>.log.1`.
|
|
79
|
-
|
|
80
|
-
```bash
|
|
81
|
-
ssh ha "sudo docker exec app_b34a1737_torrent_tv_proxy sh -c 'ls -la /data/client-*.log'"
|
|
82
|
-
ssh ha "sudo docker exec app_b34a1737_torrent_tv_proxy sh -c 'cat /data/client-20260913-*.log'"
|
|
83
|
-
```
|
|
84
|
-
|
|
85
|
-
Implementation: `utils/client-log-file.js`, route `routes/api/client-logs/post.js`.
|
|
86
|
-
|
|
87
|
-
## Quick triage
|
|
88
|
-
|
|
89
|
-
- Rewind/seek bug (hold 0ms → 500): HA `hold ... failed after 0ms → 500` + `encode-run` lines for the same `<sessionId>`, and DO `[client ...] fragLoadError / levelLoadError 500` for same `sn`.
|
|
90
|
-
- Cushion / link budget: HA `memory: rss=... anon=...` and `cushion` lines; DO `[eta]` / `[cushion]` from client.
|
|
91
|
-
- Update check: `ssh ha "sudo docker exec hassio_cli ha apps info b34a1737_torrent_tv_proxy | grep version"` and `ssh ha "sudo docker ps --filter name=app_b34a1737_torrent_tv_proxy"`.
|
|
92
|
-
|
|
93
|
-
## Related
|
|
94
|
-
|
|
95
|
-
- `ha-addon/CLAUDE.md` — addon build/update detour via `hassio_cli`, cache-bust via `config.yaml` version.
|
|
96
|
-
- `docs/container-architecture.md` — what container/track classes log and where.
|
|
97
|
-
- `routes/api/client-logs/post.js` and `server/routes/api/client-logs/post.js` — the two receivers; both sanitize the same way (control chars → space, `MAX_LINES`, `MAX_MSG_LEN`).
|
|
1
|
+
# Logs — where to look
|
|
2
|
+
|
|
3
|
+
No browser console copy-paste is needed: the page forwards its own log. But it
|
|
4
|
+
lands in TWO places depending on the phase of the session, and `docker logs` is
|
|
5
|
+
only one of them — see "Browser logs are in TWO places" below before concluding
|
|
6
|
+
anything from a half-session.
|
|
7
|
+
|
|
8
|
+
## HA proxy (aarch64, addon `b34a1737_torrent_tv_proxy`)
|
|
9
|
+
|
|
10
|
+
Image `b34a1737/aarch64-addon-torrent_tv_proxy:<version>` = `@torrent-tv/proxy` `<version>` (see `ha-addon/torrent_tv_proxy/config.yaml`).
|
|
11
|
+
|
|
12
|
+
- Console + file are the same: the proxy logs to both `stdout` and `/data/proxy.log` on start (`logging to /data/proxy.log as well as the console`).
|
|
13
|
+
- **Since 2.71.0 that is true of the torrent thread too, and it was not before.**
|
|
14
|
+
A worker thread loads its own instance of every module, so the logger's file
|
|
15
|
+
handle — set once, on the main thread — was null in the worker for the life of
|
|
16
|
+
the process. Measured over a whole 49 938-line file: zero lines from the piece
|
|
17
|
+
reader (`read "…"`, `supply "…"`, every wait and its cause) and zero from the
|
|
18
|
+
torrent pool, against 52 and 36 of them in `docker logs`, which every release
|
|
19
|
+
destroys. The worker now sends its lines to the main thread, which is the only
|
|
20
|
+
writer; a line from there reads `torrent-worker: …`.
|
|
21
|
+
- Via container:
|
|
22
|
+
```bash
|
|
23
|
+
ssh ha "sudo docker logs app_b34a1737_torrent_tv_proxy --tail 200"
|
|
24
|
+
ssh ha "sudo docker exec app_b34a1737_torrent_tv_proxy cat /data/proxy.log | tail -n 300"
|
|
25
|
+
```
|
|
26
|
+
- Filter by session: every transcode session logs its id, e.g. `003ed2fd-7c9b-4cd5-9d05-ff875ff2be23`, `hold segment-00068.mp4 failed`, `encode-run failed`, `EXITED_*`.
|
|
27
|
+
- Host file `/data/proxy.log` survives `docker logs` rotation; `core dumps: 1 present` line shows kept dump under `/data`.
|
|
28
|
+
|
|
29
|
+
SSH to HA requires `MACs hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,umac-128-etm@openssh.com` — server `OpenSSH_10.3` on `homeassistant.local` offers only `*-etm` (`Unable to negotiate` / `Corrupted MAC` otherwise). Already in `~/.ssh/config` `Host ha`.
|
|
30
|
+
|
|
31
|
+
## DO server (webauth.courses, `infra-server-1`)
|
|
32
|
+
|
|
33
|
+
- Server is `infra-server-1` (`ghcr.io/torrent-tv/server:latest`) on `do` (`206.189.97.152`).
|
|
34
|
+
- Frontend logs reach here only in the phases the proxy cannot be reached in — see "Browser logs are in TWO places" below. When they do, the server `console.log`s them with the prefix `[client <tag> <id> sig=<webrtcSessionId>]` (`server/routes/api/client-logs/post.js`), in the same container as its own lines.
|
|
35
|
+
- Via container:
|
|
36
|
+
```bash
|
|
37
|
+
ssh do "docker logs infra-server-1 --tail 200 | cat"
|
|
38
|
+
# filter a single viewing:
|
|
39
|
+
ssh do "docker logs infra-server-1 --tail 500 | grep 003ed2fd"
|
|
40
|
+
ssh do "docker logs infra-server-1 --tail 500 | grep '\[client'"
|
|
41
|
+
```
|
|
42
|
+
- No need to open eruda or copy the browser console on a phone — it is forwarded.
|
|
43
|
+
|
|
44
|
+
## Browser logs are in TWO places, split by phase — read both
|
|
45
|
+
|
|
46
|
+
Until 2026-09-13 the page's whole log went to the droplet's standard output,
|
|
47
|
+
which every release of the server destroys. A viewer reported a desync and a
|
|
48
|
+
frozen picture that day and the analysis ran on half the evidence, so the log
|
|
49
|
+
moved to the proxy — beside the proxy's own, on the same durable directory.
|
|
50
|
+
|
|
51
|
+
The split is by PHASE, and neither half is the whole session:
|
|
52
|
+
|
|
53
|
+
1. **Before a transport exists** — the page opening, the proxy being chosen, a
|
|
54
|
+
connection failing — there is nowhere else to send it, so it goes to the
|
|
55
|
+
SERVER (`infra-server-1`);
|
|
56
|
+
2. **From the moment the data channel is up**, the page's logger is given a
|
|
57
|
+
proxy sink (`loading.js`, `setProxySink`) and every batch goes to the PROXY
|
|
58
|
+
over that channel. This is the bulk of a viewing;
|
|
59
|
+
3. **At unload** it goes to the server again: `navigator.sendBeacon` is the only
|
|
60
|
+
thing a page that is going away can use, and a data channel is not;
|
|
61
|
+
4. **A batch that cannot reach the proxy** falls back to the server.
|
|
62
|
+
|
|
63
|
+
So a session that failed while connecting is on the droplet, a session that
|
|
64
|
+
failed while playing is on the addon host, and a complete reading of a long
|
|
65
|
+
failure needs both.
|
|
66
|
+
|
|
67
|
+
### The files on the proxy
|
|
68
|
+
|
|
69
|
+
One file per viewing session, in the proxy log's own directory (`/data` on the
|
|
70
|
+
addon host), named so the two halves join without guessing:
|
|
71
|
+
|
|
72
|
+
```
|
|
73
|
+
client-<YYYYMMDD-HHMMSS UTC of the session start>-<sessionId:8>-<torrent name:60>-<infoHash:8>.log
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
A session that has not chosen a torrent yet writes `no-torrent-yet` in that
|
|
77
|
+
last part — the file is keyed by the session id, not by the name, so choosing a
|
|
78
|
+
torrent later does NOT start a second file. Rotated at 16 MB to `<name>.log.1`.
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
ssh ha "sudo docker exec app_b34a1737_torrent_tv_proxy sh -c 'ls -la /data/client-*.log'"
|
|
82
|
+
ssh ha "sudo docker exec app_b34a1737_torrent_tv_proxy sh -c 'cat /data/client-20260913-*.log'"
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Implementation: `utils/client-log-file.js`, route `routes/api/client-logs/post.js`.
|
|
86
|
+
|
|
87
|
+
## Quick triage
|
|
88
|
+
|
|
89
|
+
- Rewind/seek bug (hold 0ms → 500): HA `hold ... failed after 0ms → 500` + `encode-run` lines for the same `<sessionId>`, and DO `[client ...] fragLoadError / levelLoadError 500` for same `sn`.
|
|
90
|
+
- Cushion / link budget: HA `memory: rss=... anon=...` and `cushion` lines; DO `[eta]` / `[cushion]` from client.
|
|
91
|
+
- Update check: `ssh ha "sudo docker exec hassio_cli ha apps info b34a1737_torrent_tv_proxy | grep version"` and `ssh ha "sudo docker ps --filter name=app_b34a1737_torrent_tv_proxy"`.
|
|
92
|
+
|
|
93
|
+
## Related
|
|
94
|
+
|
|
95
|
+
- `ha-addon/CLAUDE.md` — addon build/update detour via `hassio_cli`, cache-bust via `config.yaml` version.
|
|
96
|
+
- `docs/container-architecture.md` — what container/track classes log and where.
|
|
97
|
+
- `routes/api/client-logs/post.js` and `server/routes/api/client-logs/post.js` — the two receivers; both sanitize the same way (control chars → space, `MAX_LINES`, `MAX_MSG_LEN`).
|