studio-engine 0.95.0 → 0.96.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +131 -16
- data/README.md +193 -12
- data/app/javascript/studio/application.js +2 -0
- data/app/javascript/studio/controllers/modal_host_controller.js +40 -0
- data/app/javascript/studio/modal_host.js +574 -0
- data/app/models/concerns/sluggable.rb +11 -1
- data/app/models/studio/knowledge_doc/recording.rb +214 -0
- data/app/models/studio/knowledge_doc.rb +3 -0
- data/app/views/layouts/studio/_head.html.erb +10 -2
- data/app/views/studio/emails/index.html.erb +13 -0
- data/app/views/studio/modals/_host.html.erb +33 -597
- data/app/views/studio/modals/_load_convention.html.erb +6 -27
- data/app/views/studio/modals/_scoped_host.html.erb +14 -257
- data/app/views/studio/modals/auth/_resend_footer.html.erb +2 -2
- data/app/views/studio/modals/blocks/_processing_card.html.erb +1 -1
- data/app/views/style/_modals.html.erb +12 -15
- data/db/migrate/20261008120000_add_recording_to_studio_knowledge_docs.rb +19 -0
- data/lib/studio/component_gallery.rb +13 -3
- data/lib/studio/email_image_generator.rb +125 -0
- data/lib/studio/engine.rb +2 -5
- data/lib/studio/knowledge_recording.rb +579 -0
- data/lib/studio/knowledge_transcript.rb +185 -0
- data/lib/studio/s3/multipart.rb +193 -0
- data/lib/studio/theme_resolver.rb +11 -22
- data/lib/studio/version.rb +1 -1
- data/lib/studio.rb +33 -1
- data/lib/tasks/studio_knowledge.rake +89 -0
- metadata +10 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: f7dde1a30b166a051c73369b581da7849d842cbca707c106ee331563b88ebf5a
|
|
4
|
+
data.tar.gz: b5f02ca221a1f0e623eb6524f5d99874e4f40c22ff7ab5a1f2241f4a4893a6e3
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: c631412bfcc7c90633dd0c7b8bcab6422dc923e5f4c8dc32d26aae01d1868179f290913fe3448656a12a097cfc70b19ca9cf9d5ea26e057e7cd9e9baaa4c3904
|
|
7
|
+
data.tar.gz: 82a12672dec7dee2b1a0fc86a9c1b2c0a13e79b5ee0cb22437156b8fc93749515b21ebd457811e7c3b8063a23b3adf6201613332c8e8eaeb2d27bfce5edd5223
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,118 @@ The format is [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This pro
|
|
|
4
4
|
|
|
5
5
|
## Unreleased
|
|
6
6
|
|
|
7
|
+
## 0.96.0 — 2026-10-08
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **A knowledge document can hold its meeting's recording.** The backend only;
|
|
12
|
+
no page changes yet. `Studio::KnowledgeDoc` gains:
|
|
13
|
+
- `attach_recording!(path, filename: nil, source_url: nil)`: copies a local
|
|
14
|
+
audio or video file into the app's private bucket beside the document, in
|
|
15
|
+
16 MB parts, never whole in memory, and saves the row. Replacing a
|
|
16
|
+
recording moves the old object to `trash/` (`Studio::S3.delete`).
|
|
17
|
+
- `attach_recording_from_url!(url, filename: nil, source_url: nil)`: the same,
|
|
18
|
+
from an https download on port 443. Every hop goes through the engine's SSRF
|
|
19
|
+
guard (`Studio::ImageCache.vet_source_url!`) and connects to the address it
|
|
20
|
+
was vetted against. A download that ends short of its declared length, or
|
|
21
|
+
before a chunked body's last chunk, fails and stores nothing. Nothing of
|
|
22
|
+
the download URL is stored, printed or used as a name.
|
|
23
|
+
- `recording?`, `recording_kind` (`:video` or `:audio`), `recording_link` (the
|
|
24
|
+
external page, only when it is an http(s) link) and
|
|
25
|
+
`recording_url(expires_in:)`, a presigned GET good for six hours by default
|
|
26
|
+
(`RECORDING_URL_TTL`).
|
|
27
|
+
|
|
28
|
+
Only MP4, M4V, MOV, M4A, WebM, MP3, WAV and Ogg are stored, judged by the
|
|
29
|
+
file's first bytes; the stored content type comes from that check, never from
|
|
30
|
+
a remote server's header. A recording may be up to 4 GB. An operator attaches
|
|
31
|
+
one with `bin/rails studio:knowledge:attach_recording ID=<id> FILE=<path>` (or
|
|
32
|
+
`URL=<https url>`, or `URL=-` to read the URL from standard input and keep it
|
|
33
|
+
off the command line). README, *Knowledge recordings*, walks every phase of a
|
|
34
|
+
fetch and of an upload with its time bound, its size bound and what a failure
|
|
35
|
+
leaves behind.
|
|
36
|
+
|
|
37
|
+
**Adopting it needs a migration:** `bin/rails studio_engine:install:migrations
|
|
38
|
+
&& bin/rails db:migrate` adds `recording_key`, `recording_mime_type`,
|
|
39
|
+
`recording_byte_size` and `recording_source_url` to `studio_knowledge_docs`.
|
|
40
|
+
An app that has not run it keeps working: `recording?` is false, and only the
|
|
41
|
+
two attach methods raise (`MissingRecordingColumns`, naming the fix).
|
|
42
|
+
|
|
43
|
+
The multipart upload is tested against a stubbed client only. That a live R2
|
|
44
|
+
bucket accepts it is not yet verified.
|
|
45
|
+
- `Studio::S3::Multipart.upload_file(key:, path:, content_type:, max_bytes:)`
|
|
46
|
+
(`require "studio/s3/multipart"`), a streaming multipart upload of a local
|
|
47
|
+
file. It uses its own client with `request_checksum_calculation` and
|
|
48
|
+
`response_checksum_validation` at `when_required` and sends a Content-MD5 with
|
|
49
|
+
each part; `Studio::S3.client` is unchanged. The upload is aborted on any
|
|
50
|
+
error or signal, and an object whose size cannot be confirmed is deleted.
|
|
51
|
+
- `Studio::KnowledgeTranscript.parse(text)` (`require
|
|
52
|
+
"studio/knowledge_transcript"`): transcript text to cues of `seconds`,
|
|
53
|
+
`speaker` and `text`, for both layouts the layer holds (`0:02 - Speaker` with
|
|
54
|
+
the text indented below, and `Speaker • 0:02` with the text below), with
|
|
55
|
+
`M:SS` and `H:MM:SS` stamps. It reads at most 2 MB and 5,000 cues, never
|
|
56
|
+
raises, and answers no cues for text that is not a transcript. `read(text)`
|
|
57
|
+
also says whether a bound cut the reading short.
|
|
58
|
+
- **An "Email image generator" link at the top of `/admin/emails`, set per
|
|
59
|
+
app.** `config.email_manager_generator_url` takes an http(s) URL or a callable
|
|
60
|
+
that receives the request; `email_manager_generator_label` (default "Email image
|
|
61
|
+
generator") and `email_manager_generator_description` override the wording.
|
|
62
|
+
Unset, nil or blank draws nothing, and the page renders byte-for-byte as
|
|
63
|
+
before. A non-http(s) String raises `ArgumentError` at boot; a callable
|
|
64
|
+
answering one draws no link. The link opens in a new tab with
|
|
65
|
+
`rel="noopener noreferrer"`. See README, *The image generator link*.
|
|
66
|
+
|
|
67
|
+
### Changed
|
|
68
|
+
|
|
69
|
+
- **The modal host and the scoped host move to an ES module and a Stimulus
|
|
70
|
+
controller.** `studio/modals/_host` and `studio/modals/_scoped_host` render
|
|
71
|
+
markup and styles only; their inline scripts (the shared host's store,
|
|
72
|
+
registry and cleanup, the load convention it rendered, and the scoped host's
|
|
73
|
+
copy of the stack) are gone. The stacks, the focus trap both share, the
|
|
74
|
+
animation and card-width registries and `holdAtLeast` are `studio/modal_host`,
|
|
75
|
+
with `node:test` unit tests; each host's `<template>` carries
|
|
76
|
+
`data-studio-controller="modal-host"` and `data-modal-host-store-value`
|
|
77
|
+
(the scoped host adds `data-modal-host-scoped-value="true"`), and the
|
|
78
|
+
`modal-host` controller binds the bfcache and Turbo snapshot cleanup to it.
|
|
79
|
+
`layouts/studio/_head` imports `studio/modal_host` by its own nonced module
|
|
80
|
+
tag, as it does `studio/alpine_stores`, so a failed boot keeps the stores.
|
|
81
|
+
Stores register on `alpine:init`, before a Turbo render, and on connect, and
|
|
82
|
+
only where a host renders: an app with no shared host still has no
|
|
83
|
+
`$store.modals`. `$store.modals`, every scoped `$store.<name>`,
|
|
84
|
+
`window.ModalAnimations` and `window.StudioModals` keep their names and API.
|
|
85
|
+
- **`window.ModalAnimations` and `window.StudioModals` (`CARD_WIDTHS`,
|
|
86
|
+
`DEFAULT_CARD_WIDTH`, `MIN_LOAD_MS`, `holdAtLeast`) are published on every
|
|
87
|
+
page**, once the document has parsed, merged over whatever the app defined
|
|
88
|
+
first. They were published only by a host's inline script, so an app's
|
|
89
|
+
definition had to come before the host render; anywhere in the page works now.
|
|
90
|
+
- **`studio/modals/_load_convention` renders nothing.** The convention it carried
|
|
91
|
+
is `studio/modal_host`'s; the partial stays so a render of it by name still
|
|
92
|
+
resolves.
|
|
93
|
+
|
|
94
|
+
### Breaking
|
|
95
|
+
|
|
96
|
+
- **A direct write to a persisted `Sluggable` slug is invalid.** `update(slug:)`,
|
|
97
|
+
or assigning `slug` and saving, on a row that already has a slug fails
|
|
98
|
+
validation with `errors[:slug]` of kind `:readonly` ("changes only through
|
|
99
|
+
rename_slug!"); `update!` and `save!` raise `ActiveRecord::RecordInvalid`.
|
|
100
|
+
Change a slug with `rename_slug!` or `rename_slug`, which also cascade to the
|
|
101
|
+
child columns. Filling a blank slug and writes that skip validations
|
|
102
|
+
(`update_column`, `update_all`) are unaffected.
|
|
103
|
+
|
|
104
|
+
### Fixed
|
|
105
|
+
|
|
106
|
+
- **The component gallery is not mounted when lookbook loads before
|
|
107
|
+
studio-engine.** A host that lists `gem "lookbook"` above `gem "studio-engine"`
|
|
108
|
+
drew the gallery anyway; it now draws no gallery or asset route, and the boot
|
|
109
|
+
warning says so and names the fix: move `gem "lookbook"` below
|
|
110
|
+
`gem "studio-engine"` in the Gemfile and restart.
|
|
111
|
+
- **`--color-danger-ink` clears AA on its own 10% tint.** It was derived
|
|
112
|
+
against the theme surfaces only, so `bg-danger/10 text-danger-ink` (the danger
|
|
113
|
+
badge and chip) measured 4.10:1 light and 4.18:1 dark; it now counts its tint
|
|
114
|
+
as the warning and success inks do, and measures 4.61:1 and 4.53:1. The default
|
|
115
|
+
ink moves from `#BA3535` to `#AC3131` (light) and `#F48484` to `#F58F8F`
|
|
116
|
+
(dark). A consumer test that pins either hex, or that expects danger-ink to
|
|
117
|
+
fail on a danger tint, needs updating.
|
|
118
|
+
|
|
7
119
|
## 0.95.0 — 2026-10-07
|
|
8
120
|
|
|
9
121
|
### Security
|
|
@@ -127,24 +239,8 @@ The format is [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This pro
|
|
|
127
239
|
|
|
128
240
|
## 0.93.0 — 2026-10-07
|
|
129
241
|
|
|
130
|
-
## 0.92.2 — 2026-10-07
|
|
131
|
-
|
|
132
|
-
## 0.92.1 — 2026-10-07
|
|
133
|
-
|
|
134
242
|
### Added
|
|
135
243
|
|
|
136
|
-
- **The engine owns `error_logs`, `theme_settings` and `image_caches`.** Three
|
|
137
|
-
migrations (`ensure_error_logs_table`, `ensure_theme_settings_table`,
|
|
138
|
-
`ensure_image_caches_table`) create each table when it is absent and, on an app
|
|
139
|
-
that already has it, add only the engine columns it lacks, as nullable columns;
|
|
140
|
-
nothing is altered or dropped and no index is added. Adopt with
|
|
141
|
-
`bin/rails studio_engine:install:migrations` and `bin/rails db:migrate`; the
|
|
142
|
-
release's lock bump does both. Hosts no longer copy these tables from
|
|
143
|
-
`docs/NEW_APP_SETUP.md`.
|
|
144
|
-
- **A boot check names a missing engine column** (`Studio::HostSchema`). It
|
|
145
|
-
raises `Studio::HostSchemaError` in development and test, and logs (and reports
|
|
146
|
-
to Sentry) in production; `Studio.host_schema_check = :raise | :log | false`
|
|
147
|
-
overrides. Extra host columns are never reported, and rake tasks skip it.
|
|
148
244
|
- **`Sluggable#rename_slug!` changes a slug on purpose and cascades.** It updates
|
|
149
245
|
the row and every child column that holds the old slug in one transaction:
|
|
150
246
|
each `has_many`/`has_one` declared with `primary_key: :slug`, plus the pairs a
|
|
@@ -166,6 +262,25 @@ The format is [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This pro
|
|
|
166
262
|
Code that relied on a save moving the slug calls `rename_slug!(name_slug)`
|
|
167
263
|
instead.
|
|
168
264
|
|
|
265
|
+
## 0.92.2 — 2026-10-07
|
|
266
|
+
|
|
267
|
+
## 0.92.1 — 2026-10-07
|
|
268
|
+
|
|
269
|
+
### Added
|
|
270
|
+
|
|
271
|
+
- **The engine owns `error_logs`, `theme_settings` and `image_caches`.** Three
|
|
272
|
+
migrations (`ensure_error_logs_table`, `ensure_theme_settings_table`,
|
|
273
|
+
`ensure_image_caches_table`) create each table when it is absent and, on an app
|
|
274
|
+
that already has it, add only the engine columns it lacks, as nullable columns;
|
|
275
|
+
nothing is altered or dropped and no index is added. Adopt with
|
|
276
|
+
`bin/rails studio_engine:install:migrations` and `bin/rails db:migrate`; the
|
|
277
|
+
release's lock bump does both. Hosts no longer copy these tables from
|
|
278
|
+
`docs/NEW_APP_SETUP.md`.
|
|
279
|
+
- **A boot check names a missing engine column** (`Studio::HostSchema`). It
|
|
280
|
+
raises `Studio::HostSchemaError` in development and test, and logs (and reports
|
|
281
|
+
to Sentry) in production; `Studio.host_schema_check = :raise | :log | false`
|
|
282
|
+
overrides. Extra host columns are never reported, and rake tasks skip it.
|
|
283
|
+
|
|
169
284
|
### Fixed
|
|
170
285
|
|
|
171
286
|
- **A theme save no longer raises on apps without `theme_settings.slug`**
|
data/README.md
CHANGED
|
@@ -35,7 +35,7 @@ resolved.
|
|
|
35
35
|
- **Theme system**: Dynamic CSS custom properties generated from 7 role colors (primary, dark, light, success, accent, warning, danger). Dark/light mode toggle. Admin theme editor at `/admin/theme`.
|
|
36
36
|
- **UI primitives**: Shared component partials and CSS primitives such as `components/emoji_swap` for nav/sidebar emoji hover transitions.
|
|
37
37
|
- **Operator tooling**: Shared `studio/banners/environment` banner with Dev Mode + email connector controls and `studio/banners/impersonation`.
|
|
38
|
-
- **Sluggable concern**: a human-readable slug written once at create (`to_param` returns it); `rename_slug!` changes it and cascades to every child column in one transaction, and a refused rename answers 422
|
|
38
|
+
- **Sluggable concern**: a human-readable slug written once at create (`to_param` returns it); `rename_slug!` changes it and cascades to every child column in one transaction, any other write that changes a persisted slug is invalid, and a refused rename answers 422
|
|
39
39
|
- **ThemeSetting model**: Per-app DB overrides with fallback to config defaults
|
|
40
40
|
- **Geo**: `Studio::GeoDetection` places every visitor (IP → country + subdivision, session-cached), `Studio::GeoSetting` stores the operator's blocked countries and regions, `require_geo_allowed` locks whichever surfaces an app chooses, and the shared badge + `/admin/geo` manager ship with it. See [`docs/GEO.md`](docs/GEO.md).
|
|
41
41
|
- **Site identity and link previews**: `Studio::SiteIdentity` holds the app's title, description and image, edited at `/admin/link_preview` beside a live unfurl card and read anywhere through `Studio.site_identity`. Every page unfurls with it unless it calls `link_preview image:, title:, description:`, and `Studio::LinkPreviewBots` serves preview fetchers a slim page under iMessage's 1 MiB limit. Adopt with `bin/rails g studio:site_identity`. See [`docs/LINK_PREVIEW.md`](docs/LINK_PREVIEW.md).
|
|
@@ -477,9 +477,12 @@ Recognized props: `dismissible: false` disables escape/click-outside dismissal
|
|
|
477
477
|
animation from the registry (`'pop'` default, `'shake'`, `'slide'`).
|
|
478
478
|
|
|
479
479
|
`window.ModalAnimations` is the animation registry — each key maps a CSS class
|
|
480
|
-
to its duration.
|
|
481
|
-
|
|
482
|
-
|
|
480
|
+
to its duration. `studio/modal_host` (an ES module the engine head imports on
|
|
481
|
+
every page, which also owns `$store.modals`, the focus trap and
|
|
482
|
+
`window.StudioModals`) merges an app's entries over the engine defaults once
|
|
483
|
+
the document has parsed. To add a custom animation, define
|
|
484
|
+
`window.ModalAnimations` in a script anywhere in the page with your extra keys
|
|
485
|
+
and ship the matching CSS class in the app stylesheet:
|
|
483
486
|
|
|
484
487
|
```html
|
|
485
488
|
<script>
|
|
@@ -487,16 +490,17 @@ defaults) and ship the matching CSS class in the app stylesheet:
|
|
|
487
490
|
</script>
|
|
488
491
|
```
|
|
489
492
|
|
|
490
|
-
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
|
|
493
|
+
An inline script anywhere in the page, as above, or a host module (it
|
|
494
|
+
evaluates before the engine's) both merge. A script that assigns
|
|
495
|
+
`window.ModalAnimations` LATER, after the page has loaded, **replaces** the
|
|
496
|
+
merged registry rather than extending it: the store guards against this
|
|
497
|
+
(unknown keys and gutted registries fall back to the built-in `'pop'`), but
|
|
498
|
+
your other custom keys are lost unless the late script merges into the existing
|
|
499
|
+
object instead of assigning over it.
|
|
496
500
|
|
|
497
501
|
`window.StudioModals.CARD_WIDTHS` is the **card-width registry**, and it works
|
|
498
|
-
exactly like the animation one: define it
|
|
499
|
-
|
|
502
|
+
exactly like the animation one: define it in the page and your entries merge
|
|
503
|
+
over the engine defaults. Keys are modal ids, values are the
|
|
500
504
|
Tailwind `max-w-*` class that card should use. `DEFAULT_CARD_WIDTH` (default
|
|
501
505
|
`max-w-sm`) covers every id you don't name:
|
|
502
506
|
|
|
@@ -707,6 +711,29 @@ its helper names; drawing them there raises at route-load and kills every route
|
|
|
707
711
|
in the app. The gate covers only the page — the registry and the inherited
|
|
708
712
|
defaults are always on.
|
|
709
713
|
|
|
714
|
+
### The image generator link
|
|
715
|
+
|
|
716
|
+
An app can put a link to its own banner generator at the top of the page: a
|
|
717
|
+
short line and a button that opens the generator in a new tab. Unset (the
|
|
718
|
+
default), nil or blank draws nothing, and the page renders exactly as it did
|
|
719
|
+
before the setting existed.
|
|
720
|
+
|
|
721
|
+
```ruby
|
|
722
|
+
# config/initializers/studio.rb
|
|
723
|
+
config.email_manager_generator_url = "https://example.com/email-art"
|
|
724
|
+
# or worked out per request; the callable receives the request:
|
|
725
|
+
config.email_manager_generator_url = ->(request) { "#{request.base_url}/admin/email-art" }
|
|
726
|
+
|
|
727
|
+
config.email_manager_generator_label = "Header generator" # default "Email image generator"
|
|
728
|
+
config.email_manager_generator_description = "Open it, copy the prompt, paste it into Claude Code."
|
|
729
|
+
```
|
|
730
|
+
|
|
731
|
+
The default line reads "Make a new header with <app name>'s character model:
|
|
732
|
+
open the generator, copy the prompt, paste it into Claude Code." The URL must be
|
|
733
|
+
an absolute `http`/`https` URL with a host: any other String (a `javascript:`
|
|
734
|
+
URL, a relative path) raises `ArgumentError` at boot, and a callable that
|
|
735
|
+
answers one draws no link. Every value is escaped.
|
|
736
|
+
|
|
710
737
|
### The catalog
|
|
711
738
|
|
|
712
739
|
A registered email carries a key, a label, a description, what **type** it is,
|
|
@@ -1086,6 +1113,160 @@ destroyed before the object was trashed, so the task prints the
|
|
|
1086
1113
|
and the record must be re-attached by hand. `filename` is known only when the
|
|
1087
1114
|
upload carried a Content-Disposition; supply it otherwise.
|
|
1088
1115
|
|
|
1116
|
+
## Knowledge recordings
|
|
1117
|
+
|
|
1118
|
+
A knowledge document (`Studio::KnowledgeDoc`) is usually a meeting's transcript.
|
|
1119
|
+
Its recording can be stored beside it, in the same private bucket, so the audio
|
|
1120
|
+
or video does not stay on a notetaker's servers. This is the backend; the
|
|
1121
|
+
document page does not show a player yet.
|
|
1122
|
+
|
|
1123
|
+
**Adopting it needs a migration.** Run `bin/rails studio_engine:install:migrations
|
|
1124
|
+
&& bin/rails db:migrate`. It adds four nullable columns to
|
|
1125
|
+
`studio_knowledge_docs`: `recording_key`, `recording_mime_type`,
|
|
1126
|
+
`recording_byte_size` (bigint) and `recording_source_url`. An app that has not
|
|
1127
|
+
run it keeps working: `recording?` is false and `recording_link` is nil, and only
|
|
1128
|
+
the two attach methods raise (`Studio::KnowledgeDoc::Recording::MissingRecordingColumns`,
|
|
1129
|
+
before any byte moves).
|
|
1130
|
+
|
|
1131
|
+
```ruby
|
|
1132
|
+
doc.attach_recording!("/tmp/standup.mp4", source_url: "https://notes.example.com/calls/abc")
|
|
1133
|
+
doc.attach_recording_from_url!("https://files.example.com/standup.mp4", filename: "standup.mp4")
|
|
1134
|
+
doc.recording? # => true
|
|
1135
|
+
doc.recording_kind # => :video (or :audio)
|
|
1136
|
+
doc.recording_url # presigned GET, six hours
|
|
1137
|
+
doc.recording_link # the external page, when it is an http(s) link
|
|
1138
|
+
```
|
|
1139
|
+
|
|
1140
|
+
From a one-off dyno or a laptop:
|
|
1141
|
+
|
|
1142
|
+
```bash
|
|
1143
|
+
bin/rails studio:knowledge:attach_recording ID=42 FILE=/tmp/standup.mp4
|
|
1144
|
+
bin/rails studio:knowledge:attach_recording ID=42 URL=https://files.example.com/standup.mp4
|
|
1145
|
+
echo "$DOWNLOAD_URL" | bin/rails studio:knowledge:attach_recording ID=42 URL=-
|
|
1146
|
+
```
|
|
1147
|
+
|
|
1148
|
+
`NAME=<file name>` names the object (a fetched recording is otherwise
|
|
1149
|
+
`recording.<ext>`) and `SOURCE_URL=<page>` sets the link.
|
|
1150
|
+
|
|
1151
|
+
**Keep a download URL off the command line.** It carries a credential, and a
|
|
1152
|
+
command line is logged: by shell history, by `ps`, and by a platform's record of
|
|
1153
|
+
the command it started. `URL=-` reads the URL from standard input instead.
|
|
1154
|
+
Whether a given platform's one-off runner passes standard input through has not
|
|
1155
|
+
been tested here; check before relying on it.
|
|
1156
|
+
|
|
1157
|
+
**What is printed and stored.** The task prints the document's id, the bucket,
|
|
1158
|
+
the key, the type and the size. It does not print the title or the link (a
|
|
1159
|
+
title is content of the layer, and the output goes to a log). The download URL
|
|
1160
|
+
is never stored, never printed, and no part of it names the object: a failure
|
|
1161
|
+
message names the URL's scheme and host only. `SOURCE_URL` is the one URL kept.
|
|
1162
|
+
|
|
1163
|
+
**What is stored.** Audio and video only: MP4, M4V, MOV, M4A, WebM, MP3, WAV and
|
|
1164
|
+
Ogg. The file's first 16 bytes decide, and the stored content type comes from
|
|
1165
|
+
that check (`Studio::KnowledgeRecording::TYPES`), never from a header a remote
|
|
1166
|
+
server sent. A file name's extension picks within a container (`.m4a` is
|
|
1167
|
+
`audio/mp4`) and must agree with the bytes; an extension off the list is
|
|
1168
|
+
refused, and a name with no extension is judged by its bytes alone. An old
|
|
1169
|
+
QuickTime file with no `ftyp` box is refused; re-export it as MP4.
|
|
1170
|
+
|
|
1171
|
+
**The object.** `knowledge/<entity>/<path>/<timestamp>-<random>-recording-<name>`,
|
|
1172
|
+
beside the document's own object. Replacing a recording uploads the new object,
|
|
1173
|
+
saves the row, then moves the old object to `trash/` for three days (see
|
|
1174
|
+
[Trash and restore](#trash-and-restore)).
|
|
1175
|
+
|
|
1176
|
+
**The playback URL** is good for six hours (`RECORDING_URL_TTL`), not the
|
|
1177
|
+
document link's fifteen minutes: a player makes a new request each time the
|
|
1178
|
+
viewer seeks, and each one is checked against the URL's expiry.
|
|
1179
|
+
|
|
1180
|
+
### The fetch, phase by phase
|
|
1181
|
+
|
|
1182
|
+
`attach_recording_from_url!` is a server-side fetch of a URL the caller
|
|
1183
|
+
supplied, run with the bucket's credentials in the process. Each hop, the first
|
|
1184
|
+
URL and every redirect, must be https on port 443, may carry no user or
|
|
1185
|
+
password, and goes through `Studio::ImageCache.vet_source_url!` (see
|
|
1186
|
+
[Remote image URLs](#remote-image-urls)); the connection is then made to an
|
|
1187
|
+
address that hop was vetted against. A host that vets to no address is refused.
|
|
1188
|
+
|
|
1189
|
+
The whole fetch has one deadline, `FETCH_DEADLINE` (3,600 seconds), and every
|
|
1190
|
+
time bound below is also cut to what is left of it. It is enforced on every
|
|
1191
|
+
read and write of the socket, by `Studio::KnowledgeRecording::Meter`, not only
|
|
1192
|
+
as body arrives.
|
|
1193
|
+
|
|
1194
|
+
| Phase | Time bound | Size bound | Left if the process dies here |
|
|
1195
|
+
|-------|------------|------------|-------------------------------|
|
|
1196
|
+
| URL check and DNS, each hop | 6 s a lookup (`Studio::ImageCache::RESOLVE_DEADLINE`). **Not cut to the fetch's time left**: a lookup in flight can run the fetch up to 6 s over | URL 8,192 bytes (`MAX_URL_BYTES`) | an empty temporary file |
|
|
1197
|
+
| Connect and TLS, each address | 20 s (`CONNECT_DEADLINE`); at most 4 addresses a hop (`MAX_ADDRESSES`) | the TLS library's own handshake limits | an empty temporary file |
|
|
1198
|
+
| Sending the request | the fetch's time left | the URL | an empty temporary file |
|
|
1199
|
+
| Status line, `1xx` responses, headers | 30 s for all of them together (`HEADER_DEADLINE`) | 64 KB for all of them together (`MAX_UNDELIVERED_BYTES`) | an empty temporary file |
|
|
1200
|
+
| A redirect | counts as a hop: 3 followed (`MAX_REDIRECTS`), each vetted and bounded as above | Location 8,192 bytes; its body is never read | an empty temporary file |
|
|
1201
|
+
| Chunk-size lines and trailers | the fetch's time left; 30 s for any one silent wait | 64 KB between two pieces of body | a partial temporary file |
|
|
1202
|
+
| The body | the fetch's time left; 30 s for any one silent wait (`Studio::ImageCache::READ_TIMEOUT`) | 4 GB (`MAX_BYTES`), counted before each write; a declared `Content-Length` over it is refused before the body | a partial temporary file |
|
|
1203
|
+
|
|
1204
|
+
- **A complete body or nothing.** A response that declares a `Content-Length`
|
|
1205
|
+
and sends any other number of bytes fails (`FetchFailed`) before anything is
|
|
1206
|
+
uploaded, and so does a chunked body that ends before its last chunk. The
|
|
1207
|
+
recording already stored is untouched. **A body with no `Content-Length` and
|
|
1208
|
+
no chunking cannot be checked**: it ends when the server closes the
|
|
1209
|
+
connection, and a download cut short there looks the same as one that
|
|
1210
|
+
finished. Compare the printed size with the source's when a server sends such
|
|
1211
|
+
a response.
|
|
1212
|
+
- **The temporary file** is in the system temporary directory, takes up to the
|
|
1213
|
+
recording's size (4 GB at most) and is deleted on every way out, an interrupt
|
|
1214
|
+
or `SIGTERM` included. After a `SIGKILL` it stays on that machine's disk; on a
|
|
1215
|
+
one-off dyno that disk goes with the dyno.
|
|
1216
|
+
- A compressed body is refused, a redirect or error response's body is never
|
|
1217
|
+
read, a failed request is not retried, and only a failure to connect moves on
|
|
1218
|
+
to the host's next address.
|
|
1219
|
+
- Any port but 443 is refused, on the first URL and on every redirect.
|
|
1220
|
+
- The meter reads Net::HTTP's socket through two of its instance variables,
|
|
1221
|
+
because net-http has no hook for it. On a net-http where they are not what it
|
|
1222
|
+
expects, the fetch is refused instead of run unmetered. Written against
|
|
1223
|
+
net-http 0.9.1 and net-protocol 0.2.2.
|
|
1224
|
+
|
|
1225
|
+
### The upload, phase by phase
|
|
1226
|
+
|
|
1227
|
+
`Studio::S3::Multipart.upload_file` sends the file in 16 MB parts, one in memory
|
|
1228
|
+
at a time (`PART_SIZE`), at most 10,000 of them (`MAX_PARTS`). The whole upload
|
|
1229
|
+
has a deadline, `UPLOAD_DEADLINE` (7,200 seconds), read before each part. One
|
|
1230
|
+
request to the store is bounded by the SDK client, not by that deadline: 15 s to
|
|
1231
|
+
connect, 60 s for any one silent read, 3 retries (the SDK's defaults, pinned by
|
|
1232
|
+
a test).
|
|
1233
|
+
|
|
1234
|
+
| Phase | Time bound | Size bound | Left in the bucket if it fails here | Left after a `SIGKILL` here |
|
|
1235
|
+
|-------|------------|------------|--------------------------------------|------------------------------|
|
|
1236
|
+
| Create the upload | one SDK request | none | nothing | an upload with no parts |
|
|
1237
|
+
| Each part | `UPLOAD_DEADLINE` before it starts; one SDK request | 16 MB; the file's size at open; 4 GB | nothing: the upload is aborted on any error or signal | the parts sent so far |
|
|
1238
|
+
| Complete | one SDK request | none | nothing: aborted. If the store completed and the answer was lost, a whole object no row points at | the parts, or a whole object |
|
|
1239
|
+
| Confirm the stored size | one SDK request | none | nothing: the object is deleted on a wrong size, an error or a signal | a whole object no row points at |
|
|
1240
|
+
| Save the row | the database's own statement timeout; the engine sets none | none | nothing: the new object is moved to `trash/` on any error or signal | a whole object no row points at |
|
|
1241
|
+
| Trash the replaced recording | three SDK requests | 5 GiB a copy (`Studio::S3::Trash::MAX_COPY_BYTES`) | the row points at the new recording; the old object stays where it was | the same |
|
|
1242
|
+
|
|
1243
|
+
- **Parts left by a killed process** are not visible in a bucket listing. The
|
|
1244
|
+
bucket's abort-incomplete-multipart lifecycle rule removes them; R2's default
|
|
1245
|
+
rule does so after 7 days. No process can clean up after its own `SIGKILL`.
|
|
1246
|
+
- **A whole object no row points at** sits under
|
|
1247
|
+
`knowledge/<entity>/<path>/` with `-recording-` in its name. List that prefix
|
|
1248
|
+
and compare against `recording_key` to find one.
|
|
1249
|
+
- **Two attaches to one document at the same time** leave one recording and no
|
|
1250
|
+
orphan: the save reads the key it replaces from the database under a row
|
|
1251
|
+
lock, so whichever saves second trashes the first one's object.
|
|
1252
|
+
|
|
1253
|
+
**Checksums and R2.** `Studio::S3::Multipart` uploads through its own client,
|
|
1254
|
+
built from `Studio::S3.client_options` with `request_checksum_calculation` and
|
|
1255
|
+
`response_checksum_validation` at `when_required`, and sends a Content-MD5 with
|
|
1256
|
+
each part. Newer `aws-sdk-s3` versions add a CRC32 by default, and R2 has refused
|
|
1257
|
+
some of the combinations that produces. The request shape is tested against a
|
|
1258
|
+
stubbed client; **a multipart upload to a live R2 bucket has not been verified.**
|
|
1259
|
+
|
|
1260
|
+
### Transcripts
|
|
1261
|
+
|
|
1262
|
+
`Studio::KnowledgeTranscript.parse(text)` (`require
|
|
1263
|
+
"studio/knowledge_transcript"`) turns transcript text into cues of `seconds`,
|
|
1264
|
+
`speaker` and `text`. It reads two layouts, `0:02 - Speaker` with the text
|
|
1265
|
+
indented below and `Speaker • 0:02` with the text below, and `M:SS` or
|
|
1266
|
+
`H:MM:SS` stamps. It reads at most 2 MB of text and 5,000 cues, keeps 8 KB of
|
|
1267
|
+
text per cue, never raises, and answers no cues for text that is not a
|
|
1268
|
+
transcript. `read(text)` answers the cues and whether a bound cut them short.
|
|
1269
|
+
|
|
1089
1270
|
## Remote image URLs
|
|
1090
1271
|
|
|
1091
1272
|
`Studio::ImageCache.validate_source_url!(url)` is the check to run before the
|
|
@@ -18,6 +18,7 @@
|
|
|
18
18
|
// (Studio::Engine.javascript_boot_graph), so the boot costs one round trip.
|
|
19
19
|
import { Application, defaultSchema } from "@hotwired/stimulus"
|
|
20
20
|
import NavCollapseController from "studio/controllers/nav_collapse_controller"
|
|
21
|
+
import ModalHostController from "studio/controllers/modal_host_controller"
|
|
21
22
|
import { startPinnedStack } from "studio/pinned_stack"
|
|
22
23
|
import { installAlpineShims } from "studio/alpine_shims"
|
|
23
24
|
|
|
@@ -33,5 +34,6 @@ export const schema = {
|
|
|
33
34
|
|
|
34
35
|
const application = Application.start(document.documentElement, schema)
|
|
35
36
|
application.register("nav-collapse", NavCollapseController)
|
|
37
|
+
application.register("modal-host", ModalHostController)
|
|
36
38
|
|
|
37
39
|
export { application }
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
// modal-host: one modal host's lifetime, on the engine's Stimulus application
|
|
2
|
+
// (studio/application, which reads data-studio-controller). It sits on the
|
|
3
|
+
// host's outer <template>:
|
|
4
|
+
//
|
|
5
|
+
// <template x-if="$store.modals.current()"
|
|
6
|
+
// data-studio-controller="modal-host"
|
|
7
|
+
// data-modal-host-store-value="modals"> (studio/modals/_host)
|
|
8
|
+
//
|
|
9
|
+
// <template x-if="$store.labModals.current()"
|
|
10
|
+
// data-studio-controller="modal-host"
|
|
11
|
+
// data-modal-host-store-value="labModals"
|
|
12
|
+
// data-modal-host-scoped-value="true"> (studio/modals/_scoped_host)
|
|
13
|
+
//
|
|
14
|
+
// The stack, the focus trap and the registries are studio/modal_host, which
|
|
15
|
+
// registers each host's store on alpine:init (a full load, where a controller
|
|
16
|
+
// connects too late) and before a Turbo render. connect() registers it too, for
|
|
17
|
+
// a Turbo visit and for markup that arrived any other way, and binds the
|
|
18
|
+
// bfcache and Turbo snapshot cleanup to this host while it is on the page.
|
|
19
|
+
// Registered by studio/application.
|
|
20
|
+
import { Controller } from "@hotwired/stimulus"
|
|
21
|
+
import { registerModalStore, clearStaleModals } from "studio/modal_host"
|
|
22
|
+
|
|
23
|
+
export default class extends Controller {
|
|
24
|
+
static values = { store: { type: String, default: "modals" }, scoped: Boolean }
|
|
25
|
+
|
|
26
|
+
connect() {
|
|
27
|
+
const name = this.storeValue
|
|
28
|
+
registerModalStore(window.Alpine, { store: name, scoped: this.scopedValue }, { doc: document, win: window })
|
|
29
|
+
|
|
30
|
+
this.onPageShow = (event) => { if (event.persisted) clearStaleModals(window, name) }
|
|
31
|
+
this.onBeforeCache = () => clearStaleModals(window, name)
|
|
32
|
+
window.addEventListener("pageshow", this.onPageShow)
|
|
33
|
+
document.addEventListener("turbo:before-cache", this.onBeforeCache)
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
disconnect() {
|
|
37
|
+
window.removeEventListener("pageshow", this.onPageShow)
|
|
38
|
+
document.removeEventListener("turbo:before-cache", this.onBeforeCache)
|
|
39
|
+
}
|
|
40
|
+
}
|