@valbuild/cli 0.140.0 → 0.141.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +80 -0
- package/cli/dist/valbuild-cli-cli.cjs.dev.js +157 -9
- package/cli/dist/valbuild-cli-cli.cjs.prod.js +157 -9
- package/cli/dist/valbuild-cli-cli.esm.js +159 -11
- package/package.json +7 -7
- package/src/cli.ts +1 -1
- package/src/debug/context.ts +1 -1
- package/src/debug/moduleClosure.ts +3 -1
- package/src/listUnusedFiles.ts +107 -9
- package/src/publish/__fixtures__/buildPublishProbe.ts +1 -1
- package/src/publish/runPublish.test.ts +73 -0
- package/src/publish/runPublish.ts +53 -2
- package/src/runValidation.ts +12 -0
- package/src/runValidation.videoset.test.ts +302 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,85 @@
|
|
|
1
1
|
# @valbuild/cli
|
|
2
2
|
|
|
3
|
+
## 0.141.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- [#802](https://github.com/valbuild/val/pull/802) [`199e214`](https://github.com/valbuild/val/commit/199e21471d98806fb34c579cb876560078c928b9) Thanks [@freekh](https://github.com/freekh)! - New: `files: { remote: true }` in `val.config.ts` stores every image, file and video on Val's remote content host.
|
|
8
|
+
|
|
9
|
+
```ts
|
|
10
|
+
const { s, c, val, config } = initVal({
|
|
11
|
+
project: "myorg/myproject",
|
|
12
|
+
files: { remote: true },
|
|
13
|
+
});
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
With it, `s.image()`, `s.file()`, `s.video()`, `s.imageset()`, `s.fileset()`, `s.videoset()` and `s.richtext({ img: true })` behave as though `.remote()` had been written on them. Images already in the project are reported as `image:upload-remote` until they are uploaded; `npx val validate --fix` does that.
|
|
17
|
+
|
|
18
|
+
**Required in the Val app.** A project running there (`VAL_ENV=app`) refuses to start without it, with an error that says what to add, and `val publish` refuses before it builds.
|
|
19
|
+
|
|
20
|
+
**Breaking:** `files.directory` has been removed from `val.config.ts`. Local files that have no `dir` of their own are stored in `/public/val`, which was the default.
|
|
21
|
+
|
|
22
|
+
- [#779](https://github.com/valbuild/val/pull/779) [`ba3f330`](https://github.com/valbuild/val/commit/ba3f3300a3dd8649e154097cdfca3b64db209d89) Thanks [@freekh](https://github.com/freekh)! - New: `s.video()`, for mp4/WebM files and HLS streams.
|
|
23
|
+
|
|
24
|
+
```ts
|
|
25
|
+
const schema = s.object({
|
|
26
|
+
intro: s.video({ stream: { type: "hls" } }),
|
|
27
|
+
});
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
- **Upload and play in the Studio.** A video field shows a player, the file's size and length, and progress while it uploads.
|
|
31
|
+
- **HLS streaming without a video service.** With `stream: { type: "hls" }`, the Studio converts an upload to an HLS stream in the editor's browser before it uploads it. It makes one H.264 rendition per height (1080p, 720p and 480p by default, and never larger than the upload) and stores them as a folder of files next to your other uploads. Nothing is installed in your app for this: the converter ships inside the Studio and is only downloaded when someone uploads to a streaming field. A browser that can't convert (no WebCodecs, or a non-https address) uploads the original file and says so.
|
|
32
|
+
- **Poster:** pause on a frame and choose "Use current frame". The frame is saved as an image, and `posterTime` records where it was taken.
|
|
33
|
+
- **Start and end:** play part of a video without cutting the file.
|
|
34
|
+
- **Focal point:** marks what must stay in frame when the page crops the video.
|
|
35
|
+
- **Description:** text for people who can't see the video.
|
|
36
|
+
- **Captions:** one WebVTT file per language. `.srt` files are converted on upload.
|
|
37
|
+
- **`<ValVideo>`** in `@valbuild/next` and `@valbuild/tanstack` renders all of the above and is click-to-edit. For HLS in browsers that can't play it natively, pass `hls={() => import("hls.js")}`. It is only loaded when it's needed.
|
|
38
|
+
- **Rename** a video in the Studio. An HLS stream is renamed as a whole: every playlist and segment moves to the new folder.
|
|
39
|
+
- `npx val validate --fix` fills in `mimeType`, `width`, `height` and `duration` for a hand-written `.mp4`, `.mov`, `.webm`, `.mkv` or HLS video, on disk or on Val Remote. A remote video is not downloaded: only its headers are read, usually in one to three small requests.
|
|
40
|
+
- `.remote()` works for videos. `npx val validate --fix` moves a video to Val Remote, or back into the project, together with its poster, captions and every file of an HLS stream.
|
|
41
|
+
|
|
42
|
+
A video value always has a `mimeType`: it is how a page knows whether it has an `.mp4` or an `.m3u8`.
|
|
43
|
+
|
|
44
|
+
New: `s.videoset()`, a collection of videos, like `s.imageset()` for images.
|
|
45
|
+
|
|
46
|
+
```ts
|
|
47
|
+
const videosVal = c.define(
|
|
48
|
+
"/content/videos.val.ts",
|
|
49
|
+
s.videoset({ dir: "/public/val/videos", stream: { type: "hls" } }),
|
|
50
|
+
{},
|
|
51
|
+
);
|
|
52
|
+
const schema = s.object({ intro: s.video(videosVal) });
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
- **Upload once, set it up once, use anywhere.** A set's entry holds what is true of the file (its type, size and length) and, as defaults, a description, poster, start and end, focal point and captions. A field that picks from the set (`s.video(videosVal)`) gets all of those, and can override any of them on its own. For example, the same clip can open one page at 0:02 and every other page where the set says.
|
|
56
|
+
- **Media in the Studio.** A set is listed under Media and opens as a gallery of videos, with each video's poster as its thumbnail. Hovering a video previews it, and the selected video plays in a real player (HLS streams too) beside the poster, times, focal point and captions you edit for it. You can upload into the set (with `stream`, uploads become HLS streams, and each upload gets a poster), rename a video, and delete a video that nothing uses. Renaming a video updates every field that uses it. An HLS stream is one entry, and renaming or deleting it moves or removes all of its files.
|
|
57
|
+
- **Picking.** A set-backed field picks from the set, or uploads into it. Each value the set provides is marked "From gallery · Override", and once overridden, "Overridden · Use gallery's".
|
|
58
|
+
- `useVal` and `fetchVal` fill in what the set holds (`mimeType`, `width`, `height`, `duration`, and every default the field does not override), so `<ValVideo>` works the same either way.
|
|
59
|
+
- `.remote()` works for sets. `npx val validate --fix` fills in missing metadata for an entry, adds untracked videos found in the set's directory, and moves a set's videos to Val Remote.
|
|
60
|
+
|
|
61
|
+
Image and file galleries use the same new gallery layout: a grid or list of entries, with the selected one open in a panel beside it.
|
|
62
|
+
|
|
63
|
+
- An `s.imageset()` entry can hold a default focal point, as well as its description. An `s.image(galleryVal)` field shows both, and can override either.
|
|
64
|
+
- Fixed: outside draft mode, `useVal` and `fetchVal` now fill a gallery-backed image or file from its gallery (dimensions, mime type, description). Before, a published page got only the `path`.
|
|
65
|
+
|
|
66
|
+
### Patch Changes
|
|
67
|
+
|
|
68
|
+
- Updated dependencies [[`199e214`](https://github.com/valbuild/val/commit/199e21471d98806fb34c579cb876560078c928b9), [`ba3f330`](https://github.com/valbuild/val/commit/ba3f3300a3dd8649e154097cdfca3b64db209d89)]:
|
|
69
|
+
- @valbuild/core@0.141.0
|
|
70
|
+
- @valbuild/server@0.141.0
|
|
71
|
+
- @valbuild/shared@0.141.0
|
|
72
|
+
- @valbuild/language-server@0.141.0
|
|
73
|
+
|
|
74
|
+
## 0.140.2
|
|
75
|
+
|
|
76
|
+
### Patch Changes
|
|
77
|
+
|
|
78
|
+
- Updated dependencies [[`df1e5f5`](https://github.com/valbuild/val/commit/df1e5f5f94391f25c71839cc476a51198b90db4d)]:
|
|
79
|
+
- @valbuild/server@0.140.2
|
|
80
|
+
- @valbuild/shared@0.140.2
|
|
81
|
+
- @valbuild/language-server@0.140.2
|
|
82
|
+
|
|
3
83
|
## 0.140.0
|
|
4
84
|
|
|
5
85
|
### Patch Changes
|
|
@@ -359,6 +359,18 @@ async function* runValidation({
|
|
|
359
359
|
}, !!fix, sourcePath, v, remoteFiles, valModule.source, valModule.schema);
|
|
360
360
|
if (fix && fixPatch !== null && fixPatch !== void 0 && fixPatch.patch && (fixPatch === null || fixPatch === void 0 ? void 0 : fixPatch.patch.length) > 0) {
|
|
361
361
|
await service.patch(moduleFilePath, fixPatch.patch);
|
|
362
|
+
// A fix that renames something other modules name (a video
|
|
363
|
+
// set's key) rewrites them too, and only once its own patch
|
|
364
|
+
// is in: a reference to a key that was never renamed is the
|
|
365
|
+
// one thing worse than a stale one.
|
|
366
|
+
for (const other of result.otherModulePatches ?? []) {
|
|
367
|
+
await service.patch(other.moduleFilePath, other.patch);
|
|
368
|
+
yield {
|
|
369
|
+
type: "fix-applied",
|
|
370
|
+
file: other.moduleFilePath.slice(1),
|
|
371
|
+
sourcePath: other.moduleFilePath
|
|
372
|
+
};
|
|
373
|
+
}
|
|
362
374
|
fixedErrors += 1;
|
|
363
375
|
yield {
|
|
364
376
|
type: "fix-applied",
|
|
@@ -975,11 +987,10 @@ async function watchAndValidate(projectRoot, runOnce) {
|
|
|
975
987
|
async function listUnusedFiles({
|
|
976
988
|
root
|
|
977
989
|
}) {
|
|
978
|
-
var _valConfigFile$files;
|
|
979
990
|
const projectRoot = root ? path__default["default"].resolve(root) : process.cwd();
|
|
980
|
-
|
|
981
|
-
//
|
|
982
|
-
const managedDir =
|
|
991
|
+
|
|
992
|
+
// Relative to the project root.
|
|
993
|
+
const managedDir = "public/val";
|
|
983
994
|
const service = await server.createService(projectRoot);
|
|
984
995
|
const registered = new Set(service.getModuleFilePaths());
|
|
985
996
|
const valFiles = await fastGlob.glob("**/*.val.{js,ts}", {
|
|
@@ -996,6 +1007,54 @@ async function listUnusedFiles({
|
|
|
996
1007
|
const valModule = await service.get(moduleId, "", {
|
|
997
1008
|
validate: true
|
|
998
1009
|
});
|
|
1010
|
+
// A video is found by walking the source, not by its validation errors:
|
|
1011
|
+
// unlike an image, a video with all its metadata reports none, so the
|
|
1012
|
+
// errors below would call every finished video unused. And a video names
|
|
1013
|
+
// more than its `path` — a poster, caption tracks, and a stream's
|
|
1014
|
+
// playlists and segments — which `filesOfVideo` is the one answer to.
|
|
1015
|
+
if (valModule.source !== undefined && valModule.schema) {
|
|
1016
|
+
internal.traverseSchemaSource(valModule.source, valModule.schema, moduleId, ({
|
|
1017
|
+
source,
|
|
1018
|
+
schema
|
|
1019
|
+
}) => {
|
|
1020
|
+
if (schema.type !== "video") {
|
|
1021
|
+
return;
|
|
1022
|
+
}
|
|
1023
|
+
for (const ref of server.filesOfVideo(source, {
|
|
1024
|
+
projectRoot
|
|
1025
|
+
})) {
|
|
1026
|
+
if (!core.Internal.isRemoteMediaPath(ref)) {
|
|
1027
|
+
filesUsedByVal.push(path__default["default"].join(projectRoot, ...ref.split("/")));
|
|
1028
|
+
}
|
|
1029
|
+
}
|
|
1030
|
+
});
|
|
1031
|
+
}
|
|
1032
|
+
// A media collection (`s.imageset()`, `s.fileset()`, `s.videoset()`) is
|
|
1033
|
+
// keyed by its files, and an entry that is fine reports no error either —
|
|
1034
|
+
// so, as for a video, they are found by walking the source. A video set's
|
|
1035
|
+
// stream holds every playlist and segment its master names, and an entry
|
|
1036
|
+
// its default poster and captions.
|
|
1037
|
+
if (valModule.source !== undefined && valModule.schema) {
|
|
1038
|
+
forEachMediaCollection(valModule.source, valModule.schema, (entries, mediaType) => {
|
|
1039
|
+
for (const [key, entry] of Object.entries(entries)) {
|
|
1040
|
+
const refs = mediaType === "videos" ?
|
|
1041
|
+
// The entry's poster and captions are its own files too.
|
|
1042
|
+
server.filesOfVideo(isObject(entry) ? {
|
|
1043
|
+
...entry,
|
|
1044
|
+
path: key
|
|
1045
|
+
} : {
|
|
1046
|
+
path: key
|
|
1047
|
+
}, {
|
|
1048
|
+
projectRoot
|
|
1049
|
+
}) : [key];
|
|
1050
|
+
for (const ref of refs) {
|
|
1051
|
+
if (!core.Internal.isRemoteMediaPath(ref)) {
|
|
1052
|
+
filesUsedByVal.push(path__default["default"].join(projectRoot, ...ref.split("/")));
|
|
1053
|
+
}
|
|
1054
|
+
}
|
|
1055
|
+
}
|
|
1056
|
+
});
|
|
1057
|
+
}
|
|
999
1058
|
// TODO: not sure using validation is the best way to do this, but it works currently.
|
|
1000
1059
|
if (valModule.errors) {
|
|
1001
1060
|
if (valModule.errors.validation) {
|
|
@@ -1040,6 +1099,44 @@ async function listUnusedFiles({
|
|
|
1040
1099
|
function isFileRef(value) {
|
|
1041
1100
|
return !!value && typeof value === "object" && "path" in value && typeof value.path === "string";
|
|
1042
1101
|
}
|
|
1102
|
+
function isObject(value) {
|
|
1103
|
+
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
1104
|
+
}
|
|
1105
|
+
|
|
1106
|
+
/**
|
|
1107
|
+
* Every media collection in a module's source, with its entries.
|
|
1108
|
+
*
|
|
1109
|
+
* `traverseSchemaSource` visits leaves, and a collection is a record whose
|
|
1110
|
+
* KEYS are what matters, so it never shows up there.
|
|
1111
|
+
*/
|
|
1112
|
+
function forEachMediaCollection(source, schema, visit) {
|
|
1113
|
+
if (schema.type === "record") {
|
|
1114
|
+
if (!isObject(source)) {
|
|
1115
|
+
return;
|
|
1116
|
+
}
|
|
1117
|
+
if (schema.mediaType) {
|
|
1118
|
+
visit(source, schema.mediaType);
|
|
1119
|
+
return;
|
|
1120
|
+
}
|
|
1121
|
+
for (const value of Object.values(source)) {
|
|
1122
|
+
forEachMediaCollection(value, schema.item, visit);
|
|
1123
|
+
}
|
|
1124
|
+
} else if (schema.type === "object") {
|
|
1125
|
+
if (!isObject(source)) {
|
|
1126
|
+
return;
|
|
1127
|
+
}
|
|
1128
|
+
for (const [key, item] of Object.entries(schema.items)) {
|
|
1129
|
+
forEachMediaCollection(source[key], item, visit);
|
|
1130
|
+
}
|
|
1131
|
+
} else if (schema.type === "array") {
|
|
1132
|
+
if (!Array.isArray(source)) {
|
|
1133
|
+
return;
|
|
1134
|
+
}
|
|
1135
|
+
for (const value of source) {
|
|
1136
|
+
forEachMediaCollection(value, schema.item, visit);
|
|
1137
|
+
}
|
|
1138
|
+
}
|
|
1139
|
+
}
|
|
1043
1140
|
|
|
1044
1141
|
/**
|
|
1045
1142
|
* The versions of the Val packages this project has installed.
|
|
@@ -1220,7 +1317,6 @@ async function lsp() {
|
|
|
1220
1317
|
|
|
1221
1318
|
class DebugContextError extends Error {}
|
|
1222
1319
|
async function createDebugContext(options) {
|
|
1223
|
-
var _config$files;
|
|
1224
1320
|
const projectRoot = options.root ? path__default["default"].resolve(options.root) : process.cwd();
|
|
1225
1321
|
if (!fs__default$1["default"].existsSync(projectRoot)) {
|
|
1226
1322
|
throw new DebugContextError(`Project root does not exist: ${projectRoot}`);
|
|
@@ -1231,7 +1327,7 @@ async function createDebugContext(options) {
|
|
|
1231
1327
|
}
|
|
1232
1328
|
const valModules = server.loadValModules(projectRoot);
|
|
1233
1329
|
const contentUrl = process.env.VAL_CONTENT_URL || core.DEFAULT_CONTENT_HOST;
|
|
1234
|
-
const filesDirectory =
|
|
1330
|
+
const filesDirectory = "/public/val";
|
|
1235
1331
|
const project = config.project || process.env.VAL_PROJECT || null;
|
|
1236
1332
|
const git = await server.safeReadGit(projectRoot);
|
|
1237
1333
|
const branch = options.branch || config.gitBranch || process.env.VAL_GIT_BRANCH || git.branch || null;
|
|
@@ -1321,7 +1417,8 @@ function readAuth(projectRoot) {
|
|
|
1321
1417
|
* Starts from the modules the pending patches touch and closes over the
|
|
1322
1418
|
* cross-module references a schema can hold:
|
|
1323
1419
|
* - `keyOf` points at another module through its `path` (a SourcePath)
|
|
1324
|
-
* - `image`/`file` point at a gallery module through
|
|
1420
|
+
* - `image`/`file`/`video` point at a gallery (or video set) module through
|
|
1421
|
+
* `referencedModule`
|
|
1325
1422
|
* - route validation cross-references *every* router module, so if any
|
|
1326
1423
|
* included module has a route or router we need all of them
|
|
1327
1424
|
*/
|
|
@@ -1410,6 +1507,7 @@ function collectSchemaReferences(schema) {
|
|
|
1410
1507
|
break;
|
|
1411
1508
|
case "file":
|
|
1412
1509
|
case "image":
|
|
1510
|
+
case "video":
|
|
1413
1511
|
if (node.referencedModule) {
|
|
1414
1512
|
refs.referencedModules.push(node.referencedModule);
|
|
1415
1513
|
}
|
|
@@ -3113,7 +3211,39 @@ async function runPublish(options) {
|
|
|
3113
3211
|
if (git.status === "error") {
|
|
3114
3212
|
return git;
|
|
3115
3213
|
}
|
|
3116
|
-
const
|
|
3214
|
+
const read = await server.findAndEvalValConfigFile(root).then(config => ({
|
|
3215
|
+
config,
|
|
3216
|
+
error: null
|
|
3217
|
+
}), error => ({
|
|
3218
|
+
config: null,
|
|
3219
|
+
error
|
|
3220
|
+
}));
|
|
3221
|
+
const config = read.config;
|
|
3222
|
+
/*
|
|
3223
|
+
* The Val app stores all media remotely, and the deployed server refuses to
|
|
3224
|
+
* start for a project that does not say so. Refused here too, so that a
|
|
3225
|
+
* developer who removes `files: { remote: true }` finds out from this
|
|
3226
|
+
* command -- not from the site, after the build has gone out.
|
|
3227
|
+
*
|
|
3228
|
+
* A config that cannot be read is refused as well when this builds the
|
|
3229
|
+
* checkout, because the commonest way to lose the setting is to shorten
|
|
3230
|
+
* `initVal({ files: { remote: true } })` to `initVal()`, which exports no
|
|
3231
|
+
* config at all. Skipping the check then would be skipping it exactly when
|
|
3232
|
+
* it was needed.
|
|
3233
|
+
*
|
|
3234
|
+
* A directory of artifacts built elsewhere is the exception: it may have no
|
|
3235
|
+
* config beside it, or none this machine can evaluate, and the server's own
|
|
3236
|
+
* check still stands there.
|
|
3237
|
+
*/
|
|
3238
|
+
const refusal = remoteFilesRefusal(read, {
|
|
3239
|
+
building: !prebuilt
|
|
3240
|
+
});
|
|
3241
|
+
if (refusal !== null) {
|
|
3242
|
+
return {
|
|
3243
|
+
status: "error",
|
|
3244
|
+
message: refusal
|
|
3245
|
+
};
|
|
3246
|
+
}
|
|
3117
3247
|
const project = (config === null || config === void 0 ? void 0 : config.project) ?? env.VAL_PROJECT ?? null;
|
|
3118
3248
|
const credential = await resolvePublishCredential({
|
|
3119
3249
|
root,
|
|
@@ -3671,6 +3801,24 @@ function formatBytes(bytes) {
|
|
|
3671
3801
|
}
|
|
3672
3802
|
return `${value.toFixed(1)} ${units[unit]}`;
|
|
3673
3803
|
}
|
|
3804
|
+
/**
|
|
3805
|
+
* Why this project cannot be published to the Val app as its media is
|
|
3806
|
+
* configured, or null when it can. See the call site.
|
|
3807
|
+
*/
|
|
3808
|
+
function remoteFilesRefusal(read, {
|
|
3809
|
+
building
|
|
3810
|
+
}) {
|
|
3811
|
+
if (read.config !== null) {
|
|
3812
|
+
var _read$config$files;
|
|
3813
|
+
return ((_read$config$files = read.config.files) === null || _read$config$files === void 0 ? void 0 : _read$config$files.remote) === true ? null : server.APP_MODE_REQUIRES_REMOTE_FILES;
|
|
3814
|
+
}
|
|
3815
|
+
if (!building) return null;
|
|
3816
|
+
if (read.error !== null) {
|
|
3817
|
+
const reason = read.error instanceof Error ? read.error.message : String(read.error);
|
|
3818
|
+
return `${server.APP_MODE_REQUIRES_REMOTE_FILES}\n\nval.config.ts could not be read, so this could not be checked: ${reason}`;
|
|
3819
|
+
}
|
|
3820
|
+
return `${server.APP_MODE_REQUIRES_REMOTE_FILES}\n\nThere is no val.config.ts (or val.config.js) in this project to say it.`;
|
|
3821
|
+
}
|
|
3674
3822
|
|
|
3675
3823
|
/**
|
|
3676
3824
|
* `val publish` - a built project in, a live site out.
|
|
@@ -3976,7 +4124,7 @@ async function main() {
|
|
|
3976
4124
|
|
|
3977
4125
|
Command: list-unused-files
|
|
3978
4126
|
Description: EXPERIMENTAL.
|
|
3979
|
-
List files that are in the configured files directory (
|
|
4127
|
+
List files that are in the configured files directory (public/val) but not in use by any Val module.
|
|
3980
4128
|
This is useful for cleaning up unused files.
|
|
3981
4129
|
Options:
|
|
3982
4130
|
--root [root], -r [root] Set project root directory (default process.cwd())
|
|
@@ -359,6 +359,18 @@ async function* runValidation({
|
|
|
359
359
|
}, !!fix, sourcePath, v, remoteFiles, valModule.source, valModule.schema);
|
|
360
360
|
if (fix && fixPatch !== null && fixPatch !== void 0 && fixPatch.patch && (fixPatch === null || fixPatch === void 0 ? void 0 : fixPatch.patch.length) > 0) {
|
|
361
361
|
await service.patch(moduleFilePath, fixPatch.patch);
|
|
362
|
+
// A fix that renames something other modules name (a video
|
|
363
|
+
// set's key) rewrites them too, and only once its own patch
|
|
364
|
+
// is in: a reference to a key that was never renamed is the
|
|
365
|
+
// one thing worse than a stale one.
|
|
366
|
+
for (const other of result.otherModulePatches ?? []) {
|
|
367
|
+
await service.patch(other.moduleFilePath, other.patch);
|
|
368
|
+
yield {
|
|
369
|
+
type: "fix-applied",
|
|
370
|
+
file: other.moduleFilePath.slice(1),
|
|
371
|
+
sourcePath: other.moduleFilePath
|
|
372
|
+
};
|
|
373
|
+
}
|
|
362
374
|
fixedErrors += 1;
|
|
363
375
|
yield {
|
|
364
376
|
type: "fix-applied",
|
|
@@ -975,11 +987,10 @@ async function watchAndValidate(projectRoot, runOnce) {
|
|
|
975
987
|
async function listUnusedFiles({
|
|
976
988
|
root
|
|
977
989
|
}) {
|
|
978
|
-
var _valConfigFile$files;
|
|
979
990
|
const projectRoot = root ? path__default["default"].resolve(root) : process.cwd();
|
|
980
|
-
|
|
981
|
-
//
|
|
982
|
-
const managedDir =
|
|
991
|
+
|
|
992
|
+
// Relative to the project root.
|
|
993
|
+
const managedDir = "public/val";
|
|
983
994
|
const service = await server.createService(projectRoot);
|
|
984
995
|
const registered = new Set(service.getModuleFilePaths());
|
|
985
996
|
const valFiles = await fastGlob.glob("**/*.val.{js,ts}", {
|
|
@@ -996,6 +1007,54 @@ async function listUnusedFiles({
|
|
|
996
1007
|
const valModule = await service.get(moduleId, "", {
|
|
997
1008
|
validate: true
|
|
998
1009
|
});
|
|
1010
|
+
// A video is found by walking the source, not by its validation errors:
|
|
1011
|
+
// unlike an image, a video with all its metadata reports none, so the
|
|
1012
|
+
// errors below would call every finished video unused. And a video names
|
|
1013
|
+
// more than its `path` — a poster, caption tracks, and a stream's
|
|
1014
|
+
// playlists and segments — which `filesOfVideo` is the one answer to.
|
|
1015
|
+
if (valModule.source !== undefined && valModule.schema) {
|
|
1016
|
+
internal.traverseSchemaSource(valModule.source, valModule.schema, moduleId, ({
|
|
1017
|
+
source,
|
|
1018
|
+
schema
|
|
1019
|
+
}) => {
|
|
1020
|
+
if (schema.type !== "video") {
|
|
1021
|
+
return;
|
|
1022
|
+
}
|
|
1023
|
+
for (const ref of server.filesOfVideo(source, {
|
|
1024
|
+
projectRoot
|
|
1025
|
+
})) {
|
|
1026
|
+
if (!core.Internal.isRemoteMediaPath(ref)) {
|
|
1027
|
+
filesUsedByVal.push(path__default["default"].join(projectRoot, ...ref.split("/")));
|
|
1028
|
+
}
|
|
1029
|
+
}
|
|
1030
|
+
});
|
|
1031
|
+
}
|
|
1032
|
+
// A media collection (`s.imageset()`, `s.fileset()`, `s.videoset()`) is
|
|
1033
|
+
// keyed by its files, and an entry that is fine reports no error either —
|
|
1034
|
+
// so, as for a video, they are found by walking the source. A video set's
|
|
1035
|
+
// stream holds every playlist and segment its master names, and an entry
|
|
1036
|
+
// its default poster and captions.
|
|
1037
|
+
if (valModule.source !== undefined && valModule.schema) {
|
|
1038
|
+
forEachMediaCollection(valModule.source, valModule.schema, (entries, mediaType) => {
|
|
1039
|
+
for (const [key, entry] of Object.entries(entries)) {
|
|
1040
|
+
const refs = mediaType === "videos" ?
|
|
1041
|
+
// The entry's poster and captions are its own files too.
|
|
1042
|
+
server.filesOfVideo(isObject(entry) ? {
|
|
1043
|
+
...entry,
|
|
1044
|
+
path: key
|
|
1045
|
+
} : {
|
|
1046
|
+
path: key
|
|
1047
|
+
}, {
|
|
1048
|
+
projectRoot
|
|
1049
|
+
}) : [key];
|
|
1050
|
+
for (const ref of refs) {
|
|
1051
|
+
if (!core.Internal.isRemoteMediaPath(ref)) {
|
|
1052
|
+
filesUsedByVal.push(path__default["default"].join(projectRoot, ...ref.split("/")));
|
|
1053
|
+
}
|
|
1054
|
+
}
|
|
1055
|
+
}
|
|
1056
|
+
});
|
|
1057
|
+
}
|
|
999
1058
|
// TODO: not sure using validation is the best way to do this, but it works currently.
|
|
1000
1059
|
if (valModule.errors) {
|
|
1001
1060
|
if (valModule.errors.validation) {
|
|
@@ -1040,6 +1099,44 @@ async function listUnusedFiles({
|
|
|
1040
1099
|
function isFileRef(value) {
|
|
1041
1100
|
return !!value && typeof value === "object" && "path" in value && typeof value.path === "string";
|
|
1042
1101
|
}
|
|
1102
|
+
function isObject(value) {
|
|
1103
|
+
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
1104
|
+
}
|
|
1105
|
+
|
|
1106
|
+
/**
|
|
1107
|
+
* Every media collection in a module's source, with its entries.
|
|
1108
|
+
*
|
|
1109
|
+
* `traverseSchemaSource` visits leaves, and a collection is a record whose
|
|
1110
|
+
* KEYS are what matters, so it never shows up there.
|
|
1111
|
+
*/
|
|
1112
|
+
function forEachMediaCollection(source, schema, visit) {
|
|
1113
|
+
if (schema.type === "record") {
|
|
1114
|
+
if (!isObject(source)) {
|
|
1115
|
+
return;
|
|
1116
|
+
}
|
|
1117
|
+
if (schema.mediaType) {
|
|
1118
|
+
visit(source, schema.mediaType);
|
|
1119
|
+
return;
|
|
1120
|
+
}
|
|
1121
|
+
for (const value of Object.values(source)) {
|
|
1122
|
+
forEachMediaCollection(value, schema.item, visit);
|
|
1123
|
+
}
|
|
1124
|
+
} else if (schema.type === "object") {
|
|
1125
|
+
if (!isObject(source)) {
|
|
1126
|
+
return;
|
|
1127
|
+
}
|
|
1128
|
+
for (const [key, item] of Object.entries(schema.items)) {
|
|
1129
|
+
forEachMediaCollection(source[key], item, visit);
|
|
1130
|
+
}
|
|
1131
|
+
} else if (schema.type === "array") {
|
|
1132
|
+
if (!Array.isArray(source)) {
|
|
1133
|
+
return;
|
|
1134
|
+
}
|
|
1135
|
+
for (const value of source) {
|
|
1136
|
+
forEachMediaCollection(value, schema.item, visit);
|
|
1137
|
+
}
|
|
1138
|
+
}
|
|
1139
|
+
}
|
|
1043
1140
|
|
|
1044
1141
|
/**
|
|
1045
1142
|
* The versions of the Val packages this project has installed.
|
|
@@ -1220,7 +1317,6 @@ async function lsp() {
|
|
|
1220
1317
|
|
|
1221
1318
|
class DebugContextError extends Error {}
|
|
1222
1319
|
async function createDebugContext(options) {
|
|
1223
|
-
var _config$files;
|
|
1224
1320
|
const projectRoot = options.root ? path__default["default"].resolve(options.root) : process.cwd();
|
|
1225
1321
|
if (!fs__default$1["default"].existsSync(projectRoot)) {
|
|
1226
1322
|
throw new DebugContextError(`Project root does not exist: ${projectRoot}`);
|
|
@@ -1231,7 +1327,7 @@ async function createDebugContext(options) {
|
|
|
1231
1327
|
}
|
|
1232
1328
|
const valModules = server.loadValModules(projectRoot);
|
|
1233
1329
|
const contentUrl = process.env.VAL_CONTENT_URL || core.DEFAULT_CONTENT_HOST;
|
|
1234
|
-
const filesDirectory =
|
|
1330
|
+
const filesDirectory = "/public/val";
|
|
1235
1331
|
const project = config.project || process.env.VAL_PROJECT || null;
|
|
1236
1332
|
const git = await server.safeReadGit(projectRoot);
|
|
1237
1333
|
const branch = options.branch || config.gitBranch || process.env.VAL_GIT_BRANCH || git.branch || null;
|
|
@@ -1321,7 +1417,8 @@ function readAuth(projectRoot) {
|
|
|
1321
1417
|
* Starts from the modules the pending patches touch and closes over the
|
|
1322
1418
|
* cross-module references a schema can hold:
|
|
1323
1419
|
* - `keyOf` points at another module through its `path` (a SourcePath)
|
|
1324
|
-
* - `image`/`file` point at a gallery module through
|
|
1420
|
+
* - `image`/`file`/`video` point at a gallery (or video set) module through
|
|
1421
|
+
* `referencedModule`
|
|
1325
1422
|
* - route validation cross-references *every* router module, so if any
|
|
1326
1423
|
* included module has a route or router we need all of them
|
|
1327
1424
|
*/
|
|
@@ -1410,6 +1507,7 @@ function collectSchemaReferences(schema) {
|
|
|
1410
1507
|
break;
|
|
1411
1508
|
case "file":
|
|
1412
1509
|
case "image":
|
|
1510
|
+
case "video":
|
|
1413
1511
|
if (node.referencedModule) {
|
|
1414
1512
|
refs.referencedModules.push(node.referencedModule);
|
|
1415
1513
|
}
|
|
@@ -3113,7 +3211,39 @@ async function runPublish(options) {
|
|
|
3113
3211
|
if (git.status === "error") {
|
|
3114
3212
|
return git;
|
|
3115
3213
|
}
|
|
3116
|
-
const
|
|
3214
|
+
const read = await server.findAndEvalValConfigFile(root).then(config => ({
|
|
3215
|
+
config,
|
|
3216
|
+
error: null
|
|
3217
|
+
}), error => ({
|
|
3218
|
+
config: null,
|
|
3219
|
+
error
|
|
3220
|
+
}));
|
|
3221
|
+
const config = read.config;
|
|
3222
|
+
/*
|
|
3223
|
+
* The Val app stores all media remotely, and the deployed server refuses to
|
|
3224
|
+
* start for a project that does not say so. Refused here too, so that a
|
|
3225
|
+
* developer who removes `files: { remote: true }` finds out from this
|
|
3226
|
+
* command -- not from the site, after the build has gone out.
|
|
3227
|
+
*
|
|
3228
|
+
* A config that cannot be read is refused as well when this builds the
|
|
3229
|
+
* checkout, because the commonest way to lose the setting is to shorten
|
|
3230
|
+
* `initVal({ files: { remote: true } })` to `initVal()`, which exports no
|
|
3231
|
+
* config at all. Skipping the check then would be skipping it exactly when
|
|
3232
|
+
* it was needed.
|
|
3233
|
+
*
|
|
3234
|
+
* A directory of artifacts built elsewhere is the exception: it may have no
|
|
3235
|
+
* config beside it, or none this machine can evaluate, and the server's own
|
|
3236
|
+
* check still stands there.
|
|
3237
|
+
*/
|
|
3238
|
+
const refusal = remoteFilesRefusal(read, {
|
|
3239
|
+
building: !prebuilt
|
|
3240
|
+
});
|
|
3241
|
+
if (refusal !== null) {
|
|
3242
|
+
return {
|
|
3243
|
+
status: "error",
|
|
3244
|
+
message: refusal
|
|
3245
|
+
};
|
|
3246
|
+
}
|
|
3117
3247
|
const project = (config === null || config === void 0 ? void 0 : config.project) ?? env.VAL_PROJECT ?? null;
|
|
3118
3248
|
const credential = await resolvePublishCredential({
|
|
3119
3249
|
root,
|
|
@@ -3671,6 +3801,24 @@ function formatBytes(bytes) {
|
|
|
3671
3801
|
}
|
|
3672
3802
|
return `${value.toFixed(1)} ${units[unit]}`;
|
|
3673
3803
|
}
|
|
3804
|
+
/**
|
|
3805
|
+
* Why this project cannot be published to the Val app as its media is
|
|
3806
|
+
* configured, or null when it can. See the call site.
|
|
3807
|
+
*/
|
|
3808
|
+
function remoteFilesRefusal(read, {
|
|
3809
|
+
building
|
|
3810
|
+
}) {
|
|
3811
|
+
if (read.config !== null) {
|
|
3812
|
+
var _read$config$files;
|
|
3813
|
+
return ((_read$config$files = read.config.files) === null || _read$config$files === void 0 ? void 0 : _read$config$files.remote) === true ? null : server.APP_MODE_REQUIRES_REMOTE_FILES;
|
|
3814
|
+
}
|
|
3815
|
+
if (!building) return null;
|
|
3816
|
+
if (read.error !== null) {
|
|
3817
|
+
const reason = read.error instanceof Error ? read.error.message : String(read.error);
|
|
3818
|
+
return `${server.APP_MODE_REQUIRES_REMOTE_FILES}\n\nval.config.ts could not be read, so this could not be checked: ${reason}`;
|
|
3819
|
+
}
|
|
3820
|
+
return `${server.APP_MODE_REQUIRES_REMOTE_FILES}\n\nThere is no val.config.ts (or val.config.js) in this project to say it.`;
|
|
3821
|
+
}
|
|
3674
3822
|
|
|
3675
3823
|
/**
|
|
3676
3824
|
* `val publish` - a built project in, a live site out.
|
|
@@ -3976,7 +4124,7 @@ async function main() {
|
|
|
3976
4124
|
|
|
3977
4125
|
Command: list-unused-files
|
|
3978
4126
|
Description: EXPERIMENTAL.
|
|
3979
|
-
List files that are in the configured files directory (
|
|
4127
|
+
List files that are in the configured files directory (public/val) but not in use by any Val module.
|
|
3980
4128
|
This is useful for cleaning up unused files.
|
|
3981
4129
|
Options:
|
|
3982
4130
|
--root [root], -r [root] Set project root directory (default process.cwd())
|