@kubohiroya/sb3-toolchain 0.1.1 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +59 -96
- package/docs/extension-api-compatibility.md +169 -0
- package/docs/extension-bundles.md +446 -0
- package/docs/extension-id-migration.md +44 -33
- package/docs/ja/extension-api-compatibility.md +167 -0
- package/docs/source-format-v1.md +134 -62
- package/docs/workflows.md +249 -0
- package/package.json +1 -1
- package/src/build.js +21 -7
- package/src/cli.js +176 -3
- package/src/constants.js +1 -1
- package/src/extension-api-manifest.js +336 -0
- package/src/extension-bundle-archive.js +422 -0
- package/src/extension-bundle-configuration.js +247 -0
- package/src/extension-bundle.js +640 -0
- package/src/extension-dependencies.js +16 -1
- package/src/extension-id-migration.js +74 -3
- package/src/extension-sync.js +189 -20
- package/src/import.js +28 -12
- package/src/index.js +25 -0
- package/src/source.js +48 -7
package/README.md
CHANGED
|
@@ -1,94 +1,49 @@
|
|
|
1
1
|
# sb3-toolchain
|
|
2
2
|
|
|
3
|
-
Scratch 3
|
|
4
|
-
bit-for-bit
|
|
3
|
+
A Node.js toolchain for managing Scratch 3 and TurboWarp `.sb3` projects as Git-diffable
|
|
4
|
+
expanded sources and rebuilding bit-for-bit identical SB3 files from the same input.
|
|
5
5
|
|
|
6
|
-
##
|
|
6
|
+
## Features
|
|
7
7
|
|
|
8
|
-
- SB3
|
|
9
|
-
-
|
|
10
|
-
-
|
|
11
|
-
-
|
|
12
|
-
-
|
|
13
|
-
|
|
14
|
-
-
|
|
8
|
+
- Safely expand an SB3 into formatted `project.source.json`, assets, and embedded extensions
|
|
9
|
+
- Validate asset references, MD5 hashes, ZIP entries, and embedded extension mappings
|
|
10
|
+
- Record commits and SHA-256 hashes for GitHub-hosted embedded extensions and verify them offline
|
|
11
|
+
- Optionally compare versioned extension API manifests before replacing embedded JavaScript
|
|
12
|
+
- Statically bundle multiple extensions into one permission unit without deleting their original
|
|
13
|
+
JavaScript, then restore them from either the expanded source or the bundled SB3
|
|
14
|
+
- Produce deterministic builds with fixed ZIP entry order, timestamps, and compression settings
|
|
15
|
+
- Protect uncommitted Git changes when importing
|
|
16
|
+
- Protect existing output through transactional replacement and rollback
|
|
17
|
+
- Provide both a CLI and a JavaScript API
|
|
15
18
|
|
|
16
|
-
TMPose
|
|
17
|
-
|
|
19
|
+
TMPose kamishibai script conversion, project-specific data, and web application generation with
|
|
20
|
+
TurboWarp Packager are outside the scope of this package.
|
|
18
21
|
|
|
19
|
-
##
|
|
22
|
+
## Requirements
|
|
20
23
|
|
|
21
|
-
- Node.js 22.12.0
|
|
24
|
+
- Node.js 22.12.0 or later
|
|
22
25
|
- pnpm 11
|
|
23
26
|
|
|
24
|
-
##
|
|
27
|
+
## Installation
|
|
25
28
|
|
|
26
|
-
|
|
29
|
+
Pin the verified npm version for reproducible installation.
|
|
27
30
|
|
|
28
31
|
```bash
|
|
29
|
-
pnpm add --save-dev --save-exact @kubohiroya/sb3-toolchain@0.
|
|
32
|
+
pnpm add --save-dev --save-exact @kubohiroya/sb3-toolchain@0.3.0
|
|
30
33
|
```
|
|
31
34
|
|
|
32
|
-
##
|
|
35
|
+
## Quick start
|
|
33
36
|
|
|
34
|
-
TurboWarp
|
|
37
|
+
Expand an SB3 saved by TurboWarp, validate it, and rebuild it.
|
|
35
38
|
|
|
36
39
|
```bash
|
|
37
40
|
sb3-toolchain import tmp/project.sb3 --output app
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
展開ソースを検証します。
|
|
41
|
-
|
|
42
|
-
```bash
|
|
43
41
|
sb3-toolchain check app
|
|
44
|
-
```
|
|
45
|
-
|
|
46
|
-
決定的なSB3を生成します。
|
|
47
|
-
|
|
48
|
-
```bash
|
|
49
42
|
sb3-toolchain build app --output dist/project.sb3
|
|
50
43
|
```
|
|
51
44
|
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
```bash
|
|
55
|
-
sb3-toolchain extensions status app
|
|
56
|
-
```
|
|
57
|
-
|
|
58
|
-
記録済みの固定commitから実ファイルを復元します。refは解決せず、メタデータも変更しません。
|
|
59
|
-
|
|
60
|
-
```bash
|
|
61
|
-
sb3-toolchain extensions sync app
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
追跡中のrefを最新commitへ解決し、成果物と`resolvedCommit`/`integrity`を更新します。
|
|
65
|
-
IDを省略すると、すべての管理対象拡張を一つのtransactionとして更新します。
|
|
66
|
-
|
|
67
|
-
```bash
|
|
68
|
-
sb3-toolchain extensions update app
|
|
69
|
-
sb3-toolchain extensions update app example
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
読み込み済みの拡張IDを変更する場合は、まずdry-runで分類済みの変更件数と、変更しない
|
|
73
|
-
未分類参照を確認します。`--yes`を付けたときだけ適用します。
|
|
74
|
-
|
|
75
|
-
```bash
|
|
76
|
-
sb3-toolchain extensions migrate-id app --from oldId --to newid
|
|
77
|
-
sb3-toolchain extensions migrate-id app --from oldId --to newid --yes
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
GitHub管理対象では、新IDを宣言する新しい成果物の取得とID移行を一つのtransactionにします。
|
|
81
|
-
リモートの成果物パスも変わる場合は`--artifact`で明示します。
|
|
82
|
-
|
|
83
|
-
```bash
|
|
84
|
-
sb3-toolchain extensions update app oldId \
|
|
85
|
-
--migrate-id newid \
|
|
86
|
-
--artifact dist/newid.js
|
|
87
|
-
```
|
|
88
|
-
|
|
89
|
-
既存出力と内容が同じ場合は更新時刻を変えません。異なる既存出力の置換には対話確認
|
|
90
|
-
または`--yes`が必要です。import先に未コミット変更がある場合、`--yes`だけでは置換せず、
|
|
91
|
-
明示的な`--discard-local-changes`も要求します。
|
|
45
|
+
See [`docs/workflows.md`](docs/workflows.md) for the recommended source-of-truth, re-import,
|
|
46
|
+
replacement protection, extension update, and CI workflows for a project repository.
|
|
92
47
|
|
|
93
48
|
## JavaScript API
|
|
94
49
|
|
|
@@ -97,6 +52,7 @@ import {readFile} from 'node:fs/promises';
|
|
|
97
52
|
|
|
98
53
|
import {
|
|
99
54
|
buildSb3,
|
|
55
|
+
bundleExtensions,
|
|
100
56
|
createDeterministicSb3,
|
|
101
57
|
extensionIntegrity,
|
|
102
58
|
extensionStatus,
|
|
@@ -104,6 +60,8 @@ import {
|
|
|
104
60
|
migrateExtensionId,
|
|
105
61
|
planExtensionIdMigration,
|
|
106
62
|
syncExtensions,
|
|
63
|
+
unbundleSb3,
|
|
64
|
+
unbundleExtensions,
|
|
107
65
|
updateExtensions,
|
|
108
66
|
validateSb3Source,
|
|
109
67
|
} from '@kubohiroya/sb3-toolchain';
|
|
@@ -142,45 +100,50 @@ await updateExtensions({
|
|
|
142
100
|
extensionId: 'oldId',
|
|
143
101
|
migrateToId: 'newid',
|
|
144
102
|
sourceArtifact: 'dist/newid.js',
|
|
103
|
+
apiManifestArtifact: 'dist/newid.manifest.json',
|
|
104
|
+
yes: true,
|
|
105
|
+
});
|
|
106
|
+
// After reviewing a reported breaking API change, opt in explicitly:
|
|
107
|
+
await updateExtensions({sourceDirectory: 'app', allowBreakingApi: true, yes: true});
|
|
108
|
+
await bundleExtensions({
|
|
109
|
+
sourceDirectory: 'app',
|
|
110
|
+
bundleId: 'projectbundle',
|
|
111
|
+
bundleName: 'Project Extension Bundle',
|
|
112
|
+
extensionIds: ['extensionone', 'extensiontwo'],
|
|
113
|
+
yes: true,
|
|
114
|
+
});
|
|
115
|
+
await unbundleExtensions({
|
|
116
|
+
sourceDirectory: 'app',
|
|
117
|
+
bundleId: 'projectbundle',
|
|
118
|
+
yes: true,
|
|
119
|
+
});
|
|
120
|
+
await unbundleSb3({
|
|
121
|
+
inputPath: 'dist/project.sb3',
|
|
122
|
+
outputPath: 'dist/project.unbundled.sb3',
|
|
123
|
+
bundleId: 'projectbundle',
|
|
145
124
|
yes: true,
|
|
146
125
|
});
|
|
147
126
|
```
|
|
148
127
|
|
|
149
|
-
##
|
|
150
|
-
|
|
151
|
-
`embedded-extensions.json`の拡張エントリには、任意の`source`メタデータとして
|
|
152
|
-
GitHubリポジトリ、追跡するref、解決済みcommit、成果物パス、SHA-256を記録できます。
|
|
153
|
-
`check`と`build`はネットワークへ接続せず、ローカルのJavaScriptが記録済みハッシュおよび
|
|
154
|
-
`// ID: <extensionId>`ヘッダーと一致することを検証します。
|
|
155
|
-
|
|
156
|
-
既存の展開ディレクトリへSB3を再importする場合、同じIDとパスを持つ拡張の`source`は
|
|
157
|
-
維持されます。SB3内の実ファイルが記録済みの内容から変わっている場合は、既存出力を
|
|
158
|
-
変更せずにimportを拒否します。メタデータ形式は
|
|
159
|
-
[`docs/source-format-v1.md`](docs/source-format-v1.md)を参照してください。
|
|
160
|
-
|
|
161
|
-
`extensions sync`は`resolvedCommit`だけから取得するため、mutableなbranchやtagを
|
|
162
|
-
再現処理に使いません。`extensions update`だけが`ref`を解決します。取得先はGitHubの
|
|
163
|
-
HTTPS endpointに限定し、redirectと5 MiBを超える成果物を拒否します。取得したJavaScriptは
|
|
164
|
-
実行せず、IDとSHA-256を検証してから展開ディレクトリを置換します。複数拡張の一部だけが
|
|
165
|
-
失敗した場合は、ファイルもメタデータも変更しません。
|
|
166
|
-
|
|
167
|
-
ID移行はTurboWarp公式形式の`[a-z0-9]+`を新IDに要求し、任意の文字列置換を行いません。
|
|
168
|
-
対象schema、dry-run、未分類参照の詳細は
|
|
169
|
-
[`docs/extension-id-migration.md`](docs/extension-id-migration.md)を参照してください。
|
|
128
|
+
## Documentation
|
|
170
129
|
|
|
171
|
-
|
|
130
|
+
- [`docs/workflows.md`](docs/workflows.md): SB3 source and extension management workflows
|
|
131
|
+
- [`docs/source-format-v1.md`](docs/source-format-v1.md): expanded source format and deterministic output
|
|
132
|
+
- [`docs/extension-id-migration.md`](docs/extension-id-migration.md): migration of extension IDs already used by a project
|
|
133
|
+
- [`docs/extension-api-compatibility.md`](docs/extension-api-compatibility.md) ([日本語](docs/ja/extension-api-compatibility.md)): opt-in static API compatibility checks for extension updates
|
|
134
|
+
- [`docs/extension-bundles.md`](docs/extension-bundles.md): static bundling into one permission unit and reversible unbundling
|
|
172
135
|
|
|
173
|
-
##
|
|
136
|
+
## Development
|
|
174
137
|
|
|
175
138
|
```bash
|
|
176
139
|
pnpm install
|
|
177
140
|
pnpm check
|
|
178
141
|
```
|
|
179
142
|
|
|
180
|
-
##
|
|
143
|
+
## License
|
|
181
144
|
|
|
182
145
|
[Mozilla Public License 2.0](LICENSE)
|
|
183
146
|
|
|
184
|
-
|
|
185
|
-
[`kubohiroya/tmpose-kamishibai`](https://github.com/kubohiroya/tmpose-kamishibai)
|
|
186
|
-
|
|
147
|
+
This implementation extracts the general SB3 source-management mechanism developed for
|
|
148
|
+
[`kubohiroya/tmpose-kamishibai`](https://github.com/kubohiroya/tmpose-kamishibai) from its
|
|
149
|
+
project- and TMPose-specific processing.
|
|
@@ -0,0 +1,169 @@
|
|
|
1
|
+
# Extension API compatibility manifests
|
|
2
|
+
|
|
3
|
+
[日本語版](ja/extension-api-compatibility.md)
|
|
4
|
+
|
|
5
|
+
An embedded extension can opt in to a versioned API manifest so that `sb3-toolchain` checks the
|
|
6
|
+
saved-project contract before replacing its JavaScript. The manifest is validation metadata: it is
|
|
7
|
+
kept in expanded source, fetched from the same immutable commit as the JavaScript, and never embedded
|
|
8
|
+
in the generated SB3.
|
|
9
|
+
|
|
10
|
+
This feature is optional. Existing unmanaged extensions and managed extensions without
|
|
11
|
+
`source.apiManifest` keep their previous behavior and deterministic SB3 output.
|
|
12
|
+
|
|
13
|
+
## Why a second artifact is necessary
|
|
14
|
+
|
|
15
|
+
Commit pinning and SHA-256 answer whether a JavaScript artifact is authentic and reproducible. They
|
|
16
|
+
do not answer whether an update still provides the opcodes and argument shapes already stored in a
|
|
17
|
+
project.
|
|
18
|
+
|
|
19
|
+
```mermaid
|
|
20
|
+
flowchart LR
|
|
21
|
+
Ref["Tracked Git ref"] --> Commit["Resolved commit"]
|
|
22
|
+
Commit --> JS["Extension JavaScript"]
|
|
23
|
+
Commit --> API["API manifest v1"]
|
|
24
|
+
JS --> Identity["Header ID + SHA-256"]
|
|
25
|
+
API --> Contract["Opcode / argument / menu comparison"]
|
|
26
|
+
Identity --> Candidate["Validated candidate"]
|
|
27
|
+
Contract --> Candidate
|
|
28
|
+
Candidate --> Transaction["One transactional source update"]
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
The manifest is parsed as JSON and is never executed. `sb3-toolchain` validates manifest v1 locally
|
|
32
|
+
instead of downloading a schema during a check or build.
|
|
33
|
+
|
|
34
|
+
## Expanded-source metadata
|
|
35
|
+
|
|
36
|
+
Add `apiManifest` below an existing GitHub `source` object:
|
|
37
|
+
|
|
38
|
+
```json
|
|
39
|
+
{
|
|
40
|
+
"id": "example",
|
|
41
|
+
"path": "extensions/example.js",
|
|
42
|
+
"mediaType": "text/javascript",
|
|
43
|
+
"parameters": [],
|
|
44
|
+
"encoding": "base64",
|
|
45
|
+
"source": {
|
|
46
|
+
"provider": "github",
|
|
47
|
+
"repository": "owner/example-extension",
|
|
48
|
+
"ref": "main",
|
|
49
|
+
"resolvedCommit": "0123456789abcdef0123456789abcdef01234567",
|
|
50
|
+
"artifact": "dist/example.js",
|
|
51
|
+
"integrity": "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=",
|
|
52
|
+
"apiManifest": {
|
|
53
|
+
"artifact": "dist/extension-manifest.json",
|
|
54
|
+
"path": "extensions/example.manifest.json",
|
|
55
|
+
"formatVersion": 1,
|
|
56
|
+
"integrity": "sha256-BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB="
|
|
57
|
+
}
|
|
58
|
+
}
|
|
59
|
+
}
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
- `artifact` is the manifest path in the GitHub repository.
|
|
63
|
+
- `path` must be `extensions/<extensionId>.manifest.json` in expanded source.
|
|
64
|
+
- `formatVersion` must be `1`.
|
|
65
|
+
- `integrity` is the SHA-256 SRI value of the installed manifest file.
|
|
66
|
+
|
|
67
|
+
The JavaScript and manifest artifacts use the same repository and `resolvedCommit`. Separate
|
|
68
|
+
repositories or commits are intentionally unsupported because they would make the recorded API
|
|
69
|
+
contract ambiguous.
|
|
70
|
+
|
|
71
|
+
## Manifest v1 contract
|
|
72
|
+
|
|
73
|
+
Manifest v1 contains:
|
|
74
|
+
|
|
75
|
+
- the extension ID;
|
|
76
|
+
- block opcode and block type;
|
|
77
|
+
- argument ID, type, and optional menu reference;
|
|
78
|
+
- menu ID and `acceptReporters`.
|
|
79
|
+
|
|
80
|
+
Unknown properties, duplicate identifiers, unknown menu references, unsupported versions, and ID
|
|
81
|
+
mismatches are rejected. Array order is normalized before comparison.
|
|
82
|
+
|
|
83
|
+
Display text, descriptions, default values, static menu items, labels, separators, and palette order
|
|
84
|
+
are outside manifest v1. They do not identify saved-project API references. Extension bundling
|
|
85
|
+
therefore continues to use `extensionBundles[].members` and runtime `getInfo().blocks` order; it does
|
|
86
|
+
not use manifest order to build the palette.
|
|
87
|
+
|
|
88
|
+
## Compatibility policy
|
|
89
|
+
|
|
90
|
+
| Candidate change | Classification | Reason |
|
|
91
|
+
| ---------------------------- | -------------- | ---------------------------------------------------- |
|
|
92
|
+
| Add a block | Compatible | Existing saved blocks keep their handler |
|
|
93
|
+
| Add an unreferenced menu | Compatible | No existing argument contract changes |
|
|
94
|
+
| Remove an unreferenced menu | Compatible | No saved argument refers to the menu |
|
|
95
|
+
| Remove a block | Breaking | Saved opcodes lose their handler |
|
|
96
|
+
| Change a block type | Breaking | Saved block shape or evaluation changes |
|
|
97
|
+
| Add or remove an argument | Breaking | Manifest v1 cannot prove a safe default or migration |
|
|
98
|
+
| Change argument type or menu | Breaking | Saved input interpretation changes |
|
|
99
|
+
| Remove a referenced menu | Breaking | Existing menu references can no longer resolve |
|
|
100
|
+
| Change `acceptReporters` | Breaking | Accepted saved input shapes change |
|
|
101
|
+
|
|
102
|
+
Compatibility reports use stable paths such as:
|
|
103
|
+
|
|
104
|
+
```text
|
|
105
|
+
/blocks/speak/blockType
|
|
106
|
+
/blocks/speak/arguments/VOICE/menu
|
|
107
|
+
/menus/voices/acceptReporters
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
An ID migration explicitly normalizes the old and new top-level IDs before comparing the remaining
|
|
111
|
+
contract. Any other API difference follows the same policy.
|
|
112
|
+
|
|
113
|
+
If the remote manifest filename changes with the extension ID, supply the new path explicitly:
|
|
114
|
+
|
|
115
|
+
```bash
|
|
116
|
+
sb3-toolchain extensions update app OLD_ID --migrate-id NEW_ID \
|
|
117
|
+
--artifact dist/newid.js \
|
|
118
|
+
--api-manifest-artifact dist/newid.manifest.json
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Both artifact paths and their integrity values are updated in the same transaction.
|
|
122
|
+
|
|
123
|
+
## Status, sync, and update
|
|
124
|
+
|
|
125
|
+
`extensions status` validates both installed files offline when reporting `local=valid` or
|
|
126
|
+
`local=modified`. It resolves only the tracked Git ref and does not download a candidate manifest.
|
|
127
|
+
|
|
128
|
+
`extensions sync` downloads both artifacts from the recorded `resolvedCommit`. Both must match their
|
|
129
|
+
recorded integrity and IDs before either local file is replaced.
|
|
130
|
+
|
|
131
|
+
`extensions update` resolves `ref`, downloads both artifacts from the new commit, validates them, and
|
|
132
|
+
compares the candidate API with the installed manifest. New blocks are reported as compatible.
|
|
133
|
+
Breaking changes are rejected before a candidate directory is installed.
|
|
134
|
+
|
|
135
|
+
To apply an intentionally breaking update, review every reported path and use both explicit flags:
|
|
136
|
+
|
|
137
|
+
```bash
|
|
138
|
+
sb3-toolchain extensions update app EXTENSION_ID --allow-breaking-api --yes
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
`--allow-breaking-api` without `--yes` is rejected. This prevents an ordinary replacement
|
|
142
|
+
confirmation from also authorizing an API break.
|
|
143
|
+
|
|
144
|
+
The JavaScript, API manifest, `resolvedCommit`, and both integrity values are written in one
|
|
145
|
+
transaction. If one download or validation fails, expanded source is unchanged.
|
|
146
|
+
|
|
147
|
+
## Check, build, and import
|
|
148
|
+
|
|
149
|
+
`check` and `build` verify the installed API manifest without network access. The manifest does not
|
|
150
|
+
enter `project.json` or the SB3 ZIP, so opting in does not change the generated SB3 bytes.
|
|
151
|
+
|
|
152
|
+
Re-importing an unchanged SB3 into an existing managed expanded source preserves both
|
|
153
|
+
`source.apiManifest` and its installed manifest file. A new import cannot infer GitHub provenance or
|
|
154
|
+
manifest metadata, so it does not enable the feature automatically. Existing manifest metadata is
|
|
155
|
+
read only for an extension with the same imported ID and path, and its local path is validated before
|
|
156
|
+
the manifest file is opened.
|
|
157
|
+
|
|
158
|
+
## Rollback
|
|
159
|
+
|
|
160
|
+
For a project update, revert the project commit so that JavaScript, API manifest,
|
|
161
|
+
`resolvedCommit`, and both integrity values return together.
|
|
162
|
+
|
|
163
|
+
To stop API compatibility tracking without changing the embedded JavaScript:
|
|
164
|
+
|
|
165
|
+
1. Remove `source.apiManifest` from the extension entry.
|
|
166
|
+
2. Remove `extensions/<extensionId>.manifest.json`.
|
|
167
|
+
3. Run `sb3-toolchain check` and rebuild the SB3.
|
|
168
|
+
|
|
169
|
+
The extension then follows the previous JavaScript provenance and integrity workflow.
|