@kubohiroya/sb3-toolchain 0.1.1 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,94 +1,49 @@
1
1
  # sb3-toolchain
2
2
 
3
- Scratch 3/TurboWarpの`.sb3`をGit差分可能な展開ソースとして管理し、同じ入力から
4
- bit-for-bitで同一のSB3を再生成するNode.jsツールチェーンです。
3
+ A Node.js toolchain for managing Scratch 3 and TurboWarp `.sb3` projects as Git-diffable
4
+ expanded sources and rebuilding bit-for-bit identical SB3 files from the same input.
5
5
 
6
- ## 主な機能
6
+ ## Features
7
7
 
8
- - SB3を整形済み`project.source.json`、アセット、埋め込み拡張へ安全に展開
9
- - アセット参照、MD5、ZIPエントリ、埋め込み拡張対応を検証
10
- - GitHub由来の埋め込み拡張についてcommitとSHA-256を記録し、オフラインで検証
11
- - ZIPエントリ順、タイムスタンプ、圧縮条件を固定した決定的ビルド
12
- - Git管理中の未コミット変更を保護したimport
13
- - 既存出力を保護する原子的な置換とロールバック
14
- - CLIとJavaScript API
8
+ - Safely expand an SB3 into formatted `project.source.json`, assets, and embedded extensions
9
+ - Validate asset references, MD5 hashes, ZIP entries, and embedded extension mappings
10
+ - Record commits and SHA-256 hashes for GitHub-hosted embedded extensions and verify them offline
11
+ - Optionally compare versioned extension API manifests before replacing embedded JavaScript
12
+ - Statically bundle multiple extensions into one permission unit without deleting their original
13
+ JavaScript, then restore them from either the expanded source or the bundled SB3
14
+ - Produce deterministic builds with fixed ZIP entry order, timestamps, and compression settings
15
+ - Protect uncommitted Git changes when importing
16
+ - Protect existing output through transactional replacement and rollback
17
+ - Provide both a CLI and a JavaScript API
15
18
 
16
- TMPose紙芝居の台本変換、特定作品のデータ、TurboWarp PackagerによるWebアプリ生成は
17
- このパッケージの対象外です。
19
+ TMPose kamishibai script conversion, project-specific data, and web application generation with
20
+ TurboWarp Packager are outside the scope of this package.
18
21
 
19
- ## 必要な環境
22
+ ## Requirements
20
23
 
21
- - Node.js 22.12.0以上
24
+ - Node.js 22.12.0 or later
22
25
  - pnpm 11
23
26
 
24
- ## インストール
27
+ ## Installation
25
28
 
26
- 再現可能な導入のため、npmで検証済みバージョンを固定します。
29
+ Pin the verified npm version for reproducible installation.
27
30
 
28
31
  ```bash
29
- pnpm add --save-dev --save-exact @kubohiroya/sb3-toolchain@0.1.1
32
+ pnpm add --save-dev --save-exact @kubohiroya/sb3-toolchain@0.3.0
30
33
  ```
31
34
 
32
- ## CLI
35
+ ## Quick start
33
36
 
34
- TurboWarpで保存したSB3を展開します。
37
+ Expand an SB3 saved by TurboWarp, validate it, and rebuild it.
35
38
 
36
39
  ```bash
37
40
  sb3-toolchain import tmp/project.sb3 --output app
38
- ```
39
-
40
- 展開ソースを検証します。
41
-
42
- ```bash
43
41
  sb3-toolchain check app
44
- ```
45
-
46
- 決定的なSB3を生成します。
47
-
48
- ```bash
49
42
  sb3-toolchain build app --output dist/project.sb3
50
43
  ```
51
44
 
52
- 管理対象の埋め込み拡張について、追跡中のGitHub refに更新があるか確認します。
53
-
54
- ```bash
55
- sb3-toolchain extensions status app
56
- ```
57
-
58
- 記録済みの固定commitから実ファイルを復元します。refは解決せず、メタデータも変更しません。
59
-
60
- ```bash
61
- sb3-toolchain extensions sync app
62
- ```
63
-
64
- 追跡中のrefを最新commitへ解決し、成果物と`resolvedCommit`/`integrity`を更新します。
65
- IDを省略すると、すべての管理対象拡張を一つのtransactionとして更新します。
66
-
67
- ```bash
68
- sb3-toolchain extensions update app
69
- sb3-toolchain extensions update app example
70
- ```
71
-
72
- 読み込み済みの拡張IDを変更する場合は、まずdry-runで分類済みの変更件数と、変更しない
73
- 未分類参照を確認します。`--yes`を付けたときだけ適用します。
74
-
75
- ```bash
76
- sb3-toolchain extensions migrate-id app --from oldId --to newid
77
- sb3-toolchain extensions migrate-id app --from oldId --to newid --yes
78
- ```
79
-
80
- GitHub管理対象では、新IDを宣言する新しい成果物の取得とID移行を一つのtransactionにします。
81
- リモートの成果物パスも変わる場合は`--artifact`で明示します。
82
-
83
- ```bash
84
- sb3-toolchain extensions update app oldId \
85
- --migrate-id newid \
86
- --artifact dist/newid.js
87
- ```
88
-
89
- 既存出力と内容が同じ場合は更新時刻を変えません。異なる既存出力の置換には対話確認
90
- または`--yes`が必要です。import先に未コミット変更がある場合、`--yes`だけでは置換せず、
91
- 明示的な`--discard-local-changes`も要求します。
45
+ See [`docs/workflows.md`](docs/workflows.md) for the recommended source-of-truth, re-import,
46
+ replacement protection, extension update, and CI workflows for a project repository.
92
47
 
93
48
  ## JavaScript API
94
49
 
@@ -97,6 +52,7 @@ import {readFile} from 'node:fs/promises';
97
52
 
98
53
  import {
99
54
  buildSb3,
55
+ bundleExtensions,
100
56
  createDeterministicSb3,
101
57
  extensionIntegrity,
102
58
  extensionStatus,
@@ -104,6 +60,8 @@ import {
104
60
  migrateExtensionId,
105
61
  planExtensionIdMigration,
106
62
  syncExtensions,
63
+ unbundleSb3,
64
+ unbundleExtensions,
107
65
  updateExtensions,
108
66
  validateSb3Source,
109
67
  } from '@kubohiroya/sb3-toolchain';
@@ -142,45 +100,50 @@ await updateExtensions({
142
100
  extensionId: 'oldId',
143
101
  migrateToId: 'newid',
144
102
  sourceArtifact: 'dist/newid.js',
103
+ apiManifestArtifact: 'dist/newid.manifest.json',
104
+ yes: true,
105
+ });
106
+ // After reviewing a reported breaking API change, opt in explicitly:
107
+ await updateExtensions({sourceDirectory: 'app', allowBreakingApi: true, yes: true});
108
+ await bundleExtensions({
109
+ sourceDirectory: 'app',
110
+ bundleId: 'projectbundle',
111
+ bundleName: 'Project Extension Bundle',
112
+ extensionIds: ['extensionone', 'extensiontwo'],
113
+ yes: true,
114
+ });
115
+ await unbundleExtensions({
116
+ sourceDirectory: 'app',
117
+ bundleId: 'projectbundle',
118
+ yes: true,
119
+ });
120
+ await unbundleSb3({
121
+ inputPath: 'dist/project.sb3',
122
+ outputPath: 'dist/project.unbundled.sb3',
123
+ bundleId: 'projectbundle',
145
124
  yes: true,
146
125
  });
147
126
  ```
148
127
 
149
- ## 管理対象の埋め込み拡張
150
-
151
- `embedded-extensions.json`の拡張エントリには、任意の`source`メタデータとして
152
- GitHubリポジトリ、追跡するref、解決済みcommit、成果物パス、SHA-256を記録できます。
153
- `check`と`build`はネットワークへ接続せず、ローカルのJavaScriptが記録済みハッシュおよび
154
- `// ID: <extensionId>`ヘッダーと一致することを検証します。
155
-
156
- 既存の展開ディレクトリへSB3を再importする場合、同じIDとパスを持つ拡張の`source`は
157
- 維持されます。SB3内の実ファイルが記録済みの内容から変わっている場合は、既存出力を
158
- 変更せずにimportを拒否します。メタデータ形式は
159
- [`docs/source-format-v1.md`](docs/source-format-v1.md)を参照してください。
160
-
161
- `extensions sync`は`resolvedCommit`だけから取得するため、mutableなbranchやtagを
162
- 再現処理に使いません。`extensions update`だけが`ref`を解決します。取得先はGitHubの
163
- HTTPS endpointに限定し、redirectと5 MiBを超える成果物を拒否します。取得したJavaScriptは
164
- 実行せず、IDとSHA-256を検証してから展開ディレクトリを置換します。複数拡張の一部だけが
165
- 失敗した場合は、ファイルもメタデータも変更しません。
166
-
167
- ID移行はTurboWarp公式形式の`[a-z0-9]+`を新IDに要求し、任意の文字列置換を行いません。
168
- 対象schema、dry-run、未分類参照の詳細は
169
- [`docs/extension-id-migration.md`](docs/extension-id-migration.md)を参照してください。
128
+ ## Documentation
170
129
 
171
- 展開形式の詳細は[`docs/source-format-v1.md`](docs/source-format-v1.md)を参照してください。
130
+ - [`docs/workflows.md`](docs/workflows.md): SB3 source and extension management workflows
131
+ - [`docs/source-format-v1.md`](docs/source-format-v1.md): expanded source format and deterministic output
132
+ - [`docs/extension-id-migration.md`](docs/extension-id-migration.md): migration of extension IDs already used by a project
133
+ - [`docs/extension-api-compatibility.md`](docs/extension-api-compatibility.md) ([日本語](docs/ja/extension-api-compatibility.md)): opt-in static API compatibility checks for extension updates
134
+ - [`docs/extension-bundles.md`](docs/extension-bundles.md): static bundling into one permission unit and reversible unbundling
172
135
 
173
- ## 開発
136
+ ## Development
174
137
 
175
138
  ```bash
176
139
  pnpm install
177
140
  pnpm check
178
141
  ```
179
142
 
180
- ## ライセンス
143
+ ## License
181
144
 
182
145
  [Mozilla Public License 2.0](LICENSE)
183
146
 
184
- 本実装は
185
- [`kubohiroya/tmpose-kamishibai`](https://github.com/kubohiroya/tmpose-kamishibai)
186
- で開発したSB3ソース管理機構を、作品およびTMPose固有処理から分離したものです。
147
+ This implementation extracts the general SB3 source-management mechanism developed for
148
+ [`kubohiroya/tmpose-kamishibai`](https://github.com/kubohiroya/tmpose-kamishibai) from its
149
+ project- and TMPose-specific processing.
@@ -0,0 +1,169 @@
1
+ # Extension API compatibility manifests
2
+
3
+ [日本語版](ja/extension-api-compatibility.md)
4
+
5
+ An embedded extension can opt in to a versioned API manifest so that `sb3-toolchain` checks the
6
+ saved-project contract before replacing its JavaScript. The manifest is validation metadata: it is
7
+ kept in expanded source, fetched from the same immutable commit as the JavaScript, and never embedded
8
+ in the generated SB3.
9
+
10
+ This feature is optional. Existing unmanaged extensions and managed extensions without
11
+ `source.apiManifest` keep their previous behavior and deterministic SB3 output.
12
+
13
+ ## Why a second artifact is necessary
14
+
15
+ Commit pinning and SHA-256 answer whether a JavaScript artifact is authentic and reproducible. They
16
+ do not answer whether an update still provides the opcodes and argument shapes already stored in a
17
+ project.
18
+
19
+ ```mermaid
20
+ flowchart LR
21
+ Ref["Tracked Git ref"] --> Commit["Resolved commit"]
22
+ Commit --> JS["Extension JavaScript"]
23
+ Commit --> API["API manifest v1"]
24
+ JS --> Identity["Header ID + SHA-256"]
25
+ API --> Contract["Opcode / argument / menu comparison"]
26
+ Identity --> Candidate["Validated candidate"]
27
+ Contract --> Candidate
28
+ Candidate --> Transaction["One transactional source update"]
29
+ ```
30
+
31
+ The manifest is parsed as JSON and is never executed. `sb3-toolchain` validates manifest v1 locally
32
+ instead of downloading a schema during a check or build.
33
+
34
+ ## Expanded-source metadata
35
+
36
+ Add `apiManifest` below an existing GitHub `source` object:
37
+
38
+ ```json
39
+ {
40
+ "id": "example",
41
+ "path": "extensions/example.js",
42
+ "mediaType": "text/javascript",
43
+ "parameters": [],
44
+ "encoding": "base64",
45
+ "source": {
46
+ "provider": "github",
47
+ "repository": "owner/example-extension",
48
+ "ref": "main",
49
+ "resolvedCommit": "0123456789abcdef0123456789abcdef01234567",
50
+ "artifact": "dist/example.js",
51
+ "integrity": "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=",
52
+ "apiManifest": {
53
+ "artifact": "dist/extension-manifest.json",
54
+ "path": "extensions/example.manifest.json",
55
+ "formatVersion": 1,
56
+ "integrity": "sha256-BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB="
57
+ }
58
+ }
59
+ }
60
+ ```
61
+
62
+ - `artifact` is the manifest path in the GitHub repository.
63
+ - `path` must be `extensions/<extensionId>.manifest.json` in expanded source.
64
+ - `formatVersion` must be `1`.
65
+ - `integrity` is the SHA-256 SRI value of the installed manifest file.
66
+
67
+ The JavaScript and manifest artifacts use the same repository and `resolvedCommit`. Separate
68
+ repositories or commits are intentionally unsupported because they would make the recorded API
69
+ contract ambiguous.
70
+
71
+ ## Manifest v1 contract
72
+
73
+ Manifest v1 contains:
74
+
75
+ - the extension ID;
76
+ - block opcode and block type;
77
+ - argument ID, type, and optional menu reference;
78
+ - menu ID and `acceptReporters`.
79
+
80
+ Unknown properties, duplicate identifiers, unknown menu references, unsupported versions, and ID
81
+ mismatches are rejected. Array order is normalized before comparison.
82
+
83
+ Display text, descriptions, default values, static menu items, labels, separators, and palette order
84
+ are outside manifest v1. They do not identify saved-project API references. Extension bundling
85
+ therefore continues to use `extensionBundles[].members` and runtime `getInfo().blocks` order; it does
86
+ not use manifest order to build the palette.
87
+
88
+ ## Compatibility policy
89
+
90
+ | Candidate change | Classification | Reason |
91
+ | ---------------------------- | -------------- | ---------------------------------------------------- |
92
+ | Add a block | Compatible | Existing saved blocks keep their handler |
93
+ | Add an unreferenced menu | Compatible | No existing argument contract changes |
94
+ | Remove an unreferenced menu | Compatible | No saved argument refers to the menu |
95
+ | Remove a block | Breaking | Saved opcodes lose their handler |
96
+ | Change a block type | Breaking | Saved block shape or evaluation changes |
97
+ | Add or remove an argument | Breaking | Manifest v1 cannot prove a safe default or migration |
98
+ | Change argument type or menu | Breaking | Saved input interpretation changes |
99
+ | Remove a referenced menu | Breaking | Existing menu references can no longer resolve |
100
+ | Change `acceptReporters` | Breaking | Accepted saved input shapes change |
101
+
102
+ Compatibility reports use stable paths such as:
103
+
104
+ ```text
105
+ /blocks/speak/blockType
106
+ /blocks/speak/arguments/VOICE/menu
107
+ /menus/voices/acceptReporters
108
+ ```
109
+
110
+ An ID migration explicitly normalizes the old and new top-level IDs before comparing the remaining
111
+ contract. Any other API difference follows the same policy.
112
+
113
+ If the remote manifest filename changes with the extension ID, supply the new path explicitly:
114
+
115
+ ```bash
116
+ sb3-toolchain extensions update app OLD_ID --migrate-id NEW_ID \
117
+ --artifact dist/newid.js \
118
+ --api-manifest-artifact dist/newid.manifest.json
119
+ ```
120
+
121
+ Both artifact paths and their integrity values are updated in the same transaction.
122
+
123
+ ## Status, sync, and update
124
+
125
+ `extensions status` validates both installed files offline when reporting `local=valid` or
126
+ `local=modified`. It resolves only the tracked Git ref and does not download a candidate manifest.
127
+
128
+ `extensions sync` downloads both artifacts from the recorded `resolvedCommit`. Both must match their
129
+ recorded integrity and IDs before either local file is replaced.
130
+
131
+ `extensions update` resolves `ref`, downloads both artifacts from the new commit, validates them, and
132
+ compares the candidate API with the installed manifest. New blocks are reported as compatible.
133
+ Breaking changes are rejected before a candidate directory is installed.
134
+
135
+ To apply an intentionally breaking update, review every reported path and use both explicit flags:
136
+
137
+ ```bash
138
+ sb3-toolchain extensions update app EXTENSION_ID --allow-breaking-api --yes
139
+ ```
140
+
141
+ `--allow-breaking-api` without `--yes` is rejected. This prevents an ordinary replacement
142
+ confirmation from also authorizing an API break.
143
+
144
+ The JavaScript, API manifest, `resolvedCommit`, and both integrity values are written in one
145
+ transaction. If one download or validation fails, expanded source is unchanged.
146
+
147
+ ## Check, build, and import
148
+
149
+ `check` and `build` verify the installed API manifest without network access. The manifest does not
150
+ enter `project.json` or the SB3 ZIP, so opting in does not change the generated SB3 bytes.
151
+
152
+ Re-importing an unchanged SB3 into an existing managed expanded source preserves both
153
+ `source.apiManifest` and its installed manifest file. A new import cannot infer GitHub provenance or
154
+ manifest metadata, so it does not enable the feature automatically. Existing manifest metadata is
155
+ read only for an extension with the same imported ID and path, and its local path is validated before
156
+ the manifest file is opened.
157
+
158
+ ## Rollback
159
+
160
+ For a project update, revert the project commit so that JavaScript, API manifest,
161
+ `resolvedCommit`, and both integrity values return together.
162
+
163
+ To stop API compatibility tracking without changing the embedded JavaScript:
164
+
165
+ 1. Remove `source.apiManifest` from the extension entry.
166
+ 2. Remove `extensions/<extensionId>.manifest.json`.
167
+ 3. Run `sb3-toolchain check` and rebuild the SB3.
168
+
169
+ The extension then follows the previous JavaScript provenance and integrity workflow.