@magicpixelart/cli 0.5.13 → 0.5.20

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,97 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## 0.5.20
11
+
12
+ - `sync` no longer deletes a PNG that was edited in your project when the sprite
13
+ leaves the manifest — un-checking "Sync to Unity", trashing the document, or a
14
+ rename/previous-key sweep now keep the local edit and report it instead.
15
+
16
+ ## 0.5.19
17
+
18
+ - `sync` no longer overwrites a PNG that changed both on disk and in MagicPixel. Such sprites are reported as conflicts, kept on disk, never pruned, and re-reported until you `push` them or delete the local file.
19
+ - A full reconcile is promoted automatically every 15 minutes (and whenever the server reports it truncated its removal list), so sprites deleted permanently in MagicPixel stop lingering in the game project.
20
+
21
+ ## [0.5.18] — 2026-08-27
22
+
23
+ ### Fixed
24
+ - Trashed documents (and documents converted into component masters) no longer
25
+ linger in the game project. The manifest now excludes them, and incremental
26
+ syncs receive an explicit `removed_keys` list so previously pulled sprites are
27
+ pruned on the next run. Sprites authored in Unity are still never touched.
28
+ - Renaming a folder or toggling its "Sync to Unity" flag now invalidates the
29
+ manifest cache, so the CLI stops receiving a stale `304 Not Modified`.
30
+
31
+ ## [0.5.17] — 2026-08-27
32
+
33
+ ### Fixed
34
+ - Watch-mode idle backoff now measures wall-clock seconds since the last change
35
+ instead of `ticks x interval`. Once the watcher had stepped down to 5s/10s
36
+ ticks the old estimate undercounted elapsed time, so the 5-minute step landed
37
+ well past 5 minutes.
38
+ - Pushing a locally edited sprite counts as activity again: a run that only
39
+ pushes (an artist working entirely inside their engine) resets the hot window
40
+ to the fast interval instead of sliding to the 10s idle poll.
41
+
42
+ ### Changed
43
+ - Idle incremental ticks no longer walk the whole output directory. The listing
44
+ is only needed by the full-sync orphan sweep and the rename fallback, so it is
45
+ now computed lazily — an unchanged (304) tick does no filesystem scan.
46
+
47
+ ## [0.5.16] — 2026-08-27
48
+
49
+ ### Changed
50
+ - Manifest validators are persisted in `.magicpixel/state.json`, so a one-shot
51
+ `sync` (CI, cron, a manual run) gets the same idle `304 Not Modified` a
52
+ watcher already got on its second tick. Validators are keyed by endpoint plus
53
+ exact query, so a staging/self-hosted swap or an edited `connect`/`exclude`
54
+ glob can never be answered from a stale validator.
55
+
56
+ ## [0.5.15] — 2026-08-26
57
+
58
+ ### Changed
59
+ - `sync --watch` sends conditional manifest requests (`If-None-Match`), so an
60
+ idle tick costs a `304` with no body instead of a full manifest render.
61
+ Validators are only used on incremental fetches — a full fetch, which prune
62
+ compares against, always reads the body.
63
+ - Idle backoff is now a hot window: the configured interval applies for ~1 min
64
+ after the last change, then steps to 5s, then to 10s after ~5 min idle
65
+ (previously ~3 min / ~15 min).
66
+
67
+ ### Fixed
68
+ - `safeFetch` no longer treated `304` as a malformed redirect. This made every
69
+ conditional asset download retry five times and then fail as a network error,
70
+ so unchanged PNGs were re-downloaded in full.
71
+
72
+
73
+ ## [0.5.14] — 2026-08-26
74
+
75
+ ### Fixed
76
+
77
+ - **Watch no longer re-downloads the whole game on catch-up.** Connect
78
+ originals almost never match MagicPixel's re-encoded composite hash, so
79
+ comparing those bytes queued ~every PNG (`492/3116…`) and would overwrite
80
+ the Unity files. Pull now skips a connected sprite unless the cloud
81
+ composite actually changed (or an editor save cleared the cached hash).
82
+ Ingest collision slugs (`ground-ice-snow-4`) alias onto the original PNG
83
+ instead of dumping copies into `Assets/MagicPixel`.
84
+ - **`connect <folder>` actually narrows the default `**` working set.** Adding
85
+ a folder used to OR with `**` and keep every sprite. `connect '**'` restores
86
+ all sprites. Godot's `.godot/` import cache is not ingested.
87
+ - **Editor saves of connected sprites still pull when the Unity flag is
88
+ missing.** A save nulls `artboard_index`; the next manifest often omits
89
+ `unity`. Watch writes those pixels back to the original game PNG. `lastSync`
90
+ advances past every row the fetch saw, including skipped drafts.
91
+ - **Watch no longer sits silent on `Checking your game files…`.** After a
92
+ pull, the upload half hashes local PNGs; the status line now shows
93
+ `1,200 / 3,126` instead of a frozen timestamp. The game-tree walk is reused
94
+ for 30s so each 2s poll does not re-list thousands of folders.
95
+ - **Ctrl+C drain message** says `Ctrl+C again` instead of `undefined`.
96
+ - **Single-artboard downloads no longer composite in the edge worker.**
97
+ Unity ingest sheets (including 2048/4096 tiles) are served from storage
98
+ so a catch-up download does not 546 the isolate. The CLI writes those
99
+ bytes even when they differ from a cached composite hash.
100
+
10
101
  ## [0.5.13] — 2026-08-26
11
102
 
12
103
  ### Fixed
@@ -16,10 +107,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
16
107
  rest of the project root blocked the spinner before any sprite was found.
17
108
  Directory reads stream so the path keeps updating (`Assets/Sprites`), and
18
109
  nested walks no longer deadlock on the folder limiter.
110
+ - **Push no longer aborts the rest of a connect on a 502.** Batches of 8
111
+ (down from 20) persist after each success. A gateway error splits the
112
+ batch instead of crashing; already-uploaded sprites are remembered so a
113
+ re-run skips them.
19
114
  - **`connect` copy.** Re-running `connect '**'` no longer prints
20
115
  `connect pattern already present: **`. First-time connect says
21
116
  `now syncing all sprites in your game`.
22
117
 
118
+ ### Changed
119
+
120
+ - **Unity/Godot/GameMaker default to all game sprites.** `start` and `sync`
121
+ ingest the game tree without a separate `connect '**'`. Use `connect` only
122
+ to narrow folders. Daily command is `sync --watch`.
123
+
23
124
  ## [0.5.12] — 2026-08-26
24
125
 
25
126
  ### Fixed
package/README.md CHANGED
@@ -24,14 +24,15 @@ Unity project or UPM package (`Assets`/`assets`, `Editor`/`Runtime`), Godot
24
24
  npx @magicpixelart/cli start
25
25
  ```
26
26
 
27
- That walks you through everything: detects your framework, writes `magicpixel.json`, prompts for your API key (paste it from [magicpixel.art/settings](https://magicpixel.art/settings) → API Keys), pulls flagged sprites, and indexes game PNGs locally. Nothing is imported into the library until you connect a working set:
27
+ That walks you through everything: detects your engine or framework, writes `magicpixel.json`, prompts for your API key (paste it from [magicpixel.art/settings](https://magicpixel.art/settings) → API Keys), and syncs sprites both ways. Unity, Godot, and GameMaker default to all game PNGs.
28
+
29
+ Then leave this running while you work:
28
30
 
29
31
  ```bash
30
- npx @magicpixelart/cli search hero
31
- npx @magicpixelart/cli connect 'assets/Sprites/**'
32
+ npx @magicpixelart/cli sync --watch
32
33
  ```
33
34
 
34
- Connected sprites write back to their original game path. New MagicPixel-only art still lands in `outDir`.
35
+ After `npm i -D @magicpixelart/cli` you can type `magicpixel sync --watch`. Connected sprites write back to their original game path. New MagicPixel-only art still lands in `outDir`. To sync only some folders: `npx @magicpixelart/cli connect 'assets/Sprites/**'`.
35
36
 
36
37
  When it finishes:
37
38
 
@@ -109,9 +110,9 @@ No bundler config, no runtime, no extra package.
109
110
  | `logout` | Remove the stored API key. |
110
111
  | `doctor` | Print a one-screen diagnostic — paste it to your AI agent when something breaks. |
111
112
  | `repair [--dry-run] [-y]` | Self-heal a broken sync: validate key → quarantine `state.json` → prune empty dirs → full re-sync. |
112
- | `sync [...flags]` | Fetch manifest, diff against disk, download changed assets, prune orphans. |
113
- | `push [--dry-run] [--flatten]` | Upload local PNG edits (and the `connect` working set) back to MagicPixel. |
114
- | `connect <glob>` | Add a game-sprite glob to the working set and ingest matching PNGs. |
113
+ | `sync [...flags]` | Two-way: pull MagicPixel edits and push changed game sprites. |
114
+ | `push [--dry-run] [--flatten]` | Upload local PNG edits (and connected game sprites) back to MagicPixel. |
115
+ | `connect <glob>` | Limit which game folders sync (default is all sprites on Unity/Godot/GameMaker). |
115
116
  | `search <query>` | Search indexed game PNGs (no network). |
116
117
  | `add <glob>` / `remove <glob>` | Manage `include` patterns. |
117
118
  | `list` | Print matching manifest as a table. |
@@ -146,7 +147,7 @@ Sync is built to be cheap: a no-op run is one small manifest request, zero PNG b
146
147
  | `outDir` | `string` | framework-dependent | Where PNGs (and `index.ts`) are written. |
147
148
  | `include` | `string[]` | `["**/*"]` | Globs (picomatch) matched against `folder/slug`. |
148
149
  | `exclude` | `string[]` | `[]` | Globs to exclude. |
149
- | `connect` | `string[]` | `[]` | Working-set globs of game PNGs to adopt into Connected. Empty = index only. |
150
+ | `connect` | `string[]` | `["**"]` on engines | Game PNG globs to keep in Connected. Empty on JS projects. |
150
151
  | `emitIndex`| `boolean` | `true` | Emit `<outDir>/index.ts` with typed asset map. |
151
152
  | `unityPpu` | `number?` | `32` | Unity only: pixels-per-unit in generated `.meta`. |
152
153
  | `unitySyncAll` | `boolean?` | `false` | Unity only: sync every artboard, not just those flagged in the editor. |
package/dist/api.js CHANGED
@@ -2,6 +2,52 @@ import { resolveEndpoint, getApiKey, CLI_VERSION } from './config.js';
2
2
  import { etagForSha256, MAX_ASSET_BYTES, readBodyWithLimit, safeFetch, validateEndpointUrl, } from './util/security.js';
3
3
  import { authHeaders } from './util/authHeaders.js';
4
4
  import { cmd } from './util/invoke.js';
5
+ /**
6
+ * Manifest validators, so an idle poll costs a conditional GET (304, no body)
7
+ * instead of a full manifest render. Keyed by endpoint + exact query string: a
8
+ * different `since`/glob set — or a different server — is a different
9
+ * projection and must not reuse a validator.
10
+ *
11
+ * Bounded so a long-lived watcher walking many `since` values can't grow it
12
+ * without limit; eviction is FIFO and only costs one non-conditional GET.
13
+ *
14
+ * `sync` persists this map into state.json (see `manifestEtagsSnapshot` /
15
+ * `primeManifestEtags`) so one-shot runs — CI, cron, a manual `sync` — get the
16
+ * same 304 as a watcher's second tick.
17
+ */
18
+ const manifestEtags = new Map();
19
+ const MAX_MANIFEST_ETAGS = 32;
20
+ function rememberManifestEtag(key, etag) {
21
+ manifestEtags.delete(key);
22
+ manifestEtags.set(key, etag);
23
+ while (manifestEtags.size > MAX_MANIFEST_ETAGS) {
24
+ const oldest = manifestEtags.keys().next().value;
25
+ if (oldest === undefined)
26
+ break;
27
+ manifestEtags.delete(oldest);
28
+ }
29
+ }
30
+ /**
31
+ * Seed validators persisted by an earlier run. Ignores non-string junk so a
32
+ * hand-edited state.json can't inject a bogus header value; a wrong-but-typed
33
+ * validator is harmless (the server just answers 200 with a body).
34
+ */
35
+ export function primeManifestEtags(entries) {
36
+ if (!entries || typeof entries !== 'object' || Array.isArray(entries))
37
+ return;
38
+ for (const [key, etag] of Object.entries(entries)) {
39
+ if (typeof etag === 'string' && etag)
40
+ rememberManifestEtag(key, etag);
41
+ }
42
+ }
43
+ /** Snapshot of the current validators, for persisting into state.json. */
44
+ export function manifestEtagsSnapshot() {
45
+ return Object.fromEntries(manifestEtags);
46
+ }
47
+ /** Test seam: drop cached manifest validators between cases. */
48
+ export function __resetManifestEtagsForTesting() {
49
+ manifestEtags.clear();
50
+ }
5
51
  /**
6
52
  * Last-seen project metadata from any manifest response this process made.
7
53
  * Populated as a side-effect of `fetchManifestPage`; read by `whoami`,
@@ -84,17 +130,32 @@ export async function fetchManifestPage(opts) {
84
130
  url.searchParams.set('cursor', opts.cursor);
85
131
  if (opts.limit)
86
132
  url.searchParams.set('limit', String(opts.limit));
133
+ // Conditional GET: an unchanged scope comes back as a 304 with no body.
134
+ // Only for un-paginated requests — the server doesn't validate cursor pages.
135
+ // Key includes the origin+path so a persisted validator can't be replayed
136
+ // against a different endpoint (self-hosted / staging swap).
137
+ const etagKey = opts.conditional && !opts.cursor ? `${url.origin}${url.pathname}${url.search}` : null;
138
+ const knownEtag = etagKey ? manifestEtags.get(etagKey) : undefined;
87
139
  // Reuse the same transient-failure policy as asset downloads, with extra
88
140
  // 546 attempts so first-run can ride out a worker recycle without stalling
89
141
  // every PNG download for ~30s.
90
142
  return retryTransient(`manifest`, async () => {
91
- const { headers, requestId } = buildHeaders();
143
+ const { headers, requestId } = buildHeaders(knownEtag ? { 'If-None-Match': knownEtag } : undefined);
92
144
  const res = await safeFetch(url.href, { headers });
93
145
  const serverRequestId = res.headers.get('x-request-id') ?? requestId;
146
+ if (res.status === 304) {
147
+ const minCli = res.headers.get('x-magicpixel-min-cli-version');
148
+ if (minCli)
149
+ maybeWarnStaleCli(minCli);
150
+ return { items: [], nextCursor: null, count: 0 };
151
+ }
94
152
  if (res.status >= 200 && res.status < 300) {
95
153
  const minCli = res.headers.get('x-magicpixel-min-cli-version');
96
154
  if (minCli)
97
155
  maybeWarnStaleCli(minCli);
156
+ const etag = res.headers.get('etag');
157
+ if (etagKey && etag)
158
+ rememberManifestEtag(etagKey, etag);
98
159
  const data = (await res.json());
99
160
  // Shape-guard the response. A malformed edge response (null,
100
161
  // {items: null}, nextCursor: 42, etc.) would otherwise either crash
@@ -214,18 +275,33 @@ function compareSemver(a, b) {
214
275
  }
215
276
  return 0;
216
277
  }
217
- export async function fetchAllManifest(config, since) {
278
+ /** Full manifest walk, including the cloud's removal list. */
279
+ export async function fetchManifestSnapshot(config, since) {
218
280
  const out = [];
281
+ const removed = new Set();
282
+ let removalsTruncated = false;
219
283
  let cursor;
284
+ // Conditional GETs only on incremental fetches. An incremental 304 means
285
+ // "nothing changed since `since`", which is indistinguishable from the empty
286
+ // incremental page callers already handle. A full fetch (`since` unset) is
287
+ // the cloud truth that prune compares against, so it always reads the body.
288
+ const conditional = !!since;
220
289
  // Cycle guard: caps total pages so a buggy server cursor can't hang the CLI.
221
290
  // At 500 entries/page this is 100k assets — well past any realistic project.
222
291
  const MAX_PAGES = 200;
223
292
  for (let page = 0; page < MAX_PAGES; page++) {
224
- const res = await fetchManifestPage({ config, since, cursor, limit: 500 });
293
+ const res = await fetchManifestPage({ config, since, cursor, limit: 500, conditional });
225
294
  out.push(...res.items);
295
+ if (Array.isArray(res.removed_keys)) {
296
+ for (const k of res.removed_keys)
297
+ if (typeof k === 'string' && k)
298
+ removed.add(k);
299
+ }
300
+ if (res.removals_truncated === true)
301
+ removalsTruncated = true;
226
302
  const nextCursor = res.nextCursor ?? undefined;
227
303
  if (!nextCursor)
228
- return out;
304
+ return { entries: out, removedKeys: [...removed], removalsTruncated };
229
305
  // Detect a stuck cursor (server returns the same token it just received)
230
306
  // in O(1) round-trips instead of burning the full page budget.
231
307
  if (nextCursor === cursor) {
@@ -237,6 +313,10 @@ export async function fetchAllManifest(config, since) {
237
313
  throw new Error(`manifest: pagination exceeded ${MAX_PAGES} pages — possible server cursor loop.\n` +
238
314
  ` Fix: re-run with --full, or report at https://github.com/magicpixel/cli/issues.`);
239
315
  }
316
+ /** Entries only — for read-only commands (`list`, `status`). */
317
+ export async function fetchAllManifest(config, since) {
318
+ return (await fetchManifestSnapshot(config, since)).entries;
319
+ }
240
320
  /**
241
321
  * Download a single asset by key. Returns null on 304 (not modified).
242
322
  * Shares the retryTransient policy with manifest fetches — including
@@ -280,15 +360,15 @@ export function retryAfterMsFromResponse(res) {
280
360
  }
281
361
  return 0;
282
362
  }
283
- /** Max sprites the server accepts per request (mirrors MAX_SPRITES_PER_REQUEST). */
284
- export const PUSH_BATCH_SIZE = 20;
363
+ /** CLI send size. The server allows 20; smaller batches survive Worker 502s. */
364
+ export const PUSH_BATCH_SIZE = 8;
285
365
  const DEFAULT_INGEST_ENDPOINT = 'https://magicpixel.art/api/public/integration/ingest';
286
366
  export function resolveIngestEndpoint() {
287
367
  const override = process.env.MAGICPIXEL_INGEST_ENDPOINT?.trim();
288
368
  return override ? validateEndpointUrl(override) : DEFAULT_INGEST_ENDPOINT;
289
369
  }
290
370
  /** Post one batch of sprites. Shares the transient-retry policy with pulls. */
291
- export async function pushSprites(sprites) {
371
+ export async function pushSprites(sprites, retryOpts) {
292
372
  const url = resolveIngestEndpoint();
293
373
  return retryTransient('push', async () => {
294
374
  const { headers, requestId } = buildHeaders({ 'Content-Type': 'application/json' });
@@ -308,5 +388,41 @@ export async function pushSprites(sprites) {
308
388
  }
309
389
  const bodyText = await res.text();
310
390
  throw new ApiError(res.status, withRequestId(friendly(res.status, bodyText, 'push'), serverRequestId), serverRequestId, retryAfterMsFromResponse(res));
311
- });
391
+ }, retryOpts);
392
+ }
393
+ function isTransientPushError(e) {
394
+ return e instanceof ApiError && (e.status >= 500 || e.status === 429);
395
+ }
396
+ /**
397
+ * Push a batch, splitting in half on persistent 502/5xx so one fat request
398
+ * cannot abort the rest of a connect/sync.
399
+ */
400
+ export async function pushSpritesAdaptive(sprites, retryOpts) {
401
+ if (sprites.length === 0)
402
+ return [];
403
+ try {
404
+ return await pushSprites(sprites, retryOpts);
405
+ }
406
+ catch (e) {
407
+ if (!isTransientPushError(e))
408
+ throw e;
409
+ if (sprites.length === 1) {
410
+ return [{ key: sprites[0].key, status: 'error', message: e.message }];
411
+ }
412
+ const mid = Math.ceil(sprites.length / 2);
413
+ const left = await pushSpritesAdaptive(sprites.slice(0, mid), retryOpts);
414
+ try {
415
+ const right = await pushSpritesAdaptive(sprites.slice(mid), retryOpts);
416
+ return [...left, ...right];
417
+ }
418
+ catch (rightErr) {
419
+ // Left half already landed — don't drop those results when the rest fails.
420
+ const right = sprites.slice(mid).map((s) => ({
421
+ key: s.key,
422
+ status: 'error',
423
+ message: rightErr.message,
424
+ }));
425
+ return [...left, ...right];
426
+ }
427
+ }
312
428
  }
@@ -4,26 +4,25 @@ import { loadConfig, saveConfig } from '../config.js';
4
4
  import { assertSafeGlob } from '../util/security.js';
5
5
  import { detectProjectKind, isEngineKind } from '../util/framework.js';
6
6
  import { indexGamePngs, matchConnectGlobs, GAME_INDEX_CAP_HINT, connectCapMessage, countingSpritesText } from '../util/gameScan.js';
7
+ import { isAllSpritesGlob, nextConnectGlobs } from '../util/engineConnect.js';
7
8
  import { cmd } from '../util/invoke.js';
8
9
  import { runPush } from './push.js';
9
10
  /** Plain-language label for a connect glob (`**` is not meaningful to players). */
10
11
  export function describeWorkingSet(glob) {
11
- const g = glob.trim();
12
- if (g === '**' || g === '**/*' || g === '**/**' || g === '**/*.png' || g === '**/**/*.png') {
13
- return 'all sprites in your game';
14
- }
15
- return g;
12
+ return isAllSpritesGlob(glob) ? 'all sprites in your game' : glob.trim();
16
13
  }
17
14
  /**
18
- * Add a working-set glob and ingest matching game PNGs into Connected.
19
- * Sync writes edits back to the original files — not a copy under outDir.
15
+ * Add or narrow a working-set glob, then ingest matching game PNGs.
16
+ * A specific folder replaces the default `**`. Sync writes edits back to
17
+ * the original files — not a copy under outDir.
20
18
  */
21
19
  export async function connectCommand(glob) {
22
20
  const pattern = assertSafeGlob(glob);
23
21
  const config = await loadConfig();
24
- const already = config.connect.includes(pattern);
25
- if (!already) {
26
- config.connect.push(pattern);
22
+ const next = nextConnectGlobs(config.connect, pattern);
23
+ const changed = next.length !== config.connect.length || next.some((g, i) => g !== config.connect[i]);
24
+ if (changed) {
25
+ config.connect = next;
27
26
  await saveConfig(config);
28
27
  console.log(kleur.green(`✓ now syncing ${describeWorkingSet(pattern)}`));
29
28
  }
@@ -49,7 +48,7 @@ export async function connectCommand(glob) {
49
48
  return;
50
49
  }
51
50
  const n = matched.entries.length;
52
- console.log(kleur.dim(` ${n} sprite${n === 1 ? '' : 's'} in working set (of ${index.files.length} indexed` +
51
+ console.log(kleur.dim(` ${n} sprite${n === 1 ? '' : 's'} to sync (of ${index.files.length} in your game` +
53
52
  (matched.capped ? `, ${matched.total} matched` : '') +
54
53
  `).`));
55
54
  if (n >= 200) {
@@ -9,6 +9,7 @@ import { detectProjectKind, suggestOutDir, hasPackageJson, isStaticOutDir, suppo
9
9
  import { assertSafeOutDir } from '../util/security.js';
10
10
  import { atomicWrite } from '../util/atomicWrite.js';
11
11
  import { cmd } from '../util/invoke.js';
12
+ import { DEFAULT_ENGINE_CONNECT } from '../util/engineConnect.js';
12
13
  const WATCH_SCRIPT_NAME = 'magicpixel:watch';
13
14
  const WATCH_SCRIPT_CMD = 'magicpixel sync --watch';
14
15
  export async function initCommand(opts) {
@@ -23,6 +24,8 @@ export async function initCommand(opts) {
23
24
  const kind = await detectProjectKind();
24
25
  const suggestedOutDir = suggestOutDir(kind);
25
26
  config.outDir = suggestedOutDir;
27
+ if (isEngineKind(kind))
28
+ config.connect = [...DEFAULT_ENGINE_CONNECT];
26
29
  const pkgExists = hasPackageJson();
27
30
  const pkgPath = resolve(process.cwd(), 'package.json');
28
31
  let addWatchScript = pkgExists && !isEngineKind(kind);
@@ -9,10 +9,10 @@
9
9
  import kleur from 'kleur';
10
10
  import ora from 'ora';
11
11
  import { mkdir, readFile, rename } from 'node:fs/promises';
12
- import { dirname } from 'node:path';
13
- import { relative, resolve } from 'node:path';
12
+ import { dirname, relative, resolve } from 'node:path';
14
13
  import { loadConfig, loadState, saveState } from '../config.js';
15
- import { PUSH_BATCH_SIZE, pushSprites } from '../api.js';
14
+ import { ensureEngineConnect } from '../util/engineConnect.js';
15
+ import { PUSH_BATCH_SIZE, pushSpritesAdaptive } from '../api.js';
16
16
  import { hashFile } from '../util/hash.js';
17
17
  import { createLimit } from '../util/limit.js';
18
18
  import { assetDiskPathFromKey, walkOutDirPngs } from '../util/paths.js';
@@ -35,7 +35,7 @@ export async function pushCommand(opts = {}) {
35
35
  * set is ingested without a second command).
36
36
  */
37
37
  export async function runPush(opts = {}) {
38
- const config = await loadConfig();
38
+ const config = await ensureEngineConnect(await loadConfig(), process.cwd(), !opts.dryRun);
39
39
  const state = await loadState();
40
40
  return runPushWith(config, state, opts);
41
41
  }
@@ -68,10 +68,22 @@ export async function runPushWith(config, state, opts = {}) {
68
68
  const candidates = [];
69
69
  const cwdAbs = resolve(process.cwd());
70
70
  let skippedUnsafe = 0;
71
- const synced = state.synced ?? {};
72
- const byRel = indexSyncedBySourceRel(synced);
71
+ const knownSynced = state.synced ?? {};
72
+ const byRel = indexSyncedBySourceRel(knownSynced);
73
73
  const scan = createLimit(PUSH_SCAN_CONCURRENCY);
74
- await Promise.all([...absByKey.entries()].map(([key, abs]) => scan(async () => {
74
+ const toHash = [...absByKey.entries()];
75
+ let hashed = 0;
76
+ const reportHash = () => {
77
+ hashed++;
78
+ const n = hashed.toLocaleString('en-US');
79
+ const t = toHash.length.toLocaleString('en-US');
80
+ if (spinner)
81
+ spinner.text = `Checking your game files… ${n} / ${t}`;
82
+ else if (opts.onStatus && (hashed === 1 || hashed === toHash.length || hashed % 25 === 0)) {
83
+ opts.onStatus(`Checking your game files… ${n} / ${t}`);
84
+ }
85
+ };
86
+ await Promise.all(toHash.map(([key, abs]) => scan(async () => {
75
87
  try {
76
88
  if (!verifiedIoPaths.has(abs)) {
77
89
  await assertSafeIoPath(abs, cwdAbs);
@@ -80,29 +92,33 @@ export async function runPushWith(config, state, opts = {}) {
80
92
  }
81
93
  catch {
82
94
  skippedUnsafe++;
95
+ reportHash();
83
96
  return;
84
97
  }
85
98
  const entry = entryByKey.get(key);
86
99
  const sourceRel = entry?.sourceRel ?? sourceByKey.get(key);
87
- const stateKey = resolveSyncedKey({ key, sourceRel }, synced, byRel);
88
- const known = stateKey ? synced[stateKey] : undefined;
100
+ const stateKey = resolveSyncedKey({ key, sourceRel }, knownSynced, byRel);
101
+ const known = stateKey ? knownSynced[stateKey] : undefined;
89
102
  const hint = known?.diskSha256 && known.diskMtimeMs != null && known.diskSize != null
90
103
  ? { sha256: known.diskSha256, mtimeMs: known.diskMtimeMs, size: known.diskSize }
91
104
  : undefined;
92
- const hashed = await hashFile(abs, hint);
93
- if (!hashed)
105
+ const hashedFile = await hashFile(abs, hint);
106
+ if (!hashedFile) {
107
+ reportHash();
94
108
  return;
109
+ }
95
110
  const segments = entry
96
111
  ? entry.adoptRel.replace(/\.png$/i, '').split('/')
97
112
  : key.split('/');
98
113
  candidates.push({
99
114
  key,
100
115
  segments,
101
- diskSha256: hashed.sha256,
116
+ diskSha256: hashedFile.sha256,
102
117
  ...(sourceRel ? { sourceRel } : {}),
103
- diskMtimeMs: hashed.mtimeMs,
104
- diskSize: hashed.size,
118
+ diskMtimeMs: hashedFile.mtimeMs,
119
+ diskSize: hashedFile.size,
105
120
  });
121
+ reportHash();
106
122
  })));
107
123
  const folderTreeKeys = new Set(matched.entries.map((e) => e.key));
108
124
  for (const k of absByKey.keys()) {
@@ -183,43 +199,37 @@ export async function runPushWith(config, state, opts = {}) {
183
199
  }
184
200
  const progress = quiet ? null : ora(`Pushing 0/${sprites.length}…`).start();
185
201
  const results = [];
186
- for (let i = 0; i < sprites.length; i += PUSH_BATCH_SIZE) {
187
- const batch = sprites.slice(i, i + PUSH_BATCH_SIZE);
188
- const batchResults = await pushSprites(batch);
189
- results.push(...batchResults);
190
- if (progress)
191
- progress.text = `Pushing ${Math.min(i + batch.length, sprites.length)}/${sprites.length}…`;
202
+ let synced = { ...(state.synced ?? {}) };
203
+ try {
204
+ for (let i = 0; i < sprites.length; i += PUSH_BATCH_SIZE) {
205
+ const batch = sprites.slice(i, i + PUSH_BATCH_SIZE);
206
+ const batchResults = await pushSpritesAdaptive(batch);
207
+ results.push(...batchResults);
208
+ synced = await commitPushResults({
209
+ synced,
210
+ batchResults,
211
+ candidates,
212
+ sourceByKey,
213
+ shaByKey,
214
+ fpByKey,
215
+ outDir: config.outDir,
216
+ });
217
+ await saveState({ ...state, synced });
218
+ if (progress)
219
+ progress.text = `Pushing ${Math.min(i + batch.length, sprites.length)}/${sprites.length}…`;
220
+ }
221
+ }
222
+ catch (e) {
223
+ progress?.fail(`Push stopped after ${results.length}/${sprites.length}. ${e.message}`);
224
+ if (results.length > 0)
225
+ await saveState({ ...state, synced });
226
+ throw e;
192
227
  }
193
228
  const counts = { created: 0, updated: 0, unchanged: 0, conflict: 0, error: 0 };
194
- const failedKeys = new Set(results.filter((r) => r.status === 'conflict' || r.status === 'error').map((r) => r.key));
195
- const fingerprinted = applyDiskFingerprints(state.synced ?? {}, candidates.filter((c) => !failedKeys.has(c.key)));
196
- const nextSynced = { ...fingerprinted.next };
197
229
  for (const r of results) {
198
- counts[r.status] = (counts[r.status] ?? 0) + 1;
199
- const stateKey = r.cloudKey && r.cloudKey !== r.key ? r.cloudKey : r.key;
200
- const sourceRel = sourceByKey.get(r.key) ?? sourceByKey.get(stateKey);
201
- if (stateKey !== r.key && (r.status === 'created' || r.status === 'updated')) {
202
- if (!sourceRel)
203
- await renameLocalSprite(config.outDir, r.key, stateKey);
204
- }
205
- if ((r.status === 'created' || r.status === 'updated' || r.status === 'unchanged') && r.assetId && r.sha256) {
206
- if (stateKey !== r.key)
207
- delete nextSynced[r.key];
208
- const diskSha256 = shaByKey.get(r.key) ?? shaByKey.get(stateKey) ?? nextSynced[stateKey]?.diskSha256;
209
- const fp = fpByKey.get(r.key) ?? fpByKey.get(stateKey);
210
- nextSynced[stateKey] = {
211
- assetId: r.assetId,
212
- layerIdx: typeof r.layerIdx === 'number' ? r.layerIdx : (nextSynced[stateKey]?.layerIdx ?? 0),
213
- sha256: r.sha256,
214
- layers: 1,
215
- ...(diskSha256 ? { diskSha256 } : {}),
216
- ...(fp?.diskMtimeMs != null ? { diskMtimeMs: fp.diskMtimeMs } : {}),
217
- ...(fp?.diskSize != null ? { diskSize: fp.diskSize } : {}),
218
- ...(sourceRel ? { sourceRel } : {}),
219
- };
220
- }
230
+ if (r.status in counts)
231
+ counts[r.status]++;
221
232
  }
222
- await saveState({ ...state, synced: nextSynced });
223
233
  const failed = counts.conflict + counts.error;
224
234
  const summary = `created ${counts.created}, updated ${counts.updated}, unchanged ${counts.unchanged}` +
225
235
  (counts.conflict ? `, conflicts ${counts.conflict}` : '') +
@@ -248,6 +258,38 @@ export async function runPushWith(config, state, opts = {}) {
248
258
  process.exitCode = 1;
249
259
  return { ...counts, imported: matched.entries.length };
250
260
  }
261
+ async function commitPushResults(opts) {
262
+ const { batchResults, candidates, sourceByKey, shaByKey, fpByKey, outDir } = opts;
263
+ const failedKeys = new Set(batchResults.filter((r) => r.status === 'conflict' || r.status === 'error').map((r) => r.key));
264
+ const batchKeys = new Set(batchResults.map((r) => r.key));
265
+ const fingerprinted = applyDiskFingerprints(opts.synced, candidates.filter((c) => batchKeys.has(c.key) && !failedKeys.has(c.key)));
266
+ const nextSynced = { ...fingerprinted.next };
267
+ for (const r of batchResults) {
268
+ const stateKey = r.cloudKey && r.cloudKey !== r.key ? r.cloudKey : r.key;
269
+ const sourceRel = sourceByKey.get(r.key) ?? sourceByKey.get(stateKey);
270
+ if (stateKey !== r.key && (r.status === 'created' || r.status === 'updated')) {
271
+ if (!sourceRel)
272
+ await renameLocalSprite(outDir, r.key, stateKey);
273
+ }
274
+ if ((r.status === 'created' || r.status === 'updated' || r.status === 'unchanged') && r.assetId && r.sha256) {
275
+ if (stateKey !== r.key)
276
+ delete nextSynced[r.key];
277
+ const diskSha256 = shaByKey.get(r.key) ?? shaByKey.get(stateKey) ?? nextSynced[stateKey]?.diskSha256;
278
+ const fp = fpByKey.get(r.key) ?? fpByKey.get(stateKey);
279
+ nextSynced[stateKey] = {
280
+ assetId: r.assetId,
281
+ layerIdx: typeof r.layerIdx === 'number' ? r.layerIdx : (nextSynced[stateKey]?.layerIdx ?? 0),
282
+ sha256: r.sha256,
283
+ layers: 1,
284
+ ...(diskSha256 ? { diskSha256 } : {}),
285
+ ...(fp?.diskMtimeMs != null ? { diskMtimeMs: fp.diskMtimeMs } : {}),
286
+ ...(fp?.diskSize != null ? { diskSize: fp.diskSize } : {}),
287
+ ...(sourceRel ? { sourceRel } : {}),
288
+ };
289
+ }
290
+ }
291
+ return nextSynced;
292
+ }
251
293
  /** Persist disk sha256 onto existing synced rows so the next scan can skip. */
252
294
  async function persistDiskFingerprints(state, candidates) {
253
295
  const { next, changed } = applyDiskFingerprints(state.synced ?? {}, candidates);
@@ -19,7 +19,7 @@ import { cmd } from '../util/invoke.js';
19
19
  */
20
20
  export async function startCommand(opts = {}) {
21
21
  console.log(kleur.bold('🪄 MagicPixel — first-run setup'));
22
- console.log(kleur.dim(' This links your game project. Sprites are indexed locally; nothing is imported until you connect a working set.'));
22
+ console.log(kleur.dim(' This links your game project and syncs sprites both ways.'));
23
23
  console.log();
24
24
  const kind = await detectProjectKind();
25
25
  // Engine projects (including Unity UPM packages) have no package.json
@@ -109,8 +109,7 @@ export async function startCommand(opts = {}) {
109
109
  console.log(` ${kleur.green('▶')} ${kleur.bold(`${cmd('sync')} --watch`)} ${kleur.dim('# keeps sprites fresh while you edit them in MagicPixel')}`);
110
110
  }
111
111
  console.log();
112
- console.log(kleur.dim(` Find sprites with \`${cmd('search')} hero\`, then \`${cmd('connect')} 'assets/Sprites/**'\` to add a working set.`));
113
- console.log(kleur.dim(' Connected sprites write back to their original game path. New MagicPixel art lands in outDir.'));
112
+ console.log(kleur.dim(' Game sprites write back to their original path. New MagicPixel art lands in outDir.'));
114
113
  console.log();
115
114
  if (await hasDevScript()) {
116
115
  const watchCmd = hasWatch ? 'npm run magicpixel:watch' : `${cmd('sync')} --watch`;