virtualmatter 0.6.2 → 0.6.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -44,6 +44,11 @@ was stopped), your copy stays put and the remote version lands next to it as
44
44
  `<name>.remote-conflict`. Merge, save, and it pushes. Edits made while sync
45
45
  was stopped go out when it starts again.
46
46
 
47
+ Large voxel and terrain files (several GB is normal) are streamed to disk,
48
+ never loaded into memory. Sync does not push a file over 8 MB, the most the
49
+ world accepts in one write. It warns once and leaves that file as it is on
50
+ both sides.
51
+
47
52
  `make`, `pull` and `sync` print the world's editor link ending in `?chat=closed`.
48
53
  Open it in any browser to watch changes land live - no install needed.
49
54
 
@@ -121,8 +126,8 @@ deploying it.
121
126
  | `pull [target] [dir]` | Mirror a world's file tree. `target` is any link to the world (or your website) or a framing id, and always resolves to the world's editor; omitted, it uses this folder's world or your only one. |
122
127
  | `sync [dir]` | Watch + two-way sync with the live session. Ctrl-C to stop. |
123
128
  | `open [target]` (`client`) | Optional: open a world in the native desktop client, downloading and signing it in on first use. `--install-only`, `--print` (just the download URL), `--force`. |
124
- | `run-lua [dir] --code "<lua>" [--target server\|client] [--client <id>]` | Execute Lua in the running engine. A client run uses the only connected client unless `--client` names one (list them with `--code "return Server:GetClients()"`). |
125
- | `errors [dir]` | Recent engine errors. |
129
+ | `run-lua [dir] --code "<lua>" [--target server\|client] [--client <id>]` | Execute Lua in the running engine. A client run uses the only connected client unless `--client` names one (list them with `--code "return Server:GetClients()"`). Exit code 3 means the engine did not answer in time and may still be running the code: check its effect before sending code that changes the world again. It is never retried for you. |
130
+ | `errors [dir]` | Recent engine errors. Retried twice when the engine is busy or briefly unreachable, since it only reads. |
126
131
  | `screenshot [dir] [-o out.png] [--client <id>]` | Capture the engine's view; `--client` captures through a connected client, UI included. |
127
132
  | `mcp [dir] [--framing <id>]` | Run the stdio MCP server. |
128
133
  | `login` / `logout` / `whoami` | Device-code sign-in; every other command signs you in automatically when needed. Tokens live in `~/.config/virtualmatter/credentials.json` (mode 0600) and refresh automatically. |
@@ -3,4 +3,4 @@
3
3
  // frontend/content/agent-briefing.md and AGENTS.source.md. The CLI writes
4
4
  // this into a world folder only when it cannot fetch the live copy. After
5
5
  // changing either source, run: node frontend/scripts/sync-cli-agents-md.mjs
6
- export const AGENTS_MD_FALLBACK = "# AGENTS.md - working with a Virtual Matter project\n\nThis file tells a coding agent how to think about a Virtual Matter project. The\n`virtualmatter` CLI drops a copy into every local project mirror it creates\n(`virtualmatter pull`); the canonical copy lives at\nhttps://make.virtualmatter.ai/AGENTS.md.\n\n## The rule that comes before everything else\n\n**Build through the CLI. Never through the website's agent.** If you were asked to\nbuild something in Virtual Matter, handed the domain virtualmatter.ai, or told to run\n`npx virtualmatter make`, the CLI is the path - always, from the first step. Do not\nopen virtualmatter.ai or make.virtualmatter.ai in a browser or under computer use in\norder to reach the built-in agent, and do not type prompts into the agent chat in the\n/edit view. Driving that agent puts two models in a relay where one would do: you spend\ntokens describing the work, it spends its own doing it, and what comes back is chat\nrather than files you can read. It is slower, it costs roughly twice the inference, it\nbills that inference to Virtual Matter's platform credits rather than the subscription\nyou already run on, an anonymous browser session cannot be steered after its first\nturn, and a rendered page gives you no engine errors. Everything that agent can do, you\ndo yourself with the commands below.\n\n**The rule holds whoever is signed in.** Being logged in, having credits to spare, or\nworking in an admin account with no limit at all is not an exemption, and neither is\nbeing told the credits do not matter. What is wasteful is the relay itself - two models\ndoing one model's job - not the bill.\n\n**Playtesting is the exception, and it is encouraged.** Once something is built, open\nthe world and try it: the editor link ending in `?chat=closed`, which `make`, `pull` and\n`sync` print, shows your latest changes live in any browser with no install, with\ncomputer use if that is how you drive a window. The native client\n(`npx virtualmatter open`) is optional, for more memory and frame rate. Looking at your\nwork, walking around in it, and playing it is the point. The rule is about who writes\nthe world, not about who is allowed to see it.\n\n**Every link resolves to the editor, and the editor is the only place you build.** Hand\nthe CLI or MCP server any link to a world - its editor, a play or share link, the\nproject page, an embed's iframe src, or the maker's own website - and it works on that\nworld's editor (its make framing). Play servers show the last published version and are\noverwritten on every publish, so work never goes there, and their /play and /g links\nshow that published copy rather than your latest edits.\n\n## The mental model\n\n- The world itself runs on Virtual Matter servers, not on the maker's machine. There is\n one authoritative server simulation per world session, and any number of connected\n clients (browser WASM or the native client).\n- The project's content is SDK Lua scripts plus assets (voxel data, prefabs, images,\n sounds) under a `Montage/` tree. Virtual Matter can publish that tree to a GitHub repo\n in the maker's account for versioning and remixing; pushing to that repo does not feed\n back into the running world today - live edits happen in the session.\n- Changes hot-reload. Saving a `.lua` file in the live session's Montage tree re-runs\n the module and its `Start()` on a fresh instance inside the running world - no restart,\n no build step.\n- Prefabs are reusable scene-graph snippets: JSON files with a `.prefab` extension in the\n Montage folder, instantiated at runtime via `Server:InsertPrefab(asset)`.\n- There is no local runtime - the world cannot run on the maker's machine. The\n `virtualmatter` CLI (npm) is how you reach it from a local harness, and it needs no\n setup: the first command that needs an account signs you in with a device code.\n `npx virtualmatter make \"My world\"` makes a world and mirrors its Montage tree\n into `./my-world`; `npx virtualmatter pull <any Virtual Matter link>` mirrors an\n existing one (/edit, /play, /g, /p, /projects links all work, with or without a\n readable slug); `npx virtualmatter list` shows your worlds; `npx virtualmatter sync`\n live-pushes saves into the running session (which hot-reloads them);\n `npx virtualmatter run-lua` / `errors` / `screenshot` drive the engine;\n `npx virtualmatter open` optionally opens the world in the native desktop client,\n downloaded and signed in on first use; and `npx virtualmatter mcp` serves\n all of it over MCP. A mirrored folder carries `.mcp.json` and `.cursor/mcp.json`\n (so Claude Code and Cursor register the MCP server on their own), this file, and a\n `CLAUDE.md` that imports it. Folders from virtualmatter 0.5.0 on also carry\n `.codex/config.toml`, which Codex loads once you trust the folder; with an older\n CLI, or if Codex does not list the server, register it once with\n `codex mcp add virtualmatter -- npx -y virtualmatter mcp`. In Claude Code outside a\n mirrored folder, use `claude mcp add virtualmatter -- npx -y virtualmatter mcp`. The\n server works before a world\n is selected (`list_projects`, `create_project`, `select_project`), then exposes\n `list_files`, `read_file`, `write_file`, `run_lua`, `get_engine_errors`,\n `capture_screenshot`, `open_native_client`, and `world_info`.\n- Work through the CLI, not the website - see the rule above. Prompting the built-in\n agent to do your building is the one thing to avoid; opening the world to play it is\n not.\n- Seeing your work: `virtualmatter screenshot` shoots a default overview of the world\n origin, `--target <object>` frames one object by name or id, and `--at x,y,z --rot\n yaw,pitch,roll` places the camera exactly. Y is up and -Z is forward, and the\n rotation really is yaw first: yaw 0 faces -Z, pitch -90 looks straight down, pitch 0\n is the horizon. The MCP `capture_screenshot`\n tool takes the same arguments. Verify a change with a screenshot plus `errors` rather\n than assuming a write worked.\n- The mirrored folder's AGENTS.md is this guide followed by the engine SDK's own agent guide\n (the AGENTS.md that lives in every Montage tree and inside the desktop client's\n Data/Sdk/Montage/). The engine guide assumes an in-session agent driving the engine\n through `atomo`; the merged file maps each `atomo` step to the CLI or MCP equivalent, and\n the `Skills/*.md` references it points at are in the folder. The SDK's own tooling\n (`atomo`, `vm_auth.py`, the agent-log hooks) is not mirrored: it only works inside a\n session. Sandboxed agents need network access for every CLI command.\n\n## Reference\n\nThe same facts as terse lists, for quick lookup.\n\n### Project model\n- A project (\"montage\") is one voxel world hosted with Virtual Matter.\n- A project has framings: \"make\" framings (editing sessions) and \"play\"\n framings (public play sessions). A framing is a running session slot\n on a voxel host.\n- Project content = SDK Lua scripts + assets under a Montage/ tree.\n Edits made in the live session hot-reload into the running world.\n The tree can be published to a maker-owned GitHub repo.\n\n### URL shapes (make.virtualmatter.ai)\n- /new builder for people: create a world from a prompt\n (also at https://virtualmatter.ai/); coding agents\n use npx virtualmatter make instead\n- /edit/<framing-id> maker session: live world + agent chat. The chat is\n for people; a coding agent builds with the CLI and\n never prompts it\n (also /edit/<slug>-<framing-id>; the CLI accepts both)\n- /play/<framing-id> play session for a specific framing\n- /g/<id> share URL resolver: 302s to a live session\n\n### APIs (unauthenticated)\n- GET https://make.virtualmatter.ai/api/v1/public/native-clients\n JSON: { iteration, clients: [{ platform, kind, variant, url, filename,\n file_size, branch, commit, match }] } - per-platform native client\n installers (kind \"download\") or store links (kind \"store\", iOS).\n Windows and Linux list two downloads each: variant \"installer\" /\n \"flatpak\" for people, and \"portable\" / \"tarball\", the file tree the\n CLI installs.\n\n### CLI + MCP (npm package \"virtualmatter\", Node >= 20)\n- npx virtualmatter make \"<name>\" make a world, mirror it into ./<slug>\n- npx virtualmatter pull <link> mirror an existing world (any link shape)\n- npx virtualmatter list your worlds with framing ids + URLs\n- npx virtualmatter sync live-push saves into the running world\n- npx virtualmatter open optional: open it in the native client\n- npx virtualmatter mcp stdio MCP server (list_projects,\n create_project, select_project, list_files, read_file, write_file,\n run_lua, get_engine_errors, capture_screenshot, open_native_client,\n world_info)\n- Sign-in happens on first use (device code); no separate login step.\n- These commands are the whole build path. Never prompt the built-in\n agent on the site to do the work; a browser or the native client is\n for playtesting what you built.\n- Any link to a world (editor, play or share link, project page, embed,\n or the maker's own website) resolves to its editor, the only place\n the CLI and MCP server work. Play servers show the last published\n version and are overwritten on publish.\n- Watch changes land live at the editor link ending in ?chat=closed,\n in any browser, no install. The native client is optional.\n- A mirrored folder carries .mcp.json, .cursor/mcp.json, AGENTS.md and\n CLAUDE.md, so Claude Code and Cursor register the server on their own.\n- Codex: folders from 0.5.0 on carry .codex/config.toml, loaded once the\n folder is trusted; otherwise register the server once:\n codex mcp add virtualmatter -- npx -y virtualmatter mcp\n\n## SDK Lua cheat sheet\n\nThe scripting language is Lua 5.4, sandboxed: `os`, `io`, `require`, `package`,\n`dofile`, `loadfile`, `loadstring` are nil. `math`, `string`, `table`, `coroutine`\nremain. No `os.time` - use `Time.time` (sim time), `Time.frame`, or\n`AE:GetDebugTime()` (wall clock).\n\n### Script shape\n\nEvery persistent behavior is a `.lua` file in the Montage tree returning a `self` table:\n\n```lua\nlocal self = {}\nfunction self:Start() end\nfunction self:Update(deltaTime) end\nreturn self\n```\n\nAttach to an object with `obj:AddScript(\"My Folder/Example.lua\", sync)` - the path is\nrelative to the Montage root; `sync = true` replicates the script to clients.\n`obj:FindScript(\"Example\")` returns the live instance.\n\n### Server / client split\n\nThe same script runs on the server and (when synced) on every client. Branch with\n`self.onServer` / `self.onClient`:\n\n```lua\nfunction self:Update(dt)\n if self.onClient then return end -- voxel edits are server-only\n -- authoritative logic here\nend\n```\n\nNothing replicates automatically: set `syncToClients = true` on the script or\nVoxelData component, and use `util:makeNetworkedTable(self, { hp = 100 })` for\nproperties that should sync (server writes, clients read, deltas only).\n`obj.pos` / `obj.rot` do not auto-sync - replicate them yourself.\n\n### RPC\n\n```lua\nself:RPC(\"serverDoSomething\", pos, dir) -- on a client: goes to the server\nfunction self:serverDoSomething(pos, dir, clientID) -- clientID auto-appended\n assert(self.onServer)\nend\n```\n\nOn the server, `self:RPC(...)` fans out to all clients. Reliable, FIFO per direction.\nRequires `self.component.syncToClients = true`.\n\n### Scene and objects (server)\n\n```lua\nlocal ob = Scene:CreateObject(\"Name\")\nob.save = true\nob:AddScript(\"Path/Script.lua\")\nScene:GetObjectByName(\"Name\")\nScene:CloneObject(ob) -- prefer over rebuilding\nob.active = false -- prefer over destroy\nob:AddTag(\"Enemy\"); FindObjectsWithTag(\"Enemy\") -- tags are runtime-only state\n```\n\n### Voxel editing (server only, async)\n\n```lua\nVox:Add(Sphere(pos, 2)):Color(1, 0, 0):Run()\nVox:Add(Box(Vec3(0, -1, 0), Vec3(20, 2, 20))):ForceStatic():Run()\n```\n\nShapes: `Box(center, fullSize)`, `Sphere(center, r)`, `Capsule(p1, p2, r)`,\n`Cylinder(p1, p2, r1[, r2])`. `Run()` is async - chain `:OnFinished(fn)` before\n`:Run()` to read post-commit state. Big edits stall the frame for every client;\nkeep in-game edits small and infrequent.\n\n### Input (event-driven, client input reaches the server)\n\n```lua\nself.component.syncToClients = true\nevents.keyDown.addListener(self, function(key, from) self.keys[key] = true end)\nevents.keyUp.addListener(self, function(key, from) self.keys[key] = false end)\n```\n\nPlayer-driven movement must be client-predicted - never gate the player's own\nfeedback on the server round-trip.\n\n### Physics\n\n```lua\nlocal rb = obj:AddComponent(\"RigidBody\")\nrb:AddImpulse(Vec3(0, 50, 0), obj.pos)\nrb.velocity; rb.mass; rb.gravityScale\n```\n\n### Time and diagnostics\n\n```lua\nTime.dt; Time.time; Time.frame; Time.timeScale\nAE:GetLogValue(\"Raycasts\") -- engine counters, e.g. raycasts this frame\nAE:GetAssets() -- list all assets (APIs take Assets, not paths)\n```\n\n### UI (client-side HUD)\n\nBuild screen UI with the MUI builder: `UI:AddPanel|AddButton|AddLabel|AddSlider|...`\nchained with `:Set{...}`; flexbox-like row-wrap layout.\n\n## Conventions\n\n- Coordinates: Y is up, -Z is forward, X is right.\n- Script files are named in capital case with spaces (`Character Controller.lua`).\n- Check whether a script is already attached before adding it - duplicates run twice.\n- The server is for authority (world edits, spawns, state); clients are for feel\n (prediction, FX, UI).\n\n## Embed or build a world for an existing website\n\nFor a shared project link, fetch `/api/v1/public/embed?target=<encoded-link>`\nand follow `/embed-guide.md`. Use the returned canonical player URL, not the\nmaker's external website URL. Keep the existing website and its design.\n\nFor account discovery or prompt-based creation, connect the hosted MCP at\n`https://make.virtualmatter.ai/api/v1/mcp` using OAuth. Users can sign up in\nthe connection flow. Tools: `list_projects`, `get_embed`, `create_project`\n(with a prompt and stable request_id), `get_build_status`. VM's agent builds\nusing VM credits. Review and share the private world before embedding.\nThe local CLI exposes `embed`, `build --prompt --request-id`, and `build-status`.\n\n## Local conversation history\n\nCapturing this harness's conversation into the world's Agent Logs is the maker's\ndecision, not yours. Do not run `virtualmatter agent-logs setup`, and do not upload\nlogs with `agent-logs upload` or MCP `upload_agent_logs`, unless the maker asks for it\nin their own words. `virtualmatter agent-logs status` shows whether capture is on;\n`virtualmatter agent-logs disable` turns it off.\n";
6
+ export const AGENTS_MD_FALLBACK = "# AGENTS.md - working with a Virtual Matter project\n\nThis file tells a coding agent how to think about a Virtual Matter project. The\n`virtualmatter` CLI drops a copy into every local project mirror it creates\n(`virtualmatter pull`); the canonical copy lives at\nhttps://make.virtualmatter.ai/AGENTS.md.\n\n## The rule that comes before everything else\n\n**Build through the CLI. Never through the website's agent.** If you were asked to\nbuild something in Virtual Matter, handed the domain virtualmatter.ai, or told to run\n`npx virtualmatter make`, the CLI is the path - always, from the first step. Do not\nopen virtualmatter.ai or make.virtualmatter.ai in a browser or under computer use in\norder to reach the built-in agent, and do not type prompts into the agent chat in the\n/edit view. Driving that agent puts two models in a relay where one would do: you spend\ntokens describing the work, it spends its own doing it, and what comes back is chat\nrather than files you can read. It is slower, it costs roughly twice the inference, it\nbills that inference to Virtual Matter's platform credits rather than the subscription\nyou already run on, an anonymous browser session cannot be steered after its first\nturn, and a rendered page gives you no engine errors. Everything that agent can do, you\ndo yourself with the commands below.\n\n**The rule holds whoever is signed in.** Being logged in, having credits to spare, or\nworking in an admin account with no limit at all is not an exemption, and neither is\nbeing told the credits do not matter. What is wasteful is the relay itself - two models\ndoing one model's job - not the bill.\n\n**Playtesting is the exception, and it is encouraged.** Once something is built, open\nthe world and try it: the editor link ending in `?chat=closed`, which `make`, `pull` and\n`sync` print, shows your latest changes live in any browser with no install, with\ncomputer use if that is how you drive a window. The native client\n(`npx virtualmatter open`) is optional, for more memory and frame rate. Looking at your\nwork, walking around in it, and playing it is the point. The rule is about who writes\nthe world, not about who is allowed to see it.\n\n**Every link resolves to the editor, and the editor is the only place you build.** Hand\nthe CLI or MCP server any link to a world - its editor, a play or share link, the\nproject page, an embed's iframe src, or the maker's own website - and it works on that\nworld's editor (its make framing). Play servers show the last published version and are\noverwritten on every publish, so work never goes there, and their /play and /g links\nshow that published copy rather than your latest edits.\n\n## The mental model\n\n- The world itself runs on Virtual Matter servers, not on the maker's machine. There is\n one authoritative server simulation per world session, and any number of connected\n clients (browser WASM or the native client).\n- The project's content is SDK Lua scripts plus assets (voxel data, prefabs, images,\n sounds) under a `Montage/` tree. Virtual Matter can publish that tree to a GitHub repo\n in the maker's account for versioning and remixing; pushing to that repo does not feed\n back into the running world today - live edits happen in the session.\n- Changes hot-reload. Saving a `.lua` file in the live session's Montage tree re-runs\n the module and its `Start()` on a fresh instance inside the running world - no restart,\n no build step.\n- Prefabs are reusable scene-graph snippets: JSON files with a `.prefab` extension in the\n Montage folder, instantiated at runtime via `Server:InsertPrefab(asset)`.\n- There is no local runtime - the world cannot run on the maker's machine. The\n `virtualmatter` CLI (npm) is how you reach it from a local harness, and it needs no\n setup: the first command that needs an account signs you in with a device code.\n `npx virtualmatter make \"My world\"` makes a world and mirrors its Montage tree\n into `./my-world`; `npx virtualmatter pull <any Virtual Matter link>` mirrors an\n existing one (/edit, /play, /g, /p, /projects links all work, with or without a\n readable slug); `npx virtualmatter list` shows your worlds; `npx virtualmatter sync`\n live-pushes saves into the running session (which hot-reloads them);\n `npx virtualmatter run-lua` / `errors` / `screenshot` drive the engine;\n `npx virtualmatter open` optionally opens the world in the native desktop client,\n downloaded and signed in on first use; and `npx virtualmatter mcp` serves\n all of it over MCP. A mirrored folder carries `.mcp.json` and `.cursor/mcp.json`\n (so Claude Code and Cursor register the MCP server on their own), this file, and a\n `CLAUDE.md` that imports it. Folders from virtualmatter 0.5.0 on also carry\n `.codex/config.toml`, which Codex loads once you trust the folder; with an older\n CLI, or if Codex does not list the server, register it once with\n `codex mcp add virtualmatter -- npx -y virtualmatter mcp`. In Claude Code outside a\n mirrored folder, use `claude mcp add virtualmatter -- npx -y virtualmatter mcp`. The\n server works before a world\n is selected (`list_projects`, `create_project`, `select_project`), then exposes\n `list_files`, `read_file`, `write_file`, `run_lua`, `get_engine_errors`,\n `capture_screenshot`, `open_native_client`, and `world_info`.\n- Work through the CLI, not the website - see the rule above. Prompting the built-in\n agent to do your building is the one thing to avoid; opening the world to play it is\n not.\n- Seeing your work: `virtualmatter screenshot` shoots a default overview of the world\n origin, `--target <object>` frames one object by name or id, and `--at x,y,z --rot\n yaw,pitch,roll` places the camera exactly. Y is up and -Z is forward, and the\n rotation really is yaw first: yaw 0 faces -Z, pitch -90 looks straight down, pitch 0\n is the horizon. The MCP `capture_screenshot`\n tool takes the same arguments. Verify a change with a screenshot plus `errors` rather\n than assuming a write worked.\n- A `run-lua` that fails with HTTP 504 (`run-lua timed out`, exit code 3 from CLI\n 0.6.4 on; the MCP `run_lua` tool returns the same text) reached the engine, which\n may still be running it. Do not send code that changes the world again: check its\n effect first, by reading back what it sets or taking a screenshot. Any other\n failure never reached the engine and is safe to retry.\n- The mirrored folder's AGENTS.md is this guide followed by the engine SDK's own agent guide\n (the AGENTS.md that lives in every Montage tree and inside the desktop client's\n Data/Sdk/Montage/). The engine guide assumes an in-session agent driving the engine\n through `atomo`; the merged file maps each `atomo` step to the CLI or MCP equivalent, and\n the `Skills/*.md` references it points at are in the folder. The SDK's own tooling\n (`atomo`, `vm_auth.py`, the agent-log hooks) is not mirrored: it only works inside a\n session. Sandboxed agents need network access for every CLI command.\n\n## Reference\n\nThe same facts as terse lists, for quick lookup.\n\n### Project model\n- A project (\"montage\") is one voxel world hosted with Virtual Matter.\n- A project has framings: \"make\" framings (editing sessions) and \"play\"\n framings (public play sessions). A framing is a running session slot\n on a voxel host.\n- Project content = SDK Lua scripts + assets under a Montage/ tree.\n Edits made in the live session hot-reload into the running world.\n The tree can be published to a maker-owned GitHub repo.\n\n### URL shapes (make.virtualmatter.ai)\n- /new builder for people: create a world from a prompt\n (also at https://virtualmatter.ai/); coding agents\n use npx virtualmatter make instead\n- /edit/<framing-id> maker session: live world + agent chat. The chat is\n for people; a coding agent builds with the CLI and\n never prompts it\n (also /edit/<slug>-<framing-id>; the CLI accepts both)\n- /play/<framing-id> play session for a specific framing\n- /g/<id> share URL resolver: 302s to a live session\n\n### APIs (unauthenticated)\n- GET https://make.virtualmatter.ai/api/v1/public/native-clients\n JSON: { iteration, clients: [{ platform, kind, variant, url, filename,\n file_size, branch, commit, match }] } - per-platform native client\n installers (kind \"download\") or store links (kind \"store\", iOS).\n Windows and Linux list two downloads each: variant \"installer\" /\n \"flatpak\" for people, and \"portable\" / \"tarball\", the file tree the\n CLI installs.\n\n### CLI + MCP (npm package \"virtualmatter\", Node >= 20)\n- npx virtualmatter make \"<name>\" make a world, mirror it into ./<slug>\n- npx virtualmatter pull <link> mirror an existing world (any link shape)\n- npx virtualmatter list your worlds with framing ids + URLs\n- npx virtualmatter sync live-push saves into the running world\n- npx virtualmatter open optional: open it in the native client\n- npx virtualmatter mcp stdio MCP server (list_projects,\n create_project, select_project, list_files, read_file, write_file,\n run_lua, get_engine_errors, capture_screenshot, open_native_client,\n world_info)\n- Sign-in happens on first use (device code); no separate login step.\n- These commands are the whole build path. Never prompt the built-in\n agent on the site to do the work; a browser or the native client is\n for playtesting what you built.\n- Any link to a world (editor, play or share link, project page, embed,\n or the maker's own website) resolves to its editor, the only place\n the CLI and MCP server work. Play servers show the last published\n version and are overwritten on publish.\n- Watch changes land live at the editor link ending in ?chat=closed,\n in any browser, no install. The native client is optional.\n- A mirrored folder carries .mcp.json, .cursor/mcp.json, AGENTS.md and\n CLAUDE.md, so Claude Code and Cursor register the server on their own.\n- Codex: folders from 0.5.0 on carry .codex/config.toml, loaded once the\n folder is trusted; otherwise register the server once:\n codex mcp add virtualmatter -- npx -y virtualmatter mcp\n\n## SDK Lua cheat sheet\n\nThe scripting language is Lua 5.4, sandboxed: `os`, `io`, `require`, `package`,\n`dofile`, `loadfile`, `loadstring` are nil. `math`, `string`, `table`, `coroutine`\nremain. No `os.time` - use `Time.time` (sim time), `Time.frame`, or\n`AE:GetDebugTime()` (wall clock).\n\n### Script shape\n\nEvery persistent behavior is a `.lua` file in the Montage tree returning a `self` table:\n\n```lua\nlocal self = {}\nfunction self:Start() end\nfunction self:Update(deltaTime) end\nreturn self\n```\n\nAttach to an object with `obj:AddScript(\"My Folder/Example.lua\", sync)` - the path is\nrelative to the Montage root; `sync = true` replicates the script to clients.\n`obj:FindScript(\"Example\")` returns the live instance.\n\n### Server / client split\n\nThe same script runs on the server and (when synced) on every client. Branch with\n`self.onServer` / `self.onClient`:\n\n```lua\nfunction self:Update(dt)\n if self.onClient then return end -- voxel edits are server-only\n -- authoritative logic here\nend\n```\n\nNothing replicates automatically: set `syncToClients = true` on the script or\nVoxelData component, and use `util:makeNetworkedTable(self, { hp = 100 })` for\nproperties that should sync (server writes, clients read, deltas only).\n`obj.pos` / `obj.rot` do not auto-sync - replicate them yourself.\n\n### RPC\n\n```lua\nself:RPC(\"serverDoSomething\", pos, dir) -- on a client: goes to the server\nfunction self:serverDoSomething(pos, dir, clientID) -- clientID auto-appended\n assert(self.onServer)\nend\n```\n\nOn the server, `self:RPC(...)` fans out to all clients. Reliable, FIFO per direction.\nRequires `self.component.syncToClients = true`.\n\n### Scene and objects (server)\n\n```lua\nlocal ob = Scene:CreateObject(\"Name\")\nob.save = true\nob:AddScript(\"Path/Script.lua\")\nScene:GetObjectByName(\"Name\")\nScene:CloneObject(ob) -- prefer over rebuilding\nob.active = false -- prefer over destroy\nob:AddTag(\"Enemy\"); FindObjectsWithTag(\"Enemy\") -- tags are runtime-only state\n```\n\n### Voxel editing (server only, async)\n\n```lua\nVox:Add(Sphere(pos, 2)):Color(1, 0, 0):Run()\nVox:Add(Box(Vec3(0, -1, 0), Vec3(20, 2, 20))):ForceStatic():Run()\n```\n\nShapes: `Box(center, fullSize)`, `Sphere(center, r)`, `Capsule(p1, p2, r)`,\n`Cylinder(p1, p2, r1[, r2])`. `Run()` is async - chain `:OnFinished(fn)` before\n`:Run()` to read post-commit state. Big edits stall the frame for every client;\nkeep in-game edits small and infrequent.\n\n### Input (event-driven, client input reaches the server)\n\n```lua\nself.component.syncToClients = true\nevents.keyDown.addListener(self, function(key, from) self.keys[key] = true end)\nevents.keyUp.addListener(self, function(key, from) self.keys[key] = false end)\n```\n\nPlayer-driven movement must be client-predicted - never gate the player's own\nfeedback on the server round-trip.\n\n### Physics\n\n```lua\nlocal rb = obj:AddComponent(\"RigidBody\")\nrb:AddImpulse(Vec3(0, 50, 0), obj.pos)\nrb.velocity; rb.mass; rb.gravityScale\n```\n\n### Time and diagnostics\n\n```lua\nTime.dt; Time.time; Time.frame; Time.timeScale\nAE:GetLogValue(\"Raycasts\") -- engine counters, e.g. raycasts this frame\nAE:GetAssets() -- list all assets (APIs take Assets, not paths)\n```\n\n### UI (client-side HUD)\n\nBuild screen UI with the MUI builder: `UI:AddPanel|AddButton|AddLabel|AddSlider|...`\nchained with `:Set{...}`; flexbox-like row-wrap layout.\n\n## Conventions\n\n- Coordinates: Y is up, -Z is forward, X is right.\n- Script files are named in capital case with spaces (`Character Controller.lua`).\n- Check whether a script is already attached before adding it - duplicates run twice.\n- The server is for authority (world edits, spawns, state); clients are for feel\n (prediction, FX, UI).\n\n## Embed or build a world for an existing website\n\nFor a shared project link, fetch `/api/v1/public/embed?target=<encoded-link>`\nand follow `/embed-guide.md`. Use the returned canonical player URL, not the\nmaker's external website URL. Keep the existing website and its design.\n\nFor account discovery or prompt-based creation, connect the hosted MCP at\n`https://make.virtualmatter.ai/api/v1/mcp` using OAuth. Users can sign up in\nthe connection flow. Tools: `list_projects`, `get_embed`, `create_project`\n(with a prompt and stable request_id), `get_build_status`. VM's agent builds\nusing VM credits. Review and share the private world before embedding.\nThe local CLI exposes `embed`, `build --prompt --request-id`, and `build-status`.\n\n## Local conversation history\n\nCapturing this harness's conversation into the world's Agent Logs is the maker's\ndecision, not yours. Do not run `virtualmatter agent-logs setup`, and do not upload\nlogs with `agent-logs upload` or MCP `upload_agent_logs`, unless the maker asks for it\nin their own words. `virtualmatter agent-logs status` shows whether capture is on;\n`virtualmatter agent-logs disable` turns it off.\n";
package/dist/files.js CHANGED
@@ -2,7 +2,25 @@
2
2
  * Client for the session file + engine API served at
3
3
  * https://<voxel_host>/s/<session_id>/api/montage/<framing_id>/...
4
4
  */
5
+ import crypto from "node:crypto";
6
+ import fs from "node:fs";
7
+ import path from "node:path";
8
+ import { Readable, Transform } from "node:stream";
9
+ import { pipeline } from "node:stream/promises";
5
10
  import { authorizedFetch } from "./api.js";
11
+ /**
12
+ * The session API's per-write cap (session-agent server/api/files.py
13
+ * MAX_WRITE_BYTES). A bigger PUT is answered 413, so sync skips such files
14
+ * up front instead of reading gigabytes of voxel data just to be refused.
15
+ */
16
+ export const MAX_WRITE_BYTES = 8 * 1024 * 1024;
17
+ /**
18
+ * Largest body `get()` reads into memory. Node refuses a Buffer past
19
+ * 2^31-1 bytes, and worlds do carry multi-GB voxel/terrain files - Rooftop
20
+ * Rumble's 2.7 GB one crashed every pull. Anything that writes a file to
21
+ * disk uses `download()`, which streams and has no limit.
22
+ */
23
+ export const MAX_BUFFERED_BYTES = 256 * 1024 * 1024;
6
24
  /**
7
25
  * The reason an engine call failed, read out of the response body.
8
26
  *
@@ -12,23 +30,42 @@ import { authorizedFetch } from "./api.js";
12
30
  * browser instead. Falls back to the status when the body says nothing.
13
31
  */
14
32
  export async function engineErrorMessage(res, what) {
15
- let detail = "";
33
+ const detail = await errorDetail(res);
34
+ return `${what} failed: HTTP ${res.status}${detail ? ` - ${detail}` : ""}`;
35
+ }
36
+ async function errorDetail(res) {
16
37
  try {
17
38
  const body = (await res.json());
18
39
  const d = body.detail;
19
40
  if (typeof d === "string")
20
- detail = d;
21
- else if (d && typeof d === "object" && typeof d.message === "string") {
22
- detail = d.message;
41
+ return d;
42
+ if (d && typeof d === "object" && typeof d.message === "string") {
43
+ return d.message;
23
44
  }
24
- else if (d)
25
- detail = JSON.stringify(d);
45
+ return d ? JSON.stringify(d) : "";
26
46
  }
27
47
  catch {
28
- /* non-JSON body */
48
+ return ""; // non-JSON body, e.g. a proxy's own 504 page
29
49
  }
30
- return `${what} failed: HTTP ${res.status}${detail ? ` - ${detail}` : ""}`;
31
50
  }
51
+ /**
52
+ * The engine did not answer an engine call in time (HTTP 504). The session
53
+ * relay answers 504 `engine_timeout` only when the call reached the engine,
54
+ * so for run-lua the snippet may still be running: resending it can run a
55
+ * mutation twice. Nothing retries it, and the CLI exits with
56
+ * ENGINE_TIMEOUT_EXIT_CODE so a script can tell it from a plain failure.
57
+ */
58
+ export class EngineTimeoutError extends Error {
59
+ constructor(message) {
60
+ super(message);
61
+ this.name = "EngineTimeoutError";
62
+ }
63
+ }
64
+ export const ENGINE_TIMEOUT_EXIT_CODE = 3;
65
+ const RUN_LUA_TIMEOUT = "The engine did not answer in time; it may still be running this Lua. Do not resend code that changes the world: check its effect first.";
66
+ /** How long `engineErrors` waits before each retry of a 503/504; it only reads. */
67
+ const READ_RETRY_DELAYS_MS = [1_000, 3_000];
68
+ const defaultSleep = (ms) => new Promise((r) => setTimeout(r, ms));
32
69
  export class ConflictError extends Error {
33
70
  filePath;
34
71
  constructor(filePath) {
@@ -37,6 +74,10 @@ export class ConflictError extends Error {
37
74
  this.name = "ConflictError";
38
75
  }
39
76
  }
77
+ function tooBigToBuffer(filePath, size, limit) {
78
+ const mb = (n) => `${Math.round(n / (1024 * 1024))} MB`;
79
+ return `${filePath} is ${mb(size)}, too large to read into memory (limit ${mb(limit)}). It is in the folder \`virtualmatter pull\` mirrored - read it from disk.`;
80
+ }
40
81
  function encodePath(p) {
41
82
  return p.split("/").map(encodeURIComponent).join("/");
42
83
  }
@@ -58,14 +99,49 @@ export function normalizeEtag(v) {
58
99
  s = s.slice(1, -1);
59
100
  return s;
60
101
  }
102
+ /**
103
+ * Write a response body to `destAbs` without holding it in memory, hashing
104
+ * it on the way. Lands in a temp file beside the target and is renamed into
105
+ * place, so the target is either the old file or the whole new one - never a
106
+ * truncated half that a later sync would push back as an edit.
107
+ */
108
+ export async function streamToFile(res, destAbs) {
109
+ fs.mkdirSync(path.dirname(destAbs), { recursive: true });
110
+ const tmp = `${destAbs}.vmdownload-${process.pid}-${crypto.randomBytes(4).toString("hex")}`;
111
+ const hasher = crypto.createHash("sha256");
112
+ let size = 0;
113
+ const tap = new Transform({
114
+ transform(chunk, _enc, cb) {
115
+ hasher.update(chunk);
116
+ size += chunk.length;
117
+ cb(null, chunk);
118
+ },
119
+ });
120
+ try {
121
+ const source = res.body
122
+ ? Readable.fromWeb(res.body)
123
+ : Readable.from([]);
124
+ await pipeline(source, tap, fs.createWriteStream(tmp));
125
+ fs.renameSync(tmp, destAbs);
126
+ }
127
+ catch (err) {
128
+ fs.rmSync(tmp, { force: true });
129
+ throw err;
130
+ }
131
+ return { hash: hasher.digest("hex"), size };
132
+ }
61
133
  export class FilesClient {
62
134
  sessionBase;
63
135
  framingId;
64
136
  fetchFn;
65
- constructor(sessionBase, framingId, fetchFn = fetch) {
137
+ maxBufferedBytes;
138
+ sleep;
139
+ constructor(sessionBase, framingId, fetchFn = fetch, maxBufferedBytes = MAX_BUFFERED_BYTES, sleep = defaultSleep) {
66
140
  this.sessionBase = sessionBase;
67
141
  this.framingId = framingId;
68
142
  this.fetchFn = fetchFn;
143
+ this.maxBufferedBytes = maxBufferedBytes;
144
+ this.sleep = sleep;
69
145
  }
70
146
  get apiRoot() {
71
147
  return `${this.sessionBase}/api/montage/${encodeURIComponent(this.framingId)}`;
@@ -87,8 +163,30 @@ export class FilesClient {
87
163
  const res = await authorizedFetch(`${this.apiRoot}/files/content/${encodePath(filePath)}`, {}, this.fetchFn);
88
164
  if (!res.ok)
89
165
  throw new Error(`Downloading ${filePath} failed: HTTP ${res.status}`);
90
- const bytes = Buffer.from(await res.arrayBuffer());
91
- return { bytes, etag: normalizeEtag(res.headers.get("ETag")) };
166
+ const declared = Number(res.headers.get("Content-Length"));
167
+ if (Number.isFinite(declared) && declared > this.maxBufferedBytes) {
168
+ await res.body?.cancel();
169
+ throw new Error(tooBigToBuffer(filePath, declared, this.maxBufferedBytes));
170
+ }
171
+ // No (or a lying) Content-Length: count as it arrives and stop at the limit.
172
+ const chunks = [];
173
+ let size = 0;
174
+ if (res.body) {
175
+ for await (const chunk of Readable.fromWeb(res.body)) {
176
+ size += chunk.length;
177
+ if (size > this.maxBufferedBytes)
178
+ throw new Error(tooBigToBuffer(filePath, size, this.maxBufferedBytes));
179
+ chunks.push(chunk);
180
+ }
181
+ }
182
+ return { bytes: Buffer.concat(chunks, size), etag: normalizeEtag(res.headers.get("ETag")) };
183
+ }
184
+ async download(filePath, destAbs) {
185
+ const res = await authorizedFetch(`${this.apiRoot}/files/content/${encodePath(filePath)}`, {}, this.fetchFn);
186
+ if (!res.ok)
187
+ throw new Error(`Downloading ${filePath} failed: HTTP ${res.status}`);
188
+ const { hash, size } = await streamToFile(res, destAbs);
189
+ return { etag: normalizeEtag(res.headers.get("ETag")), hash, size };
92
190
  }
93
191
  async put(filePath, body, opts) {
94
192
  const headers = { "Content-Type": "application/octet-stream" };
@@ -121,6 +219,11 @@ export class FilesClient {
121
219
  headers: { "Content-Type": "application/json" },
122
220
  body: JSON.stringify(payload),
123
221
  }, this.fetchFn);
222
+ if (res.status === 504) {
223
+ // Never retried: the engine may still be running the snippet.
224
+ const detail = (await errorDetail(res)) || RUN_LUA_TIMEOUT;
225
+ throw new EngineTimeoutError(`run-lua timed out: HTTP 504 - ${detail}`);
226
+ }
124
227
  if (!res.ok)
125
228
  throw new Error(await engineErrorMessage(res, "run-lua"));
126
229
  // Server envelope: {output: <text>} - the engine bridge's text
@@ -129,11 +232,24 @@ export class FilesClient {
129
232
  return body.output;
130
233
  }
131
234
  async engineErrors() {
132
- const res = await authorizedFetch(`${this.apiRoot}/engine/errors`, {}, this.fetchFn);
133
- if (!res.ok)
134
- throw new Error(await engineErrorMessage(res, "Fetching engine errors"));
135
- const body = (await res.json());
136
- return body.output;
235
+ // Reading the error buffer changes nothing, so a busy (504) or briefly
236
+ // unreachable (503) engine is worth another try.
237
+ for (let attempt = 0;; attempt++) {
238
+ const res = await authorizedFetch(`${this.apiRoot}/engine/errors`, {}, this.fetchFn);
239
+ if (res.ok) {
240
+ const body = (await res.json());
241
+ return body.output;
242
+ }
243
+ const retryable = res.status === 503 || res.status === 504;
244
+ const delay = READ_RETRY_DELAYS_MS[attempt];
245
+ if (retryable && delay !== undefined) {
246
+ await res.body?.cancel();
247
+ await this.sleep(delay);
248
+ continue;
249
+ }
250
+ const message = await engineErrorMessage(res, "Fetching engine errors");
251
+ throw res.status === 504 ? new EngineTimeoutError(message) : new Error(message);
252
+ }
137
253
  }
138
254
  async screenshot(opts = {}) {
139
255
  const body = {};
package/dist/index.js CHANGED
@@ -9,7 +9,7 @@ import { NotLoggedInError, clearCredentials, loadCredentials, pollForToken, save
9
9
  import { openInBrowser } from "./browser.js";
10
10
  import { detectPlatform, ensureClientInstalled, launchClient, seedNativeAuth } from "./client.js";
11
11
  import { apiBase, packageVersion } from "./config.js";
12
- import { FilesClient } from "./files.js";
12
+ import { ENGINE_TIMEOUT_EXIT_CODE, EngineTimeoutError, FilesClient } from "./files.js";
13
13
  import { runMcpServer } from "./mcp.js";
14
14
  import { pullCommand } from "./pull.js";
15
15
  import { NeedsChoiceError, formatProjectList, makeFramingOf, resolveImplicitTarget, resolveTarget, } from "./resolve.js";
@@ -42,9 +42,15 @@ program
42
42
  "install. The native client (`virtualmatter open`) is optional, for more memory",
43
43
  "and frame rate.",
44
44
  ].join("\n"));
45
- function fail(message) {
45
+ function fail(message, code = 1) {
46
46
  console.error(message);
47
- process.exit(1);
47
+ process.exit(code);
48
+ }
49
+ /** Exit with the failure's own code: an engine timeout is not a plain failure. */
50
+ function failWith(err) {
51
+ if (err instanceof EngineTimeoutError)
52
+ fail(err.message, ENGINE_TIMEOUT_EXIT_CODE);
53
+ fail(err instanceof Error ? err.message : String(err));
48
54
  }
49
55
  /** The device-code sign-in, shared by `login` and the automatic first-use sign-in. */
50
56
  async function signIn() {
@@ -76,10 +82,10 @@ async function run(fn) {
76
82
  return;
77
83
  }
78
84
  catch (again) {
79
- fail(again instanceof Error ? again.message : String(again));
85
+ failWith(again);
80
86
  }
81
87
  }
82
- fail(err instanceof Error ? err.message : String(err));
88
+ failWith(err);
83
89
  }
84
90
  }
85
91
  /** Resolve a session-scoped FilesClient from a synced dir's state file. */
@@ -115,6 +121,14 @@ function slugFor(name) {
115
121
  }
116
122
  function reportPull(result, dest, watchUrl) {
117
123
  console.log(`Pulled ${result.fileCount} files into ${dest}.`);
124
+ if (result.failed.length > 0) {
125
+ // Reported, not fatal: the folder and its state file are usable, and the
126
+ // first `sync` retries whatever is missing.
127
+ console.error(`${result.failed.length} file(s) could not be downloaded (\`virtualmatter sync\` will retry them):`);
128
+ for (const f of result.failed)
129
+ console.error(` ${f.path}: ${f.error}`);
130
+ process.exitCode = 1;
131
+ }
118
132
  const files = result.agentFiles;
119
133
  if (files && files.written.length > 0) {
120
134
  console.log(`Wrote ${files.written.join(", ")} so Claude Code, Codex, and Cursor find their way around.`);
@@ -296,7 +310,7 @@ program
296
310
  }));
297
311
  program
298
312
  .command("run-lua")
299
- .description("Execute Lua in the running engine")
313
+ .description("Execute Lua in the running engine. Exit code 3 means the engine did not answer in time and may still be running the code: check its effect before sending it again")
300
314
  .argument("[dir]", "a synced folder", ".")
301
315
  .requiredOption("--code <lua>", "Lua source to execute")
302
316
  .option("--target <target>", "server or client", "server")
package/dist/mcp.js CHANGED
@@ -195,7 +195,7 @@ export async function runMcpServer(dir, framingFlag) {
195
195
  let pulled = null;
196
196
  if (pull_to) {
197
197
  const result = await pullCommand(framingId, pull_to);
198
- pulled = { dir: result.dir, file_count: result.fileCount, agent_files: result.agentFiles };
198
+ pulled = { dir: result.dir, file_count: result.fileCount, failed: result.failed, agent_files: result.agentFiles };
199
199
  }
200
200
  return textResult(JSON.stringify({ ...projectSummary(project), selected: true, pulled }, null, 2));
201
201
  });
@@ -211,7 +211,7 @@ export async function runMcpServer(dir, framingFlag) {
211
211
  let pulled = null;
212
212
  if (pull_to) {
213
213
  const result = await pullCommand(resolved.framingId, pull_to);
214
- pulled = { dir: result.dir, file_count: result.fileCount, agent_files: result.agentFiles };
214
+ pulled = { dir: result.dir, file_count: result.fileCount, failed: result.failed, agent_files: result.agentFiles };
215
215
  }
216
216
  return textResult(JSON.stringify({
217
217
  framing_id: resolved.framingId,
@@ -249,7 +249,7 @@ export async function runMcpServer(dir, framingFlag) {
249
249
  return textResult(JSON.stringify({ ok: true, path: filePath, etag }));
250
250
  });
251
251
  server.registerTool("run_lua", {
252
- description: "Execute a Lua snippet in the running Virtual Matter engine and return its result. target \"server\" (default) runs in the server-side Lua state where game logic lives; \"client\" runs in a connected client (someone must have the world open) - that is where UI, input and the camera live. With several clients connected, pass client_id; list them with target server and code 'return Server:GetClients()'. Use this to inspect live state, call engine APIs, or nudge the scene without editing files.",
252
+ description: "Execute a Lua snippet in the running Virtual Matter engine and return its result. target \"server\" (default) runs in the server-side Lua state where game logic lives; \"client\" runs in a connected client (someone must have the world open) - that is where UI, input and the camera live. With several clients connected, pass client_id; list them with target server and code 'return Server:GetClients()'. Use this to inspect live state, call engine APIs, or nudge the scene without editing files. If it fails with \"run-lua timed out\", the engine may still be running the snippet: check its effect before sending code that changes the world again.",
253
253
  inputSchema: {
254
254
  code: z.string().describe("Lua source to execute"),
255
255
  target: z
package/dist/pull.js CHANGED
@@ -5,7 +5,7 @@ import { resolveEditSession, sessionBaseUrl } from "./api.js";
5
5
  import { composeAgentsMd, fetchAgentsMd, writeAgentFiles } from "./agentfiles.js";
6
6
  import { FilesClient } from "./files.js";
7
7
  import { isIgnoredPath } from "./ignore.js";
8
- import { recordLocal, saveState } from "./state.js";
8
+ import { adoptLocal, saveState } from "./state.js";
9
9
  export { fetchAgentsMd } from "./agentfiles.js";
10
10
  export { AGENTS_MD_FALLBACK } from "./agents-md.generated.js";
11
11
  /** Download workers per pull. The tree is ~1000 small files; sequential
@@ -52,6 +52,7 @@ export async function pullTree(client, dir, framingId) {
52
52
  const remote = (await client.list()).filter((f) => !isIgnoredPath(f.path));
53
53
  const state = { framing_id: framingId, etags: {}, local: {} };
54
54
  const progress = makeProgress(remote.length);
55
+ const failed = [];
55
56
  let next = 0;
56
57
  const worker = async () => {
57
58
  for (;;) {
@@ -59,20 +60,26 @@ export async function pullTree(client, dir, framingId) {
59
60
  if (index >= remote.length)
60
61
  return;
61
62
  const file = remote[index];
62
- const { bytes, etag } = await client.get(file.path);
63
- const absPath = path.join(dir, file.path);
64
- fs.mkdirSync(path.dirname(absPath), { recursive: true });
65
- fs.writeFileSync(absPath, bytes);
66
- state.etags[file.path] = etag ?? file.etag;
67
- // what the disk holds now, so the first sync can tell local edits apart
68
- recordLocal(state, dir, file.path, bytes);
63
+ try {
64
+ // streamed to disk: a multi-GB voxel file must not pass through a Buffer
65
+ const { etag, hash } = await client.download(file.path, path.join(dir, file.path));
66
+ state.etags[file.path] = etag ?? file.etag;
67
+ // what the disk holds now, so the first sync can tell local edits apart
68
+ adoptLocal(state, dir, file.path, hash);
69
+ }
70
+ catch (err) {
71
+ // One bad file must not cost the other thousand, nor the state file:
72
+ // without it the folder cannot sync at all. A file left out here has
73
+ // no etag, so the first `sync` pulls it again.
74
+ failed.push({ path: file.path, error: err instanceof Error ? err.message : String(err) });
75
+ }
69
76
  progress.tick(file.path);
70
77
  }
71
78
  };
72
79
  await Promise.all(Array.from({ length: Math.min(PULL_CONCURRENCY, remote.length) }, () => worker()));
73
80
  progress.done();
74
81
  saveState(dir, state);
75
- return { dir, framingId, fileCount: remote.length };
82
+ return { dir, framingId, fileCount: remote.length - failed.length, failed };
76
83
  }
77
84
  export async function pullCommand(framingId, dir) {
78
85
  console.log(`Resolving session for framing ${framingId} (this can cold-start one) ...`);
package/dist/state.js CHANGED
@@ -34,6 +34,27 @@ export function requireState(dir) {
34
34
  export function sha256Hex(bytes) {
35
35
  return crypto.createHash("sha256").update(bytes).digest("hex");
36
36
  }
37
+ /**
38
+ * sha256 of a file read in chunks. `readFileSync` cannot load a file past
39
+ * 2 GiB at all, and multi-GB voxel files are normal in a world.
40
+ */
41
+ export function hashFileSync(abs) {
42
+ const hasher = crypto.createHash("sha256");
43
+ const buf = Buffer.allocUnsafe(1024 * 1024);
44
+ const fd = fs.openSync(abs, "r");
45
+ try {
46
+ for (;;) {
47
+ const n = fs.readSync(fd, buf, 0, buf.length, null);
48
+ if (n === 0)
49
+ break;
50
+ hasher.update(buf.subarray(0, n));
51
+ }
52
+ }
53
+ finally {
54
+ fs.closeSync(fd);
55
+ }
56
+ return hasher.digest("hex");
57
+ }
37
58
  /**
38
59
  * Record a file whose content hash is already known to match its last sync,
39
60
  * from a stat alone. How a state file from before `local` records existed
@@ -69,7 +90,7 @@ export function localHash(state, dir, relPath) {
69
90
  const rec = state.local?.[relPath];
70
91
  if (rec && rec.size === st.size && rec.mtime_ms === st.mtimeMs)
71
92
  return rec.hash;
72
- return sha256Hex(fs.readFileSync(abs));
93
+ return hashFileSync(abs);
73
94
  }
74
95
  const SHA256_HEX = /^[0-9a-f]{64}$/;
75
96
  /**
package/dist/sync.js CHANGED
@@ -14,7 +14,7 @@ import { startLogSync } from "./agent-logs.js";
14
14
  */
15
15
  import fs from "node:fs";
16
16
  import path from "node:path";
17
- import { ConflictError } from "./files.js";
17
+ import { ConflictError, MAX_WRITE_BYTES } from "./files.js";
18
18
  import { isIgnoredPath } from "./ignore.js";
19
19
  import { loadState, adoptLocal, localHash, recordLocal, saveState, sha256Hex, syncedHash, } from "./state.js";
20
20
  const consoleLogger = {
@@ -34,6 +34,8 @@ export class SyncEngine {
34
34
  selfWrites = new Map();
35
35
  /** State changed without being written yet (see flush). */
36
36
  dirty = false;
37
+ /** relPath -> "size:mtime" of an oversized file already warned about, so the scan does not repeat it every few seconds. */
38
+ tooBigWarned = new Map();
37
39
  constructor(client, dir, state, opts = {}) {
38
40
  this.client = client;
39
41
  this.dir = dir;
@@ -102,13 +104,12 @@ export class SyncEngine {
102
104
  this.persist();
103
105
  }
104
106
  async pullFile(relPath, etag) {
105
- const { bytes, etag: gotEtag } = await this.client.get(relPath);
106
- const absPath = this.abs(relPath);
107
- fs.mkdirSync(path.dirname(absPath), { recursive: true });
108
107
  this.recordSelfWrite(relPath);
109
- fs.writeFileSync(absPath, bytes);
108
+ // streamed: voxel and terrain files run to several GB
109
+ const { etag: gotEtag, hash } = await this.client.download(relPath, this.abs(relPath));
110
+ this.recordSelfWrite(relPath); // the rename is the write the watcher sees; restart the grace from it
110
111
  this.state.etags[relPath] = gotEtag ?? etag;
111
- recordLocal(this.state, this.dir, relPath, bytes);
112
+ adoptLocal(this.state, this.dir, relPath, hash);
112
113
  this.persist();
113
114
  }
114
115
  /**
@@ -119,10 +120,8 @@ export class SyncEngine {
119
120
  */
120
121
  async writeConflict(relPath, remoteEtag) {
121
122
  try {
122
- const { bytes, etag } = await this.client.get(relPath);
123
123
  const conflictPath = this.abs(relPath) + CONFLICT_SUFFIX;
124
- fs.mkdirSync(path.dirname(conflictPath), { recursive: true });
125
- fs.writeFileSync(conflictPath, bytes);
124
+ const { etag } = await this.client.download(relPath, conflictPath);
126
125
  const e = etag ?? remoteEtag;
127
126
  if (e)
128
127
  this.state.etags[relPath] = e;
@@ -253,6 +252,8 @@ export class SyncEngine {
253
252
  }
254
253
  /** Push the local file. Returns false when there was nothing to send or it became a conflict. */
255
254
  async pushFile(relPath, opts) {
255
+ if (this.tooBigToPush(relPath))
256
+ return false;
256
257
  const body = fs.readFileSync(this.abs(relPath));
257
258
  const known = this.state.etags[relPath];
258
259
  // unchanged since the last sync: nothing to send (a touch, or the
@@ -276,6 +277,32 @@ export class SyncEngine {
276
277
  throw err;
277
278
  }
278
279
  }
280
+ /**
281
+ * Over the session API's write cap: the server would answer 413, so do not
282
+ * read the file at all (a multi-GB read would also exhaust memory). Warned
283
+ * once per version of the file. The file stays as it is on both sides.
284
+ */
285
+ tooBigToPush(relPath) {
286
+ let st;
287
+ try {
288
+ st = fs.statSync(this.abs(relPath));
289
+ }
290
+ catch {
291
+ return false;
292
+ }
293
+ if (st.size <= MAX_WRITE_BYTES) {
294
+ this.tooBigWarned.delete(relPath);
295
+ return false;
296
+ }
297
+ const key = `${st.size}:${st.mtimeMs}`;
298
+ if (this.tooBigWarned.get(relPath) !== key) {
299
+ this.tooBigWarned.set(relPath, key);
300
+ const mb = (n) => `${(n / (1024 * 1024)).toFixed(1)} MB`;
301
+ this.log.warn(`not pushing ${relPath} (${mb(st.size)}): the world accepts at most ${mb(MAX_WRITE_BYTES)} per file. ` +
302
+ `Edit large voxel/terrain data in the engine instead; it is left unchanged in the world.`);
303
+ }
304
+ return true;
305
+ }
279
306
  /** Watcher callback for a local add/change. */
280
307
  async handleLocalChange(relPath) {
281
308
  const norm = relPath.replace(/\\/g, "/");
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "virtualmatter",
3
- "version": "0.6.2",
3
+ "version": "0.6.4",
4
4
  "description": "CLI + MCP server for building with Virtual Matter - list and create worlds, sync their files, run Lua, capture screenshots, open the native client, and wire coding agents into a live session.",
5
5
  "license": "MIT",
6
6
  "type": "module",