@pascal-app/mcp 0.3.0 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +71 -0
- package/dist/bin/pascal-mcp.js +0 -0
- package/dist/bridge/scene-bridge.d.ts.map +1 -1
- package/dist/bridge/scene-bridge.js +6 -0
- package/dist/operations/scene-operations.js +1 -0
- package/dist/resources/constraints.js +1 -1
- package/dist/tools/construction-tools.d.ts +21 -21
- package/dist/tools/construction-tools.d.ts.map +1 -1
- package/dist/tools/construction-tools.js +45 -19
- package/dist/tools/create-level.d.ts +1 -1
- package/dist/tools/create-level.d.ts.map +1 -1
- package/dist/tools/create-level.js +5 -1
- package/dist/tools/create-wall.d.ts +2 -2
- package/dist/tools/create-wall.d.ts.map +1 -1
- package/dist/tools/create-wall.js +6 -2
- package/dist/tools/cut-opening.d.ts +3 -3
- package/dist/tools/cut-opening.d.ts.map +1 -1
- package/dist/tools/cut-opening.js +3 -2
- package/dist/tools/find-nodes.d.ts +2 -2
- package/dist/tools/find-nodes.js +1 -1
- package/dist/tools/measurement.d.ts +37 -0
- package/dist/tools/measurement.d.ts.map +1 -0
- package/dist/tools/measurement.js +94 -0
- package/dist/tools/photo-to-scene/photo-to-scene.d.ts +2 -2
- package/dist/tools/photo-to-scene/photo-to-scene.d.ts.map +1 -1
- package/dist/tools/photo-to-scene/photo-to-scene.js +9 -2
- package/dist/tools/place-item.d.ts +1 -1
- package/dist/tools/place-item.d.ts.map +1 -1
- package/dist/tools/place-item.js +2 -1
- package/dist/tools/room-tools.d.ts +8 -8
- package/dist/tools/room-tools.d.ts.map +1 -1
- package/dist/tools/room-tools.js +17 -7
- package/dist/tools/scene-query.js +2 -2
- package/examples/coordinate-conventions-demo.json +968 -0
- package/examples/coordinate-conventions-demo.md +68 -0
- package/examples/embed-in-agent.ts +82 -0
- package/examples/generate-apartment.md +242 -0
- package/examples/photo-to-scene.md +119 -0
- package/examples/renovate-from-photos.md +202 -0
- package/package.json +8 -6
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# Coordinate-conventions demo
|
|
2
|
+
|
|
3
|
+
Companion scene for the **Coordinate conventions** section of
|
|
4
|
+
[`../README.md`](../README.md). Stress-tests every claim that section
|
|
5
|
+
makes against real Pascal-generated geometry, built entirely through the
|
|
6
|
+
MCP API.
|
|
7
|
+
|
|
8
|
+
## Load it
|
|
9
|
+
|
|
10
|
+
```bash
|
|
11
|
+
pascal-mcp --stdio --scene examples/coordinate-conventions-demo.json
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
The scene fits in a 50 m × 50 m ground-plane footprint at `level: 0`.
|
|
15
|
+
|
|
16
|
+
## What's in the scene
|
|
17
|
+
|
|
18
|
+
A flat ground level with four demos, plus a labelled compass at the
|
|
19
|
+
origin so the world axes are unambiguous regardless of which viewport
|
|
20
|
+
you're in.
|
|
21
|
+
|
|
22
|
+
| Section | What | Why |
|
|
23
|
+
|---|---|---|
|
|
24
|
+
| Reference compass at the origin | 12 m `+X` and `+Z` axis bars with arrowhead tips, short `-X` / `-Z` stubs, an origin marker, plus zone labels naming each component of `[x, z]`. | Establishes which way `+X` and `+Z` actually point in world space, independent of viewport rotation. |
|
|
25
|
+
| **Demo A** at world `(18, 0)` | Axis-aligned 6 m × 4 m rectangle. | Baseline that matches the only example currently in `examples/generate-apartment.md` style (`[0,0] → [10,0]`). |
|
|
26
|
+
| **Demo B** at world `(18, 10)` | The proposed README example, **verbatim**: `polygon: [[0,0],[5.196,3.0],[3.196,6.464],[-2.0,3.464]]` at this offset. | Programmatically verified to be a 6 m × 4 m rectangle whose first edge is heading 30° CCW from `+X` (side lengths 6/4/6/4 m to 4 dp, `AB · AD = 0`, heading = 30.0007°). Confirms the README's worked example produces the geometry it claims. |
|
|
27
|
+
| **Demo C** at world `(0, 22)` and `(10, 22)` | An **L** authored on a north-up page (page-+x right, page-+y up) drawn at half scale and faded; alongside it, the same L pasted into `[x, z]` **uncorrected**, drawn at full scale and vivid. | Makes the external-coordinate gotcha visual: the author's page-+y direction lands on world +Z, which does not correspond to "screen-up" in any of Pascal's viewports. |
|
|
28
|
+
| **Demo D** at world `(22, 22)` and `(32, 22)` | Same pairing for the L with its second coordinate reflected (`z → 5 − z`) before paste. | Makes plain that z-reflection only "corrects" a true mirror; Pascal's viewports apply a *rotation*, so the reflection produces a mirror-image of Demo C rather than a page-correct L. |
|
|
29
|
+
| Takeaway band on the south edge | Short labels summarising the convention. | Self-documenting; readable in any viewport without an external README. |
|
|
30
|
+
|
|
31
|
+
## What it confirms
|
|
32
|
+
|
|
33
|
+
Open the scene in Pascal (or render it from the JSON) and you can read
|
|
34
|
+
each claim directly off the geometry:
|
|
35
|
+
|
|
36
|
+
1. **`[x, z] → world (x, 0, z)` is exact, no sign flip.** Demo A and
|
|
37
|
+
Demo B both sit flat on the floor at Y = 0; their polygon vertices
|
|
38
|
+
round-trip through `save_scene` / `get_scene` byte-identical to what
|
|
39
|
+
was authored (graph hash matches, see `validate_scene` output).
|
|
40
|
+
|
|
41
|
+
2. **Demo B is the rectangle the README says it is.** A simple analytic
|
|
42
|
+
check on its 4 vertices yields side lengths 6, 4, 6, 4 m (to 4 dp)
|
|
43
|
+
with the first edge at 30.0007° from +X and perpendicular adjacent
|
|
44
|
+
edges. No rendering required.
|
|
45
|
+
|
|
46
|
+
3. **External page coordinates *rotate* (not mirror) when pasted as
|
|
47
|
+
`[x, z]` and viewed in Pascal.** Inspecting Demo C in the 2-D plan
|
|
48
|
+
panel: the page-up L lands rotated 90° clockwise, so the author's
|
|
49
|
+
"stem-up, foot-bottom-right" reads as "stem-on-right, foot-along-top"
|
|
50
|
+
on screen. Inspecting Demo D right next to it: the z-reflected
|
|
51
|
+
variant lands as the *mirror* of Demo C — neither matches a
|
|
52
|
+
page-correct L. That demonstrates why "reflect across the axis" is
|
|
53
|
+
the wrong corrective for an issue that is, in this viewport, a
|
|
54
|
+
rotation.
|
|
55
|
+
|
|
56
|
+
4. **The 3-D "top-down" snap is offset 45° from world axes by default.**
|
|
57
|
+
Viewing the scene in 3-D and snapping to top-down from the default
|
|
58
|
+
iso camera shows the rectangular site polygon as a *diamond* (its
|
|
59
|
+
long edges are diagonals on screen, not axis-aligned). That's the
|
|
60
|
+
camera's "up" vector being inherited from the iso start; once you
|
|
61
|
+
orbit to a true axis-aligned top-down it goes away. The label
|
|
62
|
+
inviting reviewers to "verify against a guide image" exists for this
|
|
63
|
+
reason.
|
|
64
|
+
|
|
65
|
+
## Reproducibility
|
|
66
|
+
|
|
67
|
+
The canonical demo is the JSON file in this directory; load that file to compare
|
|
68
|
+
the geometry against the notes above.
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Programmatic `@pascal-app/mcp` usage.
|
|
3
|
+
*
|
|
4
|
+
* Runs a full MCP client/server pair over the in-memory transport inside a
|
|
5
|
+
* single Bun process. Useful for agent frameworks and tests that want to
|
|
6
|
+
* drive Pascal without spawning a subprocess.
|
|
7
|
+
*
|
|
8
|
+
* Compile with the package's `tsc --build`, or run directly with Bun:
|
|
9
|
+
*
|
|
10
|
+
* bun run packages/mcp/examples/embed-in-agent.ts
|
|
11
|
+
*/
|
|
12
|
+
|
|
13
|
+
import { Client } from '@modelcontextprotocol/sdk/client/index.js'
|
|
14
|
+
import { InMemoryTransport } from '@modelcontextprotocol/sdk/inMemory.js'
|
|
15
|
+
import { createPascalMcpServer, SceneBridge } from '@pascal-app/mcp'
|
|
16
|
+
|
|
17
|
+
async function main(): Promise<void> {
|
|
18
|
+
// 1. Spin up the headless bridge. `loadDefault()` seeds a Site → Building →
|
|
19
|
+
// Level stack so the client has something to query immediately.
|
|
20
|
+
const bridge = new SceneBridge()
|
|
21
|
+
bridge.loadDefault()
|
|
22
|
+
const server = createPascalMcpServer({ bridge })
|
|
23
|
+
|
|
24
|
+
// 2. Link the server to an in-memory client. Exactly the same API surface
|
|
25
|
+
// as the stdio / HTTP transports, but without any process boundary.
|
|
26
|
+
const [srvT, cliT] = InMemoryTransport.createLinkedPair()
|
|
27
|
+
const client = new Client({ name: 'my-agent', version: '0.1.0' })
|
|
28
|
+
await Promise.all([server.connect(srvT), client.connect(cliT)])
|
|
29
|
+
|
|
30
|
+
// 3. Discover available capabilities.
|
|
31
|
+
const tools = await client.listTools()
|
|
32
|
+
console.log(
|
|
33
|
+
'available tools:',
|
|
34
|
+
tools.tools.map((t) => t.name),
|
|
35
|
+
)
|
|
36
|
+
|
|
37
|
+
// 4. Inspect the current scene.
|
|
38
|
+
const scene = await client.callTool({ name: 'get_scene', arguments: {} })
|
|
39
|
+
console.log('scene snapshot:', JSON.stringify(scene, null, 2))
|
|
40
|
+
|
|
41
|
+
// 5. Find the default level, create a 5 m wall, and undo it.
|
|
42
|
+
const levels = await client.callTool({
|
|
43
|
+
name: 'find_nodes',
|
|
44
|
+
arguments: { type: 'level' },
|
|
45
|
+
})
|
|
46
|
+
const levelId = (levels.structuredContent as { nodes: Array<{ id: string }> }).nodes[0]?.id
|
|
47
|
+
|
|
48
|
+
if (levelId) {
|
|
49
|
+
const created = await client.callTool({
|
|
50
|
+
name: 'create_wall',
|
|
51
|
+
arguments: {
|
|
52
|
+
levelId,
|
|
53
|
+
start: [0, 0],
|
|
54
|
+
end: [5, 0],
|
|
55
|
+
thickness: 0.2,
|
|
56
|
+
height: 2.5,
|
|
57
|
+
},
|
|
58
|
+
})
|
|
59
|
+
console.log('created wall:', created.structuredContent)
|
|
60
|
+
|
|
61
|
+
const undone = await client.callTool({ name: 'undo', arguments: { steps: 1 } })
|
|
62
|
+
console.log('undone:', undone.structuredContent)
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
// 6. Validate and export.
|
|
66
|
+
const validation = await client.callTool({ name: 'validate_scene', arguments: {} })
|
|
67
|
+
console.log('validation:', validation.structuredContent)
|
|
68
|
+
|
|
69
|
+
const exported = await client.callTool({
|
|
70
|
+
name: 'export_json',
|
|
71
|
+
arguments: { pretty: true },
|
|
72
|
+
})
|
|
73
|
+
console.log('export size:', (exported.structuredContent as { json: string }).json.length)
|
|
74
|
+
|
|
75
|
+
await client.close()
|
|
76
|
+
await server.close()
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
main().catch((err) => {
|
|
80
|
+
console.error(err)
|
|
81
|
+
process.exit(1)
|
|
82
|
+
})
|
|
@@ -0,0 +1,242 @@
|
|
|
1
|
+
# Generate a 2-bed apartment from a brief
|
|
2
|
+
|
|
3
|
+
This example walks through a realistic session with an MCP host (Claude
|
|
4
|
+
Desktop, Claude Code, or Cursor) that has `pascal-mcp` configured. The agent
|
|
5
|
+
uses the `from_brief` prompt to turn a short brief into a concrete scene.
|
|
6
|
+
|
|
7
|
+
## The brief
|
|
8
|
+
|
|
9
|
+
> **User:** Claude, create a 2-bedroom 1-bath apartment in 80 m² in Spain.
|
|
10
|
+
|
|
11
|
+
The host UI lets the user select the **`from_brief`** prompt and fills in:
|
|
12
|
+
|
|
13
|
+
```text
|
|
14
|
+
brief: "2-bedroom 1-bath apartment in 80 m² in Spain, open-plan living /
|
|
15
|
+
kitchen, bathroom on the interior wall"
|
|
16
|
+
constraints: "Spanish building regulations; ceiling height 2.5 m"
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
## What the agent does
|
|
20
|
+
|
|
21
|
+
The prompt returns a system message instructing the agent to start from an
|
|
22
|
+
empty site, read the current scene, and emit incremental `apply_patch` calls.
|
|
23
|
+
The agent proceeds roughly like this:
|
|
24
|
+
|
|
25
|
+
### 1. Inspect the current scene
|
|
26
|
+
|
|
27
|
+
```jsonc
|
|
28
|
+
// tool: get_scene
|
|
29
|
+
{ "name": "get_scene", "arguments": {} }
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Response (trimmed):
|
|
33
|
+
|
|
34
|
+
```jsonc
|
|
35
|
+
{
|
|
36
|
+
"nodes": {
|
|
37
|
+
"site-1": { "type": "site", "id": "site-1", "children": [/* ... */] },
|
|
38
|
+
"building-1": { "type": "building", "id": "building-1", "parentId": "site-1" },
|
|
39
|
+
"level-1": { "type": "level", "id": "level-1", "parentId": "building-1",
|
|
40
|
+
"elevation": 0, "height": 2.5 }
|
|
41
|
+
},
|
|
42
|
+
"rootNodeIds": ["site-1"]
|
|
43
|
+
}
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
The default scene is a Site → Building → Level stack with no walls. The
|
|
47
|
+
agent decides to work on `level-1` and targets a 10 m × 8 m = 80 m² outline.
|
|
48
|
+
|
|
49
|
+
### 2. Create the perimeter walls
|
|
50
|
+
|
|
51
|
+
The agent chooses a rectangular outline with its origin at (0, 0):
|
|
52
|
+
|
|
53
|
+
```jsonc
|
|
54
|
+
// tool: apply_patch
|
|
55
|
+
{
|
|
56
|
+
"name": "apply_patch",
|
|
57
|
+
"arguments": {
|
|
58
|
+
"patches": [
|
|
59
|
+
{ "op": "create", "parentId": "level-1",
|
|
60
|
+
"node": { "type": "wall", "start": [0, 0], "end": [10, 0],
|
|
61
|
+
"thickness": 0.2, "height": 2.5 } },
|
|
62
|
+
{ "op": "create", "parentId": "level-1",
|
|
63
|
+
"node": { "type": "wall", "start": [10, 0], "end": [10, 8],
|
|
64
|
+
"thickness": 0.2, "height": 2.5 } },
|
|
65
|
+
{ "op": "create", "parentId": "level-1",
|
|
66
|
+
"node": { "type": "wall", "start": [10, 8], "end": [0, 8],
|
|
67
|
+
"thickness": 0.2, "height": 2.5 } },
|
|
68
|
+
{ "op": "create", "parentId": "level-1",
|
|
69
|
+
"node": { "type": "wall", "start": [0, 8], "end": [0, 0],
|
|
70
|
+
"thickness": 0.2, "height": 2.5 } }
|
|
71
|
+
]
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Response:
|
|
77
|
+
|
|
78
|
+
```jsonc
|
|
79
|
+
{ "applied": 4 }
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
### 3. Create interior partitions
|
|
83
|
+
|
|
84
|
+
Two bedrooms on the east side, bathroom on the interior wall, open-plan
|
|
85
|
+
living / kitchen on the west.
|
|
86
|
+
|
|
87
|
+
```jsonc
|
|
88
|
+
// tool: apply_patch
|
|
89
|
+
{
|
|
90
|
+
"name": "apply_patch",
|
|
91
|
+
"arguments": {
|
|
92
|
+
"patches": [
|
|
93
|
+
{ "op": "create", "parentId": "level-1",
|
|
94
|
+
"node": { "type": "wall", "start": [5.5, 0], "end": [5.5, 8],
|
|
95
|
+
"thickness": 0.15, "height": 2.5 } },
|
|
96
|
+
{ "op": "create", "parentId": "level-1",
|
|
97
|
+
"node": { "type": "wall", "start": [5.5, 4], "end": [10, 4],
|
|
98
|
+
"thickness": 0.15, "height": 2.5 } },
|
|
99
|
+
{ "op": "create", "parentId": "level-1",
|
|
100
|
+
"node": { "type": "wall", "start": [5.5, 5.5], "end": [8, 5.5],
|
|
101
|
+
"thickness": 0.15, "height": 2.5 } },
|
|
102
|
+
{ "op": "create", "parentId": "level-1",
|
|
103
|
+
"node": { "type": "wall", "start": [8, 4], "end": [8, 5.5],
|
|
104
|
+
"thickness": 0.15, "height": 2.5 } }
|
|
105
|
+
]
|
|
106
|
+
}
|
|
107
|
+
}
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
### 4. Define zones
|
|
111
|
+
|
|
112
|
+
The agent declares the rooms so later queries and item placement can target
|
|
113
|
+
them by name:
|
|
114
|
+
|
|
115
|
+
```jsonc
|
|
116
|
+
// tool: set_zone (called once per zone)
|
|
117
|
+
{
|
|
118
|
+
"name": "set_zone",
|
|
119
|
+
"arguments": {
|
|
120
|
+
"levelId": "level-1",
|
|
121
|
+
"label": "Living / Kitchen",
|
|
122
|
+
"polygon": [[0, 0], [5.5, 0], [5.5, 8], [0, 8]]
|
|
123
|
+
}
|
|
124
|
+
}
|
|
125
|
+
// → { "zoneId": "zone-living" }
|
|
126
|
+
|
|
127
|
+
{
|
|
128
|
+
"name": "set_zone",
|
|
129
|
+
"arguments": {
|
|
130
|
+
"levelId": "level-1",
|
|
131
|
+
"label": "Bedroom 1",
|
|
132
|
+
"polygon": [[5.5, 0], [10, 0], [10, 4], [5.5, 4]]
|
|
133
|
+
}
|
|
134
|
+
}
|
|
135
|
+
// → { "zoneId": "zone-bed1" }
|
|
136
|
+
|
|
137
|
+
{
|
|
138
|
+
"name": "set_zone",
|
|
139
|
+
"arguments": {
|
|
140
|
+
"levelId": "level-1",
|
|
141
|
+
"label": "Bedroom 2",
|
|
142
|
+
"polygon": [[5.5, 5.5], [10, 5.5], [10, 8], [5.5, 8]]
|
|
143
|
+
}
|
|
144
|
+
}
|
|
145
|
+
// → { "zoneId": "zone-bed2" }
|
|
146
|
+
|
|
147
|
+
{
|
|
148
|
+
"name": "set_zone",
|
|
149
|
+
"arguments": {
|
|
150
|
+
"levelId": "level-1",
|
|
151
|
+
"label": "Bathroom",
|
|
152
|
+
"polygon": [[5.5, 4], [8, 4], [8, 5.5], [5.5, 5.5]]
|
|
153
|
+
}
|
|
154
|
+
}
|
|
155
|
+
// → { "zoneId": "zone-bath" }
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
### 5. Cut doors and windows
|
|
159
|
+
|
|
160
|
+
The agent uses `cut_opening` to add entry doors on each interior partition
|
|
161
|
+
and windows on the south and east façades:
|
|
162
|
+
|
|
163
|
+
```jsonc
|
|
164
|
+
// tool: cut_opening (called once per opening)
|
|
165
|
+
{
|
|
166
|
+
"name": "cut_opening",
|
|
167
|
+
"arguments": {
|
|
168
|
+
"wallId": "wall-south", // perimeter wall [0,0] → [10,0]
|
|
169
|
+
"type": "window",
|
|
170
|
+
"position": 0.25, // 25% along centerline
|
|
171
|
+
"width": 1.2,
|
|
172
|
+
"height": 1.2
|
|
173
|
+
}
|
|
174
|
+
}
|
|
175
|
+
// → { "openingId": "window-south-1" }
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
```jsonc
|
|
179
|
+
{
|
|
180
|
+
"name": "cut_opening",
|
|
181
|
+
"arguments": {
|
|
182
|
+
"wallId": "wall-bed1", // partition wall to Bedroom 1
|
|
183
|
+
"type": "door",
|
|
184
|
+
"position": 0.4,
|
|
185
|
+
"width": 0.9,
|
|
186
|
+
"height": 2.1
|
|
187
|
+
}
|
|
188
|
+
}
|
|
189
|
+
// → { "openingId": "door-bed1" }
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
The agent repeats this for Bedroom 2's door, the bathroom door, and two
|
|
193
|
+
more windows on the east façade.
|
|
194
|
+
|
|
195
|
+
### 6. Validate and report
|
|
196
|
+
|
|
197
|
+
```jsonc
|
|
198
|
+
// tool: validate_scene
|
|
199
|
+
{ "name": "validate_scene", "arguments": {} }
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
Response:
|
|
203
|
+
|
|
204
|
+
```jsonc
|
|
205
|
+
{ "valid": true, "errors": [] }
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
The agent then reads the scene summary for its response to the user:
|
|
209
|
+
|
|
210
|
+
```jsonc
|
|
211
|
+
// resource: pascal://scene/current/summary
|
|
212
|
+
{ "uri": "pascal://scene/current/summary" }
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
The host displays the returned Markdown: 1 site, 1 building, 1 level, 8
|
|
216
|
+
walls, 4 zones, 3 doors, 3 windows; usable area ~78 m²; perimeter ~36 m.
|
|
217
|
+
|
|
218
|
+
### 7. Iterate
|
|
219
|
+
|
|
220
|
+
The user follows up:
|
|
221
|
+
|
|
222
|
+
> **User:** Swap the bathroom and bedroom 2 — I want the bathroom near the
|
|
223
|
+
> entrance.
|
|
224
|
+
|
|
225
|
+
The agent loads the `iterate_on_feedback` prompt and issues a single
|
|
226
|
+
`apply_patch` that updates the polygon of `zone-bath` and `zone-bed2` and
|
|
227
|
+
moves the corresponding partition walls. Because mutation goes through the
|
|
228
|
+
Zustand store, the user can `undo` the change if they dislike it:
|
|
229
|
+
|
|
230
|
+
```jsonc
|
|
231
|
+
{ "name": "undo", "arguments": { "steps": 1 } }
|
|
232
|
+
// → { "undone": 1 }
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
## Takeaways
|
|
236
|
+
|
|
237
|
+
- Mutations batch inside a single `apply_patch` so that `undo` rolls back
|
|
238
|
+
the whole logical change.
|
|
239
|
+
- Zones are not walls — they're polygon annotations that make later queries
|
|
240
|
+
(`find_nodes({ zoneId })`) and planning steps much easier for the agent.
|
|
241
|
+
- The agent never needs to speak to `@pascal-app/viewer`: everything the
|
|
242
|
+
host sees flows through tools + resources + prompts.
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
# Floor-plan photo to Pascal scene
|
|
2
|
+
|
|
3
|
+
The `photo_to_scene` orchestrator takes a single floor-plan photo and
|
|
4
|
+
returns a saved, navigable Pascal scene. It chains vision (via MCP
|
|
5
|
+
sampling) → scene build → save in one call, so an agent doesn't have to
|
|
6
|
+
stitch three tools together manually.
|
|
7
|
+
|
|
8
|
+
> **Note:** `photo_to_scene` uses MCP sampling to call the host's model.
|
|
9
|
+
> Hosts that do not advertise `sampling` capability will receive a
|
|
10
|
+
> structured `sampling_unavailable` error; fall back to the text-only
|
|
11
|
+
> `from_brief` prompt in that case.
|
|
12
|
+
|
|
13
|
+
## The brief
|
|
14
|
+
|
|
15
|
+
A user drops a photo of a hand-drawn floor plan into the chat and types:
|
|
16
|
+
|
|
17
|
+
> **User:** here's a floor plan photo, turn it into a Pascal scene.
|
|
18
|
+
|
|
19
|
+
## The tool call
|
|
20
|
+
|
|
21
|
+
The agent reads the attachment as a data URI and issues a single tool call:
|
|
22
|
+
|
|
23
|
+
```jsonc
|
|
24
|
+
// tool: photo_to_scene
|
|
25
|
+
{
|
|
26
|
+
"name": "photo_to_scene",
|
|
27
|
+
"arguments": {
|
|
28
|
+
"image": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAYABgAAD...",
|
|
29
|
+
"scaleHint": "1 cm = 1 m, approx 20 m²",
|
|
30
|
+
"name": "Weekend flat"
|
|
31
|
+
}
|
|
32
|
+
}
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Optional knobs:
|
|
36
|
+
|
|
37
|
+
- `save` (default `true`) — if `false`, the response includes `graph`
|
|
38
|
+
inline instead of persisting to the `SceneStore`.
|
|
39
|
+
- `defaultWallThickness` (default `0.2` m) — used when the vision model
|
|
40
|
+
doesn't propose a per-wall thickness.
|
|
41
|
+
- `defaultWallHeight` (default `2.6` m) — applied to every generated wall
|
|
42
|
+
since the vision schema only captures 2D geometry.
|
|
43
|
+
|
|
44
|
+
## What happens under the hood
|
|
45
|
+
|
|
46
|
+
1. The orchestrator issues an MCP sampling request to the host with the
|
|
47
|
+
image and a structured JSON-only system prompt, mirroring
|
|
48
|
+
`analyze_floorplan_image`. The host's model returns walls, rooms, and
|
|
49
|
+
approximate dimensions as JSON.
|
|
50
|
+
2. The reply is validated against a strict Zod schema. Unparseable or
|
|
51
|
+
schema-failing responses surface as `sampling_response_unparseable` /
|
|
52
|
+
`sampling_response_invalid` MCP errors.
|
|
53
|
+
3. A fresh `SceneGraph` is built using the core schema factories: a
|
|
54
|
+
`site` → `building` → `level 0` skeleton, then one `WallNode` per
|
|
55
|
+
vision wall and one `ZoneNode` per vision room. Each node is
|
|
56
|
+
re-parsed with `AnyNode.safeParse`; invalid ones are dropped with a
|
|
57
|
+
warning appended to `notes`.
|
|
58
|
+
4. `bridge.setScene(...)` swaps the live scene so any follow-up MCP call
|
|
59
|
+
(`find_nodes`, `measure`, `apply_patch`, ...) operates on the new
|
|
60
|
+
geometry.
|
|
61
|
+
5. If `save: true`, the graph is persisted via `SceneStore.save` and the
|
|
62
|
+
response carries `sceneId` + `url: /scene/<id>`.
|
|
63
|
+
|
|
64
|
+
## The response
|
|
65
|
+
|
|
66
|
+
```jsonc
|
|
67
|
+
{
|
|
68
|
+
"sceneId": "scene_01hx8a...",
|
|
69
|
+
"url": "/scene/scene_01hx8a...",
|
|
70
|
+
"walls": 4,
|
|
71
|
+
"rooms": 1,
|
|
72
|
+
"confidence": 0.82
|
|
73
|
+
}
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
When `save: false` instead:
|
|
77
|
+
|
|
78
|
+
```jsonc
|
|
79
|
+
{
|
|
80
|
+
"walls": 4,
|
|
81
|
+
"rooms": 1,
|
|
82
|
+
"confidence": 0.82,
|
|
83
|
+
"graph": {
|
|
84
|
+
"nodes": { /* flat id → node dict */ },
|
|
85
|
+
"rootNodeIds": ["site_..."],
|
|
86
|
+
"collections": {}
|
|
87
|
+
}
|
|
88
|
+
}
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
If any wall or room failed schema validation, the response includes a
|
|
92
|
+
`notes` string summarising what was dropped.
|
|
93
|
+
|
|
94
|
+
## Opening the scene
|
|
95
|
+
|
|
96
|
+
The user follows `url` in their browser:
|
|
97
|
+
|
|
98
|
+
```
|
|
99
|
+
https://your-pascal-host/scene/scene_01hx8a...
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
...and lands in the editor with the new scene loaded, camera auto-framed
|
|
103
|
+
on the building footprint.
|
|
104
|
+
|
|
105
|
+
## Follow-up prompts
|
|
106
|
+
|
|
107
|
+
Because the bridge now holds the new scene, subsequent agent turns can
|
|
108
|
+
operate on it without reloading:
|
|
109
|
+
|
|
110
|
+
> **User:** add a door on the south wall between Living and Kitchen.
|
|
111
|
+
|
|
112
|
+
The agent calls `find_nodes({ type: "wall" })`, picks the appropriate
|
|
113
|
+
wall, and issues `cut_opening` — no extra wiring needed.
|
|
114
|
+
|
|
115
|
+
## Takeaways
|
|
116
|
+
|
|
117
|
+
- `photo_to_scene` is a one-shot primitive: one call, one scene.
|
|
118
|
+
- Vision confidence is surfaced so the agent can warn the user.
|
|
119
|
+
- v0.1 covers walls + zones; doors, windows, items are follow-up tools.
|