@fugood/bricks-project 2.25.0-beta.4 → 2.25.0-beta.40
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/compile/action-name-map.ts +63 -0
- package/compile/index.ts +208 -19
- package/package.json +4 -3
- package/package.json.bak +4 -3
- package/skills/bricks-ctor/SKILL.md +21 -17
- package/skills/bricks-ctor/{rules → references}/animation.md +3 -2
- package/skills/bricks-ctor/{rules → references}/architecture-patterns.md +12 -0
- package/skills/bricks-ctor/{rules → references}/automations.md +11 -0
- package/skills/bricks-ctor/references/buttress.md +245 -0
- package/skills/bricks-ctor/{rules → references}/data-calculation.md +5 -5
- package/skills/bricks-ctor/references/simulator.md +115 -0
- package/skills/bricks-ctor/references/verification-toolchain.md +179 -0
- package/skills/bricks-design/SKILL.md +160 -45
- package/skills/bricks-design/references/architecture-truths.md +132 -0
- package/skills/bricks-design/references/avoiding-complexity.md +91 -0
- package/skills/bricks-design/references/design-critique.md +195 -0
- package/skills/bricks-design/references/design-languages.md +265 -0
- package/skills/bricks-design/references/performance.md +116 -0
- package/skills/bricks-design/references/presentation-and-slideshow.md +137 -0
- package/skills/bricks-design/references/translating-inputs.md +152 -0
- package/skills/bricks-design/references/variations-and-tweaks.md +124 -0
- package/skills/bricks-design/references/when-the-brief-is-branded.md +284 -0
- package/skills/bricks-design/references/when-the-brief-is-vague.md +85 -0
- package/skills/bricks-design/references/workflow.md +134 -0
- package/skills/bricks-ux/SKILL.md +120 -0
- package/skills/bricks-ux/references/accessibility.md +162 -0
- package/skills/bricks-ux/references/flow-states.md +175 -0
- package/skills/bricks-ux/references/interaction-archetypes.md +189 -0
- package/skills/bricks-ux/references/monitoring-screens.md +153 -0
- package/skills/bricks-ux/references/pressable-composition.md +126 -0
- package/skills/bricks-ux/references/user-journey.md +168 -0
- package/skills/bricks-ux/references/ux-critique.md +256 -0
- package/tools/_git-author.ts +10 -2
- package/tools/_last-pushed-commit.ts +28 -0
- package/tools/_shell.ts +8 -1
- package/tools/deploy.ts +15 -0
- package/tools/mcp-tools/compile.ts +19 -9
- package/tools/mcp-tools/media.ts +4 -1
- package/tools/pull.ts +91 -16
- package/tools/push-config.ts +118 -0
- package/tools/{preview-main.mjs → simulator-main.mjs} +207 -12
- package/tools/simulator-preload.cjs +16 -0
- package/tools/{preview.ts → simulator.ts} +4 -4
- package/types/{animation.ts → animation.d.ts} +24 -8
- package/types/{automation.ts → automation.d.ts} +16 -20
- package/types/{brick-base.ts → brick-base.d.ts} +1 -1
- package/types/bricks/{Camera.ts → Camera.d.ts} +8 -8
- package/types/bricks/{Chart.ts → Chart.d.ts} +4 -4
- package/types/bricks/{GenerativeMedia.ts → GenerativeMedia.d.ts} +15 -15
- package/types/bricks/{Icon.ts → Icon.d.ts} +7 -7
- package/types/bricks/{Image.ts → Image.d.ts} +21 -9
- package/types/bricks/{Items.ts → Items.d.ts} +7 -7
- package/types/bricks/{Lottie.ts → Lottie.d.ts} +10 -10
- package/types/bricks/{Maps.ts → Maps.d.ts} +11 -11
- package/types/bricks/{QrCode.ts → QrCode.d.ts} +7 -7
- package/types/bricks/{Rect.ts → Rect.d.ts} +7 -7
- package/types/bricks/{RichText.ts → RichText.d.ts} +12 -9
- package/types/bricks/{Rive.ts → Rive.d.ts} +9 -9
- package/types/bricks/Scene3D.d.ts +596 -0
- package/types/bricks/{Sketch.ts → Sketch.d.ts} +6 -6
- package/types/bricks/{Slideshow.ts → Slideshow.d.ts} +7 -7
- package/types/bricks/{Svg.ts → Svg.d.ts} +7 -7
- package/types/bricks/{Text.ts → Text.d.ts} +9 -9
- package/types/bricks/{TextInput.ts → TextInput.d.ts} +10 -10
- package/types/bricks/{Video.ts → Video.d.ts} +12 -12
- package/types/bricks/{VideoStreaming.ts → VideoStreaming.d.ts} +10 -10
- package/types/bricks/{WebRtcStream.ts → WebRtcStream.d.ts} +1 -1
- package/types/bricks/{WebView.ts → WebView.d.ts} +4 -4
- package/types/bricks/{index.ts → index.d.ts} +1 -0
- package/types/{common.ts → common.d.ts} +3 -6
- package/types/data-calc-command/base.d.ts +57 -0
- package/types/data-calc-command/collection.d.ts +418 -0
- package/types/data-calc-command/color.d.ts +432 -0
- package/types/data-calc-command/constant.d.ts +50 -0
- package/types/data-calc-command/datetime.d.ts +147 -0
- package/types/data-calc-command/file.d.ts +129 -0
- package/types/data-calc-command/index.d.ts +13 -0
- package/types/data-calc-command/iteratee.d.ts +23 -0
- package/types/data-calc-command/logictype.d.ts +190 -0
- package/types/data-calc-command/math.d.ts +275 -0
- package/types/data-calc-command/object.d.ts +119 -0
- package/types/data-calc-command/sandbox.d.ts +58 -0
- package/types/data-calc-command/string.d.ts +407 -0
- package/types/{data-calc.ts → data-calc.d.ts} +1 -0
- package/types/{data.ts → data.d.ts} +4 -2
- package/types/generators/{Assistant.ts → Assistant.d.ts} +19 -0
- package/types/generators/{LlmGgml.ts → LlmGgml.d.ts} +43 -1
- package/types/generators/{LlmMlx.ts → LlmMlx.d.ts} +1 -0
- package/types/generators/{RerankerGgml.ts → RerankerGgml.d.ts} +5 -1
- package/types/generators/{SoundRecorder.ts → SoundRecorder.d.ts} +10 -1
- package/types/generators/{SpeechToTextGgml.ts → SpeechToTextGgml.d.ts} +6 -1
- package/types/generators/{SttAppleBuiltin.ts → SttAppleBuiltin.d.ts} +27 -4
- package/types/generators/{ThermalPrinter.ts → ThermalPrinter.d.ts} +9 -7
- package/types/generators/{VadGgml.ts → VadGgml.d.ts} +12 -2
- package/types/{subspace.ts → subspace.d.ts} +1 -1
- package/utils/data.ts +2 -2
- package/utils/event-props.ts +17 -0
- package/utils/id.ts +78 -27
- package/skills/bricks-ctor/rules/buttress.md +0 -156
- package/skills/bricks-design/LICENSE.txt +0 -180
- package/types/data-calc-command.ts +0 -7005
- /package/skills/bricks-ctor/{rules → references}/local-sync.md +0 -0
- /package/skills/bricks-ctor/{rules → references}/media-flow.md +0 -0
- /package/skills/bricks-ctor/{rules → references}/remote-data-bank.md +0 -0
- /package/skills/bricks-ctor/{rules → references}/standby-transition.md +0 -0
- /package/types/{canvas.ts → canvas.d.ts} +0 -0
- /package/types/{data-calc-script.ts → data-calc-script.d.ts} +0 -0
- /package/types/generators/{AlarmClock.ts → AlarmClock.d.ts} +0 -0
- /package/types/generators/{BleCentral.ts → BleCentral.d.ts} +0 -0
- /package/types/generators/{BlePeripheral.ts → BlePeripheral.d.ts} +0 -0
- /package/types/generators/{CanvasMap.ts → CanvasMap.d.ts} +0 -0
- /package/types/generators/{CastlesPay.ts → CastlesPay.d.ts} +0 -0
- /package/types/generators/{DataBank.ts → DataBank.d.ts} +0 -0
- /package/types/generators/{File.ts → File.d.ts} +0 -0
- /package/types/generators/{GraphQl.ts → GraphQl.d.ts} +0 -0
- /package/types/generators/{Http.ts → Http.d.ts} +0 -0
- /package/types/generators/{HttpServer.ts → HttpServer.d.ts} +0 -0
- /package/types/generators/{Information.ts → Information.d.ts} +0 -0
- /package/types/generators/{Intent.ts → Intent.d.ts} +0 -0
- /package/types/generators/{Iterator.ts → Iterator.d.ts} +0 -0
- /package/types/generators/{Keyboard.ts → Keyboard.d.ts} +0 -0
- /package/types/generators/{LlmAnthropicCompat.ts → LlmAnthropicCompat.d.ts} +0 -0
- /package/types/generators/{LlmAppleBuiltin.ts → LlmAppleBuiltin.d.ts} +0 -0
- /package/types/generators/{LlmMediaTekNeuroPilot.ts → LlmMediaTekNeuroPilot.d.ts} +0 -0
- /package/types/generators/{LlmOnnx.ts → LlmOnnx.d.ts} +0 -0
- /package/types/generators/{LlmOpenAiCompat.ts → LlmOpenAiCompat.d.ts} +0 -0
- /package/types/generators/{LlmQualcommAiEngine.ts → LlmQualcommAiEngine.d.ts} +0 -0
- /package/types/generators/{Mcp.ts → Mcp.d.ts} +0 -0
- /package/types/generators/{McpServer.ts → McpServer.d.ts} +0 -0
- /package/types/generators/{MediaFlow.ts → MediaFlow.d.ts} +0 -0
- /package/types/generators/{MqttBroker.ts → MqttBroker.d.ts} +0 -0
- /package/types/generators/{MqttClient.ts → MqttClient.d.ts} +0 -0
- /package/types/generators/{Question.ts → Question.d.ts} +0 -0
- /package/types/generators/{RealtimeTranscription.ts → RealtimeTranscription.d.ts} +0 -0
- /package/types/generators/{SerialPort.ts → SerialPort.d.ts} +0 -0
- /package/types/generators/{SoundPlayer.ts → SoundPlayer.d.ts} +0 -0
- /package/types/generators/{SpeechToTextOnnx.ts → SpeechToTextOnnx.d.ts} +0 -0
- /package/types/generators/{SpeechToTextPlatform.ts → SpeechToTextPlatform.d.ts} +0 -0
- /package/types/generators/{SqLite.ts → SqLite.d.ts} +0 -0
- /package/types/generators/{Step.ts → Step.d.ts} +0 -0
- /package/types/generators/{Tcp.ts → Tcp.d.ts} +0 -0
- /package/types/generators/{TcpServer.ts → TcpServer.d.ts} +0 -0
- /package/types/generators/{TextToSpeechAppleBuiltin.ts → TextToSpeechAppleBuiltin.d.ts} +0 -0
- /package/types/generators/{TextToSpeechGgml.ts → TextToSpeechGgml.d.ts} +0 -0
- /package/types/generators/{TextToSpeechOnnx.ts → TextToSpeechOnnx.d.ts} +0 -0
- /package/types/generators/{TextToSpeechOpenAiLike.ts → TextToSpeechOpenAiLike.d.ts} +0 -0
- /package/types/generators/{Tick.ts → Tick.d.ts} +0 -0
- /package/types/generators/{Udp.ts → Udp.d.ts} +0 -0
- /package/types/generators/{VadOnnx.ts → VadOnnx.d.ts} +0 -0
- /package/types/generators/{VadTraditional.ts → VadTraditional.d.ts} +0 -0
- /package/types/generators/{VectorStore.ts → VectorStore.d.ts} +0 -0
- /package/types/generators/{Watchdog.ts → Watchdog.d.ts} +0 -0
- /package/types/generators/{WebCrawler.ts → WebCrawler.d.ts} +0 -0
- /package/types/generators/{WebRtc.ts → WebRtc.d.ts} +0 -0
- /package/types/generators/{WebSocket.ts → WebSocket.d.ts} +0 -0
- /package/types/generators/{index.ts → index.d.ts} +0 -0
- /package/types/{index.ts → index.d.ts} +0 -0
- /package/types/{switch.ts → switch.d.ts} +0 -0
- /package/types/{system.ts → system.d.ts} +0 -0
|
@@ -0,0 +1,245 @@
|
|
|
1
|
+
# Buttress (Remote Inference)
|
|
2
|
+
|
|
3
|
+
Backend system for offloading compute-intensive AI generator tasks from BRICKS devices to more powerful machines.
|
|
4
|
+
|
|
5
|
+
## Purpose
|
|
6
|
+
|
|
7
|
+
When mobile devices or embedded systems lack hardware for local AI inference (LLM, speech-to-text), Buttress transparently delegates work to a server with appropriate resources (GPU).
|
|
8
|
+
|
|
9
|
+
## How It Works
|
|
10
|
+
|
|
11
|
+
1. **Capability Exchange**: Client and server share hardware capabilities
|
|
12
|
+
2. **Strategy Selection**: System decides local vs. remote execution
|
|
13
|
+
3. **Transparent Offloading**: Generator operates same way, execution happens remotely
|
|
14
|
+
|
|
15
|
+
## Supported Generators
|
|
16
|
+
|
|
17
|
+
- LLM (GGML) (LlmMlx.ts) - Local Large Language Model inference with GGML
|
|
18
|
+
- LLM (MLX) (LlmGgml.ts) - Local Large Language Model inference with MLX
|
|
19
|
+
- Speech-to-Text (GGML) (SpeechToTextGgml.ts) - Local Speech-to-Text inference with GGML
|
|
20
|
+
|
|
21
|
+
## Client Configuration
|
|
22
|
+
|
|
23
|
+
In generator properties, configure `buttressConnectionSettings`:
|
|
24
|
+
|
|
25
|
+
| Setting | Description |
|
|
26
|
+
|---------|-------------|
|
|
27
|
+
| `enabled` | Toggle Buttress offloading |
|
|
28
|
+
| `autoDiscoverType` | `auto` (LAN auto-discovery, recommended) or `manual` (typed URL) |
|
|
29
|
+
| `url` | Server URL (`manual` mode only — leave empty for `auto`) |
|
|
30
|
+
| `fallbackType` | If Buttress is unreachable: `use-local` runs locally, `no-op` blocks the load |
|
|
31
|
+
| `strategy` | Execution preference (see below) |
|
|
32
|
+
|
|
33
|
+
The launcher provides the access token automatically — there is **no user-facing access-token setting**. In `auto` mode the launcher discovers servers bound to the device's workspace and uses its workspace-scoped JWT; in `manual` mode it sends the same JWT only when the typed URL belongs to a server bound to the same workspace (verified against the server's `/buttress/info`).
|
|
34
|
+
|
|
35
|
+
### `autoDiscoverType` options
|
|
36
|
+
|
|
37
|
+
| Mode | When to use | What the device does |
|
|
38
|
+
|------|-------------|----------------------|
|
|
39
|
+
| `auto` | LAN deployment with a workspace-bound buttress-server | Listens for UDP announcements and picks the strongest server bound to its workspace; reconnects automatically when the workspace flips |
|
|
40
|
+
| `manual` | Internet-reachable server, mixed networks, or a public/unbound server | Connects to the typed `url` directly; sends a token only when the server is bound to the same workspace |
|
|
41
|
+
|
|
42
|
+
### Strategy options
|
|
43
|
+
|
|
44
|
+
| Strategy | Description |
|
|
45
|
+
|----------|-------------|
|
|
46
|
+
| `prefer-local` | Use local if capable, fallback to Buttress |
|
|
47
|
+
| `prefer-buttress` | Use Buttress if available, fallback to local |
|
|
48
|
+
| `prefer-best` | Auto-select based on capability comparison |
|
|
49
|
+
|
|
50
|
+
## Generator Configuration Examples
|
|
51
|
+
|
|
52
|
+
### Auto-discovery (recommended)
|
|
53
|
+
|
|
54
|
+
Workspace-bound launcher + workspace-bound buttress-server on the same LAN. No URL needed — discovery picks the highest-scoring server that runs the requested backend.
|
|
55
|
+
|
|
56
|
+
```typescript
|
|
57
|
+
import { makeId } from 'bricks-ctor'
|
|
58
|
+
|
|
59
|
+
const llmGenerator: GeneratorLLM = {
|
|
60
|
+
__typename: 'Generator',
|
|
61
|
+
templateKey: 'GENERATOR_LLM',
|
|
62
|
+
id: makeId('generator'),
|
|
63
|
+
title: 'Chat LLM',
|
|
64
|
+
description: '',
|
|
65
|
+
property: {
|
|
66
|
+
modelUrl: 'https://huggingface.co/ggml-org/gemma-3-12b-it-qat-GGUF/resolve/main/gemma-3-12b-it-qat-q4_0.gguf',
|
|
67
|
+
contextSize: 8192,
|
|
68
|
+
buttressConnectionSettings: {
|
|
69
|
+
enabled: true,
|
|
70
|
+
autoDiscoverType: 'auto',
|
|
71
|
+
fallbackType: 'use-local',
|
|
72
|
+
strategy: 'prefer-best',
|
|
73
|
+
},
|
|
74
|
+
},
|
|
75
|
+
events: {},
|
|
76
|
+
switches: [],
|
|
77
|
+
}
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
### Manual URL
|
|
81
|
+
|
|
82
|
+
Use when the server isn't on the launcher's broadcast domain (cross-subnet, internet, behind a reverse proxy) or you want to point at a specific public/unbound server.
|
|
83
|
+
|
|
84
|
+
```typescript
|
|
85
|
+
buttressConnectionSettings: {
|
|
86
|
+
enabled: true,
|
|
87
|
+
autoDiscoverType: 'manual',
|
|
88
|
+
url: 'http://192.168.1.100:2080',
|
|
89
|
+
fallbackType: 'use-local',
|
|
90
|
+
strategy: 'prefer-best',
|
|
91
|
+
}
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
> ⚠️ With `fallbackType: 'no-op'` the generator refuses to load locally if Buttress is unreachable. Use `'use-local'` whenever the device can also run the model standalone.
|
|
95
|
+
|
|
96
|
+
## Integrating a discovered buttress-server
|
|
97
|
+
|
|
98
|
+
The ctor-desktop "Local Devices" panel (and the `bricks buttress scan` CLI) can hand you a server's identity, workspace, and announced generator caps. When the user asks to integrate one:
|
|
99
|
+
|
|
100
|
+
1. **Confirm workspace match.** The discovery message says whether the server's workspace matches this project's. If it doesn't, do NOT add the integration — instruct the user to run `bricks buttress unbind && bricks buttress bind` from this workspace's owner CLI on the server host. The launcher won't trust a cross-workspace server.
|
|
101
|
+
|
|
102
|
+
2. **Map announced types to generator templates.** For each `generators[].type` in the announce, target the matching templateKey (create the generator if absent):
|
|
103
|
+
|
|
104
|
+
| Announced type | templateKey | Default model |
|
|
105
|
+
|----------------|------------------------------|---------------|
|
|
106
|
+
| `ggml-llm` | `GENERATOR_LLM` | `ggml-org/gemma-3-12b-it-qat-GGUF` (≥20 GB usable) or `ggml-org/gpt-oss-20b-GGUF` (~12 GB usable) |
|
|
107
|
+
| `mlx-llm` | `GENERATOR_MLX_LLM` | `mlx-community/Qwen3-4B-4bit` (or a larger 4-bit quant if `usableBytes` allows) |
|
|
108
|
+
| `ggml-stt` | `GENERATOR_SPEECH_INFERENCE` | `BricksDisplay/whisper-ggml` (filename `ggml-small-q8_0.bin`) |
|
|
109
|
+
|
|
110
|
+
Pick a model that fits the announced `usableBytes`. Set `contextSize` to match.
|
|
111
|
+
|
|
112
|
+
3. **Use the canonical auto-discovery config** from "Auto-discovery (recommended)" above. Don't switch to manual mode just because the server is on the LAN — auto mode picks the workspace-bound server automatically and rebinds when the device's workspace flips.
|
|
113
|
+
|
|
114
|
+
### Real-device caveat
|
|
115
|
+
|
|
116
|
+
Buttress LAN auto-discovery uses native UDP via `react-native-jsi-udp`. The iOS Simulator silently fails to bind UDP, so auto mode is a no-op there. With `fallbackType: 'use-local'` the generator falls back to local inference in the simulator — dev/test stays functional. **Validate the buttress path itself only when deploying to a real Foundation device; don't gate the build on simulator validation.**
|
|
117
|
+
|
|
118
|
+
## Server Setup
|
|
119
|
+
|
|
120
|
+
### Requirements
|
|
121
|
+
- [Bun](https://bun.sh) v1.3+
|
|
122
|
+
- GPU recommended for LLM/STT
|
|
123
|
+
|
|
124
|
+
### Installation
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
bun add -g @fugood/buttress-server
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
### Start Server
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
bricks-buttress
|
|
134
|
+
# or with config
|
|
135
|
+
bricks-buttress --config ./config.toml
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
### CLI Options
|
|
139
|
+
|
|
140
|
+
| Option | Description |
|
|
141
|
+
|--------|-------------|
|
|
142
|
+
| `-p, --port` | Port (default: 2080) |
|
|
143
|
+
| `-c, --config` | TOML config file path |
|
|
144
|
+
| `-v, --version` | Show version |
|
|
145
|
+
| `-h, --help` | Show help |
|
|
146
|
+
|
|
147
|
+
### Environment Variables
|
|
148
|
+
|
|
149
|
+
| Variable | Description |
|
|
150
|
+
|----------|-------------|
|
|
151
|
+
| `HF_TOKEN` | Hugging Face token for model downloads |
|
|
152
|
+
| `ENABLE_OPENAI_COMPAT_ENDPOINT` | Set to `1` for OpenAI-compatible API |
|
|
153
|
+
| `BRICKS_BUTTRESS_STATE_DIR` | Override the workspace state directory (default `~/.bricks-cli/buttress`) |
|
|
154
|
+
|
|
155
|
+
## Bind the Server to a Workspace
|
|
156
|
+
|
|
157
|
+
To use `autoDiscoverType: 'auto'` — and to require workspace-scoped JWT auth — pair the buttress-server with a BRICKS workspace using the `bricks buttress` CLI commands. An unbound server stays in legacy public mode and accepts any LAN client.
|
|
158
|
+
|
|
159
|
+
### One-time bind
|
|
160
|
+
|
|
161
|
+
```bash
|
|
162
|
+
# Log into the cloud bricks-server with the workspace owner's account
|
|
163
|
+
bricks auth login
|
|
164
|
+
|
|
165
|
+
# Pair the buttress-server running on this machine with the active workspace
|
|
166
|
+
bricks buttress bind
|
|
167
|
+
|
|
168
|
+
# Restart bricks-buttress so it picks up the new state.json
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
`bricks buttress bind` writes `~/.bricks-cli/buttress/state.json` containing the workspace id, the issuer's Ed25519 public key, and a stable `serverId` (defaults to `buttress-<machineId>`). Override location with `--state-dir` or `$BRICKS_BUTTRESS_STATE_DIR`. For headless setups, use `bricks buttress bind --print` to emit state.json to stdout.
|
|
172
|
+
|
|
173
|
+
### Verify and manage
|
|
174
|
+
|
|
175
|
+
```bash
|
|
176
|
+
# Show local binding + workspace-side bound list
|
|
177
|
+
bricks buttress status
|
|
178
|
+
|
|
179
|
+
# Discover buttress-servers on the LAN (with version, auth state, generator caps)
|
|
180
|
+
bricks buttress scan
|
|
181
|
+
|
|
182
|
+
# Issue a long-lived workspace access token (CI / ctor agents that already hold a workspace token)
|
|
183
|
+
bricks buttress issue-token --ttl 86400
|
|
184
|
+
|
|
185
|
+
# Detach this server from the workspace
|
|
186
|
+
bricks buttress unbind
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
After binding, launchers in the same workspace will auto-discover this server; manual-mode generators will only send their access token if the server's reported workspace matches their own.
|
|
190
|
+
|
|
191
|
+
## Server Configuration (TOML)
|
|
192
|
+
|
|
193
|
+
```toml
|
|
194
|
+
[server]
|
|
195
|
+
port = 2080
|
|
196
|
+
|
|
197
|
+
[runtime]
|
|
198
|
+
cache_dir = "./.buttress-cache"
|
|
199
|
+
n_threads = 6
|
|
200
|
+
flash_attn_type = "on"
|
|
201
|
+
cache_type_k = "q8_0"
|
|
202
|
+
cache_type_v = "q8_0"
|
|
203
|
+
|
|
204
|
+
# LLM Generator
|
|
205
|
+
[[generators]]
|
|
206
|
+
type = "ggml-llm"
|
|
207
|
+
[generators.backend]
|
|
208
|
+
variant_preference = ["cuda", "vulkan", "default"]
|
|
209
|
+
gpu_memory_fraction = 0.95
|
|
210
|
+
[generators.model]
|
|
211
|
+
repo_id = "ggml-org/gemma-3-12b-it-qat-GGUF"
|
|
212
|
+
download = true
|
|
213
|
+
n_ctx = 8192
|
|
214
|
+
|
|
215
|
+
# STT Generator
|
|
216
|
+
[[generators]]
|
|
217
|
+
type = "ggml-stt"
|
|
218
|
+
[generators.backend]
|
|
219
|
+
variant_preference = ["cuda", "vulkan", "default"]
|
|
220
|
+
[generators.model]
|
|
221
|
+
repo_id = "BricksDisplay/whisper-ggml"
|
|
222
|
+
filename = "ggml-small-q8_0.bin"
|
|
223
|
+
download = true
|
|
224
|
+
use_gpu = true
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
## Use Cases
|
|
228
|
+
|
|
229
|
+
### Resource-Constrained Devices
|
|
230
|
+
Digital signage with basic hardware offloads LLM to powerful server.
|
|
231
|
+
|
|
232
|
+
### Shared GPU Resources
|
|
233
|
+
Multiple devices share single GPU server for inference.
|
|
234
|
+
|
|
235
|
+
### Development Testing
|
|
236
|
+
Test AI features on lightweight dev machines by connecting to beefy server.
|
|
237
|
+
|
|
238
|
+
## Best Practices
|
|
239
|
+
|
|
240
|
+
1. **Network reliability**: Ensure stable LAN connection to Buttress server
|
|
241
|
+
2. **Fallback strategy**: Configure appropriate fallback for critical features
|
|
242
|
+
3. **Server monitoring**: Monitor Buttress server resource usage
|
|
243
|
+
4. **Model consistency**: Ensure client and server use compatible models
|
|
244
|
+
5. **Security**: Run Buttress on private network, not public internet
|
|
245
|
+
6. **Latency awareness**: Account for network latency in UX design
|
|
@@ -167,7 +167,7 @@ code: `
|
|
|
167
167
|
|
|
168
168
|
## Triggering Manually
|
|
169
169
|
|
|
170
|
-
Use `PROPERTY_BANK_COMMAND` system action to trigger a manual calculation:
|
|
170
|
+
Use `PROPERTY_BANK_COMMAND` system action to trigger a manual calculation. `input` references a Data that is a trigger input (`trigger: true`) of the target calc — the system runs the calc(s) that data feeds into. It does NOT reference the DataCalculation itself.
|
|
171
171
|
|
|
172
172
|
```typescript
|
|
173
173
|
const triggerCalc: EventAction = {
|
|
@@ -176,12 +176,14 @@ const triggerCalc: EventAction = {
|
|
|
176
176
|
__actionName: 'PROPERTY_BANK_COMMAND',
|
|
177
177
|
parent: 'System',
|
|
178
178
|
dataParams: [
|
|
179
|
-
{ input: () =>
|
|
179
|
+
{ input: () => priceData }, // Reference to a trigger Data input of the calc
|
|
180
180
|
],
|
|
181
181
|
},
|
|
182
182
|
}
|
|
183
183
|
```
|
|
184
184
|
|
|
185
|
+
When the same chain writes a calc input (e.g. `PROPERTY_BANK` setting `dLastButton`) then issues `PROPERTY_BANK_COMMAND`, set `waitAsync: true` on the write so the calc reads the new value rather than the pre-chain snapshot. See [Event Action Chains](architecture-patterns.md#event-action-chains-priority-2).
|
|
186
|
+
|
|
185
187
|
## Best Practices
|
|
186
188
|
|
|
187
189
|
1. **Avoid circular deps**: Set non-triggering inputs (`trigger: false`) or use `manual` mode
|
|
@@ -194,9 +196,7 @@ const triggerCalc: EventAction = {
|
|
|
194
196
|
See [Architecture Patterns](architecture-patterns.md) for the full pattern selection guide.
|
|
195
197
|
|
|
196
198
|
### Using Data Calc as an orchestrator
|
|
197
|
-
Scripts that manage state machines, control UI flow, or coordinate multi-step processes belong in Event Action Chains,
|
|
198
|
-
|
|
199
|
-
**Symptom**: Script has if/else branches deciding "what happens next" or tracks "current step".
|
|
199
|
+
Scripts that manage state machines, control UI flow, or coordinate multi-step processes belong in Event Action Chains. Symptoms: if/else on "what happens next", mirror `dFooResult` outputs that copy back to `dFoo` via `valueChange`, or a `dLastInput` field set-then-cleared to force an auto calc. See the "user-driven state machine" recipe in [Architecture Patterns](architecture-patterns.md).
|
|
200
200
|
|
|
201
201
|
### Quick reference
|
|
202
202
|
|
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
# Simulator
|
|
2
|
+
|
|
3
|
+
What the Simulator can and can't faithfully reproduce — so a passing Simulator run means what you think it means. This is the fidelity companion to Path 1 in [Verification Toolchain](verification-toolchain.md); read that for how to *drive* the Simulator (compile, screenshot, Automations).
|
|
4
|
+
|
|
5
|
+
## What it is
|
|
6
|
+
|
|
7
|
+
- The default Path 1 target: it renders your compiled project on your computer, with no device required.
|
|
8
|
+
- It runs in Chromium/Electron — a browser-class runtime, not the device runtime. Two consequences to design verification around: it has a browser's capabilities and limits (below), and its JavaScript engine differs from the device's (iOS runs JSC, Android runs Hermes), so engine-specific behavior won't surface here.
|
|
9
|
+
|
|
10
|
+
A Simulator screenshot proves **layout, navigation, Standby Transitions, Data wiring, and generator/Automation wiring**. It does *not* prove anything that depends on real hardware, real models, or capabilities the preview runtime lacks.
|
|
11
|
+
|
|
12
|
+
## Automation screenshots
|
|
13
|
+
|
|
14
|
+
`match_screenshot` stores Simulator automation artifacts at the project level:
|
|
15
|
+
|
|
16
|
+
```text
|
|
17
|
+
.bricks/automation_screenshots/
|
|
18
|
+
last/
|
|
19
|
+
failed/
|
|
20
|
+
diff/
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
There is no application-id subdirectory; in CTOR the project is the application. Generated project `.gitignore` files exclude this directory.
|
|
24
|
+
|
|
25
|
+
When a `match_screenshot` baseline is missing or updated, the baseline goes under `last/`. When a screenshot mismatch occurs, the current capture goes under `failed/` and the pixel diff goes under `diff/`. File names follow `<screenshotName>+local+<timestamp>.png`.
|
|
26
|
+
|
|
27
|
+
In CTOR Desktop, `simulator_invoke` reports any automation screenshot files created or changed by the invocation in an `Automation screenshots:` section. It still separately reports the ordinary preview capture at `.bricks/simulator-screenshot.jpg`.
|
|
28
|
+
|
|
29
|
+
## What the Simulator can't reproduce
|
|
30
|
+
|
|
31
|
+
Because it runs in a browser-class preview rather than on device hardware, the Simulator cannot exercise these — a green run says nothing about them. Verify on a real device (Path 2):
|
|
32
|
+
|
|
33
|
+
| Area | In the Simulator |
|
|
34
|
+
|------|------------------|
|
|
35
|
+
| BLE (Central / Peripheral), Video Streaming | Not available |
|
|
36
|
+
| Raw network sockets — UDP, TCP, TCP Server, MQTT Broker | Not available (also why Buttress LAN discovery is a no-op — see [Buttress](buttress.md)) |
|
|
37
|
+
| Android Intent | Not available (Android-only) |
|
|
38
|
+
| On-device database **persistence** (SQLite / Vector Store) | Not available — both run in-memory in the Simulator (see caveats below), so data never survives a reload |
|
|
39
|
+
| GGML Text-to-Speech | No vocoder in the preview — use an ONNX TTS model to hear speech |
|
|
40
|
+
|
|
41
|
+
These work, but with browser caveats:
|
|
42
|
+
|
|
43
|
+
| Area | Caveat |
|
|
44
|
+
|------|--------|
|
|
45
|
+
| Serial Port | Works via WebSerial — requires browser support and a user gesture |
|
|
46
|
+
| WebView / WebCrawler | Subject to browser CORS — a load/fetch that works on device may be blocked |
|
|
47
|
+
| On-device AI (LLM / STT / VAD / Vector Store / Reranker) | Runs **single-threaded** — far slower than the device, not representative of real latency. Also subject to the model fallbacks below |
|
|
48
|
+
| On-device database (SQLite — `GENERATOR_SQLITE`) | Runs for real on the in-memory WASM `sqlite-vec` build — `execute` / `query` / `transaction` / batch all work. `storageType: file` is transparently treated as in-memory, so nothing persists across reloads (see above) |
|
|
49
|
+
| Scene3D / Maps / Sketch / WebRTC | Supported |
|
|
50
|
+
|
|
51
|
+
Feature availability also varies across the device platforms themselves (iOS / tvOS / Android / the desktop OSes). When a deployment targets a specific platform's capability, confirm it on that platform.
|
|
52
|
+
|
|
53
|
+
## Network security
|
|
54
|
+
|
|
55
|
+
The Simulator blocks insecure connections from `applicationPreview` by default: `http://` and `ws://` requests fail, while `https://` and `wss://` are allowed. The Simulator internally allows its own preview host when running the development preview.
|
|
56
|
+
|
|
57
|
+
Generated projects include this project-level setting:
|
|
58
|
+
|
|
59
|
+
```json
|
|
60
|
+
{
|
|
61
|
+
"allowedInsecureHosts": []
|
|
62
|
+
}
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Add explicit entries only when you deliberately need to test an insecure endpoint in the Simulator. A bare `host:port` allows both `http://` and `ws://` for that host:
|
|
66
|
+
|
|
67
|
+
```json
|
|
68
|
+
{
|
|
69
|
+
"allowedInsecureHosts": ["localhost:8080"]
|
|
70
|
+
}
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
Use a URL form when only one insecure protocol should be allowed:
|
|
74
|
+
|
|
75
|
+
```json
|
|
76
|
+
{
|
|
77
|
+
"allowedInsecureHosts": ["http://localhost:8080"]
|
|
78
|
+
}
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
In that example, `http://localhost:8080` is allowed and `ws://localhost:8080` remains blocked.
|
|
82
|
+
|
|
83
|
+
Prefer HTTPS/WSS for anything that is not local development.
|
|
84
|
+
|
|
85
|
+
## Feature fallbacks — what the Simulator fakes
|
|
86
|
+
|
|
87
|
+
So that camera and AI features are usable without device permissions, multi-gigabyte downloads, API keys, or a remote inference backend, the Simulator **transparently substitutes a lightweight stand-in** for a few bricks/generators. The config you author is untouched — only the Simulator's runtime behavior differs. **A fallback render proves wiring and layout, not the real feature.**
|
|
88
|
+
|
|
89
|
+
| Brick / Generator | In the Simulator | Does NOT prove |
|
|
90
|
+
|-------------------|------------------|----------------|
|
|
91
|
+
| Camera (`BRICK_CAMERA`) | A 3D mock canvas, no camera permission prompt. `takePicture` snapshots the canvas; recording produces a placeholder clip | Real camera feed, focus, recording, permission flow |
|
|
92
|
+
| Thermal Printer (`GENERATOR_THERMAL_PRINTER`) | A simulated printer — `init` / `checkStatus` / `scan` fake per-driver status and discovered devices (ESC/POS, Star, TSC, Castles); `print` renders an approximate on-screen receipt. A bottom-left bubble shows live status with a fault toggle to exercise error wiring | Real device connection, actual paper output, exact native driver status codes |
|
|
93
|
+
| LLM (`GENERATOR_LLM`) | Swapped to a tiny local stand-in model | Output quality / latency of your real model |
|
|
94
|
+
| Reranker — GGML (`GENERATOR_RERANKER`) | Swapped to a small local multilingual reranker model | Ranking quality / latency of your real model |
|
|
95
|
+
| Speech-to-Text — GGML (`GENERATOR_SPEECH_INFERENCE`) | Swapped to a tiny local model | Accuracy / latency of your real model |
|
|
96
|
+
| Speech-to-Text — ONNX (`GENERATOR_ONNX_STT`) | Swapped to a tiny whisper model | Accuracy / latency of your real model |
|
|
97
|
+
| Text-to-Speech — ONNX (`GENERATOR_TTS`) | Swapped to a tiny VITS model (no vocoder or speaker embedding) | Voice / quality / latency of your real model |
|
|
98
|
+
| Vector Store (`GENERATOR_VECTOR_STORE`) | Runs with a tiny local embedding model — an OpenAI-source config works **without a key** | Real embeddings / dimensions / recall (and no persistence — see above) |
|
|
99
|
+
| Buttress (all generators) | **Disabled** — there is no remote inference backend in the Simulator | The Buttress offload path — see [Buttress](buttress.md) |
|
|
100
|
+
| MLX LLM (`GENERATOR_MLX_LLM`) | Runs as authored, except Buttress is disabled | The MLX / Buttress path |
|
|
101
|
+
|
|
102
|
+
These swaps make the AI generators runnable in the Simulator for wiring checks — validate real models, quality, and latency on a device (Path 2).
|
|
103
|
+
|
|
104
|
+
### Running the real implementation instead
|
|
105
|
+
|
|
106
|
+
Each substituted brick/generator can be switched back to its real implementation per item: open the **gear (Simulator settings)** in the editor's preview toolbar, uncheck the item, and **Apply**. Apply persists the choice and reloads the preview so it takes effect (a plain refresh won't). Use this to, e.g., point a Vector Store at a real API key in the preview. The browser limits above still apply, and **Buttress stays disabled regardless** — there's no backend for it here.
|
|
107
|
+
|
|
108
|
+
The Thermal Printer is the exception: it has no real web implementation to switch to (the native drivers can't run in a browser), so it is **always simulated** and is not in the gear list.
|
|
109
|
+
|
|
110
|
+
## Enough vs. escalate
|
|
111
|
+
|
|
112
|
+
- **Simulator is enough for:** layout, navigation, Standby Transitions, Data wiring, Automation / state-machine happy paths, and confirming a generator fires and handles its events.
|
|
113
|
+
- **Escalate to a real device for:** anything in the "can't reproduce" tables, real camera / peripherals / payment / identity, real model quality & latency, multi-device Local Sync, on-device persistence, the Buttress offload path, and engine-specific behavior.
|
|
114
|
+
|
|
115
|
+
The full Path 1/2/3 decision rule lives in [Verification Toolchain](verification-toolchain.md).
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
# Verification Toolchain
|
|
2
|
+
|
|
3
|
+
Three runtime targets, one decision rule. Verify against the cheapest path that proves what you need to prove.
|
|
4
|
+
|
|
5
|
+
## Definition of done — the hard gate
|
|
6
|
+
|
|
7
|
+
Before the agent is allowed to claim work is complete, **all** of the following must be true:
|
|
8
|
+
|
|
9
|
+
1. **Compile clean.** Latest source passes `compile` with no errors.
|
|
10
|
+
2. **Every Canvas screenshot captured.** A rendered screenshot of every Canvas in the deliverable, captured via the available preview tool (`responseImage: true` when the selected model supports vision) at the target hardware resolution and orientation. Canvases gated behind events are reached via Automation runs (`testId` / `testTitleLike` with `brick_press` / `wait_until_canvas_change` cases). Single-Canvas Subspaces still require a captured screenshot.
|
|
11
|
+
3. **Every screenshot reviewed by the agent.** The agent must read each captured screenshot back through the host's image-reading capability — saving the file is not the same as seeing it. The model that produced the Canvas must also have seen the rendered result, or the verification gate has not passed.
|
|
12
|
+
4. **Reference comparison report.** If the user supplied any reference material (Figma / website / HTML / screenshot / PDF / brand book / competitor), produce a short delta report per Canvas:
|
|
13
|
+
- What matches the reference.
|
|
14
|
+
- What intentionally diverges, and why (deployment constraint, BRICKS-runtime semantic, system commitment).
|
|
15
|
+
- What unintentionally diverges, and the planned fix.
|
|
16
|
+
5. **Path appropriate to deployment executed.** Path 1 by default. Path 2 (on-device) when the deployment depends on real hardware behaviour, or whenever the brief touches payment, identity, peripherals, or LocalSync.
|
|
17
|
+
6. **Console clean.** No 404s on media, no Data-key-undefined warnings, no Generator init failures, no React-style warnings — every console line is either zero or accept-with-a-comment.
|
|
18
|
+
|
|
19
|
+
What does **not** count as done:
|
|
20
|
+
|
|
21
|
+
- A single hero-Canvas screenshot.
|
|
22
|
+
- "Looks roughly like the reference" with no per-Canvas comparison.
|
|
23
|
+
- "Compiled successfully" with no rendered preview.
|
|
24
|
+
- A claim of done with no screenshots in evidence.
|
|
25
|
+
- Captured screenshots saved to disk but never read back by the agent.
|
|
26
|
+
|
|
27
|
+
If any item is unmet, the work is mid-iteration. Say so explicitly to the user; offer a precise list of what remains.
|
|
28
|
+
|
|
29
|
+
## Path 1 — Simulator (no device required)
|
|
30
|
+
|
|
31
|
+
The default loop. Always available; deterministic; no device wear; safe for side-effecting flows because nothing real fires.
|
|
32
|
+
|
|
33
|
+
Before trusting a Simulator screenshot, know what it can and can't reproduce: it runs in the Electron preview (a browser-class runtime, not the device — and not the device's JS engine), can't exercise hardware/peripheral features (BLE, printers, raw sockets, SQLite, …), and substitutes fallbacks for Camera and the AI generators (LLM/STT/Vector Store) with Buttress disabled. See [Simulator](simulator.md) for the fidelity boundaries and the per-item gear to run the real implementation — it's what tells you when a green run is enough vs. when to escalate to Path 2.
|
|
34
|
+
|
|
35
|
+
`bun update-app` and `bun deploy-app` publish to the BRICKS portal. The local preview reads `.bricks/build/application-config.json` (produced by `compile`) directly.
|
|
36
|
+
|
|
37
|
+
### Compile tool
|
|
38
|
+
|
|
39
|
+
Typecheck + compile the project. Gate every iteration on this; everything below assumes a clean compile.
|
|
40
|
+
|
|
41
|
+
Agent invocation: call the MCP tool `compile` exposed by the `bricks-ctor` MCP server registered for the project. No arguments.
|
|
42
|
+
|
|
43
|
+
### Simulator tool
|
|
44
|
+
|
|
45
|
+
Use the preview implementation exposed by the current harness:
|
|
46
|
+
|
|
47
|
+
- **CTOR Desktop agent session:** use `simulator_invoke`. CTOR Desktop disables the `bricks-ctor` MCP `simulator` tool and routes screenshots/automation through the desktop preview pane instead. If the user already opened the simulator pane, `simulator_invoke` should reuse it rather than start a separate preview.
|
|
48
|
+
- **Pure `bricks-ctor` project / other agent harness:** use the `bricks-ctor` MCP `simulator` tool when `simulator_invoke` is not available.
|
|
49
|
+
|
|
50
|
+
Both forms are the same verification primitive: launch or reuse Electron preview, take a screenshot, and optionally run a named Automation test by id or partial title. Do not hard-code one tool name into the workflow; choose the available preview tool for the environment.
|
|
51
|
+
|
|
52
|
+
Common arguments:
|
|
53
|
+
- `delay` (ms before screenshot) — default is harness-specific: usually 3000 for a fresh/standalone preview; CTOR Desktop may capture immediately when its preview already has a settled latest config. Increase if Standby Transitions are still in flight.
|
|
54
|
+
- `width`, `height` — screenshot dimensions in px.
|
|
55
|
+
- `responseImage: true` — return the screenshot as image content when the selected model supports vision. If vision is unavailable, save the screenshot and report the path.
|
|
56
|
+
- `testId` or `testTitleLike` — run a project Automation before the screenshot. Use `testTitleLike` for fuzzy matches (case-insensitive partial title).
|
|
57
|
+
|
|
58
|
+
Returns text log + saved screenshot path + image content when supported.
|
|
59
|
+
|
|
60
|
+
### `bun invoke-simulator` (project script)
|
|
61
|
+
|
|
62
|
+
Sustained dev session — watches `subspaces/`, recompiles on save, exposes CDP at `localhost:19852` (configurable via `--cdp-port`), writes `.bricks/devtools.json` with port/pid for downstream tooling.
|
|
63
|
+
|
|
64
|
+
Useful flags:
|
|
65
|
+
- `--screenshot` + `--screenshot-delay/width/height/path` — drive screenshot capture without keeping the window open.
|
|
66
|
+
- `--show-menu` — surface the Electron menu (debugging).
|
|
67
|
+
- `--test-id` / `--test-title-like` — run an Automation in this preview.
|
|
68
|
+
- `--no-keep-open` — exit after one screenshot.
|
|
69
|
+
- `--no-cdp` — disable CDP server (rare; default is to expose it).
|
|
70
|
+
- `--clear-cache` — reset cached state between runs.
|
|
71
|
+
|
|
72
|
+
For ad-hoc CDP inspection against this local preview, connect any CDP client to `localhost:19852` — Chrome DevTools front-end works directly. For an agent-friendly CLI over CDP (screenshot, brick tree/query, input emulation, storage reads, runtime eval, network capture), the `bricks-cli` skill documents the `bricks devtools` command surface — read that skill if it is installed in this workspace. If it is not installed, run `bricks --help` and `bricks devtools --help`; the CLI's own help output is authoritative.
|
|
73
|
+
|
|
74
|
+
### Project Automations
|
|
75
|
+
|
|
76
|
+
E2E tests authored in TypeScript inside the project (`AutomationTest` / `TestCase`). Test cases include:
|
|
77
|
+
|
|
78
|
+
- `brick_press` — synthesize a press on a Brick.
|
|
79
|
+
- `wait_until_canvas_change` — assert navigation.
|
|
80
|
+
- `assert_property` — assert a Data value equals expected.
|
|
81
|
+
- `wait_property_update` — wait for a Data value to change.
|
|
82
|
+
- `match_screenshot` — visual regression with stored reference image.
|
|
83
|
+
|
|
84
|
+
Trigger types: `launch` (runs on app start), `anytime` (manual), `cron` (scheduled).
|
|
85
|
+
|
|
86
|
+
Important: the automation map id must be `'AUTOMATION_MAP_DEFAULT'` (not a generated id) — the preview test runner reads from `automationMap['AUTOMATION_MAP_DEFAULT']?.map`.
|
|
87
|
+
|
|
88
|
+
Run a single test from the agent: call the available preview tool (`simulator_invoke` in CTOR Desktop, otherwise `bricks-ctor` MCP `preview`) with `testId` or `testTitleLike`.
|
|
89
|
+
|
|
90
|
+
## Path 2 — Real device with DevTools enabled
|
|
91
|
+
|
|
92
|
+
Required when the deployment depends on hardware behaviour the Electron preview cannot reproduce: physical orientation/DPI, real touch hardware (IR overlays, multi-touch), peripherals (camera, BLE, MQTT, NFC, payment, printer, sensors), watchdog cycles on the actual launcher build, fleet-managed reboot semantics, LocalSync across multiple devices, the actual color/luminance of the panel.
|
|
93
|
+
|
|
94
|
+
**Always Path 2** when the brief touches: payment, identity capture, real-time peripheral data, multi-device LocalSync, certified hardware.
|
|
95
|
+
|
|
96
|
+
### One-time manual setup (the agent cannot do this)
|
|
97
|
+
|
|
98
|
+
Ask the user to:
|
|
99
|
+
|
|
100
|
+
1. On the device, open **Settings** → advanced settings.
|
|
101
|
+
2. Toggle **Chrome DevTools** on. The device starts a DevTools server on the local network on port `19851` (auto-increments if taken).
|
|
102
|
+
3. Confirm **Enable LAN Discovery** is on (default) so the device is discoverable.
|
|
103
|
+
4. Note the device passcode if displayed.
|
|
104
|
+
|
|
105
|
+
Requires BRICKS Foundation **≥ 2.24** for CDP commands.
|
|
106
|
+
|
|
107
|
+
### Endpoints exposed once enabled
|
|
108
|
+
|
|
109
|
+
| Endpoint | URL shape | Use |
|
|
110
|
+
|---|---|---|
|
|
111
|
+
| Web UI | `http://<ip>:19851` | DevTools landing in a browser |
|
|
112
|
+
| CDP | `http://<ip>:19851/devtools-frontend/inspector.html?ws=<ip>:19851/ws/<passcode>` | Chrome DevTools front-end |
|
|
113
|
+
| MCP | `http://<ip>:19851/mcp` | MCP endpoint (Bearer-token auth with passcode) |
|
|
114
|
+
| MCP SSE | `http://<ip>:19851/sse` | Same, SSE transport |
|
|
115
|
+
| Info | `http://<ip>:19851/devtools/info` | Device / server metadata |
|
|
116
|
+
|
|
117
|
+
### Driving the device
|
|
118
|
+
|
|
119
|
+
Once DevTools is on, ask the user how they want to drive the verification — Chrome DevTools front-end, an MCP client they already run, or screenshot export from their own tooling — and follow their lead. Capture every Canvas screenshot through whichever path the user picks; the verification gate is about the evidence, not the tool.
|
|
120
|
+
|
|
121
|
+
For agent-driven CDP/MCP work against the device (`bricks devtools …` with `-a <ip> --passcode <pc>`, plus bridging the device MCP endpoint into an MCP client), the same `bricks-cli` skill referenced in Path 1 covers the on-device case — read it if installed. If not installed, run `bricks --help` and `bricks devtools --help` for the authoritative command listing.
|
|
122
|
+
|
|
123
|
+
### Real-device side-effects warning
|
|
124
|
+
|
|
125
|
+
Real devices fire real peripherals. Payment terminals charge. MQTT broadcasts on shared topics. BLE advertises to bystanders. Use a **staging** device for verification cycles; never iterate on a production-deployed device unless the user explicitly approves.
|
|
126
|
+
|
|
127
|
+
## Path 3 — Remote debugging via BRICKS Controller (off-LAN)
|
|
128
|
+
|
|
129
|
+
Out of scope for the standard verification loop. It exists for ops scenarios where the device is not on the same network. If the user asks for it, point them at the BRICKS Controller documentation rather than attempting it as part of verification.
|
|
130
|
+
|
|
131
|
+
## Decision rule
|
|
132
|
+
|
|
133
|
+
```
|
|
134
|
+
default Path 1.
|
|
135
|
+
|
|
136
|
+
if deployment depends on:
|
|
137
|
+
physical orientation / DPI / touch HW / peripherals /
|
|
138
|
+
watchdog on real launcher / LocalSync / certified hardware
|
|
139
|
+
→ escalate to Path 2 before declaring done.
|
|
140
|
+
|
|
141
|
+
if brief touches payment, identity, peripherals, LocalSync
|
|
142
|
+
→ Path 2 is mandatory, not optional.
|
|
143
|
+
|
|
144
|
+
if user is off-LAN
|
|
145
|
+
→ Path 3 (out of scope here).
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
## What to actually verify (per shape)
|
|
149
|
+
|
|
150
|
+
### Static signage (single-canvas glance loop)
|
|
151
|
+
- Path 1 screenshot captures the loop frame.
|
|
152
|
+
- Watch one full rotation in `bun invoke-simulator` to see all queue items.
|
|
153
|
+
- Cut network mid-loop; confirm cached media plays.
|
|
154
|
+
- Path 2 only if the panel is OLED / has burn-in concerns or specific color profile.
|
|
155
|
+
|
|
156
|
+
### Multi-canvas state machine (kiosk)
|
|
157
|
+
- Path 1: Automation walking the happy path with `brick_press` + `wait_until_canvas_change` + `assert_property` + `match_screenshot` per Canvas.
|
|
158
|
+
- Cancel + inactivity reset paths verified as Automations.
|
|
159
|
+
- Watchdog reset: kill the runtime, restart, confirm boot Canvas resets transient flow Data.
|
|
160
|
+
- Path 2 mandatory if payment / identity is in the flow — fire a real test transaction on staging hardware.
|
|
161
|
+
|
|
162
|
+
### Subspace-driven composition
|
|
163
|
+
- Path 1: open the Subspace standalone if the preview supports it; verify in isolation.
|
|
164
|
+
- Embedded test: walk the host's flow; assert Outlets fire as expected via `wait_property_update`.
|
|
165
|
+
- Re-mount test: cause the host to re-bind the Subspace; confirm internal state resets.
|
|
166
|
+
|
|
167
|
+
### Generator-driven reactive
|
|
168
|
+
- Path 1: drive the source by writing to Data from the runtime (whatever CDP path the user prefers); observe Brick re-render.
|
|
169
|
+
- Throttle / firehose tests: write 100 updates rapidly; confirm the canvas doesn't choke (frame budget intact).
|
|
170
|
+
- Disconnect: simulate source disconnect; confirm UI surfaces stale state, not blank.
|
|
171
|
+
- Threshold transitions: cross every Switch boundary; confirm visual reaction.
|
|
172
|
+
- Path 2 mandatory if the source is a real peripheral — Electron cannot fake the timing or the failure modes.
|
|
173
|
+
|
|
174
|
+
### Multi-device LocalSync
|
|
175
|
+
- Path 2 mandatory. Two devices on the same LAN with DevTools on. Drive one, observe the other.
|
|
176
|
+
|
|
177
|
+
## Browser / runtime log discipline
|
|
178
|
+
|
|
179
|
+
Throughout: the DevTools console must be clean. 404s on media, "Data key not defined" warnings, Generator init failures, React-style warnings — every one is a defect or accept-with-comment. Don't ship around them.
|