roam-research-mcp 3.2.0 → 4.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +28 -0
- package/build/Roam_Markdown_Cheatsheet.md +129 -224
- package/build/cli/utils/output.js +36 -7
- package/build/markdown-utils.js +14 -7
- package/build/server/roam-server.js +1 -1
- package/build/shared/block-escaping.js +74 -0
- package/build/tools/operations/full-page-view.js +21 -6
- package/build/tools/operations/pages.js +136 -6
- package/build/tools/tool-handlers.js +3 -1
- package/package.json +3 -2
package/README.md
CHANGED
|
@@ -20,6 +20,24 @@ Whether you want to give Claude superpowers over your knowledge base or just wan
|
|
|
20
20
|
|
|
21
21
|

|
|
22
22
|
|
|
23
|
+
## What's New in v4.0
|
|
24
|
+
|
|
25
|
+
**In one line:** a block containing a soft line break (Shift+Enter) now survives a page rewrite. Read a page, write it back, and nothing moves.
|
|
26
|
+
|
|
27
|
+
Until now, a multi-line block rendered as two physical lines, the second at column 0. That reset the parser's indentation baseline, so every block after it collapsed toward the root and `roam_update_page_markdown` dutifully generated the moves to make your real page match. Reading a page and writing back a revision, the documented purpose of the tool, was enough to trigger it. Callout bodies and fenced code blocks are exactly the blocks that carry soft breaks.
|
|
28
|
+
|
|
29
|
+
- **Soft breaks render as `⏎`.** A page containing one gains a leading `<!-- roam:escaped-newlines -->` marker line; keep it if you write the markdown back. Pages with no multi-line block render byte-identical to 3.x, with no marker and no encoding.
|
|
30
|
+
- **Backslashes are never special.** The earlier design escaped newlines as `\n`, which is also a common prefix in authored text: `\nabla`, `\neq`, `C:\newdir`. The sentinel needs no such rule, so all of those are written exactly as typed, everywhere.
|
|
31
|
+
- **Verbatim round-trips are no-ops.** Renderer output submitted back unchanged, title header included, produces zero actions. The CLI shares the fix: `roam get` piped into `roam save --update` leaves the page as it was.
|
|
32
|
+
- **Two more guards on page rewrites.** Non-empty markdown that parses to zero blocks is now refused rather than deleting every block on the page (genuinely empty markdown still clears a page, as documented). And a hand-authored first block that happens to be an H1 echoing the page title is no longer stripped on an ordinary update.
|
|
33
|
+
- **Linked references are encoded too.** `roam_fetch_page_full_view` escapes soft breaks in referring blocks and breadcrumbs, not just the page's own content.
|
|
34
|
+
- **Browser clients pass CORS preflight.** The HTTP transport now allows `MCP-Protocol-Version` and `Last-Event-ID`, both of which a client must send after initialization. Non-browser clients were never affected.
|
|
35
|
+
- **The MCP SDK is pinned** to the version the test suite runs against, so a fresh install gets the protocol surface that was tested rather than whatever npm serves that day.
|
|
36
|
+
|
|
37
|
+
**Why a major.** Four read surfaces return different bytes for any page containing a multi-line block: `roam_fetch_page_by_title` (`format: "markdown"`), `roam_fetch_page_full_view`, `roam_get_subpages`, and `roam get`. If you use the server through an AI assistant, nothing is required of you. A script that parses markdown output of multi-line pages sees the new encoding. If you pinned `roam-research-mcp@3`, you keep 3.2.0's fixes and its documented multi-line limitation until you re-pin.
|
|
38
|
+
|
|
39
|
+
Full detail, including the corner cases and how each fix was verified against the prior state, is in the [changelog](CHANGELOG.md).
|
|
40
|
+
|
|
23
41
|
## How this differs from Roam's official MCP server
|
|
24
42
|
|
|
25
43
|
Roam Research ships its own MCP server and CLI ([`@roam-research/roam-mcp`](https://github.com/Roam-Research/roam-tools)). It is a good tool, and this project is not trying to replace it. **They talk to two different Roam APIs, which is the difference everything else follows from.**
|
|
@@ -162,6 +180,8 @@ Three things worth knowing:
|
|
|
162
180
|
- **Read tools deliberately have neither.** They already serialise their whole result into the text channel, so a schema would just double the payload.
|
|
163
181
|
- **These fields are additive-only.** Some clients validate live responses against a cached tool list, so a field will be added or deprecated — never renamed or removed outside a major version.
|
|
164
182
|
|
|
183
|
+
> **Upgrading to 4.0.0:** markdown reads of a page containing a soft line break (Shift+Enter) now render that break as `⏎` and carry a leading `<!-- roam:escaped-newlines -->` marker, so the page survives a write-back intact. Pages without multi-line blocks are byte-identical to 3.x. AI-assistant users need do nothing; scripts parsing markdown output of multi-line pages see the new encoding. See the [changelog](CHANGELOG.md).
|
|
184
|
+
|
|
165
185
|
> **Upgrading from 2.x:** three write-result fields were renamed — `uid` → `page_uid` (`roam_create_page`), `created_uids` → `created_blocks` (`roam_create_outline`, `roam_import_markdown`) and `preservedUids` → `preserved_uids` (`roam_update_page_markdown`). This only affects code that reads those names; if you use the server through an AI assistant, nothing changes. See the [changelog](CHANGELOG.md) for why.
|
|
166
186
|
|
|
167
187
|
---
|
|
@@ -208,6 +228,14 @@ Results are cached for 30 seconds — an edit to the page takes effect without a
|
|
|
208
228
|
|
|
209
229
|
Note that guidelines are read through the normal page path, so blocks tagged `#.rm-hide` / `#.rm-private` are withheld from them too — see below.
|
|
210
230
|
|
|
231
|
+
Reads of a page containing a soft line break render it as `⏎` so each block
|
|
232
|
+
stays on one line — an unescaped newline lands at column 0 and reparents
|
|
233
|
+
everything after it on write-back. Such payloads carry a leading
|
|
234
|
+
`<!-- roam:escaped-newlines -->` marker; keep it if you write the markdown
|
|
235
|
+
back. Markdown you author is never decoded: backslashes are not special, and
|
|
236
|
+
only `⏎` inside a marked payload is interpreted. `roam_get_guidelines` output
|
|
237
|
+
is plain prose — no sentinel, no marker.
|
|
238
|
+
|
|
211
239
|
---
|
|
212
240
|
|
|
213
241
|
## Hiding content from the AI
|
|
@@ -1,16 +1,22 @@
|
|
|
1
|
-
# Roam Markdown Cheatsheet v2.
|
|
1
|
+
# Roam Markdown Cheatsheet v2.8.0
|
|
2
2
|
|
|
3
3
|
## Core Syntax
|
|
4
4
|
|
|
5
5
|
### Formatting
|
|
6
6
|
`**bold**` · `__italic__` · `^^highlight^^` · `~~strike~~` · `` `code` `` · `$$LaTeX$$`
|
|
7
7
|
|
|
8
|
+
⚠️ `*italic*` and `_italic_` are normalized to `__italic__` by the markdown tools, but `roam_process_batch_actions` writes strings literally and Roam does not render single markers. Write `__italic__` everywhere so reads and writes agree.
|
|
9
|
+
|
|
10
|
+
### Headings
|
|
11
|
+
`# H1` · `## H2` · `### H3` at the start of a block. Roam has H1–H3 only; `####` and deeper are not headings and stay literal text. A heading is a property of its own block: it does not nest the blocks after it. Indent to nest.
|
|
12
|
+
|
|
8
13
|
### Links & References
|
|
9
14
|
- **Page ref:** `[[Page Name]]` — creates/links to page
|
|
10
15
|
- **Block ref:** `((block-uid))` — embeds block content inline
|
|
11
16
|
- **Block embed:** `{{[[embed]]: ((block-uid))}}` — full block with children
|
|
12
17
|
- **Embed children:** `{{[[embed-children]]: ((block-uid))}}` — children only (not the parent block)
|
|
13
18
|
- **Embed path:** `{{[[embed-path]]: ((block-uid))}}` — block with its ancestor path
|
|
19
|
+
- **Page embed:** `{{[[embed]]: [[Page Name]]}}` — the whole page inline
|
|
14
20
|
- **External:** `[text](URL)`
|
|
15
21
|
- **Aliased page:** `[display text]([[Actual Page]])`
|
|
16
22
|
- **Aliased block:** `[display text](<((block-uid))>)` — note the angle brackets
|
|
@@ -32,6 +38,8 @@ Always ordinal format: `[[January 1st, 2025]]`, `[[December 23rd, 2024]]`
|
|
|
32
38
|
- Todo: `{{[[TODO]]}} task`
|
|
33
39
|
- Done: `{{[[DONE]]}} task`
|
|
34
40
|
|
|
41
|
+
⚠️ `{{[[TODO]]}}` must lead the block. Anywhere else it renders a checkbox that does not toggle to DONE.
|
|
42
|
+
|
|
35
43
|
### Callouts
|
|
36
44
|
Styled blockquotes with an icon and colour. Two page refs open the block: `[[>]]` marks it a callout, `[[!TYPE]]` picks the style.
|
|
37
45
|
|
|
@@ -49,18 +57,30 @@ Append `+` or `-` to make it foldable — `[[!TIP]]+` starts expanded, `[[!TIP]]
|
|
|
49
57
|
⚠️ `[[>]]` and `[[!TIP]]` are real page references, so every callout backlinks to those pages. That is normal and how the feature works — don't "clean it up."
|
|
50
58
|
⚠️ A plain `> quote` is an ordinary blockquote, not a callout. The two are unrelated.
|
|
51
59
|
|
|
52
|
-
###
|
|
60
|
+
### Soft line breaks
|
|
53
61
|
|
|
54
|
-
A block can hold a **soft line break** (Shift+Enter
|
|
55
|
-
|
|
62
|
+
A block can hold more than one line — a **soft line break** (Shift+Enter in the
|
|
63
|
+
UI). It stays *one* block, which is what callout bodies and fenced code blocks
|
|
64
|
+
rely on.
|
|
56
65
|
|
|
57
|
-
**
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
66
|
+
**You cannot write one through the markdown tools.** `roam_create_page`,
|
|
67
|
+
`roam_import_markdown`, `roam_create_outline` and `roam_update_page_markdown`
|
|
68
|
+
all treat a line break as a block break. Use `roam_process_batch_actions`, which
|
|
69
|
+
writes block strings literally — put a real newline in the `string`.
|
|
61
70
|
|
|
62
|
-
|
|
63
|
-
|
|
71
|
+
The one exception: inside a payload carrying the `<!-- roam:escaped-newlines -->`
|
|
72
|
+
marker (below), a `⏎` decodes to a real soft line break on write. So keeping —
|
|
73
|
+
or adding — a `⏎` in a marker-carrying payload handed to
|
|
74
|
+
`roam_update_page_markdown` *is* the one markdown-tools path that writes one.
|
|
75
|
+
Without the marker, `⏎` is just the character; it does not decode.
|
|
76
|
+
|
|
77
|
+
⚠️ Reads render a soft line break as `⏎` so the block stays on one line, and a
|
|
78
|
+
payload containing any is marked with a leading `<!-- roam:escaped-newlines -->`
|
|
79
|
+
comment. If you edit that markdown and pass it back to
|
|
80
|
+
`roam_update_page_markdown`, **keep the marker line** — it is what tells the
|
|
81
|
+
server `⏎` means a line break there. Content you add yourself is safe either
|
|
82
|
+
way: backslashes are never special, and `$$\nabla f$$` or `C:\newdir` are
|
|
83
|
+
written exactly as typed.
|
|
64
84
|
|
|
65
85
|
### Attributes
|
|
66
86
|
```
|
|
@@ -84,6 +104,9 @@ Rating:: 4/5
|
|
|
84
104
|
- Grandchild
|
|
85
105
|
```
|
|
86
106
|
|
|
107
|
+
### Numbered & document lists
|
|
108
|
+
A numbered list is a **view type on the parent**, not `1.` markers in the text. Set `children-view-type: "numbered"` (or `"document"` for no bullets) on the parent through `roam_process_batch_actions` (`create-block` / `update-block`) or a `roam_create_page` content item. `1.` markers written into block text stay literal.
|
|
109
|
+
|
|
87
110
|
### Code Blocks
|
|
88
111
|
````
|
|
89
112
|
```javascript
|
|
@@ -91,6 +114,8 @@ const x = 1;
|
|
|
91
114
|
```
|
|
92
115
|
````
|
|
93
116
|
|
|
117
|
+
⚠️ Through `roam_process_batch_actions` (literal), do not leave a trailing newline before the closing fence. Roam stores it verbatim and renders it wrong. The markdown tools normalize it away.
|
|
118
|
+
|
|
94
119
|
### Queries
|
|
95
120
|
```
|
|
96
121
|
{{[[query]]: {and: [[tag1]] [[tag2]]}}}
|
|
@@ -169,13 +194,15 @@ Diagram definition via nested bullets or a code block child:
|
|
|
169
194
|
- A[Start] --> B{Decision}
|
|
170
195
|
- B -->|Yes| C[Action]
|
|
171
196
|
```
|
|
172
|
-
|
|
197
|
+
Per-diagram theme: first child line `%%{init: {"theme":"forest"}}%%`. Graph-wide via CSS: `:root { --mermaidjs-theme: dark; }` (in `roam/css`)
|
|
173
198
|
|
|
174
199
|
### Hiccup
|
|
175
200
|
`:hiccup [:iframe {:width "600" :height "400" :src "URL"}]`
|
|
176
201
|
|
|
177
202
|
## Advanced Components
|
|
178
203
|
|
|
204
|
+
Write the `{{[[name]]: arg}}` form. The `/name` slash commands are UI-only and do nothing in a written block string.
|
|
205
|
+
|
|
179
206
|
### Dropdowns & Tooltips
|
|
180
207
|
- **Dropdown:** `{{or: option A|option B|option C}}` — select from options, display chosen one
|
|
181
208
|
- **Tooltip:** `{{=:text|hidden content}}` — click to reveal/hide content
|
|
@@ -188,7 +215,28 @@ Theme via CSS: `:root { --mermaidjs-theme: dark; }` (in `roam/css`)
|
|
|
188
215
|
- **Datalog block query:** `{{datalog-block-query: [:find ?b :where [?b :block/string "text"]]}}` — renders results like native queries
|
|
189
216
|
- **Datalog table:** `:q [:find ?title :where [?p :node/title ?title]]` — renders results in sortable table
|
|
190
217
|
- Supports column transforms, date arithmetic, resizable columns, pagination
|
|
191
|
-
-
|
|
218
|
+
- `:q` extensions, usable **only inside a `:q` block in the graph**: date symbols `ms/today-start`, `ms/this-week-start`, `ms/+1D-start`, `dnp/today`, `dnp/-1D` (a daily-page title), `current/page-title`; inbuilt rules `(created-by ?user ?b)`, `(edited-by ?user ?b)`, `(by ?user ?b)`, `(refs-page ?title ?b)`, `(block-or-parent-refs-page ?title ?b)`, `(created-between ?t1 ?t2 ?b)`, `(edited-between ?t1 ?t2 ?b)`, `(in-dnp ?dnp ?b)`, `(refs-dnp ?dnp ?b)`, `(in-dnp-between ?start ?end ?b)`
|
|
219
|
+
|
|
220
|
+
#### `roam_datomic_query` runs plain DataScript only
|
|
221
|
+
⚠️ The `:q` extensions above do **not** work through `roam_datomic_query`. A rule such as `(refs-page "X" ?b)` fails with `Missing rules var '%'`. Write the raw clauses instead, `[?p :node/title "X"] [?b :block/refs ?p]`, and use epoch-millisecond literals for time bounds.
|
|
222
|
+
|
|
223
|
+
**Schema** (what `:where` clauses match):
|
|
224
|
+
- Pages: `:node/title`
|
|
225
|
+
- Blocks: `:block/string` (raw stored text), `:block/uid`, `:block/order`, `:block/page`, `:block/children` (immediate), `:block/parents` (all ancestors), `:block/refs` (outgoing), `:block/heading` (1–3), `:children/view-type`
|
|
226
|
+
- Both: `:create/time`, `:edit/time` (epoch ms); `:create/user`, `:edit/user` point at a user entity with `:user/uid` (every user) and `:user/display-page`
|
|
227
|
+
|
|
228
|
+
Three gotchas:
|
|
229
|
+
- **`:user/email` exists only on human users.** API-token and AI writers have none, so a join on `:user/email` silently drops their blocks. Match on `:user/uid`.
|
|
230
|
+
- **Results are unordered.** Sort client-side; `:create/time` descending for newest-first.
|
|
231
|
+
- **Recursive child pulls cap at 1000 per level** unless written `{[:block/children :limit nil] ...}`. `[*]` alone is immediate children only.
|
|
232
|
+
|
|
233
|
+
```clojure
|
|
234
|
+
;; blocks referencing a page (the portable form of refs-page)
|
|
235
|
+
[:find ?uid :where [?p :node/title "Project Alpha"] [?b :block/refs ?p] [?b :block/uid ?uid]]
|
|
236
|
+
|
|
237
|
+
;; a page and its full block tree, uncapped
|
|
238
|
+
[:find (pull ?e [* {[:block/children :limit nil] ...}]) :where [?e :node/title "Page Name"]]
|
|
239
|
+
```
|
|
192
240
|
|
|
193
241
|
### Document Mode
|
|
194
242
|
`:document` — opens inline WYSIWYG text editor in the block
|
|
@@ -199,6 +247,15 @@ Theme via CSS: `:root { --mermaidjs-theme: dark; }` (in `roam/css`)
|
|
|
199
247
|
- `{{word-count}}` — displays word count for the block
|
|
200
248
|
- `{{chart: ATTR_PAGE_TO_CHART}}` — chart component
|
|
201
249
|
- `{{a}}` — anonymous slider (shared graphs)
|
|
250
|
+
- `{{[[video]]: URL}}` — YouTube / Vimeo / Loom
|
|
251
|
+
- `{{[[mentions]]: [[Page]]}}` — inline a page's linked and unlinked references
|
|
252
|
+
- `{{date}}` — date picker that inserts a date-page ref
|
|
253
|
+
- `{{[[slider]]}}` — inline rating control, e.g. `certainty:: {{[[slider]]}}`
|
|
254
|
+
- `{{[[streak]]: [[Goal]]}}` — heatmap of how often a ref appears in daily notes
|
|
255
|
+
- `{{roam/render: ((codeUid))}}` — render a referenced code block as a component
|
|
256
|
+
- `{{[[roam/css]]}}` — a child fenced `css` block applies graph-wide styling
|
|
257
|
+
- `{{diagram: Title}}` — 2D canvas; each node is a real block
|
|
258
|
+
- `{{encrypt}}` — write it bare; Roam converts it once the user supplies content and a password
|
|
202
259
|
|
|
203
260
|
### CSS Tags
|
|
204
261
|
- `#.classname` — applies CSS class `.classname` to the block
|
|
@@ -207,10 +264,13 @@ Theme via CSS: `:root { --mermaidjs-theme: dark; }` (in `roam/css`)
|
|
|
207
264
|
| Tag | Effect |
|
|
208
265
|
|-----|--------|
|
|
209
266
|
| `#.rm-E` | Display children horizontally |
|
|
210
|
-
| `#.rm-g` |
|
|
211
|
-
| `#.rm-hide` |
|
|
267
|
+
| `#.rm-g` | Promote children up a level; `[[.rm-g]]` keeps the container visible |
|
|
268
|
+
| `#.rm-hide` | Collapse to a clickable bar in the UI; **withheld from every AI read** |
|
|
269
|
+
| `#.rm-private` | Roam's hidden-from-other-users tag; **also withheld from every AI read** |
|
|
212
270
|
| `#.rm-hide-for-readers` | Hide block for read-only users |
|
|
213
271
|
|
|
272
|
+
⚠️ `#.rm-hide` and `#.rm-private` remove the block **and its whole subtree** from what the read tools return, with no marker in the output. A page can look complete when it is not.
|
|
273
|
+
|
|
214
274
|
## Anti-Patterns
|
|
215
275
|
|
|
216
276
|
| ❌ Wrong | ✅ Correct |
|
|
@@ -229,6 +289,11 @@ Theme via CSS: `:root { --mermaidjs-theme: dark; }` (in `roam/css`)
|
|
|
229
289
|
| `{{[[query]]: {search: text}}}` | `{{[[query]]: {and: {search: text}}}}` (never standalone) |
|
|
230
290
|
| `> [[!TIP]] Title` | `[[>]] [[!TIP]] Title` |
|
|
231
291
|
| callout body as a child block | body as `\n` in the same block |
|
|
292
|
+
| `*italic*` in batch actions | `__italic__` |
|
|
293
|
+
| `#### Heading` | `### Heading` (H1–H3 only) |
|
|
294
|
+
| `1. item` as a numbered list | `children-view-type: "numbered"` on the parent |
|
|
295
|
+
| `- [ ] task` | `{{[[TODO]]}} task` |
|
|
296
|
+
| `(refs-page "X" ?b)` in `roam_datomic_query` | `[?p :node/title "X"] [?b :block/refs ?p]` |
|
|
232
297
|
|
|
233
298
|
## Tool Selection
|
|
234
299
|
|
|
@@ -288,222 +353,62 @@ Use `{{uid:name}}` for parent refs in batch actions:
|
|
|
288
353
|
```
|
|
289
354
|
Server returns `{"uid_map": {"parent": "Xk7mN2pQ9"}}`.
|
|
290
355
|
|
|
356
|
+
⚠️ `delete-block` breaks every `((uid))` that pointed at the block, and Roam has no undo for API writes. Check `roam_search_block_refs` before deleting a block others may reference.
|
|
357
|
+
|
|
291
358
|
## Structural Defaults
|
|
292
359
|
|
|
293
360
|
- **Hierarchy:** 2-4 levels preferred, rarely exceed 5
|
|
294
361
|
- **Blocks:** One idea per block
|
|
295
|
-
- **Page refs vs tags:** `[[Page]]` for expandable concepts, `#tag` for filtering
|
|
296
362
|
- **Embed vs ref:** `((uid))` inline, `{{[[embed]]: ((uid))}}` with children, `{{[[embed-children]]: ((uid))}}` children only, `{{[[embed-path]]: ((uid))}}` with ancestors, `[text](<((uid))>)` link only
|
|
297
|
-
- **No empty blocks
|
|
298
|
-
|
|
299
|
-
## Output Conventions
|
|
300
|
-
|
|
301
|
-
**Quote:** `<text> —[[Author]] #quote`
|
|
302
|
-
**Definition:** `Term:: definition #definition`
|
|
303
|
-
**Open question:** `{{[[TODO]]}} Research: <question> #[[open questions]]`
|
|
304
|
-
|
|
305
|
-
---
|
|
306
|
-
# Roam Preferences — Personalization Layer
|
|
307
|
-
|
|
308
|
-
> This section contains YOUR specific conventions, tagging philosophy, and graph-specific rules. Customize to match your workflow.
|
|
309
|
-
|
|
310
|
-
---
|
|
363
|
+
- **No empty blocks** — use hierarchy for visual separation
|
|
364
|
+
- **Never invent a `((uid))`:** use only uids a tool actually returned. A fabricated ref is a broken link.
|
|
311
365
|
|
|
312
|
-
##
|
|
366
|
+
## Conventions
|
|
313
367
|
|
|
314
|
-
|
|
315
|
-
<!-- CUSTOMIZE: What should happen when a new page is created? -->
|
|
316
|
-
- After creating a new page, add a reference block on today's daily page: `Created page: [[New Page Name]]`
|
|
317
|
-
- <!-- Add any naming conventions, required metadata, etc. -->
|
|
318
|
-
|
|
319
|
-
### On Adding Content
|
|
320
|
-
<!-- CUSTOMIZE: Any rules about where/how content gets added? -->
|
|
321
|
-
- Default location for quick captures: Daily page
|
|
322
|
-
- Long-form content: Create dedicated page, link from daily page
|
|
323
|
-
- <!-- Your preferences here -->
|
|
324
|
-
|
|
325
|
-
---
|
|
326
|
-
|
|
327
|
-
## Tagging Philosophy
|
|
328
|
-
|
|
329
|
-
### Core Principle
|
|
330
|
-
> Tag for **intellectual collision** and **future discovery**, not just categorization. Every tag should maximize potential for unexpected connections.
|
|
331
|
-
|
|
332
|
-
### The Serendipity Test
|
|
333
|
-
Before tagging, ask: *"Could this concept surprise me by connecting to something completely unrelated?"*
|
|
334
|
-
|
|
335
|
-
### What To Tag — Decision Framework
|
|
336
|
-
|
|
337
|
-
```
|
|
338
|
-
ASK YOURSELF:
|
|
339
|
-
┌─ How will Future Me find this?
|
|
340
|
-
│ └─ Tag by retrieval context, not just content
|
|
341
|
-
│
|
|
342
|
-
├─ What domain does this belong to?
|
|
343
|
-
│ └─ Use broad category tags: #[[knowledge management]], #[[decision-making]]
|
|
344
|
-
│
|
|
345
|
-
├─ Is this a proper noun?
|
|
346
|
-
│ └─ YES → Wrap name (no titles): [[Werner Erhard]], [[NASA]]
|
|
347
|
-
│ └─ For abbreviations: [NASA]([[National Aeronautics and Space Administration (NASA)]])
|
|
348
|
-
│
|
|
349
|
-
├─ Could this alias to existing page?
|
|
350
|
-
│ └─ YES → [displayed phrase]([[existing page name]])
|
|
351
|
-
│ └─ Example: [frameworks for decisions]([[decision-making frameworks]])
|
|
352
|
-
│
|
|
353
|
-
└─ Parent block with children?
|
|
354
|
-
└─ Tag parent when category applies to all children
|
|
355
|
-
└─ Tag individual children for specific categorization
|
|
356
|
-
```
|
|
357
|
-
|
|
358
|
-
### Tag Type Selection
|
|
359
|
-
|
|
360
|
-
| Use This | When |
|
|
361
|
-
|----------|------|
|
|
362
|
-
| `[[Page Reference]]` | Concept deserves its own page, will be expanded |
|
|
363
|
-
| `#[[hashtag]]` | Categorization, filtering, won't be a standalone page |
|
|
364
|
-
| `#single-word` | Simple, unambiguous category |
|
|
365
|
-
| Attribute `Type::` | Structured metadata for queries |
|
|
366
|
-
|
|
367
|
-
### WHEN creating Endnotes/Footnotes:
|
|
368
|
-
- Find/Create the block with heading "Footnotes::" and nest footnote item below. (Footnotes do not need to be on the same page as the block to which it references. Typically on the same page unless instructed otherwise.)
|
|
369
|
-
- If not known, retrieve the block_uid reference for this footnote item.
|
|
370
|
-
- In the block referencing the footnote, append the reference with footnote-item-block_id, example: "- <block_text> #ref ((block_uid))"
|
|
371
|
-
|
|
372
|
-
### Structural Tagging (Beyond Content)
|
|
373
|
-
|
|
374
|
-
Tag by **patterns and mechanisms**, not just subjects:
|
|
375
|
-
|
|
376
|
-
| Structural Tag | Connects |
|
|
377
|
-
|----------------|----------|
|
|
378
|
-
| `#[[has feedback loops]]` | Systems, habits, markets, conversations |
|
|
379
|
-
| `#[[requires calibration]]` | Instruments, relationships, AI prompts |
|
|
380
|
-
| `#[[exhibits emergence]]` | Complexity, culture, creativity |
|
|
381
|
-
| `#[[perspective switching]]` | Photography, negotiation, analysis |
|
|
382
|
-
| `#[[flow dynamics]]` | Fluids, music, conversation, sequences |
|
|
383
|
-
|
|
384
|
-
### Problem-Oriented Tagging
|
|
385
|
-
|
|
386
|
-
Tag by problems solved, not methods used:
|
|
387
|
-
|
|
388
|
-
- `#[[breaking cognitive constraints]]`
|
|
389
|
-
- `#[[expanding solution spaces]]`
|
|
390
|
-
- `#[[preventing expert blindness]]`
|
|
391
|
-
|
|
392
|
-
### Temporal & State-Based Tags
|
|
393
|
-
|
|
394
|
-
| Tag Type | Examples |
|
|
395
|
-
|----------|----------|
|
|
396
|
-
| Future relevance | `#[[will be relevant in 5 years]]`, `#[[connects to unborn projects]]` |
|
|
397
|
-
| Mental state triggers | `#[[feeling stuck in patterns]]`, `#[[needing fresh perspective]]` |
|
|
398
|
-
| Review scheduling | `[[For review]]: [[August 12th, 2026]]` |
|
|
368
|
+
This cheatsheet is syntax only. How to tag, when to use a page ref versus a hashtag, and the shape of a quote, definition, TODO, or footnote are conventions, and they belong to the graph's `[[roam/agent guidelines]]` page (returned by `roam_get_guidelines`) and the personalization layer appended below. Where a convention and this sheet appear to disagree, the convention wins on style; this sheet wins on what Roam will actually render.
|
|
399
369
|
|
|
400
370
|
---
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
###
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
###
|
|
426
|
-
When
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
-
|
|
442
|
-
-
|
|
443
|
-
-
|
|
444
|
-
|
|
445
|
-
- **Bold Attributes** - ❌ `**Attribute**::`, ✅ `Attribute::` (Roam auto-formats)
|
|
446
|
-
- **Separators** - `---` Don't use them.
|
|
447
|
-
|
|
448
|
-
### DO
|
|
449
|
-
- **Think retrieval-first** — How will you search for this later?
|
|
450
|
-
- **Cross-pollinate domains** — Force unlikely intellectual meetings
|
|
451
|
-
- **Update aging tags** — As interests evolve, so should tag vocabulary
|
|
452
|
-
- **Track surprise discoveries** — When unexpected connections yield insights, engineer more of those patterns
|
|
453
|
-
|
|
454
|
-
---
|
|
455
|
-
|
|
456
|
-
## Custom Rules
|
|
457
|
-
|
|
458
|
-
<!--
|
|
459
|
-
CUSTOMIZE THIS SECTION with your specific conventions:
|
|
460
|
-
- Naming patterns for certain page types
|
|
461
|
-
- Required attributes for books/articles/people
|
|
462
|
-
- Project-specific tagging schemes
|
|
463
|
-
- Integration rules with other tools
|
|
464
|
-
- etc.
|
|
465
|
-
-->
|
|
466
|
-
|
|
467
|
-
### Example Custom Rules (modify as needed):
|
|
468
|
-
|
|
469
|
-
**Books:**
|
|
470
|
-
```
|
|
471
|
-
[[Book/<title> | <author>]]
|
|
472
|
-
Type:: Book
|
|
473
|
-
Author:: [[Author Name]]
|
|
474
|
-
Status:: Reading | Completed | Abandoned
|
|
475
|
-
Rating:: X/5
|
|
476
|
-
```
|
|
477
|
-
|
|
478
|
-
**People:**
|
|
479
|
-
```
|
|
480
|
-
[[Person Name]]
|
|
481
|
-
Type:: Person
|
|
482
|
-
Context:: How I know them
|
|
483
|
-
```
|
|
484
|
-
- When linking bibliographic references —>
|
|
485
|
-
Example: `McAdams, D.P. (2001) [The Psychology of Life Stories](https://journals.sagepub.com/doi/10.1037/1089-2680.5.2.100) — foundational paper`
|
|
486
|
-
- [McAdams, D.P.]([[Dan McAdams]]) - author's name in the graph
|
|
487
|
-
- If source URL, link to source: [The Psychology of Life Stories](https://journals.sagepub.com/doi/10.1037/1089-2680.5.2.100)
|
|
488
|
-
- If notes page exists or will exist in Roam: append ` | [Notes]([[Article/The Psychology of Life Stories]]), if not, just leave it without link.
|
|
489
|
-
|
|
490
|
-
**Projects:**
|
|
491
|
-
```
|
|
492
|
-
[[Project/<project anme>]]
|
|
493
|
-
Status:: Active | Paused | Completed
|
|
494
|
-
Start:: [[Date]]
|
|
495
|
-
```
|
|
496
|
-
---
|
|
497
|
-
|
|
498
|
-
## Integration Notes
|
|
499
|
-
|
|
500
|
-
<!-- CUSTOMIZE: Any rules about how Roam integrates with your other tools/systems -->
|
|
501
|
-
|
|
502
|
-
- Daily pages serve as: <!-- inbox / journal / task list / etc. -->
|
|
503
|
-
- Weekly reviews occur on: <!-- day of week -->
|
|
504
|
-
- Content flows from: <!-- capture tools, read-later apps, etc. -->
|
|
505
|
-
- Content flows to: <!-- publishing, archives, etc. -->
|
|
506
|
-
|
|
507
|
-
---
|
|
508
|
-
|
|
509
|
-
*End of Personalization Layer*
|
|
371
|
+
<personalization_layer>
|
|
372
|
+
# Roam: Nova operating notes (server-side supplement)
|
|
373
|
+
|
|
374
|
+
Style conventions (voice, `#Nova` attribution, tagging, namespacing, formatting) live on the `[[roam/agent guidelines]]` page and arrive through `roam_get_guidelines`. This file carries only what that page deliberately leaves out: the mechanics behind the `#@Nova` handshake, so an interactive session produces the same artifacts as the nightly job and neither one answers a block twice.
|
|
375
|
+
|
|
376
|
+
## The `#@Nova` handshake: mechanics
|
|
377
|
+
|
|
378
|
+
Ownership rules are on the guidelines page (`#@Nova` is Ian's, `#@Nova-DONE` is Nova's, never self-tag). What follows is the output shape the nightly job writes and checks for.
|
|
379
|
+
|
|
380
|
+
### Reading the request
|
|
381
|
+
- The request is the tagged block's text with `#@Nova` removed, plus every child block that is not a `[NOVA]` reply, read as further instructions.
|
|
382
|
+
- Context is the page title, the ancestor chain, and the sibling blocks under the same parent. Fetch any URL present before answering.
|
|
383
|
+
- Empty tag line and no children means "review and respond to the surrounding context."
|
|
384
|
+
|
|
385
|
+
### Writing the reply
|
|
386
|
+
- First child of the tagged block: `[NOVA] <one-line summary, at most 150 characters>`.
|
|
387
|
+
- The rest of the reply nests under that `[NOVA]` block as ordinary blocks. Headings and bullets become nesting.
|
|
388
|
+
- Then flip the tag in place: `#@Nova` becomes `#@Nova-DONE`. Leave the rest of the block text untouched. A `{{[[TODO]]}}` on the block stays as-is; that checkbox is Ian's.
|
|
389
|
+
- Then embed on today's daily page under a `## Nova Responses` heading: a block `Nova response on [[<origin page>]]` with the child `{{[[embed]]: ((<uid of the [NOVA] block>))}}`. The guidelines' root-level rule for daily pages applies to memories; Nova responses go under this heading.
|
|
390
|
+
|
|
391
|
+
### Idempotency
|
|
392
|
+
- A block that already has a `[NOVA]` child is answered. Do not answer it again, even if the tag still reads `#@Nova`. The nightly cleanup pass removes stray duplicates; do not create them.
|
|
393
|
+
- `#@Nova-DONE` is never a work queue. Search `@Nova` for open items and skip anything containing `-DONE`.
|
|
394
|
+
|
|
395
|
+
### Who runs when (America/Los_Angeles)
|
|
396
|
+
| When | Job | Scope |
|
|
397
|
+
|---|---|---|
|
|
398
|
+
| 03:00 nightly | AI Intel sweep, `Skills/AIIntel/Tools/ProcessTriage.ts` | `#@Nova` and `#Status/*` on the `AI Intel Report` and `PAI-Anthropic Report` pages |
|
|
399
|
+
| 04:30 nightly | `Skills/Productivity/scripts/ProcessNovaTodos.ts`, EventBridge job `nova_todo_processing` | `#@Nova` on every other page |
|
|
400
|
+
| Interactive | This session, when Ian points at a block or asks for a sweep | Same output shape as above |
|
|
401
|
+
|
|
402
|
+
- Both nightly jobs post their own Telegram summary. An interactive answer does not need one.
|
|
403
|
+
- Dry run by hand: `bun $PAI_DIR/Skills/Productivity/scripts/ProcessNovaTodos.ts --dry-run`. Add `--cleanup` to only remove duplicate `[NOVA]` blocks.
|
|
404
|
+
- On AI Intel pages, `#Status/ExecSum`, `#Status/Explore`, `#Status/Backburner`, and `#Status/Reject` run unattended. `#Status/Implement`, `#Status/PRD`, and `#@Nova` wait for an interactive session. Processed items end as `#Status/Processed`.
|
|
405
|
+
|
|
406
|
+
## Retired conventions
|
|
407
|
+
- `#PAI/do`, `#PAI/done`, `#PAI/hold`, `#PAI/ignore`: replaced by `#@Nova` and `#@Nova-DONE`. Nothing processes them and no block carried them as of September 6th, 2026. Never create them. If one turns up, treat it as `#@Nova`.
|
|
408
|
+
- The `| [[Date]]` suffix once used to surface nested results on the daily page: replaced by the embed above.
|
|
409
|
+
|
|
410
|
+
## Ian's capture tags (not Nova's queue)
|
|
411
|
+
- `{{[[TODO]]}} #task <action>` on the daily page is Ian's capture inbox. ProcessFlow routes it to OmniFocus weekly. Do not convert `#task` to `#@Nova` unprompted; Ian decides what to delegate.
|
|
412
|
+
- `#researchThis` means look into this later. It is not a task and not a delegation.
|
|
413
|
+
- `#suggested-next-action` is a follow-up Nova proposes under a reply. Ian adds `#@Nova` to delegate it or checks it `{{[[DONE]]}}` to close it himself.
|
|
414
|
+
</personalization_layer>
|
|
@@ -1,26 +1,49 @@
|
|
|
1
|
+
import { escapeBlockString, needsNewlineEscaping, ESCAPED_NEWLINES_MARKER } from '../../shared/block-escaping.js';
|
|
1
2
|
/**
|
|
2
|
-
* Convert RoamBlock hierarchy to markdown with proper indentation
|
|
3
|
+
* Convert RoamBlock hierarchy to markdown with proper indentation.
|
|
4
|
+
*
|
|
5
|
+
* `escape` mirrors `PageOperations.fetchPageByTitle`'s markdown branch: a
|
|
6
|
+
* block may contain a soft line break (Shift+Enter), and this renderer emits
|
|
7
|
+
* one `- ` line per block, so an unescaped newline spills onto a second
|
|
8
|
+
* physical line with no bullet, resetting the parser's indentation baseline
|
|
9
|
+
* and reparenting everything after it. Callers that render a full page
|
|
10
|
+
* compute whether escaping is needed and set this flag; callers that never
|
|
11
|
+
* feed their output back through `updatePageMarkdown` can leave it off.
|
|
3
12
|
*/
|
|
4
|
-
export function blocksToMarkdown(blocks, level = 0) {
|
|
13
|
+
export function blocksToMarkdown(blocks, level = 0, escape = false) {
|
|
5
14
|
return blocks
|
|
6
15
|
.map(block => {
|
|
7
16
|
const indent = ' '.repeat(level);
|
|
8
17
|
let md;
|
|
18
|
+
const text = escape ? escapeBlockString(block.string) : block.string;
|
|
9
19
|
// Check block heading level and format accordingly
|
|
10
20
|
if (block.heading && block.heading > 0) {
|
|
11
21
|
const hashtags = '#'.repeat(block.heading);
|
|
12
|
-
md = `${indent}${hashtags} ${
|
|
22
|
+
md = `${indent}${hashtags} ${text}`;
|
|
13
23
|
}
|
|
14
24
|
else {
|
|
15
|
-
md = `${indent}- ${
|
|
25
|
+
md = `${indent}- ${text}`;
|
|
16
26
|
}
|
|
17
27
|
if (block.children && block.children.length > 0) {
|
|
18
|
-
md += '\n' + blocksToMarkdown(block.children, level + 1);
|
|
28
|
+
md += '\n' + blocksToMarkdown(block.children, level + 1, escape);
|
|
19
29
|
}
|
|
20
30
|
return md;
|
|
21
31
|
})
|
|
22
32
|
.join('\n');
|
|
23
33
|
}
|
|
34
|
+
/**
|
|
35
|
+
* Collect every block string in a tree, for deciding whether a page needs
|
|
36
|
+
* newline escaping at all (see `blocksToMarkdown`).
|
|
37
|
+
*/
|
|
38
|
+
function collectBlockStrings(blocks, out = []) {
|
|
39
|
+
for (const block of blocks) {
|
|
40
|
+
out.push(block.string);
|
|
41
|
+
if (block.children && block.children.length > 0) {
|
|
42
|
+
collectBlockStrings(block.children, out);
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
return out;
|
|
46
|
+
}
|
|
24
47
|
/**
|
|
25
48
|
* Flatten block hierarchy to single-level list
|
|
26
49
|
*/
|
|
@@ -42,7 +65,11 @@ export function formatPageOutput(title, blocks, options) {
|
|
|
42
65
|
return JSON.stringify({ title, children: data }, null, 2);
|
|
43
66
|
}
|
|
44
67
|
const displayBlocks = options.flat ? flattenBlocks(blocks) : blocks;
|
|
45
|
-
|
|
68
|
+
const escape = needsNewlineEscaping(collectBlockStrings(displayBlocks));
|
|
69
|
+
const body = blocksToMarkdown(displayBlocks, 0, escape);
|
|
70
|
+
return escape
|
|
71
|
+
? `# ${title}\n${ESCAPED_NEWLINES_MARKER}\n\n${body}`
|
|
72
|
+
: `# ${title}\n\n${body}`;
|
|
46
73
|
}
|
|
47
74
|
/**
|
|
48
75
|
* Format block content for output
|
|
@@ -53,7 +80,9 @@ export function formatBlockOutput(block, options) {
|
|
|
53
80
|
return JSON.stringify(data, null, 2);
|
|
54
81
|
}
|
|
55
82
|
const displayBlocks = options.flat ? flattenBlocks([block]) : [block];
|
|
56
|
-
|
|
83
|
+
const escape = needsNewlineEscaping(collectBlockStrings(displayBlocks));
|
|
84
|
+
const body = blocksToMarkdown(displayBlocks, 0, escape);
|
|
85
|
+
return escape ? `${ESCAPED_NEWLINES_MARKER}\n\n${body}` : body;
|
|
57
86
|
}
|
|
58
87
|
/**
|
|
59
88
|
* Format search results for output
|
package/build/markdown-utils.js
CHANGED
|
@@ -111,13 +111,18 @@ function convertToRoamMarkdown(text) {
|
|
|
111
111
|
* That was a real, unrecoverable defect: `- wrap it in ``` to make code`
|
|
112
112
|
* followed by three blocks parsed to a single block "wrap it in", and
|
|
113
113
|
* roam_update_page_markdown deleted the other three. Roam has no undo.
|
|
114
|
+
*
|
|
115
|
+
* This subsumes the earlier balanced-fence guard: a line carrying both an
|
|
116
|
+
* opening and a closing fence necessarily has content after the opener, so it
|
|
117
|
+
* never matches this shape.
|
|
114
118
|
*/
|
|
115
119
|
function isBulletFenceOpener(trimmedLine) {
|
|
116
120
|
return /^\s*[-*+]\s+```[A-Za-z0-9_+-]*\s*$/.test(trimmedLine);
|
|
117
121
|
}
|
|
118
122
|
/**
|
|
119
123
|
* A fence line with content after its opening ``` is content, not a region
|
|
120
|
-
* opener. Guards the bare (non-bullet) case the splice rule cannot see
|
|
124
|
+
* opener. Guards the bare (non-bullet) case that the splice rule cannot see,
|
|
125
|
+
* e.g. a line rendered as ```js\ncode\n``` with no bullet prefix.
|
|
121
126
|
*/
|
|
122
127
|
function fenceHasTrailingContent(trimmedLine) {
|
|
123
128
|
const open = trimmedLine.indexOf('```');
|
|
@@ -135,12 +140,14 @@ function parseMarkdown(markdown) {
|
|
|
135
140
|
const codeStartIndex = trimmedLine.indexOf('```');
|
|
136
141
|
if (codeStartIndex > 0 && isBulletFenceOpener(trimmedLine)) {
|
|
137
142
|
// Under this rule the text before the fence is ALWAYS just the bullet
|
|
138
|
-
// marker
|
|
139
|
-
//
|
|
140
|
-
//
|
|
141
|
-
//
|
|
142
|
-
//
|
|
143
|
-
//
|
|
143
|
+
// marker (isBulletFenceOpener only matches "bullet + nothing but the
|
|
144
|
+
// fence"), so there is no real content to preserve as its own node.
|
|
145
|
+
// Pushing it anyway used to leave a bare "-" line that the parser can't
|
|
146
|
+
// recognise as a bullet once trimmed (no trailing content survives
|
|
147
|
+
// trimEnd), so it fell through to the plain-line branch and emitted a
|
|
148
|
+
// spurious "-" block ahead of the code block it introduces. Dropping it
|
|
149
|
+
// loses nothing: the fence line below carries the same leading
|
|
150
|
+
// whitespace, so indentation-based nesting is unaffected.
|
|
144
151
|
const indentationWhitespace = line.match(/^\s*/)?.[0] ?? '';
|
|
145
152
|
processedLines.push(indentationWhitespace + trimmedLine.substring(codeStartIndex));
|
|
146
153
|
}
|
|
@@ -331,7 +331,7 @@ export class RoamServer {
|
|
|
331
331
|
res.setHeader('Access-Control-Allow-Origin', '*');
|
|
332
332
|
}
|
|
333
333
|
res.setHeader('Access-Control-Allow-Methods', 'GET, POST, DELETE, OPTIONS');
|
|
334
|
-
res.setHeader('Access-Control-Allow-Headers', 'Content-Type, Authorization, Mcp-Session-Id');
|
|
334
|
+
res.setHeader('Access-Control-Allow-Headers', 'Content-Type, Authorization, Mcp-Session-Id, MCP-Protocol-Version, Last-Event-ID');
|
|
335
335
|
res.setHeader('Access-Control-Expose-Headers', 'Mcp-Session-Id');
|
|
336
336
|
res.setHeader('Access-Control-Allow-Credentials', 'true');
|
|
337
337
|
// Handle preflight OPTIONS requests
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Encoding a Roam block string so it occupies exactly one line of markdown.
|
|
3
|
+
*
|
|
4
|
+
* A Roam block may contain a newline — a soft line break (Shift+Enter). Our
|
|
5
|
+
* markdown renderers emit one `- ` line per block, so an unescaped newline
|
|
6
|
+
* spills onto a second physical line with no bullet and no indentation. The
|
|
7
|
+
* parser then reads it as a separate block AT ROOT LEVEL, which resets the
|
|
8
|
+
* indentation baseline and reparents everything after it. A single soft line
|
|
9
|
+
* break flattens the rest of the page.
|
|
10
|
+
*
|
|
11
|
+
* Keeping each block on one line is therefore not cosmetic: it is what makes
|
|
12
|
+
* the round trip preserve hierarchy at all.
|
|
13
|
+
*
|
|
14
|
+
* A soft break is rendered as a single sentinel character, U+23CE `⏎`, rather
|
|
15
|
+
* than a backslash escape. Two earlier revisions used `\n` (with backslash
|
|
16
|
+
* doubling to disambiguate a literal backslash-n); both corrupted ordinary
|
|
17
|
+
* content, because `\n` is a common PREFIX in authored text — LaTeX commands
|
|
18
|
+
* (`\nabla`, `\neq`) and Windows paths (`C:\new…`) all begin with it, so
|
|
19
|
+
* decoding `\n` anywhere corrupted them. `⏎` essentially never occurs in
|
|
20
|
+
* authored text, so decoding it is safe even for blocks the agent wrote
|
|
21
|
+
* itself. No backslash rules exist at all any more.
|
|
22
|
+
*
|
|
23
|
+
* See `docs/multiline-block-roundtrip-spec.md`.
|
|
24
|
+
*/
|
|
25
|
+
/**
|
|
26
|
+
* The single-line stand-in for a soft line break: U+23CE RETURN SYMBOL.
|
|
27
|
+
*
|
|
28
|
+
* Revision 2 used `\n` escapes with backslash doubling. That failed twice:
|
|
29
|
+
* `\n` is a common PREFIX in ordinary content (`\nabla`, `\neq`, `C:\new…`),
|
|
30
|
+
* so decoding it anywhere an agent may have authored text corrupted LaTeX and
|
|
31
|
+
* Windows paths — even the marker could not make it safe per-block. `⏎`
|
|
32
|
+
* essentially never occurs in authored text, so decoding it inside a marked
|
|
33
|
+
* payload is safe INCLUDING for blocks the agent wrote. No backslash rules
|
|
34
|
+
* exist at all any more.
|
|
35
|
+
*
|
|
36
|
+
* The accepted corner: a block genuinely containing a literal `⏎` round-trips
|
|
37
|
+
* it into a newline. Content-level, degrades to a soft break, documented, and
|
|
38
|
+
* pinned by a test so it stays a choice rather than an accident.
|
|
39
|
+
*/
|
|
40
|
+
export const SOFT_BREAK_SENTINEL = '⏎';
|
|
41
|
+
/** Render a block string as a single line. Identity for newline-free text. */
|
|
42
|
+
export function escapeBlockString(text) {
|
|
43
|
+
return text.includes('\n') ? text.replace(/\n/g, SOFT_BREAK_SENTINEL) : text;
|
|
44
|
+
}
|
|
45
|
+
/** Restore soft line breaks. Call ONLY on marker-carrying payloads. */
|
|
46
|
+
export function unescapeBlockString(text) {
|
|
47
|
+
return text.includes(SOFT_BREAK_SENTINEL)
|
|
48
|
+
? text.replace(/⏎/g, '\n')
|
|
49
|
+
: text;
|
|
50
|
+
}
|
|
51
|
+
/**
|
|
52
|
+
* Marks a rendered payload whose block strings were escaped.
|
|
53
|
+
*
|
|
54
|
+
* Revision 1 decoded `\n` unconditionally, which corrupted every LaTeX command
|
|
55
|
+
* and Windows path that begins `\n` — `$$\nabla f$$` became `$$<newline>abla
|
|
56
|
+
* f$$`. An escape sequence built from characters that occur in ordinary content
|
|
57
|
+
* can only be decoded where the ENCODER is known to have run. This marker is
|
|
58
|
+
* how the decoder knows.
|
|
59
|
+
*
|
|
60
|
+
* An HTML comment on purpose: inert everywhere, and harmless if it ever lands
|
|
61
|
+
* in a block by accident.
|
|
62
|
+
*/
|
|
63
|
+
export const ESCAPED_NEWLINES_MARKER = '<!-- roam:escaped-newlines -->';
|
|
64
|
+
/**
|
|
65
|
+
* Does this page need escaping at all?
|
|
66
|
+
*
|
|
67
|
+
* Escaping is conditional so a page with no multi-line block renders exactly
|
|
68
|
+
* as it did before any of this work — no marker, no backslash doubling. That
|
|
69
|
+
* is the overwhelming majority of pages, and it keeps the blast radius of the
|
|
70
|
+
* encoding near zero.
|
|
71
|
+
*/
|
|
72
|
+
export function needsNewlineEscaping(blockStrings) {
|
|
73
|
+
return blockStrings.some((s) => s.includes('\n'));
|
|
74
|
+
}
|
|
@@ -4,6 +4,7 @@ import { getPageUid as getPageUidHelper } from '../helpers/page-resolution.js';
|
|
|
4
4
|
import { resolveRefs } from '../helpers/refs.js';
|
|
5
5
|
import { fetchChildrenByDepth } from '../helpers/fetch-children.js';
|
|
6
6
|
import { collectHiddenUids, pruneHiddenBlocks, isHiddenBlockString } from '../helpers/hidden.js';
|
|
7
|
+
import { escapeBlockString } from '../../shared/block-escaping.js';
|
|
7
8
|
export class FullPageViewOperations {
|
|
8
9
|
constructor(graph, pageOps) {
|
|
9
10
|
this.graph = graph;
|
|
@@ -107,7 +108,11 @@ export class FullPageViewOperations {
|
|
|
107
108
|
});
|
|
108
109
|
}
|
|
109
110
|
const linkedReferenceGroups = Array.from(groupMap.values());
|
|
110
|
-
// 7. Render as markdown
|
|
111
|
+
// 7. Render as markdown. This output is display-only — never valid
|
|
112
|
+
// `roam_update_page_markdown` input, and nothing decodes it — so
|
|
113
|
+
// `renderBlocks` applies the `⏎` sentinel to every block string
|
|
114
|
+
// unconditionally (it's the identity for newline-free text) rather than
|
|
115
|
+
// gating on a per-page predicate.
|
|
111
116
|
return this.renderMarkdown(title, pageBlocks, linkedReferenceGroups, truncated ? allUniqueRefs.length : undefined);
|
|
112
117
|
}
|
|
113
118
|
// ─── Private: fetch all blocks that reference this page ──────────────────────
|
|
@@ -268,11 +273,17 @@ export class FullPageViewOperations {
|
|
|
268
273
|
// This mirrors Roam's ancestor context display
|
|
269
274
|
for (let i = 0; i < ref.breadcrumbs.length; i++) {
|
|
270
275
|
const prefix = '> '.repeat(i + 1);
|
|
271
|
-
|
|
276
|
+
// Same reasoning as `renderBlocks` below: a breadcrumb string can
|
|
277
|
+
// itself carry a soft line break, and an unescaped one spills onto
|
|
278
|
+
// a bare physical line with no `> ` prefix at all.
|
|
279
|
+
lines.push(`${prefix}${escapeBlockString(ref.breadcrumbs[i].string)}`);
|
|
272
280
|
}
|
|
273
|
-
// The referring block itself, indented to sit visually under its
|
|
281
|
+
// The referring block itself, indented to sit visually under its
|
|
282
|
+
// breadcrumbs. Escaped for the same reason `renderBlocks` escapes
|
|
283
|
+
// every other block string in this file: an unescaped newline spills
|
|
284
|
+
// the rest of the block onto a bare physical line with no bullet.
|
|
274
285
|
const refIndent = ' '.repeat(ref.breadcrumbs.length);
|
|
275
|
-
lines.push(`${refIndent}- ${ref.block.string}`);
|
|
286
|
+
lines.push(`${refIndent}- ${escapeBlockString(ref.block.string)}`);
|
|
276
287
|
// Children of the referring block
|
|
277
288
|
if (ref.block.children.length > 0) {
|
|
278
289
|
lines.push(this.renderBlocks(ref.block.children, ref.breadcrumbs.length + 1));
|
|
@@ -286,13 +297,17 @@ export class FullPageViewOperations {
|
|
|
286
297
|
renderBlocks(blocks, baseIndent) {
|
|
287
298
|
const renderBlock = (block, depth) => {
|
|
288
299
|
const indent = ' '.repeat(depth);
|
|
300
|
+
// Unconditional: `escapeBlockString` is the identity for newline-free
|
|
301
|
+
// text, so pages with no soft line breaks render byte-identical to how
|
|
302
|
+
// they did before this feature existed — no predicate needed.
|
|
303
|
+
const text = escapeBlockString(block.string);
|
|
289
304
|
let line;
|
|
290
305
|
if (block.heading && block.heading > 0) {
|
|
291
306
|
const hashes = '#'.repeat(block.heading);
|
|
292
|
-
line = `${indent}${hashes} ${
|
|
307
|
+
line = `${indent}${hashes} ${text}`;
|
|
293
308
|
}
|
|
294
309
|
else {
|
|
295
|
-
line = `${indent}- ${
|
|
310
|
+
line = `${indent}- ${text}`;
|
|
296
311
|
}
|
|
297
312
|
const childLines = block.children.map(c => renderBlock(c, depth + 1)).join('\n');
|
|
298
313
|
return childLines ? `${line}\n${childLines}` : line;
|
|
@@ -7,10 +7,11 @@ import { executeBatch, executeBatchSafe } from '../helpers/batch-utils.js';
|
|
|
7
7
|
import { pruneHiddenBlocks } from '../helpers/hidden.js';
|
|
8
8
|
import { convertToRoamMarkdown, generateBlockUid } from '../../markdown-utils.js';
|
|
9
9
|
import { executeStagedBatch } from '../../shared/staged-batch.js';
|
|
10
|
+
import { escapeBlockString, unescapeBlockString, needsNewlineEscaping, ESCAPED_NEWLINES_MARKER, SOFT_BREAK_SENTINEL } from '../../shared/block-escaping.js';
|
|
10
11
|
import { pageUidCache } from '../../cache/page-uid-cache.js';
|
|
11
12
|
import { buildTableActions } from './table.js';
|
|
12
13
|
import { BatchOperations } from './batch.js';
|
|
13
|
-
import { parseExistingBlocks, pruneHiddenExistingBlocks, countHiddenExistingBlocks, markdownToBlocks, diffBlockTrees, generateBatchActions, getDiffStats, isDiffEmpty, summarizeActions, } from '../../diff/index.js';
|
|
14
|
+
import { parseExistingBlocks, pruneHiddenExistingBlocks, countHiddenExistingBlocks, flattenExistingBlocks, markdownToBlocks, diffBlockTrees, generateBatchActions, getDiffStats, isDiffEmpty, summarizeActions, } from '../../diff/index.js';
|
|
14
15
|
// Helper to get ordinal suffix for dates
|
|
15
16
|
function getOrdinalSuffix(day) {
|
|
16
17
|
if (day > 3 && day < 21)
|
|
@@ -419,7 +420,17 @@ export class PageOperations {
|
|
|
419
420
|
// renders a page (markdown, raw, structure) inherits it.
|
|
420
421
|
return { title, blocks: pruneHiddenBlocks(rootBlocks) };
|
|
421
422
|
}
|
|
422
|
-
async fetchPageByTitle(title, format = 'raw'
|
|
423
|
+
async fetchPageByTitle(title, format = 'raw',
|
|
424
|
+
/**
|
|
425
|
+
* Encode newlines so each block is one line (`shared/block-escaping.ts`).
|
|
426
|
+
* Required by anything whose output may be fed back to
|
|
427
|
+
* `roam_update_page_markdown`; wrong for anything shown as prose.
|
|
428
|
+
*
|
|
429
|
+
* Defaults to OFF so a caller that has not considered this renders today's
|
|
430
|
+
* output rather than silently acquiring doubled backslashes. `guidelines.ts`
|
|
431
|
+
* relies on that default.
|
|
432
|
+
*/
|
|
433
|
+
options = {}) {
|
|
423
434
|
if (!title) {
|
|
424
435
|
throw new McpError(ErrorCode.InvalidRequest, 'title is required');
|
|
425
436
|
}
|
|
@@ -592,21 +603,36 @@ export class PageOperations {
|
|
|
592
603
|
await Promise.all(visibleBlocks.map(async (b) => {
|
|
593
604
|
b.string = await resolveRefs(this.graph, b.string);
|
|
594
605
|
}));
|
|
606
|
+
// Collect every visible block string to decide whether this page needs the
|
|
607
|
+
// encoding at all. A page with no soft line break renders exactly as it
|
|
608
|
+
// did before this feature existed.
|
|
609
|
+
const allStrings = [];
|
|
610
|
+
const collectStrings = (blocks) => {
|
|
611
|
+
for (const b of blocks) {
|
|
612
|
+
allStrings.push(b.string);
|
|
613
|
+
collectStrings(b.children);
|
|
614
|
+
}
|
|
615
|
+
};
|
|
616
|
+
collectStrings(visibleRoots);
|
|
617
|
+
const escaping = options.escapeNewlines === true && needsNewlineEscaping(allStrings);
|
|
595
618
|
// Convert to markdown with proper nesting
|
|
596
619
|
const toMarkdown = (blocks, level = 0) => {
|
|
597
620
|
return blocks
|
|
598
621
|
.map(block => {
|
|
599
622
|
const indent = ' '.repeat(level);
|
|
600
623
|
let md;
|
|
624
|
+
const text = escaping
|
|
625
|
+
? escapeBlockString(block.string)
|
|
626
|
+
: block.string;
|
|
601
627
|
// Check block heading level and format accordingly
|
|
602
628
|
if (block.heading && block.heading > 0) {
|
|
603
629
|
// Format as heading with appropriate number of hashtags
|
|
604
630
|
const hashtags = '#'.repeat(block.heading);
|
|
605
|
-
md = `${indent}${hashtags} ${
|
|
631
|
+
md = `${indent}${hashtags} ${text}`;
|
|
606
632
|
}
|
|
607
633
|
else {
|
|
608
634
|
// No heading, use bullet point (current behavior)
|
|
609
|
-
md = `${indent}- ${
|
|
635
|
+
md = `${indent}- ${text}`;
|
|
610
636
|
}
|
|
611
637
|
if (block.children.length > 0) {
|
|
612
638
|
md += '\n' + toMarkdown(block.children, level + 1);
|
|
@@ -615,7 +641,10 @@ export class PageOperations {
|
|
|
615
641
|
})
|
|
616
642
|
.join('\n');
|
|
617
643
|
};
|
|
618
|
-
|
|
644
|
+
const body = toMarkdown(visibleRoots);
|
|
645
|
+
return escaping
|
|
646
|
+
? `# ${title}\n${ESCAPED_NEWLINES_MARKER}\n\n${body}`
|
|
647
|
+
: `# ${title}\n\n${body}`;
|
|
619
648
|
}
|
|
620
649
|
/**
|
|
621
650
|
* Update an existing page with new markdown content using smart diff.
|
|
@@ -664,7 +693,108 @@ export class PageOperations {
|
|
|
664
693
|
const hiddenCount = countHiddenExistingBlocks(allExistingBlocks);
|
|
665
694
|
const existingBlocks = pruneHiddenExistingBlocks(allExistingBlocks);
|
|
666
695
|
// 4. Convert new markdown to block structure
|
|
667
|
-
|
|
696
|
+
//
|
|
697
|
+
// Decode ONLY when our own renderer said it encoded. The marker may be
|
|
698
|
+
// the first non-empty line (header stripped by the caller) or the first
|
|
699
|
+
// non-empty line after a single leading `#` header (payload submitted
|
|
700
|
+
// verbatim — the path `roam save --update` takes). Revision 2 checked
|
|
701
|
+
// only the first line; a verbatim submit therefore never decoded, wrote
|
|
702
|
+
// the marker as a block, and deleted the blocks it rewrote. The tests
|
|
703
|
+
// that should have caught it stripped the header themselves.
|
|
704
|
+
const lines = markdown.split('\n');
|
|
705
|
+
let markerAt = -1;
|
|
706
|
+
let nonEmptySeen = 0;
|
|
707
|
+
for (let i = 0; i < lines.length; i++) {
|
|
708
|
+
const t = lines[i].trim();
|
|
709
|
+
if (t.length === 0)
|
|
710
|
+
continue;
|
|
711
|
+
nonEmptySeen++;
|
|
712
|
+
if (t === ESCAPED_NEWLINES_MARKER) {
|
|
713
|
+
markerAt = i;
|
|
714
|
+
break;
|
|
715
|
+
}
|
|
716
|
+
// A single leading header line may precede the marker; anything else
|
|
717
|
+
// (or a second line that is not the marker) means an unmarked payload.
|
|
718
|
+
if (nonEmptySeen === 1 && t.startsWith('#'))
|
|
719
|
+
continue;
|
|
720
|
+
break;
|
|
721
|
+
}
|
|
722
|
+
const isEscaped = markerAt !== -1;
|
|
723
|
+
// `fetchPageByTitle`'s markdown branch always prepends `# ${title}\n` --
|
|
724
|
+
// it is page-level metadata describing what page this is, never content.
|
|
725
|
+
// Tolerating the marker after that header (above) is not enough on its
|
|
726
|
+
// own: a verbatim round trip still hands that literal `# Title` line to
|
|
727
|
+
// the parser, which turns it into a real heading block with no match in
|
|
728
|
+
// the existing tree, so the diff creates it and reparents every sibling
|
|
729
|
+
// after it -- exactly the "no-op round trip" this marker exists to
|
|
730
|
+
// guarantee. Strip it whenever the first non-empty line matches this
|
|
731
|
+
// page's own title exactly -- but ONLY when the payload demonstrably came
|
|
732
|
+
// from our own renderer (`isEscaped`, or the body still carries the
|
|
733
|
+
// `⏎` sentinel -- the signature of a marker-dropped degradation, since
|
|
734
|
+
// that scenario is "an agent rebuilt our output and lost the marker" and
|
|
735
|
+
// the sentinel is what that rebuilding could not have removed). A FRESH,
|
|
736
|
+
// hand-authored page can legitimately open with an H1 that echoes its own
|
|
737
|
+
// title (imported docs; an author who titles their own first line) --
|
|
738
|
+
// essentially never with `⏎` in it. The two wrong calls are not
|
|
739
|
+
// symmetric: preserving the header when it should have been stripped
|
|
740
|
+
// creates a harmless, visible stray block; stripping it when it should
|
|
741
|
+
// not have been touched deletes -- or reparents into corruption -- real,
|
|
742
|
+
// unrecoverable content, with no undo. The gate always takes the harmless
|
|
743
|
+
// direction whenever the payload carries no provenance signal.
|
|
744
|
+
const trimmedTitle = String(title).trim();
|
|
745
|
+
const firstNonEmptyAt = lines.findIndex((l) => l.trim().length > 0);
|
|
746
|
+
const hasRendererProvenance = isEscaped || markdown.includes(SOFT_BREAK_SENTINEL);
|
|
747
|
+
const titleHeaderAt = hasRendererProvenance &&
|
|
748
|
+
firstNonEmptyAt !== -1 &&
|
|
749
|
+
lines[firstNonEmptyAt].trim() === `# ${trimmedTitle}`
|
|
750
|
+
? firstNonEmptyAt
|
|
751
|
+
: -1;
|
|
752
|
+
let effectiveMarkdown = markdown;
|
|
753
|
+
const linesToStrip = [isEscaped ? markerAt : -1, titleHeaderAt]
|
|
754
|
+
.filter((i) => i !== -1)
|
|
755
|
+
.sort((a, b) => b - a);
|
|
756
|
+
if (linesToStrip.length > 0) {
|
|
757
|
+
// Strip the marker line itself, or it becomes a stray block on the
|
|
758
|
+
// page. Descending order so removing one index never shifts the other.
|
|
759
|
+
for (const idx of linesToStrip)
|
|
760
|
+
lines.splice(idx, 1);
|
|
761
|
+
effectiveMarkdown = lines.join('\n');
|
|
762
|
+
}
|
|
763
|
+
// Decode AFTER parsing, per block -- never on the whole document before
|
|
764
|
+
// parsing. Decoding the document first would turn every `\n` into a real
|
|
765
|
+
// newline and then split on newlines, which is exactly the flattening
|
|
766
|
+
// bug this whole effort exists to fix.
|
|
767
|
+
const newBlocks = markdownToBlocks(effectiveMarkdown, pageUid);
|
|
768
|
+
// Submitting empty/whitespace markdown is the documented way to clear a
|
|
769
|
+
// page, and stays untouched below. This guards the DIFFERENT case: markdown
|
|
770
|
+
// that is NOT empty but still parsed to zero blocks. That only happens when
|
|
771
|
+
// the parser swallowed the payload -- the known cause is a first block
|
|
772
|
+
// whose string is a bare fence opener (e.g. a line reading "- ```js" with
|
|
773
|
+
// nothing to close it), which is exactly the shape our own renderer emits
|
|
774
|
+
// for a Roam block whose string starts with a fence, and the shape a
|
|
775
|
+
// pasted code snippet produces. `markdownToBlocks` then returns an empty
|
|
776
|
+
// array, and diffing 0 new blocks against N existing ones deletes all N --
|
|
777
|
+
// silently, against an API with no undo. The asymmetry is deliberate: an
|
|
778
|
+
// empty parse of EMPTY input is the documented clear-the-page instruction;
|
|
779
|
+
// an empty parse of NON-EMPTY input is the parser losing the payload, never
|
|
780
|
+
// an instruction to delete anything.
|
|
781
|
+
if (effectiveMarkdown.trim().length > 0 && newBlocks.length === 0) {
|
|
782
|
+
const existingCount = flattenExistingBlocks(existingBlocks).length;
|
|
783
|
+
if (existingCount > 0) {
|
|
784
|
+
throw new McpError(ErrorCode.InvalidParams, `markdown parsed to zero blocks even though it is not empty. This usually ` +
|
|
785
|
+
`means an unterminated \`\`\` fence swallowed the whole payload (a first ` +
|
|
786
|
+
`block that opens a code fence and never closes it consumes every line ` +
|
|
787
|
+
`after it). Refusing to write: this would have deleted all ${existingCount} ` +
|
|
788
|
+
`existing block${existingCount === 1 ? '' : 's'} on "${title}". Check the ` +
|
|
789
|
+
`markdown for an unbalanced fence, or call again with dry_run: true to ` +
|
|
790
|
+
`inspect the planned actions before writing.`);
|
|
791
|
+
}
|
|
792
|
+
}
|
|
793
|
+
if (isEscaped) {
|
|
794
|
+
for (const block of newBlocks) {
|
|
795
|
+
block.text = unescapeBlockString(block.text);
|
|
796
|
+
}
|
|
797
|
+
}
|
|
668
798
|
// 5. Compute diff
|
|
669
799
|
const diff = diffBlockTrees(existingBlocks, newBlocks, pageUid);
|
|
670
800
|
// 6. Generate ordered batch actions
|
|
@@ -41,7 +41,9 @@ export class ToolHandlers {
|
|
|
41
41
|
return this.pageOps.createPage(title, content);
|
|
42
42
|
}
|
|
43
43
|
async fetchPageByTitle(title, format) {
|
|
44
|
-
|
|
44
|
+
// The MCP read tool's markdown may be handed straight back to
|
|
45
|
+
// roam_update_page_markdown, so it must be round-trippable.
|
|
46
|
+
return this.pageOps.fetchPageByTitle(title, format, { escapeNewlines: true });
|
|
45
47
|
}
|
|
46
48
|
async fetchPageFullView(title, children_depth, max_references) {
|
|
47
49
|
return this.fullPageViewOps.fetchPageFullView(title, children_depth, max_references);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "roam-research-mcp",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "4.0.1",
|
|
4
4
|
"description": "MCP server and CLI for Roam Research",
|
|
5
5
|
"private": false,
|
|
6
6
|
"repository": {
|
|
@@ -31,6 +31,7 @@
|
|
|
31
31
|
"scripts": {
|
|
32
32
|
"build": "echo \"Using custom instructions: .roam/${CUSTOM_INSTRUCTIONS_PREFIX}custom-instructions.md\" && tsc && cat Roam_Markdown_Cheatsheet.md .roam/${CUSTOM_INSTRUCTIONS_PREFIX}custom-instructions.md > build/Roam_Markdown_Cheatsheet.md && chmod 755 build/index.js build/cli/roam.js",
|
|
33
33
|
"clean": "rm -rf build",
|
|
34
|
+
"prepublishOnly": "node scripts/check-bundled-cheatsheet.mjs",
|
|
34
35
|
"watch": "tsc --watch",
|
|
35
36
|
"inspector": "npx @modelcontextprotocol/inspector build/index.js",
|
|
36
37
|
"start": "node build/index.js",
|
|
@@ -45,7 +46,7 @@
|
|
|
45
46
|
"test:watch": "vitest"
|
|
46
47
|
},
|
|
47
48
|
"dependencies": {
|
|
48
|
-
"@modelcontextprotocol/sdk": "
|
|
49
|
+
"@modelcontextprotocol/sdk": "1.25.1",
|
|
49
50
|
"@roam-research/roam-api-sdk": "^0.10.0",
|
|
50
51
|
"commander": "^14.0.2",
|
|
51
52
|
"dotenv": "^16.4.7"
|