roam-research-mcp 3.1.0 → 3.2.0

package/README.md

@@ -183,6 +183,14 @@ This is distinct from `CUSTOM_INSTRUCTIONS_PATH`, and the two compose:
 
 If the page doesn't exist, the tool returns `exists: false` rather than failing, so it is always safe to call.
 
+### It also returns the rules that aren't yours to set
+
+Alongside your conventions, every `roam_get_guidelines` response carries a `roamSyntax` field: the short list of things that *destroy* content — `roam_update_page_markdown` deleting every block your markdown omits, truncated `structure` previews written back as if they were content, block references retyped as plain text — plus a caution that reads silently exclude `#.rm-hide` subtrees, and the handful of places Roam's markdown inverts standard markdown.
+
+Two reasons it rides here rather than in the cheatsheet. It reaches **every** client, including one that never calls `roam_markdown_cheatsheet`; and it is returned even when a graph has **no** guidelines page, which is exactly the case where an agent has least context. The layering is deliberate: **your conventions win on style, `roamSyntax` wins on data safety.** No convention can make a truncated preview complete.
+
+The full syntax reference — components, queries, embeds, tool selection — stays in `roam_markdown_cheatsheet`. `roamSyntax` is ~800 tokens and deliberately capped.
+
 Each graph can point at a different page, or turn it off:
 
 ```bash
@@ -210,6 +218,8 @@ This follows the same convention as Roam's official MCP server, so a block tagge
 
 Applied to: `roam_fetch_page_by_title`, `roam_fetch_block`, `roam_fetch_page_full_view`, `roam_get_subpages`, `roam_search_by_text`, `roam_search_for_tag`, `roam_search_by_status`, `roam_search_block_refs`, `roam_search_hierarchy`, `roam_search_by_date`.
 
+**Hidden blocks are also excluded from the page-rewrite diff**, which is what stops them being *deleted* for being absent from markdown the agent could not have written. `roam_update_page_markdown` (and `roam save --update`) replaces a page with what you give it, deleting whatever your markdown omits — so its baseline is pruned by this same filter, on the rule that **the baseline a diff deletes from must be the same page the caller was allowed to read.** It reports `preserved_hidden` when it protected anything. Content is preserved; exact ordering relative to visible siblings may shift. This was a real data-loss bug before the fix — see the [changelog](CHANGELOG.md).
+
 **This is a convenience filter, not a security guarantee.** `roam_datomic_query` reads the database directly and deliberately does **not** apply it, so a capable agent can still surface hidden blocks through raw Datalog. Treat these tags as "keep it out of the AI's way," not "keep it secret."
 
 Tag matching is case-insensitive, and only exact tags match — `#.rm-hidden` and `#.rm-highlight` are left alone. The set of hidden UIDs is cached for 30 seconds, so a block tagged just now may remain visible for up to that long.

package/build/Roam_Markdown_Cheatsheet.md

@@ -1,4 +1,4 @@
-# Roam Markdown Cheatsheet v2.3.0
+# Roam Markdown Cheatsheet v2.4.0
 
 ## Core Syntax
 
@@ -32,6 +32,36 @@ Always ordinal format: `[[January 1st, 2025]]`, `[[December 23rd, 2024]]`
 - Todo: `{{[[TODO]]}} task`
 - Done: `{{[[DONE]]}} task`
 
+### Callouts
+Styled blockquotes with an icon and colour. Two page refs open the block: `[[>]]` marks it a callout, `[[!TYPE]]` picks the style.
+
+```
+[[>]] [[!TIP]] Title text
+Body on the next line
+```
+
+The body is a **soft line break inside the same block** (Shift+Enter in the UI, `\n` in the block string) — not a child block. A child block renders as a nested bullet inside the callout instead, which is usually not what you want.
+
+Types: `NOTE` `INFO` `SUMMARY` `TIP` `SUCCESS` `QUESTION` `WARNING` `FAILURE` `DANGER` `BUG` `EXAMPLE` `QUOTE`
+
+Append `+` or `-` to make it foldable — `[[!TIP]]+` starts expanded, `[[!TIP]]-` starts collapsed.
+
+⚠️ `[[>]]` and `[[!TIP]]` are real page references, so every callout backlinks to those pages. That is normal and how the feature works — don't "clean it up."
+⚠️ A plain `> quote` is an ordinary blockquote, not a callout. The two are unrelated.
+
+### ⚠️ Multi-line blocks and page rewrites
+
+A block can hold a **soft line break** (Shift+Enter, stored as `\n` in the block
+string). Callout bodies require one; fenced code blocks are full of them.
+
+**Do not run `roam_update_page_markdown` — or `roam save --update` — on a page
+containing a callout, a fenced code block, or any Shift+Enter line break.** It
+will split those blocks and **flatten the hierarchy of everything after them**,
+reparenting blocks under the wrong ancestors.
+
+Use `roam_process_batch_actions` for those pages. It writes block strings
+literally, and is currently the only way to create a soft line break at all.
+
 ### Attributes
 ```
 Type:: Book
@@ -67,8 +97,32 @@ const x = 1;
 {{[[query]]: {or: [[A]] [[B]]}}}
 {{[[query]]: {not: [[exclude]]}}}
 {{[[query]]: {between: [[January 1st, 2025]] [[January 31st, 2025]]}}}
+{{[[query]]: {and: [[Project]] {search: mobile}}}}
 ```
 
+Clauses nest: `{and: [[Project]] {not: [[DONE]]}}`.
+
+**Queries match REFERENCES, not text.** Operands must be `[[Page]]` or `((block-uid))` — bare or quoted words do not match. `{and: TODO}` and `{and: "project alpha"}` both find nothing; write `{and: [[TODO]]}` and `{and: [[project alpha]]}`. For free text use `roam_search_by_text`, or a `{search:}` clause.
+
+**`{search:}` only works nested inside `{and:}` or `{or:}`** — never on its own, and it is the one clause that takes plain text rather than a reference.
+
+**`{between:}` only works on Daily Notes pages.** It filters by the daily page a block lives on, so it does nothing for content on ordinary pages. It accepts shorthands: `[[today]]`, `[[yesterday]]`, `[[last week]]`, `[[next month]]`.
+
+Also available: `{created-by: [[User]]}`, `{edited-by: [[User]]}`, `{by: [[User]]}`.
+
+#### Page-ref inheritance — the non-obvious one
+**A block inherits its parent's page refs for query matching.** So this TODO matches `{and: [[TODO]] [[Project Alpha]]}` even though it contains no reference to Project Alpha:
+
+```
+- Notes on [[Project Alpha]]
+    - {{[[TODO]]}} Ship the thing
+```
+
+Three consequences:
+- **Don't tag every child with the parent's ref** — it is already inherited, and the duplication just clutters the backlinks.
+- **Do tag a child explicitly** if you want it to match *independently* of where it sits. Move it later and inherited matching goes with the old parent.
+- **Reading results:** a returned block may not visibly contain the thing you queried for. The matching ref can be on an ancestor. Don't report the result as wrong, and don't "fix" the block by adding the tag.
+
 ### Calculator
 `{{[[calc]]: 2 + 2}}`
 
@@ -170,6 +224,11 @@ Theme via CSS: `:root { --mermaidjs-theme: dark; }` (in `roam/css`)
 | `- *bullet` | `- bullet` |
 | `* bullet` | `- bullet` |
 | `**Attr**:: val` | `Attr:: val` |
+| `{and: TODO}` | `{and: [[TODO]]}` (queries match refs, not words) |
+| `{and: "project alpha"}` | `{and: [[project alpha]]}` |
+| `{{[[query]]: {search: text}}}` | `{{[[query]]: {and: {search: text}}}}` (never standalone) |
+| `> [[!TIP]] Title` | `[[>]] [[!TIP]] Title` |
+| callout body as a child block | body as `\n` in the same block |
 
 ## Tool Selection
 

package/build/diff/index.js

@@ -34,7 +34,7 @@
  */
 export { getDiffStats, isDiffEmpty } from './types.js';
 // Parser
-export { parseExistingBlock, parseExistingBlocks, flattenExistingBlocks, markdownToBlocks, getBlockDepth, } from './parser.js';
+export { parseExistingBlock, parseExistingBlocks, pruneHiddenExistingBlocks, countHiddenExistingBlocks, flattenExistingBlocks, markdownToBlocks, getBlockDepth, } from './parser.js';
 // Matcher
 export { normalizeText, normalizeForMatching, matchBlocks, groupByParent } from './matcher.js';
 // Diff

package/build/diff/parser.js

@@ -5,6 +5,7 @@
  * and provides utilities for flattening block trees.
  */
 import { generateBlockUid, parseMarkdown } from '../markdown-utils.js';
+import { isHiddenBlockString } from '../tools/helpers/hidden.js';
 /**
  * Parse a raw Roam API block into an ExistingBlock structure.
  * Recursively processes children and sorts them by order.
@@ -38,6 +39,56 @@ export function parseExistingBlocks(pageData) {
     const childrenSorted = [...childrenRaw].sort((a, b) => (a[':block/order'] ?? 0) - (b[':block/order'] ?? 0));
     return childrenSorted.map((c) => parseExistingBlock(c, null));
 }
+/**
+ * Drop `#.rm-hide` / `#.rm-private` subtrees from a diff baseline.
+ *
+ * THE RULE THIS ENFORCES: **the baseline a diff deletes from must be the same
+ * page the caller was allowed to read.**
+ *
+ * Every read path filters these subtrees out, so an agent composing replacement
+ * markdown cannot include what it was never shown. The diff then compared that
+ * markdown against the *unfiltered* page, found the hidden blocks unaccounted
+ * for, and deleted them — turning "hide this from the AI" into "let the AI
+ * delete this", with no undo. Pruning the baseline makes the two views agree:
+ * a block that is invisible is also unmatched-against, so it is never a
+ * deletion candidate.
+ *
+ * WHAT THIS DOES NOT PRESERVE: position. A surviving hidden block keeps its
+ * original `:block/order`, while the new blocks are numbered from the markdown,
+ * so a hidden block can end up sharing an order with a visible sibling and
+ * settle either side of it. Ordering is a sort key, not content — and the
+ * alternative, reserving slots for blocks the caller cannot see, would let
+ * hidden content dictate visible layout. Content survives; exact position may
+ * not.
+ *
+ * @param blocks - Parsed baseline, straight from `parseExistingBlocks`
+ * @returns The same trees with hidden blocks and their descendants removed
+ */
+export function pruneHiddenExistingBlocks(blocks) {
+    const kept = [];
+    for (const block of blocks) {
+        // The whole subtree goes: a visible child of a hidden parent is still
+        // content the caller never saw, so it is equally undeletable.
+        if (isHiddenBlockString(block.text))
+            continue;
+        kept.push({ ...block, children: pruneHiddenExistingBlocks(block.children) });
+    }
+    return kept;
+}
+/** How many blocks `pruneHiddenExistingBlocks` would remove, counting subtrees. */
+export function countHiddenExistingBlocks(blocks) {
+    let count = 0;
+    for (const block of blocks) {
+        if (isHiddenBlockString(block.text)) {
+            // The block plus everything under it, all of which is being protected.
+            count += 1 + flattenExistingBlocks(block.children).length;
+        }
+        else {
+            count += countHiddenExistingBlocks(block.children);
+        }
+    }
+    return count;
+}
 /**
  * Flatten a tree of existing blocks into a single array.
  * Preserves parent-child relationships through parentUid property.

package/build/markdown-utils.js

@@ -100,6 +100,31 @@ function convertToRoamMarkdown(text) {
     });
     return text;
 }
+/**
+ * Is this line a bullet whose only content is a code-fence opener?
+ *
+ * Only such a line may be spliced into "bullet" + "fence" so the fence state
+ * machine can gather the following lines. Any line with content AFTER the
+ * fence is a block that merely CONTAINS backticks — splicing it opens a region
+ * that never closes, and the parser then consumes the rest of the document.
+ *
+ * That was a real, unrecoverable defect: `- wrap it in ``` to make code`
+ * followed by three blocks parsed to a single block "wrap it in", and
+ * roam_update_page_markdown deleted the other three. Roam has no undo.
+ */
+function isBulletFenceOpener(trimmedLine) {
+    return /^\s*[-*+]\s+```[A-Za-z0-9_+-]*\s*$/.test(trimmedLine);
+}
+/**
+ * A fence line with content after its opening ``` is content, not a region
+ * opener. Guards the bare (non-bullet) case the splice rule cannot see.
+ */
+function fenceHasTrailingContent(trimmedLine) {
+    const open = trimmedLine.indexOf('```');
+    if (open === -1)
+        return false;
+    return trimmedLine.slice(open + 3).replace(/^[A-Za-z0-9_+-]*/, '').trim().length > 0;
+}
 function parseMarkdown(markdown) {
     markdown = convertToRoamMarkdown(markdown);
     const originalLines = markdown.split('\n');
@@ -108,9 +133,15 @@ function parseMarkdown(markdown) {
     for (const line of originalLines) {
         const trimmedLine = line.trimEnd();
         const codeStartIndex = trimmedLine.indexOf('```');
-        if (codeStartIndex > 0) {
+        if (codeStartIndex > 0 && isBulletFenceOpener(trimmedLine)) {
+            // Under this rule the text before the fence is ALWAYS just the bullet
+            // marker, so there is no real content to preserve as its own node.
+            // Pushing it anyway left a bare "-" line that the parser could not
+            // recognise as a bullet once trimmed, so it emitted a spurious "-" block
+            // ahead of the code block it introduces. Dropping it loses nothing: the
+            // fence line below carries the same leading whitespace, so
+            // indentation-based nesting is unaffected.
             const indentationWhitespace = line.match(/^\s*/)?.[0] ?? '';
-            processedLines.push(indentationWhitespace + trimmedLine.substring(0, codeStartIndex));
             processedLines.push(indentationWhitespace + trimmedLine.substring(codeStartIndex));
         }
         else {
@@ -123,7 +154,7 @@ function parseMarkdown(markdown) {
     let inCodeBlockFirstPass = false;
     for (const line of processedLines) {
         const trimmedLine = line.trimEnd();
-        if (trimmedLine.match(/^(\s*)```/)) {
+        if (trimmedLine.match(/^(\s*)```/) && !fenceHasTrailingContent(trimmedLine)) {
             inCodeBlockFirstPass = !inCodeBlockFirstPass;
             if (!inCodeBlockFirstPass)
                 continue; // Skip closing ```
@@ -176,7 +207,7 @@ function parseMarkdown(markdown) {
     for (let i = 0; i < processedLines.length; i++) {
         const line = processedLines[i];
         const trimmedLine = line.trimEnd();
-        if (trimmedLine.match(/^(\s*)```/)) {
+        if (trimmedLine.match(/^(\s*)```/) && !fenceHasTrailingContent(trimmedLine)) {
             if (!inCodeBlock) {
                 inCodeBlock = true;
                 codeBlockContent = trimmedLine.trimStart() + '\n';

package/build/tools/operations/guidelines.js

@@ -10,10 +10,16 @@
  * server's mechanics and lives in a file. Guidelines are per-graph, live-edited
  * from inside Roam, and answer "how does this user want their graph handled".
  *
+ * The result also carries `roamSyntax` (see `../roam-syntax.ts`) on every path,
+ * whether or not a guidelines page exists. Conventions are the user's to supply
+ * and may be absent; the data-safety rules are the server's and never are.
+ *
  * Read by default; set `guidelinesPage: false` on a graph to disable it there.
+ * Disabling suppresses the user's conventions, not `roamSyntax`.
  */
 import { PageOperations } from './pages.js';
 import { formatRoamDate } from '../../utils/helpers.js';
+import { ROAM_SYNTAX } from '../roam-syntax.js';
 /** The shared convention, read by default and by Roam's own MCP server. */
 export const DEFAULT_GUIDELINES_PAGE = 'roam/agent guidelines';
 /**
@@ -43,8 +49,9 @@ export class GuidelinesOperations {
                 page: null,
                 exists: false,
                 guidelines: null,
+                roamSyntax: ROAM_SYNTAX,
                 todaysDailyNote: today,
-                nextSteps: 'Guidelines are disabled for this graph. Proceed using the Roam Markdown Cheatsheet for syntax.',
+                nextSteps: 'Guidelines are disabled for this graph. Follow `roamSyntax` below, and load the Roam Markdown Cheatsheet for anything it does not cover.',
             };
         }
         const perGraph = cache.get(this.graph) ?? new Map();
@@ -60,9 +67,11 @@ export class GuidelinesOperations {
                     page: this.guidelinesPage,
                     exists: false,
                     guidelines: null,
+                    roamSyntax: ROAM_SYNTAX,
                     todaysDailyNote: today,
                     nextSteps: `No "${this.guidelinesPage}" page exists in this graph, so there are no user conventions to follow. ` +
-                        `Do not call this tool again for this graph this session. Proceed using the Roam Markdown Cheatsheet for syntax. ` +
+                        `Do not call this tool again for this graph this session. The \`roamSyntax\` rules below still apply — they are about data safety, not convention. ` +
+                        `Load the Roam Markdown Cheatsheet for syntax they do not cover. ` +
                         `The user can create the page at any time to set conventions.`,
                 };
             }
@@ -75,9 +84,11 @@ export class GuidelinesOperations {
                     page: this.guidelinesPage,
                     exists: true,
                     guidelines,
+                    roamSyntax: ROAM_SYNTAX,
                     todaysDailyNote: today,
                     nextSteps: `You now have this graph's conventions. Do not call this tool again for this graph this session — you already have what you need. ` +
                         `Apply these conventions to reads as well as writes: they change how results should be interpreted and presented, not just how content is written. ` +
+                        `Where they are silent, the \`roamSyntax\` rules govern; where they conflict, conventions win on style and \`roamSyntax\` wins on data safety. ` +
                         `Today's daily note is "${today}".`,
                 };
             }
@@ -88,9 +99,12 @@ export class GuidelinesOperations {
                 page: this.guidelinesPage,
                 exists: false,
                 guidelines: null,
+                roamSyntax: ROAM_SYNTAX,
                 todaysDailyNote: today,
                 nextSteps: `Could not read "${this.guidelinesPage}" (${error instanceof Error ? error.message : String(error)}). ` +
-                    `Proceed using the Roam Markdown Cheatsheet for syntax.`,
+                    `The graph's conventions are unavailable, so be conservative about placement and style. ` +
+                    `The \`roamSyntax\` rules below are unaffected — they ship with the server. ` +
+                    `Load the Roam Markdown Cheatsheet for syntax they do not cover.`,
             };
         }
         perGraph.set(this.guidelinesPage, { at: Date.now(), result });

package/build/tools/operations/pages.js

@@ -10,7 +10,7 @@ import { executeStagedBatch } from '../../shared/staged-batch.js';
 import { pageUidCache } from '../../cache/page-uid-cache.js';
 import { buildTableActions } from './table.js';
 import { BatchOperations } from './batch.js';
-import { parseExistingBlocks, markdownToBlocks, diffBlockTrees, generateBatchActions, getDiffStats, isDiffEmpty, summarizeActions, } from '../../diff/index.js';
+import { parseExistingBlocks, pruneHiddenExistingBlocks, countHiddenExistingBlocks, markdownToBlocks, diffBlockTrees, generateBatchActions, getDiffStats, isDiffEmpty, summarizeActions, } from '../../diff/index.js';
 // Helper to get ordinal suffix for dates
 function getOrdinalSuffix(day) {
     if (day > 3 && day < 21)
@@ -524,12 +524,30 @@ export class PageOperations {
             return JSON.stringify(visibleRoots);
         }
         if (format === 'structure') {
+            // Flatten the tree into a list optimized for surgical updates: this
+            // format exists to hand an agent the UIDs and shape of a page cheaply, so
+            // `text` is a PREVIEW, cut at PREVIEW_CHARS.
+            //
+            // That cut is the format's one sharp edge. The tool description sells the
+            // output as "optimized for surgical updates", and the obvious next move —
+            // feed these entries to roam_process_batch_actions as update-block
+            // strings — silently replaces every long block with its own first 80
+            // characters. The `...` suffix was the only signal, and an agent
+            // reassembling content does not reliably read punctuation as a warning.
+            //
+            // So a cut entry now says so in a field: `truncated: true`, plus
+            // `full_length` so the agent can see how much is missing. The payload
+            // also carries a one-line instruction, but ONLY when something was
+            // actually cut — a warning present on every response is a warning that
+            // gets skimmed. Widening the cut, or removing it, would change what an
+            // unchanged call returns; marking it does not.
+            const PREVIEW_CHARS = 80;
             const flattenBlocks = (blocks, depth, parentUid) => {
                 const result = [];
                 for (const block of blocks) {
-                    // Truncate text for preview (keep first 80 chars)
-                    const preview = block.string.length > 80
-                        ? block.string.substring(0, 80) + '...'
+                    const isTruncated = block.string.length > PREVIEW_CHARS;
+                    const preview = isTruncated
+                        ? block.string.substring(0, PREVIEW_CHARS) + '...'
                         : block.string;
                     const entry = {
                         uid: block.uid,
@@ -538,6 +556,10 @@ export class PageOperations {
                         depth,
                         parent_uid: parentUid
                     };
+                    if (isTruncated) {
+                        entry.truncated = true;
+                        entry.full_length = block.string.length;
+                    }
                     if (block.heading) {
                         entry.heading = block.heading;
                     }
@@ -550,10 +572,19 @@ export class PageOperations {
                 return result;
             };
             const structureBlocks = flattenBlocks(visibleRoots, 0, uid);
+            const truncatedCount = structureBlocks.filter((b) => b.truncated).length;
             return JSON.stringify({
                 page_uid: uid,
                 title: title,
                 block_count: structureBlocks.length,
+                ...(truncatedCount > 0 && {
+                    truncated_count: truncatedCount,
+                    warning: `${truncatedCount} block${truncatedCount === 1 ? '' : 's'} shown here ` +
+                        `${truncatedCount === 1 ? 'is' : 'are'} cut off at ${PREVIEW_CHARS} characters ` +
+                        `(marked \`truncated: true\`). Their \`text\` is a preview for orientation, not content. ` +
+                        `Writing it back would replace the block with its opening fragment — ` +
+                        `fetch the block with roam_fetch_block to get its full string before editing it.`
+                }),
                 blocks: structureBlocks
             });
         }
@@ -621,7 +652,17 @@ export class PageOperations {
             throw new McpError(ErrorCode.InternalError, `Failed to fetch page data for "${title}"`);
         }
         // 3. Parse existing blocks into our format
-        const existingBlocks = parseExistingBlocks(pageData);
+        const allExistingBlocks = parseExistingBlocks(pageData);
+        // 3a. Withhold #.rm-hide / #.rm-private subtrees from the BASELINE, not
+        // just from reads. The query above deliberately pulls the whole page —
+        // block UIDs and ordering have to be complete for the diff to preserve
+        // references — but diffing against blocks the caller was never shown is
+        // how the hide filter became a deletion mechanism: unseen content cannot
+        // appear in replacement markdown, and this diff deletes whatever the
+        // markdown does not account for. The baseline must match what could be
+        // read. See `pruneHiddenExistingBlocks`.
+        const hiddenCount = countHiddenExistingBlocks(allExistingBlocks);
+        const existingBlocks = pruneHiddenExistingBlocks(allExistingBlocks);
         // 4. Convert new markdown to block structure
         const newBlocks = markdownToBlocks(markdown, pageUid);
         // 5. Compute diff
@@ -643,12 +684,23 @@ export class PageOperations {
                 throw new McpError(ErrorCode.InternalError, `Failed to apply changes: ${error instanceof Error ? error.message : String(error)}`);
             }
         }
+        // Report the protection rather than applying it silently. Without this, a
+        // caller told "3 blocks deleted, 5 created" has no way to explain the
+        // blocks still on the page afterwards, and a user debugging that has
+        // nothing to go on. It is a count, never content — and these tags are
+        // documented as "keep it out of the AI's way", not a secrecy boundary
+        // (`tools/helpers/hidden.ts`), so a count is well within their contract.
+        const preservationNote = hiddenCount > 0
+            ? ` ${hiddenCount} hidden block${hiddenCount === 1 ? '' : 's'} ` +
+                `(#.rm-hide / #.rm-private) ${hiddenCount === 1 ? 'was' : 'were'} excluded from the diff and left untouched.`
+            : '';
         return {
             success: true,
             actions,
             stats,
             preserved_uids: [...diff.preservedUids],
-            summary: dryRun ? `[DRY RUN] ${summary}` : summary
+            ...(hiddenCount > 0 && { preserved_hidden: hiddenCount }),
+            summary: (dryRun ? `[DRY RUN] ${summary}` : summary) + preservationNote
         };
     }
     /**

package/build/tools/roam-syntax.js

@@ -0,0 +1,65 @@
+/**
+ * ROAM_SYNTAX — the data-safety and syntax rules an agent needs before its
+ * first write, returned by `roam_get_guidelines` as the `roamSyntax` field.
+ *
+ * WHY IT LIVES ON THE GUIDELINES RESULT. Every write tool's description already
+ * says "load the Roam Markdown Cheatsheet", but that is a second, voluntary
+ * tool call, and a model under budget pressure skips it. The guidelines call is
+ * the one we successfully insist on. Anything that must reach every client —
+ * including clients with no skill installed and no cheatsheet fetch — has to
+ * ride on a response the agent already asked for.
+ *
+ * SCOPE. This is not the cheatsheet. `Roam_Markdown_Cheatsheet.md` is the
+ * complete syntax reference (components, queries, embeds, CSS tags) and stays
+ * the place to look things up. This blob is only the subset where being wrong
+ * DESTROYS something — content, references, or the user's trust in the graph —
+ * and it stays short enough that its cost is never the reason to skip it.
+ *
+ * ORDERING IS DELIBERATE. Sections run in descending order of how much damage
+ * getting them wrong does, not in taxonomic order, and the closing line repeats
+ * the destructive cases. Models attend hardest to the beginning and end of a
+ * blob, so the irreversible things occupy both.
+ *
+ * LAYERING. The user's own `[[roam/agent guidelines]]` page governs style and
+ * convention and can override anything about voice, tagging, or placement. It
+ * cannot override the facts below: no convention makes a truncated preview
+ * complete, or brings back a block that a whole-page rewrite deleted.
+ *
+ * PRIOR ART. The idea of returning a compact syntax blob from the guidelines
+ * call, the damage-ranked ordering, and the closing checksum are borrowed from
+ * Roam Research's own MCP server (`@roam-research/roam-mcp`, the `roamSyntax`
+ * field of its `get_graph_guidelines`, added 2026-08). That project publishes
+ * no licence, so nothing here is copied from it: the text below is written for
+ * this server's tools and describes this server's failure modes, which are not
+ * the same ones — it talks to Roam's backend REST API, not Roam Desktop, and
+ * has no `<roam/>` wire format to teach.
+ *
+ * WHEN EDITING: this is the canonical home for the load-bearing warnings. If a
+ * fact here also appears in the cheatsheet, change it here first;
+ * `roam-syntax.test.ts` pins the invariants that must not quietly disappear.
+ */
+/**
+ * The blob itself. Assembled from an array so each section is independently
+ * readable in source and the joins can't drift.
+ */
+export const ROAM_SYNTAX = [
+    'ROAM DATA SAFETY — read before writing. Roam is an outliner, its markdown is not standard markdown, and it has no undo for API writes. The graph guidelines above govern style and convention; the rules below govern data integrity and hold regardless of them.',
+    '',
+    'DESTRUCTIVE: WHOLE-PAGE REWRITES DELETE. `roam_update_page_markdown` diffs the markdown you pass against the ENTIRE existing page and deletes every block your markdown does not account for. It is not an append. Pass the complete intended page, or use `roam_process_batch_actions` / `roam_create_outline` to touch only what you mean to touch. Use `dry_run: true` first when unsure — it returns the actions without applying them.',
+    '',
+    'DESTRUCTIVE: PREVIEWS ARE NOT CONTENT. `roam_fetch_page_by_title` with `format: "structure"` cuts each block at 80 characters and marks the entry `truncated: true`. That text is for orientation only. Writing it back replaces the block with its own opening fragment. Re-read the block with `roam_fetch_block` before editing any text you got from a preview.',
+    '',
+    'DESTRUCTIVE: REFERENCES ARE LIVE — DO NOT RETYPE THEM. `((block-uid))` renders the referenced block; replacing it with the text it displayed turns a live reference into a stale copy, and the link is not recoverable from the result. Preserve `((uid))` spans exactly as read, and never invent a uid — use only uids a tool actually returned to you.',
+    '',
+    'YOUR READS ARE INCOMPLETE. Blocks tagged `#.rm-hide` or `#.rm-private`, and everything nested under them, are withheld from every read tool, and nothing in the output marks the gap — a page can look complete when it is not. This will not destroy anything: those subtrees are excluded from the `roam_update_page_markdown` diff too, so a rewrite leaves them alone (it reports `preserved_hidden` when it does). But do not claim a page contains only what you were shown, and be careful answering "is X here?" — you cannot tell absence from concealment.',
+    '',
+    'FORMATTING (differs from standard markdown). Italics is `__text__`; bold is `**text**` — the two are swapped relative to standard markdown, so `__x__` written as bold silently renders italic. Highlight `^^text^^`, strikethrough `~~text~~`. A task is `{{[[TODO]]}}` at the START of the block, never `- [ ]`, or it will not toggle. Headings are `#`/`##`/`###` only, H1–H3. Nest by indenting: a heading does not pull the blocks after it underneath itself.',
+    '',
+    'LINKS MINT PAGES. `[[Name]]` and `#Name` both create the page if it does not exist, so link deliberately rather than bracketing every noun. Multi-word tags need brackets: `#[[two words]]`, not `#twowords`. Because `#` always makes a page, a bare `#1` creates a page named "1" — write `Step 1` or quote it. Dates are ordinal page names: `[[January 3rd, 2026]]`.',
+    '',
+    'ATTRIBUTES. `Key:: value` at the start of a block creates a queryable attribute. Roam bolds the key itself, so write `Type:: Book`, never `**Type**:: Book`. Use `::` only for values you would query across the graph; otherwise write a bold label.',
+    '',
+    'ESCAPING. Markup you write goes live immediately. When your text DOCUMENTS syntax rather than using it, wrap it in backticks — an unescaped `[[example]]` in a note about linking creates a real page and a real backlink. This is how an agent quietly pollutes a graph.',
+    '',
+    'BEFORE EVERY WRITE: is this a whole-page rewrite that will delete what I did not include; did I get this text from a truncated preview; am I retyping a `((uid))` reference instead of preserving it; is the syntax I am documenting inside backticks?',
+].join('\n');

package/build/tools/schemas.js

@@ -163,7 +163,7 @@ export const toolSchemas = {
     roam_get_guidelines: {
         name: 'roam_get_guidelines',
         annotations: READ,
-        description: 'Retrieve this graph\'s user-defined agent conventions, read from the `[[roam/agent guidelines]]` page inside the graph (configurable per graph). These are the user\'s own rules — how they tag, how they name and namespace pages, what to never do, how they want your voice attributed.\n\nCall this ONCE per graph per session, before other tools, INCLUDING for reads: conventions change how results should be interpreted and presented, not just how content is written. Returns today\'s daily note title as orientation.\n\nDistinct from `roam_markdown_cheatsheet`, which covers Roam syntax and this server\'s mechanics. Guidelines answer "how does this user want their graph handled". Returns exists:false rather than failing when no page has been created.',
+        description: 'Retrieve this graph\'s user-defined agent conventions, read from the `[[roam/agent guidelines]]` page inside the graph (configurable per graph). These are the user\'s own rules — how they tag, how they name and namespace pages, what to never do, how they want your voice attributed.\n\nAlso returns `roamSyntax`: the rules whose violation destroys content — whole-page rewrites that delete, truncated previews written back as content, retyped block references, and the syntax that differs from standard markdown. These are returned on every call, including when the graph has no guidelines page, and they hold regardless of what the conventions say.\n\nCall this ONCE per graph per session, before other tools, INCLUDING for reads: conventions change how results should be interpreted and presented, not just how content is written. Returns today\'s daily note title as orientation.\n\nDistinct from `roam_markdown_cheatsheet`, which is the complete syntax reference — components, queries, embeds, tool selection. Call that when you need to look something up; this one you need before writing at all. Returns exists:false rather than failing when no page has been created.',
         inputSchema: {
             type: 'object',
             properties: withMultiGraphParams({}),
@@ -204,7 +204,7 @@ export const toolSchemas = {
                     type: 'string',
                     enum: ['markdown', 'raw', 'structure'],
                     default: 'raw',
-                    description: "Format output as markdown, JSON, or structure. 'markdown' returns readable string; 'raw' returns full JSON with nested blocks; 'structure' returns flattened list optimized for surgical updates (uid, order, text preview, depth, parent_uid)"
+                    description: "Format output as markdown, JSON, or structure. 'markdown' returns readable string; 'raw' returns full JSON with nested blocks; 'structure' returns a flattened list (uid, order, text, depth, parent_uid) for locating blocks to update. In 'structure', `text` is a PREVIEW cut at 80 characters — an entry marked `truncated: true` is a fragment, and writing it back would replace the block with its own opening. Use it to find the uid, then fetch that block with roam_fetch_block before editing its text."
                 }
             }),
             required: ['title']
@@ -616,7 +616,7 @@ export const toolSchemas = {
     roam_markdown_cheatsheet: {
         name: 'roam_markdown_cheatsheet',
         annotations: READ,
-        description: 'Provides the comprehensive Roam syntax reference. Covers: formatting, links & references (page refs, block refs, embeds including embed-children and embed-path), tags, dates, tasks, attributes, queries (native and :q Datalog tables with built-in rules), tables, kanban, mermaid diagrams (with theme support), advanced components (dropdowns, tooltips, templates, document mode, word-count), CSS tags (#.rm-E, #.rm-hide, etc.), anti-patterns, tool selection guide, and API efficiency tips.\n\n**IMPORTANT:** Always load this cheatsheet before creating or updating Roam content. It prevents common syntax errors and guides tool selection.\n\nIMPORTANT: call roam_get_guidelines for this graph once per session before using this tool, reads included — conventions change how results are read, not just written.',
+        description: 'Provides the comprehensive Roam syntax reference. Covers: formatting, links & references (page refs, block refs, embeds including embed-children and embed-path), tags, dates, tasks, callouts, attributes, queries (native `{{query}}` with its clause rules and page-ref inheritance, plus :q Datalog tables with built-in rules), tables, kanban, mermaid diagrams (with theme support), advanced components (dropdowns, tooltips, templates, document mode, word-count), CSS tags (#.rm-E, #.rm-hide, etc.), anti-patterns, tool selection guide, and API efficiency tips.\n\n**IMPORTANT:** Always load this cheatsheet before creating or updating Roam content. It prevents common syntax errors and guides tool selection.\n\nIMPORTANT: call roam_get_guidelines for this graph once per session before using this tool, reads included — conventions change how results are read, not just written.',
         inputSchema: {
             type: 'object',
             properties: withMultiGraphParams({}),
@@ -906,9 +906,13 @@ export const toolSchemas = {
                 items: { type: 'string' },
                 description: 'Blocks whose UIDs survived the diff, so refs to them still resolve'
             },
+            preserved_hidden: {
+                type: 'number',
+                description: 'Present only when non-zero: how many #.rm-hide / #.rm-private blocks were excluded from the diff and left on the page untouched'
+            },
             summary: { type: 'string' }
         }, ['success', 'actions', 'stats', 'preserved_uids', 'summary']),
-        description: 'Update an existing page with new markdown content using smart diff. Preserves block UIDs where possible and generates minimal changes. This is ideal for:\n- Syncing external markdown files to Roam\n- AI-assisted content updates that preserve references\n- Batch content modifications without losing block references\n\n**How it works:**\n1. Fetches existing page blocks\n2. Matches new content to existing blocks by text similarity\n3. Generates minimal create/update/move/delete operations\n4. Preserves UIDs for matched blocks (keeping references intact)\n\n\nIMPORTANT: call roam_get_guidelines for this graph once per session, and load the Roam Markdown Cheatsheet, before using this tool.',
+        description: 'Update an existing page with new markdown content using smart diff. Preserves block UIDs where possible and generates minimal changes. This is ideal for:\n- Syncing external markdown files to Roam\n- AI-assisted content updates that preserve references\n- Batch content modifications without losing block references\n\n**⚠️ This REPLACES the page, it does not append.** Any block your markdown does not account for is deleted. Pass the complete intended page, or use `roam_process_batch_actions` / `roam_create_outline` to change only part of one. Use `dry_run: true` to see the actions first.\n\n**How it works:**\n1. Fetches existing page blocks\n2. Matches new content to existing blocks by text similarity\n3. Generates minimal create/update/move/delete operations\n4. Preserves UIDs for matched blocks (keeping references intact)\n\n`#.rm-hide` / `#.rm-private` subtrees are excluded from the diff and left untouched — you cannot see them, so you cannot be asked to account for them. `preserved_hidden` reports how many, when any.\n\nIMPORTANT: call roam_get_guidelines for this graph once per session, and load the Roam Markdown Cheatsheet, before using this tool.',
         inputSchema: {
             type: 'object',
             properties: withMultiGraphParams({

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "roam-research-mcp",
-  "version": "3.1.0",
+  "version": "3.2.0",
   "description": "MCP server and CLI for Roam Research",
   "private": false,
   "repository": {