mdld-parse 0.9.9 → 1.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. package/README.md +263 -326
  2. package/dist/index.js +1860 -1337
  3. package/package.json +4 -4
  4. package/dist/.nojekyll +0 -0
  5. package/dist/CNAME +0 -1
  6. package/dist/README.md +0 -500
  7. package/dist/docs/API.md +0 -435
  8. package/dist/docs/Architecture.md +0 -190
  9. package/dist/docs/Guide.md +0 -816
  10. package/dist/docs/LLM-MDLD-Inducer.md +0 -550
  11. package/dist/docs/LiterateRDF.md +0 -472
  12. package/dist/docs/Parser.md +0 -720
  13. package/dist/docs/Performance.md +0 -238
  14. package/dist/docs/Spec-render.md +0 -1439
  15. package/dist/docs/Subject.md +0 -180
  16. package/dist/docs/Syntax.md +0 -463
  17. package/dist/docs/TodoList.md +0 -215
  18. package/dist/docs/Use-Cases.md +0 -354
  19. package/dist/docs/guide-m.md +0 -56
  20. package/dist/docs/index.html +0 -108
  21. package/dist/docs/index.md +0 -56
  22. package/dist/docs/one-page.md +0 -322
  23. package/dist/docs/origin.md +0 -460
  24. package/dist/docs/polarity.md +0 -586
  25. package/dist/docs/primary-subject.md +0 -326
  26. package/dist/docs/statements.md +0 -226
  27. package/dist/docs/tokenization-performance-report.md +0 -164
  28. package/dist/examples/Cookbook.md +0 -32
  29. package/dist/examples/Status-SHACL.md +0 -31
  30. package/dist/examples/demo/index.md +0 -35
  31. package/dist/examples/demo/journal.md +0 -58
  32. package/dist/examples/demo/minimal.md +0 -23
  33. package/dist/examples/demo/project.md +0 -84
  34. package/dist/examples/demo/prov.md +0 -107
  35. package/dist/examples/demo/rdf.md +0 -69
  36. package/dist/examples/demo/recipe.md +0 -50
  37. package/dist/examples/demo/research.md +0 -84
  38. package/dist/examples/demo/shacl.md +0 -161
  39. package/dist/examples/demo/space-mission.md +0 -58
  40. package/dist/examples/demo/tasks.md +0 -175
  41. package/dist/examples/demo/xsd.md +0 -91
  42. package/dist/examples/dogfood.md +0 -288
  43. package/dist/examples/few-shot.md +0 -309
  44. package/dist/examples/graph/rdf-graph.html +0 -473
  45. package/dist/examples/graph/rdf_force_graph_web_component.html +0 -506
  46. package/dist/examples/index.md +0 -45
  47. package/dist/examples/llm-time-workflow.md +0 -173
  48. package/dist/examples/statements.md +0 -43
  49. package/dist/examples/workflow.md +0 -150
  50. package/dist/index.html +0 -3049
  51. package/dist/logo.svg +0 -7
  52. package/dist/spec/Spec-compact.md +0 -368
  53. package/dist/spec/Spec-ultra.md +0 -232
  54. package/dist/spec/Spec.md +0 -1116
  55. package/dist/spec/index.md +0 -46
package/README.md CHANGED
@@ -1,10 +1,39 @@
1
1
  # MD-LD
2
2
 
3
- **Markdown-Linked Data (MD-LD)** — a deterministic, streaming-friendly RDF authoring format that extends Markdown with explicit `{...}` annotations.
3
+ **Markdown-Linked Data** — Human-friendly knowledge graph authoring.
4
+
5
+ Write semantic data in natural Markdown, parse to RDF quads, build deterministic state machines.
4
6
 
5
7
  [![NPM](https://img.shields.io/npm/v/mdld-parse)](https://www.npmjs.com/package/mdld-parse)
6
8
 
7
- [Demo](https://mdld.js.org) | [Repository](https://github.com/davay42/mdld-parse)
9
+ [Demo](https://mdld.js.org) | [Repository](https://github.com/davay42/mdld-parse)
10
+
11
+ ## 🎯 What is MD-LD?
12
+
13
+ MD-LD is not just another RDF syntax. It's a **universal semantic writing interface** that removes the intermediary between human text and machine-readable graphs.
14
+
15
+ Traditional systems require:
16
+ ```
17
+ Human → UI → App Logic → Hidden Database → APIs → Exports
18
+ ```
19
+
20
+ MD-LD enables:
21
+ ```
22
+ Human text → Graph immediately
23
+ ```
24
+
25
+ **Core value:** Author and maintain knowledge graphs as plain text with deterministic round-trip safety. No platforms, databases, or proprietary SaaS mediation required.
26
+
27
+ ```markdown
28
+ [ex] <http://example.org/>
29
+
30
+ # Document {=ex:doc .Article label}
31
+
32
+ [Alice] {=ex:alice ?ex:author .prov:Person ex:firstName label}
33
+ [Smith] {ex:lastName}
34
+ ```
35
+
36
+ **Generates RDF quads** that work with n3.js, rdflib, and any RDF/JS-compatible library.
8
37
 
9
38
  ## 🚀 Quick Start
10
39
 
@@ -13,66 +42,107 @@ pnpm install mdld-parse
13
42
  ```
14
43
 
15
44
  ```javascript
16
- import { parse } from 'mdld-parse';
45
+ import { parse, generate, merge } from 'mdld-parse';
17
46
 
18
- const result = parse(`
19
- [ex] <http://example.org/>
47
+ // Parse MDLD to RDF quads
48
+ const result = parse({ text: mdldString });
49
+ console.log(result.quads); // RDF/JS quads
50
+ console.log(result.primary); // Primary metadata (subject, type, label, comment)
51
+ console.log(result.statements); // Elevated statements
52
+ console.log(result.origin); // Provenance tracking
20
53
 
21
- # Document {=ex:doc .ex:Article label}
22
-
23
- [Alice] {?ex:author =ex:alice .prov:Person ex:firstName label}
24
- [Smith] {ex:lastName}`);
25
-
26
- console.log(result.quads);
27
- // RDF/JS quads ready for n3.js, rdflib, etc.
28
- // @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
29
- // @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>.
30
- // @prefix prov: <http://www.w3.org/ns/prov#>.
31
- // @prefix ex: <http://example.org/>.
32
-
33
- // ex:doc a ex:Article;
34
- // rdfs:label "Document";
35
- // ex:author ex:alice.
36
- // ex:alice a prov:Person;
37
- // rdfs:label "Alice";
38
- // ex:firstName "Alice";
39
- // ex:lastName "Smith".
54
+ // Generate MDLD from quads
55
+ const { text } = generate({ quads: result.quads });
56
+
57
+ // Merge multiple documents (CRDT-style)
58
+ const merged = merge([doc1, doc2, doc3]);
40
59
  ```
41
60
 
42
- ## 📚 Documentation Hub
61
+ ## 💡 Why MD-LD?
43
62
 
44
- - **📖 [Documentation](./docs/index.md)** - Complete documentation with guides and references
45
- - **🎯 [Examples](./examples/index.md)** - Real-world MD-LD examples and use cases
46
- - **📋 [Specification](./spec/index.md)** - Formal specification and test suite
63
+ ### The Problem with Current Systems
47
64
 
48
- ## ✨ Core Features
65
+ Most software today uses graphs internally but hides them behind UIs:
66
+ - **Notion, Slack, Google Docs** — Human interfaces over hidden graphs
67
+ - **CRMs, task apps, note apps** — Proprietary data silos
68
+ - **Social networks** — Platform-controlled knowledge prisons
69
+
70
+ Users cannot access the graph directly. Semantics are hidden. Data is locked in products.
71
+
72
+ ### The MD-LD Solution
73
+
74
+ MD-LD removes the intermediary. Writing becomes publishing. Publishing becomes graph construction.
75
+
76
+ **Key benefits:**
77
+ - **Graph sovereignty** — You own text, graph, provenance, execution, history
78
+ - **No central platform required** — Works offline, in browsers, on servers
79
+ - **Universal semantic substrate** — Agents can read, reason, write, execute, validate
80
+ - **Continuous semantic narrative** — Unifies chat, tasks, notes, emails, calendar, files
81
+ - **Native time dimension** — Every action, statement, correction becomes part of the graph
82
+ - **Decentralized authority** — RFC 4151 tag: URIs enable self-sovereign identity without central registries
83
+
84
+ ### Real-World Applications
85
+
86
+ #### Personal Knowledge Management
87
+ ```markdown
88
+ [alice] <tag:alice@example.com,2026:>
89
+
90
+ # Meeting Notes {=alice:meeting-2024-01-15 .alice:Meeting}
91
+
92
+ Attendees:
93
+ - **Alice** {+alice:alice ?alice:attendee label}
94
+ - **Bob** {+alice:bob ?alice:attendee label}
95
+
96
+ Action items:
97
+ - **Review proposal** {+alice:task-1 ?alice:actionItem label}
98
+ ```
49
99
 
50
- - **🔗 Prefix folding** - Build hierarchical namespaces with lightweight IRI authoring
51
- - **📍 Subject declarations** - `{=IRI}` and `{=#fragment}` for context setting
52
- - **🎯 Object IRIs** - `{+IRI}` and `{+#fragment}` for temporary object declarations
53
- - **🔄 Three predicate forms** - `p` (S→L), `?p` (S→O), `!p` (O→S)
54
- - **🏷️ Type declarations** - `.Class` for rdf:type triples
55
- - **📅 Datatypes & language** - `^^xsd:date` and `@en` support
56
- - **🧩 Fragments** - Document structuring with `{=#fragment}`
57
- - **⚡ Polarity system** - Sophisticated diff authoring with `+` and `-` prefixes
58
- - **📍 Origin tracking** - Complete provenance with lean quad-to-source mapping
59
- - **🎯 Elevated statements** - Automatic rdf:Statement pattern detection for "golden" graph extraction
60
- - **🏷️ Primary metadata** - Structured primary object with subject, type, and label for document identity
100
+ #### Developer Documentation
101
+ ```markdown
102
+ # API Endpoint {=api:/users/:id .api:Endpoint}
103
+
104
+ [GET] {api:method}
105
+ [/users/:id] {api:path}
106
+ ```
61
107
 
62
- ## 🌟 What is MD-LD?
108
+ #### Academic Research
109
+ ```markdown
110
+ # Paper {=alice:paper-semantic-markdown .alice:ScholarlyArticle}
63
111
 
64
- MD-LD allows you to author RDF graphs directly in Markdown using explicit `{...}` annotations:
112
+ [Semantic Web] {label}
113
+ [Alice Johnson] {=alice:alice-johnson ?alice:author}
114
+ [2024-01] {alice:datePublished ^^xsd:gYear}
115
+ ```
65
116
 
117
+ #### Content Management
66
118
  ```markdown
67
- [my] <tag:alice@example.com,2026:>
68
- # 2024-07-18 {=my:journal-2024-07-18 .my:Event my:date ^^xsd:date}
69
- ## A good day {label}
70
- Mood: [Happy] {my:mood}
71
- Energy level: [8] {my:energyLevel ^^xsd:integer}
72
- Met [Sam] {+my:sam .my:Person ?my:attendee} on my regular walk at [Central Park] {+my:central-park ?my:location .my:Place label @en} and talked about [Sunny] {my:weather} weather.
119
+ # Understanding MD-LD {=blog:post-mdld .blog:Post}
120
+
121
+ [MD-LD] {blog:emphasized label} allows you to embed RDF directly in Markdown.
73
122
  ```
74
123
 
75
- Generates valid RDF triples with complete provenance tracking.
124
+ ## 📚 Documentation Hub
125
+
126
+ - **📖 [Documentation](./docs/index.md)** — Complete documentation with guides and references
127
+ - **🎯 [Examples](./examples/index.md)** — Real-world MD-LD examples and use cases
128
+ - **📋 [Specification](./spec/index.md)** — Formal specification and test suite
129
+ - **🔄 [Diff Generation](./docs/diff.md)** — Automatic diff document generation for CRDT workflows
130
+
131
+ ## ✨ Core Features
132
+
133
+ - **🔗 Prefix folding** — Build hierarchical namespaces with CURIE-based IRI authoring
134
+ - **📍 Subject declarations** — `{=IRI}` and `{=#fragment}` for context setting
135
+ - **🎯 Object IRIs** — `{+IRI}` and `{+#fragment}` for temporary object declarations
136
+ - **🔄 Three predicate forms** — `p` (S→L), `?p` (S→O), `!p` (O→S)
137
+ - **🏷️ Type declarations** — `.Class` for rdf:type triples
138
+ - **📅 Datatypes & language** — `^^xsd:date` and `@en` support
139
+ - **🧩 Fragments** — Document structuring with `{=#fragment}`
140
+ - **⚡ Polarity system** — Sophisticated diff authoring with `+` and `-` prefixes
141
+ - **📍 Origin tracking** — Complete provenance with lean quad-to-source mapping
142
+ - **🔗 Span chains** — Walkable textual topology between semantic blocks for context recovery and resonance
143
+ - **🎯 Elevated statements** — Automatic rdf:Statement pattern detection
144
+ - **🏷️ Primary metadata quartet** — Subject, type, label, comment for document identity
145
+ - **🔄 Round-trip safety** — Deterministic parse ↔ generate cycles
76
146
 
77
147
  ## 📦 Installation
78
148
 
@@ -83,20 +153,19 @@ pnpm install mdld-parse
83
153
  ```
84
154
 
85
155
  ```javascript
86
- import { parse } from 'mdld-parse';
87
-
88
- const markdown = `[ex] <tag:alice@example.org,2026:>
156
+ import { parse, generate, merge } from 'mdld-parse';
89
157
 
90
- # Demo document {=ex:example/doc .prov:Entity label}
158
+ const mdld = `[ex] <http://example.org/>
91
159
 
92
- > A demo document for MD-LD {comment}
93
- [Alice] {+ex:alice ?ex:author .prov:Person label}
94
- `;
160
+ # Document {=ex:doc .Article label}
95
161
 
96
- const result = parse({ text: markdown });
162
+ [Alice] {=ex:alice ?ex:author .prov:Person ex:firstName label}
163
+ [Smith] {ex:lastName}`;
97
164
 
98
- console.log(result.quads);
99
- // RDF/JS quads ready for n3.js, rdflib, etc.
165
+ const result = parse({ text: mdld });
166
+ console.log(result.quads); // RDF/JS quads
167
+ console.log(result.primary); // { subject, type, label, comment }
168
+ console.log(result.statements); // Elevated statements
100
169
  ```
101
170
 
102
171
  ### Browser (ES Modules)
@@ -104,11 +173,12 @@ console.log(result.quads);
104
173
  ```html
105
174
  <script type="module">
106
175
  import { parse } from 'https://cdn.jsdelivr.net/npm/mdld-parse/+esm';
107
-
108
- const result = parse('[ex] <tag:alice@example.org,2026:>\n\n# Hello {=ex:hello label}');
176
+ const result = parse('[ex] <http://example.org/>\n\n# Hello {=ex:hello label}');
109
177
  </script>
110
178
  ```
111
179
 
180
+ **Bundle size:** 84KB unminified, 19KB gzipped
181
+
112
182
  ## 🧠 Semantic Model
113
183
 
114
184
  MD-LD encodes a directed labeled multigraph where three nodes may be in scope:
@@ -117,7 +187,7 @@ MD-LD encodes a directed labeled multigraph where three nodes may be in scope:
117
187
  - **O** — object resource (IRI from link/image)
118
188
  - **L** — literal value (string + optional datatype/language)
119
189
 
120
- ### Predicate Routing (§8.1)
190
+ ### Predicate Routing
121
191
 
122
192
  Each predicate form determines the graph edge:
123
193
 
@@ -127,23 +197,6 @@ Each predicate form determines the graph edge:
127
197
  | `?p` | S → O | `[NASA] {=ex:nasa ?org}` | object property |
128
198
  | `!p` | O → S | `[Parent] {=ex:p !hasPart}` | reverse object |
129
199
 
130
- ## 📍 Elevated Statements
131
-
132
- MD-LD automatically detects `rdf:Statement` patterns during parsing and extracts elevated SPO quads for convenient consumption by applications.
133
-
134
- ### Pattern Detection
135
-
136
- When the parser encounters a complete `rdf:Statement` pattern with `rdf:subject`, `rdf:predicate`, and `rdf:object`, it automatically adds the corresponding SPO quad to the `statements` array:
137
-
138
- ```markdown
139
- [ex] <http://example.org/>
140
-
141
- ## Elevated statement {=ex:stmt1 .rdf:Statement}
142
- **Alice** {+ex:alice ?rdf:subject} *knows* {+ex:knows ?rdf:predicate} **Bob** {+ex:bob ?rdf:object}
143
-
144
- Direct statement:**Alice** {=ex:alice} knows **Bob** {?ex:knows +ex:bob}
145
- ```
146
-
147
200
  ## 🎨 Syntax Quick Reference
148
201
 
149
202
  ### Subject Declaration
@@ -152,19 +205,10 @@ Set current subject (emits no quads):
152
205
  ## Apollo 11 {=ex:apollo11}
153
206
  ```
154
207
 
155
- ### Fragment Syntax
156
- Create fragment IRIs relative to current subject:
157
- ```markdown
158
- # Document {=ex:document}
159
- {=#summary}
160
- [Content] {label}
161
- ```
162
- Fragments replace any existing fragment and require a current subject.
163
-
164
208
  ### Type Declaration
165
209
  Emit `rdf:type` triple:
166
210
  ```markdown
167
- ## Apollo 11 {=ex:apollo11 .ex:SpaceMission .ex:Event}
211
+ ## Apollo 11 {=ex:apollo11 .SpaceMission .Event}
168
212
  ```
169
213
 
170
214
  ### Literal Properties
@@ -184,50 +228,47 @@ Links create relationships (use `?` prefix):
184
228
  ```
185
229
 
186
230
  ### Resource Declaration
187
- Declare resources inline with `{=iri}`:
231
+ Declare resources inline with `{+iri}`:
188
232
  ```markdown
189
233
  # Mission {=ex:apollo11}
190
- [Neil Armstrong] {=ex:armstrong ?ex:commander .prov:Person}
234
+ [Neil Armstrong] {+ex:armstrong ?ex:commander .Person}
235
+ ```
236
+
237
+ ### Diff Authoring (Polarity)
238
+ Use `+` and `-` for retractions:
239
+ ```markdown
240
+ # Update value
241
+ [Person] {=ex:person}
242
+ [Alice] {-ex:name}
243
+ [Bob] {ex:name}
244
+
245
+ # Remove triple
246
+ [Person] {=ex:person}
247
+ [Alice] {-ex:friend}
191
248
  ```
192
249
 
193
250
  ## 🔧 API Reference
194
251
 
195
252
  ### `parse({ text, context, dataFactory, graph })`
196
253
 
197
- Parse MD-LD markdown and return RDF quads with lean origin tracking.
254
+ Parse MDLD to RDF quads with lean origin tracking.
198
255
 
199
- **Parameters (named object):**
200
- - `text` (string, required) — MD-LD formatted text
201
- - `context` (object, optional) — Prefix mappings (default: `{ '@vocab': 'http://www.w3.org/2000/01/rdf-schema#', rdf, rdfs, xsd, sh, prov }`)
256
+ **Parameters:**
257
+ - `text` (string, required) — MDLD formatted text
258
+ - `context` (object, optional) — Prefix mappings
202
259
  - `dataFactory` (object, optional) — Custom RDF/JS DataFactory
203
260
  - `graph` (string, optional) — Named graph IRI
204
261
 
205
262
  **Returns:** `{ quads, remove, statements, origin, context, primarySubject, primary, md }`
206
263
 
207
- - `quads` — Array of RDF/JS Quads (final resolved graph state)
208
- - `remove` — Array of RDF/JS Quads (external retractions targeting prior state)
209
- - `statements` — Array of elevated RDF/JS Quads extracted from rdf:Statement patterns
210
- - `origin` — Lean origin tracking object with quadIndex for UI navigation
211
- - `context` — Final context used (includes prefixes)
264
+ - `quads` — RDF/JS Quads (final resolved graph state)
265
+ - `remove` — RDF/JS Quads (external retractions for diff workflows)
266
+ - `statements` — Elevated SPO quads from rdf:Statement patterns
267
+ - `origin` — Lean origin tracking: `quadIndex`, `blocks`, `spans`, `documentStructure`
268
+ - `context` — Final context with prefixes
212
269
  - `primarySubject` — String IRI or null (canonical append identity)
213
- - `primary` — Object containing primary metadata (semantic surface descriptor)
214
- - `md` — Clean Markdown with all MD-LD annotations stripped (round-trip safe)
215
-
216
- **Dual-Layer Architecture:**
217
-
218
- | Layer | Field | Purpose | Use Cases |
219
- |-------|-------|---------|-----------|
220
- | **Canonical Identity** | `primarySubject` | Append routing, storage, synchronization | `append()`, file placement, authority validation |
221
- | **Semantic Surface** | `primary` | UI, indexing, navigation, agent orientation | Dashboards, search, previews, timelines |
222
-
223
- **Primary Object Structure:**
224
- ```javascript
225
- primary: {
226
- subject: string | null, // First non-fragment subject declaration
227
- type: string | null, // First rdf:type declaration
228
- label: string | null // First rdfs:label literal
229
- }
230
- ```
270
+ - `primary` — Primary metadata quartet: `{ subject, type, label, comment }`
271
+ - `md` — Clean Markdown with annotations stripped
231
272
 
232
273
  ### `merge(docs, options)`
233
274
 
@@ -236,265 +277,161 @@ Merge multiple MDLD documents with diff polarity resolution.
236
277
  **Parameters:**
237
278
  - `docs` (array) — Array of markdown strings or ParseResult objects
238
279
  - `options` (object, optional):
239
- - `context` (object) — Prefix mappings (merged with DEFAULT_CONTEXT)
280
+ - `context` (object) — Prefix mappings
240
281
 
241
- **Returns:** `{ quads, remove, origin, context, primarySubjects, primary }`
282
+ **Returns:** `{ quads, remove, statements, origin, context, primarySubjects, primary }`
242
283
 
243
- - `quads` — Array of RDF/JS Quads (final resolved graph state)
244
- - `remove` — Array of RDF/JS Quads (external retractions targeting prior state)
284
+ - `quads` — RDF/JS Quads (final resolved graph state)
285
+ - `remove` — RDF/JS Quads (external retractions)
286
+ - `statements` — Elevated statements from all documents
245
287
  - `origin` — Merge origin with document tracking
246
- - `context` — Final context used (includes prefixes)
247
- - `primarySubjects` — Array of string IRIs (canonical append identities, ordered by merge)
248
- - `primary` — Array of primary objects (semantic surface descriptors, ordered by merge)
249
-
250
- **Dual-Layer Architecture:**
251
-
252
- | Layer | Field | Purpose | Use Cases |
253
- |-------|-------|---------|-----------|
254
- | **Canonical Identity** | `primarySubjects` | Append routing, storage, synchronization | Multi-document append, file organization |
255
- | **Semantic Surface** | `primary` | UI, indexing, navigation, agent orientation | Vault indexing, document discovery, search |
256
-
257
- **Primary Object Array Structure:**
258
- ```javascript
259
- primary: [
260
- {
261
- subject: string | null, // First non-fragment subject declaration
262
- type: string | null, // First rdf:type declaration
263
- label: string | null // First rdfs:label literal
264
- },
265
- // ... one object per document
266
- ]
267
- ```
268
-
269
-
270
- ### `generate({ quads, context, primarySubject })`
271
-
272
- Generate deterministic MDLD from RDF quads with visual styling.
273
-
274
- **Parameters (named object):**
275
- - `quads` (array, required) — Array of RDF/JS Quads to convert
276
- - `context` (object, optional) — Prefix mappings (default: `{}`)
277
- - `primarySubject` (string, optional) — String IRI to place first in output (ensures round-trip safety). If not provided, falls back to the first subject from quads.
278
-
279
- **Returns:** `{ text, context }`
280
-
281
- **Features:**
282
- - Visual carrier styles based on datatype (code spans for numbers, bold booleans, etc.)
283
- - Label-in-heading: Uses `rdfs:label` in subject headings when available
284
- - Multiple labels: First label in heading, additional labels rendered as literals
285
- - Round-trip safe: All data preserved through parse → generate → parse
286
- - Composable: `generate(parse(text))` extracts semantics; `parse(generate({quads}))` normalizes quads
288
+ - `context` — Final context with prefixes
289
+ - `primarySubjects` — Array of string IRIs (canonical identities)
290
+ - `primary` — Array of primary metadata objects
287
291
 
288
- ### `generateNode({ quads, focusIRI, context })`
292
+ **Use case:** CRDT-style state management with append-only documents.
289
293
 
290
- Generate node-centric MDLD showing all quads where a specific IRI appears in any position.
294
+ ### `generate({ quads, context, primarySubject, compactInline, renderReverse, remove, lang })`
291
295
 
292
- **Parameters (named object):**
293
- - `quads` (array, required) — Array of RDF/JS Quads to search
294
- - `focusIRI` (string, required) — The IRI to center the view on
295
- - `context` (object, optional) — Prefix mappings (default: `{}`)
296
+ Generate deterministic MDLD from RDF quads.
296
297
 
297
- **Returns:** `{ text, context }`
298
-
299
- **Behavior (Safety-First):**
300
- - If `focusIRI` is null/undefined: Returns empty text
301
- - If `focusIRI` not in graph: Returns empty text (never falls back to all data)
302
- - If `quads` is empty: Returns empty text
303
-
304
- **Safety rationale:** Prevents accidental rendering of entire databases on misspelled IRIs—critical for production use with LLM cost per token. Explicit emptiness signals "not found" to the caller.
298
+ **Parameters:**
299
+ - `quads` (array, required) — RDF/JS Quads to convert
300
+ - `context` (object, optional) — Prefix mappings
301
+ - `primarySubject` (string, optional) — IRI to place first in output
302
+ - `compactInline` (boolean, optional) — Inline type/label compaction (default: `false`)
303
+ - `renderReverse` (boolean, optional) — Reverse connections as `!p` (default: `false`)
304
+ - `remove` (array, optional) — RDF/JS Quads to retract (for diff generation)
305
+ - `lang` (string, optional) — Preferred language for labels (e.g., `'en'`, `'es'`, `'fr'`). Priority: specified lang → untagged → English → any language
305
306
 
306
- **Use case:** Perfect for exploring a specific node and all its relationships—where it appears as subject, object, predicate, type, or datatype. Creates an exhaustive view of everything related to the focus IRI. Ideal for node-centric knowledge graph explorers.
307
+ **Returns:** `{ text, context, compactStats }`
307
308
 
308
- ### `updateValue()` — Update Quad Carrier Text in Source Text
309
+ - `text` — Generated MDLD text
310
+ - `context` — Full context with prefixes
311
+ - `compactStats` — Compaction metrics
309
312
 
310
- Update the carrier text of a literal quad in MDLD text. Only the carrier content is replaced — datatype (`^^xsd:integer`) and language (`@en`) annotations inside the `{…}` block are preserved as-is, since they are part of the annotation, not the carrier.
313
+ **Features:** Visual styling, label-in-heading, round-trip safe, diff generation, language preference.
311
314
 
315
+ **Example with language preference:**
312
316
  ```javascript
313
- import { parse, updateValue } from 'mdld-parse';
314
-
315
- const mdld = `[ex] <http://example.org/>
316
-
317
- # Article {=ex:article .ex:Article}
318
-
319
- [Alice Smith] {ex:author}`;
320
-
321
- const result = parse({ text: mdld });
322
- const authorQuad = result.quads.find(q =>
323
- q.subject.value === 'http://example.org/article' &&
324
- q.predicate.value === 'http://example.org/author'
325
- );
326
-
327
- const updatedText = updateValue({
328
- text: mdld,
329
- quad: authorQuad,
330
- value: 'Bob Johnson',
331
- origin: result.origin // optional (auto-parses if not provided)
317
+ const { text } = generate({
318
+ quads: result.quads,
319
+ lang: 'es' // Prefer Spanish labels
332
320
  });
333
-
334
- console.log(updatedText);
335
- // [ex] <http://example.org/>
336
- //
337
- // # Article {=ex:article .ex:Article}
338
- //
339
- // [Bob Johnson] {ex:author}
340
321
  ```
341
322
 
342
- **Annotation annotations are preserved:**
343
- ```javascript
344
- // Original: > 25 {ex:age ^^xsd:integer}
345
- updateValue({ text, quad, value: '30' });
346
- // Result: > 30 {ex:age ^^xsd:integer} ← datatype preserved
323
+ ### `generateNode({ quads, focusIRI, context, compactInline, renderReverse, lang })`
347
324
 
348
- // Original: > Hello {ex:greeting @en}
349
- updateValue({ text, quad, value: 'Good morning' });
350
- // Result: > Good morning {ex:greeting @en} ← language tag preserved
351
- ```
325
+ Generate node-centric MDLD for a specific IRI.
352
326
 
353
327
  **Parameters:**
354
- - `text` (string) — The original MDLD text
355
- - `quad` (object) — The quad to update (subject, predicate, object)
356
- - `value` (string) — The new carrier text to set
357
- - `origin` (object, optional) — ParseResult.origin (auto-parses if not provided)
358
-
359
- **Returns:** Updated MDLD text, or original text if update fails (fail-safe)
360
-
361
- **How it works:**
362
- - Uses `locate()` to find the quad's position in the source text
363
- - Uses `valueRange` to replace only the carrier text (excluding carrier markers like `[`, `]`)
364
- - Annotation block `{…}` with predicate, datatype, language is untouched
365
- - Auto-parses if result not provided (convenient but less efficient)
366
-
367
- **Fail-safe behavior:**
368
- - Returns original text if quad cannot be located
369
- - Returns original text if valueRange is not available
370
- - Never corrupts the source text
371
-
372
- **Use case:** Perfect for editor applications that need to update literal values while preserving carrier syntax, datatype annotations, and language tags.
328
+ - `quads` (array, required) — RDF/JS Quads to search
329
+ - `focusIRI` (string, required) — IRI to center view on
330
+ - `context` (object, optional) — Prefix mappings
331
+ - `compactInline` (boolean, optional) — Inline compaction (default: `true`)
332
+ - `renderReverse` (boolean, optional) — Reverse connections (default: `true`)
333
+ - `lang` (string, optional) — Preferred language for labels (e.g., `'en'`, `'es'`, `'fr'`). Priority: specified lang → untagged → English → any language
373
334
 
374
- ### Composition Patterns
335
+ **Returns:** `{ text, context, compactStats }`
375
336
 
376
- With the unified named parameter API, `parse()` and `generate()` compose seamlessly through object spreading:
337
+ **Safety:** Returns empty text if focusIRI not found (prevents accidental full database rendering).
377
338
 
378
- ```javascript
379
- import { parse, generate, generateNode } from 'mdld-parse';
380
-
381
- // Pattern 1: parse → generate (semantic extraction)
382
- const canonical = generate({ ...parse({ text, context }) });
383
- // text → quads → canonical MDLD (deterministic, visual styling applied)
384
-
385
- // Pattern 2: generate → parse (normalize external RDF)
386
- const normalized = parse({ ...generate({ quads: externalQuads, context }) });
387
- // external quads → MDLD → validated quads (DataFactory-safe, no blank nodes)
388
-
389
- // Pattern 3: parse → generateNode (node-centric exploration)
390
- const nodeView = generateNode({ ...parse({ text }), focusIRI: 'http://example.org/alice' });
391
- // full graph → isolated node view (safe: returns empty if IRI not found)
392
- ```
393
-
394
- **Why this works:**
395
- - `parse()` returns `{ quads, context, primarySubject, md, ... }`
396
- - `generate()` accepts `{ quads, context, primarySubject }`
397
- - `generateNode()` accepts `{ quads, context, focusIRI }` (with focusIRI override)
398
- - Perfect shape alignment enables elegant `{ ...spread }` composition
399
-
400
- ### The `md` Field — Clean Markdown Extraction
401
-
402
- Every `parse()` result includes a `md` field containing the original Markdown with all MD-LD annotations stripped:
403
-
404
- ```javascript
405
- const result = parse({
406
- text: `# Document {=ex:doc .Article}\n[Content] {ex:content}`,
407
- context: { ex: 'http://example.org/' }
408
- });
409
-
410
- console.log(result.md);
411
- // # Document\nContent
412
-
413
- // Round-trip safety: re-parsing clean MD produces zero quads
414
- const reparsed = parse({ text: result.md });
415
- console.log(reparsed.quads.length); // 0
416
- ```
417
-
418
- **Behavior:**
419
- - Valid MD-LD annotations (`{=...}`, `{+...}`, `{...}`) are completely removed
420
- - Content from value carriers (`[text]`, `**bold**`, `` `code` ``) is preserved
421
- - Invalid syntax (annotations not at end-of-line) is preserved as visible markers
422
- - Headings, lists, blockquotes, code blocks maintain their structure
423
- - Prefix declarations at start of line are stripped
424
- - Standalone subject declarations (`{=ex:subject}`) are stripped
425
-
426
- **Use cases:**
427
- - **Content extraction** — Get readable Markdown without semantic markup
428
- - **Syntax validation** — Remaining `{...}` patterns indicate invalid MD-LD syntax
429
- - **Round-trip testing** — `parse(md).md` should parse to zero quads
430
- - **Preview generation** — Show clean document before publishing
339
+ ### `updateValue({ text, quad, value, origin })`
431
340
 
432
- ### `locate(quad, origin)`
433
-
434
- Locate the origin entry for a quad using the lean origin system.
341
+ Update carrier text of a literal quad in MDLD text.
435
342
 
436
343
  **Parameters:**
437
- - `quad` (object) — The quad to locate (subject, predicate, object)
438
- - `origin` (object) — Origin object containing quadIndex
344
+ - `text` (string) — Original MDLD text
345
+ - `quad` (object) — Quad to update
346
+ - `value` (string) — New carrier text
347
+ - `origin` (object, optional) — ParseResult.origin
439
348
 
440
- **Returns:** `{ blockId, range, valueRange, carrierType, subject, predicate, context, value, polarity }` or `null`
349
+ **Returns:** Updated MDLD text (fail-safe)
441
350
 
442
- - `range`: Full character range including carrier markers (e.g., `[`, `]`, `{`, `}`)
443
- - `valueRange`: Character range excluding carrier markers (null if not available)
351
+ **Use case:** Editor applications updating literal values.
444
352
 
445
- ### `render(quads, options)`
446
-
447
- Render RDF quads as HTML+RDFa for web display.
353
+ ### `locate(quad, origin)`
448
354
 
449
- **Parameters:**
450
- - `quads` (array) — Array of RDF/JS Quads to render
451
- - `options` (object, optional):
452
- - `context` (object) — Prefix mappings for CURIE shortening
453
- - `baseIRI` (string) — Base IRI for resolving relative references
355
+ Locate quad origin entry for UI navigation.
454
356
 
455
- **Returns:** `{ html, context }`
357
+ **Returns:** `{ blockId, range, valueRange, carrierType, ... }` or `null`
456
358
 
457
359
  ### Utility Functions
458
360
 
459
361
  ```javascript
460
362
  import {
461
363
  DEFAULT_CONTEXT, // Default prefix mappings
462
- DataFactory, // RDF/JS DataFactory instance
463
- hash, // String hashing function
464
- expandIRI, // IRI expansion with context
465
- shortenIRI, // IRI shortening with context
466
- parseSemanticBlock // Parse semantic block syntax
364
+ DataFactory, // RDF/JS DataFactory
365
+ hash, // String hashing
366
+ expandIRI, // IRI expansion
367
+ shortenIRI, // IRI shortening
368
+ parseSemanticBlock // Semantic block parsing
467
369
  } from 'mdld-parse';
468
370
  ```
469
371
 
470
372
  ## 🏗️ Architecture
471
373
 
472
374
  ### Design Principles
473
- - **Zero dependencies** — Pure JavaScript, ~15KB minified
375
+ - **Zero dependencies** — Pure JavaScript, 84KB unminified (19KB gzipped)
474
376
  - **Streaming-first** — Single-pass parsing, O(n) complexity
475
- - **Standards-compliant** — RDF/JS data model
476
- - **Origin tracking** — Full round-trip support with source maps
377
+ - **Character-based tokenization** — 20-28% faster than regex-based approaches
378
+ - **Standards-compliant** — RDF/JS data model, W3C CURIE 1.0 syntax
379
+ - **Deterministic** — Same input always produces same output
477
380
  - **Explicit semantics** — No guessing, inference, or heuristics
381
+ - **Dual-layer origin** — Every parse emits both a semantic quad graph and a walkable textual topology graph simultaneously
382
+
383
+ ### Origin: Blocks and Spans
384
+
385
+ The parser output includes a complete document chain at no extra cost:
386
+
387
+ ```
388
+ [Block] --(Span)-- [Block] --(Span)-- [Block]
389
+ ```
390
+
391
+ - **Blocks** (`origin.blocks`) — semantic anchors: tokens that produced RDF quads, with `prevSpanId`/`nextSpanId` links
392
+ - **Spans** (`origin.spans`) — textual observations: raw byte ranges between blocks, with bidirectional block and span links
393
+
394
+ Spans store no text — content is always recovered via `sourceText.slice(span.range[0], span.range[1])`. This unlocks context-aware UI, autocomplete neighborhood retrieval, and cross-document topology without any parser-level interpretation.
395
+
396
+ ### Performance Characteristics
397
+ - **Real-time (60fps):** Up to 4,527 quads per frame
398
+ - **Batch processing:** Up to 225,059 quads per second
399
+ - **Memory efficient:** ~640 bytes per quad retained after GC
400
+ - **Streaming-friendly:** Full document never in memory
478
401
 
479
402
  ### RDF/JS Compatibility
480
- Quads are compatible with:
403
+ Quads work with:
481
404
  - [`n3.js`](https://github.com/rdfjs/N3.js) — Turtle/N-Triples serialization
482
- - [`rdflib.js`](https://github.com/linkeddata/rdflib.js) — RDF store and reasoning
405
+ - [`rdflib.js`](https://github.com/linkeddata/rdflib.js) — RDF stores
483
406
  - [`sparqljs`](https://github.com/RubenVerborgh/SPARQL.js) — SPARQL queries
484
- - [`rdf-ext`](https://github.com/rdf-ext/rdf-ext) — Extended RDF utilities
407
+ - [`rdf-ext`](https://github.com/rdf-ext/rdf-ext) — RDF utilities
485
408
 
486
- ## 🧪 Testing
409
+ ### Standards Compliance
410
+ - **RDF 1.1** — Core RDF concepts
411
+ - **RDFS** — Schema vocabulary
412
+ - **PROV-O** — Provenance ontology
413
+ - **SHACL** — Constraint validation
414
+ - **W3C CURIE 1.0** — Compact URI syntax
487
415
 
488
- The parser includes comprehensive tests covering all spec requirements:
416
+ ## 🧪 Testing
489
417
 
490
418
  ```bash
491
419
  pnpm test
492
420
  ```
493
421
 
494
- Tests validate:
495
- - Subject declaration and context
496
- - All predicate forms (p, ?p, !p)
497
- - Datatypes and language tags
498
- - Explicit list item annotations
499
- - Code blocks and blockquotes
500
- - Round-trip serialization
422
+ Comprehensive test suite covering:
423
+ - Syntax parsing and tokenization
424
+ - Context management and prefix folding
425
+ - Polarity system and retractions
426
+ - Elevated statements detection
427
+ - Primary metadata extraction
428
+ - Round-trip parse/generate cycles
429
+ - Origin tracking and provenance
430
+
431
+ ## 📚 Additional Resources
432
+
433
+ - **[Documentation](./docs/index.md)** — Complete guides and references
434
+ - **[Specification](./spec/index.md)** — Formal specification
435
+ - **[Examples](./examples/index.md)** — Real-world examples
436
+ - **[Grammar](./grammar/mdld.ebnf)** — EBNF grammar specification
437
+ - **[VS Code Extension](https://marketplace.visualstudio.com/)** — Syntax highlighting (coming soon)