mdld-parse 0.9.9 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +263 -326
- package/dist/index.js +1860 -1337
- package/package.json +4 -4
- package/dist/.nojekyll +0 -0
- package/dist/CNAME +0 -1
- package/dist/README.md +0 -500
- package/dist/docs/API.md +0 -435
- package/dist/docs/Architecture.md +0 -190
- package/dist/docs/Guide.md +0 -816
- package/dist/docs/LLM-MDLD-Inducer.md +0 -550
- package/dist/docs/LiterateRDF.md +0 -472
- package/dist/docs/Parser.md +0 -720
- package/dist/docs/Performance.md +0 -238
- package/dist/docs/Spec-render.md +0 -1439
- package/dist/docs/Subject.md +0 -180
- package/dist/docs/Syntax.md +0 -463
- package/dist/docs/TodoList.md +0 -215
- package/dist/docs/Use-Cases.md +0 -354
- package/dist/docs/guide-m.md +0 -56
- package/dist/docs/index.html +0 -108
- package/dist/docs/index.md +0 -56
- package/dist/docs/one-page.md +0 -322
- package/dist/docs/origin.md +0 -460
- package/dist/docs/polarity.md +0 -586
- package/dist/docs/primary-subject.md +0 -326
- package/dist/docs/statements.md +0 -226
- package/dist/docs/tokenization-performance-report.md +0 -164
- package/dist/examples/Cookbook.md +0 -32
- package/dist/examples/Status-SHACL.md +0 -31
- package/dist/examples/demo/index.md +0 -35
- package/dist/examples/demo/journal.md +0 -58
- package/dist/examples/demo/minimal.md +0 -23
- package/dist/examples/demo/project.md +0 -84
- package/dist/examples/demo/prov.md +0 -107
- package/dist/examples/demo/rdf.md +0 -69
- package/dist/examples/demo/recipe.md +0 -50
- package/dist/examples/demo/research.md +0 -84
- package/dist/examples/demo/shacl.md +0 -161
- package/dist/examples/demo/space-mission.md +0 -58
- package/dist/examples/demo/tasks.md +0 -175
- package/dist/examples/demo/xsd.md +0 -91
- package/dist/examples/dogfood.md +0 -288
- package/dist/examples/few-shot.md +0 -309
- package/dist/examples/graph/rdf-graph.html +0 -473
- package/dist/examples/graph/rdf_force_graph_web_component.html +0 -506
- package/dist/examples/index.md +0 -45
- package/dist/examples/llm-time-workflow.md +0 -173
- package/dist/examples/statements.md +0 -43
- package/dist/examples/workflow.md +0 -150
- package/dist/index.html +0 -3049
- package/dist/logo.svg +0 -7
- package/dist/spec/Spec-compact.md +0 -368
- package/dist/spec/Spec-ultra.md +0 -232
- package/dist/spec/Spec.md +0 -1116
- package/dist/spec/index.md +0 -46
package/README.md
CHANGED
|
@@ -1,10 +1,39 @@
|
|
|
1
1
|
# MD-LD
|
|
2
2
|
|
|
3
|
-
**Markdown-Linked Data
|
|
3
|
+
**Markdown-Linked Data** — Human-friendly knowledge graph authoring.
|
|
4
|
+
|
|
5
|
+
Write semantic data in natural Markdown, parse to RDF quads, build deterministic state machines.
|
|
4
6
|
|
|
5
7
|
[](https://www.npmjs.com/package/mdld-parse)
|
|
6
8
|
|
|
7
|
-
[Demo](https://mdld.js.org) | [Repository](https://github.com/davay42/mdld-parse)
|
|
9
|
+
[Demo](https://mdld.js.org) | [Repository](https://github.com/davay42/mdld-parse)
|
|
10
|
+
|
|
11
|
+
## 🎯 What is MD-LD?
|
|
12
|
+
|
|
13
|
+
MD-LD is not just another RDF syntax. It's a **universal semantic writing interface** that removes the intermediary between human text and machine-readable graphs.
|
|
14
|
+
|
|
15
|
+
Traditional systems require:
|
|
16
|
+
```
|
|
17
|
+
Human → UI → App Logic → Hidden Database → APIs → Exports
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
MD-LD enables:
|
|
21
|
+
```
|
|
22
|
+
Human text → Graph immediately
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
**Core value:** Author and maintain knowledge graphs as plain text with deterministic round-trip safety. No platforms, databases, or proprietary SaaS mediation required.
|
|
26
|
+
|
|
27
|
+
```markdown
|
|
28
|
+
[ex] <http://example.org/>
|
|
29
|
+
|
|
30
|
+
# Document {=ex:doc .Article label}
|
|
31
|
+
|
|
32
|
+
[Alice] {=ex:alice ?ex:author .prov:Person ex:firstName label}
|
|
33
|
+
[Smith] {ex:lastName}
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
**Generates RDF quads** that work with n3.js, rdflib, and any RDF/JS-compatible library.
|
|
8
37
|
|
|
9
38
|
## 🚀 Quick Start
|
|
10
39
|
|
|
@@ -13,66 +42,107 @@ pnpm install mdld-parse
|
|
|
13
42
|
```
|
|
14
43
|
|
|
15
44
|
```javascript
|
|
16
|
-
import { parse } from 'mdld-parse';
|
|
45
|
+
import { parse, generate, merge } from 'mdld-parse';
|
|
17
46
|
|
|
18
|
-
|
|
19
|
-
|
|
47
|
+
// Parse MDLD to RDF quads
|
|
48
|
+
const result = parse({ text: mdldString });
|
|
49
|
+
console.log(result.quads); // RDF/JS quads
|
|
50
|
+
console.log(result.primary); // Primary metadata (subject, type, label, comment)
|
|
51
|
+
console.log(result.statements); // Elevated statements
|
|
52
|
+
console.log(result.origin); // Provenance tracking
|
|
20
53
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
console.log(result.quads);
|
|
27
|
-
// RDF/JS quads ready for n3.js, rdflib, etc.
|
|
28
|
-
// @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
|
|
29
|
-
// @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>.
|
|
30
|
-
// @prefix prov: <http://www.w3.org/ns/prov#>.
|
|
31
|
-
// @prefix ex: <http://example.org/>.
|
|
32
|
-
|
|
33
|
-
// ex:doc a ex:Article;
|
|
34
|
-
// rdfs:label "Document";
|
|
35
|
-
// ex:author ex:alice.
|
|
36
|
-
// ex:alice a prov:Person;
|
|
37
|
-
// rdfs:label "Alice";
|
|
38
|
-
// ex:firstName "Alice";
|
|
39
|
-
// ex:lastName "Smith".
|
|
54
|
+
// Generate MDLD from quads
|
|
55
|
+
const { text } = generate({ quads: result.quads });
|
|
56
|
+
|
|
57
|
+
// Merge multiple documents (CRDT-style)
|
|
58
|
+
const merged = merge([doc1, doc2, doc3]);
|
|
40
59
|
```
|
|
41
60
|
|
|
42
|
-
##
|
|
61
|
+
## 💡 Why MD-LD?
|
|
43
62
|
|
|
44
|
-
|
|
45
|
-
- **🎯 [Examples](./examples/index.md)** - Real-world MD-LD examples and use cases
|
|
46
|
-
- **📋 [Specification](./spec/index.md)** - Formal specification and test suite
|
|
63
|
+
### The Problem with Current Systems
|
|
47
64
|
|
|
48
|
-
|
|
65
|
+
Most software today uses graphs internally but hides them behind UIs:
|
|
66
|
+
- **Notion, Slack, Google Docs** — Human interfaces over hidden graphs
|
|
67
|
+
- **CRMs, task apps, note apps** — Proprietary data silos
|
|
68
|
+
- **Social networks** — Platform-controlled knowledge prisons
|
|
69
|
+
|
|
70
|
+
Users cannot access the graph directly. Semantics are hidden. Data is locked in products.
|
|
71
|
+
|
|
72
|
+
### The MD-LD Solution
|
|
73
|
+
|
|
74
|
+
MD-LD removes the intermediary. Writing becomes publishing. Publishing becomes graph construction.
|
|
75
|
+
|
|
76
|
+
**Key benefits:**
|
|
77
|
+
- **Graph sovereignty** — You own text, graph, provenance, execution, history
|
|
78
|
+
- **No central platform required** — Works offline, in browsers, on servers
|
|
79
|
+
- **Universal semantic substrate** — Agents can read, reason, write, execute, validate
|
|
80
|
+
- **Continuous semantic narrative** — Unifies chat, tasks, notes, emails, calendar, files
|
|
81
|
+
- **Native time dimension** — Every action, statement, correction becomes part of the graph
|
|
82
|
+
- **Decentralized authority** — RFC 4151 tag: URIs enable self-sovereign identity without central registries
|
|
83
|
+
|
|
84
|
+
### Real-World Applications
|
|
85
|
+
|
|
86
|
+
#### Personal Knowledge Management
|
|
87
|
+
```markdown
|
|
88
|
+
[alice] <tag:alice@example.com,2026:>
|
|
89
|
+
|
|
90
|
+
# Meeting Notes {=alice:meeting-2024-01-15 .alice:Meeting}
|
|
91
|
+
|
|
92
|
+
Attendees:
|
|
93
|
+
- **Alice** {+alice:alice ?alice:attendee label}
|
|
94
|
+
- **Bob** {+alice:bob ?alice:attendee label}
|
|
95
|
+
|
|
96
|
+
Action items:
|
|
97
|
+
- **Review proposal** {+alice:task-1 ?alice:actionItem label}
|
|
98
|
+
```
|
|
49
99
|
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
- **⚡ Polarity system** - Sophisticated diff authoring with `+` and `-` prefixes
|
|
58
|
-
- **📍 Origin tracking** - Complete provenance with lean quad-to-source mapping
|
|
59
|
-
- **🎯 Elevated statements** - Automatic rdf:Statement pattern detection for "golden" graph extraction
|
|
60
|
-
- **🏷️ Primary metadata** - Structured primary object with subject, type, and label for document identity
|
|
100
|
+
#### Developer Documentation
|
|
101
|
+
```markdown
|
|
102
|
+
# API Endpoint {=api:/users/:id .api:Endpoint}
|
|
103
|
+
|
|
104
|
+
[GET] {api:method}
|
|
105
|
+
[/users/:id] {api:path}
|
|
106
|
+
```
|
|
61
107
|
|
|
62
|
-
|
|
108
|
+
#### Academic Research
|
|
109
|
+
```markdown
|
|
110
|
+
# Paper {=alice:paper-semantic-markdown .alice:ScholarlyArticle}
|
|
63
111
|
|
|
64
|
-
|
|
112
|
+
[Semantic Web] {label}
|
|
113
|
+
[Alice Johnson] {=alice:alice-johnson ?alice:author}
|
|
114
|
+
[2024-01] {alice:datePublished ^^xsd:gYear}
|
|
115
|
+
```
|
|
65
116
|
|
|
117
|
+
#### Content Management
|
|
66
118
|
```markdown
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
Mood: [Happy] {my:mood}
|
|
71
|
-
Energy level: [8] {my:energyLevel ^^xsd:integer}
|
|
72
|
-
Met [Sam] {+my:sam .my:Person ?my:attendee} on my regular walk at [Central Park] {+my:central-park ?my:location .my:Place label @en} and talked about [Sunny] {my:weather} weather.
|
|
119
|
+
# Understanding MD-LD {=blog:post-mdld .blog:Post}
|
|
120
|
+
|
|
121
|
+
[MD-LD] {blog:emphasized label} allows you to embed RDF directly in Markdown.
|
|
73
122
|
```
|
|
74
123
|
|
|
75
|
-
|
|
124
|
+
## 📚 Documentation Hub
|
|
125
|
+
|
|
126
|
+
- **📖 [Documentation](./docs/index.md)** — Complete documentation with guides and references
|
|
127
|
+
- **🎯 [Examples](./examples/index.md)** — Real-world MD-LD examples and use cases
|
|
128
|
+
- **📋 [Specification](./spec/index.md)** — Formal specification and test suite
|
|
129
|
+
- **🔄 [Diff Generation](./docs/diff.md)** — Automatic diff document generation for CRDT workflows
|
|
130
|
+
|
|
131
|
+
## ✨ Core Features
|
|
132
|
+
|
|
133
|
+
- **🔗 Prefix folding** — Build hierarchical namespaces with CURIE-based IRI authoring
|
|
134
|
+
- **📍 Subject declarations** — `{=IRI}` and `{=#fragment}` for context setting
|
|
135
|
+
- **🎯 Object IRIs** — `{+IRI}` and `{+#fragment}` for temporary object declarations
|
|
136
|
+
- **🔄 Three predicate forms** — `p` (S→L), `?p` (S→O), `!p` (O→S)
|
|
137
|
+
- **🏷️ Type declarations** — `.Class` for rdf:type triples
|
|
138
|
+
- **📅 Datatypes & language** — `^^xsd:date` and `@en` support
|
|
139
|
+
- **🧩 Fragments** — Document structuring with `{=#fragment}`
|
|
140
|
+
- **⚡ Polarity system** — Sophisticated diff authoring with `+` and `-` prefixes
|
|
141
|
+
- **📍 Origin tracking** — Complete provenance with lean quad-to-source mapping
|
|
142
|
+
- **🔗 Span chains** — Walkable textual topology between semantic blocks for context recovery and resonance
|
|
143
|
+
- **🎯 Elevated statements** — Automatic rdf:Statement pattern detection
|
|
144
|
+
- **🏷️ Primary metadata quartet** — Subject, type, label, comment for document identity
|
|
145
|
+
- **🔄 Round-trip safety** — Deterministic parse ↔ generate cycles
|
|
76
146
|
|
|
77
147
|
## 📦 Installation
|
|
78
148
|
|
|
@@ -83,20 +153,19 @@ pnpm install mdld-parse
|
|
|
83
153
|
```
|
|
84
154
|
|
|
85
155
|
```javascript
|
|
86
|
-
import { parse } from 'mdld-parse';
|
|
87
|
-
|
|
88
|
-
const markdown = `[ex] <tag:alice@example.org,2026:>
|
|
156
|
+
import { parse, generate, merge } from 'mdld-parse';
|
|
89
157
|
|
|
90
|
-
|
|
158
|
+
const mdld = `[ex] <http://example.org/>
|
|
91
159
|
|
|
92
|
-
|
|
93
|
-
[Alice] {+ex:alice ?ex:author .prov:Person label}
|
|
94
|
-
`;
|
|
160
|
+
# Document {=ex:doc .Article label}
|
|
95
161
|
|
|
96
|
-
|
|
162
|
+
[Alice] {=ex:alice ?ex:author .prov:Person ex:firstName label}
|
|
163
|
+
[Smith] {ex:lastName}`;
|
|
97
164
|
|
|
98
|
-
|
|
99
|
-
// RDF/JS quads
|
|
165
|
+
const result = parse({ text: mdld });
|
|
166
|
+
console.log(result.quads); // RDF/JS quads
|
|
167
|
+
console.log(result.primary); // { subject, type, label, comment }
|
|
168
|
+
console.log(result.statements); // Elevated statements
|
|
100
169
|
```
|
|
101
170
|
|
|
102
171
|
### Browser (ES Modules)
|
|
@@ -104,11 +173,12 @@ console.log(result.quads);
|
|
|
104
173
|
```html
|
|
105
174
|
<script type="module">
|
|
106
175
|
import { parse } from 'https://cdn.jsdelivr.net/npm/mdld-parse/+esm';
|
|
107
|
-
|
|
108
|
-
const result = parse('[ex] <tag:alice@example.org,2026:>\n\n# Hello {=ex:hello label}');
|
|
176
|
+
const result = parse('[ex] <http://example.org/>\n\n# Hello {=ex:hello label}');
|
|
109
177
|
</script>
|
|
110
178
|
```
|
|
111
179
|
|
|
180
|
+
**Bundle size:** 84KB unminified, 19KB gzipped
|
|
181
|
+
|
|
112
182
|
## 🧠 Semantic Model
|
|
113
183
|
|
|
114
184
|
MD-LD encodes a directed labeled multigraph where three nodes may be in scope:
|
|
@@ -117,7 +187,7 @@ MD-LD encodes a directed labeled multigraph where three nodes may be in scope:
|
|
|
117
187
|
- **O** — object resource (IRI from link/image)
|
|
118
188
|
- **L** — literal value (string + optional datatype/language)
|
|
119
189
|
|
|
120
|
-
### Predicate Routing
|
|
190
|
+
### Predicate Routing
|
|
121
191
|
|
|
122
192
|
Each predicate form determines the graph edge:
|
|
123
193
|
|
|
@@ -127,23 +197,6 @@ Each predicate form determines the graph edge:
|
|
|
127
197
|
| `?p` | S → O | `[NASA] {=ex:nasa ?org}` | object property |
|
|
128
198
|
| `!p` | O → S | `[Parent] {=ex:p !hasPart}` | reverse object |
|
|
129
199
|
|
|
130
|
-
## 📍 Elevated Statements
|
|
131
|
-
|
|
132
|
-
MD-LD automatically detects `rdf:Statement` patterns during parsing and extracts elevated SPO quads for convenient consumption by applications.
|
|
133
|
-
|
|
134
|
-
### Pattern Detection
|
|
135
|
-
|
|
136
|
-
When the parser encounters a complete `rdf:Statement` pattern with `rdf:subject`, `rdf:predicate`, and `rdf:object`, it automatically adds the corresponding SPO quad to the `statements` array:
|
|
137
|
-
|
|
138
|
-
```markdown
|
|
139
|
-
[ex] <http://example.org/>
|
|
140
|
-
|
|
141
|
-
## Elevated statement {=ex:stmt1 .rdf:Statement}
|
|
142
|
-
**Alice** {+ex:alice ?rdf:subject} *knows* {+ex:knows ?rdf:predicate} **Bob** {+ex:bob ?rdf:object}
|
|
143
|
-
|
|
144
|
-
Direct statement:**Alice** {=ex:alice} knows **Bob** {?ex:knows +ex:bob}
|
|
145
|
-
```
|
|
146
|
-
|
|
147
200
|
## 🎨 Syntax Quick Reference
|
|
148
201
|
|
|
149
202
|
### Subject Declaration
|
|
@@ -152,19 +205,10 @@ Set current subject (emits no quads):
|
|
|
152
205
|
## Apollo 11 {=ex:apollo11}
|
|
153
206
|
```
|
|
154
207
|
|
|
155
|
-
### Fragment Syntax
|
|
156
|
-
Create fragment IRIs relative to current subject:
|
|
157
|
-
```markdown
|
|
158
|
-
# Document {=ex:document}
|
|
159
|
-
{=#summary}
|
|
160
|
-
[Content] {label}
|
|
161
|
-
```
|
|
162
|
-
Fragments replace any existing fragment and require a current subject.
|
|
163
|
-
|
|
164
208
|
### Type Declaration
|
|
165
209
|
Emit `rdf:type` triple:
|
|
166
210
|
```markdown
|
|
167
|
-
## Apollo 11 {=ex:apollo11 .
|
|
211
|
+
## Apollo 11 {=ex:apollo11 .SpaceMission .Event}
|
|
168
212
|
```
|
|
169
213
|
|
|
170
214
|
### Literal Properties
|
|
@@ -184,50 +228,47 @@ Links create relationships (use `?` prefix):
|
|
|
184
228
|
```
|
|
185
229
|
|
|
186
230
|
### Resource Declaration
|
|
187
|
-
Declare resources inline with `{
|
|
231
|
+
Declare resources inline with `{+iri}`:
|
|
188
232
|
```markdown
|
|
189
233
|
# Mission {=ex:apollo11}
|
|
190
|
-
[Neil Armstrong] {
|
|
234
|
+
[Neil Armstrong] {+ex:armstrong ?ex:commander .Person}
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
### Diff Authoring (Polarity)
|
|
238
|
+
Use `+` and `-` for retractions:
|
|
239
|
+
```markdown
|
|
240
|
+
# Update value
|
|
241
|
+
[Person] {=ex:person}
|
|
242
|
+
[Alice] {-ex:name}
|
|
243
|
+
[Bob] {ex:name}
|
|
244
|
+
|
|
245
|
+
# Remove triple
|
|
246
|
+
[Person] {=ex:person}
|
|
247
|
+
[Alice] {-ex:friend}
|
|
191
248
|
```
|
|
192
249
|
|
|
193
250
|
## 🔧 API Reference
|
|
194
251
|
|
|
195
252
|
### `parse({ text, context, dataFactory, graph })`
|
|
196
253
|
|
|
197
|
-
Parse
|
|
254
|
+
Parse MDLD to RDF quads with lean origin tracking.
|
|
198
255
|
|
|
199
|
-
**Parameters
|
|
200
|
-
- `text` (string, required) —
|
|
201
|
-
- `context` (object, optional) — Prefix mappings
|
|
256
|
+
**Parameters:**
|
|
257
|
+
- `text` (string, required) — MDLD formatted text
|
|
258
|
+
- `context` (object, optional) — Prefix mappings
|
|
202
259
|
- `dataFactory` (object, optional) — Custom RDF/JS DataFactory
|
|
203
260
|
- `graph` (string, optional) — Named graph IRI
|
|
204
261
|
|
|
205
262
|
**Returns:** `{ quads, remove, statements, origin, context, primarySubject, primary, md }`
|
|
206
263
|
|
|
207
|
-
- `quads` —
|
|
208
|
-
- `remove` —
|
|
209
|
-
- `statements` —
|
|
210
|
-
- `origin` — Lean origin tracking
|
|
211
|
-
- `context` — Final context
|
|
264
|
+
- `quads` — RDF/JS Quads (final resolved graph state)
|
|
265
|
+
- `remove` — RDF/JS Quads (external retractions for diff workflows)
|
|
266
|
+
- `statements` — Elevated SPO quads from rdf:Statement patterns
|
|
267
|
+
- `origin` — Lean origin tracking: `quadIndex`, `blocks`, `spans`, `documentStructure`
|
|
268
|
+
- `context` — Final context with prefixes
|
|
212
269
|
- `primarySubject` — String IRI or null (canonical append identity)
|
|
213
|
-
- `primary` —
|
|
214
|
-
- `md` — Clean Markdown with
|
|
215
|
-
|
|
216
|
-
**Dual-Layer Architecture:**
|
|
217
|
-
|
|
218
|
-
| Layer | Field | Purpose | Use Cases |
|
|
219
|
-
|-------|-------|---------|-----------|
|
|
220
|
-
| **Canonical Identity** | `primarySubject` | Append routing, storage, synchronization | `append()`, file placement, authority validation |
|
|
221
|
-
| **Semantic Surface** | `primary` | UI, indexing, navigation, agent orientation | Dashboards, search, previews, timelines |
|
|
222
|
-
|
|
223
|
-
**Primary Object Structure:**
|
|
224
|
-
```javascript
|
|
225
|
-
primary: {
|
|
226
|
-
subject: string | null, // First non-fragment subject declaration
|
|
227
|
-
type: string | null, // First rdf:type declaration
|
|
228
|
-
label: string | null // First rdfs:label literal
|
|
229
|
-
}
|
|
230
|
-
```
|
|
270
|
+
- `primary` — Primary metadata quartet: `{ subject, type, label, comment }`
|
|
271
|
+
- `md` — Clean Markdown with annotations stripped
|
|
231
272
|
|
|
232
273
|
### `merge(docs, options)`
|
|
233
274
|
|
|
@@ -236,265 +277,161 @@ Merge multiple MDLD documents with diff polarity resolution.
|
|
|
236
277
|
**Parameters:**
|
|
237
278
|
- `docs` (array) — Array of markdown strings or ParseResult objects
|
|
238
279
|
- `options` (object, optional):
|
|
239
|
-
- `context` (object) — Prefix mappings
|
|
280
|
+
- `context` (object) — Prefix mappings
|
|
240
281
|
|
|
241
|
-
**Returns:** `{ quads, remove, origin, context, primarySubjects, primary }`
|
|
282
|
+
**Returns:** `{ quads, remove, statements, origin, context, primarySubjects, primary }`
|
|
242
283
|
|
|
243
|
-
- `quads` —
|
|
244
|
-
- `remove` —
|
|
284
|
+
- `quads` — RDF/JS Quads (final resolved graph state)
|
|
285
|
+
- `remove` — RDF/JS Quads (external retractions)
|
|
286
|
+
- `statements` — Elevated statements from all documents
|
|
245
287
|
- `origin` — Merge origin with document tracking
|
|
246
|
-
- `context` — Final context
|
|
247
|
-
- `primarySubjects` — Array of string IRIs (canonical
|
|
248
|
-
- `primary` — Array of primary objects
|
|
249
|
-
|
|
250
|
-
**Dual-Layer Architecture:**
|
|
251
|
-
|
|
252
|
-
| Layer | Field | Purpose | Use Cases |
|
|
253
|
-
|-------|-------|---------|-----------|
|
|
254
|
-
| **Canonical Identity** | `primarySubjects` | Append routing, storage, synchronization | Multi-document append, file organization |
|
|
255
|
-
| **Semantic Surface** | `primary` | UI, indexing, navigation, agent orientation | Vault indexing, document discovery, search |
|
|
256
|
-
|
|
257
|
-
**Primary Object Array Structure:**
|
|
258
|
-
```javascript
|
|
259
|
-
primary: [
|
|
260
|
-
{
|
|
261
|
-
subject: string | null, // First non-fragment subject declaration
|
|
262
|
-
type: string | null, // First rdf:type declaration
|
|
263
|
-
label: string | null // First rdfs:label literal
|
|
264
|
-
},
|
|
265
|
-
// ... one object per document
|
|
266
|
-
]
|
|
267
|
-
```
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
### `generate({ quads, context, primarySubject })`
|
|
271
|
-
|
|
272
|
-
Generate deterministic MDLD from RDF quads with visual styling.
|
|
273
|
-
|
|
274
|
-
**Parameters (named object):**
|
|
275
|
-
- `quads` (array, required) — Array of RDF/JS Quads to convert
|
|
276
|
-
- `context` (object, optional) — Prefix mappings (default: `{}`)
|
|
277
|
-
- `primarySubject` (string, optional) — String IRI to place first in output (ensures round-trip safety). If not provided, falls back to the first subject from quads.
|
|
278
|
-
|
|
279
|
-
**Returns:** `{ text, context }`
|
|
280
|
-
|
|
281
|
-
**Features:**
|
|
282
|
-
- Visual carrier styles based on datatype (code spans for numbers, bold booleans, etc.)
|
|
283
|
-
- Label-in-heading: Uses `rdfs:label` in subject headings when available
|
|
284
|
-
- Multiple labels: First label in heading, additional labels rendered as literals
|
|
285
|
-
- Round-trip safe: All data preserved through parse → generate → parse
|
|
286
|
-
- Composable: `generate(parse(text))` extracts semantics; `parse(generate({quads}))` normalizes quads
|
|
288
|
+
- `context` — Final context with prefixes
|
|
289
|
+
- `primarySubjects` — Array of string IRIs (canonical identities)
|
|
290
|
+
- `primary` — Array of primary metadata objects
|
|
287
291
|
|
|
288
|
-
|
|
292
|
+
**Use case:** CRDT-style state management with append-only documents.
|
|
289
293
|
|
|
290
|
-
|
|
294
|
+
### `generate({ quads, context, primarySubject, compactInline, renderReverse, remove, lang })`
|
|
291
295
|
|
|
292
|
-
|
|
293
|
-
- `quads` (array, required) — Array of RDF/JS Quads to search
|
|
294
|
-
- `focusIRI` (string, required) — The IRI to center the view on
|
|
295
|
-
- `context` (object, optional) — Prefix mappings (default: `{}`)
|
|
296
|
+
Generate deterministic MDLD from RDF quads.
|
|
296
297
|
|
|
297
|
-
**
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
-
|
|
301
|
-
-
|
|
302
|
-
-
|
|
303
|
-
|
|
304
|
-
|
|
298
|
+
**Parameters:**
|
|
299
|
+
- `quads` (array, required) — RDF/JS Quads to convert
|
|
300
|
+
- `context` (object, optional) — Prefix mappings
|
|
301
|
+
- `primarySubject` (string, optional) — IRI to place first in output
|
|
302
|
+
- `compactInline` (boolean, optional) — Inline type/label compaction (default: `false`)
|
|
303
|
+
- `renderReverse` (boolean, optional) — Reverse connections as `!p` (default: `false`)
|
|
304
|
+
- `remove` (array, optional) — RDF/JS Quads to retract (for diff generation)
|
|
305
|
+
- `lang` (string, optional) — Preferred language for labels (e.g., `'en'`, `'es'`, `'fr'`). Priority: specified lang → untagged → English → any language
|
|
305
306
|
|
|
306
|
-
**
|
|
307
|
+
**Returns:** `{ text, context, compactStats }`
|
|
307
308
|
|
|
308
|
-
|
|
309
|
+
- `text` — Generated MDLD text
|
|
310
|
+
- `context` — Full context with prefixes
|
|
311
|
+
- `compactStats` — Compaction metrics
|
|
309
312
|
|
|
310
|
-
|
|
313
|
+
**Features:** Visual styling, label-in-heading, round-trip safe, diff generation, language preference.
|
|
311
314
|
|
|
315
|
+
**Example with language preference:**
|
|
312
316
|
```javascript
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
# Article {=ex:article .ex:Article}
|
|
318
|
-
|
|
319
|
-
[Alice Smith] {ex:author}`;
|
|
320
|
-
|
|
321
|
-
const result = parse({ text: mdld });
|
|
322
|
-
const authorQuad = result.quads.find(q =>
|
|
323
|
-
q.subject.value === 'http://example.org/article' &&
|
|
324
|
-
q.predicate.value === 'http://example.org/author'
|
|
325
|
-
);
|
|
326
|
-
|
|
327
|
-
const updatedText = updateValue({
|
|
328
|
-
text: mdld,
|
|
329
|
-
quad: authorQuad,
|
|
330
|
-
value: 'Bob Johnson',
|
|
331
|
-
origin: result.origin // optional (auto-parses if not provided)
|
|
317
|
+
const { text } = generate({
|
|
318
|
+
quads: result.quads,
|
|
319
|
+
lang: 'es' // Prefer Spanish labels
|
|
332
320
|
});
|
|
333
|
-
|
|
334
|
-
console.log(updatedText);
|
|
335
|
-
// [ex] <http://example.org/>
|
|
336
|
-
//
|
|
337
|
-
// # Article {=ex:article .ex:Article}
|
|
338
|
-
//
|
|
339
|
-
// [Bob Johnson] {ex:author}
|
|
340
321
|
```
|
|
341
322
|
|
|
342
|
-
|
|
343
|
-
```javascript
|
|
344
|
-
// Original: > 25 {ex:age ^^xsd:integer}
|
|
345
|
-
updateValue({ text, quad, value: '30' });
|
|
346
|
-
// Result: > 30 {ex:age ^^xsd:integer} ← datatype preserved
|
|
323
|
+
### `generateNode({ quads, focusIRI, context, compactInline, renderReverse, lang })`
|
|
347
324
|
|
|
348
|
-
|
|
349
|
-
updateValue({ text, quad, value: 'Good morning' });
|
|
350
|
-
// Result: > Good morning {ex:greeting @en} ← language tag preserved
|
|
351
|
-
```
|
|
325
|
+
Generate node-centric MDLD for a specific IRI.
|
|
352
326
|
|
|
353
327
|
**Parameters:**
|
|
354
|
-
- `
|
|
355
|
-
- `
|
|
356
|
-
- `
|
|
357
|
-
- `
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
**How it works:**
|
|
362
|
-
- Uses `locate()` to find the quad's position in the source text
|
|
363
|
-
- Uses `valueRange` to replace only the carrier text (excluding carrier markers like `[`, `]`)
|
|
364
|
-
- Annotation block `{…}` with predicate, datatype, language is untouched
|
|
365
|
-
- Auto-parses if result not provided (convenient but less efficient)
|
|
366
|
-
|
|
367
|
-
**Fail-safe behavior:**
|
|
368
|
-
- Returns original text if quad cannot be located
|
|
369
|
-
- Returns original text if valueRange is not available
|
|
370
|
-
- Never corrupts the source text
|
|
371
|
-
|
|
372
|
-
**Use case:** Perfect for editor applications that need to update literal values while preserving carrier syntax, datatype annotations, and language tags.
|
|
328
|
+
- `quads` (array, required) — RDF/JS Quads to search
|
|
329
|
+
- `focusIRI` (string, required) — IRI to center view on
|
|
330
|
+
- `context` (object, optional) — Prefix mappings
|
|
331
|
+
- `compactInline` (boolean, optional) — Inline compaction (default: `true`)
|
|
332
|
+
- `renderReverse` (boolean, optional) — Reverse connections (default: `true`)
|
|
333
|
+
- `lang` (string, optional) — Preferred language for labels (e.g., `'en'`, `'es'`, `'fr'`). Priority: specified lang → untagged → English → any language
|
|
373
334
|
|
|
374
|
-
|
|
335
|
+
**Returns:** `{ text, context, compactStats }`
|
|
375
336
|
|
|
376
|
-
|
|
337
|
+
**Safety:** Returns empty text if focusIRI not found (prevents accidental full database rendering).
|
|
377
338
|
|
|
378
|
-
|
|
379
|
-
import { parse, generate, generateNode } from 'mdld-parse';
|
|
380
|
-
|
|
381
|
-
// Pattern 1: parse → generate (semantic extraction)
|
|
382
|
-
const canonical = generate({ ...parse({ text, context }) });
|
|
383
|
-
// text → quads → canonical MDLD (deterministic, visual styling applied)
|
|
384
|
-
|
|
385
|
-
// Pattern 2: generate → parse (normalize external RDF)
|
|
386
|
-
const normalized = parse({ ...generate({ quads: externalQuads, context }) });
|
|
387
|
-
// external quads → MDLD → validated quads (DataFactory-safe, no blank nodes)
|
|
388
|
-
|
|
389
|
-
// Pattern 3: parse → generateNode (node-centric exploration)
|
|
390
|
-
const nodeView = generateNode({ ...parse({ text }), focusIRI: 'http://example.org/alice' });
|
|
391
|
-
// full graph → isolated node view (safe: returns empty if IRI not found)
|
|
392
|
-
```
|
|
393
|
-
|
|
394
|
-
**Why this works:**
|
|
395
|
-
- `parse()` returns `{ quads, context, primarySubject, md, ... }`
|
|
396
|
-
- `generate()` accepts `{ quads, context, primarySubject }`
|
|
397
|
-
- `generateNode()` accepts `{ quads, context, focusIRI }` (with focusIRI override)
|
|
398
|
-
- Perfect shape alignment enables elegant `{ ...spread }` composition
|
|
399
|
-
|
|
400
|
-
### The `md` Field — Clean Markdown Extraction
|
|
401
|
-
|
|
402
|
-
Every `parse()` result includes a `md` field containing the original Markdown with all MD-LD annotations stripped:
|
|
403
|
-
|
|
404
|
-
```javascript
|
|
405
|
-
const result = parse({
|
|
406
|
-
text: `# Document {=ex:doc .Article}\n[Content] {ex:content}`,
|
|
407
|
-
context: { ex: 'http://example.org/' }
|
|
408
|
-
});
|
|
409
|
-
|
|
410
|
-
console.log(result.md);
|
|
411
|
-
// # Document\nContent
|
|
412
|
-
|
|
413
|
-
// Round-trip safety: re-parsing clean MD produces zero quads
|
|
414
|
-
const reparsed = parse({ text: result.md });
|
|
415
|
-
console.log(reparsed.quads.length); // 0
|
|
416
|
-
```
|
|
417
|
-
|
|
418
|
-
**Behavior:**
|
|
419
|
-
- Valid MD-LD annotations (`{=...}`, `{+...}`, `{...}`) are completely removed
|
|
420
|
-
- Content from value carriers (`[text]`, `**bold**`, `` `code` ``) is preserved
|
|
421
|
-
- Invalid syntax (annotations not at end-of-line) is preserved as visible markers
|
|
422
|
-
- Headings, lists, blockquotes, code blocks maintain their structure
|
|
423
|
-
- Prefix declarations at start of line are stripped
|
|
424
|
-
- Standalone subject declarations (`{=ex:subject}`) are stripped
|
|
425
|
-
|
|
426
|
-
**Use cases:**
|
|
427
|
-
- **Content extraction** — Get readable Markdown without semantic markup
|
|
428
|
-
- **Syntax validation** — Remaining `{...}` patterns indicate invalid MD-LD syntax
|
|
429
|
-
- **Round-trip testing** — `parse(md).md` should parse to zero quads
|
|
430
|
-
- **Preview generation** — Show clean document before publishing
|
|
339
|
+
### `updateValue({ text, quad, value, origin })`
|
|
431
340
|
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
Locate the origin entry for a quad using the lean origin system.
|
|
341
|
+
Update carrier text of a literal quad in MDLD text.
|
|
435
342
|
|
|
436
343
|
**Parameters:**
|
|
437
|
-
- `
|
|
438
|
-
- `
|
|
344
|
+
- `text` (string) — Original MDLD text
|
|
345
|
+
- `quad` (object) — Quad to update
|
|
346
|
+
- `value` (string) — New carrier text
|
|
347
|
+
- `origin` (object, optional) — ParseResult.origin
|
|
439
348
|
|
|
440
|
-
**Returns:**
|
|
349
|
+
**Returns:** Updated MDLD text (fail-safe)
|
|
441
350
|
|
|
442
|
-
|
|
443
|
-
- `valueRange`: Character range excluding carrier markers (null if not available)
|
|
351
|
+
**Use case:** Editor applications updating literal values.
|
|
444
352
|
|
|
445
|
-
### `
|
|
446
|
-
|
|
447
|
-
Render RDF quads as HTML+RDFa for web display.
|
|
353
|
+
### `locate(quad, origin)`
|
|
448
354
|
|
|
449
|
-
|
|
450
|
-
- `quads` (array) — Array of RDF/JS Quads to render
|
|
451
|
-
- `options` (object, optional):
|
|
452
|
-
- `context` (object) — Prefix mappings for CURIE shortening
|
|
453
|
-
- `baseIRI` (string) — Base IRI for resolving relative references
|
|
355
|
+
Locate quad origin entry for UI navigation.
|
|
454
356
|
|
|
455
|
-
**Returns:** `{
|
|
357
|
+
**Returns:** `{ blockId, range, valueRange, carrierType, ... }` or `null`
|
|
456
358
|
|
|
457
359
|
### Utility Functions
|
|
458
360
|
|
|
459
361
|
```javascript
|
|
460
362
|
import {
|
|
461
363
|
DEFAULT_CONTEXT, // Default prefix mappings
|
|
462
|
-
DataFactory, // RDF/JS DataFactory
|
|
463
|
-
hash, // String hashing
|
|
464
|
-
expandIRI, // IRI expansion
|
|
465
|
-
shortenIRI, // IRI shortening
|
|
466
|
-
parseSemanticBlock //
|
|
364
|
+
DataFactory, // RDF/JS DataFactory
|
|
365
|
+
hash, // String hashing
|
|
366
|
+
expandIRI, // IRI expansion
|
|
367
|
+
shortenIRI, // IRI shortening
|
|
368
|
+
parseSemanticBlock // Semantic block parsing
|
|
467
369
|
} from 'mdld-parse';
|
|
468
370
|
```
|
|
469
371
|
|
|
470
372
|
## 🏗️ Architecture
|
|
471
373
|
|
|
472
374
|
### Design Principles
|
|
473
|
-
- **Zero dependencies** — Pure JavaScript,
|
|
375
|
+
- **Zero dependencies** — Pure JavaScript, 84KB unminified (19KB gzipped)
|
|
474
376
|
- **Streaming-first** — Single-pass parsing, O(n) complexity
|
|
475
|
-
- **
|
|
476
|
-
- **
|
|
377
|
+
- **Character-based tokenization** — 20-28% faster than regex-based approaches
|
|
378
|
+
- **Standards-compliant** — RDF/JS data model, W3C CURIE 1.0 syntax
|
|
379
|
+
- **Deterministic** — Same input always produces same output
|
|
477
380
|
- **Explicit semantics** — No guessing, inference, or heuristics
|
|
381
|
+
- **Dual-layer origin** — Every parse emits both a semantic quad graph and a walkable textual topology graph simultaneously
|
|
382
|
+
|
|
383
|
+
### Origin: Blocks and Spans
|
|
384
|
+
|
|
385
|
+
The parser output includes a complete document chain at no extra cost:
|
|
386
|
+
|
|
387
|
+
```
|
|
388
|
+
[Block] --(Span)-- [Block] --(Span)-- [Block]
|
|
389
|
+
```
|
|
390
|
+
|
|
391
|
+
- **Blocks** (`origin.blocks`) — semantic anchors: tokens that produced RDF quads, with `prevSpanId`/`nextSpanId` links
|
|
392
|
+
- **Spans** (`origin.spans`) — textual observations: raw byte ranges between blocks, with bidirectional block and span links
|
|
393
|
+
|
|
394
|
+
Spans store no text — content is always recovered via `sourceText.slice(span.range[0], span.range[1])`. This unlocks context-aware UI, autocomplete neighborhood retrieval, and cross-document topology without any parser-level interpretation.
|
|
395
|
+
|
|
396
|
+
### Performance Characteristics
|
|
397
|
+
- **Real-time (60fps):** Up to 4,527 quads per frame
|
|
398
|
+
- **Batch processing:** Up to 225,059 quads per second
|
|
399
|
+
- **Memory efficient:** ~640 bytes per quad retained after GC
|
|
400
|
+
- **Streaming-friendly:** Full document never in memory
|
|
478
401
|
|
|
479
402
|
### RDF/JS Compatibility
|
|
480
|
-
Quads
|
|
403
|
+
Quads work with:
|
|
481
404
|
- [`n3.js`](https://github.com/rdfjs/N3.js) — Turtle/N-Triples serialization
|
|
482
|
-
- [`rdflib.js`](https://github.com/linkeddata/rdflib.js) — RDF
|
|
405
|
+
- [`rdflib.js`](https://github.com/linkeddata/rdflib.js) — RDF stores
|
|
483
406
|
- [`sparqljs`](https://github.com/RubenVerborgh/SPARQL.js) — SPARQL queries
|
|
484
|
-
- [`rdf-ext`](https://github.com/rdf-ext/rdf-ext) —
|
|
407
|
+
- [`rdf-ext`](https://github.com/rdf-ext/rdf-ext) — RDF utilities
|
|
485
408
|
|
|
486
|
-
|
|
409
|
+
### Standards Compliance
|
|
410
|
+
- **RDF 1.1** — Core RDF concepts
|
|
411
|
+
- **RDFS** — Schema vocabulary
|
|
412
|
+
- **PROV-O** — Provenance ontology
|
|
413
|
+
- **SHACL** — Constraint validation
|
|
414
|
+
- **W3C CURIE 1.0** — Compact URI syntax
|
|
487
415
|
|
|
488
|
-
|
|
416
|
+
## 🧪 Testing
|
|
489
417
|
|
|
490
418
|
```bash
|
|
491
419
|
pnpm test
|
|
492
420
|
```
|
|
493
421
|
|
|
494
|
-
|
|
495
|
-
-
|
|
496
|
-
-
|
|
497
|
-
-
|
|
498
|
-
-
|
|
499
|
-
-
|
|
500
|
-
- Round-trip
|
|
422
|
+
Comprehensive test suite covering:
|
|
423
|
+
- Syntax parsing and tokenization
|
|
424
|
+
- Context management and prefix folding
|
|
425
|
+
- Polarity system and retractions
|
|
426
|
+
- Elevated statements detection
|
|
427
|
+
- Primary metadata extraction
|
|
428
|
+
- Round-trip parse/generate cycles
|
|
429
|
+
- Origin tracking and provenance
|
|
430
|
+
|
|
431
|
+
## 📚 Additional Resources
|
|
432
|
+
|
|
433
|
+
- **[Documentation](./docs/index.md)** — Complete guides and references
|
|
434
|
+
- **[Specification](./spec/index.md)** — Formal specification
|
|
435
|
+
- **[Examples](./examples/index.md)** — Real-world examples
|
|
436
|
+
- **[Grammar](./grammar/mdld.ebnf)** — EBNF grammar specification
|
|
437
|
+
- **[VS Code Extension](https://marketplace.visualstudio.com/)** — Syntax highlighting (coming soon)
|