mdld-parse 0.9.8 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +243 -262
- package/dist/index.js +1797 -1324
- package/package.json +5 -5
- package/dist/.nojekyll +0 -0
- package/dist/CNAME +0 -1
- package/dist/README.md +0 -431
- package/dist/docs/API.md +0 -435
- package/dist/docs/Architecture.md +0 -190
- package/dist/docs/Guide.md +0 -816
- package/dist/docs/LLM-MDLD-Inducer.md +0 -550
- package/dist/docs/LiterateRDF.md +0 -472
- package/dist/docs/Parser.md +0 -720
- package/dist/docs/Performance.md +0 -238
- package/dist/docs/Spec-render.md +0 -1439
- package/dist/docs/Subject.md +0 -180
- package/dist/docs/Syntax.md +0 -463
- package/dist/docs/TodoList.md +0 -215
- package/dist/docs/Use-Cases.md +0 -354
- package/dist/docs/guide-m.md +0 -56
- package/dist/docs/index.html +0 -108
- package/dist/docs/index.md +0 -56
- package/dist/docs/one-page.md +0 -322
- package/dist/docs/origin.md +0 -459
- package/dist/docs/polarity.md +0 -586
- package/dist/docs/primary-subject.md +0 -326
- package/dist/docs/statements.md +0 -226
- package/dist/docs/tokenization-performance-report.md +0 -164
- package/dist/examples/Cookbook.md +0 -32
- package/dist/examples/Status-SHACL.md +0 -31
- package/dist/examples/demo/index.md +0 -35
- package/dist/examples/demo/journal.md +0 -60
- package/dist/examples/demo/minimal.md +0 -23
- package/dist/examples/demo/project.md +0 -84
- package/dist/examples/demo/prov.md +0 -107
- package/dist/examples/demo/rdf.md +0 -69
- package/dist/examples/demo/recipe.md +0 -50
- package/dist/examples/demo/research.md +0 -84
- package/dist/examples/demo/shacl.md +0 -161
- package/dist/examples/demo/space-mission.md +0 -58
- package/dist/examples/demo/tasks.md +0 -175
- package/dist/examples/demo/xsd.md +0 -91
- package/dist/examples/dogfood.md +0 -288
- package/dist/examples/few-shot.md +0 -309
- package/dist/examples/graph/rdf-graph.html +0 -473
- package/dist/examples/graph/rdf_force_graph_web_component.html +0 -506
- package/dist/examples/index.md +0 -45
- package/dist/examples/llm-time-workflow.md +0 -173
- package/dist/examples/statements.md +0 -43
- package/dist/examples/workflow.md +0 -150
- package/dist/index.html +0 -3023
- package/dist/logo.svg +0 -7
- package/dist/spec/Spec-compact.md +0 -368
- package/dist/spec/Spec-ultra.md +0 -232
- package/dist/spec/Spec.md +0 -1116
- package/dist/spec/index.md +0 -46
package/dist/docs/API.md
DELETED
|
@@ -1,435 +0,0 @@
|
|
|
1
|
-
# MD-LD API Reference
|
|
2
|
-
|
|
3
|
-
Complete API documentation for MD-LD parser with tested examples.
|
|
4
|
-
|
|
5
|
-
## Core Functions
|
|
6
|
-
|
|
7
|
-
### `parse({ text, context, dataFactory, graph })`
|
|
8
|
-
|
|
9
|
-
Parse MD-LD markdown and return RDF quads.
|
|
10
|
-
|
|
11
|
-
**Parameters (named object):**
|
|
12
|
-
- `text` (string, required) — MD-LD formatted text
|
|
13
|
-
- `context` (object, optional) — Prefix mappings (default: `{}`)
|
|
14
|
-
- `dataFactory` (object, optional) — Custom RDF/JS DataFactory
|
|
15
|
-
- `graph` (string, optional) — Named graph IRI
|
|
16
|
-
|
|
17
|
-
**Returns:** `{ quads, context, primarySubject, primary, origin, remove, statements }`
|
|
18
|
-
|
|
19
|
-
- `quads` — Array of RDF/JS Quads (final resolved graph state)
|
|
20
|
-
- `remove` — Array of RDF/JS Quads (external retractions targeting prior state)
|
|
21
|
-
- `origin` — Lean origin tracking object with quadIndex for UI navigation
|
|
22
|
-
- `context` — Final context used (includes prefixes)
|
|
23
|
-
- `primarySubject` — String IRI or null (canonical append identity)
|
|
24
|
-
- `primary` — Object containing primary metadata (semantic surface descriptor)
|
|
25
|
-
|
|
26
|
-
**Dual-Layer Architecture:**
|
|
27
|
-
|
|
28
|
-
| Layer | Field | Purpose | Use Cases |
|
|
29
|
-
|-------|-------|---------|-----------|
|
|
30
|
-
| **Canonical Identity** | `primarySubject` | Append routing, storage, synchronization | `append()`, file placement, authority validation |
|
|
31
|
-
| **Semantic Surface** | `primary` | UI, indexing, navigation, agent orientation | Dashboards, search, previews, timelines |
|
|
32
|
-
|
|
33
|
-
**Primary Object Structure:**
|
|
34
|
-
```javascript
|
|
35
|
-
primary: {
|
|
36
|
-
subject: string | null, // First non-fragment subject declaration
|
|
37
|
-
type: string | null, // First rdf:type declaration
|
|
38
|
-
label: string | null // First rdfs:label literal
|
|
39
|
-
}
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
> Legacy: `parse(text, options)` supported for backward compatibility (deprecated)
|
|
43
|
-
|
|
44
|
-
#### Basic Example
|
|
45
|
-
|
|
46
|
-
```javascript
|
|
47
|
-
import { parse } from 'mdld-parse';
|
|
48
|
-
|
|
49
|
-
const result = parse(`
|
|
50
|
-
[ex] <http://example.org/>
|
|
51
|
-
|
|
52
|
-
# Document {=ex:doc .ex:Article}
|
|
53
|
-
[Alice] {?ex:author =ex:alice .prov:Person ex:firstName label}
|
|
54
|
-
[Smith] {ex:lastName}`);
|
|
55
|
-
|
|
56
|
-
console.log(result.quads);
|
|
57
|
-
// [
|
|
58
|
-
// {
|
|
59
|
-
// subject: { termType: 'NamedNode', value: 'http://example.org/doc' },
|
|
60
|
-
// predicate: { termType: 'NamedNode', value: 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' },
|
|
61
|
-
// object: { termType: 'NamedNode', value: 'http://example.org/Article' }
|
|
62
|
-
// },
|
|
63
|
-
// {
|
|
64
|
-
// subject: { termType: 'NamedNode', value: 'http://example.org/doc' },
|
|
65
|
-
// predicate: { termType: 'NamedNode', value: 'http://example.org/author' },
|
|
66
|
-
// object: { termType: 'NamedNode', value: 'http://example.org/alice' }
|
|
67
|
-
// },
|
|
68
|
-
// {
|
|
69
|
-
// subject: { termType: 'NamedNode', value: 'http://example.org/alice' },
|
|
70
|
-
// predicate: { termType: 'NamedNode', value: 'http://www.w3.org/2000/01/rdf-schema#label' },
|
|
71
|
-
// object: { termType: 'Literal', value: 'Alice' }
|
|
72
|
-
// },
|
|
73
|
-
// {
|
|
74
|
-
// subject: { termType: 'NamedNode', value: 'http://example.org/alice' },
|
|
75
|
-
// predicate: { termType: 'NamedNode', value: 'http://example.org/firstName' },
|
|
76
|
-
// object: { termType: 'Literal', value: 'Alice' }
|
|
77
|
-
// },
|
|
78
|
-
// {
|
|
79
|
-
// subject: { termType: 'NamedNode', value: 'http://example.org/alice' },
|
|
80
|
-
// predicate: { termType: 'NamedNode', value: 'http://example.org/lastName' },
|
|
81
|
-
// object: { termType: 'Literal', value: 'Smith' }
|
|
82
|
-
// }
|
|
83
|
-
// ]
|
|
84
|
-
```
|
|
85
|
-
|
|
86
|
-
#### Polarity Example
|
|
87
|
-
|
|
88
|
-
```javascript
|
|
89
|
-
const result = parse({
|
|
90
|
-
text: `# Article {=ex:article .ex:Article}
|
|
91
|
-
|
|
92
|
-
[Bob] {+ex:bob -?ex:author} is not an author of this article, it's [Alice] {+ex:alice ?ex:author}.
|
|
93
|
-
`, // Remove previous author
|
|
94
|
-
context: { ex: 'http://example.org/' }
|
|
95
|
-
});
|
|
96
|
-
|
|
97
|
-
console.log(result.quads);
|
|
98
|
-
// [
|
|
99
|
-
// {
|
|
100
|
-
// subject: { termType: 'NamedNode', value: 'http://example.org/article' },
|
|
101
|
-
// predicate: { termType: 'NamedNode', value: 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' },
|
|
102
|
-
// object: { termType: 'NamedNode', value: 'http://example.org/Article' }
|
|
103
|
-
// },
|
|
104
|
-
// {
|
|
105
|
-
// subject: { termType: 'NamedNode', value: 'http://example.org/article' },
|
|
106
|
-
// predicate: { termType: 'NamedNode', value: 'http://example.org/author' },
|
|
107
|
-
// object: { termType: 'NamedNode', value: 'http://example.org/alice' }
|
|
108
|
-
// }
|
|
109
|
-
// ]
|
|
110
|
-
|
|
111
|
-
console.log(result.remove);
|
|
112
|
-
// [] - Empty because Alice author was cancelled in-stream
|
|
113
|
-
```
|
|
114
|
-
|
|
115
|
-
#### External Retraction Example
|
|
116
|
-
|
|
117
|
-
```javascript
|
|
118
|
-
const result2 = parse({ text:
|
|
119
|
-
`# Article {=ex:article}
|
|
120
|
-
[Alice] {+ex:alice -?ex:author}`, // External retract (Alice not in current state)
|
|
121
|
-
context: { ex: 'http://example.org/' } }
|
|
122
|
-
);
|
|
123
|
-
|
|
124
|
-
console.log(result2.remove);
|
|
125
|
-
// [
|
|
126
|
-
// {
|
|
127
|
-
// subject: { termType: 'NamedNode', value: 'http://example.org/article' },
|
|
128
|
-
// predicate: { termType: 'NamedNode', value: 'http://example.org/author' },
|
|
129
|
-
// object: { termType: 'NamedNode', value: 'http://example.org/alice' }
|
|
130
|
-
// }
|
|
131
|
-
// ]
|
|
132
|
-
```
|
|
133
|
-
|
|
134
|
-
### `merge(docs, options)`
|
|
135
|
-
|
|
136
|
-
Merge multiple MDLD documents with diff polarity resolution.
|
|
137
|
-
|
|
138
|
-
**Parameters:**
|
|
139
|
-
- `docs` (array) — Array of markdown strings or ParseResult objects
|
|
140
|
-
- `options` (object, optional):
|
|
141
|
-
- `context` (object) — Prefix mappings (merged with DEFAULT_CONTEXT)
|
|
142
|
-
|
|
143
|
-
**Returns:** `{ quads, remove, origin, context, primarySubjects, primary }`
|
|
144
|
-
|
|
145
|
-
- `quads` — Merged array of RDF/JS Quads
|
|
146
|
-
- `remove` — Array of retractions from merge process
|
|
147
|
-
- `origin` — Merge origin with document chain:
|
|
148
|
-
- `documents` — Array of document metadata
|
|
149
|
-
- `quadIndex` — Combined quad index from all documents
|
|
150
|
-
- `context` — Final merged context
|
|
151
|
-
- `primarySubjects` — Array of string IRIs (canonical append identities, ordered by merge)
|
|
152
|
-
- `primary` — Array of primary objects (semantic surface descriptors, ordered by merge)
|
|
153
|
-
|
|
154
|
-
**Dual-Layer Architecture:**
|
|
155
|
-
|
|
156
|
-
| Layer | Field | Purpose | Use Cases |
|
|
157
|
-
|-------|-------|---------|-----------|
|
|
158
|
-
| **Canonical Identity** | `primarySubjects` | Append routing, storage, synchronization | Multi-document append, file organization |
|
|
159
|
-
| **Semantic Surface** | `primary` | UI, indexing, navigation, agent orientation | Vault indexing, document discovery, search |
|
|
160
|
-
|
|
161
|
-
**Primary Object Array Structure:**
|
|
162
|
-
```javascript
|
|
163
|
-
primary: [
|
|
164
|
-
{
|
|
165
|
-
subject: string | null, // First non-fragment subject declaration
|
|
166
|
-
type: string | null, // First rdf:type declaration
|
|
167
|
-
label: string | null // First rdfs:label literal
|
|
168
|
-
},
|
|
169
|
-
// ... one object per document
|
|
170
|
-
]
|
|
171
|
-
```
|
|
172
|
-
|
|
173
|
-
#### Basic Merge Example
|
|
174
|
-
|
|
175
|
-
```javascript
|
|
176
|
-
import { merge } from 'mdld-parse';
|
|
177
|
-
|
|
178
|
-
const merged = merge([
|
|
179
|
-
`# Article {=ex:article}
|
|
180
|
-
[Bob] {author}`,
|
|
181
|
-
`# Article {=ex:article}
|
|
182
|
-
[Bob] {-author}
|
|
183
|
-
[Charlie] {author}`
|
|
184
|
-
], { context: { ex: 'http://example.org/' } });
|
|
185
|
-
|
|
186
|
-
console.log(merged.quads.length); // 2 (type + Charlie author)
|
|
187
|
-
console.log(merged.remove.length); // 1 (Bob author removal)
|
|
188
|
-
```
|
|
189
|
-
|
|
190
|
-
#### Version Control Example
|
|
191
|
-
|
|
192
|
-
```javascript
|
|
193
|
-
const v1 = `# Article {=ex:article .ex:Article}
|
|
194
|
-
[Alice] {author}
|
|
195
|
-
[Draft] {status}`;
|
|
196
|
-
|
|
197
|
-
const v2 = `# Article {=ex:article}
|
|
198
|
-
[Alice] {-author} // Retract original
|
|
199
|
-
[Bob] {author} // Add new author
|
|
200
|
-
[Draft] {-status} // Retract draft status
|
|
201
|
-
[Published] {status} // Add published status`;
|
|
202
|
-
|
|
203
|
-
const merged = merge([v1, v2]);
|
|
204
|
-
console.log(merged.quads.map(q => `${q.subject.value.split('/').pop()} ${q.predicate.value} ${q.object.value}`));
|
|
205
|
-
// ['article rdf:type Article', 'article author Bob', 'article status Published']
|
|
206
|
-
```
|
|
207
|
-
|
|
208
|
-
### `generate({ quads, context, primarySubject })`
|
|
209
|
-
|
|
210
|
-
Generate deterministic MDLD from RDF quads with visual styling.
|
|
211
|
-
|
|
212
|
-
**Parameters (named object):**
|
|
213
|
-
- `quads` (array, required) — Array of RDF/JS Quads to convert
|
|
214
|
-
- `context` (object, optional) — Prefix mappings (default: `{}`)
|
|
215
|
-
- `primarySubject` (string, optional) — IRI to place first in output (ensures round-trip safety)
|
|
216
|
-
|
|
217
|
-
**Returns:** `{ text, context }`
|
|
218
|
-
|
|
219
|
-
- `text` — Generated MDLD markdown
|
|
220
|
-
- `context` — Final context used (includes defaults)
|
|
221
|
-
|
|
222
|
-
#### Generate Example
|
|
223
|
-
|
|
224
|
-
```javascript
|
|
225
|
-
import { generate } from 'mdld-parse';
|
|
226
|
-
|
|
227
|
-
const quads = [
|
|
228
|
-
{
|
|
229
|
-
subject: { termType: 'NamedNode', value: 'http://example.org/article' },
|
|
230
|
-
predicate: { termType: 'NamedNode', value: 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type' },
|
|
231
|
-
object: { termType: 'NamedNode', value: 'http://example.org/Article' }
|
|
232
|
-
},
|
|
233
|
-
{
|
|
234
|
-
subject: { termType: 'NamedNode', value: 'http://example.org/article' },
|
|
235
|
-
predicate: { termType: 'NamedNode', value: 'http://example.org/author' },
|
|
236
|
-
object: { termType: 'NamedNode', value: 'http://example.org/alice' }
|
|
237
|
-
}
|
|
238
|
-
];
|
|
239
|
-
|
|
240
|
-
const result = generate({
|
|
241
|
-
quads,
|
|
242
|
-
context: { ex: 'http://example.org/' }
|
|
243
|
-
});
|
|
244
|
-
|
|
245
|
-
console.log(result.text);
|
|
246
|
-
// # Article {=ex:article .ex:Article}
|
|
247
|
-
//
|
|
248
|
-
// > alice {+ex:alice ?ex:author}
|
|
249
|
-
```
|
|
250
|
-
|
|
251
|
-
### `locate(quad, origin)`
|
|
252
|
-
|
|
253
|
-
Locate the origin entry for a quad using the lean origin system.
|
|
254
|
-
|
|
255
|
-
**Parameters:**
|
|
256
|
-
- `quad` (object) — The quad to locate (subject, predicate, object)
|
|
257
|
-
- `origin` (object) — Origin object containing quadIndex
|
|
258
|
-
|
|
259
|
-
**Returns:** `{ blockId, range, carrierType, subject, predicate, context, value, polarity }` or `null`
|
|
260
|
-
|
|
261
|
-
- `blockId` — ID of the containing block
|
|
262
|
-
- `range` — Character range of the carrier in the source text
|
|
263
|
-
- `carrierType` — Type of carrier (heading, blockquote, span)
|
|
264
|
-
- `subject` — Subject IRI of the quad
|
|
265
|
-
- `predicate` — Predicate IRI of the quad
|
|
266
|
-
- `context` — Context object inherited from parsing
|
|
267
|
-
- `value` — Raw carrier text content
|
|
268
|
-
- `polarity` — '+' for assertions, '-' for retractions
|
|
269
|
-
|
|
270
|
-
#### Locate Example
|
|
271
|
-
|
|
272
|
-
```javascript
|
|
273
|
-
import { parse, locate } from 'mdld-parse';
|
|
274
|
-
|
|
275
|
-
const result = parse({ text: mdldText, context: { ex: 'http://example.org/' } });
|
|
276
|
-
const quad = result.quads[0]; // Find a quad to locate
|
|
277
|
-
|
|
278
|
-
const location = locate(quad, result.origin);
|
|
279
|
-
|
|
280
|
-
console.log(location.range); // { start: 38, end: 44 }
|
|
281
|
-
console.log(location.value); // "Alice"
|
|
282
|
-
console.log(location.carrierType); // "blockquote"
|
|
283
|
-
```
|
|
284
|
-
|
|
285
|
-
### `render(quads, options)`
|
|
286
|
-
|
|
287
|
-
Render RDF quads as HTML+RDFa for web display.
|
|
288
|
-
|
|
289
|
-
**Parameters:**
|
|
290
|
-
- `quads` (array) — Array of RDF/JS Quads to render
|
|
291
|
-
- `options` (object, optional):
|
|
292
|
-
- `context` (object) — Prefix mappings for CURIE shortening
|
|
293
|
-
- `baseIRI` (string) — Base IRI for resolving relative references
|
|
294
|
-
|
|
295
|
-
**Returns:** `{ html, context }`
|
|
296
|
-
|
|
297
|
-
- `html` — HTML string with RDFa annotations
|
|
298
|
-
- `context` — Context used for rendering
|
|
299
|
-
|
|
300
|
-
#### Render Example
|
|
301
|
-
|
|
302
|
-
```javascript
|
|
303
|
-
import { parse, render } from 'mdld-parse';
|
|
304
|
-
|
|
305
|
-
const result = parse({ text: mdldText});
|
|
306
|
-
const rendered = render(result.quads, { ex: 'http://example.org/' });
|
|
307
|
-
|
|
308
|
-
console.log(rendered.html);
|
|
309
|
-
// <div typeof="ex:Article">
|
|
310
|
-
// <span property="ex:author">Alice</span>
|
|
311
|
-
// </div>
|
|
312
|
-
```
|
|
313
|
-
|
|
314
|
-
## Utility Functions
|
|
315
|
-
|
|
316
|
-
```javascript
|
|
317
|
-
import {
|
|
318
|
-
DEFAULT_CONTEXT, // Default prefix mappings
|
|
319
|
-
DataFactory, // RDF/JS DataFactory instance
|
|
320
|
-
hash, // String hashing function
|
|
321
|
-
expandIRI, // IRI expansion with context
|
|
322
|
-
shortenIRI, // IRI shortening with context
|
|
323
|
-
parseSemanticBlock // Parse semantic block syntax
|
|
324
|
-
} from 'mdld-parse';
|
|
325
|
-
```
|
|
326
|
-
|
|
327
|
-
### DEFAULT_CONTEXT
|
|
328
|
-
|
|
329
|
-
```javascript
|
|
330
|
-
{
|
|
331
|
-
'@vocab': 'http://www.w3.org/2000/01/rdf-schema#',
|
|
332
|
-
'rdf': 'http://www.w3.org/1999/02/22-rdf-syntax-ns#',
|
|
333
|
-
'rdfs': 'http://www.w3.org/2000/01/rdf-schema#',
|
|
334
|
-
'xsd': 'http://www.w3.org/2001/XMLSchema#',
|
|
335
|
-
'sh': 'http://www.w3.org/ns/shacl#',
|
|
336
|
-
'prov': 'http://www.w3.org/ns/prov#'
|
|
337
|
-
}
|
|
338
|
-
```
|
|
339
|
-
|
|
340
|
-
### DataFactory
|
|
341
|
-
|
|
342
|
-
RDF/JS DataFactory instance for creating RDF terms:
|
|
343
|
-
|
|
344
|
-
```javascript
|
|
345
|
-
DataFactory.namedNode('http://example.org/subject')
|
|
346
|
-
DataFactory.literal('value')
|
|
347
|
-
DataFactory.quad(subject, predicate, object, graph)
|
|
348
|
-
```
|
|
349
|
-
|
|
350
|
-
### Hash Function
|
|
351
|
-
|
|
352
|
-
Generate consistent hashes for identifiers:
|
|
353
|
-
|
|
354
|
-
```javascript
|
|
355
|
-
hash('content') // Returns: '5f4dcc3b5aa765d61d8327deb882cf99'
|
|
356
|
-
```
|
|
357
|
-
|
|
358
|
-
### IRI Expansion/Shortening
|
|
359
|
-
|
|
360
|
-
Context-aware IRI manipulation:
|
|
361
|
-
|
|
362
|
-
```javascript
|
|
363
|
-
const context = { ex: 'http://example.org/' };
|
|
364
|
-
|
|
365
|
-
expandIRI('ex:term', context) // 'http://example.org/term'
|
|
366
|
-
shortenIRI('http://example.org/term', context) // 'ex:term'
|
|
367
|
-
```
|
|
368
|
-
|
|
369
|
-
### Semantic Block Parser
|
|
370
|
-
|
|
371
|
-
Parse individual annotation blocks:
|
|
372
|
-
|
|
373
|
-
```javascript
|
|
374
|
-
parseSemanticBlock('{author}', context) // Returns parsed annotation object
|
|
375
|
-
```
|
|
376
|
-
|
|
377
|
-
## Error Handling
|
|
378
|
-
|
|
379
|
-
### Common Errors
|
|
380
|
-
|
|
381
|
-
- **Invalid syntax** - Malformed annotations or prefixes
|
|
382
|
-
- **Missing context** - Undefined prefixes in IRI expansion
|
|
383
|
-
- **Type errors** - Invalid datatype or language combinations
|
|
384
|
-
|
|
385
|
-
### Error Messages
|
|
386
|
-
|
|
387
|
-
The parser provides clear error messages with line numbers:
|
|
388
|
-
|
|
389
|
-
```javascript
|
|
390
|
-
try {
|
|
391
|
-
const result = parse({text:invalidMarkdown});
|
|
392
|
-
} catch (error) {
|
|
393
|
-
console.error(error.message); // "Invalid annotation at line 5: ..."
|
|
394
|
-
}
|
|
395
|
-
```
|
|
396
|
-
|
|
397
|
-
## Performance Characteristics
|
|
398
|
-
|
|
399
|
-
- **O(n) parsing** - Single pass, linear time complexity
|
|
400
|
-
- **Memory efficient** - Streaming-friendly, minimal state
|
|
401
|
-
- **Deterministic** - Same input always produces same output
|
|
402
|
-
- **Zero dependencies** - Pure JavaScript implementation
|
|
403
|
-
|
|
404
|
-
## RDF/JS Compatibility
|
|
405
|
-
|
|
406
|
-
Generated quads are standard RDF/JS Quad objects compatible with:
|
|
407
|
-
|
|
408
|
-
- **n3.js** - Turtle/N-Triples serialization
|
|
409
|
-
- **rdflib.js** - RDF store and reasoning
|
|
410
|
-
- **sparqljs** - SPARQL queries and updates
|
|
411
|
-
- **rdf-ext** - Extended RDF utilities
|
|
412
|
-
|
|
413
|
-
## Browser Support
|
|
414
|
-
|
|
415
|
-
MD-LD works in modern browsers with ES module support:
|
|
416
|
-
|
|
417
|
-
```html
|
|
418
|
-
<script type="module">
|
|
419
|
-
import { parse } from 'https://cdn.jsdelivr.net/npm/mdld-parse/+esm';
|
|
420
|
-
|
|
421
|
-
const result = parse('# Hello {=ex:hello label}');
|
|
422
|
-
</script>
|
|
423
|
-
```
|
|
424
|
-
|
|
425
|
-
## Node.js Support
|
|
426
|
-
|
|
427
|
-
Full Node.js support with CommonJS and ES modules:
|
|
428
|
-
|
|
429
|
-
```javascript
|
|
430
|
-
// ES Modules
|
|
431
|
-
import { parse } from 'mdld-parse';
|
|
432
|
-
|
|
433
|
-
// CommonJS
|
|
434
|
-
const { parse } = require('mdld-parse');
|
|
435
|
-
```
|
|
@@ -1,190 +0,0 @@
|
|
|
1
|
-
# MDLD Architecture
|
|
2
|
-
|
|
3
|
-
## Design Principles
|
|
4
|
-
|
|
5
|
-
MDLD follows strict engineering principles for reliability and performance:
|
|
6
|
-
|
|
7
|
-
- **Zero dependencies** — Pure JavaScript, ~15KB minified
|
|
8
|
-
- **Streaming-first** — Single-pass parsing, O(n) complexity
|
|
9
|
-
- **Character-based tokenization** — 20-28% faster than regex-based approaches
|
|
10
|
-
- **Memory-efficient** — ~640 bytes per quad retained after GC
|
|
11
|
-
- **Standards-compliant** — RDF/JS data model
|
|
12
|
-
- **Origin tracking** — Full round-trip support with source maps
|
|
13
|
-
- **Explicit semantics** — No guessing, inference, or heuristics
|
|
14
|
-
|
|
15
|
-
## Processing Pipeline
|
|
16
|
-
|
|
17
|
-
Document processing follows this pipeline:
|
|
18
|
-
|
|
19
|
-
1. **Line-by-line scanning** — Sequential token creation
|
|
20
|
-
2. **Context resolution** — Prefix and vocabulary expansion
|
|
21
|
-
3. **Subject tracking** — Current subject management
|
|
22
|
-
4. **Annotation processing** — Semantic block evaluation
|
|
23
|
-
5. **Quad emission** — RDF triple generation
|
|
24
|
-
|
|
25
|
-
## Core Parser Structures
|
|
26
|
-
|
|
27
|
-
### Token Processing Pipeline
|
|
28
|
-
|
|
29
|
-
The parser uses character-based tokenizers for optimal performance:
|
|
30
|
-
|
|
31
|
-
```javascript
|
|
32
|
-
const PROCESSORS = [
|
|
33
|
-
{ type: 'fence', test: line => detectFence(line.trim()), process: handleFence },
|
|
34
|
-
{ type: 'content', test: () => codeBlock, process: line => codeBlock.content.push(line) },
|
|
35
|
-
{ type: 'prefix', test: line => detectPrefix(line), process: handlePrefix },
|
|
36
|
-
{ type: 'standalone', test: line => detectStandaloneSubject(line), process: handleStandaloneSubject },
|
|
37
|
-
{ type: 'heading', test: line => detectHeading(line), process: handleHeading },
|
|
38
|
-
{ type: 'list', test: line => detectList(line), process: handleList },
|
|
39
|
-
{ type: 'blockquote', test: line => detectBlockquote(line), process: handleBlockquote },
|
|
40
|
-
{ type: 'para', test: line => line.trim(), process: handlePara }
|
|
41
|
-
];
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
### Character-Based Tokenization
|
|
45
|
-
|
|
46
|
-
Character-based tokenizers replace regex patterns for performance:
|
|
47
|
-
|
|
48
|
-
- **detectFence()** — Code block fence detection
|
|
49
|
-
- **detectPrefix()** — Namespace prefix declarations
|
|
50
|
-
- **detectHeading()** — Markdown heading detection
|
|
51
|
-
- **detectList()** — List item detection
|
|
52
|
-
- **detectBlockquote()** — Blockquote detection
|
|
53
|
-
- **detectStandaloneSubject()** — Subject annotation detection
|
|
54
|
-
|
|
55
|
-
### Semantic Block Processing
|
|
56
|
-
|
|
57
|
-
The parser processes semantic annotations in context:
|
|
58
|
-
|
|
59
|
-
```javascript
|
|
60
|
-
// Subject declaration
|
|
61
|
-
[entity] {=ex:Entity .rdfs:Class label}
|
|
62
|
-
|
|
63
|
-
// Object introduction
|
|
64
|
-
[related] {+ex:related ?rdfs:subClassOf}
|
|
65
|
-
|
|
66
|
-
// Property assertion
|
|
67
|
-
has [property] {+ex:property ?ex:hasProperty}
|
|
68
|
-
```
|
|
69
|
-
|
|
70
|
-
### Data Structures
|
|
71
|
-
|
|
72
|
-
**Parser State:**
|
|
73
|
-
```javascript
|
|
74
|
-
{
|
|
75
|
-
prefixes: Map<string, string>, // prefix → IRI mappings
|
|
76
|
-
currentSubject: NamedNode|null, // active subject
|
|
77
|
-
codeBlock: CodeBlock|null, // fenced content
|
|
78
|
-
quads: Quad[], // emitted triples
|
|
79
|
-
origins: Origin[] // source tracking
|
|
80
|
-
}
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
**Quad Generation:**
|
|
84
|
-
```javascript
|
|
85
|
-
{
|
|
86
|
-
subject: NamedNode,
|
|
87
|
-
predicate: NamedNode,
|
|
88
|
-
object: Literal|NamedNode,
|
|
89
|
-
graph: NamedNode
|
|
90
|
-
}
|
|
91
|
-
```
|
|
92
|
-
|
|
93
|
-
## Performance Architecture
|
|
94
|
-
|
|
95
|
-
### Memory Management
|
|
96
|
-
|
|
97
|
-
- **Streaming parsing** — Full document never in memory
|
|
98
|
-
- **Lazy evaluation** — Context computed on demand
|
|
99
|
-
- **Efficient indexing** — O(1) subject/object/predicate lookup
|
|
100
|
-
- **Garbage collection** — Minimal retained state
|
|
101
|
-
|
|
102
|
-
### Character-Based Optimization
|
|
103
|
-
|
|
104
|
-
Character-based tokenizers provide:
|
|
105
|
-
- **20-28% faster** parsing than regex-based approaches
|
|
106
|
-
- **Predictable performance** — O(n) complexity
|
|
107
|
-
- **Memory efficiency** — No regex engine overhead
|
|
108
|
-
- **Maintainability** — Clear character logic
|
|
109
|
-
|
|
110
|
-
### Scaling Architecture
|
|
111
|
-
|
|
112
|
-
**Real-time Applications (60fps):**
|
|
113
|
-
- Maximum: 4,527 quads per frame
|
|
114
|
-
- Use case: Interactive knowledge graphs
|
|
115
|
-
- Pattern: Incremental updates only
|
|
116
|
-
|
|
117
|
-
**Batch Processing (1-second):**
|
|
118
|
-
- Maximum: 225,059 quads per second
|
|
119
|
-
- Use case: Background reindexing, imports
|
|
120
|
-
- Pattern: Worker thread, chunked processing
|
|
121
|
-
|
|
122
|
-
**Enterprise Scale:**
|
|
123
|
-
- Small (<4K quads): Real-time updates
|
|
124
|
-
- Medium (4-225K quads): Batch reindexing
|
|
125
|
-
- Large (>225K quads): Streaming architecture
|
|
126
|
-
|
|
127
|
-
## Extensibility
|
|
128
|
-
|
|
129
|
-
### Plugin Architecture
|
|
130
|
-
|
|
131
|
-
The parser supports extensions through:
|
|
132
|
-
|
|
133
|
-
- **Custom tokenizers** — Add new block types
|
|
134
|
-
- **Semantic processors** — Custom annotation handling
|
|
135
|
-
- **Output formats** — Alternative serialization
|
|
136
|
-
- **Validation hooks** — Custom constraint checking
|
|
137
|
-
|
|
138
|
-
### Integration Points
|
|
139
|
-
|
|
140
|
-
- **Parse hooks** — Pre/post processing
|
|
141
|
-
- **Context injection** — Dynamic prefix resolution
|
|
142
|
-
- **Error handling** — Custom error strategies
|
|
143
|
-
- **Performance monitoring** — Built-in profiling
|
|
144
|
-
|
|
145
|
-
## Standards Compliance
|
|
146
|
-
|
|
147
|
-
### RDF/JS Data Model
|
|
148
|
-
|
|
149
|
-
MDLD implements the RDF/JS data model specification:
|
|
150
|
-
- **NamedNode** — IRIs and CURIEs
|
|
151
|
-
- **Literal** — Typed and language-tagged values
|
|
152
|
-
- **Quad** — RDF triples with graph context
|
|
153
|
-
- **Dataset** — Quad collections with indexing
|
|
154
|
-
|
|
155
|
-
### W3C Standards
|
|
156
|
-
|
|
157
|
-
- **RDF 1.1** — Core RDF concepts
|
|
158
|
-
- **RDFS** — Schema vocabulary
|
|
159
|
-
- **PROV-O** — Provenance ontology
|
|
160
|
-
- **SHACL** — Constraint validation
|
|
161
|
-
- **CURIE 1.0** — Compact URI syntax
|
|
162
|
-
|
|
163
|
-
## Implementation Notes
|
|
164
|
-
|
|
165
|
-
### Single-Pass Design
|
|
166
|
-
|
|
167
|
-
The parser processes documents in a single forward pass:
|
|
168
|
-
- No backtracking or look-ahead beyond current line
|
|
169
|
-
- Streaming-friendly for large documents
|
|
170
|
-
- Predictable memory usage
|
|
171
|
-
- Linear time complexity
|
|
172
|
-
|
|
173
|
-
### Error Handling
|
|
174
|
-
|
|
175
|
-
- **Graceful degradation** — Continue parsing on errors
|
|
176
|
-
- **Context preservation** — Maintain parser state
|
|
177
|
-
- **Detailed reporting** — Line/column information
|
|
178
|
-
- **Recovery strategies** — Skip invalid content
|
|
179
|
-
|
|
180
|
-
### Testing Strategy
|
|
181
|
-
|
|
182
|
-
- **Unit tests** — Individual tokenizer validation
|
|
183
|
-
- **Integration tests** — End-to-end parsing
|
|
184
|
-
- **Performance tests** — Benchmarking at scale
|
|
185
|
-
- **Real-world tests** — Actual ontology validation
|
|
186
|
-
|
|
187
|
-
---
|
|
188
|
-
|
|
189
|
-
For detailed parser internals, see **[Parser Documentation](./Parser.md)**.
|
|
190
|
-
For performance metrics and testing methods, see **[Performance Documentation](./Performance.md)**.
|