mdld-parse 0.9.9 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. package/README.md +242 -330
  2. package/dist/index.js +1793 -1336
  3. package/package.json +4 -4
  4. package/dist/.nojekyll +0 -0
  5. package/dist/CNAME +0 -1
  6. package/dist/README.md +0 -500
  7. package/dist/docs/API.md +0 -435
  8. package/dist/docs/Architecture.md +0 -190
  9. package/dist/docs/Guide.md +0 -816
  10. package/dist/docs/LLM-MDLD-Inducer.md +0 -550
  11. package/dist/docs/LiterateRDF.md +0 -472
  12. package/dist/docs/Parser.md +0 -720
  13. package/dist/docs/Performance.md +0 -238
  14. package/dist/docs/Spec-render.md +0 -1439
  15. package/dist/docs/Subject.md +0 -180
  16. package/dist/docs/Syntax.md +0 -463
  17. package/dist/docs/TodoList.md +0 -215
  18. package/dist/docs/Use-Cases.md +0 -354
  19. package/dist/docs/guide-m.md +0 -56
  20. package/dist/docs/index.html +0 -108
  21. package/dist/docs/index.md +0 -56
  22. package/dist/docs/one-page.md +0 -322
  23. package/dist/docs/origin.md +0 -460
  24. package/dist/docs/polarity.md +0 -586
  25. package/dist/docs/primary-subject.md +0 -326
  26. package/dist/docs/statements.md +0 -226
  27. package/dist/docs/tokenization-performance-report.md +0 -164
  28. package/dist/examples/Cookbook.md +0 -32
  29. package/dist/examples/Status-SHACL.md +0 -31
  30. package/dist/examples/demo/index.md +0 -35
  31. package/dist/examples/demo/journal.md +0 -58
  32. package/dist/examples/demo/minimal.md +0 -23
  33. package/dist/examples/demo/project.md +0 -84
  34. package/dist/examples/demo/prov.md +0 -107
  35. package/dist/examples/demo/rdf.md +0 -69
  36. package/dist/examples/demo/recipe.md +0 -50
  37. package/dist/examples/demo/research.md +0 -84
  38. package/dist/examples/demo/shacl.md +0 -161
  39. package/dist/examples/demo/space-mission.md +0 -58
  40. package/dist/examples/demo/tasks.md +0 -175
  41. package/dist/examples/demo/xsd.md +0 -91
  42. package/dist/examples/dogfood.md +0 -288
  43. package/dist/examples/few-shot.md +0 -309
  44. package/dist/examples/graph/rdf-graph.html +0 -473
  45. package/dist/examples/graph/rdf_force_graph_web_component.html +0 -506
  46. package/dist/examples/index.md +0 -45
  47. package/dist/examples/llm-time-workflow.md +0 -173
  48. package/dist/examples/statements.md +0 -43
  49. package/dist/examples/workflow.md +0 -150
  50. package/dist/index.html +0 -3049
  51. package/dist/logo.svg +0 -7
  52. package/dist/spec/Spec-compact.md +0 -368
  53. package/dist/spec/Spec-ultra.md +0 -232
  54. package/dist/spec/Spec.md +0 -1116
  55. package/dist/spec/index.md +0 -46
package/dist/logo.svg DELETED
@@ -1,7 +0,0 @@
1
- <svg width="1000" height="1000" xmlns="http://www.w3.org/2000/svg">
2
-
3
- <text x="500" y="500" font-family="Consolas, Menlo, Monospace, monospace" font-size="210" text-anchor="middle" style="font-weight: bold">
4
- <tspan fill="gray" style="opacity:0.5">{</tspan><tspan fill="#d9711f" >=+ .?!</tspan><tspan fill="gray" style="opacity:0.5">}</tspan>
5
- </text>
6
-
7
- </svg>
@@ -1,368 +0,0 @@
1
- # MD-LD — Core Specification (Minimal)
2
-
3
- **Markdown-Linked Data**
4
-
5
- A deterministic, streaming-friendly semantic annotation layer for CommonMark Markdown that emits RDF quads **only** from explicit `{...}` annotations.
6
-
7
- ---
8
-
9
- ## 1. Core Guarantees
10
-
11
- 1. **Markdown-preserving** - Removing `{...}` yields valid Markdown
12
- 2. **No implicit semantics** - Everything must be explicit
13
- 3. **Explicit origin** - All quads come from `{...}` blocks
14
- 4. **Single-pass streaming** - No backtracking required
15
- 5. **No blank nodes** - Every entity has an IRI
16
- 6. **No inference** - No guessing or structural heuristics
17
- 7. **Traceable** - Every quad traces to source span
18
-
19
- ---
20
-
21
- ## 2. Graph Model
22
-
23
- MD-LD creates facts: **Subject → Predicate → Object**
24
-
25
- **Building blocks:**
26
- - **IRIs** - Unique identifiers (subjects and objects)
27
- - **Literals** - Data values (objects only)
28
-
29
- **Rules:**
30
- - IRIs can be subjects or objects
31
- - Literals are always objects
32
- - Creates a directed labeled multigraph
33
-
34
- ---
35
-
36
- ## 3. Available Nodes (S, O, L)
37
-
38
- When parsing `{...}`, MD-LD knows:
39
-
40
- - **S** - Current subject (IRI, from `{=IRI}` or `{=#fragment}`)
41
- - **O** - Object resource (IRI, from links/images or `{=IRI}`)
42
- - **L** - Literal value (text from value carrier)
43
-
44
- ---
45
-
46
- ## 4. Value Carriers
47
-
48
- `{...}` attaches to exactly one value carrier:
49
-
50
- ### Inline
51
- - `[text] {...}` - Link text
52
- - `*text* {...}`, `_text_ {...}` - Emphasis
53
- - `**text** {...}`, `__text__ {...}` - Strong
54
- - `` `text` {...}`` - Code
55
-
56
- ### Block
57
- - `# Heading {...}` - Heading text
58
- - `- item {...}` - List item text
59
- - `> quote {...}` - Quote text
60
- - ````lang {...}`` - Code block content
61
-
62
- ### Links/Media
63
- - `<URL> {...}` - URL (IRI)
64
- - `[label](URL) {...}` - Link (text + URL)
65
- - `![alt](URL) {...}` - Image (alt + URL)
66
-
67
- **Rule:** `{...}` must immediately follow the carrier.
68
-
69
- ---
70
-
71
- ## 5. Semantic Block Attachment
72
-
73
- `{...}` blocks attach by:
74
-
75
- 1. **Nearest preceding value carrier**, OR
76
- 2. **Stand alone** - applies to current subject or following list
77
-
78
- **Ambiguous attachment = no output**
79
-
80
- ---
81
-
82
- ## 6. Subjects
83
-
84
- ### Full IRI
85
- ```
86
- {=IRI}
87
- ```
88
- Sets current subject. Persists until changed.
89
-
90
- ### Fragment
91
- ```
92
- {=#fragment}
93
- ```
94
- Creates `currentSubjectBase#fragment`. Requires existing subject.
95
-
96
- ### Soft Object IRI (Temporary)
97
-
98
- ```
99
- {+iri}
100
- ```
101
-
102
- Declares a temporary object for use with `?` and `!` predicates, without changing the current subject.
103
-
104
- ### Soft Fragment (Temporary)
105
-
106
- ```
107
- {+#fragment}
108
- ```
109
-
110
- Creates a temporary fragment relative to the current subject base.
111
-
112
- **Use cases:**
113
- - `[Related Item] {+ex:related ?schema:isRelatedTo}` - Links to a local fragment
114
- - `[Parent] {+ex:parent !schema:hasPart}` - Reverse relationship
115
- - `[Section] {+#section1 name ?hasPart}` - Object property with fragment
116
-
117
- The soft IRI only exists within the current annotation block.
118
-
119
- ### Primary Metadata
120
-
121
- MD-LD tracks three primary metadata fields during single-pass parsing:
122
-
123
- | Field | Source | Purpose |
124
- |-------|--------|---------|
125
- | **primarySubject** | First non-fragment `{=subject}` | Document identity |
126
- | **primaryType** | First `.Class` declaration | Document category |
127
- | **primaryLabel** | First `{label}` literal | Human-readable name |
128
-
129
- **Selection rules:**
130
- - First occurrence only (fixed once detected)
131
- - Never cleared on `{=}` reset
132
- - Returns `null` if not declared
133
-
134
- **API returns:**
135
- - `parse()`: `{ ..., primarySubject: string | null, primaryType: string | null, primaryLabel: string | null }`
136
- - `merge()`: `{ ..., primarySubjects: string[] }` (ordered by merge)
137
- - `generate({ quads, context, primarySubject })`: Places primary subject first for round-trip safety
138
-
139
- **Use cases:** Document identity, stream addressing, merge tracking, UI navigation, query optimization, content classification, search indexing.
140
-
141
- ---
142
-
143
- ## 7. Types
144
-
145
- ```
146
- .Class
147
- ```
148
- Declares type: `S rdf:type schema:Class`
149
-
150
- Multiple types allowed: `.Person .Astronaut`
151
-
152
- ---
153
-
154
- ## 8. Predicate Forms
155
-
156
- | Syntax | Direction | Use |
157
- |--------|-----------|-----|
158
- | `p` | S → L | Literal properties |
159
- | `?p` | S → O | Object properties |
160
- | `!p` | O → S | Reverse properties |
161
-
162
- **Emission rule:** Create quad only if all nodes exist and roles are valid.
163
-
164
- ### 8.1 Diff Polarity (Remove Tokens)
165
-
166
- Prefix `-` to any predicate for remove polarity:
167
-
168
- | Syntax | Effect |
169
- |--------|--------|
170
- | `-p` | Remove S→L fact |
171
- | `-?p` | Remove S→O fact |
172
- | `-!p` | Remove O→S fact |
173
- | `-.C` | Remove rdf:type fact |
174
-
175
- **Live state routing:** Cancel if quad exists in buffer, else add to remove array.
176
-
177
- **Parse result:** `{ quads, remove, origin, context }`
178
-
179
- **Invariant:** `quads ∩ remove = ∅`
180
-
181
- ---
182
-
183
- ## 9. Literals
184
-
185
- Extracted **only** from attached value carrier.
186
-
187
- ### Datatypes/Languages
188
- ```
189
- ^^datatypeIRI # Typed data
190
- @lang # Language tag
191
- ```
192
-
193
- Mutually exclusive. Common types: `xsd:string`, `xsd:integer`, `xsd:gYear`, `xsd:date`, `xsd:boolean`.
194
-
195
- ---
196
-
197
- ## 10. Object Resources
198
-
199
- Available from:
200
- - Link URLs: `[text](URL)`
201
- - Image URLs: `![alt](URL)`
202
- - Explicit IRIs: `{=IRI}`
203
-
204
- **Example:**
205
- ```md
206
- ## References {=ex:references}
207
- [Example image](https://example.com/image.jpg) {?image name}
208
- [W3C RDF](https://www.w3.org/RDF) {?dct:references name}
209
- [Alice] {=urn:person:alice ?author name}
210
- ```
211
-
212
- **Result:**
213
- ```turtle
214
- ex:references schema:image <https://example.com/image.jpg> .
215
- <https://example.com/image.jpg> schema:name "Example image" .
216
- ex:references dct:references <https://www.w3.org/RDF> .
217
- <https://www.w3.org/RDF> schema:name "W3C RDF" .
218
- ex:references schema:author <urn:person:alice> .
219
- <urn:person:alice> schema:name "Alice" .
220
- ```
221
-
222
- ---
223
-
224
- ## 11. Lists
225
-
226
- Lists are pure Markdown structure with **no semantic scope**. Each list item requires explicit annotations.
227
-
228
- ### Explicit List Annotations
229
-
230
- ```md
231
- [@vocab] <http://www.w3.org/2000/01/rdf-schema#>
232
-
233
- ## Research Team {=prj:team .Container label}
234
- Team members:
235
-
236
- - **Lead researcher** {+prj:lead ?member .Person label}
237
- - **Data analyst** {+prj:analyst ?member .Person label}
238
- ```
239
-
240
- **Key Rules:**
241
- - No semantic propagation from list scope
242
- - Each item must have explicit annotations
243
- - Use `+IRI` to maintain subject chaining for repeated object properties
244
- - Nested lists have no inheritance
245
-
246
- **Ordered lists** require explicit rdf:List construction:
247
- ```md
248
- [ex] <http://example.org/>
249
-
250
- # Manual list construction {=ex:manualList label}
251
- [head] {=ex:l1 ?sh:in .rdf:List}
252
- [a] {+ex:A ?rdf:first}
253
- [list2] {=ex:l2 ?rdf:rest}
254
- [b] {+ex:B ?rdf:first}
255
- [nil] {+rdf:nil ?rdf:rest}
256
- {=}
257
- ```
258
-
259
- ---
260
-
261
- ## 12. Reverse Relationships
262
-
263
- `!p` flips direction only, not meaning.
264
-
265
- **Example:**
266
- ```md
267
- Used in: {!hasPart}
268
- - Bread {=ex:bread}
269
- ```
270
- Creates: `ex:bread schema:hasPart S`
271
-
272
- ---
273
-
274
- ## 13. Code Blocks
275
-
276
- `{...}` on opening fence makes code block a value carrier.
277
-
278
- **Example:**
279
- ```md
280
- ```js {=ex:code .SoftwareSourceCode text}
281
- console.log("hi")
282
- ```
283
- ```
284
-
285
- **Result:**
286
- ```turtle
287
- ex:code a schema:SoftwareSourceCode ;
288
- schema:text "console.log(\"hi\")" .
289
- ```
290
-
291
- ---
292
-
293
- ## 14. Context
294
-
295
- ### Default Context
296
- ```json
297
- {
298
- "@vocab": "http://www.w3.org/2000/01/rdf-schema#",
299
- "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
300
- "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
301
- "xsd": "http://www.w3.org/2001/XMLSchema#",
302
- "sh": "http://www.w3.org/ns/shacl#",
303
- "prov": "http://www.w3.org/ns/prov#"
304
- }
305
- ```
306
-
307
- ### Prefix Declaration
308
- ```
309
- [prefix] <IRI>
310
- ```
311
-
312
- ### Prefix Folding: Hierarchical IRI Authoring
313
-
314
- Build namespace hierarchies by referencing previously declared prefixes:
315
-
316
- ```
317
- [my] <tag:mymail@domain.com,2026:>
318
- [j] <my:journal:>
319
- [p] <my:property:>
320
- [org] <https://org.example.com/>
321
- [person] <org:person/>
322
- [emp] <person:employee/>
323
- ```
324
-
325
- **Resolution Rules:**
326
- - Prefixes must be declared before they can be referenced (forward-reference only)
327
- - References are resolved immediately during declaration
328
- - Circular references are treated as literal strings
329
- - Later declarations override earlier ones
330
-
331
- ### Vocabulary Setting
332
- ```
333
- [@vocab] <IRI>
334
- ```
335
-
336
- **Rules:** Apply forward, later overrides earlier, single-pass resolution.
337
-
338
- ---
339
-
340
- ## 15. Forbidden Constructs
341
-
342
- - Implicit labels
343
- - Structural inference
344
- - Auto-subjects
345
- - Literal lists
346
- - Blank nodes
347
- - key=value syntax
348
- - Predicate guessing
349
- - Backtracking parses
350
- - CURIE in Markdown links
351
- - **Semantic list propagation**
352
-
353
- These ensure predictability, traceability, and single-pass processing.
354
-
355
- ---
356
-
357
- ## 16. Conformance
358
-
359
- A conformant MD-LD processor must:
360
-
361
- 1. Follow predicate routing rules (§8)
362
- 2. Emit quads only from `{...}` blocks
363
- 3. Implement single-pass streaming parsing
364
- 4. Provide deterministic output
365
- 5. Maintain traceable origins
366
- 6. Support all three predicate forms (`p`, `?p`, `!p`)
367
-
368
- No guessing, no blank nodes, no backtracking. Round-trip MD → RDF → MD preserves quads and origins.
@@ -1,232 +0,0 @@
1
- # MD-LD — Core Specification (Minimal)
2
-
3
- **Markdown-Linked Data**
4
-
5
- A deterministic, streaming-friendly semantic annotation layer for CommonMark Markdown that emits RDF quads **only** from explicit `{...}` annotations.
6
-
7
- ## 1. Core Rules
8
-
9
- - Markdown-preserving (remove `{...}` → valid Markdown)
10
- - No implicit semantics, explicit only
11
- - Single-pass streaming, no backtracking
12
- - No blank nodes, all IRIs
13
- - Every quad traces to `{...}` source
14
-
15
- ## 2. Graph Model
16
-
17
- **Facts**: Subject → Predicate → Object
18
- - **IRIs**: Subjects + objects (identifiers)
19
- - **Literals**: Objects only (data values)
20
-
21
- ## 3. Available Nodes
22
-
23
- - **S**: Current subject (`{=IRI}`, `{=#fragment}`)
24
- - **O**: Object resource (links, images, `{=IRI}`)
25
- - **L**: Literal value (from value carrier)
26
-
27
- ## 4. Value Carriers
28
-
29
- `{...}` attaches to exactly one:
30
-
31
- **Inline**: `[text]` `*text*` `**text**` `` `text` ``
32
- **Block**: `# Heading` `- item` `> quote` ````lang````
33
- **Links**: `http://url` `[label](url)` `![alt](url)` `<url>`
34
-
35
- **Rule**: `{...}` immediately follows carrier.
36
-
37
- ## 5. Attachment
38
-
39
- 1. Nearest preceding value carrier, OR
40
- 2. Stand alone (current subject or following list)
41
-
42
- Ambiguous = no output.
43
-
44
- ## 6. Subjects
45
-
46
- ```
47
- {=IRI} # Full IRI, persists
48
- {=#fragment} # Relative to current subject
49
- {+iri} # Temporary object for local ?/! predicates
50
- {+#fragment} # Temporary soft fragment
51
- ```
52
-
53
- ### Primary Metadata
54
-
55
- MD-LD tracks three primary metadata fields during single-pass parsing:
56
-
57
- | Field | Source | Purpose |
58
- |-------|--------|---------|
59
- | **primarySubject** | First non-fragment `{=subject}` | Document identity |
60
- | **primaryType** | First `.Class` declaration | Document category |
61
- | **primaryLabel** | First `{label}` literal | Human-readable name |
62
-
63
- **Selection Rules:**
64
- - First occurrence only (fixed once detected)
65
- - Never cleared on `{=}` reset
66
- - Returns `null` if not declared
67
-
68
- **parse()**: Returns `{ primarySubject, primaryType, primaryLabel }` (string IRI/null)
69
- **merge()**: Returns `primarySubjects` array (ordered by merge)
70
- **generate()**: Accepts `{ quads, context, primarySubject }` — places primary subject first for round-trip safety
71
-
72
- ## 7. Types
73
-
74
- ```
75
- .Class # S rdf:type schema:Class
76
- .Person .Astronaut # Multiple types
77
- ```
78
-
79
- ## 8. Predicates
80
-
81
- | Syntax | Direction | Use |
82
- |--------|-----------|-----|
83
- | `p` | S → L | Literal properties |
84
- | `?p` | S → O | Object properties |
85
- | `!p` | O → S | Reverse properties |
86
-
87
- Create quad only if all nodes exist.
88
-
89
- ### 8.1 Remove Polarity
90
-
91
- Prefix `-` for remove tokens: `-p`, `-?p`, `-!p`, `-.C`
92
-
93
- Route against live buffer: cancel if present, else add to remove array.
94
-
95
- Result: `{ quads, remove, origin, context }`
96
-
97
- Invariant: `quads ∩ remove = ∅`
98
-
99
- ## 9. Literals
100
-
101
- From value carrier only.
102
-
103
- ```
104
- ^^datatypeIRI # Typed data
105
- @lang # Language tag
106
- ```
107
-
108
- Common: `xsd:string`, `xsd:integer`, `xsd:gYear`, `xsd:date`.
109
-
110
- ## 10. Objects
111
-
112
- From: `[text](url)`, `![alt](url)`, `{=IRI}`
113
-
114
- **Example**:
115
- ```md
116
- ## References {=ex:references}
117
- [Image](https://example.com/img.jpg) {?image label}
118
- [W3C](https://www.w3.org/RDF) {?dct:references label}
119
- [Alice] {=tag:alice@example.com,2026:me ? label}
120
- ```
121
-
122
- ## 11. Lists
123
-
124
- Lists are pure Markdown structure with **no semantic scope**. Each list item requires explicit annotations.
125
-
126
- ```md
127
- [prj] <tag:project@example.com,2026-02-03:>
128
- [foaf] <http://xmlns.com/foaf/0.1/>
129
-
130
- ## Local Projects Team {=prj:team .foaf:Group label}
131
-
132
- Team members:
133
-
134
- - **Alice Cooper** {+prj:alice ?foaf:member .foaf:Person foaf:name}
135
- - **Bruce Miles** {+prj:bruce ?foaf:member .foaf:Person foaf:name}
136
- - **Eva Clark** {+prj:eva ?foaf:member .foaf:Person foaf:name}
137
- ```
138
-
139
- **Key Rules:**
140
- - No semantic propagation from list scope
141
- - Each item must have explicit annotations
142
- - Use `+IRI` to maintain subject chaining for repeated object properties
143
- - Nested lists have no inheritance
144
-
145
- **Ordered lists** require explicit rdf:List construction:
146
- ```md
147
- [ex] <http://example.org/>
148
-
149
- # Manual list construction {=ex:manualList label}
150
- [head] {=ex:l1 ?sh:in .rdf:List}
151
- [a] {+ex:A ?rdf:first}
152
- [list2] {=ex:l2 ?rdf:rest}
153
- [b] {+ex:B ?rdf:first}
154
- [nil] {+rdf:nil ?rdf:rest}
155
- {=}
156
- ```
157
-
158
- ## 12. Reverse Relations
159
-
160
- `!p` flips direction only.
161
-
162
- ```md
163
- Used in: {!schema:hasPart}
164
- - Bread {=ex:bread}
165
- # Creates: ex:bread schema:hasPart S
166
- ```
167
-
168
- ## 13. Code Blocks
169
-
170
- `{...}` on opening fence:
171
-
172
- ```md
173
- ```js {=ex:code .SoftwareSourceCode schema:text }
174
- console.log("hi")
175
- ```
176
- ```
177
-
178
- ## 14. Context
179
-
180
- ### Default Context
181
-
182
- ```json
183
- {
184
- "@vocab": "http://www.w3.org/2000/01/rdf-schema#",
185
- "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
186
- "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
187
- "xsd": "http://www.w3.org/2001/XMLSchema#",
188
- "sh": "http://www.w3.org/ns/shacl#",
189
- "prov": "http://www.w3.org/ns/prov#"
190
- }
191
- ```
192
-
193
- ### Adding Prefixes
194
-
195
- ```md
196
- [alice] <tag:alice@example.com,2026:>
197
- [wd] <https://www.wikidata.org/entity/>
198
- ```
199
-
200
- ### Prefix Folding: Hierarchical IRI Authoring
201
-
202
- Build namespace hierarchies by referencing previously declared prefixes:
203
-
204
- ```md
205
- [my] <tag:mymail@domain.com,2026:>
206
- [j] <my:journal:>
207
- [p] <my:property:>
208
- [org] <https://org.example.com/>
209
- [person] <org:person/>
210
- [emp] <person:employee/>
211
- ```
212
-
213
- **Resolution Rules:**
214
- - Prefixes must be declared before they can be referenced (forward-reference only)
215
- - Circular references are treated as literal strings
216
- - Later declarations override earlier ones
217
-
218
- ### Setting Vocabulary
219
-
220
- ```md
221
- [@vocab] <http://schema.org/>
222
- ```
223
-
224
- **Rules:** Apply forward, later overrides earlier, single-pass resolution.
225
-
226
- ## 15. Forbidden
227
-
228
- Implicit labels, structural inference, auto-subjects, literal lists, blank nodes, key=value, predicate guessing, backtracking, CURIE in links, **semantic list propagation**.
229
-
230
- ## 16. Conformance
231
-
232
- Must: Follow routing rules, emit only from `{...}`, single-pass, deterministic, traceable, support all predicate forms (`p`, `?p`, `!p`).