mdld-parse 0.9.9 → 1.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. package/README.md +263 -326
  2. package/dist/index.js +1860 -1337
  3. package/package.json +4 -4
  4. package/dist/.nojekyll +0 -0
  5. package/dist/CNAME +0 -1
  6. package/dist/README.md +0 -500
  7. package/dist/docs/API.md +0 -435
  8. package/dist/docs/Architecture.md +0 -190
  9. package/dist/docs/Guide.md +0 -816
  10. package/dist/docs/LLM-MDLD-Inducer.md +0 -550
  11. package/dist/docs/LiterateRDF.md +0 -472
  12. package/dist/docs/Parser.md +0 -720
  13. package/dist/docs/Performance.md +0 -238
  14. package/dist/docs/Spec-render.md +0 -1439
  15. package/dist/docs/Subject.md +0 -180
  16. package/dist/docs/Syntax.md +0 -463
  17. package/dist/docs/TodoList.md +0 -215
  18. package/dist/docs/Use-Cases.md +0 -354
  19. package/dist/docs/guide-m.md +0 -56
  20. package/dist/docs/index.html +0 -108
  21. package/dist/docs/index.md +0 -56
  22. package/dist/docs/one-page.md +0 -322
  23. package/dist/docs/origin.md +0 -460
  24. package/dist/docs/polarity.md +0 -586
  25. package/dist/docs/primary-subject.md +0 -326
  26. package/dist/docs/statements.md +0 -226
  27. package/dist/docs/tokenization-performance-report.md +0 -164
  28. package/dist/examples/Cookbook.md +0 -32
  29. package/dist/examples/Status-SHACL.md +0 -31
  30. package/dist/examples/demo/index.md +0 -35
  31. package/dist/examples/demo/journal.md +0 -58
  32. package/dist/examples/demo/minimal.md +0 -23
  33. package/dist/examples/demo/project.md +0 -84
  34. package/dist/examples/demo/prov.md +0 -107
  35. package/dist/examples/demo/rdf.md +0 -69
  36. package/dist/examples/demo/recipe.md +0 -50
  37. package/dist/examples/demo/research.md +0 -84
  38. package/dist/examples/demo/shacl.md +0 -161
  39. package/dist/examples/demo/space-mission.md +0 -58
  40. package/dist/examples/demo/tasks.md +0 -175
  41. package/dist/examples/demo/xsd.md +0 -91
  42. package/dist/examples/dogfood.md +0 -288
  43. package/dist/examples/few-shot.md +0 -309
  44. package/dist/examples/graph/rdf-graph.html +0 -473
  45. package/dist/examples/graph/rdf_force_graph_web_component.html +0 -506
  46. package/dist/examples/index.md +0 -45
  47. package/dist/examples/llm-time-workflow.md +0 -173
  48. package/dist/examples/statements.md +0 -43
  49. package/dist/examples/workflow.md +0 -150
  50. package/dist/index.html +0 -3049
  51. package/dist/logo.svg +0 -7
  52. package/dist/spec/Spec-compact.md +0 -368
  53. package/dist/spec/Spec-ultra.md +0 -232
  54. package/dist/spec/Spec.md +0 -1116
  55. package/dist/spec/index.md +0 -46
@@ -1,460 +0,0 @@
1
- # Origin System - Lean Quad Tracking
2
-
3
- The MDLD parser includes a sophisticated **origin tracking system** that provides complete provenance for every RDF quad generated from markdown. This system enables powerful use cases like UI navigation, debugging, and audit trails.
4
-
5
- ## Overview
6
-
7
- The origin system is a **lean, read-only source map** that connects each RDF quad back to its exact location in the original markdown text. Unlike complex mutable systems, the lean origin focuses on three core use cases:
8
-
9
- 1. **UI Navigation** - Click-to-jump functionality in editors
10
- 2. **Provenance Audit** - Complete relationship tracking for knowledge graphs
11
- 3. **Parser Error Reporting** - Precise error location reporting
12
-
13
- ## Architecture
14
-
15
- ### Lean Origin Structure
16
-
17
- ```javascript
18
- {
19
- quadIndex: Map<string, OriginEntry>
20
- }
21
- ```
22
-
23
- Each `OriginEntry` contains:
24
-
25
- ```javascript
26
- {
27
- blockId: string, // Unique identifier for the containing block
28
- range: { start: number, end: number }, // Character range including carrier markers
29
- valueRange: { start: number, end: number } | null, // Character range excluding carrier markers
30
- carrierType: string, // 'heading', 'blockquote', 'span'
31
- subject: string, // Subject IRI of the quad
32
- predicate: string, // Predicate IRI of the quad
33
- context: object, // Context object inherited from parsing
34
- value: string, // Raw carrier text content
35
- polarity: '+' | '-' // Assertion (+) or retraction (-)
36
- }
37
- ```
38
-
39
- ### Key Benefits
40
-
41
- #### 🎯 Complete Audit Trails
42
- Track full subject-predicate-object relationships with precise source locations:
43
-
44
- ```javascript
45
- const location = locate(quad, origin);
46
- console.log(location);
47
- // {
48
- // blockId: '4ac750c12',
49
- // range: { start: 33, end: 53 },
50
- // carrierType: 'blockquote',
51
- // subject: 'http://example.org/alice',
52
- // predicate: 'http://www.w3.org/2000/01/rdf-schema#name',
53
- // context: { ex: 'http://example.org/' },
54
- // value: 'Alice Smith',
55
- // polarity: '+'
56
- // }
57
- ```
58
-
59
- #### 🔍 Better Debugging
60
- Full quad information available for debugging and validation:
61
-
62
- ```javascript
63
- // Find all quads from a specific block
64
- const blockQuads = Array.from(origin.quadIndex.entries())
65
- .filter(([key, entry]) => entry.blockId === '4ac750c12')
66
- .map(([key, entry]) => ({ key, entry }));
67
-
68
- // Debug specific predicate usage
69
- const nameQuads = Array.from(origin.quadIndex.entries())
70
- .filter(([key, entry]) => entry.predicate.includes('name'));
71
- ```
72
-
73
- #### 📊 Enhanced Provenance
74
- Complete relationship tracking for knowledge graphs:
75
-
76
- ```javascript
77
- // Build provenance chain
78
- function buildProvenanceChain(quads, origin) {
79
- return quads.map(quad => {
80
- const location = locate(quad, origin);
81
- return {
82
- quad,
83
- source: {
84
- text: location.value,
85
- range: location.range,
86
- blockId: location.blockId
87
- },
88
- relationship: {
89
- subject: location.subject,
90
- predicate: location.predicate,
91
- polarity: location.polarity
92
- }
93
- };
94
- });
95
- }
96
- ```
97
-
98
- ## API Reference
99
-
100
- ### `locate(quad, origin)`
101
-
102
- Locate the origin entry for a quad using the lean origin system.
103
-
104
- **Parameters:**
105
- - `quad` (object) — The quad to locate (subject, predicate, object)
106
- - `origin` (object) — Origin object containing quadIndex
107
-
108
- **Returns:** Origin entry or `null`
109
-
110
- **Example:**
111
- ```javascript
112
- import { parse, locate } from 'mdld-parse';
113
-
114
- const result = parse({ text: mdldText, context: { ex: 'http://example.org/' } });
115
- const quad = result.quads[0];
116
-
117
- const location = locate(quad, result.origin);
118
-
119
- if (location) {
120
- console.log(`Found "${location.value}" at ${location.range.start}-${location.range.end}`);
121
- console.log(`Subject: ${location.subject}`);
122
- console.log(`Predicate: ${location.predicate}`);
123
- console.log(`Carrier: ${location.carrierType}`);
124
- }
125
- ```
126
-
127
- ## Use Cases
128
-
129
- ### 1. UI Navigation
130
-
131
- Create click-to-jump functionality in MDLD editors:
132
-
133
- ```javascript
134
- // Handle quad click in UI
135
- function onQuadClick(quad, origin, editor) {
136
- const location = locate(quad, origin);
137
- if (location) {
138
- // Jump to source location
139
- editor.setSelection(location.range.start, location.range.end);
140
- editor.focus();
141
-
142
- // Highlight the block
143
- editor.highlightBlock(location.blockId);
144
- }
145
- }
146
-
147
- // Hover preview
148
- function onQuadHover(quad, origin) {
149
- const location = locate(quad, origin);
150
- if (location) {
151
- showTooltip({
152
- text: location.value,
153
- range: location.range,
154
- predicate: location.predicate
155
- });
156
- }
157
- }
158
- ```
159
-
160
- ### 2. Provenance Audit
161
-
162
- Build complete audit trails for knowledge graphs:
163
-
164
- ```javascript
165
- function buildAuditReport(quads, origin) {
166
- const report = {
167
- totalQuads: quads.length,
168
- blocks: new Map(),
169
- predicates: new Map(),
170
- polarities: { '+': 0, '-': 0 }
171
- };
172
-
173
- quads.forEach(quad => {
174
- const location = locate(quad, origin);
175
- if (!location) return;
176
-
177
- // Track block usage
178
- if (!report.blocks.has(location.blockId)) {
179
- report.blocks.set(location.blockId, {
180
- carrierType: location.carrierType,
181
- quads: []
182
- });
183
- }
184
- report.blocks.get(location.blockId).quads.push(quad);
185
-
186
- // Track predicate usage
187
- const predCount = report.predicates.get(location.predicate) || 0;
188
- report.predicates.set(location.predicate, predCount + 1);
189
-
190
- // Track polarity
191
- report.polarities[location.polarity]++;
192
- });
193
-
194
- return report;
195
- }
196
- ```
197
-
198
- ### 3. Error Reporting
199
-
200
- Provide precise error location information:
201
-
202
- ```javascript
203
- function validateQuad(quad, origin, rules) {
204
- const location = locate(quad, origin);
205
- const errors = [];
206
-
207
- rules.forEach(rule => {
208
- if (!rule.validate(quad)) {
209
- errors.push({
210
- message: rule.message,
211
- quad,
212
- location: {
213
- text: location.value,
214
- range: location.range,
215
- blockId: location.blockId,
216
- line: getLineNumber(location.range.start, sourceText)
217
- }
218
- });
219
- }
220
- });
221
-
222
- return errors;
223
- }
224
- ```
225
-
226
- ### 4. Multi-Document Merging
227
-
228
- Track provenance across merged documents:
229
-
230
- ```javascript
231
- import { merge } from 'mdld-parse';
232
-
233
- const merged = merge([doc1, doc2, doc3]);
234
-
235
- // Build document provenance
236
- function buildDocumentProvenance(merged) {
237
- const provenance = new Map();
238
-
239
- merged.origin.documents.forEach((doc, index) => {
240
- doc.quads.forEach(quad => {
241
- const location = locate(quad, merged.origin);
242
- if (location) {
243
- provenance.set(quadKey(quad), {
244
- documentIndex: index,
245
- originalLocation: location,
246
- mergedLocation: location
247
- });
248
- }
249
- });
250
- });
251
-
252
- return provenance;
253
- }
254
- ```
255
-
256
- ## Performance Considerations
257
-
258
- The lean origin system is optimized for performance:
259
-
260
- - **Memory Efficient**: Uses Map data structure for O(1) lookups
261
- - **Streaming Friendly**: Built during single-pass parsing
262
- - **Immutable**: Origin entries never change after creation
263
- - **Minimal Overhead**: Only stores essential information
264
-
265
- ```javascript
266
- // Performance characteristics
267
- const origin = parse({text: largeMarkdown}).origin;
268
-
269
- // O(1) lookup by quad
270
- const location = locate(quad, origin);
271
-
272
- // O(n) iteration over all entries
273
- for (const [key, entry] of origin.quadIndex) {
274
- // Process entry
275
- }
276
-
277
- // Memory usage scales linearly with quad count
278
- // ~200 bytes per quad entry
279
- ```
280
-
281
- ## Migration from Legacy Origin
282
-
283
- If you were using the pre-v0.7.0 origin system, here's how to migrate:
284
-
285
- ### Old System (pre-v0.7.0)
286
- ```javascript
287
- // Complex mutable origin with slot management
288
- {
289
- blocks: Map<string, Block>,
290
- quadMap: Map<string, QuadSlot>,
291
- entries: Array<OriginEntry>
292
- }
293
-
294
- // Complex locate with auto-parsing
295
- locate(quad, origin, text, context)
296
- ```
297
-
298
- ### New Lean System (v0.7.0+)
299
- ```javascript
300
- // Simple immutable origin
301
- {
302
- quadIndex: Map<string, OriginEntry>
303
- }
304
-
305
- // Simple locate
306
- locate(quad, origin)
307
- ```
308
-
309
- ### Migration Steps
310
-
311
- 1. **Update locate calls**:
312
- ```javascript
313
- // Old
314
- const location = locate(quad, origin, text, context);
315
-
316
- // New
317
- const location = locate(quad, origin);
318
- ```
319
-
320
- 2. **Update origin access**:
321
- ```javascript
322
- // Old
323
- const block = origin.blocks.get(blockId);
324
- const slot = origin.quadMap.get(quadKey);
325
-
326
- // New
327
- const entry = origin.quadIndex.get(quadKey);
328
- ```
329
-
330
- 3. **Remove applyDiff usage**:
331
- ```javascript
332
- // Old - no longer available
333
- const updated = applyDiff({ text, diff, origin });
334
-
335
- // New - use merge for document updates
336
- const updated = merge([original, diffText]);
337
- ```
338
-
339
- ## Best Practices
340
-
341
- ### 1. Cache Location Lookups
342
- ```javascript
343
- // Cache frequently accessed locations
344
- const locationCache = new Map();
345
-
346
- function getCachedLocation(quad, origin) {
347
- const key = quadKey(quad);
348
- if (!locationCache.has(key)) {
349
- locationCache.set(key, locate(quad, origin));
350
- }
351
- return locationCache.get(key);
352
- }
353
- ```
354
-
355
- ### 2. Batch Processing
356
- ```javascript
357
- // Process multiple quads efficiently
358
- function processQuads(quads, origin, processor) {
359
- return quads.map(quad => {
360
- const location = locate(quad, origin);
361
- return processor(quad, location);
362
- });
363
- }
364
- ```
365
-
366
- ### 3. Error Handling
367
- ```javascript
368
- // Always handle null returns
369
- function safeLocate(quad, origin) {
370
- const location = locate(quad, origin);
371
- if (!location) {
372
- console.warn('Quad not found in origin:', quad);
373
- return null;
374
- }
375
- return location;
376
- }
377
- ```
378
-
379
- ### 4. Memory Management
380
- ```javascript
381
- // Clear caches when no longer needed
382
- function cleanup() {
383
- locationCache.clear();
384
- // Allow garbage collection
385
- }
386
- ```
387
-
388
- ## Examples
389
-
390
- ### Complete UI Integration
391
- ```javascript
392
- class MDLDEditor {
393
- constructor() {
394
- this.origin = null;
395
- this.quads = [];
396
- }
397
-
398
- async parse(text) {
399
- const result = parse({ text, context: this.context });
400
- this.origin = result.origin;
401
- this.quads = result.quads;
402
- this.render();
403
- }
404
-
405
- onQuadClick(quad) {
406
- const location = locate(quad, this.origin);
407
- if (location) {
408
- this.editor.setSelection(location.range.start, location.range.end);
409
- this.showQuadInfo(quad, location);
410
- }
411
- }
412
-
413
- showQuadInfo(quad, location) {
414
- const info = `
415
- <div class="quad-info">
416
- <h4>Quad Information</h4>
417
- <p><strong>Subject:</strong> ${location.subject}</p>
418
- <p><strong>Predicate:</strong> ${location.predicate}</p>
419
- <p><strong>Value:</strong> ${location.value}</p>
420
- <p><strong>Type:</strong> ${location.carrierType}</p>
421
- <p><strong>Range:</strong> ${location.range.start}-${location.range.end}</p>
422
- <p><strong>Polarity:</strong> ${location.polarity}</p>
423
- </div>
424
- `;
425
- this.sidebar.innerHTML = info;
426
- }
427
- }
428
- ```
429
-
430
- ### Audit Trail Generator
431
- ```javascript
432
- function generateAuditTrail(quads, origin) {
433
- const trail = {
434
- timestamp: new Date().toISOString(),
435
- totalQuads: quads.length,
436
- entries: []
437
- };
438
-
439
- quads.forEach((quad, index) => {
440
- const location = locate(quad, origin);
441
- if (location) {
442
- trail.entries.push({
443
- index,
444
- subject: location.subject,
445
- predicate: location.predicate,
446
- object: quad.object.value,
447
- sourceText: location.value,
448
- sourceRange: location.range,
449
- blockId: location.blockId,
450
- carrierType: location.carrierType,
451
- polarity: location.polarity
452
- });
453
- }
454
- });
455
-
456
- return trail;
457
- }
458
- ```
459
-
460
- The lean origin system provides a powerful, efficient foundation for building sophisticated MDLD applications with complete provenance tracking.