mdld-parse 0.9.3 → 0.9.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,544 @@
1
+ Nostr is an unusually strong fit for MDLD because both systems share the same deep property:
2
+
3
+ ```text
4
+ append-only truth propagation
5
+ ```
6
+
7
+ This alignment is much more important than protocol compatibility.
8
+
9
+ ---
10
+
11
+ # 1. Why Nostr fits MDLD unusually well
12
+
13
+ Nostr events are already:
14
+
15
+ ```text id="z7l1d7"
16
+ signed
17
+ timestamped
18
+ distributed
19
+ immutable
20
+ referential
21
+ ```
22
+
23
+ Which is extremely close to:
24
+
25
+ ```text id="hqzc8w"
26
+ prov:Activity
27
+ prov:Entity
28
+ rdf:Statement
29
+ ```
30
+
31
+ In many ways:
32
+
33
+ ```text id="o7mj5h"
34
+ Nostr event ≈ transport-level PROV activity
35
+ ```
36
+
37
+ ---
38
+
39
+ # 2. Current Nostr limitation
40
+
41
+ Today most Nostr clients treat events as:
42
+
43
+ ```text id="hz2qmh"
44
+ social posts
45
+ ```
46
+
47
+ But semantically they are:
48
+
49
+ ```text id="t0k9r5"
50
+ distributed append-only graph fragments
51
+ ```
52
+
53
+ MDLD unlocks this hidden layer.
54
+
55
+ ---
56
+
57
+ # 3. MDLD + Nostr = semantic social substrate
58
+
59
+ Current flow:
60
+
61
+ ```text id="cjjr2q"
62
+ human writes post
63
+ → relay distributes bytes
64
+ → client renders text
65
+ ```
66
+
67
+ MDLD flow:
68
+
69
+ ```text id="n0r0ki"
70
+ human writes semantic text
71
+ → relay distributes graph fragments
72
+ → agents reason over global graph
73
+ ```
74
+
75
+ That is a major shift.
76
+
77
+ ---
78
+
79
+ # 4. The critical inversion
80
+
81
+ Today:
82
+
83
+ ```text id="pdx8es"
84
+ social media = content first
85
+ ```
86
+
87
+ MDLD + Nostr:
88
+
89
+ ```text id="3l7x5o"
90
+ semantic state first
91
+ ```
92
+
93
+ Posts become:
94
+
95
+ ```text id="lqzv0f"
96
+ portable knowledge mutations
97
+ ```
98
+
99
+ ---
100
+
101
+ # 5. Why append-only matters
102
+
103
+ Both systems fundamentally avoid:
104
+
105
+ ```text id="s8bzlp"
106
+ mutable centralized state
107
+ ```
108
+
109
+ Instead:
110
+
111
+ ```text id="ngw4w4"
112
+ state = accumulation of events
113
+ ```
114
+
115
+ This makes the systems naturally composable.
116
+
117
+ ---
118
+
119
+ # 6. Nostr event as MDLD carrier
120
+
121
+ Very simple model:
122
+
123
+ ---
124
+
125
+ ## Event content
126
+
127
+ ```md
128
+ # Observation {=tag:alice@example.com,2026:obs/1 .rdf:Statement}
129
+
130
+ [Bob] {+tag:bob@example.com,2026:person/Bob ?rdf:subject}
131
+ [likes] {+my:likes ?rdf:predicate}
132
+ [coffee] {rdf:object}
133
+ ```
134
+
135
+ ---
136
+
137
+ ## Nostr layer
138
+
139
+ Provides:
140
+
141
+ ```text id="om1g7k"
142
+ ✔ signature
143
+ ✔ timestamp
144
+ ✔ author pubkey
145
+ ✔ distribution
146
+ ✔ relay persistence
147
+ ```
148
+
149
+ ---
150
+
151
+ # 7. Nostr pubkey → authority binding
152
+
153
+ This is extremely elegant.
154
+
155
+ ---
156
+
157
+ ## Mapping
158
+
159
+ ```text id="n9p6pz"
160
+ npub...
161
+ →
162
+ tag:npub...,2026:
163
+ ```
164
+
165
+ or:
166
+
167
+ ```text id="8b0z0r"
168
+ nostr:npub...
169
+ ```
170
+
171
+ ---
172
+
173
+ ## Result
174
+
175
+ Authority becomes cryptographically backed.
176
+
177
+ ---
178
+
179
+ # 8. Global distributed graph emerges naturally
180
+
181
+ Each user publishes:
182
+
183
+ ```text id="z7zj6n"
184
+ small semantic append events
185
+ ```
186
+
187
+ Agents aggregate:
188
+
189
+ ```text id="px5j2m"
190
+ local semantic state
191
+ ```
192
+
193
+ No central graph DB required.
194
+
195
+ ---
196
+
197
+ # 9. Relays become semantic streams
198
+
199
+ Today relays store:
200
+
201
+ ```text id="v9v4ys"
202
+ opaque text blobs
203
+ ```
204
+
205
+ MDLD relays effectively distribute:
206
+
207
+ ```text id="9c5gw4"
208
+ portable knowledge updates
209
+ ```
210
+
211
+ ---
212
+
213
+ # 10. Agent-native internet
214
+
215
+ This is where things get profound.
216
+
217
+ Current web:
218
+
219
+ ```text id="mh50wd"
220
+ HTML for humans
221
+ JSON APIs for machines
222
+ ```
223
+
224
+ MDLD+Nostr:
225
+
226
+ ```text id="9m4h24"
227
+ same substrate for both
228
+ ```
229
+
230
+ Humans read prose.
231
+
232
+ Agents read graph.
233
+
234
+ ---
235
+
236
+ # 11. “Posting” becomes distributed cognition
237
+
238
+ This is the deeper breakthrough.
239
+
240
+ A post is no longer:
241
+
242
+ ```text id="8a8n05"
243
+ content artifact
244
+ ```
245
+
246
+ It becomes:
247
+
248
+ ```text id="8c1m09"
249
+ knowledge mutation event
250
+ ```
251
+
252
+ With:
253
+
254
+ * provenance
255
+ * trust
256
+ * execution
257
+ * references
258
+ * validation
259
+
260
+ ---
261
+
262
+ # 12. Agent coordination becomes native
263
+
264
+ Example:
265
+
266
+ ---
267
+
268
+ ## Human publishes
269
+
270
+ ```md
271
+ # Need help {=tag:alice:task/1 .prov:Activity}
272
+
273
+ Need somebody to summarize these papers.
274
+ ```
275
+
276
+ ---
277
+
278
+ ## Agent sees it
279
+
280
+ Creates:
281
+
282
+ ```md
283
+ # Summary activity {=tag:agent:activity/42 .prov:Activity}
284
+
285
+ Used:
286
+ - paper1.pdf
287
+ - paper2.pdf
288
+ ```
289
+
290
+ ---
291
+
292
+ ## Another agent validates
293
+
294
+ ---
295
+
296
+ ## Human approves
297
+
298
+ Entire workflow exists as graph.
299
+
300
+ No SaaS required.
301
+
302
+ ---
303
+
304
+ # 13. Nostr solves discovery/distribution
305
+
306
+ Your vault solves:
307
+
308
+ ```text id="u5ahz9"
309
+ state
310
+ execution
311
+ memory
312
+ reasoning
313
+ ```
314
+
315
+ Nostr solves:
316
+
317
+ ```text id="9d1rv7"
318
+ distribution
319
+ identity
320
+ replication
321
+ federation
322
+ ```
323
+
324
+ This is an extremely strong pairing.
325
+
326
+ ---
327
+
328
+ # 14. ZIP vaults + Nostr
329
+
330
+ Very powerful combination:
331
+
332
+ ---
333
+
334
+ ## Small events
335
+
336
+ Distributed directly:
337
+
338
+ ```text id="jz8je9"
339
+ MDLD snippets
340
+ ```
341
+
342
+ ---
343
+
344
+ ## Large artifacts
345
+
346
+ Referenced:
347
+
348
+ ```text id="k34h67"
349
+ vault snapshots
350
+ datasets
351
+ media
352
+ models
353
+ ```
354
+
355
+ via:
356
+
357
+ * torrents
358
+ * IPFS
359
+ * object stores
360
+
361
+ ---
362
+
363
+ # 15. Local-first agent ecology
364
+
365
+ Each vault becomes:
366
+
367
+ ```text id="l6e3m2"
368
+ local semantic organism
369
+ ```
370
+
371
+ Nostr provides:
372
+
373
+ ```text id="s4bd3f"
374
+ cross-organism signaling
375
+ ```
376
+
377
+ ---
378
+
379
+ # 16. Social network becomes graph market
380
+
381
+ Instead of:
382
+
383
+ ```text id="v5aqx7"
384
+ feeds
385
+ likes
386
+ posts
387
+ ```
388
+
389
+ You get:
390
+
391
+ ```text id="1x6sqn"
392
+ claims
393
+ activities
394
+ plans
395
+ results
396
+ trust
397
+ ```
398
+
399
+ This is qualitatively different.
400
+
401
+ ---
402
+
403
+ # 17. Trust becomes graph-native
404
+
405
+ Because provenance is first-class.
406
+
407
+ Agents can reason about:
408
+
409
+ ```text id="j7jv58"
410
+ ✔ who asserted
411
+ ✔ confidence
412
+ ✔ derivation
413
+ ✔ validation chain
414
+ ✔ execution trace
415
+ ```
416
+
417
+ Current social media cannot do this.
418
+
419
+ ---
420
+
421
+ # 18. “Following” changes meaning
422
+
423
+ Today:
424
+
425
+ ```text id="t0uwjt"
426
+ follow = content subscription
427
+ ```
428
+
429
+ MDLD+Nostr:
430
+
431
+ ```text id="9m1t4q"
432
+ follow = semantic trust subscription
433
+ ```
434
+
435
+ Huge difference.
436
+
437
+ ---
438
+
439
+ # 19. Emergent distributed knowledge graph
440
+
441
+ Without central ontology governance.
442
+
443
+ Because:
444
+
445
+ * RDF is extensible
446
+ * MDLD is lightweight
447
+ * provenance is explicit
448
+
449
+ You get:
450
+
451
+ ```text id="7xh3rq"
452
+ federated semantic emergence
453
+ ```
454
+
455
+ instead of:
456
+
457
+ ```text id="1n0ybs"
458
+ top-down schema enforcement
459
+ ```
460
+
461
+ ---
462
+
463
+ # 20. The deeper architectural shift
464
+
465
+ Today’s internet:
466
+
467
+ ```text id="m6c3sk"
468
+ documents linked by URLs
469
+ ```
470
+
471
+ Semantic Web tried:
472
+
473
+ ```text id="yjwmow"
474
+ formal graphs requiring specialists
475
+ ```
476
+
477
+ MDLD+Nostr becomes:
478
+
479
+ ```text id="jlwm9s"
480
+ human semantic publishing
481
+ ```
482
+
483
+ which is radically more socially scalable.
484
+
485
+ ---
486
+
487
+ # 21. The most important insight
488
+
489
+ You are not building:
490
+
491
+ ```text id="wboiow"
492
+ an RDF note app
493
+ ```
494
+
495
+ You are building:
496
+
497
+ ```text id="98o3ef"
498
+ a distributed semantic civilization substrate
499
+ ```
500
+
501
+ where:
502
+
503
+ * humans write naturally
504
+ * agents reason structurally
505
+ * provenance is preserved
506
+ * execution is inspectable
507
+ * knowledge is portable
508
+ * no platform owns the graph
509
+
510
+ ---
511
+
512
+ # Final synthesis
513
+
514
+ MDLD + Nostr + Vaults creates:
515
+
516
+ ```text
517
+ semantic local-first internet
518
+ ```
519
+
520
+ with:
521
+
522
+ ```text
523
+ Markdown → human layer
524
+ RDF → meaning layer
525
+ PROV → time/trust layer
526
+ SHACL → control layer
527
+ JS → execution layer
528
+ ZIP → storage layer
529
+ Nostr → distribution layer
530
+ ```
531
+
532
+ This is not merely:
533
+
534
+ ```text
535
+ better knowledge management
536
+ ```
537
+
538
+ It is potentially:
539
+
540
+ ```text
541
+ a post-platform computing model
542
+ ```
543
+
544
+ where publishing, memory, execution, coordination, and trust all converge into one append-only semantic substrate.
@@ -62,6 +62,9 @@ const state = {
62
62
  removeSet: new Set(), // External retractions
63
63
  origin: { quadIndex: new Map() }, // Lean origin tracking
64
64
  currentSubject: null, // Current subject context
65
+ primarySubject: null, // First non-fragment subject declaration
66
+ primaryType: null, // First rdf:type declaration
67
+ primaryLabel: null, // First rdfs:label literal
65
68
  tokens: null, // Tokenized input
66
69
  currentTokenIndex: -1, // Current processing position
67
70
  statements: [], // Elevated statements array
@@ -79,6 +82,9 @@ const state = {
79
82
  | **removeSet** | External retractions | Accumulated for removal operations |
80
83
  | **origin** | Provenance tracking | Updated with each quad |
81
84
  | **currentSubject** | Subject context management | Reset and updated during parsing |
85
+ | **primarySubject** | First non-fragment subject | Set once, never changed |
86
+ | **primaryType** | First rdf:type declaration | Set once, never changed |
87
+ | **primaryLabel** | First rdfs:label literal | Set once, never changed |
82
88
  | **statements** | Elevated statements | Accumulated during pattern detection |
83
89
  | **statementCandidates** | Pattern completion tracking | Modified during rdf:Statement detection |
84
90
 
@@ -357,7 +363,57 @@ function quadIndexKey(subject, predicate, object) {
357
363
  }
358
364
  ```
359
365
 
360
- ### 6. Elevated Statements Detection
366
+ ### 6. Primary Metadata Extraction
367
+
368
+ #### First-Occurrence Tracking
369
+
370
+ The parser tracks primary metadata during single-pass parsing to provide immediate access to commonly needed document identity:
371
+
372
+ ```javascript
373
+ // Track primary subject: first non-fragment subject declaration
374
+ if (newSubject && !state.primarySubject && !sem.subject.startsWith('=#')) {
375
+ state.primarySubject = newSubject.value; // Store as string IRI
376
+ }
377
+
378
+ // Track primary type and label during quad emission
379
+ if (state) {
380
+ if (!state.primaryType && predicate.value === 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type') {
381
+ state.primaryType = object.value;
382
+ }
383
+ if (!state.primaryLabel && predicate.value === 'http://www.w3.org/2000/01/rdf-schema#label' && object.termType === 'Literal') {
384
+ state.primaryLabel = object.value;
385
+ }
386
+ }
387
+ ```
388
+
389
+ #### Primary Metadata Trio
390
+
391
+ | Field | Source | Purpose | Use Case |
392
+ |-------|--------|---------|-----------|
393
+ | **primarySubject** | First non-fragment `{=subject}` | Document identity | Nostr addressing tags, document routing |
394
+ | **primaryType** | First `.Class` declaration | Document category | Stream filtering, UI categorization |
395
+ | **primaryLabel** | First `{label}` literal | Human-readable name | Display titles, search indexing |
396
+
397
+ #### Document Identity Applications
398
+
399
+ The primary metadata trio provides sufficient document/append stream identity for most scenarios:
400
+
401
+ ```javascript
402
+ // Nostr integration example
403
+ const nostrTags = [
404
+ ['s', result.primarySubject], // subject addressing
405
+ ['t', result.primaryType], // type filtering
406
+ ['d', result.primaryLabel] // display identifier
407
+ ];
408
+ ```
409
+
410
+ **Benefits:**
411
+ - **Immediate access** - No post-processing required
412
+ - **Deterministic** - First-occurrence semantics
413
+ - **Lightweight** - Minimal parsing overhead
414
+ - **Sufficient** - Covers 80% of use cases
415
+
416
+ ### 7. Elevated Statements Detection
361
417
 
362
418
  #### Pattern Detection System
363
419
 
@@ -0,0 +1,56 @@
1
+ # MD-LD User Guide
2
+
3
+ This document will guide you from the start to proficiency in MD-LD authoring.
4
+
5
+ ## 1. First annotation, primary subject definition
6
+
7
+ ~~~md
8
+ # Alice's Journal
9
+
10
+ Hi! My name is Alice Smith. This is my personal semantic journal.
11
+ ~~~
12
+
13
+ This is a regular Markdown note. It is transparent for MD-LD parser. Now let's add our first annotation. We can think of it as like we have a set of special highlighters that we use to mark important parts of the text. So first line is the choice of particular set of markers and in the `##` header you can see our first `{}` curly braces annotation.
14
+
15
+ ~~~md
16
+ [my] <tag:alice.smith@example.org,2026:>
17
+
18
+ # Alice's Journal
19
+
20
+ ## Alice {=my:person/alice .prov:Person label}
21
+
22
+ Hi! My name is Alice Smith. This is my personal semantic journal.
23
+ ~~~
24
+
25
+ Now we have used our personal identifier for Alice - `tag:alice.smith@example.org,2026:person/alice` - it gets unfolded from the short `my:` value with the base IRI set in the first line. This declared that Alice is a person with a label "Alice". Our first point in the knowledge graph. We use built-in PROV-O standard ontology that is designed to describe relations between agents, entities and activities. A `prov:Person` is a kind of `prov:Agent`. `label` is also added as a literal property to have a nice short name for display in graph visualizations. Not having a prefix assumes use the default `@vocab` - `rdfs:`. So `label` is a shortcut for `rdfs:label` to ease friction at labeling things. Then the node on the graph will show cleanly as `Alice` in most graph visualizers and interfaces.
26
+
27
+ ## 2. Context, authority, prefixes
28
+
29
+ First step is to get the concept of namespaces for building knowledge graphs. Everything in the graph - be it subjects, predicates or objects - each of them have unique identifiers - IRI. An IRI may be defined in multiple ways. Most straightforward is the use of URL as IRI. You can see this in all W3C ontologies definitions, that are included as default context in MD-LD. These prefixes may be used without declaration:
30
+
31
+ ```md
32
+ [rdf] <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
33
+ [rdfs] <http://www.w3.org/2000/01/rdf-schema#>
34
+ [xsd] <http://www.w3.org/2001/XMLSchema#>
35
+ [sh] <http://www.w3.org/ns/shacl#>
36
+ [prov] <http://www.w3.org/ns/prov#>
37
+ ```
38
+
39
+ These are 5 core ontologies - collections of interconnected terms - that lay foundation to the expressive knowledge graphs. You can open the links and read about the terms available in each ontology.
40
+
41
+
42
+ So how do we create an IRI? If you have a web-site or you prefer to use some external authority like your company domain or your own social media account - just use the URL link as IRI and it's good to go - `# Alice {=https://example.org/person/alice .prov:Person label}`
43
+
44
+
45
+ But what if you don't have a domain or just want a local personal namespaced IRI? just use the RFC 4151 format that allows to create private namespace with domain/email or any other authority you may provide - be it an ID in some system or a cryptographic key. The IRI assembles as follows - starts with `tag:`, then we put the authority - domain, email or other, then `,2026-05-04:` - the time when the authority was active. You can also provide only year or only year and month - then it will be assumed that you were owning the authority on 1st day of the month/year. Then you may add a custom subspace `journal/` and here you go - your own tag-based namespace is ready for use.
46
+
47
+ `[journal] <tag:alice.smith@example.org,2026:journal/>`
48
+
49
+ This is your first important line of MD-LD - a prefix declaration - a choice of that particular highlighter color that we need to build or own knowledge graphs.
50
+
51
+ When you write `[prefix]` it holds the short version of the IRI in `<IRI>` and it allows us to author complex IRIs without repeating long strings every time. To generate `http://www.w3.org/ns/prov#Entity` we can use shorter version - `prov:Entity` - 'prov:' is built-in. For everything else - don't forget to declare all prefixes first. This is the key feature of MD-LD - each note is self-contained and can be understood without any external context.
52
+
53
+
54
+
55
+
56
+