eyeling 1.26.7 → 1.27.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/HANDBOOK.md +32 -1
- package/dist/browser/eyeling.browser.js +519 -3
- package/examples/deck/act-barley-seed-lineage.md +1 -1
- package/examples/deck/faltings-genus2-finiteness.md +1 -1
- package/examples/deck/high-trust-rdf-bloom-envelope.md +1 -1
- package/examples/deck/odrl-dpv-risk-ranked.md +1 -1
- package/examples/deck/rdf-message-cold-chain-recall.md +71 -0
- package/examples/deck/rdf-message-flow.md +1 -1
- package/examples/deck/rdf-message-ldes-incremental.md +94 -0
- package/examples/deck/rdf-message-window-repair.md +1 -1
- package/examples/deck/schema-foaf-mapping.md +2 -0
- package/examples/input/rdf-message-cold-chain-recall.trig +668 -0
- package/examples/input/rdf-message-flow.trig +3 -0
- package/examples/input/rdf-message-ldes-incremental.trig +564 -0
- package/examples/input/rdf-message-microgrid.trig +3 -0
- package/examples/input/rdf-message-window-repair.trig +3 -0
- package/examples/input/rdf-messages.trig +3 -0
- package/examples/output/rdf-message-cold-chain-recall.md +13 -0
- package/examples/output/rdf-message-ldes-incremental.md +13 -0
- package/examples/rdf-message-cold-chain-recall.n3 +229 -0
- package/examples/rdf-message-flow.n3 +3 -0
- package/examples/rdf-message-ldes-incremental.n3 +231 -0
- package/examples/rdf-message-microgrid.n3 +3 -0
- package/examples/rdf-message-window-repair.n3 +3 -0
- package/examples/rdf-messages.n3 +3 -0
- package/eyeling.js +519 -3
- package/lib/cli.js +519 -3
- package/package.json +5 -3
- package/test/examples.test.js +25 -14
- package/test/fixtures/marc-rules-stream-messages.n3 +192 -0
- package/test/stream_messages.test.js +284 -0
package/HANDBOOK.md
CHANGED
|
@@ -333,6 +333,36 @@ This is the important mental model for `-r` with RDF Messages: the parser preser
|
|
|
333
333
|
|
|
334
334
|
This keeps Eyeling's N3 model stable while allowing small RDF 1.1/RDF 1.2 dataset-shaped and message-log-shaped inputs to run through the existing `GraphTerm` machinery when the caller opts in. More exotic future RDF forms should be added only if they can be mapped cleanly onto Eyeling's quoted-formula term model.
|
|
335
335
|
|
|
336
|
+
|
|
337
|
+
#### Streaming large RDF Message Logs with `--stream-messages`
|
|
338
|
+
|
|
339
|
+
The ordinary `-r` replay described above is intentionally a faithful whole-log replay: it materializes one stream resource, the complete ordered-envelope list, all envelope links, and all payload graphs before reasoning. That is useful for examples, audits, proofs, and rules that need to see the whole log at once, but it is not the right execution model for very large append-only logs.
|
|
340
|
+
|
|
341
|
+
For large files where each RDF Message can be processed independently, use:
|
|
342
|
+
|
|
343
|
+
```sh
|
|
344
|
+
eyeling -r --stream-messages rules.n3 large-message-log.trig
|
|
345
|
+
```
|
|
346
|
+
|
|
347
|
+
`--stream-messages` keeps the rule files loaded once, then reads each RDF Message chunk from the log and runs the rules against a one-message replay view:
|
|
348
|
+
|
|
349
|
+
```n3
|
|
350
|
+
?stream a eymsg:RDFMessageStream;
|
|
351
|
+
eymsg:envelope ?envelope;
|
|
352
|
+
eymsg:firstEnvelope ?envelope;
|
|
353
|
+
eymsg:lastEnvelope ?envelope;
|
|
354
|
+
eymsg:orderedEnvelopes (?envelope).
|
|
355
|
+
|
|
356
|
+
?envelope a eymsg:MessageEnvelope;
|
|
357
|
+
eymsg:offset ?n;
|
|
358
|
+
eymsg:payloadKind eymsg:nonEmpty;
|
|
359
|
+
eymsg:payloadGraph ?payload.
|
|
360
|
+
```
|
|
361
|
+
|
|
362
|
+
The payload is still scoped behind `?payload log:nameOf { ... }`, so rules use the same `log:nameOf`/`log:includes` pattern as in whole-log replay. The important difference is lifetime: after one message has been parsed, reasoned over, and printed, its facts are discarded before the next message is read. Local files are scanned incrementally instead of first being normalized into one giant N3 document.
|
|
363
|
+
|
|
364
|
+
This mode is meant for production-style message feeds such as MARC-record streams, telemetry streams, or LDES member logs where a consumer checkpoint already tells the application which messages are new. It deliberately does not expose `eymsg:nextEnvelope` links or a complete `eymsg:messageCount`, because those require holding global stream state. Use ordinary `eyeling -r` when your rules need global ordering, sliding windows across several messages, or proof output for the entire replay.
|
|
365
|
+
|
|
336
366
|
### 4.2 Parsing triples, with Turtle-style convenience
|
|
337
367
|
|
|
338
368
|
The parser supports:
|
|
@@ -2623,6 +2653,7 @@ Options:
|
|
|
2623
2653
|
-h, --help Show this help and exit.
|
|
2624
2654
|
-p, --proof Enable proof explanations.
|
|
2625
2655
|
-r, --rdf Enable RDF/TriG input/output compatibility.
|
|
2656
|
+
--stream-messages Process RDF Message Logs one message at a time under -r.
|
|
2626
2657
|
-s, --super-restricted Disable all builtins except => and <=.
|
|
2627
2658
|
-t, --stream Stream derived triples as soon as they are derived.
|
|
2628
2659
|
-v, --version Print version and exit.
|
|
@@ -2663,7 +2694,7 @@ Quoted graphs/formulas use `{ ... }`. Inside a quoted formula, directive scope m
|
|
|
2663
2694
|
|
|
2664
2695
|
- `@prefix/@base` and `PREFIX/BASE` directives may appear at top level **or inside `{ ... }`**, and apply to the formula they occur in (formula-local scoping).
|
|
2665
2696
|
|
|
2666
|
-
With `-r, --rdf` / `{ rdf: true }`, Eyeling accepts selected RDF/TriG surface syntax before normal N3 parsing. RDF 1.2 triple-term forms such as `<<( s p o )>>` and `<<s p o ~ r>>` are compatibility spellings for singleton quoted formulas such as `{ s p o }`; feasible singleton graph terms are printed back as RDF 1.2 triple terms. TriG named graph blocks become `log:nameOf` quoted formulas. If a `VERSION "1.2-messages"`-style directive is present, top-level `MESSAGE` delimiters are replayed as `eymsg:` stream/envelope facts and per-message payload graphs. See [Chapter 4.1](#41-lexing-tokens-not-magic) for the full `-r` model and RDF Message handling.
|
|
2697
|
+
With `-r, --rdf` / `{ rdf: true }`, Eyeling accepts selected RDF/TriG surface syntax before normal N3 parsing. RDF 1.2 triple-term forms such as `<<( s p o )>>` and `<<s p o ~ r>>` are compatibility spellings for singleton quoted formulas such as `{ s p o }`; feasible singleton graph terms are printed back as RDF 1.2 triple terms. TriG named graph blocks become `log:nameOf` quoted formulas. If a `VERSION "1.2-messages"`-style directive is present, top-level `MESSAGE` delimiters are replayed as `eymsg:` stream/envelope facts and per-message payload graphs. For large independent message logs, `--stream-messages` runs the same payload-inspection style one message at a time. See [Chapter 4.1](#41-lexing-tokens-not-magic) for the full `-r` model and RDF Message handling.
|
|
2667
2698
|
|
|
2668
2699
|
For the formal grammar, see the N3 spec grammar:
|
|
2669
2700
|
|
|
@@ -4622,12 +4622,16 @@ module.exports = {
|
|
|
4622
4622
|
|
|
4623
4623
|
'use strict';
|
|
4624
4624
|
|
|
4625
|
+
const fs = require('node:fs');
|
|
4626
|
+
const os = require('node:os');
|
|
4625
4627
|
const path = require('node:path');
|
|
4626
|
-
const { pathToFileURL } = require('node:url');
|
|
4628
|
+
const { pathToFileURL, fileURLToPath } = require('node:url');
|
|
4629
|
+
const { TextDecoder } = require('node:util');
|
|
4627
4630
|
|
|
4628
4631
|
const engine = require('./engine');
|
|
4629
4632
|
const deref = require('./deref');
|
|
4630
4633
|
const { PrefixEnv } = require('./prelude');
|
|
4634
|
+
const { normalizeRdfCompatibility } = require('./lexer');
|
|
4631
4635
|
const { parseN3Text, mergeParsedDocuments } = require('./multisource');
|
|
4632
4636
|
|
|
4633
4637
|
function offsetToLineCol(text, offset) {
|
|
@@ -4666,7 +4670,6 @@ function formatN3SyntaxError(err, text, path) {
|
|
|
4666
4670
|
|
|
4667
4671
|
// CLI entry point (invoked when eyeling.js is run directly)
|
|
4668
4672
|
function readTextFromStdinSync() {
|
|
4669
|
-
const fs = require('node:fs');
|
|
4670
4673
|
return fs.readFileSync(0, { encoding: 'utf8' });
|
|
4671
4674
|
}
|
|
4672
4675
|
|
|
@@ -4674,6 +4677,10 @@ function __isNetworkOrFileIri(s) {
|
|
|
4674
4677
|
return typeof s === 'string' && /^(https?:|file:\/\/)/i.test(s);
|
|
4675
4678
|
}
|
|
4676
4679
|
|
|
4680
|
+
function __isHttpSource(s) {
|
|
4681
|
+
return typeof s === 'string' && /^https?:/i.test(s);
|
|
4682
|
+
}
|
|
4683
|
+
|
|
4677
4684
|
function __sourceLabelToBaseIri(sourceLabel) {
|
|
4678
4685
|
if (!sourceLabel || sourceLabel === '<stdin>') return '';
|
|
4679
4686
|
if (__isNetworkOrFileIri(sourceLabel)) return deref.stripFragment(sourceLabel);
|
|
@@ -4689,10 +4696,492 @@ function __readInputSourceSync(sourceLabel) {
|
|
|
4689
4696
|
return txt;
|
|
4690
4697
|
}
|
|
4691
4698
|
|
|
4692
|
-
const fs = require('node:fs');
|
|
4693
4699
|
return fs.readFileSync(sourceLabel, { encoding: 'utf8' });
|
|
4694
4700
|
}
|
|
4695
4701
|
|
|
4702
|
+
function __httpFetchScriptBody({ prefixOnly = false } = {}) {
|
|
4703
|
+
return `
|
|
4704
|
+
const fs = require('fs');
|
|
4705
|
+
const http = require('http');
|
|
4706
|
+
const https = require('https');
|
|
4707
|
+
const zlib = require('zlib');
|
|
4708
|
+
const { URL } = require('url');
|
|
4709
|
+
const urlArg = process.argv[1];
|
|
4710
|
+
const outFile = process.argv[2] || '';
|
|
4711
|
+
const limit = Math.max(1, Number(process.argv[3] || 65536));
|
|
4712
|
+
const prefixOnly = ${prefixOnly ? 'true' : 'false'};
|
|
4713
|
+
const maxRedirects = 10;
|
|
4714
|
+
function requestUrl(u, redirects) {
|
|
4715
|
+
if (redirects > maxRedirects) { console.error('Too many redirects'); process.exit(3); }
|
|
4716
|
+
const parsed = new URL(u);
|
|
4717
|
+
const mod = parsed.protocol === 'https:' ? https : parsed.protocol === 'http:' ? http : null;
|
|
4718
|
+
if (!mod) { console.error('Unsupported protocol ' + parsed.protocol); process.exit(2); }
|
|
4719
|
+
const headers = {
|
|
4720
|
+
accept: 'text/n3, text/turtle, application/trig, application/n-triples, application/n-quads, text/plain;q=0.8, */*;q=0.01',
|
|
4721
|
+
'accept-encoding': 'identity',
|
|
4722
|
+
'user-agent': 'eyeling-rdf-message-stream'
|
|
4723
|
+
};
|
|
4724
|
+
if (prefixOnly) headers.range = 'bytes=0-' + String(limit - 1);
|
|
4725
|
+
const req = mod.request({
|
|
4726
|
+
protocol: parsed.protocol,
|
|
4727
|
+
hostname: parsed.hostname,
|
|
4728
|
+
port: parsed.port || undefined,
|
|
4729
|
+
path: parsed.pathname + parsed.search,
|
|
4730
|
+
headers,
|
|
4731
|
+
}, (res) => {
|
|
4732
|
+
const sc = res.statusCode || 0;
|
|
4733
|
+
if (sc >= 300 && sc < 400 && res.headers && res.headers.location) {
|
|
4734
|
+
const next = new URL(res.headers.location, u).toString();
|
|
4735
|
+
res.resume();
|
|
4736
|
+
return requestUrl(next, redirects + 1);
|
|
4737
|
+
}
|
|
4738
|
+
if (sc < 200 || sc >= 300) {
|
|
4739
|
+
res.resume();
|
|
4740
|
+
console.error('HTTP status ' + sc);
|
|
4741
|
+
process.exit(4);
|
|
4742
|
+
}
|
|
4743
|
+
const enc = String((res.headers && res.headers['content-encoding']) || '').toLowerCase();
|
|
4744
|
+
let body = res;
|
|
4745
|
+
if (enc.includes('gzip')) body = res.pipe(zlib.createGunzip());
|
|
4746
|
+
else if (enc.includes('deflate')) body = res.pipe(zlib.createInflate());
|
|
4747
|
+
else if (enc.includes('br')) body = res.pipe(zlib.createBrotliDecompress());
|
|
4748
|
+
if (prefixOnly) {
|
|
4749
|
+
const chunks = [];
|
|
4750
|
+
let bytes = 0;
|
|
4751
|
+
let finished = false;
|
|
4752
|
+
function finish() {
|
|
4753
|
+
if (finished) return;
|
|
4754
|
+
finished = true;
|
|
4755
|
+
const buf = Buffer.concat(chunks, bytes).subarray(0, limit);
|
|
4756
|
+
process.stdout.write(buf.toString('utf8'));
|
|
4757
|
+
process.exit(0);
|
|
4758
|
+
}
|
|
4759
|
+
body.on('data', (chunk) => {
|
|
4760
|
+
if (finished) return;
|
|
4761
|
+
chunks.push(chunk);
|
|
4762
|
+
bytes += chunk.length;
|
|
4763
|
+
if (bytes >= limit) finish();
|
|
4764
|
+
});
|
|
4765
|
+
body.on('end', finish);
|
|
4766
|
+
body.on('error', (e) => { console.error(e && e.message ? e.message : String(e)); process.exit(5); });
|
|
4767
|
+
return;
|
|
4768
|
+
}
|
|
4769
|
+
const out = fs.createWriteStream(outFile);
|
|
4770
|
+
body.pipe(out);
|
|
4771
|
+
body.on('error', (e) => { console.error(e && e.message ? e.message : String(e)); process.exit(5); });
|
|
4772
|
+
out.on('error', (e) => { console.error(e && e.message ? e.message : String(e)); process.exit(6); });
|
|
4773
|
+
out.on('finish', () => process.exit(0));
|
|
4774
|
+
});
|
|
4775
|
+
req.on('error', (e) => { console.error(e && e.message ? e.message : String(e)); process.exit(5); });
|
|
4776
|
+
req.end();
|
|
4777
|
+
}
|
|
4778
|
+
requestUrl(urlArg, 0);
|
|
4779
|
+
`;
|
|
4780
|
+
}
|
|
4781
|
+
|
|
4782
|
+
function __readHttpPrefixSync(sourceLabel, byteLimit = 64 * 1024) {
|
|
4783
|
+
const cp = require('node:child_process');
|
|
4784
|
+
const r = cp.spawnSync(process.execPath, ['-e', __httpFetchScriptBody({ prefixOnly: true }), sourceLabel, '', String(byteLimit)], {
|
|
4785
|
+
encoding: 'utf8',
|
|
4786
|
+
maxBuffer: byteLimit + 16 * 1024,
|
|
4787
|
+
});
|
|
4788
|
+
if (r.status !== 0) throw new Error(`Failed to dereference ${sourceLabel}${r.stderr ? ': ' + String(r.stderr).trim() : ''}`);
|
|
4789
|
+
return r.stdout;
|
|
4790
|
+
}
|
|
4791
|
+
|
|
4792
|
+
function __downloadHttpSourceToTempFileSync(sourceLabel) {
|
|
4793
|
+
const cp = require('node:child_process');
|
|
4794
|
+
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'eyeling-rdf-message-log-'));
|
|
4795
|
+
const file = path.join(dir, 'source.txt');
|
|
4796
|
+
const r = cp.spawnSync(process.execPath, ['-e', __httpFetchScriptBody({ prefixOnly: false }), sourceLabel, file], {
|
|
4797
|
+
encoding: 'utf8',
|
|
4798
|
+
maxBuffer: 1024 * 1024,
|
|
4799
|
+
stdio: ['ignore', 'ignore', 'pipe'],
|
|
4800
|
+
});
|
|
4801
|
+
if (r.status !== 0) {
|
|
4802
|
+
try { fs.rmSync(dir, { recursive: true, force: true }); } catch {}
|
|
4803
|
+
throw new Error(`Failed to dereference ${sourceLabel}${r.stderr ? ': ' + String(r.stderr).trim() : ''}`);
|
|
4804
|
+
}
|
|
4805
|
+
return { file, cleanup: () => { try { fs.rmSync(dir, { recursive: true, force: true }); } catch {} } };
|
|
4806
|
+
}
|
|
4807
|
+
|
|
4808
|
+
|
|
4809
|
+
const RDF_MESSAGE_VERSION_RE = /^\s*(?:@version|VERSION)\s+(["'])(?:1\.1|1\.2|1\.2-basic)-messages\1\s*\.?\s*(?:#.*)?$/im;
|
|
4810
|
+
const RDF_MESSAGE_VERSION_LINE_RE = /^\s*(?:@version|VERSION)\s+(["'])(?:1\.1|1\.2|1\.2-basic)-messages\1\s*\.?\s*(?:#.*)?$/i;
|
|
4811
|
+
const RDF_DIRECTIVE_LINE_RE = /^\s*(?:@?(?:prefix|base)\b|PREFIX\b|BASE\b)/i;
|
|
4812
|
+
const RDF_MESSAGE_DELIMITER_LINE_RE = /^\s*(?:MESSAGE\b|@message\s*\.?)\s*(?:#.*)?$/i;
|
|
4813
|
+
const LOG_NAME_OF_IRI = '<http://www.w3.org/2000/10/swap/log#nameOf>';
|
|
4814
|
+
const RDF_TYPE_IRI = '<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>';
|
|
4815
|
+
const XSD_INTEGER_IRI = '<http://www.w3.org/2001/XMLSchema#integer>';
|
|
4816
|
+
const EYMSG_NS = 'https://eyereasoner.github.io/eyeling/vocab/message#';
|
|
4817
|
+
const EYMSG_IRIS = Object.freeze({
|
|
4818
|
+
RDFMessageStream: `<${EYMSG_NS}RDFMessageStream>`,
|
|
4819
|
+
MessageEnvelope: `<${EYMSG_NS}MessageEnvelope>`,
|
|
4820
|
+
envelope: `<${EYMSG_NS}envelope>`,
|
|
4821
|
+
firstEnvelope: `<${EYMSG_NS}firstEnvelope>`,
|
|
4822
|
+
lastEnvelope: `<${EYMSG_NS}lastEnvelope>`,
|
|
4823
|
+
orderedEnvelopes: `<${EYMSG_NS}orderedEnvelopes>`,
|
|
4824
|
+
offset: `<${EYMSG_NS}offset>`,
|
|
4825
|
+
payloadGraph: `<${EYMSG_NS}payloadGraph>`,
|
|
4826
|
+
payloadKind: `<${EYMSG_NS}payloadKind>`,
|
|
4827
|
+
empty: `<${EYMSG_NS}empty>`,
|
|
4828
|
+
nonEmpty: `<${EYMSG_NS}nonEmpty>`,
|
|
4829
|
+
});
|
|
4830
|
+
|
|
4831
|
+
function simpleHashText(s) {
|
|
4832
|
+
let h = 0x811c9dc5;
|
|
4833
|
+
const text = String(s || '');
|
|
4834
|
+
for (let i = 0; i < text.length; i += 1) {
|
|
4835
|
+
h ^= text.charCodeAt(i);
|
|
4836
|
+
h = Math.imul(h, 0x01000193) >>> 0;
|
|
4837
|
+
}
|
|
4838
|
+
return h.toString(16).padStart(8, '0');
|
|
4839
|
+
}
|
|
4840
|
+
|
|
4841
|
+
function __isLocalPathSource(sourceLabel) {
|
|
4842
|
+
return typeof sourceLabel === 'string' && sourceLabel !== '<stdin>' && !/^(https?:|file:\/\/)/i.test(sourceLabel);
|
|
4843
|
+
}
|
|
4844
|
+
|
|
4845
|
+
function __localPathForSource(sourceLabel) {
|
|
4846
|
+
if (__isLocalPathSource(sourceLabel)) return sourceLabel;
|
|
4847
|
+
if (typeof sourceLabel === 'string' && /^file:\/\//i.test(sourceLabel)) return fileURLToPath(sourceLabel);
|
|
4848
|
+
return null;
|
|
4849
|
+
}
|
|
4850
|
+
|
|
4851
|
+
function __sourceLooksLikeRdfMessageLogSync(sourceLabel) {
|
|
4852
|
+
const filePath = __localPathForSource(sourceLabel);
|
|
4853
|
+
if (filePath) {
|
|
4854
|
+
const fd = fs.openSync(filePath, 'r');
|
|
4855
|
+
try {
|
|
4856
|
+
const buf = Buffer.allocUnsafe(64 * 1024);
|
|
4857
|
+
const n = fs.readSync(fd, buf, 0, buf.length, 0);
|
|
4858
|
+
return RDF_MESSAGE_VERSION_RE.test(buf.toString('utf8', 0, n));
|
|
4859
|
+
} finally {
|
|
4860
|
+
fs.closeSync(fd);
|
|
4861
|
+
}
|
|
4862
|
+
}
|
|
4863
|
+
if (__isHttpSource(sourceLabel)) {
|
|
4864
|
+
return RDF_MESSAGE_VERSION_RE.test(__readHttpPrefixSync(sourceLabel));
|
|
4865
|
+
}
|
|
4866
|
+
return RDF_MESSAGE_VERSION_RE.test(__readInputSourceSync(sourceLabel));
|
|
4867
|
+
}
|
|
4868
|
+
|
|
4869
|
+
function stripRdfDirectiveLines(text) {
|
|
4870
|
+
return String(text || '')
|
|
4871
|
+
.split(/(?<=\r\n|\n|\r)/)
|
|
4872
|
+
.filter((line) => !RDF_DIRECTIVE_LINE_RE.test(line) && !RDF_MESSAGE_VERSION_LINE_RE.test(line))
|
|
4873
|
+
.join('');
|
|
4874
|
+
}
|
|
4875
|
+
|
|
4876
|
+
function hasRdfPayload(text) {
|
|
4877
|
+
return String(text || '')
|
|
4878
|
+
.split(/\r\n|\n|\r/)
|
|
4879
|
+
.some((line) => {
|
|
4880
|
+
const trimmed = line.replace(/#.*$/g, '').trim();
|
|
4881
|
+
return trimmed && !RDF_DIRECTIVE_LINE_RE.test(trimmed) && !RDF_MESSAGE_VERSION_LINE_RE.test(trimmed);
|
|
4882
|
+
});
|
|
4883
|
+
}
|
|
4884
|
+
|
|
4885
|
+
function addRdfDirective(directives, seen, line) {
|
|
4886
|
+
if (!RDF_DIRECTIVE_LINE_RE.test(line)) return;
|
|
4887
|
+
const key = line.trim();
|
|
4888
|
+
if (!key || seen.has(key)) return;
|
|
4889
|
+
seen.add(key);
|
|
4890
|
+
directives.push(line.endsWith('\n') || line.endsWith('\r') ? line : line + '\n');
|
|
4891
|
+
}
|
|
4892
|
+
|
|
4893
|
+
function normalizeStreamingPayloadChunk(chunk, directives) {
|
|
4894
|
+
const prelude = directives.join('');
|
|
4895
|
+
const normalized = normalizeRdfCompatibility(prelude + String(chunk || ''));
|
|
4896
|
+
return stripRdfDirectiveLines(normalized).trim();
|
|
4897
|
+
}
|
|
4898
|
+
|
|
4899
|
+
function buildSingleMessageReplayDocument({ sourceLabel, messageIndex, chunk, directives }) {
|
|
4900
|
+
const hash = simpleHashText(sourceLabel || '<stream>');
|
|
4901
|
+
const base = `urn:eyeling:message-stream:${hash}`;
|
|
4902
|
+
const padded = String(messageIndex).padStart(6, '0');
|
|
4903
|
+
const stream = `<${base}#stream>`;
|
|
4904
|
+
const envelope = `<${base}#m${padded}>`;
|
|
4905
|
+
const payload = `<${base}#m${padded}/payload>`;
|
|
4906
|
+
const body = normalizeStreamingPayloadChunk(chunk, directives);
|
|
4907
|
+
const hasBody = hasRdfPayload(body);
|
|
4908
|
+
const out = [];
|
|
4909
|
+
|
|
4910
|
+
out.push(...directives.map((line) => line.trim()).filter(Boolean));
|
|
4911
|
+
out.push(`${stream} ${RDF_TYPE_IRI} ${EYMSG_IRIS.RDFMessageStream} .`);
|
|
4912
|
+
out.push(`${stream} ${EYMSG_IRIS.envelope} ${envelope} .`);
|
|
4913
|
+
out.push(`${stream} ${EYMSG_IRIS.orderedEnvelopes} (${envelope}) .`);
|
|
4914
|
+
out.push(`${stream} ${EYMSG_IRIS.firstEnvelope} ${envelope} .`);
|
|
4915
|
+
out.push(`${stream} ${EYMSG_IRIS.lastEnvelope} ${envelope} .`);
|
|
4916
|
+
out.push(`${envelope} ${RDF_TYPE_IRI} ${EYMSG_IRIS.MessageEnvelope} .`);
|
|
4917
|
+
out.push(`${envelope} ${EYMSG_IRIS.offset} "${messageIndex}"^^${XSD_INTEGER_IRI} .`);
|
|
4918
|
+
out.push(`${envelope} ${EYMSG_IRIS.payloadKind} ${hasBody ? EYMSG_IRIS.nonEmpty : EYMSG_IRIS.empty} .`);
|
|
4919
|
+
if (hasBody) {
|
|
4920
|
+
out.push(`${envelope} ${EYMSG_IRIS.payloadGraph} ${payload} .`);
|
|
4921
|
+
out.push(`${payload} ${LOG_NAME_OF_IRI} {`);
|
|
4922
|
+
out.push(body);
|
|
4923
|
+
out.push(`} .`);
|
|
4924
|
+
}
|
|
4925
|
+
return out.join('\n') + '\n';
|
|
4926
|
+
}
|
|
4927
|
+
|
|
4928
|
+
function forEachRdfMessageChunkInText(text, onMessage) {
|
|
4929
|
+
const directives = [];
|
|
4930
|
+
const seenDirectives = new Set();
|
|
4931
|
+
let chunk = '';
|
|
4932
|
+
let messageIndex = 1;
|
|
4933
|
+
let sawVersion = false;
|
|
4934
|
+
let sawDelimiter = false;
|
|
4935
|
+
|
|
4936
|
+
function emit() {
|
|
4937
|
+
onMessage({ messageIndex, chunk, directives: directives.slice() });
|
|
4938
|
+
messageIndex += 1;
|
|
4939
|
+
chunk = '';
|
|
4940
|
+
}
|
|
4941
|
+
|
|
4942
|
+
const lines = String(text || '').match(/.*(?:\r\n|\n|\r)|.+$/g) || [];
|
|
4943
|
+
for (const line of lines) {
|
|
4944
|
+
if (RDF_MESSAGE_VERSION_LINE_RE.test(line)) {
|
|
4945
|
+
sawVersion = true;
|
|
4946
|
+
continue;
|
|
4947
|
+
}
|
|
4948
|
+
if (RDF_MESSAGE_DELIMITER_LINE_RE.test(line)) {
|
|
4949
|
+
emit();
|
|
4950
|
+
sawDelimiter = true;
|
|
4951
|
+
continue;
|
|
4952
|
+
}
|
|
4953
|
+
addRdfDirective(directives, seenDirectives, line);
|
|
4954
|
+
chunk += line;
|
|
4955
|
+
}
|
|
4956
|
+
if (!sawVersion) throw new Error('not an RDF Message Log: missing VERSION "*-messages" directive');
|
|
4957
|
+
if (sawDelimiter || hasRdfPayload(chunk)) emit();
|
|
4958
|
+
}
|
|
4959
|
+
|
|
4960
|
+
function forEachLineInFileSync(filePath, onLine) {
|
|
4961
|
+
const fd = fs.openSync(filePath, 'r');
|
|
4962
|
+
const decoder = new TextDecoder('utf8');
|
|
4963
|
+
const buf = Buffer.allocUnsafe(64 * 1024);
|
|
4964
|
+
let carry = '';
|
|
4965
|
+
try {
|
|
4966
|
+
for (;;) {
|
|
4967
|
+
const n = fs.readSync(fd, buf, 0, buf.length, null);
|
|
4968
|
+
if (n === 0) break;
|
|
4969
|
+
carry += decoder.decode(buf.subarray(0, n), { stream: true });
|
|
4970
|
+
for (;;) {
|
|
4971
|
+
const m = /\r\n|\n|\r/.exec(carry);
|
|
4972
|
+
if (!m) break;
|
|
4973
|
+
const end = m.index + m[0].length;
|
|
4974
|
+
onLine(carry.slice(0, end));
|
|
4975
|
+
carry = carry.slice(end);
|
|
4976
|
+
}
|
|
4977
|
+
}
|
|
4978
|
+
carry += decoder.decode();
|
|
4979
|
+
if (carry) onLine(carry);
|
|
4980
|
+
} finally {
|
|
4981
|
+
fs.closeSync(fd);
|
|
4982
|
+
}
|
|
4983
|
+
}
|
|
4984
|
+
|
|
4985
|
+
function forEachRdfMessageChunkInFileSync(filePath, onMessage) {
|
|
4986
|
+
const directives = [];
|
|
4987
|
+
const seenDirectives = new Set();
|
|
4988
|
+
let chunk = '';
|
|
4989
|
+
let messageIndex = 1;
|
|
4990
|
+
let sawVersion = false;
|
|
4991
|
+
let sawDelimiter = false;
|
|
4992
|
+
|
|
4993
|
+
function emit() {
|
|
4994
|
+
onMessage({ messageIndex, chunk, directives: directives.slice() });
|
|
4995
|
+
messageIndex += 1;
|
|
4996
|
+
chunk = '';
|
|
4997
|
+
}
|
|
4998
|
+
|
|
4999
|
+
forEachLineInFileSync(filePath, (line) => {
|
|
5000
|
+
if (RDF_MESSAGE_VERSION_LINE_RE.test(line)) {
|
|
5001
|
+
sawVersion = true;
|
|
5002
|
+
return;
|
|
5003
|
+
}
|
|
5004
|
+
if (RDF_MESSAGE_DELIMITER_LINE_RE.test(line)) {
|
|
5005
|
+
emit();
|
|
5006
|
+
sawDelimiter = true;
|
|
5007
|
+
return;
|
|
5008
|
+
}
|
|
5009
|
+
addRdfDirective(directives, seenDirectives, line);
|
|
5010
|
+
chunk += line;
|
|
5011
|
+
});
|
|
5012
|
+
|
|
5013
|
+
if (!sawVersion) throw new Error('not an RDF Message Log: missing VERSION "*-messages" directive');
|
|
5014
|
+
if (sawDelimiter || hasRdfPayload(chunk)) emit();
|
|
5015
|
+
}
|
|
5016
|
+
|
|
5017
|
+
|
|
5018
|
+
function __forEachRdfMessageChunkSync(sourceLabel, onMessage) {
|
|
5019
|
+
const filePath = __localPathForSource(sourceLabel);
|
|
5020
|
+
if (filePath) {
|
|
5021
|
+
forEachRdfMessageChunkInFileSync(filePath, onMessage);
|
|
5022
|
+
return;
|
|
5023
|
+
}
|
|
5024
|
+
if (__isHttpSource(sourceLabel)) {
|
|
5025
|
+
const tmp = __downloadHttpSourceToTempFileSync(sourceLabel);
|
|
5026
|
+
try {
|
|
5027
|
+
forEachRdfMessageChunkInFileSync(tmp.file, onMessage);
|
|
5028
|
+
} finally {
|
|
5029
|
+
tmp.cleanup();
|
|
5030
|
+
}
|
|
5031
|
+
return;
|
|
5032
|
+
}
|
|
5033
|
+
forEachRdfMessageChunkInText(__readInputSourceSync(sourceLabel), onMessage);
|
|
5034
|
+
}
|
|
5035
|
+
|
|
5036
|
+
function factsContainOutputStrings(triplesForOutput) {
|
|
5037
|
+
const LOG_OUTPUT_STRING = 'http://www.w3.org/2000/10/swap/log#outputString';
|
|
5038
|
+
return (
|
|
5039
|
+
Array.isArray(triplesForOutput) &&
|
|
5040
|
+
triplesForOutput.some(
|
|
5041
|
+
(tr) => tr && tr.p && tr.p.constructor && tr.p.constructor.name === 'Iri' && tr.p.value === LOG_OUTPUT_STRING,
|
|
5042
|
+
)
|
|
5043
|
+
);
|
|
5044
|
+
}
|
|
5045
|
+
|
|
5046
|
+
function programMayProduceOutputStrings(topLevelTriples, forwardRules, logQueryRules) {
|
|
5047
|
+
const hasOutputStringPredicate = (trs) => factsContainOutputStrings(trs);
|
|
5048
|
+
if (hasOutputStringPredicate(topLevelTriples)) return true;
|
|
5049
|
+
if (Array.isArray(forwardRules) && forwardRules.some((r) => hasOutputStringPredicate(r && r.conclusion))) return true;
|
|
5050
|
+
if (Array.isArray(logQueryRules) && logQueryRules.some((r) => hasOutputStringPredicate(r && r.conclusion))) return true;
|
|
5051
|
+
return false;
|
|
5052
|
+
}
|
|
5053
|
+
|
|
5054
|
+
function runParsedDocumentOnce(mergedDocument, { rdfMode = false, outputPrefixes = null } = {}) {
|
|
5055
|
+
const prefixes = mergedDocument.prefixes;
|
|
5056
|
+
const triples = mergedDocument.triples;
|
|
5057
|
+
const frules = mergedDocument.frules;
|
|
5058
|
+
const brules = mergedDocument.brules;
|
|
5059
|
+
const qrules = mergedDocument.logQueryRules;
|
|
5060
|
+
|
|
5061
|
+
engine.materializeRdfLists(triples, frules.concat(qrules || []), brules);
|
|
5062
|
+
const facts = triples.slice();
|
|
5063
|
+
const hasQueries = Array.isArray(qrules) && qrules.length;
|
|
5064
|
+
const mayAutoRenderOutputStrings = programMayProduceOutputStrings(triples, frules, qrules);
|
|
5065
|
+
|
|
5066
|
+
let derived = [];
|
|
5067
|
+
let outTriples = [];
|
|
5068
|
+
if (hasQueries) {
|
|
5069
|
+
const res = engine.forwardChainAndCollectLogQueryConclusions(facts, frules, brules, qrules, { prefixes });
|
|
5070
|
+
derived = res.derived;
|
|
5071
|
+
outTriples = res.queryTriples;
|
|
5072
|
+
} else {
|
|
5073
|
+
const skipDerivedCollection = mayAutoRenderOutputStrings;
|
|
5074
|
+
derived = engine.forwardChain(facts, frules, brules, null, {
|
|
5075
|
+
captureExplanations: false,
|
|
5076
|
+
collectDerived: !skipDerivedCollection,
|
|
5077
|
+
prefixes,
|
|
5078
|
+
});
|
|
5079
|
+
outTriples = skipDerivedCollection ? [] : derived.map((df) => df.fact);
|
|
5080
|
+
}
|
|
5081
|
+
|
|
5082
|
+
const renderedOutputTriples = hasQueries ? outTriples : facts;
|
|
5083
|
+
if (factsContainOutputStrings(renderedOutputTriples)) {
|
|
5084
|
+
process.stdout.write(engine.collectOutputStringsFromFacts(renderedOutputTriples, prefixes));
|
|
5085
|
+
return;
|
|
5086
|
+
}
|
|
5087
|
+
|
|
5088
|
+
const outPrefixEnv = outputPrefixes || prefixes;
|
|
5089
|
+
if (rdfMode) {
|
|
5090
|
+
for (const tr of outTriples) console.log(engine.tripleToRdfCompatible(tr, outPrefixEnv));
|
|
5091
|
+
} else if (hasQueries) {
|
|
5092
|
+
const bodyText = engine.prettyPrintQueryTriples(outTriples, outPrefixEnv);
|
|
5093
|
+
if (bodyText) process.stdout.write(String(bodyText).replace(/\s*$/g, '') + '\n');
|
|
5094
|
+
} else {
|
|
5095
|
+
for (const df of derived) console.log(engine.tripleToN3(df.fact, outPrefixEnv));
|
|
5096
|
+
}
|
|
5097
|
+
}
|
|
5098
|
+
|
|
5099
|
+
function runStreamMessagesMode(sourceLabels, { rdfMode }) {
|
|
5100
|
+
const ordinarySourceLabels = [];
|
|
5101
|
+
const messageSourceLabels = [];
|
|
5102
|
+
|
|
5103
|
+
for (const sourceLabel of sourceLabels) {
|
|
5104
|
+
try {
|
|
5105
|
+
if (__sourceLooksLikeRdfMessageLogSync(sourceLabel)) messageSourceLabels.push(sourceLabel);
|
|
5106
|
+
else ordinarySourceLabels.push(sourceLabel);
|
|
5107
|
+
} catch (e) {
|
|
5108
|
+
console.error(`Error reading source ${JSON.stringify(sourceLabel)}: ${e && e.message ? e.message : String(e)}`);
|
|
5109
|
+
process.exit(1);
|
|
5110
|
+
}
|
|
5111
|
+
}
|
|
5112
|
+
|
|
5113
|
+
if (!messageSourceLabels.length) {
|
|
5114
|
+
console.error('Error: --stream-messages did not find any RDF Message Log input.');
|
|
5115
|
+
process.exit(1);
|
|
5116
|
+
}
|
|
5117
|
+
|
|
5118
|
+
const programSources = [];
|
|
5119
|
+
for (const sourceLabel of ordinarySourceLabels) {
|
|
5120
|
+
let text;
|
|
5121
|
+
try {
|
|
5122
|
+
text = __readInputSourceSync(sourceLabel);
|
|
5123
|
+
} catch (e) {
|
|
5124
|
+
if (sourceLabel === '<stdin>') console.error(`Error reading stdin: ${e.message}`);
|
|
5125
|
+
else console.error(`Error reading source ${JSON.stringify(sourceLabel)}: ${e.message}`);
|
|
5126
|
+
process.exit(1);
|
|
5127
|
+
}
|
|
5128
|
+
|
|
5129
|
+
try {
|
|
5130
|
+
programSources.push(
|
|
5131
|
+
parseN3Text(text, {
|
|
5132
|
+
baseIri: __sourceLabelToBaseIri(sourceLabel),
|
|
5133
|
+
label: sourceLabel,
|
|
5134
|
+
keepSourceArtifacts: false,
|
|
5135
|
+
sourceLocations: false,
|
|
5136
|
+
rdf: rdfMode,
|
|
5137
|
+
}),
|
|
5138
|
+
);
|
|
5139
|
+
} catch (e) {
|
|
5140
|
+
if (e && e.name === 'N3SyntaxError') {
|
|
5141
|
+
console.error(formatN3SyntaxError(e, text, sourceLabel));
|
|
5142
|
+
process.exit(1);
|
|
5143
|
+
}
|
|
5144
|
+
throw e;
|
|
5145
|
+
}
|
|
5146
|
+
}
|
|
5147
|
+
|
|
5148
|
+
const fullIriPrefixes = new PrefixEnv({});
|
|
5149
|
+
for (const messageSourceLabel of messageSourceLabels) {
|
|
5150
|
+
try {
|
|
5151
|
+
__forEachRdfMessageChunkSync(messageSourceLabel, ({ messageIndex, chunk, directives }) => {
|
|
5152
|
+
const messageText = buildSingleMessageReplayDocument({
|
|
5153
|
+
sourceLabel: messageSourceLabel,
|
|
5154
|
+
messageIndex,
|
|
5155
|
+
chunk,
|
|
5156
|
+
directives,
|
|
5157
|
+
});
|
|
5158
|
+
let messageDoc;
|
|
5159
|
+
try {
|
|
5160
|
+
messageDoc = parseN3Text(messageText, {
|
|
5161
|
+
baseIri: `${__sourceLabelToBaseIri(messageSourceLabel)}#message-${messageIndex}`,
|
|
5162
|
+
label: `${messageSourceLabel}#message-${messageIndex}`,
|
|
5163
|
+
keepSourceArtifacts: false,
|
|
5164
|
+
sourceLocations: false,
|
|
5165
|
+
rdf: false,
|
|
5166
|
+
});
|
|
5167
|
+
} catch (e) {
|
|
5168
|
+
if (e && e.name === 'N3SyntaxError') {
|
|
5169
|
+
console.error(formatN3SyntaxError(e, messageText, `${messageSourceLabel}#message-${messageIndex}`));
|
|
5170
|
+
process.exit(1);
|
|
5171
|
+
}
|
|
5172
|
+
throw e;
|
|
5173
|
+
}
|
|
5174
|
+
|
|
5175
|
+
const merged = mergeParsedDocuments(programSources.concat([messageDoc]));
|
|
5176
|
+
runParsedDocumentOnce(merged, { rdfMode, outputPrefixes: fullIriPrefixes });
|
|
5177
|
+
});
|
|
5178
|
+
} catch (e) {
|
|
5179
|
+
console.error(`Error streaming RDF Message Log ${JSON.stringify(messageSourceLabel)}: ${e && e.message ? e.message : String(e)}`);
|
|
5180
|
+
process.exit(1);
|
|
5181
|
+
}
|
|
5182
|
+
}
|
|
5183
|
+
}
|
|
5184
|
+
|
|
4696
5185
|
function main() {
|
|
4697
5186
|
// Drop "node" and script name; keep only user-provided args
|
|
4698
5187
|
// Expand combined short options: -pt == -p -t
|
|
@@ -4723,6 +5212,7 @@ function main() {
|
|
|
4723
5212
|
` -h, --help Show this help and exit.\n` +
|
|
4724
5213
|
` -p, --proof Enable proof explanations.\n` +
|
|
4725
5214
|
` -r, --rdf Enable RDF/TriG input/output compatibility.\n` +
|
|
5215
|
+
` --stream-messages Process RDF Message Logs one message at a time under -r.\n` +
|
|
4726
5216
|
` -s, --super-restricted Disable all builtins except => and <=.\n` +
|
|
4727
5217
|
` -t, --stream Stream derived triples as soon as they are derived.\n` +
|
|
4728
5218
|
` -v, --version Print version and exit.\n`;
|
|
@@ -4764,6 +5254,7 @@ function main() {
|
|
|
4764
5254
|
|
|
4765
5255
|
const showAst = argv.includes('--ast') || argv.includes('-a');
|
|
4766
5256
|
const streamMode = argv.includes('--stream') || argv.includes('-t');
|
|
5257
|
+
const streamMessagesMode = argv.includes('--stream-messages');
|
|
4767
5258
|
const rdfMode = argv.includes('--rdf') || argv.includes('-r');
|
|
4768
5259
|
|
|
4769
5260
|
// --enforce-https: rewrite http:// -> https:// for log dereferencing builtins
|
|
@@ -4786,6 +5277,26 @@ function main() {
|
|
|
4786
5277
|
if (typeof engine.setSuperRestrictedMode === 'function') engine.setSuperRestrictedMode(true);
|
|
4787
5278
|
}
|
|
4788
5279
|
|
|
5280
|
+
|
|
5281
|
+
if (streamMessagesMode) {
|
|
5282
|
+
if (!rdfMode) {
|
|
5283
|
+
console.error('Error: --stream-messages requires -r/--rdf.');
|
|
5284
|
+
process.exit(1);
|
|
5285
|
+
}
|
|
5286
|
+
if (showAst) {
|
|
5287
|
+
console.error('Error: --stream-messages cannot be combined with --ast.');
|
|
5288
|
+
process.exit(1);
|
|
5289
|
+
}
|
|
5290
|
+
if (streamMode) {
|
|
5291
|
+
console.error('Error: --stream-messages cannot be combined with --stream.');
|
|
5292
|
+
process.exit(1);
|
|
5293
|
+
}
|
|
5294
|
+
if (engine.getProofCommentsEnabled()) {
|
|
5295
|
+
console.error('Error: --stream-messages currently does not support proof output.');
|
|
5296
|
+
process.exit(1);
|
|
5297
|
+
}
|
|
5298
|
+
}
|
|
5299
|
+
|
|
4789
5300
|
// Positional args (one or more N3 sources).
|
|
4790
5301
|
const useImplicitStdin = positional.length === 0 && !process.stdin.isTTY;
|
|
4791
5302
|
if (positional.length === 0 && !useImplicitStdin) {
|
|
@@ -4809,6 +5320,11 @@ function main() {
|
|
|
4809
5320
|
process.exit(1);
|
|
4810
5321
|
}
|
|
4811
5322
|
|
|
5323
|
+
if (streamMessagesMode) {
|
|
5324
|
+
runStreamMessagesMode(sourceLabels, { rdfMode });
|
|
5325
|
+
return;
|
|
5326
|
+
}
|
|
5327
|
+
|
|
4812
5328
|
const parsedSources = [];
|
|
4813
5329
|
for (const sourceLabel of sourceLabels) {
|
|
4814
5330
|
let text;
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# ACT barley seed lineage — can and can't
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[Start with the source](../act-barley-seed-lineage.n3) to see the ACT model itself; the [golden output](../output/act-barley-seed-lineage.md) shows the lineage conclusions that Eyeling derives from it.
|
|
4
4
|
|
|
5
5
|
The aim is to show how **Applied Constructor Theory** can describe a concrete biological case in Notation3 while doing both sides of Chiara Marletto’s formula:
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Faltings’ theorem (emulated) in Notation3: a genus‑2 curve over Q
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[Start with the source](../faltings-genus2-finiteness.n3) for the emulated theorem statement and inference rules; the [golden output](../output/faltings-genus2-finiteness.n3) shows the finite-rational-points conclusion produced by the example.
|
|
4
4
|
|
|
5
5
|
The goal is to show—at a friendly, “wide audience” level—how an N3 file can _model_ a famous mathematical implication:
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# High-trust RDF graph lookup with a decimal certificate
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[Start with the source](../high-trust-rdf-bloom-envelope.n3) for the Bloom-filter certificate and arithmetic checks; the [golden output](../output/high-trust-rdf-bloom-envelope.n3) shows the trust decision that Eyeling derives.
|
|
4
4
|
|
|
5
5
|
The goal is to show that **advanced engineering claims can be expressed and checked in N3**, even when the claim involves a **transcendental quantity** such as `exp(-k*n/m)`.
|
|
6
6
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
## Ranked, explainable output from machine-readable “Terms of Service”
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
[Start with the source](../odrl-dpv-risk-ranked.n3) to inspect the machine-readable policy and risk rules; the [golden output](../output/odrl-dpv-risk-ranked.md) shows the ranked explanation produced from them.
|
|
6
6
|
|
|
7
7
|
---
|
|
8
8
|
|