veriquote 0.2.1 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CITATION.cff +1 -1
- package/README.md +54 -34
- package/dist/cli/main.d.ts.map +1 -1
- package/dist/cli/main.js +30 -4
- package/dist/cli/main.js.map +1 -1
- package/dist/gate.d.ts +12 -2
- package/dist/gate.d.ts.map +1 -1
- package/dist/gate.js +6 -2
- package/dist/gate.js.map +1 -1
- package/dist/source/html.d.ts.map +1 -1
- package/dist/source/html.js +3 -0
- package/dist/source/html.js.map +1 -1
- package/package.json +1 -1
- package/src/cli/main.ts +28 -4
- package/src/gate.ts +14 -4
- package/src/source/html.ts +3 -0
package/CITATION.cff
CHANGED
package/README.md
CHANGED
|
@@ -6,14 +6,20 @@
|
|
|
6
6
|
**Make an LLM quote its sources, then check every quote twice: is it really in
|
|
7
7
|
the source, and does it support the claim?**
|
|
8
8
|
|
|
9
|
-
A `[1]` after a sentence looks like evidence, but
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
9
|
+
A `[1]` after a sentence looks like evidence, but checking it means opening the
|
|
10
|
+
source and finding the passage. VeriQuote makes the answering model attach a
|
|
11
|
+
verbatim quote to every citation and then checks each quote twice: a
|
|
12
|
+
deterministic matcher confirms that the quote is in the source, and an LLM
|
|
13
|
+
judge decides whether the quote supports the claim.
|
|
14
|
+
|
|
15
|
+
Both checks are needed. Strong models copy quotes reliably: the best open
|
|
16
|
+
models in my tests quoted verbatim in over 99% of cases, and a recent study
|
|
17
|
+
measured 98% for a frontier model. But a real quote is not a supported claim;
|
|
18
|
+
in that study, only 37% of the model's claims were fully supported by their
|
|
19
|
+
quotes.[^zhang] Weaker models also leave citations without any quote, and a
|
|
20
|
+
judge asked whether a quote supports a claim is not built to notice that the
|
|
21
|
+
quote was made up. VeriQuote is written in TypeScript, has no dependencies and
|
|
22
|
+
runs in Node and in the browser.
|
|
17
23
|
|
|
18
24
|
**Try it in the browser:** [rickinto.place/veriquote](https://rickinto.place/veriquote)
|
|
19
25
|
has eight worked examples, a box for your own text, and the benchmark results as
|
|
@@ -81,8 +87,8 @@ checks them. Each citation then goes through three steps:
|
|
|
81
87
|
A citation's combined score is the lower of its match score and its support
|
|
82
88
|
score. A judge error can therefore never lift a citation above what the matcher
|
|
83
89
|
found, and a failed judge call is reported as an error, never counted as
|
|
84
|
-
support.
|
|
85
|
-
|
|
90
|
+
support. The quote format is not meant to make a model more accurate; it makes
|
|
91
|
+
every claim checkable.
|
|
86
92
|
|
|
87
93
|
## Benchmarks
|
|
88
94
|
|
|
@@ -92,13 +98,14 @@ Everything is in [`bench/`](bench), including how to reproduce it.
|
|
|
92
98
|
|
|
93
99
|
<picture>
|
|
94
100
|
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/rickintoplace/veriquote/main/bench/figures/tango-dark.svg">
|
|
95
|
-
<img alt="The matcher blocks all 187 quotes that are not in the source and none of the 586 real quotes whose meaning was changed; the judge blocks 84% of unsupported citations
|
|
101
|
+
<img alt="The matcher blocks all 187 quotes that are not in the source and none of the 586 real quotes whose meaning was changed; the judge blocks 84% of unsupported citations but does not check whether a quote exists; together they cover both kinds of failure." src="https://raw.githubusercontent.com/rickintoplace/veriquote/main/bench/figures/tango-light.svg">
|
|
96
102
|
</picture>
|
|
97
103
|
|
|
98
104
|
The two checks are blind in opposite places. The matcher never reads the claim,
|
|
99
|
-
so it cannot tell whether a real quote supports it. The judge
|
|
100
|
-
|
|
101
|
-
|
|
105
|
+
so it cannot tell whether a real quote supports it. The judge is asked whether
|
|
106
|
+
the quote supports the claim, not whether the quote exists. It sees a little of
|
|
107
|
+
the source as context, but an invented quote that fits the claim can still pass
|
|
108
|
+
it: in the example above, it rates the made-up quote in `c4` as "entailed 0.90".
|
|
102
109
|
|
|
103
110
|
### Matcher
|
|
104
111
|
|
|
@@ -141,9 +148,11 @@ annotators on citation precision.
|
|
|
141
148
|
Before trusting any of this, look at the left panel: even the best judge lets
|
|
142
149
|
one unsupported citation in six through as fully supported. No judge should be
|
|
143
150
|
the only check. With only 56 unsupported pairs per run the intervals are wide,
|
|
144
|
-
so neighbouring models are not really separated.
|
|
145
|
-
|
|
146
|
-
|
|
151
|
+
so neighbouring models are not really separated. With reasoning switched off,
|
|
152
|
+
all three hybrid models caught less and agreed less while answering five to ten
|
|
153
|
+
times faster; each difference is within the sampling error, but all three point
|
|
154
|
+
the same way. These pairs come from ALCE, where the judge gets a whole passage
|
|
155
|
+
rather than a quote, which makes its task harder than in normal use.
|
|
147
156
|
|
|
148
157
|
### Answering models
|
|
149
158
|
|
|
@@ -156,12 +165,14 @@ Each model answered 18 questions twice, with the citation instructions in its
|
|
|
156
165
|
system prompt, and `parseAnswer()` decided mechanically whether it complied.
|
|
157
166
|
Every raw answer is in
|
|
158
167
|
[`bench/results/protocol-answers.jsonl`](bench/results/protocol-answers.jsonl).
|
|
159
|
-
The open models
|
|
160
|
-
others
|
|
161
|
-
appendix, yet only a third of its answers give every citation a
|
|
162
|
-
`mistral-medium` is nearly always complete,
|
|
163
|
-
|
|
164
|
-
|
|
168
|
+
The strongest open models tested follow the format almost perfectly. Two
|
|
169
|
+
others show failures a reader would not notice: `gpt-oss-120b` almost always
|
|
170
|
+
prints the appendix, yet only a third of its answers give every citation a
|
|
171
|
+
quote. `mistral-medium` is nearly always complete, but only 81% of its quotes
|
|
172
|
+
are exact copies; the rest mostly join passages with "..." or change a few
|
|
173
|
+
words, and the matcher still finds them at a lower score. On the three
|
|
174
|
+
questions the sources cannot answer, no model attached a citation without a
|
|
175
|
+
real quote.
|
|
165
176
|
|
|
166
177
|
<details>
|
|
167
178
|
<summary>The numbers behind the figures</summary>
|
|
@@ -203,20 +214,29 @@ The figures are rendered from `bench/results/*.json` by
|
|
|
203
214
|
|
|
204
215
|
An agent that reads sources and writes conclusions is exactly the case
|
|
205
216
|
VeriQuote was built for. [`verify-citations`](integrations/verify-citations) is
|
|
206
|
-
an [Agent Skill](integrations/verify-citations/SKILL.md)
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
217
|
+
an [Agent Skill](integrations/verify-citations/SKILL.md) for Claude Code and
|
|
218
|
+
other hosts that read Agent Skills. With it, a research answer goes like this:
|
|
219
|
+
|
|
220
|
+
1. The agent reads each source with `veriquote source <url>`, which prints the
|
|
221
|
+
exact text the check will use. Web-fetch tools often summarize or reformat
|
|
222
|
+
pages, so quotes copied from them may not match.
|
|
223
|
+
2. It writes the answer in the checkable format and runs
|
|
224
|
+
`veriquote check answer.md --source <url> … --json`. The CLI fetches every
|
|
225
|
+
URL itself, so a truncated or misremembered copy of a page cannot pass.
|
|
226
|
+
3. The exit code says what to do next:
|
|
211
227
|
|
|
212
228
|
```
|
|
213
|
-
exit 0 verdict "pass"
|
|
214
|
-
exit 2 verdict "revise"
|
|
215
|
-
exit 1
|
|
229
|
+
exit 0 verdict "pass" present cleanAnswer, the sources and one line on the check
|
|
230
|
+
exit 2 verdict "revise" follow instructionsForModel, then check again
|
|
231
|
+
exit 1 verdict "unverified" the judge failed; check again before claiming anything
|
|
232
|
+
exit 1 (no verdict) bad input or unreachable source
|
|
216
233
|
```
|
|
217
234
|
|
|
218
|
-
The
|
|
219
|
-
|
|
235
|
+
The user sees only the final answer, with `[n]` markers, the list of sources
|
|
236
|
+
and a line such as "Checked with VeriQuote: all 6 quotes were found in the
|
|
237
|
+
sources and judged to support their claims." The quote appendix, the JSON and
|
|
238
|
+
any revision rounds stay with the agent. The CLI works in anything that can run
|
|
239
|
+
a shell command.
|
|
220
240
|
|
|
221
241
|
## Use as a library
|
|
222
242
|
|
|
@@ -354,4 +374,4 @@ are also in [`CITATION.cff`](CITATION.cff).
|
|
|
354
374
|
|
|
355
375
|
[^zhang]: Zhang, J., Chen, Y., Commodore-Mensah, Y., & Oberst, M. (2026). *Verifiable by construction: Claim-level evaluation of verbatim citation in clinical question answering* (Version 2) [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2609.15964
|
|
356
376
|
|
|
357
|
-
[^veriquote]: Heilmann, E. (2026). *VeriQuote: Deterministic and semantic verification of quote-grounded LLM citations* (Version 0.2.
|
|
377
|
+
[^veriquote]: Heilmann, E. (2026). *VeriQuote: Deterministic and semantic verification of quote-grounded LLM citations* (Version 0.2.2) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.21552379
|
package/dist/cli/main.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"main.d.ts","sourceRoot":"","sources":["../../src/cli/main.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,KAAK,EAAE,eAAe,EAAsC,MAAM,aAAa,CAAC;AAOvF,MAAM,WAAW,KAAK;IACpB,IAAI,EAAE,MAAM,EAAE,CAAC;IACf,GAAG,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,GAAG,SAAS,CAAC,CAAC;IACxC,MAAM,EAAE,CAAC,CAAC,EAAE,MAAM,KAAK,IAAI,CAAC;IAC5B,MAAM,EAAE,CAAC,CAAC,EAAE,MAAM,KAAK,IAAI,CAAC;IAC5B,QAAQ,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,OAAO,CAAC,MAAM,CAAC,CAAC;IAC5C,SAAS,EAAE,MAAM,OAAO,CAAC,MAAM,CAAC,CAAC;IACjC,KAAK,CAAC,EAAE,OAAO,UAAU,CAAC,KAAK,CAAC;IAChC,OAAO,EAAE,MAAM,CAAC;IAChB,uCAAuC;IACvC,KAAK,EAAE,OAAO,CAAC;IACf,oDAAoD;IACpD,KAAK,CAAC,EAAE,eAAe,CAAC;CACzB;AAED,eAAO,MAAM,IAAI;;;;CAA4C,CAAC;
|
|
1
|
+
{"version":3,"file":"main.d.ts","sourceRoot":"","sources":["../../src/cli/main.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,KAAK,EAAE,eAAe,EAAsC,MAAM,aAAa,CAAC;AAOvF,MAAM,WAAW,KAAK;IACpB,IAAI,EAAE,MAAM,EAAE,CAAC;IACf,GAAG,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,GAAG,SAAS,CAAC,CAAC;IACxC,MAAM,EAAE,CAAC,CAAC,EAAE,MAAM,KAAK,IAAI,CAAC;IAC5B,MAAM,EAAE,CAAC,CAAC,EAAE,MAAM,KAAK,IAAI,CAAC;IAC5B,QAAQ,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,OAAO,CAAC,MAAM,CAAC,CAAC;IAC5C,SAAS,EAAE,MAAM,OAAO,CAAC,MAAM,CAAC,CAAC;IACjC,KAAK,CAAC,EAAE,OAAO,UAAU,CAAC,KAAK,CAAC;IAChC,OAAO,EAAE,MAAM,CAAC;IAChB,uCAAuC;IACvC,KAAK,EAAE,OAAO,CAAC;IACf,oDAAoD;IACpD,KAAK,CAAC,EAAE,eAAe,CAAC;CACzB;AAED,eAAO,MAAM,IAAI;;;;CAA4C,CAAC;AAmG9D,wBAAsB,IAAI,CAAC,EAAE,EAAE,KAAK,GAAG,OAAO,CAAC,MAAM,CAAC,CAsCrD"}
|
package/dist/cli/main.js
CHANGED
|
@@ -12,6 +12,7 @@ const HELP = `veriquote — check that an LLM answer's quotes exist in its sourc
|
|
|
12
12
|
|
|
13
13
|
Usage
|
|
14
14
|
veriquote prompt print the citation instructions for the answering model
|
|
15
|
+
veriquote source <url|file> print the text a source is checked against (quote from this)
|
|
15
16
|
veriquote check <answer> --source <url|file> [--source …]
|
|
16
17
|
veriquote check --job <file|-> JSON job: {"answer": "...", "sources": [{"text": "..."}]}
|
|
17
18
|
|
|
@@ -34,7 +35,7 @@ Judge (semantic support check), from the environment:
|
|
|
34
35
|
VERIQUOTE_JUDGE_MODEL required when a key is set
|
|
35
36
|
VERIQUOTE_JUDGE_BASE_URL default https://openrouter.ai/api/v1
|
|
36
37
|
|
|
37
|
-
Exit status: 0 pass · 2 revise · 1 error`;
|
|
38
|
+
Exit status: 0 pass · 2 revise · 1 error, or the judge failed (unverified)`;
|
|
38
39
|
class UsageError extends Error {
|
|
39
40
|
}
|
|
40
41
|
function parseArgs(argv) {
|
|
@@ -117,6 +118,13 @@ export async function main(io) {
|
|
|
117
118
|
return EXIT.pass;
|
|
118
119
|
case 'check':
|
|
119
120
|
return await check(args, io);
|
|
121
|
+
case 'source': {
|
|
122
|
+
if (args.positional.length !== 1)
|
|
123
|
+
throw new UsageError('source needs exactly one URL or file');
|
|
124
|
+
const src = await loadSource(args.positional[0], io);
|
|
125
|
+
io.stdout(`${src.text}\n`);
|
|
126
|
+
return EXIT.pass;
|
|
127
|
+
}
|
|
120
128
|
default:
|
|
121
129
|
throw new UsageError(`unknown command "${args.command}"`);
|
|
122
130
|
}
|
|
@@ -162,9 +170,21 @@ async function check(args, io) {
|
|
|
162
170
|
io.stdout(`${JSON.stringify({
|
|
163
171
|
verdict: gate.verdict,
|
|
164
172
|
judge: judgeSetup.model ?? null,
|
|
173
|
+
// The answer as the user should see it: [n] markers kept, {cX} markers and the appendix removed.
|
|
174
|
+
cleanAnswer: report.cleanText,
|
|
165
175
|
summary: report.summary,
|
|
166
176
|
problems: gate.problems,
|
|
167
177
|
uncited: gate.uncited,
|
|
178
|
+
unjudged: gate.unjudged,
|
|
179
|
+
citations: report.citations.map((c) => ({
|
|
180
|
+
claimId: c.claimId,
|
|
181
|
+
sourceIndex: c.sourceIndex,
|
|
182
|
+
match: { method: c.textMatch.method, score: round2(c.textMatch.score) },
|
|
183
|
+
judge: c.entailment
|
|
184
|
+
? { class: c.entailment.class, support: c.entailment.confidence === null ? null : round2(c.entailment.confidence) }
|
|
185
|
+
: null,
|
|
186
|
+
score: c.score === null ? null : round2(c.score),
|
|
187
|
+
})),
|
|
168
188
|
warnings: report.warnings,
|
|
169
189
|
sources: sources.map((s, i) => ({ index: i + 1, url: s.url, title: s.title, chars: s.text.length })),
|
|
170
190
|
instructionsForModel: gate.instructionsForModel,
|
|
@@ -173,8 +193,9 @@ async function check(args, io) {
|
|
|
173
193
|
else {
|
|
174
194
|
io.stdout(renderHuman(report, gate, sources, judgeSetup.model, io.color, args.minScore ?? 0.5));
|
|
175
195
|
}
|
|
176
|
-
return gate.verdict === 'pass' ? EXIT.pass : EXIT.revise;
|
|
196
|
+
return gate.verdict === 'pass' ? EXIT.pass : gate.verdict === 'revise' ? EXIT.revise : EXIT.error;
|
|
177
197
|
}
|
|
198
|
+
const round2 = (x) => Math.round(x * 100) / 100;
|
|
178
199
|
async function loadSource(ref, io) {
|
|
179
200
|
if (/^https?:\/\//i.test(ref)) {
|
|
180
201
|
return fetchSource(ref, {
|
|
@@ -242,14 +263,19 @@ function renderHuman(report, gate, sources, model, color, minScore) {
|
|
|
242
263
|
out.push(dim(`warning: ${w}`));
|
|
243
264
|
out.push('');
|
|
244
265
|
if (gate.verdict === 'pass') {
|
|
245
|
-
out.push(green(report.citations.length ? 'PASS' : 'PASS
|
|
266
|
+
out.push(green(report.citations.length ? 'PASS' : 'PASS, but the answer cites nothing'));
|
|
267
|
+
}
|
|
268
|
+
else if (gate.verdict === 'unverified') {
|
|
269
|
+
out.push(yellow(`UNVERIFIED: the judge failed on ${gate.unjudged.length} citation(s), so their support was not checked. Run the check again.`));
|
|
246
270
|
}
|
|
247
271
|
else {
|
|
248
272
|
const parts = [
|
|
249
273
|
gate.problems.length && `${gate.problems.length} failed citation(s)`,
|
|
250
274
|
gate.uncited.length && `${gate.uncited.length} uncited sentence(s)`,
|
|
251
275
|
].filter(Boolean);
|
|
252
|
-
out.push(red(`REVISE
|
|
276
|
+
out.push(red(`REVISE: ${parts.join(', ')}`));
|
|
277
|
+
if (gate.unjudged.length)
|
|
278
|
+
out.push(yellow(`The judge also failed on ${gate.unjudged.length} citation(s).`));
|
|
253
279
|
}
|
|
254
280
|
if (!model)
|
|
255
281
|
out.push(dim('Only checked that quotes exist in their sources. Set VERIQUOTE_JUDGE_API_KEY to check support.'));
|
package/dist/cli/main.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"main.js","sourceRoot":"","sources":["../../src/cli/main.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAGH,OAAO,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAC5C,OAAO,EAAE,UAAU,EAAmB,MAAM,YAAY,CAAC;AACzD,OAAO,EAAE,yBAAyB,EAAE,MAAM,uBAAuB,CAAC;AAClE,OAAO,EAAE,oBAAoB,EAAE,MAAM,wBAAwB,CAAC;AAC9D,OAAO,EAAE,WAAW,EAAE,cAAc,EAAE,MAAM,oBAAoB,CAAC;AAiBjE,MAAM,CAAC,MAAM,IAAI,GAAG,EAAE,IAAI,EAAE,CAAC,EAAE,KAAK,EAAE,CAAC,EAAE,MAAM,EAAE,CAAC,EAAW,CAAC;AAE9D,MAAM,IAAI,GAAG;;;;;;;;;;;;;;;;;;;;;;;;;;yCA0B4B,CAAC;AAgB1C,MAAM,UAAW,SAAQ,KAAK;CAAG;AAEjC,SAAS,SAAS,CAAC,IAAc;IAC/B,MAAM,CAAC,GAAS,EAAE,UAAU,EAAE,EAAE,EAAE,OAAO,EAAE,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,KAAK,EAAE,OAAO,EAAE,KAAK,EAAE,CAAC;IACvG,MAAM,KAAK,GAAG,CAAC,CAAS,EAAE,IAAY,EAAU,EAAE;QAChD,MAAM,CAAC,GAAG,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;QACtB,IAAI,CAAC,KAAK,SAAS;YAAE,MAAM,IAAI,UAAU,CAAC,GAAG,IAAI,gBAAgB,CAAC,CAAC;QACnE,OAAO,CAAC,CAAC;IACX,CAAC,CAAC;IACF,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACrC,MAAM,GAAG,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QACpB,QAAQ,GAAG,EAAE,CAAC;YACZ,KAAK,IAAI,CAAC;YACV,KAAK,UAAU;gBACb,CAAC,CAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,GAAG,CAAC,CAAC,CAAC;gBAChC,MAAM;YACR,KAAK,OAAO;gBACV,CAAC,CAAC,GAAG,GAAG,KAAK,CAAC,CAAC,EAAE,EAAE,GAAG,CAAC,CAAC;gBACxB,MAAM;YACR,KAAK,QAAQ;gBACX,CAAC,CAAC,IAAI,GAAG,IAAI,CAAC;gBACd,MAAM;YACR,KAAK,YAAY;gBACf,CAAC,CAAC,KAAK,GAAG,KAAK,CAAC;gBAChB,MAAM;YACR,KAAK,aAAa,CAAC,CAAC,CAAC;gBACnB,MAAM,CAAC,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,GAAG,CAAC,CAAC,CAAC;gBAClC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;oBAAE,MAAM,IAAI,UAAU,CAAC,qCAAqC,CAAC,CAAC;gBACrF,CAAC,CAAC,QAAQ,GAAG,CAAC,CAAC;gBACf,MAAM;YACR,CAAC;YACD,KAAK,eAAe;gBAClB,CAAC,CAAC,UAAU,GAAG,KAAK,CAAC,CAAC,EAAE,EAAE,GAAG,CAAC,CAAC;gBAC/B,MAAM;YACR,KAAK,kBAAkB;gBACrB,CAAC,CAAC,YAAY,GAAG,KAAK,CAAC,CAAC,EAAE,EAAE,GAAG,CAAC,CAAC;gBACjC,MAAM;YACR,KAAK,IAAI,CAAC;YACV,KAAK,QAAQ;gBACX,CAAC,CAAC,IAAI,GAAG,IAAI,CAAC;gBACd,MAAM;YACR,KAAK,IAAI,CAAC;YACV,KAAK,WAAW;gBACd,CAAC,CAAC,OAAO,GAAG,IAAI,CAAC;gBACjB,MAAM;YACR;gBACE,IAAI,GAAG,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,GAAG,KAAK,GAAG;oBAAE,MAAM,IAAI,UAAU,CAAC,kBAAkB,GAAG,EAAE,CAAC,CAAC;gBACtF,IAAI,CAAC,CAAC,OAAO,KAAK,SAAS;oBAAE,CAAC,CAAC,OAAO,GAAG,GAAG,CAAC;;oBACxC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QAChC,CAAC;IACH,CAAC;IACD,OAAO,CAAC,CAAC;AACX,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,IAAI,CAAC,EAAS;IAClC,IAAI,IAAU,CAAC;IACf,IAAI,CAAC;QACH,IAAI,GAAG,SAAS,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC;IAC5B,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,EAAE,CAAC,MAAM,CAAC,cAAe,GAAa,CAAC,OAAO,uCAAuC,CAAC,CAAC;QACvF,OAAO,IAAI,CAAC,KAAK,CAAC;IACpB,CAAC;IACD,IAAI,IAAI,CAAC,OAAO,EAAE,CAAC;QACjB,EAAE,CAAC,MAAM,CAAC,GAAG,EAAE,CAAC,OAAO,IAAI,CAAC,CAAC;QAC7B,OAAO,IAAI,CAAC,IAAI,CAAC;IACnB,CAAC;IACD,IAAI,IAAI,CAAC,IAAI,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,CAAC;QAC/B,EAAE,CAAC,MAAM,CAAC,GAAG,IAAI,IAAI,CAAC,CAAC;QACvB,OAAO,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC;IAC5C,CAAC;IAED,IAAI,CAAC;QACH,QAAQ,IAAI,CAAC,OAAO,EAAE,CAAC;YACrB,KAAK,QAAQ;gBACX,EAAE,CAAC,MAAM,CAAC,GAAG,yBAAyB,EAAE,IAAI,CAAC,CAAC;gBAC9C,OAAO,IAAI,CAAC,IAAI,CAAC;YACnB,KAAK,OAAO;gBACV,OAAO,MAAM,KAAK,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;YAC/B;gBACE,MAAM,IAAI,UAAU,CAAC,oBAAoB,IAAI,CAAC,OAAO,GAAG,CAAC,CAAC;QAC9D,CAAC;IACH,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,MAAM,IAAI,GAAG,GAAG,YAAY,UAAU,CAAC,CAAC,CAAC,qCAAqC,CAAC,CAAC,CAAC,EAAE,CAAC;QACpF,EAAE,CAAC,MAAM,CAAC,cAAe,GAAa,CAAC,OAAO,GAAG,IAAI,IAAI,CAAC,CAAC;QAC3D,OAAO,IAAI,CAAC,KAAK,CAAC;IACpB,CAAC;AACH,CAAC;AAED,KAAK,UAAU,KAAK,CAAC,IAAU,EAAE,EAAS;IACxC,MAAM,IAAI,GAAG,CAAC,IAAY,EAAE,EAAE,CAAC,CAAC,IAAI,KAAK,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,SAAS,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC;IACnF,IAAI,MAAc,CAAC;IACnB,IAAI,OAAyB,CAAC;IAE9B,IAAI,IAAI,CAAC,GAAG,EAAE,CAAC;QACb,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,IAAI,IAAI,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC;YAClD,MAAM,IAAI,UAAU,CAAC,+CAA+C,CAAC,CAAC;QACxE,CAAC;QACD,MAAM,GAAG,GAAG,IAAI,CAAC,KAAK,CAAC,MAAM,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,CAA4C,CAAC;QACxF,MAAM,GAAG,MAAM,CAAC,GAAG,CAAC,MAAM,IAAI,EAAE,CAAC,CAAC;QAClC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC;YAAE,MAAM,IAAI,UAAU,CAAC,8BAA8B,CAAC,CAAC;QACtF,OAAO,GAAG,GAAG,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE;YACjC,IAAI,OAAO,CAAC,EAAE,IAAI,KAAK,QAAQ;gBAAE,MAAM,IAAI,UAAU,CAAC,eAAe,CAAC,yBAAyB,CAAC,CAAC;YACjG,OAAO,CAAmB,CAAC;QAC7B,CAAC,CAAC,CAAC;IACL,CAAC;SAAM,CAAC;QACN,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,KAAK,CAAC;YAAE,MAAM,IAAI,UAAU,CAAC,4CAA4C,CAAC,CAAC;QACrG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,MAAM;YAAE,MAAM,IAAI,UAAU,CAAC,+CAA+C,CAAC,CAAC;QAChG,MAAM,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC;QACxC,OAAO,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC;IAC1E,CAAC;IACD,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE;QAAE,MAAM,IAAI,UAAU,CAAC,qBAAqB,CAAC,CAAC;IAEhE,MAAM,UAAU,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,YAAY,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,SAAS,EAAE,KAAK,EAAE,SAAS,EAAE,CAAC;IAChG,MAAM,MAAM,GAAG,MAAM,YAAY,CAAC,EAAE,MAAM,EAAE,OAAO,EAAE,KAAK,EAAE,UAAU,CAAC,KAAK,EAAE,CAAC,CAAC;IAChF,MAAM,IAAI,GAAG,UAAU,CAAC,MAAM,EAAE,MAAM,EAAE,IAAI,CAAC,QAAQ,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,CAAC,QAAQ,EAAE,CAAC,CAAC;IAExG,IAAI,IAAI,CAAC,IAAI,EAAE,CAAC;QACd,EAAE,CAAC,MAAM,CACP,GAAG,IAAI,CAAC,SAAS,CACf;YACE,OAAO,EAAE,IAAI,CAAC,OAAO;YACrB,KAAK,EAAE,UAAU,CAAC,KAAK,IAAI,IAAI;YAC/B,OAAO,EAAE,MAAM,CAAC,OAAO;YACvB,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,OAAO,EAAE,IAAI,CAAC,OAAO;YACrB,QAAQ,EAAE,MAAM,CAAC,QAAQ;YACzB,OAAO,EAAE,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,GAAG,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,EAAE,KAAK,EAAE,CAAC,CAAC,KAAK,EAAE,KAAK,EAAE,CAAC,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC,CAAC;YACpG,oBAAoB,EAAE,IAAI,CAAC,oBAAoB;SAChD,EACD,IAAI,EACJ,CAAC,CACF,IAAI,CACN,CAAC;IACJ,CAAC;SAAM,CAAC;QACN,EAAE,CAAC,MAAM,CAAC,WAAW,CAAC,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,UAAU,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,EAAE,IAAI,CAAC,QAAQ,IAAI,GAAG,CAAC,CAAC,CAAC;IAClG,CAAC;IACD,OAAO,IAAI,CAAC,OAAO,KAAK,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC;AAC3D,CAAC;AAED,KAAK,UAAU,UAAU,CAAC,GAAW,EAAE,EAAS;IAC9C,IAAI,eAAe,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC;QAC9B,OAAO,WAAW,CAAC,GAAG,EAAE;YACtB,KAAK,EAAE,EAAE,CAAC,KAAK;YACf,SAAS,EAAE,aAAa,EAAE,CAAC,OAAO,gDAAgD;SACnF,CAAC,CAAC;IACL,CAAC;IACD,IAAI,SAAS,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC;QACxB,MAAM,IAAI,KAAK,CAAC,GAAG,GAAG,2FAA2F,CAAC,CAAC;IACrH,CAAC;IACD,MAAM,IAAI,GAAG,MAAM,EAAE,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC;IACpC,MAAM,MAAM,GAAG,cAAc,CAAC,IAAI,EAAE,aAAa,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC;IAC1F,OAAO,EAAE,GAAG,MAAM,EAAE,EAAE,EAAE,GAAG,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,IAAI,GAAG,EAAE,CAAC;AAC5D,CAAC;AAED,SAAS,YAAY,CAAC,IAAU,EAAE,EAAS;IACzC,IAAI,EAAE,CAAC,KAAK;QAAE,OAAO,EAAE,KAAK,EAAE,EAAE,CAAC,KAAK,EAAE,KAAK,EAAE,IAAI,CAAC,UAAU,IAAI,QAAQ,EAAE,CAAC;IAC7E,MAAM,MAAM,GAAG,EAAE,CAAC,GAAG,CAAC,uBAAuB,IAAI,EAAE,CAAC,GAAG,CAAC,kBAAkB,CAAC;IAC3E,IAAI,CAAC,MAAM;QAAE,OAAO,EAAE,CAAC;IACvB,MAAM,KAAK,GAAG,IAAI,CAAC,UAAU,IAAI,EAAE,CAAC,GAAG,CAAC,qBAAqB,CAAC;IAC9D,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,MAAM,IAAI,UAAU,CAAC,oFAAoF,CAAC,CAAC;IAC7G,CAAC;IACD,MAAM,OAAO,GACX,IAAI,CAAC,YAAY,IAAI,EAAE,CAAC,GAAG,CAAC,wBAAwB,IAAI,EAAE,CAAC,GAAG,CAAC,kBAAkB,IAAI,8BAA8B,CAAC;IACtH,OAAO;QACL,KAAK;QACL,KAAK,EAAE,IAAI,oBAAoB,CAAC,EAAE,MAAM,EAAE,KAAK,EAAE,OAAO,EAAE,SAAS,EAAE,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC,KAAK,EAAE,CAAC;KAChG,CAAC;AACJ,CAAC;AAED,4EAA4E;AAE5E,SAAS,WAAW,CAClB,MAA0B,EAC1B,IAAgB,EAChB,OAAyB,EACzB,KAAyB,EACzB,KAAc,EACd,QAAgB;IAEhB,MAAM,CAAC,GAAG,CAAC,IAAY,EAAE,CAAS,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,QAAQ,IAAI,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAChF,MAAM,KAAK,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;IACxC,MAAM,GAAG,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;IACtC,MAAM,MAAM,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;IACzC,MAAM,GAAG,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC;IACrC,MAAM,GAAG,GAAa,EAAE,CAAC;IAEzB,GAAG,CAAC,IAAI,CACN,GAAG,CAAC,GAAG,MAAM,CAAC,SAAS,CAAC,MAAM,kBAAkB,OAAO,CAAC,MAAM,uBAAuB,KAAK,IAAI,mBAAmB,EAAE,CAAC,EACpH,EAAE,CACH,CAAC;IAEF,MAAM,MAAM,GAAG,IAAI,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,GAAG,CAAC,CAAC,OAAO,IAAI,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC;IAClF,KAAK,MAAM,GAAG,IAAI,MAAM,CAAC,SAAS,EAAE,CAAC;QACnC,MAAM,GAAG,GAAG,MAAM,CAAC,GAAG,CAAC,GAAG,GAAG,CAAC,OAAO,IAAI,GAAG,CAAC,WAAW,EAAE,CAAC,CAAC;QAC5D,MAAM,IAAI,GAAG,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,KAAK,KAAK,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;QAC5E,MAAM,KAAK,GACT,GAAG,CAAC,SAAS,CAAC,MAAM,KAAK,WAAW;YAClC,CAAC,CAAC,GAAG,CAAC,6BAA6B,GAAG,CAAC,SAAS,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,GAAG,CAAC;YACrE,CAAC,CAAC,GAAG,GAAG,CAAC,SAAS,CAAC,MAAM,KAAK,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,UAAU,IAAI,GAAG,CAAC,SAAS,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC;QACrG,MAAM,MAAM,GAAG,GAAG,CAAC,UAAU;YAC3B,CAAC,CAAC,GAAG,CAAC,UAAU,CAAC,KAAK,KAAK,OAAO;gBAChC,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC;gBACvB,CAAC,CAAC,GAAG,GAAG,CAAC,UAAU,CAAC,KAAK,IAAI,GAAG,CAAC,UAAU,CAAC,UAAU,EAAE,OAAO,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,EAAE;YACnF,CAAC,CAAC,EAAE,CAAC;QACP,GAAG,CAAC,IAAI,CAAC,GAAG,IAAI,IAAI,GAAG,CAAC,OAAO,KAAK,GAAG,CAAC,WAAW,MAAM,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;QACxG,GAAG,CAAC,IAAI,CAAC,OAAO,QAAQ,CAAC,GAAG,CAAC,SAAS,IAAI,iBAAiB,EAAE,GAAG,CAAC,EAAE,CAAC,CAAC;QACrE,IAAI,GAAG,IAAI,GAAG,CAAC,SAAS,CAAC,MAAM,KAAK,WAAW;YAAE,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,kDAAkD,CAAC,CAAC,CAAC;aAC9G,IAAI,GAAG,IAAI,GAAG,CAAC,UAAU,EAAE,OAAO,CAAC,MAAM;YAAE,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,OAAO,GAAG,CAAC,UAAU,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,OAAO,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;aAChJ,IAAI,GAAG,IAAI,GAAG,CAAC,KAAK,KAAK,IAAI,IAAI,GAAG,CAAC,SAAS,CAAC,MAAM,KAAK,WAAW,EAAE,CAAC;YAC3E,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,sBAAsB,GAAG,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,yBAAyB,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;QAC1G,CAAC;IACH,CAAC;IACD,KAAK,MAAM,CAAC,IAAI,IAAI,CAAC,OAAO;QAAE,GAAG,CAAC,IAAI,CAAC,GAAG,MAAM,CAAC,GAAG,CAAC,aAAa,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,CAAC,CAAC;IACtF,KAAK,MAAM,CAAC,IAAI,MAAM,CAAC,QAAQ;QAAE,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,EAAE,CAAC,CAAC,CAAC;IAEhE,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACb,IAAI,IAAI,CAAC,OAAO,KAAK,MAAM,EAAE,CAAC;QAC5B,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,qCAAqC,CAAC,CAAC,CAAC;IAC5F,CAAC;SAAM,CAAC;QACN,MAAM,KAAK,GAAG;YACZ,IAAI,CAAC,QAAQ,CAAC,MAAM,IAAI,GAAG,IAAI,CAAC,QAAQ,CAAC,MAAM,qBAAqB;YACpE,IAAI,CAAC,OAAO,CAAC,MAAM,IAAI,GAAG,IAAI,CAAC,OAAO,CAAC,MAAM,sBAAsB;SACpE,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;QAClB,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,YAAY,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;IAChD,CAAC;IACD,IAAI,CAAC,KAAK;QAAE,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,gGAAgG,CAAC,CAAC,CAAC;IAC5H,OAAO,GAAG,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC;AAC/B,CAAC;AAED,SAAS,QAAQ,CAAC,CAAS,EAAE,CAAS;IACpC,MAAM,IAAI,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3C,OAAO,IAAI,CAAC,MAAM,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC;AAC9D,CAAC"}
|
|
1
|
+
{"version":3,"file":"main.js","sourceRoot":"","sources":["../../src/cli/main.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAGH,OAAO,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAC5C,OAAO,EAAE,UAAU,EAAmB,MAAM,YAAY,CAAC;AACzD,OAAO,EAAE,yBAAyB,EAAE,MAAM,uBAAuB,CAAC;AAClE,OAAO,EAAE,oBAAoB,EAAE,MAAM,wBAAwB,CAAC;AAC9D,OAAO,EAAE,WAAW,EAAE,cAAc,EAAE,MAAM,oBAAoB,CAAC;AAiBjE,MAAM,CAAC,MAAM,IAAI,GAAG,EAAE,IAAI,EAAE,CAAC,EAAE,KAAK,EAAE,CAAC,EAAE,MAAM,EAAE,CAAC,EAAW,CAAC;AAE9D,MAAM,IAAI,GAAG;;;;;;;;;;;;;;;;;;;;;;;;;;;2EA2B8D,CAAC;AAgB5E,MAAM,UAAW,SAAQ,KAAK;CAAG;AAEjC,SAAS,SAAS,CAAC,IAAc;IAC/B,MAAM,CAAC,GAAS,EAAE,UAAU,EAAE,EAAE,EAAE,OAAO,EAAE,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,KAAK,EAAE,OAAO,EAAE,KAAK,EAAE,CAAC;IACvG,MAAM,KAAK,GAAG,CAAC,CAAS,EAAE,IAAY,EAAU,EAAE;QAChD,MAAM,CAAC,GAAG,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;QACtB,IAAI,CAAC,KAAK,SAAS;YAAE,MAAM,IAAI,UAAU,CAAC,GAAG,IAAI,gBAAgB,CAAC,CAAC;QACnE,OAAO,CAAC,CAAC;IACX,CAAC,CAAC;IACF,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACrC,MAAM,GAAG,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QACpB,QAAQ,GAAG,EAAE,CAAC;YACZ,KAAK,IAAI,CAAC;YACV,KAAK,UAAU;gBACb,CAAC,CAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,GAAG,CAAC,CAAC,CAAC;gBAChC,MAAM;YACR,KAAK,OAAO;gBACV,CAAC,CAAC,GAAG,GAAG,KAAK,CAAC,CAAC,EAAE,EAAE,GAAG,CAAC,CAAC;gBACxB,MAAM;YACR,KAAK,QAAQ;gBACX,CAAC,CAAC,IAAI,GAAG,IAAI,CAAC;gBACd,MAAM;YACR,KAAK,YAAY;gBACf,CAAC,CAAC,KAAK,GAAG,KAAK,CAAC;gBAChB,MAAM;YACR,KAAK,aAAa,CAAC,CAAC,CAAC;gBACnB,MAAM,CAAC,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,GAAG,CAAC,CAAC,CAAC;gBAClC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;oBAAE,MAAM,IAAI,UAAU,CAAC,qCAAqC,CAAC,CAAC;gBACrF,CAAC,CAAC,QAAQ,GAAG,CAAC,CAAC;gBACf,MAAM;YACR,CAAC;YACD,KAAK,eAAe;gBAClB,CAAC,CAAC,UAAU,GAAG,KAAK,CAAC,CAAC,EAAE,EAAE,GAAG,CAAC,CAAC;gBAC/B,MAAM;YACR,KAAK,kBAAkB;gBACrB,CAAC,CAAC,YAAY,GAAG,KAAK,CAAC,CAAC,EAAE,EAAE,GAAG,CAAC,CAAC;gBACjC,MAAM;YACR,KAAK,IAAI,CAAC;YACV,KAAK,QAAQ;gBACX,CAAC,CAAC,IAAI,GAAG,IAAI,CAAC;gBACd,MAAM;YACR,KAAK,IAAI,CAAC;YACV,KAAK,WAAW;gBACd,CAAC,CAAC,OAAO,GAAG,IAAI,CAAC;gBACjB,MAAM;YACR;gBACE,IAAI,GAAG,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,GAAG,KAAK,GAAG;oBAAE,MAAM,IAAI,UAAU,CAAC,kBAAkB,GAAG,EAAE,CAAC,CAAC;gBACtF,IAAI,CAAC,CAAC,OAAO,KAAK,SAAS;oBAAE,CAAC,CAAC,OAAO,GAAG,GAAG,CAAC;;oBACxC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QAChC,CAAC;IACH,CAAC;IACD,OAAO,CAAC,CAAC;AACX,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,IAAI,CAAC,EAAS;IAClC,IAAI,IAAU,CAAC;IACf,IAAI,CAAC;QACH,IAAI,GAAG,SAAS,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC;IAC5B,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,EAAE,CAAC,MAAM,CAAC,cAAe,GAAa,CAAC,OAAO,uCAAuC,CAAC,CAAC;QACvF,OAAO,IAAI,CAAC,KAAK,CAAC;IACpB,CAAC;IACD,IAAI,IAAI,CAAC,OAAO,EAAE,CAAC;QACjB,EAAE,CAAC,MAAM,CAAC,GAAG,EAAE,CAAC,OAAO,IAAI,CAAC,CAAC;QAC7B,OAAO,IAAI,CAAC,IAAI,CAAC;IACnB,CAAC;IACD,IAAI,IAAI,CAAC,IAAI,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,CAAC;QAC/B,EAAE,CAAC,MAAM,CAAC,GAAG,IAAI,IAAI,CAAC,CAAC;QACvB,OAAO,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC;IAC5C,CAAC;IAED,IAAI,CAAC;QACH,QAAQ,IAAI,CAAC,OAAO,EAAE,CAAC;YACrB,KAAK,QAAQ;gBACX,EAAE,CAAC,MAAM,CAAC,GAAG,yBAAyB,EAAE,IAAI,CAAC,CAAC;gBAC9C,OAAO,IAAI,CAAC,IAAI,CAAC;YACnB,KAAK,OAAO;gBACV,OAAO,MAAM,KAAK,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;YAC/B,KAAK,QAAQ,CAAC,CAAC,CAAC;gBACd,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,KAAK,CAAC;oBAAE,MAAM,IAAI,UAAU,CAAC,sCAAsC,CAAC,CAAC;gBAC/F,MAAM,GAAG,GAAG,MAAM,UAAU,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;gBACrD,EAAE,CAAC,MAAM,CAAC,GAAG,GAAG,CAAC,IAAI,IAAI,CAAC,CAAC;gBAC3B,OAAO,IAAI,CAAC,IAAI,CAAC;YACnB,CAAC;YACD;gBACE,MAAM,IAAI,UAAU,CAAC,oBAAoB,IAAI,CAAC,OAAO,GAAG,CAAC,CAAC;QAC9D,CAAC;IACH,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,MAAM,IAAI,GAAG,GAAG,YAAY,UAAU,CAAC,CAAC,CAAC,qCAAqC,CAAC,CAAC,CAAC,EAAE,CAAC;QACpF,EAAE,CAAC,MAAM,CAAC,cAAe,GAAa,CAAC,OAAO,GAAG,IAAI,IAAI,CAAC,CAAC;QAC3D,OAAO,IAAI,CAAC,KAAK,CAAC;IACpB,CAAC;AACH,CAAC;AAED,KAAK,UAAU,KAAK,CAAC,IAAU,EAAE,EAAS;IACxC,MAAM,IAAI,GAAG,CAAC,IAAY,EAAE,EAAE,CAAC,CAAC,IAAI,KAAK,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,SAAS,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC;IACnF,IAAI,MAAc,CAAC;IACnB,IAAI,OAAyB,CAAC;IAE9B,IAAI,IAAI,CAAC,GAAG,EAAE,CAAC;QACb,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,IAAI,IAAI,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC;YAClD,MAAM,IAAI,UAAU,CAAC,+CAA+C,CAAC,CAAC;QACxE,CAAC;QACD,MAAM,GAAG,GAAG,IAAI,CAAC,KAAK,CAAC,MAAM,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,CAA4C,CAAC;QACxF,MAAM,GAAG,MAAM,CAAC,GAAG,CAAC,MAAM,IAAI,EAAE,CAAC,CAAC;QAClC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC;YAAE,MAAM,IAAI,UAAU,CAAC,8BAA8B,CAAC,CAAC;QACtF,OAAO,GAAG,GAAG,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE;YACjC,IAAI,OAAO,CAAC,EAAE,IAAI,KAAK,QAAQ;gBAAE,MAAM,IAAI,UAAU,CAAC,eAAe,CAAC,yBAAyB,CAAC,CAAC;YACjG,OAAO,CAAmB,CAAC;QAC7B,CAAC,CAAC,CAAC;IACL,CAAC;SAAM,CAAC;QACN,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,KAAK,CAAC;YAAE,MAAM,IAAI,UAAU,CAAC,4CAA4C,CAAC,CAAC;QACrG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,MAAM;YAAE,MAAM,IAAI,UAAU,CAAC,+CAA+C,CAAC,CAAC;QAChG,MAAM,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC;QACxC,OAAO,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC;IAC1E,CAAC;IACD,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE;QAAE,MAAM,IAAI,UAAU,CAAC,qBAAqB,CAAC,CAAC;IAEhE,MAAM,UAAU,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,YAAY,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,SAAS,EAAE,KAAK,EAAE,SAAS,EAAE,CAAC;IAChG,MAAM,MAAM,GAAG,MAAM,YAAY,CAAC,EAAE,MAAM,EAAE,OAAO,EAAE,KAAK,EAAE,UAAU,CAAC,KAAK,EAAE,CAAC,CAAC;IAChF,MAAM,IAAI,GAAG,UAAU,CAAC,MAAM,EAAE,MAAM,EAAE,IAAI,CAAC,QAAQ,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,CAAC,QAAQ,EAAE,CAAC,CAAC;IAExG,IAAI,IAAI,CAAC,IAAI,EAAE,CAAC;QACd,EAAE,CAAC,MAAM,CACP,GAAG,IAAI,CAAC,SAAS,CACf;YACE,OAAO,EAAE,IAAI,CAAC,OAAO;YACrB,KAAK,EAAE,UAAU,CAAC,KAAK,IAAI,IAAI;YAC/B,iGAAiG;YACjG,WAAW,EAAE,MAAM,CAAC,SAAS;YAC7B,OAAO,EAAE,MAAM,CAAC,OAAO;YACvB,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,OAAO,EAAE,IAAI,CAAC,OAAO;YACrB,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,SAAS,EAAE,MAAM,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;gBACtC,OAAO,EAAE,CAAC,CAAC,OAAO;gBAClB,WAAW,EAAE,CAAC,CAAC,WAAW;gBAC1B,KAAK,EAAE,EAAE,MAAM,EAAE,CAAC,CAAC,SAAS,CAAC,MAAM,EAAE,KAAK,EAAE,MAAM,CAAC,CAAC,CAAC,SAAS,CAAC,KAAK,CAAC,EAAE;gBACvE,KAAK,EAAE,CAAC,CAAC,UAAU;oBACjB,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,UAAU,CAAC,KAAK,EAAE,OAAO,EAAE,CAAC,CAAC,UAAU,CAAC,UAAU,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,UAAU,CAAC,UAAU,CAAC,EAAE;oBACnH,CAAC,CAAC,IAAI;gBACR,KAAK,EAAE,CAAC,CAAC,KAAK,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,KAAK,CAAC;aACjD,CAAC,CAAC;YACH,QAAQ,EAAE,MAAM,CAAC,QAAQ;YACzB,OAAO,EAAE,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,GAAG,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,EAAE,KAAK,EAAE,CAAC,CAAC,KAAK,EAAE,KAAK,EAAE,CAAC,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC,CAAC;YACpG,oBAAoB,EAAE,IAAI,CAAC,oBAAoB;SAChD,EACD,IAAI,EACJ,CAAC,CACF,IAAI,CACN,CAAC;IACJ,CAAC;SAAM,CAAC;QACN,EAAE,CAAC,MAAM,CAAC,WAAW,CAAC,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,UAAU,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,EAAE,IAAI,CAAC,QAAQ,IAAI,GAAG,CAAC,CAAC,CAAC;IAClG,CAAC;IACD,OAAO,IAAI,CAAC,OAAO,KAAK,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,KAAK,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC;AACpG,CAAC;AAED,MAAM,MAAM,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,GAAG,CAAC,GAAG,GAAG,CAAC;AAExD,KAAK,UAAU,UAAU,CAAC,GAAW,EAAE,EAAS;IAC9C,IAAI,eAAe,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC;QAC9B,OAAO,WAAW,CAAC,GAAG,EAAE;YACtB,KAAK,EAAE,EAAE,CAAC,KAAK;YACf,SAAS,EAAE,aAAa,EAAE,CAAC,OAAO,gDAAgD;SACnF,CAAC,CAAC;IACL,CAAC;IACD,IAAI,SAAS,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC;QACxB,MAAM,IAAI,KAAK,CAAC,GAAG,GAAG,2FAA2F,CAAC,CAAC;IACrH,CAAC;IACD,MAAM,IAAI,GAAG,MAAM,EAAE,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC;IACpC,MAAM,MAAM,GAAG,cAAc,CAAC,IAAI,EAAE,aAAa,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC;IAC1F,OAAO,EAAE,GAAG,MAAM,EAAE,EAAE,EAAE,GAAG,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,IAAI,GAAG,EAAE,CAAC;AAC5D,CAAC;AAED,SAAS,YAAY,CAAC,IAAU,EAAE,EAAS;IACzC,IAAI,EAAE,CAAC,KAAK;QAAE,OAAO,EAAE,KAAK,EAAE,EAAE,CAAC,KAAK,EAAE,KAAK,EAAE,IAAI,CAAC,UAAU,IAAI,QAAQ,EAAE,CAAC;IAC7E,MAAM,MAAM,GAAG,EAAE,CAAC,GAAG,CAAC,uBAAuB,IAAI,EAAE,CAAC,GAAG,CAAC,kBAAkB,CAAC;IAC3E,IAAI,CAAC,MAAM;QAAE,OAAO,EAAE,CAAC;IACvB,MAAM,KAAK,GAAG,IAAI,CAAC,UAAU,IAAI,EAAE,CAAC,GAAG,CAAC,qBAAqB,CAAC;IAC9D,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,MAAM,IAAI,UAAU,CAAC,oFAAoF,CAAC,CAAC;IAC7G,CAAC;IACD,MAAM,OAAO,GACX,IAAI,CAAC,YAAY,IAAI,EAAE,CAAC,GAAG,CAAC,wBAAwB,IAAI,EAAE,CAAC,GAAG,CAAC,kBAAkB,IAAI,8BAA8B,CAAC;IACtH,OAAO;QACL,KAAK;QACL,KAAK,EAAE,IAAI,oBAAoB,CAAC,EAAE,MAAM,EAAE,KAAK,EAAE,OAAO,EAAE,SAAS,EAAE,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC,KAAK,EAAE,CAAC;KAChG,CAAC;AACJ,CAAC;AAED,4EAA4E;AAE5E,SAAS,WAAW,CAClB,MAA0B,EAC1B,IAAgB,EAChB,OAAyB,EACzB,KAAyB,EACzB,KAAc,EACd,QAAgB;IAEhB,MAAM,CAAC,GAAG,CAAC,IAAY,EAAE,CAAS,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,QAAQ,IAAI,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAChF,MAAM,KAAK,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;IACxC,MAAM,GAAG,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;IACtC,MAAM,MAAM,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;IACzC,MAAM,GAAG,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC;IACrC,MAAM,GAAG,GAAa,EAAE,CAAC;IAEzB,GAAG,CAAC,IAAI,CACN,GAAG,CAAC,GAAG,MAAM,CAAC,SAAS,CAAC,MAAM,kBAAkB,OAAO,CAAC,MAAM,uBAAuB,KAAK,IAAI,mBAAmB,EAAE,CAAC,EACpH,EAAE,CACH,CAAC;IAEF,MAAM,MAAM,GAAG,IAAI,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,GAAG,CAAC,CAAC,OAAO,IAAI,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC;IAClF,KAAK,MAAM,GAAG,IAAI,MAAM,CAAC,SAAS,EAAE,CAAC;QACnC,MAAM,GAAG,GAAG,MAAM,CAAC,GAAG,CAAC,GAAG,GAAG,CAAC,OAAO,IAAI,GAAG,CAAC,WAAW,EAAE,CAAC,CAAC;QAC5D,MAAM,IAAI,GAAG,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,KAAK,KAAK,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;QAC5E,MAAM,KAAK,GACT,GAAG,CAAC,SAAS,CAAC,MAAM,KAAK,WAAW;YAClC,CAAC,CAAC,GAAG,CAAC,6BAA6B,GAAG,CAAC,SAAS,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,GAAG,CAAC;YACrE,CAAC,CAAC,GAAG,GAAG,CAAC,SAAS,CAAC,MAAM,KAAK,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,UAAU,IAAI,GAAG,CAAC,SAAS,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC;QACrG,MAAM,MAAM,GAAG,GAAG,CAAC,UAAU;YAC3B,CAAC,CAAC,GAAG,CAAC,UAAU,CAAC,KAAK,KAAK,OAAO;gBAChC,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC;gBACvB,CAAC,CAAC,GAAG,GAAG,CAAC,UAAU,CAAC,KAAK,IAAI,GAAG,CAAC,UAAU,CAAC,UAAU,EAAE,OAAO,CAAC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,EAAE;YACnF,CAAC,CAAC,EAAE,CAAC;QACP,GAAG,CAAC,IAAI,CAAC,GAAG,IAAI,IAAI,GAAG,CAAC,OAAO,KAAK,GAAG,CAAC,WAAW,MAAM,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;QACxG,GAAG,CAAC,IAAI,CAAC,OAAO,QAAQ,CAAC,GAAG,CAAC,SAAS,IAAI,iBAAiB,EAAE,GAAG,CAAC,EAAE,CAAC,CAAC;QACrE,IAAI,GAAG,IAAI,GAAG,CAAC,SAAS,CAAC,MAAM,KAAK,WAAW;YAAE,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,kDAAkD,CAAC,CAAC,CAAC;aAC9G,IAAI,GAAG,IAAI,GAAG,CAAC,UAAU,EAAE,OAAO,CAAC,MAAM;YAAE,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,OAAO,GAAG,CAAC,UAAU,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,OAAO,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;aAChJ,IAAI,GAAG,IAAI,GAAG,CAAC,KAAK,KAAK,IAAI,IAAI,GAAG,CAAC,SAAS,CAAC,MAAM,KAAK,WAAW,EAAE,CAAC;YAC3E,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,sBAAsB,GAAG,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,yBAAyB,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;QAC1G,CAAC;IACH,CAAC;IACD,KAAK,MAAM,CAAC,IAAI,IAAI,CAAC,OAAO;QAAE,GAAG,CAAC,IAAI,CAAC,GAAG,MAAM,CAAC,GAAG,CAAC,aAAa,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,CAAC,CAAC;IACtF,KAAK,MAAM,CAAC,IAAI,MAAM,CAAC,QAAQ;QAAE,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,EAAE,CAAC,CAAC,CAAC;IAEhE,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACb,IAAI,IAAI,CAAC,OAAO,KAAK,MAAM,EAAE,CAAC;QAC5B,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,oCAAoC,CAAC,CAAC,CAAC;IAC3F,CAAC;SAAM,IAAI,IAAI,CAAC,OAAO,KAAK,YAAY,EAAE,CAAC;QACzC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,mCAAmC,IAAI,CAAC,QAAQ,CAAC,MAAM,sEAAsE,CAAC,CAAC,CAAC;IAClJ,CAAC;SAAM,CAAC;QACN,MAAM,KAAK,GAAG;YACZ,IAAI,CAAC,QAAQ,CAAC,MAAM,IAAI,GAAG,IAAI,CAAC,QAAQ,CAAC,MAAM,qBAAqB;YACpE,IAAI,CAAC,OAAO,CAAC,MAAM,IAAI,GAAG,IAAI,CAAC,OAAO,CAAC,MAAM,sBAAsB;SACpE,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;QAClB,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,WAAW,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;QAC7C,IAAI,IAAI,CAAC,QAAQ,CAAC,MAAM;YAAE,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,4BAA4B,IAAI,CAAC,QAAQ,CAAC,MAAM,eAAe,CAAC,CAAC,CAAC;IAC9G,CAAC;IACD,IAAI,CAAC,KAAK;QAAE,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,gGAAgG,CAAC,CAAC,CAAC;IAC5H,OAAO,GAAG,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC;AAC/B,CAAC;AAED,SAAS,QAAQ,CAAC,CAAS,EAAE,CAAS;IACpC,MAAM,IAAI,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC;IAC3C,OAAO,IAAI,CAAC,MAAM,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC;AAC9D,CAAC"}
|
package/dist/gate.d.ts
CHANGED
|
@@ -30,11 +30,21 @@ export interface GateProblem {
|
|
|
30
30
|
score: number | null;
|
|
31
31
|
}
|
|
32
32
|
export interface GateResult {
|
|
33
|
-
|
|
33
|
+
/**
|
|
34
|
+
* `unverified`: nothing failed, but the judge could not rate some citations,
|
|
35
|
+
* so support was not checked for them. Never report that as a pass.
|
|
36
|
+
*/
|
|
37
|
+
verdict: 'pass' | 'revise' | 'unverified';
|
|
34
38
|
problems: GateProblem[];
|
|
35
39
|
/** Factual-looking sentences without any `[n]` marker. Heuristic. */
|
|
36
40
|
uncited: string[];
|
|
37
|
-
/**
|
|
41
|
+
/** Citations the judge failed on (only when a judge ran). */
|
|
42
|
+
unjudged: {
|
|
43
|
+
claimId: string;
|
|
44
|
+
sourceIndex: number;
|
|
45
|
+
reason: string;
|
|
46
|
+
}[];
|
|
47
|
+
/** Ready-to-send correction prompt; `null` unless the verdict is `revise`. */
|
|
38
48
|
instructionsForModel: string | null;
|
|
39
49
|
}
|
|
40
50
|
/** Apply the gate to a report produced by `verifyAnswer` for `answer`. */
|
package/dist/gate.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"gate.d.ts","sourceRoot":"","sources":["../src/gate.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,KAAK,EAAwB,eAAe,EAAE,kBAAkB,EAAE,MAAM,YAAY,CAAC;AAG5F,MAAM,WAAW,cAAc;IAC7B,oDAAoD;IACpD,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,mEAAmE;IACnE,eAAe,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,MAAM,WAAW,GACnB,qBAAqB,GACrB,wBAAwB,GACxB,YAAY,GACZ,kBAAkB,CAAC;AAEvB,MAAM,WAAW,WAAW;IAC1B,OAAO,EAAE,MAAM,CAAC;IAChB,WAAW,EAAE,MAAM,CAAC;IACpB,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,EAAE,MAAM,CAAC;IACd,IAAI,EAAE,WAAW,CAAC;IAClB,SAAS,EAAE;QAAE,MAAM,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAA;KAAE,CAAC;IAC7C,UAAU,EAAE;QAAE,KAAK,EAAE,eAAe,GAAG,OAAO,CAAC;QAAC,UAAU,EAAE,MAAM,GAAG,IAAI,CAAC;QAAC,OAAO,EAAE,MAAM,EAAE,CAAA;KAAE,GAAG,IAAI,CAAC;IACtG,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;CACtB;AAED,MAAM,WAAW,UAAU;IACzB,OAAO,EAAE,MAAM,GAAG,QAAQ,CAAC;
|
|
1
|
+
{"version":3,"file":"gate.d.ts","sourceRoot":"","sources":["../src/gate.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,KAAK,EAAwB,eAAe,EAAE,kBAAkB,EAAE,MAAM,YAAY,CAAC;AAG5F,MAAM,WAAW,cAAc;IAC7B,oDAAoD;IACpD,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,mEAAmE;IACnE,eAAe,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,MAAM,WAAW,GACnB,qBAAqB,GACrB,wBAAwB,GACxB,YAAY,GACZ,kBAAkB,CAAC;AAEvB,MAAM,WAAW,WAAW;IAC1B,OAAO,EAAE,MAAM,CAAC;IAChB,WAAW,EAAE,MAAM,CAAC;IACpB,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,EAAE,MAAM,CAAC;IACd,IAAI,EAAE,WAAW,CAAC;IAClB,SAAS,EAAE;QAAE,MAAM,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAA;KAAE,CAAC;IAC7C,UAAU,EAAE;QAAE,KAAK,EAAE,eAAe,GAAG,OAAO,CAAC;QAAC,UAAU,EAAE,MAAM,GAAG,IAAI,CAAC;QAAC,OAAO,EAAE,MAAM,EAAE,CAAA;KAAE,GAAG,IAAI,CAAC;IACtG,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;CACtB;AAED,MAAM,WAAW,UAAU;IACzB;;;OAGG;IACH,OAAO,EAAE,MAAM,GAAG,QAAQ,GAAG,YAAY,CAAC;IAC1C,QAAQ,EAAE,WAAW,EAAE,CAAC;IACxB,qEAAqE;IACrE,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,6DAA6D;IAC7D,QAAQ,EAAE;QAAE,OAAO,EAAE,MAAM,CAAC;QAAC,WAAW,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC;IACrE,8EAA8E;IAC9E,oBAAoB,EAAE,MAAM,GAAG,IAAI,CAAC;CACrC;AAID,0EAA0E;AAC1E,wBAAgB,UAAU,CACxB,MAAM,EAAE,kBAAkB,EAC1B,MAAM,EAAE,MAAM,EACd,UAAU,GAAE,cAAmB,GAC9B,UAAU,CAgCZ"}
|
package/dist/gate.js
CHANGED
|
@@ -28,12 +28,16 @@ export function gateReport(report, answer, thresholds = {}) {
|
|
|
28
28
|
score: c.score === null ? null : round(c.score),
|
|
29
29
|
}));
|
|
30
30
|
const uncited = findUncitedSentences(stripEvi1Appendix(answer), t.minUncitedChars);
|
|
31
|
-
const
|
|
31
|
+
const unjudged = report.citations
|
|
32
|
+
.filter((c) => c.entailment?.class === 'error')
|
|
33
|
+
.map((c) => ({ claimId: c.claimId, sourceIndex: c.sourceIndex, reason: c.entailment?.reasons.join('; ') || 'judge error' }));
|
|
34
|
+
const verdict = problems.length || uncited.length ? 'revise' : unjudged.length ? 'unverified' : 'pass';
|
|
32
35
|
return {
|
|
33
36
|
verdict,
|
|
34
37
|
problems,
|
|
35
38
|
uncited,
|
|
36
|
-
|
|
39
|
+
unjudged,
|
|
40
|
+
instructionsForModel: verdict === 'revise' ? correctionInstructions(problems, uncited) : null,
|
|
37
41
|
};
|
|
38
42
|
}
|
|
39
43
|
function isProblem(c, minScore) {
|
package/dist/gate.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"gate.js","sourceRoot":"","sources":["../src/gate.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAGH,OAAO,EAAE,iBAAiB,EAAE,MAAM,oBAAoB,CAAC;
|
|
1
|
+
{"version":3,"file":"gate.js","sourceRoot":"","sources":["../src/gate.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAGH,OAAO,EAAE,iBAAiB,EAAE,MAAM,oBAAoB,CAAC;AAyCvD,MAAM,QAAQ,GAA6B,EAAE,QAAQ,EAAE,GAAG,EAAE,eAAe,EAAE,EAAE,EAAE,CAAC;AAElF,0EAA0E;AAC1E,MAAM,UAAU,UAAU,CACxB,MAA0B,EAC1B,MAAc,EACd,aAA6B,EAAE;IAE/B,MAAM,CAAC,GAAG,EAAE,GAAG,QAAQ,EAAE,GAAG,UAAU,EAAE,CAAC;IACzC,MAAM,QAAQ,GAAG,MAAM,CAAC,SAAS;SAC9B,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,SAAS,CAAC,CAAC,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC;SACvC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;QACX,OAAO,EAAE,CAAC,CAAC,OAAO;QAClB,WAAW,EAAE,CAAC,CAAC,WAAW;QAC1B,SAAS,EAAE,CAAC,CAAC,SAAS;QACtB,KAAK,EAAE,CAAC,CAAC,KAAK;QACd,IAAI,EAAE,WAAW,CAAC,CAAC,CAAC;QACpB,SAAS,EAAE,EAAE,MAAM,EAAE,CAAC,CAAC,SAAS,CAAC,MAAM,EAAE,KAAK,EAAE,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC,KAAK,CAAC,EAAE;QAC1E,UAAU,EAAE,CAAC,CAAC,UAAU;YACtB,CAAC,CAAC;gBACE,KAAK,EAAE,CAAC,CAAC,UAAU,CAAC,KAAK;gBACzB,UAAU,EAAE,CAAC,CAAC,UAAU,CAAC,UAAU,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,UAAU,CAAC,UAAU,CAAC;gBACpF,OAAO,EAAE,CAAC,CAAC,UAAU,CAAC,OAAO;aAC9B;YACH,CAAC,CAAC,IAAI;QACR,KAAK,EAAE,CAAC,CAAC,KAAK,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC;KAChD,CAAC,CAAC,CAAC;IACN,MAAM,OAAO,GAAG,oBAAoB,CAAC,iBAAiB,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,eAAe,CAAC,CAAC;IACnF,MAAM,QAAQ,GAAG,MAAM,CAAC,SAAS;SAC9B,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,UAAU,EAAE,KAAK,KAAK,OAAO,CAAC;SAC9C,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,CAAC,OAAO,EAAE,WAAW,EAAE,CAAC,CAAC,WAAW,EAAE,MAAM,EAAE,CAAC,CAAC,UAAU,EAAE,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,aAAa,EAAE,CAAC,CAAC,CAAC;IAC/H,MAAM,OAAO,GAAG,QAAQ,CAAC,MAAM,IAAI,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,CAAC;IACvG,OAAO;QACL,OAAO;QACP,QAAQ;QACR,OAAO;QACP,QAAQ;QACR,oBAAoB,EAAE,OAAO,KAAK,QAAQ,CAAC,CAAC,CAAC,sBAAsB,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC,IAAI;KAC9F,CAAC;AACJ,CAAC;AAED,SAAS,SAAS,CAAC,CAAuB,EAAE,QAAgB;IAC1D,IAAI,CAAC,CAAC,SAAS,CAAC,MAAM,KAAK,WAAW;QAAE,OAAO,IAAI,CAAC;IACpD,MAAM,GAAG,GAAG,CAAC,CAAC,UAAU,EAAE,KAAK,CAAC;IAChC,IAAI,GAAG,KAAK,cAAc,IAAI,GAAG,KAAK,YAAY;QAAE,OAAO,IAAI,CAAC;IAChE,qFAAqF;IACrF,OAAO,CAAC,CAAC,KAAK,KAAK,IAAI,IAAI,CAAC,CAAC,KAAK,GAAG,QAAQ,CAAC;AAChD,CAAC;AAED,SAAS,WAAW,CAAC,CAAuB;IAC1C,IAAI,CAAC,CAAC,SAAS,CAAC,MAAM,KAAK,WAAW;QAAE,OAAO,qBAAqB,CAAC;IACrE,IAAI,CAAC,CAAC,UAAU,EAAE,KAAK,KAAK,cAAc;QAAE,OAAO,wBAAwB,CAAC;IAC5E,IAAI,CAAC,CAAC,UAAU,EAAE,KAAK,KAAK,YAAY;QAAE,OAAO,YAAY,CAAC;IAC9D,OAAO,kBAAkB,CAAC;AAC5B,CAAC;AAED;;;;GAIG;AACH,SAAS,oBAAoB,CAAC,IAAY,EAAE,QAAgB;IAC1D,MAAM,GAAG,GAAa,EAAE,CAAC;IACzB,KAAK,MAAM,OAAO,IAAI,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC;QACvC,MAAM,IAAI,GAAG,OAAO,CAAC,IAAI,EAAE,CAAC;QAC5B,IAAI,CAAC,IAAI,IAAI,WAAW,CAAC,IAAI,CAAC,IAAI,CAAC;YAAE,SAAS;QAC9C,KAAK,MAAM,QAAQ,IAAI,cAAc,CAAC,IAAI,CAAC,EAAE,CAAC;YAC5C,MAAM,CAAC,GAAG,QAAQ,CAAC,IAAI,EAAE,CAAC;YAC1B,IAAI,CAAC,CAAC,MAAM,GAAG,QAAQ,IAAI,CAAC,CAAC,QAAQ,CAAC,GAAG,CAAC;gBAAE,SAAS;YACrD,IAAI,SAAS,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,YAAY,CAAC,IAAI,CAAC,CAAC,CAAC;gBAAE,SAAS;YACzD,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC;QAC7B,CAAC;IACH,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,qFAAqF;AACrF,MAAM,YAAY,GAAG,uHAAuH,CAAC;AAE7I,SAAS,cAAc,CAAC,IAAY;IAClC,MAAM,GAAG,GAAa,EAAE,CAAC;IACzB,KAAK,MAAM,KAAK,IAAI,IAAI,CAAC,KAAK,CAAC,mCAAmC,CAAC,EAAE,CAAC;QACpE,IAAI,GAAG,CAAC,MAAM,IAAI,YAAY,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;YAAE,GAAG,CAAC,GAAG,CAAC,MAAM,GAAG,CAAC,CAAC,IAAI,IAAI,KAAK,EAAE,CAAC;;YACxF,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IACvB,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,MAAM,GAAG,GAAgC;IACvC,mBAAmB,EACjB,mHAAmH;IACrH,sBAAsB,EAAE,8FAA8F;IACtH,UAAU,EAAE,yGAAyG;IACrH,gBAAgB,EAAE,mFAAmF;CACtG,CAAC;AAEF,SAAS,sBAAsB,CAAC,QAAuB,EAAE,OAAiB;IACxE,MAAM,KAAK,GAAG;QACZ,oIAAoI;KACrI,CAAC;IACF,KAAK,MAAM,CAAC,IAAI,QAAQ,EAAE,CAAC;QACzB,KAAK,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,OAAO,MAAM,QAAQ,CAAC,CAAC,CAAC,SAAS,EAAE,GAAG,CAAC,OAAO,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IACxF,CAAC;IACD,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;QACxB,KAAK,CAAC,IAAI,CAAC,iCAAiC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,yDAAyD,CAAC,CAAC;IACzH,CAAC;IACD,KAAK,CAAC,IAAI,CAAC,0EAA0E,CAAC,CAAC;IACvF,OAAO,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC1B,CAAC;AAED,SAAS,QAAQ,CAAC,CAAS,EAAE,CAAS;IACpC,OAAO,CAAC,CAAC,MAAM,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC;AACrD,CAAC;AAED,SAAS,KAAK,CAAC,CAAS;IACtB,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,GAAG,CAAC,GAAG,GAAG,CAAC;AACnC,CAAC"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"html.d.ts","sourceRoot":"","sources":["../../src/source/html.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;
|
|
1
|
+
{"version":3,"file":"html.d.ts","sourceRoot":"","sources":["../../src/source/html.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AA0BH,mFAAmF;AACnF,wBAAgB,cAAc,CAAC,CAAC,EAAE,MAAM,GAAG,MAAM,CAQhD;AAED,0CAA0C;AAC1C,wBAAgB,SAAS,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAI1D;AAED;;;GAGG;AACH,wBAAgB,eAAe,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAMhE;AAED,mEAAmE;AACnE,wBAAgB,UAAU,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,CAc/C"}
|
package/dist/source/html.js
CHANGED
|
@@ -6,6 +6,8 @@
|
|
|
6
6
|
*/
|
|
7
7
|
const DROP_ELEMENTS = /<(script|style|noscript|template|svg|iframe|head|object|canvas)\b[\s\S]*?<\/\1\s*>/gi;
|
|
8
8
|
const COMMENTS = /<!--[\s\S]*?-->/g;
|
|
9
|
+
/** Reference markers such as Wikipedia's [12]: models quoting the page leave them out. */
|
|
10
|
+
const REF_MARKERS = /<sup\b[^>]*class="[^"]*\breference\b[^"]*"[^>]*>[\s\S]*?<\/sup\s*>/gi;
|
|
9
11
|
const BLOCK_TAGS = /<\/?(?:p|div|br|hr|li|ul|ol|dl|dt|dd|h[1-6]|tr|table|thead|tbody|tfoot|section|article|aside|main|header|footer|nav|blockquote|pre|figure|figcaption|details|summary|form|fieldset|address)\b[^>]*>/gi;
|
|
10
12
|
const CELL_TAGS = /<\/?(?:td|th)\b[^>]*>/gi;
|
|
11
13
|
const ANY_TAG = /<[^>]+>/g;
|
|
@@ -55,6 +57,7 @@ export function htmlMainContent(html) {
|
|
|
55
57
|
export function htmlToText(html) {
|
|
56
58
|
const text = html
|
|
57
59
|
.replace(COMMENTS, '')
|
|
60
|
+
.replace(REF_MARKERS, '')
|
|
58
61
|
.replace(DROP_ELEMENTS, ' ')
|
|
59
62
|
.replace(BLOCK_TAGS, '\n')
|
|
60
63
|
.replace(CELL_TAGS, ' ')
|
package/dist/source/html.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"html.js","sourceRoot":"","sources":["../../src/source/html.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,MAAM,aAAa,GAAG,sFAAsF,CAAC;AAC7G,MAAM,QAAQ,GAAG,kBAAkB,CAAC;AACpC,MAAM,UAAU,GACd,uMAAuM,CAAC;AAC1M,MAAM,SAAS,GAAG,yBAAyB,CAAC;AAC5C,MAAM,OAAO,GAAG,UAAU,CAAC;AAE3B,MAAM,cAAc,GAA2B;IAC7C,GAAG,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG;IAC3D,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG;IACvE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG;IACpF,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG;IAChE,IAAI,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG;IAC3F,MAAM,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG;IAChF,IAAI,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG;IACjF,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG;IAClF,KAAK,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG;IACvF,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG;IAC5E,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG;IACzF,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG;CACrE,CAAC;AAEF,mFAAmF;AACnF,MAAM,UAAU,cAAc,CAAC,CAAS;IACtC,OAAO,CAAC,CAAC,OAAO,CAAC,uCAAuC,EAAE,CAAC,KAAK,EAAE,IAAY,EAAE,EAAE;QAChF,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,GAAG,EAAE,CAAC;YACpB,MAAM,EAAE,GAAG,IAAI,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;YAC1G,OAAO,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC,IAAI,EAAE,GAAG,CAAC,IAAI,EAAE,IAAI,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC;QAC5F,CAAC;QACD,OAAO,cAAc,CAAC,IAAI,CAAC,IAAI,KAAK,CAAC;IACvC,CAAC,CAAC,CAAC;AACL,CAAC;AAED,0CAA0C;AAC1C,MAAM,UAAU,SAAS,CAAC,IAAY;IACpC,MAAM,CAAC,GAAG,uCAAuC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC7D,MAAM,KAAK,GAAG,CAAC,CAAC,CAAC,CAAC,cAAc,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;IAC7F,OAAO,KAAK,IAAI,SAAS,CAAC;AAC5B,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,eAAe,CAAC,IAAY;IAC1C,KAAK,MAAM,GAAG,IAAI,CAAC,MAAM,EAAE,SAAS,CAAC,EAAE,CAAC;QACtC,MAAM,CAAC,GAAG,IAAI,MAAM,CAAC,IAAI,GAAG,yBAAyB,GAAG,OAAO,EAAE,GAAG,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACjF,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE;YAAE,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;IACpC,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,mEAAmE;AACnE,MAAM,UAAU,UAAU,CAAC,IAAY;IACrC,MAAM,IAAI,GAAG,IAAI;SACd,OAAO,CAAC,QAAQ,EAAE,EAAE,CAAC;SACrB,OAAO,CAAC,aAAa,EAAE,GAAG,CAAC;SAC3B,OAAO,CAAC,UAAU,EAAE,IAAI,CAAC;SACzB,OAAO,CAAC,SAAS,EAAE,GAAG,CAAC;SACvB,OAAO,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC;IACxB,OAAO,cAAc,CAAC,IAAI,CAAC;SACxB,KAAK,CAAC,IAAI,CAAC;SACX,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,OAAO,CAAC,cAAc,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC;SACvD,IAAI,CAAC,IAAI,CAAC;SACV,OAAO,CAAC,SAAS,EAAE,MAAM,CAAC;SAC1B,IAAI,EAAE,CAAC;AACZ,CAAC"}
|
|
1
|
+
{"version":3,"file":"html.js","sourceRoot":"","sources":["../../src/source/html.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,MAAM,aAAa,GAAG,sFAAsF,CAAC;AAC7G,MAAM,QAAQ,GAAG,kBAAkB,CAAC;AACpC,0FAA0F;AAC1F,MAAM,WAAW,GAAG,sEAAsE,CAAC;AAC3F,MAAM,UAAU,GACd,uMAAuM,CAAC;AAC1M,MAAM,SAAS,GAAG,yBAAyB,CAAC;AAC5C,MAAM,OAAO,GAAG,UAAU,CAAC;AAE3B,MAAM,cAAc,GAA2B;IAC7C,GAAG,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG;IAC3D,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG;IACvE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG;IACpF,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG;IAChE,IAAI,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG;IAC3F,MAAM,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG;IAChF,IAAI,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG;IACjF,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG;IAClF,KAAK,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG;IACvF,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG;IAC5E,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG;IACzF,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG;CACrE,CAAC;AAEF,mFAAmF;AACnF,MAAM,UAAU,cAAc,CAAC,CAAS;IACtC,OAAO,CAAC,CAAC,OAAO,CAAC,uCAAuC,EAAE,CAAC,KAAK,EAAE,IAAY,EAAE,EAAE;QAChF,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,GAAG,EAAE,CAAC;YACpB,MAAM,EAAE,GAAG,IAAI,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;YAC1G,OAAO,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC,IAAI,EAAE,GAAG,CAAC,IAAI,EAAE,IAAI,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC;QAC5F,CAAC;QACD,OAAO,cAAc,CAAC,IAAI,CAAC,IAAI,KAAK,CAAC;IACvC,CAAC,CAAC,CAAC;AACL,CAAC;AAED,0CAA0C;AAC1C,MAAM,UAAU,SAAS,CAAC,IAAY;IACpC,MAAM,CAAC,GAAG,uCAAuC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC7D,MAAM,KAAK,GAAG,CAAC,CAAC,CAAC,CAAC,cAAc,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;IAC7F,OAAO,KAAK,IAAI,SAAS,CAAC;AAC5B,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,eAAe,CAAC,IAAY;IAC1C,KAAK,MAAM,GAAG,IAAI,CAAC,MAAM,EAAE,SAAS,CAAC,EAAE,CAAC;QACtC,MAAM,CAAC,GAAG,IAAI,MAAM,CAAC,IAAI,GAAG,yBAAyB,GAAG,OAAO,EAAE,GAAG,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACjF,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE;YAAE,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;IACpC,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,mEAAmE;AACnE,MAAM,UAAU,UAAU,CAAC,IAAY;IACrC,MAAM,IAAI,GAAG,IAAI;SACd,OAAO,CAAC,QAAQ,EAAE,EAAE,CAAC;SACrB,OAAO,CAAC,WAAW,EAAE,EAAE,CAAC;SACxB,OAAO,CAAC,aAAa,EAAE,GAAG,CAAC;SAC3B,OAAO,CAAC,UAAU,EAAE,IAAI,CAAC;SACzB,OAAO,CAAC,SAAS,EAAE,GAAG,CAAC;SACvB,OAAO,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC;IACxB,OAAO,cAAc,CAAC,IAAI,CAAC;SACxB,KAAK,CAAC,IAAI,CAAC;SACX,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,OAAO,CAAC,cAAc,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC;SACvD,IAAI,CAAC,IAAI,CAAC;SACV,OAAO,CAAC,SAAS,EAAE,MAAM,CAAC;SAC1B,IAAI,EAAE,CAAC;AACZ,CAAC"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "veriquote",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.2",
|
|
4
4
|
"description": "Deterministic + semantic verification of quote-grounded LLM citations (EVI1 protocol): fuzzy verbatim-quote matching and an LLM entailment judge for transparent, per-claim hallucination detection.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"llm",
|
package/src/cli/main.ts
CHANGED
|
@@ -31,6 +31,7 @@ const HELP = `veriquote — check that an LLM answer's quotes exist in its sourc
|
|
|
31
31
|
|
|
32
32
|
Usage
|
|
33
33
|
veriquote prompt print the citation instructions for the answering model
|
|
34
|
+
veriquote source <url|file> print the text a source is checked against (quote from this)
|
|
34
35
|
veriquote check <answer> --source <url|file> [--source …]
|
|
35
36
|
veriquote check --job <file|-> JSON job: {"answer": "...", "sources": [{"text": "..."}]}
|
|
36
37
|
|
|
@@ -53,7 +54,7 @@ Judge (semantic support check), from the environment:
|
|
|
53
54
|
VERIQUOTE_JUDGE_MODEL required when a key is set
|
|
54
55
|
VERIQUOTE_JUDGE_BASE_URL default https://openrouter.ai/api/v1
|
|
55
56
|
|
|
56
|
-
Exit status: 0 pass · 2 revise · 1 error`;
|
|
57
|
+
Exit status: 0 pass · 2 revise · 1 error, or the judge failed (unverified)`;
|
|
57
58
|
|
|
58
59
|
interface Args {
|
|
59
60
|
command?: string;
|
|
@@ -147,6 +148,12 @@ export async function main(io: CliIo): Promise<number> {
|
|
|
147
148
|
return EXIT.pass;
|
|
148
149
|
case 'check':
|
|
149
150
|
return await check(args, io);
|
|
151
|
+
case 'source': {
|
|
152
|
+
if (args.positional.length !== 1) throw new UsageError('source needs exactly one URL or file');
|
|
153
|
+
const src = await loadSource(args.positional[0], io);
|
|
154
|
+
io.stdout(`${src.text}\n`);
|
|
155
|
+
return EXIT.pass;
|
|
156
|
+
}
|
|
150
157
|
default:
|
|
151
158
|
throw new UsageError(`unknown command "${args.command}"`);
|
|
152
159
|
}
|
|
@@ -191,9 +198,21 @@ async function check(args: Args, io: CliIo): Promise<number> {
|
|
|
191
198
|
{
|
|
192
199
|
verdict: gate.verdict,
|
|
193
200
|
judge: judgeSetup.model ?? null,
|
|
201
|
+
// The answer as the user should see it: [n] markers kept, {cX} markers and the appendix removed.
|
|
202
|
+
cleanAnswer: report.cleanText,
|
|
194
203
|
summary: report.summary,
|
|
195
204
|
problems: gate.problems,
|
|
196
205
|
uncited: gate.uncited,
|
|
206
|
+
unjudged: gate.unjudged,
|
|
207
|
+
citations: report.citations.map((c) => ({
|
|
208
|
+
claimId: c.claimId,
|
|
209
|
+
sourceIndex: c.sourceIndex,
|
|
210
|
+
match: { method: c.textMatch.method, score: round2(c.textMatch.score) },
|
|
211
|
+
judge: c.entailment
|
|
212
|
+
? { class: c.entailment.class, support: c.entailment.confidence === null ? null : round2(c.entailment.confidence) }
|
|
213
|
+
: null,
|
|
214
|
+
score: c.score === null ? null : round2(c.score),
|
|
215
|
+
})),
|
|
197
216
|
warnings: report.warnings,
|
|
198
217
|
sources: sources.map((s, i) => ({ index: i + 1, url: s.url, title: s.title, chars: s.text.length })),
|
|
199
218
|
instructionsForModel: gate.instructionsForModel,
|
|
@@ -205,9 +224,11 @@ async function check(args: Args, io: CliIo): Promise<number> {
|
|
|
205
224
|
} else {
|
|
206
225
|
io.stdout(renderHuman(report, gate, sources, judgeSetup.model, io.color, args.minScore ?? 0.5));
|
|
207
226
|
}
|
|
208
|
-
return gate.verdict === 'pass' ? EXIT.pass : EXIT.revise;
|
|
227
|
+
return gate.verdict === 'pass' ? EXIT.pass : gate.verdict === 'revise' ? EXIT.revise : EXIT.error;
|
|
209
228
|
}
|
|
210
229
|
|
|
230
|
+
const round2 = (x: number) => Math.round(x * 100) / 100;
|
|
231
|
+
|
|
211
232
|
async function loadSource(ref: string, io: CliIo): Promise<SourceDocument> {
|
|
212
233
|
if (/^https?:\/\//i.test(ref)) {
|
|
213
234
|
return fetchSource(ref, {
|
|
@@ -287,13 +308,16 @@ function renderHuman(
|
|
|
287
308
|
|
|
288
309
|
out.push('');
|
|
289
310
|
if (gate.verdict === 'pass') {
|
|
290
|
-
out.push(green(report.citations.length ? 'PASS' : 'PASS
|
|
311
|
+
out.push(green(report.citations.length ? 'PASS' : 'PASS, but the answer cites nothing'));
|
|
312
|
+
} else if (gate.verdict === 'unverified') {
|
|
313
|
+
out.push(yellow(`UNVERIFIED: the judge failed on ${gate.unjudged.length} citation(s), so their support was not checked. Run the check again.`));
|
|
291
314
|
} else {
|
|
292
315
|
const parts = [
|
|
293
316
|
gate.problems.length && `${gate.problems.length} failed citation(s)`,
|
|
294
317
|
gate.uncited.length && `${gate.uncited.length} uncited sentence(s)`,
|
|
295
318
|
].filter(Boolean);
|
|
296
|
-
out.push(red(`REVISE
|
|
319
|
+
out.push(red(`REVISE: ${parts.join(', ')}`));
|
|
320
|
+
if (gate.unjudged.length) out.push(yellow(`The judge also failed on ${gate.unjudged.length} citation(s).`));
|
|
297
321
|
}
|
|
298
322
|
if (!model) out.push(dim('Only checked that quotes exist in their sources. Set VERIQUOTE_JUDGE_API_KEY to check support.'));
|
|
299
323
|
return `${out.join('\n')}\n`;
|
package/src/gate.ts
CHANGED
|
@@ -33,11 +33,17 @@ export interface GateProblem {
|
|
|
33
33
|
}
|
|
34
34
|
|
|
35
35
|
export interface GateResult {
|
|
36
|
-
|
|
36
|
+
/**
|
|
37
|
+
* `unverified`: nothing failed, but the judge could not rate some citations,
|
|
38
|
+
* so support was not checked for them. Never report that as a pass.
|
|
39
|
+
*/
|
|
40
|
+
verdict: 'pass' | 'revise' | 'unverified';
|
|
37
41
|
problems: GateProblem[];
|
|
38
42
|
/** Factual-looking sentences without any `[n]` marker. Heuristic. */
|
|
39
43
|
uncited: string[];
|
|
40
|
-
/**
|
|
44
|
+
/** Citations the judge failed on (only when a judge ran). */
|
|
45
|
+
unjudged: { claimId: string; sourceIndex: number; reason: string }[];
|
|
46
|
+
/** Ready-to-send correction prompt; `null` unless the verdict is `revise`. */
|
|
41
47
|
instructionsForModel: string | null;
|
|
42
48
|
}
|
|
43
49
|
|
|
@@ -69,12 +75,16 @@ export function gateReport(
|
|
|
69
75
|
score: c.score === null ? null : round(c.score),
|
|
70
76
|
}));
|
|
71
77
|
const uncited = findUncitedSentences(stripEvi1Appendix(answer), t.minUncitedChars);
|
|
72
|
-
const
|
|
78
|
+
const unjudged = report.citations
|
|
79
|
+
.filter((c) => c.entailment?.class === 'error')
|
|
80
|
+
.map((c) => ({ claimId: c.claimId, sourceIndex: c.sourceIndex, reason: c.entailment?.reasons.join('; ') || 'judge error' }));
|
|
81
|
+
const verdict = problems.length || uncited.length ? 'revise' : unjudged.length ? 'unverified' : 'pass';
|
|
73
82
|
return {
|
|
74
83
|
verdict,
|
|
75
84
|
problems,
|
|
76
85
|
uncited,
|
|
77
|
-
|
|
86
|
+
unjudged,
|
|
87
|
+
instructionsForModel: verdict === 'revise' ? correctionInstructions(problems, uncited) : null,
|
|
78
88
|
};
|
|
79
89
|
}
|
|
80
90
|
|
package/src/source/html.ts
CHANGED
|
@@ -7,6 +7,8 @@
|
|
|
7
7
|
|
|
8
8
|
const DROP_ELEMENTS = /<(script|style|noscript|template|svg|iframe|head|object|canvas)\b[\s\S]*?<\/\1\s*>/gi;
|
|
9
9
|
const COMMENTS = /<!--[\s\S]*?-->/g;
|
|
10
|
+
/** Reference markers such as Wikipedia's [12]: models quoting the page leave them out. */
|
|
11
|
+
const REF_MARKERS = /<sup\b[^>]*class="[^"]*\breference\b[^"]*"[^>]*>[\s\S]*?<\/sup\s*>/gi;
|
|
10
12
|
const BLOCK_TAGS =
|
|
11
13
|
/<\/?(?:p|div|br|hr|li|ul|ol|dl|dt|dd|h[1-6]|tr|table|thead|tbody|tfoot|section|article|aside|main|header|footer|nav|blockquote|pre|figure|figcaption|details|summary|form|fieldset|address)\b[^>]*>/gi;
|
|
12
14
|
const CELL_TAGS = /<\/?(?:td|th)\b[^>]*>/gi;
|
|
@@ -61,6 +63,7 @@ export function htmlMainContent(html: string): string | undefined {
|
|
|
61
63
|
export function htmlToText(html: string): string {
|
|
62
64
|
const text = html
|
|
63
65
|
.replace(COMMENTS, '')
|
|
66
|
+
.replace(REF_MARKERS, '')
|
|
64
67
|
.replace(DROP_ELEMENTS, ' ')
|
|
65
68
|
.replace(BLOCK_TAGS, '\n')
|
|
66
69
|
.replace(CELL_TAGS, ' ')
|