@yahyahammad/venastine 0.2.0 → 0.3.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +9 -4
- package/CONFIG_ARCHITECTURE.md +2030 -0
- package/LICENSE +201 -201
- package/NOTICE +1 -1
- package/README.md +1034 -895
- package/agents/builtin/build.md +73 -0
- package/agents/builtin/compactor.md +75 -75
- package/agents/builtin/explore.md +77 -77
- package/agents/builtin/general.md +80 -0
- package/agents/builtin/pipeline-reviewer.md +100 -100
- package/agents/builtin/plan.md +106 -88
- package/agents/builtin/review.md +74 -74
- package/agents/builtin/test.md +65 -0
- package/agents/builtin/writer.md +67 -0
- package/agents/manager.py +68 -2
- package/agents/subagent_tool.py +472 -238
- package/agents/tui_commands.py +238 -151
- package/bin/venastine.mjs +150 -4
- package/bin/venastine.py +71 -0
- package/cli_commands.py +838 -0
- package/config.py +95 -1198
- package/config.yaml +1103 -0
- package/config_edit.py +1301 -0
- package/config_schema.py +971 -0
- package/config_update.py +749 -0
- package/core/agent_activity.py +256 -118
- package/core/ansi.py +452 -0
- package/core/approval.py +137 -115
- package/core/call_view.py +364 -0
- package/core/cancel.py +121 -0
- package/core/chatgpt_auth.py +747 -0
- package/core/claims_view.py +107 -0
- package/core/client.py +2715 -1559
- package/core/commands.py +460 -0
- package/core/compaction.py +1735 -971
- package/core/config_loader.py +1982 -1563
- package/core/connect.py +686 -0
- package/core/events.py +123 -61
- package/core/file_mentions.py +280 -0
- package/core/interaction.py +411 -328
- package/core/line_pattern.py +128 -0
- package/core/loop.py +2566 -1490
- package/core/memory.py +482 -370
- package/core/pipeline_models.py +190 -176
- package/core/provider_errors.py +234 -0
- package/core/reasoning/authorization.py +130 -130
- package/core/reasoning/base.py +219 -219
- package/core/reasoning/confidence_scoring.py +484 -468
- package/core/reasoning/events.py +149 -149
- package/core/reasoning/json_retry.py +19 -1
- package/core/reasoning/orchestrator.py +1561 -1382
- package/core/reasoning/output_writer.py +345 -286
- package/core/reasoning/payload_validation.py +524 -398
- package/core/reasoning/pipeline_storage.py +88 -22
- package/core/reasoning/review.py +659 -615
- package/core/reasoning/scholar.py +458 -434
- package/core/reasoning/source_corpus.py +433 -327
- package/core/reasoning/source_scoring.py +855 -855
- package/core/replay.py +315 -183
- package/core/session.py +36 -12
- package/core/session_view.py +259 -0
- package/core/session_wake.py +259 -0
- package/core/shell_sessions.py +1553 -0
- package/core/workspace_trust.py +1 -1
- package/credentials.py +148 -20
- package/database.py +333 -252
- package/{PRIVACY.md → docs/PRIVACY.md} +78 -66
- package/docs/SECURITY.md +108 -0
- package/docs/THIRD_PARTY_NOTICES.md +148 -0
- package/env_secrets.py +50 -9
- package/json_store.py +343 -123
- package/logging_setup.py +295 -185
- package/main.py +3433 -1961
- package/mcp_client/client.py +733 -499
- package/mcp_client/config.py +126 -21
- package/mcp_client/registration.py +343 -182
- package/memories/manager.py +147 -132
- package/memories/tui_commands.py +319 -244
- package/package.json +303 -73
- package/project_init/config_files.py +282 -278
- package/project_init/generator.py +728 -564
- package/project_init/manifest.py +18 -6
- package/project_init/tui_commands.py +147 -144
- package/prompts/critic_pass.md +24 -24
- package/prompts/revalidate.md +25 -25
- package/prompts/revise.md +2 -0
- package/prompts/source_grounding.md +66 -66
- package/prompts/system_prompts.py +316 -262
- package/providers.json.example +4 -2
- package/pyproject.toml +149 -100
- package/requirements.txt +295 -72
- package/safety/policy_enforcement.py +1082 -746
- package/security/askpass.py +419 -0
- package/security/capability.py +207 -177
- package/security/confine.py +484 -0
- package/security/hosted.py +182 -0
- package/security/permissions.py +237 -190
- package/security/posture.py +124 -12
- package/security/protected_paths.py +501 -309
- package/security/sandbox.py +4108 -985
- package/security/secrets.py +855 -0
- package/skills/builtin/math/formal-specification.md +67 -67
- package/skills/builtin/math/numerical-methods.md +66 -66
- package/skills/builtin/research/experiment-design.md +62 -62
- package/skills/builtin/research/reproducibility.md +55 -55
- package/skills/builtin/research/source-evaluation.md +70 -70
- package/skills/builtin/research/statistical-inference.md +63 -63
- package/skills/builtin/research/technical-writing.md +59 -59
- package/skills/builtin/software/code-review.md +61 -61
- package/skills/builtin/software/debugging.md +61 -61
- package/skills/builtin/software/test-design.md +67 -67
- package/skills/tui_commands.py +110 -111
- package/storage.py +1338 -1005
- package/tools/base.py +161 -1
- package/tools/builtin/_math_common.py +667 -377
- package/tools/builtin/_net_common.py +225 -19
- package/tools/builtin/arxiv.py +497 -39
- package/tools/builtin/ask_user.py +231 -213
- package/tools/builtin/discrete_math.py +35 -4
- package/tools/builtin/fetch_url.py +371 -153
- package/tools/builtin/file_ops.py +720 -412
- package/tools/builtin/geometry.py +48 -12
- package/tools/builtin/get_time.py +2 -2
- package/tools/builtin/linear_algebra.py +66 -23
- package/tools/builtin/load_skill.py +76 -66
- package/tools/builtin/logic.py +171 -12
- package/tools/builtin/pin.py +227 -193
- package/tools/builtin/probability_stats.py +12 -16
- package/tools/builtin/project_docs.py +464 -405
- package/tools/builtin/remember.py +175 -158
- package/tools/builtin/shell.py +980 -461
- package/tools/builtin/shell_sessions.py +613 -0
- package/tools/builtin/symbolic_math.py +80 -10
- package/tools/builtin/todo.py +149 -147
- package/tools/builtin/web_search.py +191 -13
- package/tools/context.py +159 -136
- package/tools/isolation.py +107 -34
- package/tools/registry.py +1242 -925
- package/tui/app.py +8318 -4100
- package/tui/app.tcss +747 -556
- package/tui/commands.py +26 -170
- package/tui/diffs.py +34 -13
- package/tui/history.py +128 -128
- package/tui/hosted_driver.py +102 -0
- package/tui/markdown.py +1329 -1279
- package/tui/meters.py +364 -268
- package/tui/ravens.py +146 -146
- package/tui/screens.py +1255 -943
- package/tui/themes.py +599 -559
- package/tui/widgets.py +3813 -2332
- package/SECURITY.md +0 -88
- package/THIRD_PARTY_NOTICES.md +0 -100
package/project_init/manifest.py
CHANGED
|
@@ -88,7 +88,7 @@ _MAKE_TEST_TARGET = re.compile(r"^test\s*:", re.M)
|
|
|
88
88
|
def _make_test(project_path: str) -> Optional[str]:
|
|
89
89
|
"""`make test` only when the Makefile declares that target."""
|
|
90
90
|
try:
|
|
91
|
-
with open(os.path.join(project_path, "Makefile"),
|
|
91
|
+
with open(os.path.join(project_path, "Makefile"),
|
|
92
92
|
encoding="utf-8", errors="replace") as f:
|
|
93
93
|
head = f.read(64_000)
|
|
94
94
|
except OSError:
|
|
@@ -148,16 +148,28 @@ _TEST_DIRS = ("tests", "test", "__tests__", "spec")
|
|
|
148
148
|
# anyway, so this is not the control -- it stops the agent spending one of
|
|
149
149
|
# its bounded steps discovering that, and stops a credential FILENAME (which
|
|
150
150
|
# can name a service, an account or an environment) reaching the model at
|
|
151
|
-
# all. Dotfiles are already excluded from the walk; this covers the rest
|
|
151
|
+
# all. Dotfiles are already excluded from the walk; this covers the rest:
|
|
152
|
+
# five store names, `.env*`, and since #227 the conventional shapes of key
|
|
153
|
+
# material -- an SSH private key's default name, and the suffixes private
|
|
154
|
+
# keys and keystores are saved under. Until then this comment claimed all
|
|
155
|
+
# credential filenames and the set held the first five: `id_rsa`,
|
|
156
|
+
# `deploy-key.pem` and `api.key` were listed. A public key (`id_rsa.pub`)
|
|
157
|
+
# still is: it is not a secret, and its name says only that a key exists.
|
|
152
158
|
_SECRET_FILENAMES = frozenset({
|
|
153
159
|
"providers.json", "credentials.json", "secrets.json", "secrets.yaml",
|
|
154
160
|
"secrets.yml",
|
|
155
161
|
})
|
|
162
|
+
_KEY_FILENAMES = frozenset({
|
|
163
|
+
"id_rsa", "id_dsa", "id_ecdsa", "id_ed25519", "id_ecdsa_sk",
|
|
164
|
+
"id_ed25519_sk",
|
|
165
|
+
})
|
|
166
|
+
_KEY_SUFFIXES = (".pem", ".key", ".p12", ".pfx", ".ppk", ".jks", ".keystore")
|
|
156
167
|
|
|
157
168
|
|
|
158
169
|
def _is_secret(name: str) -> bool:
|
|
159
170
|
lower = name.lower()
|
|
160
|
-
return lower in _SECRET_FILENAMES or lower.startswith(".env")
|
|
171
|
+
return (lower in _SECRET_FILENAMES or lower.startswith(".env")
|
|
172
|
+
or lower in _KEY_FILENAMES or lower.endswith(_KEY_SUFFIXES))
|
|
161
173
|
|
|
162
174
|
|
|
163
175
|
_TREE_MAX_DEPTH = 2
|
|
@@ -188,7 +200,7 @@ def _first_heading(path: str) -> str:
|
|
|
188
200
|
repo alone.
|
|
189
201
|
"""
|
|
190
202
|
try:
|
|
191
|
-
with open(path,
|
|
203
|
+
with open(path, encoding="utf-8", errors="replace") as f:
|
|
192
204
|
head = f.read(4096)
|
|
193
205
|
except OSError:
|
|
194
206
|
return ""
|
|
@@ -340,7 +352,7 @@ def _mentions(project_path: str, filenames, needle: str) -> bool:
|
|
|
340
352
|
if not os.path.isfile(path):
|
|
341
353
|
continue
|
|
342
354
|
try:
|
|
343
|
-
with open(path,
|
|
355
|
+
with open(path, encoding="utf-8", errors="replace") as f:
|
|
344
356
|
if needle in f.read(64_000):
|
|
345
357
|
return True
|
|
346
358
|
except OSError:
|
|
@@ -351,7 +363,7 @@ def _mentions(project_path: str, filenames, needle: str) -> bool:
|
|
|
351
363
|
def _package_json_test(project_path: str) -> Optional[str]:
|
|
352
364
|
path = os.path.join(project_path, "package.json")
|
|
353
365
|
try:
|
|
354
|
-
with open(path,
|
|
366
|
+
with open(path, encoding="utf-8-sig") as f:
|
|
355
367
|
data = json.load(f)
|
|
356
368
|
except (OSError, ValueError):
|
|
357
369
|
# A malformed package.json is the project's problem, not /init's.
|
|
@@ -1,144 +1,147 @@
|
|
|
1
|
-
"""
|
|
2
|
-
project_init/tui_commands.py
|
|
3
|
-
|
|
4
|
-
ROADMAP_v2 §24: `/init`, registered into §16's slash registry.
|
|
5
|
-
|
|
6
|
-
Mirrors memories/tui_commands.py, skills/tui_commands.py and
|
|
7
|
-
agents/tui_commands.py -- each section registers its own commands rather
|
|
8
|
-
than editing a match statement in the app.
|
|
9
|
-
|
|
10
|
-
THIN BY CONSTRUCTION. Every decision -- which set, whether to overwrite,
|
|
11
|
-
whether to re-grant trust -- lives in project_init/generator.py, so the CLI
|
|
12
|
-
makes the same ones. What is here is the three things only a TUI can do:
|
|
13
|
-
run the work off the UI thread, render a diff into the transcript, and put
|
|
14
|
-
a modal in front of a human.
|
|
15
|
-
"""
|
|
16
|
-
|
|
17
|
-
from tui.commands import
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
f"
|
|
54
|
-
f"
|
|
55
|
-
f"
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
the
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
#
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
from
|
|
92
|
-
from project_init
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
#
|
|
96
|
-
#
|
|
97
|
-
#
|
|
98
|
-
#
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
"
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
1
|
+
"""
|
|
2
|
+
project_init/tui_commands.py
|
|
3
|
+
|
|
4
|
+
ROADMAP_v2 §24: `/init`, registered into §16's slash registry.
|
|
5
|
+
|
|
6
|
+
Mirrors memories/tui_commands.py, skills/tui_commands.py and
|
|
7
|
+
agents/tui_commands.py -- each section registers its own commands rather
|
|
8
|
+
than editing a match statement in the app.
|
|
9
|
+
|
|
10
|
+
THIN BY CONSTRUCTION. Every decision -- which set, whether to overwrite,
|
|
11
|
+
whether to re-grant trust -- lives in project_init/generator.py, so the CLI
|
|
12
|
+
makes the same ones. What is here is the three things only a TUI can do:
|
|
13
|
+
run the work off the UI thread, render a diff into the transcript, and put
|
|
14
|
+
a modal in front of a human.
|
|
15
|
+
"""
|
|
16
|
+
|
|
17
|
+
from tui.commands import bind
|
|
18
|
+
from tui.commands import registry as commands
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
def _split_init_flags(args: str) -> tuple:
|
|
22
|
+
"""(kind, scaffold_config, error) for `/init`'s leading flags.
|
|
23
|
+
|
|
24
|
+
A LOOP over tokens, modelled on `/research`'s `_split_research_flags`,
|
|
25
|
+
so the flags compose in either written order -- `/init --config
|
|
26
|
+
--software` and `/init --software --config` are one command written
|
|
27
|
+
two ways, and handling one flag then the other in a fixed sequence is
|
|
28
|
+
exactly how that stopped being true there.
|
|
29
|
+
|
|
30
|
+
STRICTER THAN WHAT IT REPLACES, on purpose. The old parser read a
|
|
31
|
+
single token through `.lstrip("-")`, which also accepted `/init
|
|
32
|
+
software` and `/init ---software`. A command that writes files into
|
|
33
|
+
someone's project should not be reachable by a typo that happens to
|
|
34
|
+
normalise, and there is no muscle memory to protect: the documented
|
|
35
|
+
spellings are the only ones that ever appeared.
|
|
36
|
+
|
|
37
|
+
The kind flags are derived from `doc_sets.PROJECT_KINDS` rather than
|
|
38
|
+
typed out, so a third document set is spelled once.
|
|
39
|
+
"""
|
|
40
|
+
from project_init import doc_sets
|
|
41
|
+
|
|
42
|
+
usage = "Usage: /init [--software|--research] [--config]."
|
|
43
|
+
kinds = {"--" + name: name for name in doc_sets.PROJECT_KINDS}
|
|
44
|
+
kind = None
|
|
45
|
+
scaffold_config = False
|
|
46
|
+
for token in (args or "").split():
|
|
47
|
+
low = token.lower()
|
|
48
|
+
if low == "--config":
|
|
49
|
+
scaffold_config = True
|
|
50
|
+
elif low in kinds:
|
|
51
|
+
if kind is not None and kind != kinds[low]:
|
|
52
|
+
return None, False, (
|
|
53
|
+
f"/init takes at most one of "
|
|
54
|
+
f"{', '.join(sorted(kinds))} -- got both, and guessing "
|
|
55
|
+
f"which you meant would scaffold the wrong document "
|
|
56
|
+
f"set. {usage}")
|
|
57
|
+
kind = kinds[low]
|
|
58
|
+
else:
|
|
59
|
+
return None, False, f"Unknown option {token!r}. {usage}"
|
|
60
|
+
return kind, scaffold_config, None
|
|
61
|
+
|
|
62
|
+
|
|
63
|
+
def _cmd_init(app, args: str) -> None:
|
|
64
|
+
"""Scaffold this project's documentation set, its configuration, or both.
|
|
65
|
+
|
|
66
|
+
On a worker, because it is a model call over a project's documents, and
|
|
67
|
+
announced before it starts: a call the user pays for is never silent,
|
|
68
|
+
the rule /compact and /summary already follow.
|
|
69
|
+
|
|
70
|
+
`--config` ALONE runs no model call -- the worker is still where it
|
|
71
|
+
belongs, because the one consent opens a modal and the thread that
|
|
72
|
+
waits on the answer cannot be the UI thread.
|
|
73
|
+
"""
|
|
74
|
+
kind, scaffold_config, problem = _split_init_flags(args)
|
|
75
|
+
if problem:
|
|
76
|
+
app._transcript.write_error(problem)
|
|
77
|
+
return
|
|
78
|
+
# `--config` on its own is the configuration and nothing else; named
|
|
79
|
+
# with a kind it is both, under one consent. §24 I17.
|
|
80
|
+
scaffold_docs = not scaffold_config or kind is not None
|
|
81
|
+
|
|
82
|
+
if app._busy:
|
|
83
|
+
app._transcript.write_error(
|
|
84
|
+
"Still working — wait for this turn to finish.")
|
|
85
|
+
return
|
|
86
|
+
|
|
87
|
+
app._busy = True
|
|
88
|
+
transcript = app._transcript # held before any modal opens (§27's note)
|
|
89
|
+
|
|
90
|
+
def _work() -> None:
|
|
91
|
+
from core import interaction
|
|
92
|
+
from project_init import doc_sets
|
|
93
|
+
from project_init.generator import InitError, generate
|
|
94
|
+
|
|
95
|
+
# §23: generate()'s confirm/choose_kind parameters are unchanged --
|
|
96
|
+
# they are a shell-agnostic API the CLI implements too. What moved
|
|
97
|
+
# is the plumbing beneath them: both now go down the same channel
|
|
98
|
+
# as every other question, so the fail-safe decoding is the shared
|
|
99
|
+
# one rather than a guard written by hand here.
|
|
100
|
+
channel = app.response_channel()
|
|
101
|
+
|
|
102
|
+
def _confirm(summary: str) -> bool:
|
|
103
|
+
return interaction.ask(channel, interaction.Request(
|
|
104
|
+
kind=interaction.CONFIRM,
|
|
105
|
+
payload={"title": "Write these files?", "body": summary,
|
|
106
|
+
"confirm_label": "Write"}))
|
|
107
|
+
|
|
108
|
+
def _choose_kind(proposal, reason, blank):
|
|
109
|
+
return interaction.ask(channel, interaction.Request(
|
|
110
|
+
kind=interaction.CHOICE,
|
|
111
|
+
payload={"options": list(doc_sets.PROJECT_KINDS),
|
|
112
|
+
"proposal": proposal, "reason": reason,
|
|
113
|
+
"blank": blank}))
|
|
114
|
+
|
|
115
|
+
try:
|
|
116
|
+
notice = generate(
|
|
117
|
+
# §47. `getattr`, matching /summary and /compact: a stub
|
|
118
|
+
# app in the suite has no sink, and every span call site
|
|
119
|
+
# takes None without a branch.
|
|
120
|
+
activity=getattr(app, "_activity", None),
|
|
121
|
+
model=app.model,
|
|
122
|
+
provider_name=app.provider_name,
|
|
123
|
+
kind=kind,
|
|
124
|
+
confirm=_confirm,
|
|
125
|
+
notify=lambda text: app.call_from_thread(
|
|
126
|
+
transcript.write_system, text),
|
|
127
|
+
choose_kind=_choose_kind,
|
|
128
|
+
scaffold_docs=scaffold_docs,
|
|
129
|
+
scaffold_config=scaffold_config,
|
|
130
|
+
)
|
|
131
|
+
except InitError as e:
|
|
132
|
+
app.call_from_thread(transcript.write_error, str(e))
|
|
133
|
+
return
|
|
134
|
+
except Exception as e: # noqa: BLE001 — same containment as /compact
|
|
135
|
+
app.call_from_thread(transcript.write_error, f"/init failed: {e}")
|
|
136
|
+
return
|
|
137
|
+
finally:
|
|
138
|
+
app.call_from_thread(setattr, app, "_busy", False)
|
|
139
|
+
app.call_from_thread(transcript.write_system, notice["text"])
|
|
140
|
+
|
|
141
|
+
app.run_worker(_work, thread=True, exit_on_error=False, name="init")
|
|
142
|
+
|
|
143
|
+
|
|
144
|
+
def register_init_commands() -> None:
|
|
145
|
+
"""Idempotent -- registering by name overwrites, matching §16's
|
|
146
|
+
register_builtin_commands() and every section registrar since."""
|
|
147
|
+
commands.register(bind("init", _cmd_init))
|
package/prompts/critic_pass.md
CHANGED
|
@@ -1,24 +1,24 @@
|
|
|
1
|
-
# Pass 3b — Critic Pass
|
|
2
|
-
|
|
3
|
-
You will receive every surviving candidate response from Pass 1, labelled `Candidate N:` exactly as Pass 2 labelled them, the factual claims extracted from their union, and their grounding results. A claim may originate from any candidate — check it against the text it came from and against the others.
|
|
4
|
-
|
|
5
|
-
Check for:
|
|
6
|
-
- **Internal contradictions**: does any claim conflict with another claim elsewhere in the candidate responses?
|
|
7
|
-
- **Logical fallacies**: non-sequiturs, false dichotomies, correlation-causation conflation, circular reasoning, or other reasoning errors in how claims are connected or justified.
|
|
8
|
-
|
|
9
|
-
Weight severity by how well-grounded the claim already is: a contradiction involving a well-grounded (`grounded` status) claim is more serious than one involving an already-`ungrounded` guess, since undermining something otherwise-credible is a bigger problem than flagging an issue with something already known to be shaky. Use this weighting to inform the `severity` score below, not as a separate field.
|
|
10
|
-
|
|
11
|
-
Respond with ONLY this JSON structure — one entry per factual claim you were given (include claims with no issues, using empty lists and severity 0.0):
|
|
12
|
-
|
|
13
|
-
```json
|
|
14
|
-
[
|
|
15
|
-
{
|
|
16
|
-
"claim_id": "c001",
|
|
17
|
-
"fallacies": ["..."],
|
|
18
|
-
"contradictions": ["..."],
|
|
19
|
-
"severity": 0.0
|
|
20
|
-
}
|
|
21
|
-
]
|
|
22
|
-
```
|
|
23
|
-
|
|
24
|
-
`severity` is 0.0 (no issues) to 1.0 (severe — e.g. a flat contradiction undermining an otherwise well-grounded claim). Be specific in `fallacies`/`contradictions` — name the actual reasoning error or quote the conflicting claim, don't just say "there is a contradiction."
|
|
1
|
+
# Pass 3b — Critic Pass
|
|
2
|
+
|
|
3
|
+
You will receive every surviving candidate response from Pass 1, labelled `Candidate N:` exactly as Pass 2 labelled them, the factual claims extracted from their union, and their grounding results. A claim may originate from any candidate — check it against the text it came from and against the others.
|
|
4
|
+
|
|
5
|
+
Check for:
|
|
6
|
+
- **Internal contradictions**: does any claim conflict with another claim elsewhere in the candidate responses?
|
|
7
|
+
- **Logical fallacies**: non-sequiturs, false dichotomies, correlation-causation conflation, circular reasoning, or other reasoning errors in how claims are connected or justified.
|
|
8
|
+
|
|
9
|
+
Weight severity by how well-grounded the claim already is: a contradiction involving a well-grounded (`grounded` status) claim is more serious than one involving an already-`ungrounded` guess, since undermining something otherwise-credible is a bigger problem than flagging an issue with something already known to be shaky. Use this weighting to inform the `severity` score below, not as a separate field.
|
|
10
|
+
|
|
11
|
+
Respond with ONLY this JSON structure — one entry per factual claim you were given (include claims with no issues, using empty lists and severity 0.0):
|
|
12
|
+
|
|
13
|
+
```json
|
|
14
|
+
[
|
|
15
|
+
{
|
|
16
|
+
"claim_id": "c001",
|
|
17
|
+
"fallacies": ["..."],
|
|
18
|
+
"contradictions": ["..."],
|
|
19
|
+
"severity": 0.0
|
|
20
|
+
}
|
|
21
|
+
]
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
`severity` is 0.0 (no issues) to 1.0 (severe — e.g. a flat contradiction undermining an otherwise well-grounded claim). Be specific in `fallacies`/`contradictions` — name the actual reasoning error or quote the conflicting claim, don't just say "there is a contradiction."
|
package/prompts/revalidate.md
CHANGED
|
@@ -1,25 +1,25 @@
|
|
|
1
|
-
# Pass 6b — Re-validate
|
|
2
|
-
|
|
3
|
-
You will receive the revised claims from Pass 6a — ONLY the claims
|
|
4
|
-
|
|
5
|
-
Re-run the same checks Pass 3a (source grounding) and Pass 3b (critic) performed, but scoped to just this revised subset:
|
|
6
|
-
|
|
7
|
-
1. **Grounding**: for each revised claim, re-derive a search query from its (now possibly different) entity/subject and check it against available sources, the same way Pass 3a did — deduplicate entities across this batch first, search once per unique entity, map back to claims. Assign a fresh `grounded`/`partial`/`ungrounded` status based on the REVISED text, not the original.
|
|
8
|
-
2. **Critic**: re-check the revised claims for fallacies and contradictions, the same way Pass 3b did, including against the rest of the (unrevised) claim set if relevant context is available to you.
|
|
9
|
-
|
|
10
|
-
Respond with ONLY this JSON structure:
|
|
11
|
-
|
|
12
|
-
```json
|
|
13
|
-
{
|
|
14
|
-
"grounding": [
|
|
15
|
-
{"claim_id": "c001", "sources": [{"url": "...", "quote": "...", "similarity_score": 0.0, "authority_adjustment": 0.0, "authority_reason": "..."}], "status": "grounded"}
|
|
16
|
-
],
|
|
17
|
-
"critic": [
|
|
18
|
-
{"claim_id": "c001", "fallacies": ["..."], "contradictions": ["..."], "severity": 0.0}
|
|
19
|
-
]
|
|
20
|
-
}
|
|
21
|
-
```
|
|
22
|
-
|
|
23
|
-
Include an entry in both `grounding` and `critic` for every claim you were given, even if a section is empty (e.g. `"fallacies": []`).
|
|
24
|
-
|
|
25
|
-
The `sources` entries follow Pass 3a's rules exactly, and this is a re-check rather than a re-use: quote the passage verbatim from the source, score `similarity_score` against the passage you quoted and against the REVISED claim text, and leave `authority_adjustment` at `0.0` unless this specific page is stronger or weaker than its domain suggests. A source that supported the original wording may not support the revision, and carrying its old score across is the thing re-validation exists to catch.
|
|
1
|
+
# Pass 6b — Re-validate
|
|
2
|
+
|
|
3
|
+
You will receive the revised claims from Pass 6a — ONLY the claims it was just given, not the full original claim set. Most were rewritten; a claim it was asked only to check may come back with its text unchanged, and it is re-validated all the same.
|
|
4
|
+
|
|
5
|
+
Re-run the same checks Pass 3a (source grounding) and Pass 3b (critic) performed, but scoped to just this revised subset:
|
|
6
|
+
|
|
7
|
+
1. **Grounding**: for each revised claim, re-derive a search query from its (now possibly different) entity/subject and check it against available sources, the same way Pass 3a did — deduplicate entities across this batch first, search once per unique entity, map back to claims. Assign a fresh `grounded`/`partial`/`ungrounded` status based on the REVISED text, not the original.
|
|
8
|
+
2. **Critic**: re-check the revised claims for fallacies and contradictions, the same way Pass 3b did, including against the rest of the (unrevised) claim set if relevant context is available to you.
|
|
9
|
+
|
|
10
|
+
Respond with ONLY this JSON structure:
|
|
11
|
+
|
|
12
|
+
```json
|
|
13
|
+
{
|
|
14
|
+
"grounding": [
|
|
15
|
+
{"claim_id": "c001", "sources": [{"url": "...", "quote": "...", "similarity_score": 0.0, "authority_adjustment": 0.0, "authority_reason": "..."}], "status": "grounded"}
|
|
16
|
+
],
|
|
17
|
+
"critic": [
|
|
18
|
+
{"claim_id": "c001", "fallacies": ["..."], "contradictions": ["..."], "severity": 0.0}
|
|
19
|
+
]
|
|
20
|
+
}
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
Include an entry in both `grounding` and `critic` for every claim you were given, even if a section is empty (e.g. `"fallacies": []`).
|
|
24
|
+
|
|
25
|
+
The `sources` entries follow Pass 3a's rules exactly, and this is a re-check rather than a re-use: quote the passage verbatim from the source, score `similarity_score` against the passage you quoted and against the REVISED claim text, and leave `authority_adjustment` at `0.0` unless this specific page is stronger or weaker than its domain suggests. A source that supported the original wording may not support the revision, and carrying its old score across is the thing re-validation exists to catch.
|
package/prompts/revise.md
CHANGED
|
@@ -6,6 +6,8 @@ For each flagged claim, rewrite it to actually address the specific issue(s) rai
|
|
|
6
6
|
|
|
7
7
|
Do not revise every claim identically with generic hedge language ("some evidence suggests," "it is possible that") — that would defeat the purpose of this pass. Each rewrite should reflect what the SPECIFIC feedback for that claim actually said was wrong.
|
|
8
8
|
|
|
9
|
+
Some claims also carry `disagreement_feedback`: several models answered the same query independently, and only `asserted_by` of the `of` answers made this claim. That is a reason to check the claim, not evidence that it is wrong, and it may be the only reason the claim is here. Check it against its grounding results. If they support the claim as written, return its text unchanged. If they do not, narrow it to what they do support. Do not mention the disagreement in the claim itself ("not all sources agree that…"): the run already records it, and hedging is not checking.
|
|
10
|
+
|
|
9
11
|
Respond with ONLY this JSON structure — one entry per flagged claim you were given:
|
|
10
12
|
|
|
11
13
|
```json
|
|
@@ -1,66 +1,66 @@
|
|
|
1
|
-
# Pass 3a — Source Grounding
|
|
2
|
-
|
|
3
|
-
You will receive a list of factual claims, and a deduplicated list of the unique entities/subjects those claims collectively reference (already deduplicated in code — you do not need to do this yourself).
|
|
4
|
-
|
|
5
|
-
For each unique entity in the deduplicated list, search for it ONCE using the tools available to you (web search, arxiv search, or fetching a specific URL, whichever fits the entity). Do not search once per claim — search once per unique entity, then apply what you find to every claim that referenced it. If the first search result is ambiguous or clearly insufficient, you may run one additional, more specific search for that same entity — but no more than two searches per entity.
|
|
6
|
-
|
|
7
|
-
Critical: derive your search query from the entity/subject itself, not from the claim's exact wording. If a claim says "the treaty was signed in 1947," search for the treaty's actual signing date as a topic — do not search for confirmation of that literal sentence. Searching for validation of your own starting point is how unverified claims end up looking "confirmed."
|
|
8
|
-
|
|
9
|
-
For each claim, determine a grounding status:
|
|
10
|
-
- `grounded`: one or more credible sources directly support the claim.
|
|
11
|
-
- `partial`: sources exist and are relevant, but don't fully confirm the specific claim as stated (e.g. confirm the general topic but not the specific number/date/detail).
|
|
12
|
-
- `ungrounded`: no credible source found supporting the claim, or search results contradict it.
|
|
13
|
-
|
|
14
|
-
Respond with ONLY this JSON structure — one entry per factual claim you were given:
|
|
15
|
-
|
|
16
|
-
```json
|
|
17
|
-
[
|
|
18
|
-
{
|
|
19
|
-
"claim_id": "c001",
|
|
20
|
-
"sources": [
|
|
21
|
-
{
|
|
22
|
-
"url": "...",
|
|
23
|
-
"quote": "...",
|
|
24
|
-
"similarity_score": 0.0,
|
|
25
|
-
"authority_adjustment": 0.0,
|
|
26
|
-
"authority_reason": "..."
|
|
27
|
-
}
|
|
28
|
-
],
|
|
29
|
-
"status": "grounded"
|
|
30
|
-
}
|
|
31
|
-
]
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
If status is `ungrounded`, `sources` may be an empty list.
|
|
35
|
-
|
|
36
|
-
## `quote` — what the source actually says
|
|
37
|
-
|
|
38
|
-
The verbatim span from that source which supports the claim: copy it exactly as it appeared, do not paraphrase it, do not correct its spelling, and do not stitch together text from two places. One to three sentences is the right length. This is the passage your `similarity_score` is about.
|
|
39
|
-
|
|
40
|
-
If you cannot quote the source — you are citing it from a search result you did not open, or the page had nothing quotable — use an empty string rather than reconstructing what you believe it said. An invented quote is worse than an absent one: quotes are checked against the retrieved text.
|
|
41
|
-
|
|
42
|
-
## `similarity_score` (0–1) — how closely that passage matches this claim
|
|
43
|
-
|
|
44
|
-
Score the passage you quoted. Not the page's title, not the search result's summary, and not your own memory of what this source says.
|
|
45
|
-
|
|
46
|
-
| score | the passage… |
|
|
47
|
-
|---|---|
|
|
48
|
-
| 0.9–1.0 | states the claim explicitly, including its specific number, date, name or quantity |
|
|
49
|
-
| 0.7–0.9 | states the claim, but with a different scope or qualifier (a range where the claim gives a point, one country where the claim says a region) |
|
|
50
|
-
| 0.5–0.7 | supports the general assertion but not the specific detail the claim turns on |
|
|
51
|
-
| 0.3–0.5 | is on the same topic and does not address the assertion |
|
|
52
|
-
| 0.0–0.3 | mentions the entity only, or is off-topic |
|
|
53
|
-
|
|
54
|
-
A source that is authoritative but does not discuss this specific claim scores LOW here. Authority and relevance are separate questions and this is the relevance one.
|
|
55
|
-
|
|
56
|
-
## `authority_adjustment` (−0.15 to +0.15) — a correction, not a score
|
|
57
|
-
|
|
58
|
-
How authoritative a source is, is computed in code from its domain and — for published research — its venue, citation standing and retraction status. You are not asked for that number. You are asked only whether the computed one is wrong about *this specific page*, in a way a domain cannot see.
|
|
59
|
-
|
|
60
|
-
Use `0.0` and an empty `authority_reason` unless one of these applies. This is the normal answer.
|
|
61
|
-
|
|
62
|
-
Raise it when the page is a primary source that its domain does not signal: an organisation's own filing, dataset, standard or specification; a manufacturer's own technical documentation for its own product; an author's own publication of their own results.
|
|
63
|
-
|
|
64
|
-
Lower it when the page is weaker than its domain suggests: an opinion column, sponsored or promotional content, a user-editable page, a press release restating someone else's finding, or a page that is clearly outdated for a claim about the present.
|
|
65
|
-
|
|
66
|
-
`authority_reason` must name which of these it is, in a few words, whenever the adjustment is not `0.0`. An adjustment with no reason is discarded.
|
|
1
|
+
# Pass 3a — Source Grounding
|
|
2
|
+
|
|
3
|
+
You will receive a list of factual claims, and a deduplicated list of the unique entities/subjects those claims collectively reference (already deduplicated in code — you do not need to do this yourself).
|
|
4
|
+
|
|
5
|
+
For each unique entity in the deduplicated list, search for it ONCE using the tools available to you (web search, arxiv search, or fetching a specific URL, whichever fits the entity). Do not search once per claim — search once per unique entity, then apply what you find to every claim that referenced it. If the first search result is ambiguous or clearly insufficient, you may run one additional, more specific search for that same entity — but no more than two searches per entity.
|
|
6
|
+
|
|
7
|
+
Critical: derive your search query from the entity/subject itself, not from the claim's exact wording. If a claim says "the treaty was signed in 1947," search for the treaty's actual signing date as a topic — do not search for confirmation of that literal sentence. Searching for validation of your own starting point is how unverified claims end up looking "confirmed."
|
|
8
|
+
|
|
9
|
+
For each claim, determine a grounding status:
|
|
10
|
+
- `grounded`: one or more credible sources directly support the claim.
|
|
11
|
+
- `partial`: sources exist and are relevant, but don't fully confirm the specific claim as stated (e.g. confirm the general topic but not the specific number/date/detail).
|
|
12
|
+
- `ungrounded`: no credible source found supporting the claim, or search results contradict it.
|
|
13
|
+
|
|
14
|
+
Respond with ONLY this JSON structure — one entry per factual claim you were given:
|
|
15
|
+
|
|
16
|
+
```json
|
|
17
|
+
[
|
|
18
|
+
{
|
|
19
|
+
"claim_id": "c001",
|
|
20
|
+
"sources": [
|
|
21
|
+
{
|
|
22
|
+
"url": "...",
|
|
23
|
+
"quote": "...",
|
|
24
|
+
"similarity_score": 0.0,
|
|
25
|
+
"authority_adjustment": 0.0,
|
|
26
|
+
"authority_reason": "..."
|
|
27
|
+
}
|
|
28
|
+
],
|
|
29
|
+
"status": "grounded"
|
|
30
|
+
}
|
|
31
|
+
]
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
If status is `ungrounded`, `sources` may be an empty list.
|
|
35
|
+
|
|
36
|
+
## `quote` — what the source actually says
|
|
37
|
+
|
|
38
|
+
The verbatim span from that source which supports the claim: copy it exactly as it appeared, do not paraphrase it, do not correct its spelling, and do not stitch together text from two places. One to three sentences is the right length. This is the passage your `similarity_score` is about.
|
|
39
|
+
|
|
40
|
+
If you cannot quote the source — you are citing it from a search result you did not open, or the page had nothing quotable — use an empty string rather than reconstructing what you believe it said. An invented quote is worse than an absent one: quotes are checked against the retrieved text.
|
|
41
|
+
|
|
42
|
+
## `similarity_score` (0–1) — how closely that passage matches this claim
|
|
43
|
+
|
|
44
|
+
Score the passage you quoted. Not the page's title, not the search result's summary, and not your own memory of what this source says.
|
|
45
|
+
|
|
46
|
+
| score | the passage… |
|
|
47
|
+
|---|---|
|
|
48
|
+
| 0.9–1.0 | states the claim explicitly, including its specific number, date, name or quantity |
|
|
49
|
+
| 0.7–0.9 | states the claim, but with a different scope or qualifier (a range where the claim gives a point, one country where the claim says a region) |
|
|
50
|
+
| 0.5–0.7 | supports the general assertion but not the specific detail the claim turns on |
|
|
51
|
+
| 0.3–0.5 | is on the same topic and does not address the assertion |
|
|
52
|
+
| 0.0–0.3 | mentions the entity only, or is off-topic |
|
|
53
|
+
|
|
54
|
+
A source that is authoritative but does not discuss this specific claim scores LOW here. Authority and relevance are separate questions and this is the relevance one.
|
|
55
|
+
|
|
56
|
+
## `authority_adjustment` (−0.15 to +0.15) — a correction, not a score
|
|
57
|
+
|
|
58
|
+
How authoritative a source is, is computed in code from its domain and — for published research — its venue, citation standing and retraction status. You are not asked for that number. You are asked only whether the computed one is wrong about *this specific page*, in a way a domain cannot see.
|
|
59
|
+
|
|
60
|
+
Use `0.0` and an empty `authority_reason` unless one of these applies. This is the normal answer.
|
|
61
|
+
|
|
62
|
+
Raise it when the page is a primary source that its domain does not signal: an organisation's own filing, dataset, standard or specification; a manufacturer's own technical documentation for its own product; an author's own publication of their own results.
|
|
63
|
+
|
|
64
|
+
Lower it when the page is weaker than its domain suggests: an opinion column, sponsored or promotional content, a user-editable page, a press release restating someone else's finding, or a page that is clearly outdated for a claim about the present.
|
|
65
|
+
|
|
66
|
+
`authority_reason` must name which of these it is, in a few words, whenever the adjustment is not `0.0`. An adjustment with no reason is discarded.
|