@yahyahammad/venastine 0.3.17 → 0.3.18

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1058,7 +1058,7 @@ Deliberately not settings.json keys: editing these means editing the harness's o
1058
1058
 
1059
1059
  **Built:** ROADMAP.md §1–§12 (§10's ensemble mode rebuilt by its revisit — diversity now comes from a roster of different models) and ROADMAP_v2.md §13 (streaming loop), §14 (config loader + workspace trust), §15 (permissions), §16 (TUI), §17 (MCP), §18 (agents), §19 (skills), §20 (post-pipeline review), §21a (compaction + `pin`), §21b (durable memory), §22 (live pipeline progress), §25 (authorised tool use in the pipeline), §26 (research legibility: per-pass tool calls, code-stage announcements, the claims view, role colour, `/copy`), §27 (thread legibility: transcript replay on resume, a conversations-only thread picker), §21c (session summaries and cross-thread referencing — §21 is now complete), §24 (`/init`), §23 (interactive tools: the response channel, per-tool subagent sign-off, `ask_user`, and the todo list), §28 (the shell capability classifier).
1060
1060
 
1061
- **Remaining:** nothing in `ROADMAP.md` or `ROADMAP_v2.md`, and §10's revisit is now closed. `ROADMAP_v3.md` §49–§59, §61–§93, §95, §96 and §97 are built. The section still to build is §60 (Unified Harness Protocol compliance), specified and awaiting detailed design. §56 lets a decision model approve a single contained shell command under the opt-in `classified` mode; §54 completes project file/folder mentions without loading contents; §58 supplies a short harness identity that follows the current model; §83's final live checks are confirmed, and the ChatGPT header distinguishes a selected provider from an active sign-in. `TECHNICAL_DEBT.md` items 14, 17, 18, 21, 102, 116, 117, 118 and 119 are open, and so is the secrets half of item 97. The two checks recorded against §10 are answered from the providers' documentation in `ROADMAP_v3.md` §87: the default `temperature` on OpenAI and Google is 1.0, and OpenAI's reasoning models refuse one whenever reasoning effort is not `none`, which the guard now follows.
1061
+ **Remaining:** nothing in `ROADMAP.md` or `ROADMAP_v2.md`, and §10's revisit is now closed. `ROADMAP_v3.md` §49–§59, §61–§93, §95, §96 and §97 are built. The section still to build is §60 (Unified Harness Protocol compliance), specified and awaiting detailed design. §56 lets a decision model approve a single contained shell command under the opt-in `classified` mode; §54 completes project file/folder mentions without loading contents; §58 supplies a short harness identity that follows the current model; §83's final live checks are confirmed, and the ChatGPT header distinguishes a selected provider from an active sign-in. `TECHNICAL_DEBT.md` items 14, 17, 18, 21, 102, 116, 118, 119 and 120 are open, and so is the secrets half of item 97. The two checks recorded against §10 are answered from the providers' documentation in `ROADMAP_v3.md` §87: the default `temperature` on OpenAI and Google is 1.0, and OpenAI's reasoning models refuse one whenever reasoning effort is not `none`, which the guard now follows.
1062
1062
 
1063
1063
  **And both audits are closed.** Both roadmaps being built is not the same as the code
1064
1064
  being clean, and saying only the first would be the "built and runs are different claims"
@@ -1077,7 +1077,7 @@ note that shell approval is governed solely by `shell_approval_mode` -- `tool_ap
1077
1077
  deliberately has no `shell` key, so an old file carrying one is refused at startup
1078
1078
  naming the key — see `tests/BREAKING_CHANGES.md` §24.
1079
1079
 
1080
- Run the test suite with `pytest` — 9770 tests, fully offline, no API keys needed. Since batch 243 the suite holds itself to that: a test that asks DNS for a name, or writes into your home folder, fails by name, and every test process runs with HOME pointing at a throwaway folder of its own. The suite's own packages are `requirements-test.txt`, separate from `requirements.txt` since batch 136: they used to be in the shipped dependency list, so installing the harness also installed a test runner. It runs in parallel by default (`pytest-xdist`, 70% of your cores): **1:26 on a 16-core machine, against 9:06 single-process.** `pytest -n0` forces the old serial run, which is what you want under a debugger or when something is parsing the output. Eight further tests are marked `integration` and excluded by default: three spawn a real stdio MCP server, one runs an external program through the host fallback shell, one builds the hosted bundle's Docker image, and three run the installers' uv path against a real uv (`pytest -m integration`).
1080
+ Run the test suite with `pytest` — 9829 tests, fully offline, no API keys needed. Since batch 243 the suite holds itself to that: a test that asks DNS for a name, or writes into your home folder, fails by name, and every test process runs with HOME pointing at a throwaway folder of its own. The suite's own packages are `requirements-test.txt`, separate from `requirements.txt` since batch 136: they used to be in the shipped dependency list, so installing the harness also installed a test runner. It runs in parallel by default (`pytest-xdist`, 70% of your cores): **1:26 on a 16-core machine, against 9:06 single-process.** `pytest -n0` forces the old serial run, which is what you want under a debugger or when something is parsing the output. Eight further tests are marked `integration` and excluded by default: three spawn a real stdio MCP server, one runs an external program through the host fallback shell, one builds the hosted bundle's Docker image, and three run the installers' uv path against a real uv (`pytest -m integration`).
1081
1081
 
1082
1082
  ## Documentation
1083
1083
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@yahyahammad/venastine",
3
- "version": "0.3.17",
3
+ "version": "0.3.18",
4
4
  "description": "A from-scratch agentic harness: tool-using chat plus a ten-pass self-critiquing research pipeline.",
5
5
  "keywords": [
6
6
  "agent",
package/pyproject.toml CHANGED
@@ -18,7 +18,7 @@ build-backend = "setuptools.build_meta"
18
18
 
19
19
  [project]
20
20
  name = "venastine-research-harness"
21
- version = "0.3.17"
21
+ version = "0.3.18"
22
22
  description = "A from-scratch agentic harness: tool-using chat plus a ten-pass self-critiquing research pipeline."
23
23
  readme = "README.md"
24
24
  # The same words as package.json's `keywords`, in the same order: what a
@@ -234,6 +234,17 @@ _PEM_SEP = _PEM_BREAK + r"+[ \t]*"
234
234
  # key carries before its base64.
235
235
  _PEM_UNIT = r"(?:[A-Za-z0-9+/=]{16,}|(?:Proc-Type|DEK-Info):[^\r\n\\]*)"
236
236
 
237
+ # A token character that CLOSES AN ESCAPE, so a pattern that refuses to start
238
+ # after a token character still starts here: the second hex digit of `%3D`
239
+ # in a URL-encoded value, and the letter of `\n`, `\r` or `\t` in a
240
+ # JSON-escaped one. Batch 260 wrote it for GitHub's stateless form. Batch 268
241
+ # measured the same miss in every other pattern anchored that way: `hf_`,
242
+ # `ms-` and the JWT each passed whole after `%3D`, `\n` and `\t`, and the
243
+ # `Authorization` header after `\n`. They share this one copy. Each such
244
+ # start sits two characters after a `%` or a `\`, which none of their runs
245
+ # contains, so a pattern that was linear stays linear.
246
+ _AFTER_ESCAPE = r"(?<=%[0-9A-Fa-f]{2})|(?<=\\[nrt])"
247
+
237
248
  _SECRET_PATTERNS = [
238
249
  # OpenAI-style keys
239
250
  re.compile(r"sk-[a-zA-Z0-9]{20,}"),
@@ -292,14 +303,14 @@ _SECRET_PATTERNS = [
292
303
  # long run is a start that scans to the run's end: measured quadratic,
293
304
  # 99 s on a hostile megabyte against 0.03 s with it. A token character
294
305
  # is allowed before the prefix only where it closes an escape, `%3D`
295
- # or `\n`, since a URL-encoded or JSON-escaped token is still a token,
296
- # and the classic pattern catches those. Each of those starts sits two
297
- # characters after a `%` or a `\`, which no run contains, so the scan
298
- # stays linear.
306
+ # or `\n` (`_AFTER_ESCAPE`), since a URL-encoded or JSON-escaped token
307
+ # is still a token, and the classic pattern catches those. Each of those
308
+ # starts sits two characters after a `%` or a `\`, which no run
309
+ # contains, so the scan stays linear.
299
310
  # This pattern runs BEFORE the classic one, because a 36-character
300
311
  # alphanumeric run inside a segment would otherwise take a piece of
301
312
  # the token and leave the rest standing.
302
- re.compile(r"(?:(?<![A-Za-z0-9_-])|(?<=%[0-9A-Fa-f]{2})|(?<=\\[nrt]))"
313
+ re.compile(r"(?:(?<![A-Za-z0-9_-])|" + _AFTER_ESCAPE + r")"
303
314
  r"gh[ousr]_[A-Za-z0-9_-]{20,}\.[A-Za-z0-9_-]{20,}"
304
315
  r"\.[A-Za-z0-9_-]{20,}"),
305
316
  # GitHub OAuth, user-to-server, server-to-server and refresh tokens
@@ -316,14 +327,21 @@ _SECRET_PATTERNS = [
316
327
  # --- the rest of the list.
317
328
  # Hugging Face user access tokens and OAuth access tokens. An
318
329
  # identifier like `hf_hub_download` breaks the run of word characters
319
- # at its second underscore and is not matched.
320
- re.compile(r"(?<![A-Za-z0-9])hf_(?:oauth_)?[A-Za-z0-9]{30,}"),
330
+ # at its second underscore and is not matched. An escape's closing
331
+ # character may come before the prefix (`_AFTER_ESCAPE`, batch 268):
332
+ # before that, `%3Dhf_...` and `\nhf_...` were neither redacted nor
333
+ # refused.
334
+ re.compile(r"(?:(?<![A-Za-z0-9])|" + _AFTER_ESCAPE + r")"
335
+ r"hf_(?:oauth_)?[A-Za-z0-9]{30,}"),
321
336
  # OpenRouter keys, which `sk-` above misses at the second hyphen
322
337
  re.compile(r"sk-or-v1-[A-Za-z0-9]{32,}"),
323
338
  # ModelScope access tokens in their prefixed form: `ms-` and a UUID. A
324
339
  # bare UUID is NOT matched -- thread ids are UUIDs -- and neither is one
325
- # that merely ENDS a word like `items-<uuid>`.
326
- re.compile(r"(?<![A-Za-z0-9])ms-[0-9a-f]{8}-[0-9a-f]{4}"
340
+ # that merely ENDS a word like `items-<uuid>`. An escape's closing
341
+ # character is not a word (`_AFTER_ESCAPE`, batch 268), so `%3Dms-...`
342
+ # and `\nms-...` are matched, as `hf_` is.
343
+ re.compile(r"(?:(?<![A-Za-z0-9])|" + _AFTER_ESCAPE + r")"
344
+ r"ms-[0-9a-f]{8}-[0-9a-f]{4}"
327
345
  r"-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"),
328
346
  ]
329
347
 
@@ -426,19 +444,26 @@ _ASSIGNMENT_RE = re.compile(
426
444
  # `Digest username=...`, whose secret is not one value, is left alone. A
427
445
  # value with NO scheme word must also be 16 characters and carry a digit,
428
446
  # because API documentation writes "Authorization: required" and a bare
429
- # 8-character floor redacted the word.
447
+ # 8-character floor redacted the word. The header may follow an escape's
448
+ # closing character (`_AFTER_ESCAPE`, batch 268), case-sensitively, since
449
+ # `\b` fails between the `n` of a JSON-escaped `\n` and the header name: a
450
+ # captured request's `\nAuthorization: Bearer ...` passed whole before it.
430
451
  # Groups: 1 header name, 2 separator, 3 scheme (or None).
431
452
  _AUTH_HEADER_RE = re.compile(
432
- r"(?i)\b((?:proxy-)?authorization)([\"']?\s*[:=]\s*[\"']?)"
453
+ r"(?i)(?:\b|(?-i:" + _AFTER_ESCAPE + r"))"
454
+ r"((?:proxy-)?authorization)([\"']?\s*[:=]\s*[\"']?)"
433
455
  r"(?:((?:bearer|basic|token)\s+)[A-Za-z0-9._~+/=\-]{8,}"
434
456
  r"|(?=[A-Za-z._~+/=\-]*[0-9])[A-Za-z0-9._~+/=\-]{16,})")
435
457
 
436
458
  # A JSON Web Token, whole: three base64url segments, the first two of
437
459
  # which open with `eyJ` because both are JSON objects (`{"`). A bearer
438
460
  # credential wherever it appears, and the shape `_AUTH_HEADER_RE` cannot
439
- # reach when the token is pasted without its header.
461
+ # reach when the token is pasted without its header. It may follow an
462
+ # escape's closing character (`_AFTER_ESCAPE`, batch 268), where `\b` fails:
463
+ # `%3DeyJ...` and `\neyJ...` passed whole before that.
440
464
  _JWT_RE = re.compile(
441
- r"\beyJ[A-Za-z0-9_-]{10,}\.eyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}")
465
+ r"(?:\b|" + _AFTER_ESCAPE + r")"
466
+ r"eyJ[A-Za-z0-9_-]{10,}\.eyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}")
442
467
 
443
468
  # The END of a PEM key block whose BEGIN is not in this string: the footer
444
469
  # and the run of base64 lines above it. The vendor pattern takes a block
@@ -1065,7 +1090,23 @@ def check_output_policy(tool_name: str, result: dict) -> dict:
1065
1090
  # claims and reports constantly, and refusing every mention would make the
1066
1091
  # blocklist unusable while blocking nothing an attacker couldn't rephrase.
1067
1092
  # fetch_url.py keeps its own bare-domain check for the value it is given.
1068
- _URL_RE = re.compile(r"[a-zA-Z][a-zA-Z0-9+.\-]*://[^\s\"'<>]+")
1093
+ #
1094
+ # The scan starts once per RUN of scheme characters (TECHNICAL_DEBT 117,
1095
+ # batch 268). It used to be `[a-zA-Z][a-zA-Z0-9+.\-]*://...`, tried at every
1096
+ # letter, and on one long run of letters and digits each try scanned to the
1097
+ # run's end before failing to find `://`. That was quadratic: 4.8 s at
1098
+ # 100 KB and 441 s at a megabyte, before the handler ran. Now the lookbehind
1099
+ # refuses every start inside a run, the run's leading digits and `+.-` are
1100
+ # skipped, and the group captures from the run's first letter, which is
1101
+ # where the old pattern's leftmost match began. So `findall`, which returns
1102
+ # that one group, gives exactly the list it gave before (`0https://h` still
1103
+ # gives `https://h`), and the blocklist reads the same strings. Measured:
1104
+ # 0.05 s or less on each of ten hostile megabytes, and the old pattern's
1105
+ # list on 303,000 random strings. The shape the register recorded, a scheme
1106
+ # bounded to 32 characters, was linear too, but it found nothing in `a` and
1107
+ # 40 digits before `://evil.com`, where today's scan finds the host.
1108
+ _URL_RE = re.compile(
1109
+ r"(?<![a-zA-Z0-9+.\-])[0-9+.\-]*([a-zA-Z][a-zA-Z0-9+.\-]*://[^\s\"'<>]+)")
1069
1110
 
1070
1111
 
1071
1112
  def _input_leaf(text: str) -> str: