agentinputlab 0.1.0a1__py3-none-any.whl

workers/ircot/README.md

@@ -0,0 +1,167 @@
+# Pinned IRCoT worker
+
+This directory adapts the Apache-2.0 IRCoT repository at commit
+`3c1820f698eea5eeddb4fba3c56b64c961e063e4` to AgentInputLab's existing
+`IRCoTControlledWorker` contract. The upstream
+`BestFirstDecomposer.find_answer_decomp` controller remains responsible for
+the initial retrieval → CoT generation → exit check → follow-up retrieval
+loop. AgentInputLab only applies bounded controls, freezes external bindings
+for the test profile, observes controller transitions, and validates the
+resulting trajectory.
+
+## Identity profiles
+
+- `fixture.tiny.frozen.v1` is a manifest-locked `controlled_fixture`. It runs
+  the real upstream IRCoT loop against the local, checksum-pinned prompt,
+  retriever responses, and generator responses. It is deterministic, offline,
+  and may be called in-process through `PinnedIRCoTWorker`.
+- `upstream.hotpotqa.codex.v1` and
+  `upstream.hotpotqa.flan_t5_xxl.v1` are `upstream_native` identities. They
+  retain the pinned upstream HotpotQA config/prompt and require exact
+  generator, retriever, corpus, and index attestations. They must run in the
+  isolated JSONL process; the in-process adapter fails with
+  `ISOLATION_REQUIRED`.
+
+The historical clients do not expose complete token telemetry through the
+original generator interface. Their JSONL execution currently rejects with
+`TOKEN_USAGE_OBSERVER_REQUIRED` before any model request, rather than reporting
+unobserved token counts as zero. Source/identity inspection remains available.
+
+The fixture identity must not be reported as an upstream-native reproduction.
+Both identities, the upstream commit, manifest/config/prompt hashes, binding
+attestation, driver hash, and runtime-adapter hash are exposed in the worker
+fingerprint.
+
+## Fail-closed compatibility gate
+
+`compatibility_manifest.json` pins the upstream license and every source file
+used by the adapter. The driver rejects a wrong commit, dirty checkout,
+changed license/source/config/prompt/bindings, an unlisted profile, or an
+unattested production binding before execution.
+
+The historical upstream environment also needs optional packages and model
+assets that are deliberately not hidden by the adapter. Identity inspection
+reports `full_api_status=dependency_missing:<package>` when the complete
+upstream API cannot import. A production call returns a structured
+`DEPENDENCY_MISSING` response (for example when `_jsonnet` is absent) rather
+than substituting another implementation. The frozen fixture supplies only
+small compatibility shims needed to execute its offline trajectory; these do
+not apply to an upstream-native profile.
+
+Production endpoints and fingerprint hashes come only from the environment
+keys named in the manifest. Request JSON cannot provide a path, module,
+command, endpoint, or credential, and fingerprints containing credential
+fields are rejected.
+
+## Usage
+
+For the offline bridge fixture, point the runtime at an exact pinned checkout:
+
+```python
+from pathlib import Path
+
+from workers.ircot import PinnedIRCoTWorker
+
+worker = PinnedIRCoTWorker(
+    environment={
+        "AGENTINPUTLAB_UPSTREAM_ROOT": str(Path("/path/to/pinned/ircot")),
+    }
+)
+```
+
+Pass `worker` to `IRCoTControlledWorker`; its `call_controlled` method accepts
+the bridge's immutable `IRCoTWorkerRequest` directly.
+
+For a production profile, start one JSONL worker process with its environment
+prepared outside the request boundary:
+
+```bash
+AGENTINPUTLAB_UPSTREAM_ROOT=/path/to/pinned/ircot \
+AGENTINPUTLAB_IRCOT_PROFILE=upstream.hotpotqa.codex.v1 \
+python -m workers.ircot.driver
+```
+
+Use AgentInputLab's `JsonLineSubprocessWorker` as the client for that process.
+The production environment must additionally provide the exact index and
+binding hashes plus the model/retriever settings named by the selected
+manifest profile. Secrets belong only in the isolated process environment;
+they must never be included in request JSON or in a public fingerprint.
+
+## Startup-bound controlled reader transfer
+
+`PinnedIRCoTControlledRuntime` binds a toolkit backend, retriever, actual GPT-2
+tokenizer and dependency fingerprints once at process startup. It executes the
+pinned HotpotQA Codex configuration through the original
+`build_decomposer_and_models` and `BestFirstDecomposer.find_answer_decomp`, with
+the original retrieval, CoT sentence and exit participants. Native prompt
+selection/fitting, spaCy sentence parsing, query preprocessing, deduplication
+and exit decisions are preserved. The generator and retriever are controlled
+replacements, so this is **not** historical Codex/Elasticsearch result parity.
+It is also distinct from the one-pass common-support adapter.
+
+Run each worker in its own process with an explicit private `HOME` equal to
+`cache_home`; upstream import-time caches then cannot touch research caches.
+The mounted source must remain at the manifest-pinned commit with a clean tree.
+Provide real native dependencies, the native `en_core_web_sm` package, and a
+hash-closed GPT-2 tokenizer; there are no fixture dependency shims or implicit
+asset downloads in this adapter. Those startup fingerprints describe supplied
+artifacts; they do not establish authorship, license clearance or real-model
+acceptance merely by being present.
+
+Use an isolated **Python 3.10** environment and install `agentinputlab[ircot]`.
+This legacy worker extra pins spaCy 3.4.1 and Transformers 4.24.0; it is not a
+promise that this optional environment supports every Python version supported
+by the dependency-light toolkit core. It is separate from the ACE/LLMCompiler
+environments, whose native client versions differ. The controlled reader is a
+remote backend, so this extra does not install Torch or backbone weights. Mount
+the matching `en_core_web_sm` 3.4.1 pipeline, GPT-2 tokenizer files and NLTK
+English stopwords separately, recording their sources/checksums. No corpus or
+model-weight download is triggered by importing the controlled adapter.
+
+```python
+from workers.ircot import PinnedIRCoTControlledRuntime
+from agentinputlab.algorithms.acquire.ircot import IRCoTControlledWorker, IRCoTControls
+
+worker = PinnedIRCoTControlledRuntime(
+    source_root=pinned_checkout, backend=backend, retriever=retriever,
+    tokenizer=gpt2_tokenizer, tokenizer_fingerprint=trusted_tokenizer_manifest,
+    dependency_fingerprint=native_dependency_manifest,
+    corpus_revision=corpus_sha256, index_revision=index_sha256,
+    cache_home=private_worker_home, seed=17,
+    max_model_calls=7, max_output_tokens=3584, per_call_output_tokens=512,
+)
+method = IRCoTControlledWorker(
+    client=worker, corpus_revision=corpus_sha256, index_revision=index_sha256,
+    prompt_hash=worker.fingerprint()["prompt_sha256"],
+    controls=IRCoTControls(max_steps=7, max_output_tokens=3584),
+)
+```
+
+This example reserves a separate downstream call/output allowance within an
+overall eight-call/4096-output-token application budget. The caller must debit
+both stages; the worker does not make the final downstream prediction. Generation
+seeds are startup `seed + call_index`, not a request-selected seed. For reasoning
+backends, explicitly choose `FinalContentStopBackend` to apply the native logical
+newline stop to final content while charging all provider-generated tokens.
+No retry, answer extraction call, or reasoning-to-final promotion is added.
+
+Each response retains native controller events, exact actual generation and
+retrieval records, and provider-observed tokens. If a transport fails without
+usage, the error response marks `usage_is_complete=false` and records the number
+of unknown-usage attempts separately; reported token sums are observed lower
+bounds, not claims of zero consumption. Callers must preserve those error fields
+and raw records. Default CPU/GPU counters are not measured compute costs.
+
+## Verification
+
+```bash
+python -m pytest -q \
+  tests/unit/test_ircot_worker.py \
+  tests/workers/test_ircot_worker_driver.py
+python -m ruff check workers/ircot tests/workers/test_ircot_worker_driver.py
+```
+
+The worker test builds a fake pinned checkout for hermetic bridge E2E coverage.
+When `/private/tmp/agentinputlab-upstreams/ircot` is available, it additionally
+checks the real pinned checkout's API and executes the frozen fixture through
+the original upstream loop.

workers/ircot/__init__.py

@@ -0,0 +1,7 @@
+"""Pinned IRCoT worker and direct WorkerClient adapter."""
+
+from .controlled import PinnedIRCoTControlledRuntime
+from .native_qa import PinnedIRCoTNativeQARuntime
+from .runtime import PinnedIRCoTWorker
+
+__all__ = ["PinnedIRCoTControlledRuntime", "PinnedIRCoTNativeQARuntime", "PinnedIRCoTWorker"]

workers/ircot/compatibility_manifest.json

@@ -0,0 +1,121 @@
+{
+  "schema_version": "agentinputlab.ircot.compatibility.v1",
+  "upstream": {
+    "repository": "https://github.com/StonyBrookNLP/ircot",
+    "commit": "3c1820f698eea5eeddb4fba3c56b64c961e063e4",
+    "license_spdx": "Apache-2.0",
+    "license_sha256": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
+    "require_clean_checkout": true
+  },
+  "source_files": {
+    "commaqa/inference/configurable_inference.py": "6d19f148ef0637707d1dd481e604944144eb296754c88429706a514b111db54d",
+    "commaqa/inference/constants.py": "4482a7eb15e107e72f364b7357a8963662a063a730aa3861a6b51e3018d99570",
+    "commaqa/inference/data_instances.py": "018a4addbb5d6fb81c76af049e1ae1e50ac227ab48ee7a69a9cb453b00ff6317",
+    "commaqa/inference/ircot.py": "a2233cbdae107e80cd383993edadc43c2491d2360a878b05a97dfa66a0b0eebf",
+    "commaqa/inference/model_search.py": "c22ddb376bd165a0872da11fa0310cda8c25e00c62d9efa3e47cd36371889558",
+    "commaqa/inference/prompt_reader.py": "910b6e971edd00460a570260d669128163965f2f61fe230db64be055131d311e"
+  },
+  "allowed_model_names": [
+    "retrieve_and_reset_paragraphs",
+    "step_by_step_cot_gen",
+    "step_by_step_exit_controller"
+  ],
+  "profiles": {
+    "fixture.tiny.frozen.v1": {
+      "identity_kind": "controlled_fixture",
+      "config_scope": "worker",
+      "config_path": "fixture_config.json",
+      "config_sha256": "67cd9ed5011bd06b4f5b636a7e13c68095dd0f5c9d3482b1c0fba74ad63a9891",
+      "prompt_scope": "worker",
+      "prompt_path": "fixture_prompt.txt",
+      "prompt_sha256": "ec8cb750660f2bc7ed8718673e0a865582d273710ad6e44ee6403ffcec1753cb",
+      "bindings_scope": "worker",
+      "bindings_path": "fixture_bindings.json",
+      "bindings_sha256": "daa93b132cea7cdcf88cdf70c0bb6827a75dd39732ceb9d13bf4bba2867bcfbb",
+      "corpus_revision": "fixture-corpus-v1",
+      "index_revision": "fixture-index-v1",
+      "binding_attestation": "fixture_identity_only",
+      "config_ext_vars": {}
+    },
+    "upstream.hotpotqa.codex.v1": {
+      "identity_kind": "upstream_native",
+      "config_scope": "upstream",
+      "config_path": "base_configs/ircot_codex_hotpotqa.jsonnet",
+      "config_sha256": "36cdb0023d1f59266486c463a53bb4546a2364114672db8e83faac0a6d3645e3",
+      "prompt_scope": "upstream",
+      "prompt_path": "prompts/hotpotqa/gold_with_2_distractors_context_cot_qa_codex.txt",
+      "prompt_sha256": "af56b51582a5fdbebf5a5d50ab95cefcb95d92986cf820774ccbaa897a71552f",
+      "corpus_revision": "hotpotqa",
+      "index_revision_env": "AGENTINPUTLAB_IRCOT_INDEX_REVISION",
+      "binding_attestation": "exact_fingerprint_hashes_from_environment",
+      "generator_fingerprint_sha256_env": "AGENTINPUTLAB_IRCOT_GENERATOR_FINGERPRINT_SHA256",
+      "retriever_fingerprint_sha256_env": "AGENTINPUTLAB_IRCOT_RETRIEVER_FINGERPRINT_SHA256",
+      "config_ext_vars": {
+        "RETRIEVER_HOST": "AGENTINPUTLAB_IRCOT_RETRIEVER_HOST",
+        "RETRIEVER_PORT": "AGENTINPUTLAB_IRCOT_RETRIEVER_PORT"
+      },
+      "required_runtime_env": [
+        "OPENAI_API_KEY"
+      ]
+    },
+    "upstream.hotpotqa.flan_t5_xxl.v1": {
+      "identity_kind": "upstream_native",
+      "config_scope": "upstream",
+      "config_path": "base_configs/ircot_flan_t5_xxl_hotpotqa.jsonnet",
+      "config_sha256": "a2bb710cf7de45b3f148ee8eac9c7b7a09aa84979375219c48f8caf44fd6c9ac",
+      "prompt_scope": "upstream",
+      "prompt_path": "prompts/hotpotqa/gold_with_2_distractors_context_cot_qa_flan_t5.txt",
+      "prompt_sha256": "bcc08b7b4d2158b625be0825d5852d5ca013e65832882056931b93d322055db1",
+      "corpus_revision": "hotpotqa",
+      "index_revision_env": "AGENTINPUTLAB_IRCOT_INDEX_REVISION",
+      "binding_attestation": "exact_fingerprint_hashes_from_environment",
+      "generator_fingerprint_sha256_env": "AGENTINPUTLAB_IRCOT_GENERATOR_FINGERPRINT_SHA256",
+      "retriever_fingerprint_sha256_env": "AGENTINPUTLAB_IRCOT_RETRIEVER_FINGERPRINT_SHA256",
+      "config_ext_vars": {
+        "RETRIEVER_HOST": "AGENTINPUTLAB_IRCOT_RETRIEVER_HOST",
+        "RETRIEVER_PORT": "AGENTINPUTLAB_IRCOT_RETRIEVER_PORT"
+      },
+      "required_runtime_env": [
+        "LLM_SERVER_HOST",
+        "LLM_SERVER_PORT"
+      ]
+    }
+  },
+  "patches": [
+    {
+      "id": "controlled_limits_v1",
+      "upstream_symbols": [
+        "RetrieveAndResetParagraphsParticipant",
+        "StepByStepCOTGenParticipant",
+        "StepByStepExitControllerParticipant"
+      ],
+      "kind": "configuration_compatibility_patch",
+      "changes": [
+        "cap per-round retrieval, total active documents, reasoning sentences, and per-call generation",
+        "map replace_each_step to upstream cumulate_titles=false",
+        "reject deduplicate=false because the pinned upstream always de-duplicates"
+      ]
+    },
+    {
+      "id": "controller_trace_v1",
+      "upstream_symbol": "BestFirstDecomposer.find_answer_decomp",
+      "kind": "non_mutating_observability_wrapper",
+      "changes": [
+        "leave upstream best-first loop unchanged",
+        "snapshot controller inputs and outputs to reconstruct retrieval/CoT trajectory",
+        "capture accepted upstream retrieval responses without changing participant decisions"
+      ]
+    },
+    {
+      "id": "binding_gate_v1",
+      "upstream_symbol": "configurable_inference.load_config",
+      "kind": "security_patch",
+      "changes": [
+        "load only checksum-pinned config and prompt artifacts",
+        "read production endpoints only from manifest-named worker environment keys",
+        "require exact generator/retriever fingerprint hashes for upstream-native profiles",
+        "never accept request-controlled paths, imports, endpoints, commands, or credentials"
+      ]
+    }
+  ]
+}