lyingdocs 0.1.2__tar.gz → 0.1.4__tar.gz

lyingdocs-0.1.4/PKG-INFO

@@ -0,0 +1,275 @@
+Metadata-Version: 2.4
+Name: lyingdocs
+Version: 0.1.4
+Summary: Autonomous documentation-code misalignment detection using LLM agents
+License-Expression: MIT
+License-File: LICENSE
+Requires-Python: >=3.11
+Requires-Dist: anthropic>=0.40
+Requires-Dist: openai>=1.0
+Requires-Dist: python-dotenv>=1.0
+Provides-Extra: dev
+Requires-Dist: build; extra == 'dev'
+Requires-Dist: pytest; extra == 'dev'
+Requires-Dist: ruff; extra == 'dev'
+Description-Content-Type: text/markdown
+
+<p align="center">
+  <img src="assets/logo.png" alt="LyingDocs" width="200" />
+</p>
+
+<h1 align="center">LyingDocs</h1>
+
+<p align="center">
+  A trust layer for your repository.
+</p>
+
+<p align="center">
+  Detect when your docs, code, configs, and examples stop agreeing with each other.
+</p>
+
+---
+
+Modern repositories are read by more than humans.
+
+They are read by teammates, new contributors, users, reviewers, downstream integrators — and increasingly by AI agents.
+
+That only works if the repository can be trusted.
+
+But trust quietly erodes over time:
+
+- documentation describes features that were never shipped
+- code behavior drifts away from the spec
+- examples stop matching reality
+- values claimed to be configurable are hardcoded deep in the codebase
+- papers and implementation tell different stories
+
+**LyingDocs is a trust layer for your repository.**  
+It audits the gap between what your repo *says* and what your code *actually does* — before your users, contributors, or agents learn the wrong thing.
+
+---
+
+## Why LyingDocs exists
+
+Every codebase accumulates invisible trust debt.
+
+In the age of fast iteration and LLM-assisted development, teams now ship code and documentation faster than ever — but not always in sync. A repo may still look polished while becoming progressively less reliable as a source of truth.
+
+That is the problem LyingDocs is built to solve.
+
+LyingDocs is not just a documentation checker. It is a system for surfacing **repository misalignment**:
+
+- docs that overclaim
+- code paths that are undocumented
+- specs that no longer match implementation
+- "configurable" behavior that is actually fixed
+- claims in papers or READMEs that cannot be supported by the code
+
+The goal is simple:
+
+> Keep your repository trustworthy for humans and machines.
+
+---
+
+## What LyingDocs does
+
+LyingDocs deploys two autonomous agents against your repository:
+
+- **Hermes** reads your documentation, plans an audit strategy, and decides what needs to be verified
+- **Argus** investigates the actual codebase and reports what the code really does
+
+Hermes then reconciles the two and writes a structured report of the mismatches it finds.
+
+This lets you catch cases where your repository is no longer telling the truth about itself.
+
+---
+
+## How it works
+
+### 1. Hermes reads what the repo claims
+
+Hermes traverses your documentation and extracts claims, assumptions, and implementation promises from sources such as:
+
+- docs/
+- README files
+- setup guides
+- usage examples
+- configuration references
+- papers and research writeups
+
+It then plans an audit by turning those claims into targeted investigation tasks.
+
+### 2. Argus checks what the code actually does
+
+Argus executes each task against your real codebase.
+
+You can choose the backend that best fits your setup:
+
+- **`codex`** — [OpenAI Codex CLI](https://github.com/openai/codex) subprocess
+- **`claude_code`** — [Claude Code](https://docs.anthropic.com/claude/docs/claude-code) CLI subprocess (`claude -p`)
+- **`local`** — built-in minimal agent loop using filesystem tools and any OpenAI-compatible API directly
+
+### 3. LyingDocs reports the trust gaps
+
+Hermes reconciles documented claims with observed implementation behavior and outputs a report of misalignments.
+
+These findings can then be reviewed by maintainers, turned into issues, and eventually enforced in CI.
+
+---
+
+## Positioning
+
+LyingDocs is best thought of as:
+
+- a **trust layer** for your repo
+- a **docs-to-code alignment guard**
+- a **pre-user warning system** for misleading documentation
+- a future **CI / GitHub Action quality gate** for repository truthfulness
+
+It is not meant to be a tool you manually open every day.
+
+It is meant to become something your repository runs automatically:
+
+- on pull requests
+- before releases
+- during scheduled audits
+- before docs deployment
+- as part of your GitHub Actions workflow
+
+---
+
+## Installation
+
+```bash
+pip install lyingdocs
+````
+
+---
+
+## Quick Start
+
+```bash
+export OPENAI_API_KEY="sk-..."
+
+lyingdocs analyze --doc-path docs/ --code-path . -o output/audit
+```
+
+This performs a full audit of your repository and produces a report describing where documentation and implementation no longer align.
+
+---
+
+## Documentation
+
+| | |
+| --- | --- |
+| [Configuration](docs/configuration.md) | Config file schema, environment variables, layer resolution |
+| [Argus Backends](docs/backends.md) | Setup for `local`, `codex`, and `claude_code` |
+| [CLI Reference](docs/cli.md) | All flags, commands, and output artifacts |
+| [GitHub Actions](docs/guides/github-actions.md) | CI integration, authentication, triggers, and approval gates |
+| [GitHub Issues](docs/guides/github-issues.md) | Using `--gen-issue` to draft and post issues |
+
+---
+
+## Example use cases
+
+Use LyingDocs when you want to answer questions like:
+
+* Does the README still reflect the real behavior of the project?
+* Are our examples and quickstarts still valid?
+* Did code change without the docs changing with it?
+* Are we claiming configuration that does not actually exist?
+* Does our paper describe behavior the implementation does not support?
+* Can an AI agent trust this repository as a source of truth?
+
+---
+
+## Misalignment categories
+
+| Category           | Description                                           |
+| ------------------ | ----------------------------------------------------- |
+| **LogicMismatch**  | Code contradicts documentation                        |
+| **PhantomSpec**    | Documentation describes non-existent features         |
+| **ShadowLogic**    | Important code behavior exists but is undocumented    |
+| **HardcodedDrift** | Supposedly configurable values are actually hardcoded |
+
+These categories represent different ways repository trust breaks down.
+
+---
+
+## GitHub Actions
+
+LyingDocs runs natively in GitHub Actions as a trust gate for your CI pipeline.
+
+```bash
+pip install lyingdocs
+lyingdocs init-ci --doc-path docs/ --backend claude_code --claude-oauth --trigger tag,manual
+```
+
+This generates a workflow that:
+
+* audits docs-vs-code alignment on every tag push (or on demand)
+* posts findings as a PR comment
+* optionally requires manual approval before merging
+
+Supports all three backends (`local`, `codex`, `claude_code`) and both API key and OAuth token authentication for Claude Code.
+
+See the [full setup guide](docs/guides/github-actions.md) for trigger options, approval gates, and backend configuration.
+
+---
+
+## Roadmap
+
+* [x] **Multi-harness support** — Argus runs on Codex, Claude Code, or a built-in local agent
+* [x] **Issue generation** — `--gen-issue` drafts GitHub issues from findings
+* [x] **GitHub Action integration** — `lyingdocs init-ci` generates a ready-to-use workflow with configurable triggers, backend selection, PR comments, and manual approval gates
+* [ ] **One-session memory support** — Argus backends retain state across tasks for deeper multi-step investigations
+* [ ] **Deeper analysis** — multi-hop reasoning across doc hierarchies and version-aware diffing to detect when code changed but docs did not
+* [ ] **Paper mode** — treat academic papers as documentation and detect paper-to-code misalignment
+* [ ] **Auto-fix mode** — Hermes proposes doc patches for human review and application
+
+---
+
+## For researchers
+
+A paper is also documentation.
+
+It is a human-language description of code, behavior, claims, and expected results — often written under deadline, and often drifting away from the implementation over time.
+
+If you want to know whether:
+
+* your repo matches your paper
+* your claims are supported by the code
+* another researcher can trust your implementation
+
+then LyingDocs can help.
+
+The problem is the same.
+Paper is documentation for code.
+LyingDocs is for papers too.
+
+---
+
+## Why “trust layer”
+
+Because the problem is bigger than stale docs.
+
+A repository becomes untrustworthy whenever its outward description and inward behavior drift apart.
+
+That harms:
+
+* users trying to adopt the project
+* contributors trying to extend it
+* maintainers trying to review changes
+* researchers trying to reproduce results
+* AI agents trying to understand the repo
+
+LyingDocs exists to make that gap visible.
+
+Not after users complain.
+Before.
+
+---
+
+## License
+
+MIT

lyingdocs-0.1.4/README.md

@@ -0,0 +1,259 @@
+<p align="center">
+  <img src="assets/logo.png" alt="LyingDocs" width="200" />
+</p>
+
+<h1 align="center">LyingDocs</h1>
+
+<p align="center">
+  A trust layer for your repository.
+</p>
+
+<p align="center">
+  Detect when your docs, code, configs, and examples stop agreeing with each other.
+</p>
+
+---
+
+Modern repositories are read by more than humans.
+
+They are read by teammates, new contributors, users, reviewers, downstream integrators — and increasingly by AI agents.
+
+That only works if the repository can be trusted.
+
+But trust quietly erodes over time:
+
+- documentation describes features that were never shipped
+- code behavior drifts away from the spec
+- examples stop matching reality
+- values claimed to be configurable are hardcoded deep in the codebase
+- papers and implementation tell different stories
+
+**LyingDocs is a trust layer for your repository.**  
+It audits the gap between what your repo *says* and what your code *actually does* — before your users, contributors, or agents learn the wrong thing.
+
+---
+
+## Why LyingDocs exists
+
+Every codebase accumulates invisible trust debt.
+
+In the age of fast iteration and LLM-assisted development, teams now ship code and documentation faster than ever — but not always in sync. A repo may still look polished while becoming progressively less reliable as a source of truth.
+
+That is the problem LyingDocs is built to solve.
+
+LyingDocs is not just a documentation checker. It is a system for surfacing **repository misalignment**:
+
+- docs that overclaim
+- code paths that are undocumented
+- specs that no longer match implementation
+- "configurable" behavior that is actually fixed
+- claims in papers or READMEs that cannot be supported by the code
+
+The goal is simple:
+
+> Keep your repository trustworthy for humans and machines.
+
+---
+
+## What LyingDocs does
+
+LyingDocs deploys two autonomous agents against your repository:
+
+- **Hermes** reads your documentation, plans an audit strategy, and decides what needs to be verified
+- **Argus** investigates the actual codebase and reports what the code really does
+
+Hermes then reconciles the two and writes a structured report of the mismatches it finds.
+
+This lets you catch cases where your repository is no longer telling the truth about itself.
+
+---
+
+## How it works
+
+### 1. Hermes reads what the repo claims
+
+Hermes traverses your documentation and extracts claims, assumptions, and implementation promises from sources such as:
+
+- docs/
+- README files
+- setup guides
+- usage examples
+- configuration references
+- papers and research writeups
+
+It then plans an audit by turning those claims into targeted investigation tasks.
+
+### 2. Argus checks what the code actually does
+
+Argus executes each task against your real codebase.
+
+You can choose the backend that best fits your setup:
+
+- **`codex`** — [OpenAI Codex CLI](https://github.com/openai/codex) subprocess
+- **`claude_code`** — [Claude Code](https://docs.anthropic.com/claude/docs/claude-code) CLI subprocess (`claude -p`)
+- **`local`** — built-in minimal agent loop using filesystem tools and any OpenAI-compatible API directly
+
+### 3. LyingDocs reports the trust gaps
+
+Hermes reconciles documented claims with observed implementation behavior and outputs a report of misalignments.
+
+These findings can then be reviewed by maintainers, turned into issues, and eventually enforced in CI.
+
+---
+
+## Positioning
+
+LyingDocs is best thought of as:
+
+- a **trust layer** for your repo
+- a **docs-to-code alignment guard**
+- a **pre-user warning system** for misleading documentation
+- a future **CI / GitHub Action quality gate** for repository truthfulness
+
+It is not meant to be a tool you manually open every day.
+
+It is meant to become something your repository runs automatically:
+
+- on pull requests
+- before releases
+- during scheduled audits
+- before docs deployment
+- as part of your GitHub Actions workflow
+
+---
+
+## Installation
+
+```bash
+pip install lyingdocs
+````
+
+---
+
+## Quick Start
+
+```bash
+export OPENAI_API_KEY="sk-..."
+
+lyingdocs analyze --doc-path docs/ --code-path . -o output/audit
+```
+
+This performs a full audit of your repository and produces a report describing where documentation and implementation no longer align.
+
+---
+
+## Documentation
+
+| | |
+| --- | --- |
+| [Configuration](docs/configuration.md) | Config file schema, environment variables, layer resolution |
+| [Argus Backends](docs/backends.md) | Setup for `local`, `codex`, and `claude_code` |
+| [CLI Reference](docs/cli.md) | All flags, commands, and output artifacts |
+| [GitHub Actions](docs/guides/github-actions.md) | CI integration, authentication, triggers, and approval gates |
+| [GitHub Issues](docs/guides/github-issues.md) | Using `--gen-issue` to draft and post issues |
+
+---
+
+## Example use cases
+
+Use LyingDocs when you want to answer questions like:
+
+* Does the README still reflect the real behavior of the project?
+* Are our examples and quickstarts still valid?
+* Did code change without the docs changing with it?
+* Are we claiming configuration that does not actually exist?
+* Does our paper describe behavior the implementation does not support?
+* Can an AI agent trust this repository as a source of truth?
+
+---
+
+## Misalignment categories
+
+| Category           | Description                                           |
+| ------------------ | ----------------------------------------------------- |
+| **LogicMismatch**  | Code contradicts documentation                        |
+| **PhantomSpec**    | Documentation describes non-existent features         |
+| **ShadowLogic**    | Important code behavior exists but is undocumented    |
+| **HardcodedDrift** | Supposedly configurable values are actually hardcoded |
+
+These categories represent different ways repository trust breaks down.
+
+---
+
+## GitHub Actions
+
+LyingDocs runs natively in GitHub Actions as a trust gate for your CI pipeline.
+
+```bash
+pip install lyingdocs
+lyingdocs init-ci --doc-path docs/ --backend claude_code --claude-oauth --trigger tag,manual
+```
+
+This generates a workflow that:
+
+* audits docs-vs-code alignment on every tag push (or on demand)
+* posts findings as a PR comment
+* optionally requires manual approval before merging
+
+Supports all three backends (`local`, `codex`, `claude_code`) and both API key and OAuth token authentication for Claude Code.
+
+See the [full setup guide](docs/guides/github-actions.md) for trigger options, approval gates, and backend configuration.
+
+---
+
+## Roadmap
+
+* [x] **Multi-harness support** — Argus runs on Codex, Claude Code, or a built-in local agent
+* [x] **Issue generation** — `--gen-issue` drafts GitHub issues from findings
+* [x] **GitHub Action integration** — `lyingdocs init-ci` generates a ready-to-use workflow with configurable triggers, backend selection, PR comments, and manual approval gates
+* [ ] **One-session memory support** — Argus backends retain state across tasks for deeper multi-step investigations
+* [ ] **Deeper analysis** — multi-hop reasoning across doc hierarchies and version-aware diffing to detect when code changed but docs did not
+* [ ] **Paper mode** — treat academic papers as documentation and detect paper-to-code misalignment
+* [ ] **Auto-fix mode** — Hermes proposes doc patches for human review and application
+
+---
+
+## For researchers
+
+A paper is also documentation.
+
+It is a human-language description of code, behavior, claims, and expected results — often written under deadline, and often drifting away from the implementation over time.
+
+If you want to know whether:
+
+* your repo matches your paper
+* your claims are supported by the code
+* another researcher can trust your implementation
+
+then LyingDocs can help.
+
+The problem is the same.
+Paper is documentation for code.
+LyingDocs is for papers too.
+
+---
+
+## Why “trust layer”
+
+Because the problem is bigger than stale docs.
+
+A repository becomes untrustworthy whenever its outward description and inward behavior drift apart.
+
+That harms:
+
+* users trying to adopt the project
+* contributors trying to extend it
+* maintainers trying to review changes
+* researchers trying to reproduce results
+* AI agents trying to understand the repo
+
+LyingDocs exists to make that gap visible.
+
+Not after users complain.
+Before.
+
+---
+
+## License
+
+MIT

{lyingdocs-0.1.2 → lyingdocs-0.1.4}/lyingdocs/argus.py

@@ -16,14 +16,9 @@ class ArgusDispatcher:
     def __init__(self, config: dict):
         self.config = config
         self.backend = config["argus_backend"]
-        self._resolved = False
         self._codex_bin: str | None = None
         self._claude_bin: str | None = None
 
-    def _resolve_once(self) -> None:
-        if self._resolved:
-            return
-        self._resolved = True
         if self.backend == "codex":
             self._codex_bin = find_codex_binary(self.config)
             if self._codex_bin:
@@ -56,8 +51,6 @@ class ArgusDispatcher:
         task_id: str,
         focus_paths: list[str] | None = None,
     ) -> str:
-        self._resolve_once()
-
         if self.backend == "codex":
             return run_codex_task(
                 self.config,

{lyingdocs-0.1.2 → lyingdocs-0.1.4}/lyingdocs/argus_claude_code.py

@@ -49,12 +49,24 @@ def run_claude_code_task(
         focus_section = f"\nPriority files/directories:\n{paths_str}\n"
 
     full_prompt = (
-        f"You are Argus, a code analyst.\n\n"
+        f"You are Argus, a code analyst. Your job is to verify a specific claim "
+        f"about a codebase by reading the actual source files.\n\n"
         f"Task: {task_description}\n"
         f"{focus_section}\n"
-        f"Investigate the code at {code_path.resolve()} and return a report with "
-        f"a verdict (confirmed / contradicted / not-found / partial) plus concrete "
-        f"file:line references and short quoted snippets."
+        f"Code root: {code_path.resolve()}\n\n"
+        f"Instructions:\n"
+        f"1. Explore the relevant code areas and READ the files that bear on the claim in full — "
+        f"do not rely on search snippets or filenames alone.\n"
+        f"2. Pay special attention to: mathematical formulas and equations (verify "
+        f"operator-by-operator), numerical constants and thresholds, algorithm control flow, "
+        f"and configuration key names with their defaults.\n"
+        f"3. Ground every conclusion in concrete file:line references with short quoted snippets.\n\n"
+        f"Return a report with:\n"
+        f"- A one-line verdict: confirmed / contradicted / not-found / partial\n"
+        f"- Specific file:line references for every claim\n"
+        f"- Short quoted code snippets as evidence\n"
+        f"- Any caveats or ambiguities\n\n"
+        f"Do NOT speculate. Only report what the code actually shows."
     )
 
     output_file = output_dir / f"argus_task_{task_id}.txt"

{lyingdocs-0.1.2 → lyingdocs-0.1.4}/lyingdocs/argus_local.py

@@ -9,7 +9,7 @@ import logging
 import re
 from pathlib import Path
 
-from .llm import call_llm_with_tools, make_client
+from .llm import LLMResponse, call_llm_with_tools, make_client
 
 logger = logging.getLogger("lyingdocs")
 
@@ -126,9 +126,11 @@ class LocalArgus:
     def __init__(self, config: dict, code_path: Path):
         self.config = config
         self.code_root = code_path.resolve()
+        self.provider = config.get("argus_provider", "openai")
         self.client = make_client(
             api_key=config["argus_api_key"],
             base_url=config["argus_base_url"],
+            provider=self.provider,
         )
         self.model = config["argus_model"]
         self.max_iterations = int(config.get("argus_local_max_iterations", 25))
@@ -149,7 +151,8 @@ class LocalArgus:
         for iteration in range(1, self.max_iterations + 1):
             logger.info("  Argus(local) iter %d/%d", iteration, self.max_iterations)
             response = call_llm_with_tools(
-                self.client, self.model, messages, ARGUS_LOCAL_TOOL_SCHEMAS
+                self.client, self.model, messages, ARGUS_LOCAL_TOOL_SCHEMAS,
+                provider=self.provider,
             )
             messages.append(self._response_to_message(response))