clinescope 1.0.1__tar.gz

clinescope-1.0.1/.gitattributes

@@ -0,0 +1,12 @@
+# Normalize line endings to LF for text so the checked-out form is identical on
+# every platform. The gold set + example traces are a versioned contract read by a
+# byte-preserving label writer (src/clinescope/label_gold.py); a mixed CRLF/LF
+# checkout would make that writer's "leave other lines byte-identical" guarantee
+# platform-dependent. Pinning LF makes it hold on Windows and Linux/CI alike.
+* text=auto eol=lf
+
+# Explicit for the contract files (belt-and-suspenders; the wildcard already covers them).
+*.jsonl text eol=lf
+*.json  text eol=lf
+*.py    text eol=lf
+*.md    text eol=lf

clinescope-1.0.1/.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,63 @@
+name: Bug report
+description: A scorer, the CLI, or the trace loader misbehaves on a real trace.
+title: "[bug]: "
+labels: ["bug"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for the report. clinescope scores a Cline `messages.json` trace, so a bug is usually
+        reproducible from a trace + the command you ran. Please include both.
+  - type: textarea
+    id: what-happened
+    attributes:
+      label: What happened
+      description: What did clinescope do, and what did you expect instead?
+      placeholder: "e.g. diff_coherence scored 0.0 on a patch that applies cleanly in Cline."
+    validations:
+      required: true
+  - type: input
+    id: command
+    attributes:
+      label: Command you ran
+      placeholder: "clinescope path/to/messages.json --expected read_files apply_patch"
+    validations:
+      required: true
+  - type: dropdown
+    id: trace-source
+    attributes:
+      label: Trace source
+      description: Where did the trace come from?
+      options:
+        - A real Cline CLI run
+        - A real Cline VS Code session
+        - A hand-authored / synthetic trace
+        - The bundled examples/
+        - Other (describe below)
+    validations:
+      required: true
+  - type: textarea
+    id: expected-vs-actual
+    attributes:
+      label: Expected vs actual score
+      description: Which scorer, the number you expected, and the number you got.
+      placeholder: "diff_coherence — expected 1.0, got 0.0."
+    validations:
+      required: true
+  - type: textarea
+    id: repro
+    attributes:
+      label: Minimal trace to reproduce
+      description: >
+        Paste the smallest trace that triggers it (a `messages.json` snippet). Do NOT paste secrets
+        or anything you can't share publicly. Never include Cline's own golden fixture.
+      render: json
+    validations:
+      required: false
+  - type: input
+    id: environment
+    attributes:
+      label: Environment
+      placeholder: "clinescope version (or commit), Python version, OS"
+    validations:
+      required: true

clinescope-1.0.1/.github/ISSUE_TEMPLATE/scorer_request.yml

@@ -0,0 +1,48 @@
+name: New scorer / metric request
+description: Propose a new eval scorer or metric for clinescope to compute from a trace.
+title: "[scorer]: "
+labels: ["enhancement", "scorer"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        clinescope scores a coding-agent run from its trace alone (no rerun, no repo). A new scorer
+        has to be computable from the `messages.json` trace and, ideally, verifiable against a known
+        expected number. Please open this before writing code so we can agree on the shape.
+  - type: textarea
+    id: signal
+    attributes:
+      label: What signal should it measure
+      description: What quality of the run would this scorer capture that the current ones don't?
+      placeholder: "e.g. how much unrelated context the agent read before editing."
+    validations:
+      required: true
+  - type: dropdown
+    id: llm
+    attributes:
+      label: Deterministic or judge-based?
+      description: >
+        clinescope prefers deterministic, zero-LLM scorers; an LLM judge must be validated against
+        human labels (Cohen's κ) before it counts.
+      options:
+        - Deterministic (zero-LLM) — computed from the trace text
+        - LLM-judge (would need κ validation vs human labels)
+        - Not sure
+    validations:
+      required: true
+  - type: textarea
+    id: definition
+    attributes:
+      label: How would you define / compute it
+      description: The rule, and what in the trace it reads (tool calls, patch text, results, verdicts).
+    validations:
+      required: true
+  - type: textarea
+    id: expected
+    attributes:
+      label: A trace + its expected score
+      description: >
+        A scorer is only reviewable with a trace that pins its output. Sketch one case (a trace shape)
+        and the number the scorer should produce on it.
+    validations:
+      required: false

clinescope-1.0.1/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,19 @@
+<!-- See CONTRIBUTING.md for dev setup, tests, and what a scorer change needs. -->
+
+## Summary
+
+<!-- What does this change and why? -->
+
+## Linked issue
+
+<!-- Closes #... — open an issue first for anything larger than a bug/doc fix. -->
+
+## How verified
+
+<!-- The command you ran and what it returned (e.g. `pytest -q`, and a `clinescope <trace>` run). -->
+
+## Checklist
+
+- [ ] Tests + linters pass (`pytest -q`, `ruff check .`, `ruff format --check .`, `mypy src`)
+- [ ] A scorer change adds a trace + its expected score (see CONTRIBUTING.md)
+- [ ] Cline's golden fixture was not modified or copied in

clinescope-1.0.1/.github/SECURITY.md

@@ -0,0 +1,24 @@
+# Security Policy
+
+## Supported Versions
+
+clinescope is at 1.x and maintained by a single author. Security fixes land on `main` and in the
+latest tagged release only; older tags are not patched.
+
+| Version              | Supported |
+| -------------------- | --------- |
+| latest 1.x release   | ✅        |
+| `main`               | ✅        |
+| older tags (< latest)| ❌        |
+
+## Reporting a Vulnerability
+
+Please report security issues **privately**, not as a public issue.
+
+- Preferred: open a private report from the repository's **Security** tab → **Report a vulnerability**
+  (GitHub private vulnerability reporting).
+- Fallback: email **minh2416294@gmail.com**.
+
+This is a solo, best-effort project — there is no formal SLA, but I aim to acknowledge a report within
+a few days and will keep you updated on a fix. Please give me a reasonable window to address the issue
+before any public disclosure.

clinescope-1.0.1/.github/dependabot.yml

@@ -0,0 +1,9 @@
+version: 2
+updates:
+  # Keep the GitHub Actions in .github/workflows/ up to date. clinescope has no runtime
+  # dependencies (dependencies = []), so there is no pip ecosystem to watch — Actions is
+  # the only surface that drifts.
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"

clinescope-1.0.1/.github/workflows/ci.yml

@@ -0,0 +1,37 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ci-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/setup-python@v6
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install package and dev deps
+        run: pip install -e ".[dev]"
+      - name: Lint
+        run: ruff check .
+      - name: Format check
+        run: ruff format --check .
+      - name: Type-check
+        run: mypy src
+      - name: Test (with coverage gate)
+        run: pytest -q --cov=clinescope --cov-report=term-missing --cov-fail-under=90

clinescope-1.0.1/.github/workflows/release.yml

@@ -0,0 +1,52 @@
+name: Release
+
+# Fires only when a GitHub Release is published. It is inert until then — no PR,
+# push, or tag alone triggers it — so it never collides with the main-guard ruleset
+# (which gates pull_request -> main). Publishing to PyPI uses Trusted Publishing
+# (OIDC): no API token is stored in the repo; the `pypi` environment + a PyPI
+# pending-publisher authorize the upload. See CONTRIBUTING.md "Releasing".
+on:
+  release:
+    types: [published]
+
+permissions:
+  contents: read
+
+jobs:
+  build:
+    name: Build distribution
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/setup-python@v6
+        with:
+          python-version: "3.12"
+      - name: Install build backend
+        run: python -m pip install --upgrade build
+      - name: Build sdist and wheel
+        run: python -m build
+      - name: Upload dist artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    name: Publish to PyPI
+    needs: build
+    runs-on: ubuntu-latest
+    # The environment gates the release: PyPI's pending-publisher is scoped to it,
+    # and it is where an optional required-reviewer approval would apply.
+    environment:
+      name: pypi
+      url: https://pypi.org/p/clinescope
+    permissions:
+      id-token: write # OIDC token for Trusted Publishing; no stored PyPI secret.
+    steps:
+      - name: Download dist artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

clinescope-1.0.1/.gitignore

@@ -0,0 +1,40 @@
+# --- Python ---
+__pycache__/
+*.py[cod]
+*$py.class
+*.egg-info/
+.eggs/
+build/
+dist/
+*.egg
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+
+# --- Virtual environments ---
+.venv/
+venv/
+env/
+
+# --- IDE / editor ---
+.idea/
+.vscode/
+*.swp
+
+# --- OS ---
+.DS_Store
+Thumbs.db
+
+# --- Build paper trail (git-excluded by design; feeds /public-writing) ---
+writing-content/
+artifacts/
+
+# --- Machine/workflow state (skills read this back) ---
+.planning/
+
+# Windows folder settings
+desktop.ini
+
+# Coverage data file (pytest-cov / coverage.py)
+.coverage
+htmlcov/

clinescope-1.0.1/CLAUDE.md

@@ -0,0 +1,82 @@
+# CLAUDE.md — agent-eval-harness
+
+Project-local instructions for any Claude Code session opened in this repo.
+
+> **This file is a local pointer + copy of the operating protocol.** The single source of truth is
+> in the user's memory files — read them FIRST every session:
+> - **Charter (WHAT):** `~/.claude/memory/project_agent_eval_harness.md`
+> - **Operating protocol (HOW — canonical):** `~/.claude/memory/project_agent_eval_harness_protocol.md`
+>
+> The Charter's Living Log (bottom of that file) is the append-only progress record. Read the last
+> entry's EVIDENCE + NEXT ACTION before doing any work, so sessions never conflict.
+
+## What this project is (one line)
+
+A **Cline eval harness**: scores diff quality + tool-trajectory from a real Cline `messages.json` v1
+trace. Framework-agnostic core; Cline World-A adapter is the flagship + first adapter. Purpose =
+hireability + reputation, NOT paid users.
+
+**Hard floor (v1 is "done" only when BOTH are true):**
+1. Runs against Cline's golden fixture — ingests `sdk/packages/core/fixtures/messages/success.messages.json`,
+   emits a scored report (tool-selection + diff-quality + completion).
+2. Ships the diff-coherence / minimal-diff / apply-recovery scorer the incumbents don't, on ≥1 real trace.
+
+## Standing rules (check every session — from the protocol)
+
+1. **WIP = 1.** One task in flight. One task per Claude Code session.
+2. **No task without a CHECK.** The verifying command/test is written BEFORE any code. No check =
+   task doesn't enter the log.
+3. **Every task serves hard-floor criterion 1 or 2**, or it's tagged `~` (roadmap, dropped by default).
+4. **Never modify the golden fixture** at `**/fixtures/messages/success.messages.json`. The harness
+   INGESTS it from the Cline checkout; it is never copied or edited. (Hook-enforced.)
+5. **No demos disconnected from Cline's real trace.**
+6. **Max 2 weeks without a public artifact.** Silence is the #1 OSS death signal.
+7. **Tooling-config urges = procrastination.** Log the urge, return to the CHECK.
+8. **Log is append-only.** Decisions are immutable; to change one, write a NEW entry that supersedes
+   and links the old (ADR discipline). Never edit past Living-Log entries.
+
+## Sequencing law — walking skeleton
+
+Until the thin path **load World-A trace → score → emit report** runs against the golden fixture,
+EVERY build task is a skeleton segment. Keep it ugly and runnable. After the skeleton runs, every
+task FILLS it (a new scorer, a new metric — deterministic zero-LLM gates first). Never a disconnected
+module. No second adapter before a real one exists (two-adapter rule).
+
+## Task types (only three exist)
+
+| Type | Rule |
+|---|---|
+| **BUILD** | Fills the skeleton. Verification-first, sized to one session, machine-checkable. |
+| **SPIKE** | One question, hard timebox (2–3h), throwaway code, MUST end with a written decision. Timebox expiry IS the answer. Only for genuine unknowns. |
+| **PUBLISH** | Triggered by artifact milestones, never by calendar. Raw material mined from the Living Log + commits → `writing-content/`. |
+
+## Living-Log entry template (append to the Charter file, append-only)
+
+```
+## YYYY-MM-DD — <session goal, one sentence, ends in a verifiable state>
+- TYPE: BUILD | SPIKE (timebox: Xh) | PUBLISH
+- CHECK: <exact command/test + expected output that proves done — written first>
+- APPETITE: ≤ N sessions
+- CONTEXT: <files, fixture paths, relevant log entries>
+- EVIDENCE: <pasted actual output — never an assertion>
+- DECISIONS: <immutable; supersede-don't-edit>
+- OPEN QUESTIONS: <what you now don't know>
+- NEXT ACTION: <the first thing next session does>
+```
+
+## Session gates (from the user's global CLAUDE.md)
+
+- Set a `/goal` with a measurable success criterion; match `/effort` (feature work = xhigh); Plan Mode
+  for anything multi-step.
+- Demand EVIDENCE (pasted output), never assertions of "done".
+- The paper trail (`writing-content/`, `artifacts/`) is git-excluded and feeds `/public-writing`.
+
+## Layout
+
+```
+src/clinescope/           # the package (import clinescope)
+pyproject.toml            # metadata, no deps yet
+.venv/                    # local venv (git-ignored)
+writing-content/          # git-excluded build paper trail (created when a build phase runs)
+artifacts/                # git-excluded keeper artifacts
+```

clinescope-1.0.1/CODE_OF_CONDUCT.md

@@ -0,0 +1,128 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+minh2416294@gmail.com.
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
+
+Community Impact Guidelines were inspired by [Mozilla's code of conduct
+enforcement ladder](https://github.com/mozilla/diversity).
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see the FAQ at
+https://www.contributor-covenant.org/faq. Translations are available at
+https://www.contributor-covenant.org/translations.

clinescope-1.0.1/CONTRIBUTING.md

@@ -0,0 +1,100 @@
+# Contributing to clinescope
+
+Thanks for helping build clinescope. Small, discussed-first changes are the norm here — for anything
+larger than a bug fix or a doc tweak, please open an issue first so we can agree on the shape before
+you write code.
+
+This is a `src`-layout Python package (`import clinescope`), Python 3.11+.
+
+## Dev setup
+
+Fork and clone, then install the package **editable, with the dev extras**, from inside your checkout:
+
+```bash
+git clone https://github.com/<you>/clinescope
+cd clinescope
+python -m venv .venv && source .venv/Scripts/activate   # Windows Git Bash; use bin/activate on macOS/Linux
+pip install -e ".[dev]"
+```
+
+The `[dev]` extra pulls in `pytest`, `pytest-cov`, `ruff`, and `mypy` — the same tools CI runs.
+
+## Running the tests and linters
+
+Run these before you push; they are exactly what CI checks:
+
+```bash
+pytest -q                 # tests
+ruff check .              # lint
+ruff format --check .     # formatting (drop --check to auto-format)
+mypy src                  # type-check
+```
+
+`pytest` is install-independent — `pyproject.toml` sets `[tool.pytest.ini_options] pythonpath = ["src"]`,
+so `pytest -q` finds the package whether or not it's installed.
+
+CI additionally runs the suite under coverage and **fails if line coverage drops below 90%**
+(measured at 94%). To reproduce that gate locally:
+
+```bash
+pytest -q --cov=clinescope --cov-report=term-missing --cov-fail-under=90
+```
+
+A bare `pytest -q` still works without the coverage plugin — the `--cov` flags live on the CI command
+line, not in `addopts`, so base pytest isn't required to run the tests.
+
+## The multi-worktree editable-install gotcha (read this if you use git worktrees)
+
+An editable install (`pip install -e .`) writes a `.pth` into the active virtualenv that pins imports to
+**one** checkout's `src/`. If you share a single `.venv` across several git worktrees of this repo,
+whichever worktree ran `pip install -e .` **last** wins — the others import stale code, or fail with
+`No module named clinescope`, even though their own source is correct.
+
+Two ways to stay safe:
+
+- **Re-run `pip install -e .` inside the worktree you're working in** (repoints the `.pth`), **or**
+- **Use `PYTHONPATH=src`** for a one-off run, which needs no install:
+
+  ```bash
+  PYTHONPATH=src python -m clinescope examples/sample-trace.json --expected read_files apply_patch
+  ```
+
+The gotcha only bites `python -m clinescope` (and editor type-checkers), which don't read
+`pyproject.toml`; `pytest` is already immune via the setting above.
+
+## Fork, branch, PR
+
+1. Fork the repo and create a branch off `main` (`fix/…`, `feat/…`, `chore/…`).
+2. Make the change; keep the diff focused on one thing.
+3. Run the tests + linters above until they're all green.
+4. Push to your fork and open a PR against `minh2416294/clinescope:main`. CI runs on the PR.
+
+## What a PR should include
+
+- **A scorer change needs a trace + its expected score.** Add (or extend) a fixture trace and assert
+  the number the scorer should produce on it — a scorer without a test proving its output isn't
+  reviewable. Never edit or copy in Cline's own golden fixture; add your own small synthetic trace
+  (see `examples/sample-trace.json` for the World-A v1 shape).
+- **Open an issue first for anything large** — a new scorer, a new adapter, a format change. A quick
+  agreement on the approach saves a rewrite.
+- Keep behavior changes and refactors in separate commits where you can; explain *why* in the PR body,
+  not just *what*.
+
+## Releasing (maintainer only)
+
+Releases publish to PyPI automatically via **Trusted Publishing** (OIDC) — there is no PyPI API token
+stored in the repo. `.github/workflows/release.yml` builds the sdist + wheel and uploads them when a
+**GitHub Release is published** (not on a tag alone, and never on a PR).
+
+To cut a release:
+
+1. Bump `version` in `pyproject.toml` (and anywhere else it's mirrored), and land it on `main` via PR.
+2. On GitHub, **Releases → Draft a new release**, create a tag matching the version (e.g. `v1.0.1`), and
+   **Publish**. The `Release` workflow runs and uploads to PyPI.
+3. Verify the new version renders at <https://pypi.org/project/clinescope/> and installs cleanly into a
+   fresh venv (`pip install clinescope` → `clinescope-corpus` exits 0).
+
+One-time setup (already configured; documented here for the record) — on PyPI, **Account → Publishing →
+Add a pending publisher** with: PyPI Project Name `clinescope`, Owner `minh2416294`, Repository name
+`clinescope`, Workflow name `release.yml`, Environment name `pypi`; and a GitHub repo Environment named
+`pypi` (Settings → Environments), where an optional required-reviewer gate can be added.