switchit-harness 0.1.0__tar.gz

switchit_harness-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 SwitchIt! contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

switchit_harness-0.1.0/PKG-INFO

@@ -0,0 +1,188 @@
+Metadata-Version: 2.4
+Name: switchit-harness
+Version: 0.1.0
+Summary: Evidence-based migration of AI workflows between providers: import traces, replay them through adapters, and report what survives the switch.
+Author: SwitchIt! contributors
+License: MIT
+Keywords: ai,llm,migration,evaluation,traces,otel,openai,anthropic,bedrock,agents,replay,mcp
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: parquet
+Requires-Dist: pyarrow>=14; extra == "parquet"
+Provides-Extra: bedrock
+Requires-Dist: boto3>=1.34; extra == "bedrock"
+Provides-Extra: all
+Requires-Dist: pyarrow>=14; extra == "all"
+Requires-Dist: boto3>=1.34; extra == "all"
+Provides-Extra: dev
+Requires-Dist: pyarrow>=14; extra == "dev"
+Dynamic: license-file
+
+# SwitchIt!
+
+SwitchIt! helps teams move AI workflows from one provider to another with evidence that the important behavior survives the move.
+
+The harness ingests authorized traces and application configuration, maps the workflows people actually use, selects representative replay cases, runs them against a target provider, and compares task outcomes, tool behavior, latency, and token usage. It produces a migration report with failures, uncertainties, and concrete changes needed.
+
+The companion skill guides an agent through reviewing the evidence, resolving ambiguous workflows, and preparing a migration plan. The harness performs the reproducible imports, replays, comparisons, and checks. Use `$switchit` from any project to ask for a provider migration plan or a verified switch; see the [skill](skills/switchit/SKILL.md), the [system map](docs/system-map.svg), and the [migration run](docs/migration-run.svg).
+
+First milestone — shipped in `switchit/`: JSON/JSONL, OTLP, Langfuse-style, and Parquet importers; one recorded baseline adapter; mock and live (OpenAI-compatible, Anthropic, Bedrock Converse) target adapters; and a complete offline replay loop with JSON + HTML reports on synthetic traces.
+
+## Install
+
+```sh
+pip install switchit-harness        # from PyPI, once published
+# or from this repository:
+pip install .
+
+switchit --version
+python3 -m switchit.cli --version   # same thing without the entry point
+```
+
+The core is stdlib-only. Add extras only if you need them: `pip install 'switchit-harness[parquet]'` for Parquet imports, `'switchit-harness[bedrock]'` for the Bedrock adapter.
+
+To install the companion `$switchit` agent skill for tools that read `~/.agents/skills`:
+
+```sh
+./scripts/install-skill.sh
+```
+
+## Quick start (fully offline, no credentials)
+
+```sh
+# One command on the bundled synthetic demo dataset:
+python3 -m switchit.cli run --demo --state-dir /tmp/switchit-demo
+open /tmp/switchit-demo/run/report.html
+
+# Or step by step on the files in examples/:
+python3 -m switchit.cli import --jsonl examples/traces.jsonl --app-config examples/app_config.json --state-dir .switchit
+python3 -m switchit.cli map --state-dir .switchit          # provisional workflow map; edit workflows.json to correct it
+python3 -m switchit.cli map --keep-edits --state-dir .switchit   # re-runs keep your edits
+python3 -m switchit.cli cases --state-dir .switchit        # replay set; edit cases.json to add real success checks
+python3 -m switchit.cli replay --state-dir .switchit       # recorded baseline vs mock target, checkpointed
+python3 -m switchit.cli report --state-dir .switchit       # report.json + report.html
+```
+
+## What each stage produces
+
+| Stage | Artifact | Notes |
+|---|---|---|
+| `import` | `<state-dir>/store.json` | Canonical sessions, traces, calls. Secrets redacted at import — redacted content is the only content stored, replayed, or reported. Usage provenance kept separate: `recorded` / `estimated` / `unknown`. Sources: your exports **and** coding-agent session folders you already have (`--agents` reads Codex, Claude Code, and OpenCode locally). |
+| `profile` | `profile.json` + `app_config.inferred.json` | Digs through the sessions: system prompts, tool inventory with schemas inferred from observed calls, provider/model mix, error samples. The inferred app config is a ready-to-edit starting point. |
+| `map` | `workflows.json` | TF-IDF + tool-overlap grouping with confidence and uncertainty flags. Labels are triage aids; set `"edited": true` and fix names/members to correct them. |
+| `cases` | `cases.json` | One case per selected trace covering common, costly, risky, and failed workflows, with default checks. Edit to add explicit success checks — a historical model response alone is not proof of success. |
+| `replay` | `run/results.json` + `run/run_state.json` | Same cases through baseline and target adapters. Tool proposals are captured, never executed. Checkpointed after every case; a fingerprint refuses to resume after cases, adapters, or runner code changed. |
+| `report` | `run/report.json` + `run/report.html` | Verdicts (pass / fail / error / uncertain), failure groups, capability gaps, prioritized fixes, usage by provenance. Local, self-contained HTML. |
+
+## Connectors
+
+- **Inputs:** OpenTelemetry/OTLP JSON (`--otlp`), JSON/JSONL (`--jsonl`), Langfuse-style exports (`--langfuse`), and Parquet / ZIP-of-`traces.parquet` (`--parquet`, needs the optional `pyarrow` package). A simple app-configuration JSON (`--app-config`) supplies prompts and tool definitions.
+- **Providers:** recorded baseline and mock adapters for offline work; OpenAI-compatible HTTP, Anthropic Messages, and Amazon Bedrock Converse for live tests. Provider presets fill in the details: `--target grok` (xAI), `groq`, `deepseek`, `mistral`, `openrouter`, `together`, `fireworks`, `ollama` (free, fully local), `vllm` (local). Meta Llama runs through `ollama`/`vllm`/`together`/`fireworks`/`bedrock`.
+- **Secrets:** environment variables only — `OPENAI_API_KEY`, `XAI_API_KEY`, `GROQ_API_KEY`, `ANTHROPIC_API_KEY`, the standard AWS chain, and so on per provider. Keys are never accepted on the command line or written to files. Local import and analysis do not contact a provider.
+- **Memory services:** products like Supermemory are business tools your agent calls, not chat-completion providers — so they are captured as tool proposals during replay, never executed, and are not replay targets.
+
+## Live replay is an explicit opt-in
+
+```sh
+export OPENAI_API_KEY='...'
+python3 -m switchit.cli replay --target openai --target-model gpt-4o-mini --remote --state-dir .switchit
+```
+
+Without `--remote`, live targets refuse to run. With it, the redacted case requests are sent to the target provider and billed accordingly; usage is recorded from provider responses, and requests that fail before a response keep their consumption counted separately as unknown.
+
+## Traces you already have: coding agents
+
+Most developers already have rich agent traces on disk without ever instrumenting anything. SwitchIt reads them directly, with the same redaction and provenance rules as every other source:
+
+```sh
+switchit import --agents                        # Codex, Claude Code, OpenCode session folders
+switchit profile --state-dir .switchit          # prompts, tool schemas, provider mix — dug out automatically
+```
+
+- **Codex**: `~/.codex/sessions/**/*.jsonl` — model calls, `function_call`s and outputs, token counts
+- **Claude Code**: `~/.claude/projects/**/*.jsonl` — transcripts with `tool_use` blocks and per-message usage
+- **OpenCode**: `~/.local/share/opencode/opencode.db` — message/part tables (read-only SQLite)
+- **Cursor**: `~/.cursor/chats/*/*/store.db` — chat messages per workspace, system prompts, agentic tool calls (Cursor reports no usage, so those calls land in the `unknown` bucket)
+- **Cline**: `~/.cline/data/sessions/` and VS Code `globalStorage/saoudrizwan.claude-dev/tasks/` — SDK message files with per-message metrics, legacy task conversations with usage from `ui_messages.json`
+- **Gemini CLI**: `~/.gemini/tmp/*/chats/session-*.jsonl` — records with `toolCalls` (args, results, status) and per-message token counts; `$set`/`$patch`/`$rewindTo` update records applied by id
+- **Copilot CLI** (incl. Desktop-bundled sessions): `~/.copilot/session-state/*/events.jsonl` — the event log: prompts, `assistant.message` with `toolRequests`, tool results, `session.usage_record` usage, system prompt
+- **Aider**: explicit paths only (`--aider .aider.chat.history.md`) — histories live per-project. Task-level evidence: prompts, assistant text, tool output as blockquotes; no structured tool calls or usage, and traces are marked accordingly
+- **Devin** (cloud): explicit opt-in (`--devin`, needs `DEVIN_API_KEY` + `DEVIN_ORG_ID` in the environment) via the official v3 API — task-level metadata per session (prompt, status, pull requests, ACU consumption; ACUs are compute units and stay out of token totals). Message-level transcripts are not exposed by the API, so these traces support workflow discovery, not replay
+- **Windsurf**: not supported — no publicly documented chat-store format (it is mid-migration to Devin per community research); will follow once a format is public
+
+Everything downstream — workflow map, replay cases with tool fixtures, migration report — works the same on these as on any other source. `--agent-limit` caps how many session files are read (newest first, default 200).
+
+## If your workflow already uses business tools (Supermemory, internal APIs, …)
+
+Their calls are treated as first-class evidence at every stage, and a chained replay can complete the workflow offline:
+
+1. **Import** — the tool calls are already in your traces; import captures them, and the workflow map shows which workflows depend on which tools.
+2. **Cases** — each case carries `tool_fixtures`: what the tool actually returned in history, extracted automatically. Edit them in `cases.json` if needed.
+3. **Replay with `--chain`** — when the target proposes a tool call, the recorded result is fed back and the model continues, so success checks evaluate the workflow's **final answer**, not just the first proposal. If the target invents a tool history never used, it gets an explicit placeholder result and the gap is reported.
+4. **The line that never moves** — the harness never executes a real business tool. No reads against your live Supermemory, no writes to your data. Live read-only fixtures against an authorized environment are a possible future opt-in, separate from the default offline path.
+
+The same mechanism covers every business tool, not just memory services — Supermemory, Stripe, internal CRUD APIs — because the fixture layer is tool-agnostic by design.
+
+## Any agent can drive it (MCP)
+
+The harness ships a built-in MCP server, so any MCP-capable agent — Claude Code, Codex, OpenCode, Cursor, or your own — can run the whole pipeline as native tools instead of shelling out:
+
+```sh
+switchit-mcp        # stdio MCP server; register it in your client as:
+```
+
+```json
+{ "switchit": { "command": "switchit-mcp", "args": [] } }
+```
+
+Exposed tools: `switchit_import`, `switchit_profile`, `switchit_map`, `switchit_cases`, `switchit_replay`, `switchit_report`, `switchit_run`, `switchit_status`, `switchit_read_report`, `switchit_list_providers`. The safety model carries over unchanged: everything is offline until the agent explicitly passes `remote: true` with a live target, and keys still come from the environment — an agent cannot talk a key out of you through a tool argument. The `$switchit` skill and the CLI remain equivalent entry points; use whichever your agent speaks.
+
+## Tests
+
+```sh
+python3 -m unittest discover -s tests -p "test_*.py" -t .
+```
+
+All tests are offline and use only synthetic data: connector shapes, redaction, usage reconciliation ("every call counted exactly once"), grouping behavior, verdict logic, checkpoint/resume, fingerprint refusal, report contents, and the no-secrets guarantee.
+
+## Publishing
+
+```sh
+python3 -m pip install build twine
+python3 -m build                       # sdist + wheel into dist/
+python3 -m twine check dist/*          # metadata + README rendering
+python3 -m twine upload dist/*         # to PyPI (configure credentials first)
+```
+
+Cut a release by bumping `version` in `pyproject.toml` and adding a `CHANGELOG.md` entry.
+
+## Code layout
+
+```text
+switchit/
+  core.py        import, normalize, redact, workflow map, replay selection
+  providers.py   provider adapters and capability descriptions
+  runner.py      replay, compare, report, checkpoints
+  cli.py         import → map → cases → replay → report; one-command run
+  demo_data.py   bundled synthetic dataset for --demo
+tests/           offline end-to-end and connector-shape checks
+examples/        synthetic traces and offline provider config
+skills/switchit/ SKILL.md: guide the agent through evidence and migration decisions
+```
+
+## Boundaries
+
+Workflow labels and similarity scores are triage aids, not measured accuracy. For a workflow that needs a real tool to complete, a single-call replay can verify the proposed tool request but must leave the end-to-end outcome unverified until a safe tool fixture or authorized test environment exists. The Hyde assessment dataset and results are not inputs to this project.

switchit_harness-0.1.0/README.md

@@ -0,0 +1,155 @@
+# SwitchIt!
+
+SwitchIt! helps teams move AI workflows from one provider to another with evidence that the important behavior survives the move.
+
+The harness ingests authorized traces and application configuration, maps the workflows people actually use, selects representative replay cases, runs them against a target provider, and compares task outcomes, tool behavior, latency, and token usage. It produces a migration report with failures, uncertainties, and concrete changes needed.
+
+The companion skill guides an agent through reviewing the evidence, resolving ambiguous workflows, and preparing a migration plan. The harness performs the reproducible imports, replays, comparisons, and checks. Use `$switchit` from any project to ask for a provider migration plan or a verified switch; see the [skill](skills/switchit/SKILL.md), the [system map](docs/system-map.svg), and the [migration run](docs/migration-run.svg).
+
+First milestone — shipped in `switchit/`: JSON/JSONL, OTLP, Langfuse-style, and Parquet importers; one recorded baseline adapter; mock and live (OpenAI-compatible, Anthropic, Bedrock Converse) target adapters; and a complete offline replay loop with JSON + HTML reports on synthetic traces.
+
+## Install
+
+```sh
+pip install switchit-harness        # from PyPI, once published
+# or from this repository:
+pip install .
+
+switchit --version
+python3 -m switchit.cli --version   # same thing without the entry point
+```
+
+The core is stdlib-only. Add extras only if you need them: `pip install 'switchit-harness[parquet]'` for Parquet imports, `'switchit-harness[bedrock]'` for the Bedrock adapter.
+
+To install the companion `$switchit` agent skill for tools that read `~/.agents/skills`:
+
+```sh
+./scripts/install-skill.sh
+```
+
+## Quick start (fully offline, no credentials)
+
+```sh
+# One command on the bundled synthetic demo dataset:
+python3 -m switchit.cli run --demo --state-dir /tmp/switchit-demo
+open /tmp/switchit-demo/run/report.html
+
+# Or step by step on the files in examples/:
+python3 -m switchit.cli import --jsonl examples/traces.jsonl --app-config examples/app_config.json --state-dir .switchit
+python3 -m switchit.cli map --state-dir .switchit          # provisional workflow map; edit workflows.json to correct it
+python3 -m switchit.cli map --keep-edits --state-dir .switchit   # re-runs keep your edits
+python3 -m switchit.cli cases --state-dir .switchit        # replay set; edit cases.json to add real success checks
+python3 -m switchit.cli replay --state-dir .switchit       # recorded baseline vs mock target, checkpointed
+python3 -m switchit.cli report --state-dir .switchit       # report.json + report.html
+```
+
+## What each stage produces
+
+| Stage | Artifact | Notes |
+|---|---|---|
+| `import` | `<state-dir>/store.json` | Canonical sessions, traces, calls. Secrets redacted at import — redacted content is the only content stored, replayed, or reported. Usage provenance kept separate: `recorded` / `estimated` / `unknown`. Sources: your exports **and** coding-agent session folders you already have (`--agents` reads Codex, Claude Code, and OpenCode locally). |
+| `profile` | `profile.json` + `app_config.inferred.json` | Digs through the sessions: system prompts, tool inventory with schemas inferred from observed calls, provider/model mix, error samples. The inferred app config is a ready-to-edit starting point. |
+| `map` | `workflows.json` | TF-IDF + tool-overlap grouping with confidence and uncertainty flags. Labels are triage aids; set `"edited": true` and fix names/members to correct them. |
+| `cases` | `cases.json` | One case per selected trace covering common, costly, risky, and failed workflows, with default checks. Edit to add explicit success checks — a historical model response alone is not proof of success. |
+| `replay` | `run/results.json` + `run/run_state.json` | Same cases through baseline and target adapters. Tool proposals are captured, never executed. Checkpointed after every case; a fingerprint refuses to resume after cases, adapters, or runner code changed. |
+| `report` | `run/report.json` + `run/report.html` | Verdicts (pass / fail / error / uncertain), failure groups, capability gaps, prioritized fixes, usage by provenance. Local, self-contained HTML. |
+
+## Connectors
+
+- **Inputs:** OpenTelemetry/OTLP JSON (`--otlp`), JSON/JSONL (`--jsonl`), Langfuse-style exports (`--langfuse`), and Parquet / ZIP-of-`traces.parquet` (`--parquet`, needs the optional `pyarrow` package). A simple app-configuration JSON (`--app-config`) supplies prompts and tool definitions.
+- **Providers:** recorded baseline and mock adapters for offline work; OpenAI-compatible HTTP, Anthropic Messages, and Amazon Bedrock Converse for live tests. Provider presets fill in the details: `--target grok` (xAI), `groq`, `deepseek`, `mistral`, `openrouter`, `together`, `fireworks`, `ollama` (free, fully local), `vllm` (local). Meta Llama runs through `ollama`/`vllm`/`together`/`fireworks`/`bedrock`.
+- **Secrets:** environment variables only — `OPENAI_API_KEY`, `XAI_API_KEY`, `GROQ_API_KEY`, `ANTHROPIC_API_KEY`, the standard AWS chain, and so on per provider. Keys are never accepted on the command line or written to files. Local import and analysis do not contact a provider.
+- **Memory services:** products like Supermemory are business tools your agent calls, not chat-completion providers — so they are captured as tool proposals during replay, never executed, and are not replay targets.
+
+## Live replay is an explicit opt-in
+
+```sh
+export OPENAI_API_KEY='...'
+python3 -m switchit.cli replay --target openai --target-model gpt-4o-mini --remote --state-dir .switchit
+```
+
+Without `--remote`, live targets refuse to run. With it, the redacted case requests are sent to the target provider and billed accordingly; usage is recorded from provider responses, and requests that fail before a response keep their consumption counted separately as unknown.
+
+## Traces you already have: coding agents
+
+Most developers already have rich agent traces on disk without ever instrumenting anything. SwitchIt reads them directly, with the same redaction and provenance rules as every other source:
+
+```sh
+switchit import --agents                        # Codex, Claude Code, OpenCode session folders
+switchit profile --state-dir .switchit          # prompts, tool schemas, provider mix — dug out automatically
+```
+
+- **Codex**: `~/.codex/sessions/**/*.jsonl` — model calls, `function_call`s and outputs, token counts
+- **Claude Code**: `~/.claude/projects/**/*.jsonl` — transcripts with `tool_use` blocks and per-message usage
+- **OpenCode**: `~/.local/share/opencode/opencode.db` — message/part tables (read-only SQLite)
+- **Cursor**: `~/.cursor/chats/*/*/store.db` — chat messages per workspace, system prompts, agentic tool calls (Cursor reports no usage, so those calls land in the `unknown` bucket)
+- **Cline**: `~/.cline/data/sessions/` and VS Code `globalStorage/saoudrizwan.claude-dev/tasks/` — SDK message files with per-message metrics, legacy task conversations with usage from `ui_messages.json`
+- **Gemini CLI**: `~/.gemini/tmp/*/chats/session-*.jsonl` — records with `toolCalls` (args, results, status) and per-message token counts; `$set`/`$patch`/`$rewindTo` update records applied by id
+- **Copilot CLI** (incl. Desktop-bundled sessions): `~/.copilot/session-state/*/events.jsonl` — the event log: prompts, `assistant.message` with `toolRequests`, tool results, `session.usage_record` usage, system prompt
+- **Aider**: explicit paths only (`--aider .aider.chat.history.md`) — histories live per-project. Task-level evidence: prompts, assistant text, tool output as blockquotes; no structured tool calls or usage, and traces are marked accordingly
+- **Devin** (cloud): explicit opt-in (`--devin`, needs `DEVIN_API_KEY` + `DEVIN_ORG_ID` in the environment) via the official v3 API — task-level metadata per session (prompt, status, pull requests, ACU consumption; ACUs are compute units and stay out of token totals). Message-level transcripts are not exposed by the API, so these traces support workflow discovery, not replay
+- **Windsurf**: not supported — no publicly documented chat-store format (it is mid-migration to Devin per community research); will follow once a format is public
+
+Everything downstream — workflow map, replay cases with tool fixtures, migration report — works the same on these as on any other source. `--agent-limit` caps how many session files are read (newest first, default 200).
+
+## If your workflow already uses business tools (Supermemory, internal APIs, …)
+
+Their calls are treated as first-class evidence at every stage, and a chained replay can complete the workflow offline:
+
+1. **Import** — the tool calls are already in your traces; import captures them, and the workflow map shows which workflows depend on which tools.
+2. **Cases** — each case carries `tool_fixtures`: what the tool actually returned in history, extracted automatically. Edit them in `cases.json` if needed.
+3. **Replay with `--chain`** — when the target proposes a tool call, the recorded result is fed back and the model continues, so success checks evaluate the workflow's **final answer**, not just the first proposal. If the target invents a tool history never used, it gets an explicit placeholder result and the gap is reported.
+4. **The line that never moves** — the harness never executes a real business tool. No reads against your live Supermemory, no writes to your data. Live read-only fixtures against an authorized environment are a possible future opt-in, separate from the default offline path.
+
+The same mechanism covers every business tool, not just memory services — Supermemory, Stripe, internal CRUD APIs — because the fixture layer is tool-agnostic by design.
+
+## Any agent can drive it (MCP)
+
+The harness ships a built-in MCP server, so any MCP-capable agent — Claude Code, Codex, OpenCode, Cursor, or your own — can run the whole pipeline as native tools instead of shelling out:
+
+```sh
+switchit-mcp        # stdio MCP server; register it in your client as:
+```
+
+```json
+{ "switchit": { "command": "switchit-mcp", "args": [] } }
+```
+
+Exposed tools: `switchit_import`, `switchit_profile`, `switchit_map`, `switchit_cases`, `switchit_replay`, `switchit_report`, `switchit_run`, `switchit_status`, `switchit_read_report`, `switchit_list_providers`. The safety model carries over unchanged: everything is offline until the agent explicitly passes `remote: true` with a live target, and keys still come from the environment — an agent cannot talk a key out of you through a tool argument. The `$switchit` skill and the CLI remain equivalent entry points; use whichever your agent speaks.
+
+## Tests
+
+```sh
+python3 -m unittest discover -s tests -p "test_*.py" -t .
+```
+
+All tests are offline and use only synthetic data: connector shapes, redaction, usage reconciliation ("every call counted exactly once"), grouping behavior, verdict logic, checkpoint/resume, fingerprint refusal, report contents, and the no-secrets guarantee.
+
+## Publishing
+
+```sh
+python3 -m pip install build twine
+python3 -m build                       # sdist + wheel into dist/
+python3 -m twine check dist/*          # metadata + README rendering
+python3 -m twine upload dist/*         # to PyPI (configure credentials first)
+```
+
+Cut a release by bumping `version` in `pyproject.toml` and adding a `CHANGELOG.md` entry.
+
+## Code layout
+
+```text
+switchit/
+  core.py        import, normalize, redact, workflow map, replay selection
+  providers.py   provider adapters and capability descriptions
+  runner.py      replay, compare, report, checkpoints
+  cli.py         import → map → cases → replay → report; one-command run
+  demo_data.py   bundled synthetic dataset for --demo
+tests/           offline end-to-end and connector-shape checks
+examples/        synthetic traces and offline provider config
+skills/switchit/ SKILL.md: guide the agent through evidence and migration decisions
+```
+
+## Boundaries
+
+Workflow labels and similarity scores are triage aids, not measured accuracy. For a workflow that needs a real tool to complete, a single-call replay can verify the proposed tool request but must leave the end-to-end outcome unverified until a safe tool fixture or authorized test environment exists. The Hyde assessment dataset and results are not inputs to this project.

switchit_harness-0.1.0/pyproject.toml

@@ -0,0 +1,46 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "switchit-harness"
+version = "0.1.0"
+description = "Evidence-based migration of AI workflows between providers: import traces, replay them through adapters, and report what survives the switch."
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "MIT" }
+authors = [{ name = "SwitchIt! contributors" }]
+keywords = [
+  "ai", "llm", "migration", "evaluation", "traces", "otel",
+  "openai", "anthropic", "bedrock", "agents", "replay", "mcp",
+]
+classifiers = [
+  "Development Status :: 4 - Beta",
+  "Environment :: Console",
+  "Intended Audience :: Developers",
+  "License :: OSI Approved :: MIT License",
+  "Operating System :: OS Independent",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Topic :: Software Development :: Quality Assurance",
+  "Topic :: Software Development :: Testing",
+]
+# The core is stdlib-only by design: local import, mapping, and offline replay
+# never need a third-party package or a network call.
+dependencies = []
+
+[project.optional-dependencies]
+parquet = ["pyarrow>=14"]
+bedrock = ["boto3>=1.34"]
+all = ["pyarrow>=14", "boto3>=1.34"]
+dev = ["pyarrow>=14"]
+
+[project.scripts]
+switchit = "switchit.cli:main"
+switchit-mcp = "switchit.mcp_server:main"
+
+[tool.setuptools]
+packages = ["switchit"]

switchit_harness-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

switchit_harness-0.1.0/switchit/__init__.py

@@ -0,0 +1,21 @@
+"""SwitchIt! — evidence-based migration of AI workflows between providers.
+
+The harness imports authorized traces, normalizes them into a canonical
+store, maps provisional workflows, selects replay cases, replays them
+through recorded/offline or live provider adapters, and produces a local
+JSON + HTML migration report.
+
+Boundaries kept honest throughout:
+- Workflow labels and similarity scores are triage aids, not measured accuracy.
+- Proposed tool calls are captured, never executed.
+- A historical model response is reference behavior, not proof of success.
+- Token provenance stays separated: recorded, estimated, unknown.
+"""
+
+__version__ = "0.1.0"
+
+USAGE_BASES = ("recorded", "estimated", "unknown")
+
+from . import core, providers, runner  # noqa: E402,F401
+
+__all__ = ["__version__", "core", "providers", "runner", "USAGE_BASES"]

switchit_harness-0.1.0/switchit/__main__.py

@@ -0,0 +1,8 @@
+"""Allow ``python -m switchit`` as a shorthand for ``python -m switchit.cli``."""
+
+import sys
+
+from .cli import main
+
+if __name__ == "__main__":
+    sys.exit(main())