pyskillet 0.0.1__tar.gz

pyskillet-0.0.1/.claude/.gitignore

@@ -0,0 +1,2 @@
+# Generated from .claude-template/
+commands/

pyskillet-0.0.1/.claude/CLAUDE.md

@@ -0,0 +1,36 @@
+# Skillet Development
+
+## Workflow
+- Work in git worktrees, tie PRs to GitHub issues
+- Before pushing: `just lint && just test-unit`
+- After pushing: monitor CI checks
+
+## Project Structure
+- `.claude-template/` - Source templates with `{{SKILLET_DIR}}` placeholders
+- `.claude/commands/` - Generated (gitignored), built from templates
+- `scripts/build_claude_config.py` - Template builder
+- `tests/e2e/` - End-to-end tests using Claude Agent SDK
+
+## Testing
+- E2E tests auto-build `.claude/commands/` via conftest.py
+- `Conversation` helper for multi-turn test flows
+- `setting_sources=["project"]` loads slash commands from `.claude/commands/`
+- Commands in subdirs get namespaced: `skillet/add.md` -> `/skillet:add`
+
+## Key Commands
+```bash
+just build-claude    # Build .claude/commands/ from templates
+just test-e2e        # Run e2e tests
+just test-unit       # Run unit tests
+```
+
+## Commit Convention
+
+Commits must follow [Conventional Commits](https://www.conventionalcommits.org/) format for automatic version bumping:
+
+- `fix: ...` → patch release (0.1.0 → 0.1.1)
+- `feat: ...` → minor release (0.1.0 → 0.2.0)
+- `feat!: ...` or `BREAKING CHANGE:` in body → major release (0.1.0 → 1.0.0)
+- `chore:`, `docs:`, `refactor:`, `test:`, etc. → patch release
+
+Releases run nightly at 2am UTC. All commits since the last release are batched together, and the highest-priority bump type wins (breaking > feat > fix).

pyskillet-0.0.1/.claude-template/commands/skillet/add.md

@@ -0,0 +1,55 @@
+# Add an Eval
+
+The user has just noticed that your previous response wasn't what they wanted. Help them add this as an eval for skill development.
+
+## Your Task
+
+1. **Ask what they expected** — Ask the user what behavior or format they expected instead of what you produced.
+
+2. **Check existing evals** — List directories in `{{SKILLET_DIR}}/evals/` to see what eval categories already exist.
+
+3. **Suggest eval name options** — Use `AskUserQuestion` to present exactly 3 options:
+   - If an existing eval seems related to this issue, suggest it first with "(existing, N evals)"
+   - Include 1-2 new name suggestions based on the user's description
+   - The user can always type a custom name via "Other"
+
+4. **Save the eval** — Write a YAML file to `{{SKILLET_DIR}}/evals/<name>/<number>.yaml` containing:
+   - `timestamp`: Current ISO timestamp
+   - `prompt`: The user's original prompt (from conversation context)
+   - `actual`: Your response (summarized if long)
+   - `expected`: What the user said they wanted
+   - `name`: The eval name
+
+5. **Confirm** — Tell the user the eval was saved and how many evals exist for this name.
+
+## Example Interaction
+
+User runs `/add`
+
+You: "What behavior or format did you expect?"
+
+User: "Should have automatically used Playwright when WebFetch failed"
+
+You: *checks {{SKILLET_DIR}}/evals/ and finds browser-fallback/ with 2 evals*
+*uses AskUserQuestion with options:*
+- "browser-fallback (existing, 2 evals)" — Add to existing eval category
+- "webfetch-retry" — New category for fetch retry behavior
+- "auto-playwright" — New category for automatic Playwright usage
+
+User selects: "browser-fallback (existing, 2 evals)"
+
+You: *saves to {{SKILLET_DIR}}/evals/browser-fallback/003.yaml*
+"Eval saved to {{SKILLET_DIR}}/evals/browser-fallback/003.yaml
+
+  ├─ Prompt: "Read https://reddit.com/..."
+  ├─ Actual: Asked user what to do
+  ├─ Expected: Automatically use Playwright
+
+You now have 3 evals for this skill."
+
+## Important
+
+- Look at the conversation history to find the prompt and your response
+- Always check existing evals first — grouping related evals is valuable
+- Keep the YAML simple and human-readable
+- Create the {{SKILLET_DIR}}/evals directory if it doesn't exist

pyskillet-0.0.1/.github/dependabot.yml

@@ -0,0 +1,15 @@
+version: 2
+updates:
+  - package-ecosystem: "pip"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+    commit-message:
+      prefix: "deps"
+
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+    commit-message:
+      prefix: "ci"

pyskillet-0.0.1/.github/workflows/build.yml

@@ -0,0 +1,31 @@
+name: Build
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - '**.py'
+      - 'pyproject.toml'
+      - 'uv.lock'
+      - '.github/workflows/build.yml'
+  pull_request:
+    paths:
+      - '**.py'
+      - 'pyproject.toml'
+      - 'uv.lock'
+      - '.github/workflows/build.yml'
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+
+      - name: Install dependencies
+        run: uv sync --extra dev
+
+      - name: Build package
+        run: uv run just build

pyskillet-0.0.1/.github/workflows/changelog.yml

@@ -0,0 +1,31 @@
+name: Changelog
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, labeled, unlabeled]
+
+jobs:
+  check:
+    runs-on: ubuntu-latest
+    # Skip for dependabot PRs and PRs with skip-changelog label
+    if: github.actor != 'dependabot[bot]'
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+
+      - name: Skip if labeled
+        if: contains(github.event.pull_request.labels.*.name, 'skip-changelog')
+        run: echo "Skipping changelog check (skip-changelog label present)"
+
+      - name: Install uv
+        if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip-changelog') }}
+        uses: astral-sh/setup-uv@v7
+
+      - name: Install dependencies
+        if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip-changelog') }}
+        run: uv sync --extra dev
+
+      - name: Check changelog
+        if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip-changelog') }}
+        run: uv run just check-changelog origin/${{ github.base_ref }}

pyskillet-0.0.1/.github/workflows/docs.yml

@@ -0,0 +1,35 @@
+name: Docs
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - 'docs/**'
+  pull_request:
+    paths:
+      - 'docs/**'
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: docs
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: pnpm/action-setup@v4
+        with:
+          version: 10
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 20
+          cache: 'pnpm'
+          cache-dependency-path: docs/pnpm-lock.yaml
+
+      - name: Install dependencies
+        run: pnpm install
+
+      - name: Build docs
+        run: pnpm build

pyskillet-0.0.1/.github/workflows/integration.yml

@@ -0,0 +1,33 @@
+name: Integration Tests
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - '**.py'
+      - 'pyproject.toml'
+      - 'uv.lock'
+      - 'tests/integration/**'
+      - '.github/workflows/integration.yml'
+  pull_request:
+    paths:
+      - '**.py'
+      - 'pyproject.toml'
+      - 'uv.lock'
+      - 'tests/integration/**'
+      - '.github/workflows/integration.yml'
+
+jobs:
+  integration:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+
+      - name: Install dependencies
+        run: uv sync --extra dev
+
+      - name: Run integration tests
+        run: uv run just test-integration

pyskillet-0.0.1/.github/workflows/lint.yml

@@ -0,0 +1,34 @@
+name: Lint
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - '**.py'
+      - 'pyproject.toml'
+      - 'uv.lock'
+      - '.github/workflows/lint.yml'
+  pull_request:
+    paths:
+      - '**.py'
+      - 'pyproject.toml'
+      - 'uv.lock'
+      - '.github/workflows/lint.yml'
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+
+      - name: Install dependencies
+        run: uv sync --extra dev
+
+      - name: Lint
+        run: uv run just lint
+
+      - name: Format check
+        run: uv run just format-check

pyskillet-0.0.1/.github/workflows/pr-monitor.yml

@@ -0,0 +1,17 @@
+name: PR Monitor
+
+on:
+  pull_request:
+
+permissions:
+  checks: read
+
+jobs:
+  monitor:
+    name: 'CI Gate'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: clankerbot/pr-monitor@v1
+        with:
+          job-name: 'CI Gate'
+          excluded-jobs: 'deploy/netlify, Header rules, Pages changed, Redirect rules'

pyskillet-0.0.1/.github/workflows/release.yml

@@ -0,0 +1,94 @@
+name: Release
+
+on:
+  push:
+    tags:
+      - 'v*'
+
+jobs:
+  # Detect what files changed between this tag and the previous one
+  changes:
+    runs-on: ubuntu-latest
+    outputs:
+      package: ${{ steps.filter.outputs.package }}
+      docs: ${{ steps.filter.outputs.docs }}
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+
+      - name: Get previous tag
+        id: prev_tag
+        run: |
+          # Get all tags sorted by version, find the one before current
+          prev_tag=$(git tag --sort=-v:refname | grep -E '^v[0-9]' | sed -n '2p')
+          if [ -z "$prev_tag" ]; then
+            # If no previous tag, compare against initial commit
+            prev_tag=$(git rev-list --max-parents=0 HEAD)
+          fi
+          echo "prev_tag=$prev_tag" >> $GITHUB_OUTPUT
+
+      - name: Check changed files
+        id: filter
+        run: |
+          # Get list of changed files between previous tag and current
+          changed_files=$(git diff --name-only ${{ steps.prev_tag.outputs.prev_tag }}..HEAD)
+
+          # Check for package-related changes
+          if echo "$changed_files" | grep -qE '^(\.claude/|skillet/|pyproject\.toml|uv\.lock|\.python-version|LICENSE|README\.md)'; then
+            echo "package=true" >> $GITHUB_OUTPUT
+          else
+            echo "package=false" >> $GITHUB_OUTPUT
+          fi
+
+          # Check for docs changes
+          if echo "$changed_files" | grep -qE '^docs/'; then
+            echo "docs=true" >> $GITHUB_OUTPUT
+          else
+            echo "docs=false" >> $GITHUB_OUTPUT
+          fi
+
+  # Publish to PyPI if package files changed
+  pypi:
+    runs-on: ubuntu-latest
+    needs: changes
+    if: needs.changes.outputs.package == 'true'
+    permissions:
+      id-token: write  # Required for trusted publishing
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+
+      - name: Build package
+        run: uv build
+
+      - name: Publish to PyPI
+        run: uv publish --trusted-publishing always
+
+  # Deploy docs to GitHub Pages if docs files changed
+  docs:
+    runs-on: ubuntu-latest
+    needs: changes
+    if: needs.changes.outputs.docs == 'true'
+    permissions:
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Setup Pages
+        uses: actions/configure-pages@v5
+
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: 'docs/'
+
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

pyskillet-0.0.1/.github/workflows/security.yml

@@ -0,0 +1,29 @@
+name: Security
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - '**.py'
+      - 'pyproject.toml'
+      - '.github/workflows/security.yml'
+  pull_request:
+    paths:
+      - '**.py'
+      - 'pyproject.toml'
+      - '.github/workflows/security.yml'
+
+jobs:
+  bandit:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+
+      - name: Install dependencies
+        run: uv sync --extra dev
+
+      - name: Run security scan
+        run: uv run just security

pyskillet-0.0.1/.github/workflows/test.yml

@@ -0,0 +1,44 @@
+name: Test
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - '**.py'
+      - 'pyproject.toml'
+      - 'uv.lock'
+      - 'tests/**'
+      - '.github/workflows/test.yml'
+  pull_request:
+    paths:
+      - '**.py'
+      - 'pyproject.toml'
+      - 'uv.lock'
+      - 'tests/**'
+      - '.github/workflows/test.yml'
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: uv sync --extra dev
+
+      - name: Run tests
+        run: uv run just test-ci
+
+      - name: Upload coverage
+        uses: codecov/codecov-action@v5
+        with:
+          files: coverage.xml
+          fail_ci_if_error: false

pyskillet-0.0.1/.github/workflows/typecheck.yml

@@ -0,0 +1,29 @@
+name: Type Check
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - '**.py'
+      - 'pyproject.toml'
+      - '.github/workflows/typecheck.yml'
+  pull_request:
+    paths:
+      - '**.py'
+      - 'pyproject.toml'
+      - '.github/workflows/typecheck.yml'
+
+jobs:
+  typecheck:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+
+      - name: Install dependencies
+        run: uv sync --extra dev
+
+      - name: Type check
+        run: uv run just typecheck

pyskillet-0.0.1/.github/workflows/version-bump.yml

@@ -0,0 +1,97 @@
+name: Nightly Release
+
+on:
+  schedule:
+    # Run at 2:00 AM UTC every day
+    - cron: '0 2 * * *'
+  workflow_dispatch:  # Allow manual trigger
+
+jobs:
+  release:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Check for new commits since last tag
+        id: check
+        run: |
+          # Get the latest tag
+          latest_tag=$(git tag --sort=-v:refname | grep -E '^v[0-9]' | head -n1)
+
+          if [ -z "$latest_tag" ]; then
+            echo "No tags found, will create initial release"
+            echo "has_changes=true" >> $GITHUB_OUTPUT
+            echo "compare_base=$(git rev-list --max-parents=0 HEAD)" >> $GITHUB_OUTPUT
+          else
+            # Check if there are commits since the last tag
+            commits_since_tag=$(git rev-list ${latest_tag}..HEAD --count)
+            if [ "$commits_since_tag" -gt 0 ]; then
+              echo "Found $commits_since_tag commits since $latest_tag"
+              echo "has_changes=true" >> $GITHUB_OUTPUT
+              echo "compare_base=$latest_tag" >> $GITHUB_OUTPUT
+            else
+              echo "No new commits since $latest_tag"
+              echo "has_changes=false" >> $GITHUB_OUTPUT
+            fi
+          fi
+
+      - name: Configure git
+        if: steps.check.outputs.has_changes == 'true'
+        run: |
+          git config user.name "github-actions[bot]"
+          git config user.email "github-actions[bot]@users.noreply.github.com"
+
+      - name: Install uv
+        if: steps.check.outputs.has_changes == 'true'
+        uses: astral-sh/setup-uv@v7
+
+      - name: Install python-semantic-release
+        if: steps.check.outputs.has_changes == 'true'
+        run: uv tool install python-semantic-release
+
+      - name: Determine version bump
+        if: steps.check.outputs.has_changes == 'true'
+        id: version
+        run: |
+          compare_base="${{ steps.check.outputs.compare_base }}"
+
+          # Analyze all commits since last tag to determine bump type
+          # Check for breaking changes (highest priority)
+          if git log --format=%B ${compare_base}..HEAD | grep -qiE '(BREAKING CHANGE|^[a-z]+!:)'; then
+            echo "bump=major" >> $GITHUB_OUTPUT
+            echo "Detected: BREAKING CHANGE -> major bump"
+          # Check for features
+          elif git log --format=%B ${compare_base}..HEAD | grep -qE '^feat(\(.+\))?:'; then
+            echo "bump=minor" >> $GITHUB_OUTPUT
+            echo "Detected: feat commit -> minor bump"
+          # Default to patch
+          else
+            echo "bump=patch" >> $GITHUB_OUTPUT
+            echo "Detected: fixes/chores only -> patch bump"
+          fi
+
+      - name: Bump version
+        if: steps.check.outputs.has_changes == 'true'
+        id: bump
+        run: |
+          semantic-release version --${{ steps.version.outputs.bump }} --no-changelog --no-push --no-vcs-release
+          new_version=$(grep -Po '(?<=^version = ")[^"]+' pyproject.toml)
+          echo "new_version=$new_version" >> $GITHUB_OUTPUT
+
+      - name: Commit and tag
+        if: steps.check.outputs.has_changes == 'true'
+        run: |
+          git add pyproject.toml
+          git commit -m "chore: release v${{ steps.bump.outputs.new_version }}" || echo "No changes to commit"
+          git tag "v${{ steps.bump.outputs.new_version }}"
+
+      - name: Push changes and tag
+        if: steps.check.outputs.has_changes == 'true'
+        run: |
+          git push origin main
+          git push origin "v${{ steps.bump.outputs.new_version }}"

pyskillet-0.0.1/.gitignore

@@ -0,0 +1,22 @@
+# Python-generated files
+__pycache__/
+*.py[oc]
+build/
+dist/
+wheels/
+*.egg-info
+
+# Virtual environments
+.venv
+
+# Local/dev files
+scribbles
+tmp
+deprecated
+.claude/settings.local.json
+.coverage
+.pnpm-store
+.worktrees
+
+# Generated from .claude-template/
+.claude/commands/

pyskillet-0.0.1/.pre-commit-config.yaml

@@ -0,0 +1,9 @@
+repos:
+  - repo: local
+    hooks:
+      - id: pytest
+        name: pytest
+        entry: uv run pytest
+        language: system
+        types: [python]
+        pass_filenames: false

pyskillet-0.0.1/.python-version

@@ -0,0 +1 @@
+3.12

pyskillet-0.0.1/CHANGELOG.md

@@ -0,0 +1,44 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+### Added
+- Template-based command system for `{{SKILLET_DIR}}` substitution
+- `scripts/build_claude_config.py` for building `.claude/commands/` from templates
+- `just build-claude` command for template building
+- E2E test helpers with `Conversation` class for multi-turn testing
+- Auto-build of commands in e2e test conftest
+- 13 eval files for testing `/skillet:add` command behavior
+- Multi-turn conversation support in eval system
+- Session resumption for sequential prompts in evals
+- Automated nightly release workflow (2am UTC) triggered by git tags
+- Version bump automation using conventional commits
+- Conditional docs deployment to GitHub Pages
+- Commit convention documentation in `CLAUDE.md`
+
+### Changed
+- Renamed PyPI package from `skillet` to `pyskillet` (name was taken)
+- Renamed `/skillet:gap` command to `/skillet:add`
+- Increased `max_turns` from 3 to 10 for complex command workflows
+- Reorganized codebase into domain modules (eval/, compare/, tune/, skill/, gaps/)
+- Separated CLI layer from core API logic
+- Renamed public API functions: `run_eval` -> `evaluate`, `run_compare` -> `compare`, `run_tune` -> `tune`
+- Moved internal utilities to `_internal/` module
+
+### Added
+- Initial CLI with `eval`, `new`, `tune`, and `compare` commands
+- Gap capture via `/skillet:gap` Claude Code command
+- Evaluation caching system
+- LLM-as-judge for response validation
+- Live display with spinner animation for parallel evals
+- Python linting with Ruff
+- pytest testing infrastructure with pytest-describe
+- Pre-commit hooks for linting and tests
+- CI workflows for lint, test, and build
+- PR monitor for CI gating
+- PEP 561 py.typed marker for type checking support