airbyte-internal-ops 0.1.8__tar.gz

airbyte_internal_ops-0.1.8/.env.template

@@ -0,0 +1,13 @@
+## Copy this file to a new `.env` file and then fill in the values.
+## Important: DO NOT PUT SENSITIVE VALUES IN THIS FILE.
+
+# Creds for Airbyte Cloud OAuth
+AIRBYTE_CLOUD_CLIENT_ID="..."
+AIRBYTE_CLOUD_CLIENT_SECRET="..."
+
+# Required for elevated admin operations
+AIRBYTE_INTERNAL_ADMIN_FLAG=airbyte.io
+AIRBYTE_INTERNAL_ADMIN_USER={my-id}@airbyte.io
+
+# Workspace ID for Testing
+AIRBYTE_CLOUD_TEST_WORKSPACE_ID="..."

airbyte_internal_ops-0.1.8/.github/copilot-instructions.md

@@ -0,0 +1,123 @@
+# Copilot Instructions for airbyte-ops-mcp
+
+## Overview
+MCP and API interfaces for Airbyte administrative operations. Python 3.11+ package using `uv` for dependency management.
+
+**Key Info**: Package: `airbyte-internal-ops` | Module: `airbyte_ops_mcp` | Size: ~38 source files, ~29 tests | Coverage: 80% required
+
+## Structure
+- `src/airbyte_ops_mcp/` - Main source (`mcp/server.py` = MCP entry, `cli/app.py` = CLI entry)
+  - `mcp/` - MCP server tools | `cli/` - CLI commands | `cloud_admin/` - API client | `airbyte_repo/` - Repo ops
+  - `_legacy/` - **EXCLUDED** legacy code (issues #49-#56) - DO NOT modify
+- `tests/` - Tests (pytest) | `legacy/` - **EXCLUDED** from test runs
+- Config: `pyproject.toml`, `poe_tasks.toml`, `ruff.toml`, `pytest.ini`, `uv.lock` (committed)
+
+## Critical Build/Test Commands
+
+**ALWAYS run from repository root. All commands validated and timing documented.**
+
+### Setup (Required First)
+```bash
+uv sync --group dev  # ~60s first run, ~5s after. Install dev deps. REQUIRED before any work.
+```
+
+### Pre-Commit Checks (Run before every commit)
+```bash
+uv run ruff format .              # ~5s - Auto-format code
+uv run ruff check --fix .         # ~10s - Fix linting issues
+uv run pytest --exitfirst --cov=airbyte_ops_mcp --cov-report=term  # ~15s - Run tests
+```
+
+**Shortcut**: `uv run poe fix && uv run poe test-fast` (~25s total)
+
+### CI-Equivalent Checks
+```bash
+uv lock --check                   # ~5s - Verify lock file current
+uv run ruff format --check .      # ~5s - Check format (no changes)
+uv run ruff check .               # ~5s - Check linting
+uv run deptry .                   # ~10s - Check dependencies (warnings normal)
+uv run pytest --cov=airbyte_ops_mcp --cov-report=term  # ~20s - Full tests
+uv build                          # ~10s - Build package
+```
+
+**Shortcut**: `uv run poe check` (runs format-check, lint, deps, test)
+
+## CI/CD Workflows
+All run on PRs and main pushes. **Must pass to merge**:
+
+1. **Format Check**: `uv run ruff format --diff .` (~10s)
+2. **Lint Check**: `uv run ruff check .` + `uv run deptry .` (~25s)
+3. **Pytest Fast**: `uv lock --check` + `uv build` + pytest with `--exitfirst` on Python 3.11 (20min timeout)
+4. **Pytest Matrix**: Tests on Python 3.11, 3.12, 3.13 (30min timeout)
+
+**Known Issue**: CI uses wrong coverage path (`awesome_python_template` vs `airbyte_ops_mcp`) - tests still work correctly.
+
+## Common Issues & Workarounds
+
+### Legacy Code Exclusions
+**CRITICAL**: `src/airbyte_ops_mcp/_legacy/` and `tests/legacy/` are EXCLUDED from all linting/testing. Configured in `ruff.toml` and `conftest.py`. **DO NOT modify** unless working on migration (issues #49-#56).
+
+### Test Marker Warnings
+`@pytest.mark.unit` and `@pytest.mark.integration` trigger warnings (known issue in pytest.ini) but work correctly.
+
+### Environment Setup
+Create `.env` from `.env.template` (NEVER commit `.env`):
+- `AIRBYTE_CLOUD_CLIENT_ID/SECRET` - OAuth
+- `AIRBYTE_INTERNAL_ADMIN_FLAG/USER` - Admin ops
+- `AIRBYTE_CLOUD_TEST_WORKSPACE_ID` - Testing
+
+### Coverage Requirements
+80% minimum (pytest.ini). Current: ~40%. Add tests for new code.
+
+## Development Workflow
+
+**Standard sequence**:
+1. `uv sync --group dev` (if dependencies changed)
+2. Make changes
+3. `uv run ruff format .` + `uv run ruff check --fix .`
+4. `uv run pytest --exitfirst --cov=airbyte_ops_mcp --cov-report=term`
+5. `uv build` (verify package builds)
+6. Commit/push
+
+**Fast iteration**: `uv run poe fix && uv run poe test-fast`
+
+## Poe Task Reference
+```bash
+uv run poe fix           # Format + lint auto-fix
+uv run poe test-fast     # Tests with --exitfirst
+uv run poe check         # All checks (format, lint, deps, test)
+uv run poe clean         # Remove artifacts
+```
+
+## Key Files & Patterns
+
+**Config Files**:
+- `pyproject.toml` - Dependencies (use `uv sync` to update), metadata
+- `poe_tasks.toml` - Task definitions
+- `ruff.toml` - Lint/format rules, legacy exclusions
+- `pytest.ini` - Test config, 80% coverage, legacy exclusions
+
+**Important Patterns**:
+- Use `uv` exclusively (not pip/poetry)
+- Run all commands from repo root
+- Validate locally before pushing (all CI checks runnable locally)
+- Package name: `airbyte-internal-ops` | Import name: `airbyte_ops_mcp`
+- Entry points: `airbyte-ops-mcp` (MCP server), `airbyte-ops` (CLI)
+
+## Repository Details
+
+**Root Files**: `.env.template`, `.gitignore`, `.mcp.json.template`, `CONTRIBUTING.md`, `README.md`, `bin/`, `conftest.py`, `poe_tasks.toml`, `pyproject.toml`, `pytest.ini`, `ruff.toml`, `src/`, `tests/`, `uv.lock`
+
+**Dependencies**: fastmcp (MCP), airbyte/airbyte-cdk (SDK), cyclopts/click (CLI), pydantic (validation), ruff/pytest/deptry (dev tools)
+
+**Test Utilities**: `bin/test_mcp_tool.py` - Test MCP tools: `uv run poe mcp-tool-test <tool> '<json>'`
+
+## Agent Guidelines
+
+1. **Use `uv` only** - No pip/poetry
+2. **Run `uv sync --group dev` first** - Always
+3. **Test locally** - All CI checks available locally
+4. **Avoid legacy code** - Don't touch `_legacy/` dirs
+5. **Meet coverage** - 80% required
+6. **Format/lint** - CI fails without
+7. **Trust these instructions** - All validated. Search only if incomplete/incorrect.

airbyte_internal_ops-0.1.8/.github/dependabot.yml

@@ -0,0 +1,35 @@
+# To get started with Dependabot version updates, you'll need to specify which
+# package ecosystems to update and where the package manifests are located.
+# Please see the documentation for all configuration options:
+# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
+
+version: 2
+updates:
+  - package-ecosystem: pip
+    directory: "/" # Location of package manifests
+    commit-message:
+      prefix: "chore(deps): "
+    schedule:
+      interval: daily
+    labels:
+      - chore
+      - dependencies
+    open-pull-requests-limit: 8 # default is 5
+
+  - package-ecosystem: github-actions
+    open-pull-requests-limit: 5 # default is 5
+    directory: "/"
+    commit-message:
+      prefix: "ci(deps): "
+    schedule:
+      interval: monthly
+    labels:
+      - ci
+      - dependencies
+    groups:
+      # allow combining github-actions updates into a single PR
+      minor-and-patch:
+        applies-to: version-updates
+        update-types:
+          - patch
+          - minor

airbyte_internal_ops-0.1.8/.github/pr-welcome-message.md

@@ -0,0 +1,32 @@
+🎉 **Thanks for opening this pull request!**
+
+Your contribution is appreciated. Here are some helpful commands you can use:
+
+## Quick Commands
+- `/autofix` - Auto-format and fix linting issues (ruff format + ruff check --fix)
+- `/lock` - Update the uv.lock file with latest dependencies
+
+## Available Poe Tasks
+You can run any of these tasks using the slash command: `/poe <task-name>`
+
+### Core Tasks
+- `/poe test` - Run all tests
+- `/poe test-fast` - Run tests with fast exit on first failure
+- `/poe lint` - Check code style and quality  
+- `/poe format` - Format code with ruff
+- `/poe deps` - Check for unused and missing dependencies
+- `/poe check` - Run format check, linting, dependency check, and tests
+
+### Quick Fixes
+- `/poe fix` - Auto-format and fix linting issues
+- `/poe clean` - Clean up build artifacts and cache
+
+### Build & Install
+- `/poe build` - Build the package
+- `/poe install` - Install with development dependencies
+
+### Other Commands
+- `/poe version` - Show package version
+- `/poe pre-commit` - Run pre-commit style checks
+
+The CI will automatically run tests when you push commits. Happy coding! 🚀

airbyte_internal_ops-0.1.8/.github/release-drafter.yml

@@ -0,0 +1,67 @@
+name-template: "v$RESOLVED_VERSION"
+tag-template: "v$RESOLVED_VERSION"
+categories:
+  - title: "New Features ✨"
+    labels:
+      - "feature"
+      - "enhancement"
+  - title: "Bug Fixes 🐛"
+    labels:
+      - "fix"
+      - "bugfix"
+      - "bug"
+  - title: "Under the Hood ⚙️"
+    labels:
+      - "chore"
+      - "ci"
+      - "refactor"
+      - "testing"
+  - title: "Documentation 📖"
+    label: "docs"
+change-template: "- $TITLE (#$NUMBER)"
+change-title-escapes: '\<*_&' # You can add # and @ to disable mentions, and add ` to disable code blocks.
+version-resolver:
+  major:
+    labels:
+      - "major"
+  minor:
+    labels:
+      - "minor"
+  patch:
+    labels:
+      - "patch"
+  default: patch
+template: |
+  ## Changes
+
+  $CHANGES
+autolabeler:
+  - label: "chore"
+    title:
+      - '/^chore(\(.*\))?\:/i'
+  - label: "testing"
+    title:
+      - '/^tests(\(.*\))?\:/i'
+  - label: "ci"
+    title:
+      - '/^ci(\(.*\))?\:/i'
+  - label: "bug"
+    title:
+      - '/^fix(\(.*\))?\:/i'
+  - label: "enhancement"
+    title:
+      - '/^feat(\(.*\))?/i'
+  - label: "docs"
+    title:
+      - '/^docs(\(.*\))?\:/i'
+  - label: "security"
+    title:
+      - '/^security(\(.*\))?\:/i'
+      - '/^fix(\(security\))?\:/i'
+  - label: "dependencies"
+    title:
+      - '/^chore\(deps\)\:/i'
+      - '/^build\(deps\)\:/i'
+  - label: "breaking"
+    title:
+      - '/!:\s*$/i'

airbyte_internal_ops-0.1.8/.github/workflows/autofix-command.yml

@@ -0,0 +1,37 @@
+name: On-Demand Fix
+
+on:
+  workflow_dispatch:
+    inputs:
+      pr:
+        description: "PR Number"
+        type: string
+        required: false
+      comment-id:
+        description: "Comment ID (Optional)"
+        type: string
+        required: false
+
+permissions:
+  contents: write
+  pull-requests: write
+
+jobs:
+  run-autofix-command:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Authenticate as GitHub App (if available)
+        uses: actions/create-github-app-token@v2
+        id: get-app-token
+        continue-on-error: true
+        with:
+          app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
+          private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}
+
+      - name: Run Poe Autofix Command
+        uses: aaronsteers/poe-command-processor@v1.3.3
+        with:
+          pr: ${{ github.event.inputs.pr }}
+          comment-id: ${{ github.event.inputs.comment-id }}
+          github-token: ${{ steps.get-app-token.outputs.token || secrets.GITHUB_TOKEN }}
+          command: autofix

airbyte_internal_ops-0.1.8/.github/workflows/connector-live-test.yml

@@ -0,0 +1,88 @@
+name: Connector Live Test
+permissions:
+  contents: read
+
+on:
+  workflow_dispatch:
+    inputs:
+      connector_image:
+        description: "Full connector image name with tag (e.g., airbyte/source-github:1.0.0). Optional if connector_name or connection_id is provided."
+        required: false
+        type: string
+      connector_name:
+        description: "Connector name to build from source (e.g., 'source-pokeapi'). If provided, builds the image locally."
+        required: false
+        type: string
+      command:
+        description: "Airbyte command to run"
+        required: false
+        type: choice
+        options:
+          - spec
+          - check
+          - discover
+          - read
+        default: check
+      connection_id:
+        description: "Airbyte Cloud connection ID to fetch config/catalog from. If provided, connector_image can be auto-detected."
+        required: false
+        type: string
+      state_path:
+        description: "Path to state JSON file (optional for read)"
+        required: false
+        type: string
+
+jobs:
+  live-test:
+    name: Run Live Test
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          version: "latest"
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: "3.11"
+
+      - name: Install dependencies
+        run: uv sync --group dev
+
+      - name: Run Live Test
+        id: live-test
+        env:
+          AIRBYTE_CLOUD_CLIENT_ID: ${{ secrets.AIRBYTE_CLOUD_CLIENT_ID }}
+          AIRBYTE_CLOUD_CLIENT_SECRET: ${{ secrets.AIRBYTE_CLOUD_CLIENT_SECRET }}
+        run: |
+          ARGS=(--command "${{ inputs.command }}")
+
+          if [ -n "${{ inputs.connector_image }}" ]; then
+            ARGS+=(--connector-image "${{ inputs.connector_image }}")
+          fi
+
+          if [ -n "${{ inputs.connector_name }}" ]; then
+            ARGS+=(--connector-name "${{ inputs.connector_name }}")
+          fi
+
+          if [ -n "${{ inputs.connection_id }}" ]; then
+            ARGS+=(--connection-id "${{ inputs.connection_id }}")
+          fi
+
+          if [ -n "${{ inputs.state_path }}" ]; then
+            ARGS+=(--state-path "${{ inputs.state_path }}")
+          fi
+
+          uv run airbyte-ops cloud connector live-test "${ARGS[@]}"
+
+      - name: Upload Test Artifacts
+        if: always()
+        uses: actions/upload-artifact@v5
+        with:
+          name: live-test-artifacts-${{ github.run_id }}
+          path: /tmp/live_test_artifacts/
+          retention-days: 7

airbyte_internal_ops-0.1.8/.github/workflows/connector-regression-test.yml

@@ -0,0 +1,188 @@
+name: Connector Regression Test
+permissions:
+  contents: read
+  checks: write
+
+on:
+  workflow_dispatch:
+    inputs:
+      target_image:
+        description: "Target connector image (new version) with tag (e.g., airbyte/source-github:2.0.0). Optional if connector_name is provided."
+        required: false
+        type: string
+      control_image:
+        description: "Control connector image (baseline version) with tag (e.g., airbyte/source-github:1.0.0). Optional if connection_id is provided (auto-detected)."
+        required: false
+        type: string
+      connector_name:
+        description: "Connector name to build target image from source (e.g., 'source-pokeapi'). If provided, builds the target image locally."
+        required: false
+        type: string
+      command:
+        description: "Airbyte command to run"
+        required: false
+        type: choice
+        options:
+          - spec
+          - check
+          - discover
+          - read
+        default: check
+      connection_id:
+        description: "Airbyte Cloud connection ID to fetch config/catalog from. If provided, control_image can be auto-detected."
+        required: false
+        type: string
+      state_path:
+        description: "Path to state JSON file (optional for read)"
+        required: false
+        type: string
+
+  workflow_call:
+    inputs:
+      target_image:
+        description: "Target connector image (new version) with tag. Optional if connector_name is provided."
+        required: false
+        type: string
+      control_image:
+        description: "Control connector image (baseline version) with tag. Optional if connection_id is provided."
+        required: false
+        type: string
+      connector_name:
+        description: "Connector name to build target image from source (e.g., 'source-pokeapi')."
+        required: false
+        type: string
+      command:
+        description: "Airbyte command to run (spec, check, discover, read)"
+        required: false
+        type: string
+        default: check
+      connection_id:
+        description: "Airbyte Cloud connection ID to fetch config/catalog from. If provided, control_image can be auto-detected."
+        required: false
+        type: string
+      state_path:
+        description: "Path to state JSON file (optional for read)"
+        required: false
+        type: string
+    outputs:
+      success:
+        description: "True if regression test passed (no regression detected)"
+        value: ${{ jobs.regression-test.outputs.success }}
+      summary_url:
+        description: "GitHub Actions run summary URL"
+        value: ${{ jobs.regression-test.outputs.summary_url }}
+      report_url:
+        description: "Detailed regression report (GitHub Check URL)"
+        value: ${{ jobs.regression-test.outputs.report_url }}
+    secrets:
+      AIRBYTE_CLOUD_CLIENT_ID:
+        required: false
+      AIRBYTE_CLOUD_CLIENT_SECRET:
+        required: false
+
+jobs:
+  regression-test:
+    name: Run Regression Test
+    runs-on: ubuntu-latest
+    outputs:
+      success: ${{ steps.regression-test.outputs.success }}
+      summary_url: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+      report_url: ${{ steps.create-report-check.outputs.report_url }}
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          version: "latest"
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: "3.11"
+
+      - name: Install dependencies
+        run: uv sync --group dev
+
+      - name: Run Regression Test
+        id: regression-test
+        env:
+          AIRBYTE_CLOUD_CLIENT_ID: ${{ secrets.AIRBYTE_CLOUD_CLIENT_ID }}
+          AIRBYTE_CLOUD_CLIENT_SECRET: ${{ secrets.AIRBYTE_CLOUD_CLIENT_SECRET }}
+        run: |
+          ARGS=(--command "${{ inputs.command }}")
+
+          if [ -n "${{ inputs.target_image }}" ]; then
+            ARGS+=(--target-image "${{ inputs.target_image }}")
+          fi
+
+          if [ -n "${{ inputs.control_image }}" ]; then
+            ARGS+=(--control-image "${{ inputs.control_image }}")
+          fi
+
+          if [ -n "${{ inputs.connector_name }}" ]; then
+            ARGS+=(--connector-name "${{ inputs.connector_name }}")
+          fi
+
+          if [ -n "${{ inputs.connection_id }}" ]; then
+            ARGS+=(--connection-id "${{ inputs.connection_id }}")
+          fi
+
+          if [ -n "${{ inputs.state_path }}" ]; then
+            ARGS+=(--state-path "${{ inputs.state_path }}")
+          fi
+
+          uv run airbyte-ops cloud connector regression-test "${ARGS[@]}"
+
+      - name: Create Regression Test Report Check
+        id: create-report-check
+        if: always()
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const fs = require('fs');
+            const reportPath = '/tmp/regression_test_artifacts/report.md';
+            
+            let reportContent = 'Report not generated';
+            if (fs.existsSync(reportPath)) {
+              reportContent = fs.readFileSync(reportPath, 'utf8');
+            }
+            
+            const conclusion = '${{ steps.regression-test.outcome }}' === 'success' ? 'success' : 'failure';
+            
+            const check = await github.rest.checks.create({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              name: 'Regression Test Report',
+              head_sha: context.sha,
+              status: 'completed',
+              conclusion: conclusion,
+              output: {
+                title: 'Regression Test Report',
+                summary: `Regression test comparing \`${{ inputs.target_image }}\` vs \`${{ inputs.control_image }}\` using \`${{ inputs.command }}\` command.`,
+                text: reportContent
+              }
+            });
+            
+            core.setOutput('report_url', check.data.html_url);
+
+      - name: Add Report Link to Summary
+        if: always()
+        run: |
+          {
+            echo ""
+            echo "---"
+            echo ""
+            echo "### View Full Report"
+            echo ""
+            echo "[Open Regression Test Report](${{ steps.create-report-check.outputs.report_url }})"
+          } >> "$GITHUB_STEP_SUMMARY"
+
+      - name: Upload Test Artifacts
+        if: always()
+        uses: actions/upload-artifact@v5
+        with:
+          name: regression-test-artifacts-${{ github.run_id }}
+          path: /tmp/regression_test_artifacts/
+          retention-days: 7

airbyte_internal_ops-0.1.8/.github/workflows/format-check.yml

@@ -0,0 +1,31 @@
+name: Format Check
+
+on:
+  push:
+    branches:
+      - main
+  pull_request: {}
+
+jobs:
+  ruff-format-check:
+    name: Ruff Format Check
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          version: "latest"
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: "3.11"
+
+      - name: Install dependencies
+        run: uv sync --group dev
+
+      - name: Check code format
+        run: uv run ruff format --diff .

airbyte_internal_ops-0.1.8/.github/workflows/lint-check.yml

@@ -0,0 +1,80 @@
+name: Lint Check
+
+on:
+  push:
+    branches:
+      - main
+  pull_request: {}
+
+permissions:
+  contents: read
+
+jobs:
+  ruff-lint-check:
+    name: Ruff Lint Check
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          version: "latest"
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: "3.11"
+
+      - name: Install dependencies
+        run: uv sync --group dev
+
+      - name: Run lint check
+        run: uv run ruff check .
+
+  deptry-check:
+    name: Dependency Analysis
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          version: "latest"
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: "3.11"
+
+      - name: Install dependencies
+        run: uv sync --group dev
+
+      - name: Run dependency analysis
+        run: uv run deptry .
+
+  pyrefly-check:
+    name: Type Checking
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          version: "latest"
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: "3.11"
+
+      - name: Install dependencies
+        run: uv sync --group dev
+
+      - name: Run type check
+        run: uv run pyrefly check