@a5c-ai/kradle 5.0.1-staging.3abdf9534c25

package/docs/external/README.md

@@ -0,0 +1,47 @@
+# External backend integration docs
+
+## Purpose
+
+This directory defines how Kradle should integrate with GitHub first and support other externally managed backends later. External backends can implement one, two, or all three Kradle provider interfaces:
+
+1. issue tracking and work management sync;
+2. CI/CD, triggers, runners, pipelines, checks, and workflow sync;
+3. git forge sync for repositories, pull requests, refs, commits, SSH/deploy keys, collaborators, and repository policy.
+
+The design supports bidirectional, efficient sync without forcing every backend to be a full forge.
+
+## Documents
+
+- [Research results](./research-results.md) summarizes GitHub API and webhook capabilities used by this design.
+- [Pluggable backend provider catalog](./provider-catalog.md) lists GitHub, GitLab, Bitbucket, Azure DevOps, Jira, Linear, Buildkite, CircleCI, Jenkins, Gitea, Gerrit, raw Git, and custom providers by supported interface.
+
+- [Unified external backend model](./unified-external-backend-model.md) defines provider capabilities, ownership, identifiers, sync modes, and the three interface split.
+- [Provider capability manifests](./provider-capability-manifests.md) defines data-driven adapter manifests for operations, auth, webhooks, and tests.
+- [External object mapping spec](./external-object-mapping.md) defines loss-aware mappings from provider objects to Kradle resources.
+
+- [Issue tracking interface](./issue-tracking-interface.md) defines issue/comment/label/milestone/project/work-item sync.
+- [CI/CD interface](./cicd-interface.md) defines workflow/check/pipeline/job/runner/trigger sync.
+- [Git forge interface](./git-forge-interface.md) defines repository/PR/ref/commit/key/collaborator/policy sync.
+- [GitHub integration design](./github-integration-design.md) maps GitHub App, REST, GraphQL, webhooks, and Actions onto the three interfaces.
+- [Efficient bidirectional sync](./bidirectional-sync-design.md) defines webhook-first, cursor-based, idempotent reconciliation and conflict handling.
+- [External sync state machines](./sync-state-machines.md) defines stable phases for providers, bindings, deliveries, backfill, writes, and conflicts.
+
+- [External backend CRDs](./external-backend-crds.md) defines resources and schemas.
+- [External backend controllers](./external-backend-controllers.md) defines reconciliation loops and side effects.
+- [User-facing changes](./user-facing-changes.md) defines UI, settings, status, and workflow changes.
+- [External backend UI specification](./external-backend-ui-spec.md) defines org-scoped provider setup, bindings, sync health, conflicts, write intents, webhook, and repository UI.
+- [External backend UX flows](./external-backend-ux-flows.md) defines user-facing setup, mixed-provider, conflict, write approval, recovery, and rate-limit flows.
+
+
+- [Security, auth, and permissions](./security-auth-permissions.md) defines GitHub App auth, tokens, secrets, RBAC, and audit.
+- [Provider rollout and testing](./provider-rollout-testing.md) defines implementation slices, validation, and QA coverage.
+
+## Design principles
+
+- Kradle keeps its org namespace and resource model; external backends are providers, not the source of Kradle tenancy.
+- Providers declare which interfaces they support.
+- Every external object stores provider ID, native ID, global node ID when available, URL, etag/cursor, and last synced generation.
+- Webhooks are the primary freshness mechanism; polling/backfill repairs missed or truncated events.
+- Kradle can run in mirror mode, bidirectional mode, or Kradle-owned mode per interface and resource type.
+- Conflicts are explicit resources and UI states, not silent overwrites.
+- Secrets and provider tokens stay in Kubernetes Secrets and are surfaced only as grant metadata.

package/docs/external/bidirectional-sync-design.md

@@ -0,0 +1,134 @@
+# Efficient bidirectional sync design
+
+## Purpose
+
+External backends need efficient two-way sync that converges quickly without overwriting user changes or exhausting provider rate limits. This document defines the common sync strategy for all provider interfaces.
+
+## Sync layers
+
+| Layer | Purpose |
+| --- | --- |
+| Webhook ingest | near-real-time event capture and trigger source. |
+| Event normalizer | convert provider payloads into canonical Kradle sync events. |
+| Cursor backfill | recover missed events and hydrate lists. |
+| Object reconciler | compare desired/local/provider state and write projections. |
+| Write intent queue | apply Kradle-originated writes to provider with retries. |
+| Conflict detector | detect local/external divergence. |
+| Audit/event stream | explain every sync and write. |
+
+## Sync resource model
+
+```yaml
+kind: ExternalSyncState
+spec:
+  organizationRef: a5c
+  providerRef: github-a5c
+  bindingRef: github-kradle
+  interface: gitForge
+  resourceKind: PullRequest
+status:
+  highWatermark: 2026-05-11T12:00:00Z
+  cursor: opaque-provider-cursor
+  lastWebhookDeliveryId: "..."
+  lastFullBackfillAt: 2026-05-11T00:00:00Z
+  phase: Ready
+```
+
+```yaml
+kind: ExternalSyncConflict
+spec:
+  organizationRef: a5c
+  providerRef: github-a5c
+  resourceRef:
+    kind: Issue
+    name: issue-42
+  fieldConflicts:
+    - field: labels
+      local: [bug, priority]
+      external: [bug]
+  resolutionPolicy: manual
+```
+
+## Event processing
+
+```text
+provider webhook
+  -> validate signature
+  -> persist ExternalWebhookDelivery
+  -> enqueue by provider installation and repository
+  -> normalize into ExternalSyncEvent
+  -> dedupe by delivery ID + action + native object ID
+  -> apply object-specific reconcile
+  -> update Kradle projection and sync state
+  -> emit audit and watch event
+```
+
+## Backfill processing
+
+```text
+scheduled or manual backfill
+  -> read ExternalSyncState cursor/highWatermark
+  -> list changed objects from provider
+  -> hydrate missing details
+  -> upsert Kradle projections
+  -> mark deleted/missing objects according to tombstone policy
+  -> update cursor/highWatermark
+```
+
+## Write processing
+
+```text
+Kradle user/agent action
+  -> admission and RBAC
+  -> create ExternalWriteIntent
+  -> optional approval
+  -> provider write through connector
+  -> verify provider response
+  -> update local projection with provider IDs/version
+  -> wait for webhook or backfill confirmation
+  -> close write intent
+```
+
+## Efficiency rules
+
+- Prefer webhook payloads for targeted updates.
+- Use GraphQL/cursor pagination for bulk list hydration where supported.
+- Use REST endpoints for provider-specific operations and logs/artifacts.
+- Store ETag or provider resource version when available.
+- Batch by installation/org/repository to respect rate limits.
+- Lazy-load large logs, diffs, artifacts, and comments.
+- Apply bounded retries with dead-letter status for repeated provider errors.
+- Separate sync freshness from user-facing last-updated time.
+
+## Conflict rules
+
+Conflict when:
+
+- local desired generation changed after last sync and provider field also changed;
+- provider rejects a write because native version/precondition changed;
+- provider has a value Kradle cannot represent losslessly;
+- ownership mode says external-owned and Kradle has pending local mutation;
+- write intent remains unconfirmed beyond timeout.
+
+Resolution options:
+
+- prefer external;
+- prefer Kradle desired;
+- manual merge;
+- create reviewed provider-side change;
+- ignore unsupported field with warning.
+
+## Deletion and tombstones
+
+- External deletions become tombstones before local deletion when audit requires retention.
+- Kradle deletions in mirror mode should not delete provider objects.
+- Kradle-owned resources may delete provider objects if admission and provider permissions allow it.
+- PR/issue deletion may be unsupported in some providers; close/archive instead.
+
+## Acceptance criteria
+
+- Webhook replay and backfill converge to the same resource state.
+- Duplicate webhooks are idempotent.
+- Rate-limit responses slow sync without losing events.
+- Conflicts are visible in UI and API.
+- Writes are auditable from Kradle action to provider confirmation.

package/docs/external/cicd-interface.md

@@ -0,0 +1,64 @@
+# CI/CD interface
+
+## Purpose
+
+The CI/CD interface syncs external workflow/check/pipeline state into Kradle and can trigger or control external runs when allowed. It covers workflows, workflow runs, jobs, logs, artifacts, checks, commit statuses, runner groups, and self-hosted runners.
+
+## Provider contract
+
+```ts
+interface CicdProvider {
+  listPipelines(cursor): Page<ExternalPipeline>;
+  getPipeline(ref): ExternalPipeline;
+  listJobs(pipelineRef, cursor): Page<ExternalJob>;
+  getJobLog(jobRef): ExternalLogRef;
+  listArtifacts(pipelineRef, cursor): Page<ExternalArtifact>;
+  rerunPipeline(ref, options): ExternalPipeline;
+  cancelPipeline(ref): ExternalPipeline;
+  listRunners(cursor): Page<ExternalRunner>;
+  registerRunner(scope, options): RunnerRegistration;
+  createCheck(input): ExternalCheck;
+  updateCheck(ref, patch): ExternalCheck;
+}
+```
+
+Providers can implement checks/statuses without implementing runner management.
+
+## Resource mapping
+
+| External concept | Kradle resource/projection |
+| --- | --- |
+| workflow/workflow definition | `PipelineTemplate` projection or provider metadata |
+| workflow run/pipeline | `Pipeline` |
+| job/step | `Job` |
+| check run/status | `CheckRun` projection or `Job.status.checks` |
+| runner | `RunnerPool` / `Runner` projection |
+| artifact/log | `Artifact` / object-storage reference |
+| trigger event | `WebhookDelivery` / `ExternalSyncEvent` |
+
+## GitHub mapping
+
+GitHub Actions workflow runs map to `Pipeline`; workflow jobs map to `Job`; check runs and commit statuses map to check projections and PR gates. GitHub self-hosted runners map to runner inventory and runner registration flows when Kradle is allowed to manage them.
+
+## Sync rules
+
+- Webhooks handle `workflow_run`, `workflow_job`, `check_run`, `check_suite`, `status`, and `push` events.
+- Backfill periodically lists workflow runs/jobs by repository and updated timestamp.
+- Logs and artifacts are lazy-loaded and stored by digest or external URL depending on retention policy.
+- Rerun/cancel actions require permission review and provider capability.
+- External runner registration tokens are short-lived and never stored as plain status.
+
+## User-facing changes
+
+- Repository Runs page shows external pipelines next to Kradle-native runs.
+- Run detail badges show external provider and native link.
+- Rerun/cancel buttons are disabled unless provider and RBAC allow them.
+- Runner pages distinguish Kradle-managed, provider-managed, and mirrored runners.
+- Agent triggers can subscribe to external CI failure events through the same trigger rule model.
+
+## Acceptance criteria
+
+- A CI-only provider can sync pipelines/jobs without repo/issue ownership.
+- GitHub workflow jobs converge through webhook and backfill.
+- Logs/artifacts are fetched lazily and redacted according to policy.
+- Rerun/cancel actions are audited and idempotent.

package/docs/external/external-backend-controllers.md

@@ -0,0 +1,170 @@
+# External backend controllers
+
+## Purpose
+
+External backend controllers reconcile provider configuration, webhook deliveries, backfill, object projection, writes, conflicts, and status.
+
+## Controller set
+
+| Controller | Responsibilities |
+| --- | --- |
+| provider controller | validate auth, capabilities, installation access, rate limits, and status. |
+| binding controller | validate target refs, create provider webhooks, initialize sync states. |
+| webhook controller | validate signatures, persist deliveries, enqueue events, support replay. |
+| sync controller | process events/backfills and update Kradle projections. |
+| write controller | apply Kradle write intents to provider, retry, confirm, audit. |
+| conflict controller | detect field/resource conflicts and manage resolution workflow. |
+| runner/controller adapter | manage external CI runners when provider supports it. |
+| garbage/tombstone controller | handle external deletions and retention. |
+
+## Reconciliation order
+
+```text
+ExternalBackendProvider
+  -> auth/capability check
+  -> ExternalBackendBinding
+  -> webhook registration and sync state initialization
+  -> webhook events and backfill
+  -> Kradle resource projections
+  -> write intents and conflicts
+```
+
+## Provider controller
+
+- Resolve org namespace and Secret refs.
+- Validate provider type and base URLs.
+- Verify credentials without storing token values.
+- Discover capabilities where possible.
+- Track rate-limit status and degraded state.
+- Emit `AuthReady`, `InstallationReady`, and interface readiness conditions.
+
+## Binding controller
+
+- Validate target resource belongs to same org.
+- Validate provider supports requested interfaces.
+- Create/update provider webhooks when Kradle owns webhook configuration.
+- Create initial `ExternalSyncState` objects.
+- Kick off initial backfill.
+
+## Webhook controller
+
+- Validate HMAC/signature before accepting payload.
+- Persist `ExternalWebhookDelivery` with provider delivery ID.
+- Dedupe repeated deliveries.
+- Enqueue normalized `ExternalSyncEvent`.
+- Return quickly and process asynchronously.
+- Support manual replay/redelivery records.
+
+## Sync controller
+
+- Hydrate provider objects from webhook payload or API.
+- Upsert Kradle resources/projections with external identity fields.
+- Maintain high-watermarks and cursors.
+- Respect ownership mode.
+- Mark tombstones for external deletions.
+- Emit watch and audit events.
+
+## Write controller
+
+- Reads `ExternalWriteIntent` after Kradle admission and optional approval.
+- Applies provider write with provider-specific idempotency where available.
+- Handles rate limits and retryable failures.
+- Confirms via provider response, webhook, or follow-up read.
+- Creates conflict if provider state diverged.
+
+## Controller acceptance criteria
+
+- Controllers are idempotent by provider, installation, interface, native object ID, and delivery/write ID.
+- Provider outage degrades sync without corrupting Kradle state.
+- Cross-org references fail before provider calls.
+- Secret/token values never enter status, events, logs, or sync payloads.
+- Webhook replay and cursor backfill converge.
+
+## Interface adapter controllers
+
+Each interface has a provider-neutral reconciler and provider-specific adapter methods.
+
+### Issue sync controller
+
+Responsibilities:
+
+- watch issue-related webhooks;
+- backfill issues, comments, labels, milestones, project fields;
+- upsert `Issue` projections;
+- link PR-backed issue numbers to `PullRequest`;
+- process issue write intents;
+- detect comment/label/state conflicts.
+
+### CI/CD sync controller
+
+Responsibilities:
+
+- watch workflow/check/status events;
+- backfill pipelines, jobs, checks, logs, artifacts;
+- upsert `Pipeline` and `Job` projections;
+- lazy-fetch logs/artifacts on demand;
+- process rerun/cancel/check update write intents;
+- sync runner inventory where supported.
+
+### Git forge sync controller
+
+Responsibilities:
+
+- watch repository, PR, review, push, branch/tag, key, collaborator, and protection events;
+- backfill repos, pull requests, refs, branch protection, keys, collaborators;
+- upsert `Repository`, `PullRequest`, `Review`, `SSHKey`, `RepositoryPermission`, `BranchProtection`, and `RefPolicy` projections;
+- process PR, merge, key, collaborator, and branch protection writes;
+- detect force-push and stale diff/check state.
+
+## Provider adapter lifecycle
+
+```text
+load provider descriptor
+  -> validate configured interfaces
+  -> create adapter client with scoped credentials
+  -> run health/capability probe
+  -> start webhook/backfill loops
+  -> expose provider operations to sync/write controllers
+```
+
+## Rate-limit handling
+
+Controllers should:
+
+- bucket requests by provider, installation/account, org, and repository;
+- preserve webhook deliveries even when rate limited;
+- pause backfill before write intents when budget is low;
+- expose `RateLimited` conditions with reset time;
+- avoid retry storms by using exponential backoff and jitter.
+
+## Provider plugin contract
+
+Future provider plugins should implement:
+
+```ts
+interface ExternalProviderAdapter {
+  descriptor(): ProviderDescriptor;
+  health(): ProviderHealth;
+  issueTracking?: IssueTrackingProvider;
+  cicd?: CicdProvider;
+  gitForge?: GitForgeProvider;
+  normalizeWebhook(payload): NormalizedExternalEvent[];
+  verifyWebhook(request): VerificationResult;
+}
+```
+
+The core controllers own persistence, org checks, queueing, conflicts, and audit; adapters only translate provider operations.
+
+## Controller status surfaces
+
+Provider and binding status should expose:
+
+- interface readiness;
+- last successful webhook;
+- last failed webhook;
+- last backfill by interface;
+- queue depth;
+- rate limit remaining/reset;
+- conflicts count;
+- pending write count;
+- last provider error class.

package/docs/external/external-backend-crds.md

@@ -0,0 +1,234 @@
+# External backend CRDs
+
+## Purpose
+
+This document defines the resource contracts for external backend providers, bindings, sync state, write intents, conflicts, and webhook deliveries.
+
+## Config resources
+
+### `ExternalBackendProvider`
+
+```yaml
+apiVersion: kradle.a5c.ai/v1alpha1
+kind: ExternalBackendProvider
+metadata:
+  name: github-a5c
+  namespace: kradle-org-a5c
+spec:
+  organizationRef: a5c
+  providerType: github
+  displayName: GitHub a5c-ai
+  baseUrl: https://github.com
+  apiBaseUrl: https://api.github.com
+  authRef:
+    secretRef:
+      name: github-app-a5c
+  capabilities:
+    issueTracking: true
+    cicd: true
+    gitForge: true
+status:
+  phase: Ready
+  conditions: []
+```
+
+### `ExternalBackendBinding`
+
+```yaml
+kind: ExternalBackendBinding
+spec:
+  organizationRef: a5c
+  providerRef: github-a5c
+  targetRef:
+    kind: Repository
+    name: kradle
+  externalRef:
+    owner: a5c-ai
+    repository: kradle
+    installationId: 123456
+  interfaces:
+    issueTracking:
+      enabled: true
+      mode: bidirectional
+    cicd:
+      enabled: true
+      mode: external-owned
+    gitForge:
+      enabled: true
+      mode: bidirectional
+```
+
+### `ExternalBackendSyncPolicy`
+
+```yaml
+kind: ExternalBackendSyncPolicy
+spec:
+  organizationRef: a5c
+  providerRef: github-a5c
+  webhookFirst: true
+  backfill:
+    interval: 15m
+    fullResyncInterval: 24h
+  writePolicy:
+    defaultMode: reviewed-write
+    agentWriteRequiresApproval: true
+  conflictPolicy:
+    defaultResolution: manual
+```
+
+## Aggregated resources
+
+| Kind | Purpose |
+| --- | --- |
+| `ExternalWebhookDelivery` | provider webhook delivery record and processing state. |
+| `ExternalSyncEvent` | normalized provider event. |
+| `ExternalSyncState` | cursor/high-watermark per provider/interface/resource scope. |
+| `ExternalWriteIntent` | Kradle-originated write to provider. |
+| `ExternalSyncConflict` | field/resource conflict requiring resolution. |
+| `ExternalObjectLink` | external native ID/link attached to a Kradle resource. |
+
+## Required labels
+
+- `kradle.a5c.ai/org`;
+- `kradle.a5c.ai/provider`;
+- `kradle.a5c.ai/interface`;
+- `kradle.a5c.ai/repository` when repository-scoped;
+- `kradle.a5c.ai/external-owner` when provider owner/org is known.
+
+## Status conditions
+
+Providers and bindings should use:
+
+- `AuthReady`;
+- `InstallationReady`;
+- `WebhookReady`;
+- `IssueTrackingReady`;
+- `CicdReady`;
+- `GitForgeReady`;
+- `RateLimited`;
+- `BackfillHealthy`;
+- `ConflictsPresent`;
+- `Ready`.
+
+## Storage class
+
+- provider/binding/sync policy: CRD/etcd;
+- deliveries/events/state/write intents/conflicts/object links: aggregated API/Postgres;
+- large payloads/logs/artifacts: object storage by digest;
+- provider credentials: Kubernetes Secret in org namespace.
+
+## Detailed resource schemas
+
+### `ExternalWebhookDelivery.spec`
+
+```yaml
+organizationRef: a5c
+providerRef: github-a5c
+bindingRef: github-kradle
+interfaceHints: [gitForge, issueTracking]
+deliveryId: "github-delivery-guid"
+eventType: pull_request
+action: opened
+receivedAt: 2026-05-11T12:00:00Z
+signature:
+  algorithm: sha256
+  verified: true
+source:
+  owner: a5c-ai
+  repository: kradle
+payloadRef:
+  storage: object
+  digest: sha256:payload
+processing:
+  phase: Queued
+  attempts: 0
+```
+
+### `ExternalSyncEvent.spec`
+
+```yaml
+organizationRef: a5c
+providerRef: github-a5c
+bindingRef: github-kradle
+sourceDelivery: github-delivery-guid
+interface: gitForge
+resourceKind: PullRequest
+nativeId: "42"
+nodeId: PR_kwDO...
+action: opened
+eventTime: 2026-05-11T12:00:00Z
+normalized:
+  repository: kradle
+  pullRequest: 42
+  headSha: abcdef1234
+```
+
+### `ExternalWriteIntent.spec`
+
+```yaml
+organizationRef: a5c
+providerRef: github-a5c
+bindingRef: github-kradle
+interface: issueTracking
+operation: createComment
+source:
+  kind: UserAction
+  actor: user:alice
+target:
+  kind: Issue
+  name: issue-42
+nativeTarget:
+  owner: a5c-ai
+  repository: kradle
+  issueNumber: 42
+requestDigest: sha256:request
+approvalPolicy:
+  required: false
+idempotencyKey: a5c:issue-42:create-comment:01hx
+```
+
+### `ExternalObjectLink.spec`
+
+```yaml
+organizationRef: a5c
+providerRef: github-a5c
+bindingRef: github-kradle
+localRef:
+  apiVersion: kradle.a5c.ai/v1alpha1
+  kind: PullRequest
+  name: pr-42
+external:
+  interface: gitForge
+  nativeId: "42"
+  nativeNumber: 42
+  nodeId: PR_kwDO...
+  url: https://github.com/a5c-ai/kradle/pull/42
+  apiUrl: https://api.github.com/repos/a5c-ai/kradle/pulls/42
+  etag: W/"..."
+```
+
+## Provider type registry
+
+Provider types should be registered in a data-driven registry:
+
+```yaml
+providerType: github
+interfaces: [issueTracking, cicd, gitForge]
+hosting: [saas, ghe]
+authModes: [github-app, oauth-user]
+webhookSignature: hmac-sha256
+supportsGraphql: true
+supportsRest: true
+```
+
+Custom providers can be loaded later through plugin registration, but CRDs should not need a schema change for every provider.
+
+## Validation rules
+
+- `providerType` must exist in registry or use `custom` with explicit adapter ref.
+- enabled interface must be supported by provider descriptor.
+- binding target must be in the same org.
+- auth Secret must be in the org namespace.
+- write mode must be compatible with provider operations.
+- webhook endpoint must have a verification secret unless provider has a signed alternative.
+- `ExternalWriteIntent` cannot reference raw Secret values.