workspai 0.46.0 → 0.48.0

package/docs/AI_QUICKSTART.md

@@ -1,248 +1,85 @@
-# 🚀 AI Features - Quick Start Guide
+# Optional AI Module Recommendations
 
-Get started with Workspai AI recommendations in 60 seconds!
+Workspai has two different AI-facing capabilities:
 
-## Option 1: Use the bundled catalog (Recommended) ⚡
+1. **Workspace Intelligence** builds deterministic model, graph, impact,
+   verification, context, and agent artifacts. It does not require an API key.
+2. **AI module recommendations** use embeddings to suggest optional RapidKit
+   modules from a natural-language description. This page covers only that
+   optional recommender.
 
-Just run it - AI will guide you through everything:
+## Try it without an API key
 
 ```bash
-npx workspai ai recommend "user authentication"
+npx workspai ai recommend "authentication with email"
 ```
 
-Published Workspai packages normally include `data/modules-embeddings.json`, so
-recommendations can use the bundled catalog after an API key is configured.
-If that file is missing, the CLI offers this recovery flow:
+When no OpenAI key is configured, the command uses deterministic mock mode. Mock
+mode is useful for testing the command flow; its ranking is not suitable for a
+production module decision.
 
-```
-⚠️  Module embeddings not found
-AI recommendations require embeddings to be generated.
-
-? What would you like to do?
-  🚀 Generate embeddings now (requires OpenAI API key)
-  📝 Show me how to generate them manually
-  ❌ Cancel
-```
-
-Select option 1 to rebuild the missing data, or use the manual refresh flow below.
-
-## Option 2: Manual Setup (For Advanced Users)
-
-### Step 1: Get API Key (2 minutes)
-
-1. Visit [OpenAI Platform](https://platform.openai.com/api-keys)
-2. Click "Create new secret key"
-3. Copy your key
-
-### Step 2: Configure Key (30 seconds)
-
-```bash
-npx workspai config set-api-key
-# Paste your API key when prompted
-```
-
-Or set as environment variable:
-```bash
-export OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"
-```
-
-### Step 3: Generate Embeddings (typically about a minute, provider-cost dependent)
-
-```bash
-npx workspai ai generate-embeddings
-```
-
-You'll see:
-```
-🤖 Generating AI embeddings for Workspai modules...
-📡 Fetching modules from Workspai...
-✓ Found <N> modules
-
-💰 Estimated cost: shown by CLI at runtime
-   (depends on current provider pricing and module count)
-
-? Generate embeddings now? Yes
-```
-
-Embeddings are reusable until the module catalog, embedding model, or desired
-coverage changes.
-
-### Step 4: Use It! (Instant)
-
-```bash
-npx workspai ai recommend "authentication with social login"
-```
-
-Output:
-```
-📦 Recommended Modules:
-
-1. Authentication Core
-   Complete authentication with JWT, OAuth 2.0, secure sessions
-   Match: 95.2% - Matches: auth, login, oauth
-   Category: auth
-
-2. Users Core
-   User management with profiles, roles, permissions
-   Match: 88.7% - Matches: user, social
-   Category: auth
-   Requires: authentication-core
-
-💡 Quick install (top 3):
-   workspai add module authentication-core users-core
-
-? Would you like to install these modules now? Yes
-✅ Selected modules installed successfully
-```
-
-## Option 3: Test Without API Key (Mock Mode)
-
-No API key? No problem! Try it in mock mode:
-
-```bash
-npx workspai ai recommend "database with caching"
-```
-
-You'll see:
-```
-⚠️  OpenAI API key not configured - using MOCK MODE for testing
-
-📝 Note: Mock embeddings provide approximate results for testing.
-   For production, configure your OpenAI API key:
-   ...
-```
-
-Mock mode gives realistic (but not perfect) results for free!
+## Use provider-backed recommendations
 
-## 🎯 Common Use Cases
+Prefer an environment variable in CI or short-lived shells:
 
-### Find Authentication Modules
 ```bash
-npx workspai ai recommend "user authentication with email and password"
-npx workspai ai recommend "social login with Google and Facebook"
-npx workspai ai recommend "two-factor authentication"
+export OPENAI_API_KEY="<your-key>"
+npx workspai ai recommend "authentication with email" --number 5
 ```
 
-### Find Database Modules
-```bash
-npx workspai ai recommend "PostgreSQL database with migrations"
-npx workspai ai recommend "MongoDB with async operations"
-npx workspai ai recommend "database caching with Redis"
-```
-
-### Find Payment Modules
-```bash
-npx workspai ai recommend "payment processing with Stripe"
-npx workspai ai recommend "subscription billing"
-npx workspai ai recommend "invoice generation"
-```
+For local interactive use, Workspai can store the key in the user configuration:
 
-### Find Communication Modules
 ```bash
-npx workspai ai recommend "email notifications with templates"
-npx workspai ai recommend "SMS verification codes"
-npx workspai ai recommend "real-time notifications"
-```
-
-### Find Infrastructure Modules
-```bash
-npx workspai ai recommend "background job processing"
-npx workspai ai recommend "file storage with S3"
-npx workspai ai recommend "rate limiting for APIs"
-```
-
-## 💡 Pro Tips
-
-### 1. Be Specific
-❌ Bad: "authentication"  
-✅ Good: "user authentication with JWT and OAuth 2.0"
-
-### 2. Mention Technologies
-❌ Bad: "database"  
-✅ Good: "PostgreSQL database with async support"
-
-### 3. Describe Your Use Case
-❌ Bad: "payments"  
-✅ Good: "subscription payments with recurring billing"
-
-### 4. Use Natural Language
-❌ Don't: "auth jwt oauth session redis"  
-✅ Do: "I need authentication with JWT tokens and Redis sessions"
-
-### 5. Get More/Less Results
-```bash
-# Get top 3 only
-npx workspai ai recommend "auth" --number 3
-
-# Get top 10
-npx workspai ai recommend "auth" --number 10
+npx workspai config set-api-key
+npx workspai config show
 ```
 
-### 6. JSON Output for Scripts
-```bash
-npx workspai ai recommend "database" --json | jq '.recommendations[0].module.id'
-```
+Do not pass secrets through `--key` in shared terminals or CI logs. Use an
+environment secret instead.
 
-## 🔧 Quick Commands
+Published packages normally include the module embedding catalog. If the
+catalog is missing or intentionally being refreshed:
 
 ```bash
-# Get recommendations
-npx workspai ai recommend "query here"
-
-# Generate embeddings (one-time)
 npx workspai ai generate-embeddings
-
-# Update embeddings (after Workspai update)
 npx workspai ai update-embeddings
+```
 
-# View info and pricing
-npx workspai ai info
+Both commands require a provider key and may incur provider charges.
 
-# Configure API key
-npx workspai config set-api-key
+## Script-friendly output
 
-# Check current config
-npx workspai config show
+```bash
+npx workspai ai recommend "database caching" --number 3 --json
 ```
 
-## 💰 Pricing Summary
-
-| Item | Cost | When |
-|------|------|------|
-| Setup (embeddings) | Varies | One-time only |
-| Per query | Varies | Every query |
-| Ongoing usage | Varies | After setup |
+Recommendation scores are embedding similarity values. They are not confidence
+probabilities, security approvals, compatibility guarantees, or measured task
+accuracy. Verify runtime compatibility and module dependencies before install.
 
-**Note:** pricing changes over time. Check your provider dashboard for current rates.
+## Install a selected module
 
-## ❓ Troubleshooting
+Module installation is available only in projects whose validated Core bridge
+reports the `add` capability:
 
-### "Module embeddings not found"
-👉 Just follow the interactive prompts - they'll guide you!
-
-### "Invalid API key"
 ```bash
-npx workspai config set-api-key
-# Enter your correct API key
+npx workspai add module <module-id>
 ```
 
-### "Quota exceeded"
-👉 Check your billing: https://platform.openai.com/account/billing
+If the project does not expose that capability, the recommender can still return
+suggestions but Workspai will refuse the installation step with a reason.
 
-### Want to test without spending money?
-👉 Use mock mode - it works without an API key!
+## Troubleshooting
 
-## 🎓 Learn More
-
-- **Full Guide:** [AI_FEATURES.md](AI_FEATURES.md)
-- **Technical Details:** [AI_DYNAMIC_INTEGRATION.md](AI_DYNAMIC_INTEGRATION.md)
-- **Main README:** [../README.md](../README.md)
-
-## 🚀 Ready to Build?
-
-Start exploring modules with AI:
-```bash
-npx workspai ai recommend "what I want to build"
-```
+| Symptom                         | What to do                                                                 |
+| ------------------------------- | -------------------------------------------------------------------------- |
+| AI features are disabled        | `npx workspai config ai enable`                                            |
+| Provider key is missing         | Use mock mode or set `OPENAI_API_KEY`                                      |
+| Embedding catalog is missing    | Run `npx workspai ai generate-embeddings`                                  |
+| Provider returns 401            | Replace the key; never paste it into an issue                              |
+| Provider returns 429            | Check the provider quota/rate limit and retry later                        |
+| Recommendation looks irrelevant | Use a clearer requirement and treat the score as similarity, not certainty |
+| Module install is refused       | Run the command inside a module-enabled project and inspect its capability |
 
-That's it! Happy building! 🎉
+For full behavior and architecture, see [AI_FEATURES.md](./AI_FEATURES.md) and
+[AI_DYNAMIC_INTEGRATION.md](./AI_DYNAMIC_INTEGRATION.md).

package/docs/GLOSSARY.md

@@ -0,0 +1,64 @@
+# Workspai Glossary
+
+Short definitions for terms used by the CLI, its reports, IDE integrations,
+CI workflows, and AI consumers.
+
+## Core concepts
+
+| Term               | Plain-language meaning                                                                                                                                      |
+| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Workspace          | A governed collection of one or more registered projects. It is not required to be a Git monorepo.                                                          |
+| Project            | One registered application, service, library, or infrastructure unit inside or linked to a workspace.                                                       |
+| Workspace manifest | `.workspai/workspace.json`; workspace identity, profile, engine, and bootstrap metadata. It is not the project registry.                                    |
+| Workspace contract | `.workspai/workspace.contract.json`; the operational registry of projects and their declared ports, APIs, ownership, and relationships.                     |
+| Registry summary   | `.workspai/workspace-registry.v1.json`; the canonical lightweight project count and registry status for UI and CI.                                          |
+| Workspace Model    | The canonical, deterministic structural description produced from registered projects, manifests, contracts, policies, and detected facts.                  |
+| Knowledge Graph    | A proof-backed representation derived from the Workspace Model. It makes entities and relationships queryable; it is not the source of truth for the model. |
+| Dependency graph   | The project-to-project dependency subgraph embedded in the Workspace Model and used by impact and verification.                                             |
+| Provider           | A deterministic source adapter that emits facts from code, manifests, APIs, infrastructure, documentation, Git, or another supported source.                |
+| Fact               | A normalized observation about the workspace, such as a package, endpoint, import, deployment, or owner.                                                    |
+| Evidence / proof   | The source location and extraction details that justify a fact or relationship.                                                                             |
+| Proof path         | A traceable route from a query result or relationship back to its supporting evidence.                                                                      |
+| Artifact           | A versioned file written under `.workspai/`, usually `.workspai/reports/`, for people or automation to consume.                                             |
+| Command projection | The JSON returned on stdout for one command. It can include status and output metadata around a canonical artifact payload.                                 |
+
+## Intelligence and governance
+
+| Term             | Plain-language meaning                                                                                                           |
+| ---------------- | -------------------------------------------------------------------------------------------------------------------------------- |
+| Unified runner   | `workspace intelligence run`; the supported way to execute the complete intelligence chain in contract order.                    |
+| Preflight        | `sync` and baseline resolution. These prepare the run but are not stages in the 11-stage intelligence chain.                     |
+| Stage            | One contract-defined operation in the intelligence chain. Stage order comes from `workspace-intelligence-chain.v1.json`.         |
+| Gate             | A decision that can pass, need attention, or block automation according to evidence and policy.                                  |
+| Blocked          | The command completed, but evidence or policy prevents the requested release/action. The unified runner uses exit code `2`.      |
+| Failed           | Execution itself failed. The unified runner uses exit code `1`.                                                                  |
+| Needs attention  | Evidence is usable but contains warnings or non-blocking gaps. Strict policy can promote it to a blocking result.                |
+| Fresh / stale    | Whether an artifact still matches its governed inputs and dependency closure. A recent timestamp alone does not prove freshness. |
+| Snapshot         | A stable baseline of the model used for later comparison.                                                                        |
+| Diff             | The structural change between the current model and a baseline model or snapshot.                                                |
+| Impact           | Direct and transitive consequences of a model diff, including verification scope and risk.                                       |
+| Verify           | The evidence-backed gate over the affected dependency subgraph and workspace policies.                                           |
+| Context          | A bounded, agent-oriented projection of current workspace evidence.                                                              |
+| Agent sync       | Generation of tool-specific instructions, skills, prompts, indexes, and MCP metadata from canonical evidence.                    |
+| Evaluation       | A provenance-aware record of model calls, tool activity, cost, latency, and verified task outcome produced by `workspace eval`.  |
+| Verified outcome | A task result supported by the workspace verification path; smaller context alone is not treated as task success.                |
+
+## AI and integration
+
+| Term                | Plain-language meaning                                                                                                        |
+| ------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
+| Agent grounding     | Instructions and evidence references that keep an AI agent inside the correct workspace, contracts, and command loop.         |
+| Bounded retrieval   | Returning only the most relevant entities and proof paths for a question instead of injecting the complete graph or model.    |
+| Retrieval benchmark | A deterministic comparison of readable proof-source corpus size with one bounded graph response; it is not billing evidence.  |
+| Token provenance    | Whether a token value was provider-reported, counted by a named tokenizer, estimated, or unavailable.                         |
+| MCP                 | Model Context Protocol; Workspai exposes read-oriented workspace tools through `workspace mcp serve`.                         |
+| Module recommender  | The optional embedding-based FastAPI/NestJS recommendation feature. It is separate from deterministic Workspace Intelligence. |
+| Canonical path      | The current `.workspai` path that new writers and consumers should prefer.                                                    |
+| Legacy path         | A `.rapidkit` compatibility path read for older workspaces; it is not the target for new integrations.                        |
+
+## Where to continue
+
+- Run the full loop: [Unified Workspace Intelligence Runner](./workspace-intelligence-runner.md)
+- Query evidence: [Workspace Knowledge Graph](./workspace-knowledge-graph.md)
+- Find an output: [Artifact Catalog](./contracts/ARTIFACT_CATALOG.md)
+- Look up syntax: [Command Reference](./commands-reference.md)

package/docs/OPEN_SOURCE_USER_SCENARIOS.md

@@ -37,14 +37,20 @@ cd my-workspace
 npx workspai create project nextjs marketing-web --yes
 ```
 
+Success check: `workspace registry status --json` lists each project once and
+the unified runner writes
+`.workspai/reports/workspace-intelligence-run-last-run.json`.
+
 ## What changed from the old flow?
 
 Old flow (typical):
+
 - Create workspace/project
 - Run `init` / `dev`
 - Minimal governance and supply-chain controls
 
 Current flow (new baseline):
+
 - Same developer-friendly start
 - Plus optional mirror/offline controls, checksum/attestation verification, Sigstore governance, and auditable reports
 - Works for both small teams and enterprise adoption paths
@@ -76,11 +82,15 @@ npx workspai workspace intelligence run --for-agent codex --strict --json
 ```
 
 The canonical durable outputs are `.workspai/reports/workspace-model.json`,
+`.workspai/reports/workspace-knowledge-graph.json`,
 `.workspai/reports/workspace-context-agent.json`, `.workspai/reports/INDEX.json`,
 `.workspai/reports/workspace-intelligence-run-last-run.json`, and `AGENTS.md`.
 Use `npx workspai pipeline --json --strict` separately as the broader governance
 and release gate.
 
+Success check: the application starts locally, project Doctor reports the
+expected runtime, and the unified run report contains all 11 ordered stages.
+
 ### When manual vs automatic?
 
 - Manual: run commands directly in local dev.
@@ -92,8 +102,8 @@ Goal: improve stability and repeatability using mirror artifacts.
 
 ### Steps
 
-1) Define minimal mirror config (`.workspai/mirror-config.json`) with artifact sources and checksums.
-2) Run:
+1. Define minimal mirror config (`.workspai/mirror-config.json`) with artifact sources and checksums.
+2. Run:
 
 ```bash
 cd my-workspace
@@ -107,6 +117,9 @@ npx workspai init
 npx workspai dev
 ```
 
+Success check: `mirror verify` succeeds and its latest report exists under
+`.workspai/reports/` before build or test begins.
+
 ### When manual vs automatic?
 
 - Manual: initial mirror setup and local validation.
@@ -118,13 +131,14 @@ Goal: enforce stronger security controls (attestation + Sigstore governance) in
 
 ### Steps
 
-1) Configure `mirror-config.json` with:
+1. Configure `mirror-config.json` with:
+
 - `security.requireAttestation: true`
 - `security.requireSigstore: true`
 - `security.requireTransparencyLog: true`
 - environment policy allowlists (`identity`, `issuer`, `rekorUrl`)
 
-2) Run:
+2. Run:
 
 ```bash
 RAPIDKIT_ENV=stage npx workspai mirror sync --json
@@ -132,6 +146,9 @@ RAPIDKIT_ENV=stage npx workspai mirror verify --json
 npx workspai bootstrap --profile=enterprise --ci --offline --json
 ```
 
+Success check: the bootstrap compliance report records the enterprise profile,
+offline mode, checksum/attestation decisions, and a machine-readable exit.
+
 ### When manual vs automatic?
 
 - Manual: initial policy authoring and first dry run.
@@ -143,17 +160,19 @@ Goal: enforce signed governance policy bundle, generate and export audit evidenc
 
 ### Steps
 
-1) Add signed governance bundle:
+1. Add signed governance bundle:
+
 - `.workspai/governance-policy.json`
 - `.workspai/governance-policy.sig`
 - `.workspai/governance-public.pem`
 
-2) Configure in `mirror-config.json`:
+2. Configure in `mirror-config.json`:
+
 - `security.requireSignedGovernance: true`
 - `security.governanceBundle: { ... }`
 - `security.evidenceExport: { target: "file" | "http", ... }`
 
-3) Run:
+3. Run:
 
 ```bash
 RAPIDKIT_ENV=prod npx workspai mirror sync --json
@@ -161,19 +180,58 @@ RAPIDKIT_ENV=prod npx workspai mirror verify --json
 RAPIDKIT_ENV=prod npx workspai bootstrap --profile=enterprise --ci --offline --json
 ```
 
+Success check: signed-policy verification passes and the configured evidence
+sink receives the same governed run identity as the local report.
+
 ### When manual vs automatic?
 
 - Manual: key management, policy signing, endpoint provisioning.
 - Automatic: all command execution in CI/CD and release pipelines.
 
+## Scenario 5 — AI agent or IDE consumer
+
+Goal: answer a workspace question with bounded, traceable evidence instead of
+loading every source file or the complete graph into a model prompt.
+
+### Steps
+
+```bash
+cd my-workspace
+npx workspai workspace intelligence run --for-agent codex --strict --json
+npx workspai workspace graph search "authentication endpoint" --limit 12 --json
+npx workspai workspace graph benchmark "authentication endpoint" --limit 12 --json
+npx workspai workspace mcp serve
+```
+
+The runner creates and validates the model, knowledge graph, context, and agent
+surfaces. CLI consumers should start with `AGENTS.md` and
+`.workspai/reports/INDEX.json`, then call `workspace graph search` for a bounded
+result. MCP consumers should call `searchWorkspaceGraph`, follow returned proof
+references, and request the full graph only when the bounded evidence is not
+enough.
+
+The benchmark compares readable proof-source payload with the bounded retrieval
+payload. Its token value is an estimate, not a universal model-cost or
+answer-quality claim; see [Graph Benchmark Methodology](./graph-benchmark-methodology.md).
+
+### You are done when
+
+- the unified runner returns exit `0` for a release-ready workspace, or exit `2`
+  with explicit remediation evidence rather than an execution failure;
+- every search result identifies evidence or a proof path;
+- the consumer can answer the question without injecting the complete model or
+  graph by default.
+
 ## Operational outputs (for automation and auditing)
 
 Generated reports:
+
 - `.workspai/reports/bootstrap-compliance.latest.json`
 - `.workspai/reports/mirror-ops.latest.json`
 - `.workspai/reports/transparency-evidence.latest.json`
 
 Optional exported evidence sinks:
+
 - file sink (NDJSON/JSON append strategy)
 - HTTP webhook sink (SIEM/GRC intake)
 
@@ -182,6 +240,7 @@ Optional exported evidence sinks:
 - Individuals/small teams: start with Scenario 0 → 1 → 2.
 - Product teams/platform teams: adopt Scenario 3.
 - Regulated/high-compliance environments: run Scenario 4 by default.
+- AI/IDE integrations: add Scenario 5 after the workspace is registered.
 
 ## See also
 
@@ -189,3 +248,5 @@ Optional exported evidence sinks:
 - [workspace-operations.md](./workspace-operations.md)
 - [doctor-command.md](./doctor-command.md)
 - [ci-workflows.md](./ci-workflows.md) (`pipeline --json --strict`)
+- [workspace-knowledge-graph.md](./workspace-knowledge-graph.md)
+- [GLOSSARY.md](./GLOSSARY.md)