skill-flow 1.0.0 → 1.0.1

package/docs/plan/PLAN_v1.0.1.md

@@ -0,0 +1,814 @@
+# Skill Flow Implementation Plan
+
+Version: v1.0.1
+Status: ENGINEERING REVIEWED
+Date: 2026-03-22
+Product mode: Source Expansion + Discovery Entry
+Platform: macOS
+CLI entry: `skill-flow`
+State root: `~/.skillflow/`
+
+## 1. Executive Summary
+
+`v1.0.1` adds two product capabilities:
+- expanded `add` beyond plain Git clone
+- a first `find` / `search` discovery flow
+
+This release does not try to become a full ecosystem registry.
+It stays narrow:
+- Git sources still matter
+- ClawHub becomes the only new remote structured source
+- `find` only merges local inventory and ClawHub search
+
+The engineering goal is not "support every source."
+The engineering goal is:
+
+```text
+take the existing Git-only workflow manager
+  ->
+make source ingestion multi-kind without breaking state
+  ->
+add a small discovery entrypoint that always leads back to `add`
+```
+
+This plan explicitly avoids:
+- broad Git host search
+- persistent discovery cache
+- a plugin SDK
+- importing `.clawhub` local state
+- a new `install <candidate>` command
+
+## 2. Why This Release Exists
+
+`v1.0.0` shipped a narrow but real baseline:
+- Git source add / update / uninstall
+- grouped workflow management
+- config / apply / doctor
+- explicit `manifest.json + lock.json`
+
+That baseline is useful, but two gaps are now visible:
+- users still need to know a source before they can do anything
+- `add` still behaves like a Git clone wrapper instead of a source-ingestion command
+
+`v1.0.1` closes those gaps without changing the product's core identity.
+
+## 3. Research Summary
+
+Research date: 2026-03-22
+
+Primary references:
+- Vercel skills repo: <https://github.com/vercel-labs/skills>
+- Vercel `find-skills` skill: <https://github.com/vercel-labs/skills/tree/main/skills/find-skills>
+- OpenAI `skill-installer` skill: <https://github.com/openai/skills/blob/main/skills/.system/skill-installer/SKILL.md>
+- OpenClaw ClawHub docs: <https://docs.openclaw.ai/tools/clawhub>
+
+### 3.1 Vercel `skills`
+
+Observed behavior:
+- installation is still repo-centric
+- accepted inputs include `owner/repo`, full URL, repo tree URL, and local path
+- discovery is a workflow that helps users choose a source, then route into installation
+
+Takeaways:
+- `find` and `add` should be one journey
+- repo subpath install is a real need
+- discovery can start with explicit rules and curated sources
+
+### 3.2 OpenAI `skill-installer`
+
+Observed behavior:
+- supports repo, URL, local path, and repo subpath installation
+- prefers direct download for public content
+- falls back to sparse checkout when needed
+
+Takeaways:
+- path-scoped install is worth doing
+- source identity should still remain repo-first
+- direct path input does not require inventing a separate package model
+
+### 3.3 ClawHub
+
+Observed behavior:
+- `clawhub search` is a structured discovery surface
+- `clawhub install` installs named skills with optional version selection
+- installation lands in the current workspace `./skills`
+- state is tracked under `.clawhub/`
+
+Takeaways:
+- ClawHub is a real source kind, not just another Git locator
+- its identity model is `slug + version`, not `remote + commit`
+- its install UX is useful reference
+- its on-disk state model should not be reused inside `skill-flow`
+
+Engineering conclusion:
+- do not shell out to `clawhub install` and absorb `.clawhub/lock.json`
+- do implement ClawHub fetch/search against `skill-flow`'s own state and cache model
+
+## 4. What Already Exists
+
+Existing code and flows this plan must reuse:
+
+- [src/services/source-service.ts](/Users/Vint/仓库/03%20Project/skill-manager/src/services/source-service.ts)
+  Reuse:
+  - source add/update/remove orchestration
+  - leaf scan + manifest/lock write flow
+  - source-level update semantics
+
+- [src/services/skill-flow.ts](/Users/Vint/仓库/03%20Project/skill-manager/src/services/skill-flow.ts)
+  Reuse:
+  - application service entrypoints
+  - existing orchestration pattern for CLI commands
+
+- [src/state/store.ts](/Users/Vint/仓库/03%20Project/skill-manager/src/state/store.ts)
+  Reuse:
+  - manifest / lock path ownership
+  - state root initialization
+  Change:
+  - source cache path resolution must become source-kind aware
+
+- [src/tests/skill-flow.test.ts](/Users/Vint/仓库/03%20Project/skill-manager/src/tests/skill-flow.test.ts)
+  Reuse:
+  - temp repo + temp state test harness
+  - integration-heavy testing style
+  - existing backward-compat lock read pattern
+
+- [src/cli.tsx](/Users/Vint/仓库/03%20Project/skill-manager/src/cli.tsx)
+  Reuse:
+  - existing command registration shell
+  Change:
+  - add `find`
+  - add `search` alias
+  - widen `add` argument semantics
+
+Existing things this plan must not rebuild:
+- a second state layer for discovery
+- a second service layer parallel to `SkillFlowApp`
+- a generalized source plugin system
+
+## 5. NOT In Scope
+
+- Broad GitHub / GitLab keyword search
+  Reason: low-signal results and unclear trust model
+
+- Persistent discovery cache files
+  Reason: no demonstrated performance need yet
+
+- `install <candidate>` top-level command
+  Reason: `find -> add` is sufficient for this release
+
+- Importing `.clawhub` local state
+  Reason: it would couple two source-of-truth systems
+
+- Stars / install counts / leaderboard aggregation from arbitrary public hosts
+  Reason: data quality and trust are not stable enough
+
+- Registry / source plugin SDK
+  Reason: two concrete source kinds do not justify a framework
+
+## 6. Step 0 Scope Challenge
+
+This plan is intentionally split into two bounded delivery tracks:
+
+```text
+Track A: Source expansion
+  - naming alignment
+  - multi-kind storage
+  - ClawHub add/update/uninstall
+  - Git repo-subpath add
+
+Track B: Discovery
+  - find/search
+  - local + ClawHub result merge
+  - ranking
+  - candidate -> add command bridge
+```
+
+This is not a scope reduction in product value.
+It is a scope reduction in simultaneous moving parts.
+
+Without this split, one release would change:
+- naming and state paths
+- source fetch/update semantics
+- CLI surface
+- result ranking
+- follow-up action generation
+
+That is too much blast radius for one undifferentiated implementation batch.
+
+## 7. Architecture Decisions
+
+### 7.1 Phase 0: Naming And State-Root Alignment
+
+Before new behavior is added, `v1.0.1` must align plan and code on one product identity.
+
+Lock for this release:
+- CLI name: `skill-flow`
+- state root: `~/.skillflow/`
+- env var prefix: `SKILL_FLOW_`
+
+Do not mix:
+- `skill-manager`
+- `.skillmanager`
+- future rename ideas
+
+If product rename work is desired later, it must be a separate plan.
+
+### 7.2 Keep The Existing Source Of Truth
+
+Keep:
+- `manifest.json`
+- `lock.json`
+
+Do not add:
+- `inventory-index.json`
+- `discovery-cache.json`
+- `channels.json`
+
+Discovery results are ephemeral.
+Only actual installed sources modify persistent state.
+
+### 7.3 Multi-Kind Source Storage Is Explicit
+
+Current code assumes one cache root: Git.
+That is no longer enough.
+
+New required cache layout:
+
+```text
+~/.skillflow/
+├── source/
+│   ├── git/
+│   │   └── <source-id>/
+│   └── clawhub/
+│       └── <source-id>/
+├── manifest.json
+└── lock.json
+```
+
+`StateStore` owns this path resolution.
+Adapters must not construct cache paths themselves.
+
+Required API direction:
+- `getSourceRoot(kind)`
+- `getSourceCheckoutPath(kind, sourceId)`
+
+### 7.4 Source Identity Stays Source-First
+
+Direct path installs do not create a new source identity.
+
+Rules:
+- source identity is based on normalized source locator
+- `--path` is not part of source identity
+- the same repo cannot be registered twice just because the requested path differs
+- `--path` is an add-time selection filter and optional persisted preference only
+
+This preserves update and uninstall semantics at source granularity.
+
+### 7.5 Minimal Source Adapter Seam
+
+Do not build a plugin system.
+Do not let adapters own state writes or scanning.
+
+Only extract source-specific IO:
+- `resolve(locator)`
+- `fetch(resolved, destination)`
+- `update(existingSnapshot, checkoutPath)`
+
+Keep in `SourceService`:
+- manifest/lock writes
+- inventory scan
+- source diffing
+- duplicate detection
+- source lifecycle orchestration
+
+This is the smallest seam that cleanly supports:
+- Git
+- ClawHub
+
+### 7.6 `find` Is A Read-Time Aggregation Service
+
+`find` aggregates:
+- local installed inventory
+- ClawHub search metadata
+
+It does not:
+- download bundles during search
+- mutate manifest/lock
+- create background cache state
+
+Search fetch policy:
+- `find` requests metadata only
+- `add` downloads source content
+
+### 7.7 Candidate-To-Command Bridge Is A First-Class Boundary
+
+`find` output must not hand-build `add` command strings in CLI formatting code.
+
+Instead:
+- candidate assembly produces normalized `SkillCandidate`
+- a single command-builder maps candidate -> follow-up `add` command
+- text output and JSON output both consume the same mapping source
+
+This avoids formatter drift between:
+- Git candidates
+- Git + path candidates
+- ClawHub floating candidates
+- ClawHub pinned candidates
+
+## 8. Data Flow Diagrams
+
+### 8.1 Expanded `add`
+
+```text
+user input
+  |
+  v
+parse locator + optional --path
+  |
+  v
+resolve source kind
+  |
+  +--> git resolver
+  |
+  +--> clawhub resolver
+  |
+  v
+fetch source checkout into source/<kind>/<source-id>
+  |
+  v
+scan checkout for valid leafs
+  |
+  v
+apply optional path filter / preference
+  |
+  v
+write manifest + lock
+  |
+  v
+optional follow-up apply/config
+```
+
+### 8.2 `find`
+
+```text
+query
+  |
+  +--> local inventory matcher
+  |
+  +--> clawhub search metadata
+  |
+  v
+merge candidates
+  |
+  v
+rank with explicit rule table
+  |
+  v
+candidate -> add command builder
+  |
+  +--> text output
+  |
+  +--> json output
+```
+
+## 9. Command Design
+
+### 9.1 `skill-flow add`
+
+```bash
+skill-flow add <locator>
+  [--path <repoSubpath>]
+  [--channel <channel...>]
+  [--yes]
+  [--json]
+```
+
+Supported locators:
+- GitHub shorthand: `owner/repo`
+- full Git URL
+- SSH Git URL
+- local path
+- GitHub tree URL
+- `clawhub:<slug>`
+- `clawhub:<slug>@<version>`
+
+Behavior:
+- detect source kind before fetch
+- normalize source identity
+- for GitHub tree URL:
+  - resolve repo locator
+  - extract subpath as `--path` intent
+- for `--path`:
+  - narrow the initial result to matching leafs
+  - do not create a distinct source identity
+- for `--channel`:
+  - reuse existing apply flow after add
+
+### 9.2 `skill-flow find`
+
+```bash
+skill-flow find <query>
+  [--source local|clawhub|all]
+  [--limit <n>]
+  [--json]
+```
+
+Behavior:
+- default source is `all`
+- search local inventory and ClawHub metadata
+- return structured candidates
+- annotate installed matches
+- show exact follow-up `add` command for each result
+
+### 9.3 `skill-flow search`
+
+```bash
+skill-flow search <query>
+```
+
+Behavior:
+- exact alias of `find`
+- same output shape
+- same options
+
+## 10. Data Model Changes
+
+### 10.1 Source kinds
+
+```ts
+type SourceKind = "git" | "clawhub";
+```
+
+### 10.2 Source manifest
+
+Keep current manifest structure.
+Extend only enough to allow:
+- `kind = "git" | "clawhub"`
+- normalized locators for ClawHub
+
+### 10.3 Source lock
+
+Keep one `sources[]` array.
+Do not split by kind.
+
+Required fields:
+- shared:
+  - `id`
+  - `locator`
+  - `kind`
+  - `displayName`
+  - `checkoutPath`
+  - `updatedAt`
+  - `leafIds`
+  - `invalidLeafs`
+
+- Git-specific:
+  - `commitSha`
+
+- ClawHub-specific:
+  - `packageSlug`
+  - `resolvedVersion`
+  - `contentHash`
+  - `versionMode`
+
+`versionMode` is:
+- `pinned`
+- `floating`
+
+### 10.4 `SkillCandidate`
+
+`SkillCandidate` is an internal result object for `find`.
+
+Group fields by responsibility.
+
+Identity:
+- `candidateId`
+- `sourceKind`
+- `locator`
+- `skillPath?`
+- `version?`
+
+Display:
+- `title`
+- `summary`
+- `official`
+- `alreadyInstalled`
+- `installedSourceId?`
+
+Action:
+- `installCommand`
+
+Internal-only:
+- `score`
+
+Rules:
+- `score` is not a stable public contract
+- do not add leaderboard or popularity fields in this release
+
+## 11. Ranking Rules For `find`
+
+Do not implement free-form numeric tuning first.
+Implement a rule table.
+
+Ordering rules:
+
+```text
+Step 1: group by trust tier
+  Tier 0 = local installed inventory
+  Tier 1 = ClawHub registry results
+
+Step 2: within a tier, exact token overlap beats partial overlap
+
+Step 3: installed exact match beats installed partial match
+
+Step 4: remote result with exact title/path match beats remote partial text match
+
+Step 5: stable tie-break by:
+  sourceKind
+  title
+  locator
+```
+
+Notes:
+- no broad Git host search in `v1.0.1`
+- `score` may exist internally, but implementation must preserve the rule table above
+
+## 12. Implementation Plan
+
+### Phase 0: Naming And Path Alignment
+
+Goal:
+- remove naming drift before new behavior lands
+
+Tasks:
+- align plan, README, and implementation on `skill-flow`
+- align state root references on `~/.skillflow/`
+- align env var references on `SKILL_FLOW_*`
+- document that any future rename is out of scope
+
+Exit criteria:
+- no mixed `skill-manager` / `skill-flow` references remain in the active implementation plan
+
+### Phase 1: Source Storage And Resolver Refactor
+
+Goal:
+- make source ingestion multi-kind without broad refactor
+
+Tasks:
+- make `StateStore` source-kind aware
+- add a source-kind resolver
+- keep orchestration in `SourceService`
+- extract only source-specific IO seam
+- preserve existing Git behavior
+
+Exit criteria:
+- current Git add/update/uninstall tests still pass
+- source cache path is resolved by store, not by callers
+
+### Phase 2: ClawHub Source Support
+
+Goal:
+- support ClawHub as a real source kind in local state
+
+Tasks:
+- parse `clawhub:<slug>`
+- parse `clawhub:<slug>@<version>`
+- fetch source bundle into `source/clawhub/<source-id>`
+- persist ClawHub lock metadata
+- support update semantics:
+  - floating source resolves latest only on `update`
+  - pinned source remains pinned
+- support uninstall through the existing source lifecycle path
+
+Exit criteria:
+- add works for floating and pinned ClawHub locators
+- update differentiates pinned and floating behavior
+- uninstall removes ClawHub cache and deployments cleanly
+
+### Phase 3: Git Path-Scoped Add
+
+Goal:
+- support Git installs that begin from a repo subpath
+
+Tasks:
+- support `--path`
+- parse GitHub tree URLs into repo + path intent
+- validate whether requested path maps to:
+  - a valid leaf
+  - or a valid source subtree
+- reject existing-but-invalid paths clearly
+- preserve source identity without duplicating the repo
+
+Exit criteria:
+- repo path installs work without duplicating source identity
+- same repo cannot be registered twice by changing only `--path`
+
+### Phase 4: Discovery
+
+Goal:
+- provide a small, reliable discovery entrypoint
+
+Tasks:
+- add `find`
+- add `search` alias
+- build local inventory matcher
+- build ClawHub metadata search integration
+- merge candidates
+- rank by the explicit rule table
+- annotate installed state
+- use the shared candidate-to-command builder for all outputs
+
+Exit criteria:
+- local-only results work
+- remote-only results work
+- mixed local+remote results are stable
+- every actionable result produces a correct `add` command
+
+### Phase 5: Docs And Verification
+
+Goal:
+- make the release verifiable and understandable
+
+Tasks:
+- update README command examples
+- update help text
+- add diagrams to the plan
+- add any inline ASCII diagrams needed in touched implementation files
+- expand tests for new branches and failure modes
+
+Exit criteria:
+- command surface and docs match implementation
+- new codepaths are covered by tests, not just happy paths
+
+## 13. Test Plan
+
+### 13.1 New flow diagram
+
+```text
+A. Expanded add
+   git shorthand/url/local path/tree url
+   clawhub slug / slug@version
+   optional --path filter
+
+B. Discovery
+   local inventory
+   remote clawhub metadata
+   merge
+   rank
+   command build
+
+C. Mixed state
+   git-only old state
+   git + clawhub new state
+   update / uninstall / reconcile
+```
+
+### 13.2 Required unit tests
+
+- locator parsing
+  - Git shorthand
+  - Git URL
+  - local path
+  - GitHub tree URL
+  - ClawHub floating
+  - ClawHub pinned
+
+- ranking rule table
+  - local exact > local partial
+  - local > remote
+  - remote exact > remote partial
+  - stable tie-break order
+
+- candidate-to-command builder
+  - Git candidate
+  - Git candidate with path
+  - ClawHub floating candidate
+  - ClawHub pinned candidate
+
+### 13.3 Required integration tests
+
+- add Git source by shorthand / URL / local path
+- add GitHub tree URL
+- add Git source with `--path`
+- reject `--path` when it exists but is not a valid skill root
+- reject duplicate repo registration when only `--path` differs
+
+- add ClawHub floating source
+- add ClawHub pinned source
+- reject missing ClawHub package
+- reject ClawHub bundle with zero valid skills
+
+- update floating ClawHub source to newer version
+- keep pinned ClawHub source fixed on update
+- uninstall ClawHub source cleanly
+
+- `find` local-only hit
+- `find` remote-only hit
+- `find` mixed local+remote hit
+- `search` alias parity with `find`
+- `find --json` output shape
+
+### 13.4 Required failure-path tests
+
+- local inventory success + remote ClawHub failure -> return local results with warning
+- local empty + remote ClawHub failure -> clear error
+- malformed remote candidate payload -> clear drop/error behavior
+- failed ClawHub fetch must not dirty manifest/lock
+- mixed git + clawhub state survives:
+  - list
+  - update
+  - uninstall
+  - reconcile
+
+### 13.5 Backward compatibility tests
+
+- old git-only lock remains readable
+- optional ClawHub lock fields do not break Git-only reads
+- mixed git/clawhub sources coexist in one lock file
+
+## 14. Failure Modes
+
+| Codepath | Realistic failure | Test required | Error handling required | User-visible outcome |
+|---|---|---:|---:|---|
+| Git `add --path` | path exists but is not a valid skill root | yes | yes | clear validation error |
+| ClawHub `add` | package found but bundle has no valid skills | yes | yes | clear add failure |
+| ClawHub `update` | resolved version changes but lock/inventory diverge | yes | yes | clear update failure, no partial state |
+| `find` | remote search fails | yes | yes | warning + partial results, or clear error |
+| command builder | wrong follow-up `add` command | yes | yes | no silent mismatch |
+| mixed-state reconcile | Git-only assumptions break ClawHub records | yes | yes | no silent corruption |
+
+Critical rule:
+- if a failure mode has no test and no clear user-visible error, implementation is not done
+
+## 15. Open Questions And Locked Recommendations
+
+### 15.1 ClawHub fetch method
+
+Question:
+- use documented HTTP contract directly, or shell out to CLI?
+
+Locked recommendation:
+- prefer documented HTTP / bundle fetch directly
+- CLI bridge is fallback only if HTTP contract is incomplete
+
+### 15.2 Path-scoped add semantics
+
+Question:
+- should `--path` auto-select and auto-bind the leaf?
+
+Locked recommendation:
+- no hidden auto-apply
+- use `--path` only to narrow the add result / optional persisted preference
+- let `config` remain the binding surface
+
+### 15.3 Floating ClawHub version policy
+
+Question:
+- when should a floating source advance?
+
+Locked recommendation:
+- resolve version on add
+- persist resolved version in lock
+- only advance on explicit `update`
+
+## 16. Inline Diagram Targets In Code
+
+When implementing this plan, add or update inline ASCII diagrams in:
+
+- [src/services/source-service.ts](/Users/Vint/仓库/03%20Project/skill-manager/src/services/source-service.ts)
+  For:
+  - source resolve -> fetch -> scan -> persist pipeline
+
+- [src/services/skill-flow.ts](/Users/Vint/仓库/03%20Project/skill-manager/src/services/skill-flow.ts)
+  For:
+  - `find` orchestration
+  - candidate -> command output flow
+
+- new ranking / command-builder files
+  For:
+  - ranking rule table
+  - candidate-to-command decision tree
+
+## 17. Definition Of Done
+
+`v1.0.1` is done when all of the following are true:
+
+- naming and state-root references are aligned on `skill-flow`
+- source cache layout is `source/<kind>/<source-id>`
+- `add` supports Git and ClawHub locators
+- `add` supports repo subpath intent without duplicating source identity
+- `find` exists
+- `search` aliases to `find`
+- `find` merges local inventory and ClawHub metadata
+- ranking follows the explicit rule table
+- all `find` outputs use the shared candidate-to-command builder
+- partial remote failure behavior is implemented and tested
+- mixed git/clawhub lock state is implemented and tested
+- docs match the shipped command surface
+
+## 18. References
+
+- <https://github.com/vercel-labs/skills>
+- <https://github.com/vercel-labs/skills/tree/main/skills/find-skills>
+- <https://github.com/openai/skills/blob/main/skills/.system/skill-installer/SKILL.md>
+- <https://docs.openclaw.ai/tools/clawhub>