codexkit 1.0.0

package/.codex/skills/opensrc/SKILL.md

@@ -0,0 +1,127 @@
+---
+name: opensrc
+description: Fetch source code for npm, PyPI, or crates.io packages and GitHub/GitLab repos to provide AI agents with implementation context beyond types and docs. Use when needing to understand how a library works internally, debug dependency issues, or explore package implementations.
+version: 1.0.0
+tags: [research, integration]
+dependencies: []
+---
+
+# opensrc
+
+## When to Use
+
+- When you need actual dependency source code to understand behavior or debug issues.
+
+## When NOT to Use
+
+- When API docs or local type definitions are sufficient for the task.
+
+
+## When to Use
+
+- Need to understand how a library/package works internally (not just its interface)
+- Debugging issues where types alone are insufficient
+- Exploring implementation patterns in dependencies
+- Agent needs to reference actual source code of a package
+
+## Quick Start
+
+```bash
+# Install globally or use npx
+npm install -g opensrc
+
+# Fetch npm package (auto-detects installed version from lockfile)
+npx opensrc zod
+
+# Fetch from other registries
+npx opensrc pypi:requests       # Python/PyPI
+npx opensrc crates:serde        # Rust/crates.io
+
+# Fetch GitHub repo directly
+npx opensrc vercel/ai           # owner/repo shorthand
+npx opensrc github:owner/repo   # explicit prefix
+npx opensrc https://github.com/colinhacks/zod  # full URL
+
+# Fetch specific version/ref
+npx opensrc zod@3.22.0
+npx opensrc owner/repo@v1.0.0
+```
+
+## Commands
+
+| Command                 | Description                     |
+| ----------------------- | ------------------------------- |
+| `opensrc <packages...>` | Fetch source for packages/repos |
+| `opensrc list`          | List all fetched sources        |
+| `opensrc remove <name>` | Remove specific source          |
+| `opensrc clean`         | Remove all sources              |
+
+## Output Structure
+
+After fetching, sources stored in `opensrc/` directory:
+
+```
+opensrc/
+├── settings.json           # User preferences
+├── sources.json            # Index of fetched packages/repos
+└── repos/
+    └── github.com/
+        └── owner/
+            └── repo/       # Cloned source code
+```
+
+## File Modifications
+
+On first run, opensrc prompts to modify:
+
+- `.gitignore` - adds `opensrc/` to ignore list
+- `tsconfig.json` - excludes `opensrc/` from compilation
+- `AGENTS.md` - adds section pointing agents to source code
+
+Use `--modify` or `--modify=false` to skip prompt.
+
+## Key Behaviors
+
+1. **Version Detection** - For npm, auto-detects installed version from `node_modules`, `package-lock.json`, `pnpm-lock.yaml`, or `yarn.lock`
+2. **Repository Resolution** - Resolves package to its git repo via registry API, clones at matching tag
+3. **Monorepo Support** - Handles packages in monorepos via `repository.directory` field
+4. **Shallow Clone** - Uses `--depth 1` for efficient cloning, removes `.git` after clone
+5. **Tag Fallback** - Tries `v{version}`, `{version}`, then default branch if tag not found
+
+## Common Workflows
+
+### Fetching a Package
+
+```bash
+# Agent needs to understand zod's implementation
+npx opensrc zod
+# → Detects version from lockfile
+# → Finds repo URL from npm registry
+# → Clones at matching git tag
+# → Source available at opensrc/repos/github.com/colinhacks/zod/
+```
+
+### Updating Sources
+
+```bash
+# Re-run same command to update to currently installed version
+npx opensrc zod
+# → Checks if version changed
+# → Re-clones if needed
+```
+
+### Multiple Sources
+
+```bash
+# Fetch multiple at once
+npx opensrc react react-dom next
+npx opensrc zod pypi:pydantic vercel/ai
+```
+
+## References
+
+For detailed information:
+
+- [CLI Usage & Options](references/cli-usage.md) - Full command reference
+- [Architecture](references/architecture.md) - Code structure and extension
+- [Registry Support](references/registry-support.md) - npm, PyPI, crates.io details

package/.codex/skills/opensrc/references/architecture.md

@@ -0,0 +1,176 @@
+# opensrc Architecture
+
+## Contents
+- Directory structure
+- Core flow
+- Key modules
+- Type definitions
+- Extension points
+
+## Directory Structure
+
+```
+src/
+├── index.ts              # CLI entry, Commander setup
+├── types.ts              # Shared type definitions
+├── commands/
+│   ├── fetch.ts          # Main fetch logic
+│   ├── list.ts           # List sources
+│   ├── remove.ts         # Remove sources
+│   └── clean.ts          # Clean all sources
+└── lib/
+    ├── git.ts            # Clone, path management
+    ├── repo.ts           # Parse/resolve repo specs
+    ├── version.ts        # Detect installed versions
+    ├── agents.ts         # AGENTS.md/sources.json updates
+    ├── gitignore.ts      # .gitignore management
+    ├── tsconfig.ts       # tsconfig.json exclude
+    ├── settings.ts       # User preferences
+    ├── prompt.ts         # Interactive prompts
+    └── registries/
+        ├── index.ts      # Registry detection/routing
+        ├── npm.ts        # npm registry API
+        ├── pypi.ts       # PyPI API
+        └── crates.ts     # crates.io API
+```
+
+## Core Flow
+
+### Fetch Package
+
+1. **Parse input** - Detect registry prefix or repo format
+2. **Version detection** (npm only) - Check lockfiles/node_modules
+3. **Registry lookup** - Query API for repo URL
+4. **Clone** - Shallow clone at matching git tag
+5. **Update index** - Write to sources.json, optionally AGENTS.md
+
+### Input Type Detection
+
+```
+detectInputType(spec) → "package" | "repo"
+```
+
+- Registry prefix (`npm:`, `pypi:`, `crates:`) → package
+- Repo pattern (`owner/repo`, URL, `github:`) → repo
+- Default → package (npm)
+
+## Key Modules
+
+### registries/index.ts
+
+Routes to correct registry handler:
+
+```typescript
+parsePackageSpec(spec): PackageSpec
+resolvePackage(spec): ResolvedPackage
+detectRegistry(spec): { registry, cleanSpec }
+detectInputType(spec): "package" | "repo"
+```
+
+### git.ts
+
+Manages cloning and source storage:
+
+```typescript
+fetchSource(resolved, cwd): FetchResult
+fetchRepoSource(resolved, cwd): FetchResult
+listSources(cwd): { packages, repos }
+removePackageSource(name, cwd, registry): { removed, repoRemoved }
+```
+
+Storage structure: `opensrc/repos/{host}/{owner}/{repo}/`
+
+### version.ts
+
+Detects installed npm versions (priority order):
+1. `node_modules/{pkg}/package.json`
+2. `package-lock.json`
+3. `pnpm-lock.yaml`
+4. `yarn.lock`
+5. `package.json` (strips range prefixes)
+
+### repo.ts
+
+Parses repo specs, resolves via GitHub/GitLab API:
+
+```typescript
+parseRepoSpec(spec): RepoSpec | null
+resolveRepo(spec): ResolvedRepo
+isRepoSpec(spec): boolean
+```
+
+Supports: GitHub, GitLab, Bitbucket (GitHub/GitLab via API).
+
+## Type Definitions
+
+```typescript
+type Registry = "npm" | "pypi" | "crates"
+
+interface PackageSpec {
+  registry: Registry
+  name: string
+  version?: string
+}
+
+interface ResolvedPackage {
+  registry: Registry
+  name: string
+  version: string
+  repoUrl: string
+  repoDirectory?: string  // For monorepos
+  gitTag: string
+}
+
+interface RepoSpec {
+  host: string      // github.com, gitlab.com
+  owner: string
+  repo: string
+  ref?: string      // Branch, tag, commit
+}
+
+interface FetchResult {
+  package: string
+  version: string
+  path: string
+  success: boolean
+  error?: string
+  registry?: Registry
+}
+```
+
+## Extension Points
+
+### Adding a Registry
+
+1. Create `registries/{name}.ts`:
+   - `parse{Name}Spec(spec)` - Parse name@version
+   - `resolve{Name}Package(name, version)` - Query API, return ResolvedPackage
+
+2. Update `registries/index.ts`:
+   - Add prefix to `REGISTRY_PREFIXES`
+   - Add case to `parsePackageSpec` and `resolvePackage`
+
+3. Add to `Registry` type in `types.ts`
+
+### Registry Implementation Pattern
+
+```typescript
+// Parse spec into name/version
+export function parseSpec(spec: string): { name: string; version?: string }
+
+// Query registry API, extract repo URL
+async function fetchInfo(name: string): Promise<RegistryResponse>
+
+// Normalize git URLs (remove git+, .git suffix)
+function extractRepoUrl(info): string | null
+
+// Main resolver
+export async function resolve(name: string, version?: string): Promise<ResolvedPackage>
+```
+
+### Git Tag Resolution
+
+Clone tries tags in order:
+1. `v{version}` (most common)
+2. `{version}` (no prefix)
+3. Default branch (fallback with warning)

package/.codex/skills/opensrc/references/cli-usage.md

@@ -0,0 +1,176 @@
+# opensrc CLI Reference
+
+## Contents
+- Fetch command (default)
+- List command
+- Remove command
+- Clean command
+- Global options
+
+## Fetch Command (Default)
+
+Fetch source for packages/repos.
+
+```bash
+opensrc [packages...] [options]
+```
+
+### Package Formats
+
+| Format | Example | Description |
+|--------|---------|-------------|
+| `<name>` | `zod` | npm package (latest or installed version) |
+| `<name>@<version>` | `zod@3.22.0` | npm package at specific version |
+| `npm:<name>` | `npm:react` | Explicit npm prefix |
+| `pypi:<name>` | `pypi:requests` | Python package from PyPI |
+| `pip:<name>` | `pip:flask` | Alias for pypi: |
+| `crates:<name>` | `crates:serde` | Rust crate from crates.io |
+| `cargo:<name>` | `cargo:tokio` | Alias for crates: |
+
+### Repository Formats
+
+| Format | Example |
+|--------|---------|
+| `owner/repo` | `vercel/ai` |
+| `owner/repo@ref` | `vercel/ai@v1.0.0` |
+| `owner/repo#ref` | `vercel/ai#main` |
+| `github:owner/repo` | `github:facebook/react` |
+| `gitlab:owner/repo` | `gitlab:inkscape/inkscape` |
+| `https://github.com/...` | Full URL |
+
+### Options
+
+| Option | Description |
+|--------|-------------|
+| `--cwd <path>` | Working directory (default: cwd) |
+| `--modify` | Allow file modifications (skip prompt) |
+| `--modify=false` | Deny file modifications (skip prompt) |
+
+### Examples
+
+```bash
+# Single package
+opensrc zod
+
+# Multiple packages
+opensrc react react-dom next
+
+# Mixed registries
+opensrc zod pypi:requests crates:serde
+
+# Specific versions
+opensrc zod@3.22.0
+opensrc pypi:requests==2.31.0
+
+# GitHub repos
+opensrc vercel/ai
+opensrc facebook/react@v18.2.0
+```
+
+## List Command
+
+List all fetched sources.
+
+```bash
+opensrc list [options]
+```
+
+### Options
+
+| Option | Description |
+|--------|-------------|
+| `--json` | Output as JSON |
+| `--cwd <path>` | Working directory |
+
+### Output
+
+Groups by registry (npm, PyPI, crates.io) and repos:
+
+```
+npm Packages:
+
+  zod@3.22.0
+    Path: opensrc/repos/github.com/colinhacks/zod
+    Fetched: Jan 10, 2025
+
+Repositories:
+
+  github.com/vercel/ai@main
+    Path: opensrc/repos/github.com/vercel/ai
+    Fetched: Jan 10, 2025
+
+Total: 1 package(s) (1 npm), 1 repo(s)
+```
+
+## Remove Command
+
+Remove specific sources.
+
+```bash
+opensrc remove <packages...> [options]
+opensrc rm <packages...>  # alias
+```
+
+### Options
+
+| Option | Description |
+|--------|-------------|
+| `--cwd <path>` | Working directory |
+
+### Examples
+
+```bash
+opensrc remove zod
+opensrc rm pypi:requests
+opensrc remove vercel/ai
+```
+
+## Clean Command
+
+Remove all sources by category.
+
+```bash
+opensrc clean [options]
+```
+
+### Options
+
+| Option | Description |
+|--------|-------------|
+| `--packages` | Only remove packages (all registries) |
+| `--repos` | Only remove repos |
+| `--npm` | Only remove npm packages |
+| `--pypi` | Only remove PyPI packages |
+| `--crates` | Only remove crates.io packages |
+| `--cwd <path>` | Working directory |
+
+### Examples
+
+```bash
+opensrc clean              # Remove everything
+opensrc clean --packages   # Remove all packages, keep repos
+opensrc clean --npm        # Remove only npm packages
+opensrc clean --repos      # Remove only repos
+```
+
+## File Modifications
+
+On first run, prompts to modify:
+
+1. **`.gitignore`** - Adds `opensrc/` entry
+2. **`tsconfig.json`** - Adds `opensrc/` to exclude array
+3. **`AGENTS.md`** - Adds source code reference section
+
+Choice saved to `opensrc/settings.json`.
+
+### AGENTS.md Section
+
+Adds guidance for AI agents:
+
+```markdown
+<!-- opensrc:start -->
+## Source Code Reference
+Source code for dependencies is available in `opensrc/`...
+See `opensrc/sources.json` for the list of available packages.
+<!-- opensrc:end -->
+```

package/.codex/skills/opensrc/references/registry-support.md

@@ -0,0 +1,137 @@
+# Registry Support
+
+## Contents
+- npm (Node.js)
+- PyPI (Python)
+- crates.io (Rust)
+- Repository hosts
+
+## npm (Node.js)
+
+**Prefixes:** `npm:` (or none - default)
+
+**API:** `https://registry.npmjs.org/{package}`
+
+### Version Detection
+
+Auto-detects installed version from (in order):
+1. `node_modules/{pkg}/package.json`
+2. `package-lock.json` (npm v6 and v7+ formats)
+3. `pnpm-lock.yaml`
+4. `yarn.lock`
+5. `package.json` dependencies (strips `^`, `~`, etc.)
+
+### Spec Parsing
+
+```
+zod           → name: zod, version: (detected or latest)
+zod@3.22.0    → name: zod, version: 3.22.0
+@babel/core   → name: @babel/core (scoped)
+@babel/core@7.0.0 → name: @babel/core, version: 7.0.0
+```
+
+### Repo URL Extraction
+
+From `repository` field, normalizes:
+- `git+https://...` → `https://...`
+- `git://...` → `https://...`
+- `git+ssh://git@...` → `https://...`
+- Removes `.git` suffix
+- Handles `github:owner/repo` shorthand
+
+### Monorepo Support
+
+Uses `repository.directory` field for packages in monorepos.
+Path becomes: `repos/{host}/{owner}/{repo}/{directory}`
+
+## PyPI (Python)
+
+**Prefixes:** `pypi:`, `pip:`, `python:`
+
+**API:** `https://pypi.org/pypi/{package}/json`
+
+### Spec Parsing
+
+```
+requests        → name: requests
+requests==2.31.0 → name: requests, version: 2.31.0
+requests@2.31.0  → name: requests, version: 2.31.0
+```
+
+### Repo URL Extraction
+
+Checks `project_urls` in order:
+1. `Source`, `Source Code`
+2. `Repository`, `GitHub`
+3. `Code`, `Homepage`
+
+Falls back to `home_page` if it's a git URL.
+
+## crates.io (Rust)
+
+**Prefixes:** `crates:`, `cargo:`, `rust:`
+
+**API:** `https://crates.io/api/v1/crates/{crate}`
+
+### Spec Parsing
+
+```
+serde       → name: serde
+serde@1.0.0 → name: serde, version: 1.0.0
+```
+
+### Repo URL Extraction
+
+Uses `repository` field, falls back to `homepage` if it's a git URL.
+
+## Repository Hosts
+
+### GitHub
+
+**API:** `https://api.github.com/repos/{owner}/{repo}`
+
+Resolves default branch, constructs clone URL.
+
+**Rate limiting:** 60 req/hour unauthenticated. Error on 403.
+
+### GitLab
+
+**API:** `https://gitlab.com/api/v4/projects/{owner}%2F{repo}`
+
+Resolves default branch, constructs clone URL.
+
+### Bitbucket
+
+Supported for parsing (`bitbucket:owner/repo`), but no API resolution.
+Assumes `main` as default branch.
+
+## Repo Spec Formats
+
+| Format | Example | Host |
+|--------|---------|------|
+| `owner/repo` | `vercel/ai` | github.com |
+| `github:owner/repo` | `github:facebook/react` | github.com |
+| `gitlab:owner/repo` | `gitlab:inkscape/inkscape` | gitlab.com |
+| `bitbucket:owner/repo` | `bitbucket:atlassian/python-bitbucket` | bitbucket.org |
+| `host/owner/repo` | `gitlab.com/owner/repo` | (from URL) |
+| Full URL | `https://github.com/...` | (from URL) |
+
+## Ref Specifications
+
+Append to repo spec:
+
+| Suffix | Example | Meaning |
+|--------|---------|---------|
+| `@ref` | `owner/repo@v1.0.0` | Tag, branch, or commit |
+| `#ref` | `owner/repo#main` | Branch (alternative syntax) |
+
+URL paths also supported: `https://github.com/owner/repo/tree/branch`
+
+## Git Tag Patterns
+
+When cloning packages, tries tags in order:
+1. `v{version}` - Most common (e.g., `v3.22.0`)
+2. `{version}` - No prefix (e.g., `3.22.0`)
+3. Default branch - Fallback with warning
+
+Uses `--depth 1 --single-branch` for efficient cloning.