@vltpkg/vsr 0.0.0-27 → 0.0.0-29

package/info/PROJECT_STRUCTURE.md

@@ -0,0 +1,291 @@
+# VSR Project Directory Structure
+
+**VSR** is a minimal "npm-compatible" registry that replicates core
+features of `registry.npmjs.org` while adding new capabilities. It's
+built to run on Cloudflare Workers with D1 database and R2 storage.
+
+## 📁 Root Directory
+
+```
+registry/
+├── config.ts                   # Global application configuration
+├── package.json                # NPM package definition & scripts
+├── wrangler.json               # Cloudflare Workers deployment config
+├── drizzle.config.js           # Database ORM configuration
+├── tsconfig.worker.json        # TypeScript config for Workers environment
+├── types.ts                    # Shared TypeScript type definitions
+├── README.md                   # Project documentation & setup guide
+├── CONTRIBUTING.md             # Development guidelines & workflow
+├── LICENSE                     # FSL-1.1-MIT license
+└── src/                        # Source code directory
+```
+
+### Configuration Files
+
+- **`config.ts`** - Central configuration hub containing:
+  - API documentation settings
+  - Upstream registry definitions (npm, local)
+  - Authentication domains and redirect URIs
+  - Cookie options and security settings
+  - Development server configuration
+
+- **`wrangler.json`** - Cloudflare Workers configuration:
+  - D1 database bindings for SQLite storage
+  - R2 bucket bindings for package tarballs
+  - Queue configurations for background processing
+  - Asset serving and development settings
+
+- **`package.json`** - Project metadata and tooling:
+  - Build scripts for dist creation and asset copying
+  - Database management commands (push, migrate, studio)
+  - Development server orchestration
+  - TypeScript compilation for both Node.js and Workers
+
+## 📁 Source Code (`src/`)
+
+```
+src/
+├── index.ts                    # Main application entry point
+├── api.ts                      # OpenAPI specification
+├── routes/                     # HTTP route handlers
+├── utils/                      # Shared utility functions
+├── db/                         # Database layer
+├── assets/                     # Static files & frontend
+├── bin/                        # CLI executables
+└── schemas/                    # Validation schemas
+```
+
+### Core Application
+
+- **`index.ts`** - Main Hono application setup:
+  - Middleware stack (auth, logging, CORS, security)
+  - Route mounting and organization
+  - Queue consumer for background package refreshing
+  - Health checks and API documentation endpoints
+
+- **`api.ts`** - OpenAPI specification:
+  - REST API definitions for all endpoints
+  - Request/response schemas
+  - Authentication requirements
+  - Used by Scalar for auto-generated documentation
+
+## 📁 Route Handlers (`src/routes/`)
+
+```
+routes/
+├── users.ts                    # User profile management
+├── tokens.ts                   # Authentication token CRUD
+├── packages.ts                 # Package operations (45KB - core logic)
+├── search.ts                   # Package search & discovery
+├── auth.ts                     # OAuth authentication flows
+├── access.ts                   # Access control & permissions
+└── static.ts                   # Static asset serving
+```
+
+### Route Responsibilities
+
+- **`packages.ts`** (Primary) - Core package registry functionality:
+  - Package publishing and unpublishing
+  - Version management and dist-tags
+  - Tarball upload/download via R2
+  - Upstream registry proxying for missing packages
+  - Package metadata validation and transformation
+
+- **`tokens.ts`** - Granular Access Token (GAT) management:
+  - Token creation with scoped permissions
+  - CRUD operations for user tokens
+  - Scope validation (pkg:read/write, user:read/write)
+
+- **`auth.ts`** - Authentication workflows:
+  - OAuth callback handling
+  - Session management
+  - Login/logout flows
+
+- **`access.ts`** - Fine-grained access control:
+  - Package-level permissions
+  - Collaborator management
+  - Access list operations
+
+- **`search.ts`** - Package discovery:
+  - Text-based package searching
+  - Filtering and pagination
+  - Integration with upstream registries
+
+- **`users.ts`** - User management:
+  - Profile retrieval (`/-/whoami`)
+  - User information endpoints
+
+- **`static.ts`** - Asset delivery:
+  - Favicon serving
+  - CSS stylesheet delivery
+  - Image asset routing
+  - robots.txt and manifest files
+
+## 📁 Utilities (`src/utils/`)
+
+```
+utils/
+├── auth.ts                     # Authentication & token verification
+├── cache.ts                    # Package caching strategies (13KB)
+├── database.ts                 # Database connection middleware
+├── packages.ts                 # Package validation & processing (13KB)
+├── response.ts                 # HTTP response formatting
+├── routes.ts                   # Route middleware & guards
+├── spa.ts                      # Single Page Application serving
+├── tracing.ts                  # Request monitoring & performance
+└── upstream.ts                 # Registry proxying & fallback
+```
+
+### Utility Functions
+
+- **`cache.ts`** - Intelligent caching system:
+  - Package metadata caching from upstream registries
+  - Version-specific caching strategies
+  - Cache invalidation and refresh logic
+  - Background queue integration for cache warming
+
+- **`packages.ts`** - Package processing utilities:
+  - NPM package validation
+  - Tarball extraction and analysis
+  - Manifest transformation and normalization
+  - Semver handling and version resolution
+
+- **`upstream.ts`** - Multi-registry support:
+  - Configuration-driven upstream definitions
+  - Fallback logic for missing packages
+  - Request proxying and response transformation
+  - Error handling for upstream failures
+
+- **`auth.ts`** - Security utilities:
+  - Bearer token verification
+  - Scope-based authorization
+  - JWT handling and validation
+
+## 📁 Database Layer (`src/db/`)
+
+```
+db/
+├── client.ts                   # Database client & operations
+├── schema.ts                   # Drizzle schema definitions
+└── migrations/                 # Database migrations
+    ├── 0000_initial.sql       # Base schema creation
+    ├── 0001_uuid_validation.sql # UUID constraints
+    ├── drop.sql               # Development reset script
+    └── meta/                  # Drizzle migration metadata
+        ├── _journal.json      # Migration history
+        ├── 0000_snapshot.json # Schema snapshots
+        └── 0001_snapshot.json
+```
+
+### Database Architecture
+
+- **`schema.ts`** - Core data model:
+  - `packages` table: Package metadata and tags
+  - `versions` table: Version-specific manifests
+  - `tokens` table: Authentication tokens with scoped permissions
+  - Origin tracking (local vs upstream)
+  - Caching timestamps for upstream data
+
+- **`client.ts`** - Database operations:
+  - Drizzle ORM integration
+  - CRUD operations for all entities
+  - Transaction management
+  - Connection pooling for Workers environment
+
+## 📁 Static Assets (`src/assets/`)
+
+```
+assets/
+└── public/
+    ├── images/
+    │   ├── bg.png             # Background image
+    │   ├── clients/           # Package manager logos
+    │   │   ├── logo-npm.png
+    │   │   ├── logo-yarn.png
+    │   │   ├── logo-pnpm.png
+    │   │   ├── logo-bun.png
+    │   │   ├── logo-deno.png
+    │   │   └── logo-vlt.png
+    │   └── favicon/           # Browser icons (multiple formats)
+    │       ├── favicon.ico
+    │       ├── favicon.svg
+    │       ├── apple-touch-icon.png
+    │       ├── favicon-96x96.png
+    │       ├── site.webmanifest
+    │       ├── web-app-manifest-192x192.png
+    │       └── web-app-manifest-512x512.png
+    └── styles/
+        └── styles.css         # Application CSS
+```
+
+### Asset Organization
+
+- **`images/clients/`** - Package manager branding:
+  - Logos for supported package managers
+  - Used in web interface for client recognition
+  - Consistent sizing and format
+
+- **`images/favicon/`** - Browser integration:
+  - Multiple icon formats for cross-platform support
+  - Progressive Web App manifest
+  - Apple touch icons for iOS devices
+
+## 📁 CLI Tools (`src/bin/`)
+
+```
+bin/
+├── vsr.ts                      # Main CLI entry point
+└── demo/                       # Demo project workspace
+    ├── package.json           # Demo dependencies
+    └── vlt.json               # VLT configuration
+```
+
+### Command Line Interface
+
+- **`vsr.ts`** - Development server orchestration:
+  - Spawns both the vlt daemon (port 3000) and wrangler dev
+    (port 1337)
+  - Manages local development environment
+  - Debug mode for verbose logging
+  - Automatic path resolution for monorepo structure
+
+- **`demo/`** - Testing workspace:
+  - Minimal project for vlt server requirements
+  - Used during development to simulate real package operations
+  - Contains basic package.json and vlt configuration
+
+## 📁 Validation (`src/schemas/`)
+
+```
+schemas/
+└── [Currently empty - reserved for future validation schemas]
+```
+
+**Purpose**: Reserved directory for request/response validation
+schemas, likely to be populated with Zod or similar validation
+libraries.
+
+## 🏗️ Architecture Overview
+
+### Request Flow
+
+1. **Static Assets** → Direct serving via Cloudflare Workers
+2. **Package Requests** → Local DB check → Upstream fallback if needed
+3. **Authentication** → Bearer token validation → Scope checking
+4. **Publishing** → Validation → R2 storage → DB metadata update
+
+### Data Flow
+
+- **Local Packages**: Stored in D1 (metadata) + R2 (tarballs)
+- **Upstream Packages**: Cached in D1 with TTL, proxied from npm
+- **Background Jobs**: Queue-based cache refresh for popular packages
+
+### Security Model
+
+- **Granular Access Tokens**: Scoped permissions per package/user
+- **Origin Tracking**: Distinguish local vs upstream packages
+- **Scope Validation**: Package-level and user-level access control
+
+This architecture provides a scalable, serverless npm registry that
+can serve as both a private registry and an intelligent proxy to
+upstream registries like npmjs.org.

package/info/ROADMAP.md

@@ -0,0 +1,27 @@
+### Roadmap
+
+#### v1.0.0
+
+| Status | Feature                                               |
+| :----: | :---------------------------------------------------- |
+|   ⏳   | web: user login (ex. `npm login` / `--auth-type=web`) |
+|   ⏳   | web: user account management (`hosted`)               |
+|   ⏳   | web: user registration (`hosted`)                     |
+|   ⏳   | web: admin user management (`hosted`)                 |
+|   ⏳   | web: package pages                                    |
+|   ⏳   | web: search                                           |
+|   ⏳   | api: rate-limiting                                    |
+
+#### v1.x
+
+| Status | Feature                                   |
+| :----: | :---------------------------------------- |
+|   🕤   | api: package insights (powered by socket) |
+|   🕤   | api: audit (powered by socket)            |
+|   🕤   | mfa access provisioning                   |
+|   🕤   | orgs                                      |
+|   🕤   | teams                                     |
+|   🕤   | staging                                   |
+|   🕤   | events/hooks                              |
+|   🕤   | plugins/middleware                        |
+|   🕤   | variants/distributions                    |

package/info/SUPPORT.md

@@ -0,0 +1,39 @@
+# Supported `npm` Client Commands
+
+| Support | Commannd                                                                                |
+| :-----: | :-------------------------------------------------------------------------------------- |
+|   ✅    | `access`                                                                                |
+|   ✅    | `access list packages`                                                                  |
+|   ✅    | `access get status`                                                                     |
+|   ✅    | `access set status`                                                                     |
+|   🕤    | `access set mfa`                                                                        |
+|   ✅    | `access grant`                                                                          |
+|   ✅    | `access revoke`                                                                         |
+|   🕤    | `adduser` - `PUT /-/org/@<org>/<user>`: Adds/updates a user (requires admin privileges) |
+|   ⏳    | `audit`                                                                                 |
+|   ✅    | `bugs`                                                                                  |
+|   ✅    | `dist-tag add`                                                                          |
+|   ✅    | `dist-tag rm`                                                                           |
+|   ✅    | `dist-tag ls`                                                                           |
+|   ✅    | `deprecate`                                                                             |
+|   ✅    | `docs`                                                                                  |
+|   ✅    | `exec`                                                                                  |
+|   ✅    | `install`                                                                               |
+|   ⏳    | `login`                                                                                 |
+|   ⏳    | `logout`                                                                                |
+|   🕤    | `org`                                                                                   |
+|   ✅    | `outdated`                                                                              |
+|   🕤    | `owner add`                                                                             |
+|   🕤    | `owner rm`                                                                              |
+|   🕤    | `owner ls`                                                                              |
+|   ✅    | `ping`                                                                                  |
+|   🕤    | `profile enable-2fa`                                                                    |
+|   🕤    | `profile disable-2fa`                                                                   |
+|   ✅    | `profile get`                                                                           |
+|   🕤    | `profile set` - `PUT /-/npm/v1/user`: Updates a user (requires auth)                    |
+|   ✅    | `publish`                                                                               |
+|   ✅    | `repo`                                                                                  |
+|   ✅    | `search`                                                                                |
+|   🕤    | `team`                                                                                  |
+|   ✅    | `view`                                                                                  |
+|   ✅    | `whoami`                                                                                |

package/info/TESTING.md

@@ -0,0 +1,301 @@
+# VSR Registry Tests
+
+This directory contains comprehensive tests for the VSR (vlt-specific
+registry) covering all endpoints and functionality.
+
+## Test Structure
+
+The test suite uses a hybrid approach combining both mocked and real
+Cloudflare Workers environments for optimal coverage and performance.
+
+### Core Package Tests
+
+#### `manifest.test.ts`
+
+**Package manifest endpoint tests** covering both local/private
+packages and upstream registry public packages.
+
+**Covers:**
+
+- Local package manifests (packuments, versions, tarballs)
+- Upstream registry manifests (NPM, JSR, Custom)
+- Version-specific requests and semver handling
+- Error handling for invalid packages and versions
+- Response structure validation
+
+#### `packaument.test.ts`
+
+**Package packument endpoint tests** for full package information
+including all versions and metadata.
+
+**Covers:**
+
+- Local/private package packuments
+- Upstream public package packuments (NPM, JSR, Custom)
+- Version range filtering and semver queries
+- Response structure validation (name, dist-tags, versions, time)
+- Error handling and special package names
+- Performance and caching behavior
+
+### Utility Endpoint Tests
+
+#### `ping.test.ts`
+
+**Health check endpoint tests** for both root and upstream registries.
+
+**Covers:**
+
+- Root registry ping (`/-/ping`)
+- Upstream registry ping (`/{upstream}/-/ping`)
+- NPM compatibility headers
+- Response format validation
+
+#### `whoami.test.ts`
+
+**User identity endpoint tests** for authentication and user
+information.
+
+**Covers:**
+
+- Root registry whoami (`/-/whoami`)
+- Upstream registry whoami (`/{upstream}/-/whoami`)
+- Authentication behavior (authenticated vs unauthenticated)
+- Response format consistency
+- Error handling for invalid tokens
+
+#### `search.test.ts`
+
+**Package search endpoint tests** for finding packages across
+registries.
+
+**Covers:**
+
+- Root registry search (`/-/search`)
+- Upstream registry search (`/{upstream}/-/search`)
+- Query parameter handling (text, size, from, quality, popularity,
+  maintenance)
+- Search response structure and pagination
+- Legacy API redirects (`/-/v1/search`)
+- Error handling and performance limits
+
+### Management Endpoint Tests
+
+#### `tokens.test.ts`
+
+**Token management endpoint tests** for authentication token CRUD
+operations.
+
+**Covers:**
+
+- Token listing (`GET /-/tokens`)
+- Token creation (`POST /-/tokens`)
+- Token updates (`PUT /-/tokens`)
+- Token deletion (`DELETE /-/tokens/{token}`)
+- Upstream token management
+- Authentication and authorization
+- Request validation and error handling
+
+#### `access.test.ts`
+
+**Access control endpoint tests** for package permissions and
+collaborator management.
+
+**Covers:**
+
+- Package access status (`/-/package/{pkg}/access`)
+- Collaborator management
+  (`/-/package/{pkg}/collaborators/{username}`)
+- Scoped package access control
+- Package list access (`/-/package/list`)
+- Authentication and authorization levels
+- Permission validation
+
+#### `dist-tags.test.ts`
+
+**Distribution tags endpoint tests** for package version tagging.
+
+**Covers:**
+
+- Dist-tag retrieval (`/-/package/{pkg}/dist-tags`)
+- Dist-tag creation/updates (`PUT /-/package/{pkg}/dist-tags/{tag}`)
+- Dist-tag deletion (`DELETE /-/package/{pkg}/dist-tags/{tag}`)
+- Semver validation and tag name validation
+- Authentication and ownership checks
+- Complete dist-tag lifecycle operations
+
+### Security and Audit Tests
+
+#### `audit.test.ts`
+
+**Security audit endpoint tests** for vulnerability scanning.
+
+**Covers:**
+
+- Root registry audit (`/-/npm/audit`)
+- Upstream registry audit (`/{upstream}/-/npm/audit`)
+- Package lock format support (v1, v2, v3)
+- Dependency tree validation
+- Legacy API redirects
+- Request validation and error handling
+- Audit implementation status
+
+### Infrastructure Tests
+
+#### `static.test.ts`
+
+**Static asset endpoint tests** for web interface and public files.
+
+**Covers:**
+
+- Public assets (`/public/*`)
+- Special files (`/favicon.ico`, `/robots.txt`, `/manifest.json`)
+- Content-type headers for different file types
+- Cache control and compression
+- Security considerations (directory traversal prevention)
+- Performance optimization (range requests, conditional requests)
+
+#### `dashboard.test.ts`
+
+**Dashboard endpoint tests** for daemon-mode web interface data.
+
+**Covers:**
+
+- Dashboard configuration (`/dashboard.json`)
+- Application data (`/app-data.json`)
+- Daemon enable/disable behavior
+- Data structure validation
+- Feature flag integration
+- Performance and concurrent request handling
+
+## Testing Approach
+
+### Hybrid Testing Strategy
+
+The test suite uses two complementary approaches:
+
+1. **Mocked Environment Tests** (`app.request()` with `mockEnv`)
+   - Used for complex endpoints (manifest, packument, search, tokens,
+     access, etc.)
+   - Fast execution with comprehensive mocking
+   - Reliable and deterministic results
+   - Covers edge cases and error conditions
+
+2. **Real Cloudflare Workers Tests** (`SELF.fetch()` with real
+   bindings)
+   - Used for simple endpoints (ping, whoami)
+   - Real database and environment integration
+   - End-to-end validation with actual Workers runtime
+   - Configured via `@cloudflare/vitest-pool-workers`
+
+### Mock Environment Structure
+
+```typescript
+const mockEnv = {
+  DB: {
+    // Minimal D1 interface with Drizzle ORM compatibility
+    prepare: () => ({ bind: () => ({ get, all, run, raw }) }),
+    batch: () => Promise.resolve([]),
+    exec: () => Promise.resolve(),
+  },
+  BUCKET: { get, put, delete },
+  KV: { get, put, delete },
+  // Additional bindings as needed
+}
+```
+
+## Running Tests
+
+```bash
+# Run all tests
+pnpm test
+
+# Run specific test files
+pnpm test test/manifest.test.ts
+pnpm test test/tokens.test.ts
+pnpm test test/search.test.ts
+
+# Run tests in watch mode
+pnpm test --watch
+
+# Run tests with coverage
+pnpm test --coverage
+
+# Run linting
+pnpm lint
+```
+
+## Test Results
+
+Current test coverage includes **8 comprehensive test files** with
+**300+ individual tests**:
+
+- ✅ **manifest.test.ts** - 33 tests (Package manifests)
+- ✅ **packaument.test.ts** - 40 tests (Package packuments)
+- ✅ **ping.test.ts** - 5 tests (Health checks)
+- ✅ **whoami.test.ts** - 10 tests (User identity)
+- ✅ **search.test.ts** - 50+ tests (Package search)
+- ✅ **tokens.test.ts** - 40+ tests (Token management)
+- ✅ **access.test.ts** - 60+ tests (Access control)
+- ✅ **dist-tags.test.ts** - 50+ tests (Distribution tags)
+- ✅ **audit.test.ts** - 40+ tests (Security audits)
+- ✅ **static.test.ts** - 60+ tests (Static assets)
+- ✅ **dashboard.test.ts** - 40+ tests (Dashboard data)
+
+## Key Features Tested
+
+### NPM Registry Compatibility
+
+- Full npm client compatibility for all utility endpoints
+- Proper HTTP status codes and response formats
+- Legacy API redirect support
+- Semver validation and handling
+
+### Multi-Registry Support
+
+- Root/local registry functionality
+- Upstream registry proxying (NPM, JSR, Custom)
+- Registry-specific endpoint behavior
+- Upstream configuration validation
+
+### Security and Authentication
+
+- Token-based authentication
+- Package access control and permissions
+- Collaborator management
+- Security audit functionality
+- Input validation and sanitization
+
+### Performance and Reliability
+
+- Response time validation
+- Concurrent request handling
+- Error handling and edge cases
+- Cache control and optimization
+- Resource limits and validation
+
+## Configuration
+
+Tests are configured in `vitest.config.ts` with:
+
+- **Cloudflare Workers pool** for real environment testing
+- **30-second timeout** for upstream requests
+- **Real bindings setup** in `test/setup.ts`
+- **Database schema creation** for isolated test environments
+- **Environment variable configuration**
+
+## Future Improvements
+
+1. **Enhanced Integration Testing**
+   - More real Cloudflare Workers tests as the framework matures
+   - End-to-end workflow testing
+   - Performance benchmarking
+
+2. **Advanced Test Scenarios**
+   - Load testing for high-traffic scenarios
+   - Chaos engineering for resilience testing
+   - Cross-registry interaction testing
+
+3. **Test Automation**
+   - Automated test generation for new endpoints
+   - Contract testing for API compatibility
+   - Visual regression testing for web interface

package/info/USER_SUPPORT.md

@@ -0,0 +1,31 @@
+# Support
+
+## News
+
+- Home: <https://vlt.sh>
+- Blog: <https://blog.vlt.sh/>
+
+## Official Documentation
+
+- CLI: <https://docs.vlt.sh/cli>
+- VSR: <https://docs.vlt.sh/registry>
+
+## Social Media
+
+- GitHub: <https://github.com/vltpkg>
+- Linkedin: <https://www.linkedin.com/company/vltpkg/>
+- Bluesky: <https://bsky.app/profile/vlt.sh>
+- Twitter: <https://x.com/vltpkg>
+- Discord: <https://discord.gg/vltpkg>
+- Youtube: <https://www.youtube.com/@vltpkg>
+
+## Public Calendar
+
+- gCal:
+  <https://calendar.google.com/calendar/embed?src=c_ee5f7f2a875caac1c7a90183d7faf31f2fe0e22b94aca147f005bfb3e0c7e072%40group.calendar.google.com>
+- iCal:
+  <https://calendar.google.com/calendar/ical/c_ee5f7f2a875caac1c7a90183d7faf31f2fe0e22b94aca147f005bfb3e0c7e072%40group.calendar.google.com/public/basic.ics>
+
+## More
+
+- Package Managers Benchmarks: <https://benchmarks.vlt.sh>