@vltpkg/vsr 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.editorconfig +13 -0
- package/.prettierrc +7 -0
- package/CONTRIBUTING.md +228 -0
- package/LICENSE.md +110 -0
- package/README.md +373 -0
- package/bin/vsr.ts +29 -0
- package/config.ts +124 -0
- package/debug-npm.js +19 -0
- package/drizzle.config.js +33 -0
- package/package.json +80 -0
- package/pnpm-workspace.yaml +5 -0
- package/src/api.ts +2246 -0
- package/src/assets/public/images/bg.png +0 -0
- package/src/assets/public/images/clients/logo-bun.png +0 -0
- package/src/assets/public/images/clients/logo-deno.png +0 -0
- package/src/assets/public/images/clients/logo-npm.png +0 -0
- package/src/assets/public/images/clients/logo-pnpm.png +0 -0
- package/src/assets/public/images/clients/logo-vlt.png +0 -0
- package/src/assets/public/images/clients/logo-yarn.png +0 -0
- package/src/assets/public/images/favicon/apple-touch-icon.png +0 -0
- package/src/assets/public/images/favicon/favicon-96x96.png +0 -0
- package/src/assets/public/images/favicon/favicon.ico +0 -0
- package/src/assets/public/images/favicon/favicon.svg +3 -0
- package/src/assets/public/images/favicon/site.webmanifest +21 -0
- package/src/assets/public/images/favicon/web-app-manifest-192x192.png +0 -0
- package/src/assets/public/images/favicon/web-app-manifest-512x512.png +0 -0
- package/src/assets/public/styles/styles.css +219 -0
- package/src/db/client.ts +544 -0
- package/src/db/migrations/0000_faulty_ricochet.sql +14 -0
- package/src/db/migrations/0000_initial.sql +29 -0
- package/src/db/migrations/0001_uuid_validation.sql +35 -0
- package/src/db/migrations/0001_wealthy_magdalene.sql +7 -0
- package/src/db/migrations/drop.sql +3 -0
- package/src/db/migrations/meta/0000_snapshot.json +104 -0
- package/src/db/migrations/meta/0001_snapshot.json +155 -0
- package/src/db/migrations/meta/_journal.json +20 -0
- package/src/db/schema.ts +41 -0
- package/src/index.ts +709 -0
- package/src/routes/access.ts +263 -0
- package/src/routes/auth.ts +93 -0
- package/src/routes/index.ts +135 -0
- package/src/routes/packages.ts +924 -0
- package/src/routes/search.ts +50 -0
- package/src/routes/static.ts +53 -0
- package/src/routes/tokens.ts +102 -0
- package/src/routes/users.ts +14 -0
- package/src/utils/auth.ts +145 -0
- package/src/utils/cache.ts +466 -0
- package/src/utils/database.ts +44 -0
- package/src/utils/packages.ts +337 -0
- package/src/utils/response.ts +100 -0
- package/src/utils/routes.ts +47 -0
- package/src/utils/spa.ts +14 -0
- package/src/utils/tracing.ts +63 -0
- package/src/utils/upstream.ts +131 -0
- package/test/README.md +91 -0
- package/test/access.test.js +760 -0
- package/test/cloudflare-waituntil.test.js +141 -0
- package/test/db.test.js +447 -0
- package/test/dist-tag.test.js +415 -0
- package/test/e2e.test.js +904 -0
- package/test/hono-context.test.js +250 -0
- package/test/integrity-validation.test.js +183 -0
- package/test/json-response.test.js +76 -0
- package/test/manifest-slimming.test.js +449 -0
- package/test/packument-consistency.test.js +351 -0
- package/test/packument-version-range.test.js +144 -0
- package/test/performance.test.js +162 -0
- package/test/route-with-waituntil.test.js +298 -0
- package/test/run-tests.js +151 -0
- package/test/setup-cache-tests.js +190 -0
- package/test/setup.js +64 -0
- package/test/stale-while-revalidate.test.js +273 -0
- package/test/static-assets.test.js +85 -0
- package/test/upstream-routing.test.js +86 -0
- package/test/utils/test-helpers.js +84 -0
- package/test/waituntil-correct.test.js +208 -0
- package/test/waituntil-demo.test.js +138 -0
- package/test/waituntil-readme.md +113 -0
- package/tsconfig.json +37 -0
- package/types.ts +446 -0
- package/vitest.config.js +95 -0
- package/wrangler.json +58 -0
package/.editorconfig
ADDED
package/.prettierrc
ADDED
package/CONTRIBUTING.md
ADDED
|
@@ -0,0 +1,228 @@
|
|
|
1
|
+
# Contributing to VSR (vlt serverless registry)
|
|
2
|
+
|
|
3
|
+
Thank you for your interest in contributing to VSR! This document provides guidelines and instructions for development, testing, and contributing to the project.
|
|
4
|
+
|
|
5
|
+
## Table of Contents
|
|
6
|
+
|
|
7
|
+
- [Contributing to VSR (vlt serverless registry)](#contributing-to-vsr-vlt-serverless-registry)
|
|
8
|
+
- [Table of Contents](#table-of-contents)
|
|
9
|
+
- [Development Setup](#development-setup)
|
|
10
|
+
- [Project Structure](#project-structure)
|
|
11
|
+
- [Testing](#testing)
|
|
12
|
+
- [Running Tests](#running-tests)
|
|
13
|
+
- [Testing Background Refresh Functionality](#testing-background-refresh-functionality)
|
|
14
|
+
- [How `waitUntil` Works](#how-waituntil-works)
|
|
15
|
+
- [Testing Patterns](#testing-patterns)
|
|
16
|
+
- [Test Files](#test-files)
|
|
17
|
+
- [Deployment](#deployment)
|
|
18
|
+
- [Local Deployment](#local-deployment)
|
|
19
|
+
- [Production Deployment](#production-deployment)
|
|
20
|
+
- [Caching Implementation](#caching-implementation)
|
|
21
|
+
- [Implementation Details](#implementation-details)
|
|
22
|
+
- [Contribution Guidelines](#contribution-guidelines)
|
|
23
|
+
- [Pull Request Process](#pull-request-process)
|
|
24
|
+
- [Code Style](#code-style)
|
|
25
|
+
|
|
26
|
+
## Development Setup
|
|
27
|
+
|
|
28
|
+
1. **Clone the repository**:
|
|
29
|
+
```bash
|
|
30
|
+
git clone https://github.com/vltpkg/vsr.git
|
|
31
|
+
cd vsr
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
2. **Install dependencies**:
|
|
35
|
+
```bash
|
|
36
|
+
pnpm install
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
3. **Run the development server**:
|
|
40
|
+
```bash
|
|
41
|
+
pnpm run serve:dev
|
|
42
|
+
```
|
|
43
|
+
This will start a local development server at http://localhost:1337 with the Cloudflare Workers environment simulated locally.
|
|
44
|
+
|
|
45
|
+
4. **Database setup**:
|
|
46
|
+
```bash
|
|
47
|
+
pnpm run db:setup
|
|
48
|
+
```
|
|
49
|
+
This initializes the local D1 database for development.
|
|
50
|
+
|
|
51
|
+
## Project Structure
|
|
52
|
+
|
|
53
|
+
- `src/` - Core application source code
|
|
54
|
+
- `db/` - Database models and migrations
|
|
55
|
+
- `routes/` - API route handlers
|
|
56
|
+
- `middleware/` - Middleware functions
|
|
57
|
+
- `utils/` - Utility functions
|
|
58
|
+
- `test/` - Test files and utilities
|
|
59
|
+
- `scripts/` - Helper scripts
|
|
60
|
+
- `bin/` - CLI entry point
|
|
61
|
+
|
|
62
|
+
## Testing
|
|
63
|
+
|
|
64
|
+
VSR has a comprehensive test suite that covers various aspects of the codebase, including the stale-while-revalidate caching pattern implemented with Cloudflare Workers' `waitUntil` API.
|
|
65
|
+
|
|
66
|
+
### Running Tests
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
# Run all tests
|
|
70
|
+
pnpm run test
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
### Testing Background Refresh Functionality
|
|
74
|
+
|
|
75
|
+
The registry implements a "stale-while-revalidate" caching pattern for faster responses:
|
|
76
|
+
|
|
77
|
+
1. When a request is received, we first check if the data is in our cache
|
|
78
|
+
2. If the data is in the cache, we return it immediately, even if it's stale (old)
|
|
79
|
+
3. If the data is stale, we queue a background task to refresh it from the upstream registry
|
|
80
|
+
4. The background task updates the cache with fresh data for future requests
|
|
81
|
+
5. This approach ensures users get a fast response (cached data) while keeping our cache up-to-date
|
|
82
|
+
|
|
83
|
+
This pattern is implemented in the following key files:
|
|
84
|
+
- `getPackagePackument`: For fetching package metadata
|
|
85
|
+
- `getPackageManifest`: For fetching specific package versions
|
|
86
|
+
|
|
87
|
+
#### How `waitUntil` Works
|
|
88
|
+
|
|
89
|
+
Cloudflare Workers provide a special API called `waitUntil` that enables background tasks to continue running after a response has been sent. Here's a simplified example:
|
|
90
|
+
|
|
91
|
+
```javascript
|
|
92
|
+
export default {
|
|
93
|
+
async fetch(request, env, ctx) {
|
|
94
|
+
// Return a response immediately
|
|
95
|
+
const response = new Response("Hello World");
|
|
96
|
+
|
|
97
|
+
// Queue a background task that continues after response is sent
|
|
98
|
+
ctx.waitUntil(
|
|
99
|
+
(async () => {
|
|
100
|
+
// This runs in the background after response is sent
|
|
101
|
+
await doLongRunningTask();
|
|
102
|
+
})()
|
|
103
|
+
);
|
|
104
|
+
|
|
105
|
+
return response;
|
|
106
|
+
}
|
|
107
|
+
};
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
### Testing Patterns
|
|
111
|
+
|
|
112
|
+
Testing `waitUntil` behavior can be tricky because of how JavaScript executes Immediately Invoked Function Expressions (IIFEs). The common mistake is to use code like this:
|
|
113
|
+
|
|
114
|
+
```javascript
|
|
115
|
+
// ❌ PROBLEM: This executes the function immediately!
|
|
116
|
+
c.waitUntil((async () => {
|
|
117
|
+
// Background work
|
|
118
|
+
await refreshCache();
|
|
119
|
+
})());
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
Our testing approach:
|
|
123
|
+
|
|
124
|
+
1. Correctly simulates the Cloudflare `waitUntil` API
|
|
125
|
+
2. Captures background tasks without executing them immediately
|
|
126
|
+
3. Allows precise control over when background tasks run
|
|
127
|
+
|
|
128
|
+
For detailed documentation on testing this pattern, refer to `test/waituntil-README.md`.
|
|
129
|
+
|
|
130
|
+
#### Test Files
|
|
131
|
+
|
|
132
|
+
- `waituntil-correct.test.js`: Demonstrates the core pattern with simple examples
|
|
133
|
+
- `hono-context.test.js`: Shows how to mock the Hono context with proper `waitUntil` support
|
|
134
|
+
- `route-with-waituntil.test.js`: Tests a route handler implementing the stale-while-revalidate pattern
|
|
135
|
+
|
|
136
|
+
## Deployment
|
|
137
|
+
|
|
138
|
+
### Local Deployment
|
|
139
|
+
|
|
140
|
+
For local usage, you can simply run:
|
|
141
|
+
|
|
142
|
+
```bash
|
|
143
|
+
vlx vltpkg/vsr
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
This will start the registry locally at http://localhost:1337.
|
|
147
|
+
|
|
148
|
+
### Production Deployment
|
|
149
|
+
|
|
150
|
+
VSR is designed to be deployed on Cloudflare Workers. To deploy to production:
|
|
151
|
+
|
|
152
|
+
1. **Set up Cloudflare account and create necessary resources**:
|
|
153
|
+
- Create a Cloudflare account if you don't have one
|
|
154
|
+
- Create a new D1 Database
|
|
155
|
+
- Create a new R2 Bucket
|
|
156
|
+
|
|
157
|
+
2. **Configure your `wrangler.toml`**:
|
|
158
|
+
Make sure your configuration includes the correct bindings for D1 and R2.
|
|
159
|
+
|
|
160
|
+
3. **Deploy using Wrangler**:
|
|
161
|
+
```bash
|
|
162
|
+
pnpm run build # Create deployment bundle
|
|
163
|
+
pnpm run deploy # Deploy to Cloudflare Workers
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
4. **Post-deployment setup**:
|
|
167
|
+
- Initialize your database with the schema
|
|
168
|
+
- Create admin tokens as needed
|
|
169
|
+
|
|
170
|
+
## Caching Implementation
|
|
171
|
+
|
|
172
|
+
The stale-while-revalidate caching pattern is a core feature of VSR. It provides several benefits:
|
|
173
|
+
|
|
174
|
+
1. **Improved Response Times**: By returning cached data immediately, users experience fast response times.
|
|
175
|
+
2. **Reduced Load on Upstream Registry**: By caching package data, we reduce the number of requests to the upstream npm registry.
|
|
176
|
+
3. **Background Updates**: By refreshing stale data in the background, cache stays current without impacting user response time.
|
|
177
|
+
|
|
178
|
+
### Implementation Details
|
|
179
|
+
|
|
180
|
+
The caching logic is implemented as follows:
|
|
181
|
+
|
|
182
|
+
1. **Cache Check**: When a request comes in, we check if the data exists in our cache (D1 database).
|
|
183
|
+
2. **Freshness Check**: If the data exists, we check if it's fresh (typically less than 5 minutes old).
|
|
184
|
+
3. **Response Strategy**:
|
|
185
|
+
- If data is fresh: Return it immediately
|
|
186
|
+
- If data is stale: Return it immediately AND queue a background refresh
|
|
187
|
+
- If data doesn't exist: Fetch from upstream, cache it, then return
|
|
188
|
+
|
|
189
|
+
4. **Background Refresh**:
|
|
190
|
+
```javascript
|
|
191
|
+
// Example pattern (simplified)
|
|
192
|
+
if (isStale) {
|
|
193
|
+
c.executionCtx.waitUntil(new Promise(async (resolve) => {
|
|
194
|
+
const bgRefresh = async () => {
|
|
195
|
+
const freshData = await fetchFromUpstream();
|
|
196
|
+
await saveToCache(freshData);
|
|
197
|
+
resolve();
|
|
198
|
+
};
|
|
199
|
+
|
|
200
|
+
setTimeout(bgRefresh, 0);
|
|
201
|
+
}));
|
|
202
|
+
}
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
## Contribution Guidelines
|
|
206
|
+
|
|
207
|
+
1. **Create an issue** first to discuss proposed changes or report bugs
|
|
208
|
+
2. **Fork the repository** and create a feature branch for your changes
|
|
209
|
+
3. **Write tests** for new features or bug fixes
|
|
210
|
+
4. **Ensure code quality** by following the established style and patterns
|
|
211
|
+
5. **Submit a pull request** referencing the original issue
|
|
212
|
+
|
|
213
|
+
## Pull Request Process
|
|
214
|
+
|
|
215
|
+
1. Ensure your code passes all tests and linting
|
|
216
|
+
2. Update documentation if necessary
|
|
217
|
+
3. Add yourself to the contributors list (if not already there)
|
|
218
|
+
4. Submit your PR with a clear description of changes
|
|
219
|
+
5. Address any feedback from code reviews
|
|
220
|
+
|
|
221
|
+
## Code Style
|
|
222
|
+
|
|
223
|
+
- Use modern JavaScript features
|
|
224
|
+
- Follow the existing code structure and naming conventions
|
|
225
|
+
- Document new functions and components with JSDoc comments
|
|
226
|
+
- Maintain comprehensive test coverage
|
|
227
|
+
|
|
228
|
+
Thank you for contributing to VSR!
|
package/LICENSE.md
ADDED
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
# Functional Source License, Version 1.1, MIT Future License
|
|
2
|
+
|
|
3
|
+
## Abbreviation
|
|
4
|
+
|
|
5
|
+
FSL-1.1-MIT
|
|
6
|
+
|
|
7
|
+
## Notice
|
|
8
|
+
|
|
9
|
+
Copyright 2025 vlt technology inc.
|
|
10
|
+
|
|
11
|
+
## Terms and Conditions
|
|
12
|
+
|
|
13
|
+
### Licensor ("We")
|
|
14
|
+
|
|
15
|
+
The party offering the Software under these Terms and Conditions.
|
|
16
|
+
|
|
17
|
+
### The Software
|
|
18
|
+
|
|
19
|
+
The "Software" is each version of the software that we make available under
|
|
20
|
+
these Terms and Conditions, as indicated by our inclusion of these Terms and
|
|
21
|
+
Conditions with the Software.
|
|
22
|
+
|
|
23
|
+
### License Grant
|
|
24
|
+
|
|
25
|
+
Subject to your compliance with this License Grant and the Patents,
|
|
26
|
+
Redistribution and Trademark clauses below, we hereby grant you the right to
|
|
27
|
+
use, copy, modify, create derivative works, publicly perform, publicly display
|
|
28
|
+
and redistribute the Software for any Permitted Purpose identified below.
|
|
29
|
+
|
|
30
|
+
### Permitted Purpose
|
|
31
|
+
|
|
32
|
+
A Permitted Purpose is any purpose other than a Competing Use. A Competing Use
|
|
33
|
+
means making the Software available to others in a commercial product or
|
|
34
|
+
service that:
|
|
35
|
+
|
|
36
|
+
1. substitutes for the Software;
|
|
37
|
+
|
|
38
|
+
2. substitutes for any other product or service we offer using the Software
|
|
39
|
+
that exists as of the date we make the Software available; or
|
|
40
|
+
|
|
41
|
+
3. offers the same or substantially similar functionality as the Software.
|
|
42
|
+
|
|
43
|
+
Permitted Purposes specifically include using the Software:
|
|
44
|
+
|
|
45
|
+
1. for your internal use and access;
|
|
46
|
+
|
|
47
|
+
2. for non-commercial education;
|
|
48
|
+
|
|
49
|
+
3. for non-commercial research; and
|
|
50
|
+
|
|
51
|
+
4. in connection with professional services that you provide to a licensee
|
|
52
|
+
using the Software in accordance with these Terms and Conditions.
|
|
53
|
+
|
|
54
|
+
### Patents
|
|
55
|
+
|
|
56
|
+
To the extent your use for a Permitted Purpose would necessarily infringe our
|
|
57
|
+
patents, the license grant above includes a license under our patents. If you
|
|
58
|
+
make a claim against any party that the Software infringes or contributes to
|
|
59
|
+
the infringement of any patent, then your patent license to the Software ends
|
|
60
|
+
immediately.
|
|
61
|
+
|
|
62
|
+
### Redistribution
|
|
63
|
+
|
|
64
|
+
The Terms and Conditions apply to all copies, modifications and derivatives of
|
|
65
|
+
the Software.
|
|
66
|
+
|
|
67
|
+
If you redistribute any copies, modifications or derivatives of the Software,
|
|
68
|
+
you must include a copy of or a link to these Terms and Conditions and not
|
|
69
|
+
remove any copyright notices provided in or with the Software.
|
|
70
|
+
|
|
71
|
+
### Disclaimer
|
|
72
|
+
|
|
73
|
+
THE SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR
|
|
74
|
+
IMPLIED, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR
|
|
75
|
+
PURPOSE, MERCHANTABILITY, TITLE OR NON-INFRINGEMENT.
|
|
76
|
+
|
|
77
|
+
IN NO EVENT WILL WE HAVE ANY LIABILITY TO YOU ARISING OUT OF OR RELATED TO THE
|
|
78
|
+
SOFTWARE, INCLUDING INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES,
|
|
79
|
+
EVEN IF WE HAVE BEEN INFORMED OF THEIR POSSIBILITY IN ADVANCE.
|
|
80
|
+
|
|
81
|
+
### Trademarks
|
|
82
|
+
|
|
83
|
+
Except for displaying the License Details and identifying us as the origin of
|
|
84
|
+
the Software, you have no right under these Terms and Conditions to use our
|
|
85
|
+
trademarks, trade names, service marks or product names.
|
|
86
|
+
|
|
87
|
+
## Grant of Future License
|
|
88
|
+
|
|
89
|
+
We hereby irrevocably grant you an additional license to use the Software under
|
|
90
|
+
the MIT license that is effective on the second anniversary of the date we make
|
|
91
|
+
the Software available. On or after that date, you may use the Software under
|
|
92
|
+
the MIT license, in which case the following will apply:
|
|
93
|
+
|
|
94
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy of
|
|
95
|
+
this software and associated documentation files (the "Software"), to deal in
|
|
96
|
+
the Software without restriction, including without limitation the rights to
|
|
97
|
+
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
|
|
98
|
+
of the Software, and to permit persons to whom the Software is furnished to do
|
|
99
|
+
so, subject to the following conditions:
|
|
100
|
+
|
|
101
|
+
The above copyright notice and this permission notice shall be included in all
|
|
102
|
+
copies or substantial portions of the Software.
|
|
103
|
+
|
|
104
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
105
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
106
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
107
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
108
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
109
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
110
|
+
SOFTWARE.
|