dep-oracle 1.2.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +33 -8
- package/dist/action/index.js +398 -16
- package/dist/badge-5Z3WAD2B.js +89 -0
- package/dist/badge-5Z3WAD2B.js.map +1 -0
- package/dist/chunk-32B3QIPY.js +1505 -0
- package/dist/chunk-32B3QIPY.js.map +1 -0
- package/dist/chunk-7DST6SNA.js +258 -0
- package/dist/chunk-7DST6SNA.js.map +1 -0
- package/dist/{chunk-TXSNFX3N.js → chunk-DLWG22RC.js} +403 -17
- package/dist/chunk-DLWG22RC.js.map +1 -0
- package/dist/chunk-HX6MGNBD.js +271 -0
- package/dist/chunk-HX6MGNBD.js.map +1 -0
- package/dist/chunk-IVXGOPRU.js +145 -0
- package/dist/chunk-IVXGOPRU.js.map +1 -0
- package/dist/chunk-SP3VYPXX.js +218 -0
- package/dist/chunk-SP3VYPXX.js.map +1 -0
- package/dist/chunk-T5EVLWZM.js +4234 -0
- package/dist/chunk-T5EVLWZM.js.map +1 -0
- package/dist/chunk-UMB5MJHL.js +239 -0
- package/dist/chunk-UMB5MJHL.js.map +1 -0
- package/dist/cli/index.js +163 -6499
- package/dist/cli/index.js.map +1 -1
- package/dist/index.js +9 -84
- package/dist/index.js.map +1 -1
- package/dist/mcp/server.js +33 -12
- package/dist/mcp/server.js.map +1 -1
- package/dist/npm-UB54H37N.js +9 -0
- package/dist/npm-UB54H37N.js.map +1 -0
- package/dist/orchestrator-VOOYKDPT.js +8 -0
- package/dist/orchestrator-VOOYKDPT.js.map +1 -0
- package/dist/python-U4G2GK4J.js +9 -0
- package/dist/python-U4G2GK4J.js.map +1 -0
- package/dist/server-WONIBSG4.js +640 -0
- package/dist/server-WONIBSG4.js.map +1 -0
- package/dist/store-Z5UANEBB.js +8 -0
- package/dist/store-Z5UANEBB.js.map +1 -0
- package/dist/trust-score-YXYDFVPZ.js +8 -0
- package/dist/trust-score-YXYDFVPZ.js.map +1 -0
- package/package.json +1 -1
- package/server.json +2 -2
- package/dist/chunk-TXSNFX3N.js.map +0 -1
package/README.md
CHANGED
|
@@ -34,8 +34,8 @@
|
|
|
34
34
|
|
|
35
35
|
## Why?
|
|
36
36
|
|
|
37
|
-
- **
|
|
38
|
-
- The average project
|
|
37
|
+
- Supply chain attacks increased **742% since 2019** ([Sonatype 2024 Report](https://www.sonatype.com/state-of-the-software-supply-chain/introduction))
|
|
38
|
+
- The average npm project pulls in **hundreds of transitive dependencies** — any one could be compromised
|
|
39
39
|
- `npm audit` only catches **known** CVEs — dep-oracle **predicts** future risks
|
|
40
40
|
- You audit your code. But do you audit your **trust**?
|
|
41
41
|
|
|
@@ -162,6 +162,26 @@ Packages that patch vulnerabilities quickly (within 7 days) receive a **+10 bonu
|
|
|
162
162
|
|
|
163
163
|
If an API is unreachable (GitHub down, no internet, rate limited), dep-oracle doesn't crash. The missing metric weight is redistributed across available metrics. If 3+ metrics are unavailable, a reliability warning is shown.
|
|
164
164
|
|
|
165
|
+
### Blast Radius Methodology
|
|
166
|
+
|
|
167
|
+
The blast radius metric counts how many of your source files directly import a given dependency:
|
|
168
|
+
|
|
169
|
+
1. Recursively collects all `.js`, `.ts`, `.jsx`, `.tsx`, `.mjs`, `.mts`, `.cjs`, `.cts` files
|
|
170
|
+
2. Skips `node_modules`, `.git`, `dist`, `build`, `coverage`, and other build directories
|
|
171
|
+
3. Searches each file for `import ... from 'pkg'`, `require('pkg')`, and dynamic `import('pkg')` patterns
|
|
172
|
+
4. Reports the count, file paths, and percentage of codebase affected
|
|
173
|
+
|
|
174
|
+
**Current limitations:**
|
|
175
|
+
- Only scans JavaScript/TypeScript import patterns
|
|
176
|
+
- Python `import` statements are not yet analyzed (blast radius returns 0 for Python-only projects)
|
|
177
|
+
- Does not trace re-exports or barrel files — counts direct imports only
|
|
178
|
+
|
|
179
|
+
### Weight Rationale
|
|
180
|
+
|
|
181
|
+
Weights are based on the principle that **security vulnerabilities and maintainer abandonment** are the strongest predictors of supply chain risk, followed by development activity signals. Weights are fully configurable via `.dep-oraclerc.json` — enterprise teams can adjust to match their specific risk tolerance.
|
|
182
|
+
|
|
183
|
+
When data is unavailable for a metric, the score is pulled toward the midpoint (50) proportionally to the fraction of missing weight, preventing artificial inflation from missing data.
|
|
184
|
+
|
|
165
185
|
## Typosquat Detection
|
|
166
186
|
|
|
167
187
|
dep-oracle uses a multi-layer approach to catch typosquatting:
|
|
@@ -238,10 +258,13 @@ jobs:
|
|
|
238
258
|
runs-on: ubuntu-latest
|
|
239
259
|
steps:
|
|
240
260
|
- uses: actions/checkout@v4
|
|
241
|
-
- uses:
|
|
261
|
+
- uses: actions/setup-node@v4
|
|
242
262
|
with:
|
|
243
|
-
|
|
244
|
-
|
|
263
|
+
node-version: '20'
|
|
264
|
+
- name: Run dep-oracle
|
|
265
|
+
run: npx dep-oracle scan --format sarif --min-score 60
|
|
266
|
+
env:
|
|
267
|
+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
245
268
|
```
|
|
246
269
|
|
|
247
270
|
## Configuration
|
|
@@ -296,17 +319,19 @@ Or add to `package.json`:
|
|
|
296
319
|
| Feature | npm audit | Dependabot | Socket.dev | Snyk | **dep-oracle** |
|
|
297
320
|
|---------|-----------|------------|------------|------|----------------|
|
|
298
321
|
| Known CVE scan | Yes | Yes | Yes | Yes | **Yes** |
|
|
299
|
-
| Predictive risk | No | No | Partial |
|
|
322
|
+
| Predictive risk | No | No | Partial | Partial | **Yes** |
|
|
300
323
|
| Trust Score (0-100) | No | No | No | No | **Yes** |
|
|
301
324
|
| Zombie detection | No | No | No | No | **Yes** |
|
|
302
|
-
| Blast radius | No |
|
|
325
|
+
| Blast radius | No | Partial | No | No | **Yes** |
|
|
303
326
|
| Typosquat detection | No | No | Yes | No | **Yes** |
|
|
304
327
|
| Trend prediction | No | No | No | No | **Yes** |
|
|
305
328
|
| Migration advisor | No | Partial | No | Partial | **Yes (131 pkgs)** |
|
|
306
|
-
| MCP integration | No | No |
|
|
329
|
+
| MCP integration | No | No | No | No | **Yes** |
|
|
307
330
|
| Zero install (npx) | Yes | No | No | No | **Yes** |
|
|
308
331
|
| Free & open source | Yes | Yes | Freemium | Freemium | **Yes** |
|
|
309
332
|
|
|
333
|
+
> **Note:** dep-oracle is not a replacement for Snyk or Socket.dev in enterprise environments. They have dedicated security research teams and CVE databases. dep-oracle focuses on **predictive signals** (trust scores, maintenance health, funding, zombie detection) that complement existing tools.
|
|
334
|
+
|
|
310
335
|
## Programmatic API
|
|
311
336
|
|
|
312
337
|
```typescript
|
package/dist/action/index.js
CHANGED
|
@@ -5608,6 +5608,87 @@ function createLogger(label) {
|
|
|
5608
5608
|
};
|
|
5609
5609
|
}
|
|
5610
5610
|
|
|
5611
|
+
// src/utils/rate-limiter.ts
|
|
5612
|
+
var RateLimiter = class {
|
|
5613
|
+
tokens;
|
|
5614
|
+
maxTokens;
|
|
5615
|
+
refillIntervalMs;
|
|
5616
|
+
lastRefill;
|
|
5617
|
+
waitQueue = [];
|
|
5618
|
+
/**
|
|
5619
|
+
* @param maxRequests Maximum number of requests allowed in the window
|
|
5620
|
+
* @param windowMs Window duration in milliseconds
|
|
5621
|
+
*/
|
|
5622
|
+
constructor(maxRequests, windowMs) {
|
|
5623
|
+
this.maxTokens = maxRequests;
|
|
5624
|
+
this.tokens = maxRequests;
|
|
5625
|
+
this.refillIntervalMs = windowMs;
|
|
5626
|
+
this.lastRefill = Date.now();
|
|
5627
|
+
}
|
|
5628
|
+
/**
|
|
5629
|
+
* Acquire a token. Resolves immediately when tokens are available,
|
|
5630
|
+
* otherwise waits until the bucket is refilled.
|
|
5631
|
+
*/
|
|
5632
|
+
async acquire() {
|
|
5633
|
+
this.refill();
|
|
5634
|
+
if (this.tokens > 0) {
|
|
5635
|
+
this.tokens--;
|
|
5636
|
+
return;
|
|
5637
|
+
}
|
|
5638
|
+
return new Promise((resolve2) => {
|
|
5639
|
+
this.waitQueue.push(resolve2);
|
|
5640
|
+
this.scheduleRefill();
|
|
5641
|
+
});
|
|
5642
|
+
}
|
|
5643
|
+
/**
|
|
5644
|
+
* Return the number of tokens currently available (without waiting).
|
|
5645
|
+
*/
|
|
5646
|
+
get remaining() {
|
|
5647
|
+
this.refill();
|
|
5648
|
+
return this.tokens;
|
|
5649
|
+
}
|
|
5650
|
+
/**
|
|
5651
|
+
* Return the number of milliseconds until the next refill.
|
|
5652
|
+
*/
|
|
5653
|
+
get msUntilRefill() {
|
|
5654
|
+
const elapsed = Date.now() - this.lastRefill;
|
|
5655
|
+
return Math.max(0, this.refillIntervalMs - elapsed);
|
|
5656
|
+
}
|
|
5657
|
+
// -----------------------------------------------------------------------
|
|
5658
|
+
// Internal
|
|
5659
|
+
// -----------------------------------------------------------------------
|
|
5660
|
+
refill() {
|
|
5661
|
+
const now = Date.now();
|
|
5662
|
+
const elapsed = now - this.lastRefill;
|
|
5663
|
+
if (elapsed >= this.refillIntervalMs) {
|
|
5664
|
+
const periods = Math.floor(elapsed / this.refillIntervalMs);
|
|
5665
|
+
this.tokens = Math.min(this.maxTokens, this.tokens + periods * this.maxTokens);
|
|
5666
|
+
this.lastRefill = now - elapsed % this.refillIntervalMs;
|
|
5667
|
+
this.drainWaitQueue();
|
|
5668
|
+
}
|
|
5669
|
+
}
|
|
5670
|
+
scheduleRefill() {
|
|
5671
|
+
const delay = this.msUntilRefill;
|
|
5672
|
+
if (delay <= 0) {
|
|
5673
|
+
this.refill();
|
|
5674
|
+
return;
|
|
5675
|
+
}
|
|
5676
|
+
setTimeout(() => {
|
|
5677
|
+
this.refill();
|
|
5678
|
+
}, delay);
|
|
5679
|
+
}
|
|
5680
|
+
drainWaitQueue() {
|
|
5681
|
+
while (this.waitQueue.length > 0 && this.tokens > 0) {
|
|
5682
|
+
this.tokens--;
|
|
5683
|
+
const resolve2 = this.waitQueue.shift();
|
|
5684
|
+
resolve2?.();
|
|
5685
|
+
}
|
|
5686
|
+
}
|
|
5687
|
+
};
|
|
5688
|
+
var githubRateLimiter = new RateLimiter(5e3, 36e5);
|
|
5689
|
+
var npmRateLimiter = new RateLimiter(300, 6e4);
|
|
5690
|
+
var pypiRateLimiter = new RateLimiter(100, 6e4);
|
|
5691
|
+
|
|
5611
5692
|
// src/collectors/base.ts
|
|
5612
5693
|
var BaseCollector = class {
|
|
5613
5694
|
cache;
|
|
@@ -5720,6 +5801,7 @@ var RegistryCollector = class extends BaseCollector {
|
|
|
5720
5801
|
async fetchPackument(packageName) {
|
|
5721
5802
|
const url = `https://registry.npmjs.org/${encodeURIComponent(packageName)}`;
|
|
5722
5803
|
logger.debug(`Fetching packument: ${url}`);
|
|
5804
|
+
await npmRateLimiter.acquire();
|
|
5723
5805
|
const res = await fetch(url, {
|
|
5724
5806
|
headers: { Accept: "application/json" }
|
|
5725
5807
|
});
|
|
@@ -5732,6 +5814,7 @@ var RegistryCollector = class extends BaseCollector {
|
|
|
5732
5814
|
const url = `https://api.npmjs.org/downloads/point/last-week/${encodeURIComponent(packageName)}`;
|
|
5733
5815
|
logger.debug(`Fetching weekly downloads: ${url}`);
|
|
5734
5816
|
try {
|
|
5817
|
+
await npmRateLimiter.acquire();
|
|
5735
5818
|
const res = await fetch(url, {
|
|
5736
5819
|
headers: { Accept: "application/json" }
|
|
5737
5820
|
});
|
|
@@ -5759,6 +5842,166 @@ var RegistryCollector = class extends BaseCollector {
|
|
|
5759
5842
|
}
|
|
5760
5843
|
};
|
|
5761
5844
|
|
|
5845
|
+
// src/collectors/pypi-registry.ts
|
|
5846
|
+
var PyPIRegistryCollector = class extends BaseCollector {
|
|
5847
|
+
name = "pypi-registry";
|
|
5848
|
+
constructor(cache2) {
|
|
5849
|
+
super(cache2);
|
|
5850
|
+
}
|
|
5851
|
+
async collect(packageName, version) {
|
|
5852
|
+
const cached = await this.getCached(packageName, version);
|
|
5853
|
+
if (cached) return cached;
|
|
5854
|
+
try {
|
|
5855
|
+
const [metadataResult, downloadsResult] = await Promise.allSettled([
|
|
5856
|
+
this.fetchMetadata(packageName),
|
|
5857
|
+
this.fetchWeeklyDownloads(packageName)
|
|
5858
|
+
]);
|
|
5859
|
+
const metadata = metadataResult.status === "fulfilled" ? metadataResult.value : null;
|
|
5860
|
+
const downloads = downloadsResult.status === "fulfilled" ? downloadsResult.value : 0;
|
|
5861
|
+
if (!metadata) {
|
|
5862
|
+
throw new Error(`PyPI registry returned no data for ${packageName}`);
|
|
5863
|
+
}
|
|
5864
|
+
const versionCount = metadata.releases ? Object.keys(metadata.releases).length : 0;
|
|
5865
|
+
const lastPublishDate = this.findLastPublishDate(metadata.releases);
|
|
5866
|
+
const deprecated = this.checkYanked(metadata.releases, version);
|
|
5867
|
+
const data = {
|
|
5868
|
+
packageName,
|
|
5869
|
+
version: version === "latest" ? metadata.info.version : version,
|
|
5870
|
+
description: metadata.info.summary ?? null,
|
|
5871
|
+
lastPublishDate,
|
|
5872
|
+
versionCount,
|
|
5873
|
+
deprecated,
|
|
5874
|
+
weeklyDownloads: downloads,
|
|
5875
|
+
license: metadata.info.license ?? null,
|
|
5876
|
+
repositoryUrl: this.extractRepoUrl(metadata.info)
|
|
5877
|
+
};
|
|
5878
|
+
await this.setCache(packageName, version, data);
|
|
5879
|
+
return {
|
|
5880
|
+
status: "success",
|
|
5881
|
+
data,
|
|
5882
|
+
collectedAt: (/* @__PURE__ */ new Date()).toISOString()
|
|
5883
|
+
};
|
|
5884
|
+
} catch (err) {
|
|
5885
|
+
const message = err instanceof Error ? err.message : String(err);
|
|
5886
|
+
logger.error(
|
|
5887
|
+
`PyPIRegistryCollector failed for ${packageName}@${version}: ${message}`
|
|
5888
|
+
);
|
|
5889
|
+
return {
|
|
5890
|
+
status: "error",
|
|
5891
|
+
data: null,
|
|
5892
|
+
error: message,
|
|
5893
|
+
collectedAt: (/* @__PURE__ */ new Date()).toISOString()
|
|
5894
|
+
};
|
|
5895
|
+
}
|
|
5896
|
+
}
|
|
5897
|
+
// ---------------------------------------------------------------------------
|
|
5898
|
+
// Private helpers
|
|
5899
|
+
// ---------------------------------------------------------------------------
|
|
5900
|
+
async fetchMetadata(packageName) {
|
|
5901
|
+
await pypiRateLimiter.acquire();
|
|
5902
|
+
const url = `https://pypi.org/pypi/${encodeURIComponent(packageName)}/json`;
|
|
5903
|
+
logger.debug(`Fetching PyPI metadata: ${url}`);
|
|
5904
|
+
const res = await fetch(url, {
|
|
5905
|
+
headers: { Accept: "application/json" }
|
|
5906
|
+
});
|
|
5907
|
+
if (!res.ok) {
|
|
5908
|
+
if (res.status === 404) return null;
|
|
5909
|
+
throw new Error(`PyPI registry returned ${res.status} for ${packageName}`);
|
|
5910
|
+
}
|
|
5911
|
+
return await res.json();
|
|
5912
|
+
}
|
|
5913
|
+
async fetchWeeklyDownloads(packageName) {
|
|
5914
|
+
await pypiRateLimiter.acquire();
|
|
5915
|
+
const url = `https://pypistats.org/api/packages/${encodeURIComponent(packageName)}/recent`;
|
|
5916
|
+
logger.debug(`Fetching PyPI weekly downloads: ${url}`);
|
|
5917
|
+
try {
|
|
5918
|
+
const res = await fetch(url, {
|
|
5919
|
+
headers: { Accept: "application/json" }
|
|
5920
|
+
});
|
|
5921
|
+
if (!res.ok) {
|
|
5922
|
+
logger.warn(
|
|
5923
|
+
`PyPI stats API returned ${res.status} for ${packageName}`
|
|
5924
|
+
);
|
|
5925
|
+
return 0;
|
|
5926
|
+
}
|
|
5927
|
+
const body = await res.json();
|
|
5928
|
+
return body.data?.last_week ?? 0;
|
|
5929
|
+
} catch {
|
|
5930
|
+
logger.warn(`Could not fetch PyPI download stats for ${packageName}`);
|
|
5931
|
+
return 0;
|
|
5932
|
+
}
|
|
5933
|
+
}
|
|
5934
|
+
/**
|
|
5935
|
+
* Find the most recent upload date across all releases.
|
|
5936
|
+
*/
|
|
5937
|
+
findLastPublishDate(releases) {
|
|
5938
|
+
if (!releases) return null;
|
|
5939
|
+
const dates = [];
|
|
5940
|
+
for (const files of Object.values(releases)) {
|
|
5941
|
+
for (const file of files) {
|
|
5942
|
+
const dateStr = file.upload_time_iso_8601 ?? file.upload_time;
|
|
5943
|
+
if (dateStr) {
|
|
5944
|
+
const ts = new Date(dateStr).getTime();
|
|
5945
|
+
if (!isNaN(ts)) dates.push(ts);
|
|
5946
|
+
}
|
|
5947
|
+
}
|
|
5948
|
+
}
|
|
5949
|
+
if (dates.length === 0) return null;
|
|
5950
|
+
dates.sort((a, b) => b - a);
|
|
5951
|
+
return new Date(dates[0]).toISOString();
|
|
5952
|
+
}
|
|
5953
|
+
/**
|
|
5954
|
+
* Check if the requested version is yanked (PyPI's deprecation mechanism).
|
|
5955
|
+
* Returns the yank reason string, or null if not yanked.
|
|
5956
|
+
*/
|
|
5957
|
+
checkYanked(releases, version) {
|
|
5958
|
+
if (!releases || version === "latest") return null;
|
|
5959
|
+
const files = releases[version];
|
|
5960
|
+
if (!files || files.length === 0) return null;
|
|
5961
|
+
const yankedFile = files.find((f) => f.yanked);
|
|
5962
|
+
if (yankedFile) {
|
|
5963
|
+
return yankedFile.yanked_reason || "This version has been yanked";
|
|
5964
|
+
}
|
|
5965
|
+
return null;
|
|
5966
|
+
}
|
|
5967
|
+
/**
|
|
5968
|
+
* Extract a normalised repository URL from PyPI project_urls or home_page.
|
|
5969
|
+
*/
|
|
5970
|
+
extractRepoUrl(info) {
|
|
5971
|
+
const projectUrls = info.project_urls ?? {};
|
|
5972
|
+
const repoKeys = [
|
|
5973
|
+
"Source",
|
|
5974
|
+
"Source Code",
|
|
5975
|
+
"Repository",
|
|
5976
|
+
"GitHub",
|
|
5977
|
+
"Code",
|
|
5978
|
+
"Homepage",
|
|
5979
|
+
"source",
|
|
5980
|
+
"source_code",
|
|
5981
|
+
"repository",
|
|
5982
|
+
"github",
|
|
5983
|
+
"code",
|
|
5984
|
+
"homepage"
|
|
5985
|
+
];
|
|
5986
|
+
for (const key of repoKeys) {
|
|
5987
|
+
const url = projectUrls[key];
|
|
5988
|
+
if (url && (url.includes("github.com") || url.includes("gitlab.com") || url.includes("bitbucket.org"))) {
|
|
5989
|
+
return url.replace(/\.git$/, "");
|
|
5990
|
+
}
|
|
5991
|
+
}
|
|
5992
|
+
for (const url of Object.values(projectUrls)) {
|
|
5993
|
+
if (url && (url.includes("github.com") || url.includes("gitlab.com") || url.includes("bitbucket.org"))) {
|
|
5994
|
+
return url.replace(/\.git$/, "");
|
|
5995
|
+
}
|
|
5996
|
+
}
|
|
5997
|
+
const homePage = info.home_page;
|
|
5998
|
+
if (homePage && (homePage.includes("github.com") || homePage.includes("gitlab.com"))) {
|
|
5999
|
+
return homePage.replace(/\.git$/, "");
|
|
6000
|
+
}
|
|
6001
|
+
return null;
|
|
6002
|
+
}
|
|
6003
|
+
};
|
|
6004
|
+
|
|
5762
6005
|
// src/collectors/github.ts
|
|
5763
6006
|
var GitHubCollector = class extends BaseCollector {
|
|
5764
6007
|
name = "github";
|
|
@@ -5830,6 +6073,7 @@ var GitHubCollector = class extends BaseCollector {
|
|
|
5830
6073
|
async resolveRepoSlug(packageName) {
|
|
5831
6074
|
try {
|
|
5832
6075
|
const url = `https://registry.npmjs.org/${encodeURIComponent(packageName)}`;
|
|
6076
|
+
await npmRateLimiter.acquire();
|
|
5833
6077
|
const res = await fetch(url, {
|
|
5834
6078
|
headers: { Accept: "application/json" }
|
|
5835
6079
|
});
|
|
@@ -5872,6 +6116,7 @@ var GitHubCollector = class extends BaseCollector {
|
|
|
5872
6116
|
async fetchRepoInfo(owner, repo) {
|
|
5873
6117
|
const url = `https://api.github.com/repos/${owner}/${repo}`;
|
|
5874
6118
|
logger.debug(`GitHub: fetching repo info ${url}`);
|
|
6119
|
+
await githubRateLimiter.acquire();
|
|
5875
6120
|
const res = await fetch(url, { headers: this.headers() });
|
|
5876
6121
|
if (!res.ok) {
|
|
5877
6122
|
throw new Error(`GitHub API ${res.status} for ${url}`);
|
|
@@ -5886,6 +6131,7 @@ var GitHubCollector = class extends BaseCollector {
|
|
|
5886
6131
|
const url = `https://api.github.com/repos/${owner}/${repo}/contributors?per_page=1&anon=true`;
|
|
5887
6132
|
logger.debug(`GitHub: fetching contributor count ${url}`);
|
|
5888
6133
|
try {
|
|
6134
|
+
await githubRateLimiter.acquire();
|
|
5889
6135
|
const res = await fetch(url, { headers: this.headers() });
|
|
5890
6136
|
if (!res.ok) return 0;
|
|
5891
6137
|
const count = this.extractLastPage(res.headers.get("link"));
|
|
@@ -5906,6 +6152,7 @@ var GitHubCollector = class extends BaseCollector {
|
|
|
5906
6152
|
const url = `https://api.github.com/repos/${owner}/${repo}/commits?since=${since}&per_page=1`;
|
|
5907
6153
|
logger.debug(`GitHub: fetching recent commit count ${url}`);
|
|
5908
6154
|
try {
|
|
6155
|
+
await githubRateLimiter.acquire();
|
|
5909
6156
|
const res = await fetch(url, { headers: this.headers() });
|
|
5910
6157
|
if (!res.ok) return 0;
|
|
5911
6158
|
const count = this.extractLastPage(res.headers.get("link"));
|
|
@@ -5921,6 +6168,7 @@ var GitHubCollector = class extends BaseCollector {
|
|
|
5921
6168
|
const url = `https://api.github.com/repos/${owner}/${repo}/commits?per_page=1`;
|
|
5922
6169
|
logger.debug(`GitHub: fetching latest commit ${url}`);
|
|
5923
6170
|
try {
|
|
6171
|
+
await githubRateLimiter.acquire();
|
|
5924
6172
|
const res = await fetch(url, { headers: this.headers() });
|
|
5925
6173
|
if (!res.ok) return null;
|
|
5926
6174
|
const body = await res.json();
|
|
@@ -5937,6 +6185,7 @@ var GitHubCollector = class extends BaseCollector {
|
|
|
5937
6185
|
const url = `https://api.github.com/repos/${owner}/${repo}/contents/.github/FUNDING.yml`;
|
|
5938
6186
|
logger.debug(`GitHub: checking FUNDING.yml ${url}`);
|
|
5939
6187
|
try {
|
|
6188
|
+
await githubRateLimiter.acquire();
|
|
5940
6189
|
const res = await fetch(url, { headers: this.headers() });
|
|
5941
6190
|
return res.ok;
|
|
5942
6191
|
} catch {
|
|
@@ -5968,11 +6217,11 @@ var SecurityCollector = class extends BaseCollector {
|
|
|
5968
6217
|
constructor(cache2) {
|
|
5969
6218
|
super(cache2);
|
|
5970
6219
|
}
|
|
5971
|
-
async collect(packageName, version) {
|
|
6220
|
+
async collect(packageName, version, ecosystem) {
|
|
5972
6221
|
const cached = await this.getCached(packageName, version);
|
|
5973
6222
|
if (cached) return cached;
|
|
5974
6223
|
try {
|
|
5975
|
-
const vulns = await this.queryOsv(packageName);
|
|
6224
|
+
const vulns = await this.queryOsv(packageName, ecosystem);
|
|
5976
6225
|
const totalVulnerabilities = vulns.length;
|
|
5977
6226
|
const severityCounts = {
|
|
5978
6227
|
critical: 0,
|
|
@@ -6022,16 +6271,16 @@ var SecurityCollector = class extends BaseCollector {
|
|
|
6022
6271
|
// ---------------------------------------------------------------------------
|
|
6023
6272
|
// OSV API
|
|
6024
6273
|
// ---------------------------------------------------------------------------
|
|
6025
|
-
async queryOsv(packageName) {
|
|
6274
|
+
async queryOsv(packageName, ecosystem = "npm") {
|
|
6026
6275
|
const url = "https://api.osv.dev/v1/query";
|
|
6027
|
-
logger.debug(`OSV: querying vulnerabilities for ${packageName}`);
|
|
6276
|
+
logger.debug(`OSV: querying vulnerabilities for ${packageName} (ecosystem=${ecosystem})`);
|
|
6028
6277
|
const res = await fetch(url, {
|
|
6029
6278
|
method: "POST",
|
|
6030
6279
|
headers: { "Content-Type": "application/json" },
|
|
6031
6280
|
body: JSON.stringify({
|
|
6032
6281
|
package: {
|
|
6033
6282
|
name: packageName,
|
|
6034
|
-
ecosystem
|
|
6283
|
+
ecosystem
|
|
6035
6284
|
}
|
|
6036
6285
|
})
|
|
6037
6286
|
});
|
|
@@ -6195,6 +6444,7 @@ var FundingCollector = class extends BaseCollector {
|
|
|
6195
6444
|
async fetchNpmFunding(packageName) {
|
|
6196
6445
|
try {
|
|
6197
6446
|
const url = `https://registry.npmjs.org/${encodeURIComponent(packageName)}`;
|
|
6447
|
+
await npmRateLimiter.acquire();
|
|
6198
6448
|
const res = await fetch(url, {
|
|
6199
6449
|
headers: { Accept: "application/json" }
|
|
6200
6450
|
});
|
|
@@ -6229,6 +6479,7 @@ var FundingCollector = class extends BaseCollector {
|
|
|
6229
6479
|
if (this.githubToken) {
|
|
6230
6480
|
headers.Authorization = `Bearer ${this.githubToken}`;
|
|
6231
6481
|
}
|
|
6482
|
+
await githubRateLimiter.acquire();
|
|
6232
6483
|
const res = await fetch(url, { headers });
|
|
6233
6484
|
if (!res.ok) return null;
|
|
6234
6485
|
return await res.text();
|
|
@@ -6259,6 +6510,7 @@ var FundingCollector = class extends BaseCollector {
|
|
|
6259
6510
|
async resolveRepoSlug(packageName) {
|
|
6260
6511
|
try {
|
|
6261
6512
|
const url = `https://registry.npmjs.org/${encodeURIComponent(packageName)}`;
|
|
6513
|
+
await npmRateLimiter.acquire();
|
|
6262
6514
|
const res = await fetch(url, {
|
|
6263
6515
|
headers: { Accept: "application/json" }
|
|
6264
6516
|
});
|
|
@@ -6377,6 +6629,7 @@ var PopularityCollector = class extends BaseCollector {
|
|
|
6377
6629
|
const url = `https://api.npmjs.org/downloads/point/${period}/${encodeURIComponent(packageName)}`;
|
|
6378
6630
|
logger.debug(`Popularity: fetching ${period} downloads: ${url}`);
|
|
6379
6631
|
try {
|
|
6632
|
+
await npmRateLimiter.acquire();
|
|
6380
6633
|
const res = await fetch(url, {
|
|
6381
6634
|
headers: { Accept: "application/json" }
|
|
6382
6635
|
});
|
|
@@ -6404,6 +6657,7 @@ var PopularityCollector = class extends BaseCollector {
|
|
|
6404
6657
|
const url = `https://registry.npmjs.org/-/v1/search?text=${encodeURIComponent(packageName)}&size=1`;
|
|
6405
6658
|
logger.debug(`Popularity: fetching dependent count: ${url}`);
|
|
6406
6659
|
try {
|
|
6660
|
+
await npmRateLimiter.acquire();
|
|
6407
6661
|
const res = await fetch(url, {
|
|
6408
6662
|
headers: { Accept: "application/json" }
|
|
6409
6663
|
});
|
|
@@ -6429,6 +6683,7 @@ var PopularityCollector = class extends BaseCollector {
|
|
|
6429
6683
|
async fetchDependentCountFromRegistry(packageName) {
|
|
6430
6684
|
try {
|
|
6431
6685
|
const url = `https://www.npmjs.com/package/${encodeURIComponent(packageName)}`;
|
|
6686
|
+
await npmRateLimiter.acquire();
|
|
6432
6687
|
const res = await fetch(url, {
|
|
6433
6688
|
headers: {
|
|
6434
6689
|
Accept: "text/html",
|
|
@@ -6596,6 +6851,7 @@ var LicenseCollector = class extends BaseCollector {
|
|
|
6596
6851
|
async fetchLicense(packageName, version) {
|
|
6597
6852
|
const url = `https://registry.npmjs.org/${encodeURIComponent(packageName)}`;
|
|
6598
6853
|
logger.debug(`License: fetching packument ${url}`);
|
|
6854
|
+
await npmRateLimiter.acquire();
|
|
6599
6855
|
const res = await fetch(url, {
|
|
6600
6856
|
headers: { Accept: "application/json" }
|
|
6601
6857
|
});
|
|
@@ -6655,6 +6911,7 @@ var CollectorOrchestrator = class {
|
|
|
6655
6911
|
cache;
|
|
6656
6912
|
options;
|
|
6657
6913
|
registryCollector;
|
|
6914
|
+
pypiRegistryCollector;
|
|
6658
6915
|
githubCollector;
|
|
6659
6916
|
securityCollector;
|
|
6660
6917
|
fundingCollector;
|
|
@@ -6668,6 +6925,7 @@ var CollectorOrchestrator = class {
|
|
|
6668
6925
|
concurrency: options.concurrency ?? 10
|
|
6669
6926
|
};
|
|
6670
6927
|
this.registryCollector = new RegistryCollector(this.cache);
|
|
6928
|
+
this.pypiRegistryCollector = new PyPIRegistryCollector(this.cache);
|
|
6671
6929
|
this.githubCollector = new GitHubCollector(this.cache, this.options.githubToken || void 0);
|
|
6672
6930
|
this.securityCollector = new SecurityCollector(this.cache);
|
|
6673
6931
|
this.fundingCollector = new FundingCollector(this.cache, this.options.githubToken || void 0);
|
|
@@ -6681,13 +6939,15 @@ var CollectorOrchestrator = class {
|
|
|
6681
6939
|
* only the cache is consulted; if there is no cached entry the result gets
|
|
6682
6940
|
* `status: 'offline'` with `data: null`.
|
|
6683
6941
|
*/
|
|
6684
|
-
async collectAll(packageName, version) {
|
|
6942
|
+
async collectAll(packageName, version, ecosystem = "npm") {
|
|
6685
6943
|
logger.info(
|
|
6686
|
-
`Collecting data for ${packageName}@${version} (offline=${String(this.options.offline)})`
|
|
6944
|
+
`Collecting data for ${packageName}@${version} (ecosystem=${ecosystem}, offline=${String(this.options.offline)})`
|
|
6687
6945
|
);
|
|
6688
6946
|
const limit = pLimit(this.options.concurrency);
|
|
6947
|
+
const activeRegistryCollector = ecosystem === "pypi" ? this.pypiRegistryCollector : this.registryCollector;
|
|
6948
|
+
const osvEcosystem = ecosystem === "pypi" ? "PyPI" : "npm";
|
|
6689
6949
|
const entries = [
|
|
6690
|
-
{ key: "registry", collector:
|
|
6950
|
+
{ key: "registry", collector: activeRegistryCollector },
|
|
6691
6951
|
{ key: "github", collector: this.githubCollector },
|
|
6692
6952
|
{ key: "security", collector: this.securityCollector },
|
|
6693
6953
|
{ key: "funding", collector: this.fundingCollector },
|
|
@@ -6703,12 +6963,21 @@ var CollectorOrchestrator = class {
|
|
|
6703
6963
|
result = await this.offlineCollect(collector, packageName, version);
|
|
6704
6964
|
} else {
|
|
6705
6965
|
try {
|
|
6706
|
-
|
|
6707
|
-
|
|
6708
|
-
|
|
6709
|
-
|
|
6710
|
-
|
|
6711
|
-
|
|
6966
|
+
if (key === "security") {
|
|
6967
|
+
result = await Promise.race([
|
|
6968
|
+
this.securityCollector.collect(packageName, version, osvEcosystem),
|
|
6969
|
+
new Promise(
|
|
6970
|
+
(_, reject) => setTimeout(() => reject(new Error("Collector timeout")), COLLECTOR_TIMEOUT)
|
|
6971
|
+
)
|
|
6972
|
+
]);
|
|
6973
|
+
} else {
|
|
6974
|
+
result = await Promise.race([
|
|
6975
|
+
this.onlineCollect(collector, packageName, version),
|
|
6976
|
+
new Promise(
|
|
6977
|
+
(_, reject) => setTimeout(() => reject(new Error("Collector timeout")), COLLECTOR_TIMEOUT)
|
|
6978
|
+
)
|
|
6979
|
+
]);
|
|
6980
|
+
}
|
|
6712
6981
|
} catch {
|
|
6713
6982
|
logger.warn(`[${collector.name}] ${packageName}@${version} => timeout (${COLLECTOR_TIMEOUT}ms)`);
|
|
6714
6983
|
result = {
|
|
@@ -6833,7 +7102,7 @@ var TrustScoreEngine = class {
|
|
|
6833
7102
|
return {
|
|
6834
7103
|
trustScore: Math.round(trustScore),
|
|
6835
7104
|
metrics,
|
|
6836
|
-
insufficientData: unavailableMetrics.length >=
|
|
7105
|
+
insufficientData: unavailableMetrics.length >= 2,
|
|
6837
7106
|
unavailableMetrics
|
|
6838
7107
|
};
|
|
6839
7108
|
}
|
|
@@ -6975,6 +7244,11 @@ var TrustScoreEngine = class {
|
|
|
6975
7244
|
const effectiveWeight = m.weight / totalAvailableWeight;
|
|
6976
7245
|
score += m.score * effectiveWeight;
|
|
6977
7246
|
}
|
|
7247
|
+
const missingWeightFraction = 1 - totalAvailableWeight;
|
|
7248
|
+
if (missingWeightFraction > 0) {
|
|
7249
|
+
const penalty = (score - 50) * missingWeightFraction;
|
|
7250
|
+
score = score - penalty;
|
|
7251
|
+
}
|
|
6978
7252
|
return clamp(Math.round(score));
|
|
6979
7253
|
}
|
|
6980
7254
|
};
|
|
@@ -10735,7 +11009,115 @@ var POPULAR_PACKAGES = [
|
|
|
10735
11009
|
"request",
|
|
10736
11010
|
"tslint",
|
|
10737
11011
|
"node-pre-gyp",
|
|
10738
|
-
"npm-lifecycle"
|
|
11012
|
+
"npm-lifecycle",
|
|
11013
|
+
// ---------------------------------------------------------------------------
|
|
11014
|
+
// Popular Python packages (PyPI)
|
|
11015
|
+
// ---------------------------------------------------------------------------
|
|
11016
|
+
"requests",
|
|
11017
|
+
"flask",
|
|
11018
|
+
"django",
|
|
11019
|
+
"fastapi",
|
|
11020
|
+
"numpy",
|
|
11021
|
+
"pandas",
|
|
11022
|
+
"scipy",
|
|
11023
|
+
"matplotlib",
|
|
11024
|
+
"tensorflow",
|
|
11025
|
+
"torch",
|
|
11026
|
+
"scikit-learn",
|
|
11027
|
+
"keras",
|
|
11028
|
+
"pytorch-lightning",
|
|
11029
|
+
"xgboost",
|
|
11030
|
+
"lightgbm",
|
|
11031
|
+
"pytest",
|
|
11032
|
+
"black",
|
|
11033
|
+
"mypy",
|
|
11034
|
+
"ruff",
|
|
11035
|
+
"pylint",
|
|
11036
|
+
"flake8",
|
|
11037
|
+
"isort",
|
|
11038
|
+
"celery",
|
|
11039
|
+
"redis",
|
|
11040
|
+
"sqlalchemy",
|
|
11041
|
+
"alembic",
|
|
11042
|
+
"pydantic",
|
|
11043
|
+
"httpx",
|
|
11044
|
+
"aiohttp",
|
|
11045
|
+
"uvicorn",
|
|
11046
|
+
"gunicorn",
|
|
11047
|
+
"starlette",
|
|
11048
|
+
"boto3",
|
|
11049
|
+
"botocore",
|
|
11050
|
+
"awscli",
|
|
11051
|
+
"google-cloud-storage",
|
|
11052
|
+
"azure-storage-blob",
|
|
11053
|
+
"pillow",
|
|
11054
|
+
"opencv-python",
|
|
11055
|
+
"beautifulsoup4",
|
|
11056
|
+
"lxml",
|
|
11057
|
+
"scrapy",
|
|
11058
|
+
"cryptography",
|
|
11059
|
+
"paramiko",
|
|
11060
|
+
"fabric",
|
|
11061
|
+
"ansible",
|
|
11062
|
+
"click",
|
|
11063
|
+
"typer",
|
|
11064
|
+
"rich",
|
|
11065
|
+
"tqdm",
|
|
11066
|
+
"colorama",
|
|
11067
|
+
"tabulate",
|
|
11068
|
+
"setuptools",
|
|
11069
|
+
"wheel",
|
|
11070
|
+
"pip",
|
|
11071
|
+
"twine",
|
|
11072
|
+
"poetry",
|
|
11073
|
+
"pdm",
|
|
11074
|
+
"hatch",
|
|
11075
|
+
"jinja2",
|
|
11076
|
+
"mako",
|
|
11077
|
+
"markupsafe",
|
|
11078
|
+
"werkzeug",
|
|
11079
|
+
"psycopg2",
|
|
11080
|
+
"pymongo",
|
|
11081
|
+
"motor",
|
|
11082
|
+
"peewee",
|
|
11083
|
+
"tortoise-orm",
|
|
11084
|
+
"marshmallow",
|
|
11085
|
+
"attrs",
|
|
11086
|
+
"dataclasses-json",
|
|
11087
|
+
"sentry-sdk",
|
|
11088
|
+
"prometheus-client",
|
|
11089
|
+
"opentelemetry-api",
|
|
11090
|
+
"transformers",
|
|
11091
|
+
"huggingface-hub",
|
|
11092
|
+
"tokenizers",
|
|
11093
|
+
"datasets",
|
|
11094
|
+
"langchain",
|
|
11095
|
+
"openai",
|
|
11096
|
+
"anthropic",
|
|
11097
|
+
"tiktoken",
|
|
11098
|
+
"pytest-cov",
|
|
11099
|
+
"pytest-asyncio",
|
|
11100
|
+
"pytest-mock",
|
|
11101
|
+
"coverage",
|
|
11102
|
+
"pyyaml",
|
|
11103
|
+
"toml",
|
|
11104
|
+
"python-dotenv",
|
|
11105
|
+
"decouple",
|
|
11106
|
+
"arrow",
|
|
11107
|
+
"pendulum",
|
|
11108
|
+
"python-dateutil",
|
|
11109
|
+
"stripe",
|
|
11110
|
+
"twilio",
|
|
11111
|
+
"sendgrid",
|
|
11112
|
+
"selenium",
|
|
11113
|
+
"playwright",
|
|
11114
|
+
"httptools",
|
|
11115
|
+
"orjson",
|
|
11116
|
+
"ujson",
|
|
11117
|
+
"msgpack",
|
|
11118
|
+
"networkx",
|
|
11119
|
+
"sympy",
|
|
11120
|
+
"statsmodels"
|
|
10739
11121
|
];
|
|
10740
11122
|
var TyposquatDetector = class _TyposquatDetector {
|
|
10741
11123
|
popularPackages;
|