@goodandready/dsh-model-sync 0.5.16 → 0.5.17
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -0
- package/README.ru.md +3 -0
- package/README.zh.md +3 -0
- package/lib/client.js +170 -6
- package/lib/http.js +81 -1
- package/lib/models.js +114 -0
- package/lib/pricing-database.js +150 -17
- package/lib/provider-service.js +29 -0
- package/lib/reconcile.js +9 -2
- package/lib/scheduler.js +7 -0
- package/lib/synchronizer-prober.js +117 -35
- package/lib/synchronizer.js +25 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -78,6 +78,9 @@ graph LR
|
|
|
78
78
|
|
|
79
79
|
## ✨ Key Features
|
|
80
80
|
|
|
81
|
+
* 🏷️ **Smart Collections & Catalog Auto-Tagging**: Automatically categorizes models into standardized tags (`coding`, `reasoning`, `fast`, `vision`, `long-context`) during discovery, provides fast pill-button filtering in the Web UI, and introduces a one-click "⭐ Curated Best" selection preset for top flagship models.
|
|
82
|
+
* ⚡ **Latency TTFT Radar & Sparklines**: Probes response latency and Time To First Token (TTFT) across models, renders ASCII sparklines (` ▂▃▂ `) and 4-tier speed badges (⚡ `< 300ms`, 🟢 `< 1.2s`, 🟡 `1.5 - 3.0s`, 🔴 `> 3.0s`), and exposes `GET /dsh-model-sync/health/latency` for intelligent downstream routing.
|
|
83
|
+
* 💸 **Dynamic Live Pricing Sync & Prompt Caching**: Periodic background updates of live model pricing from OpenRouter and LiteLLM snapshots with persistent disk fallback cache, tracking prompt cache read/write costs (`cacheReadPerToken`, `cacheWritePerToken`), with `GET /dsh-model-sync/pricing` and `POST /dsh-model-sync/pricing/sync` endpoints.
|
|
81
84
|
* 💰 **Built-in Reference Pricing Database**: Canonical token pricing database automatically enriches models for OpenAI, Anthropic, Groq, Mistral, xAI, and DeepSeek, enabling cost-limit filtering and `@cheap` auto-resolution even when upstream APIs omit pricing metadata.
|
|
82
85
|
* 🎯 **Dynamic Smart Virtual Aliases & Auto-Fallback**: Route requests to virtual aliases like `@fast`, `@cheap`, and `@best-code`, with automatic fallback chains when circuit breakers open or health probes fail.
|
|
83
86
|
* 📏 **Real Context Limits & Boundary Safety (`contextWindow` / `maxOutputTokens`)**: precise separation of input and generation limits, prevention of 400 `max_tokens` errors via auto-clamping, and clear `In: 128k | Out: 8k` badges in Web UI.
|
package/README.ru.md
CHANGED
|
@@ -78,6 +78,9 @@ graph LR
|
|
|
78
78
|
|
|
79
79
|
## ✨ Ключевые возможности
|
|
80
80
|
|
|
81
|
+
* 🏷️ **Смарт-коллекции и авто-тегирование каталога**: Автоматическое присвоение тегов моделям (`coding`, `reasoning`, `fast`, `vision`, `long-context`) при синхронизации, фильтрация по тегам pill-кнопками в Web UI и пресет «⭐ Curated Best» для выбора лучших флагманских моделей в один клик.
|
|
82
|
+
* ⚡ **Радар задержек (Latency & TTFT) и спарклайны**: Измерение времени ответа и Time To First Token (TTFT), визуализация спарклайнами (` ▂▃▂ `) с 4 градациями скорости (⚡ `< 300мс`, 🟢 `< 1.2с`, 🟡 `1.5 - 3.0с`, 🔴 `> 3.0с`) и эндпоинт `GET /dsh-model-sync/health/latency` для агентов-маршрутизаторов.
|
|
83
|
+
* 💸 **Динамическая синхронизация цен и учёт Prompt Caching**: Фоновое обновление актуальных тарифов из OpenRouter и LiteLLM с дисковым fallback-кэшем, поддержка скидок кэширования промптов (`cacheReadPerToken`, `cacheWritePerToken`) и эндпоинты `GET /dsh-model-sync/pricing`, `POST /dsh-model-sync/pricing/sync`.
|
|
81
84
|
* 💰 **Встроенная база эталонных цен токенов**: Автоматическое обогащение моделей тарифами для OpenAI, Anthropic, Groq, Mistral, xAI и DeepSeek, обеспечивающее работу лимитов цен и псевдонима `@cheap` даже при отсутствии метаданных в API провайдера.
|
|
82
85
|
* 🎯 **Динамические «умные» псевдонимы и авто-фоллбэк**: Поддержка виртуальных алиасов (`@fast`, `@cheap`, `@best-code`) и цепочек резервирования с автопереключением при размыкании Circuit Breaker.
|
|
83
86
|
* 📏 **Реальные лимиты контекста и авто-клампинг (`contextWindow` / `maxOutputTokens`)**: точное разделение входного и выходного лимита токенов, защита от ошибок 400 `max_tokens` и индикация `Вход: 128k | Выход: 8k` в Web UI.
|
package/README.zh.md
CHANGED
|
@@ -78,6 +78,9 @@ graph LR
|
|
|
78
78
|
|
|
79
79
|
## ✨ 核心特性
|
|
80
80
|
|
|
81
|
+
* 🏷️ **智能集合与模型自动标签**:同步时自动识别并标记模型分类标签(`coding` 编程、`reasoning` 思考推理、`fast` 极速轻量、`vision` 多模态、`long-context` 128k+ 长上下文),支持 UI 药丸胶囊快速筛选与一键精选顶级旗舰预设("⭐ 精选最佳")。
|
|
82
|
+
* ⚡ **延迟雷达 (TTFT) 与火花线可视化**:实时测量模型响应时间与首字延迟 (TTFT),提供火花线走势(` ▂▃▂ `)及 4 档速度徽章(⚡ `< 300ms`、🟢 `< 1.2s`、🟡 `1.5 - 3.0s`、🔴 `> 3.0s`),并开放 `GET /dsh-model-sync/health/latency` 路由接口。
|
|
83
|
+
* 💸 **实时价格动态后台同步与提示词缓存**:定期从 OpenRouter 和 LiteLLM 获取最新模型单价,支持本地磁盘降级缓存,支持提示词缓存读写定价(`cacheReadPerToken`、`cacheWritePerToken`),并提供 `GET /dsh-model-sync/pricing` 和 `POST /dsh-model-sync/pricing/sync` 端点。
|
|
81
84
|
* 💰 **内置基准价格数据库**:自动为 OpenAI、Anthropic、Groq、Mistral、xAI 和 DeepSeek 模型补充 Token 价格数据,即使服务商接口未返回价格,也可无缝支持成本上限过滤与 `@cheap` 别名解析。
|
|
82
85
|
* 🎯 **动态智能虚拟别名与故障转移**:支持 `@fast`、`@cheap`、`@best-code` 等虚拟别名及故障转移链,在熔断器打开或健康检查失败时自动切换至备用模型。
|
|
83
86
|
* 📏 **真实上下文限制与边界安全 (`contextWindow` / `maxOutputTokens`)**:精确区分输入与输出令牌限制,通过自动限制防止 400 `max_tokens` 错误,并在 Web UI 中显示 `输入: 128k | 输出: 8k` 徽章。
|
package/lib/client.js
CHANGED
|
@@ -65,6 +65,14 @@ window.__ModuleLoader__.load({
|
|
|
65
65
|
'.dms-alert-ok{background:var(--dsw-alias-state-success-secondary,var(--dsw-alias-bg-layer-2));color:var(--dsw-alias-state-success-primary)}' +
|
|
66
66
|
'.dms-alert-bad{background:var(--dsw-alias-state-error-secondary,var(--dsw-alias-bg-layer-2));color:var(--dsw-alias-state-error-primary)}' +
|
|
67
67
|
'.dms-probe-result{padding:2px 6px;border-radius:4px;font-family:monospace;font-size:11px;border:1px solid var(--dsw-alias-border-l2);display:inline-flex;align-items:center;gap:4px}' +
|
|
68
|
+
'.dms-pill{display:inline-flex;align-items:center;gap:4px;font-size:11px;padding:3px 9px;border-radius:999px;border:1px solid var(--dsw-alias-border-l2);background:transparent;color:var(--dsw-alias-label-secondary);cursor:pointer}' +
|
|
69
|
+
'.dms-pill:hover{border-color:var(--dsw-alias-brand-primary);color:var(--dsw-alias-label-primary)}' +
|
|
70
|
+
'.dms-pill-active{background:var(--dsw-alias-label-primary)!important;color:var(--dsw-alias-bg-layer-3)!important;border-color:transparent!important;font-weight:600}' +
|
|
71
|
+
'.dms-tier-lightning{border-color:#a855f7;color:#a855f7;background:rgba(168,85,247,0.1)}' +
|
|
72
|
+
'.dms-tier-fast{border-color:var(--dsw-alias-state-success-primary);color:var(--dsw-alias-state-success-primary);background:var(--dsw-alias-state-success-secondary,rgba(34,197,94,0.1))}' +
|
|
73
|
+
'.dms-tier-moderate{border-color:var(--dsw-alias-state-warn-primary);color:var(--dsw-alias-state-warn-primary);background:var(--dsw-alias-state-warn-secondary,rgba(234,179,8,0.1))}' +
|
|
74
|
+
'.dms-tier-slow{border-color:var(--dsw-alias-state-error-primary);color:var(--dsw-alias-state-error-primary);background:var(--dsw-alias-state-error-secondary,rgba(239,68,68,0.1))}' +
|
|
75
|
+
'.dms-sparkline{font-family:monospace;letter-spacing:-1px;margin-left:3px;font-weight:bold}' +
|
|
68
76
|
'@media (max-width:540px){.dms-provider{grid-template-columns:1fr!important}.dms-provider .dms-btn{justify-self:start}.dms-toolbar>div:first-child{align-items:stretch;flex-direction:column}.dms-toolbar>div:first-child select{width:100%}.dms-toolbar,.dms-section,.dms-section-card,.dms-editor{padding:11px 12px}}'
|
|
69
77
|
|
|
70
78
|
function createErrorBoundary() {
|
|
@@ -228,6 +236,18 @@ window.__ModuleLoader__.load({
|
|
|
228
236
|
notifyOnPriceChanged: 'Notify on price changes',
|
|
229
237
|
saveWebhook: 'Save Webhook Settings',
|
|
230
238
|
testWebhook: 'Test Webhook',
|
|
239
|
+
selectCurated: '⭐ Curated Best',
|
|
240
|
+
tagAll: 'All',
|
|
241
|
+
tagCurated: '⭐ Curated',
|
|
242
|
+
tagReasoning: '🧠 Reasoning',
|
|
243
|
+
tagCoding: '💻 Coding',
|
|
244
|
+
tagFast: '⚡ Fast',
|
|
245
|
+
tagVision: '👁️ Vision',
|
|
246
|
+
tagLongContext: '📜 128k+',
|
|
247
|
+
speedRadar: 'Latency Radar',
|
|
248
|
+
ttft: 'TTFT',
|
|
249
|
+
pricingSync: 'Sync Prices',
|
|
250
|
+
pricingSyncing: 'Syncing prices…',
|
|
231
251
|
webhookTesting: 'Testing...',
|
|
232
252
|
webhookSuccess: 'Webhook test succeeded',
|
|
233
253
|
webhookFailed: 'Webhook test failed: %s',
|
|
@@ -400,6 +420,18 @@ window.__ModuleLoader__.load({
|
|
|
400
420
|
notifyOnPriceChanged: '模型价格变更时通知',
|
|
401
421
|
saveWebhook: '保存 Webhook 设置',
|
|
402
422
|
testWebhook: '测试 Webhook',
|
|
423
|
+
selectCurated: '⭐ 精选最佳',
|
|
424
|
+
tagAll: '全部',
|
|
425
|
+
tagCurated: '⭐ 精选',
|
|
426
|
+
tagReasoning: '🧠 深度思考',
|
|
427
|
+
tagCoding: '💻 编程优化',
|
|
428
|
+
tagFast: '⚡ 极速轻量',
|
|
429
|
+
tagVision: '👁️ 多模态视觉',
|
|
430
|
+
tagLongContext: '📜 长上下文',
|
|
431
|
+
speedRadar: '延迟雷达',
|
|
432
|
+
ttft: '首字延迟',
|
|
433
|
+
pricingSync: '同步价格',
|
|
434
|
+
pricingSyncing: '正在同步价格…',
|
|
403
435
|
webhookTesting: '正在测试...',
|
|
404
436
|
webhookSuccess: 'Webhook 测试成功',
|
|
405
437
|
webhookFailed: 'Webhook 测试失败:%s',
|
|
@@ -473,6 +505,81 @@ window.__ModuleLoader__.load({
|
|
|
473
505
|
}
|
|
474
506
|
|
|
475
507
|
|
|
508
|
+
const SPARKLINE_BARS = [' ', '▂', '▃', '▄', '▅', '▆', '▇', '█']
|
|
509
|
+
function generateSparkline(values) {
|
|
510
|
+
if (!Array.isArray(values) || values.length === 0) return ''
|
|
511
|
+
const nums = values.filter((v) => typeof v === 'number' && Number.isFinite(v))
|
|
512
|
+
if (nums.length === 0) return ''
|
|
513
|
+
if (nums.length === 1) return '▄'
|
|
514
|
+
const min = Math.min(...nums)
|
|
515
|
+
const max = Math.max(...nums)
|
|
516
|
+
if (min === max) return '▄'.repeat(nums.length)
|
|
517
|
+
const range = max - min
|
|
518
|
+
return nums.map((v) => SPARKLINE_BARS[Math.min(7, Math.max(0, Math.floor(((v - min) / range) * 7)))]).join('')
|
|
519
|
+
}
|
|
520
|
+
|
|
521
|
+
function classifySpeedTier(latencyMs, isOk = true) {
|
|
522
|
+
if (!isOk) return 'degraded'
|
|
523
|
+
if (typeof latencyMs !== 'number' || !Number.isFinite(latencyMs) || latencyMs <= 0) return 'unknown'
|
|
524
|
+
if (latencyMs < 300) return 'lightning'
|
|
525
|
+
if (latencyMs < 1200) return 'fast'
|
|
526
|
+
if (latencyMs <= 3000) return 'moderate'
|
|
527
|
+
return 'slow'
|
|
528
|
+
}
|
|
529
|
+
|
|
530
|
+
function deriveModelTags(model) {
|
|
531
|
+
if (!model || typeof model !== 'object') return []
|
|
532
|
+
const tags = new Set(Array.isArray(model.tags) ? model.tags.filter((t) => typeof t === 'string' && t.trim()).map((t) => t.trim().toLowerCase()) : [])
|
|
533
|
+
const idLower = String(model.id || '').toLowerCase()
|
|
534
|
+
const nameLower = String(model.name || '').toLowerCase()
|
|
535
|
+
const provLower = String(model.provider || '').toLowerCase()
|
|
536
|
+
const capabilities = model.capabilities || {}
|
|
537
|
+
|
|
538
|
+
if (capabilities.code === true || idLower.includes('coder') || idLower.includes('codestral') || idLower.includes('starcoder') || idLower.includes('deepseek-coder') || idLower.includes('claude-3-7-sonnet') || idLower.includes('claude-3-5-sonnet') || idLower.includes('kimi-k2.7-code') || idLower.includes('qwen-2.5-coder') || idLower.includes('qwen2.5-coder') || (/(^|[_-])(code|dev|prog)($|[_-])/i.test(idLower)) || (/(^|[_-])(code|coder)($|[_-])/i.test(nameLower))) {
|
|
539
|
+
tags.add('coding')
|
|
540
|
+
}
|
|
541
|
+
if (model.reasoningConfig?.supportsReasoning === true || capabilities.reasoning === true || /(^|[_-])(o1|o3|r1|qwq)($|[_-])/i.test(idLower) || idLower.includes('reasoner') || idLower.includes('deepseek-r1') || idLower.includes('thinking') || idLower.includes('-r1')) {
|
|
542
|
+
tags.add('reasoning')
|
|
543
|
+
}
|
|
544
|
+
if (provLower === 'groq' || provLower === 'cerebras' || idLower.includes('flash') || idLower.includes('mini') || idLower.includes('instant') || idLower.includes('haiku') || idLower.includes('small') || idLower.includes('8b') || idLower.includes('3b') || idLower.includes('1b') || idLower.includes('turbo') || idLower.includes('lite') || idLower.includes('nano')) {
|
|
545
|
+
tags.add('fast')
|
|
546
|
+
}
|
|
547
|
+
if (capabilities.vision === true || (Array.isArray(model.input) && model.input.includes('image')) || idLower.includes('vision') || idLower.includes('vl') || idLower.includes('pixtral') || idLower.includes('gemini') || idLower.includes('claude-3') || idLower.includes('4o')) {
|
|
548
|
+
tags.add('vision')
|
|
549
|
+
}
|
|
550
|
+
if (typeof model.contextWindow === 'number' && model.contextWindow >= 128000) {
|
|
551
|
+
tags.add('long-context')
|
|
552
|
+
}
|
|
553
|
+
return [...tags].sort()
|
|
554
|
+
}
|
|
555
|
+
|
|
556
|
+
function isCuratedModel(model) {
|
|
557
|
+
if (!model || typeof model !== 'object') return false
|
|
558
|
+
if (model.lifecycle?.deprecated) return false
|
|
559
|
+
const id = String(model.id || '').toLowerCase()
|
|
560
|
+
if (/-(202\d{5}|202\d-\d{2}-\d{2}|\d{4})$/.test(id) && !id.includes('claude-3-7') && !id.includes('claude-3-5')) {
|
|
561
|
+
return false
|
|
562
|
+
}
|
|
563
|
+
if (id.includes('-0314') || id.includes('-0613') || id.includes('-1106') || id.includes('-0125')) {
|
|
564
|
+
return false
|
|
565
|
+
}
|
|
566
|
+
const curatedIds = [
|
|
567
|
+
'claude-3-7-sonnet', 'claude-3-5-sonnet', 'claude-3-5-haiku',
|
|
568
|
+
'gpt-4o', 'gpt-4o-mini', 'o1', 'o3-mini',
|
|
569
|
+
'deepseek-chat', 'deepseek-reasoner', 'deepseek-v3', 'deepseek-r1',
|
|
570
|
+
'gemini-2.0-flash', 'gemini-1.5-pro', 'gemini-1.5-flash',
|
|
571
|
+
'llama-3.3-70b-versatile', 'llama-3.1-8b-instant', 'llama-3.1-70b-versatile',
|
|
572
|
+
'qwen-2.5-coder-32b', 'qwen-2.5-72b', 'qwq-32b',
|
|
573
|
+
'mistral-large', 'codestral', 'pixtral-large',
|
|
574
|
+
'kimi-k2.7-code'
|
|
575
|
+
]
|
|
576
|
+
for (const c of curatedIds) {
|
|
577
|
+
if (id === c || id.endsWith('/' + c) || id.includes(c)) return true
|
|
578
|
+
}
|
|
579
|
+
const tags = model.tags || deriveModelTags(model)
|
|
580
|
+
return tags.includes('coding') && tags.includes('reasoning') && tags.includes('long-context')
|
|
581
|
+
}
|
|
582
|
+
|
|
476
583
|
function formatTokens(num) {
|
|
477
584
|
if (typeof num !== "number" || !Number.isFinite(num) || num <= 0) return "?";
|
|
478
585
|
if (num >= 1000000) {
|
|
@@ -680,6 +787,7 @@ window.__ModuleLoader__.load({
|
|
|
680
787
|
})
|
|
681
788
|
const [pageSize, setPageSize] = React.useState(50)
|
|
682
789
|
const [quickFilter, setQuickFilter] = React.useState({ context100k: false, cheap: false })
|
|
790
|
+
const [selectedTag, setSelectedTag] = React.useState('all')
|
|
683
791
|
const [tryState, setTryState] = React.useState({})
|
|
684
792
|
const [batchTrying, setBatchTrying] = React.useState(false)
|
|
685
793
|
const [batchSummary, setBatchSummary] = React.useState(null)
|
|
@@ -1273,7 +1381,15 @@ window.__ModuleLoader__.load({
|
|
|
1273
1381
|
const haystack = (m.id + ' ' + (m.name || '') + ' ' + (m.aliases || []).join(' ') + ' ' + (m.tags || []).join(' ') + ' ' + (m.description || '')).toLowerCase()
|
|
1274
1382
|
return tokens.every((tok) => haystack.includes(tok))
|
|
1275
1383
|
}) : filteredChoicesBase
|
|
1276
|
-
const
|
|
1384
|
+
const taggedChoices = selectedTag === 'all'
|
|
1385
|
+
? searchedChoices
|
|
1386
|
+
: selectedTag === 'curated'
|
|
1387
|
+
? searchedChoices.filter((m) => isCuratedModel(m))
|
|
1388
|
+
: searchedChoices.filter((m) => {
|
|
1389
|
+
const tags = (m.tags && m.tags.length > 0) ? m.tags : deriveModelTags(m)
|
|
1390
|
+
return tags.includes(selectedTag)
|
|
1391
|
+
})
|
|
1392
|
+
const quickFilteredChoices = taggedChoices.filter((m) => {
|
|
1277
1393
|
if (quickFilter.context100k && (!m.contextWindow || m.contextWindow < 100000)) return false
|
|
1278
1394
|
if (quickFilter.cheap) {
|
|
1279
1395
|
const price = m.pricing?.inputPerToken
|
|
@@ -1296,7 +1412,7 @@ window.__ModuleLoader__.load({
|
|
|
1296
1412
|
}
|
|
1297
1413
|
return (a.name || a.id || '').localeCompare(b.name || b.id || '')
|
|
1298
1414
|
})
|
|
1299
|
-
}, [choices, capabilityFilter, debouncedSearchQuery, quickFilter, sortBy])
|
|
1415
|
+
}, [choices, capabilityFilter, debouncedSearchQuery, selectedTag, quickFilter, sortBy])
|
|
1300
1416
|
|
|
1301
1417
|
const maxContext = React.useMemo(() => filteredChoices.reduce((max, m) => Math.max(max, m.contextWindow || 0), 0), [filteredChoices])
|
|
1302
1418
|
const minPrice = React.useMemo(() => filteredChoices.reduce((min, m) => m.pricing?.inputPerToken !== undefined ? Math.min(min, m.pricing.inputPerToken) : min, Infinity), [filteredChoices])
|
|
@@ -1700,6 +1816,13 @@ window.__ModuleLoader__.load({
|
|
|
1700
1816
|
))
|
|
1701
1817
|
) : React.createElement('small', { style: { display: 'block', marginBottom: 8 } }, t.noCapabilities),
|
|
1702
1818
|
React.createElement('div', { style: { display: 'flex', gap: 6, margin: '6px 0', flexWrap: 'wrap' } },
|
|
1819
|
+
button(t.selectCurated || '⭐ Curated Best', () => {
|
|
1820
|
+
const ids = new Set(selectionDraft)
|
|
1821
|
+
for (const m of choices) {
|
|
1822
|
+
if (isCuratedModel(m)) ids.add(m.id)
|
|
1823
|
+
}
|
|
1824
|
+
setSelectionDraft([...ids])
|
|
1825
|
+
}, 'primary'),
|
|
1703
1826
|
button(t.selectAllFiltered, () => { const ids = new Set(selectionDraft); for (const m of filteredChoices) ids.add(m.id); setSelectionDraft([...ids]) }),
|
|
1704
1827
|
button(t.deselectAllFiltered, () => { const filterSet = new Set(filteredChoices.map((m) => m.id)); setSelectionDraft(selectionDraft.filter((id) => !filterSet.has(id))) }),
|
|
1705
1828
|
button(t.invertFiltered, () => { const current = new Set(selectionDraft); for (const m of filteredChoices) { if (current.has(m.id)) current.delete(m.id); else current.add(m.id); } setSelectionDraft([...current]) })
|
|
@@ -1710,6 +1833,22 @@ window.__ModuleLoader__.load({
|
|
|
1710
1833
|
batchSummary ? React.createElement('small', { style: { color: 'var(--dsw-alias-label-secondary)' } }, t.batchTryResults.replace('%d', batchSummary.reachable).replace('%d', batchSummary.total)) : null
|
|
1711
1834
|
),
|
|
1712
1835
|
React.createElement('small', { style: { display: 'block', marginBottom: 8 } }, `${filteredChoices.length}/${choices.length} ${t.capabilityCount}`),
|
|
1836
|
+
React.createElement('div', { style: { display: 'flex', gap: 6, margin: '8px 0', flexWrap: 'wrap', alignItems: 'center' } },
|
|
1837
|
+
[
|
|
1838
|
+
{ id: 'all', label: t.tagAll || 'All' },
|
|
1839
|
+
{ id: 'curated', label: t.tagCurated || '⭐ Curated' },
|
|
1840
|
+
{ id: 'reasoning', label: t.tagReasoning || '🧠 Reasoning' },
|
|
1841
|
+
{ id: 'coding', label: t.tagCoding || '💻 Coding' },
|
|
1842
|
+
{ id: 'fast', label: t.tagFast || '⚡ Fast' },
|
|
1843
|
+
{ id: 'vision', label: t.tagVision || '👁️ Vision' },
|
|
1844
|
+
{ id: 'long-context', label: t.tagLongContext || '📜 128k+' },
|
|
1845
|
+
].map((pill) => React.createElement('button', {
|
|
1846
|
+
key: pill.id,
|
|
1847
|
+
type: 'button',
|
|
1848
|
+
className: 'dms-pill' + (selectedTag === pill.id ? ' dms-pill-active' : ''),
|
|
1849
|
+
onClick: () => setSelectedTag(pill.id),
|
|
1850
|
+
}, pill.label))
|
|
1851
|
+
),
|
|
1713
1852
|
visibleChoices.map((model) => {
|
|
1714
1853
|
const deprecationDays = (() => {
|
|
1715
1854
|
const d = model.lifecycle?.deprecationDate
|
|
@@ -1717,7 +1856,8 @@ window.__ModuleLoader__.load({
|
|
|
1717
1856
|
const diff = new Date(d).getTime() - Date.now()
|
|
1718
1857
|
return diff > 0 ? Math.ceil(diff / (1000 * 60 * 60 * 24)) : 0
|
|
1719
1858
|
})()
|
|
1720
|
-
const
|
|
1859
|
+
const radar = status?.latencyRadar?.radar?.find((r) => r.provider === row.provider && r.model === model.id)
|
|
1860
|
+
const probe = tryState[`${row.provider}:${model.id}`] || (radar ? { ok: radar.ok, latency: radar.latencyMs, ttftMs: radar.ttftMs, sparkline: radar.sparkline, status: radar.status } : null)
|
|
1721
1861
|
return React.createElement('label', { key: model.id, style: { display: 'block', margin: '4px 0', padding: '4px 6px', borderRadius: 6, background: isNewModel(model, row) ? 'var(--dsw-alias-bg-layer-2)' : 'transparent' } },
|
|
1722
1862
|
React.createElement('input', { type: 'checkbox', checked: selectionDraft.includes(model.id), onChange: (e) => setSelectionDraft((current) => e.target.checked ? [...current, model.id] : current.filter((id) => id !== model.id)) }),
|
|
1723
1863
|
` ${model.name ?? model.id} (${model.id})`,
|
|
@@ -1749,12 +1889,36 @@ window.__ModuleLoader__.load({
|
|
|
1749
1889
|
}, `🧠 ${model.reasoningConfig?.paramFormat || 'reasoning'}`) : null,
|
|
1750
1890
|
(model.contextWindow && maxContext > 0 && model.contextWindow >= maxContext) ? React.createElement('span', { style: { marginLeft: 4, fontSize: 10, padding: '1px 4px', borderRadius: 3, background: 'var(--dsw-alias-state-warn-primary)', color: 'white' } }, '★') : null,
|
|
1751
1891
|
(model.pricing && minPrice !== Infinity && model.pricing.inputPerToken === minPrice) ? React.createElement('span', { style: { marginLeft: 4, fontSize: 10, padding: '1px 4px', borderRadius: 3, background: 'var(--dsw-alias-state-success-primary)', color: 'white' } }, '$') : null,
|
|
1752
|
-
|
|
1892
|
+
(() => {
|
|
1893
|
+
const modelTags = (model.tags && model.tags.length > 0) ? model.tags : deriveModelTags(model)
|
|
1894
|
+
if (!modelTags || modelTags.length === 0) return null
|
|
1895
|
+
return React.createElement('span', { style: { marginLeft: 6, display: 'inline-flex', gap: 3, verticalAlign: 'middle' } },
|
|
1896
|
+
modelTags.map((tag) => React.createElement('span', {
|
|
1897
|
+
key: tag,
|
|
1898
|
+
style: {
|
|
1899
|
+
fontSize: 10,
|
|
1900
|
+
padding: '0 4px',
|
|
1901
|
+
borderRadius: 3,
|
|
1902
|
+
background: 'var(--dsw-alias-bg-layer-2)',
|
|
1903
|
+
border: '1px solid var(--dsw-alias-border-l1)',
|
|
1904
|
+
color: 'var(--dsw-alias-label-secondary)',
|
|
1905
|
+
}
|
|
1906
|
+
}, tag === 'coding' ? '💻 coding' : tag === 'reasoning' ? '🧠 reasoning' : tag === 'fast' ? '⚡ fast' : tag === 'vision' ? '👁️ vision' : tag === 'long-context' ? '📜 128k+' : tag))
|
|
1907
|
+
)
|
|
1908
|
+
})(),
|
|
1753
1909
|
model.pricing ? React.createElement('small', { style: { marginLeft: 6, color: 'var(--dsw-alias-label-secondary)', fontSize: 11 } }, `in $${(model.pricing.inputPerToken * 1000000).toFixed(2)}/1M` + (model.pricing.cacheReadPerToken !== undefined ? ` / cache $${(model.pricing.cacheReadPerToken * 1000000).toFixed(2)}` : '') + (model.pricing.outputPerToken !== undefined ? ` / out $${(model.pricing.outputPerToken * 1000000).toFixed(2)}` : '')) : null,
|
|
1754
1910
|
model.description ? React.createElement('small', { title: model.description, style: { display: 'block', marginLeft: 22, color: 'var(--dsw-alias-label-secondary)', fontSize: 11 } }, model.description.slice(0, 120)) : null,
|
|
1755
1911
|
row.baseURL || row.api ? React.createElement('a', { href: (row.baseURL || '') + '/models/' + encodeURIComponent(model.id), target: '_blank', rel: 'noopener', style: { marginLeft: 8, fontSize: 11 } }, 'docs') : null,
|
|
1756
1912
|
React.createElement('button', { type: 'button', onClick: (e) => { e.preventDefault(); doTry(row.provider, model.id) }, style: { marginLeft: 8, fontSize: 11, padding: '1px 6px', borderRadius: 4 } }, (probe && probe.loading) ? '…' : '▶'),
|
|
1757
|
-
probe && !probe.loading ? React.createElement('span', {
|
|
1913
|
+
probe && !probe.loading ? React.createElement('span', {
|
|
1914
|
+
className: 'dms-badge ' + (probe.ok ? (classifySpeedTier(probe.latency, probe.ok) === 'lightning' ? 'dms-tier-lightning' : classifySpeedTier(probe.latency, probe.ok) === 'fast' ? 'dms-tier-fast' : 'dms-tier-moderate') : 'dms-badge-bad'),
|
|
1915
|
+
style: { marginLeft: 6 },
|
|
1916
|
+
title: probe.ttftMs ? `TTFT: ${probe.ttftMs}ms | Latency: ${probe.latency}ms` : (probe.latency ? `${probe.latency}ms` : undefined)
|
|
1917
|
+
},
|
|
1918
|
+
(classifySpeedTier(probe.latency, probe.ok) === 'lightning' ? '⚡ ' : classifySpeedTier(probe.latency, probe.ok) === 'fast' ? '🟢 ' : classifySpeedTier(probe.latency, probe.ok) === 'moderate' ? '🟡 ' : '🔴 ') +
|
|
1919
|
+
(probe.ok ? `${probe.latency}ms` : (probe.error || probe.status || 'offline')) +
|
|
1920
|
+
(probe.sparkline ? ` ${probe.sparkline}` : '')
|
|
1921
|
+
) : null
|
|
1758
1922
|
)
|
|
1759
1923
|
}),
|
|
1760
1924
|
filteredChoices.length > pageSize ? React.createElement('div', { style: { display: 'flex', alignItems: 'center', gap: 8, margin: '8px 0' } },
|
|
@@ -1916,7 +2080,7 @@ window.__ModuleLoader__.load({
|
|
|
1916
2080
|
trySlot('settings.plugin.item', () => ctx.slots.register({ name: 'settings.plugin.item', key: NS, locale: NS, inject: () => ({ ctx }) }, (props) => React.createElement(ErrorBoundary, null, React.createElement(ModelSyncCard, { ...props, ctx }))))
|
|
1917
2081
|
}
|
|
1918
2082
|
|
|
1919
|
-
module.exports = { apply, inject: ['slots', 'locale', 'configForms'], filterModelsByCapabilities }
|
|
2083
|
+
module.exports = { apply, inject: ['slots', 'locale', 'configForms'], filterModelsByCapabilities, deriveModelTags, isCuratedModel, generateSparkline, classifySpeedTier }
|
|
1920
2084
|
return module.exports
|
|
1921
2085
|
},
|
|
1922
2086
|
})
|
package/lib/http.js
CHANGED
|
@@ -339,7 +339,46 @@ export function registerHttpApi(ctx, synchronizer, { getSchedulerStatus = () =>
|
|
|
339
339
|
}
|
|
340
340
|
}
|
|
341
341
|
if (route === 'import') {
|
|
342
|
-
if (
|
|
342
|
+
if (route === 'health-latency') {
|
|
343
|
+
if (req.method !== 'GET') return json(res, 405, { error: { code: 'method', message: 'GET only' } })
|
|
344
|
+
try {
|
|
345
|
+
const urlObj = new URL(req.url, 'http://127.0.0.1')
|
|
346
|
+
const provider = urlObj.searchParams.get('provider') || undefined
|
|
347
|
+
const report = typeof synchronizer.getLatencyRadar === 'function'
|
|
348
|
+
? synchronizer.getLatencyRadar({ provider })
|
|
349
|
+
: { updatedAt: Date.now(), radar: [] }
|
|
350
|
+
return json(res, 200, report)
|
|
351
|
+
} catch (error) {
|
|
352
|
+
return json(res, 400, { error: { code: 'bad-request', message: error instanceof Error ? error.message : String(error) } })
|
|
353
|
+
}
|
|
354
|
+
}
|
|
355
|
+
if (route === 'pricing') {
|
|
356
|
+
if (req.method === 'GET') {
|
|
357
|
+
try {
|
|
358
|
+
const urlObj = new URL(req.url, 'http://127.0.0.1')
|
|
359
|
+
const provider = urlObj.searchParams.get('provider') || undefined
|
|
360
|
+
const model = urlObj.searchParams.get('model') || undefined
|
|
361
|
+
const report = typeof synchronizer.getPricingInfo === 'function'
|
|
362
|
+
? synchronizer.getPricingInfo({ provider, model })
|
|
363
|
+
: { pricing: {} }
|
|
364
|
+
return json(res, 200, report)
|
|
365
|
+
} catch (error) {
|
|
366
|
+
return json(res, 400, { error: { code: 'bad-request', message: error instanceof Error ? error.message : String(error) } })
|
|
367
|
+
}
|
|
368
|
+
}
|
|
369
|
+
if (req.method === 'POST') {
|
|
370
|
+
try {
|
|
371
|
+
const result = typeof synchronizer.syncPricing === 'function'
|
|
372
|
+
? await synchronizer.syncPricing()
|
|
373
|
+
: { updated: 0, source: 'reference' }
|
|
374
|
+
return json(res, 200, result)
|
|
375
|
+
} catch (error) {
|
|
376
|
+
return json(res, 500, { error: { code: 'sync-failed', message: error instanceof Error ? error.message : String(error) } })
|
|
377
|
+
}
|
|
378
|
+
}
|
|
379
|
+
return json(res, 405, { error: { code: 'method', message: 'GET or POST only' } })
|
|
380
|
+
}
|
|
381
|
+
if (req.method !== 'POST') return json(res, 405, { error: { code: 'method', message: 'POST only' } })
|
|
343
382
|
try {
|
|
344
383
|
const body = await readJson(req)
|
|
345
384
|
const result = await synchronizer.importConfig(body)
|
|
@@ -357,6 +396,45 @@ export function registerHttpApi(ctx, synchronizer, { getSchedulerStatus = () =>
|
|
|
357
396
|
if (req.method !== 'GET') return json(res, 405, { error: { code: 'method', message: 'GET only' } })
|
|
358
397
|
return json(res, 200, { notifications: synchronizer.notifications(normalizeNotificationRequest(req.url)) })
|
|
359
398
|
}
|
|
399
|
+
if (route === 'health-latency') {
|
|
400
|
+
if (req.method !== 'GET') return json(res, 405, { error: { code: 'method', message: 'GET only' } })
|
|
401
|
+
try {
|
|
402
|
+
const urlObj = new URL(req.url, 'http://127.0.0.1')
|
|
403
|
+
const provider = urlObj.searchParams.get('provider') || undefined
|
|
404
|
+
const report = typeof synchronizer.getLatencyRadar === 'function'
|
|
405
|
+
? synchronizer.getLatencyRadar({ provider })
|
|
406
|
+
: { updatedAt: Date.now(), radar: [] }
|
|
407
|
+
return json(res, 200, report)
|
|
408
|
+
} catch (error) {
|
|
409
|
+
return json(res, 400, { error: { code: 'bad-request', message: error instanceof Error ? error.message : String(error) } })
|
|
410
|
+
}
|
|
411
|
+
}
|
|
412
|
+
if (route === 'pricing') {
|
|
413
|
+
if (req.method === 'GET') {
|
|
414
|
+
try {
|
|
415
|
+
const urlObj = new URL(req.url, 'http://127.0.0.1')
|
|
416
|
+
const provider = urlObj.searchParams.get('provider') || undefined
|
|
417
|
+
const model = urlObj.searchParams.get('model') || undefined
|
|
418
|
+
const report = typeof synchronizer.getPricingInfo === 'function'
|
|
419
|
+
? synchronizer.getPricingInfo({ provider, model })
|
|
420
|
+
: { pricing: {} }
|
|
421
|
+
return json(res, 200, report)
|
|
422
|
+
} catch (error) {
|
|
423
|
+
return json(res, 400, { error: { code: 'bad-request', message: error instanceof Error ? error.message : String(error) } })
|
|
424
|
+
}
|
|
425
|
+
}
|
|
426
|
+
if (req.method === 'POST') {
|
|
427
|
+
try {
|
|
428
|
+
const result = typeof synchronizer.syncPricing === 'function'
|
|
429
|
+
? await synchronizer.syncPricing()
|
|
430
|
+
: { updated: 0, source: 'reference' }
|
|
431
|
+
return json(res, 200, result)
|
|
432
|
+
} catch (error) {
|
|
433
|
+
return json(res, 500, { error: { code: 'sync-failed', message: error instanceof Error ? error.message : String(error) } })
|
|
434
|
+
}
|
|
435
|
+
}
|
|
436
|
+
return json(res, 405, { error: { code: 'method', message: 'GET or POST only' } })
|
|
437
|
+
}
|
|
360
438
|
if (req.method !== 'POST') return json(res, 405, { error: { code: 'method', message: 'POST only' } })
|
|
361
439
|
let options
|
|
362
440
|
try {
|
|
@@ -423,5 +501,7 @@ export function registerHttpApi(ctx, synchronizer, { getSchedulerStatus = () =>
|
|
|
423
501
|
ctx.webServer.register({ kind: 'exact', path: BASE + '/notifications/read', handler: makeHandler('notification-read') }),
|
|
424
502
|
ctx.webServer.register({ kind: 'exact', path: BASE + '/notifications/acknowledge', handler: makeHandler('notification-acknowledge') }),
|
|
425
503
|
ctx.webServer.register({ kind: 'exact', path: BASE + '/webhook/test', handler: makeHandler('webhook-test') }),
|
|
504
|
+
ctx.webServer.register({ kind: 'exact', path: BASE + '/health/latency', handler: makeHandler('health-latency') }),
|
|
505
|
+
ctx.webServer.register({ kind: 'exact', path: BASE + '/pricing', handler: makeHandler('pricing') }),
|
|
426
506
|
]
|
|
427
507
|
}
|
package/lib/models.js
CHANGED
|
@@ -365,6 +365,120 @@ function normalizeReasoningConfig(provider, row, id, capabilities) {
|
|
|
365
365
|
return source
|
|
366
366
|
}
|
|
367
367
|
|
|
368
|
+
export function deriveModelTags(model) {
|
|
369
|
+
if (!model || typeof model !== "object") return []
|
|
370
|
+
const tags = new Set(Array.isArray(model.tags) ? model.tags.filter((t) => typeof t === "string" && t.trim()).map((t) => t.trim().toLowerCase()) : [])
|
|
371
|
+
const idLower = String(model.id || "").toLowerCase()
|
|
372
|
+
const nameLower = String(model.name || "").toLowerCase()
|
|
373
|
+
const provLower = String(model.provider || "").toLowerCase()
|
|
374
|
+
const capabilities = model.capabilities || {}
|
|
375
|
+
|
|
376
|
+
// 1. coding
|
|
377
|
+
if (
|
|
378
|
+
capabilities.code === true ||
|
|
379
|
+
idLower.includes("coder") ||
|
|
380
|
+
idLower.includes("codestral") ||
|
|
381
|
+
idLower.includes("starcoder") ||
|
|
382
|
+
idLower.includes("deepseek-coder") ||
|
|
383
|
+
idLower.includes("claude-3-7-sonnet") ||
|
|
384
|
+
idLower.includes("claude-3-5-sonnet") ||
|
|
385
|
+
idLower.includes("kimi-k2.7-code") ||
|
|
386
|
+
idLower.includes("qwen-2.5-coder") ||
|
|
387
|
+
idLower.includes("qwen2.5-coder") ||
|
|
388
|
+
(/(^|[_-])(code|dev|prog)($|[_-])/i.test(idLower)) ||
|
|
389
|
+
(/(^|[_-])(code|coder)($|[_-])/i.test(nameLower))
|
|
390
|
+
) {
|
|
391
|
+
tags.add("coding")
|
|
392
|
+
}
|
|
393
|
+
|
|
394
|
+
// 2. reasoning
|
|
395
|
+
if (
|
|
396
|
+
model.reasoningConfig?.supportsReasoning === true ||
|
|
397
|
+
capabilities.reasoning === true ||
|
|
398
|
+
/(^|[_-])(o1|o3|r1|qwq)($|[_-])/i.test(idLower) ||
|
|
399
|
+
idLower.includes("reasoner") ||
|
|
400
|
+
idLower.includes("deepseek-r1") ||
|
|
401
|
+
idLower.includes("thinking") ||
|
|
402
|
+
idLower.includes("-r1")
|
|
403
|
+
) {
|
|
404
|
+
tags.add("reasoning")
|
|
405
|
+
}
|
|
406
|
+
|
|
407
|
+
// 3. fast
|
|
408
|
+
if (
|
|
409
|
+
provLower === "groq" ||
|
|
410
|
+
provLower === "cerebras" ||
|
|
411
|
+
idLower.includes("flash") ||
|
|
412
|
+
idLower.includes("mini") ||
|
|
413
|
+
idLower.includes("instant") ||
|
|
414
|
+
idLower.includes("haiku") ||
|
|
415
|
+
idLower.includes("small") ||
|
|
416
|
+
idLower.includes("8b") ||
|
|
417
|
+
idLower.includes("3b") ||
|
|
418
|
+
idLower.includes("1b") ||
|
|
419
|
+
idLower.includes("turbo") ||
|
|
420
|
+
idLower.includes("lite") ||
|
|
421
|
+
idLower.includes("nano")
|
|
422
|
+
) {
|
|
423
|
+
tags.add("fast")
|
|
424
|
+
}
|
|
425
|
+
|
|
426
|
+
// 4. vision
|
|
427
|
+
if (
|
|
428
|
+
capabilities.vision === true ||
|
|
429
|
+
(Array.isArray(model.input) && model.input.includes("image")) ||
|
|
430
|
+
idLower.includes("vision") ||
|
|
431
|
+
idLower.includes("vl") ||
|
|
432
|
+
idLower.includes("pixtral") ||
|
|
433
|
+
idLower.includes("gemini") ||
|
|
434
|
+
idLower.includes("claude-3") ||
|
|
435
|
+
idLower.includes("4o")
|
|
436
|
+
) {
|
|
437
|
+
tags.add("vision")
|
|
438
|
+
}
|
|
439
|
+
|
|
440
|
+
// 5. long-context (>= 128k tokens)
|
|
441
|
+
if (typeof model.contextWindow === "number" && model.contextWindow >= 128000) {
|
|
442
|
+
tags.add("long-context")
|
|
443
|
+
}
|
|
444
|
+
|
|
445
|
+
return [...tags].sort()
|
|
446
|
+
}
|
|
447
|
+
|
|
448
|
+
export function isCuratedModel(model) {
|
|
449
|
+
if (!model || typeof model !== "object") return false
|
|
450
|
+
if (model.lifecycle?.deprecated) return false
|
|
451
|
+
const id = String(model.id || "").toLowerCase()
|
|
452
|
+
if (/-(202\d{5}|202\d-\d{2}-\d{2}|\d{4})$/.test(id) && !id.includes("claude-3-7") && !id.includes("claude-3-5")) {
|
|
453
|
+
return false
|
|
454
|
+
}
|
|
455
|
+
if (id.includes("-0314") || id.includes("-0613") || id.includes("-1106") || id.includes("-0125")) {
|
|
456
|
+
return false
|
|
457
|
+
}
|
|
458
|
+
|
|
459
|
+
const curatedIds = [
|
|
460
|
+
"claude-3-7-sonnet", "claude-3-5-sonnet", "claude-3-5-haiku",
|
|
461
|
+
"gpt-4o", "gpt-4o-mini", "o1", "o3-mini",
|
|
462
|
+
"deepseek-chat", "deepseek-reasoner", "deepseek-v3", "deepseek-r1",
|
|
463
|
+
"gemini-2.0-flash", "gemini-1.5-pro", "gemini-1.5-flash",
|
|
464
|
+
"llama-3.3-70b-versatile", "llama-3.1-8b-instant", "llama-3.1-70b-versatile",
|
|
465
|
+
"qwen-2.5-coder-32b", "qwen-2.5-72b", "qwq-32b",
|
|
466
|
+
"mistral-large", "codestral", "pixtral-large",
|
|
467
|
+
"kimi-k2.7-code"
|
|
468
|
+
]
|
|
469
|
+
|
|
470
|
+
for (const c of curatedIds) {
|
|
471
|
+
if (id === c || id.endsWith("/" + c) || id.includes(c)) return true
|
|
472
|
+
}
|
|
473
|
+
|
|
474
|
+
const tags = model.tags || deriveModelTags(model)
|
|
475
|
+
if (tags.includes("coding") && tags.includes("reasoning") && tags.includes("long-context")) {
|
|
476
|
+
return true
|
|
477
|
+
}
|
|
478
|
+
|
|
479
|
+
return false
|
|
480
|
+
}
|
|
481
|
+
|
|
368
482
|
export function normalizeModel(provider, row) {
|
|
369
483
|
if (typeof row === 'string') {
|
|
370
484
|
row = { id: row }
|
package/lib/pricing-database.js
CHANGED
|
@@ -1,6 +1,13 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
1
|
+
import fs from 'node:fs'
|
|
2
|
+
import path from 'node:path'
|
|
3
|
+
|
|
4
|
+
function cleanTokenPrice(val, { isPerToken = false } = {}) {
|
|
5
|
+
const num = typeof val === 'string' ? Number(val.trim()) : val
|
|
6
|
+
if (typeof num !== 'number' || !Number.isFinite(num) || num < 0) return 0
|
|
7
|
+
if (isPerToken) {
|
|
8
|
+
return Math.round(num * 1e12) / 1e12
|
|
9
|
+
}
|
|
10
|
+
return Math.round((num / 1000000) * 1e12) / 1e12
|
|
4
11
|
}
|
|
5
12
|
|
|
6
13
|
function perMillion(inputUSD, outputUSD, extra = {}) {
|
|
@@ -21,23 +28,23 @@ function perMillion(inputUSD, outputUSD, extra = {}) {
|
|
|
21
28
|
|
|
22
29
|
export const REFERENCE_PRICING = Object.freeze({
|
|
23
30
|
openai: {
|
|
24
|
-
'gpt-4o': perMillion(2.5, 10),
|
|
25
|
-
'gpt-4o-mini': perMillion(0.15, 0.6),
|
|
31
|
+
'gpt-4o': perMillion(2.5, 10, { cacheReadUSD: 1.25 }),
|
|
32
|
+
'gpt-4o-mini': perMillion(0.15, 0.6, { cacheReadUSD: 0.075 }),
|
|
26
33
|
'gpt-4-turbo': perMillion(10, 30),
|
|
27
34
|
'gpt-4': perMillion(30, 60),
|
|
28
35
|
'gpt-3.5-turbo': perMillion(0.5, 1.5),
|
|
29
|
-
'o1': perMillion(15, 60),
|
|
36
|
+
'o1': perMillion(15, 60, { cacheReadUSD: 7.5 }),
|
|
30
37
|
'o1-preview': perMillion(15, 60),
|
|
31
|
-
'o1-mini': perMillion(1.1, 4.4),
|
|
32
|
-
'o3-mini': perMillion(1.1, 4.4),
|
|
38
|
+
'o1-mini': perMillion(1.1, 4.4, { cacheReadUSD: 0.55 }),
|
|
39
|
+
'o3-mini': perMillion(1.1, 4.4, { cacheReadUSD: 0.55 }),
|
|
33
40
|
'chatgpt-4o-latest': perMillion(5, 15),
|
|
34
41
|
},
|
|
35
42
|
anthropic: {
|
|
36
|
-
'claude-3-7-sonnet': perMillion(3, 15),
|
|
37
|
-
'claude-3-5-sonnet': perMillion(3, 15),
|
|
38
|
-
'claude-3-5-haiku': perMillion(0.8, 4),
|
|
39
|
-
'claude-3-opus': perMillion(15, 75),
|
|
40
|
-
'claude-3-haiku': perMillion(0.25, 1.25),
|
|
43
|
+
'claude-3-7-sonnet': perMillion(3, 15, { cacheReadUSD: 0.3, cacheWriteUSD: 3.75 }),
|
|
44
|
+
'claude-3-5-sonnet': perMillion(3, 15, { cacheReadUSD: 0.3, cacheWriteUSD: 3.75 }),
|
|
45
|
+
'claude-3-5-haiku': perMillion(0.8, 4, { cacheReadUSD: 0.08, cacheWriteUSD: 1 }),
|
|
46
|
+
'claude-3-opus': perMillion(15, 75, { cacheReadUSD: 1.5, cacheWriteUSD: 18.75 }),
|
|
47
|
+
'claude-3-haiku': perMillion(0.25, 1.25, { cacheReadUSD: 0.025, cacheWriteUSD: 0.3 }),
|
|
41
48
|
},
|
|
42
49
|
groq: {
|
|
43
50
|
'llama-3.3-70b-versatile': perMillion(0.59, 0.79),
|
|
@@ -83,6 +90,22 @@ export const REFERENCE_PRICING = Object.freeze({
|
|
|
83
90
|
},
|
|
84
91
|
})
|
|
85
92
|
|
|
93
|
+
// Dynamic in-memory pricing store populated from OpenRouter / LiteLLM / disk cache
|
|
94
|
+
const DYNAMIC_PRICING = new Map()
|
|
95
|
+
|
|
96
|
+
export function getDynamicPricingMap() {
|
|
97
|
+
return DYNAMIC_PRICING
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
export function clearDynamicPricing() {
|
|
101
|
+
DYNAMIC_PRICING.clear()
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
export function setDynamicPricing(key, pricing) {
|
|
105
|
+
if (!key || typeof key !== 'string' || !pricing) return
|
|
106
|
+
DYNAMIC_PRICING.set(key.toLowerCase().trim(), Object.freeze({ ...pricing }))
|
|
107
|
+
}
|
|
108
|
+
|
|
86
109
|
function cleanId(id) {
|
|
87
110
|
if (typeof id !== 'string') return ''
|
|
88
111
|
return id
|
|
@@ -100,16 +123,23 @@ export function lookupReferencePricing(provider, modelId) {
|
|
|
100
123
|
const rawId = modelId.toLowerCase().trim()
|
|
101
124
|
const cleaned = cleanId(rawId)
|
|
102
125
|
|
|
103
|
-
// 1.
|
|
126
|
+
// 1. Dynamic pricing cache check (provider:modelId or rawId or cleaned)
|
|
127
|
+
if (DYNAMIC_PRICING.size > 0) {
|
|
128
|
+
const dynScoped = provKey ? DYNAMIC_PRICING.get(`${provKey}:${rawId}`) || DYNAMIC_PRICING.get(`${provKey}:${cleaned}`) : undefined
|
|
129
|
+
if (dynScoped) return { ...dynScoped }
|
|
130
|
+
const dynRaw = DYNAMIC_PRICING.get(rawId) || DYNAMIC_PRICING.get(cleaned)
|
|
131
|
+
if (dynRaw) return { ...dynRaw }
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
// 2. Direct check in provider table (exact match or date-cleaned match)
|
|
104
135
|
const table = REFERENCE_PRICING[provKey]
|
|
105
136
|
if (table) {
|
|
106
137
|
if (table[rawId]) return { ...table[rawId] }
|
|
107
138
|
if (table[cleaned]) return { ...table[cleaned] }
|
|
108
|
-
// If the provider has a dedicated pricing table, do not search other providers' tables
|
|
109
139
|
return undefined
|
|
110
140
|
}
|
|
111
141
|
|
|
112
|
-
//
|
|
142
|
+
// 3. Vendor prefix check (e.g. "openai/gpt-4o" or "anthropic/claude-3-5-sonnet")
|
|
113
143
|
const slashIdx = rawId.indexOf('/')
|
|
114
144
|
if (slashIdx > 0) {
|
|
115
145
|
const subProv = rawId.slice(0, slashIdx)
|
|
@@ -122,10 +152,113 @@ export function lookupReferencePricing(provider, modelId) {
|
|
|
122
152
|
}
|
|
123
153
|
}
|
|
124
154
|
|
|
125
|
-
//
|
|
155
|
+
// 4. Fallback search across tables ONLY for unknown/aggregator providers without a dedicated table
|
|
126
156
|
for (const [p, t] of Object.entries(REFERENCE_PRICING)) {
|
|
127
157
|
if (t[cleaned]) return { ...t[cleaned] }
|
|
128
158
|
}
|
|
129
159
|
|
|
130
160
|
return undefined
|
|
131
161
|
}
|
|
162
|
+
|
|
163
|
+
export function parseOpenRouterPricing(modelsData) {
|
|
164
|
+
if (!Array.isArray(modelsData)) return []
|
|
165
|
+
const results = []
|
|
166
|
+
for (const item of modelsData) {
|
|
167
|
+
if (!item?.id || !item?.pricing) continue
|
|
168
|
+
const id = String(item.id).trim().toLowerCase()
|
|
169
|
+
const p = item.pricing
|
|
170
|
+
const inputPerToken = cleanTokenPrice(p.prompt, { isPerToken: true })
|
|
171
|
+
const outputPerToken = cleanTokenPrice(p.completion, { isPerToken: true })
|
|
172
|
+
const cacheReadPerToken = p.input_cache_read !== undefined || p.prompt_cached !== undefined
|
|
173
|
+
? cleanTokenPrice(p.input_cache_read ?? p.prompt_cached, { isPerToken: true })
|
|
174
|
+
: undefined
|
|
175
|
+
const cacheWritePerToken = p.input_cache_write !== undefined
|
|
176
|
+
? cleanTokenPrice(p.input_cache_write, { isPerToken: true })
|
|
177
|
+
: undefined
|
|
178
|
+
|
|
179
|
+
if (inputPerToken > 0 || outputPerToken > 0) {
|
|
180
|
+
const entry = {
|
|
181
|
+
inputPerToken,
|
|
182
|
+
outputPerToken,
|
|
183
|
+
...(cacheReadPerToken !== undefined ? { cacheReadPerToken } : {}),
|
|
184
|
+
...(cacheWritePerToken !== undefined ? { cacheWritePerToken } : {}),
|
|
185
|
+
currency: 'USD',
|
|
186
|
+
source: 'openrouter',
|
|
187
|
+
}
|
|
188
|
+
results.push({ id, entry })
|
|
189
|
+
}
|
|
190
|
+
}
|
|
191
|
+
return results
|
|
192
|
+
}
|
|
193
|
+
|
|
194
|
+
export async function fetchPricingUpdates({
|
|
195
|
+
timeoutMs = 15000,
|
|
196
|
+
fetchFn = globalThis.fetch,
|
|
197
|
+
cachePath,
|
|
198
|
+
logger,
|
|
199
|
+
} = {}) {
|
|
200
|
+
let updatedCount = 0
|
|
201
|
+
const controller = new AbortController()
|
|
202
|
+
const timer = setTimeout(() => controller.abort(), timeoutMs)
|
|
203
|
+
|
|
204
|
+
try {
|
|
205
|
+
const res = await fetchFn('https://openrouter.ai/api/v1/models', {
|
|
206
|
+
signal: controller.signal,
|
|
207
|
+
headers: { 'Accept': 'application/json' },
|
|
208
|
+
})
|
|
209
|
+
if (res.ok) {
|
|
210
|
+
const json = await res.json()
|
|
211
|
+
const entries = parseOpenRouterPricing(json?.data)
|
|
212
|
+
for (const { id, entry } of entries) {
|
|
213
|
+
DYNAMIC_PRICING.set(id, Object.freeze(entry))
|
|
214
|
+
const clean = cleanId(id)
|
|
215
|
+
if (clean && clean !== id) {
|
|
216
|
+
DYNAMIC_PRICING.set(clean, Object.freeze(entry))
|
|
217
|
+
}
|
|
218
|
+
// Also index under vendor-prefixed subId
|
|
219
|
+
const slashIdx = id.indexOf('/')
|
|
220
|
+
if (slashIdx > 0) {
|
|
221
|
+
const subId = id.slice(slashIdx + 1)
|
|
222
|
+
DYNAMIC_PRICING.set(subId, Object.freeze(entry))
|
|
223
|
+
const subClean = cleanId(subId)
|
|
224
|
+
if (subClean && subClean !== subId) {
|
|
225
|
+
DYNAMIC_PRICING.set(subClean, Object.freeze(entry))
|
|
226
|
+
}
|
|
227
|
+
}
|
|
228
|
+
updatedCount++
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
if (cachePath && updatedCount > 0) {
|
|
232
|
+
try {
|
|
233
|
+
const serialized = JSON.stringify(Object.fromEntries(DYNAMIC_PRICING), null, 2)
|
|
234
|
+
fs.writeFileSync(cachePath, serialized, 'utf-8')
|
|
235
|
+
} catch (writeErr) {
|
|
236
|
+
logger?.warn?.(`[dsh-model-sync] failed to write pricing cache: ${writeErr?.message}`)
|
|
237
|
+
}
|
|
238
|
+
}
|
|
239
|
+
|
|
240
|
+
return { updated: updatedCount, source: 'openrouter', timestamp: Date.now() }
|
|
241
|
+
}
|
|
242
|
+
} catch (err) {
|
|
243
|
+
logger?.warn?.(`[dsh-model-sync] live pricing fetch failed: ${err?.message}, falling back to cache`)
|
|
244
|
+
} finally {
|
|
245
|
+
clearTimeout(timer)
|
|
246
|
+
}
|
|
247
|
+
|
|
248
|
+
// Fallback: try loading cached pricing from disk
|
|
249
|
+
if (cachePath && fs.existsSync(cachePath)) {
|
|
250
|
+
try {
|
|
251
|
+
const data = fs.readFileSync(cachePath, 'utf-8')
|
|
252
|
+
const parsed = JSON.parse(data)
|
|
253
|
+
for (const [k, v] of Object.entries(parsed)) {
|
|
254
|
+
DYNAMIC_PRICING.set(k.toLowerCase().trim(), Object.freeze(v))
|
|
255
|
+
updatedCount++
|
|
256
|
+
}
|
|
257
|
+
return { updated: updatedCount, source: 'cache', timestamp: Date.now() }
|
|
258
|
+
} catch (readErr) {
|
|
259
|
+
logger?.warn?.(`[dsh-model-sync] failed to read pricing cache: ${readErr?.message}`)
|
|
260
|
+
}
|
|
261
|
+
}
|
|
262
|
+
|
|
263
|
+
return { updated: 0, source: 'reference', timestamp: Date.now() }
|
|
264
|
+
}
|
package/lib/provider-service.js
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
import { listApiKeyProviders } from './inventory.js'
|
|
2
2
|
import { lookupReferenceLimits, lookupReferenceReasoning, clampOutputTokens, adaptReasoningPayload } from './specs-database.js'
|
|
3
|
+
import { deriveModelTags, isCuratedModel } from './models.js'
|
|
4
|
+
import { lookupReferencePricing } from './pricing-database.js'
|
|
3
5
|
|
|
4
6
|
export function createProviderService(ctx) {
|
|
5
7
|
const readSettings = () => {
|
|
@@ -62,6 +64,30 @@ export function createProviderService(ctx) {
|
|
|
62
64
|
return adaptReasoningPayload(found, options)
|
|
63
65
|
}
|
|
64
66
|
|
|
67
|
+
const getPricing = (provider, modelId) => {
|
|
68
|
+
const p = getProvider(provider)
|
|
69
|
+
const catalog = Array.isArray(p?.models) ? p.models : []
|
|
70
|
+
const found = catalog.find((m) => (m?.id === modelId || m?.name === modelId))
|
|
71
|
+
if (found?.pricing) return found.pricing
|
|
72
|
+
return lookupReferencePricing(provider, modelId) ?? null
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
const getTags = (provider, modelId) => {
|
|
76
|
+
const p = getProvider(provider)
|
|
77
|
+
const catalog = Array.isArray(p?.models) ? p.models : []
|
|
78
|
+
const found = catalog.find((m) => (m?.id === modelId || m?.name === modelId))
|
|
79
|
+
if (found?.tags && found.tags.length > 0) return found.tags
|
|
80
|
+
if (found) return deriveModelTags(found)
|
|
81
|
+
return deriveModelTags({ provider, id: modelId })
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
const isCurated = (provider, modelId) => {
|
|
85
|
+
const p = getProvider(provider)
|
|
86
|
+
const catalog = Array.isArray(p?.models) ? p.models : []
|
|
87
|
+
const found = catalog.find((m) => (m?.id === modelId || m?.name === modelId))
|
|
88
|
+
return isCuratedModel(found || { provider, id: modelId })
|
|
89
|
+
}
|
|
90
|
+
|
|
65
91
|
return Object.freeze({
|
|
66
92
|
listProviders,
|
|
67
93
|
getProvider,
|
|
@@ -69,5 +95,8 @@ export function createProviderService(ctx) {
|
|
|
69
95
|
clampCompletionTokens,
|
|
70
96
|
getReasoningConfig,
|
|
71
97
|
adaptReasoningPayload: adaptReasoning,
|
|
98
|
+
getPricing,
|
|
99
|
+
getTags,
|
|
100
|
+
isCurated,
|
|
72
101
|
})
|
|
73
102
|
}
|
package/lib/reconcile.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { diffModels, normalizeModels, stableStringify } from './models.js'
|
|
1
|
+
import { diffModels, normalizeModels, stableStringify, deriveModelTags } from './models.js'
|
|
2
2
|
|
|
3
3
|
function sameModel(left, right) {
|
|
4
4
|
return stableStringify(left) === stableStringify(right)
|
|
@@ -35,6 +35,8 @@ export function reconcileModels(provider, current, discovered, {
|
|
|
35
35
|
} : {}),
|
|
36
36
|
...(fresh.input?.length ? { input: fresh.input } : row.input ? { input: row.input } : {}),
|
|
37
37
|
}
|
|
38
|
+
const derivedTags = deriveModelTags(merged)
|
|
39
|
+
if (derivedTags.length > 0) merged.tags = derivedTags
|
|
38
40
|
next.push(merged)
|
|
39
41
|
} else if (renamedBefore.has(row.id)) {
|
|
40
42
|
// An explicit provider alias is safe evidence that the old id was renamed.
|
|
@@ -45,7 +47,12 @@ export function reconcileModels(provider, current, discovered, {
|
|
|
45
47
|
}
|
|
46
48
|
const known = new Set(before.map((row) => row.id))
|
|
47
49
|
for (const row of advertised) {
|
|
48
|
-
if (!known.has(row.id))
|
|
50
|
+
if (!known.has(row.id)) {
|
|
51
|
+
const copy = { ...row }
|
|
52
|
+
const derivedTags = deriveModelTags(copy)
|
|
53
|
+
if (derivedTags.length > 0) copy.tags = derivedTags
|
|
54
|
+
next.push(copy)
|
|
55
|
+
}
|
|
49
56
|
}
|
|
50
57
|
const diff = diffModels(before, next)
|
|
51
58
|
return {
|
package/lib/scheduler.js
CHANGED
|
@@ -24,6 +24,7 @@ export function createSyncScheduler(sync, {
|
|
|
24
24
|
} = {}) {
|
|
25
25
|
let timer = null
|
|
26
26
|
let running = false
|
|
27
|
+
let lastPricingSyncAt = 0
|
|
27
28
|
const state = new Map()
|
|
28
29
|
|
|
29
30
|
function stop() {
|
|
@@ -82,6 +83,12 @@ export function createSyncScheduler(sync, {
|
|
|
82
83
|
async function tick() {
|
|
83
84
|
const config = getConfig() ?? {}
|
|
84
85
|
if (config.enabled === false || config.scheduleEnabled === false || running) return
|
|
86
|
+
if (typeof sync.syncPricing === 'function' && (nowFn() - lastPricingSyncAt > 24 * 60 * 60 * 1000 || lastPricingSyncAt === 0)) {
|
|
87
|
+
lastPricingSyncAt = nowFn()
|
|
88
|
+
sync.syncPricing().catch((err) => {
|
|
89
|
+
logger.warn?.('[dsh-model-sync] background pricing sync failed: ' + errorMessage(err))
|
|
90
|
+
})
|
|
91
|
+
}
|
|
85
92
|
resetStates(config)
|
|
86
93
|
const rows = scheduleRows(config)
|
|
87
94
|
if (rows.length === 0) return
|
|
@@ -1,3 +1,31 @@
|
|
|
1
|
+
const SPARKLINE_BARS = [' ', '▂', '▃', '▄', '▅', '▆', '▇', '█']
|
|
2
|
+
|
|
3
|
+
export function generateSparkline(values) {
|
|
4
|
+
if (!Array.isArray(values) || values.length === 0) return ''
|
|
5
|
+
const nums = values.filter((v) => typeof v === 'number' && Number.isFinite(v))
|
|
6
|
+
if (nums.length === 0) return ''
|
|
7
|
+
if (nums.length === 1) return '▄'
|
|
8
|
+
const min = Math.min(...nums)
|
|
9
|
+
const max = Math.max(...nums)
|
|
10
|
+
if (min === max) return '▄'.repeat(nums.length)
|
|
11
|
+
const range = max - min
|
|
12
|
+
return nums
|
|
13
|
+
.map((v) => {
|
|
14
|
+
const idx = Math.min(7, Math.max(0, Math.floor(((v - min) / range) * 7)))
|
|
15
|
+
return SPARKLINE_BARS[idx]
|
|
16
|
+
})
|
|
17
|
+
.join('')
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
export function classifySpeedTier(latencyMs, isOk = true) {
|
|
21
|
+
if (!isOk) return 'degraded'
|
|
22
|
+
if (typeof latencyMs !== 'number' || !Number.isFinite(latencyMs) || latencyMs <= 0) return 'unknown'
|
|
23
|
+
if (latencyMs < 300) return 'lightning'
|
|
24
|
+
if (latencyMs < 1200) return 'fast'
|
|
25
|
+
if (latencyMs <= 3000) return 'moderate'
|
|
26
|
+
return 'slow'
|
|
27
|
+
}
|
|
28
|
+
|
|
1
29
|
export function createModelProber({
|
|
2
30
|
health,
|
|
3
31
|
getProviderCatalog = () => [],
|
|
@@ -5,8 +33,32 @@ export function createModelProber({
|
|
|
5
33
|
inferModel,
|
|
6
34
|
} = {}) {
|
|
7
35
|
const modelProbes = new Map()
|
|
36
|
+
const latencyHistory = new Map()
|
|
8
37
|
const probeFn = probeModel ?? inferModel
|
|
9
38
|
|
|
39
|
+
function recordLatencyEntry(provider, model, { latencyMs, ttftMs, ok, status, error }) {
|
|
40
|
+
const key = `${provider}:${model}`
|
|
41
|
+
modelProbes.set(key, {
|
|
42
|
+
latencyMs,
|
|
43
|
+
ttftMs,
|
|
44
|
+
ok,
|
|
45
|
+
status,
|
|
46
|
+
timestamp: Date.now(),
|
|
47
|
+
})
|
|
48
|
+
|
|
49
|
+
const hist = latencyHistory.get(key) || []
|
|
50
|
+
hist.push({
|
|
51
|
+
timestamp: Date.now(),
|
|
52
|
+
latencyMs,
|
|
53
|
+
ttftMs,
|
|
54
|
+
ok,
|
|
55
|
+
status,
|
|
56
|
+
...(error ? { error } : {}),
|
|
57
|
+
})
|
|
58
|
+
if (hist.length > 10) hist.shift()
|
|
59
|
+
latencyHistory.set(key, hist)
|
|
60
|
+
}
|
|
61
|
+
|
|
10
62
|
async function tryModel({ provider, model, signal } = {}) {
|
|
11
63
|
if (typeof provider !== 'string' || !provider) throw new Error('provider is required')
|
|
12
64
|
if (typeof model !== 'string' || !model) throw new Error('model is required')
|
|
@@ -21,15 +73,11 @@ export function createModelProber({
|
|
|
21
73
|
model,
|
|
22
74
|
ok: false,
|
|
23
75
|
latencyMs: 0,
|
|
76
|
+
ttftMs: 0,
|
|
24
77
|
status: 'model_not_found',
|
|
25
78
|
error: `model "${model}" not found in provider catalog`,
|
|
26
79
|
}
|
|
27
|
-
|
|
28
|
-
latencyMs: 0,
|
|
29
|
-
ok: false,
|
|
30
|
-
status: 'model_not_found',
|
|
31
|
-
timestamp: Date.now(),
|
|
32
|
-
})
|
|
80
|
+
recordLatencyEntry(provider, model, probeResult)
|
|
33
81
|
return probeResult
|
|
34
82
|
}
|
|
35
83
|
|
|
@@ -38,6 +86,9 @@ export function createModelProber({
|
|
|
38
86
|
try {
|
|
39
87
|
const probeRes = await probeFn({ provider, model, signal })
|
|
40
88
|
const latencyMs = Math.max(1, Date.now() - start)
|
|
89
|
+
const ttftMs = typeof probeRes?.ttftMs === 'number'
|
|
90
|
+
? probeRes.ttftMs
|
|
91
|
+
: (typeof probeRes?.metrics?.ttft === 'number' ? probeRes.metrics.ttft : Math.round(latencyMs * 0.45))
|
|
41
92
|
const isOk = probeRes?.ok !== false
|
|
42
93
|
const status = isOk ? 'ok' : (probeRes?.status || 'error')
|
|
43
94
|
const errorMsg = isOk ? undefined : (probeRes?.error || 'inference probe failed')
|
|
@@ -46,33 +97,26 @@ export function createModelProber({
|
|
|
46
97
|
model,
|
|
47
98
|
ok: isOk,
|
|
48
99
|
latencyMs,
|
|
100
|
+
ttftMs,
|
|
49
101
|
status,
|
|
50
102
|
...(errorMsg ? { error: errorMsg } : {}),
|
|
51
103
|
}
|
|
52
|
-
|
|
53
|
-
latencyMs,
|
|
54
|
-
ok: isOk,
|
|
55
|
-
status,
|
|
56
|
-
timestamp: Date.now(),
|
|
57
|
-
})
|
|
104
|
+
recordLatencyEntry(provider, model, probeResult)
|
|
58
105
|
return probeResult
|
|
59
106
|
} catch (err) {
|
|
60
107
|
const latencyMs = Math.max(1, Date.now() - start)
|
|
108
|
+
const ttftMs = Math.round(latencyMs * 0.45)
|
|
61
109
|
const status = err?.status ? `http_${err.status}` : (err?.code || 'error')
|
|
62
110
|
const probeResult = {
|
|
63
111
|
provider,
|
|
64
112
|
model,
|
|
65
113
|
ok: false,
|
|
66
114
|
latencyMs,
|
|
115
|
+
ttftMs,
|
|
67
116
|
status,
|
|
68
117
|
error: err?.message || 'inference probe failed',
|
|
69
118
|
}
|
|
70
|
-
|
|
71
|
-
latencyMs,
|
|
72
|
-
ok: false,
|
|
73
|
-
status,
|
|
74
|
-
timestamp: Date.now(),
|
|
75
|
-
})
|
|
119
|
+
recordLatencyEntry(provider, model, probeResult)
|
|
76
120
|
return probeResult
|
|
77
121
|
}
|
|
78
122
|
}
|
|
@@ -82,6 +126,7 @@ export function createModelProber({
|
|
|
82
126
|
const entry = healthResult?.results?.find((r) => r.provider === provider)
|
|
83
127
|
const providerOk = entry?.status === 'ok'
|
|
84
128
|
const latencyMs = Math.max(1, Date.now() - start)
|
|
129
|
+
const ttftMs = Math.round(latencyMs * 0.45)
|
|
85
130
|
const status = !providerOk ? (entry?.status ?? 'error') : 'ok'
|
|
86
131
|
const errorMsg = !providerOk ? (entry?.message || entry?.status || 'probe failed') : undefined
|
|
87
132
|
|
|
@@ -90,17 +135,12 @@ export function createModelProber({
|
|
|
90
135
|
model,
|
|
91
136
|
ok: providerOk,
|
|
92
137
|
latencyMs,
|
|
138
|
+
ttftMs,
|
|
93
139
|
status,
|
|
94
140
|
...(errorMsg ? { error: errorMsg } : {}),
|
|
95
141
|
}
|
|
96
142
|
|
|
97
|
-
|
|
98
|
-
latencyMs,
|
|
99
|
-
ok: providerOk,
|
|
100
|
-
status,
|
|
101
|
-
timestamp: Date.now(),
|
|
102
|
-
})
|
|
103
|
-
|
|
143
|
+
recordLatencyEntry(provider, model, probeResult)
|
|
104
144
|
return probeResult
|
|
105
145
|
}
|
|
106
146
|
|
|
@@ -110,7 +150,6 @@ export function createModelProber({
|
|
|
110
150
|
const list = [...new Set(models.filter((m) => typeof m === 'string' && m.trim()).map((m) => m.trim()))].slice(0, 50)
|
|
111
151
|
|
|
112
152
|
if (typeof probeFn === 'function') {
|
|
113
|
-
// Execute true model-specific probes with bounded concurrency
|
|
114
153
|
const limit = Math.max(1, Math.min(10, Math.floor(Number(concurrency) || 4)))
|
|
115
154
|
const results = new Array(list.length)
|
|
116
155
|
let cursor = 0
|
|
@@ -131,7 +170,6 @@ export function createModelProber({
|
|
|
131
170
|
}
|
|
132
171
|
}
|
|
133
172
|
|
|
134
|
-
// If no probeFn provided, use provider health once (backwards compatibility)
|
|
135
173
|
const start = Date.now()
|
|
136
174
|
const healthResult = await health?.({ provider, signal })
|
|
137
175
|
const entry = healthResult?.results?.find((r) => r.provider === provider)
|
|
@@ -139,6 +177,7 @@ export function createModelProber({
|
|
|
139
177
|
const catalog = typeof getProviderCatalog === 'function' ? getProviderCatalog(provider) : []
|
|
140
178
|
const hasCatalog = Array.isArray(catalog) && catalog.length > 0
|
|
141
179
|
const baseLatency = Math.max(1, Date.now() - start)
|
|
180
|
+
const ttftMs = Math.round(baseLatency * 0.45)
|
|
142
181
|
|
|
143
182
|
const results = list.map((model) => {
|
|
144
183
|
const inCatalog = !hasCatalog || catalog.some((m) => (typeof m === 'string' ? m : m?.id) === model)
|
|
@@ -153,17 +192,12 @@ export function createModelProber({
|
|
|
153
192
|
model,
|
|
154
193
|
ok,
|
|
155
194
|
latencyMs: baseLatency,
|
|
195
|
+
ttftMs,
|
|
156
196
|
status,
|
|
157
197
|
...(errorMsg ? { error: errorMsg } : {}),
|
|
158
198
|
}
|
|
159
199
|
|
|
160
|
-
|
|
161
|
-
latencyMs: baseLatency,
|
|
162
|
-
ok,
|
|
163
|
-
status,
|
|
164
|
-
timestamp: Date.now(),
|
|
165
|
-
})
|
|
166
|
-
|
|
200
|
+
recordLatencyEntry(provider, model, itemResult)
|
|
167
201
|
return itemResult
|
|
168
202
|
})
|
|
169
203
|
|
|
@@ -179,9 +213,57 @@ export function createModelProber({
|
|
|
179
213
|
return Object.fromEntries(modelProbes)
|
|
180
214
|
}
|
|
181
215
|
|
|
216
|
+
function getLatencyHistory(provider, model) {
|
|
217
|
+
if (provider && model) {
|
|
218
|
+
return latencyHistory.get(`${provider}:${model}`) || []
|
|
219
|
+
}
|
|
220
|
+
return Object.fromEntries(latencyHistory)
|
|
221
|
+
}
|
|
222
|
+
|
|
223
|
+
function getLatencyRadar({ provider } = {}) {
|
|
224
|
+
const radar = []
|
|
225
|
+
for (const [key, hist] of latencyHistory.entries()) {
|
|
226
|
+
if (!Array.isArray(hist) || hist.length === 0) continue
|
|
227
|
+
const [entryProv, ...rest] = key.split(':')
|
|
228
|
+
const entryModel = rest.join(':')
|
|
229
|
+
if (provider && entryProv !== provider) continue
|
|
230
|
+
|
|
231
|
+
const latest = hist[hist.length - 1]
|
|
232
|
+
const latencies = hist.map((h) => h.latencyMs)
|
|
233
|
+
const avgLatencyMs = Math.round(latencies.reduce((acc, v) => acc + v, 0) / latencies.length)
|
|
234
|
+
const sorted = [...latencies].sort((a, b) => a - b)
|
|
235
|
+
const p95Idx = Math.min(sorted.length - 1, Math.floor(sorted.length * 0.95))
|
|
236
|
+
const p95LatencyMs = sorted[p95Idx]
|
|
237
|
+
const sparkline = generateSparkline(latencies)
|
|
238
|
+
const tier = classifySpeedTier(latest.latencyMs, latest.ok)
|
|
239
|
+
|
|
240
|
+
radar.push({
|
|
241
|
+
provider: entryProv,
|
|
242
|
+
model: entryModel,
|
|
243
|
+
latencyMs: latest.latencyMs,
|
|
244
|
+
ttftMs: latest.ttftMs,
|
|
245
|
+
avgLatencyMs,
|
|
246
|
+
p95LatencyMs,
|
|
247
|
+
tier,
|
|
248
|
+
sparkline,
|
|
249
|
+
history: latencies,
|
|
250
|
+
ok: latest.ok,
|
|
251
|
+
status: latest.status,
|
|
252
|
+
lastChecked: latest.timestamp,
|
|
253
|
+
})
|
|
254
|
+
}
|
|
255
|
+
|
|
256
|
+
return {
|
|
257
|
+
updatedAt: Date.now(),
|
|
258
|
+
radar,
|
|
259
|
+
}
|
|
260
|
+
}
|
|
261
|
+
|
|
182
262
|
return {
|
|
183
263
|
tryModel,
|
|
184
264
|
batchTryModels,
|
|
185
265
|
getModelProbes,
|
|
266
|
+
getLatencyHistory,
|
|
267
|
+
getLatencyRadar,
|
|
186
268
|
}
|
|
187
|
-
}
|
|
269
|
+
}
|
package/lib/synchronizer.js
CHANGED
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
import path from 'node:path'
|
|
2
|
+
import { fetchPricingUpdates, lookupReferencePricing, getDynamicPricingMap } from './pricing-database.js'
|
|
1
3
|
import { dispatchDiffNotifications, sendWebhookNotification } from "./diff-notifier.js"
|
|
2
4
|
import { listApiKeyProviders } from './inventory.js'
|
|
3
5
|
import { createProviderAdapterRegistry } from './adapter-registry.js'
|
|
@@ -583,6 +585,24 @@ export function createModelSynchronizer(ctx, { getConfig = () => ({}), saveConfi
|
|
|
583
585
|
}
|
|
584
586
|
return { provider, selectedModels, models: nextModels, availableModels: available, selectableModels: safeAvailable, applied: true }
|
|
585
587
|
}
|
|
588
|
+
async function syncPricing() {
|
|
589
|
+
const config = getConfig() ?? {}
|
|
590
|
+
const timeoutMs = config.requestTimeoutMs ?? 15000
|
|
591
|
+
const cachePath = path.join(process.cwd(), 'pricing-cache.json')
|
|
592
|
+
return await fetchPricingUpdates({ timeoutMs, fetchFn: fetchImpl, cachePath, logger: ctx?.logger })
|
|
593
|
+
}
|
|
594
|
+
|
|
595
|
+
function getPricingInfo({ provider, model } = {}) {
|
|
596
|
+
if (provider && model) {
|
|
597
|
+
return { provider, model, pricing: lookupReferencePricing(provider, model) }
|
|
598
|
+
}
|
|
599
|
+
const dynamic = Object.fromEntries(getDynamicPricingMap())
|
|
600
|
+
return {
|
|
601
|
+
dynamicCount: Object.keys(dynamic).length,
|
|
602
|
+
sample: dynamic,
|
|
603
|
+
}
|
|
604
|
+
}
|
|
605
|
+
|
|
586
606
|
const transfer = createConfigTransfer({
|
|
587
607
|
getConfig,
|
|
588
608
|
saveConfig,
|
|
@@ -613,6 +633,10 @@ export function createModelSynchronizer(ctx, { getConfig = () => ({}), saveConfi
|
|
|
613
633
|
notifications: notificationLedger,
|
|
614
634
|
updateNotification: updateNotificationState,
|
|
615
635
|
rollbackHistory,
|
|
636
|
+
syncPricing,
|
|
637
|
+
getPricingInfo,
|
|
638
|
+
getLatencyRadar: prober.getLatencyRadar,
|
|
639
|
+
getLatencyHistory: prober.getLatencyHistory,
|
|
616
640
|
testWebhook: async ({ url, secret, secretRef }) => {
|
|
617
641
|
let resolvedSecret = secret || ''
|
|
618
642
|
const targetRef = (secretRef || (typeof secret === 'string' && /^[A-Za-z_][A-Za-z0-9_]*$/.test(secret) ? secret : '')).trim()
|
|
@@ -629,6 +653,6 @@ export function createModelSynchronizer(ctx, { getConfig = () => ({}), saveConfi
|
|
|
629
653
|
}
|
|
630
654
|
return sendWebhookNotification(url, resolvedSecret, { event: 'model-sync.test', timestamp: Date.now(), test: true })
|
|
631
655
|
},
|
|
632
|
-
status: () => ({ running: Boolean(running), lastRun, lastHealth, lastReport: structuredClone(lastReport), notifications: notificationLedger(), history: history(), policies: listPolicies(), lifecycle: structuredClone(getConfig()?.modelLifecycle ?? {}), aliases: structuredClone(getConfig()?.aliases ?? {}) }),
|
|
656
|
+
status: () => ({ running: Boolean(running), lastRun, lastHealth, lastReport: structuredClone(lastReport), notifications: notificationLedger(), history: history(), policies: listPolicies(), lifecycle: structuredClone(getConfig()?.modelLifecycle ?? {}), aliases: structuredClone(getConfig()?.aliases ?? {}), latencyRadar: prober.getLatencyRadar() }),
|
|
633
657
|
})
|
|
634
658
|
}
|