lynkr 9.6.0 → 9.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +32 -302
- package/bin/cli.js +6 -0
- package/bin/lynkr-init.js +679 -0
- package/bin/wrap.js +686 -0
- package/install.sh +26 -81
- package/package.json +4 -3
- package/scripts/build-knn-index.js +1 -1
- package/scripts/convert-routellm.py +105 -0
- package/src/api/router.js +694 -21
- package/src/auth-mode.js +116 -0
- package/src/clients/databricks.js +416 -67
- package/src/clients/prompt-cache-injection.js +15 -0
- package/src/orchestrator/index.js +120 -60
- package/src/routing/index.js +23 -4
- package/src/routing/knn-router.js +9 -2
- package/src/routing/model-tiers.js +34 -0
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Lynkr
|
|
2
2
|
|
|
3
|
-
###
|
|
3
|
+
### An LLM Gateway which optimises your token usage.
|
|
4
4
|
|
|
5
5
|
**87.6% fewer tokens on JSON tool results. 53% fewer tokens on tool-heavy requests. 171ms semantic cache hits. Zero code changes.**
|
|
6
6
|
|
|
@@ -24,107 +24,67 @@
|
|
|
24
24
|
|
|
25
25
|
---
|
|
26
26
|
|
|
27
|
-
##
|
|
27
|
+
## 🚀 New: Wrap Mode for AI Coding Tools
|
|
28
28
|
|
|
29
|
-
|
|
29
|
+
**Use Lynkr's routing with your AI coding assistant — maximize your subscription value:**
|
|
30
30
|
|
|
31
31
|
```bash
|
|
32
32
|
npm install -g lynkr
|
|
33
|
-
```
|
|
34
33
|
|
|
35
|
-
### 2. Configure Lynkr
|
|
36
34
|
|
|
37
|
-
|
|
35
|
+
# Claude Code Pro/Max
|
|
36
|
+
lynkr wrap claude
|
|
38
37
|
|
|
39
|
-
**Option A: Free & Local (Ollama) - Recommended for Testing**
|
|
40
38
|
|
|
41
|
-
```bash
|
|
42
|
-
# Install Ollama first: https://ollama.com
|
|
43
|
-
ollama pull qwen2.5-coder:latest
|
|
44
39
|
```
|
|
45
40
|
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
# Ollama Configuration
|
|
53
|
-
OLLAMA_ENDPOINT=http://localhost:11434
|
|
54
|
-
OLLAMA_MODEL=qwen2.5-coder:latest
|
|
41
|
+
**Wrapping gives you:**
|
|
42
|
+
- ✅ Tier routing (send simple tasks to free Ollama, complex to your subscription/API)
|
|
43
|
+
- ✅ TOON/RTK compression (87% token reduction on tool outputs)
|
|
44
|
+
- ✅ Semantic caching (171ms cache hits)
|
|
45
|
+
- ✅ **3-5x more usage from the same subscription limits**
|
|
46
|
+
- ✅ Works with OAuth (Claude, Copilot, Cursor) or API keys (Aider, Codex)
|
|
55
47
|
|
|
56
|
-
|
|
57
|
-
PORT=8081
|
|
48
|
+
[Full wrap guide →](docs/wrap-guide.md)
|
|
58
49
|
|
|
59
|
-
|
|
60
|
-
POLICY_MAX_STEPS=50
|
|
61
|
-
POLICY_MAX_TOOL_CALLS=100
|
|
50
|
+
---
|
|
62
51
|
|
|
63
|
-
|
|
64
|
-
POLICY_SAFE_COMMANDS_ENABLED=false
|
|
65
|
-
```
|
|
52
|
+
## Quick Start (2 Minutes)
|
|
66
53
|
|
|
67
|
-
|
|
54
|
+
### 1. Install Lynkr
|
|
68
55
|
|
|
69
56
|
```bash
|
|
70
|
-
|
|
57
|
+
npm install -g lynkr
|
|
71
58
|
```
|
|
72
59
|
|
|
73
|
-
|
|
74
|
-
```bash
|
|
75
|
-
# Provider
|
|
76
|
-
MODEL_PROVIDER=openrouter
|
|
77
|
-
OPENROUTER_API_KEY=sk-or-v1-your-key-here
|
|
78
|
-
FALLBACK_ENABLED=false
|
|
60
|
+
### 2. Configure Lynkr
|
|
79
61
|
|
|
80
|
-
|
|
81
|
-
PORT=8081
|
|
62
|
+
The fastest path is the interactive wizard:
|
|
82
63
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
POLICY_MAX_TOOL_CALLS=100
|
|
86
|
-
|
|
87
|
-
# Optional: Enable caching
|
|
88
|
-
PROMPT_CACHE_ENABLED=true
|
|
89
|
-
SEMANTIC_CACHE_ENABLED=true
|
|
64
|
+
```bash
|
|
65
|
+
lynkr init
|
|
90
66
|
```
|
|
91
67
|
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
Create/edit `.env`:
|
|
95
|
-
```bash
|
|
96
|
-
# Provider
|
|
97
|
-
MODEL_PROVIDER=bedrock
|
|
98
|
-
AWS_BEDROCK_API_KEY=your-aws-key
|
|
99
|
-
AWS_BEDROCK_MODEL_ID=anthropic.claude-3-5-sonnet-20241022-v2:0
|
|
100
|
-
FALLBACK_ENABLED=false
|
|
68
|
+
It asks four questions — usage mode (Claude Pro/Max via wrap, or direct API keys), tier picks for SIMPLE/MEDIUM/COMPLEX/REASONING across the 12 supported providers, credentials for what you chose, and a few routing-intelligence knobs — then writes a fully-populated `.env` with sensible production defaults for everything else (caching, compression, policy budgets, MCP sandbox, agents, rate limiting).
|
|
101
69
|
|
|
102
|
-
|
|
103
|
-
PORT=8081
|
|
70
|
+
Useful flags:
|
|
104
71
|
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
POLICY_MAX_TOOL_CALLS=100
|
|
72
|
+
```bash
|
|
73
|
+
lynkr init --force # overwrite an existing .env
|
|
108
74
|
```
|
|
109
75
|
|
|
110
|
-
|
|
76
|
+
See [`docs/init.md`](docs/init.md) for the full wizard reference.
|
|
111
77
|
|
|
112
|
-
|
|
113
|
-
```bash
|
|
114
|
-
# Provider
|
|
115
|
-
MODEL_PROVIDER=databricks
|
|
116
|
-
DATABRICKS_API_BASE=https://your-workspace.cloud.databricks.com
|
|
117
|
-
DATABRICKS_API_KEY=your-token
|
|
118
|
-
FALLBACK_ENABLED=false
|
|
78
|
+
If you'd rather configure by hand, the manual options below still work — copy `.env.example` to `.env` and edit it directly:
|
|
119
79
|
|
|
120
|
-
|
|
121
|
-
PORT=8081
|
|
80
|
+
**Option A: Free & Local (Ollama) - Recommended for Testing**
|
|
122
81
|
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
82
|
+
```bash
|
|
83
|
+
# Install Ollama first: https://ollama.com
|
|
84
|
+
ollama pull qwen2.5-coder:latest
|
|
126
85
|
```
|
|
127
86
|
|
|
87
|
+
|
|
128
88
|
Then start Lynkr:
|
|
129
89
|
|
|
130
90
|
```bash
|
|
@@ -133,21 +93,6 @@ lynkr start
|
|
|
133
93
|
|
|
134
94
|
### 3. Connect Your Tool
|
|
135
95
|
|
|
136
|
-
**Claude Code**
|
|
137
|
-
|
|
138
|
-
**Windows (Command Prompt):**
|
|
139
|
-
```cmd
|
|
140
|
-
set ANTHROPIC_BASE_URL=http://localhost:8081
|
|
141
|
-
set ANTHROPIC_API_KEY=dummy
|
|
142
|
-
claude "write a hello world in python"
|
|
143
|
-
```
|
|
144
|
-
|
|
145
|
-
**Linux/macOS:**
|
|
146
|
-
```bash
|
|
147
|
-
export ANTHROPIC_BASE_URL=http://localhost:8081
|
|
148
|
-
export ANTHROPIC_API_KEY=dummy
|
|
149
|
-
claude "write a hello world in python"
|
|
150
|
-
```
|
|
151
96
|
|
|
152
97
|
**Cursor IDE**
|
|
153
98
|
- Settings → Models → Override Base URL
|
|
@@ -300,225 +245,10 @@ Lynkr analyzes each request and routes it to the appropriate tier. Simple questi
|
|
|
300
245
|
|
|
301
246
|
**Result:** 70-90% of requests use cheaper/faster models. Only hard problems hit expensive models.
|
|
302
247
|
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
## Complete .env Examples
|
|
306
|
-
|
|
307
|
-
### MVP: Minimal Working Setup (Ollama)
|
|
308
|
-
|
|
309
|
-
Copy-paste ready configuration for immediate use:
|
|
310
|
-
|
|
311
|
-
```bash
|
|
312
|
-
# .env - Minimal Ollama Setup
|
|
313
|
-
|
|
314
|
-
# ============================================
|
|
315
|
-
# REQUIRED: Provider Configuration
|
|
316
|
-
# ============================================
|
|
317
|
-
MODEL_PROVIDER=ollama
|
|
318
|
-
FALLBACK_ENABLED=false
|
|
319
|
-
|
|
320
|
-
# ============================================
|
|
321
|
-
# REQUIRED: Ollama Settings
|
|
322
|
-
# ============================================
|
|
323
|
-
OLLAMA_ENDPOINT=http://localhost:11434
|
|
324
|
-
OLLAMA_MODEL=qwen2.5-coder:latest
|
|
325
|
-
|
|
326
|
-
# ============================================
|
|
327
|
-
# REQUIRED: Server Configuration
|
|
328
|
-
# ============================================
|
|
329
|
-
PORT=8081
|
|
330
|
-
HOST=0.0.0.0
|
|
331
|
-
|
|
332
|
-
# ============================================
|
|
333
|
-
# REQUIRED: Claude Code/Cursor Compatibility
|
|
334
|
-
# ============================================
|
|
335
|
-
POLICY_MAX_STEPS=50
|
|
336
|
-
POLICY_MAX_TOOL_CALLS=100
|
|
337
|
-
POLICY_SAFE_COMMANDS_ENABLED=false
|
|
338
|
-
|
|
339
|
-
# ============================================
|
|
340
|
-
# OPTIONAL: Performance (Recommended)
|
|
341
|
-
# ============================================
|
|
342
|
-
LOG_LEVEL=warn
|
|
343
|
-
LOAD_SHEDDING_ENABLED=true
|
|
344
|
-
LOAD_SHEDDING_HEAP_THRESHOLD=0.85
|
|
345
|
-
```
|
|
346
|
-
|
|
347
|
-
**Steps:**
|
|
348
|
-
1. Install Ollama: `curl -fsSL https://ollama.com/install.sh | sh`
|
|
349
|
-
2. Pull model: `ollama pull qwen2.5-coder:latest`
|
|
350
|
-
3. Copy above to `.env` in your project directory
|
|
351
|
-
4. Run: `lynkr start`
|
|
352
|
-
|
|
353
|
-
---
|
|
354
|
-
|
|
355
|
-
### Production: Cloud with Tier Routing (OpenRouter)
|
|
356
|
-
|
|
357
|
-
Optimized for cost savings with smart routing:
|
|
358
|
-
|
|
359
|
-
```bash
|
|
360
|
-
# .env - Production OpenRouter Setup
|
|
361
|
-
|
|
362
|
-
# ============================================
|
|
363
|
-
# REQUIRED: Provider Configuration
|
|
364
|
-
# ============================================
|
|
365
|
-
MODEL_PROVIDER=openrouter
|
|
366
|
-
OPENROUTER_API_KEY=sk-or-v1-your-key-here
|
|
367
|
-
FALLBACK_ENABLED=false
|
|
368
|
-
|
|
369
|
-
# ============================================
|
|
370
|
-
# REQUIRED: Server Configuration
|
|
371
|
-
# ============================================
|
|
372
|
-
PORT=8081
|
|
373
|
-
HOST=0.0.0.0
|
|
374
|
-
|
|
375
|
-
# ============================================
|
|
376
|
-
# TIER ROUTING: Smart Cost Optimization
|
|
377
|
-
# ============================================
|
|
378
|
-
# Simple queries → Cheap/fast model
|
|
379
|
-
TIER_SIMPLE=openrouter:google/gemini-flash-1.5
|
|
380
|
-
|
|
381
|
-
# Normal coding → Balanced model
|
|
382
|
-
TIER_MEDIUM=openrouter:anthropic/claude-3.5-sonnet
|
|
383
|
-
|
|
384
|
-
# Complex refactoring → Powerful model
|
|
385
|
-
TIER_COMPLEX=openrouter:anthropic/claude-opus-4
|
|
386
|
-
|
|
387
|
-
# Deep reasoning → Most capable model
|
|
388
|
-
TIER_REASONING=openrouter:anthropic/claude-opus-4
|
|
389
|
-
|
|
390
|
-
# ============================================
|
|
391
|
-
# REQUIRED: Claude Code/Cursor Compatibility
|
|
392
|
-
# ============================================
|
|
393
|
-
POLICY_MAX_STEPS=50
|
|
394
|
-
POLICY_MAX_TOOL_CALLS=100
|
|
395
|
-
POLICY_SAFE_COMMANDS_ENABLED=false
|
|
396
|
-
|
|
397
|
-
# ============================================
|
|
398
|
-
# OPTIONAL: Token Optimization (60-80% savings)
|
|
399
|
-
# ============================================
|
|
400
|
-
PROMPT_CACHE_ENABLED=true
|
|
401
|
-
SEMANTIC_CACHE_ENABLED=true
|
|
402
|
-
SEMANTIC_CACHE_THRESHOLD=0.95
|
|
403
|
-
TOOL_INJECTION_ENABLED=false
|
|
404
|
-
|
|
405
|
-
# ============================================
|
|
406
|
-
# OPTIONAL: Performance Tuning
|
|
407
|
-
# ============================================
|
|
408
|
-
LOG_LEVEL=warn
|
|
409
|
-
LOAD_SHEDDING_ENABLED=true
|
|
410
|
-
LOAD_SHEDDING_HEAP_THRESHOLD=0.85
|
|
411
|
-
```
|
|
412
|
-
|
|
413
|
-
**Expected savings:** 70-90% of requests use Gemini Flash ($). Only 10-30% use Claude Opus ($$$).
|
|
248
|
+
Tier configuration is strictly authoritative — bandit exploration is constrained to the models you've listed in `TIER_*`, and multi-turn conversations score with a recency-weighted sliding window so context isn't lost on short follow-ups. See [`docs/intent-window-routing.md`](docs/intent-window-routing.md).
|
|
414
249
|
|
|
415
250
|
---
|
|
416
251
|
|
|
417
|
-
### Enterprise: Databricks Foundation Models
|
|
418
|
-
|
|
419
|
-
For teams using Databricks Model Serving:
|
|
420
|
-
|
|
421
|
-
```bash
|
|
422
|
-
# .env - Enterprise Databricks Setup
|
|
423
|
-
|
|
424
|
-
# ============================================
|
|
425
|
-
# REQUIRED: Provider Configuration
|
|
426
|
-
# ============================================
|
|
427
|
-
MODEL_PROVIDER=databricks
|
|
428
|
-
DATABRICKS_API_BASE=https://your-workspace.cloud.databricks.com
|
|
429
|
-
DATABRICKS_API_KEY=dapi1234567890abcdef
|
|
430
|
-
FALLBACK_ENABLED=false
|
|
431
|
-
|
|
432
|
-
# ============================================
|
|
433
|
-
# REQUIRED: Model Configuration
|
|
434
|
-
# ============================================
|
|
435
|
-
# Option 1: Single model (no tier routing)
|
|
436
|
-
DATABRICKS_MODEL=databricks-meta-llama-3-1-405b-instruct
|
|
437
|
-
|
|
438
|
-
# Option 2: Tier routing (comment out above, uncomment below)
|
|
439
|
-
# TIER_SIMPLE=databricks:databricks-meta-llama-3-1-70b-instruct
|
|
440
|
-
# TIER_MEDIUM=databricks:databricks-claude-sonnet-4-5
|
|
441
|
-
# TIER_COMPLEX=databricks:databricks-claude-opus-4-6
|
|
442
|
-
# TIER_REASONING=databricks:databricks-claude-opus-4-6
|
|
443
|
-
|
|
444
|
-
# ============================================
|
|
445
|
-
# REQUIRED: Server Configuration
|
|
446
|
-
# ============================================
|
|
447
|
-
PORT=8081
|
|
448
|
-
HOST=0.0.0.0
|
|
449
|
-
|
|
450
|
-
# ============================================
|
|
451
|
-
# REQUIRED: Claude Code/Cursor Compatibility
|
|
452
|
-
# ============================================
|
|
453
|
-
POLICY_MAX_STEPS=50
|
|
454
|
-
POLICY_MAX_TOOL_CALLS=100
|
|
455
|
-
POLICY_SAFE_COMMANDS_ENABLED=false
|
|
456
|
-
|
|
457
|
-
# ============================================
|
|
458
|
-
# OPTIONAL: Enterprise Features
|
|
459
|
-
# ============================================
|
|
460
|
-
LOG_LEVEL=info
|
|
461
|
-
LOAD_SHEDDING_ENABLED=true
|
|
462
|
-
LOAD_SHEDDING_HEAP_THRESHOLD=0.85
|
|
463
|
-
|
|
464
|
-
# Optional: Metrics for monitoring
|
|
465
|
-
# PROMETHEUS_METRICS_ENABLED=true
|
|
466
|
-
```
|
|
467
|
-
|
|
468
|
-
---
|
|
469
|
-
|
|
470
|
-
### Hybrid: Local + Cloud Fallback
|
|
471
|
-
|
|
472
|
-
Use free Ollama, fallback to cloud when needed:
|
|
473
|
-
|
|
474
|
-
```bash
|
|
475
|
-
# .env - Hybrid Setup (Advanced)
|
|
476
|
-
|
|
477
|
-
# ============================================
|
|
478
|
-
# PRIMARY: Local Ollama
|
|
479
|
-
# ============================================
|
|
480
|
-
MODEL_PROVIDER=ollama
|
|
481
|
-
OLLAMA_ENDPOINT=http://localhost:11434
|
|
482
|
-
OLLAMA_MODEL=qwen2.5-coder:latest
|
|
483
|
-
|
|
484
|
-
# ============================================
|
|
485
|
-
# FALLBACK: Cloud Provider
|
|
486
|
-
# ============================================
|
|
487
|
-
FALLBACK_ENABLED=true
|
|
488
|
-
FALLBACK_PROVIDER=openrouter
|
|
489
|
-
OPENROUTER_API_KEY=sk-or-v1-your-key-here
|
|
490
|
-
|
|
491
|
-
# ============================================
|
|
492
|
-
# TIER ROUTING: Mix Local + Cloud
|
|
493
|
-
# ============================================
|
|
494
|
-
TIER_SIMPLE=ollama:qwen2.5:3b
|
|
495
|
-
TIER_MEDIUM=ollama:qwen2.5:7b
|
|
496
|
-
TIER_COMPLEX=openrouter:anthropic/claude-3.5-sonnet
|
|
497
|
-
TIER_REASONING=openrouter:anthropic/claude-opus-4
|
|
498
|
-
|
|
499
|
-
# ============================================
|
|
500
|
-
# REQUIRED: Server Configuration
|
|
501
|
-
# ============================================
|
|
502
|
-
PORT=8081
|
|
503
|
-
HOST=0.0.0.0
|
|
504
|
-
|
|
505
|
-
# ============================================
|
|
506
|
-
# REQUIRED: Claude Code/Cursor Compatibility
|
|
507
|
-
# ============================================
|
|
508
|
-
POLICY_MAX_STEPS=50
|
|
509
|
-
POLICY_MAX_TOOL_CALLS=100
|
|
510
|
-
POLICY_SAFE_COMMANDS_ENABLED=false
|
|
511
|
-
|
|
512
|
-
# ============================================
|
|
513
|
-
# OPTIONAL: Performance
|
|
514
|
-
# ============================================
|
|
515
|
-
LOG_LEVEL=warn
|
|
516
|
-
LOAD_SHEDDING_ENABLED=true
|
|
517
|
-
```
|
|
518
|
-
|
|
519
|
-
**Best of both worlds:** 80% of requests stay local (free). Complex tasks use cloud (paid).
|
|
520
|
-
|
|
521
|
-
---
|
|
522
252
|
|
|
523
253
|
## Common Issues & Fixes
|
|
524
254
|
|
package/bin/cli.js
CHANGED
|
@@ -8,11 +8,16 @@ const pkg = require('../package.json');
|
|
|
8
8
|
const SUBCOMMANDS = {
|
|
9
9
|
usage: path.join(__dirname, "lynkr-usage.js"),
|
|
10
10
|
trajectory: path.join(__dirname, "lynkr-trajectory.js"),
|
|
11
|
+
wrap: path.join(__dirname, "wrap.js"),
|
|
12
|
+
init: path.join(__dirname, "lynkr-init.js"),
|
|
11
13
|
};
|
|
12
14
|
|
|
13
15
|
const sub = process.argv[2];
|
|
14
16
|
if (sub && Object.prototype.hasOwnProperty.call(SUBCOMMANDS, sub)) {
|
|
15
17
|
process.argv.splice(2, 1); // drop the subcommand token so the script's own arg parser is happy
|
|
18
|
+
// Subcommand scripts check this to decide whether to invoke their main()
|
|
19
|
+
// when they're require()'d (vs being loaded by a test for unit-checking).
|
|
20
|
+
process.env._LYNKR_SUBCMD = sub;
|
|
16
21
|
require(SUBCOMMANDS[sub]);
|
|
17
22
|
return;
|
|
18
23
|
}
|
|
@@ -30,6 +35,7 @@ ${pkg.description}
|
|
|
30
35
|
|
|
31
36
|
Usage:
|
|
32
37
|
lynkr [options] Start the proxy server (default)
|
|
38
|
+
lynkr wrap <target> [options] Wrap CLI tools through Lynkr proxy
|
|
33
39
|
lynkr usage [options] Show AI spend report and tier-routing savings
|
|
34
40
|
lynkr trajectory [options] Export agent trajectories as JSONL training data
|
|
35
41
|
|