@dizzlkheinz/ynab-mcpb 0.16.1 → 0.17.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (92) hide show
  1. package/.code/agents/0098661e-0fa3-4990-beb9-c0cbf3f123aa/status.txt +1 -0
  2. package/.code/agents/1324/exec-call_tIpx9uV1TpARbAMZonRQm8AO.txt +757 -0
  3. package/.code/agents/1572/exec-call_GjVFBFOWcY7lE0idc5nWlLNh.txt +781 -0
  4. package/.code/agents/1846/exec-call_1YNAVD18RjrMN7JnfkkQhUP3.txt +766 -0
  5. package/.code/agents/1846/exec-call_lh3lDzE4WJAh1lFiomiiZ73D.txt +766 -0
  6. package/.code/agents/2038/exec-call_DYwOukaYsL8VCONWmV2rUW5u.txt +766 -0
  7. package/.code/agents/2038/exec-call_c7fOQ7UrpVcTtvdfGBRM146V.txt +652 -0
  8. package/.code/agents/2038/exec-call_ySNyq9Mm55jWE480s54r5QcA.txt +766 -0
  9. package/.code/agents/2256/exec-call_AtPcRWPmFPMcmX6qOFm1fCEY.txt +766 -0
  10. package/.code/agents/2454/exec-call_aFJpupwjfZeOBm7ixI5Vc8z2.txt +766 -0
  11. package/.code/agents/2454/exec-call_wogZ4HfXTodTEXvdgXlVUBpv.txt +766 -0
  12. package/.code/agents/2e905864-aa07-4314-bcf9-c5b32277e4ac/result.txt +36 -0
  13. package/.code/agents/3073/exec-call_Peeagc9DxGYLgE6pNdMZhqIE.txt +766 -0
  14. package/.code/agents/3073/exec-call_d2YSE3hXF08KRSoUM3qd8Z3x.txt +766 -0
  15. package/.code/agents/335aa031-466d-4fb7-925f-3cd864e264d0/result.txt +191 -0
  16. package/.code/agents/3364/exec-call_NbhIrsM5HhyDZDmJZG5CuCYL.txt +766 -0
  17. package/.code/agents/3364/exec-call_cKtJg0NrXiwXEFwlsE3uPZRA.txt +766 -0
  18. package/.code/agents/36d98414-5cde-4d9d-9a67-a240a18c1f07/result.txt +189 -0
  19. package/.code/agents/4604e866-b7b8-44f5-992f-2f683b0a523b/status.txt +1 -0
  20. package/.code/agents/5f8dc01c-47b3-4163-b0b3-aa31be89fcdc/status.txt +1 -0
  21. package/.code/agents/7/exec-call_HltHpkDox0Zm1vGEjdksUgpE.txt +1120 -0
  22. package/.code/agents/7/exec-call_LCATrOPPAgbxW9Q1z0XaVi2E.txt +2646 -0
  23. package/.code/agents/7/exec-call_W8DeRfNG9hvbgVFvf0clBf6R.txt +2646 -0
  24. package/.code/agents/94a0ddf3-a304-4ec3-913e-3cceef509948/error.txt +1 -0
  25. package/.code/agents/e2c752b7-711d-423a-af57-f53c809deb84/result.txt +160 -0
  26. package/.code/agents/e6601719-c31f-4a0e-8c71-d70787d0ab71/status.txt +1 -0
  27. package/.code/agents/f250b7ed-5bd5-4036-aa8c-ce63caee7d61/result.txt +20 -0
  28. package/AGENTS.md +1 -36
  29. package/CLAUDE.md +28 -43
  30. package/NUL +0 -1
  31. package/README.md +8 -10
  32. package/dist/bundle/index.cjs +41 -41
  33. package/dist/server/YNABMCPServer.js +28 -381
  34. package/dist/server/config.d.ts +2 -0
  35. package/dist/server/config.js +1 -0
  36. package/dist/tools/accountTools.d.ts +2 -0
  37. package/dist/tools/accountTools.js +45 -0
  38. package/dist/tools/adapters.d.ts +12 -0
  39. package/dist/tools/adapters.js +25 -0
  40. package/dist/tools/budgetTools.d.ts +2 -0
  41. package/dist/tools/budgetTools.js +30 -0
  42. package/dist/tools/categoryTools.d.ts +2 -0
  43. package/dist/tools/categoryTools.js +45 -0
  44. package/dist/tools/monthTools.d.ts +2 -0
  45. package/dist/tools/monthTools.js +32 -0
  46. package/dist/tools/payeeTools.d.ts +2 -0
  47. package/dist/tools/payeeTools.js +32 -0
  48. package/dist/tools/reconciliation/index.d.ts +2 -0
  49. package/dist/tools/reconciliation/index.js +33 -0
  50. package/dist/tools/schemas/common.d.ts +3 -0
  51. package/dist/tools/schemas/common.js +3 -0
  52. package/dist/tools/schemas/outputs/comparisonOutputs.d.ts +1 -1
  53. package/dist/tools/transactionTools.d.ts +2 -0
  54. package/dist/tools/transactionTools.js +124 -0
  55. package/dist/tools/utilityTools.d.ts +3 -1
  56. package/dist/tools/utilityTools.js +32 -2
  57. package/dist/types/index.d.ts +1 -0
  58. package/dist/types/toolRegistration.d.ts +27 -0
  59. package/dist/types/toolRegistration.js +1 -0
  60. package/package.json +2 -2
  61. package/scripts/run-domain-integration-tests.js +4 -1
  62. package/src/__tests__/workflows.e2e.test.ts +1 -7
  63. package/src/server/YNABMCPServer.ts +33 -519
  64. package/src/server/__tests__/toolRegistration.test.ts +236 -0
  65. package/src/server/config.ts +1 -0
  66. package/src/tools/__tests__/adapters.test.ts +113 -0
  67. package/src/tools/__tests__/utilityTools.test.ts +7 -7
  68. package/src/tools/accountTools.ts +53 -0
  69. package/src/tools/adapters.ts +74 -0
  70. package/src/tools/budgetTools.ts +37 -0
  71. package/src/tools/categoryTools.ts +53 -0
  72. package/src/tools/monthTools.ts +39 -0
  73. package/src/tools/payeeTools.ts +39 -0
  74. package/src/tools/reconciliation/index.ts +45 -0
  75. package/src/tools/schemas/common.ts +18 -0
  76. package/src/tools/transactionTools.ts +140 -0
  77. package/src/tools/utilityTools.ts +42 -2
  78. package/src/types/index.ts +3 -0
  79. package/src/types/toolRegistration.ts +88 -0
  80. package/.github/workflows/pr-description-check.yml +0 -88
  81. package/docs/README.md +0 -72
  82. package/docs/getting-started/CONFIGURATION.md +0 -175
  83. package/docs/getting-started/INSTALLATION.md +0 -333
  84. package/docs/getting-started/QUICKSTART.md +0 -282
  85. package/docs/guides/ARCHITECTURE.md +0 -533
  86. package/docs/guides/DEPLOYMENT.md +0 -189
  87. package/docs/guides/INTEGRATION_TESTING.md +0 -730
  88. package/docs/guides/TESTING.md +0 -591
  89. package/docs/reconciliation-flow.md +0 -83
  90. package/docs/reference/EXAMPLES.md +0 -946
  91. package/docs/reference/TOOLS.md +0 -348
  92. package/docs/reference/TROUBLESHOOTING.md +0 -481
@@ -1,730 +0,0 @@
1
- # Integration Testing Strategy for YNAB MCP Server
2
-
3
- ## Philosophy
4
-
5
- **Integration tests should test real API interactions, not mocks.** This project uses a tiered testing strategy that works within YNAB's rate limits while maintaining test quality and fast feedback loops.
6
-
7
- ## YNAB API Constraints
8
-
9
- - **Rate Limit**: 200 requests per access token per hour (rolling window)
10
- - **No Official Sandbox**: YNAB does not provide a test/sandbox environment for API development
11
- - **Real Data**: Tests run against real YNAB budgets (recommend using a dedicated test budget)
12
-
13
- ---
14
-
15
- ## Testing Tiers
16
-
17
- ### Tier 1: Core/Smoke Tests (~10-15 API calls, <3 minutes)
18
-
19
- **Purpose**: Essential functionality verification during active development
20
-
21
- **What's Tested**:
22
- - List budgets
23
- - Get single budget by ID
24
- - List accounts for a budget
25
- - Basic transaction CRUD (create, read, update, delete - one of each)
26
- - Get user info
27
-
28
- **When to Run**:
29
- - Before every commit
30
- - During active development (frequent)
31
- - In CI/CD pipelines
32
-
33
- **Rate Limit Impact**: Negligible (~10-15 calls = ~7-8% of hourly limit)
34
-
35
- **NPM Script**: `npm run test:integration:core`
36
-
37
- ---
38
-
39
- ### Tier 2: Domain Tests (~30-50 API calls per domain, 5-10 minutes)
40
-
41
- **Purpose**: Comprehensive testing of specific feature areas you're actively working on
42
-
43
- **Domains**:
44
- - **Budget Tools**: Budget listing, retrieval, settings
45
- - **Account Tools**: Account management, balances, reconciliation
46
- - **Transaction Tools**: Full CRUD, bulk operations, filtering
47
- - **Category Tools**: Category management, activity tracking
48
- - **Payee Tools**: Payee management and transactions
49
- - **Month Tools**: Monthly budget data, rollover calculations
50
- - **Delta Operations**: Server knowledge tracking, incremental updates
51
- - **Reconciliation**: Account reconciliation workflows, recommendation engine
52
-
53
- **When to Run**:
54
- - When working on a specific domain (selective testing)
55
- - Before committing changes to a domain
56
- - In PR validation for changed domains only
57
-
58
- **Rate Limit Impact**: Moderate (one domain = ~15-25% of hourly limit)
59
-
60
- **NPM Scripts**:
61
- - `npm run test:integration:budgets`
62
- - `npm run test:integration:accounts`
63
- - `npm run test:integration:transactions`
64
- - `npm run test:integration:categories`
65
- - `npm run test:integration:payees`
66
- - `npm run test:integration:reconciliation`
67
- - `npm run test:integration:delta`
68
-
69
- **Throttling**: Light delays between test files (5-10 seconds) to avoid bursts
70
-
71
- ---
72
-
73
- ### Tier 3: Full Integration Suite (~200 API calls, 2-3 hours)
74
-
75
- **Purpose**: Comprehensive validation of entire system before releases or major changes
76
-
77
- **What's Tested**: All integration tests across all domains
78
-
79
- **When to Run**:
80
- - Before releases (manual trigger)
81
- - Nightly or weekly (scheduled)
82
- - After major refactoring
83
- - On-demand for comprehensive validation
84
-
85
- **Rate Limit Impact**: Uses full hourly quota (~100% of rate limit)
86
-
87
- **NPM Script**: `npm run test:integration:full`
88
-
89
- **Throttling**: Intelligent rate-limit-aware pacing:
90
- - Monitors API rate limit headers (`X-Rate-Limit-Remaining`)
91
- - Automatically delays test execution to stay under 200 requests/hour
92
- - Spreads tests over 2-3 hours with smart scheduling
93
- - Safe to run unattended - will never hit rate limits
94
-
95
- ---
96
-
97
- ## Test Organization
98
-
99
- ### File Structure
100
-
101
- ```
102
- src/
103
- ├── __tests__/
104
- │ ├── unit/ # Unit tests (mocked, no API)
105
- │ └── testUtils.ts # Shared test utilities
106
- ├── server/__tests__/
107
- │ ├── *.test.ts # Unit tests
108
- │ └── *.integration.test.ts # Integration tests (tagged by tier)
109
- └── tools/__tests__/
110
- ├── *.test.ts # Unit tests
111
- ├── *.integration.test.ts # Integration tests (tagged by tier)
112
- └── *.delta.integration.test.ts # Delta-specific integration tests
113
- ```
114
-
115
- ### Test Tagging
116
-
117
- Integration tests are tagged with metadata to enable tier-based filtering:
118
-
119
- ```typescript
120
- import { describe, it, expect } from 'vitest';
121
-
122
- describe('Budget Tools Integration', () => {
123
- it('should list budgets', {
124
- meta: { tier: 'core', domain: 'budgets' }
125
- }, async () => {
126
- // Test implementation
127
- });
128
-
129
- it('should get budget settings', {
130
- meta: { tier: 'domain', domain: 'budgets' }
131
- }, async () => {
132
- // Test implementation
133
- });
134
- });
135
- ```
136
-
137
- ---
138
-
139
- ## Rate Limit Management
140
-
141
- ### Throttling Implementation
142
-
143
- **Core Tests**: No throttling needed (fast execution)
144
-
145
- **Domain Tests**: Light throttling between test files
146
- ```typescript
147
- // Delay between test files in same domain
148
- const DOMAIN_TEST_DELAY_MS = 5000; // 5 seconds
149
- ```
150
-
151
- **Full Suite**: Intelligent throttling based on rate limit headers
152
- ```typescript
153
- // Rate limit aware test scheduler
154
- class RateLimitScheduler {
155
- async scheduleTest(testFn: () => Promise<void>) {
156
- const remainingCalls = this.getRemainingFromHeaders();
157
- const estimatedCallsInTest = this.estimateAPICalls(testFn);
158
-
159
- if (remainingCalls < estimatedCallsInTest + 20) { // 20 call buffer
160
- const waitTime = this.calculateWaitTime();
161
- console.log(`Rate limit approaching, waiting ${waitTime}ms...`);
162
- await this.sleep(waitTime);
163
- }
164
-
165
- await testFn();
166
- }
167
-
168
- private calculateWaitTime(): number {
169
- // Calculate time until rate limit window resets
170
- // Based on X-Rate-Limit-Reset header
171
- return this.getResetTime() - Date.now();
172
- }
173
- }
174
- ```
175
-
176
- ### Best Practices
177
-
178
- 1. **Monitor Rate Limit Headers**: Always check `X-Rate-Limit-Remaining` and `X-Rate-Limit-Reset` in responses
179
- 2. **Fail Gracefully**: If rate limit hit, pause tests and resume after reset window
180
- 3. **Estimate API Calls**: Track approximate API calls per test for scheduling
181
- 4. **Use Delta Requests**: Prefer delta/incremental API calls when possible to reduce response size
182
- 5. **Cache Aggressively**: Use cached data in tests when fresh data isn't required
183
-
184
- ---
185
-
186
- ## Vitest Configuration
187
-
188
- ### Project Setup
189
-
190
- ```typescript
191
- // vitest.config.ts
192
- import { defineConfig } from 'vitest/config';
193
-
194
- const integrationFiles = ['src/**/*.integration.test.ts'];
195
-
196
- export default defineConfig({
197
- test: {
198
- environment: 'node',
199
- globals: true,
200
- setupFiles: ['src/__tests__/setup.ts'],
201
- projects: [
202
- {
203
- test: {
204
- name: 'unit',
205
- include: ['src/**/*.{test,spec}.ts'],
206
- exclude: [
207
- 'src/**/*.integration.test.ts',
208
- 'src/**/*.e2e.test.ts',
209
- 'src/server/__tests__/YNABMCPServer.test.ts',
210
- ],
211
- },
212
- },
213
- {
214
- test: {
215
- name: 'integration:core',
216
- include: integrationFiles,
217
- env: { INTEGRATION_TEST_TIER: 'core' },
218
- testTimeout: 30000,
219
- hookTimeout: 10000,
220
- },
221
- },
222
- {
223
- test: {
224
- name: 'integration:domain',
225
- include: integrationFiles,
226
- env: { INTEGRATION_TEST_TIER: 'domain' },
227
- testTimeout: 60000,
228
- hookTimeout: 15000,
229
- },
230
- },
231
- {
232
- test: {
233
- name: 'integration:full',
234
- include: integrationFiles,
235
- env: { INTEGRATION_TEST_TIER: 'full' },
236
- testTimeout: 120000,
237
- hookTimeout: 30000,
238
- fileParallelism: false,
239
- maxWorkers: 1,
240
- },
241
- },
242
- {
243
- test: {
244
- name: 'e2e',
245
- include: ['src/**/*.e2e.test.ts'],
246
- },
247
- },
248
- ],
249
- },
250
- });
251
- ```
252
-
253
- ---
254
-
255
- ## NPM Scripts
256
-
257
- ```json
258
- {
259
- "scripts": {
260
- "test": "vitest run --project unit",
261
- "test:unit": "vitest run --project unit",
262
- "test:watch": "vitest --project unit",
263
-
264
- "test:integration": "npm run test:integration:core",
265
- "test:integration:core": "vitest run --project integration:core",
266
- "test:integration:full": "node scripts/run-throttled-integration-tests.js",
267
-
268
- "test:integration:budgets": "node scripts/run-domain-integration-tests.js budgets",
269
- "test:integration:accounts": "node scripts/run-domain-integration-tests.js accounts",
270
- "test:integration:transactions": "node scripts/run-domain-integration-tests.js transactions",
271
- "test:integration:categories": "node scripts/run-domain-integration-tests.js categories",
272
- "test:integration:payees": "node scripts/run-domain-integration-tests.js payees",
273
- "test:integration:months": "node scripts/run-domain-integration-tests.js months",
274
- "test:integration:reconciliation": "node scripts/run-domain-integration-tests.js reconciliation",
275
- "test:integration:delta": "node scripts/run-domain-integration-tests.js delta",
276
-
277
- "test:all": "npm run test:unit && npm run test:integration:core",
278
- "test:coverage": "vitest run --coverage --project unit"
279
- }
280
- }
281
- ```
282
-
283
- Each domain script routes through `scripts/run-domain-integration-tests.js`, which sets the `INTEGRATION_TEST_DOMAINS` environment variable so only tests tagged with those domains execute. You can target multiple domains at once (for example `node scripts/run-domain-integration-tests.js budgets accounts`) or pass additional Vitest flags after `--`.
284
-
285
- ---
286
-
287
- ## CI/CD Integration
288
-
289
- ### GitHub Actions Example
290
-
291
- ```yaml
292
- name: CI Tests
293
-
294
- on: [push, pull_request]
295
-
296
- jobs:
297
- unit-tests:
298
- runs-on: ubuntu-latest
299
- steps:
300
- - uses: actions/checkout@v4
301
- - uses: actions/setup-node@v4
302
- with:
303
- node-version: '20'
304
- - run: npm ci
305
- - run: npm run test:unit
306
-
307
- integration-core:
308
- runs-on: ubuntu-latest
309
- needs: unit-tests
310
- steps:
311
- - uses: actions/checkout@v4
312
- - uses: actions/setup-node@v4
313
- with:
314
- node-version: '20'
315
- - run: npm ci
316
- - run: npm run test:integration:core
317
- env:
318
- YNAB_ACCESS_TOKEN: ${{ secrets.YNAB_ACCESS_TOKEN }}
319
-
320
- integration-full:
321
- runs-on: ubuntu-latest
322
- # Only run on schedule (nightly) or manual trigger
323
- if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
324
- steps:
325
- - uses: actions/checkout@v4
326
- - uses: actions/setup-node@v4
327
- with:
328
- node-version: '20'
329
- - run: npm ci
330
- - run: npm run test:integration:full
331
- env:
332
- YNAB_ACCESS_TOKEN: ${{ secrets.YNAB_ACCESS_TOKEN }}
333
- ```
334
-
335
- **CI Strategy**:
336
- - **Every Push**: Unit tests only
337
- - **Every PR**: Unit tests + core integration tests (~10-15 API calls)
338
- - **Nightly/Weekly**: Full integration suite (scheduled)
339
- - **Manual**: Full integration suite (on-demand via workflow_dispatch)
340
-
341
- ---
342
-
343
- ## Environment Variables
344
-
345
- ```bash
346
- # .env.test
347
- # YNAB API Configuration
348
- YNAB_ACCESS_TOKEN=your_token_here
349
-
350
- # Test Configuration
351
- TEST_BUDGET_ID=optional_specific_budget_id
352
- TEST_ACCOUNT_ID=optional_specific_account_id
353
-
354
- # Integration Test Behavior
355
- INTEGRATION_TEST_TIER=core # core | domain | full
356
- INTEGRATION_TEST_DOMAINS= # comma-separated domains for domain tier
357
- SKIP_INTEGRATION_TESTS=false # Set to true to skip all integration tests
358
- INTEGRATION_THROTTLE_MS=5000 # Delay between domain tests (milliseconds)
359
-
360
- # Rate Limit Configuration
361
- RATE_LIMIT_PER_HOUR=200 # Base rate limit before applying buffer
362
- RATE_LIMIT_BUFFER=20 # Reserve N calls before throttling
363
- RATE_LIMIT_WINDOW_MS=3600000 # Window length (1 hour)
364
- RATE_LIMIT_MAX_WAIT_MS=3600000 # Max wait time for rate limit reset (1 hour)
365
- ```
366
-
367
- ---
368
-
369
- ## Tier Assignment Guidelines
370
-
371
- ### Core Tier (Fast, Frequent)
372
-
373
- Assign tests to **core** tier if they:
374
- - ✅ Test fundamental, critical-path functionality
375
- - ✅ Are needed for basic confidence before committing
376
- - ✅ Make 1-2 API calls per test
377
- - ✅ Run in under 5 seconds (including network latency)
378
- - ✅ Don't depend on complex state or multiple resources
379
-
380
- **Example Core Tests**:
381
- - `GET /budgets` - List all budgets
382
- - `GET /budgets/{budget_id}` - Get single budget
383
- - `GET /budgets/{budget_id}/accounts` - List accounts
384
- - `POST /budgets/{budget_id}/transactions` - Create transaction
385
- - `GET /user` - Get user info
386
-
387
- ### Domain Tier (Selective, Thorough)
388
-
389
- Assign tests to **domain** tier if they:
390
- - ✅ Test comprehensive functionality within a specific domain
391
- - ✅ Are needed when working on that specific feature area
392
- - ✅ Make 3-10 API calls per test
393
- - ✅ Test edge cases, error handling, or complex workflows
394
- - ✅ May depend on multiple resources or state
395
-
396
- **Example Domain Tests**:
397
- - Full transaction CRUD workflow
398
- - Account reconciliation with recommendations
399
- - Category group management and reordering
400
- - Delta request handling with server_knowledge
401
- - Bulk transaction operations
402
- - Month-to-month rollover calculations
403
-
404
- ### Full Suite (Comprehensive, Scheduled)
405
-
406
- All integration tests run in **full** tier, including:
407
- - ✅ All core tests
408
- - ✅ All domain tests
409
- - ✅ Performance tests
410
- - ✅ Stress tests
411
- - ✅ Long-running scenarios
412
-
413
- ---
414
-
415
- ## Delta Integration Tests
416
-
417
- Delta tests (`*.delta.integration.test.ts`) require special handling due to their stateful nature:
418
-
419
- ### Challenge
420
- Delta requests track `server_knowledge` - a monotonically increasing value that represents server state. Sequential delta requests depend on previous responses.
421
-
422
- ### Strategy
423
- 1. **Initialize Fresh**: Each delta test starts with a fresh API client (no cached server_knowledge)
424
- 2. **Test Sequence**: Test both initial fetch AND subsequent delta request in same test
425
- 3. **Validate Incremental**: Verify that delta responses are smaller than full responses
426
- 4. **Reset State**: Clean up any created test data to avoid polluting future delta tests
427
-
428
- ### Example Delta Test
429
-
430
- ```typescript
431
- describe('Delta Integration', () => {
432
- it('should fetch initial data and then delta updates', {
433
- meta: { tier: 'domain', domain: 'delta' }
434
- }, async () => {
435
- // Initial fetch (no server_knowledge)
436
- const initial = await api.budgets.getBudgetById(budgetId);
437
- const serverKnowledge = initial.data.server_knowledge;
438
-
439
- // Create a change
440
- await api.transactions.createTransaction(budgetId, { /* ... */ });
441
-
442
- // Delta fetch (with server_knowledge)
443
- const delta = await api.budgets.getBudgetById(budgetId, serverKnowledge);
444
-
445
- // Verify delta is smaller than full response
446
- expect(delta.data.transactions.length).toBeLessThan(initial.data.transactions.length);
447
- expect(delta.data.server_knowledge).toBeGreaterThan(serverKnowledge);
448
- });
449
- });
450
- ```
451
-
452
- ---
453
-
454
- ## Throttled Test Runner
455
-
456
- The full integration suite uses a custom test runner that intelligently throttles API calls:
457
-
458
- ```typescript
459
- // scripts/run-throttled-integration-tests.js
460
- import { spawn } from 'child_process';
461
- import { readFileSync } from 'fs';
462
-
463
- class ThrottledTestRunner {
464
- constructor() {
465
- this.rateLimit = 200; // requests per hour
466
- this.rateLimitWindow = 3600000; // 1 hour in ms
467
- this.buffer = 20; // safety buffer
468
- this.requestHistory = [];
469
- }
470
-
471
- async run() {
472
- // Get all integration test files
473
- const testFiles = this.getIntegrationTests();
474
-
475
- for (const testFile of testFiles) {
476
- await this.runWithThrottling(testFile);
477
- }
478
- }
479
-
480
- async runWithThrottling(testFile) {
481
- // Check if we need to throttle
482
- const recentRequests = this.getRecentRequests();
483
-
484
- if (recentRequests >= this.rateLimit - this.buffer) {
485
- const waitTime = this.calculateWaitTime();
486
- console.log(`⏳ Rate limit approaching (${recentRequests}/${this.rateLimit} calls)`);
487
- console.log(` Waiting ${Math.round(waitTime / 60000)} minutes...`);
488
- await this.sleep(waitTime);
489
- }
490
-
491
- // Run test
492
- console.log(`▶️ Running ${testFile}...`);
493
- await this.runTest(testFile);
494
-
495
- // Track request count (estimate based on test file)
496
- const estimatedCalls = this.estimateAPICalls(testFile);
497
- this.requestHistory.push({
498
- timestamp: Date.now(),
499
- calls: estimatedCalls,
500
- });
501
- }
502
-
503
- getRecentRequests() {
504
- const cutoff = Date.now() - this.rateLimitWindow;
505
- this.requestHistory = this.requestHistory.filter(r => r.timestamp > cutoff);
506
- return this.requestHistory.reduce((sum, r) => sum + r.calls, 0);
507
- }
508
-
509
- calculateWaitTime() {
510
- if (this.requestHistory.length === 0) return 0;
511
-
512
- const oldestRequest = this.requestHistory[0];
513
- const timeUntilExpiry = (oldestRequest.timestamp + this.rateLimitWindow) - Date.now();
514
-
515
- return Math.max(timeUntilExpiry, 60000); // At least 1 minute
516
- }
517
-
518
- estimateAPICalls(testFile) {
519
- // Parse test file for API call estimates
520
- // Could use static analysis or embedded metadata
521
- // For now, use conservative estimates based on test type
522
-
523
- if (testFile.includes('delta')) return 15;
524
- if (testFile.includes('reconciliation')) return 25;
525
- if (testFile.includes('transaction')) return 12;
526
- if (testFile.includes('budget')) return 8;
527
-
528
- return 10; // default estimate
529
- }
530
-
531
- async runTest(testFile) {
532
- return new Promise((resolve, reject) => {
533
- const proc = spawn('npx', ['vitest', 'run', testFile], {
534
- stdio: 'inherit',
535
- env: { ...process.env },
536
- });
537
-
538
- proc.on('close', (code) => {
539
- if (code === 0) resolve();
540
- else reject(new Error(`Test failed with code ${code}`));
541
- });
542
- });
543
- }
544
-
545
- sleep(ms) {
546
- return new Promise(resolve => setTimeout(resolve, ms));
547
- }
548
-
549
- getIntegrationTests() {
550
- // Use glob or similar to find all *.integration.test.ts files
551
- // Return sorted array of test file paths
552
- }
553
- }
554
-
555
- // Run it
556
- const runner = new ThrottledTestRunner();
557
- runner.run().catch(console.error);
558
- ```
559
-
560
- ---
561
-
562
- ## Migration Path
563
-
564
- ### Current State (v0.10.0)
565
- - ✅ 21 integration test files exist
566
- - ⚠️ All tests hit real API without throttling
567
- - ⚠️ No tier organization
568
- - ⚠️ Rate limits cause test failures
569
-
570
- ### Target State
571
- - ✅ Integration tests organized into 3 tiers
572
- - ✅ Core tests run frequently (CI, pre-commit)
573
- - ✅ Domain tests run selectively (feature work)
574
- - ✅ Full suite runs scheduled/on-demand with throttling
575
- - ✅ CI runs only unit + core tests
576
- - ✅ Never hit rate limits
577
-
578
- ### Migration Steps
579
-
580
- 1. **Tag Existing Tests** (1-2 hours)
581
- - Add `meta: { tier, domain }` to all integration tests
582
- - Identify 8-10 tests for core tier
583
- - Organize remaining tests by domain
584
-
585
- 2. **Update Vitest Config** (30 minutes)
586
- - Create `integration:core`, `integration:domain`, `integration:full` projects
587
- - Configure test filtering by tier
588
- - Set appropriate timeouts per tier
589
-
590
- 3. **Create NPM Scripts** (15 minutes)
591
- - Add tier-specific test scripts
592
- - Add domain-specific test scripts
593
- - Update default `test:integration` to run core tier
594
-
595
- 4. **Implement Throttled Runner** (2-3 hours)
596
- - Create `scripts/run-throttled-integration-tests.js`
597
- - Implement rate limit tracking and scheduling
598
- - Add API call estimation logic
599
- - Test with subset of integration tests
600
-
601
- 5. **Update CI Configuration** (30 minutes)
602
- - Configure CI to run unit + core only on PRs
603
- - Set up scheduled full suite runs (nightly/weekly)
604
- - Add manual trigger for full suite
605
-
606
- 6. **Document Workflow** (1 hour)
607
- - Update developer documentation
608
- - Create examples for each tier
609
- - Document when to run which tier
610
-
611
- 7. **Validate** (2-3 hours)
612
- - Run core tier locally (should be fast)
613
- - Run domain tiers selectively
614
- - Run full suite with throttling (validate no rate limit hits)
615
- - Verify CI runs successfully
616
-
617
- **Total Effort**: ~8-12 hours
618
-
619
- ---
620
-
621
- ## Best Practices
622
-
623
- ### 1. Test Data Management
624
- - **Use Dedicated Test Budget**: Create a YNAB budget specifically for testing
625
- - **Consistent Test Data**: Use predictable account/category names for easier test writing
626
- - **Clean Up After Tests**: Delete created transactions/payees to keep budget clean
627
- - **Idempotent Tests**: Tests should be repeatable without manual cleanup
628
-
629
- ### 2. Rate Limit Awareness
630
- - **Monitor Headers**: Always log rate limit headers during full suite runs
631
- - **Buffer Safety**: Never use more than 180 calls in full suite (leave 20 call buffer)
632
- - **Graceful Degradation**: If rate limit hit, pause and resume (don't fail)
633
- - **Estimate Conservatively**: Overestimate API calls per test for safer throttling
634
-
635
- ### 3. Test Organization
636
- - **Core = Critical Path**: Only include must-have functionality in core tier
637
- - **Domain = Feature Work**: Group tests by domain for selective running
638
- - **Full = Scheduled**: Accept that full suite is slow, run overnight/weekly
639
- - **Tag Consistently**: Use consistent tier/domain tags for filtering
640
-
641
- ### 4. CI/CD Strategy
642
- - **Fast Feedback**: Unit + core tests should complete in <5 minutes
643
- - **PR Validation**: Core tests catch regressions without burning rate limits
644
- - **Scheduled Deep Tests**: Full suite runs when you're not waiting for results
645
- - **Manual Override**: Allow manual full suite runs for pre-release validation
646
-
647
- ---
648
-
649
- ## Troubleshooting
650
-
651
- ### Rate Limit Exceeded During Tests
652
-
653
- **Symptoms**: Tests fail with 429 responses or HTML error pages
654
-
655
- **Solutions**:
656
- 1. Check recent API usage: `grep "X-Rate-Limit-Remaining" test-output.log`
657
- 2. Wait for rate limit window to reset (check `X-Rate-Limit-Reset` header)
658
- 3. Reduce test scope: Run domain tests instead of full suite
659
- 4. Increase throttling: Adjust `INTEGRATION_THROTTLE_MS` environment variable
660
-
661
- ### Tests Failing Inconsistently
662
-
663
- **Symptoms**: Same test passes/fails on different runs
664
-
665
- **Possible Causes**:
666
- - Network issues (timeouts, latency)
667
- - YNAB API throttling (not rate limit, but request pacing)
668
- - State pollution (previous test didn't clean up)
669
- - Delta state issues (server_knowledge out of sync)
670
-
671
- **Solutions**:
672
- - Increase test timeouts: `testTimeout: 60000`
673
- - Add retry logic for network errors
674
- - Ensure tests clean up created resources
675
- - Reset delta state between tests
676
-
677
- ### Full Suite Takes Too Long
678
-
679
- **Symptoms**: Full suite exceeds expected 2-3 hour window
680
-
681
- **Solutions**:
682
- - Reduce throttling buffer: Lower `RATE_LIMIT_BUFFER` (but risk hitting limits)
683
- - Optimize tests: Combine multiple assertions into single API call
684
- - Remove redundant tests: Eliminate duplicate coverage
685
- - Use delta requests: Prefer incremental fetches over full data
686
-
687
- ### CI Running Out of Rate Limits
688
-
689
- **Symptoms**: CI fails due to rate limits on every PR
690
-
691
- **Solutions**:
692
- - Verify CI only runs core tier: Check workflow uses `test:integration:core`
693
- - Reduce core tier tests: Move less critical tests to domain tier
694
- - Use separate YNAB token for CI: Isolate CI rate limits from dev
695
- - Implement CI-level throttling: Add delays between workflow runs
696
-
697
- ---
698
-
699
- ## Resources
700
-
701
- ### YNAB API
702
- - **API Documentation**: https://api.youneedabudget.com/
703
- - **Rate Limits**: 200 requests/hour per token (rolling window)
704
- - **SDK**: https://github.com/ynab/ynab-sdk-js
705
- - **Support**: https://support.youneedabudget.com/
706
-
707
- ### Testing Tools
708
- - **Vitest**: https://vitest.dev/
709
- - **Vitest Projects**: https://vitest.dev/guide/workspace
710
- - **GitHub Actions**: https://docs.github.com/en/actions
711
-
712
- ### Related Documentation
713
- - [Testing Guide](./TESTING.md) - Comprehensive testing documentation
714
- - [Build Guide](../development/BUILD.md) - Build and development workflow
715
- - [Deployment Guide](./DEPLOYMENT.md) - Deployment and packaging
716
-
717
- ---
718
-
719
- ## Conclusion
720
-
721
- This tiered integration testing strategy provides:
722
-
723
- ✅ **Fast Feedback**: Core tests run in <3 minutes for quick validation
724
- ✅ **Selective Testing**: Domain tests focus on areas you're actively developing
725
- ✅ **Comprehensive Coverage**: Full suite validates entire system before releases
726
- ✅ **Rate Limit Safety**: Intelligent throttling prevents API limit exhaustion
727
- ✅ **CI/CD Friendly**: Lightweight core tests don't burn rate limits on every PR
728
- ✅ **Real API Testing**: No mocks - tests validate actual YNAB API behavior
729
-
730
- **The key insight**: Not all integration tests need to run all the time. Organize tests by purpose and run them at the right frequency to balance speed, coverage, and rate limits.