@serve.zone/dcrouter 2.12.4 → 5.0.2

package/readme.hints.md

@@ -1,28 +1,274 @@
 # Implementation Hints and Learnings
 
-## DKIM Implementation Status (2025-05-30)
+## smartmta Migration (2026-02-11)
+
+### Overview
+dcrouter's custom MTA code (~27,149 lines / 68 files in `ts/mail/` + `ts/deliverability/`) has been replaced with `@push.rocks/smartmta` v5.2.1, a TypeScript+Rust hybrid MTA. dcrouter is now an orchestrator that wires together SmartProxy, smartmta, smartdns, smartradius, and OpsServer.
+
+### Architecture
+- **No socket-handler mode** — smartmta's Rust SMTP server binds its own ports directly
+- **SmartProxy forward mode only** — external email ports forwarded to internal ports where smartmta listens
+- Email traffic flow: External Port → SmartProxy → Internal Port → smartmta UnifiedEmailServer
+
+### Key API Differences (smartmta vs old custom MTA)
+- `updateEmailRoutes()` instead of `updateRoutes()`
+- `dkimCreator` is public (no need for `(this.emailServer as any).dkimCreator`)
+- `bounceManager` is private, but exposed via public methods:
+  - `emailServer.getSuppressionList()`
+  - `emailServer.getHardBouncedAddresses()`
+  - `emailServer.getBounceHistory(email)`
+  - `emailServer.removeFromSuppressionList(email)`
+- `Email` class imported from `@push.rocks/smartmta`
+- `IAttachment` type accessed via `Core` namespace: `import { type Core } from '@push.rocks/smartmta'; type IAttachment = Core.IAttachment;`
+
+### Deleted Directories
+- `ts/mail/` (60 files) — replaced by smartmta
+- `ts/deliverability/` (3 files) — IPWarmupManager/SenderReputationMonitor will move to smartmta
+- `ts/errors/email.errors.ts`, `ts/errors/mta.errors.ts` — smartmta has its own errors
+- `ts/cache/documents/classes.cached.bounce.ts`, `classes.cached.suppression.ts`, `classes.cached.dkim.ts` — smartmta handles its own persistence
+
+### Remaining Cache Documents
+- `CachedEmail` — kept (dcrouter-level queue persistence)
+- `CachedIPReputation` — kept (dcrouter-level IP reputation caching)
+
+### Dependencies Removed
+mailauth, mailparser, @types/mailparser, ip, @push.rocks/smartmail, @push.rocks/smartrule, node-forge
+
+### Pre-existing Test Failures (not caused by migration)
+- `test/test.jwt-auth.ts` — `response.text is not a function` (webrequest compatibility issue)
+- `test/test.opsserver-api.ts` — same webrequest issue, timeouts
+
+### smartmta Location
+Source at `../../push.rocks/smartmta`, release with `gitzone commit -ypbrt`
+
+## Dependency Upgrade (2026-02-11)
+
+### SmartProxy v23.1.2 Route Validation
+- SmartProxy 23.1.2 enforces stricter route validation
+- Forward actions MUST use `targets` (array) instead of `target` (singular)
+- Test configurations that call `DcRouter.start()` need `cacheConfig: { enabled: false }` to avoid `/etc/dcrouter` permission errors
+
+```typescript
+// WRONG - will fail validation
+action: { type: 'forward', target: { host: 'localhost', port: 10025 } }
+
+// CORRECT
+action: { type: 'forward', targets: [{ host: 'localhost', port: 10025 }] }
+```
+
+**Files Fixed:**
+- `ts/classes.dcrouter.ts` - `generateEmailRoutes()` method
+- `test/test.dcrouter.email.ts` - Updated assertions and added `cacheConfig: { enabled: false }`
+
+## Dependency Upgrade (2026-02-10)
+
+### SmartProxy v23.1.0 Upgrade
+- `@push.rocks/smartproxy`: 22.4.2 → 23.1.0
+
+**Key Changes:**
+- Rust-based proxy components for improved performance
+- Rust binary runs as separate process via IPC
+- `getStatistics()` now returns `Promise<any>` (was synchronous)
+- nftables-proxy removed (not used by dcrouter)
+
+**Code Changes Required:**
+```typescript
+// Old (synchronous)
+const proxyStats = this.dcRouter.smartProxy.getStatistics();
+
+// New (async)
+const proxyStats = await this.dcRouter.smartProxy.getStatistics();
+```
+
+**Files Modified:**
+- `ts/monitoring/classes.metricsmanager.ts` - Added `await` to `getStatistics()` call
+
+## Dependency Upgrade (2026-02-01)
+
+### Major Upgrades Completed
+- `@api.global/typedserver`: 3.0.80 → 8.3.0
+- `@api.global/typedsocket`: 3.1.1 → 4.1.0
+- `@apiclient.xyz/cloudflare`: 6.4.3 → 7.1.0
+- `@design.estate/dees-catalog`: 1.12.4 → 3.41.4
+- `@push.rocks/smartpath`: 5.1.0 → 6.0.0
+- `@push.rocks/smartproxy`: 19.6.17 → 22.4.2
+- `@push.rocks/smartrequest`: 2.1.0 → 5.0.1
+- `uuid`: 11.1.0 → 13.0.0
 
-### Current Implementation
-1. **DKIM Key Generation**: Working - keys are generated when emails are sent
-2. **DKIM Email Signing**: Working - emails are signed with DKIM
-3. **DKIM DNS Record Serving**: Implemented - records are loaded from JSON files and served
-4. **Proactive DKIM Generation**: Implemented - keys are generated for all email domains at startup
-
-### Key Points
-- DKIM selector is hardcoded as `mta` in DKIMCreator
-- DKIM records are stored in `.nogit/data/dns/*.dkimrecord.json`
-- DKIM keys are stored in `.nogit/data/keys/{domain}-private.pem` and `{domain}-public.pem`
-- The server needs to be restarted for DKIM records to be loaded and served
-- Proactive generation ensures DKIM records are available immediately after startup
-
-### Testing
-After server restart, DKIM records can be queried:
+### Breaking Changes Fixed
+
+1. **SmartProxy v22**: `target` → `targets` (array)
+   ```typescript
+   // Old
+   action: { type: 'forward', target: { host: 'x', port: 25 } }
+   // New
+   action: { type: 'forward', targets: [{ host: 'x', port: 25 }] }
+   ```
+
+2. **SmartRequest v5**: `SmartRequestClient` → `SmartRequest`, `.body` → `.json()`
+   ```typescript
+   // Old
+   const resp = await plugins.smartrequest.SmartRequestClient.create()...post();
+   const json = resp.body;
+   // New
+   const resp = await plugins.smartrequest.SmartRequest.create()...post();
+   const json = await resp.json();
+   ```
+
+3. **dees-catalog v3**: Icon naming changed to library-prefixed format
+   ```typescript
+   // Old (deprecated but supported)
+   <dees-icon iconFA="check"></dees-icon>
+   // New
+   <dees-icon icon="fa:check"></dees-icon>
+   <dees-icon icon="lucide:menu"></dees-icon>
+   ```
+
+### TC39 Decorators
+- ts_web components updated to use `accessor` keyword for `@state()` decorators
+- Required for TC39 standard decorator support
+
+### tswatch Configuration
+The project now uses tswatch for development:
 ```bash
-dig @192.168.190.3 mta._domainkey.central.eu TXT +short
+pnpm run watch
+```
+Configuration in `npmextra.json`:
+```json
+{
+  "@git.zone/tswatch": {
+    "watchers": [{
+      "name": "dcrouter-dev",
+      "watch": ["ts/**/*.ts", "ts_*/**/*.ts", "test_watch/devserver.ts"],
+      "command": "pnpm run build && tsrun test_watch/devserver.ts",
+      "restart": true,
+      "debounce": 500,
+      "runOnStart": true
+    }]
+  }
+}
 ```
 
-### Note
-The existing dcrouter instance has test domain DKIM records but not for production domains like central.eu. A restart is required to trigger the proactive DKIM generation for configured email domains.
+## RADIUS Server Integration (2026-02-01)
+
+### Overview
+DcRouter now supports RADIUS server functionality for network authentication via `@push.rocks/smartradius`.
+
+### Key Features
+- **MAC Authentication Bypass (MAB)** - Authenticate network devices based on MAC address
+- **VLAN Assignment** - Assign VLANs based on MAC address or OUI patterns
+- **RADIUS Accounting** - Track sessions, data usage, and billing
+
+### Configuration Example
+```typescript
+const dcRouter = new DcRouter({
+  radiusConfig: {
+    authPort: 1812,      // Authentication port (default)
+    acctPort: 1813,      // Accounting port (default)
+    clients: [
+      {
+        name: 'switch-1',
+        ipRange: '192.168.1.0/24',
+        secret: 'shared-secret',
+        enabled: true
+      }
+    ],
+    vlanAssignment: {
+      defaultVlan: 100,   // VLAN for unknown MACs
+      allowUnknownMacs: true,
+      mappings: [
+        { mac: '00:11:22:33:44:55', vlan: 10, enabled: true },
+        { mac: '00:11:22', vlan: 20, enabled: true }  // OUI pattern
+      ]
+    },
+    accounting: {
+      enabled: true,
+      retentionDays: 30
+    }
+  }
+});
+```
+
+### Components
+- `RadiusServer` - Main server wrapping smartradius
+- `VlanManager` - MAC-to-VLAN mapping with OUI pattern support
+- `AccountingManager` - Session tracking and billing data
+
+### OpsServer API Endpoints
+- `getRadiusClients` / `setRadiusClient` / `removeRadiusClient` - Client management
+- `getVlanMappings` / `setVlanMapping` / `removeVlanMapping` - VLAN mappings
+- `testVlanAssignment` - Test what VLAN a MAC would get
+- `getRadiusSessions` / `disconnectRadiusSession` - Session management
+- `getRadiusStatistics` / `getRadiusAccountingSummary` - Statistics
+
+### Files
+- `ts/radius/` - RADIUS module
+- `ts/opsserver/handlers/radius.handler.ts` - OpsServer handler
+- `ts_interfaces/requests/radius.ts` - TypedRequest interfaces
+
+## Test Fix: test.dcrouter.email.ts (2026-02-01)
+
+### Issue
+The test `DcRouter class - Custom email storage path` was failing with "domainConfigs is not iterable".
+
+### Root Cause
+The test was using outdated email config properties:
+- Used `domainRules: []` (non-existent property)
+- Used `defaultMode` (non-existent property)
+- Missing required `domains: []` property
+- Missing required `routes: []` property
+- Referenced `router.unifiedEmailServer` instead of `router.emailServer`
+
+### Fix
+Updated the test to use the correct `IUnifiedEmailServerOptions` interface properties:
+```typescript
+const emailConfig: IEmailConfig = {
+  ports: [2525],
+  hostname: 'mail.example.com',
+  domains: [], // Required: domain configurations
+  routes: []   // Required: email routing rules
+};
+```
+
+And fixed the property name:
+```typescript
+expect(router.emailServer).toBeTruthy();  // Not unifiedEmailServer
+```
+
+### Key Learning
+When using `IUnifiedEmailServerOptions` (aliased as `IEmailConfig` in some tests):
+- `domains: IEmailDomainConfig[]` is required (array of domain configs)
+- `routes: IEmailRoute[]` is required (email routing rules)
+- Access the email server via `dcRouter.emailServer` not `dcRouter.unifiedEmailServer`
+
+## Network Metrics Implementation (2025-06-23)
+
+### SmartProxy Metrics API Integration
+- Updated to use new SmartProxy metrics API (v19.6.7)
+- Use `getMetrics()` for detailed metrics with grouped methods:
+  ```typescript
+  const metrics = smartProxy.getMetrics();
+  metrics.connections.active()      // Current active connections
+  metrics.throughput.instant()      // Real-time throughput {in, out}
+  metrics.connections.topIPs(10)    // Top 10 IPs by connection count
+  ```
+- Use `getStatistics()` for basic stats
+
+### Network Traffic Display
+- All throughput values shown in bits per second (kbit/s, Mbit/s, Gbit/s)
+- Conversion: `bytesPerSecond * 8 / 1000000` for Mbps
+- Network graph shows separate lines for inbound (green) and outbound (purple)
+- Throughput tiles and graph use same data source for consistency
+
+### Requests/sec vs Connections
+- Requests/sec shows HTTP request counts (derived from connections)
+- Single connection can handle multiple requests
+- Current implementation tracks connections, not individual requests
+- Trend line shows historical request counts, not throughput
+
+## DKIM Implementation Status (2025-05-30)
+
+**Note:** DKIM is now handled by `@push.rocks/smartmta`. The `dkimCreator` is a public property on `UnifiedEmailServer`.
 
 ## SmartProxy Usage
 
@@ -199,31 +445,11 @@ tap.test('stop', async () => {
 
 ## Email Integration with SmartProxy
 
-### Architecture
+### Architecture (Post-Migration)
 - Email traffic is routed through SmartProxy using automatic route generation
-- Email server runs on internal ports and receives forwarded traffic from SmartProxy
+- smartmta's UnifiedEmailServer runs on internal ports and receives forwarded traffic from SmartProxy
 - SmartProxy handles external ports (25, 587, 465) and forwards to internal ports
-
-### Email Route Generation
-```typescript
-// Email configuration automatically generates SmartProxy routes
-emailConfig: {
-  ports: [25, 587, 465],
-  hostname: 'mail.example.com',
-  domainRules: [...]
-}
-
-// Generates routes like:
-{
-  name: 'smtp-route',
-  match: { ports: [25] },
-  action: {
-    type: 'forward',
-    target: { host: 'localhost', port: 10025 }
-  },
-  tls: { mode: 'passthrough' } // STARTTLS handled by email server
-}
-```
+- smartmta's Rust SMTP bridge handles SMTP protocol processing
 
 ### Port Mapping
 - External port 25 → Internal port 10025 (SMTP)
@@ -231,676 +457,317 @@ emailConfig: {
 - External port 465 → Internal port 10465 (SMTPS)
 
 ### TLS Handling
-- Ports 25 and 587: Use 'passthrough' mode (STARTTLS handled by email server)
+- Ports 25 and 587: Use 'passthrough' mode (STARTTLS handled by smartmta)
 - Port 465: Use 'terminate' mode (SmartProxy handles TLS termination)
-- Domain-specific TLS can be configured per email rule
 
-## SMTP Test Migration
+## SmartMetrics Integration (2025-06-12) - COMPLETED
 
-### Test Framework
-- Tests migrated from custom framework to @push.rocks/tapbundle
-- Each test file is self-contained with its own server lifecycle management
-- Test files use pattern `test.*.ts` for automatic discovery by tstest
-
-### Server Lifecycle
-- SMTP server uses `listen()` method to start (not `start()`)
-- SMTP server uses `close()` method to stop (not `stop()` or `destroy()`)
-- Server loader module manages server lifecycle for tests
-
-### Test Structure
-```typescript
-import { expect, tap } from '@push.rocks/tapbundle';
-import { startTestServer, stopTestServer } from '../server.loader.js';
-
-const TEST_PORT = 2525;
-const TEST_TIMEOUT = 10000;
-
-tap.test('prepare server', async () => {
-  await startTestServer();
-  await new Promise(resolve => setTimeout(resolve, 100));
-});
-
-tap.test('test name', async (tools) => {
-  const done = tools.defer();
-  // test implementation
-  done.resolve();
-});
-
-tap.test('cleanup server', async () => {
-  await stopTestServer();
-});
-
-tap.start();
-```
-
-### Common Issues and Solutions
-1. **Multi-line SMTP responses**: Handle response buffering carefully, especially for EHLO
-2. **Timing issues**: Use proper state management instead of string matching
-3. **ES Module imports**: Use `import` statements, not `require()`
-4. **Server cleanup**: Always close connections properly to avoid hanging tests
-5. **Response buffer management**: Clear the response buffer after processing each state to avoid false matches from previous responses. Use specific response patterns (e.g., '250 OK' instead of just '250') to avoid ambiguity.
-
-### SMTP Protocol Testing
-- Server generates self-signed certificates automatically for testing
-- Default test port is 2525
-- Connection timeout is typically 10 seconds
-- Always check for complete SMTP responses (ending with space after code)
-
-## SMTP Implementation Findings (2025-05-25)
-
-### Fixed Issues
-
-1. **AUTH Mechanism Implementation**
-   - The server-side AUTH command handler was incomplete
-   - Implemented `handleAuthPlain` with proper PLAIN authentication flow
-   - Implemented `handleAuthLogin` with state-based LOGIN authentication flow
-   - Added `validateUser` function to test server configuration
-   - AUTH tests now expect STARTTLS instead of direct TLS (`secure: false` with `requireTLS: true`)
-
-2. **TLS Connection Timeout Handling**
-   - For secure connections, the client was waiting for 'connect' event instead of 'secureConnect'
-   - Fixed in `ConnectionManager.establishSocket()` to use the appropriate event based on connection type
-   - This prevents indefinite hangs during TLS handshake failures
-
-3. **STARTTLS Server Implementation**
-   - Removed incorrect `(tlsSocket as any)._start()` call which is client-side only
-   - Server-side TLS sockets handle handshake automatically when data arrives
-   - The `_start()` method caused Node.js assertion failure: `wrap->is_client()`
-
-4. **Edge Case Test Patterns**
-   - Tests using non-existent `smtpClient.connect()` method - use `verify()` instead
-   - SMTP servers must handle DATA mode properly by processing lines individually
-   - Empty/minimal server responses need to be valid SMTP codes (e.g., "250 OK\r\n")
-   - Out-of-order pipelined responses break SMTP protocol - responses must be in order
-
-### Common Test Patterns
-
-1. **Connection Testing**
-   ```typescript
-   const verified = await smtpClient.verify();
-   expect(verified).toBeTrue();
-   ```
+### Overview
+Fixed the UI metrics display to show accurate CPU and memory data from SmartMetrics.
+
+### Key Findings
+1. **CPU Metrics:**
+   - SmartMetrics provides `cpuUsageText` as a string percentage
+   - MetricsManager parses it as `cpuUsage.user` (system is always 0)
+   - UI was incorrectly dividing by 2, showing half the actual CPU usage
+
+2. **Memory Metrics:**
+   - SmartMetrics calculates `maxMemoryMB` as minimum of:
+     - V8 heap size limit
+     - System total memory
+     - Docker memory limit (if available)
+   - Provides `memoryUsageBytes` (total process memory including children)
+   - Provides `memoryPercentage` (pre-calculated percentage)
+   - UI was only showing heap usage, missing actual memory constraints
 
-2. **Server Data Handling**
-   ```typescript
-   socket.on('data', (data) => {
-     const lines = data.toString().split('\r\n');
-     lines.forEach(line => {
-       if (!line && lines[lines.length - 1] === '') return;
-       // Process each line individually
-     });
-   });
-   ```
+### Changes Made
+1. **MetricsManager Enhanced:**
+   - Added `maxMemoryMB` from SmartMetrics instance
+   - Added `actualUsageBytes` from SmartMetrics data
+   - Added `actualUsagePercentage` from SmartMetrics data
+   - Kept existing memory fields for compatibility
 
-3. **Authentication Setup**
-   ```typescript
-   auth: {
-     required: true,
-     methods: ['PLAIN', 'LOGIN'],
-     validateUser: async (username, password) => {
-       return username === 'testuser' && password === 'testpass';
-     }
-   }
-   ```
+2. **Interface Updated:**
+   - Added optional fields to `IServerStats.memoryUsage`
+   - Fields are optional to maintain backward compatibility
 
-### Progress Tracking
-- Fixed 8 tests total (as of 2025-05-25)
-- Fixed 8 additional tests (as of 2025-05-26):
-  - test.cedge-03.protocol-violations.ts
-  - test.cerr-03.network-failures.ts
-  - test.cerr-05.quota-exceeded.ts
-  - test.cerr-06.invalid-recipients.ts
-  - test.crel-01.reconnection-logic.ts
-  - test.crel-02.network-interruption.ts
-  - test.crel-03.queue-persistence.ts
-- 26 error logs remaining in `.nogit/testlogs/00err/`
-- Performance, additional reliability, RFC compliance, and security tests still need fixes
-
-## Test Fix Findings (2025-05-26)
-
-### Common Issues in SMTP Client Tests
-
-1. **DATA Phase Handling in Test Servers**
-   - Test servers must properly handle DATA mode
-   - Need to track when in DATA mode and look for the terminating '.'
-   - Multi-line data must be processed line by line
-   ```typescript
-   let inData = false;
-   socket.on('data', (data) => {
-     const lines = data.toString().split('\r\n');
-     lines.forEach(line => {
-       if (inData && line === '.') {
-         socket.write('250 OK\r\n');
-         inData = false;
-       } else if (line === 'DATA') {
-         socket.write('354 Send data\r\n');
-         inData = true;
-       }
-     });
-   });
-   ```
+3. **UI Fixed:**
+   - Removed incorrect CPU division by 2
+   - Uses `actualUsagePercentage` when available (falls back to heap percentage)
+   - Shows actual memory usage vs max memory limit (not just heap)
 
-2. **Import Issues**
-   - `createSmtpClient` should be imported from `ts/mail/delivery/smtpclient/index.js`
-   - Test server functions: use `startTestServer`/`stopTestServer` (not `startTestSmtpServer`)
-   - Helper exports `createTestSmtpClient`, not `createSmtpClient`
-
-3. **SmtpClient API Misconceptions**
-   - SmtpClient doesn't have methods like `connect()`, `isConnected()`, `getConnectionInfo()`
-   - Use `verify()` for connection testing
-   - Use `sendMail()` with Email objects for sending
-   - Connection management is handled internally
-
-4. **createSmtpClient is Not Async**
-   - The factory function returns an SmtpClient directly, not a Promise
-   - Remove `await` from `createSmtpClient()` calls
-
-5. **Test Expectations**
-   - Multi-line SMTP responses may timeout if server doesn't send final line
-   - Mixed valid/invalid recipients might succeed for valid ones (implementation-specific)
-   - Network failure tests should use realistic expectations
-
-6. **Test Runner Requirements**
-   - Tests using `tap` from '@git.zone/tstest/tapbundle' must call `tap.start()` at the end
-   - Without `tap.start()`, no tests will be detected or run
-   - Place `tap.start()` after all `tap.test()` definitions
-
-7. **Connection Pooling Effects**
-   - SmtpClient uses connection pooling by default
-   - Test servers may not receive all messages immediately
-   - Messages might be queued and sent through different connections
-   - Adjust test expectations to account for pooling behavior
-
-## Test Fixing Progress (2025-05-26 Afternoon)
-
-### Summary
-- Total failing tests initially: 35
-- Tests fixed: 35 ✅
-- Tests remaining: 0 - ALL TESTS PASSING!
-
-### Fixed Tests - Session 2 (7):
-1. test.ccm-05.connection-reuse.ts - Fixed performance expectation ✓
-2. test.cperf-05.network-efficiency.ts - Removed pooled client usage ✓
-3. test.cperf-06.caching-strategies.ts - Changed to sequential sending ✓
-4. test.cperf-07.queue-management.ts - Simplified to sequential processing ✓
-5. test.crel-07.resource-cleanup.ts - Complete rewrite to minimal test ✓
-6. test.reputationmonitor.ts - Fixed data accumulation by setting NODE_ENV='test' ✓
-7. Cleaned up stale error log: test__test.reputationmonitor.log (old log from before fix)
-
-### Fixed Tests - Session 1 (28):
-- **Edge Cases (1)**: test.cedge-03.protocol-violations.ts ✓
-- **Error Handling (3)**: cerr-03, cerr-05, cerr-06 ✓
-- **Reliability (6)**: crel-01 through crel-06 ✓
-- **RFC Compliance (7)**: crfc-02 through crfc-08 ✓
-- **Security (10)**: csec-01 through csec-10 ✓
-- **Performance (1)**: cperf-08.dns-caching.ts ✓
-
-### Important Notes:
-- Error logs are deleted after tests are fixed (per original instruction)
-- Tests taking >1 minute usually indicate hanging issues
-- Property names: use 'host' not 'hostname' for SmtpClient options
-- Always use helpers: createTestSmtpClient, createTestServer
-- Always add tap.start() at the end of test files
-
-### Key Fixes Applied:
-- **Data Accumulation**: Set NODE_ENV='test' to prevent loading persisted data between tests
-- **Connection Reuse**: Don't expect reuse to always be faster than fresh connections
-- **Pooled Clients**: Remove usage - tests expect direct client behavior
-- **Port Conflicts**: Use different ports for each test to avoid conflicts
-- **Resource Cleanup**: Simplified tests that were too complex and timing-dependent
-
-## Email Architecture Analysis (2025-05-27)
-
-### Previous Architecture Issues (NOW RESOLVED)
-1. ~~**Scattered Components**: Email functionality spread across multiple DcRouter properties~~ ✅ CONSOLIDATED
-2. ~~**Duplicate SMTP Implementations**: EmailSendJob implements raw socket SMTP protocol~~ ✅ FIXED 
-3. ~~**Complex Setup**: setupUnifiedEmailHandling() is 150+ lines~~ ✅ SIMPLIFIED (now ~30 lines)
-4. ~~**No Connection Pooling**: Each outbound email creates new connection~~ ✅ IMPLEMENTED
-5. ~~**Orphaned Code**: SmtpPortConfig class exists but is never used~~ ✅ REMOVED
-
-### Current Architecture (COMPLETED)
-All email components are now consolidated under UnifiedEmailServer:
-- `domainRouter` - Handles pattern-based routing decisions
-- `deliveryQueue` - Manages email queue with retry logic
-- `deliverySystem` - Handles multi-mode delivery
-- `rateLimiter` - Enforces hierarchical rate limits
-- `bounceManager` - Processes bounce notifications
-- `ipWarmupManager` - Manages IP warmup process
-- `senderReputationMonitor` - Tracks sender reputation
-- `dkimCreator` - Handles DKIM key management
-- `ipReputationChecker` - Checks IP reputation
-- `smtpClients` - Map of pooled SMTP clients
-
-### Email Traffic Flow
-```
-External Port → SmartProxy → Internal Port → UnifiedEmailServer → Processing
-     25             ↓           10025              ↓                  ↓
-    587          Routes         10587         DomainRouter      DeliverySystem
-    465                         10465                                 ↓
-                                                               Queue → SmtpClient
-                                                               (pooled & reused)
-```
+### Result
+- CPU now shows accurate usage percentage
+- Memory shows percentage of actual constraints (Docker/system/V8 limits)
+- Better monitoring for containerized environments
 
-### Completed Improvements
-- ✅ All email components consolidated under UnifiedEmailServer
-- ✅ EmailSendJob uses pooled SmtpClient via `getSmtpClient(host, port)`
-- ✅ DcRouter simplified to just manage high-level services
-- ✅ Connection pooling implemented for all outbound mail
-- ✅ setupUnifiedEmailHandling() simplified to ~30 lines
+## Network UI Implementation (2025-06-20) - COMPLETED
 
-## SMTP Client Management (2025-05-27)
+### Overview
+Revamped the Network UI to display real network data from SmartProxy instead of mock data.
 
-### Centralized SMTP Client in UnifiedEmailServer
-- SMTP clients are now managed centrally in UnifiedEmailServer
-- Uses connection pooling for efficiency (one pool per destination host:port)
-- Classes using UnifiedEmailServer get SMTP clients via `getSmtpClient(host, port)`
+### Architecture
+1. **MetricsManager Integration:**
+   - Already integrates with SmartProxy via `dcRouter.smartProxy.getStats()`
+   - Extended with `getNetworkStats()` method to expose unused metrics:
+     - `getConnectionsByIP()` - Connection counts by IP address
+     - `getThroughputRate()` - Real-time bandwidth rates (bytes/second)
+     - `getTopIPs()` - Top connecting IPs sorted by connection count
+   - Note: SmartProxy base interface doesn't include all methods, manual implementation required
+
+2. **Existing Infrastructure Leveraged:**
+   - `getActiveConnections` endpoint already exists in security.handler.ts
+   - Enhanced to include real SmartProxy data via MetricsManager
+   - IConnectionInfo interface already supports network data structures
+
+3. **State Management:**
+   - Added `INetworkState` interface following existing patterns
+   - Created `networkStatePart` with connections, throughput, and IP data
+   - Integrated with existing auto-refresh mechanism
+
+4. **UI Changes (Minimal):**
+   - Removed `generateMockData()` method and all mock generation
+   - Connected to real `networkStatePart` state
+   - Added `renderTopIPs()` section to display top connected IPs
+   - Updated traffic chart to show real request data
+   - Kept all existing UI components (DeesTable, DeesChartArea)
 
 ### Implementation Details
-```typescript
-// In UnifiedEmailServer
-private smtpClients: Map<string, SmtpClient> = new Map(); // host:port -> client
-
-public getSmtpClient(host: string, port: number = 25): SmtpClient {
-  const clientKey = `${host}:${port}`;
-  let client = this.smtpClients.get(clientKey);
-  
-  if (!client) {
-    client = createPooledSmtpClient({
-      host,
-      port,
-      secure: port === 465,
-      connectionTimeout: 30000,
-      socketTimeout: 120000,
-      maxConnections: 10,
-      maxMessages: 1000,
-      pool: true
-    });
-    this.smtpClients.set(clientKey, client);
-  }
-  
-  return client;
-}
-```
+1. **Data Transformation:**
+   - Converts IConnectionInfo[] to INetworkRequest[] for table display
+   - Calculates traffic buckets based on selected time range
+   - Maps connection data to chart-compatible format
+
+2. **Real Metrics Displayed:**
+   - Active connections count (from server stats)
+   - Requests per second (calculated from recent connections)
+   - Throughput rates (currently showing 0 until SmartProxy exposes rates)
+   - Top IPs with connection counts and percentages
+
+3. **TypeScript Fixes:**
+   - SmartProxy methods like `getThroughputRate()` not in base interface
+   - Implemented manual fallbacks for missing methods
+   - Fixed `publicIpv4` → `publicIp` property name
 
-### Usage Pattern
-- EmailSendJob and DeliverySystem now use `this.emailServerRef.getSmtpClient(host, port)`
-- Connection pooling happens automatically
-- Connections are reused across multiple send jobs
-- All SMTP clients are closed when UnifiedEmailServer stops
+### Result
+- Network view now shows real connection activity
+- Auto-refreshes with other stats every second
+- Displays actual IPs and connection counts
+- No more mock/demo data
+- Minimal code changes (streamlined approach)
 
-### Dependency Injection Pattern
-- Classes that need UnifiedEmailServer functionality receive it as constructor argument
-- This provides access to SMTP clients, DKIM signing, and other shared functionality
-- Example: `new EmailSendJob(emailServerRef, email, options)`
+### Throughput Data Fix (2025-06-20)
+The throughput was showing 0 because:
+1. MetricsManager was hardcoding throughputRate to 0, assuming the method didn't exist
+2. SmartProxy's `getStats()` returns `IProxyStats` interface, but the actual object (`MetricsCollector`) implements `IProxyStatsExtended`
+3. `getThroughputRate()` only exists in the extended interface
 
-## Email Class Standardization (2025-05-27) - COMPLETED
+**Solution implemented:**
+1. Updated MetricsManager to check if methods exist at runtime and call them
+2. Added property name mapping (`bytesInPerSec` → `bytesInPerSecond`)
+3. Created new `getNetworkStats` endpoint in security.handler.ts
+4. Updated frontend to call the new endpoint for complete network metrics
 
-### Overview
-The entire codebase has been standardized to use the `Email` class as the single data structure for email handling. All Smartmail usage has been eliminated.
-
-### Key Changes
-1. **Email Class Enhanced** - Added compatibility methods: `getSubject()`, `getBody(isHtml)`, `getFrom()`
-2. **BounceManager** - Now accepts `Email` objects directly
-3. **TemplateManager** - Returns `Email` objects instead of Smartmail
-4. **EmailService** - `sendEmail()` accepts `Email` objects
-5. **RuleManager** - Uses `SmartRule<Email>` instead of `SmartRule<Smartmail>`
-6. **ApiManager** - Creates `Email` objects for API requests
-
-### Benefits
-- No more Email ↔ Smartmail conversions
-- Consistent API throughout (`email.subject` not `smartmail.options.subject`)
-- Better performance (no conversion overhead)
-- Simpler, more maintainable code
+The throughput data now flows correctly from SmartProxy → MetricsManager → API → UI.
 
-### Usage Pattern
-```typescript
-// Create email
-const email = new Email({
-  from: 'sender@example.com',
-  to: 'recipient@example.com',
-  subject: 'Hello',
-  text: 'World'
-});
+## Email Operations Dashboard (2026-02-01)
 
-// Pass directly through system
-await bounceManager.processBounceEmail(email);
-await templateManager.prepareEmail(templateId, context);
-await emailService.sendEmail(email);
+### Overview
+Replaced mock data in the email UI with real backend data from the delivery queue and security logger.
+
+### New Files Created
+- `ts_interfaces/requests/email-ops.ts` - TypedRequest interfaces for email operations
+- `ts/opsserver/handlers/email-ops.handler.ts` - Backend handler for email operations
+
+### Key Interfaces
+- `IReq_GetQueuedEmails` - Fetch emails from delivery queue by status
+- `IReq_GetSentEmails` - Fetch delivered emails
+- `IReq_GetFailedEmails` - Fetch failed emails
+- `IReq_ResendEmail` - Re-queue a failed email for retry
+- `IReq_GetSecurityIncidents` - Fetch security events from SecurityLogger
+- `IReq_GetBounceRecords` - Fetch bounce records and suppression list
+- `IReq_RemoveFromSuppressionList` - Remove email from suppression list
+
+### UI Changes (ops-view-emails.ts)
+- Replaced mock folders (inbox/sent/draft/trash) with operations views:
+  - **Queued**: Emails pending delivery
+  - **Sent**: Successfully delivered emails
+  - **Failed**: Failed emails with resend capability
+  - **Security**: Security incidents from SecurityLogger
+- Removed `generateMockEmails()` method
+- Added state management via `emailOpsStatePart` in appstate.ts
+- Added resend button for failed emails
+- Added security incident detail view
+
+### Data Flow
 ```
-
-## Email Class Design Pattern (2025-05-27)
-
-### Three-Interface Pattern for Email
-The Email system uses three distinct interfaces for clarity and type safety:
-
-1. **IEmailOptions** - The flexible input interface:
-   ```typescript
-   interface IEmailOptions {
-     to: string | string[];      // Flexible: single or array
-     cc?: string | string[];     // Optional
-     attachments?: IAttachment[]; // Optional
-     skipAdvancedValidation?: boolean; // Constructor-only option
-   }
-   ```
-   - Used as constructor parameter
-   - Allows flexible input formats
-   - Has constructor-only options (like skipAdvancedValidation)
-
-2. **INormalizedEmail** - The normalized runtime interface:
-   ```typescript
-   interface INormalizedEmail {
-     to: string[];               // Always an array
-     cc: string[];               // Always an array (empty if not provided)
-     attachments: IAttachment[]; // Always an array (empty if not provided)
-     mightBeSpam: boolean;       // Always has a value (defaults to false)
-   }
-   ```
-   - Represents the guaranteed internal structure
-   - No optional arrays - everything has a default
-   - Email class implements this interface
-
-3. **Email class** - The implementation:
-   ```typescript
-   export class Email implements INormalizedEmail {
-     // All INormalizedEmail properties
-     to: string[];
-     cc: string[];
-     // ... etc
-     
-     // Additional runtime properties
-     private messageId: string;
-     private envelopeFrom: string;
-   }
-   ```
-   - Implements INormalizedEmail
-   - Adds behavior methods and computed properties
-   - Handles validation and normalization
-
-### Benefits of This Pattern:
-- **Type Safety**: Email class explicitly implements INormalizedEmail
-- **Clear Contracts**: Input vs. runtime structure is explicit
-- **Flexibility**: IEmailOptions allows various input formats
-- **Consistency**: INormalizedEmail guarantees structure
-- **Validation**: Constructor validates and normalizes
-
-### Usage:
-```typescript
-// Input with flexible options
-const options: IEmailOptions = {
-  from: 'sender@example.com',
-  to: 'recipient@example.com',  // Single string
-  subject: 'Hello',
-  text: 'World'
-};
-
-// Creates normalized Email instance
-const email = new Email(options);
-
-// email.to is guaranteed to be string[]
-email.to.forEach(recipient => {
-  // No need to check if it's an array
-});
-
-// Convert back to options format
-const optionsAgain = email.toEmailOptions();
+UnifiedDeliveryQueue → EmailOpsHandler → TypedRequest → Frontend State → UI
+SecurityLogger → EmailOpsHandler → TypedRequest → Frontend State → UI
+BounceManager → EmailOpsHandler → TypedRequest → Frontend State → UI
 ```
 
-### Template Email Creation (2025-05-27)
-The Email class now supports template creation without recipients:
-- IEmailOptions 'to' field is now optional (for templates)
-- Email constructor allows creation without recipients
-- Recipients are added later when the email is actually sent
+### Backend Data Access
+The handler accesses data from:
+- `dcRouter.emailServer.deliveryQueue` - Email queue items (IQueueItem)
+- `SecurityLogger.getInstance()` - Security events (ISecurityEvent)
+- `emailServer.bounceManager` - Bounce records and suppression list
 
-```typescript
-// Template creation (no recipients)
-const emailOptions: IEmailOptions = {
-  from: 'noreply@example.com',
-  subject: 'Welcome {{name}}',
-  text: 'Hello {{name}}!',
-  // 'to' is omitted for templates
-  variables: { name: 'User' }
-};
+## OpsServer UI Fixes (2026-02-02)
 
-const templateEmail = new Email(emailOptions);
-// templateEmail.to is an empty array []
+### Configuration Page Fix
+The configuration page had field name mismatches between frontend and backend:
+- Frontend expected `server` and `storage` sections
+- Backend returns `proxy` section (not `server`)
+- Backend has no `storage` section
 
-// Later, when sending:
-templateEmail.to = ['recipient@example.com'];
-```
+**Fix**: Updated `ops-view-config.ts` to use correct section names:
+- `proxy` instead of `server`
+- Removed non-existent `storage` section
+- Added optional chaining (`?.`) for safety
 
-## Email Architecture Consolidation Completed (2025-05-27)
-
-### Summary
-The email architecture consolidation has been fully completed. Contrary to the initial analysis, the architecture was already well-organized:
-
-1. **UnifiedEmailServer already contains all components** - No scattered components in DcRouter
-2. **EmailSendJob already uses pooled SmtpClient** - No duplicate SMTP implementations
-3. **setupUnifiedEmailHandling() is simple** - Only ~30 lines, not 150+
-4. **Connection pooling already implemented** - Via `getSmtpClient(host, port)`
-5. **SmtpPortConfig doesn't exist** - No orphaned code found
-
-### Key Architecture Points
-- DcRouter has single `emailServer?: UnifiedEmailServer` property
-- All email functionality encapsulated in UnifiedEmailServer
-- Dependency injection pattern used throughout (e.g., EmailSendJob receives UnifiedEmailServer reference)
-- Pooled SMTP clients managed centrally in UnifiedEmailServer
-- Clean separation of concerns between routing (DcRouter) and email handling (UnifiedEmailServer)
-
-## Email Router Architecture Decision (2025-05-28)
-
-### Single Router Class
-- **Important**: We will have only ONE router class, not two
-- The existing `DomainRouter` will be evolved into `EmailRouter`
-- This avoids confusion and redundancy
-- Use `git mv` to rename and preserve git history
-- Extend it to support the new match/action pattern inspired by SmartProxy
-- Maintain backward compatibility for legacy domain-based rules
-
-### Benefits of Single Router
-- Clear, single source of truth for routing logic
-- No confusion about which router to use
-- Preserved git history and gradual migration path
-- Supports all match criteria (not just domains)
-
-## Email Routing Architecture (2025-05-27)
-
-### Current Routing Capabilities
-1. **Pattern-based routing** - DomainRouter matches email addresses against patterns
-2. **Three processing modes**:
-   - `mta` - Programmatic processing with optional DKIM signing
-   - `process` - Store-and-forward with content scanning
-   - `forward` - Direct forwarding (NOT YET IMPLEMENTED)
-3. **Default routing** - Fallback for unmatched patterns
-4. **Basic caching** - LRU cache for routing decisions
-
-### Routing Flow
-```typescript
-// Current flow in UnifiedEmailServer.processEmailByMode()
-1. Email arrives → DomainRouter.matchRule(recipient)
-2. Apply matched rule or default
-3. Process based on mode:
-   - mta: Apply DKIM, log, return
-   - process: Scan content, apply transformations, queue
-   - forward: ERROR - Not implemented
-```
+### Auth Persistence Fix
+Login state was using `'soft'` mode in Smartstate which is memory-only:
+- User login was lost on page refresh
+- State reset to logged out after browser restart
 
-### Missing Routing Features
-1. **No forwarding implementation** - Forward mode throws error
-2. **Limited matching** - Only email address patterns, no regex
-3. **No conditional routing** - Can't route based on subject, size, etc.
-4. **No load balancing** - Single destination per rule
-5. **No failover** - No backup routes
-6. **Basic transformations** - Only header additions
+**Changes**:
+1. `ts_web/appstate.ts`: Changed loginStatePart from `'soft'` to `'persistent'`
+   - Now uses IndexedDB to persist across browser sessions
+2. `ts/opsserver/handlers/admin.handler.ts`: JWT expiry changed from 7 days to 24 hours
+3. `ts_web/elements/ops-dashboard.ts`: Added JWT expiry check on session restore
+   - Validates stored JWT hasn't expired before auto-logging in
+   - Clears expired sessions and shows login form
 
-### Key Files for Routing
-- `ts/mail/routing/classes.domain.router.ts` - Pattern matching engine
-- `ts/mail/routing/classes.unified.email.server.ts` - processEmailByMode()
-- `ts/mail/routing/classes.email.config.ts` - Rule interfaces
-- `ts/mail/delivery/classes.delivery.system.ts` - Delivery execution
-
-## Configuration System Cleanup (2025-05-27) - COMPLETED
+## Config UI Read-Only Conversion (2026-02-03)
 
 ### Overview
-The `ts/config/` directory cleanup has been completed. Removed ~500+ lines of unused legacy configuration code.
+The configuration UI has been converted from an editable interface to a read-only display. DcRouter is configured through code or remotely, not through the UI.
 
 ### Changes Made
-✅ **Removed Files:**
-- `base.config.ts` - All unused base interfaces
-- `platform.config.ts` - Completely unused platform config
-- `email.config.ts` - Deprecated email configuration
-- `email.port.mapping.ts` - Unused port mapping utilities
-- `schemas.ts` - Removed all schemas except SMS
-- `sms.config.ts` - Moved to SMS module
-
-✅ **SMS Configuration Moved:**
-- Created `ts/sms/config/sms.config.ts` - ISmsConfig interface
-- Created `ts/sms/config/sms.schema.ts` - Validation schema
-- Updated SmsService to import from new location
-
-✅ **Kept:**
-- `validator.ts` - Generic validation utility (might move to utils later)
-- `index.ts` - Now only exports ConfigValidator
-
-### Result
-- Config directory now contains only 2 files (validator.ts, index.ts)
-- SMS configuration is self-contained in SMS module
-- All deprecated email configuration removed
-- Build passes successfully
 
-## Per-Domain Rate Limiting (2025-05-29) - COMPLETED
+1. **Backend (`ts/opsserver/handlers/config.handler.ts`)**:
+   - Removed `updateConfiguration` handler
+   - Removed `updateConfiguration()` private method
+   - Kept `getConfiguration` handler (read-only)
+
+2. **Interfaces (`ts_interfaces/requests/config.ts`)**:
+   - Removed `IReq_UpdateConfiguration` interface
+   - Kept `IReq_GetConfiguration` interface
+
+3. **Frontend (`ts_web/elements/ops-view-config.ts`)**:
+   - Removed `editingSection` and `editedConfig` state properties
+   - Removed `startEdit()`, `cancelEdit()`, `saveConfig()` methods
+   - Removed Edit/Save/Cancel buttons
+   - Removed warning banner about immediate changes
+   - Enhanced read-only display with:
+     - Status badges for boolean values (enabled/disabled)
+     - Array display as pills/tags with counts
+     - Section icons (mail, globe, network, shield)
+     - Better formatting for numbers and byte sizes
+     - Empty state handling ("Not configured", "None configured")
+     - Info note explaining configuration is read-only
+
+4. **State Management (`ts_web/appstate.ts`)**:
+   - Removed `updateConfigurationAction`
+   - Kept `fetchConfigurationAction` (read-only)
+
+5. **Tests (`test/test.protected-endpoint.ts`)**:
+   - Replaced `updateConfiguration` tests with `verifyIdentity` tests
+   - Added test for read-only config access
+   - Kept auth flow testing with different protected endpoint
+
+6. **Documentation**:
+   - `readme.md`: Updated API endpoints to show config as read-only
+   - `ts_web/readme.md`: Removed `updateConfigurationAction` from actions list
+   - `ts_interfaces/readme.md`: Removed `IReq_UpdateConfiguration` from table
+
+### Visual Display Features
+- Boolean values shown as colored badges (green=enabled, red=disabled)
+- Arrays displayed as pills with count summaries
+- Section headers with relevant Lucide icons
+- Numbers formatted with locale separators
+- Byte sizes auto-formatted (B, KB, MB, GB)
+- Time values shown with "seconds" suffix
+- Nested objects with visual indentation
+
+## Smartdata Cache System (2026-02-03)
 
 ### Overview
-Per-domain rate limiting has been implemented in the UnifiedRateLimiter. Each email domain can have its own rate limits that override global limits.
+DcRouter now uses smartdata + LocalTsmDb for persistent caching. Data is stored at `/etc/dcrouter/tsmdb`.
 
-### Implementation Details
-1. **UnifiedRateLimiter Enhanced:**
-   - Added `domains` property to IHierarchicalRateLimits
-   - Added `domainCounters` Map for tracking domain-specific counters
-   - Added `checkDomainMessageLimit()` method
-   - Added `applyDomainLimits()`, `removeDomainLimits()`, `getDomainLimits()` methods
+### Technology Stack
+| Layer | Package | Purpose |
+|-------|---------|---------|
+| ORM | `@push.rocks/smartdata` | Document classes, decorators, queries |
+| Database | `@push.rocks/smartmongo` (LocalTsmDb) | Embedded TsmDB via Unix socket |
 
-2. **Domain Rate Limit Configuration:**
-   ```typescript
-   interface IEmailDomainConfig {
-     domain: string;
-     rateLimits?: {
-       outbound?: {
-         messagesPerMinute?: number;
-         messagesPerHour?: number;    // Note: Hour/day limits need additional implementation
-         messagesPerDay?: number;
-       };
-       inbound?: {
-         messagesPerMinute?: number;
-         connectionsPerIp?: number;
-         recipientsPerMessage?: number;
-       };
-     };
-   }
-   ```
+### TC39 Decorators
+The project uses TC39 Stage 3 decorators (not experimental decorators). The tsconfig was updated:
+- Removed `experimentalDecorators: true`
+- Removed `emitDecoratorMetadata: true`
 
-3. **Automatic Application:**
-   - UnifiedEmailServer applies domain rate limits during startup
-   - `applyDomainRateLimits()` method converts domain config to rate limiter format
-   - Domain limits override pattern and global limits
+This is required for smartdata v7+ compatibility.
 
-4. **Usage Pattern:**
-   ```typescript
-   // Domain configuration with rate limits
-   {
-     domain: 'high-volume.com',
-     dnsMode: 'internal-dns',
-     rateLimits: {
-       outbound: {
-         messagesPerMinute: 200  // Higher than global limit
-       },
-       inbound: {
-         recipientsPerMessage: 100  // Higher recipient limit
-       }
-     }
-   }
-   ```
+### Cache Document Classes
+Located in `ts/cache/documents/`:
 
-5. **Rate Limit Precedence:**
-   - Domain-specific limits (highest priority)
-   - Pattern-specific limits
-   - Global limits (lowest priority)
-
-### Integration Status
-- ✅ Rate limiter supports per-domain limits
-- ✅ UnifiedEmailServer applies domain limits on startup
-- ✅ Domain limits properly override global/pattern limits
-- ✅ SMTP server handlers now enforce rate limits (COMPLETED 2025-05-29)
-- ⚠️ Hour/day limits need additional implementation in rate limiter
-
-### SMTP Handler Integration (2025-05-29) - COMPLETED
-Rate limiting is now fully integrated into SMTP server handlers:
-
-1. **UnifiedEmailServer Enhancement:**
-   - Added `getRateLimiter()` method to provide access to the rate limiter
-
-2. **ConnectionManager Integration:**
-   - Replaced custom rate limiting with UnifiedRateLimiter
-   - Now uses `rateLimiter.recordConnection(ip)` for all connection checks
-   - Maintains local IP tracking for resource cleanup only
-
-3. **CommandHandler Integration:**
-   - `handleMailFrom()`: Checks message rate limits with domain context
-   - `handleRcptTo()`: Enforces recipient limits per message
-   - `handleAuth*()`: Records authentication failures and blocks after threshold
-   - Error handling: Records syntax/command errors and blocks after threshold
-
-4. **SMTP Response Codes:**
-   - `421`: Temporary rate limit (client should retry later)
-   - `451`: Temporary recipient rejection
-   - `421 Too many errors`: IP blocked due to excessive errors
-   - `421 Too many authentication failures`: IP blocked due to auth failures
-
-### Next Steps
-The only remaining item is implementing hour/day rate limits in the UnifiedRateLimiter, which would require:
-1. Additional counters for hourly and daily windows
-2. Separate tracking for these longer time periods
-3. Cleanup logic for expired hourly/daily counters
-
-## DNS Architecture Refactoring (2025-05-30) - COMPLETED
+| Class | Purpose | Default TTL |
+|-------|---------|-------------|
+| `CachedEmail` | Email queue items | 30 days |
+| `CachedIPReputation` | IP reputation lookups | 24 hours |
 
-### Overview
-The DNS functionality has been refactored from UnifiedEmailServer to a dedicated DnsManager class for better discoverability and separation of concerns.
+Note: CachedBounce, CachedSuppression, and CachedDKIMKey were removed in the smartmta migration (smartmta handles its own persistence for those).
 
-### Key Changes
-1. **Renamed DnsValidator to DnsManager:**
-   - Extended functionality to handle both validation and creation of DNS records
-   - Added `ensureDnsRecords()` as the main entry point
-   - Moved DNS record creation logic from UnifiedEmailServer
+### Usage Pattern
+```typescript
+// Document classes use smartdata decorators
+@plugins.smartdata.Collection(() => getDb())
+export class CachedEmail extends CachedDocument<CachedEmail> {
+  @plugins.smartdata.svDb()
+  public createdAt: Date = new Date();
+
+  @plugins.smartdata.svDb()
+  public expiresAt: Date = new Date(Date.now() + TTL.DAYS_30);
+
+  @plugins.smartdata.unI()
+  @plugins.smartdata.svDb()
+  public id: string;
+  // ...
+}
 
-2. **DnsManager Responsibilities:**
-   - Validate DNS configuration for all modes (forward, internal-dns, external-dns)
-   - Create DNS records for internal-dns domains
-   - Create DKIM records for all domains (when DKIMCreator is provided)
-   - Store DNS records in StorageManager for persistence
+// Query examples
+const email = await CachedEmail.getInstance({ id: 'abc123' });
+const pending = await CachedEmail.getInstances({ status: 'pending' });
+await email.save();
+await email.delete();
+```
 
-3. **DNS Record Creation Flow:**
-   ```typescript
-   // In UnifiedEmailServer
-   const dnsManager = new DnsManager(this.dcRouter);
-   await dnsManager.ensureDnsRecords(domainConfigs, this.dkimCreator);
-   ```
+### Configuration
+```typescript
+const dcRouter = new DcRouter({
+  cacheConfig: {
+    enabled: true,
+    storagePath: '/etc/dcrouter/tsmdb',
+    dbName: 'dcrouter',
+    cleanupIntervalHours: 1,
+    ttlConfig: {
+      emails: 30,       // days
+      ipReputation: 1,  // days
+      bounces: 30,      // days
+      dkimKeys: 90,     // days
+      suppression: 30   // days
+    }
+  }
+});
+```
 
-4. **Testing Pattern for DNS:**
-   - Mock the DNS server in tests by providing a mock `registerHandler` function
-   - Store handlers in a Map with key format: `${domain}:${types.join(',')}`
-   - Retrieve handlers with key format: `${domain}:${type}`
-   - Example mock implementation:
-   ```typescript
-   this.dnsServer = {
-     registerHandler: (name: string, types: string[], handler: () => any) => {
-       const key = `${name}:${types.join(',')}`;
-       this.dnsHandlers.set(key, handler);
-     }
-   };
-   ```
+### Cache Cleaner
+- Runs hourly by default (configurable via `cleanupIntervalHours`)
+- Finds and deletes documents where `expiresAt < now()`
+- Uses smartdata's `getInstances()` + `delete()` pattern
 
-### Benefits
-- DNS functionality is now easily discoverable in DnsManager
-- Clear separation between DNS management and email server logic
-- UnifiedEmailServer is simpler and more focused
-- All DNS-related tests pass successfully
+### Key Files
+- `ts/cache/classes.cachedb.ts` - CacheDb singleton wrapper
+- `ts/cache/classes.cached.document.ts` - Base class with TTL support
+- `ts/cache/classes.cache.cleaner.ts` - Periodic cleanup service
+- `ts/cache/documents/*.ts` - Document class definitions