log-collector-async 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +58 -0
- package/README.md +866 -0
- package/__tests__/client.test.js +418 -0
- package/example_express.js +99 -0
- package/example_global_error_handler.js +77 -0
- package/example_user_context.js +305 -0
- package/jest.config.js +13 -0
- package/package.json +60 -0
- package/src/browser-client.js +382 -0
- package/src/browser-worker.js +118 -0
- package/src/index.js +29 -0
- package/src/node-client.js +561 -0
- package/src/node-worker.js +127 -0
- package/test-manual.js +29 -0
- package/test_local.js +45 -0
package/README.md
ADDED
|
@@ -0,0 +1,866 @@
|
|
|
1
|
+
# Log Collector - JavaScript/Node.js Client
|
|
2
|
+
|
|
3
|
+
고성능 비동기 로그 수집 클라이언트 for JavaScript/Node.js
|
|
4
|
+
|
|
5
|
+
[](https://nodejs.org/)
|
|
6
|
+
[](LICENSE)
|
|
7
|
+
|
|
8
|
+
## 📋 Prerequisites
|
|
9
|
+
|
|
10
|
+
Before using this library, ensure you have:
|
|
11
|
+
|
|
12
|
+
- **Node.js 14+** or **Browser** (modern browsers with Web Worker support)
|
|
13
|
+
- **Package manager**: npm or yarn
|
|
14
|
+
- **Log server running**: See [Log Save Server Setup](../../services/log-save-server/README.md)
|
|
15
|
+
- **PostgreSQL database**: For log storage (v12+)
|
|
16
|
+
- **Basic async knowledge**: Understanding of async/await and Promise patterns
|
|
17
|
+
|
|
18
|
+
## 🎯 Why Use This Library?
|
|
19
|
+
|
|
20
|
+
### The Problem
|
|
21
|
+
Traditional logging blocks your application, creating performance bottlenecks:
|
|
22
|
+
- Each log = 1 HTTP request = ~50ms blocked time
|
|
23
|
+
- 100 logs/sec = 5 seconds of blocking per second (impossible!)
|
|
24
|
+
- Application threads wait for network I/O
|
|
25
|
+
- Database connection pool exhaustion
|
|
26
|
+
|
|
27
|
+
### The Solution
|
|
28
|
+
Asynchronous batch logging with zero blocking:
|
|
29
|
+
- ✅ **~0.01ms per log** - App never blocks waiting for network
|
|
30
|
+
- ✅ **Batches 1000 logs** - Single HTTP request instead of 1000
|
|
31
|
+
- ✅ **Background workers** - Web Worker/Worker Threads handle transmission
|
|
32
|
+
- ✅ **Auto compression** - gzip reduces bandwidth by ~70%
|
|
33
|
+
- ✅ **Reliable delivery** - Automatic retries with exponential backoff
|
|
34
|
+
- ✅ **Graceful shutdown** - Flushes queue before exit, zero log loss
|
|
35
|
+
|
|
36
|
+
### When to Use This
|
|
37
|
+
- High-traffic applications (>100 requests/sec)
|
|
38
|
+
- Performance-critical paths where blocking is unacceptable
|
|
39
|
+
- Microservices needing centralized structured logging
|
|
40
|
+
- Distributed tracing across services
|
|
41
|
+
- PostgreSQL-based log analysis and querying
|
|
42
|
+
|
|
43
|
+
### When NOT to Use This
|
|
44
|
+
- Low-traffic apps (<10 req/sec) - simple file logging is fine
|
|
45
|
+
- Quick debugging sessions - use console.log for speed
|
|
46
|
+
- Need real-time log streaming - use dedicated streaming solutions
|
|
47
|
+
- Cannot run log server infrastructure - use cloud logging services
|
|
48
|
+
|
|
49
|
+
## 🚀 Quick Start (30 seconds)
|
|
50
|
+
|
|
51
|
+
### Step 1: Install
|
|
52
|
+
```bash
|
|
53
|
+
npm install log-collector-async
|
|
54
|
+
# or
|
|
55
|
+
yarn add log-collector-async
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
### Step 2: Use in your app
|
|
59
|
+
```javascript
|
|
60
|
+
import { createLogClient } from 'log-collector-async';
|
|
61
|
+
|
|
62
|
+
// Initialize logger
|
|
63
|
+
const logger = createLogClient('http://localhost:8000');
|
|
64
|
+
|
|
65
|
+
// Send logs - non-blocking, ~0.01ms
|
|
66
|
+
logger.info('Hello world!', { user_id: '123', action: 'test' });
|
|
67
|
+
logger.warn('High memory usage', { memory_mb: 512 });
|
|
68
|
+
logger.error('Database error', { error: 'connection timeout' });
|
|
69
|
+
|
|
70
|
+
// Logs are batched and sent automatically every 1 second or 1000 logs
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
### Step 3: Check logs in database
|
|
74
|
+
```bash
|
|
75
|
+
psql -h localhost -U postgres -d logs_db \
|
|
76
|
+
-c "SELECT * FROM logs ORDER BY created_at DESC LIMIT 5;"
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
**Want more details?** See [Framework Integration](#http-컨텍스트-자동-수집) below.
|
|
80
|
+
|
|
81
|
+
**Want a working example?** Check out [Demo Applications](#-live-demo).
|
|
82
|
+
|
|
83
|
+
## 📺 Live Demo
|
|
84
|
+
|
|
85
|
+
See working examples with full context tracking:
|
|
86
|
+
|
|
87
|
+
### JavaScript + Express
|
|
88
|
+
- **Location**: [tests/demo-app/backend/](../../tests/demo-app/backend/)
|
|
89
|
+
- **Features**: Login, CRUD operations, error handling, slow API testing
|
|
90
|
+
- **Run**: `node tests/demo-app/backend/server.js`
|
|
91
|
+
|
|
92
|
+
### Python + FastAPI
|
|
93
|
+
- **Location**: [tests/demo-app/backend-python/](../../tests/demo-app/backend-python/)
|
|
94
|
+
- **Features**: Same features but with Python
|
|
95
|
+
- **Run**: `python tests/demo-app/backend-python/server.py`
|
|
96
|
+
|
|
97
|
+
### Frontend Integration
|
|
98
|
+
- **Location**: [tests/demo-app/frontend/](../../tests/demo-app/frontend/)
|
|
99
|
+
- **Features**: Browser-based logging with proper CORS setup
|
|
100
|
+
- **Run**: Open `tests/demo-app/frontend/index.html` in browser
|
|
101
|
+
|
|
102
|
+
### Quick Demo Setup
|
|
103
|
+
```bash
|
|
104
|
+
# 1. Start log server (in Docker)
|
|
105
|
+
cd services/log-save-server
|
|
106
|
+
docker-compose up
|
|
107
|
+
|
|
108
|
+
# 2. Start backend (JavaScript or Python)
|
|
109
|
+
cd tests/demo-app/backend
|
|
110
|
+
node server.js
|
|
111
|
+
|
|
112
|
+
# 3. Open frontend
|
|
113
|
+
open ../frontend/index.html
|
|
114
|
+
|
|
115
|
+
# 4. Interact with app, then check logs
|
|
116
|
+
psql -h localhost -U postgres -d logs_db \
|
|
117
|
+
-c "SELECT service, level, message FROM logs ORDER BY created_at DESC LIMIT 10;"
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
## 🔗 Integration with Full System
|
|
121
|
+
|
|
122
|
+
This client is part of a complete log analysis system. See the [main README](../../README.md) for the full picture.
|
|
123
|
+
|
|
124
|
+
### System Architecture
|
|
125
|
+
|
|
126
|
+
```
|
|
127
|
+
[Your App] → [JavaScript Client] → [Log Save Server] → [PostgreSQL] → [Analysis Server] → [Frontend]
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
### Related Components
|
|
131
|
+
|
|
132
|
+
- **Log Save Server**: Receives logs via HTTP POST ([README](../../services/log-save-server/README.md))
|
|
133
|
+
- **Log Analysis Server**: Text-to-SQL with Claude Sonnet 4.5 ([README](../../services/log-analysis-server/README.md))
|
|
134
|
+
- **Frontend Dashboard**: Svelte 5 web interface ([README](../../frontend/README.md))
|
|
135
|
+
- **Python Client**: Python async log collection ([README](../python/README.md))
|
|
136
|
+
- **Database Schema**: PostgreSQL 15 with 21 fields ([schema.sql](../../database/schema.sql))
|
|
137
|
+
|
|
138
|
+
### Quick System Setup
|
|
139
|
+
|
|
140
|
+
For a complete local environment with all components:
|
|
141
|
+
|
|
142
|
+
```bash
|
|
143
|
+
# From root directory
|
|
144
|
+
docker-compose up -d
|
|
145
|
+
# Starts: PostgreSQL, Log Save Server, Log Analysis Server, Frontend
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
See [QUICKSTART.md](../../QUICKSTART.md) for detailed setup.
|
|
149
|
+
|
|
150
|
+
## ✨ 주요 기능
|
|
151
|
+
|
|
152
|
+
- ⚡ **비블로킹 로깅** - 앱 블로킹 < 0.01ms (Web Worker/Worker Threads)
|
|
153
|
+
- 🚀 **배치 전송** - 1000건 or 1초마다 자동 전송
|
|
154
|
+
- 📦 **자동 압축** - gzip 압축으로 네트워크 비용 절감
|
|
155
|
+
- 🔄 **Graceful Shutdown** - 앱 종료 시 큐 자동 flush
|
|
156
|
+
- 🎯 **자동 필드 수집** - 호출 위치, HTTP 컨텍스트, 사용자 컨텍스트 자동 포함
|
|
157
|
+
- 🌐 **웹 프레임워크 통합** - Express, Fastify, Koa 지원
|
|
158
|
+
- 🔍 **분산 추적** - trace_id로 마이크로서비스 간 요청 추적
|
|
159
|
+
|
|
160
|
+
## 📦 Installation
|
|
161
|
+
|
|
162
|
+
```bash
|
|
163
|
+
npm install log-collector-async
|
|
164
|
+
# or
|
|
165
|
+
yarn add log-collector-async
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
## 💡 Basic Usage
|
|
169
|
+
|
|
170
|
+
### Node.js
|
|
171
|
+
|
|
172
|
+
```javascript
|
|
173
|
+
import { createLogClient } from 'log-collector-async';
|
|
174
|
+
|
|
175
|
+
// Initialize with options
|
|
176
|
+
const logger = createLogClient('http://localhost:8000', {
|
|
177
|
+
service: 'my-service',
|
|
178
|
+
environment: 'production',
|
|
179
|
+
serviceVersion: 'v1.0.0'
|
|
180
|
+
});
|
|
181
|
+
|
|
182
|
+
// Send logs (non-blocking, batched automatically)
|
|
183
|
+
logger.info('Application started');
|
|
184
|
+
logger.warn('High memory usage detected', { memory_mb: 512 });
|
|
185
|
+
logger.error('Database connection failed', { db_host: 'localhost' });
|
|
186
|
+
|
|
187
|
+
// Automatic graceful shutdown on process exit
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
### Browser
|
|
191
|
+
|
|
192
|
+
```javascript
|
|
193
|
+
import { WebWorkerLogClient } from 'log-collector-async/browser';
|
|
194
|
+
|
|
195
|
+
const logger = new WebWorkerLogClient('http://localhost:8000', {
|
|
196
|
+
service: 'web-app',
|
|
197
|
+
environment: 'production'
|
|
198
|
+
});
|
|
199
|
+
|
|
200
|
+
logger.info('User action', { page: '/dashboard' });
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
### Environment Variables
|
|
204
|
+
|
|
205
|
+
`.env` file or environment variables (Vite, Webpack supported):
|
|
206
|
+
```bash
|
|
207
|
+
VITE_LOG_SERVER_URL=http://localhost:8000
|
|
208
|
+
VITE_SERVICE_NAME=payment-api
|
|
209
|
+
VITE_ENVIRONMENT=production
|
|
210
|
+
VITE_SERVICE_VERSION=v1.2.3
|
|
211
|
+
VITE_LOG_TYPE=BACKEND
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
```javascript
|
|
215
|
+
// Auto-load from environment variables
|
|
216
|
+
const logger = createLogClient();
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
## 🎯 Feature 1: 자동 호출 위치 추적
|
|
220
|
+
|
|
221
|
+
**모든 로그에 `function_name`, `file_path` 자동 포함!**
|
|
222
|
+
|
|
223
|
+
```javascript
|
|
224
|
+
function processPayment(amount) {
|
|
225
|
+
logger.info('Processing payment', { amount });
|
|
226
|
+
// → function_name="processPayment", file_path="/app/payment.js" 자동 포함!
|
|
227
|
+
}
|
|
228
|
+
|
|
229
|
+
// 비활성화도 가능
|
|
230
|
+
logger.log('INFO', 'Manual log', { autoCaller: false });
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
**PostgreSQL 분석:**
|
|
234
|
+
```sql
|
|
235
|
+
SELECT function_name, COUNT(*) as call_count
|
|
236
|
+
FROM logs
|
|
237
|
+
WHERE created_at > NOW() - INTERVAL '1 hour'
|
|
238
|
+
GROUP BY function_name
|
|
239
|
+
ORDER BY call_count DESC;
|
|
240
|
+
```
|
|
241
|
+
|
|
242
|
+
## 🌐 Feature 2: HTTP 컨텍스트 자동 수집
|
|
243
|
+
|
|
244
|
+
**웹 프레임워크 환경에서 `path`, `method`, `ip` 자동 포함!**
|
|
245
|
+
|
|
246
|
+
### Express 통합
|
|
247
|
+
|
|
248
|
+
```javascript
|
|
249
|
+
import express from 'express';
|
|
250
|
+
import crypto from 'crypto';
|
|
251
|
+
import { createLogClient } from 'log-collector-async';
|
|
252
|
+
|
|
253
|
+
const app = express();
|
|
254
|
+
const logger = createLogClient('http://localhost:8000');
|
|
255
|
+
|
|
256
|
+
// HTTP 컨텍스트 미들웨어 - 요청마다 컨텍스트 객체 생성
|
|
257
|
+
app.use((req, res, next) => {
|
|
258
|
+
// 요청 컨텍스트를 req 객체에 저장
|
|
259
|
+
req.logContext = {
|
|
260
|
+
path: req.path,
|
|
261
|
+
method: req.method,
|
|
262
|
+
ip: req.ip,
|
|
263
|
+
trace_id: req.headers['x-trace-id'] || crypto.randomUUID().replace(/-/g, '').substring(0, 32)
|
|
264
|
+
};
|
|
265
|
+
|
|
266
|
+
// 사용자 ID가 있으면 추가
|
|
267
|
+
if (req.headers['x-user-id']) {
|
|
268
|
+
req.logContext.user_id = req.headers['x-user-id'];
|
|
269
|
+
}
|
|
270
|
+
|
|
271
|
+
// 요청 시작 로그
|
|
272
|
+
logger.info('Request received', req.logContext);
|
|
273
|
+
|
|
274
|
+
const startTime = Date.now();
|
|
275
|
+
|
|
276
|
+
// 응답 완료 시 로그
|
|
277
|
+
res.on('finish', () => {
|
|
278
|
+
const duration_ms = Date.now() - startTime;
|
|
279
|
+
logger.info('Request completed', {
|
|
280
|
+
...req.logContext,
|
|
281
|
+
status_code: res.statusCode,
|
|
282
|
+
duration_ms
|
|
283
|
+
});
|
|
284
|
+
});
|
|
285
|
+
|
|
286
|
+
next();
|
|
287
|
+
});
|
|
288
|
+
|
|
289
|
+
app.get('/api/users/:userId', (req, res) => {
|
|
290
|
+
// 라우트 핸들러에서 컨텍스트를 메타데이터로 전달
|
|
291
|
+
logger.info(`Fetching user ${req.params.userId}`, {
|
|
292
|
+
...req.logContext,
|
|
293
|
+
user_id_param: req.params.userId
|
|
294
|
+
});
|
|
295
|
+
// → path, method, ip, trace_id 모두 자동 포함!
|
|
296
|
+
res.json({ userId: req.params.userId });
|
|
297
|
+
});
|
|
298
|
+
|
|
299
|
+
app.post('/api/todos', (req, res) => {
|
|
300
|
+
logger.info('Creating todo', {
|
|
301
|
+
...req.logContext,
|
|
302
|
+
todo_text: req.body.text
|
|
303
|
+
});
|
|
304
|
+
// ... handle todo creation
|
|
305
|
+
res.json({ success: true });
|
|
306
|
+
});
|
|
307
|
+
|
|
308
|
+
app.listen(3000);
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
### Fastify 통합
|
|
312
|
+
|
|
313
|
+
```javascript
|
|
314
|
+
import Fastify from 'fastify';
|
|
315
|
+
import crypto from 'crypto';
|
|
316
|
+
import { createLogClient } from 'log-collector-async';
|
|
317
|
+
|
|
318
|
+
const fastify = Fastify();
|
|
319
|
+
const logger = createLogClient('http://localhost:8000');
|
|
320
|
+
|
|
321
|
+
// onRequest Hook: HTTP 컨텍스트 생성
|
|
322
|
+
fastify.addHook('onRequest', async (request, reply) => {
|
|
323
|
+
// 요청 컨텍스트를 request 객체에 저장
|
|
324
|
+
request.logContext = {
|
|
325
|
+
path: request.url,
|
|
326
|
+
method: request.method,
|
|
327
|
+
ip: request.ip,
|
|
328
|
+
trace_id: request.headers['x-trace-id'] || crypto.randomUUID().replace(/-/g, '').substring(0, 32)
|
|
329
|
+
};
|
|
330
|
+
|
|
331
|
+
// 사용자 ID가 있으면 추가
|
|
332
|
+
if (request.headers['x-user-id']) {
|
|
333
|
+
request.logContext.user_id = request.headers['x-user-id'];
|
|
334
|
+
}
|
|
335
|
+
|
|
336
|
+
// 요청 시작 시간 기록
|
|
337
|
+
request.startTime = Date.now();
|
|
338
|
+
|
|
339
|
+
// 요청 시작 로그
|
|
340
|
+
logger.info('Request received', request.logContext);
|
|
341
|
+
});
|
|
342
|
+
|
|
343
|
+
// onResponse Hook: 응답 완료 로그
|
|
344
|
+
fastify.addHook('onResponse', async (request, reply) => {
|
|
345
|
+
const duration_ms = Date.now() - request.startTime;
|
|
346
|
+
logger.info('Request completed', {
|
|
347
|
+
...request.logContext,
|
|
348
|
+
status_code: reply.statusCode,
|
|
349
|
+
duration_ms
|
|
350
|
+
});
|
|
351
|
+
});
|
|
352
|
+
|
|
353
|
+
fastify.get('/api/users/:userId', async (request, reply) => {
|
|
354
|
+
// 라우트 핸들러에서 컨텍스트를 메타데이터로 전달
|
|
355
|
+
logger.info(`Fetching user ${request.params.userId}`, {
|
|
356
|
+
...request.logContext,
|
|
357
|
+
user_id_param: request.params.userId
|
|
358
|
+
});
|
|
359
|
+
// → path, method, ip, trace_id 모두 자동 포함!
|
|
360
|
+
return { userId: request.params.userId };
|
|
361
|
+
});
|
|
362
|
+
|
|
363
|
+
fastify.post('/api/todos', async (request, reply) => {
|
|
364
|
+
logger.info('Creating todo', {
|
|
365
|
+
...request.logContext,
|
|
366
|
+
todo_text: request.body.text
|
|
367
|
+
});
|
|
368
|
+
// ... handle todo creation
|
|
369
|
+
return { success: true };
|
|
370
|
+
});
|
|
371
|
+
|
|
372
|
+
await fastify.listen({ port: 3000 });
|
|
373
|
+
```
|
|
374
|
+
|
|
375
|
+
### Koa 통합
|
|
376
|
+
|
|
377
|
+
```javascript
|
|
378
|
+
import Koa from 'koa';
|
|
379
|
+
import crypto from 'crypto';
|
|
380
|
+
import { createLogClient } from 'log-collector-async';
|
|
381
|
+
|
|
382
|
+
const app = new Koa();
|
|
383
|
+
const logger = createLogClient('http://localhost:8000');
|
|
384
|
+
|
|
385
|
+
// HTTP 컨텍스트 미들웨어
|
|
386
|
+
app.use(async (ctx, next) => {
|
|
387
|
+
// 요청 컨텍스트를 ctx.state에 저장
|
|
388
|
+
ctx.state.logContext = {
|
|
389
|
+
path: ctx.path,
|
|
390
|
+
method: ctx.method,
|
|
391
|
+
ip: ctx.ip,
|
|
392
|
+
trace_id: ctx.headers['x-trace-id'] || crypto.randomUUID().replace(/-/g, '').substring(0, 32)
|
|
393
|
+
};
|
|
394
|
+
|
|
395
|
+
// 사용자 ID가 있으면 추가
|
|
396
|
+
if (ctx.headers['x-user-id']) {
|
|
397
|
+
ctx.state.logContext.user_id = ctx.headers['x-user-id'];
|
|
398
|
+
}
|
|
399
|
+
|
|
400
|
+
// 요청 시작 시간 기록
|
|
401
|
+
const startTime = Date.now();
|
|
402
|
+
|
|
403
|
+
// 요청 시작 로그
|
|
404
|
+
logger.info('Request received', ctx.state.logContext);
|
|
405
|
+
|
|
406
|
+
try {
|
|
407
|
+
await next();
|
|
408
|
+
|
|
409
|
+
// 응답 완료 로그
|
|
410
|
+
const duration_ms = Date.now() - startTime;
|
|
411
|
+
logger.info('Request completed', {
|
|
412
|
+
...ctx.state.logContext,
|
|
413
|
+
status_code: ctx.status,
|
|
414
|
+
duration_ms
|
|
415
|
+
});
|
|
416
|
+
} catch (err) {
|
|
417
|
+
// 에러 발생 로그
|
|
418
|
+
logger.error('Request failed', {
|
|
419
|
+
...ctx.state.logContext,
|
|
420
|
+
error: err.message,
|
|
421
|
+
stack_trace: err.stack
|
|
422
|
+
});
|
|
423
|
+
throw err;
|
|
424
|
+
}
|
|
425
|
+
});
|
|
426
|
+
|
|
427
|
+
// 라우트 핸들러
|
|
428
|
+
app.use(async (ctx) => {
|
|
429
|
+
if (ctx.path === '/api/users' && ctx.method === 'GET') {
|
|
430
|
+
logger.info('Fetching users', ctx.state.logContext);
|
|
431
|
+
// → path, method, ip, trace_id 모두 자동 포함!
|
|
432
|
+
ctx.body = { users: [] };
|
|
433
|
+
} else if (ctx.path === '/api/todos' && ctx.method === 'POST') {
|
|
434
|
+
logger.info('Creating todo', {
|
|
435
|
+
...ctx.state.logContext,
|
|
436
|
+
todo_text: ctx.request.body.text
|
|
437
|
+
});
|
|
438
|
+
ctx.body = { success: true };
|
|
439
|
+
} else {
|
|
440
|
+
ctx.body = { message: 'Hello' };
|
|
441
|
+
}
|
|
442
|
+
});
|
|
443
|
+
|
|
444
|
+
app.listen(3000);
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
## 👤 Feature 3: 사용자 컨텍스트 관리
|
|
448
|
+
|
|
449
|
+
**`user_id`, `trace_id`, `session_id` 등을 모든 로그에 자동 포함!**
|
|
450
|
+
|
|
451
|
+
### runWithUserContext 방식 (권장)
|
|
452
|
+
|
|
453
|
+
```javascript
|
|
454
|
+
import { createLogClient } from 'log-collector-async';
|
|
455
|
+
|
|
456
|
+
const logger = createLogClient('http://localhost:8000');
|
|
457
|
+
|
|
458
|
+
// 특정 블록에만 컨텍스트 적용
|
|
459
|
+
logger.constructor.runWithUserContext({
|
|
460
|
+
user_id: 'user_123',
|
|
461
|
+
trace_id: 'trace_xyz',
|
|
462
|
+
session_id: 'sess_abc'
|
|
463
|
+
}, () => {
|
|
464
|
+
logger.info('User logged in');
|
|
465
|
+
// → user_id, trace_id, session_id 자동 포함!
|
|
466
|
+
|
|
467
|
+
processPayment();
|
|
468
|
+
logger.info('Payment completed');
|
|
469
|
+
// → 하위 함수에서도 자동으로 컨텍스트 유지!
|
|
470
|
+
});
|
|
471
|
+
|
|
472
|
+
// 블록 벗어나면 자동 초기화
|
|
473
|
+
```
|
|
474
|
+
|
|
475
|
+
### 비동기 함수와 함께 사용
|
|
476
|
+
|
|
477
|
+
```javascript
|
|
478
|
+
await logger.constructor.runWithUserContext({
|
|
479
|
+
user_id: 'user_456',
|
|
480
|
+
trace_id: 'trace_async_123'
|
|
481
|
+
}, async () => {
|
|
482
|
+
logger.info('Async operation started');
|
|
483
|
+
// → user_id, trace_id 자동 포함
|
|
484
|
+
|
|
485
|
+
await fetchUserData();
|
|
486
|
+
await processData();
|
|
487
|
+
|
|
488
|
+
logger.info('Async operation completed');
|
|
489
|
+
// → 같은 컨텍스트 유지됨
|
|
490
|
+
});
|
|
491
|
+
```
|
|
492
|
+
|
|
493
|
+
### 중첩 컨텍스트 (자동 병합)
|
|
494
|
+
|
|
495
|
+
```javascript
|
|
496
|
+
// 외부: tenant_id
|
|
497
|
+
logger.constructor.runWithUserContext({ tenant_id: 'tenant_1' }, () => {
|
|
498
|
+
logger.info('Tenant operation');
|
|
499
|
+
// → tenant_id="tenant_1"
|
|
500
|
+
|
|
501
|
+
// 내부: user_id 추가
|
|
502
|
+
logger.constructor.runWithUserContext({ user_id: 'user_123' }, () => {
|
|
503
|
+
logger.info('User operation');
|
|
504
|
+
// → tenant_id="tenant_1", user_id="user_123" 둘 다 포함!
|
|
505
|
+
});
|
|
506
|
+
});
|
|
507
|
+
```
|
|
508
|
+
|
|
509
|
+
### 분산 추적 (Distributed Tracing)
|
|
510
|
+
|
|
511
|
+
```javascript
|
|
512
|
+
import { v4 as uuidv4 } from 'uuid';
|
|
513
|
+
|
|
514
|
+
function handleRequest() {
|
|
515
|
+
// 요청마다 고유한 trace_id 생성 (32자, 대시 제거)
|
|
516
|
+
const traceId = uuidv4().replace(/-/g, '');
|
|
517
|
+
|
|
518
|
+
logger.constructor.runWithUserContext({
|
|
519
|
+
trace_id: traceId,
|
|
520
|
+
user_id: 'user_123'
|
|
521
|
+
}, () => {
|
|
522
|
+
logger.info('Request received');
|
|
523
|
+
callServiceA(); // Service A 호출
|
|
524
|
+
callServiceB(); // Service B 호출
|
|
525
|
+
logger.info('Request completed');
|
|
526
|
+
// → 모든 로그가 같은 trace_id로 추적 가능!
|
|
527
|
+
});
|
|
528
|
+
}
|
|
529
|
+
```
|
|
530
|
+
|
|
531
|
+
**PostgreSQL 분석:**
|
|
532
|
+
```sql
|
|
533
|
+
-- trace_id로 전체 요청 흐름 추적
|
|
534
|
+
SELECT created_at, service, function_name, message, duration_ms
|
|
535
|
+
FROM logs
|
|
536
|
+
WHERE metadata->>'trace_id' = 'your-trace-id'
|
|
537
|
+
ORDER BY created_at;
|
|
538
|
+
```
|
|
539
|
+
|
|
540
|
+
### Set/Clear 방식
|
|
541
|
+
|
|
542
|
+
```javascript
|
|
543
|
+
// 로그인 시
|
|
544
|
+
logger.constructor.setUserContext({
|
|
545
|
+
user_id: 'user_123',
|
|
546
|
+
session_id: 'sess_abc'
|
|
547
|
+
});
|
|
548
|
+
|
|
549
|
+
logger.info('User action');
|
|
550
|
+
// → user_id, session_id 자동 포함
|
|
551
|
+
|
|
552
|
+
// 로그아웃 시
|
|
553
|
+
logger.constructor.clearUserContext();
|
|
554
|
+
```
|
|
555
|
+
|
|
556
|
+
## 🔧 고급 기능
|
|
557
|
+
|
|
558
|
+
### 타이머 측정
|
|
559
|
+
|
|
560
|
+
```javascript
|
|
561
|
+
// 수동 타이머
|
|
562
|
+
const timer = logger.startTimer();
|
|
563
|
+
const result = expensiveOperation();
|
|
564
|
+
logger.endTimer(timer, 'INFO', 'Operation completed');
|
|
565
|
+
// → duration_ms 자동 계산
|
|
566
|
+
|
|
567
|
+
// 함수 래퍼 (동기/비동기 자동 감지)
|
|
568
|
+
const result = logger.measure(() => expensiveOperation());
|
|
569
|
+
```
|
|
570
|
+
|
|
571
|
+
### 에러 추적
|
|
572
|
+
|
|
573
|
+
```javascript
|
|
574
|
+
try {
|
|
575
|
+
riskyOperation();
|
|
576
|
+
} catch (err) {
|
|
577
|
+
logger.errorWithTrace('Operation failed', err);
|
|
578
|
+
// → stack_trace, error_type, function_name, file_path 자동 포함!
|
|
579
|
+
}
|
|
580
|
+
```
|
|
581
|
+
|
|
582
|
+
### 수동 Flush
|
|
583
|
+
|
|
584
|
+
```javascript
|
|
585
|
+
// 중요한 로그를 즉시 전송
|
|
586
|
+
logger.flush();
|
|
587
|
+
```
|
|
588
|
+
|
|
589
|
+
### 클라이언트 종료
|
|
590
|
+
|
|
591
|
+
```javascript
|
|
592
|
+
// Graceful shutdown
|
|
593
|
+
await logger.close();
|
|
594
|
+
```
|
|
595
|
+
|
|
596
|
+
## ⚙️ 설정 옵션
|
|
597
|
+
|
|
598
|
+
```javascript
|
|
599
|
+
const logger = createLogClient('http://localhost:8000', {
|
|
600
|
+
service: 'payment-api',
|
|
601
|
+
environment: 'production',
|
|
602
|
+
serviceVersion: 'v1.2.3',
|
|
603
|
+
logType: 'BACKEND',
|
|
604
|
+
batchSize: 1000, // 배치 크기 (기본: 1000)
|
|
605
|
+
flushInterval: 1000, // Flush 간격 ms (기본: 1000)
|
|
606
|
+
enableCompression: true // gzip 압축 (기본: true)
|
|
607
|
+
});
|
|
608
|
+
```
|
|
609
|
+
|
|
610
|
+
## 📊 성능
|
|
611
|
+
|
|
612
|
+
- **앱 블로킹**: < 0.01ms per log (Web Worker/Worker Threads)
|
|
613
|
+
- **처리량**: > 10,000 logs/sec
|
|
614
|
+
- **메모리**: < 10MB (1000건 큐)
|
|
615
|
+
- **압축률**: ~70% (100건 이상 시 자동 압축)
|
|
616
|
+
|
|
617
|
+
## 🧪 테스트
|
|
618
|
+
|
|
619
|
+
```bash
|
|
620
|
+
# 단위 테스트
|
|
621
|
+
npm test
|
|
622
|
+
|
|
623
|
+
# 통합 테스트 (로그 서버 필요)
|
|
624
|
+
npm run test:integration
|
|
625
|
+
|
|
626
|
+
# 커버리지
|
|
627
|
+
npm run test:coverage
|
|
628
|
+
```
|
|
629
|
+
|
|
630
|
+
## 📝 로그 레벨
|
|
631
|
+
|
|
632
|
+
```javascript
|
|
633
|
+
logger.trace('Trace message'); // TRACE
|
|
634
|
+
logger.debug('Debug message'); // DEBUG
|
|
635
|
+
logger.info('Info message'); // INFO
|
|
636
|
+
logger.warn('Warning message'); // WARN
|
|
637
|
+
logger.error('Error message'); // ERROR
|
|
638
|
+
logger.fatal('Fatal message'); // FATAL
|
|
639
|
+
```
|
|
640
|
+
|
|
641
|
+
## 🔍 PostgreSQL 쿼리 예제
|
|
642
|
+
|
|
643
|
+
### 사용자별 로그 조회
|
|
644
|
+
```sql
|
|
645
|
+
SELECT * FROM logs
|
|
646
|
+
WHERE metadata->>'user_id' = 'user_123'
|
|
647
|
+
ORDER BY created_at DESC
|
|
648
|
+
LIMIT 100;
|
|
649
|
+
```
|
|
650
|
+
|
|
651
|
+
### 에러 발생률
|
|
652
|
+
```sql
|
|
653
|
+
SELECT
|
|
654
|
+
metadata->>'path' as path,
|
|
655
|
+
metadata->>'method' as method,
|
|
656
|
+
COUNT(*) as total_requests,
|
|
657
|
+
COUNT(CASE WHEN level = 'ERROR' THEN 1 END) as errors,
|
|
658
|
+
ROUND(100.0 * COUNT(CASE WHEN level = 'ERROR' THEN 1 END) / COUNT(*), 2) as error_rate
|
|
659
|
+
FROM logs
|
|
660
|
+
WHERE created_at > NOW() - INTERVAL '1 hour'
|
|
661
|
+
GROUP BY metadata->>'path', metadata->>'method'
|
|
662
|
+
ORDER BY error_rate DESC;
|
|
663
|
+
```
|
|
664
|
+
|
|
665
|
+
### 함수별 성능
|
|
666
|
+
```sql
|
|
667
|
+
SELECT
|
|
668
|
+
function_name,
|
|
669
|
+
COUNT(*) as calls,
|
|
670
|
+
AVG((metadata->>'duration_ms')::numeric) as avg_ms,
|
|
671
|
+
MAX((metadata->>'duration_ms')::numeric) as max_ms
|
|
672
|
+
FROM logs
|
|
673
|
+
WHERE metadata->>'duration_ms' IS NOT NULL
|
|
674
|
+
GROUP BY function_name
|
|
675
|
+
ORDER BY avg_ms DESC;
|
|
676
|
+
```
|
|
677
|
+
|
|
678
|
+
## 🚨 주의사항
|
|
679
|
+
|
|
680
|
+
1. **민감한 정보 포함 금지**
|
|
681
|
+
```javascript
|
|
682
|
+
// ❌ 절대 안 됨!
|
|
683
|
+
logger.info('Login', { password: 'secret' });
|
|
684
|
+
|
|
685
|
+
// ✅ 식별자만 사용
|
|
686
|
+
logger.info('Login successful', { user_id: 'user_123' });
|
|
687
|
+
```
|
|
688
|
+
|
|
689
|
+
2. **과도한 로깅 피하기**
|
|
690
|
+
```javascript
|
|
691
|
+
// ❌ 루프 내부에서 과도한 로깅
|
|
692
|
+
for (let i = 0; i < 10000; i++) {
|
|
693
|
+
logger.debug(`Processing ${i}`);
|
|
694
|
+
}
|
|
695
|
+
|
|
696
|
+
// ✅ 주요 이벤트만 로깅
|
|
697
|
+
logger.info('Batch processing started', { count: 10000 });
|
|
698
|
+
```
|
|
699
|
+
|
|
700
|
+
## 🔧 Troubleshooting
|
|
701
|
+
|
|
702
|
+
### Logs not appearing in database
|
|
703
|
+
|
|
704
|
+
**Symptoms**:
|
|
705
|
+
- `logger.info()` runs without errors
|
|
706
|
+
- No logs visible in PostgreSQL
|
|
707
|
+
- No errors in console
|
|
708
|
+
|
|
709
|
+
**Checklist**:
|
|
710
|
+
1. ✅ **Log server running?**
|
|
711
|
+
```bash
|
|
712
|
+
curl http://localhost:8000/
|
|
713
|
+
# Should return: {"status": "ok"}
|
|
714
|
+
```
|
|
715
|
+
|
|
716
|
+
2. ✅ **PostgreSQL running?**
|
|
717
|
+
```bash
|
|
718
|
+
psql -h localhost -U postgres -d logs_db -c "SELECT 1;"
|
|
719
|
+
```
|
|
720
|
+
|
|
721
|
+
3. ✅ **Schema created?**
|
|
722
|
+
```bash
|
|
723
|
+
psql -h localhost -U postgres -d logs_db -c "\dt"
|
|
724
|
+
# Should show 'logs' table
|
|
725
|
+
```
|
|
726
|
+
|
|
727
|
+
4. ✅ **Batch flushed?**
|
|
728
|
+
- Wait 1 second (default flush interval)
|
|
729
|
+
- OR manually flush: `await logger.close()`
|
|
730
|
+
|
|
731
|
+
5. ✅ **Check server logs**:
|
|
732
|
+
```bash
|
|
733
|
+
cd services/log-save-server
|
|
734
|
+
docker-compose logs -f
|
|
735
|
+
# Look for "Received X logs" messages
|
|
736
|
+
```
|
|
737
|
+
|
|
738
|
+
---
|
|
739
|
+
|
|
740
|
+
### "Connection refused" errors
|
|
741
|
+
|
|
742
|
+
**Symptoms**:
|
|
743
|
+
```
|
|
744
|
+
Error: connect ECONNREFUSED 127.0.0.1:8000
|
|
745
|
+
```
|
|
746
|
+
|
|
747
|
+
**Cause**: Log server not running
|
|
748
|
+
|
|
749
|
+
**Solution**:
|
|
750
|
+
```bash
|
|
751
|
+
cd services/log-save-server
|
|
752
|
+
docker-compose up -d
|
|
753
|
+
|
|
754
|
+
# Verify it's running
|
|
755
|
+
curl http://localhost:8000/
|
|
756
|
+
```
|
|
757
|
+
|
|
758
|
+
---
|
|
759
|
+
|
|
760
|
+
### CORS errors (browser only)
|
|
761
|
+
|
|
762
|
+
**Symptoms**:
|
|
763
|
+
```
|
|
764
|
+
Access to fetch at 'http://localhost:8000/logs' from origin 'http://localhost:3000'
|
|
765
|
+
has been blocked by CORS policy
|
|
766
|
+
```
|
|
767
|
+
|
|
768
|
+
**Cause**: Log server missing CORS configuration
|
|
769
|
+
|
|
770
|
+
**Solution**: Ensure log server has CORS middleware enabled (already configured in log-save-server)
|
|
771
|
+
|
|
772
|
+
---
|
|
773
|
+
|
|
774
|
+
### High memory usage
|
|
775
|
+
|
|
776
|
+
**Symptoms**:
|
|
777
|
+
- Application memory grows over time
|
|
778
|
+
- Eventually crashes with OOM error
|
|
779
|
+
|
|
780
|
+
**Cause**: Batch size too large or flush interval too long
|
|
781
|
+
|
|
782
|
+
**Solution**: Reduce batching parameters
|
|
783
|
+
```javascript
|
|
784
|
+
const logger = createLogClient('http://localhost:8000', {
|
|
785
|
+
batchSize: 500, // Reduce from 1000
|
|
786
|
+
flushInterval: 500 // Reduce from 1000
|
|
787
|
+
});
|
|
788
|
+
```
|
|
789
|
+
|
|
790
|
+
---
|
|
791
|
+
|
|
792
|
+
### Logs delayed or not sent on app shutdown
|
|
793
|
+
|
|
794
|
+
**Symptoms**:
|
|
795
|
+
- Last few logs before shutdown are missing
|
|
796
|
+
- Queue not flushing properly
|
|
797
|
+
|
|
798
|
+
**Cause**: App exits before background worker flushes
|
|
799
|
+
|
|
800
|
+
**Solution**: Call close before exit
|
|
801
|
+
```javascript
|
|
802
|
+
// Graceful shutdown
|
|
803
|
+
process.on('SIGTERM', async () => {
|
|
804
|
+
await logger.close(); // Flushes queue before closing
|
|
805
|
+
process.exit(0);
|
|
806
|
+
});
|
|
807
|
+
|
|
808
|
+
// Or manually before exit
|
|
809
|
+
await logger.close();
|
|
810
|
+
```
|
|
811
|
+
|
|
812
|
+
---
|
|
813
|
+
|
|
814
|
+
### Worker thread/Web Worker not starting
|
|
815
|
+
|
|
816
|
+
**Symptoms**:
|
|
817
|
+
- Console warnings about worker initialization
|
|
818
|
+
- Logs sent synchronously instead of async
|
|
819
|
+
|
|
820
|
+
**Cause**: Worker script not found or CORS issues in browser
|
|
821
|
+
|
|
822
|
+
**Solution (Node.js)**: Ensure worker script is included in deployment
|
|
823
|
+
```javascript
|
|
824
|
+
// worker-node.js should be in node_modules/log-collector-async/dist/
|
|
825
|
+
```
|
|
826
|
+
|
|
827
|
+
**Solution (Browser)**: Serve worker script with correct MIME type
|
|
828
|
+
```javascript
|
|
829
|
+
// Ensure worker.js is served as application/javascript
|
|
830
|
+
```
|
|
831
|
+
|
|
832
|
+
## 📋 Version Compatibility
|
|
833
|
+
|
|
834
|
+
| Component | Minimum Version | Tested Version | Notes |
|
|
835
|
+
|-----------|----------------|----------------|-------|
|
|
836
|
+
| **This Client** | 1.0.0 | 1.0.0 | Current release |
|
|
837
|
+
| **Log Save Server** | 1.0.0 | 1.0.0 | FastAPI 0.104+ |
|
|
838
|
+
| **PostgreSQL** | 12 | 15 | Requires JSONB support |
|
|
839
|
+
| **Log Analysis Server** | 1.0.0 | 1.0.0 | Optional (for Text-to-SQL) |
|
|
840
|
+
| **Node.js** | 14 | 18 | Runtime environment |
|
|
841
|
+
|
|
842
|
+
### Breaking Changes
|
|
843
|
+
|
|
844
|
+
- **v1.0.0**: Initial release
|
|
845
|
+
|
|
846
|
+
### Upgrade Guide
|
|
847
|
+
|
|
848
|
+
No upgrades yet. This is the initial release.
|
|
849
|
+
|
|
850
|
+
## 📚 추가 문서
|
|
851
|
+
|
|
852
|
+
- [HTTP-CONTEXT-GUIDE.md](../HTTP-CONTEXT-GUIDE.md) - HTTP 컨텍스트 완전 가이드
|
|
853
|
+
- [USER-CONTEXT-GUIDE.md](../USER-CONTEXT-GUIDE.md) - 사용자 컨텍스트 완전 가이드
|
|
854
|
+
- [FIELD-AUTO-COLLECTION.md](../FIELD-AUTO-COLLECTION.md) - 자동 필드 수집 상세
|
|
855
|
+
|
|
856
|
+
## 🤝 기여
|
|
857
|
+
|
|
858
|
+
기여는 언제나 환영합니다!
|
|
859
|
+
|
|
860
|
+
## 📄 라이선스
|
|
861
|
+
|
|
862
|
+
MIT License - 자유롭게 사용하세요!
|
|
863
|
+
|
|
864
|
+
---
|
|
865
|
+
|
|
866
|
+
**Made with ❤️ by Log Analysis System Team**
|