http-log-replay 1.0.0 → 1.1.1

package/README.md

@@ -49,16 +49,20 @@ This will launch the **Traffic Mirror Dashboard** at [http://localhost:4200](htt
 The Web UI is the best way to get started. It guides you through the entire workflow in three simple tabs.
 
 ### 1. 🔴 Record / Generate
-- **Manual Mode**: Start the proxy, point your application/client to it, and use your app normally. Requests are saved to a file.
+
+- **Manual Mode**: Start the proxy, point your application/client to it, and select desired HTTP methods (e.g., GET, POST).
 - **Auto-Generate**: Upload an OpenAPI/Swagger JSON file to automatically generate realistic traffic patterns.
+- **YAML Config Mode**: Load a `config.yaml` file to pre-configure Traffic Generation (target, source, exclusions, timeouts, methods).
 
 ### 2. ▶️ Replay
+
 - Configure your **Primary** (Stable) and **Secondary** (Test) environments.
 - Set **Concurrency** to speed up large suites.
 - Add **Ignore Fields** (e.g., `createdAt`, `traceId`) to filter out noise in the comparison.
 - Click **Replay & Compare**.
 
 ### 3. 📄 Report
+
 - Instantly view the results.
 - **Green**: Exact match.
 - **Red**: Mismatch (click to expand the JSON diff).
@@ -70,24 +74,36 @@ The Web UI is the best way to get started. It guides you through the entire work
 Perfect for **CI/CD pipelines** or headless environments.
 
 ### 1. Record Traffic
+
 Start a recording proxy on port `3000` forwarding to your real API at `localhost:8080`.
 
 ```bash
-npx http-log-replay record --target http://localhost:8080 --port 3000 --out traffic.jsonl
+# Run from source
+node index.js record --target http://localhost:8080 --port 3000 --out traffic.jsonl
 ```
 
 ### 2. Auto-Generate from Swagger
-Generate traffic without manual clicking.
+
+Generate traffic without manual clicking. You can use command line flags or a YAML configuration file.
+
+**Using Configuration File (Recommended):**
 
 ```bash
-npx http-log-replay generate --file ./openapi.json --target http://localhost:3000
+node index.js generate --config config.example.yaml
+```
+
+**Using Flags:**
+
+```bash
+node index.js generate --file ./openapi.json --target http://localhost:3000
 ```
 
 ### 3. Replay & Verify
+
 Replay recorded traffic against two environments and generate an HTML report.
 
 ```bash
-npx http-log-replay replay \
+node index.js replay \
   --log traffic.jsonl \
   --primary http://prod-api.com \
   --secondary http://staging-api.com \
@@ -102,18 +118,23 @@ npx http-log-replay replay \
 We provide optimized Docker images for both the UI and CLI.
 
 ### Prerequisites
+
 - Docker & Docker Compose installed.
 
 ### Option A: Complete Environment (Recommended)
+
 Use the included `docker-compose.yml` to run everything.
 
 **Start the GUI:**
+
 ```bash
 docker-compose up gui
 ```
+
 > Access at [http://localhost:4200](http://localhost:4200). Data is persisted to your host folder.
 
 **Run CLI Commands:**
+
 ```bash
 docker-compose run cli record --target http://host.docker.internal:8080 ...
 ```
@@ -121,11 +142,13 @@ docker-compose run cli record --target http://host.docker.internal:8080 ...
 ### Option B: Manual Docker Run
 
 **UI Image:**
+
 ```bash
 docker run -p 4200:4200 -v $(pwd):/app traffic-mirror-gui
 ```
 
 **CLI Image:**
+
 ```bash
 docker run -v $(pwd):/app traffic-mirror-cli --help
 ```
@@ -137,6 +160,7 @@ docker run -v $(pwd):/app traffic-mirror-cli --help
 We welcome contributions! Here is how to run the project locally for development.
 
 ### 1. Setup
+
 ```bash
 git clone https://github.com/your-username/http-log-replay.git
 cd http-log-replay
@@ -144,7 +168,9 @@ npm install
 ```
 
 ### 2. Build the UI
+
 The UI is built with Angular. You must build it before running the app.
+
 ```bash
 cd ui
 npm install
@@ -153,12 +179,14 @@ cd ..
 ```
 
 ### 3. Run Locally (Dev Mode)
+
 ```bash
 # Start the full app (Frontend + Backend)
 node index.js ui --port 4200
 ```
 
 ### 4. Testing & Code Quality
+
 We use **Jest** for testing and **ESLint** for code quality.
 
 ```bash
@@ -170,13 +198,35 @@ npm run lint
 
 # Format Code
 npm run format
+
+# Production Start (PM2)
+npm run start:pm2
 ```
 
+### 5. Contribution Guidelines
+
+We use **Semantic Release** to automate version management and package publishing. To ensure this works correctly, please follow the **Conventional Commits** specification for your commit messages.
+
+**Commit Message Format:**
+
+```
+<type>(<scope>): <subject>
+```
+
+**Common Types:**
+
+- **`feat`**: A new feature (triggers a **MINOR** release).
+- **`fix`**: A bug fix (triggers a **PATCH** release).
+- **`docs`**: Documentation only changes (triggers a **PATCH** release).
+- **`refactor`**: A code change that neither fixes a bug nor adds a feature (triggers a **PATCH** release).
+- **`perf`**, **`test`**, **`ci`**, **`chore`**: Changes that do not trigger a release (unless configured otherwise).
+
 ---
 
 ## 📦 Maintenance
 
 ### Publishing to NPM
+
 1.  **Bump Version**: Update `version` in `package.json`.
 2.  **Build UI**: The `prepublishOnly` script will automatically build the Angular UI.
 3.  **Publish**:
@@ -185,6 +235,7 @@ npm run format
     ```
 
 ### File Structure
+
 - `index.js`: CLI entry point.
 - `recorder.js`: Proxy logic.
 - `replayer.js`: Replay & Diff logic.

package/index.js

@@ -1,6 +1,8 @@
 #!/usr/bin/env node
 /* eslint-disable no-console */
 const { Command } = require('commander');
+const fs = require('fs');
+const yaml = require('js-yaml');
 const recorder = require('./recorder');
 const replayAndDiff = require('./replayer');
 const startServer = require('./server');
@@ -64,26 +66,34 @@ program
 
 program
   .command('generate')
-  .description('Auto-generate traffic from Swagger file')
-  .option('-t, --target <url>', 'Proxy URL', 'http://localhost:3000')
-  .option('-f, --file <path>', 'Swagger file path', './full_documentation.json')
-  .option('-x, --exclude <items>', 'Comma separated list of endpoints to exclude', (val) =>
+  .description('Auto-generate traffic from Swagger file using a configuration file')
+  .option('-c, --config <path>', 'Path to YAML configuration file', 'config.yaml')
+  .option('-m, --methods <items>', 'Comma separated list of HTTP methods', (val) =>
     val.split(',')
-  ) // <--- NEW OPTION
-  .option('-s, --source <url>', 'Source Server URL', 'http://localhost:1338')
+  )
   .action(async (options) => {
     try {
       console.log('🚀 Starting Traffic Generation...');
 
-      // Pass options.exclude (or empty array) as 3rd arg
+      let config = {};
+      if (options.config && fs.existsSync(options.config)) {
+        console.log(`Loading configuration from ${options.config}...`);
+        const raw = fs.readFileSync(options.config, 'utf8');
+        config = yaml.load(raw);
+        
+        // Override methods if provided via CLI
+        if (options.methods && options.methods.length > 0) {
+           config.methods = options.methods;
+        }
+      } else {
+         throw new Error(`Configuration file not found at ${options.config}`);
+      }
+
       await generator.run(
-        options.target,
-        options.file,
-        options.exclude || [],
+        config,
         (log) => {
           console.log(log.message);
-        },
-        options.source
+        }
       );
 
       console.log('✅ Done.');

package/package.json

@@ -1,55 +1,60 @@
-{
-  "name": "http-log-replay",
-  "version": "1.0.0",
-  "description": "Traffic Mirror is a regression testing tool that records HTTP traffic from a live environment and replays it against two different environments (e.g., Stable vs. Canary) to detect differences in responses.",
-  "author": "Xhani Manolis Trungu",
-  "main": "index.js",
-  "bin": {
-    "traffic-mirror": "./index.js"
-  },
-  "files": [
-    "*.js",
-    "ui/dist"
-  ],
-  "scripts": {
-    "test": "jest tests/",
-    "lint": "eslint .",
-    "format": "prettier --write .",
-    "start:pm2": "pm2 start ecosystem.config.js",
-    "prepublishOnly": "cd ui && npm install && npm run build"
-  },
-  "keywords": [
-    "http-log",
-    "http-log-replay",
-    "traffic-replay",
-    "regression-testing",
-    "api-regression",
-    "api-testing"
-  ],
-  "license": "ISC",
-  "type": "commonjs",
-  "dependencies": {
-    "axios": "^1.13.2",
-    "colors": "^1.4.0",
-    "commander": "^14.0.2",
-    "cors": "^2.8.5",
-    "deep-diff": "^1.0.2",
-    "diff": "^8.0.2",
-    "express": "^5.2.1",
-    "http-proxy": "^1.18.1",
-    "open": "^11.0.0",
-    "readline": "^1.3.0",
-    "socket.io": "^4.8.1"
-  },
-  "devDependencies": {
-    "eslint": "^9.39.2",
-    "eslint-config-prettier": "^10.1.8",
-    "jest": "^30.2.0",
-    "pm2": "^6.0.14",
-    "prettier": "^3.7.4"
-  },
-  "repository": {
-    "type": "git",
-    "url": "https://github.com/xhani-manolis-trungu/traffic-mirror"
-  }
-}
+{
+  "name": "http-log-replay",
+  "version": "1.1.1",
+  "description": "Traffic Mirror is a regression testing tool that records HTTP traffic from a live environment and replays it against two different environments (e.g., Stable vs. Canary) to detect differences in responses.",
+  "author": "Xhani Manolis Trungu",
+  "main": "index.js",
+  "bin": {
+    "http-log-replay": "./index.js"
+  },
+  "files": [
+    "*.js",
+    "ui/dist"
+  ],
+  "scripts": {
+    "test": "jest tests/",
+    "lint": "eslint .",
+    "format": "prettier --write .",
+    "start:pm2": "pm2 start ecosystem.config.js",
+    "prepublishOnly": "cd ui && npm install && npm run build"
+  },
+  "keywords": [
+    "http-log",
+    "http-log-replay",
+    "traffic-replay",
+    "regression-testing",
+    "api-regression",
+    "api-testing"
+  ],
+  "license": "ISC",
+  "type": "commonjs",
+  "dependencies": {
+    "axios": "^1.13.2",
+    "colors": "^1.4.0",
+    "commander": "^14.0.2",
+    "cors": "^2.8.5",
+    "deep-diff": "^1.0.2",
+    "diff": "^8.0.2",
+    "express": "^5.2.1",
+    "http-proxy": "^1.18.1",
+    "js-yaml": "^4.1.1",
+    "open": "^11.0.0",
+    "readline": "^1.3.0",
+    "socket.io": "^4.8.1"
+  },
+  "devDependencies": {
+    "@semantic-release/changelog": "^6.0.3",
+    "@semantic-release/git": "^10.0.1",
+    "conventional-changelog-conventionalcommits": "^8.0.0",
+    "eslint": "^9.39.2",
+    "eslint-config-prettier": "^10.1.8",
+    "jest": "^30.2.0",
+    "pm2": "^6.0.14",
+    "prettier": "^3.7.4",
+    "semantic-release": "^25.0.2"
+  },
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/xhani-manolis-trungu/traffic-mirror"
+  }
+}

package/recorder.js

@@ -1,114 +1,114 @@
-/* eslint-disable no-console */
-const http = require('http');
-const httpProxy = require('http-proxy');
-const fs = require('fs');
-
-class Recorder {
-  constructor() {
-    this.server = null;
-    this.proxy = httpProxy.createProxyServer({});
-
-    // 1. Handle Proxy Errors (Critical for preventing crashes)
-    this.proxy.on('error', (err, req, res) => {
-      console.error(`❌ Proxy Error [${req.url}]:`, err.message);
-      if (!res.headersSent) {
-        res.writeHead(502, { 'Content-Type': 'application/json' });
-      }
-      res.end(JSON.stringify({ error: 'Proxy Request Failed', details: err.message }));
-    });
-  }
-
-  start(target, port, outputFile, onLog = () => { }) {
-    if (this.server) throw new Error('Recorder already running');
-
-    console.log(`📝 Recording to file: ${outputFile}`);
-    const stream = fs.createWriteStream(outputFile, { flags: 'a' });
-
-    // Handle file write errors
-    stream.on('error', (err) => {
-      console.error('❌ File Write Error:', err.message);
-    });
-
-    this.server = http.createServer((req, res) => {
-      // 2. Capture Request Body safely
-      // We use a separate array but we DO NOT attach 'data' listeners directly
-      // unless we plan to buffer it for the proxy.
-      // For simplicity, let's rely on the response capture primarily.
-
-      const reqChunks = [];
-      // We tap into the stream without consuming it destructively if possible,
-      // but the safest way with http-proxy is often just to listen to the proxy events.
-      // HOWEVER, for your specific "Request Stealing" fix:
-      req.on('data', (chunk) => reqChunks.push(chunk));
-
-      // --- Response Capture Logic ---
-      const originalWrite = res.write;
-      const originalEnd = res.end;
-      const resChunks = [];
-
-      res.write = function (chunk, ...args) {
-        if (chunk) resChunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk));
-        return originalWrite.apply(res, [chunk, ...args]);
-      };
-
-      res.end = function (chunk, ...args) {
-        if (chunk) resChunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk));
-
-        // WRAP IN TRY/CATCH to prevent silent crashes
-        try {
-          const reqBody = Buffer.concat(reqChunks).toString();
-          const resBody = Buffer.concat(resChunks).toString('utf8');
-
-          const logData = {
-            timestamp: new Date().toISOString(),
-            method: req.method,
-            url: req.url,
-            status: res.statusCode,
-            requestBody: reqBody || '',
-            responseBody: resBody || '',
-            // requestHeaders: req.headers, // Uncomment if needed (verbose)
-          };
-
-          const jsonLine = JSON.stringify(logData) + '\n';
-
-          // 3. Write and Log
-          stream.write(jsonLine);
-          onLog(logData);
-
-          // Debug Log to prove it worked
-          process.stdout.write('.'); // Prints a dot for every logged request
-        } catch (e) {
-          console.error('\n❌ Error generating log:', e.message);
-        }
-
-        return originalEnd.apply(res, [chunk, ...args]);
-      };
-
-      // 4. Force Plain Text (Disable Gzip)
-      delete req.headers['accept-encoding'];
-
-      // 5. Forward to Target
-      // We must set 'buffer' to the request stream if we consumed it,
-      // but since we are just tapping 'data' without pausing, http-proxy might miss it.
-      // A simple workaround for node streams in this context:
-      this.proxy.web(req, res, { target });
-    });
-
-    this.server.listen(port, () => {
-      console.log(`\n⏺️  Recorder listening on port ${port}`);
-      console.log(`➡️  Forwarding to ${target}`);
-    });
-
-    return `Recorder started`;
-  }
-
-  stop() {
-    if (this.server) {
-      this.server.close();
-      this.server = null;
-      return 'Recorder stopped';
-    }
-  }
-}
-
-module.exports = new Recorder();
+/* eslint-disable no-console */
+const http = require('http');
+const httpProxy = require('http-proxy');
+const fs = require('fs');
+
+class Recorder {
+  constructor() {
+    this.server = null;
+    this.proxy = httpProxy.createProxyServer({});
+
+    // 1. Handle Proxy Errors (Critical for preventing crashes)
+    this.proxy.on('error', (err, req, res) => {
+      console.error(`❌ Proxy Error [${req.url}]:`, err.message);
+      if (!res.headersSent) {
+        res.writeHead(502, { 'Content-Type': 'application/json' });
+      }
+      res.end(JSON.stringify({ error: 'Proxy Request Failed', details: err.message }));
+    });
+  }
+
+  start(target, port, outputFile, onLog = () => { }) {
+    if (this.server) throw new Error('Recorder already running');
+
+    console.log(`📝 Recording to file: ${outputFile}`);
+    const stream = fs.createWriteStream(outputFile, { flags: 'a' });
+
+    // Handle file write errors
+    stream.on('error', (err) => {
+      console.error('❌ File Write Error:', err.message);
+    });
+
+    this.server = http.createServer((req, res) => {
+      // 2. Capture Request Body safely
+      // We use a separate array but we DO NOT attach 'data' listeners directly
+      // unless we plan to buffer it for the proxy.
+      // For simplicity, let's rely on the response capture primarily.
+
+      const reqChunks = [];
+      // We tap into the stream without consuming it destructively if possible,
+      // but the safest way with http-proxy is often just to listen to the proxy events.
+      // HOWEVER, for your specific "Request Stealing" fix:
+      req.on('data', (chunk) => reqChunks.push(chunk));
+
+      // --- Response Capture Logic ---
+      const originalWrite = res.write;
+      const originalEnd = res.end;
+      const resChunks = [];
+
+      res.write = function (chunk, ...args) {
+        if (chunk) resChunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk));
+        return originalWrite.apply(res, [chunk, ...args]);
+      };
+
+      res.end = function (chunk, ...args) {
+        if (chunk) resChunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk));
+
+        // WRAP IN TRY/CATCH to prevent silent crashes
+        try {
+          const reqBody = Buffer.concat(reqChunks).toString();
+          const resBody = Buffer.concat(resChunks).toString('utf8');
+
+          const logData = {
+            timestamp: new Date().toISOString(),
+            method: req.method,
+            url: req.url,
+            status: res.statusCode,
+            requestBody: reqBody || '',
+            responseBody: resBody || '',
+            // requestHeaders: req.headers, // Uncomment if needed (verbose)
+          };
+
+          const jsonLine = JSON.stringify(logData) + '\n';
+
+          // 3. Write and Log
+          stream.write(jsonLine);
+          onLog(logData);
+
+          // Debug Log to prove it worked
+          process.stdout.write('.'); // Prints a dot for every logged request
+        } catch (e) {
+          console.error('\n❌ Error generating log:', e.message);
+        }
+
+        return originalEnd.apply(res, [chunk, ...args]);
+      };
+
+      // 4. Force Plain Text (Disable Gzip)
+      delete req.headers['accept-encoding'];
+
+      // 5. Forward to Target
+      // We must set 'buffer' to the request stream if we consumed it,
+      // but since we are just tapping 'data' without pausing, http-proxy might miss it.
+      // A simple workaround for node streams in this context:
+      this.proxy.web(req, res, { target });
+    });
+
+    this.server.listen(port, () => {
+      console.log(`\n⏺️  Recorder listening on port ${port}`);
+      console.log(`➡️  Forwarding to ${target}`);
+    });
+
+    return `Recorder started`;
+  }
+
+  stop() {
+    if (this.server) {
+      this.server.close();
+      this.server = null;
+      return 'Recorder stopped';
+    }
+  }
+}
+
+module.exports = new Recorder();