rollbar 2.26.4 → 3.0.0-alpha.2

package/CLAUDE.testrunner.md

@@ -0,0 +1,470 @@
+# Test Runner Migration: Karma → Web Test Runner
+
+## Table of Contents
+1. [Overview](#overview)
+2. [Understanding JavaScript Testing Environments](#understanding-javascript-testing-environments)
+3. [Current State Analysis](#current-state-analysis)
+4. [Migration Strategy](#migration-strategy)
+5. [Implementation Plan](#implementation-plan)
+6. [Technical Details](#technical-details)
+7. [Success Metrics](#success-metrics)
+8. [Rollback Plan](#rollback-plan)
+
+## Overview
+
+This document outlines the migration from the deprecated Karma test runner to the modern Web Test Runner (@web/test-runner) for the Rollbar.js SDK. This migration is part of the broader modernization effort to enable TypeScript support and improve developer experience.
+
+### Goals
+- Replace deprecated Karma with actively maintained Web Test Runner
+- **Eliminate Grunt dependency** - run tests directly via npm scripts
+- Improve test execution speed by 50-70%
+- Enable native TypeScript support for future migration
+- Simplify test configuration and maintenance
+- Maintain 100% test compatibility
+
+### Approach
+- **Strategy**: Incremental migration with parallel validation
+- **Risk Level**: Low (keeping existing setup as fallback)
+
+## Understanding JavaScript Testing Environments
+
+### The Two Worlds of JavaScript
+
+JavaScript runs in two fundamentally different environments, and understanding this distinction is crucial for our testing strategy:
+
+#### 1. Browser Environment
+- **What it has**: DOM, window object, document, localStorage, fetch API
+- **What it lacks**: Filesystem access, process control, native modules
+- **Security**: Sandboxed for safety (can't read your hard drive!)
+- **Examples**: Chrome, Firefox, Safari, Edge
+
+#### 2. Node.js Environment
+- **What it has**: Filesystem (`fs`), process object, Buffer, native modules
+- **What it lacks**: DOM, window, browser-specific APIs
+- **Security**: Full system access (can read/write files)
+- **Examples**: Your terminal, web servers, build tools
+
+### Why This Matters for Rollbar.js
+
+The Rollbar SDK must work in BOTH environments:
+
+```javascript
+// Browser usage
+<script src="rollbar.min.js"></script>
+<script>
+  Rollbar.init({ accessToken: 'abc123' });
+</script>
+
+// Node.js usage
+import Rollbar from 'rollbar';
+const rollbar = new Rollbar({ accessToken: 'abc123' });
+```
+
+This dual-environment requirement means we need different test strategies:
+- **Browser tests**: Need a real browser to test DOM manipulation, window.onerror, etc.
+- **Server tests**: Need Node.js to test filesystem operations, process handling, etc.
+
+### Test Framework vs Test Runner
+
+**Test Framework (Mocha)**
+- Provides the structure for writing tests
+- Defines `describe()`, `it()`, `beforeEach()`, etc.
+- Framework-agnostic (works in any JavaScript environment)
+
+```javascript
+describe('Calculator', () => {
+  it('should add numbers', () => {
+    expect(1 + 1).to.equal(2);
+  });
+});
+```
+
+**Test Runner**
+- Executes the tests in the appropriate environment
+- For browsers: Launches browser, serves files, collects results
+- For Node.js: Just runs the JavaScript file directly
+
+### Our Testing Architecture
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                      Rollbar.js Tests                       │
+├─────────────────────────────┬───────────────────────────────┤
+│      Browser Tests (~478)   │      Server Tests (~126)      │
+├─────────────────────────────┼───────────────────────────────┤
+│   Environment: Browser      │   Environment: Node.js        │
+│   Runner: Web Test Runner   │   Runner: Node.js (direct)    │
+│   Browser: Playwright       │   Browser: N/A                │
+│   Framework: Mocha          │   Framework: Mocha            │
+│   Assertions: Chai/Expect   │   Assertions: Chai            │
+└─────────────────────────────┴───────────────────────────────┘
+```
+
+## Current State Analysis
+
+### Test Infrastructure Metrics
+- **Total Tests**: ~604 (478 browser + 126 server)
+- **Test Runner**: Karma 6.4.4 (deprecated, last update 2023)
+- **Execution Time**: ~27 seconds (23s browser + 4s server)
+- **Configuration Complexity**: 91 lines Karma + 273 lines Grunt
+
+### Current Grunt Dependency
+The tests currently run through Grunt tasks:
+- `grunt test` → runs `grunt test-browser`
+- `grunt test-browser` → orchestrates multiple Karma runs
+- `grunt test-replay` → runs replay-specific tests
+- `grunt karma:api`, `grunt karma:browser.core`, etc.
+
+**We will eliminate this entire layer**, running Web Test Runner directly via npm scripts.
+
+### Current Pain Points
+
+1. **Maintenance Burden**
+   - Karma is deprecated with security vulnerabilities
+   - Complex configuration spread across multiple files
+   - Difficult debugging experience
+
+2. **Developer Experience**
+   - Poor error messages from Karma
+   - No stack traces on test failures (obfuscated/minified code)
+   - No way to manually test directly in browsers
+   - No native ES module support
+   - Complex watch mode setup
+
+3. **Performance Issues**
+   - Sequential test suite execution
+   - Webpack rebundling for each suite (500ms overhead × 13 suites)
+   - Bundle sizes up to 4.82MB for single test suites
+
+### Test Categories and Distribution
+
+| Category       | Test Count | Current Runner | Bundle Size |
+| -------------- | ---------- | -------------- | ----------- |
+| API            | 5          | Karma          | 691 KB      |
+| API Utility    | 14         | Karma          | 425 KB      |
+| Browser Core   | 16         | Karma          | 2.31 MB     |
+| DOM Utility    | 15         | Karma          | 73.4 KB     |
+| Predicates     | 4          | Karma          | 369 KB      |
+| Replay Tests   | 50         | Karma          | 2.43 MB     |
+| Browser Tests  | 67         | Karma          | 4.82 MB     |
+| Transforms     | 42         | Karma          | 1.74 MB     |
+| Utility        | 37         | Karma          | 395 KB      |
+| Tracing        | 28         | Karma          | 444 KB      |
+| Examples       | 163        | Karma          | Various     |
+| Server Tests   | 126        | Mocha          | N/A         |
+
+## Migration Strategy
+
+### Architectural Decision: Two Test Runners
+
+After careful consideration, we'll maintain two separate test runners:
+
+1. **Web Test Runner** for browser tests (using **Playwright** for browser automation)
+2. **Direct Mocha execution** for server tests
+
+**Rationale:**
+- Clear separation of concerns
+- Optimal performance for each environment
+- Simpler debugging and maintenance
+- Aligns with JavaScript ecosystem best practices
+
+**Browser Automation: Playwright**
+We're using Playwright as our browser launcher because:
+- Cross-browser support (Chromium, Firefox, WebKit/Safari)
+- Actively maintained by Microsoft
+- Better performance than legacy WebDriver solutions
+- Native support in Web Test Runner
+- Future-proof for advanced testing scenarios
+
+### Phased Approach
+
+We'll migrate in 5 phases to minimize risk and ensure thorough validation:
+
+#### Phase 1: Foundation & Proof of Concept
+- Install and configure Web Test Runner
+- Migrate simplest test suite (API - 5 tests)
+- Validate core functionality
+- Document learnings
+
+#### Phase 2: Core Test Migration
+- Migrate utility and transform tests
+- Set up HTML fixture loading
+- Configure proxy routes
+- Establish patterns for common scenarios
+
+#### Phase 3: Complex Test Migration
+- Migrate browser-specific tests
+- Handle replay and tracing tests
+- Address edge cases
+- Performance optimization
+
+#### Phase 4: CI/CD Integration
+- Update GitHub Actions workflows
+- Parallel test execution
+- Coverage reporting
+- Remove Karma dependencies
+
+#### Phase 5: Documentation & Cleanup
+- Update developer documentation
+- Remove old configurations
+- Team training
+- Final validation
+
+## Implementation Plan
+
+### Phase 1: Foundation Setup
+
+#### 1.1 Install Dependencies
+```bash
+npm install --save-dev @web/test-runner @web/test-runner-mocha @web/test-runner-playwright
+```
+
+Note: `@web/test-runner-playwright` provides the Playwright integration for browser automation.
+
+#### 1.2 Create Basic Configuration
+Create `web-test-runner.config.mjs`:
+```javascript
+import { playwrightLauncher } from '@web/test-runner-playwright';
+
+export default {
+  files: 'test/**/*.test.js',
+  nodeResolve: true,
+  
+  browsers: [
+    playwrightLauncher({ product: 'chromium' }),
+  ],
+
+  testFramework: {
+    config: {
+      ui: 'bdd',
+      timeout: 5000,
+    },
+  },
+};
+```
+
+#### 1.3 Migrate First Test Suite
+Start with the API tests (smallest suite):
+1. Copy existing test file
+2. Update imports for ES modules
+3. Run with Web Test Runner
+4. Compare results with Karma output
+
+### Phase 2: Progressive Migration
+
+#### 2.1 Migration Order
+Based on complexity and dependencies:
+
+1. **Simple Tests**
+   - api (5 tests)
+   - apiUtility (14 tests)
+   - predicates (4 tests)
+
+2. **Utility Tests**
+   - utility (37 tests)
+   - transforms (42 tests)
+   - domUtility (15 tests)
+
+3. **Core Tests**
+   - browser.core (16 tests)
+   - browser.init (23 tests)
+   - browser.rollbar (67 tests)
+
+4. **Feature Tests**
+   - replay.* (50 tests)
+   - tracing.* (28 tests)
+
+5. **Example Tests**
+   - All example tests (163 tests)
+
+#### 2.2 Configuration Enhancements
+Add support for:
+- HTML fixtures
+- Proxy routes for `/dist/` and `/examples/`
+- Custom CSP headers
+- jQuery and other global dependencies
+
+### Phase 3: Advanced Features
+
+#### 3.1 Parallel Execution
+Configure concurrent test execution:
+```javascript
+export default {
+  concurrency: 4,
+  concurrentBrowsers: 2,
+  // ... rest of config
+};
+```
+
+#### 3.2 Coverage Reporting
+Add code coverage:
+```javascript
+import { esbuildPlugin } from '@web/dev-server-esbuild';
+
+export default {
+  coverage: true,
+  coverageConfig: {
+    include: ['src/**/*.js'],
+    threshold: {
+      statements: 90,
+      branches: 85,
+      functions: 85,
+      lines: 90,
+    },
+  },
+  plugins: [esbuildPlugin({ js: true })],
+};
+```
+
+### Phase 4: CI/CD Updates
+
+#### 4.1 GitHub Actions Workflow
+Update `.github/workflows/test.yml`:
+```yaml
+- name: Run browser tests
+  run: npm run test:browser
+  
+- name: Run server tests
+  run: npm run test:server
+```
+
+#### 4.2 NPM Scripts (Direct Execution - No Grunt!)
+Update package.json:
+```json
+{
+  "scripts": {
+    "test": "npm run test:browser && npm run test:server",
+    "test:browser": "web-test-runner",
+    "test:server": "mocha 'test/server.*.test.mjs'",
+    "test:watch": "web-test-runner --watch",
+    "test:replay": "web-test-runner 'test/**/replay/**/*.test.js'",
+    "test:unit": "web-test-runner 'test/**/*.test.js' --exclude '**/integration/**'"
+  }
+}
+```
+
+**Command Comparison:**
+| Current (via Grunt)               | New (Direct)                  | Benefit                |
+| --------------------------------- | ----------------------------- | ---------------------- |
+| `npm test` → `grunt test` → Karma | `npm test` → Web Test Runner  | Remove 1 layer         |
+| `grunt test-replay`               | `npm run test:replay`         | Direct execution       |
+| `grunt karma:api`                 | Covered by `npm test:browser` | Unified approach       |
+| 273 lines of Grunt config         | 0 lines                       | Massive simplification |
+
+## Technical Details
+
+### Handling Special Requirements
+
+#### HTML Fixtures
+```javascript
+// Current (Karma)
+__html__['test/fixtures/frame.html']
+
+// New (Web Test Runner)
+import { fixture, html } from '@open-wc/testing-helpers';
+const el = await fixture(html`<div>content</div>`);
+```
+
+#### Proxy Routes
+```javascript
+export default {
+  middleware: [
+    // Serve dist files
+    function serveDistFiles(context, next) {
+      if (context.path.startsWith('/dist/')) {
+        context.path = context.path.replace('/dist/', '/');
+      }
+      return next();
+    },
+  ],
+};
+```
+
+#### Global Dependencies
+```javascript
+// For tests requiring jQuery
+export default {
+  testRunnerHtml: testFramework => `
+    <html>
+      <body>
+        <script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
+        <script type="module" src="${testFramework}"></script>
+      </body>
+    </html>
+  `,
+};
+```
+
+### Performance Optimizations
+
+1. **Parallel Execution**: Run up to 4 test files concurrently
+2. **Shared Dependencies**: Load common libraries once
+3. **No Bundling**: Use native ES modules
+4. **Smart Watch Mode**: Only re-run affected tests
+
+### Expected Performance Gains
+
+| Metric      | Current (Karma) | Expected (WTR) | Improvement |
+| ----------- | --------------- | -------------- | ----------- |
+| Total Time  | 23 seconds      | 8-10 seconds   | 57-65%      |
+| Startup     | 3 seconds       | 0.5 seconds    | 83%         |
+| Per Test    | 48ms            | 17ms           | 65%         |
+| Bundle Size | 4.82MB max      | 0 (native)     | 100%        |
+
+## Success Metrics
+
+### Quantitative Metrics
+- ✅ All 478 browser tests passing
+- ✅ Test execution time < 10 seconds
+- ✅ Code coverage maintained at > 90%
+- ✅ Zero test flakiness
+- ✅ CI pipeline success rate 100%
+
+### Qualitative Metrics
+- ✅ Improved developer experience
+- ✅ Easier debugging workflow
+- ✅ Simplified configuration
+- ✅ Better error messages
+- ✅ TypeScript-ready infrastructure
+
+## Rollback Plan
+
+If issues arise during migration:
+
+1. **Immediate**: Tests continue running on Karma (parallel setup)
+2. **Short-term**: Revert Web Test Runner changes
+3. **Investigation**: Debug and fix issues
+4. **Retry**: Attempt migration with fixes
+
+### Risk Mitigation
+- Keep Karma configuration until fully validated
+- Run both test runners in CI during migration
+- Incremental migration allows partial rollback
+- Each phase independently validated
+
+## Next Steps
+
+1. **Review**: Team reviews this plan
+2. **Approval**: Get stakeholder sign-off
+3. **Setup**: Create feature branch `feature/web-test-runner`
+4. **Execute**: Begin Phase 1 implementation
+5. **Validate**: Run parallel validation until confident
+6. **Complete**: Remove Karma after full validation
+
+## Appendix: Learning Resources
+
+### For JavaScript Testing Beginners
+- [MDN: Introduction to automated testing](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Automated_testing)
+- [Modern Web: Web Test Runner docs](https://modern-web.dev/docs/test-runner/overview/)
+- [Mocha documentation](https://mochajs.org/)
+
+### Understanding the Migration
+- Why Karma is deprecated: Maintenance burden, architectural limitations
+- Why Web Test Runner: Modern architecture, native ES modules, active development
+- Why keep Mocha: Minimal change, widely understood, works everywhere
+
+### Key Concepts
+- **Test Framework**: Provides test structure (describe/it)
+- **Test Runner**: Executes tests in target environment
+- **Assertion Library**: Provides expectation syntax (expect/assert)
+- **Test Environment**: Where tests execute (browser/Node.js)

package/Gruntfile.js

@@ -112,7 +112,6 @@ module.exports = function (grunt) {
   grunt.loadNpmTasks('grunt-karma');
   grunt.loadNpmTasks('grunt-webpack');
   grunt.loadNpmTasks('grunt-text-replace');
-  grunt.loadNpmTasks('grunt-vows');
 
   var rollbarJsSnippet = fs.readFileSync('dist/rollbar.snippet.js');
   var rollbarjQuerySnippet = fs.readFileSync('dist/plugins/jquery.min.js');
@@ -120,14 +119,6 @@ module.exports = function (grunt) {
   grunt.initConfig({
     pkg: pkg,
     webpack: webpackConfig,
-    vows: {
-      all: {
-        options: {
-          reporter: 'spec',
-        },
-        src: ['test/server.*.test.js'],
-      },
-    },
 
     karma: buildGruntKarmaConfig(singleRun, browserTests, reporters),
 
@@ -145,7 +136,6 @@ module.exports = function (grunt) {
           // Main rollbar snippet
           {
             from: new RegExp(
-              /* eslint-disable-next-line no-control-regex */
               '^(.*// Rollbar Snippet)[\n\r]+(.*[\n\r])*(.*// End Rollbar Snippet)',
               'm',
             ),
@@ -157,7 +147,6 @@ module.exports = function (grunt) {
           // jQuery rollbar plugin snippet
           {
             from: new RegExp(
-              /* eslint-disable-next-line no-control-regex */
               '^(.*// Rollbar jQuery Snippet)[\n\r]+(.*[\n\r])*(.*// End Rollbar jQuery Snippet)',
               'm',
             ),
@@ -183,13 +172,11 @@ module.exports = function (grunt) {
 
   grunt.registerTask('build', ['webpack', 'replace:snippets']);
   grunt.registerTask('default', ['build']);
-  grunt.registerTask('test', ['test-server', 'test-browser']);
+  grunt.registerTask('test', [
+    'test-browser',
+  ]);
   grunt.registerTask('release', ['build', 'copyrelease']);
 
-  grunt.registerTask('test-server', function (_target) {
-    var tasks = ['vows'];
-    grunt.task.run.apply(grunt.task, tasks);
-  });
 
   grunt.registerTask('test-browser', function (target) {
     var karmaTask = 'karma' + (target ? ':' + target : '');
@@ -197,6 +184,62 @@ module.exports = function (grunt) {
     grunt.task.run.apply(grunt.task, tasks);
   });
 
+  function findReplayTests() {
+    return Object.keys(browserTests).filter(function (testName) {
+      var testPath = browserTests[testName];
+      return (
+        testPath.includes('test/replay/') ||
+        testPath.includes('test/tracing/') ||
+        testPath.match(/test\/browser\.replay\..*\.test\.js/)
+      );
+    });
+  }
+
+  grunt.registerTask('test-replay', function () {
+    var replayTests = findReplayTests();
+
+    grunt.log.writeln('Running all replay-related tests:');
+    replayTests.forEach(function (testName) {
+      grunt.log.writeln('- ' + testName + ' (' + browserTests[testName] + ')');
+    });
+
+    replayTests.forEach(function (testName) {
+      grunt.task.run('karma:' + testName);
+    });
+  });
+
+  grunt.registerTask('test-replay-unit', function () {
+    var unitTests = findReplayTests().filter(function (testName) {
+      var testPath = browserTests[testName];
+      return testPath.includes('/replay/unit/');
+    });
+
+    grunt.log.writeln('Running replay unit tests:');
+    unitTests.forEach(function (testName) {
+      grunt.log.writeln('- ' + testName + ' (' + browserTests[testName] + ')');
+    });
+
+    unitTests.forEach(function (testName) {
+      grunt.task.run('karma:' + testName);
+    });
+  });
+
+  grunt.registerTask('test-replay-integration', function () {
+    var integrationTests = findReplayTests().filter(function (testName) {
+      var testPath = browserTests[testName];
+      return testPath.includes('/replay/integration/');
+    });
+
+    grunt.log.writeln('Running replay integration tests:');
+    integrationTests.forEach(function (testName) {
+      grunt.log.writeln('- ' + testName + ' (' + browserTests[testName] + ')');
+    });
+
+    integrationTests.forEach(function (testName) {
+      grunt.task.run('karma:' + testName);
+    });
+  });
+
   grunt.registerTask('copyrelease', function createRelease() {
     var version = pkg.version;
     var builds = ['', '.umd'];

package/Makefile

@@ -17,8 +17,8 @@ test-server:
 	@npm run test-server
 	@echo ""
 
-test_ci:
-	@npm run test_ci
+test-ci:
+	@npm run test-ci
 	@echo ""
 
-.PHONY: test test_ci
+.PHONY: test test-ci

package/SECURITY.md

@@ -0,0 +1,5 @@
+# Security Policy
+
+## Reporting a Vulnerability
+
+Please report security issues to `security@rollbar.com`

package/babel.config.json

@@ -0,0 +1,9 @@
+{
+  "parserOpts": {
+    "ecmaVersion": 2021,
+    "sourceType": "module",
+  },
+  "presets": [
+    "@babel/preset-env"
+  ]
+}