@flakiness/vitest 0.1.0-alpha.2 → 1.1.0

package/README.md

@@ -11,7 +11,7 @@ A custom Vitest reporter that generates Flakiness Reports from your Vitest test
 - [Viewing Reports](#viewing-reports)
 - [Features](#features)
   - [Test Location Tracking](#test-location-tracking)
-  - [Duplicate Test Detection](#duplicate-test-detection)
+  - [Handling Test Duplicates](#handling-test-duplicates)
   - [Environment Detection](#environment-detection)
   - [CI Integration](#ci-integration)
 - [Configuration Options](#configuration-options)
@@ -19,7 +19,7 @@ A custom Vitest reporter that generates Flakiness Reports from your Vitest test
   - [`endpoint?: string`](#endpoint-string)
   - [`token?: string`](#token-string)
   - [`outputFolder?: string`](#outputfolder-string)
-  - [`open?: 'always' | 'never' | 'on-failure'`](#open-always--never--on-failure)
+  - [`duplicates?: 'fail' | 'rename'`](#duplicates-fail--rename)
   - [`disableUpload?: boolean`](#disableupload-boolean)
 - [Environment Variables](#environment-variables)
 - [Example Configuration](#example-configuration)
@@ -47,13 +47,18 @@ export default defineConfig({
   test: {
     includeTaskLocation: true,
     reporters: [
-      ['@flakiness/vitest']
+      'default',
+      ['@flakiness/vitest', { flakinessProject: 'my-org/my-project' }]
     ],
   },
 });
 ```
 
-> **Note:** Setting `includeTaskLocation: true` is recommended to enable test locations the report.
+> [!NOTE]
+> The flakiness reporter should be added alongside your other reporters. Include `'default'` (or another built-in reporter) to retain the standard terminal output, since Vitest only uses the reporters listed in the `reporters` array.
+
+> [!TIP]
+> Setting `includeTaskLocation: true` is recommended to enable test locations in the report.
 
 Run your tests. The report will be automatically generated in the `./flakiness-report` folder:
 
@@ -92,16 +97,30 @@ npx flakiness show ./flakiness-report
 
 When `includeTaskLocation: true` is set in your Vitest config, the reporter records the exact file, line, and column for each test. This enables precise navigation from the Flakiness.io dashboard back to your source code.
 
-### Duplicate Test Detection
+### Handling Test Duplicates
 
-Vitest allows multiple tests to share the same name:
+Vitest allows creating tests with identical names:
 
 ```typescript
+// Trivial duplicates
 it('should work', () => { /* ... */ });
 it('should work', () => { /* ... */ });
+
+// More common way to end up with test duplicates:
+it.for([
+  { input: 1, expected: 2 },
+  { input: 1, expected: 3 },
+])('should handle $input', ({ input, expected }) => { /* ... */ });
 ```
 
-Flakiness.io relies on full test names (all parent suite names + test name) to construct test history, so each test must have a unique full name. The reporter detects duplicates, issues warnings, and marks them as failed with a descriptive error message. Rename your tests to have unique full names to resolve this.
+Flakiness.io, however, **does not allow** test duplicates.
+
+Flakiness.io considers two tests to be *duplicates* when they:
+1. Share the same parent suite hierarchy
+2. Have the same test name
+3. And run in the same Vitest project
+
+Flakiness.io relies on full test names to construct test history, so each test must have a unique full name. When Vitest reporter detects tests with identical full names, it issues warnings and handles them according to the [`duplicates`](#duplicates-fail--rename) option.
 
 ### Environment Detection
 
@@ -183,20 +202,23 @@ reporters: [
 ]
 ```
 
-### `open?: 'always' | 'never' | 'on-failure'`
+### `duplicates?: 'fail' | 'rename'`
+
+Controls how the reporter handles tests with duplicate full names. Defaults to `'fail'`.
 
-Controls when the report viewer should automatically open in your browser after test completion.
+> [!WARNING]
+> The `'rename'` mode is **not recommended** for regular use. There is no guarantee that test histories will remain stable for duplicate tests, since the renaming is based on internal Vitest identifiers that may change between runs. This mode exists to help evaluate the reporter against large Vitest projects that have not yet resolved their duplicate test names.
 
-- **`'on-failure'`** (default): Opens the report only if tests failed and running in an interactive terminal (not in CI)
-- **`'always'`**: Always opens the report after test completion (when running in an interactive terminal)
-- **`'never'`**: Never automatically opens the report
+- **`'fail'`** (default): Duplicate tests are marked as failed with a descriptive error message and a `dupe` annotation. The first duplicate gets a single failed attempt explaining the problem; the remaining duplicates are stripped of their attempts (effectively hidden from the report). This is the recommended mode — it surfaces the problem so you can fix it by renaming your tests.
+- **`'rename'`**: Duplicate tests are automatically renamed by appending a suffix (e.g., ` – dupe #2`, ` – dupe #3`) to their titles. Each renamed test also receives a `dupe` annotation on all its attempts. The first test with a given name keeps its original title; only the subsequent duplicates are renamed.
 
 ```typescript
 reporters: [
-  ['@flakiness/vitest', { open: 'always' }]
+  ['@flakiness/vitest', { duplicates: 'rename' }]
 ]
 ```
 
+
 ### `disableUpload?: boolean`
 
 When set to `true`, prevents uploading the report to Flakiness.io. The report is still generated locally. Can also be controlled via the `FLAKINESS_DISABLE_UPLOAD` environment variable.
@@ -227,12 +249,13 @@ export default defineConfig({
   test: {
     includeTaskLocation: true,
     reporters: [
+      'default',
       ['@flakiness/vitest', {
         flakinessProject: 'my-org/my-project',
         endpoint: process.env.FLAKINESS_ENDPOINT,
         token: process.env.FLAKINESS_ACCESS_TOKEN,
         outputFolder: './flakiness-report',
-        open: 'on-failure',
+        duplicates: 'fail',
         disableUpload: false,
       }]
     ],

package/lib/reporter.js

@@ -1,5 +1,5 @@
 import { FlakinessReport as FK } from "@flakiness/flakiness-report";
-import { CIUtils, CPUUtilization, GitWorktree, RAMUtilization, ReportUtils, showReport, uploadReport, writeReport } from "@flakiness/sdk";
+import { CIUtils, CPUUtilization, GitWorktree, RAMUtilization, ReportUtils, uploadReport, writeReport } from "@flakiness/sdk";
 import chalk from "chalk";
 import crypto from "crypto";
 import assert from "node:assert";
@@ -9,6 +9,7 @@ class FKVitestReporter {
     this._options = _options;
   }
   _impl;
+  _vitest;
   _logger = {
     warn: (txt) => console.warn(chalk.yellow(txt)),
     error: (txt) => console.error(chalk.red(txt)),
@@ -17,58 +18,47 @@ class FKVitestReporter {
   setLoggerForTest(logger) {
     this._logger = logger;
   }
-  onUserConsoleLog(log) {
-    this._impl?.onUserConsoleLog(log);
+  async onUserConsoleLog(log) {
+    await this._impl?.onUserConsoleLog(log);
   }
   onInit(vitest) {
-    this._impl = ReporterImpl.create(vitest.config.root, this._options, this._logger);
-    this._impl?.onInit(vitest);
-  }
-  /**
-  * Called when annotation is added via the `task.annotate` API.
-  */
-  onTestCaseAnnotate(testCase, annotation) {
-    this._impl?.onTestCaseAnnotate(testCase, annotation);
+    this._vitest = vitest;
   }
   onTestRunStart() {
-    this._impl?.onTestRunStart();
-  }
-  async onTestCaseResult(testCase) {
-    this._impl?.onTestCaseResult(testCase);
+    assert(this._vitest, "onInit must be called before onTestRunStart");
+    this._impl = ReporterImpl.create(this._vitest.config.root, this._options, this._logger, this._vitest.config.config);
   }
   async onTestRunEnd(testModules, unhandledErrors, reason) {
-    this._impl?.onTestRunEnd(testModules, unhandledErrors, reason);
+    await this._impl?.onTestRunEnd(testModules, unhandledErrors, reason);
+    this._impl = void 0;
   }
 }
+const gTestToTestCaseId = /* @__PURE__ */ new WeakMap();
 class ReporterImpl {
-  constructor(_worktree, _commitId, _options, _logger) {
+  constructor(_worktree, _commitId, _options, _logger, _configPath) {
     this._worktree = _worktree;
     this._commitId = _commitId;
     this._options = _options;
     this._logger = _logger;
+    this._configPath = _configPath;
     this._sampleSystem = this._sampleSystem.bind(this);
+    this._sampleSystem();
   }
   _telemetryTimer;
   _cpuUtilization = new CPUUtilization({ precision: 10 });
   _ramUtilization = new RAMUtilization({ precision: 10 });
   _startTimestamp = Date.now();
-  _testCaseIdToTest = /* @__PURE__ */ new Map();
-  _testToTestCaseId = /* @__PURE__ */ new Map();
   _stdio = /* @__PURE__ */ new Map();
-  _annotations = /* @__PURE__ */ new Map();
   // In Vitest, all projects MUST HAVE UNIQUE NAMES.
   // So we create environments per project name.
   _environments = [];
-  _allSuites = /* @__PURE__ */ new Map();
-  _fileSuites = /* @__PURE__ */ new Map();
-  _configPath;
-  static create(rootPath, options, logger) {
+  static create(rootPath, options, logger, config) {
     let commitId;
     let worktree;
     try {
       worktree = GitWorktree.create(rootPath);
       commitId = worktree.headCommitId();
-      return new ReporterImpl(worktree, commitId, options, logger);
+      return new ReporterImpl(worktree, commitId, options, logger, config);
     } catch (e) {
       logger.warn(`[flakiness.io] Failed to fetch commit info - is this a git repo?`);
       logger.error(`[flakiness.io] Report is NOT generated.`);
@@ -80,9 +70,6 @@ class ReporterImpl {
     this._ramUtilization.sample();
     this._telemetryTimer = setTimeout(this._sampleSystem, 1e3);
   }
-  onInit(vitest) {
-    this._configPath = vitest.config.config;
-  }
   onUserConsoleLog(log) {
     if (!log.taskId)
       return;
@@ -93,70 +80,6 @@ class ReporterImpl {
     }
     entries.push(log);
   }
-  onTestCaseAnnotate(testCase, annotation) {
-    let entries = this._annotations.get(testCase.id);
-    if (!entries) {
-      entries = [];
-      this._annotations.set(testCase.id, entries);
-    }
-    entries.push(annotation);
-  }
-  onTestRunStart() {
-    this._startTimestamp = Date.now();
-    this._sampleSystem();
-  }
-  _ensureFKSuite(p) {
-    let suite = this._allSuites.get(p.id);
-    if (suite)
-      return suite;
-    if ("name" in p) {
-      const parent = this._ensureFKSuite(p.parent);
-      suite = {
-        type: "suite",
-        title: p.name,
-        location: p.location ? {
-          file: this._worktree.gitPath(p.module.moduleId),
-          column: p.location.column,
-          line: p.location.line
-        } : void 0
-      };
-      parent.suites ??= [];
-      parent.suites.push(suite);
-    } else {
-      suite = {
-        type: "file",
-        title: p.relativeModuleId,
-        location: {
-          file: this._worktree.gitPath(p.moduleId),
-          column: 0,
-          line: 0
-        }
-      };
-      this._fileSuites.set(p.id, suite);
-    }
-    this._allSuites.set(p.id, suite);
-    return suite;
-  }
-  _ensureTest(testCase) {
-    let fkTest = this._testCaseIdToTest.get(testCase.id);
-    if (!fkTest) {
-      fkTest = {
-        attempts: [],
-        title: testCase.name,
-        location: testCase.location ? {
-          file: this._worktree.gitPath(testCase.module.moduleId),
-          column: testCase.location.column,
-          line: testCase.location.line
-        } : void 0
-      };
-      this._testToTestCaseId.set(fkTest, testCase.id);
-      this._testCaseIdToTest.set(testCase.id, fkTest);
-      const parent = this._ensureFKSuite(testCase.parent);
-      parent.tests ??= [];
-      parent.tests.push(fkTest);
-    }
-    return fkTest;
-  }
   _ensureEnvironmentIdx(testCase) {
     const projectName = testCase.project.name || "vitest";
     let idx = this._environments.findIndex((env) => env.name === projectName);
@@ -176,13 +99,42 @@ class ReporterImpl {
       column: frame.column
     } : void 0;
   }
-  async onTestCaseResult(testCase) {
+  _collectSuite(fkParent, suite) {
+    const fkSuite = {
+      type: "suite",
+      title: suite.name,
+      location: suite.location ? {
+        file: this._worktree.gitPath(suite.module.moduleId),
+        column: suite.location.column,
+        line: suite.location.line
+      } : void 0
+    };
+    fkParent.suites ??= [];
+    fkParent.suites.push(fkSuite);
+    for (const s of suite.children.suites())
+      this._collectSuite(fkSuite, s);
+    for (const t of suite.children.tests())
+      this._collectTest(fkSuite, t);
+  }
+  async _collectTest(fkParent, testCase) {
     const environmentIdx = this._ensureEnvironmentIdx(testCase);
-    const fkTest = this._ensureTest(testCase);
+    const fkTest = {
+      attempts: [],
+      title: testCase.name,
+      location: testCase.location ? {
+        file: this._worktree.gitPath(testCase.module.moduleId),
+        column: testCase.location.column,
+        line: testCase.location.line
+      } : void 0
+    };
+    fkParent.tests ??= [];
+    fkParent.tests.push(fkTest);
+    gTestToTestCaseId.set(fkTest, testCase.id);
     const result = testCase.result();
-    if (result.state === "skipped") {
+    if (result.state === "skipped" || result.state === "pending") {
       fkTest.attempts.push({
         environmentIdx,
+        annotations: testCase.options.mode === "skip" ? [{ type: "skip" }] : testCase.options.mode === "todo" ? [{ type: "todo" }] : void 0,
         startTimestamp: Date.now(),
         duration: 0,
         status: "skipped",
@@ -190,11 +142,11 @@ class ReporterImpl {
       });
       return;
     }
-    const diag = testCase.diagnostic();
-    assert(diag, `Diagnostic must be present in finished test cases`);
-    ;
+    const startTime = testCase.diagnostic()?.startTime ?? this._startTimestamp;
+    const duration = testCase.diagnostic()?.duration ?? testCase.module.diagnostic()?.duration ?? 0;
+    const retryCount = testCase.diagnostic()?.retryCount ?? 0;
     const stdio = [];
-    let ts = diag.startTime;
+    let ts = startTime;
     for (const entry of this._stdio.get(testCase.id) ?? []) {
       stdio.push({
         text: entry.content,
@@ -210,7 +162,7 @@ class ReporterImpl {
       stack: error.stack,
       location: this._errorLocation(error.stacks, testFile)
     }));
-    const annotations = (this._annotations.get(testCase.id) ?? []).map((annotation) => ({
+    const annotations = testCase.annotations().map((annotation) => ({
       type: annotation.type,
       description: annotation.message,
       location: annotation.location ? {
@@ -219,12 +171,14 @@ class ReporterImpl {
         column: annotation.location.column
       } : void 0
     }));
+    if (testCase.options.fails)
+      annotations.push({ type: "fail" });
     const expectedStatus = testCase.options.fails ? "failed" : "passed";
     const oppositeStatus = expectedStatus === "failed" ? "passed" : "failed";
-    for (let i = 0; i < diag.retryCount; ++i) {
+    for (let i = 0; i < retryCount; ++i) {
       fkTest.attempts.push({
         environmentIdx,
-        startTimestamp: diag.startTime,
+        startTimestamp: startTime,
         duration: 0,
         // retries have an opposite status from expected status to
         // trigger retry.
@@ -232,15 +186,17 @@ class ReporterImpl {
         expectedStatus,
         // TODO: ideally, we can differentiate STDIO between attempts.
         // However, vitest doesn't let us do so easily.
-        stdio,
-        errors,
-        annotations
+        // We slice these arrays just to be safe: if someone downstream from us
+        // decides to push a new annotation, then they can do it per-attempt safely.
+        stdio: stdio.slice(),
+        errors: errors.slice(),
+        annotations: annotations.slice()
       });
     }
     fkTest.attempts.push({
       environmentIdx,
-      startTimestamp: diag.startTime,
-      duration: diag.duration,
+      startTimestamp: startTime,
+      duration,
       status: result.state === "failed" ? oppositeStatus : expectedStatus,
       expectedStatus,
       // TODO: ideally, we can differentiate STDIO between attempts.
@@ -250,38 +206,17 @@ class ReporterImpl {
       annotations
     });
   }
-  _detectAndHandleTestDuplicates() {
-    const TITLE_SEPARATOR = crypto.randomBytes(128).toString("base64");
-    const testIdToTests = /* @__PURE__ */ new Map();
-    const testFullNames = /* @__PURE__ */ new Map();
-    function visitSuite(suite, parentTitles = []) {
-      parentTitles.push(suite.title);
-      for (const childSuite of suite.suites ?? [])
-        visitSuite(childSuite, parentTitles);
-      for (const test of suite.tests ?? []) {
-        const envs = Array.from(new Set(test.attempts.map((attempt) => attempt.environmentIdx ?? 0))).sort((a, b) => a - b);
-        const testId = [`[${envs.join(", ")}]`, ...parentTitles, test.title].join(TITLE_SEPARATOR);
-        testFullNames.set(test, [...parentTitles, test.title].join(" > "));
-        let tests = testIdToTests.get(testId);
-        if (!tests) {
-          tests = [];
-          testIdToTests.set(testId, tests);
-        }
-        tests.push(test);
-      }
-      parentTitles.pop();
-    }
-    for (const fileSuite of this._fileSuites.values())
-      visitSuite(fileSuite);
-    if (Array.from(testIdToTests.values()).every((tests) => tests.length <= 1))
-      return;
+  _warnDuplicates(duplicates) {
     const warnMessages = [];
-    for (const [testId, tests] of testIdToTests) {
-      if (tests.length <= 1)
-        continue;
-      const testFullName = testFullNames.get(tests[0]);
-      const warnMessage = `${tests.length} tests: ${testFullName}`;
-      warnMessages.push(warnMessage);
+    for (const duplicate of duplicates)
+      warnMessages.push(`${duplicate.tests.length} tests: ${duplicate.fullName}`);
+    this._logger.warn(`[flakiness.io] \u26A0 Detected test with duplicate names!`);
+    for (const warnMessage of warnMessages)
+      this._logger.warn(`[flakiness.io] - ${warnMessage}`);
+    this._logger.warn(`[flakiness.io] Please rename tests so that they all have unique full names.`);
+  }
+  _failDuplicates(duplicates) {
+    for (const { fullName, tests } of duplicates) {
       const test = tests[0];
       const envs = new Set(test.attempts.map((a) => a.environmentIdx ?? 0));
       test.attempts = Array.from(envs).map((envIdx) => ({
@@ -292,15 +227,15 @@ class ReporterImpl {
         status: "failed",
         errors: [{
           message: [
-            `Flakiness.io detected ${tests.length} tests with identical full name "${testFullName}"`,
+            `Flakiness.io detected ${tests.length} tests with identical full name "${fullName}"`,
             `Please rename tests to ensure they all have unique full names.`
           ].join("\n")
         }],
         annotations: [{
-          type: "duplicates",
+          type: "dupe",
           description: [
             `Flakiness.io failed to process this test because there are ${tests.length} tests with`,
-            `identical full name: "${testFullName}".`,
+            `identical full name: "${fullName}".`,
             `Please make sure that all your tests have unique full names.`
           ].join("\n")
         }]
@@ -308,18 +243,88 @@ class ReporterImpl {
       for (let i = 1; i < tests.length; ++i)
         tests[i].attempts = [];
     }
-    if (warnMessages.length) {
-      this._logger.warn(`[flakiness.io] \u26A0 Detected test with duplicate names!`);
-      for (const warnMessage of warnMessages)
-        this._logger.warn(`[flakiness.io] - ${warnMessage}`);
-      this._logger.warn(`[flakiness.io] Please rename tests so that they all have unique full names.`);
+  }
+  _renameDuplicates(duplicates, testIdToTests) {
+    for (const { tests, testId } of duplicates) {
+      tests.sort((test1, test2) => {
+        const id1 = gTestToTestCaseId.get(test1) ?? "";
+        const id2 = gTestToTestCaseId.get(test2) ?? "";
+        return id1 < id2 ? -1 : id1 > id2 ? 1 : 0;
+      });
+      let dupeIndex = 2;
+      for (let i = 1; i < tests.length; ++i) {
+        while (testIdToTests.has(testId + dupeSuffix(dupeIndex)))
+          ++dupeIndex;
+        tests[i].title += dupeSuffix(dupeIndex);
+        tests[i].attempts.forEach((attempt) => {
+          attempt.annotations ??= [];
+          attempt.annotations.push({ type: "dupe" });
+        });
+        testIdToTests.set(testId + dupeSuffix(dupeIndex), [tests[i]]);
+      }
     }
   }
+  _detectDuplicates(fileSuites) {
+    const TITLE_SEPARATOR = crypto.randomBytes(128).toString("base64");
+    const testIdToTests = /* @__PURE__ */ new Map();
+    const testFullNames = /* @__PURE__ */ new Map();
+    function visitSuite(suite, parentTitles = []) {
+      parentTitles.push(suite.title);
+      for (const childSuite of suite.suites ?? [])
+        visitSuite(childSuite, parentTitles);
+      for (const test of suite.tests ?? []) {
+        const envs = Array.from(new Set(test.attempts.map((attempt) => attempt.environmentIdx ?? 0))).sort((a, b) => a - b);
+        const testId = [`[${envs.join(", ")}]`, ...parentTitles, test.title].join(TITLE_SEPARATOR);
+        testFullNames.set(test, [...parentTitles, test.title].join(" > "));
+        let tests = testIdToTests.get(testId);
+        if (!tests) {
+          tests = [];
+          testIdToTests.set(testId, tests);
+        }
+        tests.push(test);
+      }
+      parentTitles.pop();
+    }
+    for (const fileSuite of fileSuites)
+      visitSuite(fileSuite);
+    const duplicates = [];
+    for (const [testId, tests] of testIdToTests) {
+      if (tests.length <= 1)
+        continue;
+      const fullName = testFullNames.get(tests[0]);
+      duplicates.push({ fullName, testId, tests });
+    }
+    return { duplicates, testIdToTests };
+  }
   async onTestRunEnd(testModules, unhandledErrors, reason) {
+    const fileSuites = testModules.map((file) => {
+      const fkFileSuite = {
+        type: "file",
+        title: file.relativeModuleId,
+        location: {
+          file: this._worktree.gitPath(file.moduleId),
+          column: 0,
+          line: 0
+        }
+      };
+      for (const suite of file.children.suites())
+        this._collectSuite(fkFileSuite, suite);
+      for (const test of file.children.tests())
+        this._collectTest(fkFileSuite, test);
+      return fkFileSuite;
+    });
     clearTimeout(this._telemetryTimer);
     this._cpuUtilization.sample();
     this._ramUtilization.sample();
-    this._detectAndHandleTestDuplicates();
+    const { duplicates, testIdToTests } = this._detectDuplicates(fileSuites);
+    if (duplicates.length) {
+      this._warnDuplicates(duplicates);
+      if (this._options.duplicates === "rename") {
+        this._renameDuplicates(duplicates, testIdToTests);
+      } else {
+        this._failDuplicates(duplicates);
+      }
+    }
     const duration = Date.now() - this._startTimestamp;
     const report = ReportUtils.normalizeReport({
       url: CIUtils.runUrl(),
@@ -330,7 +335,7 @@ class ReporterImpl {
       environments: this._environments,
       startTimestamp: this._startTimestamp,
       duration,
-      suites: Array.from(this._fileSuites.values()),
+      suites: fileSuites,
       unattributedErrors: unhandledErrors.map((error) => ({
         message: error.message,
         stack: error.stack,
@@ -353,19 +358,13 @@ class ReporterImpl {
         logger: this._logger
       });
     }
-    const openMode = this._options.open ?? "on-failure";
-    const shouldOpen = process.stdin.isTTY && !process.env.CI && (openMode === "always" || openMode === "on-failure" && reason === "failed");
-    if (shouldOpen) {
-      await showReport(outputFolder);
-    } else {
-      const defaultOutputFolder = path.join(process.cwd(), "flakiness-report");
-      const folder = defaultOutputFolder === outputFolder ? "" : path.relative(process.cwd(), outputFolder);
-      this._logger.log(`
+    const defaultOutputFolder = path.join(process.cwd(), "flakiness-report");
+    const folder = defaultOutputFolder === outputFolder ? "" : path.relative(process.cwd(), outputFolder);
+    this._logger.log(`
 To open last Flakiness report, run:
 
   ${chalk.cyan(`npx flakiness show ${folder}`)}
-      `);
-    }
+    `);
   }
 }
 function dupeSuffix(dupeIndex) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@flakiness/vitest",
-  "version": "0.1.0-alpha.2",
+  "version": "1.1.0",
   "description": "",
   "main": "lib/reporter.js",
   "keywords": [],
@@ -12,6 +12,7 @@
   },
   "type": "module",
   "devDependencies": {
+    "@flakiness/vitest": "0.1.0-alpha.2",
     "@types/node": "^25.3.3",
     "@vitest/utils": "^4.0.18",
     "esbuild": "^0.27.3",

package/types/src/reporter.d.ts

@@ -1,4 +1,4 @@
-import type { SerializedError, TestCase, TestModule, TestRunEndReason, Vitest } from 'vitest/node';
+import type { SerializedError, TestModule, TestRunEndReason, Vitest } from 'vitest/node';
 import type { Reporter } from 'vitest/reporters';
 interface UserConsoleLog {
     content: string;
@@ -9,36 +9,13 @@ interface UserConsoleLog {
     time: number;
     size: number;
 }
-interface TestArtifactLocation {
-    /** Line number in the source file (1-indexed) */
-    line: number;
-    /** Column number in the line (1-indexed) */
-    column: number;
-    /** Path to the source file */
-    file: string;
-}
-interface TestAttachment {
-    /** MIME type of the attachment (e.g., 'image/png', 'text/plain') */
-    contentType?: string;
-    /** File system path to the attachment */
-    path?: string;
-    /** Inline attachment content as a string or raw binary data */
-    body?: string | Uint8Array;
-}
-interface TestAnnotation {
-    message: string;
-    type: string;
-    location?: TestArtifactLocation;
-    attachment?: TestAttachment;
-}
-export type OpenMode = 'always' | 'never' | 'on-failure';
 export type FKVitestReporterOptions = {
     disableUpload?: boolean;
     flakinessProject?: string;
     endpoint?: string;
     token?: string;
     outputFolder?: string;
-    open?: OpenMode;
+    duplicates?: 'fail' | 'rename';
 };
 export interface FKVitestLogger {
     log(txt: string): void;
@@ -48,17 +25,13 @@ export interface FKVitestLogger {
 export default class FKVitestReporter implements Reporter {
     private _options;
     private _impl?;
+    private _vitest?;
     private _logger;
     constructor(_options: FKVitestReporterOptions);
     setLoggerForTest(logger: FKVitestLogger): void;
-    onUserConsoleLog(log: UserConsoleLog): void;
+    onUserConsoleLog(log: UserConsoleLog): Promise<void>;
     onInit(vitest: Vitest): void;
-    /**
-    * Called when annotation is added via the `task.annotate` API.
-    */
-    onTestCaseAnnotate(testCase: TestCase, annotation: TestAnnotation): void;
     onTestRunStart(): void;
-    onTestCaseResult(testCase: TestCase): Promise<void>;
     onTestRunEnd(testModules: ReadonlyArray<TestModule>, unhandledErrors: ReadonlyArray<SerializedError>, reason: TestRunEndReason): Promise<void>;
 }
 export {};

package/types/src/reporter.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"reporter.d.ts","sourceRoot":"","sources":["../../src/reporter.ts"],"names":[],"mappings":"AAOA,OAAO,KAAK,EAAE,eAAe,EAAE,QAAQ,EAAE,UAAU,EAAE,gBAAgB,EAAa,MAAM,EAAE,MAAM,aAAa,CAAC;AAC9G,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAIjD,UAAU,cAAc;IACvB,OAAO,EAAE,MAAM,CAAC;IAChB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,IAAI,EAAE,QAAQ,GAAG,QAAQ,CAAC;IAC1B,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,MAAM,CAAC;CACb;AACD,UAAU,oBAAoB;IAC7B,iDAAiD;IACjD,IAAI,EAAE,MAAM,CAAC;IACb,4CAA4C;IAC5C,MAAM,EAAE,MAAM,CAAC;IACf,8BAA8B;IAC9B,IAAI,EAAE,MAAM,CAAC;CACb;AACD,UAAU,cAAc;IACvB,oEAAoE;IACpE,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,yCAAyC;IACzC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,+DAA+D;IAC/D,IAAI,CAAC,EAAE,MAAM,GAAG,UAAU,CAAC;CAC3B;AACD,UAAU,cAAc;IACvB,OAAO,EAAE,MAAM,CAAC;IAChB,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,CAAC,EAAE,oBAAoB,CAAC;IAChC,UAAU,CAAC,EAAE,cAAc,CAAC;CAC5B;AAED,MAAM,MAAM,QAAQ,GAAG,QAAQ,GAAG,OAAO,GAAG,YAAY,CAAC;AAEzD,MAAM,MAAM,uBAAuB,GAAG;IACpC,aAAa,CAAC,EAAE,OAAO,CAAC;IACxB,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,IAAI,CAAC,EAAE,QAAQ,CAAC;CACjB,CAAA;AAED,MAAM,WAAW,cAAc;IAC7B,GAAG,CAAC,GAAG,EAAE,MAAM,GAAG,IAAI,CAAC;IACvB,IAAI,CAAC,GAAG,EAAE,MAAM,GAAG,IAAI,CAAC;IACxB,KAAK,CAAC,GAAG,EAAE,MAAM,GAAG,IAAI,CAAC;CAC1B;AAED,MAAM,CAAC,OAAO,OAAO,gBAAiB,YAAW,QAAQ;IAQ3C,OAAO,CAAC,QAAQ;IAP5B,OAAO,CAAC,KAAK,CAAC,CAAe;IAC7B,OAAO,CAAC,OAAO,CAId;gBAEmB,QAAQ,EAAE,uBAAuB;IAGrD,gBAAgB,CAAC,MAAM,EAAE,cAAc;IAIvC,gBAAgB,CAAC,GAAG,EAAE,cAAc;IAIpC,MAAM,CAAC,MAAM,EAAE,MAAM;IAKtB;;MAEE;IACF,kBAAkB,CAAC,QAAQ,EAAE,QAAQ,EAAE,UAAU,EAAE,cAAc;IAIhE,cAAc;IAIR,gBAAgB,CAAC,QAAQ,EAAE,QAAQ;IAInC,YAAY,CAAC,WAAW,EAAE,aAAa,CAAC,UAAU,CAAC,EAAE,eAAe,EAAE,aAAa,CAAC,eAAe,CAAC,EAAE,MAAM,EAAE,gBAAgB;CAGrI"}
+{"version":3,"file":"reporter.d.ts","sourceRoot":"","sources":["../../src/reporter.ts"],"names":[],"mappings":"AAOA,OAAO,KAAK,EAAE,eAAe,EAAY,UAAU,EAAE,gBAAgB,EAAa,MAAM,EAAE,MAAM,aAAa,CAAC;AAC9G,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAIjD,UAAU,cAAc;IACtB,OAAO,EAAE,MAAM,CAAC;IAChB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,IAAI,EAAE,QAAQ,GAAG,QAAQ,CAAC;IAC1B,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,MAAM,CAAC;CACd;AAED,MAAM,MAAM,uBAAuB,GAAG;IACpC,aAAa,CAAC,EAAE,OAAO,CAAC;IACxB,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,UAAU,CAAC,EAAE,MAAM,GAAC,QAAQ,CAAC;CAC9B,CAAA;AAED,MAAM,WAAW,cAAc;IAC7B,GAAG,CAAC,GAAG,EAAE,MAAM,GAAG,IAAI,CAAC;IACvB,IAAI,CAAC,GAAG,EAAE,MAAM,GAAG,IAAI,CAAC;IACxB,KAAK,CAAC,GAAG,EAAE,MAAM,GAAG,IAAI,CAAC;CAC1B;AAED,MAAM,CAAC,OAAO,OAAO,gBAAiB,YAAW,QAAQ;IAS3C,OAAO,CAAC,QAAQ;IAR5B,OAAO,CAAC,KAAK,CAAC,CAAe;IAC7B,OAAO,CAAC,OAAO,CAAC,CAAS;IACzB,OAAO,CAAC,OAAO,CAId;gBAEmB,QAAQ,EAAE,uBAAuB;IAGrD,gBAAgB,CAAC,MAAM,EAAE,cAAc;IAIjC,gBAAgB,CAAC,GAAG,EAAE,cAAc;IAI1C,MAAM,CAAC,MAAM,EAAE,MAAM;IAIrB,cAAc;IAOR,YAAY,CAAC,WAAW,EAAE,aAAa,CAAC,UAAU,CAAC,EAAE,eAAe,EAAE,aAAa,CAAC,eAAe,CAAC,EAAE,MAAM,EAAE,gBAAgB;CAIrI"}