cyborg-hunter 0.3.0

package/README.md

@@ -0,0 +1,145 @@
+# cyborg-hunter
+
+Detects AI-tool use during browser-based behavioral experiments. Captures paste, copy, drag, tab-away, mouse trajectories, browser sidebar openings, and other signals that compromise data quality on Prolific / MTurk / classroom studies. Produces a triage report ranking participants by suspiciousness.
+
+### Example: 100-participant Prolific pilot
+
+Top of the triage list (excerpt — full ranked CSV in the report):
+
+| Rank | Participant | Score | Hard | Reason |
+|------|-------------|-------|------|--------|
+| 1 | 67c1d6d98c8787be36609212 | 4078 | no | 1 tab-away 3–10s; 2 flickers <3s; fast typing on 9 trials; **1015 synthetic insertions** |
+| 2 | 69bc21dd22f17b2337957511 | 179 | **YES** | 21 paste events; 3 copy events; 9 tab-aways ≥10s; 3 sidebar events; 21 layout shifts |
+| 3 | 69b9f3184e0896ef41b99aa8 | 152 | no | 38 synthetic insertions |
+| 4 | 5d350282cec7150015d16494 | 131 | **YES** | 5 paste events; 10 copy events; 22 tab-aways ≥10s; 5 tab-aways 3–10s; 15 flickers <3s |
+| 5 | 696a9773a84c5f9d4a930874 | 128 | **YES** | 15 paste events; 23 tab-aways ≥10s; 11 tab-aways 3–10s; 9 flickers <3s |
+| 6 | 69b594dccc64df571f092147 | 123 | **YES** | 12 paste events; 16 tab-aways ≥10s; 4 tab-aways 3–10s |
+| 7 | 69cd5149cf6fc07126897938 | 120 | **YES** | 15 paste events; 1 copy events; 19 tab-aways ≥10s; 16 flickers <3s |
+| 8 | 6314bc49055e150c92f4736e | 112 | **YES** | 12 paste events; 14 tab-aways ≥10s; 46 tab-aways 3–10s; 3 flickers <3s |
+
+Of 100 participants, 6 hard-flagged (count thresholds crossed) and 12 soft-flagged (weighted score over threshold). Per-participant tab-away timeline below — this is rank 4, with 42 tab-away events across 15 trials. Reviewers can scan the timeline visually instead of staring at a 30-column CSV:
+
+![Tab-away timeline showing 42 tab-away events across 15 trials. Most trials have multiple long red bars (≥10s tab-aways), with some shorter orange (3-10s) and grey flicker (<3s) events.](docs/images/example-tab-timeline.png)
+
+Two pieces:
+
+- A small JS library you load in your experiment that records signals as participants work.
+- A CLI that reads the resulting data files and renders an HTML report.
+
+## Install
+
+For analysis (CLI), once published to npm:
+
+```bash
+npm install -g cyborg-hunter   # not yet published — clone the repo for now
+```
+
+For the experiment page (library), copy `dist/cyborg-hunter.min.js` and `dist/jspsych-cyborg-hunter.js` from this repo into your project. The library is currently distributed as files, not via a CDN.
+
+## Plug into a jsPsych experiment
+
+```javascript
+const jsPsych = initJsPsych({
+  extensions: [
+    { type: jsPsychCyborgHunter, params: { participantId: subject.id, preset: 'standard' } }
+  ],
+  on_finish: function () {
+    // REQUIRED: jsPsych 7 has no on_finish_experiment extension hook, so the
+    // session-level signals must be flushed manually before saving data.
+    jsPsych.extensions['cyborg-hunter'].finalize();
+    jsPsych.data.get().localSave('csv', 'data.csv');
+  }
+});
+
+// Opt every trial in to monitoring (single forEach beats editing N trials):
+timeline.forEach(t => {
+  t.extensions = (t.extensions || []).concat([{ type: jsPsychCyborgHunter }]);
+});
+
+jsPsych.run(timeline);
+```
+
+Full setup walk-through, including standalone (non-jsPsych) usage and per-trial parameters: [`docs/using-cyborg-hunter.md`](docs/using-cyborg-hunter.md).
+
+## Generate a report
+
+After data collection, in the directory holding your data files:
+
+```bash
+cyborg-hunter init      # writes cyborg-hunter.config.json
+# edit config: dataDir, filePattern, participantIdField
+cyborg-hunter report    # writes ./cyborg-hunter-report/
+open cyborg-hunter-report/index.html
+```
+
+The report contains:
+
+- `summary.csv` — one row per participant, every signal as a column
+- `triage.md` — ranked list with one-line "why flagged" per participant
+- `event-log.csv` — chronological copy/paste/drop/tab-away events
+- `images/` — per-participant mouse trajectories, tab timelines, typing-speed profiles
+- `index.html` — landing page that ties it together
+
+## What it detects
+
+| Signal | How | Class |
+|---|---|---|
+| Paste | Clipboard `paste` events | Hard (count threshold) |
+| Drag-and-drop | `drop` events on inputs | Hard (count threshold) |
+| Copy | Clipboard `copy` events | Soft (weighted) |
+| Tab-away | `visibilitychange` + `blur`/`focus` | Soft (weighted) |
+| Browser sidebar | `innerWidth` delta + layout compression | Soft (weighted) |
+| Suspicious typing speed | chars/sec exceeding the preset's threshold | Soft (weighted) |
+| Synthetic insertion | text appearing without keystrokes | Soft (weighted) |
+| Foreign input | typing landing outside the experiment container | Soft (weighted) |
+| Idle gaps | input inactivity (potential context-switch out) | Soft (weighted) |
+| AI-extension content scripts | DOM scan for known extension selectors | Soft (weighted) |
+| Mouse trajectories | 20Hz polling + path-efficiency / direction-change metrics | Diagnostic |
+| Window/screen geometry | polled + resize-event capture, with zoom inference | Diagnostic |
+
+Three presets — `permissive` (pilot), `standard` (default), `strict` (high-stakes). Per-signal thresholds documented in [`docs/signals-reference.md`](docs/signals-reference.md).
+
+## What it doesn't detect
+
+- **Native browser AI sidebars.** Chrome's built-in Gemini panel and similar are not browser extensions and leave no extension content scripts. The `innerWidth_delta` heuristic still catches them as a generic sidebar event, but the named-extension column will be empty.
+- **Screen-share / second device.** A participant reading questions on screen 1 and querying ChatGPT on a phone produces no detectable trace from the browser.
+- **AI text edited and retyped.** A determined participant who copies AI output, retypes it character-by-character, and never tab-switches will look clean. The mouse-trajectory and typing-rhythm signals make this much harder than it sounds, but the tool isn't a polygraph.
+
+These are stated up front so reviewers know what flagging really represents.
+
+## How it works
+
+The library hooks browser events that are universally available (Clipboard API, Visibility API, MouseEvent, Performance API, MutationObserver) and tracks two scopes in parallel:
+
+- **Session-scoped monitors** start at experiment load and run continuously: tab-away, sidebar/extension detection, layout shifts, window position polling.
+- **Trial-scoped trackers** bracket each trial: mouse path, typing speed, paste/copy/drop within the trial's response window, idle gaps.
+
+At the end of each trial, an "integrity" object is attached to that trial's data. At the end of the experiment, session-level rollups are attached to the last trial. The CLI ingests both standard jsPsych CSV output and project-specific JSON formats (`docs/configuration.md` describes the config-file fields).
+
+## Documentation
+
+- [`docs/using-cyborg-hunter.md`](docs/using-cyborg-hunter.md) — full integration guide (jsPsych and standalone)
+- [`docs/signals-reference.md`](docs/signals-reference.md) — every signal with thresholds per preset
+- [`docs/configuration.md`](docs/configuration.md) — config file fields and CLI flags
+- [`docs/cli-reference.md`](docs/cli-reference.md) — commands and output structure
+- [`docs/plans/2026-04-03-integrity-library-and-calibration-design.md`](docs/plans/2026-04-03-integrity-library-and-calibration-design.md) — design rationale (two-tier scoring, signal taxonomy, preset choice)
+
+## Citation
+
+If you use cyborg-hunter in academic work, please cite:
+
+```bibtex
+@software{konuk2026cyborghunter,
+  author  = {Konuk, Can},
+  title   = {cyborg-hunter: integrity monitoring for online behavioral experiments},
+  year    = {2026},
+  url     = {https://github.com/konukcan/cyborg-hunter},
+  version = {0.3.0}
+}
+```
+
+GitHub also renders a "Cite this repository" button (sidebar of the repo page) from the CITATION.cff file in the root.
+
+## License
+
+MIT

package/bin/cyborg-hunter.js

@@ -0,0 +1,20 @@
+#!/usr/bin/env node
+// CLI entry point for Cyborg Hunter.
+// Usage:
+//   npx cyborg-hunter init                       # generate starter config
+//   npx cyborg-hunter report [--config path] ...  # generate report
+
+import { run } from '../src/cli/report.js';
+import { runInit } from '../src/cli/init.js';
+
+const args = process.argv.slice(2);
+const subcommand = args[0];
+
+if (subcommand === 'init') {
+  runInit().catch(err => { console.error('Error:', err.message); process.exit(1); });
+} else if (subcommand === 'report' || !subcommand) {
+  run(args).catch(err => { console.error('Error:', err.message); process.exit(1); });
+} else {
+  console.error(`Unknown command: ${subcommand}. Use "report" or "init".`);
+  process.exit(1);
+}