task-summary-extractor 8.1.0 → 9.0.0

package/README.md

@@ -1,16 +1,17 @@
 # Task Summary Extractor
 
-> **v8.1.0** — AI-powered meeting analysis & document generation CLI. Install globally, run anywhere.
+> **v9.0.0** — AI-powered content analysis CLI — meetings, recordings, documents, or any mix. Install globally, run anywhere.
 
 <p align="center">
   <img src="https://img.shields.io/badge/node-%3E%3D18.0.0-green" alt="Node.js" />
   <img src="https://img.shields.io/badge/gemini-2.5--flash-blue" alt="Gemini" />
   <img src="https://img.shields.io/badge/firebase-11.x-orange" alt="Firebase" />
-  <img src="https://img.shields.io/badge/version-8.1.0-brightgreen" alt="Version" />
+  <img src="https://img.shields.io/badge/version-9.0.0-brightgreen" alt="Version" />
+  <img src="https://img.shields.io/badge/tests-285%20passing-brightgreen" alt="Tests" />
   <img src="https://img.shields.io/badge/npm-task--summary--extractor-red" alt="npm" />
 </p>
 
-**Record a meeting → get a structured task document.** Or point it at any folder and generate docs from context.
+**Analyze any content → get a structured task document.** Feed it meeting recordings, audio files, documents, or any mix — it extracts work items, action items, blockers, and more. Or point it at any folder and generate docs from context.
 
 📖 **New here?** Jump to [Setup (3 steps)](#setup-3-steps) — you'll be running in under 5 minutes.
 
@@ -18,9 +19,9 @@
 
 ## What It Does
 
-### 🎥 Video Analysis (default mode)
+### 🎥 Content Analysis (default mode)
 
-Drop a recording in a folder → run the tool → get a Markdown task document with:
+Drop a recording (video/audio) or documents in a folder → run the tool → get a Markdown task document with:
 
 - **Tickets** — ID, title, status, assignee, confidence score
 - **Change Requests** — what changed, where, how, why
@@ -168,6 +169,9 @@ These are the ones you'll actually use:
 | `--resume` | Continue an interrupted run | `--resume` |
 | `--reanalyze` | Force fresh analysis (ignore cache) | `--reanalyze` |
 | `--dry-run` | Preview what would run, without running | `--dry-run` |
+| `--format <type>` | Output format: `md`, `html`, `json`, `pdf`, `docx`, `all` (default: `md`) | `--format html` |
+| `--min-confidence <level>` | Filter items by confidence: `high`, `medium`, `low` | `--min-confidence high` |
+| `--no-html` | Suppress HTML report generation | `--no-html` |
 
 **Typical usage:**
 
@@ -191,7 +195,7 @@ Choose what the tool does. Only use one at a time:
 
 | Flag | Mode | What You Get |
 |------|------|-------------|
-| *(none)* | **Video analysis** | `results.md` — structured task document |
+| *(none)* | **Content analysis** | `results.md` + `results.html` — structured task document |
 | `--dynamic` | **Doc generation** | `INDEX.md` + 3–15 topic documents |
 | `--deep-dive` | **Topic explainers** | `INDEX.md` + per-topic deep-dive docs |
 | `--update-progress` | **Progress check** | `progress.md` — item status via git |
@@ -257,6 +261,8 @@ CONFIG     --gemini-key  --firebase-key  --firebase-project
            --firebase-bucket  --firebase-domain
 MODES      --dynamic  --deep-dive  --update-progress
 CORE       --name  --model  --skip-upload  --resume  --reanalyze  --dry-run
+OUTPUT     --format <md|html|json|pdf|docx|all>  --min-confidence <high|medium|low>
+           --no-html
 UPLOAD     --force-upload  --no-storage-url
 SKIP       --skip-compression  --skip-gemini
 DYNAMIC    --request <text>
@@ -271,11 +277,12 @@ INFO       --help (-h)  --version (-v)
 
 ## Output
 
-### Video Analysis
+### Content Analysis
 
 ```
 my-meeting/runs/{timestamp}/
 ├── results.md            ← Open this — your task document
+├── results.html          ← Interactive HTML report (self-contained)
 ├── results.json          ← Full pipeline data
 └── compilation.json      ← All extracted items (JSON)
 ```
@@ -312,12 +319,12 @@ my-meeting/runs/{timestamp}/
 
 ## Folder Setup Tips
 
-Drop docs alongside your video to give the AI context. **More context = better extraction.**
+Drop content files and supporting docs in a folder. **More context = better extraction.**
 
 ```
 my-meeting/
-├── Recording.mp4                  ← Video (required for video mode)
-├── Recording.vtt                  ← Subtitles (highly recommended)
+├── Recording.mp4                  ← Video recording (primary for video mode)
+├── Recording.vtt                  ← Subtitles (highly recommended for recordings)
 ├── agenda.md                      ← Loose docs at root are fine
 │
 ├── .tasks/                        ← Gets priority weighting (optional)
@@ -328,7 +335,7 @@ my-meeting/
     └── requirements.md
 ```
 
-**Supported formats:** `.mp4` `.mkv` `.webm` `.avi` `.mov` (video) · `.vtt` `.srt` `.txt` `.md` `.csv` `.pdf` (docs)
+**Supported formats:** `.mp4` `.mkv` `.webm` `.avi` `.mov` (video) · `.mp3` `.wav` `.ogg` `.m4a` (audio) · `.vtt` `.srt` `.txt` `.md` `.csv` `.pdf` (docs)
 
 The tool **recursively scans all subfolders**. `.tasks/` gets highest priority weighting but everything is included.
 
@@ -406,7 +413,7 @@ GEMINI_API_KEY=your-key-here
 
 | Feature | Description |
 |---------|-------------|
-| **Video Compression** | H.264 CRF 24, text-optimized sharpening, configurable speed |
+| **Video/Audio Compression** | H.264 CRF 24, text-optimized sharpening, configurable speed |
 | **Smart Segmentation** | ≤5 min chunks with boundary-aware splitting |
 | **Cross-Segment Continuity** | Ticket IDs, names, and context carry forward |
 | **Document Discovery** | Auto-finds docs in all subfolders |
@@ -423,6 +430,11 @@ GEMINI_API_KEY=your-key-here
 | **Git Progress Tracking** | Correlates commits with extracted items |
 | **Deep Dive** | Explanatory docs per topic discussed |
 | **Dynamic Mode** | Generate docs from any content mix |
+| **Progress Bar** | Real-time visual progress with phase tracking, ETA, and cost display |
+| **HTML Report** | Self-contained HTML report with collapsible sections, filtering, dark mode |
+| **JSON Schema Validation** | Validates AI output against JSON Schema (segment + compiled) |
+| **Confidence Filter** | `--min-confidence` flag to exclude low-confidence items from output |
+| **Multi-Format Output** | `--format` flag: Markdown, HTML, JSON, PDF, DOCX, or all formats at once |
 | **Interactive CLI** | Run with no args → guided experience |
 | **Resume / Checkpoint** | `--resume` continues interrupted runs |
 | **Firebase Upload** | Team access via cloud (optional) |
@@ -468,23 +480,57 @@ Your call folders, `.env`, logs, and videos are all `.gitignore`d — nothing ge
 task-summary-extractor/
 ├── bin/
 │   └── taskex.js               Global CLI entry point
-├── process_and_upload.js       Backward-compatible entry point
+├── process_and_upload.js       Backward-compatible entry (delegates to bin/taskex)
 ├── setup.js                    First-time setup & validation
 ├── package.json                Dependencies, scripts, bin config
 ├── prompt.json                 Gemini extraction prompt
+├── vitest.config.js            Test configuration
 │
 ├── src/
 │   ├── config.js               Config, model registry, env vars
 │   ├── logger.js               Structured JSONL logger (triple output)
-│   ├── pipeline.js             Multi-mode orchestrator (1,985 lines)
+│   ├── pipeline.js             Multi-mode orchestrator (~920 lines)
+│   ├── phases/                 Decomposed pipeline phases (9 modules)
+│   │   ├── _shared.js          Shared phase utilities
+│   │   ├── init.js             Phase 1: CLI parsing, config validation
+│   │   ├── discover.js         Phase 2: Find videos, docs, resolve user
+│   │   ├── services.js         Phase 3: Firebase auth, Gemini init
+│   │   ├── process-media.js    Phase 4: Compress, upload, analyze
+│   │   ├── compile.js          Phase 5: Cross-segment compilation
+│   │   ├── output.js           Phase 6: Write JSON, render MD + HTML
+│   │   ├── summary.js          Phase 8: Save learning, print summary
+│   │   └── deep-dive.js        Phase 9: Optional deep-dive generation
 │   ├── services/
 │   │   ├── gemini.js           Gemini AI — 3-strategy file resolution + External URL support
 │   │   ├── firebase.js         Firebase Storage (async I/O)
 │   │   ├── video.js            ffmpeg compression
-│   │   └── git.js              Git CLI wrapper
+│   │   ├── git.js              Git CLI wrapper
+│   │   └── doc-parser.js       Document text extraction (DOCX, XLSX, PPTX, etc.)
+│   ├── modes/                  AI-heavy pipeline phase modules
+│   │   ├── deep-dive.js        Topic discovery & deep-dive doc generation
+│   │   ├── dynamic-mode.js     Dynamic document planning & generation
+│   │   ├── focused-reanalysis.js  Targeted reanalysis of weak segments
+│   │   ├── progress-updater.js Git-based progress assessment
+│   │   └── change-detector.js  Git change correlation engine
 │   ├── renderers/
-│   │   └── markdown.js         Report renderer
-│   └── utils/                  21 modules — see ARCHITECTURE.md
+│   │   ├── markdown.js         Markdown report renderer
+│   │   ├── html.js             HTML report renderer (self-contained)
+│   │   ├── pdf.js              PDF report renderer (HTML → PDF via puppeteer)
+│   │   ├── docx.js             DOCX report renderer (programmatic Word document)
+│   │   └── shared.js           Shared renderer utilities
+│   ├── schemas/
+│   │   ├── analysis-segment.schema.json   Segment analysis JSON Schema
+│   │   └── analysis-compiled.schema.json  Compiled analysis JSON Schema
+│   └── utils/                  Pure utilities — parsing, retry, budget, config
+│       ├── colors.js           Zero-dep ANSI color utility
+│       ├── progress-bar.js     Visual progress bar (TTY-aware)
+│       ├── confidence-filter.js  Confidence level filtering
+│       ├── schema-validator.js JSON Schema validation (ajv)
+│       └── ... (15 more utility modules)
+│
+├── tests/                      Test suite — 285 tests across 13 files (vitest)
+│   ├── utils/                  Utility module tests
+│   └── renderers/              Renderer tests
 │
 ├── QUICK_START.md              Step-by-step setup guide
 ├── ARCHITECTURE.md             Technical deep dive
@@ -505,6 +551,9 @@ task-summary-extractor/
 | `npm run check` | Validate environment |
 | `npm start` | Run the pipeline |
 | `npm run help` | Show CLI help |
+| `npm test` | Run test suite (285 tests) |
+| `npm run test:watch` | Run tests in watch mode |
+| `npm run test:coverage` | Run tests with coverage report |
 
 ---
 
@@ -512,6 +561,9 @@ task-summary-extractor/
 
 | Version | Highlights |
 |---------|-----------|
+| **v9.0.0** | **CLI UX upgrade** — colors & progress bar, HTML reports, PDF & DOCX output (via puppeteer and docx npm package), JSON Schema validation, confidence filter (`--min-confidence`), pipeline decomposition (`src/phases/` — 9 modules), test suite (285 tests via vitest), multi-format output (`--format`: md/html/json/pdf/docx/all), doc-parser service, shared renderer utilities |
+| **v8.3.0** | **Universal content analysis** — prompt v4.0.0 supports video, audio, documents, and mixed content; input type auto-detection; timestamps conditional on content type; gemini.js bridge text generalized; all markdown docs updated |
+| **v8.2.0** | **Architecture cleanup** — `src/modes/` for AI pipeline phases, `retry.js` self-contained defaults, dead code removal, export trimming, `process_and_upload.js` slim shim, `progress.js` → `checkpoint.js`, merged `prompt.js` into `cli.js` |
 | **v8.1.0** | **Smart global config** — `taskex config` persistent setup (`~/.taskexrc`), first-run prompting, 5-level config resolution, production audit fixes, shared CLI flag injection, boolean flag parser fix |
 | **v8.0.0** | **npm package** — `npm i -g task-summary-extractor`, `taskex` global CLI, `--gemini-key` / `--firebase-*` config flags, run from anywhere, CWD-first `.env` resolution |
 | **v7.2.3** | Production hardening — cross-platform ffmpeg, shell injection fix, auto git init for progress tracking, `runs/` excluded from doc discovery |
@@ -541,4 +593,4 @@ task-summary-extractor/
 
 ## License
 
-MIT — © 2026 Youssef Adel
+Proprietary — © 2026 Youssef Adel. All rights reserved.

package/bin/taskex.js

@@ -1,4 +1,4 @@
-#!/usr/bin/env node
+#!/usr/bin/env node
 /**
  * taskex — AI-powered meeting analysis & document generation.
  *
@@ -7,6 +7,8 @@
  * Usage:   taskex [options] [folder]
  *
  * Subcommands:
+ *   taskex setup               Full interactive setup (prerequisites, deps, .env, sample folder)
+ *   taskex setup --check       Validation only (no changes)
  *   taskex config              Interactive global config setup (~/.taskexrc)
  *   taskex config --show       Show saved config (masked secrets)
  *   taskex config --clear      Remove global config file
@@ -24,7 +26,7 @@
 
 'use strict';
 
-// ── Handle `taskex config` subcommand before anything else ────────────────
+// ── Handle subcommands before anything else ──────────────────────────────
 const rawArgs = process.argv.slice(2);
 if (rawArgs[0] === 'config') {
   const hasShow  = rawArgs.includes('--show');
@@ -36,6 +38,10 @@ if (rawArgs[0] === 'config') {
     process.stderr.write(`\nError: ${err.message}\n`);
     process.exit(1);
   });
+} else if (rawArgs[0] === 'setup') {
+  // Delegate to setup.js — supports --check and --silent
+  const setupPath = require('path').join(__dirname, '..', 'setup.js');
+  require('child_process').execFileSync(process.execPath, [setupPath, ...rawArgs.slice(1)], { stdio: 'inherit' });
 } else {
   // ── Inject CLI config flags into process.env ────────────────────────────
   // Must run BEFORE any require() that touches config.js / dotenv
@@ -44,6 +50,7 @@ if (rawArgs[0] === 'config') {
 
   // ── Delegate to pipeline ────────────────────────────────────────────────
   const { run, getLog } = require('../src/pipeline');
+  const { c } = require('../src/utils/colors');
 
   run().catch(err => {
     if (err.code === 'HELP_SHOWN' || err.code === 'VERSION_SHOWN') {
@@ -57,8 +64,8 @@ if (rawArgs[0] === 'config') {
       log.step('FAILED');
       log.close();
     }
-    process.stderr.write(`\nFATAL: ${err.message || err}\n`);
-    process.stderr.write(`${err.stack || ''}\n`);
+    process.stderr.write(`\n${c.error(`FATAL: ${err.message || err}`)}\n`);
+    process.stderr.write(`${c.dim(err.stack || '')}\n`);
     process.exit(1);
   });
 }

package/package.json

@@ -1,10 +1,10 @@
 {
   "name": "task-summary-extractor",
-  "version": "8.1.0",
+  "version": "9.0.0",
   "description": "AI-powered meeting analysis & document generation CLI — video + document processing, deep dive docs, dynamic mode, interactive CLI with model selection, confidence scoring, learning loop, git progress tracking",
   "main": "process_and_upload.js",
   "bin": {
-    "taskex": "./bin/taskex.js"
+    "taskex": "bin/taskex.js"
   },
   "files": [
     "bin/",
@@ -12,6 +12,7 @@
     "prompt.json",
     "process_and_upload.js",
     "setup.js",
+    ".env.example",
     "README.md",
     "QUICK_START.md",
     "ARCHITECTURE.md",
@@ -19,15 +20,37 @@
   ],
   "scripts": {
     "setup": "node setup.js",
-    "check": "node setup.js --check",
+    "setup:check": "node setup.js --check",
+    "setup:silent": "node setup.js --silent",
     "start": "node process_and_upload.js",
     "process": "node process_and_upload.js",
     "process:local": "node process_and_upload.js --skip-upload",
     "process:resume": "node process_and_upload.js --resume",
+    "process:reanalyze": "node process_and_upload.js --reanalyze",
     "process:dry-run": "node process_and_upload.js --dry-run",
+    "process:fast": "node process_and_upload.js --parallel 5 --parallel-analysis 3",
+    "process:turbo": "node process_and_upload.js --parallel 8 --parallel-analysis 3",
+    "process:serial": "node process_and_upload.js --parallel 1 --parallel-analysis 1",
+    "process:minimal": "node process_and_upload.js --skip-upload --no-focused-pass --no-learning --no-diff",
+    "process:no-ai": "node process_and_upload.js --skip-gemini",
+    "process:upload-only": "node process_and_upload.js --skip-compression --skip-gemini",
+    "deep-dive": "node process_and_upload.js --deep-dive",
+    "deep-dive:local": "node process_and_upload.js --deep-dive --skip-upload",
+    "deep-dive:fast": "node process_and_upload.js --deep-dive --parallel 5 --parallel-analysis 3",
+    "deep-dive:resume": "node process_and_upload.js --deep-dive --resume",
     "dynamic": "node process_and_upload.js --dynamic",
+    "dynamic:local": "node process_and_upload.js --dynamic --skip-upload",
+    "dynamic:fast": "node process_and_upload.js --dynamic --parallel 5 --parallel-analysis 3",
+    "dynamic:resume": "node process_and_upload.js --dynamic --resume",
+    "progress": "node process_and_upload.js --update-progress",
+    "config": "node process_and_upload.js config",
+    "config:show": "node process_and_upload.js config --show",
+    "config:clear": "node process_and_upload.js config --clear",
     "help": "node process_and_upload.js --help",
-    "version": "node process_and_upload.js --version"
+    "version": "node process_and_upload.js --version",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "test:coverage": "vitest run --coverage"
   },
   "keywords": [
     "meeting",
@@ -51,13 +74,23 @@
     "url": "https://github.com/youssefadel94/task-summary-extractor/issues"
   },
   "author": "Youssef Adel",
-  "license": "MIT",
+  "license": "UNLICENSED",
   "dependencies": {
     "@google/genai": "^1.42.0",
+    "ajv": "^8.18.0",
+    "docx": "^9.6.0",
     "dotenv": "^17.3.1",
-    "firebase": "^12.9.0"
+    "firebase": "^12.9.0",
+    "mammoth": "^1.11.0",
+    "officeparser": "^6.0.4",
+    "puppeteer": "^24.37.5",
+    "xlsx": "^0.18.5"
   },
   "engines": {
     "node": ">=18.0.0"
+  },
+  "devDependencies": {
+    "@vitest/coverage-v8": "^4.0.18",
+    "vitest": "^4.0.18"
   }
 }

package/process_and_upload.js

@@ -1,107 +1,14 @@
 #!/usr/bin/env node
 /**
- * taskex — AI-powered meeting analysis & document generation.
+ * Backward-compatible entry point — delegates to bin/taskex.js.
  *
- * Backward-compatible entry point — delegates to src/pipeline.js.
- * For global installs, use the `taskex` command directly.
+ * For global installs, use the `taskex` CLI command directly.
+ * This file exists for `node process_and_upload.js` and `npm start` compatibility.
  *
  * Usage:
- *   taskex [options] [folder]
- *   node process_and_upload.js [options] "C:\path\to\call folder"
+ *   taskex [options] [folder]                      (recommended)
+ *   node process_and_upload.js [options] [folder]   (legacy)
  *
- * Config flags (override .env):
- *   --gemini-key <key>          Gemini API key
- *   --firebase-key <key>        Firebase API key
- *   --firebase-project <id>     Firebase project ID
- *   --firebase-bucket <bucket>  Firebase storage bucket
- *   --firebase-domain <domain>  Firebase auth domain
- *
- * Options:
- *   --name <name>              Your name (skips interactive prompt)
- *   --model <id>               Gemini model (default: gemini-2.5-flash)
- *   --skip-upload              Skip Firebase Storage uploads
- *   --force-upload             Upload even if remote file exists
- *   --no-storage-url           Disable Storage URL strategy for Gemini
- *   --skip-compression         Skip video compression (use existing segments)
- *   --skip-gemini              Skip Gemini AI analysis
- *   --resume                   Resume from last checkpoint
- *   --reanalyze                Force re-analysis of all segments
- *   --parallel <n>             Max parallel uploads (default: 3)
- *   --parallel-analysis <n>    Max concurrent Gemini analyses (default: 2)
- *   --thinking-budget <n>      Gemini thinking token budget
- *   --compilation-thinking-budget <n>  Compilation thinking budget
- *   --log-level <level>        Log level: debug, info, warn, error
- *   --output <dir>             Custom output directory
- *   --dry-run                  Show what would be done without executing
- *   --dynamic                  Document-only mode (no video required)
- *   --deep-dive                Generate deep-dive documents after analysis
- *   --request <text>           Custom research prompt for deep-dive/dynamic
- *   --update-progress          Smart change detection & progress update
- *   --repo <path>              Git repo path for progress tracking
- *   --no-focused-pass          Disable focused re-analysis pass
- *   --no-learning              Disable learning loop
- *   --no-diff                  Disable diff against previous run
- *   --help, -h                 Show help
- *   --version, -v              Show version
- *
- * Project structure:
- *   src/
- *     config.js               — Environment-based config with validation
- *     logger.js               — Buffered dual-file logger with levels
- *     pipeline.js             — Main orchestrator with CLI flags & progress
- *     services/
- *       firebase.js           — Firebase init, upload with retry, exists checks
- *       gemini.js             — Gemini init, segment analysis with retry
- *       git.js                — Git CLI wrapper for change detection
- *       video.js              — ffmpeg compression, segmentation, probing
- *     renderers/
- *       markdown.js           — Action-focused Markdown renderer
- *     utils/
- *       adaptive-budget.js    — Transcript complexity → thinking budget
- *       change-detector.js    — Git + document change correlation engine
- *       cli.js                — CLI argument parser & interactive prompts
- *       context-manager.js    — Smart context prioritization for Gemini
- *       cost-tracker.js       — Model-specific token cost tracking
- *       deep-dive.js          — AI topic discovery & document generation
- *       diff-engine.js        — Compilation diff between runs
- *       dynamic-mode.js       — Document-only analysis mode
- *       focused-reanalysis.js — Second-pass extraction for weak dimensions
- *       format.js             — Duration/size formatting helpers
- *       fs.js                 — Recursive file discovery
- *       health-dashboard.js   — Quality report builder
- *       json-parser.js        — Robust JSON extraction from AI output
- *       learning-loop.js      — Cross-run history & trend analysis
- *       progress.js           — Pipeline checkpoint/resume persistence
- *       progress-updater.js   — Smart progress assessment & rendering
- *       prompt.js             — Interactive CLI prompts (stdin/stdout)
- *       quality-gate.js       — Multi-dimension confidence scoring
- *       retry.js              — Exponential backoff retry with parallelMap
+ * Run `taskex --help` for full CLI reference.
  */
-
-'use strict';
-
-// ── Inject CLI config flags into process.env ──────────────────────────────
-// Must run BEFORE any require() that touches config.js / dotenv
-const { injectCliFlags } = require('./src/utils/inject-cli-flags');
-injectCliFlags();
-
-// ── Delegate to pipeline ──────────────────────────────────────────────────
-const { run, getLog } = require('./src/pipeline');
-
-run().catch(err => {
-  // showHelp() throws with code HELP_SHOWN — clean exit, not an error
-  if (err.code === 'HELP_SHOWN' || err.code === 'VERSION_SHOWN') {
-    process.exit(0);
-  }
-
-  const log = getLog();
-  if (log) {
-    log.error(`FATAL: ${err.message || err}`);
-    log.error(err.stack || '');
-    log.step('FAILED');
-    log.close();
-  }
-  process.stderr.write(`\nFATAL: ${err.message || err}\n`);
-  process.stderr.write(`${err.stack || ''}\n`);
-  process.exit(1);
-});
+require('./bin/taskex');