@thunderkiller/video-clipper 1.2.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +19 -0
- package/CONTRIBUTING.md +100 -0
- package/LICENSE +15 -0
- package/commitlint.config.js +25 -0
- package/package.json +3 -1
- package/.github/workflows/ci.yml +0 -42
- package/.github/workflows/release.yml +0 -76
- package/.husky/pre-commit +0 -3
- package/.prettierignore +0 -6
- package/.prettierrc +0 -7
- package/.releaserc.json +0 -21
- package/AGENTS.md +0 -122
- package/docs/free-models.md +0 -78
- package/docs/plan.md +0 -442
- package/docs/refactorPhases.md +0 -105
- package/docs/yt-downloader.md +0 -440
- package/requirements.txt +0 -5
- package/scripts/detect_events.py +0 -81
- package/scripts/detect_events_whisper.py +0 -101
- package/scripts/transcribe_whisper.py +0 -70
- package/src/cli.ts +0 -186
- package/src/config/env.ts +0 -18
- package/src/config/index.ts +0 -2
- package/src/index.ts +0 -46
- package/src/pipeline/runner.ts +0 -147
- package/src/pipeline/stages/audioProcessor.ts +0 -127
- package/src/pipeline/stages/clipExporter.ts +0 -76
- package/src/pipeline/stages/segmentAnalyzer.ts +0 -72
- package/src/pipeline/stages/segmentSelector.ts +0 -39
- package/src/pipeline/stages/videoResolver.ts +0 -44
- package/src/services/audioAnalyzers/base.ts +0 -32
- package/src/services/audioAnalyzers/factory.ts +0 -69
- package/src/services/audioAnalyzers/gemini.ts +0 -136
- package/src/services/audioAnalyzers/index.ts +0 -6
- package/src/services/audioAnalyzers/whisper.ts +0 -80
- package/src/services/audioAnalyzers/yamnet.ts +0 -54
- package/src/services/audioDownloader/index.ts +0 -102
- package/src/services/chunkBuilder/index.ts +0 -82
- package/src/services/clipGenerator/index.ts +0 -210
- package/src/services/clipRefiner/index.ts +0 -141
- package/src/services/eventDetector/index.ts +0 -68
- package/src/services/llmAnalyzer/LLMAnalyzer.ts +0 -98
- package/src/services/llmAnalyzer/index.ts +0 -231
- package/src/services/metadataExtractor/index.ts +0 -83
- package/src/services/segmentRanker/index.ts +0 -88
- package/src/services/signalMerger/index.ts +0 -53
- package/src/services/transcriptAnalyzers/base.ts +0 -26
- package/src/services/transcriptAnalyzers/factory.ts +0 -66
- package/src/services/transcriptAnalyzers/gemini.ts +0 -24
- package/src/services/transcriptAnalyzers/index.ts +0 -6
- package/src/services/transcriptAnalyzers/whisper.ts +0 -68
- package/src/services/transcriptAnalyzers/ytdlp.ts +0 -19
- package/src/services/transcriptDetector/index.ts +0 -122
- package/src/services/transcriptFetcher/index.ts +0 -147
- package/src/services/urlParser/index.ts +0 -52
- package/src/services/videoDownloader/index.ts +0 -268
- package/src/types/analyzer.ts +0 -23
- package/src/types/audio.ts +0 -19
- package/src/types/cache.ts +0 -8
- package/src/types/cli.ts +0 -22
- package/src/types/config.ts +0 -151
- package/src/types/downloader.ts +0 -15
- package/src/types/factory.ts +0 -3
- package/src/types/index.ts +0 -40
- package/src/types/pipeline.ts +0 -60
- package/src/types/segment.ts +0 -43
- package/src/types/transcript.ts +0 -22
- package/src/types/video.ts +0 -18
- package/src/utils/cache.ts +0 -224
- package/src/utils/chunker.ts +0 -60
- package/src/utils/dumper.ts +0 -41
- package/src/utils/format.ts +0 -10
- package/src/utils/logger.ts +0 -17
- package/src/utils/modelFactory.ts +0 -71
- package/src/utils/redactConfig.ts +0 -23
- package/src/utils/sliceAudio.ts +0 -35
- package/test-trigger.txt +0 -1
- package/tests/analyzerFactory.test.ts +0 -146
- package/tests/audioEventDetector.test.ts +0 -69
- package/tests/cache.test.ts +0 -203
- package/tests/chunkBuilder.test.ts +0 -146
- package/tests/chunker.test.ts +0 -95
- package/tests/eventDetector.test.ts +0 -103
- package/tests/llmAnalyzer.test.ts +0 -283
- package/tests/segmentRanker.test.ts +0 -133
- package/tests/setup.ts +0 -48
- package/tests/signalMerger.test.ts +0 -197
- package/tests/transcriptDetector.test.ts +0 -150
- package/tests/transcriptFetcher.test.ts +0 -179
- package/tests/urlParser.test.ts +0 -70
- package/tsconfig.json +0 -16
- package/tsconfig.test.json +0 -8
- package/vitest.config.ts +0 -8
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,22 @@
|
|
|
1
|
+
# [1.4.0](https://github.com/AmreetKumarkhuntia/video-clipper/compare/v1.3.1...v1.4.0) (2026-03-20)
|
|
2
|
+
|
|
3
|
+
### Features
|
|
4
|
+
|
|
5
|
+
- **commitlint:** enforce conventional commit message format ([9d99882](https://github.com/AmreetKumarkhuntia/video-clipper/commit/9d998828617be72a81fbe4e72caa15eb28e88b82))
|
|
6
|
+
|
|
7
|
+
## [1.3.1](https://github.com/AmreetKumarkhuntia/video-clipper/compare/v1.3.0...v1.3.1) (2026-03-20)
|
|
8
|
+
|
|
9
|
+
### Bug Fixes
|
|
10
|
+
|
|
11
|
+
- **release:** add @semantic-release/npm plugin to update package.json version ([fe6498d](https://github.com/AmreetKumarkhuntia/video-clipper/commit/fe6498d779459455387e2ca2736862e704ea8b1c))
|
|
12
|
+
- **release:** add @semantic-release/npm to update package.json version ([9de3397](https://github.com/AmreetKumarkhuntia/video-clipper/commit/9de33977fe9f856aa8749436cbee7422dd622322))
|
|
13
|
+
|
|
14
|
+
# [1.3.0](https://github.com/AmreetKumarkhuntia/video-clipper/compare/v1.2.0...v1.3.0) (2026-03-19)
|
|
15
|
+
|
|
16
|
+
### Features
|
|
17
|
+
|
|
18
|
+
- **npm:** configure npm package publishing ([b268345](https://github.com/AmreetKumarkhuntia/video-clipper/commit/b26834559ce97db683c393db765d7d5c7e989236))
|
|
19
|
+
|
|
1
20
|
# [1.2.0](https://github.com/AmreetKumarkhuntia/video-clipper/compare/v1.1.1...v1.2.0) (2026-03-19)
|
|
2
21
|
|
|
3
22
|
### Features
|
package/CONTRIBUTING.md
ADDED
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
# Contributing
|
|
2
|
+
|
|
3
|
+
## Getting Started
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
pnpm install
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
This will install all dependencies and set up git hooks via Husky.
|
|
10
|
+
|
|
11
|
+
## Commit Message Format
|
|
12
|
+
|
|
13
|
+
This project uses [Conventional Commits](https://www.conventionalcommits.org/) enforced by [commitlint](https://commitlint.js.org/).
|
|
14
|
+
|
|
15
|
+
### Format
|
|
16
|
+
|
|
17
|
+
```
|
|
18
|
+
<type>(<scope>): <short description>
|
|
19
|
+
|
|
20
|
+
- <detail 1>
|
|
21
|
+
- <detail 2>
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
### Types
|
|
25
|
+
|
|
26
|
+
| Type | Description | Version Bump |
|
|
27
|
+
| ---------- | ------------------------------------------------------- | ---------------------- |
|
|
28
|
+
| `feat` | New feature | minor (1.0.0 -> 1.1.0) |
|
|
29
|
+
| `fix` | Bug fix | patch (1.0.0 -> 1.0.1) |
|
|
30
|
+
| `docs` | Documentation only | none |
|
|
31
|
+
| `refactor` | Code change that neither fixes a bug nor adds a feature | none |
|
|
32
|
+
| `test` | Adding or updating tests | none |
|
|
33
|
+
| `chore` | Maintenance tasks (deps, config, etc.) | none |
|
|
34
|
+
| `style` | Formatting, missing semicolons, etc. (no code change) | none |
|
|
35
|
+
| `perf` | Performance improvement | none |
|
|
36
|
+
| `ci` | CI/CD configuration changes | none |
|
|
37
|
+
| `build` | Build system or external dependency changes | none |
|
|
38
|
+
| `revert` | Reverts a previous commit | none |
|
|
39
|
+
|
|
40
|
+
### Rules
|
|
41
|
+
|
|
42
|
+
- **type** (required): one from the table above
|
|
43
|
+
- **scope** (recommended): kebab-case name of the feature area, e.g. `release`, `transcript-fetcher`, `llm-analyzer`
|
|
44
|
+
- **short description** (required): lowercase, imperative mood, max 100 chars, no period at end
|
|
45
|
+
- **body** (optional): pointwise with `-`, max 500 chars total
|
|
46
|
+
|
|
47
|
+
### Examples
|
|
48
|
+
|
|
49
|
+
```
|
|
50
|
+
feat(clip-refiner): add overlap detection for adjacent segments
|
|
51
|
+
|
|
52
|
+
- detect when refined segments overlap by more than 2s
|
|
53
|
+
- merge overlapping segments and keep the higher-scored one
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
fix(release): add @semantic-release/npm to update package.json version
|
|
58
|
+
|
|
59
|
+
- add @semantic-release/npm with npmPublish: false
|
|
60
|
+
- npm publish remains a separate workflow step
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
```
|
|
64
|
+
docs(readme): add advanced examples section
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
### Breaking Changes
|
|
68
|
+
|
|
69
|
+
Add `BREAKING CHANGE:` in the commit body or `!` after the type to trigger a major version bump:
|
|
70
|
+
|
|
71
|
+
```
|
|
72
|
+
feat(config)!: rename ENV_VAR to NEW_ENV_VAR
|
|
73
|
+
|
|
74
|
+
BREAKING CHANGE: ENV_VAR is no longer supported, use NEW_ENV_VAR instead
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## Git Hooks
|
|
78
|
+
|
|
79
|
+
The following hooks run automatically on every commit:
|
|
80
|
+
|
|
81
|
+
| Hook | What it runs |
|
|
82
|
+
| ------------ | ------------------------------------------------------------- |
|
|
83
|
+
| `pre-commit` | `lint-staged` (Prettier), `type-check` (tsc), `test` (vitest) |
|
|
84
|
+
| `commit-msg` | `commitlint` (validates commit message format) |
|
|
85
|
+
|
|
86
|
+
If your commit is rejected, check the error output — it will tell you exactly which rule was violated.
|
|
87
|
+
|
|
88
|
+
## Code Style
|
|
89
|
+
|
|
90
|
+
- TypeScript only — no plain `.js` files
|
|
91
|
+
- Prettier handles formatting automatically via `lint-staged`
|
|
92
|
+
- Run `pnpm format` to format all files manually
|
|
93
|
+
- Run `pnpm type-check` to check for type errors
|
|
94
|
+
|
|
95
|
+
## Testing
|
|
96
|
+
|
|
97
|
+
- Write unit tests for pure functions
|
|
98
|
+
- Test files live in `tests/` at the project root
|
|
99
|
+
- Run `pnpm test` to run all tests
|
|
100
|
+
- Run `pnpm test:watch` for watch mode
|
package/LICENSE
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
ISC License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025, @thunderkiller/video-clipper
|
|
4
|
+
|
|
5
|
+
Permission to use, copy, modify, and/or distribute this software for any
|
|
6
|
+
purpose with or without fee is hereby granted, provided that the above
|
|
7
|
+
copyright notice and this permission notice appear in all copies.
|
|
8
|
+
|
|
9
|
+
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
|
10
|
+
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
|
11
|
+
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
|
12
|
+
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
|
13
|
+
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
|
14
|
+
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
|
15
|
+
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
export default {
|
|
2
|
+
extends: ['@commitlint/config-conventional'],
|
|
3
|
+
rules: {
|
|
4
|
+
'type-enum': [
|
|
5
|
+
2,
|
|
6
|
+
'always',
|
|
7
|
+
[
|
|
8
|
+
'feat',
|
|
9
|
+
'fix',
|
|
10
|
+
'docs',
|
|
11
|
+
'refactor',
|
|
12
|
+
'test',
|
|
13
|
+
'chore',
|
|
14
|
+
'style',
|
|
15
|
+
'perf',
|
|
16
|
+
'ci',
|
|
17
|
+
'build',
|
|
18
|
+
'revert',
|
|
19
|
+
],
|
|
20
|
+
],
|
|
21
|
+
'scope-case': [2, 'always', 'kebab-case'],
|
|
22
|
+
'subject-max-length': [2, 'always', 100],
|
|
23
|
+
'body-max-line-length': [2, 'always', 500],
|
|
24
|
+
},
|
|
25
|
+
};
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@thunderkiller/video-clipper",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.4.0",
|
|
4
4
|
"description": "CLI that analyzes YouTube transcripts with an LLM to find interesting moments and cut clips",
|
|
5
5
|
"license": "ISC",
|
|
6
6
|
"author": "",
|
|
@@ -49,6 +49,8 @@
|
|
|
49
49
|
"zod": "^4.3.6"
|
|
50
50
|
},
|
|
51
51
|
"devDependencies": {
|
|
52
|
+
"@commitlint/cli": "^20.5.0",
|
|
53
|
+
"@commitlint/config-conventional": "^20.5.0",
|
|
52
54
|
"@semantic-release/changelog": "^6.0.3",
|
|
53
55
|
"@semantic-release/git": "^10.0.1",
|
|
54
56
|
"@semantic-release/github": "^12.0.6",
|
package/.github/workflows/ci.yml
DELETED
|
@@ -1,42 +0,0 @@
|
|
|
1
|
-
name: CI
|
|
2
|
-
|
|
3
|
-
on:
|
|
4
|
-
push:
|
|
5
|
-
branches:
|
|
6
|
-
- main
|
|
7
|
-
- master
|
|
8
|
-
pull_request:
|
|
9
|
-
branches:
|
|
10
|
-
- main
|
|
11
|
-
- master
|
|
12
|
-
jobs:
|
|
13
|
-
ci:
|
|
14
|
-
name: Type-check, Format, Test & Build
|
|
15
|
-
runs-on: ubuntu-latest
|
|
16
|
-
|
|
17
|
-
steps:
|
|
18
|
-
- name: Checkout
|
|
19
|
-
uses: actions/checkout@v4
|
|
20
|
-
|
|
21
|
-
- name: Setup Node.js (LTS)
|
|
22
|
-
uses: actions/setup-node@v4
|
|
23
|
-
with:
|
|
24
|
-
node-version: '24'
|
|
25
|
-
|
|
26
|
-
- name: Setup pnpm
|
|
27
|
-
uses: pnpm/action-setup@v4
|
|
28
|
-
|
|
29
|
-
- name: Install dependencies
|
|
30
|
-
run: pnpm install --frozen-lockfile
|
|
31
|
-
|
|
32
|
-
- name: Type-check
|
|
33
|
-
run: pnpm type-check
|
|
34
|
-
|
|
35
|
-
- name: Format check
|
|
36
|
-
run: pnpm format:check
|
|
37
|
-
|
|
38
|
-
- name: Test
|
|
39
|
-
run: pnpm test
|
|
40
|
-
|
|
41
|
-
- name: Build
|
|
42
|
-
run: pnpm build
|
|
@@ -1,76 +0,0 @@
|
|
|
1
|
-
name: Release
|
|
2
|
-
|
|
3
|
-
on:
|
|
4
|
-
push:
|
|
5
|
-
branches:
|
|
6
|
-
- master
|
|
7
|
-
|
|
8
|
-
jobs:
|
|
9
|
-
release:
|
|
10
|
-
name: Semantic Release
|
|
11
|
-
runs-on: ubuntu-latest
|
|
12
|
-
permissions:
|
|
13
|
-
contents: write
|
|
14
|
-
id-token: write
|
|
15
|
-
|
|
16
|
-
steps:
|
|
17
|
-
- name: Checkout
|
|
18
|
-
uses: actions/checkout@v4
|
|
19
|
-
with:
|
|
20
|
-
fetch-depth: 0
|
|
21
|
-
persist-credentials: false
|
|
22
|
-
|
|
23
|
-
- name: Setup pnpm
|
|
24
|
-
uses: pnpm/action-setup@v4
|
|
25
|
-
|
|
26
|
-
- name: Setup Node.js
|
|
27
|
-
uses: actions/setup-node@v4
|
|
28
|
-
with:
|
|
29
|
-
node-version: '24'
|
|
30
|
-
registry-url: 'https://registry.npmjs.org'
|
|
31
|
-
|
|
32
|
-
- name: Setup npm auth
|
|
33
|
-
run: echo "//registry.npmjs.org/:_authToken=\${{ secrets.NPM_TOKEN }}" > ~/.npmrc
|
|
34
|
-
|
|
35
|
-
- name: Install dependencies
|
|
36
|
-
run: pnpm install --frozen-lockfile
|
|
37
|
-
|
|
38
|
-
- name: Build
|
|
39
|
-
run: pnpm build
|
|
40
|
-
|
|
41
|
-
- name: Configure Git
|
|
42
|
-
run: |
|
|
43
|
-
git config user.name "github-actions[bot]"
|
|
44
|
-
git config user.email "github-actions[bot]@users.noreply.github.com"
|
|
45
|
-
git remote set-url origin https://x-access-token:${{ secrets.PUSH_TOKEN }}@github.com/${{ github.repository }}.git
|
|
46
|
-
|
|
47
|
-
- name: Stage 1 GitHub Release (versioning, tags, releases)
|
|
48
|
-
id: github_release
|
|
49
|
-
run: npx semantic-release
|
|
50
|
-
env:
|
|
51
|
-
GITHUB_TOKEN: ${{ secrets.PUSH_TOKEN }}
|
|
52
|
-
|
|
53
|
-
- name: Pull updated package.json after semantic-release
|
|
54
|
-
if: success()
|
|
55
|
-
run: git pull origin master
|
|
56
|
-
|
|
57
|
-
- name: Verify GitHub Release
|
|
58
|
-
if: success()
|
|
59
|
-
run: |
|
|
60
|
-
echo "Checking for new git tags..."
|
|
61
|
-
CURRENT_TAG=$(git describe --tags --abbrev=0 2>/dev/null || echo "none")
|
|
62
|
-
echo "Current tag: $CURRENT_TAG"
|
|
63
|
-
|
|
64
|
-
if [ "$CURRENT_TAG" != "none" ]; then
|
|
65
|
-
echo "✅ New tag created: $CURRENT_TAG"
|
|
66
|
-
echo "Proceeding to npm publish..."
|
|
67
|
-
else
|
|
68
|
-
echo "❌ No new tag created - skipping npm publish"
|
|
69
|
-
exit 1
|
|
70
|
-
fi
|
|
71
|
-
|
|
72
|
-
- name: Stage 2 Publish to NPM
|
|
73
|
-
if: success()
|
|
74
|
-
run: npm publish
|
|
75
|
-
env:
|
|
76
|
-
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
|
package/.husky/pre-commit
DELETED
package/.prettierignore
DELETED
package/.prettierrc
DELETED
package/.releaserc.json
DELETED
|
@@ -1,21 +0,0 @@
|
|
|
1
|
-
{
|
|
2
|
-
"branches": ["master"],
|
|
3
|
-
"plugins": [
|
|
4
|
-
"@semantic-release/commit-analyzer",
|
|
5
|
-
"@semantic-release/release-notes-generator",
|
|
6
|
-
[
|
|
7
|
-
"@semantic-release/changelog",
|
|
8
|
-
{
|
|
9
|
-
"changelogFile": "CHANGELOG.md"
|
|
10
|
-
}
|
|
11
|
-
],
|
|
12
|
-
[
|
|
13
|
-
"@semantic-release/git",
|
|
14
|
-
{
|
|
15
|
-
"assets": ["package.json", "CHANGELOG.md"],
|
|
16
|
-
"message": "chore(release): ${nextRelease.version} [skip ci]"
|
|
17
|
-
}
|
|
18
|
-
],
|
|
19
|
-
"@semantic-release/github"
|
|
20
|
-
]
|
|
21
|
-
}
|
package/AGENTS.md
DELETED
|
@@ -1,122 +0,0 @@
|
|
|
1
|
-
# Agent Instructions
|
|
2
|
-
|
|
3
|
-
## Project
|
|
4
|
-
|
|
5
|
-
TypeScript CLI that analyzes YouTube transcripts with an LLM to find interesting moments and optionally cut video clips.
|
|
6
|
-
|
|
7
|
-
See `docs/plan.md` for the full architecture.
|
|
8
|
-
|
|
9
|
-
## Stack
|
|
10
|
-
|
|
11
|
-
- TypeScript (Node.js 18+)
|
|
12
|
-
- Vercel AI SDK (`ai`, `@ai-sdk/openai`, `@ai-sdk/anthropic`, `@ai-sdk/google`, `@ai-sdk/xai`, `@ai-sdk/mistral`, `@ai-sdk/groq`) with `generateObject` + `zod` for structured LLM output
|
|
13
|
-
- Multi-provider support: OpenAI, Anthropic, Google, XAI, Mistral, Groq, Zai, OpenRouter
|
|
14
|
-
- `youtube-transcript` for transcript fetching
|
|
15
|
-
- `yt-dlp` + `execa` for video download
|
|
16
|
-
- `fluent-ffmpeg` for clip cutting
|
|
17
|
-
- `zod` for config validation at startup
|
|
18
|
-
- `p-limit` for concurrency control
|
|
19
|
-
|
|
20
|
-
## Project Structure
|
|
21
|
-
|
|
22
|
-
```
|
|
23
|
-
src/
|
|
24
|
-
config/ # zod-validated env config — import this, never read process.env directly
|
|
25
|
-
index.ts
|
|
26
|
-
env.ts
|
|
27
|
-
services/ # core pipeline modules
|
|
28
|
-
urlParser/
|
|
29
|
-
metadataExtractor/
|
|
30
|
-
transcriptFetcher/
|
|
31
|
-
chunkBuilder/
|
|
32
|
-
llmAnalyzer/
|
|
33
|
-
segmentRanker/
|
|
34
|
-
clipRefiner/
|
|
35
|
-
videoDownloader/
|
|
36
|
-
clipGenerator/
|
|
37
|
-
types/ # shared TypeScript types
|
|
38
|
-
index.ts
|
|
39
|
-
config.ts
|
|
40
|
-
segment.ts
|
|
41
|
-
transcript.ts
|
|
42
|
-
video.ts
|
|
43
|
-
youtube-transcript.d.ts
|
|
44
|
-
utils/ # utility functions
|
|
45
|
-
cache.ts # transcript + chunk LLM result caching
|
|
46
|
-
dumper.ts # transcript/analysis JSON dumps
|
|
47
|
-
redactConfig.ts # config formatting for logs (redacts API keys)
|
|
48
|
-
format.ts # timestamp formatting utilities
|
|
49
|
-
logger.ts # logging utilities
|
|
50
|
-
modelFactory.ts # LLM provider factory
|
|
51
|
-
index.ts # CLI entrypoint
|
|
52
|
-
tests/ # all unit tests (mirrors module names)
|
|
53
|
-
urlParser.test.ts
|
|
54
|
-
chunkBuilder.test.ts
|
|
55
|
-
segmentRanker.test.ts
|
|
56
|
-
downloads/ # yt-dlp output (gitignored)
|
|
57
|
-
outputs/ # ffmpeg clip output, caches, dumps (gitignored)
|
|
58
|
-
cache/ # transcript + LLM result cache
|
|
59
|
-
transcript/ # transcript dumps
|
|
60
|
-
analysis/ # analysis dumps
|
|
61
|
-
docs/
|
|
62
|
-
plan.md
|
|
63
|
-
free-models.md
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
## Code Rules
|
|
67
|
-
|
|
68
|
-
- All code in TypeScript — no plain `.js` files
|
|
69
|
-
- Every function must have explicit input/output types; avoid `any`
|
|
70
|
-
- Use `zod` for all external data validation (LLM output, env vars, API responses)
|
|
71
|
-
- Never read `process.env` directly — always import from `src/config.ts`
|
|
72
|
-
- Never hardcode API keys, model names, thresholds, or directory paths — all come from config
|
|
73
|
-
- Use `async/await` — no raw `.then()` chains
|
|
74
|
-
- Use `Promise.allSettled` for parallel LLM calls so one failure doesn't abort the rest
|
|
75
|
-
- Handle errors explicitly — no silent catches. Log a warning with the chunk index and reason on skip.
|
|
76
|
-
|
|
77
|
-
## LLM Usage
|
|
78
|
-
|
|
79
|
-
- Use `generateObject` (not `generateText`) for all LLM calls that return structured data
|
|
80
|
-
- Define a `zod` schema for every LLM response before writing the prompt
|
|
81
|
-
- Keep prompts in the same file as the function that uses them
|
|
82
|
-
- Do not retry on malformed JSON — `generateObject` handles structured output natively
|
|
83
|
-
- On any LLM call failure, catch and log, then continue — never crash the pipeline
|
|
84
|
-
|
|
85
|
-
## Module Conventions
|
|
86
|
-
|
|
87
|
-
Each module in `src/modules/` should:
|
|
88
|
-
|
|
89
|
-
- Export a single main function named after the module (e.g. `fetchTranscript`, `buildChunks`)
|
|
90
|
-
- Accept typed inputs and return typed outputs (no `any`)
|
|
91
|
-
- Not import from other modules except `config.ts` and `types.ts` unless there is a clear dependency
|
|
92
|
-
|
|
93
|
-
## Transcript Notes
|
|
94
|
-
|
|
95
|
-
- `youtube-transcript` returns `offset` in **milliseconds** — normalize to seconds immediately after fetching
|
|
96
|
-
- Micro-blocks group raw lines into ~15s windows before chunking
|
|
97
|
-
- LLM chunks are 120s windows with 20s overlap — built from micro-blocks, not raw lines
|
|
98
|
-
|
|
99
|
-
## Naming
|
|
100
|
-
|
|
101
|
-
- Files: `camelCase.ts`
|
|
102
|
-
- Functions: `camelCase`
|
|
103
|
-
- Types/interfaces: `PascalCase`
|
|
104
|
-
- Constants: `UPPER_SNAKE_CASE`
|
|
105
|
-
- Zod schemas: `PascalCase` + `Schema` suffix (e.g. `SegmentSchema`)
|
|
106
|
-
|
|
107
|
-
## Testing
|
|
108
|
-
|
|
109
|
-
- Write unit tests for pure functions (URL parser, chunker, ranker, deduplicator)
|
|
110
|
-
- Do not unit test functions that call external services (LLM, yt-dlp, ffmpeg) — integration test those separately
|
|
111
|
-
- Test files live in `tests/` at the project root, mirroring the module name (e.g. `tests/urlParser.test.ts`)
|
|
112
|
-
|
|
113
|
-
## Git
|
|
114
|
-
|
|
115
|
-
- Commit messages: lowercase, imperative
|
|
116
|
-
- One logical change per commit
|
|
117
|
-
- Never commit `.env`, `downloads/`, or `outputs/`
|
|
118
|
-
- It should be pointwise `-`
|
|
119
|
-
- Shouldn't be more than 500 characters all together
|
|
120
|
-
- it should be categorized as feat: , fix:, docs:, refactor etc...
|
|
121
|
-
- also it should include specifics like feat(<feature>)
|
|
122
|
-
- so final structure is <change ex: feat, docs, fix>: <short desc> followed by <full desc> pointwise
|
package/docs/free-models.md
DELETED
|
@@ -1,78 +0,0 @@
|
|
|
1
|
-
# Free Models via OpenRouter
|
|
2
|
-
|
|
3
|
-
This project supports [OpenRouter](https://openrouter.ai) as an LLM provider, which gives access to a number of completely free models (no cost per token).
|
|
4
|
-
|
|
5
|
-
## Setup
|
|
6
|
-
|
|
7
|
-
```env
|
|
8
|
-
LLM_PROVIDER=openrouter
|
|
9
|
-
OPENROUTER_API_KEY=sk-or-...
|
|
10
|
-
LLM_MODEL=<model-id-from-table-below>
|
|
11
|
-
```
|
|
12
|
-
|
|
13
|
-
---
|
|
14
|
-
|
|
15
|
-
## Recommended Free Models for Transcript Analysis
|
|
16
|
-
|
|
17
|
-
These models have sufficient context length and instruction-following capability to work well with `generateObject` and structured transcript analysis.
|
|
18
|
-
|
|
19
|
-
| Model ID | Name | Context |
|
|
20
|
-
| ----------------------------------------------- | --------------------------------- | ------- |
|
|
21
|
-
| `meta-llama/llama-3.3-70b-instruct:free` | Meta: Llama 3.3 70B Instruct | 128K |
|
|
22
|
-
| `google/gemma-3-27b-it:free` | Google: Gemma 3 27B | 131K |
|
|
23
|
-
| `mistralai/mistral-small-3.1-24b-instruct:free` | Mistral: Mistral Small 3.1 24B | 128K |
|
|
24
|
-
| `nousresearch/hermes-3-llama-3.1-405b:free` | Nous: Hermes 3 405B Instruct | 131K |
|
|
25
|
-
| `qwen/qwen3-next-80b-a3b-instruct:free` | Qwen: Qwen3 Next 80B A3B Instruct | 262K |
|
|
26
|
-
|
|
27
|
-
**Recommended default:** `meta-llama/llama-3.3-70b-instruct:free`
|
|
28
|
-
|
|
29
|
-
- Most battle-tested for structured JSON output
|
|
30
|
-
- Reliable with `generateObject` (which requires strict schema adherence)
|
|
31
|
-
- 128K context handles most YouTube transcripts comfortably
|
|
32
|
-
|
|
33
|
-
---
|
|
34
|
-
|
|
35
|
-
## All Available Free Models (as of March 2026)
|
|
36
|
-
|
|
37
|
-
| Model ID | Name | Context |
|
|
38
|
-
| --------------------------------------------------------------- | --------------------------------- | --------- |
|
|
39
|
-
| `openrouter/hunter-alpha` | Hunter Alpha | 1,048,576 |
|
|
40
|
-
| `openrouter/healer-alpha` | Healer Alpha | 262,144 |
|
|
41
|
-
| `nvidia/nemotron-3-super-120b-a12b:free` | NVIDIA: Nemotron 3 Super | 262,144 |
|
|
42
|
-
| `minimax/minimax-m2.5:free` | MiniMax: MiniMax M2.5 | 196,608 |
|
|
43
|
-
| `openrouter/free` | Free Models Router | 200,000 |
|
|
44
|
-
| `stepfun/step-3.5-flash:free` | StepFun: Step 3.5 Flash | 256,000 |
|
|
45
|
-
| `arcee-ai/trinity-large-preview:free` | Arcee AI: Trinity Large Preview | 131,000 |
|
|
46
|
-
| `liquid/lfm-2.5-1.2b-thinking:free` | LiquidAI: LFM2.5-1.2B-Thinking | 32,768 |
|
|
47
|
-
| `liquid/lfm-2.5-1.2b-instruct:free` | LiquidAI: LFM2.5-1.2B-Instruct | 32,768 |
|
|
48
|
-
| `nvidia/nemotron-3-nano-30b-a3b:free` | NVIDIA: Nemotron 3 Nano 30B A3B | 256,000 |
|
|
49
|
-
| `arcee-ai/trinity-mini:free` | Arcee AI: Trinity Mini | 131,072 |
|
|
50
|
-
| `nvidia/nemotron-nano-12b-v2-vl:free` | NVIDIA: Nemotron Nano 12B 2 VL | 128,000 |
|
|
51
|
-
| `qwen/qwen3-next-80b-a3b-instruct:free` | Qwen: Qwen3 Next 80B A3B Instruct | 262,144 |
|
|
52
|
-
| `nvidia/nemotron-nano-9b-v2:free` | NVIDIA: Nemotron Nano 9B V2 | 128,000 |
|
|
53
|
-
| `openai/gpt-oss-120b:free` | OpenAI: gpt-oss-120b | 131,072 |
|
|
54
|
-
| `openai/gpt-oss-20b:free` | OpenAI: gpt-oss-20b | 131,072 |
|
|
55
|
-
| `z-ai/glm-4.5-air:free` | Z.ai: GLM 4.5 Air | 131,072 |
|
|
56
|
-
| `qwen/qwen3-coder:free` | Qwen: Qwen3 Coder 480B A35B | 262,000 |
|
|
57
|
-
| `cognitivecomputations/dolphin-mistral-24b-venice-edition:free` | Venice: Uncensored | 32,768 |
|
|
58
|
-
| `google/gemma-3n-e2b-it:free` | Google: Gemma 3n 2B | 8,192 |
|
|
59
|
-
| `google/gemma-3n-e4b-it:free` | Google: Gemma 3n 4B | 8,192 |
|
|
60
|
-
| `qwen/qwen3-4b:free` | Qwen: Qwen3 4B | 40,960 |
|
|
61
|
-
| `mistralai/mistral-small-3.1-24b-instruct:free` | Mistral: Mistral Small 3.1 24B | 128,000 |
|
|
62
|
-
| `google/gemma-3-4b-it:free` | Google: Gemma 3 4B | 32,768 |
|
|
63
|
-
| `google/gemma-3-12b-it:free` | Google: Gemma 3 12B | 32,768 |
|
|
64
|
-
| `google/gemma-3-27b-it:free` | Google: Gemma 3 27B | 131,072 |
|
|
65
|
-
| `meta-llama/llama-3.3-70b-instruct:free` | Meta: Llama 3.3 70B Instruct | 128,000 |
|
|
66
|
-
| `meta-llama/llama-3.2-3b-instruct:free` | Meta: Llama 3.2 3B Instruct | 131,072 |
|
|
67
|
-
| `nousresearch/hermes-3-llama-3.1-405b:free` | Nous: Hermes 3 405B Instruct | 131,072 |
|
|
68
|
-
|
|
69
|
-
> Free models may have rate limits or availability constraints. Check [openrouter.ai/models](https://openrouter.ai/models) for the latest list.
|
|
70
|
-
|
|
71
|
-
---
|
|
72
|
-
|
|
73
|
-
## Notes
|
|
74
|
-
|
|
75
|
-
- Free models on OpenRouter are subject to rate limits (typically lower than paid tiers).
|
|
76
|
-
- Avoid models with context < 32K for long videos — chunks may not fit.
|
|
77
|
-
- Models marked `:free` are permanently free; others (like `openrouter/hunter-alpha`) may change pricing.
|
|
78
|
-
- If `generateObject` fails with a free model, it usually means the model doesn't follow JSON schema strictly enough — switch to `meta-llama/llama-3.3-70b-instruct:free` as a fallback.
|