@fettstorch/clai 0.1.8 → 0.1.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/settings.local.json +9 -0
- package/CLAUDE.md +66 -0
- package/dist/cli.js +638 -606
- package/dist/index.js +191 -270
- package/package.json +1 -1
- package/src/cli.ts +4 -0
- package/src/scraper.ts +12 -0
package/CLAUDE.md
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
# CLAUDE.md
|
|
2
|
+
|
|
3
|
+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
+
|
|
5
|
+
## Development Commands
|
|
6
|
+
|
|
7
|
+
- **Development**: `bun dev` - Watch mode with hot reload
|
|
8
|
+
- **Build**: `bun run build` - Build both CLI and library outputs
|
|
9
|
+
- **Start**: `bun start` - Run CLI directly from source
|
|
10
|
+
- **Linting**: Use Biome for formatting and linting (config in biome.json)
|
|
11
|
+
- **Testing**: No test framework currently configured
|
|
12
|
+
|
|
13
|
+
## Environment Setup
|
|
14
|
+
|
|
15
|
+
- Requires `OPENAI_API_KEY` environment variable for API access
|
|
16
|
+
- Uses Bun as runtime and package manager
|
|
17
|
+
- Node.js compatibility maintained for distribution
|
|
18
|
+
|
|
19
|
+
## Architecture Overview
|
|
20
|
+
|
|
21
|
+
**Core Components:**
|
|
22
|
+
- `src/cli.ts` - Main CLI entry point with Commander.js interface
|
|
23
|
+
- `src/index.ts` - Library API exposing the main `clai()` function
|
|
24
|
+
- `src/scraper.ts` - Multi-engine web scraping with anti-detection techniques
|
|
25
|
+
- `src/summarizer.ts` - OpenAI-powered content summarization
|
|
26
|
+
- `src/openai.ts` - OpenAI client wrapper with structured response support
|
|
27
|
+
|
|
28
|
+
**Data Flow:**
|
|
29
|
+
1. CLI/API accepts URL or search query
|
|
30
|
+
2. Scraper attempts multiple search engines (SearX → Google → DuckDuckGo → Wikipedia)
|
|
31
|
+
3. Content extraction using Cheerio from scraped pages
|
|
32
|
+
4. OpenAI summarization with structured response schema
|
|
33
|
+
5. Terminal display with animated text and interactive link selection
|
|
34
|
+
|
|
35
|
+
**Key Design Patterns:**
|
|
36
|
+
- Multi-fallback strategy for search engines to avoid blocking
|
|
37
|
+
- Enhanced fetch with browser-like headers for anti-bot detection
|
|
38
|
+
- Structured OpenAI responses using function calling
|
|
39
|
+
- Interactive CLI with inquirer prompts and ora spinners
|
|
40
|
+
|
|
41
|
+
## Code Conventions
|
|
42
|
+
|
|
43
|
+
- Uses tabs for indentation (configured in biome.json)
|
|
44
|
+
- Single quotes, trailing commas, semicolons as needed
|
|
45
|
+
- TypeScript with strict typing
|
|
46
|
+
- Readonly arrays and interfaces for immutable data
|
|
47
|
+
- Error handling with graceful fallbacks
|
|
48
|
+
|
|
49
|
+
## Notable Dependencies
|
|
50
|
+
|
|
51
|
+
- `@fettstorch/jule` - Utility library for functional programming
|
|
52
|
+
- `cheerio` - Server-side HTML parsing
|
|
53
|
+
- `commander` - CLI framework
|
|
54
|
+
- `inquirer` - Interactive CLI prompts
|
|
55
|
+
- `ora` - Terminal spinners
|
|
56
|
+
- `chalk` - Terminal colors
|
|
57
|
+
- `openai` - Official OpenAI client
|
|
58
|
+
|
|
59
|
+
## Scraper Anti-Detection Strategy
|
|
60
|
+
|
|
61
|
+
The scraper uses sophisticated techniques to avoid being blocked:
|
|
62
|
+
- Rotates realistic User-Agent strings
|
|
63
|
+
- Complete browser header fingerprinting
|
|
64
|
+
- Proper Sec-Fetch metadata and client hints
|
|
65
|
+
- Multiple search engine fallbacks
|
|
66
|
+
- Respectful request patterns without rapid-fire requests
|