llms-generator-cli 1.0.0 → 1.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +174 -0
  2. package/package.json +2 -2
package/README.md ADDED
@@ -0,0 +1,174 @@
1
+ # llms-gen
2
+
3
+ > A fast, lightweight CLI tool to generate `llms.txt` and `llms-full.txt` from any website URL — making your content instantly readable by AI language models.
4
+
5
+ [![npm version](https://img.shields.io/badge/version-1.0.0-blue.svg)](https://www.npmjs.com/)
6
+ [![license](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
7
+ [![node](https://img.shields.io/badge/node-%3E%3D14.0.0-brightgreen.svg)](https://nodejs.org/)
8
+
9
+ ---
10
+
11
+ ## What is llms.txt?
12
+
13
+ [`llms.txt`](https://llmstxt.org/) is an emerging standard that helps AI language models better understand your website. Similar to how `robots.txt` guides search engine crawlers, `llms.txt` provides structured, clean Markdown context that LLMs can efficiently read and reference.
14
+
15
+ **llms-gen** automates this process — just point it at a URL and it handles the rest.
16
+
17
+ ---
18
+
19
+ ## Features
20
+
21
+ - 🔍 **Auto-scrapes** any public URL using Axios + Cheerio
22
+ - 📝 **Converts HTML → clean Markdown** via Turndown
23
+ - 📄 **Generates `llms.txt`** — a concise, LLM-optimized summary
24
+ - 🚀 **Generates `llms-full.txt`** — full-page deep context (with `--advanced` flag)
25
+ - ⚡ **Zero config** — works out of the box
26
+ - 🛠️ **Globally installable** as a CLI command
27
+
28
+ ---
29
+
30
+ ## Installation
31
+
32
+ ### Option 1: Run directly with npx (no install needed)
33
+
34
+ ```bash
35
+ npx llms-generator-cli <url>
36
+ ```
37
+
38
+ ### Option 2: Install globally
39
+
40
+ ```bash
41
+ npm install -g llms-generator-cli
42
+ ```
43
+
44
+ ### Option 3: Clone and use locally
45
+
46
+ ```bash
47
+ git clone https://github.com/your-username/llms-generator-cli.git
48
+ cd llms-generator-cli
49
+ npm install
50
+ npm link
51
+ ```
52
+
53
+ ---
54
+
55
+ ## Usage
56
+
57
+ ### Basic — Generate `llms.txt`
58
+
59
+ ```bash
60
+ llms-gen https://example.com
61
+ ```
62
+
63
+ This fetches the page, extracts the main content, and writes a clean `llms.txt` file in your current directory.
64
+
65
+ ### Advanced — Generate `llms-full.txt`
66
+
67
+ ```bash
68
+ llms-gen https://example.com --advanced
69
+ ```
70
+
71
+ or with the shorthand flag:
72
+
73
+ ```bash
74
+ llms-gen https://example.com -a
75
+ ```
76
+
77
+ This generates **both** `llms.txt` (concise) and `llms-full.txt` (complete page context).
78
+
79
+ ---
80
+
81
+ ## Output Format
82
+
83
+ ### `llms.txt` (Standard)
84
+
85
+ ```
86
+ # Page Title
87
+
88
+ > Meta description of the page
89
+
90
+ ## Info
91
+ - URL: https://example.com
92
+
93
+ ## Core Content
94
+ ... (first ~2000 characters of clean Markdown content)
95
+ ```
96
+
97
+ ### `llms-full.txt` (Advanced)
98
+
99
+ ```
100
+ # Full Context: Page Title
101
+
102
+ ... (complete page content in clean Markdown, no truncation)
103
+ ```
104
+
105
+ ---
106
+
107
+ ## Examples
108
+
109
+ ```bash
110
+ # Generate llms.txt for a documentation site
111
+ llms-gen https://docs.example.com
112
+
113
+ # Generate full context for a product landing page
114
+ llms-gen https://myproduct.com -a
115
+
116
+ # Generate for a blog post
117
+ llms-gen https://myblog.com/my-article --advanced
118
+ ```
119
+
120
+ ---
121
+
122
+ ## Options
123
+
124
+ | Option | Alias | Description |
125
+ |--------|-------|-------------|
126
+ | `--advanced` | `-a` | Also generate `llms-full.txt` with full page content |
127
+ | `--version` | `-V` | Show version number |
128
+ | `--help` | `-h` | Show help information |
129
+
130
+ ---
131
+
132
+ ## Requirements
133
+
134
+ - **Node.js** v14.0.0 or higher
135
+ - **npm** v6.0.0 or higher
136
+
137
+ ---
138
+
139
+ ## Dependencies
140
+
141
+ | Package | Purpose |
142
+ |---------|---------|
143
+ | [`axios`](https://axios-http.com/) | HTTP requests to fetch web pages |
144
+ | [`cheerio`](https://cheerio.js.org/) | Fast, jQuery-like HTML parsing |
145
+ | [`turndown`](https://github.com/mixmark-io/turndown) | HTML to Markdown conversion |
146
+ | [`commander`](https://github.com/tj/commander.js/) | CLI argument and option parsing |
147
+
148
+ ---
149
+
150
+ ## How It Works
151
+
152
+ 1. **Fetch** — Downloads the HTML of the target URL via Axios
153
+ 2. **Parse** — Loads the HTML into Cheerio and extracts metadata (title, description) and the main content (`<main>` or `<body>`)
154
+ 3. **Convert** — Transforms the HTML into clean Markdown using Turndown
155
+ 4. **Format** — Structures the output into the `llms.txt` standard format
156
+ 5. **Save** — Writes the file(s) to your current working directory
157
+
158
+ ---
159
+
160
+ ## Contributing
161
+
162
+ Contributions are welcome! Feel free to open an issue or submit a pull request.
163
+
164
+ 1. Fork the repository
165
+ 2. Create your feature branch: `git checkout -b feature/my-feature`
166
+ 3. Commit your changes: `git commit -m 'Add my feature'`
167
+ 4. Push to the branch: `git push origin feature/my-feature`
168
+ 5. Open a Pull Request
169
+
170
+ ---
171
+
172
+ ## License
173
+
174
+ [MIT](LICENSE) © 2024 llms-generator-cli contributors
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "llms-generator-cli",
3
- "version": "1.0.0",
3
+ "version": "1.0.1",
4
4
  "description": "Generate llms.txt and advanced LLM contexts from a URL",
5
5
  "main": "index.js",
6
6
  "type": "module",
@@ -13,4 +13,4 @@
13
13
  "commander": "^11.1.0",
14
14
  "turndown": "^7.1.2"
15
15
  }
16
- }
16
+ }