gh-load-issue 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,24 @@
1
+ This is free and unencumbered software released into the public domain.
2
+
3
+ Anyone is free to copy, modify, publish, use, compile, sell, or
4
+ distribute this software, either in source code form or as a compiled
5
+ binary, for any purpose, commercial or non-commercial, and by any
6
+ means.
7
+
8
+ In jurisdictions that recognize copyright laws, the author or authors
9
+ of this software dedicate any and all copyright interest in the
10
+ software to the public domain. We make this dedication for the benefit
11
+ of the public at large and to the detriment of our heirs and
12
+ successors. We intend this dedication to be an overt act of
13
+ relinquishment in perpetuity of all present and future rights to this
14
+ software under copyright law.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
19
+ IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
20
+ OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
21
+ ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
22
+ OTHER DEALINGS IN THE SOFTWARE.
23
+
24
+ For more information, please refer to <https://unlicense.org>
package/README.md ADDED
@@ -0,0 +1,300 @@
1
+ # gh-download-issue
2
+
3
+ [![npm version](https://img.shields.io/npm/v/gh-download-issue)](https://www.npmjs.com/package/gh-download-issue)
4
+
5
+ A CLI tool to download GitHub issues and convert them to markdown - perfect for AI processing and offline analysis. Automatically downloads embedded images to prevent "Could not process image" errors when using with Claude Code CLI.
6
+
7
+ ## Features
8
+
9
+ - 📥 **Download Issues**: Fetch complete GitHub issues with all comments
10
+ - 📷 **Image Downloading**: Automatically download and validate embedded images
11
+ - 📝 **Markdown Export**: Convert issues to well-formatted markdown files
12
+ - 📊 **JSON Export**: Export structured data for programmatic use
13
+ - 🔐 **Smart Authentication**: Automatic GitHub CLI integration or token support
14
+ - ⚡ **Simple CLI**: Easy-to-use command-line interface
15
+ - 🎯 **Flexible Input**: Support for full URLs or short format (owner/repo#123)
16
+ - ✅ **Image Validation**: Validates downloaded images by checking magic bytes
17
+
18
+ ## Quick Start
19
+
20
+ ```bash
21
+ # Download issue using full URL
22
+ gh-download-issue https://github.com/owner/repo/issues/123
23
+
24
+ # Download issue using short format
25
+ gh-download-issue owner/repo#123
26
+
27
+ # Save to specific file
28
+ gh-download-issue owner/repo#123 -o my-issue.md
29
+
30
+ # Export as JSON
31
+ gh-download-issue owner/repo#123 --format json
32
+
33
+ # Skip image downloading
34
+ gh-download-issue owner/repo#123 --no-download-images
35
+
36
+ # Use specific GitHub token
37
+ gh-download-issue owner/repo#123 --token ghp_xxx
38
+ ```
39
+
40
+ ## Installation
41
+
42
+ ### Global Installation (Recommended)
43
+
44
+ Install globally for system-wide access:
45
+
46
+ ```bash
47
+ # Using bun
48
+ bun install -g gh-download-issue
49
+
50
+ # Using npm
51
+ npm install -g gh-download-issue
52
+
53
+ # After installation, use anywhere:
54
+ gh-download-issue --help
55
+ ```
56
+
57
+ ### Uninstall
58
+
59
+ Remove the global installation:
60
+
61
+ ```bash
62
+ # Using bun
63
+ bun uninstall -g gh-download-issue
64
+
65
+ # Using npm
66
+ npm uninstall -g gh-download-issue
67
+ ```
68
+
69
+ ### Local Installation
70
+
71
+ ```bash
72
+ # Clone the repository
73
+ git clone https://github.com/link-foundation/gh-download-issue.git
74
+ cd gh-download-issue
75
+
76
+ # Make the script executable
77
+ chmod +x gh-download-issue.mjs
78
+
79
+ # Run it
80
+ ./gh-download-issue.mjs --help
81
+ ```
82
+
83
+ ## Usage
84
+
85
+ ```
86
+ Usage: gh-download-issue <issue-url> [options]
87
+
88
+ Options:
89
+ --version Show version number
90
+ -t, --token GitHub personal access token (optional for public issues)
91
+ -o, --output Output directory or file path (default: current directory)
92
+ --download-images Download embedded images (default: true)
93
+ -f, --format Output format: markdown, json (default: markdown)
94
+ -v, --verbose Enable verbose logging
95
+ -h, --help Show help
96
+ ```
97
+
98
+ ## Image Handling
99
+
100
+ The tool automatically downloads and validates all images found in issues:
101
+
102
+ ### Supported Image Formats
103
+
104
+ - PNG, JPEG, GIF, WebP
105
+ - BMP, ICO, SVG
106
+
107
+ ### Image Features
108
+
109
+ - **Automatic Download**: Images in both markdown (`![alt](url)`) and HTML (`<img src>`) syntax are detected
110
+ - **Magic Bytes Validation**: Images are validated by content, not just file extension
111
+ - **GitHub Authentication**: Uses your GitHub token for private repository images
112
+ - **Redirect Handling**: Properly follows S3 signed URLs and other redirects
113
+ - **Error Handling**: Gracefully handles missing/expired URLs with warnings
114
+ - **Local References**: Markdown is updated to reference downloaded images
115
+
116
+ ### Output Structure
117
+
118
+ ```
119
+ issue-123.md # Issue body and comments in markdown
120
+ issue-123-images/ # Directory with downloaded images
121
+ image-1.png
122
+ image-2.jpg
123
+ issue-123.json # Optional JSON export (with --format json)
124
+ ```
125
+
126
+ ## Authentication
127
+
128
+ The tool supports multiple authentication methods for accessing private issues:
129
+
130
+ ### 1. GitHub CLI (Recommended)
131
+
132
+ If you have [GitHub CLI](https://cli.github.com/) installed and authenticated, the script will automatically use your credentials:
133
+
134
+ ```bash
135
+ # Authenticate with GitHub CLI (one-time setup)
136
+ gh auth login
137
+
138
+ # Script automatically detects and uses gh CLI authentication
139
+ gh-download-issue owner/repo#123 # Works with private issues!
140
+ ```
141
+
142
+ ### 2. Environment Variable
143
+
144
+ Set the `GITHUB_TOKEN` environment variable:
145
+
146
+ ```bash
147
+ export GITHUB_TOKEN=ghp_your_token_here
148
+ gh-download-issue owner/repo#123
149
+ ```
150
+
151
+ ### 3. Command Line Token
152
+
153
+ Pass the token directly with `--token`:
154
+
155
+ ```bash
156
+ gh-download-issue owner/repo#123 --token ghp_your_token_here
157
+ ```
158
+
159
+ ### Authentication Priority
160
+
161
+ The script uses this fallback chain:
162
+
163
+ 1. `--token` command line argument (highest priority)
164
+ 2. `GITHUB_TOKEN` environment variable
165
+ 3. GitHub CLI authentication (if `gh` is installed and authenticated)
166
+ 4. No authentication (public issues only)
167
+
168
+ ## Examples
169
+
170
+ ```bash
171
+ # Basic usage - download a public issue
172
+ gh-download-issue https://github.com/torvalds/linux/issues/123
173
+
174
+ # Use short format
175
+ gh-download-issue torvalds/linux#123
176
+
177
+ # Download private issue (using GitHub CLI auth)
178
+ gh-download-issue myorg/private-repo#456
179
+
180
+ # Save to specific location
181
+ gh-download-issue owner/repo#789 --output ./issues/issue-789.md
182
+
183
+ # Export as JSON for programmatic use
184
+ gh-download-issue owner/repo#123 --format json
185
+
186
+ # Verbose mode for debugging
187
+ gh-download-issue owner/repo#123 --verbose
188
+
189
+ # Skip image download (faster, text only)
190
+ gh-download-issue owner/repo#123 --no-download-images
191
+
192
+ # Use explicit token
193
+ gh-download-issue owner/repo#123 --token ghp_your_token_here
194
+ ```
195
+
196
+ ## Output Format
197
+
198
+ ### Markdown Output
199
+
200
+ The generated markdown file includes:
201
+
202
+ - **Issue Title** - As the main heading
203
+ - **Metadata** - Issue number, author, state, dates, labels, assignees, milestone
204
+ - **Description** - The issue body content with local image references
205
+ - **Comments** - All comments with author and timestamp
206
+
207
+ Example output structure:
208
+
209
+ ```markdown
210
+ # Issue Title
211
+
212
+ **Issue:** #123
213
+ **Author:** @username
214
+ **State:** open
215
+ **Created:** 1/1/2025, 12:00:00 PM
216
+ **Updated:** 1/2/2025, 3:30:00 PM
217
+ **Labels:** `bug`, `enhancement`
218
+
219
+ ---
220
+
221
+ ## Description
222
+
223
+ [Issue body content with local image references]
224
+
225
+ ![screenshot](issue-123-images/image-1.png)
226
+
227
+ ---
228
+
229
+ ## Comments (2)
230
+
231
+ ### Comment 1 by @user1
232
+
233
+ _Posted on 1/1/2025, 2:00:00 PM_
234
+
235
+ [Comment content here]
236
+
237
+ ---
238
+
239
+ ### Comment 2 by @user2
240
+
241
+ _Posted on 1/2/2025, 3:30:00 PM_
242
+
243
+ [Comment content here]
244
+ ```
245
+
246
+ ### JSON Output
247
+
248
+ The JSON format includes:
249
+
250
+ - Full issue data (title, body, state, labels, etc.)
251
+ - All comments with metadata
252
+ - Image download results (downloaded, failed, skipped)
253
+ - Download metadata (timestamp, tool version)
254
+
255
+ ## Requirements
256
+
257
+ - [Bun](https://bun.sh/) (>=1.2.0) or [Node.js](https://nodejs.org/) (>=20.0.0) runtime
258
+ - For private issues (optional):
259
+ - [GitHub CLI](https://cli.github.com/) (recommended) OR
260
+ - GitHub personal access token (via `--token` or `GITHUB_TOKEN` env var)
261
+
262
+ ## Testing
263
+
264
+ The project includes a test suite:
265
+
266
+ ```bash
267
+ # Run all tests
268
+ npm test
269
+
270
+ # Or run directly
271
+ cd tests
272
+ ./test-all.mjs
273
+ ```
274
+
275
+ ## Use Cases
276
+
277
+ - **AI Processing**: Download issues with images for AI analysis without "Could not process image" errors
278
+ - **Claude Code CLI**: Perfect companion for using issues with Claude Code
279
+ - **Offline Access**: Keep local copies of important issues for reference
280
+ - **Documentation**: Export issues as markdown for documentation purposes
281
+ - **Backup**: Archive issues before repository changes or migrations
282
+ - **Analysis**: Collect issues for trend analysis or reporting
283
+
284
+ ## Rate Limits
285
+
286
+ - **Unauthenticated**: 60 requests per hour (public issues only)
287
+ - **Authenticated**: 5,000 requests per hour (includes private issues)
288
+ - Authentication is automatically handled if GitHub CLI is set up
289
+
290
+ ## License
291
+
292
+ This project is released into the public domain under The Unlicense - see [LICENSE](LICENSE) file for details.
293
+
294
+ ## Contributing
295
+
296
+ Contributions are welcome! This is an MVP implementation focusing on core functionality.
297
+
298
+ ## Related Projects
299
+
300
+ - [gh-pull-all](https://github.com/link-foundation/gh-pull-all) - Sync all repositories from a GitHub organization or user
@@ -0,0 +1,883 @@
1
+ #!/usr/bin/env sh
2
+ ':'; // # ; exec "$(command -v bun || command -v node)" "$0" "$@"
3
+
4
+ // Import built-in Node.js modules
5
+ import path from 'path';
6
+ import { fileURLToPath } from 'url';
7
+ import https from 'https';
8
+ import http from 'http';
9
+
10
+ // Import npm dependencies
11
+ import { Octokit } from '@octokit/rest';
12
+ import fs from 'fs-extra';
13
+ import yargs from 'yargs';
14
+ import { hideBin } from 'yargs/helpers';
15
+
16
+ // Get __dirname equivalent for ES modules
17
+ const __filename = fileURLToPath(import.meta.url);
18
+ const __dirname = path.dirname(__filename);
19
+
20
+ // Get version from package.json or fallback
21
+ let version = '0.1.0'; // Fallback version
22
+
23
+ try {
24
+ const packagePath = path.join(__dirname, 'package.json');
25
+ if (await fs.pathExists(packagePath)) {
26
+ const packageJson = await fs.readJson(packagePath);
27
+ version = packageJson.version;
28
+ }
29
+ } catch (_error) {
30
+ // Use fallback version if package.json can't be read
31
+ }
32
+
33
+ // Colors for console output
34
+ const colors = {
35
+ green: '\x1b[32m',
36
+ yellow: '\x1b[33m',
37
+ blue: '\x1b[34m',
38
+ red: '\x1b[31m',
39
+ cyan: '\x1b[36m',
40
+ magenta: '\x1b[35m',
41
+ dim: '\x1b[2m',
42
+ bold: '\x1b[1m',
43
+ reset: '\x1b[0m',
44
+ };
45
+
46
+ // Verbose logging state
47
+ let verboseMode = false;
48
+
49
+ const log = (color, message) =>
50
+ console.log(`${colors[color]}${message}${colors.reset}`);
51
+
52
+ const logVerbose = (color, message) => {
53
+ if (verboseMode) {
54
+ console.log(`${colors[color]}${message}${colors.reset}`);
55
+ }
56
+ };
57
+
58
+ // Helper function to check if gh CLI is installed
59
+ async function isGhInstalled() {
60
+ try {
61
+ const { execSync } = await import('child_process');
62
+ execSync('gh --version', { stdio: 'pipe' });
63
+ return true;
64
+ } catch (_error) {
65
+ return false;
66
+ }
67
+ }
68
+
69
+ // Helper function to get GitHub token from gh CLI if available
70
+ async function getGhToken() {
71
+ try {
72
+ if (!(await isGhInstalled())) {
73
+ return null;
74
+ }
75
+
76
+ const { execSync } = await import('child_process');
77
+ const token = execSync('gh auth token', {
78
+ encoding: 'utf8',
79
+ stdio: 'pipe',
80
+ }).trim();
81
+ return token;
82
+ } catch (_error) {
83
+ return null;
84
+ }
85
+ }
86
+
87
+ // Parse GitHub issue URL to extract owner, repo, and issue number
88
+ function parseIssueUrl(url) {
89
+ // Support both full URLs and short formats like "owner/repo#123"
90
+ const fullUrlMatch = url.match(
91
+ /github\.com\/([^/]+)\/([^/]+)\/issues\/(\d+)/
92
+ );
93
+ if (fullUrlMatch) {
94
+ return {
95
+ owner: fullUrlMatch[1],
96
+ repo: fullUrlMatch[2],
97
+ issueNumber: parseInt(fullUrlMatch[3], 10),
98
+ };
99
+ }
100
+
101
+ const shortMatch = url.match(/^([^/]+)\/([^#]+)#(\d+)$/);
102
+ if (shortMatch) {
103
+ return {
104
+ owner: shortMatch[1],
105
+ repo: shortMatch[2],
106
+ issueNumber: parseInt(shortMatch[3], 10),
107
+ };
108
+ }
109
+
110
+ return null;
111
+ }
112
+
113
+ // Image magic bytes for validation (reference documentation)
114
+ // PNG: [0x89, 0x50, 0x4e, 0x47]
115
+ // JPEG: [0xff, 0xd8, 0xff]
116
+ // GIF: [0x47, 0x49, 0x46]
117
+ // WebP: [0x52, 0x49, 0x46, 0x46] (RIFF header) + WEBP at offset 8
118
+ // BMP: [0x42, 0x4d]
119
+ // ICO: [0x00, 0x00, 0x01, 0x00]
120
+ // SVG: starts with <?xml or <svg
121
+
122
+ // Validate image by checking magic bytes
123
+ function validateImageBytes(buffer) {
124
+ if (!buffer || buffer.length < 4) {
125
+ return { valid: false, type: null, reason: 'Buffer too small' };
126
+ }
127
+
128
+ const bytes = [...buffer.slice(0, 12)];
129
+
130
+ // Check for PNG
131
+ if (
132
+ bytes[0] === 0x89 &&
133
+ bytes[1] === 0x50 &&
134
+ bytes[2] === 0x4e &&
135
+ bytes[3] === 0x47
136
+ ) {
137
+ return { valid: true, type: 'png' };
138
+ }
139
+
140
+ // Check for JPEG
141
+ if (bytes[0] === 0xff && bytes[1] === 0xd8 && bytes[2] === 0xff) {
142
+ return { valid: true, type: 'jpeg' };
143
+ }
144
+
145
+ // Check for GIF
146
+ if (bytes[0] === 0x47 && bytes[1] === 0x49 && bytes[2] === 0x46) {
147
+ return { valid: true, type: 'gif' };
148
+ }
149
+
150
+ // Check for WebP (RIFF....WEBP)
151
+ if (
152
+ bytes[0] === 0x52 &&
153
+ bytes[1] === 0x49 &&
154
+ bytes[2] === 0x46 &&
155
+ bytes[3] === 0x46
156
+ ) {
157
+ if (buffer.length >= 12) {
158
+ const webpCheck = buffer.slice(8, 12).toString('ascii');
159
+ if (webpCheck === 'WEBP') {
160
+ return { valid: true, type: 'webp' };
161
+ }
162
+ }
163
+ }
164
+
165
+ // Check for BMP
166
+ if (bytes[0] === 0x42 && bytes[1] === 0x4d) {
167
+ return { valid: true, type: 'bmp' };
168
+ }
169
+
170
+ // Check for ICO
171
+ if (
172
+ bytes[0] === 0x00 &&
173
+ bytes[1] === 0x00 &&
174
+ bytes[2] === 0x01 &&
175
+ bytes[3] === 0x00
176
+ ) {
177
+ return { valid: true, type: 'ico' };
178
+ }
179
+
180
+ // Check for SVG (text-based, check for <?xml or <svg)
181
+ const textStart = buffer.slice(0, 100).toString('utf8').trim().toLowerCase();
182
+ if (textStart.startsWith('<?xml') || textStart.startsWith('<svg')) {
183
+ return { valid: true, type: 'svg' };
184
+ }
185
+
186
+ // Check if it looks like HTML (error page)
187
+ if (
188
+ textStart.includes('<!doctype html') ||
189
+ textStart.includes('<html') ||
190
+ textStart.includes('404')
191
+ ) {
192
+ return {
193
+ valid: false,
194
+ type: 'html',
195
+ reason: 'Received HTML instead of image (likely 404 page)',
196
+ };
197
+ }
198
+
199
+ return { valid: false, type: null, reason: 'Unknown file format' };
200
+ }
201
+
202
+ // Get file extension from image type
203
+ function getExtensionForType(type) {
204
+ const extensions = {
205
+ png: '.png',
206
+ jpeg: '.jpg',
207
+ gif: '.gif',
208
+ webp: '.webp',
209
+ bmp: '.bmp',
210
+ ico: '.ico',
211
+ svg: '.svg',
212
+ };
213
+ return extensions[type] || '.bin';
214
+ }
215
+
216
+ // Extract images from markdown content
217
+ function extractImagesFromMarkdown(content) {
218
+ const images = [];
219
+
220
+ if (!content) {
221
+ return images;
222
+ }
223
+
224
+ // Match markdown image syntax: ![alt](url) or ![alt](url "title")
225
+ const markdownImageRegex = /!\[([^\]]*)\]\(([^)\s]+)(?:\s+"[^"]*")?\)/g;
226
+ let match;
227
+
228
+ while ((match = markdownImageRegex.exec(content)) !== null) {
229
+ images.push({
230
+ original: match[0],
231
+ alt: match[1],
232
+ url: match[2],
233
+ type: 'markdown',
234
+ });
235
+ }
236
+
237
+ // Match HTML img tags: <img src="url" ... /> or <img src="url" ... >
238
+ const htmlImageRegex = /<img[^>]+src=["']([^"']+)["'][^>]*\/?>/gi;
239
+ while ((match = htmlImageRegex.exec(content)) !== null) {
240
+ images.push({
241
+ original: match[0],
242
+ alt: '',
243
+ url: match[1],
244
+ type: 'html',
245
+ });
246
+ }
247
+
248
+ return images;
249
+ }
250
+
251
+ // Download image with redirect and authentication support
252
+ function downloadImage(url, token, maxRedirects = 5) {
253
+ return new Promise((resolve, reject) => {
254
+ if (maxRedirects <= 0) {
255
+ reject(new Error('Too many redirects'));
256
+ return;
257
+ }
258
+
259
+ const parsedUrl = new URL(url);
260
+ const protocol = parsedUrl.protocol === 'https:' ? https : http;
261
+
262
+ const headers = {
263
+ 'User-Agent': 'gh-download-issue',
264
+ Accept: 'image/*,*/*',
265
+ };
266
+
267
+ // Add authorization for GitHub URLs
268
+ if (
269
+ token &&
270
+ (parsedUrl.hostname.includes('github.com') ||
271
+ parsedUrl.hostname.includes('githubusercontent.com') ||
272
+ parsedUrl.hostname.includes('github.githubassets.com'))
273
+ ) {
274
+ headers.Authorization = `Bearer ${token}`;
275
+ }
276
+
277
+ const options = {
278
+ hostname: parsedUrl.hostname,
279
+ port: parsedUrl.port,
280
+ path: parsedUrl.pathname + parsedUrl.search,
281
+ method: 'GET',
282
+ headers,
283
+ };
284
+
285
+ logVerbose('dim', ` Downloading from: ${url}`);
286
+
287
+ const req = protocol.request(options, (res) => {
288
+ // Handle redirects
289
+ if (
290
+ res.statusCode >= 300 &&
291
+ res.statusCode < 400 &&
292
+ res.headers.location
293
+ ) {
294
+ logVerbose('dim', ` Following redirect to: ${res.headers.location}`);
295
+ let redirectUrl = res.headers.location;
296
+
297
+ // Handle relative redirects
298
+ if (!redirectUrl.startsWith('http')) {
299
+ redirectUrl = new URL(redirectUrl, url).href;
300
+ }
301
+
302
+ downloadImage(redirectUrl, token, maxRedirects - 1)
303
+ .then(resolve)
304
+ .catch(reject);
305
+ return;
306
+ }
307
+
308
+ if (res.statusCode !== 200) {
309
+ reject(new Error(`HTTP ${res.statusCode}: ${res.statusMessage}`));
310
+ return;
311
+ }
312
+
313
+ const chunks = [];
314
+ res.on('data', (chunk) => chunks.push(chunk));
315
+ res.on('end', () => {
316
+ const buffer = Buffer.concat(chunks);
317
+ resolve(buffer);
318
+ });
319
+ res.on('error', reject);
320
+ });
321
+
322
+ req.on('error', reject);
323
+ req.setTimeout(30000, () => {
324
+ req.destroy();
325
+ reject(new Error('Request timeout'));
326
+ });
327
+ req.end();
328
+ });
329
+ }
330
+
331
+ // Download all images from content and save to directory
332
+ async function downloadImages(content, imageDir, token) {
333
+ const images = extractImagesFromMarkdown(content);
334
+ const imageMap = new Map(); // Original URL -> local path
335
+ const results = {
336
+ downloaded: [],
337
+ failed: [],
338
+ skipped: [],
339
+ };
340
+
341
+ if (images.length === 0) {
342
+ return { imageMap, results };
343
+ }
344
+
345
+ log('blue', `📷 Found ${images.length} image(s) to download...`);
346
+
347
+ // Create image directory if needed
348
+ await fs.ensureDir(imageDir);
349
+
350
+ let imageIndex = 0;
351
+ for (const image of images) {
352
+ imageIndex++;
353
+ const url = image.url;
354
+
355
+ // Skip if already processed (duplicate URL)
356
+ if (imageMap.has(url)) {
357
+ logVerbose('dim', ` Skipping duplicate: ${url}`);
358
+ results.skipped.push({ url, reason: 'duplicate' });
359
+ continue;
360
+ }
361
+
362
+ // Skip data URLs
363
+ if (url.startsWith('data:')) {
364
+ logVerbose('dim', ` Skipping data URL`);
365
+ results.skipped.push({ url, reason: 'data URL' });
366
+ continue;
367
+ }
368
+
369
+ // Skip relative URLs that don't start with http
370
+ if (!url.startsWith('http://') && !url.startsWith('https://')) {
371
+ logVerbose('yellow', ` Skipping non-HTTP URL: ${url}`);
372
+ results.skipped.push({ url, reason: 'non-HTTP URL' });
373
+ continue;
374
+ }
375
+
376
+ try {
377
+ logVerbose(
378
+ 'blue',
379
+ ` [${imageIndex}/${images.length}] Downloading: ${url.substring(0, 80)}...`
380
+ );
381
+
382
+ const buffer = await downloadImage(url, token);
383
+
384
+ // Validate the downloaded content
385
+ const validation = validateImageBytes(buffer);
386
+
387
+ if (!validation.valid) {
388
+ log(
389
+ 'yellow',
390
+ `⚠️ Invalid image (${validation.reason}): ${url.substring(0, 60)}...`
391
+ );
392
+ results.failed.push({ url, reason: validation.reason });
393
+ continue;
394
+ }
395
+
396
+ // Determine filename
397
+ const ext = getExtensionForType(validation.type);
398
+ const filename = `image-${imageIndex}${ext}`;
399
+ const localPath = path.join(imageDir, filename);
400
+
401
+ // Save the image
402
+ await fs.writeFile(localPath, buffer);
403
+
404
+ imageMap.set(url, {
405
+ localPath,
406
+ relativePath: path.join(path.basename(imageDir), filename),
407
+ type: validation.type,
408
+ size: buffer.length,
409
+ });
410
+
411
+ results.downloaded.push({
412
+ url,
413
+ localPath,
414
+ type: validation.type,
415
+ size: buffer.length,
416
+ });
417
+
418
+ logVerbose(
419
+ 'green',
420
+ ` ✓ Saved as ${filename} (${validation.type}, ${buffer.length} bytes)`
421
+ );
422
+ } catch (error) {
423
+ log('yellow', `⚠️ Failed to download image: ${error.message}`);
424
+ logVerbose('dim', ` URL: ${url}`);
425
+ results.failed.push({ url, reason: error.message });
426
+ }
427
+ }
428
+
429
+ // Summary
430
+ if (results.downloaded.length > 0) {
431
+ log('green', `✅ Downloaded ${results.downloaded.length} image(s)`);
432
+ }
433
+ if (results.failed.length > 0) {
434
+ log('yellow', `⚠️ Failed to download ${results.failed.length} image(s)`);
435
+ }
436
+
437
+ return { imageMap, results };
438
+ }
439
+
440
+ // Replace image URLs in content with local paths
441
+ function replaceImageUrls(content, imageMap) {
442
+ let updatedContent = content;
443
+
444
+ for (const [originalUrl, imageInfo] of imageMap) {
445
+ // Replace in markdown syntax
446
+ const markdownRegex = new RegExp(
447
+ `(!\\[[^\\]]*\\]\\()${escapeRegex(originalUrl)}((?:\\s+"[^"]*")?\\))`,
448
+ 'g'
449
+ );
450
+ updatedContent = updatedContent.replace(
451
+ markdownRegex,
452
+ `$1${imageInfo.relativePath}$2`
453
+ );
454
+
455
+ // Replace in HTML img tags
456
+ const htmlRegex = new RegExp(
457
+ `(<img[^>]+src=["'])${escapeRegex(originalUrl)}(["'])`,
458
+ 'gi'
459
+ );
460
+ updatedContent = updatedContent.replace(
461
+ htmlRegex,
462
+ `$1${imageInfo.relativePath}$2`
463
+ );
464
+ }
465
+
466
+ return updatedContent;
467
+ }
468
+
469
+ // Escape special regex characters
470
+ function escapeRegex(string) {
471
+ return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
472
+ }
473
+
474
+ // Fetch issue data from GitHub API
475
+ async function fetchIssue(owner, repo, issueNumber, token) {
476
+ try {
477
+ log('blue', `🔍 Fetching issue #${issueNumber} from ${owner}/${repo}...`);
478
+
479
+ const octokit = new Octokit({
480
+ auth: token,
481
+ baseUrl: 'https://api.github.com',
482
+ });
483
+
484
+ // Fetch the issue
485
+ const { data: issue } = await octokit.rest.issues.get({
486
+ owner,
487
+ repo,
488
+ issue_number: issueNumber,
489
+ });
490
+
491
+ // Fetch comments
492
+ const { data: comments } = await octokit.rest.issues.listComments({
493
+ owner,
494
+ repo,
495
+ issue_number: issueNumber,
496
+ });
497
+
498
+ log(
499
+ 'green',
500
+ `✅ Successfully fetched issue with ${comments.length} comments`
501
+ );
502
+
503
+ return { issue, comments };
504
+ } catch (error) {
505
+ if (error.status === 404) {
506
+ log('red', `❌ Issue #${issueNumber} not found in ${owner}/${repo}`);
507
+ } else if (error.status === 401) {
508
+ log(
509
+ 'red',
510
+ `❌ Authentication failed. Please provide a valid GitHub token`
511
+ );
512
+ } else {
513
+ log('red', `❌ Failed to fetch issue: ${error.message}`);
514
+ }
515
+ throw error;
516
+ }
517
+ }
518
+
519
+ // Convert issue to markdown format
520
+ function issueToMarkdown(issueData, imageMap = null) {
521
+ const { issue, comments } = issueData;
522
+ let markdown = '';
523
+
524
+ // Title
525
+ markdown += `# ${issue.title}\n\n`;
526
+
527
+ // Metadata
528
+ markdown += `**Issue:** [#${issue.number}](${issue.html_url}) \n`;
529
+ markdown += `**Author:** [@${issue.user.login}](${issue.user.html_url}) \n`;
530
+ markdown += `**State:** ${issue.state} \n`;
531
+ markdown += `**Created:** ${new Date(issue.created_at).toLocaleString()} \n`;
532
+ markdown += `**Updated:** ${new Date(issue.updated_at).toLocaleString()} \n`;
533
+
534
+ if (issue.labels && issue.labels.length > 0) {
535
+ const labels = issue.labels.map((label) => `\`${label.name}\``).join(', ');
536
+ markdown += `**Labels:** ${labels} \n`;
537
+ }
538
+
539
+ if (issue.assignees && issue.assignees.length > 0) {
540
+ const assignees = issue.assignees
541
+ .map((a) => `[@${a.login}](${a.html_url})`)
542
+ .join(', ');
543
+ markdown += `**Assignees:** ${assignees} \n`;
544
+ }
545
+
546
+ if (issue.milestone) {
547
+ markdown += `**Milestone:** [${issue.milestone.title}](${issue.milestone.html_url}) \n`;
548
+ }
549
+
550
+ markdown += '\n---\n\n';
551
+
552
+ // Body
553
+ if (issue.body) {
554
+ markdown += '## Description\n\n';
555
+ let body = issue.body;
556
+ if (imageMap && imageMap.size > 0) {
557
+ body = replaceImageUrls(body, imageMap);
558
+ }
559
+ markdown += body;
560
+ markdown += '\n\n';
561
+ }
562
+
563
+ // Comments
564
+ if (comments && comments.length > 0) {
565
+ markdown += '---\n\n';
566
+ markdown += `## Comments (${comments.length})\n\n`;
567
+
568
+ comments.forEach((comment, index) => {
569
+ markdown += `### Comment ${index + 1} by [@${comment.user.login}](${comment.user.html_url})\n\n`;
570
+ markdown += `*Posted on ${new Date(comment.created_at).toLocaleString()}*\n\n`;
571
+ let commentBody = comment.body;
572
+ if (imageMap && imageMap.size > 0) {
573
+ commentBody = replaceImageUrls(commentBody, imageMap);
574
+ }
575
+ markdown += commentBody;
576
+ markdown += '\n\n---\n\n';
577
+ });
578
+ }
579
+
580
+ return markdown;
581
+ }
582
+
583
+ // Convert issue to JSON format
584
+ function issueToJson(issueData, imageResults = null) {
585
+ const { issue, comments } = issueData;
586
+
587
+ return {
588
+ issue: {
589
+ number: issue.number,
590
+ title: issue.title,
591
+ state: issue.state,
592
+ html_url: issue.html_url,
593
+ author: {
594
+ login: issue.user.login,
595
+ html_url: issue.user.html_url,
596
+ },
597
+ created_at: issue.created_at,
598
+ updated_at: issue.updated_at,
599
+ labels: issue.labels.map((l) => ({
600
+ name: l.name,
601
+ color: l.color,
602
+ description: l.description,
603
+ })),
604
+ assignees: issue.assignees.map((a) => ({
605
+ login: a.login,
606
+ html_url: a.html_url,
607
+ })),
608
+ milestone: issue.milestone
609
+ ? {
610
+ title: issue.milestone.title,
611
+ html_url: issue.milestone.html_url,
612
+ }
613
+ : null,
614
+ body: issue.body,
615
+ },
616
+ comments: comments.map((comment) => ({
617
+ id: comment.id,
618
+ author: {
619
+ login: comment.user.login,
620
+ html_url: comment.user.html_url,
621
+ },
622
+ created_at: comment.created_at,
623
+ updated_at: comment.updated_at,
624
+ body: comment.body,
625
+ })),
626
+ images: imageResults || null,
627
+ metadata: {
628
+ downloaded_at: new Date().toISOString(),
629
+ tool_version: version,
630
+ },
631
+ };
632
+ }
633
+
634
+ // Configure CLI arguments
635
+ const scriptName = path.basename(process.argv[1]);
636
+
637
+ async function main() {
638
+ // Check for --help or --version before yargs parsing for faster response
639
+ const args = process.argv.slice(2);
640
+ if (args.includes('--help') || args.includes('-h')) {
641
+ console.log(`Usage: ${scriptName} <issue-url> [options]
642
+
643
+ Positionals:
644
+ issue GitHub issue URL or short format (owner/repo#123) [string]
645
+
646
+ Options:
647
+ --version Show version number [boolean]
648
+ -t, --token GitHub personal access token (optional for public
649
+ issues) [string]
650
+ -o, --output Output directory or file path (default: current
651
+ directory) [string]
652
+ --download-images Download embedded images (default: true) [boolean]
653
+ -f, --format Output format: markdown, json (default: markdown)
654
+ [string]
655
+ -v, --verbose Enable verbose logging [boolean]
656
+ -h, --help Show help [boolean]
657
+
658
+ Examples:
659
+ ${scriptName} https://github.com/owner/repo/issues/123 Download issue #123
660
+ ${scriptName} owner/repo#123 Download issue #123 using short format
661
+ ${scriptName} owner/repo#123 -o my-issue.md Save to specific file
662
+ ${scriptName} owner/repo#123 --token ghp_xxx Use specific GitHub token
663
+ ${scriptName} owner/repo#123 --format json Export as JSON
664
+ ${scriptName} owner/repo#123 --no-download-images Skip image download`);
665
+ process.exit(0);
666
+ }
667
+
668
+ if (args.includes('--version')) {
669
+ console.log(version);
670
+ process.exit(0);
671
+ }
672
+
673
+ // Create yargs instance with proper configuration
674
+ const yargsInstance = yargs(hideBin(process.argv))
675
+ .scriptName(scriptName)
676
+ .version(version)
677
+ .usage('Usage: $0 <issue-url> [options]')
678
+ .command(
679
+ '$0 [issue]',
680
+ 'Download a GitHub issue and convert it to markdown',
681
+ (yargs) => {
682
+ yargs.positional('issue', {
683
+ describe: 'GitHub issue URL or short format (owner/repo#123)',
684
+ type: 'string',
685
+ });
686
+ }
687
+ )
688
+ .option('token', {
689
+ alias: 't',
690
+ type: 'string',
691
+ describe: 'GitHub personal access token (optional for public issues)',
692
+ default: process.env.GITHUB_TOKEN,
693
+ })
694
+ .option('output', {
695
+ alias: 'o',
696
+ type: 'string',
697
+ describe: 'Output directory or file path (default: current directory)',
698
+ })
699
+ .option('download-images', {
700
+ type: 'boolean',
701
+ describe: 'Download embedded images (default: true)',
702
+ default: true,
703
+ })
704
+ .option('format', {
705
+ alias: 'f',
706
+ type: 'string',
707
+ describe: 'Output format: markdown, json (default: markdown)',
708
+ choices: ['markdown', 'json'],
709
+ default: 'markdown',
710
+ })
711
+ .option('verbose', {
712
+ alias: 'v',
713
+ type: 'boolean',
714
+ describe: 'Enable verbose logging',
715
+ default: false,
716
+ })
717
+ .help(false) // Disable yargs built-in help since we handle it manually
718
+ .version(false) // Disable yargs built-in version since we handle it manually
719
+ .example(
720
+ '$0 https://github.com/owner/repo/issues/123',
721
+ 'Download issue #123'
722
+ )
723
+ .example('$0 owner/repo#123', 'Download issue #123 using short format')
724
+ .example('$0 owner/repo#123 -o my-issue.md', 'Save to specific file')
725
+ .example('$0 owner/repo#123 --token ghp_xxx', 'Use specific GitHub token')
726
+ .example('$0 owner/repo#123 --format json', 'Export as JSON')
727
+ .example('$0 owner/repo#123 --no-download-images', 'Skip image download');
728
+
729
+ const argv = await yargsInstance.parseAsync();
730
+
731
+ const { issue: issueInput, output, format, verbose } = argv;
732
+ const downloadImages_flag = argv['download-images'];
733
+ let { token } = argv;
734
+
735
+ // Set verbose mode
736
+ verboseMode = verbose;
737
+
738
+ // Check if issue URL was provided
739
+ if (!issueInput) {
740
+ log('red', '❌ No issue URL provided');
741
+ log(
742
+ 'yellow',
743
+ ' Expected: https://github.com/owner/repo/issues/123 or owner/repo#123'
744
+ );
745
+ log('yellow', ' Run with --help for more information');
746
+ process.exit(1);
747
+ }
748
+
749
+ // Parse the issue URL
750
+ const parsed = parseIssueUrl(issueInput);
751
+ if (!parsed) {
752
+ log('red', '❌ Invalid issue URL or format');
753
+ log(
754
+ 'yellow',
755
+ ' Expected: https://github.com/owner/repo/issues/123 or owner/repo#123'
756
+ );
757
+ process.exit(1);
758
+ }
759
+
760
+ const { owner, repo, issueNumber } = parsed;
761
+
762
+ // If no token provided, try to get it from gh CLI
763
+ if (!token || token === undefined) {
764
+ const ghToken = await getGhToken();
765
+ if (ghToken) {
766
+ token = ghToken;
767
+ log('cyan', '🔑 Using GitHub token from gh CLI');
768
+ }
769
+ }
770
+
771
+ // Fetch the issue
772
+ let issueData;
773
+ try {
774
+ issueData = await fetchIssue(owner, repo, issueNumber, token);
775
+ } catch (_error) {
776
+ process.exit(1);
777
+ }
778
+
779
+ // Determine output paths
780
+ let outputDir = process.cwd();
781
+ let outputFilename;
782
+
783
+ if (output) {
784
+ // Check if output looks like a file path (has extension) or directory
785
+ const ext = path.extname(output);
786
+ if (ext === '.md' || ext === '.json') {
787
+ outputDir = path.dirname(output);
788
+ outputFilename = path.basename(output, ext);
789
+ } else if (ext) {
790
+ outputDir = path.dirname(output);
791
+ outputFilename = path.basename(output);
792
+ } else {
793
+ // Treat as directory
794
+ outputDir = output;
795
+ }
796
+ }
797
+
798
+ // Ensure output directory exists
799
+ await fs.ensureDir(outputDir);
800
+
801
+ // Default filename if not specified
802
+ if (!outputFilename) {
803
+ outputFilename = `issue-${issueNumber}`;
804
+ }
805
+
806
+ // Download images if enabled
807
+ let imageMap = null;
808
+ let imageResults = null;
809
+
810
+ if (downloadImages_flag) {
811
+ const imageDir = path.join(outputDir, `${outputFilename}-images`);
812
+
813
+ // Collect all content for image extraction
814
+ let allContent = issueData.issue.body || '';
815
+ for (const comment of issueData.comments) {
816
+ allContent += `\n${comment.body || ''}`;
817
+ }
818
+
819
+ const { imageMap: downloadedMap, results } = await downloadImages(
820
+ allContent,
821
+ imageDir,
822
+ token
823
+ );
824
+
825
+ imageMap = downloadedMap;
826
+ imageResults = results;
827
+
828
+ // Clean up empty image directory
829
+ if (results.downloaded.length === 0 && (await fs.pathExists(imageDir))) {
830
+ try {
831
+ const files = await fs.readdir(imageDir);
832
+ if (files.length === 0) {
833
+ await fs.rmdir(imageDir);
834
+ }
835
+ } catch (_error) {
836
+ // Ignore cleanup errors
837
+ }
838
+ }
839
+ }
840
+
841
+ // Generate output based on format
842
+ log('blue', `📝 Converting to ${format}...`);
843
+
844
+ if (format === 'json') {
845
+ const jsonOutput = issueToJson(issueData, imageResults);
846
+ const outputPath = path.join(outputDir, `${outputFilename}.json`);
847
+
848
+ try {
849
+ await fs.writeFile(
850
+ outputPath,
851
+ JSON.stringify(jsonOutput, null, 2),
852
+ 'utf8'
853
+ );
854
+ log('green', `✅ Issue saved to: ${outputPath}`);
855
+ } catch (error) {
856
+ log('red', `❌ Failed to write file: ${error.message}`);
857
+ process.exit(1);
858
+ }
859
+ } else {
860
+ // Markdown format
861
+ const markdown = issueToMarkdown(issueData, imageMap);
862
+ const outputPath = path.join(outputDir, `${outputFilename}.md`);
863
+
864
+ try {
865
+ await fs.writeFile(outputPath, markdown, 'utf8');
866
+ log('green', `✅ Issue saved to: ${outputPath}`);
867
+ } catch (error) {
868
+ log('red', `❌ Failed to write file: ${error.message}`);
869
+ process.exit(1);
870
+ }
871
+ }
872
+
873
+ // Summary
874
+ if (imageResults && imageResults.downloaded.length > 0) {
875
+ const imageDir = path.join(outputDir, `${outputFilename}-images`);
876
+ log('green', `📁 Images saved to: ${imageDir}`);
877
+ }
878
+ }
879
+
880
+ main().catch((error) => {
881
+ log('red', `💥 Script failed: ${error.message}`);
882
+ process.exit(1);
883
+ });
package/package.json ADDED
@@ -0,0 +1,77 @@
1
+ {
2
+ "name": "gh-load-issue",
3
+ "version": "0.3.0",
4
+ "description": "A CLI tool to download GitHub issues and convert them to markdown",
5
+ "type": "module",
6
+ "main": "gh-download-issue.mjs",
7
+ "bin": {
8
+ "gh-download-issue": "./gh-download-issue.mjs"
9
+ },
10
+ "scripts": {
11
+ "test": "node --test tests/test-all.mjs",
12
+ "lint": "eslint .",
13
+ "lint:fix": "eslint . --fix",
14
+ "format": "prettier --write .",
15
+ "format:check": "prettier --check .",
16
+ "check:file-size": "node scripts/check-file-size.mjs",
17
+ "check:duplication": "jscpd .",
18
+ "check": "npm run lint && npm run format:check && npm run check:file-size && npm run check:duplication",
19
+ "prepare": "husky || true",
20
+ "changeset": "changeset",
21
+ "changeset:version": "node scripts/changeset-version.mjs",
22
+ "changeset:publish": "changeset publish",
23
+ "changeset:status": "changeset status --since=origin/main"
24
+ },
25
+ "engines": {
26
+ "node": ">=20.0.0"
27
+ },
28
+ "dependencies": {
29
+ "@octokit/rest": "^22.0.0",
30
+ "fs-extra": "^11.3.0",
31
+ "yargs": "^17.7.2"
32
+ },
33
+ "devDependencies": {
34
+ "@changesets/cli": "^2.29.7",
35
+ "eslint": "^9.38.0",
36
+ "eslint-config-prettier": "^10.1.8",
37
+ "eslint-plugin-prettier": "^5.5.4",
38
+ "husky": "^9.1.7",
39
+ "jscpd": "^4.0.5",
40
+ "lint-staged": "^16.2.6",
41
+ "prettier": "^3.6.2",
42
+ "test-anywhere": "^0.8.48"
43
+ },
44
+ "lint-staged": {
45
+ "*.{js,mjs,cjs}": [
46
+ "eslint --fix --max-warnings 0",
47
+ "prettier --write",
48
+ "prettier --check"
49
+ ],
50
+ "*.md": [
51
+ "prettier --write",
52
+ "prettier --check"
53
+ ]
54
+ },
55
+ "keywords": [
56
+ "github",
57
+ "issue",
58
+ "markdown",
59
+ "cli",
60
+ "tool"
61
+ ],
62
+ "author": "Link.Foundation",
63
+ "license": "Unlicense",
64
+ "repository": {
65
+ "type": "git",
66
+ "url": "git+https://github.com/link-foundation/gh-download-issue.git"
67
+ },
68
+ "bugs": {
69
+ "url": "https://github.com/link-foundation/gh-download-issue/issues"
70
+ },
71
+ "homepage": "https://github.com/link-foundation/gh-download-issue#readme",
72
+ "files": [
73
+ "gh-download-issue.mjs",
74
+ "README.md",
75
+ "LICENSE"
76
+ ]
77
+ }