gh-load-issue 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +24 -0
- package/README.md +300 -0
- package/gh-download-issue.mjs +883 -0
- package/package.json +77 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
This is free and unencumbered software released into the public domain.
|
|
2
|
+
|
|
3
|
+
Anyone is free to copy, modify, publish, use, compile, sell, or
|
|
4
|
+
distribute this software, either in source code form or as a compiled
|
|
5
|
+
binary, for any purpose, commercial or non-commercial, and by any
|
|
6
|
+
means.
|
|
7
|
+
|
|
8
|
+
In jurisdictions that recognize copyright laws, the author or authors
|
|
9
|
+
of this software dedicate any and all copyright interest in the
|
|
10
|
+
software to the public domain. We make this dedication for the benefit
|
|
11
|
+
of the public at large and to the detriment of our heirs and
|
|
12
|
+
successors. We intend this dedication to be an overt act of
|
|
13
|
+
relinquishment in perpetuity of all present and future rights to this
|
|
14
|
+
software under copyright law.
|
|
15
|
+
|
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
|
19
|
+
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
|
|
20
|
+
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
|
|
21
|
+
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
|
22
|
+
OTHER DEALINGS IN THE SOFTWARE.
|
|
23
|
+
|
|
24
|
+
For more information, please refer to <https://unlicense.org>
|
package/README.md
ADDED
|
@@ -0,0 +1,300 @@
|
|
|
1
|
+
# gh-download-issue
|
|
2
|
+
|
|
3
|
+
[](https://www.npmjs.com/package/gh-download-issue)
|
|
4
|
+
|
|
5
|
+
A CLI tool to download GitHub issues and convert them to markdown - perfect for AI processing and offline analysis. Automatically downloads embedded images to prevent "Could not process image" errors when using with Claude Code CLI.
|
|
6
|
+
|
|
7
|
+
## Features
|
|
8
|
+
|
|
9
|
+
- 📥 **Download Issues**: Fetch complete GitHub issues with all comments
|
|
10
|
+
- 📷 **Image Downloading**: Automatically download and validate embedded images
|
|
11
|
+
- 📝 **Markdown Export**: Convert issues to well-formatted markdown files
|
|
12
|
+
- 📊 **JSON Export**: Export structured data for programmatic use
|
|
13
|
+
- 🔐 **Smart Authentication**: Automatic GitHub CLI integration or token support
|
|
14
|
+
- ⚡ **Simple CLI**: Easy-to-use command-line interface
|
|
15
|
+
- 🎯 **Flexible Input**: Support for full URLs or short format (owner/repo#123)
|
|
16
|
+
- ✅ **Image Validation**: Validates downloaded images by checking magic bytes
|
|
17
|
+
|
|
18
|
+
## Quick Start
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
# Download issue using full URL
|
|
22
|
+
gh-download-issue https://github.com/owner/repo/issues/123
|
|
23
|
+
|
|
24
|
+
# Download issue using short format
|
|
25
|
+
gh-download-issue owner/repo#123
|
|
26
|
+
|
|
27
|
+
# Save to specific file
|
|
28
|
+
gh-download-issue owner/repo#123 -o my-issue.md
|
|
29
|
+
|
|
30
|
+
# Export as JSON
|
|
31
|
+
gh-download-issue owner/repo#123 --format json
|
|
32
|
+
|
|
33
|
+
# Skip image downloading
|
|
34
|
+
gh-download-issue owner/repo#123 --no-download-images
|
|
35
|
+
|
|
36
|
+
# Use specific GitHub token
|
|
37
|
+
gh-download-issue owner/repo#123 --token ghp_xxx
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## Installation
|
|
41
|
+
|
|
42
|
+
### Global Installation (Recommended)
|
|
43
|
+
|
|
44
|
+
Install globally for system-wide access:
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
# Using bun
|
|
48
|
+
bun install -g gh-download-issue
|
|
49
|
+
|
|
50
|
+
# Using npm
|
|
51
|
+
npm install -g gh-download-issue
|
|
52
|
+
|
|
53
|
+
# After installation, use anywhere:
|
|
54
|
+
gh-download-issue --help
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
### Uninstall
|
|
58
|
+
|
|
59
|
+
Remove the global installation:
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
# Using bun
|
|
63
|
+
bun uninstall -g gh-download-issue
|
|
64
|
+
|
|
65
|
+
# Using npm
|
|
66
|
+
npm uninstall -g gh-download-issue
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### Local Installation
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
# Clone the repository
|
|
73
|
+
git clone https://github.com/link-foundation/gh-download-issue.git
|
|
74
|
+
cd gh-download-issue
|
|
75
|
+
|
|
76
|
+
# Make the script executable
|
|
77
|
+
chmod +x gh-download-issue.mjs
|
|
78
|
+
|
|
79
|
+
# Run it
|
|
80
|
+
./gh-download-issue.mjs --help
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
## Usage
|
|
84
|
+
|
|
85
|
+
```
|
|
86
|
+
Usage: gh-download-issue <issue-url> [options]
|
|
87
|
+
|
|
88
|
+
Options:
|
|
89
|
+
--version Show version number
|
|
90
|
+
-t, --token GitHub personal access token (optional for public issues)
|
|
91
|
+
-o, --output Output directory or file path (default: current directory)
|
|
92
|
+
--download-images Download embedded images (default: true)
|
|
93
|
+
-f, --format Output format: markdown, json (default: markdown)
|
|
94
|
+
-v, --verbose Enable verbose logging
|
|
95
|
+
-h, --help Show help
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
## Image Handling
|
|
99
|
+
|
|
100
|
+
The tool automatically downloads and validates all images found in issues:
|
|
101
|
+
|
|
102
|
+
### Supported Image Formats
|
|
103
|
+
|
|
104
|
+
- PNG, JPEG, GIF, WebP
|
|
105
|
+
- BMP, ICO, SVG
|
|
106
|
+
|
|
107
|
+
### Image Features
|
|
108
|
+
|
|
109
|
+
- **Automatic Download**: Images in both markdown (``) and HTML (`<img src>`) syntax are detected
|
|
110
|
+
- **Magic Bytes Validation**: Images are validated by content, not just file extension
|
|
111
|
+
- **GitHub Authentication**: Uses your GitHub token for private repository images
|
|
112
|
+
- **Redirect Handling**: Properly follows S3 signed URLs and other redirects
|
|
113
|
+
- **Error Handling**: Gracefully handles missing/expired URLs with warnings
|
|
114
|
+
- **Local References**: Markdown is updated to reference downloaded images
|
|
115
|
+
|
|
116
|
+
### Output Structure
|
|
117
|
+
|
|
118
|
+
```
|
|
119
|
+
issue-123.md # Issue body and comments in markdown
|
|
120
|
+
issue-123-images/ # Directory with downloaded images
|
|
121
|
+
image-1.png
|
|
122
|
+
image-2.jpg
|
|
123
|
+
issue-123.json # Optional JSON export (with --format json)
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
## Authentication
|
|
127
|
+
|
|
128
|
+
The tool supports multiple authentication methods for accessing private issues:
|
|
129
|
+
|
|
130
|
+
### 1. GitHub CLI (Recommended)
|
|
131
|
+
|
|
132
|
+
If you have [GitHub CLI](https://cli.github.com/) installed and authenticated, the script will automatically use your credentials:
|
|
133
|
+
|
|
134
|
+
```bash
|
|
135
|
+
# Authenticate with GitHub CLI (one-time setup)
|
|
136
|
+
gh auth login
|
|
137
|
+
|
|
138
|
+
# Script automatically detects and uses gh CLI authentication
|
|
139
|
+
gh-download-issue owner/repo#123 # Works with private issues!
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
### 2. Environment Variable
|
|
143
|
+
|
|
144
|
+
Set the `GITHUB_TOKEN` environment variable:
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
export GITHUB_TOKEN=ghp_your_token_here
|
|
148
|
+
gh-download-issue owner/repo#123
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
### 3. Command Line Token
|
|
152
|
+
|
|
153
|
+
Pass the token directly with `--token`:
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
gh-download-issue owner/repo#123 --token ghp_your_token_here
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
### Authentication Priority
|
|
160
|
+
|
|
161
|
+
The script uses this fallback chain:
|
|
162
|
+
|
|
163
|
+
1. `--token` command line argument (highest priority)
|
|
164
|
+
2. `GITHUB_TOKEN` environment variable
|
|
165
|
+
3. GitHub CLI authentication (if `gh` is installed and authenticated)
|
|
166
|
+
4. No authentication (public issues only)
|
|
167
|
+
|
|
168
|
+
## Examples
|
|
169
|
+
|
|
170
|
+
```bash
|
|
171
|
+
# Basic usage - download a public issue
|
|
172
|
+
gh-download-issue https://github.com/torvalds/linux/issues/123
|
|
173
|
+
|
|
174
|
+
# Use short format
|
|
175
|
+
gh-download-issue torvalds/linux#123
|
|
176
|
+
|
|
177
|
+
# Download private issue (using GitHub CLI auth)
|
|
178
|
+
gh-download-issue myorg/private-repo#456
|
|
179
|
+
|
|
180
|
+
# Save to specific location
|
|
181
|
+
gh-download-issue owner/repo#789 --output ./issues/issue-789.md
|
|
182
|
+
|
|
183
|
+
# Export as JSON for programmatic use
|
|
184
|
+
gh-download-issue owner/repo#123 --format json
|
|
185
|
+
|
|
186
|
+
# Verbose mode for debugging
|
|
187
|
+
gh-download-issue owner/repo#123 --verbose
|
|
188
|
+
|
|
189
|
+
# Skip image download (faster, text only)
|
|
190
|
+
gh-download-issue owner/repo#123 --no-download-images
|
|
191
|
+
|
|
192
|
+
# Use explicit token
|
|
193
|
+
gh-download-issue owner/repo#123 --token ghp_your_token_here
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
## Output Format
|
|
197
|
+
|
|
198
|
+
### Markdown Output
|
|
199
|
+
|
|
200
|
+
The generated markdown file includes:
|
|
201
|
+
|
|
202
|
+
- **Issue Title** - As the main heading
|
|
203
|
+
- **Metadata** - Issue number, author, state, dates, labels, assignees, milestone
|
|
204
|
+
- **Description** - The issue body content with local image references
|
|
205
|
+
- **Comments** - All comments with author and timestamp
|
|
206
|
+
|
|
207
|
+
Example output structure:
|
|
208
|
+
|
|
209
|
+
```markdown
|
|
210
|
+
# Issue Title
|
|
211
|
+
|
|
212
|
+
**Issue:** #123
|
|
213
|
+
**Author:** @username
|
|
214
|
+
**State:** open
|
|
215
|
+
**Created:** 1/1/2025, 12:00:00 PM
|
|
216
|
+
**Updated:** 1/2/2025, 3:30:00 PM
|
|
217
|
+
**Labels:** `bug`, `enhancement`
|
|
218
|
+
|
|
219
|
+
---
|
|
220
|
+
|
|
221
|
+
## Description
|
|
222
|
+
|
|
223
|
+
[Issue body content with local image references]
|
|
224
|
+
|
|
225
|
+

|
|
226
|
+
|
|
227
|
+
---
|
|
228
|
+
|
|
229
|
+
## Comments (2)
|
|
230
|
+
|
|
231
|
+
### Comment 1 by @user1
|
|
232
|
+
|
|
233
|
+
_Posted on 1/1/2025, 2:00:00 PM_
|
|
234
|
+
|
|
235
|
+
[Comment content here]
|
|
236
|
+
|
|
237
|
+
---
|
|
238
|
+
|
|
239
|
+
### Comment 2 by @user2
|
|
240
|
+
|
|
241
|
+
_Posted on 1/2/2025, 3:30:00 PM_
|
|
242
|
+
|
|
243
|
+
[Comment content here]
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
### JSON Output
|
|
247
|
+
|
|
248
|
+
The JSON format includes:
|
|
249
|
+
|
|
250
|
+
- Full issue data (title, body, state, labels, etc.)
|
|
251
|
+
- All comments with metadata
|
|
252
|
+
- Image download results (downloaded, failed, skipped)
|
|
253
|
+
- Download metadata (timestamp, tool version)
|
|
254
|
+
|
|
255
|
+
## Requirements
|
|
256
|
+
|
|
257
|
+
- [Bun](https://bun.sh/) (>=1.2.0) or [Node.js](https://nodejs.org/) (>=20.0.0) runtime
|
|
258
|
+
- For private issues (optional):
|
|
259
|
+
- [GitHub CLI](https://cli.github.com/) (recommended) OR
|
|
260
|
+
- GitHub personal access token (via `--token` or `GITHUB_TOKEN` env var)
|
|
261
|
+
|
|
262
|
+
## Testing
|
|
263
|
+
|
|
264
|
+
The project includes a test suite:
|
|
265
|
+
|
|
266
|
+
```bash
|
|
267
|
+
# Run all tests
|
|
268
|
+
npm test
|
|
269
|
+
|
|
270
|
+
# Or run directly
|
|
271
|
+
cd tests
|
|
272
|
+
./test-all.mjs
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
## Use Cases
|
|
276
|
+
|
|
277
|
+
- **AI Processing**: Download issues with images for AI analysis without "Could not process image" errors
|
|
278
|
+
- **Claude Code CLI**: Perfect companion for using issues with Claude Code
|
|
279
|
+
- **Offline Access**: Keep local copies of important issues for reference
|
|
280
|
+
- **Documentation**: Export issues as markdown for documentation purposes
|
|
281
|
+
- **Backup**: Archive issues before repository changes or migrations
|
|
282
|
+
- **Analysis**: Collect issues for trend analysis or reporting
|
|
283
|
+
|
|
284
|
+
## Rate Limits
|
|
285
|
+
|
|
286
|
+
- **Unauthenticated**: 60 requests per hour (public issues only)
|
|
287
|
+
- **Authenticated**: 5,000 requests per hour (includes private issues)
|
|
288
|
+
- Authentication is automatically handled if GitHub CLI is set up
|
|
289
|
+
|
|
290
|
+
## License
|
|
291
|
+
|
|
292
|
+
This project is released into the public domain under The Unlicense - see [LICENSE](LICENSE) file for details.
|
|
293
|
+
|
|
294
|
+
## Contributing
|
|
295
|
+
|
|
296
|
+
Contributions are welcome! This is an MVP implementation focusing on core functionality.
|
|
297
|
+
|
|
298
|
+
## Related Projects
|
|
299
|
+
|
|
300
|
+
- [gh-pull-all](https://github.com/link-foundation/gh-pull-all) - Sync all repositories from a GitHub organization or user
|
|
@@ -0,0 +1,883 @@
|
|
|
1
|
+
#!/usr/bin/env sh
|
|
2
|
+
':'; // # ; exec "$(command -v bun || command -v node)" "$0" "$@"
|
|
3
|
+
|
|
4
|
+
// Import built-in Node.js modules
|
|
5
|
+
import path from 'path';
|
|
6
|
+
import { fileURLToPath } from 'url';
|
|
7
|
+
import https from 'https';
|
|
8
|
+
import http from 'http';
|
|
9
|
+
|
|
10
|
+
// Import npm dependencies
|
|
11
|
+
import { Octokit } from '@octokit/rest';
|
|
12
|
+
import fs from 'fs-extra';
|
|
13
|
+
import yargs from 'yargs';
|
|
14
|
+
import { hideBin } from 'yargs/helpers';
|
|
15
|
+
|
|
16
|
+
// Get __dirname equivalent for ES modules
|
|
17
|
+
const __filename = fileURLToPath(import.meta.url);
|
|
18
|
+
const __dirname = path.dirname(__filename);
|
|
19
|
+
|
|
20
|
+
// Get version from package.json or fallback
|
|
21
|
+
let version = '0.1.0'; // Fallback version
|
|
22
|
+
|
|
23
|
+
try {
|
|
24
|
+
const packagePath = path.join(__dirname, 'package.json');
|
|
25
|
+
if (await fs.pathExists(packagePath)) {
|
|
26
|
+
const packageJson = await fs.readJson(packagePath);
|
|
27
|
+
version = packageJson.version;
|
|
28
|
+
}
|
|
29
|
+
} catch (_error) {
|
|
30
|
+
// Use fallback version if package.json can't be read
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
// Colors for console output
|
|
34
|
+
const colors = {
|
|
35
|
+
green: '\x1b[32m',
|
|
36
|
+
yellow: '\x1b[33m',
|
|
37
|
+
blue: '\x1b[34m',
|
|
38
|
+
red: '\x1b[31m',
|
|
39
|
+
cyan: '\x1b[36m',
|
|
40
|
+
magenta: '\x1b[35m',
|
|
41
|
+
dim: '\x1b[2m',
|
|
42
|
+
bold: '\x1b[1m',
|
|
43
|
+
reset: '\x1b[0m',
|
|
44
|
+
};
|
|
45
|
+
|
|
46
|
+
// Verbose logging state
|
|
47
|
+
let verboseMode = false;
|
|
48
|
+
|
|
49
|
+
const log = (color, message) =>
|
|
50
|
+
console.log(`${colors[color]}${message}${colors.reset}`);
|
|
51
|
+
|
|
52
|
+
const logVerbose = (color, message) => {
|
|
53
|
+
if (verboseMode) {
|
|
54
|
+
console.log(`${colors[color]}${message}${colors.reset}`);
|
|
55
|
+
}
|
|
56
|
+
};
|
|
57
|
+
|
|
58
|
+
// Helper function to check if gh CLI is installed
|
|
59
|
+
async function isGhInstalled() {
|
|
60
|
+
try {
|
|
61
|
+
const { execSync } = await import('child_process');
|
|
62
|
+
execSync('gh --version', { stdio: 'pipe' });
|
|
63
|
+
return true;
|
|
64
|
+
} catch (_error) {
|
|
65
|
+
return false;
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
// Helper function to get GitHub token from gh CLI if available
|
|
70
|
+
async function getGhToken() {
|
|
71
|
+
try {
|
|
72
|
+
if (!(await isGhInstalled())) {
|
|
73
|
+
return null;
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
const { execSync } = await import('child_process');
|
|
77
|
+
const token = execSync('gh auth token', {
|
|
78
|
+
encoding: 'utf8',
|
|
79
|
+
stdio: 'pipe',
|
|
80
|
+
}).trim();
|
|
81
|
+
return token;
|
|
82
|
+
} catch (_error) {
|
|
83
|
+
return null;
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
// Parse GitHub issue URL to extract owner, repo, and issue number
|
|
88
|
+
function parseIssueUrl(url) {
|
|
89
|
+
// Support both full URLs and short formats like "owner/repo#123"
|
|
90
|
+
const fullUrlMatch = url.match(
|
|
91
|
+
/github\.com\/([^/]+)\/([^/]+)\/issues\/(\d+)/
|
|
92
|
+
);
|
|
93
|
+
if (fullUrlMatch) {
|
|
94
|
+
return {
|
|
95
|
+
owner: fullUrlMatch[1],
|
|
96
|
+
repo: fullUrlMatch[2],
|
|
97
|
+
issueNumber: parseInt(fullUrlMatch[3], 10),
|
|
98
|
+
};
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
const shortMatch = url.match(/^([^/]+)\/([^#]+)#(\d+)$/);
|
|
102
|
+
if (shortMatch) {
|
|
103
|
+
return {
|
|
104
|
+
owner: shortMatch[1],
|
|
105
|
+
repo: shortMatch[2],
|
|
106
|
+
issueNumber: parseInt(shortMatch[3], 10),
|
|
107
|
+
};
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
return null;
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
// Image magic bytes for validation (reference documentation)
|
|
114
|
+
// PNG: [0x89, 0x50, 0x4e, 0x47]
|
|
115
|
+
// JPEG: [0xff, 0xd8, 0xff]
|
|
116
|
+
// GIF: [0x47, 0x49, 0x46]
|
|
117
|
+
// WebP: [0x52, 0x49, 0x46, 0x46] (RIFF header) + WEBP at offset 8
|
|
118
|
+
// BMP: [0x42, 0x4d]
|
|
119
|
+
// ICO: [0x00, 0x00, 0x01, 0x00]
|
|
120
|
+
// SVG: starts with <?xml or <svg
|
|
121
|
+
|
|
122
|
+
// Validate image by checking magic bytes
|
|
123
|
+
function validateImageBytes(buffer) {
|
|
124
|
+
if (!buffer || buffer.length < 4) {
|
|
125
|
+
return { valid: false, type: null, reason: 'Buffer too small' };
|
|
126
|
+
}
|
|
127
|
+
|
|
128
|
+
const bytes = [...buffer.slice(0, 12)];
|
|
129
|
+
|
|
130
|
+
// Check for PNG
|
|
131
|
+
if (
|
|
132
|
+
bytes[0] === 0x89 &&
|
|
133
|
+
bytes[1] === 0x50 &&
|
|
134
|
+
bytes[2] === 0x4e &&
|
|
135
|
+
bytes[3] === 0x47
|
|
136
|
+
) {
|
|
137
|
+
return { valid: true, type: 'png' };
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
// Check for JPEG
|
|
141
|
+
if (bytes[0] === 0xff && bytes[1] === 0xd8 && bytes[2] === 0xff) {
|
|
142
|
+
return { valid: true, type: 'jpeg' };
|
|
143
|
+
}
|
|
144
|
+
|
|
145
|
+
// Check for GIF
|
|
146
|
+
if (bytes[0] === 0x47 && bytes[1] === 0x49 && bytes[2] === 0x46) {
|
|
147
|
+
return { valid: true, type: 'gif' };
|
|
148
|
+
}
|
|
149
|
+
|
|
150
|
+
// Check for WebP (RIFF....WEBP)
|
|
151
|
+
if (
|
|
152
|
+
bytes[0] === 0x52 &&
|
|
153
|
+
bytes[1] === 0x49 &&
|
|
154
|
+
bytes[2] === 0x46 &&
|
|
155
|
+
bytes[3] === 0x46
|
|
156
|
+
) {
|
|
157
|
+
if (buffer.length >= 12) {
|
|
158
|
+
const webpCheck = buffer.slice(8, 12).toString('ascii');
|
|
159
|
+
if (webpCheck === 'WEBP') {
|
|
160
|
+
return { valid: true, type: 'webp' };
|
|
161
|
+
}
|
|
162
|
+
}
|
|
163
|
+
}
|
|
164
|
+
|
|
165
|
+
// Check for BMP
|
|
166
|
+
if (bytes[0] === 0x42 && bytes[1] === 0x4d) {
|
|
167
|
+
return { valid: true, type: 'bmp' };
|
|
168
|
+
}
|
|
169
|
+
|
|
170
|
+
// Check for ICO
|
|
171
|
+
if (
|
|
172
|
+
bytes[0] === 0x00 &&
|
|
173
|
+
bytes[1] === 0x00 &&
|
|
174
|
+
bytes[2] === 0x01 &&
|
|
175
|
+
bytes[3] === 0x00
|
|
176
|
+
) {
|
|
177
|
+
return { valid: true, type: 'ico' };
|
|
178
|
+
}
|
|
179
|
+
|
|
180
|
+
// Check for SVG (text-based, check for <?xml or <svg)
|
|
181
|
+
const textStart = buffer.slice(0, 100).toString('utf8').trim().toLowerCase();
|
|
182
|
+
if (textStart.startsWith('<?xml') || textStart.startsWith('<svg')) {
|
|
183
|
+
return { valid: true, type: 'svg' };
|
|
184
|
+
}
|
|
185
|
+
|
|
186
|
+
// Check if it looks like HTML (error page)
|
|
187
|
+
if (
|
|
188
|
+
textStart.includes('<!doctype html') ||
|
|
189
|
+
textStart.includes('<html') ||
|
|
190
|
+
textStart.includes('404')
|
|
191
|
+
) {
|
|
192
|
+
return {
|
|
193
|
+
valid: false,
|
|
194
|
+
type: 'html',
|
|
195
|
+
reason: 'Received HTML instead of image (likely 404 page)',
|
|
196
|
+
};
|
|
197
|
+
}
|
|
198
|
+
|
|
199
|
+
return { valid: false, type: null, reason: 'Unknown file format' };
|
|
200
|
+
}
|
|
201
|
+
|
|
202
|
+
// Get file extension from image type
|
|
203
|
+
function getExtensionForType(type) {
|
|
204
|
+
const extensions = {
|
|
205
|
+
png: '.png',
|
|
206
|
+
jpeg: '.jpg',
|
|
207
|
+
gif: '.gif',
|
|
208
|
+
webp: '.webp',
|
|
209
|
+
bmp: '.bmp',
|
|
210
|
+
ico: '.ico',
|
|
211
|
+
svg: '.svg',
|
|
212
|
+
};
|
|
213
|
+
return extensions[type] || '.bin';
|
|
214
|
+
}
|
|
215
|
+
|
|
216
|
+
// Extract images from markdown content
|
|
217
|
+
function extractImagesFromMarkdown(content) {
|
|
218
|
+
const images = [];
|
|
219
|
+
|
|
220
|
+
if (!content) {
|
|
221
|
+
return images;
|
|
222
|
+
}
|
|
223
|
+
|
|
224
|
+
// Match markdown image syntax:  or 
|
|
225
|
+
const markdownImageRegex = /!\[([^\]]*)\]\(([^)\s]+)(?:\s+"[^"]*")?\)/g;
|
|
226
|
+
let match;
|
|
227
|
+
|
|
228
|
+
while ((match = markdownImageRegex.exec(content)) !== null) {
|
|
229
|
+
images.push({
|
|
230
|
+
original: match[0],
|
|
231
|
+
alt: match[1],
|
|
232
|
+
url: match[2],
|
|
233
|
+
type: 'markdown',
|
|
234
|
+
});
|
|
235
|
+
}
|
|
236
|
+
|
|
237
|
+
// Match HTML img tags: <img src="url" ... /> or <img src="url" ... >
|
|
238
|
+
const htmlImageRegex = /<img[^>]+src=["']([^"']+)["'][^>]*\/?>/gi;
|
|
239
|
+
while ((match = htmlImageRegex.exec(content)) !== null) {
|
|
240
|
+
images.push({
|
|
241
|
+
original: match[0],
|
|
242
|
+
alt: '',
|
|
243
|
+
url: match[1],
|
|
244
|
+
type: 'html',
|
|
245
|
+
});
|
|
246
|
+
}
|
|
247
|
+
|
|
248
|
+
return images;
|
|
249
|
+
}
|
|
250
|
+
|
|
251
|
+
// Download image with redirect and authentication support
|
|
252
|
+
function downloadImage(url, token, maxRedirects = 5) {
|
|
253
|
+
return new Promise((resolve, reject) => {
|
|
254
|
+
if (maxRedirects <= 0) {
|
|
255
|
+
reject(new Error('Too many redirects'));
|
|
256
|
+
return;
|
|
257
|
+
}
|
|
258
|
+
|
|
259
|
+
const parsedUrl = new URL(url);
|
|
260
|
+
const protocol = parsedUrl.protocol === 'https:' ? https : http;
|
|
261
|
+
|
|
262
|
+
const headers = {
|
|
263
|
+
'User-Agent': 'gh-download-issue',
|
|
264
|
+
Accept: 'image/*,*/*',
|
|
265
|
+
};
|
|
266
|
+
|
|
267
|
+
// Add authorization for GitHub URLs
|
|
268
|
+
if (
|
|
269
|
+
token &&
|
|
270
|
+
(parsedUrl.hostname.includes('github.com') ||
|
|
271
|
+
parsedUrl.hostname.includes('githubusercontent.com') ||
|
|
272
|
+
parsedUrl.hostname.includes('github.githubassets.com'))
|
|
273
|
+
) {
|
|
274
|
+
headers.Authorization = `Bearer ${token}`;
|
|
275
|
+
}
|
|
276
|
+
|
|
277
|
+
const options = {
|
|
278
|
+
hostname: parsedUrl.hostname,
|
|
279
|
+
port: parsedUrl.port,
|
|
280
|
+
path: parsedUrl.pathname + parsedUrl.search,
|
|
281
|
+
method: 'GET',
|
|
282
|
+
headers,
|
|
283
|
+
};
|
|
284
|
+
|
|
285
|
+
logVerbose('dim', ` Downloading from: ${url}`);
|
|
286
|
+
|
|
287
|
+
const req = protocol.request(options, (res) => {
|
|
288
|
+
// Handle redirects
|
|
289
|
+
if (
|
|
290
|
+
res.statusCode >= 300 &&
|
|
291
|
+
res.statusCode < 400 &&
|
|
292
|
+
res.headers.location
|
|
293
|
+
) {
|
|
294
|
+
logVerbose('dim', ` Following redirect to: ${res.headers.location}`);
|
|
295
|
+
let redirectUrl = res.headers.location;
|
|
296
|
+
|
|
297
|
+
// Handle relative redirects
|
|
298
|
+
if (!redirectUrl.startsWith('http')) {
|
|
299
|
+
redirectUrl = new URL(redirectUrl, url).href;
|
|
300
|
+
}
|
|
301
|
+
|
|
302
|
+
downloadImage(redirectUrl, token, maxRedirects - 1)
|
|
303
|
+
.then(resolve)
|
|
304
|
+
.catch(reject);
|
|
305
|
+
return;
|
|
306
|
+
}
|
|
307
|
+
|
|
308
|
+
if (res.statusCode !== 200) {
|
|
309
|
+
reject(new Error(`HTTP ${res.statusCode}: ${res.statusMessage}`));
|
|
310
|
+
return;
|
|
311
|
+
}
|
|
312
|
+
|
|
313
|
+
const chunks = [];
|
|
314
|
+
res.on('data', (chunk) => chunks.push(chunk));
|
|
315
|
+
res.on('end', () => {
|
|
316
|
+
const buffer = Buffer.concat(chunks);
|
|
317
|
+
resolve(buffer);
|
|
318
|
+
});
|
|
319
|
+
res.on('error', reject);
|
|
320
|
+
});
|
|
321
|
+
|
|
322
|
+
req.on('error', reject);
|
|
323
|
+
req.setTimeout(30000, () => {
|
|
324
|
+
req.destroy();
|
|
325
|
+
reject(new Error('Request timeout'));
|
|
326
|
+
});
|
|
327
|
+
req.end();
|
|
328
|
+
});
|
|
329
|
+
}
|
|
330
|
+
|
|
331
|
+
// Download all images from content and save to directory
|
|
332
|
+
async function downloadImages(content, imageDir, token) {
|
|
333
|
+
const images = extractImagesFromMarkdown(content);
|
|
334
|
+
const imageMap = new Map(); // Original URL -> local path
|
|
335
|
+
const results = {
|
|
336
|
+
downloaded: [],
|
|
337
|
+
failed: [],
|
|
338
|
+
skipped: [],
|
|
339
|
+
};
|
|
340
|
+
|
|
341
|
+
if (images.length === 0) {
|
|
342
|
+
return { imageMap, results };
|
|
343
|
+
}
|
|
344
|
+
|
|
345
|
+
log('blue', `📷 Found ${images.length} image(s) to download...`);
|
|
346
|
+
|
|
347
|
+
// Create image directory if needed
|
|
348
|
+
await fs.ensureDir(imageDir);
|
|
349
|
+
|
|
350
|
+
let imageIndex = 0;
|
|
351
|
+
for (const image of images) {
|
|
352
|
+
imageIndex++;
|
|
353
|
+
const url = image.url;
|
|
354
|
+
|
|
355
|
+
// Skip if already processed (duplicate URL)
|
|
356
|
+
if (imageMap.has(url)) {
|
|
357
|
+
logVerbose('dim', ` Skipping duplicate: ${url}`);
|
|
358
|
+
results.skipped.push({ url, reason: 'duplicate' });
|
|
359
|
+
continue;
|
|
360
|
+
}
|
|
361
|
+
|
|
362
|
+
// Skip data URLs
|
|
363
|
+
if (url.startsWith('data:')) {
|
|
364
|
+
logVerbose('dim', ` Skipping data URL`);
|
|
365
|
+
results.skipped.push({ url, reason: 'data URL' });
|
|
366
|
+
continue;
|
|
367
|
+
}
|
|
368
|
+
|
|
369
|
+
// Skip relative URLs that don't start with http
|
|
370
|
+
if (!url.startsWith('http://') && !url.startsWith('https://')) {
|
|
371
|
+
logVerbose('yellow', ` Skipping non-HTTP URL: ${url}`);
|
|
372
|
+
results.skipped.push({ url, reason: 'non-HTTP URL' });
|
|
373
|
+
continue;
|
|
374
|
+
}
|
|
375
|
+
|
|
376
|
+
try {
|
|
377
|
+
logVerbose(
|
|
378
|
+
'blue',
|
|
379
|
+
` [${imageIndex}/${images.length}] Downloading: ${url.substring(0, 80)}...`
|
|
380
|
+
);
|
|
381
|
+
|
|
382
|
+
const buffer = await downloadImage(url, token);
|
|
383
|
+
|
|
384
|
+
// Validate the downloaded content
|
|
385
|
+
const validation = validateImageBytes(buffer);
|
|
386
|
+
|
|
387
|
+
if (!validation.valid) {
|
|
388
|
+
log(
|
|
389
|
+
'yellow',
|
|
390
|
+
`⚠️ Invalid image (${validation.reason}): ${url.substring(0, 60)}...`
|
|
391
|
+
);
|
|
392
|
+
results.failed.push({ url, reason: validation.reason });
|
|
393
|
+
continue;
|
|
394
|
+
}
|
|
395
|
+
|
|
396
|
+
// Determine filename
|
|
397
|
+
const ext = getExtensionForType(validation.type);
|
|
398
|
+
const filename = `image-${imageIndex}${ext}`;
|
|
399
|
+
const localPath = path.join(imageDir, filename);
|
|
400
|
+
|
|
401
|
+
// Save the image
|
|
402
|
+
await fs.writeFile(localPath, buffer);
|
|
403
|
+
|
|
404
|
+
imageMap.set(url, {
|
|
405
|
+
localPath,
|
|
406
|
+
relativePath: path.join(path.basename(imageDir), filename),
|
|
407
|
+
type: validation.type,
|
|
408
|
+
size: buffer.length,
|
|
409
|
+
});
|
|
410
|
+
|
|
411
|
+
results.downloaded.push({
|
|
412
|
+
url,
|
|
413
|
+
localPath,
|
|
414
|
+
type: validation.type,
|
|
415
|
+
size: buffer.length,
|
|
416
|
+
});
|
|
417
|
+
|
|
418
|
+
logVerbose(
|
|
419
|
+
'green',
|
|
420
|
+
` ✓ Saved as ${filename} (${validation.type}, ${buffer.length} bytes)`
|
|
421
|
+
);
|
|
422
|
+
} catch (error) {
|
|
423
|
+
log('yellow', `⚠️ Failed to download image: ${error.message}`);
|
|
424
|
+
logVerbose('dim', ` URL: ${url}`);
|
|
425
|
+
results.failed.push({ url, reason: error.message });
|
|
426
|
+
}
|
|
427
|
+
}
|
|
428
|
+
|
|
429
|
+
// Summary
|
|
430
|
+
if (results.downloaded.length > 0) {
|
|
431
|
+
log('green', `✅ Downloaded ${results.downloaded.length} image(s)`);
|
|
432
|
+
}
|
|
433
|
+
if (results.failed.length > 0) {
|
|
434
|
+
log('yellow', `⚠️ Failed to download ${results.failed.length} image(s)`);
|
|
435
|
+
}
|
|
436
|
+
|
|
437
|
+
return { imageMap, results };
|
|
438
|
+
}
|
|
439
|
+
|
|
440
|
+
// Replace image URLs in content with local paths
|
|
441
|
+
function replaceImageUrls(content, imageMap) {
|
|
442
|
+
let updatedContent = content;
|
|
443
|
+
|
|
444
|
+
for (const [originalUrl, imageInfo] of imageMap) {
|
|
445
|
+
// Replace in markdown syntax
|
|
446
|
+
const markdownRegex = new RegExp(
|
|
447
|
+
`(!\\[[^\\]]*\\]\\()${escapeRegex(originalUrl)}((?:\\s+"[^"]*")?\\))`,
|
|
448
|
+
'g'
|
|
449
|
+
);
|
|
450
|
+
updatedContent = updatedContent.replace(
|
|
451
|
+
markdownRegex,
|
|
452
|
+
`$1${imageInfo.relativePath}$2`
|
|
453
|
+
);
|
|
454
|
+
|
|
455
|
+
// Replace in HTML img tags
|
|
456
|
+
const htmlRegex = new RegExp(
|
|
457
|
+
`(<img[^>]+src=["'])${escapeRegex(originalUrl)}(["'])`,
|
|
458
|
+
'gi'
|
|
459
|
+
);
|
|
460
|
+
updatedContent = updatedContent.replace(
|
|
461
|
+
htmlRegex,
|
|
462
|
+
`$1${imageInfo.relativePath}$2`
|
|
463
|
+
);
|
|
464
|
+
}
|
|
465
|
+
|
|
466
|
+
return updatedContent;
|
|
467
|
+
}
|
|
468
|
+
|
|
469
|
+
// Escape special regex characters
|
|
470
|
+
function escapeRegex(string) {
|
|
471
|
+
return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
472
|
+
}
|
|
473
|
+
|
|
474
|
+
// Fetch issue data from GitHub API
|
|
475
|
+
async function fetchIssue(owner, repo, issueNumber, token) {
|
|
476
|
+
try {
|
|
477
|
+
log('blue', `🔍 Fetching issue #${issueNumber} from ${owner}/${repo}...`);
|
|
478
|
+
|
|
479
|
+
const octokit = new Octokit({
|
|
480
|
+
auth: token,
|
|
481
|
+
baseUrl: 'https://api.github.com',
|
|
482
|
+
});
|
|
483
|
+
|
|
484
|
+
// Fetch the issue
|
|
485
|
+
const { data: issue } = await octokit.rest.issues.get({
|
|
486
|
+
owner,
|
|
487
|
+
repo,
|
|
488
|
+
issue_number: issueNumber,
|
|
489
|
+
});
|
|
490
|
+
|
|
491
|
+
// Fetch comments
|
|
492
|
+
const { data: comments } = await octokit.rest.issues.listComments({
|
|
493
|
+
owner,
|
|
494
|
+
repo,
|
|
495
|
+
issue_number: issueNumber,
|
|
496
|
+
});
|
|
497
|
+
|
|
498
|
+
log(
|
|
499
|
+
'green',
|
|
500
|
+
`✅ Successfully fetched issue with ${comments.length} comments`
|
|
501
|
+
);
|
|
502
|
+
|
|
503
|
+
return { issue, comments };
|
|
504
|
+
} catch (error) {
|
|
505
|
+
if (error.status === 404) {
|
|
506
|
+
log('red', `❌ Issue #${issueNumber} not found in ${owner}/${repo}`);
|
|
507
|
+
} else if (error.status === 401) {
|
|
508
|
+
log(
|
|
509
|
+
'red',
|
|
510
|
+
`❌ Authentication failed. Please provide a valid GitHub token`
|
|
511
|
+
);
|
|
512
|
+
} else {
|
|
513
|
+
log('red', `❌ Failed to fetch issue: ${error.message}`);
|
|
514
|
+
}
|
|
515
|
+
throw error;
|
|
516
|
+
}
|
|
517
|
+
}
|
|
518
|
+
|
|
519
|
+
// Convert issue to markdown format
|
|
520
|
+
function issueToMarkdown(issueData, imageMap = null) {
|
|
521
|
+
const { issue, comments } = issueData;
|
|
522
|
+
let markdown = '';
|
|
523
|
+
|
|
524
|
+
// Title
|
|
525
|
+
markdown += `# ${issue.title}\n\n`;
|
|
526
|
+
|
|
527
|
+
// Metadata
|
|
528
|
+
markdown += `**Issue:** [#${issue.number}](${issue.html_url}) \n`;
|
|
529
|
+
markdown += `**Author:** [@${issue.user.login}](${issue.user.html_url}) \n`;
|
|
530
|
+
markdown += `**State:** ${issue.state} \n`;
|
|
531
|
+
markdown += `**Created:** ${new Date(issue.created_at).toLocaleString()} \n`;
|
|
532
|
+
markdown += `**Updated:** ${new Date(issue.updated_at).toLocaleString()} \n`;
|
|
533
|
+
|
|
534
|
+
if (issue.labels && issue.labels.length > 0) {
|
|
535
|
+
const labels = issue.labels.map((label) => `\`${label.name}\``).join(', ');
|
|
536
|
+
markdown += `**Labels:** ${labels} \n`;
|
|
537
|
+
}
|
|
538
|
+
|
|
539
|
+
if (issue.assignees && issue.assignees.length > 0) {
|
|
540
|
+
const assignees = issue.assignees
|
|
541
|
+
.map((a) => `[@${a.login}](${a.html_url})`)
|
|
542
|
+
.join(', ');
|
|
543
|
+
markdown += `**Assignees:** ${assignees} \n`;
|
|
544
|
+
}
|
|
545
|
+
|
|
546
|
+
if (issue.milestone) {
|
|
547
|
+
markdown += `**Milestone:** [${issue.milestone.title}](${issue.milestone.html_url}) \n`;
|
|
548
|
+
}
|
|
549
|
+
|
|
550
|
+
markdown += '\n---\n\n';
|
|
551
|
+
|
|
552
|
+
// Body
|
|
553
|
+
if (issue.body) {
|
|
554
|
+
markdown += '## Description\n\n';
|
|
555
|
+
let body = issue.body;
|
|
556
|
+
if (imageMap && imageMap.size > 0) {
|
|
557
|
+
body = replaceImageUrls(body, imageMap);
|
|
558
|
+
}
|
|
559
|
+
markdown += body;
|
|
560
|
+
markdown += '\n\n';
|
|
561
|
+
}
|
|
562
|
+
|
|
563
|
+
// Comments
|
|
564
|
+
if (comments && comments.length > 0) {
|
|
565
|
+
markdown += '---\n\n';
|
|
566
|
+
markdown += `## Comments (${comments.length})\n\n`;
|
|
567
|
+
|
|
568
|
+
comments.forEach((comment, index) => {
|
|
569
|
+
markdown += `### Comment ${index + 1} by [@${comment.user.login}](${comment.user.html_url})\n\n`;
|
|
570
|
+
markdown += `*Posted on ${new Date(comment.created_at).toLocaleString()}*\n\n`;
|
|
571
|
+
let commentBody = comment.body;
|
|
572
|
+
if (imageMap && imageMap.size > 0) {
|
|
573
|
+
commentBody = replaceImageUrls(commentBody, imageMap);
|
|
574
|
+
}
|
|
575
|
+
markdown += commentBody;
|
|
576
|
+
markdown += '\n\n---\n\n';
|
|
577
|
+
});
|
|
578
|
+
}
|
|
579
|
+
|
|
580
|
+
return markdown;
|
|
581
|
+
}
|
|
582
|
+
|
|
583
|
+
// Convert issue to JSON format
|
|
584
|
+
function issueToJson(issueData, imageResults = null) {
|
|
585
|
+
const { issue, comments } = issueData;
|
|
586
|
+
|
|
587
|
+
return {
|
|
588
|
+
issue: {
|
|
589
|
+
number: issue.number,
|
|
590
|
+
title: issue.title,
|
|
591
|
+
state: issue.state,
|
|
592
|
+
html_url: issue.html_url,
|
|
593
|
+
author: {
|
|
594
|
+
login: issue.user.login,
|
|
595
|
+
html_url: issue.user.html_url,
|
|
596
|
+
},
|
|
597
|
+
created_at: issue.created_at,
|
|
598
|
+
updated_at: issue.updated_at,
|
|
599
|
+
labels: issue.labels.map((l) => ({
|
|
600
|
+
name: l.name,
|
|
601
|
+
color: l.color,
|
|
602
|
+
description: l.description,
|
|
603
|
+
})),
|
|
604
|
+
assignees: issue.assignees.map((a) => ({
|
|
605
|
+
login: a.login,
|
|
606
|
+
html_url: a.html_url,
|
|
607
|
+
})),
|
|
608
|
+
milestone: issue.milestone
|
|
609
|
+
? {
|
|
610
|
+
title: issue.milestone.title,
|
|
611
|
+
html_url: issue.milestone.html_url,
|
|
612
|
+
}
|
|
613
|
+
: null,
|
|
614
|
+
body: issue.body,
|
|
615
|
+
},
|
|
616
|
+
comments: comments.map((comment) => ({
|
|
617
|
+
id: comment.id,
|
|
618
|
+
author: {
|
|
619
|
+
login: comment.user.login,
|
|
620
|
+
html_url: comment.user.html_url,
|
|
621
|
+
},
|
|
622
|
+
created_at: comment.created_at,
|
|
623
|
+
updated_at: comment.updated_at,
|
|
624
|
+
body: comment.body,
|
|
625
|
+
})),
|
|
626
|
+
images: imageResults || null,
|
|
627
|
+
metadata: {
|
|
628
|
+
downloaded_at: new Date().toISOString(),
|
|
629
|
+
tool_version: version,
|
|
630
|
+
},
|
|
631
|
+
};
|
|
632
|
+
}
|
|
633
|
+
|
|
634
|
+
// Configure CLI arguments
|
|
635
|
+
const scriptName = path.basename(process.argv[1]);
|
|
636
|
+
|
|
637
|
+
async function main() {
|
|
638
|
+
// Check for --help or --version before yargs parsing for faster response
|
|
639
|
+
const args = process.argv.slice(2);
|
|
640
|
+
if (args.includes('--help') || args.includes('-h')) {
|
|
641
|
+
console.log(`Usage: ${scriptName} <issue-url> [options]
|
|
642
|
+
|
|
643
|
+
Positionals:
|
|
644
|
+
issue GitHub issue URL or short format (owner/repo#123) [string]
|
|
645
|
+
|
|
646
|
+
Options:
|
|
647
|
+
--version Show version number [boolean]
|
|
648
|
+
-t, --token GitHub personal access token (optional for public
|
|
649
|
+
issues) [string]
|
|
650
|
+
-o, --output Output directory or file path (default: current
|
|
651
|
+
directory) [string]
|
|
652
|
+
--download-images Download embedded images (default: true) [boolean]
|
|
653
|
+
-f, --format Output format: markdown, json (default: markdown)
|
|
654
|
+
[string]
|
|
655
|
+
-v, --verbose Enable verbose logging [boolean]
|
|
656
|
+
-h, --help Show help [boolean]
|
|
657
|
+
|
|
658
|
+
Examples:
|
|
659
|
+
${scriptName} https://github.com/owner/repo/issues/123 Download issue #123
|
|
660
|
+
${scriptName} owner/repo#123 Download issue #123 using short format
|
|
661
|
+
${scriptName} owner/repo#123 -o my-issue.md Save to specific file
|
|
662
|
+
${scriptName} owner/repo#123 --token ghp_xxx Use specific GitHub token
|
|
663
|
+
${scriptName} owner/repo#123 --format json Export as JSON
|
|
664
|
+
${scriptName} owner/repo#123 --no-download-images Skip image download`);
|
|
665
|
+
process.exit(0);
|
|
666
|
+
}
|
|
667
|
+
|
|
668
|
+
if (args.includes('--version')) {
|
|
669
|
+
console.log(version);
|
|
670
|
+
process.exit(0);
|
|
671
|
+
}
|
|
672
|
+
|
|
673
|
+
// Create yargs instance with proper configuration
|
|
674
|
+
const yargsInstance = yargs(hideBin(process.argv))
|
|
675
|
+
.scriptName(scriptName)
|
|
676
|
+
.version(version)
|
|
677
|
+
.usage('Usage: $0 <issue-url> [options]')
|
|
678
|
+
.command(
|
|
679
|
+
'$0 [issue]',
|
|
680
|
+
'Download a GitHub issue and convert it to markdown',
|
|
681
|
+
(yargs) => {
|
|
682
|
+
yargs.positional('issue', {
|
|
683
|
+
describe: 'GitHub issue URL or short format (owner/repo#123)',
|
|
684
|
+
type: 'string',
|
|
685
|
+
});
|
|
686
|
+
}
|
|
687
|
+
)
|
|
688
|
+
.option('token', {
|
|
689
|
+
alias: 't',
|
|
690
|
+
type: 'string',
|
|
691
|
+
describe: 'GitHub personal access token (optional for public issues)',
|
|
692
|
+
default: process.env.GITHUB_TOKEN,
|
|
693
|
+
})
|
|
694
|
+
.option('output', {
|
|
695
|
+
alias: 'o',
|
|
696
|
+
type: 'string',
|
|
697
|
+
describe: 'Output directory or file path (default: current directory)',
|
|
698
|
+
})
|
|
699
|
+
.option('download-images', {
|
|
700
|
+
type: 'boolean',
|
|
701
|
+
describe: 'Download embedded images (default: true)',
|
|
702
|
+
default: true,
|
|
703
|
+
})
|
|
704
|
+
.option('format', {
|
|
705
|
+
alias: 'f',
|
|
706
|
+
type: 'string',
|
|
707
|
+
describe: 'Output format: markdown, json (default: markdown)',
|
|
708
|
+
choices: ['markdown', 'json'],
|
|
709
|
+
default: 'markdown',
|
|
710
|
+
})
|
|
711
|
+
.option('verbose', {
|
|
712
|
+
alias: 'v',
|
|
713
|
+
type: 'boolean',
|
|
714
|
+
describe: 'Enable verbose logging',
|
|
715
|
+
default: false,
|
|
716
|
+
})
|
|
717
|
+
.help(false) // Disable yargs built-in help since we handle it manually
|
|
718
|
+
.version(false) // Disable yargs built-in version since we handle it manually
|
|
719
|
+
.example(
|
|
720
|
+
'$0 https://github.com/owner/repo/issues/123',
|
|
721
|
+
'Download issue #123'
|
|
722
|
+
)
|
|
723
|
+
.example('$0 owner/repo#123', 'Download issue #123 using short format')
|
|
724
|
+
.example('$0 owner/repo#123 -o my-issue.md', 'Save to specific file')
|
|
725
|
+
.example('$0 owner/repo#123 --token ghp_xxx', 'Use specific GitHub token')
|
|
726
|
+
.example('$0 owner/repo#123 --format json', 'Export as JSON')
|
|
727
|
+
.example('$0 owner/repo#123 --no-download-images', 'Skip image download');
|
|
728
|
+
|
|
729
|
+
const argv = await yargsInstance.parseAsync();
|
|
730
|
+
|
|
731
|
+
const { issue: issueInput, output, format, verbose } = argv;
|
|
732
|
+
const downloadImages_flag = argv['download-images'];
|
|
733
|
+
let { token } = argv;
|
|
734
|
+
|
|
735
|
+
// Set verbose mode
|
|
736
|
+
verboseMode = verbose;
|
|
737
|
+
|
|
738
|
+
// Check if issue URL was provided
|
|
739
|
+
if (!issueInput) {
|
|
740
|
+
log('red', '❌ No issue URL provided');
|
|
741
|
+
log(
|
|
742
|
+
'yellow',
|
|
743
|
+
' Expected: https://github.com/owner/repo/issues/123 or owner/repo#123'
|
|
744
|
+
);
|
|
745
|
+
log('yellow', ' Run with --help for more information');
|
|
746
|
+
process.exit(1);
|
|
747
|
+
}
|
|
748
|
+
|
|
749
|
+
// Parse the issue URL
|
|
750
|
+
const parsed = parseIssueUrl(issueInput);
|
|
751
|
+
if (!parsed) {
|
|
752
|
+
log('red', '❌ Invalid issue URL or format');
|
|
753
|
+
log(
|
|
754
|
+
'yellow',
|
|
755
|
+
' Expected: https://github.com/owner/repo/issues/123 or owner/repo#123'
|
|
756
|
+
);
|
|
757
|
+
process.exit(1);
|
|
758
|
+
}
|
|
759
|
+
|
|
760
|
+
const { owner, repo, issueNumber } = parsed;
|
|
761
|
+
|
|
762
|
+
// If no token provided, try to get it from gh CLI
|
|
763
|
+
if (!token || token === undefined) {
|
|
764
|
+
const ghToken = await getGhToken();
|
|
765
|
+
if (ghToken) {
|
|
766
|
+
token = ghToken;
|
|
767
|
+
log('cyan', '🔑 Using GitHub token from gh CLI');
|
|
768
|
+
}
|
|
769
|
+
}
|
|
770
|
+
|
|
771
|
+
// Fetch the issue
|
|
772
|
+
let issueData;
|
|
773
|
+
try {
|
|
774
|
+
issueData = await fetchIssue(owner, repo, issueNumber, token);
|
|
775
|
+
} catch (_error) {
|
|
776
|
+
process.exit(1);
|
|
777
|
+
}
|
|
778
|
+
|
|
779
|
+
// Determine output paths
|
|
780
|
+
let outputDir = process.cwd();
|
|
781
|
+
let outputFilename;
|
|
782
|
+
|
|
783
|
+
if (output) {
|
|
784
|
+
// Check if output looks like a file path (has extension) or directory
|
|
785
|
+
const ext = path.extname(output);
|
|
786
|
+
if (ext === '.md' || ext === '.json') {
|
|
787
|
+
outputDir = path.dirname(output);
|
|
788
|
+
outputFilename = path.basename(output, ext);
|
|
789
|
+
} else if (ext) {
|
|
790
|
+
outputDir = path.dirname(output);
|
|
791
|
+
outputFilename = path.basename(output);
|
|
792
|
+
} else {
|
|
793
|
+
// Treat as directory
|
|
794
|
+
outputDir = output;
|
|
795
|
+
}
|
|
796
|
+
}
|
|
797
|
+
|
|
798
|
+
// Ensure output directory exists
|
|
799
|
+
await fs.ensureDir(outputDir);
|
|
800
|
+
|
|
801
|
+
// Default filename if not specified
|
|
802
|
+
if (!outputFilename) {
|
|
803
|
+
outputFilename = `issue-${issueNumber}`;
|
|
804
|
+
}
|
|
805
|
+
|
|
806
|
+
// Download images if enabled
|
|
807
|
+
let imageMap = null;
|
|
808
|
+
let imageResults = null;
|
|
809
|
+
|
|
810
|
+
if (downloadImages_flag) {
|
|
811
|
+
const imageDir = path.join(outputDir, `${outputFilename}-images`);
|
|
812
|
+
|
|
813
|
+
// Collect all content for image extraction
|
|
814
|
+
let allContent = issueData.issue.body || '';
|
|
815
|
+
for (const comment of issueData.comments) {
|
|
816
|
+
allContent += `\n${comment.body || ''}`;
|
|
817
|
+
}
|
|
818
|
+
|
|
819
|
+
const { imageMap: downloadedMap, results } = await downloadImages(
|
|
820
|
+
allContent,
|
|
821
|
+
imageDir,
|
|
822
|
+
token
|
|
823
|
+
);
|
|
824
|
+
|
|
825
|
+
imageMap = downloadedMap;
|
|
826
|
+
imageResults = results;
|
|
827
|
+
|
|
828
|
+
// Clean up empty image directory
|
|
829
|
+
if (results.downloaded.length === 0 && (await fs.pathExists(imageDir))) {
|
|
830
|
+
try {
|
|
831
|
+
const files = await fs.readdir(imageDir);
|
|
832
|
+
if (files.length === 0) {
|
|
833
|
+
await fs.rmdir(imageDir);
|
|
834
|
+
}
|
|
835
|
+
} catch (_error) {
|
|
836
|
+
// Ignore cleanup errors
|
|
837
|
+
}
|
|
838
|
+
}
|
|
839
|
+
}
|
|
840
|
+
|
|
841
|
+
// Generate output based on format
|
|
842
|
+
log('blue', `📝 Converting to ${format}...`);
|
|
843
|
+
|
|
844
|
+
if (format === 'json') {
|
|
845
|
+
const jsonOutput = issueToJson(issueData, imageResults);
|
|
846
|
+
const outputPath = path.join(outputDir, `${outputFilename}.json`);
|
|
847
|
+
|
|
848
|
+
try {
|
|
849
|
+
await fs.writeFile(
|
|
850
|
+
outputPath,
|
|
851
|
+
JSON.stringify(jsonOutput, null, 2),
|
|
852
|
+
'utf8'
|
|
853
|
+
);
|
|
854
|
+
log('green', `✅ Issue saved to: ${outputPath}`);
|
|
855
|
+
} catch (error) {
|
|
856
|
+
log('red', `❌ Failed to write file: ${error.message}`);
|
|
857
|
+
process.exit(1);
|
|
858
|
+
}
|
|
859
|
+
} else {
|
|
860
|
+
// Markdown format
|
|
861
|
+
const markdown = issueToMarkdown(issueData, imageMap);
|
|
862
|
+
const outputPath = path.join(outputDir, `${outputFilename}.md`);
|
|
863
|
+
|
|
864
|
+
try {
|
|
865
|
+
await fs.writeFile(outputPath, markdown, 'utf8');
|
|
866
|
+
log('green', `✅ Issue saved to: ${outputPath}`);
|
|
867
|
+
} catch (error) {
|
|
868
|
+
log('red', `❌ Failed to write file: ${error.message}`);
|
|
869
|
+
process.exit(1);
|
|
870
|
+
}
|
|
871
|
+
}
|
|
872
|
+
|
|
873
|
+
// Summary
|
|
874
|
+
if (imageResults && imageResults.downloaded.length > 0) {
|
|
875
|
+
const imageDir = path.join(outputDir, `${outputFilename}-images`);
|
|
876
|
+
log('green', `📁 Images saved to: ${imageDir}`);
|
|
877
|
+
}
|
|
878
|
+
}
|
|
879
|
+
|
|
880
|
+
main().catch((error) => {
|
|
881
|
+
log('red', `💥 Script failed: ${error.message}`);
|
|
882
|
+
process.exit(1);
|
|
883
|
+
});
|
package/package.json
ADDED
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "gh-load-issue",
|
|
3
|
+
"version": "0.3.0",
|
|
4
|
+
"description": "A CLI tool to download GitHub issues and convert them to markdown",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"main": "gh-download-issue.mjs",
|
|
7
|
+
"bin": {
|
|
8
|
+
"gh-download-issue": "./gh-download-issue.mjs"
|
|
9
|
+
},
|
|
10
|
+
"scripts": {
|
|
11
|
+
"test": "node --test tests/test-all.mjs",
|
|
12
|
+
"lint": "eslint .",
|
|
13
|
+
"lint:fix": "eslint . --fix",
|
|
14
|
+
"format": "prettier --write .",
|
|
15
|
+
"format:check": "prettier --check .",
|
|
16
|
+
"check:file-size": "node scripts/check-file-size.mjs",
|
|
17
|
+
"check:duplication": "jscpd .",
|
|
18
|
+
"check": "npm run lint && npm run format:check && npm run check:file-size && npm run check:duplication",
|
|
19
|
+
"prepare": "husky || true",
|
|
20
|
+
"changeset": "changeset",
|
|
21
|
+
"changeset:version": "node scripts/changeset-version.mjs",
|
|
22
|
+
"changeset:publish": "changeset publish",
|
|
23
|
+
"changeset:status": "changeset status --since=origin/main"
|
|
24
|
+
},
|
|
25
|
+
"engines": {
|
|
26
|
+
"node": ">=20.0.0"
|
|
27
|
+
},
|
|
28
|
+
"dependencies": {
|
|
29
|
+
"@octokit/rest": "^22.0.0",
|
|
30
|
+
"fs-extra": "^11.3.0",
|
|
31
|
+
"yargs": "^17.7.2"
|
|
32
|
+
},
|
|
33
|
+
"devDependencies": {
|
|
34
|
+
"@changesets/cli": "^2.29.7",
|
|
35
|
+
"eslint": "^9.38.0",
|
|
36
|
+
"eslint-config-prettier": "^10.1.8",
|
|
37
|
+
"eslint-plugin-prettier": "^5.5.4",
|
|
38
|
+
"husky": "^9.1.7",
|
|
39
|
+
"jscpd": "^4.0.5",
|
|
40
|
+
"lint-staged": "^16.2.6",
|
|
41
|
+
"prettier": "^3.6.2",
|
|
42
|
+
"test-anywhere": "^0.8.48"
|
|
43
|
+
},
|
|
44
|
+
"lint-staged": {
|
|
45
|
+
"*.{js,mjs,cjs}": [
|
|
46
|
+
"eslint --fix --max-warnings 0",
|
|
47
|
+
"prettier --write",
|
|
48
|
+
"prettier --check"
|
|
49
|
+
],
|
|
50
|
+
"*.md": [
|
|
51
|
+
"prettier --write",
|
|
52
|
+
"prettier --check"
|
|
53
|
+
]
|
|
54
|
+
},
|
|
55
|
+
"keywords": [
|
|
56
|
+
"github",
|
|
57
|
+
"issue",
|
|
58
|
+
"markdown",
|
|
59
|
+
"cli",
|
|
60
|
+
"tool"
|
|
61
|
+
],
|
|
62
|
+
"author": "Link.Foundation",
|
|
63
|
+
"license": "Unlicense",
|
|
64
|
+
"repository": {
|
|
65
|
+
"type": "git",
|
|
66
|
+
"url": "git+https://github.com/link-foundation/gh-download-issue.git"
|
|
67
|
+
},
|
|
68
|
+
"bugs": {
|
|
69
|
+
"url": "https://github.com/link-foundation/gh-download-issue/issues"
|
|
70
|
+
},
|
|
71
|
+
"homepage": "https://github.com/link-foundation/gh-download-issue#readme",
|
|
72
|
+
"files": [
|
|
73
|
+
"gh-download-issue.mjs",
|
|
74
|
+
"README.md",
|
|
75
|
+
"LICENSE"
|
|
76
|
+
]
|
|
77
|
+
}
|