repo-digest 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- repo_digest-0.1.0/.github/workflows/ci.yml +99 -0
- repo_digest-0.1.0/LICENSE +21 -0
- repo_digest-0.1.0/MVP_PLAN.md +193 -0
- repo_digest-0.1.0/PKG-INFO +75 -0
- repo_digest-0.1.0/README.md +54 -0
- repo_digest-0.1.0/export_repo_as_text.py +281 -0
- repo_digest-0.1.0/pyproject.toml +32 -0
- repo_digest-0.1.0/src/repo_digest/__init__.py +5 -0
- repo_digest-0.1.0/src/repo_digest/cli.py +43 -0
- repo_digest-0.1.0/src/repo_digest/core.py +340 -0
- repo_digest-0.1.0/tests/__init__.py +0 -0
- repo_digest-0.1.0/tests/test_core.py +231 -0
@@ -0,0 +1,99 @@
|
|
1
|
+
name: CI
|
2
|
+
|
3
|
+
on:
|
4
|
+
push:
|
5
|
+
branches: [ main, develop ]
|
6
|
+
pull_request:
|
7
|
+
branches: [ main ]
|
8
|
+
|
9
|
+
jobs:
|
10
|
+
test:
|
11
|
+
runs-on: ubuntu-latest
|
12
|
+
strategy:
|
13
|
+
matrix:
|
14
|
+
python-version: [3.8, 3.9, "3.10", "3.11"]
|
15
|
+
|
16
|
+
steps:
|
17
|
+
- uses: actions/checkout@v4
|
18
|
+
|
19
|
+
- name: Set up Python ${{ matrix.python-version }}
|
20
|
+
uses: actions/setup-python@v4
|
21
|
+
with:
|
22
|
+
python-version: ${{ matrix.python-version }}
|
23
|
+
|
24
|
+
- name: Install dependencies
|
25
|
+
run: |
|
26
|
+
python -m pip install --upgrade pip
|
27
|
+
pip install pytest
|
28
|
+
pip install -e .
|
29
|
+
|
30
|
+
- name: Run tests
|
31
|
+
run: |
|
32
|
+
python -m pytest tests/ -v
|
33
|
+
|
34
|
+
- name: Test CLI functionality
|
35
|
+
run: |
|
36
|
+
repo-digest --help
|
37
|
+
repo-digest . --preview
|
38
|
+
|
39
|
+
lint:
|
40
|
+
runs-on: ubuntu-latest
|
41
|
+
|
42
|
+
steps:
|
43
|
+
- uses: actions/checkout@v4
|
44
|
+
|
45
|
+
- name: Set up Python
|
46
|
+
uses: actions/setup-python@v4
|
47
|
+
with:
|
48
|
+
python-version: "3.10"
|
49
|
+
|
50
|
+
- name: Install dependencies
|
51
|
+
run: |
|
52
|
+
python -m pip install --upgrade pip
|
53
|
+
pip install flake8 black isort
|
54
|
+
|
55
|
+
- name: Lint with flake8
|
56
|
+
run: |
|
57
|
+
# Stop the build if there are Python syntax errors or undefined names
|
58
|
+
flake8 src/ tests/ --count --select=E9,F63,F7,F82 --show-source --statistics
|
59
|
+
# Exit-zero treats all errors as warnings. Line length set to 88 (black default)
|
60
|
+
flake8 src/ tests/ --count --exit-zero --max-complexity=10 --max-line-length=88 --statistics
|
61
|
+
|
62
|
+
- name: Check code formatting with black
|
63
|
+
run: |
|
64
|
+
black --check --diff src/ tests/
|
65
|
+
|
66
|
+
- name: Check import sorting with isort
|
67
|
+
run: |
|
68
|
+
isort --check-only --diff src/ tests/
|
69
|
+
|
70
|
+
build:
|
71
|
+
runs-on: ubuntu-latest
|
72
|
+
needs: [test, lint]
|
73
|
+
|
74
|
+
steps:
|
75
|
+
- uses: actions/checkout@v4
|
76
|
+
|
77
|
+
- name: Set up Python
|
78
|
+
uses: actions/setup-python@v4
|
79
|
+
with:
|
80
|
+
python-version: "3.10"
|
81
|
+
|
82
|
+
- name: Install build dependencies
|
83
|
+
run: |
|
84
|
+
python -m pip install --upgrade pip
|
85
|
+
pip install build twine
|
86
|
+
|
87
|
+
- name: Build package
|
88
|
+
run: |
|
89
|
+
python -m build
|
90
|
+
|
91
|
+
- name: Check package
|
92
|
+
run: |
|
93
|
+
python -m twine check dist/*
|
94
|
+
|
95
|
+
- name: Upload build artifacts
|
96
|
+
uses: actions/upload-artifact@v3
|
97
|
+
with:
|
98
|
+
name: dist
|
99
|
+
path: dist/
|
@@ -0,0 +1,21 @@
|
|
1
|
+
MIT License
|
2
|
+
|
3
|
+
Copyright (c) 2025 Your Name
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
13
|
+
copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
21
|
+
SOFTWARE.
|
@@ -0,0 +1,193 @@
|
|
1
|
+
# repo-digest MVP Plan
|
2
|
+
|
3
|
+
## Project Overview
|
4
|
+
Transform the existing `export_repo_as_text.py` script into a professional, beginner-friendly PyPI package called `repo-digest`.
|
5
|
+
|
6
|
+
**Goal**: Create a dead-simple tool for anyone to turn a local repository into an AI-ready text bundle with sensible defaults and safety guardrails.
|
7
|
+
|
8
|
+
**Target Users**:
|
9
|
+
- Absolute beginners using ChatGPT/Claude who want to "paste" their repo context
|
10
|
+
- Developers who want a quick repo digest
|
11
|
+
|
12
|
+
## Core Value Proposition
|
13
|
+
- **One-liner**: Turn any repository into a safe, structured text export ready for LLMs
|
14
|
+
- **Differentiators**:
|
15
|
+
- Sensible, safe defaults (strong excludes, secret patterns blocked by default)
|
16
|
+
- Accurate token counting with optional tiktoken (fallback to words if not installed)
|
17
|
+
- Clean summary and directory tree for quick repo understanding
|
18
|
+
- Works out-of-the-box on macOS/Linux/Windows
|
19
|
+
|
20
|
+
## MVP Scope (No Overengineering)
|
21
|
+
|
22
|
+
### ✅ COMPLETED
|
23
|
+
- [x] **Project Structure**: Created professional package structure with `src/repo_digest/`
|
24
|
+
- [x] **PyPI Package Setup**: `pyproject.toml` with metadata, optional tiktoken extra, console script
|
25
|
+
- [x] **Core Module**: Ported original script to `src/repo_digest/core.py` with type hints
|
26
|
+
- [x] **CLI Interface**: Basic argparse CLI in `src/repo_digest/cli.py` with MVP flags
|
27
|
+
- [x] **Documentation**: Beginner-friendly README.md with quickstart, safety, examples
|
28
|
+
- [x] **License**: MIT license for maximum adoption
|
29
|
+
- [x] **Package Structure**:
|
30
|
+
```
|
31
|
+
repo-digest/
|
32
|
+
├── pyproject.toml
|
33
|
+
├── README.md
|
34
|
+
├── LICENSE
|
35
|
+
├── src/
|
36
|
+
│ └── repo_digest/
|
37
|
+
│ ├── __init__.py
|
38
|
+
│ ├── core.py
|
39
|
+
│ └── cli.py
|
40
|
+
```
|
41
|
+
|
42
|
+
### 🔧 IN PROGRESS
|
43
|
+
- [ ] **Safety Logic Fix**: Complete allow_secrets implementation in core.py
|
44
|
+
- [ ] **CLI Testing**: Verify all flags work correctly
|
45
|
+
|
46
|
+
### 📋 PENDING (MVP Deliverables)
|
47
|
+
- [ ] **Testing**: Basic functionality tests
|
48
|
+
- [ ] **CI/CD**: GitHub Actions for lint + build
|
49
|
+
- [ ] **Package Build**: Test local build and installation
|
50
|
+
- [ ] **PyPI Release**: Publish v0.1.0 to PyPI
|
51
|
+
- [ ] **Demo Content**: Create GIF/screenshots for README
|
52
|
+
|
53
|
+
## CLI Design (MVP)
|
54
|
+
|
55
|
+
### Commands
|
56
|
+
```bash
|
57
|
+
# Basic usage
|
58
|
+
repo-digest . -o repo.txt
|
59
|
+
|
60
|
+
# With options
|
61
|
+
repo-digest ~/project -o export.txt --preview
|
62
|
+
repo-digest . -o repo.txt --max-bytes 5000000
|
63
|
+
repo-digest . -o repo.txt --allow-secrets --no-gitignore
|
64
|
+
```
|
65
|
+
|
66
|
+
### Flags
|
67
|
+
- `--preview`: Show counts only; don't write output
|
68
|
+
- `--max-bytes N`: Fail if estimated total bytes exceed limit
|
69
|
+
- `--allow-secrets`: Allow files matching sensitive patterns (off by default)
|
70
|
+
- `--no-gitignore`: Ignore .gitignore (default respects it)
|
71
|
+
|
72
|
+
### Exit Codes
|
73
|
+
- `0`: Success
|
74
|
+
- `1`: Runtime error (bad path, permission)
|
75
|
+
- `2`: Safety violation (secrets detected and not allowed)
|
76
|
+
- `3`: Exceeded size/limits
|
77
|
+
|
78
|
+
## Safety and Guardrails
|
79
|
+
|
80
|
+
### ✅ Implemented
|
81
|
+
- Comprehensive exclusion patterns (build dirs, node_modules, etc.)
|
82
|
+
- Binary file exclusions
|
83
|
+
- Gitignore respect by default
|
84
|
+
- Sensitive pattern detection
|
85
|
+
|
86
|
+
### 🔧 Needs Fix
|
87
|
+
- Secret blocking logic (currently has implementation issue)
|
88
|
+
- Clear safety banners in output
|
89
|
+
|
90
|
+
### Safety Features
|
91
|
+
- **Secrets blocked by default**: `.env`, `*secret*`, `*password*`, `*token*`, `*key*`, `*.pem`, etc.
|
92
|
+
- **Binary exclusions**: Images, videos, archives, compiled files
|
93
|
+
- **Build artifact exclusions**: `node_modules`, `__pycache__`, `dist`, `build`, etc.
|
94
|
+
- **Large data exclusions**: `.csv`, `.h5`, `.parquet`, etc.
|
95
|
+
|
96
|
+
## Installation and Distribution
|
97
|
+
|
98
|
+
### Package Details
|
99
|
+
- **Name**: `repo-digest`
|
100
|
+
- **PyPI**: `pip install repo-digest`
|
101
|
+
- **Optional tiktoken**: `pip install "repo-digest[tiktoken]"`
|
102
|
+
- **Entry point**: `repo-digest` console script
|
103
|
+
- **Python support**: >=3.8
|
104
|
+
|
105
|
+
### Distribution Channels (MVP)
|
106
|
+
- [x] PyPI package
|
107
|
+
- [ ] GitHub releases
|
108
|
+
- [ ] Basic documentation
|
109
|
+
|
110
|
+
### Future Distribution (Post-MVP)
|
111
|
+
- Homebrew formula for macOS
|
112
|
+
- Snap package for Linux
|
113
|
+
- Chocolatey for Windows
|
114
|
+
|
115
|
+
## Launch Plan (7-10 days)
|
116
|
+
|
117
|
+
### Phase 1: Complete MVP (Days 1-2)
|
118
|
+
- [ ] Fix safety logic in core.py
|
119
|
+
- [ ] Add basic tests
|
120
|
+
- [ ] Set up GitHub Actions (lint + build)
|
121
|
+
- [ ] Test local installation
|
122
|
+
- [ ] Release v0.1.0 to PyPI
|
123
|
+
|
124
|
+
### Phase 2: Demo and Launch (Days 3-4)
|
125
|
+
- [ ] Create demo GIF showing: run CLI → open output → highlight tree and summary
|
126
|
+
- [ ] Polish README with demo
|
127
|
+
- [ ] Create GitHub repository with good README and tags
|
128
|
+
|
129
|
+
### Phase 3: Promotion (Days 5-10)
|
130
|
+
- [ ] Launch on Reddit: r/Python, r/learnprogramming, r/programming
|
131
|
+
- [ ] Twitter/X thread with GIF demo
|
132
|
+
- [ ] Position as: "Paste your repo into ChatGPT/Claude in one go"
|
133
|
+
- [ ] Add "Good First Issues" and "Help Wanted" labels
|
134
|
+
- [ ] Respond quickly to feedback
|
135
|
+
|
136
|
+
## Success Metrics (MVP)
|
137
|
+
|
138
|
+
### Technical Metrics
|
139
|
+
- **Installation friction**: Time-to-first-export < 2 minutes
|
140
|
+
- **Functionality**: Users can export without reading more than Quickstart
|
141
|
+
|
142
|
+
### Community Metrics
|
143
|
+
- **GitHub**: 50-100 stars in first 2 weeks
|
144
|
+
- **PyPI**: 200-500 downloads in first month
|
145
|
+
- **Feedback**: At least 5 real user issues/requests (validation signal)
|
146
|
+
|
147
|
+
## Roadmap After MVP (Only When Demand Validated)
|
148
|
+
|
149
|
+
### Phase 2 Features
|
150
|
+
- Output formats: Markdown and JSON
|
151
|
+
- Config file support (repo-to-text.yaml)
|
152
|
+
- Platform integrations (GitHub repo URL)
|
153
|
+
|
154
|
+
### Phase 3 Features
|
155
|
+
- Chunking large repos into multiple files with manifest
|
156
|
+
- Simple GUI (only if users ask)
|
157
|
+
- Advanced filtering options
|
158
|
+
|
159
|
+
### Phase 4 Ecosystem
|
160
|
+
- Plugin system for custom processors
|
161
|
+
- Integration with popular AI tools
|
162
|
+
- Enterprise features
|
163
|
+
|
164
|
+
## Technical Decisions Made
|
165
|
+
|
166
|
+
### Core Choices
|
167
|
+
- **Language**: Python (matches original script)
|
168
|
+
- **CLI Framework**: argparse (simple, no dependencies)
|
169
|
+
- **Package Manager**: pip/PyPI (standard Python distribution)
|
170
|
+
- **License**: MIT (maximum adoption)
|
171
|
+
- **Limit Flag**: `--max-bytes` (simple and predictable vs `--max-tokens`)
|
172
|
+
|
173
|
+
### Architecture Decisions
|
174
|
+
- **Module Structure**: Clean separation of core logic and CLI
|
175
|
+
- **Type Hints**: Added for better code quality
|
176
|
+
- **Error Handling**: Structured exit codes for automation
|
177
|
+
- **Safety First**: Secrets blocked by default, explicit override required
|
178
|
+
|
179
|
+
## Current Status Summary
|
180
|
+
|
181
|
+
**✅ Foundation Complete**: Package structure, core functionality, CLI interface, documentation
|
182
|
+
**🔧 Minor Fixes Needed**: Safety logic implementation, testing
|
183
|
+
**📋 Ready for Launch**: Once fixes complete, ready for PyPI release and promotion
|
184
|
+
|
185
|
+
**Next Immediate Steps**:
|
186
|
+
1. Fix allow_secrets logic in core.py
|
187
|
+
2. Add basic tests
|
188
|
+
3. Test local installation
|
189
|
+
4. Release to PyPI
|
190
|
+
5. Create demo content
|
191
|
+
6. Launch promotion campaign
|
192
|
+
|
193
|
+
The MVP is 90% complete and ready for launch within 1-2 days of completing the remaining technical fixes.
|
@@ -0,0 +1,75 @@
|
|
1
|
+
Metadata-Version: 2.4
|
2
|
+
Name: repo-digest
|
3
|
+
Version: 0.1.0
|
4
|
+
Summary: Turn any repository into an AI-ready text bundle with safe defaults and rich analytics.
|
5
|
+
Author: Your Name
|
6
|
+
License: MIT
|
7
|
+
License-File: LICENSE
|
8
|
+
Keywords: ai,analysis,export,llm,repository,text,tiktoken,tokens
|
9
|
+
Classifier: Environment :: Console
|
10
|
+
Classifier: License :: OSI Approved :: MIT License
|
11
|
+
Classifier: Operating System :: OS Independent
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
13
|
+
Classifier: Topic :: Software Development :: Build Tools
|
14
|
+
Classifier: Topic :: Utilities
|
15
|
+
Requires-Python: >=3.8
|
16
|
+
Provides-Extra: dev
|
17
|
+
Requires-Dist: pytest>=7.0.0; extra == 'dev'
|
18
|
+
Provides-Extra: tiktoken
|
19
|
+
Requires-Dist: tiktoken>=0.4.0; extra == 'tiktoken'
|
20
|
+
Description-Content-Type: text/markdown
|
21
|
+
|
22
|
+
# repo-digest
|
23
|
+
|
24
|
+
Turn any repository into an AI-ready text bundle with safe defaults and rich analytics.
|
25
|
+
|
26
|
+
Who is this for?
|
27
|
+
- Anyone who wants to paste a project into ChatGPT/Claude or create a quick, comprehensive repo digest.
|
28
|
+
- Works out-of-the-box on macOS, Linux, Windows.
|
29
|
+
|
30
|
+
Quickstart (60 seconds)
|
31
|
+
1) Install
|
32
|
+
- pip install repo-digest
|
33
|
+
- For precise token counts (optional): pip install "repo-digest[tiktoken]"
|
34
|
+
|
35
|
+
2) Export your repo
|
36
|
+
- repo-digest . -o repo.txt
|
37
|
+
|
38
|
+
3) Preview first (optional)
|
39
|
+
- repo-digest . -o repo.txt --preview
|
40
|
+
|
41
|
+
Safety first (defaults)
|
42
|
+
- Secrets are blocked by default (e.g., .env, *secret*, *.key, *.pem)
|
43
|
+
- Binary/large data files are excluded
|
44
|
+
- .gitignore respected by default
|
45
|
+
- To override secrets blocking (NOT recommended): --allow-secrets
|
46
|
+
|
47
|
+
Examples
|
48
|
+
- Export current repo: repo-digest . -o repo.txt
|
49
|
+
- Preview and check size: repo-digest . -o repo.txt --preview
|
50
|
+
- Enforce a size limit (bytes): repo-digest . -o repo.txt --max-bytes 5000000
|
51
|
+
- Ignore .gitignore: repo-digest . -o repo.txt --no-gitignore
|
52
|
+
|
53
|
+
Exit codes
|
54
|
+
- 0 success
|
55
|
+
- 1 runtime error (bad path, permission)
|
56
|
+
- 2 safety violation (secrets detected and not allowed)
|
57
|
+
- 3 exceeded size/limits
|
58
|
+
|
59
|
+
Troubleshooting
|
60
|
+
- Windows long paths: try running from a shorter path (e.g., C:\src)
|
61
|
+
- Encoding issues: files are read as UTF-8 with errors ignored
|
62
|
+
- Large repos: use --preview to estimate and --max-bytes to cap
|
63
|
+
|
64
|
+
FAQ
|
65
|
+
- Why are some files missing? They’re excluded by default to keep the export safe and useful. Use --no-gitignore or tweak locally if needed.
|
66
|
+
- Why token counts differ from my model? Install tiktoken for tokenizer parity; fallback uses an approximate word count.
|
67
|
+
- Can I include secrets? Not recommended. If you must: --allow-secrets (and understand the risk).
|
68
|
+
|
69
|
+
Roadmap (post-MVP)
|
70
|
+
- Markdown/JSON outputs, config file support
|
71
|
+
- GitHub URL input, chunking for huge repos
|
72
|
+
- Simple GUI if user demand is strong
|
73
|
+
|
74
|
+
License
|
75
|
+
- MIT
|
@@ -0,0 +1,54 @@
|
|
1
|
+
# repo-digest
|
2
|
+
|
3
|
+
Turn any repository into an AI-ready text bundle with safe defaults and rich analytics.
|
4
|
+
|
5
|
+
Who is this for?
|
6
|
+
- Anyone who wants to paste a project into ChatGPT/Claude or create a quick, comprehensive repo digest.
|
7
|
+
- Works out-of-the-box on macOS, Linux, Windows.
|
8
|
+
|
9
|
+
Quickstart (60 seconds)
|
10
|
+
1) Install
|
11
|
+
- pip install repo-digest
|
12
|
+
- For precise token counts (optional): pip install "repo-digest[tiktoken]"
|
13
|
+
|
14
|
+
2) Export your repo
|
15
|
+
- repo-digest . -o repo.txt
|
16
|
+
|
17
|
+
3) Preview first (optional)
|
18
|
+
- repo-digest . -o repo.txt --preview
|
19
|
+
|
20
|
+
Safety first (defaults)
|
21
|
+
- Secrets are blocked by default (e.g., .env, *secret*, *.key, *.pem)
|
22
|
+
- Binary/large data files are excluded
|
23
|
+
- .gitignore respected by default
|
24
|
+
- To override secrets blocking (NOT recommended): --allow-secrets
|
25
|
+
|
26
|
+
Examples
|
27
|
+
- Export current repo: repo-digest . -o repo.txt
|
28
|
+
- Preview and check size: repo-digest . -o repo.txt --preview
|
29
|
+
- Enforce a size limit (bytes): repo-digest . -o repo.txt --max-bytes 5000000
|
30
|
+
- Ignore .gitignore: repo-digest . -o repo.txt --no-gitignore
|
31
|
+
|
32
|
+
Exit codes
|
33
|
+
- 0 success
|
34
|
+
- 1 runtime error (bad path, permission)
|
35
|
+
- 2 safety violation (secrets detected and not allowed)
|
36
|
+
- 3 exceeded size/limits
|
37
|
+
|
38
|
+
Troubleshooting
|
39
|
+
- Windows long paths: try running from a shorter path (e.g., C:\src)
|
40
|
+
- Encoding issues: files are read as UTF-8 with errors ignored
|
41
|
+
- Large repos: use --preview to estimate and --max-bytes to cap
|
42
|
+
|
43
|
+
FAQ
|
44
|
+
- Why are some files missing? They’re excluded by default to keep the export safe and useful. Use --no-gitignore or tweak locally if needed.
|
45
|
+
- Why token counts differ from my model? Install tiktoken for tokenizer parity; fallback uses an approximate word count.
|
46
|
+
- Can I include secrets? Not recommended. If you must: --allow-secrets (and understand the risk).
|
47
|
+
|
48
|
+
Roadmap (post-MVP)
|
49
|
+
- Markdown/JSON outputs, config file support
|
50
|
+
- GitHub URL input, chunking for huge repos
|
51
|
+
- Simple GUI if user demand is strong
|
52
|
+
|
53
|
+
License
|
54
|
+
- MIT
|