src2md 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- src2md-0.1.0/LICENSE +21 -0
- src2md-0.1.0/PKG-INFO +128 -0
- src2md-0.1.0/README.md +103 -0
- src2md-0.1.0/setup.cfg +4 -0
- src2md-0.1.0/setup.py +29 -0
- src2md-0.1.0/src2md/__init__.py +0 -0
- src2md-0.1.0/src2md/cli.py +102 -0
- src2md-0.1.0/src2md/utils.py +255 -0
- src2md-0.1.0/src2md.egg-info/PKG-INFO +128 -0
- src2md-0.1.0/src2md.egg-info/SOURCES.txt +12 -0
- src2md-0.1.0/src2md.egg-info/dependency_links.txt +1 -0
- src2md-0.1.0/src2md.egg-info/entry_points.txt +2 -0
- src2md-0.1.0/src2md.egg-info/requires.txt +2 -0
- src2md-0.1.0/src2md.egg-info/top_level.txt +1 -0
src2md-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) [year] [fullname]
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
src2md-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,128 @@
|
|
|
1
|
+
Metadata-Version: 2.2
|
|
2
|
+
Name: src2md
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: A tool to convert source code directories into Markdown documentation.
|
|
5
|
+
Home-page: https://github.com/queelius/src2md
|
|
6
|
+
Author: Alex Towell
|
|
7
|
+
Author-email: lex@example.com
|
|
8
|
+
Classifier: Programming Language :: Python :: 3
|
|
9
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
10
|
+
Classifier: Operating System :: OS Independent
|
|
11
|
+
Requires-Python: >=3.6
|
|
12
|
+
Description-Content-Type: text/markdown
|
|
13
|
+
License-File: LICENSE
|
|
14
|
+
Requires-Dist: pypandoc
|
|
15
|
+
Requires-Dist: pathspec
|
|
16
|
+
Dynamic: author
|
|
17
|
+
Dynamic: author-email
|
|
18
|
+
Dynamic: classifier
|
|
19
|
+
Dynamic: description
|
|
20
|
+
Dynamic: description-content-type
|
|
21
|
+
Dynamic: home-page
|
|
22
|
+
Dynamic: requires-dist
|
|
23
|
+
Dynamic: requires-python
|
|
24
|
+
Dynamic: summary
|
|
25
|
+
|
|
26
|
+
# src2md
|
|
27
|
+
|
|
28
|
+

|
|
29
|
+

|
|
30
|
+

|
|
31
|
+
|
|
32
|
+
**src2md** is a versatile tool that converts source code directories into comprehensive Markdown documentation. It scans your project structure, includes documentation files like `README.md` or `README.rst`, and embeds source code snippets with proper syntax highlighting. Ideal for generating project overviews, documentation for large codebases, or providing context to language models.
|
|
33
|
+
|
|
34
|
+
## 🚀 **Features**
|
|
35
|
+
|
|
36
|
+
- **Automatic Documentation Aggregation**: Collects and prioritizes documentation files (e.g., `README.md`, `README.rst`).
|
|
37
|
+
- **Syntax-Highlighted Source Code**: Embeds source code files with appropriate language identifiers for Markdown.
|
|
38
|
+
- **Flexible File Patterns**: Customize which files to include as documentation or source code using patterns.
|
|
39
|
+
- **Ignore Hidden Files and Directories**: Automatically excludes hidden files and directories (those starting with `.`).
|
|
40
|
+
- **Leverages `.gitignore`**: Respects your `.gitignore` settings to exclude unwanted files.
|
|
41
|
+
- **Simple CLI Interface**: Easy-to-use command-line interface for generating documentation.
|
|
42
|
+
|
|
43
|
+
## 📦 **Installation**
|
|
44
|
+
|
|
45
|
+
You can install `src2md` via [PyPI](https://pypi.org/) using `pip`:
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
pip install src2md
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Alternatively, install it directly from the source:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
git clone https://github.com/yourusername/src2md.git
|
|
55
|
+
cd src2md
|
|
56
|
+
pip install .
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## 🛠️ Usage
|
|
60
|
+
Once installed, you can use src2md from the command line to generate Markdown documentation for your project.
|
|
61
|
+
|
|
62
|
+
### Basic Command
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
src2md path/to/source -o output.md
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
- `path/to/source`: Path to your project's root directory.
|
|
69
|
+
- `-o output.md`: (Optional) Specifies the output Markdown file name. Defaults to project_documentation.md.
|
|
70
|
+
|
|
71
|
+
## Advanced Options
|
|
72
|
+
|
|
73
|
+
- **Specify Documentation Patterns**
|
|
74
|
+
|
|
75
|
+
Include additional documentation file types.
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
src2md path/to/source -o output.md --doc-patterns '*.md' '*.rst' '*.lark'
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
- **Specify Source Code Patterns**
|
|
82
|
+
|
|
83
|
+
Customize which source files to include. Use '*' to include all files.
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
src2md path/to/source -o output.md --src-patterns '*.py' '*.js'
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
To include all files as source code:
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
src2md path/to/source -o output.md --src-patterns '*'
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
- **Add Additional Ignore Patterns**
|
|
96
|
+
|
|
97
|
+
Exclude specific files or directories.
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
src2md path/to/source -o output.md --ignore-patterns '*.pyc' 'build/' 'dist/'
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
- **Use a Custom Ignore File**
|
|
104
|
+
|
|
105
|
+
Specify a custom ignore file.
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
src2md path/to/source -o output.md --ignore-file .src2mdignore
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
- **Disable .gitignore Usage**
|
|
112
|
+
|
|
113
|
+
Do not use .gitignore for ignoring files.
|
|
114
|
+
|
|
115
|
+
```bash
|
|
116
|
+
src2md path/to/source -o output.md --no-use-gitignore
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
## Full Example
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
src2md ./my_project -o my_project_docs.md \
|
|
123
|
+
--doc-patterns '*.md' '*.rst' '*.lark' \
|
|
124
|
+
--src-patterns '*.py' '*.cpp' '*.js' \
|
|
125
|
+
--ignore-patterns '*.pyc' 'build/' 'dist/'
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
This command generates my_project_docs.md by including all .md, .rst, and .lark documentation files, embedding .py, .cpp, and .js source files, and excluding any .pyc files and directories named build or dist.
|
src2md-0.1.0/README.md
ADDED
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
# src2md
|
|
2
|
+
|
|
3
|
+

|
|
4
|
+

|
|
5
|
+

|
|
6
|
+
|
|
7
|
+
**src2md** is a versatile tool that converts source code directories into comprehensive Markdown documentation. It scans your project structure, includes documentation files like `README.md` or `README.rst`, and embeds source code snippets with proper syntax highlighting. Ideal for generating project overviews, documentation for large codebases, or providing context to language models.
|
|
8
|
+
|
|
9
|
+
## 🚀 **Features**
|
|
10
|
+
|
|
11
|
+
- **Automatic Documentation Aggregation**: Collects and prioritizes documentation files (e.g., `README.md`, `README.rst`).
|
|
12
|
+
- **Syntax-Highlighted Source Code**: Embeds source code files with appropriate language identifiers for Markdown.
|
|
13
|
+
- **Flexible File Patterns**: Customize which files to include as documentation or source code using patterns.
|
|
14
|
+
- **Ignore Hidden Files and Directories**: Automatically excludes hidden files and directories (those starting with `.`).
|
|
15
|
+
- **Leverages `.gitignore`**: Respects your `.gitignore` settings to exclude unwanted files.
|
|
16
|
+
- **Simple CLI Interface**: Easy-to-use command-line interface for generating documentation.
|
|
17
|
+
|
|
18
|
+
## 📦 **Installation**
|
|
19
|
+
|
|
20
|
+
You can install `src2md` via [PyPI](https://pypi.org/) using `pip`:
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
pip install src2md
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Alternatively, install it directly from the source:
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
git clone https://github.com/yourusername/src2md.git
|
|
30
|
+
cd src2md
|
|
31
|
+
pip install .
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
## 🛠️ Usage
|
|
35
|
+
Once installed, you can use src2md from the command line to generate Markdown documentation for your project.
|
|
36
|
+
|
|
37
|
+
### Basic Command
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
src2md path/to/source -o output.md
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
- `path/to/source`: Path to your project's root directory.
|
|
44
|
+
- `-o output.md`: (Optional) Specifies the output Markdown file name. Defaults to project_documentation.md.
|
|
45
|
+
|
|
46
|
+
## Advanced Options
|
|
47
|
+
|
|
48
|
+
- **Specify Documentation Patterns**
|
|
49
|
+
|
|
50
|
+
Include additional documentation file types.
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
src2md path/to/source -o output.md --doc-patterns '*.md' '*.rst' '*.lark'
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
- **Specify Source Code Patterns**
|
|
57
|
+
|
|
58
|
+
Customize which source files to include. Use '*' to include all files.
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
src2md path/to/source -o output.md --src-patterns '*.py' '*.js'
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
To include all files as source code:
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
src2md path/to/source -o output.md --src-patterns '*'
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
- **Add Additional Ignore Patterns**
|
|
71
|
+
|
|
72
|
+
Exclude specific files or directories.
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
src2md path/to/source -o output.md --ignore-patterns '*.pyc' 'build/' 'dist/'
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
- **Use a Custom Ignore File**
|
|
79
|
+
|
|
80
|
+
Specify a custom ignore file.
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
src2md path/to/source -o output.md --ignore-file .src2mdignore
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
- **Disable .gitignore Usage**
|
|
87
|
+
|
|
88
|
+
Do not use .gitignore for ignoring files.
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
src2md path/to/source -o output.md --no-use-gitignore
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
## Full Example
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
src2md ./my_project -o my_project_docs.md \
|
|
98
|
+
--doc-patterns '*.md' '*.rst' '*.lark' \
|
|
99
|
+
--src-patterns '*.py' '*.cpp' '*.js' \
|
|
100
|
+
--ignore-patterns '*.pyc' 'build/' 'dist/'
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
This command generates my_project_docs.md by including all .md, .rst, and .lark documentation files, embedding .py, .cpp, and .js source files, and excluding any .pyc files and directories named build or dist.
|
src2md-0.1.0/setup.cfg
ADDED
src2md-0.1.0/setup.py
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
from setuptools import setup, find_packages
|
|
2
|
+
|
|
3
|
+
setup(
|
|
4
|
+
name='src2md',
|
|
5
|
+
version='0.1.0',
|
|
6
|
+
author='Alex Towell',
|
|
7
|
+
author_email='lex@example.com',
|
|
8
|
+
description='A tool to convert source code directories into Markdown documentation.',
|
|
9
|
+
long_description=open('README.md').read(),
|
|
10
|
+
long_description_content_type='text/markdown',
|
|
11
|
+
url='https://github.com/queelius/src2md',
|
|
12
|
+
packages=find_packages(),
|
|
13
|
+
include_package_data=True,
|
|
14
|
+
classifiers=[
|
|
15
|
+
'Programming Language :: Python :: 3',
|
|
16
|
+
'License :: OSI Approved :: MIT License',
|
|
17
|
+
'Operating System :: OS Independent',
|
|
18
|
+
],
|
|
19
|
+
python_requires='>=3.6',
|
|
20
|
+
install_requires=[
|
|
21
|
+
'pypandoc',
|
|
22
|
+
'pathspec',
|
|
23
|
+
],
|
|
24
|
+
entry_points={
|
|
25
|
+
'console_scripts': [
|
|
26
|
+
'src2md=src2md.cli:main',
|
|
27
|
+
],
|
|
28
|
+
},
|
|
29
|
+
)
|
|
File without changes
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
"""
|
|
2
|
+
src2md - Source Code to Markdown
|
|
3
|
+
|
|
4
|
+
A Python package to map a source code directory into a single Markdown file,
|
|
5
|
+
including documentation files (e.g., README.md, README.rst), and source files
|
|
6
|
+
whose filenames match specified patterns.
|
|
7
|
+
|
|
8
|
+
This is primarily used to provide an LLM context about your project, which may
|
|
9
|
+
be distributed over multiple file and sub-directories.
|
|
10
|
+
|
|
11
|
+
Usage: src2md path/to/source -o output.md
|
|
12
|
+
[--doc-pat PATTERN [PATTERN ...]]
|
|
13
|
+
[--src-pat PATTERN [PATTERN ...]]
|
|
14
|
+
[--ignore-pat PATTERN [PATTERN ...]]
|
|
15
|
+
[--ignore-file FILE]
|
|
16
|
+
|
|
17
|
+
By default, --ignore-file is set to '.src2mdignore', --doc-pat is set to ['*.md', '*.rst'],
|
|
18
|
+
--src-pat is set to all the keys in the `default_langs` dictionary, and --ignore-pat is set to ['.*']
|
|
19
|
+
which ignores all hidden files and directories.
|
|
20
|
+
"""
|
|
21
|
+
import argparse
|
|
22
|
+
from pathlib import Path
|
|
23
|
+
import sys
|
|
24
|
+
from .utils import generate_markdown, default_langs, load_ignore_file
|
|
25
|
+
|
|
26
|
+
def main():
|
|
27
|
+
"""
|
|
28
|
+
Main function to parse arguments and generate the Markdown documentation.
|
|
29
|
+
"""
|
|
30
|
+
|
|
31
|
+
parser = argparse.ArgumentParser(
|
|
32
|
+
description="Map a source code directory to a Markdown file."
|
|
33
|
+
)
|
|
34
|
+
parser.add_argument(
|
|
35
|
+
"path",
|
|
36
|
+
help="Path to the source directory."
|
|
37
|
+
)
|
|
38
|
+
parser.add_argument(
|
|
39
|
+
"-o", "--out",
|
|
40
|
+
default="project.md",
|
|
41
|
+
help="Output Markdown file name. Defaults to 'project.md'."
|
|
42
|
+
)
|
|
43
|
+
parser.add_argument(
|
|
44
|
+
"--doc-pat",
|
|
45
|
+
nargs='+',
|
|
46
|
+
default=['*.md', '*.rst'],
|
|
47
|
+
help="Patterns to identify documentation files (e.g., *.md, *.rst). README.* is prioritized. Defaults to ['*.md', '*.rst']."
|
|
48
|
+
)
|
|
49
|
+
parser.add_argument(
|
|
50
|
+
"--src-pat",
|
|
51
|
+
nargs='+',
|
|
52
|
+
default=[f"*{ext}" for ext in default_langs.keys()],
|
|
53
|
+
help="Whitelist patterns to identify files to include (e.g., *.py, *.cpp, *.js). Use '*' to include all files. Defaults to all the keys in `default_langs` dictionary."
|
|
54
|
+
)
|
|
55
|
+
parser.add_argument(
|
|
56
|
+
"--ignore-pat",
|
|
57
|
+
nargs='+',
|
|
58
|
+
default=['.*'],
|
|
59
|
+
help="Blacklist patterns to identify files to ignore (e.g., *.py, *.cpp, *.js). Use '*' to include all files. Defaults to ['.*'], which ignores all hidden files and directories."
|
|
60
|
+
)
|
|
61
|
+
parser.add_argument(
|
|
62
|
+
"--ignore-file",
|
|
63
|
+
default=".src2mdignore",
|
|
64
|
+
help="File with patterns to ignore (e.g., *.py, *.cpp, *.js). Defaults to '.src2mdignore'.")
|
|
65
|
+
|
|
66
|
+
args = parser.parse_args()
|
|
67
|
+
|
|
68
|
+
path = Path(args.path)
|
|
69
|
+
if not path.is_dir():
|
|
70
|
+
print(f"Error: {args.path} is not a valid directory.", file=sys.stderr)
|
|
71
|
+
sys.exit(1)
|
|
72
|
+
|
|
73
|
+
ignore_pat = load_ignore_file(path / args.ignore_file)
|
|
74
|
+
if ignore_pat:
|
|
75
|
+
args.ignore_pat.extend(ignore_pat)
|
|
76
|
+
|
|
77
|
+
# If '*' is in src_pat, adjust to include all files except those in doc_pat
|
|
78
|
+
if '*' in args.src_pat:
|
|
79
|
+
# Replace '*' with a pattern that matches all files
|
|
80
|
+
args.src_pat = ['*']
|
|
81
|
+
|
|
82
|
+
try:
|
|
83
|
+
markdown = generate_markdown(
|
|
84
|
+
args.path,
|
|
85
|
+
doc_pat=args.doc_pat,
|
|
86
|
+
src_pat=args.src_pat,
|
|
87
|
+
ignore_pat=args.ignore_pat)
|
|
88
|
+
except Exception as e:
|
|
89
|
+
print(f"An error occurred while generating documentation: {e}", file=sys.stderr)
|
|
90
|
+
sys.exit(1)
|
|
91
|
+
|
|
92
|
+
try:
|
|
93
|
+
with open(args.out, 'w', encoding='utf-8') as f:
|
|
94
|
+
f.write(markdown)
|
|
95
|
+
print(f"Documentation generated and saved to '{args.out}'")
|
|
96
|
+
except IOError as e:
|
|
97
|
+
print(f"Error writing to output file {args.out}: {e}", file=sys.stderr)
|
|
98
|
+
sys.exit(1)
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
if __name__ == "__main__":
|
|
102
|
+
main()
|
|
@@ -0,0 +1,255 @@
|
|
|
1
|
+
import os
|
|
2
|
+
from pathlib import Path
|
|
3
|
+
import pypandoc
|
|
4
|
+
import sys
|
|
5
|
+
import fnmatch
|
|
6
|
+
import pathspec
|
|
7
|
+
|
|
8
|
+
default_langs = {
|
|
9
|
+
'.py': 'python',
|
|
10
|
+
'.cpp': 'cpp',
|
|
11
|
+
'.cxx': 'cpp',
|
|
12
|
+
'.cc': 'cpp',
|
|
13
|
+
'.c': 'c',
|
|
14
|
+
'.js': 'javascript',
|
|
15
|
+
'.java': 'java',
|
|
16
|
+
'.ts': 'typescript',
|
|
17
|
+
'.go': 'go',
|
|
18
|
+
'.rb': 'ruby',
|
|
19
|
+
'.php': 'php',
|
|
20
|
+
'.swift': 'swift',
|
|
21
|
+
'.kt': 'kotlin',
|
|
22
|
+
'.rs': 'rust',
|
|
23
|
+
'.lark': 'lark',
|
|
24
|
+
'.sh': 'bash',
|
|
25
|
+
'.bash': 'bash',
|
|
26
|
+
'.bat': 'batch',
|
|
27
|
+
'.sql': 'sql',
|
|
28
|
+
'.html': 'html',
|
|
29
|
+
'.css': 'css',
|
|
30
|
+
'.json': 'json',
|
|
31
|
+
'.xml': 'xml',
|
|
32
|
+
'.yaml': 'yaml',
|
|
33
|
+
'.yml': 'yaml',
|
|
34
|
+
'.md': 'markdown',
|
|
35
|
+
'.rst': 'markdown'
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
def convert_rst_to_md(rst_content):
|
|
39
|
+
"""
|
|
40
|
+
Convert reStructuredText content to Markdown using pypandoc.
|
|
41
|
+
|
|
42
|
+
Args:
|
|
43
|
+
rst_content (str): Content in reStructuredText format.
|
|
44
|
+
|
|
45
|
+
Returns:
|
|
46
|
+
str: Converted Markdown content.
|
|
47
|
+
"""
|
|
48
|
+
try:
|
|
49
|
+
md_content = pypandoc.convert_text(rst_content, 'md', format='rst')
|
|
50
|
+
return md_content
|
|
51
|
+
except (RuntimeError, OSError) as e:
|
|
52
|
+
print(f"Error converting RST to MD: {e}", file=sys.stderr)
|
|
53
|
+
return rst_content # Fallback to original content if conversion fails
|
|
54
|
+
|
|
55
|
+
def find_documentation_files(source_path, doc_patterns, already_included=None):
|
|
56
|
+
"""
|
|
57
|
+
Find documentation files within the source directory based on patterns.
|
|
58
|
+
|
|
59
|
+
Args:
|
|
60
|
+
source_path (Path): Path to the source directory.
|
|
61
|
+
doc_patterns (list): List of documentation file patterns.
|
|
62
|
+
already_included (set): Set of already included documentation files.
|
|
63
|
+
|
|
64
|
+
Returns:
|
|
65
|
+
list: List of documentation file paths.
|
|
66
|
+
"""
|
|
67
|
+
doc_files = []
|
|
68
|
+
for root, dirs, files in os.walk(source_path):
|
|
69
|
+
dirs[:] = [d for d in dirs if not d.startswith('.')]
|
|
70
|
+
|
|
71
|
+
for file in files:
|
|
72
|
+
if file.startswith('.'):
|
|
73
|
+
continue # Skip hidden files
|
|
74
|
+
file_path = Path(root) / file
|
|
75
|
+
relative_path = file_path.relative_to(source_path)
|
|
76
|
+
rel_path_str = str(relative_path)
|
|
77
|
+
for pattern in doc_patterns:
|
|
78
|
+
if fnmatch.fnmatch(file.lower(), pattern.lower()):
|
|
79
|
+
if already_included and rel_path_str in already_included:
|
|
80
|
+
continue # Skip if already included
|
|
81
|
+
doc_files.append(file_path)
|
|
82
|
+
if already_included is not None:
|
|
83
|
+
already_included.add(rel_path_str)
|
|
84
|
+
break # Stop checking other patterns once matched
|
|
85
|
+
return doc_files
|
|
86
|
+
|
|
87
|
+
|
|
88
|
+
def include_documentation_files(source_path, markdown_content, doc_patterns):
|
|
89
|
+
"""
|
|
90
|
+
Include documentation files into the markdown content, prioritizing README.*.
|
|
91
|
+
|
|
92
|
+
Args:
|
|
93
|
+
source_path (Path): Path to the source directory.
|
|
94
|
+
markdown_content (list): List to append markdown content to.
|
|
95
|
+
doc_patterns (list): List of documentation file patterns.
|
|
96
|
+
"""
|
|
97
|
+
# Prioritize README.* files
|
|
98
|
+
prioritized_patterns = ['README.*']
|
|
99
|
+
other_patterns = [pattern for pattern in doc_patterns if pattern not in prioritized_patterns]
|
|
100
|
+
|
|
101
|
+
already_included = set()
|
|
102
|
+
prioritized_docs = find_documentation_files(source_path, prioritized_patterns, already_included)
|
|
103
|
+
other_docs = find_documentation_files(source_path, other_patterns, already_included)
|
|
104
|
+
|
|
105
|
+
# Combine with prioritized docs first
|
|
106
|
+
doc_files = prioritized_docs + other_docs
|
|
107
|
+
|
|
108
|
+
if not doc_files:
|
|
109
|
+
return
|
|
110
|
+
|
|
111
|
+
markdown_content.append("## Documentation Files\n")
|
|
112
|
+
markdown_content.append("\n")
|
|
113
|
+
|
|
114
|
+
for doc_file in doc_files:
|
|
115
|
+
rel_path = doc_file.relative_to(source_path)
|
|
116
|
+
markdown_content.append(f"### {rel_path}\n")
|
|
117
|
+
markdown_content.append("\n")
|
|
118
|
+
|
|
119
|
+
try:
|
|
120
|
+
with open(doc_file, 'r', encoding='utf-8') as f:
|
|
121
|
+
content = f.read()
|
|
122
|
+
except (IOError, UnicodeDecodeError) as e:
|
|
123
|
+
print(f"Error reading documentation file {doc_file}: {e}", file=sys.stderr)
|
|
124
|
+
content = f"Error reading file: {e}"
|
|
125
|
+
|
|
126
|
+
# Convert to Markdown if .rst
|
|
127
|
+
if doc_file.suffix.lower() == '.rst':
|
|
128
|
+
content = convert_rst_to_md(content)
|
|
129
|
+
|
|
130
|
+
markdown_content.append(content)
|
|
131
|
+
markdown_content.append("\n---\n\n")
|
|
132
|
+
|
|
133
|
+
|
|
134
|
+
def get_language_identifier(file_extension):
|
|
135
|
+
"""
|
|
136
|
+
Map file extension to language identifier for syntax highlighting in Markdown.
|
|
137
|
+
|
|
138
|
+
Args:
|
|
139
|
+
file_extension (str): File extension (e.g., '.py').
|
|
140
|
+
|
|
141
|
+
Returns:
|
|
142
|
+
str: Language identifier (e.g., 'python').
|
|
143
|
+
"""
|
|
144
|
+
|
|
145
|
+
return default_langs.get(file_extension.lower(), '')
|
|
146
|
+
|
|
147
|
+
|
|
148
|
+
def generate_markdown(path, doc_pat=None, src_pat=None, ignore_pat=None):
|
|
149
|
+
"""
|
|
150
|
+
Generate Markdown content from the source directory.
|
|
151
|
+
|
|
152
|
+
Args:
|
|
153
|
+
source_path (Path): Path to the source directory.
|
|
154
|
+
doc_patterns (list): List of documentation file patterns.
|
|
155
|
+
src_patterns (list): List of source file patterns.
|
|
156
|
+
ignore_patterns (list): List of patterns to ignore.
|
|
157
|
+
|
|
158
|
+
Returns:
|
|
159
|
+
str: Complete Markdown content.
|
|
160
|
+
"""
|
|
161
|
+
markdown_content = []
|
|
162
|
+
path = Path(path)
|
|
163
|
+
|
|
164
|
+
# Add project name as title
|
|
165
|
+
markdown_content.append(f"# Project: {path.name}\n\n")
|
|
166
|
+
|
|
167
|
+
# Define default documentation patterns
|
|
168
|
+
if doc_pat is None:
|
|
169
|
+
doc_pat = ['*.md', '*.rst']
|
|
170
|
+
|
|
171
|
+
# Include documentation files like README.md, README.rst, etc.
|
|
172
|
+
include_documentation_files(path, markdown_content, doc_pat)
|
|
173
|
+
|
|
174
|
+
# Collect documentation file paths to exclude them from source files
|
|
175
|
+
doc_files = find_documentation_files(path, doc_pat)
|
|
176
|
+
doc_file_paths = set(str(p.relative_to(path)) for p in doc_files)
|
|
177
|
+
|
|
178
|
+
# Traverse the directory
|
|
179
|
+
for root, dirs, files in os.walk(path):
|
|
180
|
+
# Modify dirs in-place to exclude hidden directories
|
|
181
|
+
dirs[:] = [d for d in dirs if not any(fnmatch.fnmatch(d, pattern) for pattern in ignore_pat)]
|
|
182
|
+
|
|
183
|
+
# Compute relative path from source_path
|
|
184
|
+
rel_dir = Path(root).relative_to(path)
|
|
185
|
+
# Determine heading level based on directory depth
|
|
186
|
+
if rel_dir == Path('.'):
|
|
187
|
+
heading_level = 2 # ## for top-level
|
|
188
|
+
else:
|
|
189
|
+
heading_level = 2 + len(rel_dir.parts) # Increase heading level based on depth
|
|
190
|
+
|
|
191
|
+
# Add directory heading
|
|
192
|
+
if rel_dir != Path('.'):
|
|
193
|
+
markdown_content.append(f"{'#' * heading_level} Directory: `{rel_dir}`\n\n")
|
|
194
|
+
|
|
195
|
+
for file in sorted(files):
|
|
196
|
+
#if file.startswith('.'):
|
|
197
|
+
# continue # Skip hidden files
|
|
198
|
+
file_path = Path(root) / file
|
|
199
|
+
rel_path = file_path.relative_to(path)
|
|
200
|
+
rel_path_str = str(rel_path)
|
|
201
|
+
|
|
202
|
+
if any(fnmatch.fnmatch(file, pattern) for pattern in ignore_pat):
|
|
203
|
+
continue # Skip this file
|
|
204
|
+
|
|
205
|
+
if rel_path_str in doc_file_paths:
|
|
206
|
+
continue # Skip documentation files
|
|
207
|
+
|
|
208
|
+
# Check if the file matches any source pattern
|
|
209
|
+
if not any(fnmatch.fnmatch(file, pattern) for pattern in src_pat):
|
|
210
|
+
continue
|
|
211
|
+
|
|
212
|
+
try:
|
|
213
|
+
with open(file_path, 'r', encoding='utf-8') as f:
|
|
214
|
+
source_code = f.read()
|
|
215
|
+
except (IOError, UnicodeDecodeError) as e:
|
|
216
|
+
print(f"Error reading source file {file_path}: {e}", file=sys.stderr)
|
|
217
|
+
continue
|
|
218
|
+
|
|
219
|
+
# Determine language identifier
|
|
220
|
+
lang = get_language_identifier(file_path.suffix)
|
|
221
|
+
|
|
222
|
+
# Add source file section
|
|
223
|
+
markdown_content.append(f"### Source File: `{rel_path}`\n\n")
|
|
224
|
+
|
|
225
|
+
# Include source code
|
|
226
|
+
markdown_content.append("#### Source Code\n")
|
|
227
|
+
markdown_content.append("\n```")
|
|
228
|
+
markdown_content.append(f"{lang}\n")
|
|
229
|
+
markdown_content.append(f"{source_code}\n")
|
|
230
|
+
markdown_content.append("```\n\n")
|
|
231
|
+
|
|
232
|
+
markdown_content.append("---\n\n")
|
|
233
|
+
|
|
234
|
+
return ''.join(markdown_content)
|
|
235
|
+
|
|
236
|
+
def load_ignore_file(path):
|
|
237
|
+
"""
|
|
238
|
+
Load patterns to ignore from a file. A reasonable default is
|
|
239
|
+
to ignore hidden files and directories. One could also copy
|
|
240
|
+
.gitignore or .dockerignore to .src2mdignore.
|
|
241
|
+
|
|
242
|
+
Args:
|
|
243
|
+
path (Path): Path to the ignore file.
|
|
244
|
+
|
|
245
|
+
Returns:
|
|
246
|
+
list: List of patterns to ignore
|
|
247
|
+
"""
|
|
248
|
+
pats = []
|
|
249
|
+
if os.path.exists(path):
|
|
250
|
+
with open(path, 'r', encoding='utf-8') as f:
|
|
251
|
+
for line in f:
|
|
252
|
+
line = line.strip()
|
|
253
|
+
if line and not line.startswith("#"):
|
|
254
|
+
pats.append(line)
|
|
255
|
+
return pats
|
|
@@ -0,0 +1,128 @@
|
|
|
1
|
+
Metadata-Version: 2.2
|
|
2
|
+
Name: src2md
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: A tool to convert source code directories into Markdown documentation.
|
|
5
|
+
Home-page: https://github.com/queelius/src2md
|
|
6
|
+
Author: Alex Towell
|
|
7
|
+
Author-email: lex@example.com
|
|
8
|
+
Classifier: Programming Language :: Python :: 3
|
|
9
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
10
|
+
Classifier: Operating System :: OS Independent
|
|
11
|
+
Requires-Python: >=3.6
|
|
12
|
+
Description-Content-Type: text/markdown
|
|
13
|
+
License-File: LICENSE
|
|
14
|
+
Requires-Dist: pypandoc
|
|
15
|
+
Requires-Dist: pathspec
|
|
16
|
+
Dynamic: author
|
|
17
|
+
Dynamic: author-email
|
|
18
|
+
Dynamic: classifier
|
|
19
|
+
Dynamic: description
|
|
20
|
+
Dynamic: description-content-type
|
|
21
|
+
Dynamic: home-page
|
|
22
|
+
Dynamic: requires-dist
|
|
23
|
+
Dynamic: requires-python
|
|
24
|
+
Dynamic: summary
|
|
25
|
+
|
|
26
|
+
# src2md
|
|
27
|
+
|
|
28
|
+

|
|
29
|
+

|
|
30
|
+

|
|
31
|
+
|
|
32
|
+
**src2md** is a versatile tool that converts source code directories into comprehensive Markdown documentation. It scans your project structure, includes documentation files like `README.md` or `README.rst`, and embeds source code snippets with proper syntax highlighting. Ideal for generating project overviews, documentation for large codebases, or providing context to language models.
|
|
33
|
+
|
|
34
|
+
## 🚀 **Features**
|
|
35
|
+
|
|
36
|
+
- **Automatic Documentation Aggregation**: Collects and prioritizes documentation files (e.g., `README.md`, `README.rst`).
|
|
37
|
+
- **Syntax-Highlighted Source Code**: Embeds source code files with appropriate language identifiers for Markdown.
|
|
38
|
+
- **Flexible File Patterns**: Customize which files to include as documentation or source code using patterns.
|
|
39
|
+
- **Ignore Hidden Files and Directories**: Automatically excludes hidden files and directories (those starting with `.`).
|
|
40
|
+
- **Leverages `.gitignore`**: Respects your `.gitignore` settings to exclude unwanted files.
|
|
41
|
+
- **Simple CLI Interface**: Easy-to-use command-line interface for generating documentation.
|
|
42
|
+
|
|
43
|
+
## 📦 **Installation**
|
|
44
|
+
|
|
45
|
+
You can install `src2md` via [PyPI](https://pypi.org/) using `pip`:
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
pip install src2md
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Alternatively, install it directly from the source:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
git clone https://github.com/yourusername/src2md.git
|
|
55
|
+
cd src2md
|
|
56
|
+
pip install .
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## 🛠️ Usage
|
|
60
|
+
Once installed, you can use src2md from the command line to generate Markdown documentation for your project.
|
|
61
|
+
|
|
62
|
+
### Basic Command
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
src2md path/to/source -o output.md
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
- `path/to/source`: Path to your project's root directory.
|
|
69
|
+
- `-o output.md`: (Optional) Specifies the output Markdown file name. Defaults to project_documentation.md.
|
|
70
|
+
|
|
71
|
+
## Advanced Options
|
|
72
|
+
|
|
73
|
+
- **Specify Documentation Patterns**
|
|
74
|
+
|
|
75
|
+
Include additional documentation file types.
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
src2md path/to/source -o output.md --doc-patterns '*.md' '*.rst' '*.lark'
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
- **Specify Source Code Patterns**
|
|
82
|
+
|
|
83
|
+
Customize which source files to include. Use '*' to include all files.
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
src2md path/to/source -o output.md --src-patterns '*.py' '*.js'
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
To include all files as source code:
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
src2md path/to/source -o output.md --src-patterns '*'
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
- **Add Additional Ignore Patterns**
|
|
96
|
+
|
|
97
|
+
Exclude specific files or directories.
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
src2md path/to/source -o output.md --ignore-patterns '*.pyc' 'build/' 'dist/'
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
- **Use a Custom Ignore File**
|
|
104
|
+
|
|
105
|
+
Specify a custom ignore file.
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
src2md path/to/source -o output.md --ignore-file .src2mdignore
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
- **Disable .gitignore Usage**
|
|
112
|
+
|
|
113
|
+
Do not use .gitignore for ignoring files.
|
|
114
|
+
|
|
115
|
+
```bash
|
|
116
|
+
src2md path/to/source -o output.md --no-use-gitignore
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
## Full Example
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
src2md ./my_project -o my_project_docs.md \
|
|
123
|
+
--doc-patterns '*.md' '*.rst' '*.lark' \
|
|
124
|
+
--src-patterns '*.py' '*.cpp' '*.js' \
|
|
125
|
+
--ignore-patterns '*.pyc' 'build/' 'dist/'
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
This command generates my_project_docs.md by including all .md, .rst, and .lark documentation files, embedding .py, .cpp, and .js source files, and excluding any .pyc files and directories named build or dist.
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
LICENSE
|
|
2
|
+
README.md
|
|
3
|
+
setup.py
|
|
4
|
+
src2md/__init__.py
|
|
5
|
+
src2md/cli.py
|
|
6
|
+
src2md/utils.py
|
|
7
|
+
src2md.egg-info/PKG-INFO
|
|
8
|
+
src2md.egg-info/SOURCES.txt
|
|
9
|
+
src2md.egg-info/dependency_links.txt
|
|
10
|
+
src2md.egg-info/entry_points.txt
|
|
11
|
+
src2md.egg-info/requires.txt
|
|
12
|
+
src2md.egg-info/top_level.txt
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
src2md
|