nbdevAuto 0.3.61__tar.gz → 0.3.64__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- nbdevauto-0.3.64/PKG-INFO +124 -0
- nbdevauto-0.3.64/README.md +94 -0
- nbdevauto-0.3.64/nbdevAuto/__init__.py +1 -0
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/nbdevAuto/_modidx.py +6 -5
- nbdevauto-0.3.64/nbdevAuto/automate.py +209 -0
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/nbdevAuto/github.py +26 -11
- nbdevauto-0.3.64/nbdevAuto.egg-info/PKG-INFO +124 -0
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/nbdevAuto.egg-info/entry_points.txt +0 -5
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/nbdevAuto.egg-info/requires.txt +1 -0
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/pyproject.toml +1 -6
- nbdevauto-0.3.61/PKG-INFO +0 -134
- nbdevauto-0.3.61/README.md +0 -105
- nbdevauto-0.3.61/nbdevAuto/__init__.py +0 -1
- nbdevauto-0.3.61/nbdevAuto/automate.py +0 -141
- nbdevauto-0.3.61/nbdevAuto.egg-info/PKG-INFO +0 -134
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/LICENSE +0 -0
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/MANIFEST.in +0 -0
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/nbdevAuto/core.py +0 -0
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/nbdevAuto/functions.py +0 -0
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/nbdevAuto/pdf.py +0 -0
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/nbdevAuto.egg-info/SOURCES.txt +0 -0
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/nbdevAuto.egg-info/dependency_links.txt +0 -0
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/nbdevAuto.egg-info/top_level.txt +0 -0
- {nbdevauto-0.3.61 → nbdevauto-0.3.64}/setup.cfg +0 -0
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: nbdevAuto
|
|
3
|
+
Version: 0.3.64
|
|
4
|
+
Summary: automating nbdev
|
|
5
|
+
Author-email: Benedict Thekkel <bthekkel1@gmail.com>
|
|
6
|
+
License-Expression: Apache-2.0
|
|
7
|
+
Project-URL: Repository, https://github.com/bthek1/nbdevAuto
|
|
8
|
+
Project-URL: Documentation, https://bthek1.github.io/nbdevAuto
|
|
9
|
+
Project-URL: Issues, https://github.com/bthek1/nbdevAuto/issues
|
|
10
|
+
Keywords: nbdev,jupyter,notebook,python
|
|
11
|
+
Classifier: Natural Language :: English
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Development Status :: 3 - Alpha
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
+
Requires-Python: >=3.10
|
|
22
|
+
Description-Content-Type: text/markdown
|
|
23
|
+
License-File: LICENSE
|
|
24
|
+
Requires-Dist: graphviz
|
|
25
|
+
Requires-Dist: fastcore
|
|
26
|
+
Requires-Dist: rich
|
|
27
|
+
Provides-Extra: gh
|
|
28
|
+
Requires-Dist: githubkit>=0.12; extra == "gh"
|
|
29
|
+
Dynamic: license-file
|
|
30
|
+
|
|
31
|
+
# nbdevAuto
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
|
|
35
|
+
|
|
36
|
+
Unlike most sites in this collection, this one documents an **installable library** rather than a topic. `nbdevAuto` exists because the nbdev release cycle is a fixed sequence of commands typed in the same order every time: export the notebooks, run the tests, clean them, bump the version, commit, push, build the docs. Each step is one nbdev call, and forgetting one produces a confusing failure two steps later.
|
|
37
|
+
|
|
38
|
+
The library collapses that sequence into single-word shell commands. It is what `just upload` and `just full_upload` invoke under the hood for every submodule in the parent Knowledge repo.
|
|
39
|
+
|
|
40
|
+
------------------------------------------------------------------------
|
|
41
|
+
|
|
42
|
+
## Install
|
|
43
|
+
|
|
44
|
+
``` sh
|
|
45
|
+
pip install nbdevAuto # the publish-cycle commands and the helpers
|
|
46
|
+
pip install 'nbdevAuto[gh]' # also installs githubkit, which `ghstatus` needs
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Requires Python 3.10 or newer.
|
|
50
|
+
|
|
51
|
+
## Use
|
|
52
|
+
|
|
53
|
+
Every exported function is also a console script, so the common case is a bare word in the terminal at the root of an nbdev project:
|
|
54
|
+
|
|
55
|
+
``` sh
|
|
56
|
+
upload -m "commit message" # export, test, clean, then add/commit/push
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Or import the helpers into a notebook:
|
|
60
|
+
|
|
61
|
+
``` python
|
|
62
|
+
from nbdevAuto.functions import download_search_images, classify_images
|
|
63
|
+
from nbdevAuto.pdf import PDFreader
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
------------------------------------------------------------------------
|
|
67
|
+
|
|
68
|
+
## Contents
|
|
69
|
+
|
|
70
|
+
| Page | Covers |
|
|
71
|
+
|----|----|
|
|
72
|
+
| [Functions](00_Functions.ipynb) | Dataset and image helpers: reading a list out of a text file, downloading single images or whole search-driven datasets, verifying and resizing what came back, building the folder layout a classifier expects, running a fastai learner over one image, Kaggle competition and dataset shortcuts, and [`graph`](https://bthek1.github.io/nbdevAuto/functions.html#graph), a `graphviz.Digraph` subclass preloaded with a rounded, filled style |
|
|
73
|
+
| [Automate](01_Automate.ipynb) | The command line surface: the prep/commit/push pipeline, the GitHub and PyPI release halves, and the help output that lists them all |
|
|
74
|
+
| [PDF Reader](02_PDF_reader.ipynb) | [`PDFreader`](https://bthek1.github.io/nbdevAuto/pdf_reader.html#pdfreader), a class that converts a PDF through `pdf2image` and renders page ranges inline when you slice it (`pdf[0:5]`) |
|
|
75
|
+
| [GitHub](03_GitHub.ipynb) | [`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus): reading `.gitmodules`, resolving a token, and asking the GitHub API about every submodule at once - CI runs, Pages deploys, Pages builds, remote `main` shas - plus the `--report` and `--audit` views and the exit codes that keep “absent” apart from “cannot ask” |
|
|
76
|
+
|
|
77
|
+
------------------------------------------------------------------------
|
|
78
|
+
|
|
79
|
+
## The Command Line Surface
|
|
80
|
+
|
|
81
|
+
Installing the package puts these on your `PATH`. Most are ordinary functions in [Automate](01_Automate.ipynb), exposed as scripts through `[project.scripts]`; [`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus) comes from [GitHub](03_GitHub.ipynb).
|
|
82
|
+
|
|
83
|
+
| Command | Does |
|
|
84
|
+
|----|----|
|
|
85
|
+
| [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) | Bump the version, then export, test and clean the notebooks, refreshing `_quarto.yml` and the README |
|
|
86
|
+
| [`gacp`](https://bthek1.github.io/nbdevAuto/automate.html#gacp) | git add, commit, and push. Without `-m` the commit message is the status output |
|
|
87
|
+
| [`status`](https://bthek1.github.io/nbdevAuto/automate.html#status) | Show the working tree state |
|
|
88
|
+
| [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) | [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) then [`gacp`](https://bthek1.github.io/nbdevAuto/automate.html#gacp), the one you actually type |
|
|
89
|
+
| `gitrelease` | Bump the **minor** version, then tag and create a GitHub release |
|
|
90
|
+
| `piprelease` | Build the sdist and wheel, then upload to PyPI with twine |
|
|
91
|
+
| [`release`](https://bthek1.github.io/nbdevAuto/automate.html#release) | `gitrelease` then `piprelease` |
|
|
92
|
+
| [`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus) | Remote status for every submodule of an nbdev superproject, from the GitHub API. `--report` for the sync view, `--audit` to check `.gitmodules` against the repos the account owns, `--json` for machine output |
|
|
93
|
+
| `h` | Print the list of available commands |
|
|
94
|
+
|
|
95
|
+
Both [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) and [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) take `-p` to choose which version part to increment; it defaults to 2, the patch, so every [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) ships a new version.
|
|
96
|
+
|
|
97
|
+
`piprelease` is the manual path. Pushing to `main` publishes to PyPI on its own: `publish.yaml` runs after CI passes, and skips quietly when the version is already there.
|
|
98
|
+
|
|
99
|
+
[`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus) needs the `gh` extra (`pip install 'nbdevAuto[gh]'`), which brings in `githubkit`. It is an extra rather than a dependency because githubkit pulls pydantic, httpx and hishel behind it, which is a lot of weight for anyone who only wants [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload). Without it the command exits 3 with an install hint rather than a traceback. It takes its token from `$GITHUB_TOKEN` / `$GH_TOKEN`, then `gh auth token`.
|
|
100
|
+
|
|
101
|
+
------------------------------------------------------------------------
|
|
102
|
+
|
|
103
|
+
## Where It Is Used
|
|
104
|
+
|
|
105
|
+
The parent Knowledge repo’s `justfile` calls [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) once per submodule. `just upload` runs it only where the working tree is dirty, `just full_upload` runs it everywhere. That is the main consumer, so a change to `automate.py` affects the publish path for every site in the collection.
|
|
106
|
+
|
|
107
|
+
[`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus) is the other half of that loop. [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) pushes to 19 repos and then goes blind: whether each repo’s CI passed and whether its Pages deploy actually landed is invisible from the terminal. `just ci`, `just report` and `just audit` in the parent repo are all thin wrappers around this one command, which answers for the whole fleet in about 1.5 seconds instead of 19 browser tabs.
|
|
108
|
+
|
|
109
|
+
## Releases
|
|
110
|
+
|
|
111
|
+
This package publishes itself. A `publish.yaml` workflow runs after CI passes on `main` and uploads to PyPI through Trusted Publishing, so no API token is stored in the repository. It reads `__version__` out of `nbdevAuto/__init__.py` and asks PyPI whether that version already exists: a new version is published, an existing one is skipped without failing the build.
|
|
112
|
+
|
|
113
|
+
Because [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) bumps the patch version on every push, an ordinary commit made with [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) ships a release. A plain `git commit` leaves the version alone and the workflow skips. `gitrelease`, which bumps the minor version and creates the GitHub release and tag, stays a deliberate manual step.
|
|
114
|
+
|
|
115
|
+
------------------------------------------------------------------------
|
|
116
|
+
|
|
117
|
+
## Not Covered Yet
|
|
118
|
+
|
|
119
|
+
- **No test notebook for the helpers.** `03_GitHub.ipynb` carries unit tests for its pure functions, but the dataset and image helpers are exercised only by being used, so a broken one surfaces in a downstream repo rather than here.
|
|
120
|
+
- **Undeclared dependencies.** `pyproject.toml` declares `graphviz` and `fastcore`, plus `githubkit` behind the `gh` extra. The helpers additionally import `fastai`, `fastbook`, `fastdownload`, `PIL`, `matplotlib`, `tqdm`, `pdf2image` (with poppler behind it), `nbdev` and `kaggle` (which also needs its credentials). All of those are imported lazily inside the function that needs them, so installing the package and running the console scripts still works - only the specific helper you call fails. Install them yourself.
|
|
121
|
+
- **`core.py` is an empty nbdev stub** (a single `foo`) left over from `00_core.ipynb`, which no longer exists.
|
|
122
|
+
- The docstrings are one-liners, so the rendered API pages are thin. The notebooks carry the real explanation.
|
|
123
|
+
|
|
124
|
+
------------------------------------------------------------------------
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# nbdevAuto
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
|
|
5
|
+
|
|
6
|
+
Unlike most sites in this collection, this one documents an **installable library** rather than a topic. `nbdevAuto` exists because the nbdev release cycle is a fixed sequence of commands typed in the same order every time: export the notebooks, run the tests, clean them, bump the version, commit, push, build the docs. Each step is one nbdev call, and forgetting one produces a confusing failure two steps later.
|
|
7
|
+
|
|
8
|
+
The library collapses that sequence into single-word shell commands. It is what `just upload` and `just full_upload` invoke under the hood for every submodule in the parent Knowledge repo.
|
|
9
|
+
|
|
10
|
+
------------------------------------------------------------------------
|
|
11
|
+
|
|
12
|
+
## Install
|
|
13
|
+
|
|
14
|
+
``` sh
|
|
15
|
+
pip install nbdevAuto # the publish-cycle commands and the helpers
|
|
16
|
+
pip install 'nbdevAuto[gh]' # also installs githubkit, which `ghstatus` needs
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Requires Python 3.10 or newer.
|
|
20
|
+
|
|
21
|
+
## Use
|
|
22
|
+
|
|
23
|
+
Every exported function is also a console script, so the common case is a bare word in the terminal at the root of an nbdev project:
|
|
24
|
+
|
|
25
|
+
``` sh
|
|
26
|
+
upload -m "commit message" # export, test, clean, then add/commit/push
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
Or import the helpers into a notebook:
|
|
30
|
+
|
|
31
|
+
``` python
|
|
32
|
+
from nbdevAuto.functions import download_search_images, classify_images
|
|
33
|
+
from nbdevAuto.pdf import PDFreader
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
------------------------------------------------------------------------
|
|
37
|
+
|
|
38
|
+
## Contents
|
|
39
|
+
|
|
40
|
+
| Page | Covers |
|
|
41
|
+
|----|----|
|
|
42
|
+
| [Functions](00_Functions.ipynb) | Dataset and image helpers: reading a list out of a text file, downloading single images or whole search-driven datasets, verifying and resizing what came back, building the folder layout a classifier expects, running a fastai learner over one image, Kaggle competition and dataset shortcuts, and [`graph`](https://bthek1.github.io/nbdevAuto/functions.html#graph), a `graphviz.Digraph` subclass preloaded with a rounded, filled style |
|
|
43
|
+
| [Automate](01_Automate.ipynb) | The command line surface: the prep/commit/push pipeline, the GitHub and PyPI release halves, and the help output that lists them all |
|
|
44
|
+
| [PDF Reader](02_PDF_reader.ipynb) | [`PDFreader`](https://bthek1.github.io/nbdevAuto/pdf_reader.html#pdfreader), a class that converts a PDF through `pdf2image` and renders page ranges inline when you slice it (`pdf[0:5]`) |
|
|
45
|
+
| [GitHub](03_GitHub.ipynb) | [`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus): reading `.gitmodules`, resolving a token, and asking the GitHub API about every submodule at once - CI runs, Pages deploys, Pages builds, remote `main` shas - plus the `--report` and `--audit` views and the exit codes that keep “absent” apart from “cannot ask” |
|
|
46
|
+
|
|
47
|
+
------------------------------------------------------------------------
|
|
48
|
+
|
|
49
|
+
## The Command Line Surface
|
|
50
|
+
|
|
51
|
+
Installing the package puts these on your `PATH`. Most are ordinary functions in [Automate](01_Automate.ipynb), exposed as scripts through `[project.scripts]`; [`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus) comes from [GitHub](03_GitHub.ipynb).
|
|
52
|
+
|
|
53
|
+
| Command | Does |
|
|
54
|
+
|----|----|
|
|
55
|
+
| [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) | Bump the version, then export, test and clean the notebooks, refreshing `_quarto.yml` and the README |
|
|
56
|
+
| [`gacp`](https://bthek1.github.io/nbdevAuto/automate.html#gacp) | git add, commit, and push. Without `-m` the commit message is the status output |
|
|
57
|
+
| [`status`](https://bthek1.github.io/nbdevAuto/automate.html#status) | Show the working tree state |
|
|
58
|
+
| [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) | [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) then [`gacp`](https://bthek1.github.io/nbdevAuto/automate.html#gacp), the one you actually type |
|
|
59
|
+
| `gitrelease` | Bump the **minor** version, then tag and create a GitHub release |
|
|
60
|
+
| `piprelease` | Build the sdist and wheel, then upload to PyPI with twine |
|
|
61
|
+
| [`release`](https://bthek1.github.io/nbdevAuto/automate.html#release) | `gitrelease` then `piprelease` |
|
|
62
|
+
| [`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus) | Remote status for every submodule of an nbdev superproject, from the GitHub API. `--report` for the sync view, `--audit` to check `.gitmodules` against the repos the account owns, `--json` for machine output |
|
|
63
|
+
| `h` | Print the list of available commands |
|
|
64
|
+
|
|
65
|
+
Both [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) and [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) take `-p` to choose which version part to increment; it defaults to 2, the patch, so every [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) ships a new version.
|
|
66
|
+
|
|
67
|
+
`piprelease` is the manual path. Pushing to `main` publishes to PyPI on its own: `publish.yaml` runs after CI passes, and skips quietly when the version is already there.
|
|
68
|
+
|
|
69
|
+
[`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus) needs the `gh` extra (`pip install 'nbdevAuto[gh]'`), which brings in `githubkit`. It is an extra rather than a dependency because githubkit pulls pydantic, httpx and hishel behind it, which is a lot of weight for anyone who only wants [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload). Without it the command exits 3 with an install hint rather than a traceback. It takes its token from `$GITHUB_TOKEN` / `$GH_TOKEN`, then `gh auth token`.
|
|
70
|
+
|
|
71
|
+
------------------------------------------------------------------------
|
|
72
|
+
|
|
73
|
+
## Where It Is Used
|
|
74
|
+
|
|
75
|
+
The parent Knowledge repo’s `justfile` calls [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) once per submodule. `just upload` runs it only where the working tree is dirty, `just full_upload` runs it everywhere. That is the main consumer, so a change to `automate.py` affects the publish path for every site in the collection.
|
|
76
|
+
|
|
77
|
+
[`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus) is the other half of that loop. [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) pushes to 19 repos and then goes blind: whether each repo’s CI passed and whether its Pages deploy actually landed is invisible from the terminal. `just ci`, `just report` and `just audit` in the parent repo are all thin wrappers around this one command, which answers for the whole fleet in about 1.5 seconds instead of 19 browser tabs.
|
|
78
|
+
|
|
79
|
+
## Releases
|
|
80
|
+
|
|
81
|
+
This package publishes itself. A `publish.yaml` workflow runs after CI passes on `main` and uploads to PyPI through Trusted Publishing, so no API token is stored in the repository. It reads `__version__` out of `nbdevAuto/__init__.py` and asks PyPI whether that version already exists: a new version is published, an existing one is skipped without failing the build.
|
|
82
|
+
|
|
83
|
+
Because [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) bumps the patch version on every push, an ordinary commit made with [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) ships a release. A plain `git commit` leaves the version alone and the workflow skips. `gitrelease`, which bumps the minor version and creates the GitHub release and tag, stays a deliberate manual step.
|
|
84
|
+
|
|
85
|
+
------------------------------------------------------------------------
|
|
86
|
+
|
|
87
|
+
## Not Covered Yet
|
|
88
|
+
|
|
89
|
+
- **No test notebook for the helpers.** `03_GitHub.ipynb` carries unit tests for its pure functions, but the dataset and image helpers are exercised only by being used, so a broken one surfaces in a downstream repo rather than here.
|
|
90
|
+
- **Undeclared dependencies.** `pyproject.toml` declares `graphviz` and `fastcore`, plus `githubkit` behind the `gh` extra. The helpers additionally import `fastai`, `fastbook`, `fastdownload`, `PIL`, `matplotlib`, `tqdm`, `pdf2image` (with poppler behind it), `nbdev` and `kaggle` (which also needs its credentials). All of those are imported lazily inside the function that needs them, so installing the package and running the console scripts still works - only the specific helper you call fails. Install them yourself.
|
|
91
|
+
- **`core.py` is an empty nbdev stub** (a single `foo`) left over from `00_core.ipynb`, which no longer exists.
|
|
92
|
+
- The docstrings are one-liners, so the rendered API pages are thin. The notebooks carry the real explanation.
|
|
93
|
+
|
|
94
|
+
------------------------------------------------------------------------
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
__version__ = "0.3.64"
|
|
@@ -5,18 +5,18 @@ d = { 'settings': { 'branch': 'main',
|
|
|
5
5
|
'doc_host': 'https://bthek1.github.io',
|
|
6
6
|
'git_url': 'https://github.com/bthek1/nbdevAuto',
|
|
7
7
|
'lib_path': 'nbdevAuto'},
|
|
8
|
-
'syms': { 'nbdevAuto.automate': { 'nbdevAuto.automate.
|
|
9
|
-
'nbdevAuto.automate.
|
|
8
|
+
'syms': { 'nbdevAuto.automate': { 'nbdevAuto.automate._banner': ('automate.html#_banner', 'nbdevAuto/automate.py'),
|
|
9
|
+
'nbdevAuto.automate._done': ('automate.html#_done', 'nbdevAuto/automate.py'),
|
|
10
|
+
'nbdevAuto.automate._dur': ('automate.html#_dur', 'nbdevAuto/automate.py'),
|
|
11
|
+
'nbdevAuto.automate._status_table': ('automate.html#_status_table', 'nbdevAuto/automate.py'),
|
|
12
|
+
'nbdevAuto.automate._step': ('automate.html#_step', 'nbdevAuto/automate.py'),
|
|
10
13
|
'nbdevAuto.automate.gacp': ('automate.html#gacp', 'nbdevAuto/automate.py'),
|
|
11
|
-
'nbdevAuto.automate.hello': ('automate.html#hello', 'nbdevAuto/automate.py'),
|
|
12
14
|
'nbdevAuto.automate.help_output': ('automate.html#help_output', 'nbdevAuto/automate.py'),
|
|
13
15
|
'nbdevAuto.automate.prep': ('automate.html#prep', 'nbdevAuto/automate.py'),
|
|
14
|
-
'nbdevAuto.automate.reinstall': ('automate.html#reinstall', 'nbdevAuto/automate.py'),
|
|
15
16
|
'nbdevAuto.automate.release': ('automate.html#release', 'nbdevAuto/automate.py'),
|
|
16
17
|
'nbdevAuto.automate.release_git': ('automate.html#release_git', 'nbdevAuto/automate.py'),
|
|
17
18
|
'nbdevAuto.automate.release_pypi': ('automate.html#release_pypi', 'nbdevAuto/automate.py'),
|
|
18
19
|
'nbdevAuto.automate.status': ('automate.html#status', 'nbdevAuto/automate.py'),
|
|
19
|
-
'nbdevAuto.automate.update': ('automate.html#update', 'nbdevAuto/automate.py'),
|
|
20
20
|
'nbdevAuto.automate.upload': ('automate.html#upload', 'nbdevAuto/automate.py')},
|
|
21
21
|
'nbdevAuto.functions': { 'nbdevAuto.functions.classify_images': ('functions.html#classify_images', 'nbdevAuto/functions.py'),
|
|
22
22
|
'nbdevAuto.functions.create_data_folder': ( 'functions.html#create_data_folder',
|
|
@@ -49,6 +49,7 @@ d = { 'settings': { 'branch': 'main',
|
|
|
49
49
|
'nbdevAuto.github.ahead_behind': ('github.html#ahead_behind', 'nbdevAuto/github.py'),
|
|
50
50
|
'nbdevAuto.github.api_error': ('github.html#api_error', 'nbdevAuto/github.py'),
|
|
51
51
|
'nbdevAuto.github.audit_result': ('github.html#audit_result', 'nbdevAuto/github.py'),
|
|
52
|
+
'nbdevAuto.github.client': ('github.html#client', 'nbdevAuto/github.py'),
|
|
52
53
|
'nbdevAuto.github.failures': ('github.html#failures', 'nbdevAuto/github.py'),
|
|
53
54
|
'nbdevAuto.github.find_root': ('github.html#find_root', 'nbdevAuto/github.py'),
|
|
54
55
|
'nbdevAuto.github.format_audit': ('github.html#format_audit', 'nbdevAuto/github.py'),
|
|
@@ -0,0 +1,209 @@
|
|
|
1
|
+
"""Automate
|
|
2
|
+
|
|
3
|
+
Docs: https://bthek1.github.io/nbdevAuto/automate.html.md"""
|
|
4
|
+
|
|
5
|
+
# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/01_Automate.ipynb.
|
|
6
|
+
|
|
7
|
+
# %% auto #0
|
|
8
|
+
__all__ = ['prep', 'gacp', 'status', 'upload', 'release_git', 'release_pypi', 'release', 'help_output']
|
|
9
|
+
|
|
10
|
+
# %% ../nbs/01_Automate.ipynb #6d1499d1-aa55-47d8-92fd-fa056b1fb135
|
|
11
|
+
from fastcore.script import *
|
|
12
|
+
from rich import box
|
|
13
|
+
from rich.console import Console
|
|
14
|
+
from rich.markup import escape
|
|
15
|
+
from rich.table import Table
|
|
16
|
+
|
|
17
|
+
# %% ../nbs/01_Automate.ipynb #7c1e5a90-3f52-4d21-9b6a-2c0f4e8d1a37
|
|
18
|
+
from contextlib import contextmanager
|
|
19
|
+
from time import perf_counter
|
|
20
|
+
|
|
21
|
+
from nbdevAuto import __version__
|
|
22
|
+
|
|
23
|
+
console = Console()
|
|
24
|
+
|
|
25
|
+
# `git status -s` codes: colour, and the word for the third column.
|
|
26
|
+
_CODE_STYLE = {"M":"yellow", "A":"green", "D":"red", "R":"cyan", "C":"cyan", "U":"magenta", "?":"dim"}
|
|
27
|
+
_CODE_LABEL = {"M":"modified", "A":"added", "D":"deleted", "R":"renamed", "C":"copied",
|
|
28
|
+
"U":"conflict", "?":"untracked"}
|
|
29
|
+
|
|
30
|
+
def _dur(t0):
|
|
31
|
+
"Elapsed time since `t0`, at a readable scale"
|
|
32
|
+
s = perf_counter() - t0
|
|
33
|
+
if s < 1: return f"{s*1000:.0f}ms"
|
|
34
|
+
if s < 60: return f"{s:.1f}s"
|
|
35
|
+
return f"{int(s)//60}m{int(s)%60:02d}s"
|
|
36
|
+
|
|
37
|
+
def _banner(cmd):
|
|
38
|
+
"Command header, with the package version alongside"
|
|
39
|
+
console.rule(f"[bold cyan]{cmd}[/bold cyan] [dim]nbdevAuto {__version__}[/dim]", style="cyan")
|
|
40
|
+
|
|
41
|
+
def _done(cmd, t0):
|
|
42
|
+
"Closing summary for a multi-stage command"
|
|
43
|
+
console.rule(f"[green]{cmd} finished[/green] [dim]in {_dur(t0)}[/dim]", style="green")
|
|
44
|
+
|
|
45
|
+
@contextmanager
|
|
46
|
+
def _step(n, total, label):
|
|
47
|
+
"Run one stage, reporting its outcome and duration. Re-raises so nothing is swallowed"
|
|
48
|
+
console.print(f"[dim]{n}/{total}[/dim] [bold]{label}[/bold]")
|
|
49
|
+
t0 = perf_counter()
|
|
50
|
+
try:
|
|
51
|
+
yield
|
|
52
|
+
except BaseException:
|
|
53
|
+
console.print(f" [bold red]FAIL[/bold red] [dim]{_dur(t0)}[/dim]\n")
|
|
54
|
+
raise
|
|
55
|
+
console.print(f" [green]OK[/green] [dim]{_dur(t0)}[/dim]\n")
|
|
56
|
+
|
|
57
|
+
def _status_table(porcelain):
|
|
58
|
+
"Render `git status -s` output as a table, one row per path"
|
|
59
|
+
t = Table(box=box.SIMPLE, show_header=False, pad_edge=False, expand=False)
|
|
60
|
+
t.add_column("code", no_wrap=True, justify="right")
|
|
61
|
+
t.add_column("path", overflow="fold")
|
|
62
|
+
t.add_column("what", style="dim", no_wrap=True)
|
|
63
|
+
n = 0
|
|
64
|
+
for line in porcelain.splitlines():
|
|
65
|
+
if not line.strip(): continue
|
|
66
|
+
code, path = line[:2].strip(), line[3:]
|
|
67
|
+
style = _CODE_STYLE.get(code[:1], "white")
|
|
68
|
+
t.add_row(f"[{style}]{code}[/{style}]", escape(path), _CODE_LABEL.get(code[:1], ""))
|
|
69
|
+
n += 1
|
|
70
|
+
return t, n
|
|
71
|
+
|
|
72
|
+
# %% ../nbs/01_Automate.ipynb #3a6f81d2-5c47-4e19-8b03-9d2a7f6c0e15
|
|
73
|
+
@call_parse
|
|
74
|
+
def prep(
|
|
75
|
+
p:int = 2, # Increment Part
|
|
76
|
+
):
|
|
77
|
+
"Bump version part `p`, then export, test and clean the notebooks, refreshing _quarto.yml and README"
|
|
78
|
+
|
|
79
|
+
import nbdev.test, nbdev.clean, nbdev.quarto, nbdev.release
|
|
80
|
+
_banner("prep")
|
|
81
|
+
t0 = perf_counter()
|
|
82
|
+
with _step(1, 6, f"bump version (part {p})"):
|
|
83
|
+
nbdev.release.nbdev_bump_version(p)
|
|
84
|
+
with _step(2, 6, "nbdev_export"):
|
|
85
|
+
nbdev.quarto.nbdev_export.__wrapped__()
|
|
86
|
+
with _step(3, 6, "nbdev_test"):
|
|
87
|
+
nbdev.test.nbdev_test.__wrapped__(
|
|
88
|
+
n_workers = 8, # Number of workers
|
|
89
|
+
timing = True, # Time each notebook to see which are slow
|
|
90
|
+
)
|
|
91
|
+
with _step(4, 6, "nbdev_clean"):
|
|
92
|
+
nbdev.clean.nbdev_clean.__wrapped__()
|
|
93
|
+
with _step(5, 6, "refresh_quarto_yml"):
|
|
94
|
+
nbdev.quarto.refresh_quarto_yml()
|
|
95
|
+
with _step(6, 6, "nbdev_readme"):
|
|
96
|
+
nbdev.quarto.nbdev_readme.__wrapped__(chk_time=True)
|
|
97
|
+
_done("prep", t0)
|
|
98
|
+
|
|
99
|
+
# %% ../nbs/01_Automate.ipynb #f66ef35f-b7fa-4039-aa07-e9b782d204bb
|
|
100
|
+
@call_parse
|
|
101
|
+
def gacp(
|
|
102
|
+
m:str = '', # Commit message
|
|
103
|
+
):
|
|
104
|
+
"git add, commit and push. Without `-m` the commit message is the status output"
|
|
105
|
+
|
|
106
|
+
import subprocess
|
|
107
|
+
_banner("gacp")
|
|
108
|
+
t0 = perf_counter()
|
|
109
|
+
with _step(1, 3, "git add"):
|
|
110
|
+
subprocess.run(["git", "add", "."])
|
|
111
|
+
status = subprocess.check_output(["git", "status", "-s"]).decode('utf-8')
|
|
112
|
+
if status.strip():
|
|
113
|
+
table, n = _status_table(status)
|
|
114
|
+
console.print(table)
|
|
115
|
+
console.print(f"[dim]{n} path{'' if n == 1 else 's'} staged[/dim]\n")
|
|
116
|
+
else:
|
|
117
|
+
console.print("[dim]nothing staged, the tree is clean[/dim]\n")
|
|
118
|
+
|
|
119
|
+
msg = m if m != '' else status
|
|
120
|
+
with _step(2, 3, "git commit"):
|
|
121
|
+
subprocess.run(["git", "commit", "-m", msg])
|
|
122
|
+
with _step(3, 3, "git push"):
|
|
123
|
+
subprocess.run(["git", "push"])
|
|
124
|
+
_done("gacp", t0)
|
|
125
|
+
|
|
126
|
+
# %% ../nbs/01_Automate.ipynb #88017a1b-55c1-4833-aeb6-277a7e5b6314
|
|
127
|
+
def status():
|
|
128
|
+
"Show the working tree state"
|
|
129
|
+
import subprocess
|
|
130
|
+
|
|
131
|
+
def _git(*a, **kw): return subprocess.check_output(["git", *a], **kw).decode('utf-8').strip()
|
|
132
|
+
|
|
133
|
+
_banner("status")
|
|
134
|
+
branch = _git("branch", "--show-current") or "DETACHED"
|
|
135
|
+
line = f"on [bold]{branch}[/bold]"
|
|
136
|
+
try:
|
|
137
|
+
behind, ahead = _git("rev-list", "--left-right", "--count", "@{upstream}...HEAD",
|
|
138
|
+
stderr=subprocess.DEVNULL).split()
|
|
139
|
+
if behind != "0": line += f" [yellow]behind {behind}[/yellow]"
|
|
140
|
+
if ahead != "0": line += f" [cyan]ahead {ahead}[/cyan]"
|
|
141
|
+
if behind == ahead == "0": line += " [dim]in step with upstream[/dim]"
|
|
142
|
+
except subprocess.CalledProcessError:
|
|
143
|
+
line += " [dim]no upstream[/dim]"
|
|
144
|
+
console.print(line)
|
|
145
|
+
|
|
146
|
+
# NOT via _git: `git status -s` pads the code to two columns, so stripping the
|
|
147
|
+
# output would eat the leading space of the first line and shift its path by one.
|
|
148
|
+
porcelain = subprocess.check_output(["git", "status", "-s"]).decode('utf-8')
|
|
149
|
+
if not porcelain:
|
|
150
|
+
console.print("[green]clean[/green]")
|
|
151
|
+
return
|
|
152
|
+
table, n = _status_table(porcelain)
|
|
153
|
+
console.print(table)
|
|
154
|
+
console.print(f"[dim]{n} path{'' if n == 1 else 's'} changed[/dim]")
|
|
155
|
+
|
|
156
|
+
# %% ../nbs/01_Automate.ipynb #8d777a41-0160-47f8-b51c-b14360b62da4
|
|
157
|
+
@call_parse
|
|
158
|
+
def upload(
|
|
159
|
+
m:str = '', # Commit message
|
|
160
|
+
p:int = 2, # Increment part
|
|
161
|
+
):
|
|
162
|
+
"prep then gacp: the everyday command for shipping a notebook change"
|
|
163
|
+
prep(p)
|
|
164
|
+
gacp(m)
|
|
165
|
+
|
|
166
|
+
# %% ../nbs/01_Automate.ipynb #dee37b06-c946-4b40-be0f-056e133751ea
|
|
167
|
+
def release_git():
|
|
168
|
+
"Bump the MINOR version, then tag and create a GitHub release"
|
|
169
|
+
import nbdev.release
|
|
170
|
+
console.rule("[bold]gitrelease")
|
|
171
|
+
with _step("bump minor version"):
|
|
172
|
+
nbdev.release.nbdev_bump_version(1)
|
|
173
|
+
with _step("tag and create the GitHub release"):
|
|
174
|
+
nbdev.release.release_git()
|
|
175
|
+
|
|
176
|
+
# %% ../nbs/01_Automate.ipynb #288708c6-194a-4059-9ff0-ae330079c199
|
|
177
|
+
def release_pypi():
|
|
178
|
+
"Build the sdist and wheel, then upload to PyPI with twine. CI publishes on push, so this is the manual path"
|
|
179
|
+
import nbdev.release
|
|
180
|
+
console.rule("[bold]piprelease")
|
|
181
|
+
with _step("build and upload to PyPI"):
|
|
182
|
+
nbdev.release.release_pypi()
|
|
183
|
+
|
|
184
|
+
# %% ../nbs/01_Automate.ipynb #dd798178-9d58-457e-969a-d86a5e8bf875
|
|
185
|
+
def release():
|
|
186
|
+
"release_git then release_pypi"
|
|
187
|
+
release_git()
|
|
188
|
+
release_pypi()
|
|
189
|
+
|
|
190
|
+
# %% ../nbs/01_Automate.ipynb #c4d2bd52-04c4-43fe-890b-b4cb5da8d206
|
|
191
|
+
def help_output():
|
|
192
|
+
"Print every console script this package installs, with its help"
|
|
193
|
+
from importlib.metadata import distribution
|
|
194
|
+
try:
|
|
195
|
+
eps = [e for e in distribution("nbdevAuto").entry_points if e.group == "console_scripts"]
|
|
196
|
+
except Exception:
|
|
197
|
+
# Running from a source tree with nothing installed: fall back to fastcore.
|
|
198
|
+
from fastcore.xtras import console_help
|
|
199
|
+
return console_help('nbdevAuto')
|
|
200
|
+
_banner("commands")
|
|
201
|
+
t = Table(box=box.SIMPLE, show_header=False, pad_edge=False, expand=False)
|
|
202
|
+
t.add_column("command", style="bold cyan", no_wrap=True)
|
|
203
|
+
t.add_column("does", overflow="fold")
|
|
204
|
+
for e in sorted(eps, key=lambda e: e.name):
|
|
205
|
+
try: doc = escape((e.load().__doc__ or "").strip().splitlines()[0])
|
|
206
|
+
except Exception: doc = "[red]could not load[/red]"
|
|
207
|
+
t.add_row(e.name, doc)
|
|
208
|
+
console.print(t)
|
|
209
|
+
console.print(escape("ghstatus needs the gh extra: pip install 'nbdevAuto[gh]'"), style="dim")
|
|
@@ -6,9 +6,9 @@ Docs: https://bthek1.github.io/nbdevAuto/github.html.md"""
|
|
|
6
6
|
|
|
7
7
|
# %% auto #0
|
|
8
8
|
__all__ = ['Submodule', 'CI_WORKFLOWS', 'PAGES_WORKFLOWS', 'FAILED', 'RUNNING', 'BOLD', 'RED', 'YELLOW', 'CYAN', 'DIM', 'OFF',
|
|
9
|
-
'find_root', 'parse_gitmodules', 'resolve_token', 'local_sha', 'api_error', 'repo_status', '
|
|
10
|
-
'format_table', 'failures', 'has_failure', 'git', 'local_state', 'ahead_behind',
|
|
11
|
-
'format_report', 'stale_sections', 'owned_repos', 'self_repo', 'audit_result', 'gather_audit',
|
|
9
|
+
'find_root', 'parse_gitmodules', 'resolve_token', 'local_sha', 'api_error', 'repo_status', 'client',
|
|
10
|
+
'gather_status', 'format_table', 'failures', 'has_failure', 'git', 'local_state', 'ahead_behind',
|
|
11
|
+
'add_local', 'format_report', 'stale_sections', 'owned_repos', 'self_repo', 'audit_result', 'gather_audit',
|
|
12
12
|
'format_audit', 'ghstatus']
|
|
13
13
|
|
|
14
14
|
# %% ../nbs/03_GitHub.ipynb #519f31cd-5e6f-4ff9-9137-0293e57c0929
|
|
@@ -139,15 +139,33 @@ async def repo_status(gh, sub, root=None):
|
|
|
139
139
|
"ci": ci, "deploy": deploy, "built": built, "remote": remote,
|
|
140
140
|
"local": local_sha(Path(root or find_root())/sub.path)}
|
|
141
141
|
|
|
142
|
-
|
|
143
|
-
"
|
|
142
|
+
def client(token=None):
|
|
143
|
+
"One githubkit client, with githubkit's response cache turned off"
|
|
144
144
|
try:
|
|
145
145
|
from githubkit import GitHub
|
|
146
146
|
except ImportError:
|
|
147
147
|
raise ImportError("githubkit is not installed: pip install 'nbdevAuto[gh]'") from None
|
|
148
|
-
|
|
148
|
+
# githubkit defaults to http_cache=True. Its default MemCacheStrategy sounds
|
|
149
|
+
# like a dict but is not: githubkit's AsyncMemoryStorage subclasses hishel's
|
|
150
|
+
# AsyncSqliteStorage over a ":memory:" database, so every cached response is a
|
|
151
|
+
# transaction on one shared anysqlite connection. This tool issues ~76 requests
|
|
152
|
+
# concurrently through a single client, and those transactions collide: runs
|
|
153
|
+
# die intermittently on "cannot start a transaction within a transaction" or
|
|
154
|
+
# "bad parameter or other API misuse" and exit 3, which reads as "cannot ask
|
|
155
|
+
# GitHub" on a perfectly healthy fleet. Observed about twice in 40 runs.
|
|
156
|
+
#
|
|
157
|
+
# Caching is the wrong default for this question anyway - the whole point is to
|
|
158
|
+
# see whether the thing just pushed has built yet - and 76 calls against a
|
|
159
|
+
# 5000/hour budget do not need it. Turning it off removes the shared connection
|
|
160
|
+
# from the concurrent path entirely.
|
|
161
|
+
return GitHub(token or resolve_token(), http_cache=False)
|
|
162
|
+
|
|
163
|
+
async def gather_status(subs, token=None, root=None):
|
|
164
|
+
"Every row, concurrently, through one client and one connection pool"
|
|
165
|
+
async with client(token) as gh:
|
|
149
166
|
return await asyncio.gather(*[repo_status(gh, s, root) for s in subs])
|
|
150
167
|
|
|
168
|
+
|
|
151
169
|
# %% ../nbs/03_GitHub.ipynb #d4d8fd92-b4a2-4479-bd76-df3cc7fc9f77
|
|
152
170
|
FAILED = {"failure", "timed_out", "cancelled", "action_required", "startup_failure"}
|
|
153
171
|
RUNNING = {"in_progress", "queued", "requested", "waiting", "pending"}
|
|
@@ -287,15 +305,12 @@ def audit_result(subs, login, every, active, stale, exclude=()):
|
|
|
287
305
|
|
|
288
306
|
async def gather_audit(subs, root, token=None):
|
|
289
307
|
"The audit, one client, one pass over the account's repos"
|
|
290
|
-
|
|
291
|
-
from githubkit import GitHub
|
|
292
|
-
except ImportError:
|
|
293
|
-
raise ImportError("githubkit is not installed: pip install 'nbdevAuto[gh]'") from None
|
|
294
|
-
async with GitHub(token or resolve_token()) as gh:
|
|
308
|
+
async with client(token) as gh:
|
|
295
309
|
login, every, active = await owned_repos(gh)
|
|
296
310
|
return audit_result(subs, login, every, active, stale_sections(root),
|
|
297
311
|
exclude=[self_repo(root)])
|
|
298
312
|
|
|
313
|
+
|
|
299
314
|
# %% ../nbs/03_GitHub.ipynb #5e8ec6fc-9878-422c-b4bd-a0ce64dd8234
|
|
300
315
|
def _wrap(items, width=92):
|
|
301
316
|
"Comma-joined `items`, wrapped to `width` columns"
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: nbdevAuto
|
|
3
|
+
Version: 0.3.64
|
|
4
|
+
Summary: automating nbdev
|
|
5
|
+
Author-email: Benedict Thekkel <bthekkel1@gmail.com>
|
|
6
|
+
License-Expression: Apache-2.0
|
|
7
|
+
Project-URL: Repository, https://github.com/bthek1/nbdevAuto
|
|
8
|
+
Project-URL: Documentation, https://bthek1.github.io/nbdevAuto
|
|
9
|
+
Project-URL: Issues, https://github.com/bthek1/nbdevAuto/issues
|
|
10
|
+
Keywords: nbdev,jupyter,notebook,python
|
|
11
|
+
Classifier: Natural Language :: English
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Development Status :: 3 - Alpha
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
+
Requires-Python: >=3.10
|
|
22
|
+
Description-Content-Type: text/markdown
|
|
23
|
+
License-File: LICENSE
|
|
24
|
+
Requires-Dist: graphviz
|
|
25
|
+
Requires-Dist: fastcore
|
|
26
|
+
Requires-Dist: rich
|
|
27
|
+
Provides-Extra: gh
|
|
28
|
+
Requires-Dist: githubkit>=0.12; extra == "gh"
|
|
29
|
+
Dynamic: license-file
|
|
30
|
+
|
|
31
|
+
# nbdevAuto
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
|
|
35
|
+
|
|
36
|
+
Unlike most sites in this collection, this one documents an **installable library** rather than a topic. `nbdevAuto` exists because the nbdev release cycle is a fixed sequence of commands typed in the same order every time: export the notebooks, run the tests, clean them, bump the version, commit, push, build the docs. Each step is one nbdev call, and forgetting one produces a confusing failure two steps later.
|
|
37
|
+
|
|
38
|
+
The library collapses that sequence into single-word shell commands. It is what `just upload` and `just full_upload` invoke under the hood for every submodule in the parent Knowledge repo.
|
|
39
|
+
|
|
40
|
+
------------------------------------------------------------------------
|
|
41
|
+
|
|
42
|
+
## Install
|
|
43
|
+
|
|
44
|
+
``` sh
|
|
45
|
+
pip install nbdevAuto # the publish-cycle commands and the helpers
|
|
46
|
+
pip install 'nbdevAuto[gh]' # also installs githubkit, which `ghstatus` needs
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Requires Python 3.10 or newer.
|
|
50
|
+
|
|
51
|
+
## Use
|
|
52
|
+
|
|
53
|
+
Every exported function is also a console script, so the common case is a bare word in the terminal at the root of an nbdev project:
|
|
54
|
+
|
|
55
|
+
``` sh
|
|
56
|
+
upload -m "commit message" # export, test, clean, then add/commit/push
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Or import the helpers into a notebook:
|
|
60
|
+
|
|
61
|
+
``` python
|
|
62
|
+
from nbdevAuto.functions import download_search_images, classify_images
|
|
63
|
+
from nbdevAuto.pdf import PDFreader
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
------------------------------------------------------------------------
|
|
67
|
+
|
|
68
|
+
## Contents
|
|
69
|
+
|
|
70
|
+
| Page | Covers |
|
|
71
|
+
|----|----|
|
|
72
|
+
| [Functions](00_Functions.ipynb) | Dataset and image helpers: reading a list out of a text file, downloading single images or whole search-driven datasets, verifying and resizing what came back, building the folder layout a classifier expects, running a fastai learner over one image, Kaggle competition and dataset shortcuts, and [`graph`](https://bthek1.github.io/nbdevAuto/functions.html#graph), a `graphviz.Digraph` subclass preloaded with a rounded, filled style |
|
|
73
|
+
| [Automate](01_Automate.ipynb) | The command line surface: the prep/commit/push pipeline, the GitHub and PyPI release halves, and the help output that lists them all |
|
|
74
|
+
| [PDF Reader](02_PDF_reader.ipynb) | [`PDFreader`](https://bthek1.github.io/nbdevAuto/pdf_reader.html#pdfreader), a class that converts a PDF through `pdf2image` and renders page ranges inline when you slice it (`pdf[0:5]`) |
|
|
75
|
+
| [GitHub](03_GitHub.ipynb) | [`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus): reading `.gitmodules`, resolving a token, and asking the GitHub API about every submodule at once - CI runs, Pages deploys, Pages builds, remote `main` shas - plus the `--report` and `--audit` views and the exit codes that keep “absent” apart from “cannot ask” |
|
|
76
|
+
|
|
77
|
+
------------------------------------------------------------------------
|
|
78
|
+
|
|
79
|
+
## The Command Line Surface
|
|
80
|
+
|
|
81
|
+
Installing the package puts these on your `PATH`. Most are ordinary functions in [Automate](01_Automate.ipynb), exposed as scripts through `[project.scripts]`; [`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus) comes from [GitHub](03_GitHub.ipynb).
|
|
82
|
+
|
|
83
|
+
| Command | Does |
|
|
84
|
+
|----|----|
|
|
85
|
+
| [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) | Bump the version, then export, test and clean the notebooks, refreshing `_quarto.yml` and the README |
|
|
86
|
+
| [`gacp`](https://bthek1.github.io/nbdevAuto/automate.html#gacp) | git add, commit, and push. Without `-m` the commit message is the status output |
|
|
87
|
+
| [`status`](https://bthek1.github.io/nbdevAuto/automate.html#status) | Show the working tree state |
|
|
88
|
+
| [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) | [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) then [`gacp`](https://bthek1.github.io/nbdevAuto/automate.html#gacp), the one you actually type |
|
|
89
|
+
| `gitrelease` | Bump the **minor** version, then tag and create a GitHub release |
|
|
90
|
+
| `piprelease` | Build the sdist and wheel, then upload to PyPI with twine |
|
|
91
|
+
| [`release`](https://bthek1.github.io/nbdevAuto/automate.html#release) | `gitrelease` then `piprelease` |
|
|
92
|
+
| [`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus) | Remote status for every submodule of an nbdev superproject, from the GitHub API. `--report` for the sync view, `--audit` to check `.gitmodules` against the repos the account owns, `--json` for machine output |
|
|
93
|
+
| `h` | Print the list of available commands |
|
|
94
|
+
|
|
95
|
+
Both [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) and [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) take `-p` to choose which version part to increment; it defaults to 2, the patch, so every [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) ships a new version.
|
|
96
|
+
|
|
97
|
+
`piprelease` is the manual path. Pushing to `main` publishes to PyPI on its own: `publish.yaml` runs after CI passes, and skips quietly when the version is already there.
|
|
98
|
+
|
|
99
|
+
[`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus) needs the `gh` extra (`pip install 'nbdevAuto[gh]'`), which brings in `githubkit`. It is an extra rather than a dependency because githubkit pulls pydantic, httpx and hishel behind it, which is a lot of weight for anyone who only wants [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload). Without it the command exits 3 with an install hint rather than a traceback. It takes its token from `$GITHUB_TOKEN` / `$GH_TOKEN`, then `gh auth token`.
|
|
100
|
+
|
|
101
|
+
------------------------------------------------------------------------
|
|
102
|
+
|
|
103
|
+
## Where It Is Used
|
|
104
|
+
|
|
105
|
+
The parent Knowledge repo’s `justfile` calls [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) once per submodule. `just upload` runs it only where the working tree is dirty, `just full_upload` runs it everywhere. That is the main consumer, so a change to `automate.py` affects the publish path for every site in the collection.
|
|
106
|
+
|
|
107
|
+
[`ghstatus`](https://bthek1.github.io/nbdevAuto/github.html#ghstatus) is the other half of that loop. [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) pushes to 19 repos and then goes blind: whether each repo’s CI passed and whether its Pages deploy actually landed is invisible from the terminal. `just ci`, `just report` and `just audit` in the parent repo are all thin wrappers around this one command, which answers for the whole fleet in about 1.5 seconds instead of 19 browser tabs.
|
|
108
|
+
|
|
109
|
+
## Releases
|
|
110
|
+
|
|
111
|
+
This package publishes itself. A `publish.yaml` workflow runs after CI passes on `main` and uploads to PyPI through Trusted Publishing, so no API token is stored in the repository. It reads `__version__` out of `nbdevAuto/__init__.py` and asks PyPI whether that version already exists: a new version is published, an existing one is skipped without failing the build.
|
|
112
|
+
|
|
113
|
+
Because [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) bumps the patch version on every push, an ordinary commit made with [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) ships a release. A plain `git commit` leaves the version alone and the workflow skips. `gitrelease`, which bumps the minor version and creates the GitHub release and tag, stays a deliberate manual step.
|
|
114
|
+
|
|
115
|
+
------------------------------------------------------------------------
|
|
116
|
+
|
|
117
|
+
## Not Covered Yet
|
|
118
|
+
|
|
119
|
+
- **No test notebook for the helpers.** `03_GitHub.ipynb` carries unit tests for its pure functions, but the dataset and image helpers are exercised only by being used, so a broken one surfaces in a downstream repo rather than here.
|
|
120
|
+
- **Undeclared dependencies.** `pyproject.toml` declares `graphviz` and `fastcore`, plus `githubkit` behind the `gh` extra. The helpers additionally import `fastai`, `fastbook`, `fastdownload`, `PIL`, `matplotlib`, `tqdm`, `pdf2image` (with poppler behind it), `nbdev` and `kaggle` (which also needs its credentials). All of those are imported lazily inside the function that needs them, so installing the package and running the console scripts still works - only the specific helper you call fails. Install them yourself.
|
|
121
|
+
- **`core.py` is an empty nbdev stub** (a single `foo`) left over from `00_core.ipynb`, which no longer exists.
|
|
122
|
+
- The docstrings are one-liners, so the rendered API pages are thin. The notebooks carry the real explanation.
|
|
123
|
+
|
|
124
|
+
------------------------------------------------------------------------
|
|
@@ -1,17 +1,12 @@
|
|
|
1
1
|
[console_scripts]
|
|
2
|
-
create_env = nbdevAuto.automate:create_env
|
|
3
|
-
everything = nbdevAuto.automate:everything
|
|
4
2
|
gacp = nbdevAuto.automate:gacp
|
|
5
3
|
ghstatus = nbdevAuto.github:ghstatus
|
|
6
4
|
gitrelease = nbdevAuto.automate:release_git
|
|
7
5
|
h = nbdevAuto.automate:help_output
|
|
8
|
-
hello = nbdevAuto.automate:hello
|
|
9
6
|
piprelease = nbdevAuto.automate:release_pypi
|
|
10
7
|
prep = nbdevAuto.automate:prep
|
|
11
|
-
reinstall = nbdevAuto.automate:reinstall
|
|
12
8
|
release = nbdevAuto.automate:release
|
|
13
9
|
status = nbdevAuto.automate:status
|
|
14
|
-
update = nbdevAuto.automate:update
|
|
15
10
|
upload = nbdevAuto.automate:upload
|
|
16
11
|
|
|
17
12
|
[nbdev]
|
|
@@ -24,7 +24,7 @@ classifiers = [
|
|
|
24
24
|
"Programming Language :: Python :: 3.13",
|
|
25
25
|
"Programming Language :: Python :: 3.14",
|
|
26
26
|
]
|
|
27
|
-
dependencies = ['graphviz', 'fastcore']
|
|
27
|
+
dependencies = ['graphviz', 'fastcore', 'rich']
|
|
28
28
|
|
|
29
29
|
[project.optional-dependencies]
|
|
30
30
|
# An extra, not a dependency: githubkit pulls pydantic + httpx + hishel, which is
|
|
@@ -46,15 +46,10 @@ prep = "nbdevAuto.automate:prep"
|
|
|
46
46
|
gacp = "nbdevAuto.automate:gacp"
|
|
47
47
|
status = "nbdevAuto.automate:status"
|
|
48
48
|
upload = "nbdevAuto.automate:upload"
|
|
49
|
-
reinstall = "nbdevAuto.automate:reinstall"
|
|
50
|
-
update = "nbdevAuto.automate:update"
|
|
51
49
|
gitrelease = "nbdevAuto.automate:release_git"
|
|
52
50
|
piprelease = "nbdevAuto.automate:release_pypi"
|
|
53
51
|
release = "nbdevAuto.automate:release"
|
|
54
52
|
h = "nbdevAuto.automate:help_output"
|
|
55
|
-
everything = "nbdevAuto.automate:everything"
|
|
56
|
-
create_env = "nbdevAuto.automate:create_env"
|
|
57
|
-
hello = "nbdevAuto.automate:hello"
|
|
58
53
|
ghstatus = "nbdevAuto.github:ghstatus"
|
|
59
54
|
|
|
60
55
|
[tool.setuptools.dynamic]
|
nbdevauto-0.3.61/PKG-INFO
DELETED
|
@@ -1,134 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: nbdevAuto
|
|
3
|
-
Version: 0.3.61
|
|
4
|
-
Summary: automating nbdev
|
|
5
|
-
Author-email: Benedict Thekkel <bthekkel1@gmail.com>
|
|
6
|
-
License-Expression: Apache-2.0
|
|
7
|
-
Project-URL: Repository, https://github.com/bthek1/nbdevAuto
|
|
8
|
-
Project-URL: Documentation, https://bthek1.github.io/nbdevAuto
|
|
9
|
-
Project-URL: Issues, https://github.com/bthek1/nbdevAuto/issues
|
|
10
|
-
Keywords: nbdev,jupyter,notebook,python
|
|
11
|
-
Classifier: Natural Language :: English
|
|
12
|
-
Classifier: Intended Audience :: Developers
|
|
13
|
-
Classifier: Development Status :: 3 - Alpha
|
|
14
|
-
Classifier: Programming Language :: Python :: 3
|
|
15
|
-
Classifier: Programming Language :: Python :: 3 :: Only
|
|
16
|
-
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
-
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
-
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
-
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
-
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
-
Requires-Python: >=3.10
|
|
22
|
-
Description-Content-Type: text/markdown
|
|
23
|
-
License-File: LICENSE
|
|
24
|
-
Requires-Dist: graphviz
|
|
25
|
-
Requires-Dist: fastcore
|
|
26
|
-
Provides-Extra: gh
|
|
27
|
-
Requires-Dist: githubkit>=0.12; extra == "gh"
|
|
28
|
-
Dynamic: license-file
|
|
29
|
-
|
|
30
|
-
# nbdevAuto
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
|
|
34
|
-
|
|
35
|
-
Unlike most sites in this collection, this one documents an
|
|
36
|
-
**installable library** rather than a topic. `nbdevAuto` exists because
|
|
37
|
-
the nbdev release cycle is a fixed sequence of commands typed in the
|
|
38
|
-
same order every time: export the notebooks, run the tests, clean them,
|
|
39
|
-
bump the version, commit, push, build the docs. Each step is one nbdev
|
|
40
|
-
call, and forgetting one produces a confusing failure two steps later.
|
|
41
|
-
|
|
42
|
-
The library collapses that sequence into single-word shell commands. It
|
|
43
|
-
is what `just upload` and `just full_upload` invoke under the hood for
|
|
44
|
-
every submodule in the parent Knowledge repo.
|
|
45
|
-
|
|
46
|
-
------------------------------------------------------------------------
|
|
47
|
-
|
|
48
|
-
## Install
|
|
49
|
-
|
|
50
|
-
``` sh
|
|
51
|
-
pip install nbdevAuto
|
|
52
|
-
```
|
|
53
|
-
|
|
54
|
-
## Use
|
|
55
|
-
|
|
56
|
-
Every exported function is also a console script, so the common case is
|
|
57
|
-
a bare word in the terminal at the root of an nbdev project:
|
|
58
|
-
|
|
59
|
-
``` sh
|
|
60
|
-
upload -m "commit message" # export, test, clean, then add/commit/push
|
|
61
|
-
```
|
|
62
|
-
|
|
63
|
-
Or import the helpers into a notebook:
|
|
64
|
-
|
|
65
|
-
``` python
|
|
66
|
-
from nbdevAuto.functions import download_search_images, classify_images
|
|
67
|
-
from nbdevAuto.pdf import PDFreader
|
|
68
|
-
```
|
|
69
|
-
|
|
70
|
-
------------------------------------------------------------------------
|
|
71
|
-
|
|
72
|
-
## Contents
|
|
73
|
-
|
|
74
|
-
| Page | Covers |
|
|
75
|
-
|----|----|
|
|
76
|
-
| [Functions](00_Functions.ipynb) | Dataset and image helpers: reading a list out of a text file, downloading single images or whole search-driven datasets, verifying and resizing what came back, building the folder layout a classifier expects, running a fastai learner over one image, Kaggle competition and dataset shortcuts, and [`graph`](https://bthek1.github.io/nbdevAuto/functions.html#graph), a `graphviz.Digraph` subclass preloaded with a rounded, filled style |
|
|
77
|
-
| [Automate](01_Automate.ipynb) | The command line surface: the prep/commit/push pipeline, GitHub and PyPI releases, reinstall and update, conda environment creation, and the help output that lists them all |
|
|
78
|
-
| [PDF Reader](02_PDF_reader.ipynb) | [`PDFreader`](https://bthek1.github.io/nbdevAuto/pdf_reader.html#pdfreader), a class that converts a PDF through `pdf2image` and renders page ranges inline when you slice it (`pdf[0:5]`) |
|
|
79
|
-
|
|
80
|
-
------------------------------------------------------------------------
|
|
81
|
-
|
|
82
|
-
## The Command Line Surface
|
|
83
|
-
|
|
84
|
-
Installing the package puts these on your `PATH`. They are ordinary
|
|
85
|
-
functions in [Automate](01_Automate.ipynb), exposed as scripts through
|
|
86
|
-
`[project.scripts]`.
|
|
87
|
-
|
|
88
|
-
| Command | Does |
|
|
89
|
-
|----|----|
|
|
90
|
-
| [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) | Export, test, and clean the notebooks, rendering the README if it needs it |
|
|
91
|
-
| [`gacp`](https://bthek1.github.io/nbdevAuto/automate.html#gacp) | git add, commit, and push, with `-m` for the message |
|
|
92
|
-
| [`status`](https://bthek1.github.io/nbdevAuto/automate.html#status) | Show the working tree state |
|
|
93
|
-
| [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) | [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) then [`gacp`](https://bthek1.github.io/nbdevAuto/automate.html#gacp), the one you actually type |
|
|
94
|
-
| [`release`](https://bthek1.github.io/nbdevAuto/automate.html#release) | Bump the version, then release to both GitHub and PyPI |
|
|
95
|
-
| `gitrelease` / `piprelease` | The two halves of [`release`](https://bthek1.github.io/nbdevAuto/automate.html#release), separately |
|
|
96
|
-
| [`reinstall`](https://bthek1.github.io/nbdevAuto/automate.html#reinstall) / [`update`](https://bthek1.github.io/nbdevAuto/automate.html#update) | Reinstall or update the package locally |
|
|
97
|
-
| [`everything`](https://bthek1.github.io/nbdevAuto/automate.html#everything) | [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload), [`update`](https://bthek1.github.io/nbdevAuto/automate.html#update), [`release`](https://bthek1.github.io/nbdevAuto/automate.html#release), [`reinstall`](https://bthek1.github.io/nbdevAuto/automate.html#reinstall), in that order |
|
|
98
|
-
| [`create_env`](https://bthek1.github.io/nbdevAuto/automate.html#create_env) | Create a conda environment for AI work, named `fast` by default |
|
|
99
|
-
| `h` | Print the list of available commands |
|
|
100
|
-
|
|
101
|
-
Both [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) and
|
|
102
|
-
[`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) take
|
|
103
|
-
`-p` to choose which version part to increment.
|
|
104
|
-
|
|
105
|
-
------------------------------------------------------------------------
|
|
106
|
-
|
|
107
|
-
## Where It Is Used
|
|
108
|
-
|
|
109
|
-
The parent Knowledge repo’s `justfile` calls
|
|
110
|
-
[`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) once
|
|
111
|
-
per submodule. `just upload` runs it only where the working tree is
|
|
112
|
-
dirty, `just full_upload` runs it everywhere. That is the main consumer,
|
|
113
|
-
so a change to `automate.py` affects the publish path for every site in
|
|
114
|
-
the collection.
|
|
115
|
-
|
|
116
|
-
------------------------------------------------------------------------
|
|
117
|
-
|
|
118
|
-
## Not Covered Yet
|
|
119
|
-
|
|
120
|
-
- **No test notebook.** The functions are exercised only by being used,
|
|
121
|
-
so a broken helper surfaces in a downstream repo rather than here.
|
|
122
|
-
- **Undeclared dependencies.** `pyproject.toml` lists only `graphviz`,
|
|
123
|
-
but the image helpers import `fastai`, the Kaggle shortcuts need the
|
|
124
|
-
Kaggle CLI and its credentials, and
|
|
125
|
-
[`PDFreader`](https://bthek1.github.io/nbdevAuto/pdf_reader.html#pdfreader)
|
|
126
|
-
needs `pdf2image` with poppler behind it, plus matplotlib. Install
|
|
127
|
-
those yourself.
|
|
128
|
-
- **`core.py` is an empty nbdev stub** (a single
|
|
129
|
-
[`foo`](https://bthek1.github.io/nbdevAuto/core.html#foo)), with no
|
|
130
|
-
notebook behind it.
|
|
131
|
-
- The docstrings are one-liners, so the rendered API pages are thin. The
|
|
132
|
-
notebooks carry the real explanation.
|
|
133
|
-
|
|
134
|
-
------------------------------------------------------------------------
|
nbdevauto-0.3.61/README.md
DELETED
|
@@ -1,105 +0,0 @@
|
|
|
1
|
-
# nbdevAuto
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
|
|
5
|
-
|
|
6
|
-
Unlike most sites in this collection, this one documents an
|
|
7
|
-
**installable library** rather than a topic. `nbdevAuto` exists because
|
|
8
|
-
the nbdev release cycle is a fixed sequence of commands typed in the
|
|
9
|
-
same order every time: export the notebooks, run the tests, clean them,
|
|
10
|
-
bump the version, commit, push, build the docs. Each step is one nbdev
|
|
11
|
-
call, and forgetting one produces a confusing failure two steps later.
|
|
12
|
-
|
|
13
|
-
The library collapses that sequence into single-word shell commands. It
|
|
14
|
-
is what `just upload` and `just full_upload` invoke under the hood for
|
|
15
|
-
every submodule in the parent Knowledge repo.
|
|
16
|
-
|
|
17
|
-
------------------------------------------------------------------------
|
|
18
|
-
|
|
19
|
-
## Install
|
|
20
|
-
|
|
21
|
-
``` sh
|
|
22
|
-
pip install nbdevAuto
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
## Use
|
|
26
|
-
|
|
27
|
-
Every exported function is also a console script, so the common case is
|
|
28
|
-
a bare word in the terminal at the root of an nbdev project:
|
|
29
|
-
|
|
30
|
-
``` sh
|
|
31
|
-
upload -m "commit message" # export, test, clean, then add/commit/push
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
Or import the helpers into a notebook:
|
|
35
|
-
|
|
36
|
-
``` python
|
|
37
|
-
from nbdevAuto.functions import download_search_images, classify_images
|
|
38
|
-
from nbdevAuto.pdf import PDFreader
|
|
39
|
-
```
|
|
40
|
-
|
|
41
|
-
------------------------------------------------------------------------
|
|
42
|
-
|
|
43
|
-
## Contents
|
|
44
|
-
|
|
45
|
-
| Page | Covers |
|
|
46
|
-
|----|----|
|
|
47
|
-
| [Functions](00_Functions.ipynb) | Dataset and image helpers: reading a list out of a text file, downloading single images or whole search-driven datasets, verifying and resizing what came back, building the folder layout a classifier expects, running a fastai learner over one image, Kaggle competition and dataset shortcuts, and [`graph`](https://bthek1.github.io/nbdevAuto/functions.html#graph), a `graphviz.Digraph` subclass preloaded with a rounded, filled style |
|
|
48
|
-
| [Automate](01_Automate.ipynb) | The command line surface: the prep/commit/push pipeline, GitHub and PyPI releases, reinstall and update, conda environment creation, and the help output that lists them all |
|
|
49
|
-
| [PDF Reader](02_PDF_reader.ipynb) | [`PDFreader`](https://bthek1.github.io/nbdevAuto/pdf_reader.html#pdfreader), a class that converts a PDF through `pdf2image` and renders page ranges inline when you slice it (`pdf[0:5]`) |
|
|
50
|
-
|
|
51
|
-
------------------------------------------------------------------------
|
|
52
|
-
|
|
53
|
-
## The Command Line Surface
|
|
54
|
-
|
|
55
|
-
Installing the package puts these on your `PATH`. They are ordinary
|
|
56
|
-
functions in [Automate](01_Automate.ipynb), exposed as scripts through
|
|
57
|
-
`[project.scripts]`.
|
|
58
|
-
|
|
59
|
-
| Command | Does |
|
|
60
|
-
|----|----|
|
|
61
|
-
| [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) | Export, test, and clean the notebooks, rendering the README if it needs it |
|
|
62
|
-
| [`gacp`](https://bthek1.github.io/nbdevAuto/automate.html#gacp) | git add, commit, and push, with `-m` for the message |
|
|
63
|
-
| [`status`](https://bthek1.github.io/nbdevAuto/automate.html#status) | Show the working tree state |
|
|
64
|
-
| [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) | [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) then [`gacp`](https://bthek1.github.io/nbdevAuto/automate.html#gacp), the one you actually type |
|
|
65
|
-
| [`release`](https://bthek1.github.io/nbdevAuto/automate.html#release) | Bump the version, then release to both GitHub and PyPI |
|
|
66
|
-
| `gitrelease` / `piprelease` | The two halves of [`release`](https://bthek1.github.io/nbdevAuto/automate.html#release), separately |
|
|
67
|
-
| [`reinstall`](https://bthek1.github.io/nbdevAuto/automate.html#reinstall) / [`update`](https://bthek1.github.io/nbdevAuto/automate.html#update) | Reinstall or update the package locally |
|
|
68
|
-
| [`everything`](https://bthek1.github.io/nbdevAuto/automate.html#everything) | [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload), [`update`](https://bthek1.github.io/nbdevAuto/automate.html#update), [`release`](https://bthek1.github.io/nbdevAuto/automate.html#release), [`reinstall`](https://bthek1.github.io/nbdevAuto/automate.html#reinstall), in that order |
|
|
69
|
-
| [`create_env`](https://bthek1.github.io/nbdevAuto/automate.html#create_env) | Create a conda environment for AI work, named `fast` by default |
|
|
70
|
-
| `h` | Print the list of available commands |
|
|
71
|
-
|
|
72
|
-
Both [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) and
|
|
73
|
-
[`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) take
|
|
74
|
-
`-p` to choose which version part to increment.
|
|
75
|
-
|
|
76
|
-
------------------------------------------------------------------------
|
|
77
|
-
|
|
78
|
-
## Where It Is Used
|
|
79
|
-
|
|
80
|
-
The parent Knowledge repo’s `justfile` calls
|
|
81
|
-
[`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) once
|
|
82
|
-
per submodule. `just upload` runs it only where the working tree is
|
|
83
|
-
dirty, `just full_upload` runs it everywhere. That is the main consumer,
|
|
84
|
-
so a change to `automate.py` affects the publish path for every site in
|
|
85
|
-
the collection.
|
|
86
|
-
|
|
87
|
-
------------------------------------------------------------------------
|
|
88
|
-
|
|
89
|
-
## Not Covered Yet
|
|
90
|
-
|
|
91
|
-
- **No test notebook.** The functions are exercised only by being used,
|
|
92
|
-
so a broken helper surfaces in a downstream repo rather than here.
|
|
93
|
-
- **Undeclared dependencies.** `pyproject.toml` lists only `graphviz`,
|
|
94
|
-
but the image helpers import `fastai`, the Kaggle shortcuts need the
|
|
95
|
-
Kaggle CLI and its credentials, and
|
|
96
|
-
[`PDFreader`](https://bthek1.github.io/nbdevAuto/pdf_reader.html#pdfreader)
|
|
97
|
-
needs `pdf2image` with poppler behind it, plus matplotlib. Install
|
|
98
|
-
those yourself.
|
|
99
|
-
- **`core.py` is an empty nbdev stub** (a single
|
|
100
|
-
[`foo`](https://bthek1.github.io/nbdevAuto/core.html#foo)), with no
|
|
101
|
-
notebook behind it.
|
|
102
|
-
- The docstrings are one-liners, so the rendered API pages are thin. The
|
|
103
|
-
notebooks carry the real explanation.
|
|
104
|
-
|
|
105
|
-
------------------------------------------------------------------------
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
__version__ = "0.3.61"
|
|
@@ -1,141 +0,0 @@
|
|
|
1
|
-
"""Automate
|
|
2
|
-
|
|
3
|
-
Docs: https://bthek1.github.io/nbdevAuto/automate.html.md"""
|
|
4
|
-
|
|
5
|
-
# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/01_Automate.ipynb.
|
|
6
|
-
|
|
7
|
-
# %% auto #0
|
|
8
|
-
__all__ = ['prep', 'gacp', 'status', 'upload', 'reinstall', 'update', 'release_git', 'release_pypi', 'release', 'help_output',
|
|
9
|
-
'everything', 'create_env', 'hello']
|
|
10
|
-
|
|
11
|
-
# %% ../nbs/01_Automate.ipynb #6d1499d1-aa55-47d8-92fd-fa056b1fb135
|
|
12
|
-
from fastcore.script import *
|
|
13
|
-
|
|
14
|
-
@call_parse
|
|
15
|
-
def prep(
|
|
16
|
-
p:int = 2, # Increment Part
|
|
17
|
-
):
|
|
18
|
-
"Export, test, and clean notebooks, and render README if needed"
|
|
19
|
-
|
|
20
|
-
import nbdev.test, nbdev.clean, nbdev.quarto, nbdev.release
|
|
21
|
-
nbdev.release.nbdev_bump_version(p)
|
|
22
|
-
|
|
23
|
-
nbdev.quarto.nbdev_export.__wrapped__()
|
|
24
|
-
print(f'### nbdev_export finished ###')
|
|
25
|
-
nbdev.test.nbdev_test.__wrapped__(
|
|
26
|
-
n_workers = 8, # Number of workers
|
|
27
|
-
timing = True, # Time each notebook to see which are slow
|
|
28
|
-
)
|
|
29
|
-
print(f'### nbdev_test finished ###')
|
|
30
|
-
nbdev.clean.nbdev_clean.__wrapped__()
|
|
31
|
-
print(f'### nbdev_clean finished ###')
|
|
32
|
-
nbdev.quarto.refresh_quarto_yml()
|
|
33
|
-
print(f'### refresh_quarto_yml finished ###')
|
|
34
|
-
nbdev.quarto.nbdev_readme.__wrapped__(chk_time=True)
|
|
35
|
-
print(f'### nbdev_readme finished ###')
|
|
36
|
-
|
|
37
|
-
# %% ../nbs/01_Automate.ipynb #f66ef35f-b7fa-4039-aa07-e9b782d204bb
|
|
38
|
-
@call_parse
|
|
39
|
-
def gacp(
|
|
40
|
-
m:str = '', # Commit message
|
|
41
|
-
):
|
|
42
|
-
"git add, commit and push to github"
|
|
43
|
-
|
|
44
|
-
import subprocess
|
|
45
|
-
subprocess.run(["git", "add", "."])
|
|
46
|
-
print(f'### git added ###')
|
|
47
|
-
status = subprocess.check_output(["git", "status", "-s"]).decode('utf-8')
|
|
48
|
-
print(f'### git status: "{status}" ###')
|
|
49
|
-
|
|
50
|
-
if m != '':
|
|
51
|
-
subprocess.run(["git", "commit", "-m", f'{m}'])
|
|
52
|
-
print(f'### git commited "{m}"###')
|
|
53
|
-
else:
|
|
54
|
-
subprocess.run(["git", "commit", "-m", f'{status}'])
|
|
55
|
-
print(f'### git commited status ###')
|
|
56
|
-
subprocess.run(["git", "push"])
|
|
57
|
-
print(f'### git pushed ###')
|
|
58
|
-
|
|
59
|
-
# %% ../nbs/01_Automate.ipynb #88017a1b-55c1-4833-aeb6-277a7e5b6314
|
|
60
|
-
def status():
|
|
61
|
-
"git status"
|
|
62
|
-
import subprocess
|
|
63
|
-
subprocess.run(["git", "status"])
|
|
64
|
-
|
|
65
|
-
# %% ../nbs/01_Automate.ipynb #8d777a41-0160-47f8-b51c-b14360b62da4
|
|
66
|
-
@call_parse
|
|
67
|
-
def upload(
|
|
68
|
-
m:str = '', # Commit message
|
|
69
|
-
p:int = 2, #Increment part
|
|
70
|
-
):
|
|
71
|
-
"prep, then gacp"
|
|
72
|
-
prep(p)
|
|
73
|
-
gacp(m)
|
|
74
|
-
|
|
75
|
-
# %% ../nbs/01_Automate.ipynb #0f096fc0-0e97-4592-975f-e81e583b2adb
|
|
76
|
-
def reinstall():
|
|
77
|
-
"runs pip install -e ."
|
|
78
|
-
import subprocess
|
|
79
|
-
subprocess.run(["pip", "install", "-e", "."])
|
|
80
|
-
|
|
81
|
-
# %% ../nbs/01_Automate.ipynb #fa41bc21-ba9f-4097-8c70-4fc1404d325c
|
|
82
|
-
def update():
|
|
83
|
-
"prep, then reinstall"
|
|
84
|
-
prep()
|
|
85
|
-
reinstall()
|
|
86
|
-
|
|
87
|
-
# %% ../nbs/01_Automate.ipynb #dee37b06-c946-4b40-be0f-056e133751ea
|
|
88
|
-
def release_git():
|
|
89
|
-
"release to git"
|
|
90
|
-
import nbdev.release
|
|
91
|
-
nbdev.release.nbdev_bump_version(1)
|
|
92
|
-
nbdev.release.release_git()
|
|
93
|
-
|
|
94
|
-
# %% ../nbs/01_Automate.ipynb #288708c6-194a-4059-9ff0-ae330079c199
|
|
95
|
-
def release_pypi():
|
|
96
|
-
"release to pypi"
|
|
97
|
-
import nbdev.release
|
|
98
|
-
nbdev.release.release_pypi()
|
|
99
|
-
|
|
100
|
-
# %% ../nbs/01_Automate.ipynb #dd798178-9d58-457e-969a-d86a5e8bf875
|
|
101
|
-
def release():
|
|
102
|
-
"release to github and pip"
|
|
103
|
-
release_git()
|
|
104
|
-
release_pypi()
|
|
105
|
-
|
|
106
|
-
# %% ../nbs/01_Automate.ipynb #c4d2bd52-04c4-43fe-890b-b4cb5da8d206
|
|
107
|
-
def help_output():
|
|
108
|
-
"Show help for all console scripts"
|
|
109
|
-
from fastcore.xtras import console_help
|
|
110
|
-
console_help('nbdevAuto')
|
|
111
|
-
|
|
112
|
-
# %% ../nbs/01_Automate.ipynb #3606bcc9-57b1-42e7-842a-5c889485cf88
|
|
113
|
-
def everything():
|
|
114
|
-
"prep, gacp, release, reinstall"
|
|
115
|
-
upload()
|
|
116
|
-
update()
|
|
117
|
-
release()
|
|
118
|
-
|
|
119
|
-
# %% ../nbs/01_Automate.ipynb #8ca135ed-6377-4da1-98e8-fdab288921fb
|
|
120
|
-
@call_parse
|
|
121
|
-
def create_env(
|
|
122
|
-
n:str = 'fast', #Name of the environment
|
|
123
|
-
):
|
|
124
|
-
"create conda env for AI"
|
|
125
|
-
from pathlib import Path
|
|
126
|
-
import subprocess
|
|
127
|
-
|
|
128
|
-
subprocess.run(["conda", "create", "-n", f"{n}", "python"])
|
|
129
|
-
subprocess.run(["conda", "init"])
|
|
130
|
-
subprocess.run(["conda", "activate", f"{n}"], shell=True)
|
|
131
|
-
# subprocess.run(["pip", "install", "fastai", "fastcore"])
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
# %% ../nbs/01_Automate.ipynb #16615f52-30f8-4a42-8d73-52355f2c574a
|
|
135
|
-
def hello():
|
|
136
|
-
import argparse
|
|
137
|
-
parser = argparse.ArgumentParser(description='My CLI function.')
|
|
138
|
-
parser.add_argument('-n', '--name', type=str, help='Name of the environment', default='fast')
|
|
139
|
-
args = parser.parse_args()
|
|
140
|
-
|
|
141
|
-
print(f'Hello, {args.name}!')
|
|
@@ -1,134 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: nbdevAuto
|
|
3
|
-
Version: 0.3.61
|
|
4
|
-
Summary: automating nbdev
|
|
5
|
-
Author-email: Benedict Thekkel <bthekkel1@gmail.com>
|
|
6
|
-
License-Expression: Apache-2.0
|
|
7
|
-
Project-URL: Repository, https://github.com/bthek1/nbdevAuto
|
|
8
|
-
Project-URL: Documentation, https://bthek1.github.io/nbdevAuto
|
|
9
|
-
Project-URL: Issues, https://github.com/bthek1/nbdevAuto/issues
|
|
10
|
-
Keywords: nbdev,jupyter,notebook,python
|
|
11
|
-
Classifier: Natural Language :: English
|
|
12
|
-
Classifier: Intended Audience :: Developers
|
|
13
|
-
Classifier: Development Status :: 3 - Alpha
|
|
14
|
-
Classifier: Programming Language :: Python :: 3
|
|
15
|
-
Classifier: Programming Language :: Python :: 3 :: Only
|
|
16
|
-
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
-
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
-
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
-
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
-
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
-
Requires-Python: >=3.10
|
|
22
|
-
Description-Content-Type: text/markdown
|
|
23
|
-
License-File: LICENSE
|
|
24
|
-
Requires-Dist: graphviz
|
|
25
|
-
Requires-Dist: fastcore
|
|
26
|
-
Provides-Extra: gh
|
|
27
|
-
Requires-Dist: githubkit>=0.12; extra == "gh"
|
|
28
|
-
Dynamic: license-file
|
|
29
|
-
|
|
30
|
-
# nbdevAuto
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
|
|
34
|
-
|
|
35
|
-
Unlike most sites in this collection, this one documents an
|
|
36
|
-
**installable library** rather than a topic. `nbdevAuto` exists because
|
|
37
|
-
the nbdev release cycle is a fixed sequence of commands typed in the
|
|
38
|
-
same order every time: export the notebooks, run the tests, clean them,
|
|
39
|
-
bump the version, commit, push, build the docs. Each step is one nbdev
|
|
40
|
-
call, and forgetting one produces a confusing failure two steps later.
|
|
41
|
-
|
|
42
|
-
The library collapses that sequence into single-word shell commands. It
|
|
43
|
-
is what `just upload` and `just full_upload` invoke under the hood for
|
|
44
|
-
every submodule in the parent Knowledge repo.
|
|
45
|
-
|
|
46
|
-
------------------------------------------------------------------------
|
|
47
|
-
|
|
48
|
-
## Install
|
|
49
|
-
|
|
50
|
-
``` sh
|
|
51
|
-
pip install nbdevAuto
|
|
52
|
-
```
|
|
53
|
-
|
|
54
|
-
## Use
|
|
55
|
-
|
|
56
|
-
Every exported function is also a console script, so the common case is
|
|
57
|
-
a bare word in the terminal at the root of an nbdev project:
|
|
58
|
-
|
|
59
|
-
``` sh
|
|
60
|
-
upload -m "commit message" # export, test, clean, then add/commit/push
|
|
61
|
-
```
|
|
62
|
-
|
|
63
|
-
Or import the helpers into a notebook:
|
|
64
|
-
|
|
65
|
-
``` python
|
|
66
|
-
from nbdevAuto.functions import download_search_images, classify_images
|
|
67
|
-
from nbdevAuto.pdf import PDFreader
|
|
68
|
-
```
|
|
69
|
-
|
|
70
|
-
------------------------------------------------------------------------
|
|
71
|
-
|
|
72
|
-
## Contents
|
|
73
|
-
|
|
74
|
-
| Page | Covers |
|
|
75
|
-
|----|----|
|
|
76
|
-
| [Functions](00_Functions.ipynb) | Dataset and image helpers: reading a list out of a text file, downloading single images or whole search-driven datasets, verifying and resizing what came back, building the folder layout a classifier expects, running a fastai learner over one image, Kaggle competition and dataset shortcuts, and [`graph`](https://bthek1.github.io/nbdevAuto/functions.html#graph), a `graphviz.Digraph` subclass preloaded with a rounded, filled style |
|
|
77
|
-
| [Automate](01_Automate.ipynb) | The command line surface: the prep/commit/push pipeline, GitHub and PyPI releases, reinstall and update, conda environment creation, and the help output that lists them all |
|
|
78
|
-
| [PDF Reader](02_PDF_reader.ipynb) | [`PDFreader`](https://bthek1.github.io/nbdevAuto/pdf_reader.html#pdfreader), a class that converts a PDF through `pdf2image` and renders page ranges inline when you slice it (`pdf[0:5]`) |
|
|
79
|
-
|
|
80
|
-
------------------------------------------------------------------------
|
|
81
|
-
|
|
82
|
-
## The Command Line Surface
|
|
83
|
-
|
|
84
|
-
Installing the package puts these on your `PATH`. They are ordinary
|
|
85
|
-
functions in [Automate](01_Automate.ipynb), exposed as scripts through
|
|
86
|
-
`[project.scripts]`.
|
|
87
|
-
|
|
88
|
-
| Command | Does |
|
|
89
|
-
|----|----|
|
|
90
|
-
| [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) | Export, test, and clean the notebooks, rendering the README if it needs it |
|
|
91
|
-
| [`gacp`](https://bthek1.github.io/nbdevAuto/automate.html#gacp) | git add, commit, and push, with `-m` for the message |
|
|
92
|
-
| [`status`](https://bthek1.github.io/nbdevAuto/automate.html#status) | Show the working tree state |
|
|
93
|
-
| [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) | [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) then [`gacp`](https://bthek1.github.io/nbdevAuto/automate.html#gacp), the one you actually type |
|
|
94
|
-
| [`release`](https://bthek1.github.io/nbdevAuto/automate.html#release) | Bump the version, then release to both GitHub and PyPI |
|
|
95
|
-
| `gitrelease` / `piprelease` | The two halves of [`release`](https://bthek1.github.io/nbdevAuto/automate.html#release), separately |
|
|
96
|
-
| [`reinstall`](https://bthek1.github.io/nbdevAuto/automate.html#reinstall) / [`update`](https://bthek1.github.io/nbdevAuto/automate.html#update) | Reinstall or update the package locally |
|
|
97
|
-
| [`everything`](https://bthek1.github.io/nbdevAuto/automate.html#everything) | [`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload), [`update`](https://bthek1.github.io/nbdevAuto/automate.html#update), [`release`](https://bthek1.github.io/nbdevAuto/automate.html#release), [`reinstall`](https://bthek1.github.io/nbdevAuto/automate.html#reinstall), in that order |
|
|
98
|
-
| [`create_env`](https://bthek1.github.io/nbdevAuto/automate.html#create_env) | Create a conda environment for AI work, named `fast` by default |
|
|
99
|
-
| `h` | Print the list of available commands |
|
|
100
|
-
|
|
101
|
-
Both [`prep`](https://bthek1.github.io/nbdevAuto/automate.html#prep) and
|
|
102
|
-
[`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) take
|
|
103
|
-
`-p` to choose which version part to increment.
|
|
104
|
-
|
|
105
|
-
------------------------------------------------------------------------
|
|
106
|
-
|
|
107
|
-
## Where It Is Used
|
|
108
|
-
|
|
109
|
-
The parent Knowledge repo’s `justfile` calls
|
|
110
|
-
[`upload`](https://bthek1.github.io/nbdevAuto/automate.html#upload) once
|
|
111
|
-
per submodule. `just upload` runs it only where the working tree is
|
|
112
|
-
dirty, `just full_upload` runs it everywhere. That is the main consumer,
|
|
113
|
-
so a change to `automate.py` affects the publish path for every site in
|
|
114
|
-
the collection.
|
|
115
|
-
|
|
116
|
-
------------------------------------------------------------------------
|
|
117
|
-
|
|
118
|
-
## Not Covered Yet
|
|
119
|
-
|
|
120
|
-
- **No test notebook.** The functions are exercised only by being used,
|
|
121
|
-
so a broken helper surfaces in a downstream repo rather than here.
|
|
122
|
-
- **Undeclared dependencies.** `pyproject.toml` lists only `graphviz`,
|
|
123
|
-
but the image helpers import `fastai`, the Kaggle shortcuts need the
|
|
124
|
-
Kaggle CLI and its credentials, and
|
|
125
|
-
[`PDFreader`](https://bthek1.github.io/nbdevAuto/pdf_reader.html#pdfreader)
|
|
126
|
-
needs `pdf2image` with poppler behind it, plus matplotlib. Install
|
|
127
|
-
those yourself.
|
|
128
|
-
- **`core.py` is an empty nbdev stub** (a single
|
|
129
|
-
[`foo`](https://bthek1.github.io/nbdevAuto/core.html#foo)), with no
|
|
130
|
-
notebook behind it.
|
|
131
|
-
- The docstrings are one-liners, so the rendered API pages are thin. The
|
|
132
|
-
notebooks carry the real explanation.
|
|
133
|
-
|
|
134
|
-
------------------------------------------------------------------------
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|