rolling-pin 0.9.3__tar.gz → 0.9.6__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {rolling-pin-0.9.3 → rolling-pin-0.9.6}/LICENSE +1 -1
- rolling-pin-0.9.6/PKG-INFO +429 -0
- rolling-pin-0.9.6/README.md +408 -0
- {rolling-pin-0.9.3 → rolling-pin-0.9.6}/pyproject.toml +4 -6
- {rolling-pin-0.9.3 → rolling-pin-0.9.6}/rolling_pin/command.py +2 -2
- {rolling-pin-0.9.3 → rolling-pin-0.9.6}/rolling_pin/repo_etl.py +4 -2
- {rolling-pin-0.9.3 → rolling-pin-0.9.6}/rolling_pin/tools.py +2 -2
- rolling-pin-0.9.3/PKG-INFO +0 -57
- rolling-pin-0.9.3/README.md +0 -36
- {rolling-pin-0.9.3 → rolling-pin-0.9.6}/rolling_pin/__init__.py +0 -0
- {rolling-pin-0.9.3 → rolling-pin-0.9.6}/rolling_pin/blob_etl.py +0 -0
- {rolling-pin-0.9.3 → rolling-pin-0.9.6}/rolling_pin/conform_config.py +0 -0
- {rolling-pin-0.9.3 → rolling-pin-0.9.6}/rolling_pin/conform_etl.py +0 -0
- {rolling-pin-0.9.3 → rolling-pin-0.9.6}/rolling_pin/py.typed +0 -0
- {rolling-pin-0.9.3 → rolling-pin-0.9.6}/rolling_pin/radon_etl.py +0 -0
- {rolling-pin-0.9.3 → rolling-pin-0.9.6}/rolling_pin/toml_etl.py +0 -0
|
@@ -0,0 +1,429 @@
|
|
|
1
|
+
Metadata-Version: 2.1
|
|
2
|
+
Name: rolling-pin
|
|
3
|
+
Version: 0.9.6
|
|
4
|
+
Summary: A library of generic tools for ETL work and visualization of JSON blobs and python repositories
|
|
5
|
+
License: MIT
|
|
6
|
+
Keywords: ETL,blob,dependency,graph,svg,networkx,transform,code metrics,dependency diagram,build system
|
|
7
|
+
Author-email: Alex Braun <alexander.g.braun@gmail.com>
|
|
8
|
+
Requires-Python: >=3.8
|
|
9
|
+
Classifier: Development Status :: 4 - Beta
|
|
10
|
+
Classifier: Intended Audience :: Developers
|
|
11
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.8
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
16
|
+
Classifier: Typing :: Typed
|
|
17
|
+
Project-URL: documentation, https://thenewflesh.github.io/rolling-pin
|
|
18
|
+
Project-URL: repository, https://github.com/theNewFlesh/rolling-pin
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
|
|
21
|
+
<p>
|
|
22
|
+
<a href="https://www.linkedin.com/in/alexandergbraun" rel="nofollow noreferrer">
|
|
23
|
+
<img src="https://www.gomezaparicio.com/wp-content/uploads/2012/03/linkedin-logo-1-150x150.png"
|
|
24
|
+
alt="linkedin" width="30px" height="30px"
|
|
25
|
+
>
|
|
26
|
+
</a>
|
|
27
|
+
<a href="https://github.com/theNewFlesh" rel="nofollow noreferrer">
|
|
28
|
+
<img src="https://tadeuzagallo.com/GithubPulse/assets/img/app-icon-github.png"
|
|
29
|
+
alt="github" width="30px" height="30px"
|
|
30
|
+
>
|
|
31
|
+
</a>
|
|
32
|
+
<a href="https://pypi.org/user/the-new-flesh" rel="nofollow noreferrer">
|
|
33
|
+
<img src="https://cdn.iconscout.com/icon/free/png-256/python-2-226051.png"
|
|
34
|
+
alt="pypi" width="30px" height="30px"
|
|
35
|
+
>
|
|
36
|
+
</a>
|
|
37
|
+
<a href="http://vimeo.com/user3965452" rel="nofollow noreferrer">
|
|
38
|
+
<img src="https://cdn.iconscout.com/icon/free/png-512/movie-52-151107.png?f=avif&w=512"
|
|
39
|
+
alt="vimeo" width="30px" height="30px"
|
|
40
|
+
>
|
|
41
|
+
</a>
|
|
42
|
+
<a href="http://www.alexgbraun.com" rel="nofollow noreferrer">
|
|
43
|
+
<img src="https://i.ibb.co/fvyMkpM/logo.png"
|
|
44
|
+
alt="alexgbraun" width="30px" height="30px"
|
|
45
|
+
>
|
|
46
|
+
</a>
|
|
47
|
+
</p>
|
|
48
|
+
|
|
49
|
+
<!-- <img id="logo" src="resources/logo.png" style="max-width: 717px"> -->
|
|
50
|
+
|
|
51
|
+
[](https://github.com/theNewFlesh/rolling-pin/blob/master/LICENSE)
|
|
52
|
+
[](https://github.com/theNewFlesh/rolling-pin/blob/master/docker/config/pyproject.toml)
|
|
53
|
+
[](https://pypi.org/project/rolling-pin/)
|
|
54
|
+
[](https://pepy.tech/project/rolling-pin)
|
|
55
|
+
|
|
56
|
+
# Introduction
|
|
57
|
+
Rolling-pin is a library of generic tools for ETL work and visualization of JSON
|
|
58
|
+
blobs and python repositories
|
|
59
|
+
|
|
60
|
+
See [documentation](https://thenewflesh.github.io/rolling-pin/) for details.
|
|
61
|
+
|
|
62
|
+
On the documentation main page, under the *Architecture* section, is a
|
|
63
|
+
dynamically generated dependency graph of rolling-pin's current architecture.
|
|
64
|
+
It is generated using the RepoETL class.
|
|
65
|
+
|
|
66
|
+
If you look under the *Metrics* section you will find Radon code metric plots
|
|
67
|
+
and data of the rolling-pin source code, generated by the RadonETL class.
|
|
68
|
+
|
|
69
|
+
Also have a look at this
|
|
70
|
+
**[Jupyter notebook demo](https://github.com/theNewFlesh/rolling-pin/blob/master/notebooks/prototype_demo.ipynb)**
|
|
71
|
+
for a taste of what rolling-pin can do.
|
|
72
|
+
|
|
73
|
+
---
|
|
74
|
+
|
|
75
|
+
# Installation for Developers
|
|
76
|
+
|
|
77
|
+
### Docker
|
|
78
|
+
1. Install [docker-desktop](https://docs.docker.com/desktop/)
|
|
79
|
+
2. Ensure docker-desktop has at least 4 GB of memory allocated to it.
|
|
80
|
+
3. `git clone git@github.com:theNewFlesh/rolling-pin.git`
|
|
81
|
+
4. `cd rolling-pin`
|
|
82
|
+
5. `chmod +x bin/rolling-pin`
|
|
83
|
+
6. `bin/rolling-pin docker-start`
|
|
84
|
+
- If building on a M1 Mac run `export DOCKER_DEFAULT_PLATFORM=linux/amd64` first.
|
|
85
|
+
|
|
86
|
+
The service should take a few minutes to start up.
|
|
87
|
+
|
|
88
|
+
Run `bin/rolling-pin --help` for more help on the command line tool.
|
|
89
|
+
|
|
90
|
+
### ZSH Setup
|
|
91
|
+
1. `bin/rolling-pin` must be run from this repository's top level directory.
|
|
92
|
+
2. Therefore, if using zsh, it is recommended that you paste the following line
|
|
93
|
+
in your ~/.zshrc file:
|
|
94
|
+
- `alias rolling-pin="cd [parent dir]/rolling-pin; bin/rolling-pin"`
|
|
95
|
+
- Replace `[parent dir]` with the parent directory of this repository
|
|
96
|
+
3. Consider adding the following line to your ~/.zshrc if you are using a M1 Mac:
|
|
97
|
+
- `export DOCKER_DEFAULT_PLATFORM=linux/amd64`
|
|
98
|
+
4. Running the `zsh-complete` command will enable tab completions of the cli
|
|
99
|
+
commands, in the next shell session.
|
|
100
|
+
|
|
101
|
+
For example:
|
|
102
|
+
- `rolling-pin [tab]` will show you all the cli options, which you can press
|
|
103
|
+
tab to cycle through
|
|
104
|
+
- `rolling-pin docker-[tab]` will show you only the cli options that begin with
|
|
105
|
+
"docker-"
|
|
106
|
+
|
|
107
|
+
# Installation for Production
|
|
108
|
+
|
|
109
|
+
### Python
|
|
110
|
+
`pip install rolling-pin`
|
|
111
|
+
|
|
112
|
+
### Docker
|
|
113
|
+
1. Install [docker-desktop](https://docs.docker.com/desktop/)
|
|
114
|
+
2. `docker pull theNewFlesh/rolling-pin:[version]`
|
|
115
|
+
|
|
116
|
+
|
|
117
|
+
---
|
|
118
|
+
|
|
119
|
+
# Quickstart Guide
|
|
120
|
+
This repository contains a suite commands for the whole development process.
|
|
121
|
+
This includes everything from testing, to documentation generation and
|
|
122
|
+
publishing pip packages.
|
|
123
|
+
|
|
124
|
+
These commands can be accessed through:
|
|
125
|
+
|
|
126
|
+
- The VSCode task runner
|
|
127
|
+
- The VSCode task runner side bar
|
|
128
|
+
- A terminal running on the host OS
|
|
129
|
+
- A terminal within this repositories docker container
|
|
130
|
+
|
|
131
|
+
Running the `zsh-complete` command will enable tab completions of the CLI.
|
|
132
|
+
See the zsh setup section for more information.
|
|
133
|
+
|
|
134
|
+
### Command Groups
|
|
135
|
+
|
|
136
|
+
Development commands are grouped by one of 10 prefixes:
|
|
137
|
+
|
|
138
|
+
| Command | Description |
|
|
139
|
+
| ---------- | ---------------------------------------------------------------------------------- |
|
|
140
|
+
| build | Commands for building packages for testing and pip publishing |
|
|
141
|
+
| docker | Common docker commands such as build, start and stop |
|
|
142
|
+
| docs | Commands for generating documentation and code metrics |
|
|
143
|
+
| library | Commands for managing python package dependencies |
|
|
144
|
+
| session | Commands for starting interactive sessions such as jupyter lab and python |
|
|
145
|
+
| state | Command to display the current state of the repo and container |
|
|
146
|
+
| test | Commands for running tests, linter and type annotations |
|
|
147
|
+
| version | Commands for bumping project versions |
|
|
148
|
+
| quickstart | Display this quickstart guide |
|
|
149
|
+
| zsh | Commands for running a zsh session in the container and generating zsh completions |
|
|
150
|
+
|
|
151
|
+
### Common Commands
|
|
152
|
+
|
|
153
|
+
Here are some frequently used commands to get you started:
|
|
154
|
+
|
|
155
|
+
| Command | Description |
|
|
156
|
+
| ----------------- | --------------------------------------------------------- |
|
|
157
|
+
| docker-restart | Restart container |
|
|
158
|
+
| docker-start | Start container |
|
|
159
|
+
| docker-stop | Stop container |
|
|
160
|
+
| docs-full | Generate documentation, coverage report, diagram and code |
|
|
161
|
+
| library-add | Add a given package to a given dependency group |
|
|
162
|
+
| library-graph-dev | Graph dependencies in dev environment |
|
|
163
|
+
| library-remove | Remove a given package from a given dependency group |
|
|
164
|
+
| library-search | Search for pip packages |
|
|
165
|
+
| library-update | Update dev dependencies |
|
|
166
|
+
| session-lab | Run jupyter lab server |
|
|
167
|
+
| state | State of |
|
|
168
|
+
| test-dev | Run all tests |
|
|
169
|
+
| test-lint | Run linting and type checking |
|
|
170
|
+
| zsh | Run ZSH session inside container |
|
|
171
|
+
| zsh-complete | Generate ZSH completion script |
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
# Development CLI
|
|
176
|
+
bin/rolling-pin is a command line interface (defined in cli.py) that
|
|
177
|
+
works with any version of python 2.7 and above, as it has no dependencies.
|
|
178
|
+
Commands generally do not expect any arguments or flags.
|
|
179
|
+
|
|
180
|
+
Its usage pattern is: `bin/rolling-pin COMMAND [-a --args]=ARGS [-h --help] [--dryrun]`
|
|
181
|
+
|
|
182
|
+
### Commands
|
|
183
|
+
The following is a complete list of all available development commands:
|
|
184
|
+
|
|
185
|
+
| Command | Description |
|
|
186
|
+
| ----------------------- | ------------------------------------------------------------------- |
|
|
187
|
+
| build-package | Build production version of repo for publishing |
|
|
188
|
+
| build-prod | Publish pip package of repo to PyPi |
|
|
189
|
+
| build-publish | Run production tests first then publish pip package of repo to PyPi |
|
|
190
|
+
| build-test | Build test version of repo for prod testing |
|
|
191
|
+
| docker-build | Build Docker image |
|
|
192
|
+
| docker-build-from-cache | Build Docker image from cached image |
|
|
193
|
+
| docker-build-prod | Build production image |
|
|
194
|
+
| docker-container | Display the Docker container id |
|
|
195
|
+
| docker-destroy | Shutdown container and destroy its image |
|
|
196
|
+
| docker-destroy-prod | Shutdown production container and destroy its image |
|
|
197
|
+
| docker-image | Display the Docker image id |
|
|
198
|
+
| docker-prod | Start production container |
|
|
199
|
+
| docker-pull-dev | Pull development image from Docker registry |
|
|
200
|
+
| docker-pull-prod | Pull production image from Docker registry |
|
|
201
|
+
| docker-push-dev | Push development image to Docker registry |
|
|
202
|
+
| docker-push-dev-latest | Push development image to Docker registry with dev-latest tag |
|
|
203
|
+
| docker-push-prod | Push production image to Docker registry |
|
|
204
|
+
| docker-push-prod-latest | Push production image to Docker registry with prod-latest tag |
|
|
205
|
+
| docker-remove | Remove Docker image |
|
|
206
|
+
| docker-restart | Restart Docker container |
|
|
207
|
+
| docker-start | Start Docker container |
|
|
208
|
+
| docker-stop | Stop Docker container |
|
|
209
|
+
| docs | Generate sphinx documentation |
|
|
210
|
+
| docs-architecture | Generate architecture.svg diagram from all import statements |
|
|
211
|
+
| docs-full | Generate documentation, coverage report, diagram and code |
|
|
212
|
+
| docs-metrics | Generate code metrics report, plots and tables |
|
|
213
|
+
| library-add | Add a given package to a given dependency group |
|
|
214
|
+
| library-graph-dev | Graph dependencies in dev environment |
|
|
215
|
+
| library-graph-prod | Graph dependencies in prod environment |
|
|
216
|
+
| library-install-dev | Install all dependencies into dev environment |
|
|
217
|
+
| library-install-prod | Install all dependencies into prod environment |
|
|
218
|
+
| library-list-dev | List packages in dev environment |
|
|
219
|
+
| library-list-prod | List packages in prod environment |
|
|
220
|
+
| library-lock-dev | Resolve dev.lock file |
|
|
221
|
+
| library-lock-prod | Resolve prod.lock file |
|
|
222
|
+
| library-remove | Remove a given package from a given dependency group |
|
|
223
|
+
| library-search | Search for pip packages |
|
|
224
|
+
| library-sync-dev | Sync dev environment with packages listed in dev.lock |
|
|
225
|
+
| library-sync-prod | Sync prod environment with packages listed in prod.lock |
|
|
226
|
+
| library-update | Update dev dependencies |
|
|
227
|
+
| library-update-pdm | Update PDM |
|
|
228
|
+
| quickstart | Display quickstart guide |
|
|
229
|
+
| session-lab | Run jupyter lab server |
|
|
230
|
+
| session-python | Run python session with dev dependencies |
|
|
231
|
+
| session-server | Runn application server inside Docker container |
|
|
232
|
+
| state | State of repository and Docker container |
|
|
233
|
+
| test-coverage | Generate test coverage report |
|
|
234
|
+
| test-dev | Run all tests |
|
|
235
|
+
| test-fast | Test all code excepts tests marked with SKIP_SLOWS_TESTS decorator |
|
|
236
|
+
| test-lint | Run linting and type checking |
|
|
237
|
+
| test-prod | Run tests across all support python versions |
|
|
238
|
+
| version | Full resolution of repo: dependencies, linting, tests, docs, etc |
|
|
239
|
+
| version-bump-major | Bump pyproject major version |
|
|
240
|
+
| version-bump-minor | Bump pyproject minor version |
|
|
241
|
+
| version-bump-patch | Bump pyproject patch version |
|
|
242
|
+
| version-commit | Tag with version and commit changes to master |
|
|
243
|
+
| zsh | Run ZSH session inside Docker container |
|
|
244
|
+
| zsh-complete | Generate oh-my-zsh completions |
|
|
245
|
+
| zsh-root | Run ZSH session as root inside Docker container |
|
|
246
|
+
|
|
247
|
+
### Flags
|
|
248
|
+
|
|
249
|
+
| Short | Long | Description |
|
|
250
|
+
| ----- | --------- | ---------------------------------------------------- |
|
|
251
|
+
| -a | --args | Additional arguments, this can generally be ignored |
|
|
252
|
+
| -h | --help | Prints command help message to stdout |
|
|
253
|
+
| | --dryrun | Prints command that would otherwise be run to stdout |
|
|
254
|
+
|
|
255
|
+
---
|
|
256
|
+
|
|
257
|
+
# Production CLI
|
|
258
|
+
|
|
259
|
+
Rolling-pin comes with a command line interface defined in command.py.
|
|
260
|
+
|
|
261
|
+
Its usage pattern is: `rolling-pin COMMAND [ARGS] [FLAGS] [-h --help]`
|
|
262
|
+
|
|
263
|
+
## Commands
|
|
264
|
+
|
|
265
|
+
---
|
|
266
|
+
|
|
267
|
+
### bash-completion
|
|
268
|
+
Prints BASH completion code to be written to a _rolling-pin completion file
|
|
269
|
+
|
|
270
|
+
Usage: `rolling-pin bash-completion`
|
|
271
|
+
|
|
272
|
+
---
|
|
273
|
+
|
|
274
|
+
### zsh-completion
|
|
275
|
+
Prints ZSH completion code to be written to a _rolling-pin completion file
|
|
276
|
+
|
|
277
|
+
Usage: `rolling-pin zsh-completion`
|
|
278
|
+
|
|
279
|
+
---
|
|
280
|
+
|
|
281
|
+
### conform
|
|
282
|
+
Copies source files to target filepaths according to given conform file.
|
|
283
|
+
|
|
284
|
+
Usage: `rolling-pin conform [OPTIONS] SOURCE`
|
|
285
|
+
|
|
286
|
+
| Argument | Description |
|
|
287
|
+
| -------- | --------------------- |
|
|
288
|
+
| source | conform YAML filepath |
|
|
289
|
+
|
|
290
|
+
| Flag | Argument | Description | Default |
|
|
291
|
+
| --------- | -------- | ---------------------------------------------- | ------- |
|
|
292
|
+
| --groups | text | comma separated list of groups to be conformed | all |
|
|
293
|
+
| --dryrun | <p></p> | Print out conform table instead of run conform | <p></p> |
|
|
294
|
+
| --help | <p></p> | print help message | <p></p> |
|
|
295
|
+
|
|
296
|
+
---
|
|
297
|
+
|
|
298
|
+
### graph
|
|
299
|
+
Generate a dependency graph of a source repository and write it to a given filepath
|
|
300
|
+
|
|
301
|
+
Usage: `rolling-pin graph [OPTIONS] SOURCE TARGET`
|
|
302
|
+
|
|
303
|
+
| Argument | Description |
|
|
304
|
+
| -------- | --------------- |
|
|
305
|
+
| source | repository path |
|
|
306
|
+
| target | target filepath |
|
|
307
|
+
|
|
308
|
+
| Flag | Argument | Description | Default |
|
|
309
|
+
| --------- | -------- | ------------------------------------------- | --------- |
|
|
310
|
+
| --include | text | include files that match this regex pattern | .*\.py$' |
|
|
311
|
+
| --exclude | text | exclude files that match this regex pattern | test|mock |
|
|
312
|
+
| --orient | text | graph orientation | lr |
|
|
313
|
+
| --help | <p></p> | print help message | <p></p> |
|
|
314
|
+
|
|
315
|
+
---
|
|
316
|
+
|
|
317
|
+
### plot
|
|
318
|
+
Write radon metrics plots of given repository to given filepath.
|
|
319
|
+
|
|
320
|
+
Usage: `rolling-pin plot [OPTIONS] SOURCE TARGET`
|
|
321
|
+
|
|
322
|
+
| Argument | Description |
|
|
323
|
+
| -------- | --------------- |
|
|
324
|
+
| source | repository path |
|
|
325
|
+
| target | plot filepath |
|
|
326
|
+
|
|
327
|
+
| Flag | Description |
|
|
328
|
+
| ------ | ------------------ |
|
|
329
|
+
| --help | print help message |
|
|
330
|
+
|
|
331
|
+
---
|
|
332
|
+
|
|
333
|
+
### table
|
|
334
|
+
Write radon metrics tables of given repository to given directory.
|
|
335
|
+
|
|
336
|
+
Usage: `rolling-pin table [OPTIONS] SOURCE TARGET`
|
|
337
|
+
|
|
338
|
+
| Argument | Description |
|
|
339
|
+
| -------- | --------------- |
|
|
340
|
+
| source | repository path |
|
|
341
|
+
| target | table directory |
|
|
342
|
+
|
|
343
|
+
| Flag | Description |
|
|
344
|
+
| ------ | ------------------ |
|
|
345
|
+
| --help | print help message |
|
|
346
|
+
|
|
347
|
+
---
|
|
348
|
+
|
|
349
|
+
### toml
|
|
350
|
+
Generate a copy of a given TOML file with given edits indicated by flags. Flags
|
|
351
|
+
are evalauted in the following order: edit, delete, search. Flags may be
|
|
352
|
+
arbitrarily combined. Edit and delete flags may appear multiple times.
|
|
353
|
+
|
|
354
|
+
Usage: `rolling-pin toml [OPTIONS] SOURCE`
|
|
355
|
+
|
|
356
|
+
| Argument | Description |
|
|
357
|
+
| -------- | ------------- |
|
|
358
|
+
| source | TOML filepath |
|
|
359
|
+
|
|
360
|
+
| Flag | Argument | Description |
|
|
361
|
+
| --------- | -------- | ----------------------------------------------------------------------------------------- |
|
|
362
|
+
| --edit | text | replace key's value with given value, text is "=" separated key value pair in TOML format |
|
|
363
|
+
| --delete | text | delete keys that match this regular expression |
|
|
364
|
+
| --search | text | search for keys that match this regular expression |
|
|
365
|
+
| --target | text | target filepath to write to |
|
|
366
|
+
| --help | <p></p> | print help message |
|
|
367
|
+
|
|
368
|
+
#### *Example Usage*
|
|
369
|
+
**example file**
|
|
370
|
+
```
|
|
371
|
+
>>>cat example.toml
|
|
372
|
+
[root]
|
|
373
|
+
a = 1
|
|
374
|
+
b = 2
|
|
375
|
+
|
|
376
|
+
[root.foo.bar]
|
|
377
|
+
x = "y"
|
|
378
|
+
|
|
379
|
+
[world]
|
|
380
|
+
hello = true
|
|
381
|
+
```
|
|
382
|
+
|
|
383
|
+
**edit flag**
|
|
384
|
+
```
|
|
385
|
+
>>>rolling-pin toml foobar.toml --edit 'root.a=999'
|
|
386
|
+
[root]
|
|
387
|
+
a = 999
|
|
388
|
+
b = 2...
|
|
389
|
+
|
|
390
|
+
>>>rolling-pin toml foobar.toml \
|
|
391
|
+
--edit 'root.a=[1, 2]' \
|
|
392
|
+
--edit 'root.b="xxx"'
|
|
393
|
+
[root]
|
|
394
|
+
a = [
|
|
395
|
+
1,
|
|
396
|
+
2,
|
|
397
|
+
]
|
|
398
|
+
b = "xxx"...
|
|
399
|
+
|
|
400
|
+
>>>rolling-pin toml foobar.toml --edit 'root.foo.bar="baz"'
|
|
401
|
+
...
|
|
402
|
+
hello = true
|
|
403
|
+
|
|
404
|
+
[root.foo]
|
|
405
|
+
bar = "baz"...
|
|
406
|
+
```
|
|
407
|
+
|
|
408
|
+
**delete flag**
|
|
409
|
+
```
|
|
410
|
+
>>>rolling-pin toml foobar.toml \
|
|
411
|
+
--delete 'root.foo.bar' \
|
|
412
|
+
--delete 'root.a'
|
|
413
|
+
[root]
|
|
414
|
+
b = 2
|
|
415
|
+
|
|
416
|
+
[world]
|
|
417
|
+
hello = true
|
|
418
|
+
```
|
|
419
|
+
|
|
420
|
+
**search flag**
|
|
421
|
+
```
|
|
422
|
+
>>>rolling-pin toml foobar.toml --search 'root.foo|world'
|
|
423
|
+
[world]
|
|
424
|
+
hello = true
|
|
425
|
+
|
|
426
|
+
[root.foo.bar]
|
|
427
|
+
x = "y"
|
|
428
|
+
```
|
|
429
|
+
|
|
@@ -0,0 +1,408 @@
|
|
|
1
|
+
<p>
|
|
2
|
+
<a href="https://www.linkedin.com/in/alexandergbraun" rel="nofollow noreferrer">
|
|
3
|
+
<img src="https://www.gomezaparicio.com/wp-content/uploads/2012/03/linkedin-logo-1-150x150.png"
|
|
4
|
+
alt="linkedin" width="30px" height="30px"
|
|
5
|
+
>
|
|
6
|
+
</a>
|
|
7
|
+
<a href="https://github.com/theNewFlesh" rel="nofollow noreferrer">
|
|
8
|
+
<img src="https://tadeuzagallo.com/GithubPulse/assets/img/app-icon-github.png"
|
|
9
|
+
alt="github" width="30px" height="30px"
|
|
10
|
+
>
|
|
11
|
+
</a>
|
|
12
|
+
<a href="https://pypi.org/user/the-new-flesh" rel="nofollow noreferrer">
|
|
13
|
+
<img src="https://cdn.iconscout.com/icon/free/png-256/python-2-226051.png"
|
|
14
|
+
alt="pypi" width="30px" height="30px"
|
|
15
|
+
>
|
|
16
|
+
</a>
|
|
17
|
+
<a href="http://vimeo.com/user3965452" rel="nofollow noreferrer">
|
|
18
|
+
<img src="https://cdn.iconscout.com/icon/free/png-512/movie-52-151107.png?f=avif&w=512"
|
|
19
|
+
alt="vimeo" width="30px" height="30px"
|
|
20
|
+
>
|
|
21
|
+
</a>
|
|
22
|
+
<a href="http://www.alexgbraun.com" rel="nofollow noreferrer">
|
|
23
|
+
<img src="https://i.ibb.co/fvyMkpM/logo.png"
|
|
24
|
+
alt="alexgbraun" width="30px" height="30px"
|
|
25
|
+
>
|
|
26
|
+
</a>
|
|
27
|
+
</p>
|
|
28
|
+
|
|
29
|
+
<!-- <img id="logo" src="resources/logo.png" style="max-width: 717px"> -->
|
|
30
|
+
|
|
31
|
+
[](https://github.com/theNewFlesh/rolling-pin/blob/master/LICENSE)
|
|
32
|
+
[](https://github.com/theNewFlesh/rolling-pin/blob/master/docker/config/pyproject.toml)
|
|
33
|
+
[](https://pypi.org/project/rolling-pin/)
|
|
34
|
+
[](https://pepy.tech/project/rolling-pin)
|
|
35
|
+
|
|
36
|
+
# Introduction
|
|
37
|
+
Rolling-pin is a library of generic tools for ETL work and visualization of JSON
|
|
38
|
+
blobs and python repositories
|
|
39
|
+
|
|
40
|
+
See [documentation](https://thenewflesh.github.io/rolling-pin/) for details.
|
|
41
|
+
|
|
42
|
+
On the documentation main page, under the *Architecture* section, is a
|
|
43
|
+
dynamically generated dependency graph of rolling-pin's current architecture.
|
|
44
|
+
It is generated using the RepoETL class.
|
|
45
|
+
|
|
46
|
+
If you look under the *Metrics* section you will find Radon code metric plots
|
|
47
|
+
and data of the rolling-pin source code, generated by the RadonETL class.
|
|
48
|
+
|
|
49
|
+
Also have a look at this
|
|
50
|
+
**[Jupyter notebook demo](https://github.com/theNewFlesh/rolling-pin/blob/master/notebooks/prototype_demo.ipynb)**
|
|
51
|
+
for a taste of what rolling-pin can do.
|
|
52
|
+
|
|
53
|
+
---
|
|
54
|
+
|
|
55
|
+
# Installation for Developers
|
|
56
|
+
|
|
57
|
+
### Docker
|
|
58
|
+
1. Install [docker-desktop](https://docs.docker.com/desktop/)
|
|
59
|
+
2. Ensure docker-desktop has at least 4 GB of memory allocated to it.
|
|
60
|
+
3. `git clone git@github.com:theNewFlesh/rolling-pin.git`
|
|
61
|
+
4. `cd rolling-pin`
|
|
62
|
+
5. `chmod +x bin/rolling-pin`
|
|
63
|
+
6. `bin/rolling-pin docker-start`
|
|
64
|
+
- If building on a M1 Mac run `export DOCKER_DEFAULT_PLATFORM=linux/amd64` first.
|
|
65
|
+
|
|
66
|
+
The service should take a few minutes to start up.
|
|
67
|
+
|
|
68
|
+
Run `bin/rolling-pin --help` for more help on the command line tool.
|
|
69
|
+
|
|
70
|
+
### ZSH Setup
|
|
71
|
+
1. `bin/rolling-pin` must be run from this repository's top level directory.
|
|
72
|
+
2. Therefore, if using zsh, it is recommended that you paste the following line
|
|
73
|
+
in your ~/.zshrc file:
|
|
74
|
+
- `alias rolling-pin="cd [parent dir]/rolling-pin; bin/rolling-pin"`
|
|
75
|
+
- Replace `[parent dir]` with the parent directory of this repository
|
|
76
|
+
3. Consider adding the following line to your ~/.zshrc if you are using a M1 Mac:
|
|
77
|
+
- `export DOCKER_DEFAULT_PLATFORM=linux/amd64`
|
|
78
|
+
4. Running the `zsh-complete` command will enable tab completions of the cli
|
|
79
|
+
commands, in the next shell session.
|
|
80
|
+
|
|
81
|
+
For example:
|
|
82
|
+
- `rolling-pin [tab]` will show you all the cli options, which you can press
|
|
83
|
+
tab to cycle through
|
|
84
|
+
- `rolling-pin docker-[tab]` will show you only the cli options that begin with
|
|
85
|
+
"docker-"
|
|
86
|
+
|
|
87
|
+
# Installation for Production
|
|
88
|
+
|
|
89
|
+
### Python
|
|
90
|
+
`pip install rolling-pin`
|
|
91
|
+
|
|
92
|
+
### Docker
|
|
93
|
+
1. Install [docker-desktop](https://docs.docker.com/desktop/)
|
|
94
|
+
2. `docker pull theNewFlesh/rolling-pin:[version]`
|
|
95
|
+
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
# Quickstart Guide
|
|
100
|
+
This repository contains a suite commands for the whole development process.
|
|
101
|
+
This includes everything from testing, to documentation generation and
|
|
102
|
+
publishing pip packages.
|
|
103
|
+
|
|
104
|
+
These commands can be accessed through:
|
|
105
|
+
|
|
106
|
+
- The VSCode task runner
|
|
107
|
+
- The VSCode task runner side bar
|
|
108
|
+
- A terminal running on the host OS
|
|
109
|
+
- A terminal within this repositories docker container
|
|
110
|
+
|
|
111
|
+
Running the `zsh-complete` command will enable tab completions of the CLI.
|
|
112
|
+
See the zsh setup section for more information.
|
|
113
|
+
|
|
114
|
+
### Command Groups
|
|
115
|
+
|
|
116
|
+
Development commands are grouped by one of 10 prefixes:
|
|
117
|
+
|
|
118
|
+
| Command | Description |
|
|
119
|
+
| ---------- | ---------------------------------------------------------------------------------- |
|
|
120
|
+
| build | Commands for building packages for testing and pip publishing |
|
|
121
|
+
| docker | Common docker commands such as build, start and stop |
|
|
122
|
+
| docs | Commands for generating documentation and code metrics |
|
|
123
|
+
| library | Commands for managing python package dependencies |
|
|
124
|
+
| session | Commands for starting interactive sessions such as jupyter lab and python |
|
|
125
|
+
| state | Command to display the current state of the repo and container |
|
|
126
|
+
| test | Commands for running tests, linter and type annotations |
|
|
127
|
+
| version | Commands for bumping project versions |
|
|
128
|
+
| quickstart | Display this quickstart guide |
|
|
129
|
+
| zsh | Commands for running a zsh session in the container and generating zsh completions |
|
|
130
|
+
|
|
131
|
+
### Common Commands
|
|
132
|
+
|
|
133
|
+
Here are some frequently used commands to get you started:
|
|
134
|
+
|
|
135
|
+
| Command | Description |
|
|
136
|
+
| ----------------- | --------------------------------------------------------- |
|
|
137
|
+
| docker-restart | Restart container |
|
|
138
|
+
| docker-start | Start container |
|
|
139
|
+
| docker-stop | Stop container |
|
|
140
|
+
| docs-full | Generate documentation, coverage report, diagram and code |
|
|
141
|
+
| library-add | Add a given package to a given dependency group |
|
|
142
|
+
| library-graph-dev | Graph dependencies in dev environment |
|
|
143
|
+
| library-remove | Remove a given package from a given dependency group |
|
|
144
|
+
| library-search | Search for pip packages |
|
|
145
|
+
| library-update | Update dev dependencies |
|
|
146
|
+
| session-lab | Run jupyter lab server |
|
|
147
|
+
| state | State of |
|
|
148
|
+
| test-dev | Run all tests |
|
|
149
|
+
| test-lint | Run linting and type checking |
|
|
150
|
+
| zsh | Run ZSH session inside container |
|
|
151
|
+
| zsh-complete | Generate ZSH completion script |
|
|
152
|
+
|
|
153
|
+
---
|
|
154
|
+
|
|
155
|
+
# Development CLI
|
|
156
|
+
bin/rolling-pin is a command line interface (defined in cli.py) that
|
|
157
|
+
works with any version of python 2.7 and above, as it has no dependencies.
|
|
158
|
+
Commands generally do not expect any arguments or flags.
|
|
159
|
+
|
|
160
|
+
Its usage pattern is: `bin/rolling-pin COMMAND [-a --args]=ARGS [-h --help] [--dryrun]`
|
|
161
|
+
|
|
162
|
+
### Commands
|
|
163
|
+
The following is a complete list of all available development commands:
|
|
164
|
+
|
|
165
|
+
| Command | Description |
|
|
166
|
+
| ----------------------- | ------------------------------------------------------------------- |
|
|
167
|
+
| build-package | Build production version of repo for publishing |
|
|
168
|
+
| build-prod | Publish pip package of repo to PyPi |
|
|
169
|
+
| build-publish | Run production tests first then publish pip package of repo to PyPi |
|
|
170
|
+
| build-test | Build test version of repo for prod testing |
|
|
171
|
+
| docker-build | Build Docker image |
|
|
172
|
+
| docker-build-from-cache | Build Docker image from cached image |
|
|
173
|
+
| docker-build-prod | Build production image |
|
|
174
|
+
| docker-container | Display the Docker container id |
|
|
175
|
+
| docker-destroy | Shutdown container and destroy its image |
|
|
176
|
+
| docker-destroy-prod | Shutdown production container and destroy its image |
|
|
177
|
+
| docker-image | Display the Docker image id |
|
|
178
|
+
| docker-prod | Start production container |
|
|
179
|
+
| docker-pull-dev | Pull development image from Docker registry |
|
|
180
|
+
| docker-pull-prod | Pull production image from Docker registry |
|
|
181
|
+
| docker-push-dev | Push development image to Docker registry |
|
|
182
|
+
| docker-push-dev-latest | Push development image to Docker registry with dev-latest tag |
|
|
183
|
+
| docker-push-prod | Push production image to Docker registry |
|
|
184
|
+
| docker-push-prod-latest | Push production image to Docker registry with prod-latest tag |
|
|
185
|
+
| docker-remove | Remove Docker image |
|
|
186
|
+
| docker-restart | Restart Docker container |
|
|
187
|
+
| docker-start | Start Docker container |
|
|
188
|
+
| docker-stop | Stop Docker container |
|
|
189
|
+
| docs | Generate sphinx documentation |
|
|
190
|
+
| docs-architecture | Generate architecture.svg diagram from all import statements |
|
|
191
|
+
| docs-full | Generate documentation, coverage report, diagram and code |
|
|
192
|
+
| docs-metrics | Generate code metrics report, plots and tables |
|
|
193
|
+
| library-add | Add a given package to a given dependency group |
|
|
194
|
+
| library-graph-dev | Graph dependencies in dev environment |
|
|
195
|
+
| library-graph-prod | Graph dependencies in prod environment |
|
|
196
|
+
| library-install-dev | Install all dependencies into dev environment |
|
|
197
|
+
| library-install-prod | Install all dependencies into prod environment |
|
|
198
|
+
| library-list-dev | List packages in dev environment |
|
|
199
|
+
| library-list-prod | List packages in prod environment |
|
|
200
|
+
| library-lock-dev | Resolve dev.lock file |
|
|
201
|
+
| library-lock-prod | Resolve prod.lock file |
|
|
202
|
+
| library-remove | Remove a given package from a given dependency group |
|
|
203
|
+
| library-search | Search for pip packages |
|
|
204
|
+
| library-sync-dev | Sync dev environment with packages listed in dev.lock |
|
|
205
|
+
| library-sync-prod | Sync prod environment with packages listed in prod.lock |
|
|
206
|
+
| library-update | Update dev dependencies |
|
|
207
|
+
| library-update-pdm | Update PDM |
|
|
208
|
+
| quickstart | Display quickstart guide |
|
|
209
|
+
| session-lab | Run jupyter lab server |
|
|
210
|
+
| session-python | Run python session with dev dependencies |
|
|
211
|
+
| session-server | Runn application server inside Docker container |
|
|
212
|
+
| state | State of repository and Docker container |
|
|
213
|
+
| test-coverage | Generate test coverage report |
|
|
214
|
+
| test-dev | Run all tests |
|
|
215
|
+
| test-fast | Test all code excepts tests marked with SKIP_SLOWS_TESTS decorator |
|
|
216
|
+
| test-lint | Run linting and type checking |
|
|
217
|
+
| test-prod | Run tests across all support python versions |
|
|
218
|
+
| version | Full resolution of repo: dependencies, linting, tests, docs, etc |
|
|
219
|
+
| version-bump-major | Bump pyproject major version |
|
|
220
|
+
| version-bump-minor | Bump pyproject minor version |
|
|
221
|
+
| version-bump-patch | Bump pyproject patch version |
|
|
222
|
+
| version-commit | Tag with version and commit changes to master |
|
|
223
|
+
| zsh | Run ZSH session inside Docker container |
|
|
224
|
+
| zsh-complete | Generate oh-my-zsh completions |
|
|
225
|
+
| zsh-root | Run ZSH session as root inside Docker container |
|
|
226
|
+
|
|
227
|
+
### Flags
|
|
228
|
+
|
|
229
|
+
| Short | Long | Description |
|
|
230
|
+
| ----- | --------- | ---------------------------------------------------- |
|
|
231
|
+
| -a | --args | Additional arguments, this can generally be ignored |
|
|
232
|
+
| -h | --help | Prints command help message to stdout |
|
|
233
|
+
| | --dryrun | Prints command that would otherwise be run to stdout |
|
|
234
|
+
|
|
235
|
+
---
|
|
236
|
+
|
|
237
|
+
# Production CLI
|
|
238
|
+
|
|
239
|
+
Rolling-pin comes with a command line interface defined in command.py.
|
|
240
|
+
|
|
241
|
+
Its usage pattern is: `rolling-pin COMMAND [ARGS] [FLAGS] [-h --help]`
|
|
242
|
+
|
|
243
|
+
## Commands
|
|
244
|
+
|
|
245
|
+
---
|
|
246
|
+
|
|
247
|
+
### bash-completion
|
|
248
|
+
Prints BASH completion code to be written to a _rolling-pin completion file
|
|
249
|
+
|
|
250
|
+
Usage: `rolling-pin bash-completion`
|
|
251
|
+
|
|
252
|
+
---
|
|
253
|
+
|
|
254
|
+
### zsh-completion
|
|
255
|
+
Prints ZSH completion code to be written to a _rolling-pin completion file
|
|
256
|
+
|
|
257
|
+
Usage: `rolling-pin zsh-completion`
|
|
258
|
+
|
|
259
|
+
---
|
|
260
|
+
|
|
261
|
+
### conform
|
|
262
|
+
Copies source files to target filepaths according to given conform file.
|
|
263
|
+
|
|
264
|
+
Usage: `rolling-pin conform [OPTIONS] SOURCE`
|
|
265
|
+
|
|
266
|
+
| Argument | Description |
|
|
267
|
+
| -------- | --------------------- |
|
|
268
|
+
| source | conform YAML filepath |
|
|
269
|
+
|
|
270
|
+
| Flag | Argument | Description | Default |
|
|
271
|
+
| --------- | -------- | ---------------------------------------------- | ------- |
|
|
272
|
+
| --groups | text | comma separated list of groups to be conformed | all |
|
|
273
|
+
| --dryrun | <p></p> | Print out conform table instead of run conform | <p></p> |
|
|
274
|
+
| --help | <p></p> | print help message | <p></p> |
|
|
275
|
+
|
|
276
|
+
---
|
|
277
|
+
|
|
278
|
+
### graph
|
|
279
|
+
Generate a dependency graph of a source repository and write it to a given filepath
|
|
280
|
+
|
|
281
|
+
Usage: `rolling-pin graph [OPTIONS] SOURCE TARGET`
|
|
282
|
+
|
|
283
|
+
| Argument | Description |
|
|
284
|
+
| -------- | --------------- |
|
|
285
|
+
| source | repository path |
|
|
286
|
+
| target | target filepath |
|
|
287
|
+
|
|
288
|
+
| Flag | Argument | Description | Default |
|
|
289
|
+
| --------- | -------- | ------------------------------------------- | --------- |
|
|
290
|
+
| --include | text | include files that match this regex pattern | .*\.py$' |
|
|
291
|
+
| --exclude | text | exclude files that match this regex pattern | test|mock |
|
|
292
|
+
| --orient | text | graph orientation | lr |
|
|
293
|
+
| --help | <p></p> | print help message | <p></p> |
|
|
294
|
+
|
|
295
|
+
---
|
|
296
|
+
|
|
297
|
+
### plot
|
|
298
|
+
Write radon metrics plots of given repository to given filepath.
|
|
299
|
+
|
|
300
|
+
Usage: `rolling-pin plot [OPTIONS] SOURCE TARGET`
|
|
301
|
+
|
|
302
|
+
| Argument | Description |
|
|
303
|
+
| -------- | --------------- |
|
|
304
|
+
| source | repository path |
|
|
305
|
+
| target | plot filepath |
|
|
306
|
+
|
|
307
|
+
| Flag | Description |
|
|
308
|
+
| ------ | ------------------ |
|
|
309
|
+
| --help | print help message |
|
|
310
|
+
|
|
311
|
+
---
|
|
312
|
+
|
|
313
|
+
### table
|
|
314
|
+
Write radon metrics tables of given repository to given directory.
|
|
315
|
+
|
|
316
|
+
Usage: `rolling-pin table [OPTIONS] SOURCE TARGET`
|
|
317
|
+
|
|
318
|
+
| Argument | Description |
|
|
319
|
+
| -------- | --------------- |
|
|
320
|
+
| source | repository path |
|
|
321
|
+
| target | table directory |
|
|
322
|
+
|
|
323
|
+
| Flag | Description |
|
|
324
|
+
| ------ | ------------------ |
|
|
325
|
+
| --help | print help message |
|
|
326
|
+
|
|
327
|
+
---
|
|
328
|
+
|
|
329
|
+
### toml
|
|
330
|
+
Generate a copy of a given TOML file with given edits indicated by flags. Flags
|
|
331
|
+
are evalauted in the following order: edit, delete, search. Flags may be
|
|
332
|
+
arbitrarily combined. Edit and delete flags may appear multiple times.
|
|
333
|
+
|
|
334
|
+
Usage: `rolling-pin toml [OPTIONS] SOURCE`
|
|
335
|
+
|
|
336
|
+
| Argument | Description |
|
|
337
|
+
| -------- | ------------- |
|
|
338
|
+
| source | TOML filepath |
|
|
339
|
+
|
|
340
|
+
| Flag | Argument | Description |
|
|
341
|
+
| --------- | -------- | ----------------------------------------------------------------------------------------- |
|
|
342
|
+
| --edit | text | replace key's value with given value, text is "=" separated key value pair in TOML format |
|
|
343
|
+
| --delete | text | delete keys that match this regular expression |
|
|
344
|
+
| --search | text | search for keys that match this regular expression |
|
|
345
|
+
| --target | text | target filepath to write to |
|
|
346
|
+
| --help | <p></p> | print help message |
|
|
347
|
+
|
|
348
|
+
#### *Example Usage*
|
|
349
|
+
**example file**
|
|
350
|
+
```
|
|
351
|
+
>>>cat example.toml
|
|
352
|
+
[root]
|
|
353
|
+
a = 1
|
|
354
|
+
b = 2
|
|
355
|
+
|
|
356
|
+
[root.foo.bar]
|
|
357
|
+
x = "y"
|
|
358
|
+
|
|
359
|
+
[world]
|
|
360
|
+
hello = true
|
|
361
|
+
```
|
|
362
|
+
|
|
363
|
+
**edit flag**
|
|
364
|
+
```
|
|
365
|
+
>>>rolling-pin toml foobar.toml --edit 'root.a=999'
|
|
366
|
+
[root]
|
|
367
|
+
a = 999
|
|
368
|
+
b = 2...
|
|
369
|
+
|
|
370
|
+
>>>rolling-pin toml foobar.toml \
|
|
371
|
+
--edit 'root.a=[1, 2]' \
|
|
372
|
+
--edit 'root.b="xxx"'
|
|
373
|
+
[root]
|
|
374
|
+
a = [
|
|
375
|
+
1,
|
|
376
|
+
2,
|
|
377
|
+
]
|
|
378
|
+
b = "xxx"...
|
|
379
|
+
|
|
380
|
+
>>>rolling-pin toml foobar.toml --edit 'root.foo.bar="baz"'
|
|
381
|
+
...
|
|
382
|
+
hello = true
|
|
383
|
+
|
|
384
|
+
[root.foo]
|
|
385
|
+
bar = "baz"...
|
|
386
|
+
```
|
|
387
|
+
|
|
388
|
+
**delete flag**
|
|
389
|
+
```
|
|
390
|
+
>>>rolling-pin toml foobar.toml \
|
|
391
|
+
--delete 'root.foo.bar' \
|
|
392
|
+
--delete 'root.a'
|
|
393
|
+
[root]
|
|
394
|
+
b = 2
|
|
395
|
+
|
|
396
|
+
[world]
|
|
397
|
+
hello = true
|
|
398
|
+
```
|
|
399
|
+
|
|
400
|
+
**search flag**
|
|
401
|
+
```
|
|
402
|
+
>>>rolling-pin toml foobar.toml --search 'root.foo|world'
|
|
403
|
+
[world]
|
|
404
|
+
hello = true
|
|
405
|
+
|
|
406
|
+
[root.foo.bar]
|
|
407
|
+
x = "y"
|
|
408
|
+
```
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
[project]
|
|
2
2
|
name = "rolling-pin"
|
|
3
|
-
version = "0.9.
|
|
3
|
+
version = "0.9.6"
|
|
4
4
|
description = "A library of generic tools for ETL work and visualization of JSON blobs and python repositories"
|
|
5
5
|
readme = "README.md"
|
|
6
6
|
license = "MIT"
|
|
@@ -34,11 +34,11 @@ dependencies = [
|
|
|
34
34
|
"ipython",
|
|
35
35
|
"lunchbox",
|
|
36
36
|
"networkx",
|
|
37
|
-
"numpy>=1.23.4",
|
|
37
|
+
"numpy>=1.23.4,<=1.24.4",
|
|
38
38
|
"pandas>=1.1.5",
|
|
39
|
-
"pydot",
|
|
39
|
+
"pydot<=1.4.2",
|
|
40
40
|
"pyyaml",
|
|
41
|
-
"radon",
|
|
41
|
+
"radon<6.0.0",
|
|
42
42
|
"schematics",
|
|
43
43
|
"toml>=0.10.2",
|
|
44
44
|
"wrapt",
|
|
@@ -54,8 +54,6 @@ documentation = "https://thenewflesh.github.io/rolling-pin"
|
|
|
54
54
|
[project.scripts]
|
|
55
55
|
rolling-pin = "rolling_pin.command:main"
|
|
56
56
|
|
|
57
|
-
[project.optional-dependencies]
|
|
58
|
-
|
|
59
57
|
[build-system]
|
|
60
58
|
requires = [
|
|
61
59
|
"pdm-pep517>=1.0.0",
|
|
@@ -80,8 +80,8 @@ def graph(source, target, include, exclude, orient):
|
|
|
80
80
|
SOURCE - repository path
|
|
81
81
|
TARGET - target filepath
|
|
82
82
|
'''
|
|
83
|
-
include_ =
|
|
84
|
-
exclude_ =
|
|
83
|
+
include_ = '' if include is None else include
|
|
84
|
+
exclude_ = '' if exclude is None else exclude
|
|
85
85
|
RepoETL(source, include_, exclude_).write(target, orient=orient)
|
|
86
86
|
|
|
87
87
|
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
from typing import Any, Dict, Iterator, List, Optional, Union # noqa: F401
|
|
2
|
-
from IPython.display import HTML # noqa: F401
|
|
2
|
+
from IPython.display import HTML, Image # noqa: F401
|
|
3
3
|
|
|
4
4
|
from itertools import chain
|
|
5
5
|
from pathlib import Path
|
|
@@ -275,6 +275,8 @@ class RepoETL():
|
|
|
275
275
|
'''
|
|
276
276
|
x = anneal_axis
|
|
277
277
|
y = pin_axis
|
|
278
|
+
data[x] = data[x].astype(float)
|
|
279
|
+
data[y] = data[y].astype(float)
|
|
278
280
|
for iteration in range(iterations):
|
|
279
281
|
# create directed graph from data
|
|
280
282
|
graph = RepoETL._to_networkx_graph(data)
|
|
@@ -484,7 +486,7 @@ class RepoETL():
|
|
|
484
486
|
color_scheme=None,
|
|
485
487
|
as_png=False
|
|
486
488
|
):
|
|
487
|
-
# type: (str, bool, Optional[Dict[str, str]], bool) -> HTML
|
|
489
|
+
# type: (str, bool, Optional[Dict[str, str]], bool) -> Union[HTML, Image]
|
|
488
490
|
'''
|
|
489
491
|
For use in inline rendering of graph data in Jupyter Lab.
|
|
490
492
|
|
|
@@ -340,9 +340,9 @@ def filter_text(
|
|
|
340
340
|
'''
|
|
341
341
|
lines = text.split('\n')
|
|
342
342
|
if include_regex is not None:
|
|
343
|
-
lines = list(filter(lambda x: re.search(include_regex, x), lines))
|
|
343
|
+
lines = list(filter(lambda x: re.search(include_regex, x), lines))
|
|
344
344
|
if exclude_regex is not None:
|
|
345
|
-
lines = list(filter(lambda x: not re.search(exclude_regex, x), lines))
|
|
345
|
+
lines = list(filter(lambda x: not re.search(exclude_regex, x), lines))
|
|
346
346
|
if replace_regex is not None:
|
|
347
347
|
rep_val = replace_value or ''
|
|
348
348
|
lines = [re.sub(replace_regex, rep_val, x) for x in lines]
|
rolling-pin-0.9.3/PKG-INFO
DELETED
|
@@ -1,57 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.1
|
|
2
|
-
Name: rolling-pin
|
|
3
|
-
Version: 0.9.3
|
|
4
|
-
Summary: A library of generic tools for ETL work and visualization of JSON blobs and python repositories
|
|
5
|
-
License: MIT
|
|
6
|
-
Keywords: ETL,blob,dependency,graph,svg,networkx,transform,code metrics,dependency diagram,build system
|
|
7
|
-
Author-email: Alex Braun <alexander.g.braun@gmail.com>
|
|
8
|
-
Requires-Python: >=3.8
|
|
9
|
-
Classifier: Development Status :: 4 - Beta
|
|
10
|
-
Classifier: Intended Audience :: Developers
|
|
11
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
12
|
-
Classifier: Programming Language :: Python :: 3
|
|
13
|
-
Classifier: Programming Language :: Python :: 3.10
|
|
14
|
-
Classifier: Programming Language :: Python :: 3.8
|
|
15
|
-
Classifier: Programming Language :: Python :: 3.9
|
|
16
|
-
Classifier: Typing :: Typed
|
|
17
|
-
Project-URL: documentation, https://thenewflesh.github.io/rolling-pin
|
|
18
|
-
Project-URL: repository, https://github.com/theNewFlesh/rolling-pin
|
|
19
|
-
Description-Content-Type: text/markdown
|
|
20
|
-
|
|
21
|
-
# Introduction
|
|
22
|
-
Rolling-pin is a library of generic tools for ETL work and visualization of JSON
|
|
23
|
-
blobs and python repositories
|
|
24
|
-
|
|
25
|
-
See [documentation](https://thenewflesh.github.io/rolling-pin/) for details.
|
|
26
|
-
|
|
27
|
-
On the documentation main page, under the *Architecture* section, is a
|
|
28
|
-
dynamically generated dependency graph of rolling-pin's current architecture.
|
|
29
|
-
It is generated using the RepoETL class.
|
|
30
|
-
|
|
31
|
-
If you look under the *Metrics* section you will find Radon code metric plots
|
|
32
|
-
and data of the rolling-pin source code, generated by the RadonETL class.
|
|
33
|
-
|
|
34
|
-
Also have a look at this
|
|
35
|
-
**[Jupyter notebook demo](https://github.com/theNewFlesh/rolling-pin/blob/master/notebooks/prototype_demo.ipynb)**
|
|
36
|
-
for a taste of what rolling-pin can do.
|
|
37
|
-
|
|
38
|
-
# Installation
|
|
39
|
-
### Python
|
|
40
|
-
`pip install rolling-pin`
|
|
41
|
-
|
|
42
|
-
### Docker
|
|
43
|
-
1. Install [docker-desktop](https://docs.docker.com/desktop/)
|
|
44
|
-
2. `docker pull thenewflesh/rolling-pin:latest`
|
|
45
|
-
|
|
46
|
-
### Docker For Developers
|
|
47
|
-
1. Install [docker-desktop](https://docs.docker.com/desktop/)
|
|
48
|
-
2. Ensure docker-desktop has at least 4 GB of memory allocated to it.
|
|
49
|
-
3. `git clone git@github.com:theNewFlesh/rolling-pin.git`
|
|
50
|
-
4. `cd rolling-pin`
|
|
51
|
-
6. `chmod +x bin/rolling-pin`
|
|
52
|
-
7. `bin/rolling-pin start`
|
|
53
|
-
|
|
54
|
-
The service should take a few minutes to start up.
|
|
55
|
-
|
|
56
|
-
Run `bin/rolling-pin --help` for more help on the command line tool.
|
|
57
|
-
|
rolling-pin-0.9.3/README.md
DELETED
|
@@ -1,36 +0,0 @@
|
|
|
1
|
-
# Introduction
|
|
2
|
-
Rolling-pin is a library of generic tools for ETL work and visualization of JSON
|
|
3
|
-
blobs and python repositories
|
|
4
|
-
|
|
5
|
-
See [documentation](https://thenewflesh.github.io/rolling-pin/) for details.
|
|
6
|
-
|
|
7
|
-
On the documentation main page, under the *Architecture* section, is a
|
|
8
|
-
dynamically generated dependency graph of rolling-pin's current architecture.
|
|
9
|
-
It is generated using the RepoETL class.
|
|
10
|
-
|
|
11
|
-
If you look under the *Metrics* section you will find Radon code metric plots
|
|
12
|
-
and data of the rolling-pin source code, generated by the RadonETL class.
|
|
13
|
-
|
|
14
|
-
Also have a look at this
|
|
15
|
-
**[Jupyter notebook demo](https://github.com/theNewFlesh/rolling-pin/blob/master/notebooks/prototype_demo.ipynb)**
|
|
16
|
-
for a taste of what rolling-pin can do.
|
|
17
|
-
|
|
18
|
-
# Installation
|
|
19
|
-
### Python
|
|
20
|
-
`pip install rolling-pin`
|
|
21
|
-
|
|
22
|
-
### Docker
|
|
23
|
-
1. Install [docker-desktop](https://docs.docker.com/desktop/)
|
|
24
|
-
2. `docker pull thenewflesh/rolling-pin:latest`
|
|
25
|
-
|
|
26
|
-
### Docker For Developers
|
|
27
|
-
1. Install [docker-desktop](https://docs.docker.com/desktop/)
|
|
28
|
-
2. Ensure docker-desktop has at least 4 GB of memory allocated to it.
|
|
29
|
-
3. `git clone git@github.com:theNewFlesh/rolling-pin.git`
|
|
30
|
-
4. `cd rolling-pin`
|
|
31
|
-
6. `chmod +x bin/rolling-pin`
|
|
32
|
-
7. `bin/rolling-pin start`
|
|
33
|
-
|
|
34
|
-
The service should take a few minutes to start up.
|
|
35
|
-
|
|
36
|
-
Run `bin/rolling-pin --help` for more help on the command line tool.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|