dayhoff-tools 1.0.0__tar.gz → 1.0.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/PKG-INFO +22 -45
- dayhoff_tools-1.0.1/README.md +33 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/cli/main.py +14 -1
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/cli/utility_commands.py +56 -19
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/pyproject.toml +1 -1
- dayhoff_tools-1.0.0/README.md +0 -55
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/__init__.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/chemistry/standardizer.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/chemistry/utils.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/cli/__init__.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/cli/swarm_commands.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/deployment/base.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/deployment/deploy_aws.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/deployment/deploy_gcp.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/deployment/deploy_utils.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/deployment/job_runner.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/deployment/processors.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/deployment/swarm.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/embedders.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/fasta.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/file_ops.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/gcp.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/h5.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/kegg.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/logs.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/mmseqs.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/sqlite.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/structure.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/uniprot.py +0 -0
- {dayhoff_tools-1.0.0 → dayhoff_tools-1.0.1}/dayhoff_tools/warehouse.py +0 -0
@@ -1,6 +1,6 @@
|
|
1
1
|
Metadata-Version: 2.3
|
2
2
|
Name: dayhoff-tools
|
3
|
-
Version: 1.0.
|
3
|
+
Version: 1.0.1
|
4
4
|
Summary: Common tools for all the repos at Dayhoff Labs
|
5
5
|
Author: Daniel Martin-Alarcon
|
6
6
|
Author-email: dma@dayhofflabs.com
|
@@ -66,57 +66,34 @@ Description-Content-Type: text/markdown
|
|
66
66
|
|
67
67
|
# dayhoff-tools
|
68
68
|
|
69
|
-
A set of small, sharp tools for everyone at Dayhoff.
|
69
|
+
A set of small, sharp tools for everyone at Dayhoff. Hosted on PyPi, so you can Poetry or pip install like everything else.
|
70
70
|
|
71
|
-
##
|
71
|
+
## Installation
|
72
72
|
|
73
|
-
|
73
|
+
The base package includes minimal dependencies required for core CLI functionality (like job running):
|
74
74
|
|
75
|
-
|
76
|
-
|
77
|
-
|
78
|
-
|
79
|
-
```sh
|
80
|
-
poetry install
|
75
|
+
```bash
|
76
|
+
pip install dayhoff-tools
|
77
|
+
# or
|
78
|
+
poetry add dayhoff-tools
|
81
79
|
```
|
82
80
|
|
83
|
-
|
84
|
-
|
85
|
-
1. Update version number in `pyproject.toml`
|
86
|
-
2. Run `dh wheel`
|
87
|
-
3. In other repos, run `poetry update dayhoff-tools`
|
88
|
-
|
89
|
-
If you want to overwrite an existing wheel, you'll have to manually delete it from the `dist` folder and also the [Artifact Registry repo](https://console.cloud.google.com/artifacts/python/enzyme-discovery/us-central1/pypirate/dayhoff-tools).
|
90
|
-
|
91
|
-
## Install in other repos
|
92
|
-
|
93
|
-
Installing this library is tricky because we need GCS authentication and also a couple of plugins to install this with either Pip or Poetry. These have been incorporated into `chassis`, but it's worth noting here what the various parts are. All this info came from this [Medium post](https://medium.com/google-cloud/python-packages-via-gcps-artifact-registry-ce1714f8e7c1).
|
81
|
+
### Optional Dependencies
|
94
82
|
|
95
|
-
|
96
|
-
2. Export the SA key file, copy it to your repo, and make it available through this envvar: `export GOOGLE_APPLICATION_CREDENTIALS=github_actions_key.json`
|
83
|
+
You can install extra sets of dependencies using brackets. Available groups are:
|
97
84
|
|
98
|
-
|
85
|
+
* `core`: Includes common data science and bioinformatics tools (`biopython`, `boto3`, `docker`, `fair-esm`, `h5py`, `questionary`).
|
86
|
+
* `dev`: Includes development and testing tools (`black`, `pytest`, `pandas`, `numpy`, `torch`, etc.).
|
87
|
+
* `all`: Includes all dependencies from both `core` and `dev`.
|
99
88
|
|
100
|
-
|
101
|
-
2. `pip install keyrings.google-artifactregistry-auth`
|
102
|
-
3. `pip install --upgrade dayhoff-tools --index-url https://us-central1-python.pkg.dev/enzyme-discovery/pypirate/simple/`
|
89
|
+
**Examples:**
|
103
90
|
|
104
|
-
|
105
|
-
|
106
|
-
|
107
|
-
|
108
|
-
|
109
|
-
```toml
|
110
|
-
[tool.poetry.group.pypirate.dependencies]
|
111
|
-
dayhoff-tools = {version = "*", source = "pypirate"}
|
112
|
-
|
113
|
-
[[tool.poetry.source]]
|
114
|
-
name = "pypirate"
|
115
|
-
url = "https://us-central1-python.pkg.dev/enzyme-discovery/pypirate/simple/"
|
116
|
-
priority = "supplemental"
|
117
|
-
```
|
118
|
-
|
119
|
-
3. When building a dev container, or in other circumstances when you can't easily authenticate as above, run `poetry install --without pypirate`.
|
120
|
-
4. Otherwise, just `poetry install`.
|
121
|
-
5. To ensure you have the latest version, run `poetry update dayhoff-tools`.
|
91
|
+
```bash
|
92
|
+
# Install with core dependencies
|
93
|
+
pip install 'dayhoff-tools[core]'
|
94
|
+
poetry add 'dayhoff-tools[core]'
|
122
95
|
|
96
|
+
# Install with all dependencies
|
97
|
+
pip install 'dayhoff-tools[all]'
|
98
|
+
poetry add 'dayhoff-tools[all]'
|
99
|
+
```
|
@@ -0,0 +1,33 @@
|
|
1
|
+
# dayhoff-tools
|
2
|
+
|
3
|
+
A set of small, sharp tools for everyone at Dayhoff. Hosted on PyPi, so you can Poetry or pip install like everything else.
|
4
|
+
|
5
|
+
## Installation
|
6
|
+
|
7
|
+
The base package includes minimal dependencies required for core CLI functionality (like job running):
|
8
|
+
|
9
|
+
```bash
|
10
|
+
pip install dayhoff-tools
|
11
|
+
# or
|
12
|
+
poetry add dayhoff-tools
|
13
|
+
```
|
14
|
+
|
15
|
+
### Optional Dependencies
|
16
|
+
|
17
|
+
You can install extra sets of dependencies using brackets. Available groups are:
|
18
|
+
|
19
|
+
* `core`: Includes common data science and bioinformatics tools (`biopython`, `boto3`, `docker`, `fair-esm`, `h5py`, `questionary`).
|
20
|
+
* `dev`: Includes development and testing tools (`black`, `pytest`, `pandas`, `numpy`, `torch`, etc.).
|
21
|
+
* `all`: Includes all dependencies from both `core` and `dev`.
|
22
|
+
|
23
|
+
**Examples:**
|
24
|
+
|
25
|
+
```bash
|
26
|
+
# Install with core dependencies
|
27
|
+
pip install 'dayhoff-tools[core]'
|
28
|
+
poetry add 'dayhoff-tools[core]'
|
29
|
+
|
30
|
+
# Install with all dependencies
|
31
|
+
pip install 'dayhoff-tools[all]'
|
32
|
+
poetry add 'dayhoff-tools[all]'
|
33
|
+
```
|
@@ -19,10 +19,23 @@ app.command("gha")(test_github_actions_locally)
|
|
19
19
|
app.command("rebuild")(rebuild_devcontainer_file)
|
20
20
|
app.command("wadd")(add_to_warehouse_typer)
|
21
21
|
app.command("wancestry")(get_ancestry)
|
22
|
-
app.command("wheel")(build_and_upload_wheel)
|
23
22
|
app.command("wimport")(import_from_warehouse_typer)
|
24
23
|
|
25
24
|
|
25
|
+
@app.command("wheel")
|
26
|
+
def build_and_upload_wheel_command(
|
27
|
+
bump: str = typer.Option(
|
28
|
+
"patch",
|
29
|
+
"--bump",
|
30
|
+
"-b",
|
31
|
+
help="Which part of the version to bump: 'major', 'minor', or 'patch'.",
|
32
|
+
case_sensitive=False,
|
33
|
+
)
|
34
|
+
):
|
35
|
+
"""Build wheel, bump version, and upload to PyPI."""
|
36
|
+
build_and_upload_wheel(bump_part=bump)
|
37
|
+
|
38
|
+
|
26
39
|
# Use lazy loading for slow-loading swarm commands
|
27
40
|
@app.command("reset")
|
28
41
|
def reset_wrapper(
|
@@ -1,6 +1,7 @@
|
|
1
1
|
"""CLI commands common to all repos."""
|
2
2
|
|
3
3
|
import os
|
4
|
+
import re
|
4
5
|
import subprocess
|
5
6
|
import sys
|
6
7
|
from pathlib import Path
|
@@ -164,14 +165,40 @@ def delete_local_branch(branch_name: str, folder_path: str):
|
|
164
165
|
os.chdir(original_dir)
|
165
166
|
|
166
167
|
|
167
|
-
def
|
168
|
+
def get_current_version_from_toml(file_path="pyproject.toml"):
|
169
|
+
"""Reads the version from a pyproject.toml file."""
|
170
|
+
try:
|
171
|
+
with open(file_path, "r") as f:
|
172
|
+
content = f.read()
|
173
|
+
version_match = re.search(r'^version\s*=\s*"([^"]+)"', content, re.MULTILINE)
|
174
|
+
if version_match:
|
175
|
+
return version_match.group(1)
|
176
|
+
else:
|
177
|
+
raise ValueError(f"Could not find version string in {file_path}")
|
178
|
+
except FileNotFoundError:
|
179
|
+
raise FileNotFoundError(f"{file_path} not found.")
|
180
|
+
except Exception as e:
|
181
|
+
raise e
|
182
|
+
|
183
|
+
|
184
|
+
def build_and_upload_wheel(bump_part: str = "patch"):
|
168
185
|
"""Build a Python wheel and upload to PyPI.
|
169
186
|
|
170
|
-
Automatically increments the
|
171
|
-
|
187
|
+
Automatically increments the version number in pyproject.toml before building.
|
188
|
+
Use the bump_part argument to specify major, minor, or patch increment.
|
189
|
+
For example: 1.2.3 -> 1.2.4 (patch), 1.3.0 (minor), 2.0.0 (major)
|
172
190
|
|
173
191
|
Expects the PyPI API token to be available in the PYPI_API_TOKEN environment variable.
|
192
|
+
|
193
|
+
Args:
|
194
|
+
bump_part (str): The part of the version to bump: 'major', 'minor', or 'patch'. Defaults to 'patch'.
|
174
195
|
"""
|
196
|
+
if bump_part not in ["major", "minor", "patch"]:
|
197
|
+
print(
|
198
|
+
f"Error: Invalid bump_part '{bump_part}'. Must be 'major', 'minor', or 'patch'."
|
199
|
+
)
|
200
|
+
return
|
201
|
+
|
175
202
|
pypi_token = os.environ.get("PYPI_API_TOKEN")
|
176
203
|
if not pypi_token:
|
177
204
|
print("Error: PYPI_API_TOKEN environment variable not set.")
|
@@ -179,37 +206,45 @@ def build_and_upload_wheel():
|
|
179
206
|
return
|
180
207
|
|
181
208
|
try:
|
182
|
-
#
|
183
|
-
|
184
|
-
|
209
|
+
# Get the current version before bumping
|
210
|
+
current_version = get_current_version_from_toml()
|
211
|
+
print(f"Current version: {current_version}")
|
185
212
|
|
186
|
-
#
|
187
|
-
|
188
|
-
|
213
|
+
# Use poetry to bump the version in pyproject.toml
|
214
|
+
print(f"Bumping {bump_part} version using poetry...")
|
215
|
+
subprocess.run(["poetry", "version", bump_part], check=True)
|
189
216
|
|
190
|
-
#
|
191
|
-
|
192
|
-
|
217
|
+
# Get the new version after bumping
|
218
|
+
new_version = get_current_version_from_toml()
|
219
|
+
print(f"New version: {new_version}")
|
193
220
|
|
194
|
-
# Update
|
221
|
+
# Update other pyproject files with the new version
|
195
222
|
for pyproject_file in [
|
196
|
-
"pyproject.toml",
|
197
223
|
"pyproject_gcp.toml",
|
198
224
|
"pyproject_mac.toml",
|
199
225
|
]:
|
200
226
|
try:
|
201
227
|
with open(pyproject_file, "r") as f:
|
202
228
|
content = f.read()
|
229
|
+
# Use the current_version read earlier for replacement
|
203
230
|
new_content = content.replace(
|
204
231
|
f'version = "{current_version}"', f'version = "{new_version}"'
|
205
232
|
)
|
206
|
-
|
207
|
-
|
208
|
-
|
209
|
-
|
210
|
-
|
233
|
+
if new_content == content:
|
234
|
+
print(
|
235
|
+
f"Warning: Version string 'version = \"{current_version}\"' not found in {pyproject_file}. No update performed."
|
236
|
+
)
|
237
|
+
else:
|
238
|
+
with open(pyproject_file, "w") as f:
|
239
|
+
f.write(new_content)
|
240
|
+
print(
|
241
|
+
f"Version bumped from {current_version} to {new_version} in {pyproject_file}"
|
242
|
+
)
|
211
243
|
except FileNotFoundError:
|
212
244
|
print(f"Skipping {pyproject_file} - file not found")
|
245
|
+
except Exception as e:
|
246
|
+
print(f"Error updating {pyproject_file}: {e}")
|
247
|
+
return # Stop if update fails
|
213
248
|
|
214
249
|
# Disable keyring to avoid issues in containers/CI
|
215
250
|
print("Disabling Poetry keyring...")
|
@@ -242,3 +277,5 @@ def build_and_upload_wheel():
|
|
242
277
|
|
243
278
|
except subprocess.CalledProcessError as e:
|
244
279
|
print(f"Error during build/upload: {e}")
|
280
|
+
except Exception as e:
|
281
|
+
print(f"An unexpected error occurred: {e}")
|
dayhoff_tools-1.0.0/README.md
DELETED
@@ -1,55 +0,0 @@
|
|
1
|
-
# dayhoff-tools
|
2
|
-
|
3
|
-
A set of small, sharp tools for everyone at Dayhoff.
|
4
|
-
|
5
|
-
## Hosting and Auth
|
6
|
-
|
7
|
-
This repo uses Poetry to build and publish a package to GCP Artifact Registry, at `https://us-central1-python.pkg.dev/enzyme-discovery/pypirate/`. This depends on a Poetry plugin that's now in the standard chassis setup (`keyrings.google-artifactregistry-auth`), and also on the active service account having read access to Artifact Registry. That much is set up for the standard dev container service account, but may not be available to other intended users.
|
8
|
-
|
9
|
-
## CLI commands
|
10
|
-
|
11
|
-
Unlike all the repos that use dayhoff-tools, here you have to install the package explicitly before using the CLI:
|
12
|
-
|
13
|
-
```sh
|
14
|
-
poetry install
|
15
|
-
```
|
16
|
-
|
17
|
-
## Publish a new version
|
18
|
-
|
19
|
-
1. Update version number in `pyproject.toml`
|
20
|
-
2. Run `dh wheel`
|
21
|
-
3. In other repos, run `poetry update dayhoff-tools`
|
22
|
-
|
23
|
-
If you want to overwrite an existing wheel, you'll have to manually delete it from the `dist` folder and also the [Artifact Registry repo](https://console.cloud.google.com/artifacts/python/enzyme-discovery/us-central1/pypirate/dayhoff-tools).
|
24
|
-
|
25
|
-
## Install in other repos
|
26
|
-
|
27
|
-
Installing this library is tricky because we need GCS authentication and also a couple of plugins to install this with either Pip or Poetry. These have been incorporated into `chassis`, but it's worth noting here what the various parts are. All this info came from this [Medium post](https://medium.com/google-cloud/python-packages-via-gcps-artifact-registry-ce1714f8e7c1).
|
28
|
-
|
29
|
-
1. Get a Service Account with read access to Artifact Registry (such as `github-actions`, which I made for this purpose).
|
30
|
-
2. Export the SA key file, copy it to your repo, and make it available through this envvar: `export GOOGLE_APPLICATION_CREDENTIALS=github_actions_key.json`
|
31
|
-
|
32
|
-
### ... with Pip
|
33
|
-
|
34
|
-
1. `pip install keyring`
|
35
|
-
2. `pip install keyrings.google-artifactregistry-auth`
|
36
|
-
3. `pip install --upgrade dayhoff-tools --index-url https://us-central1-python.pkg.dev/enzyme-discovery/pypirate/simple/`
|
37
|
-
|
38
|
-
### ... with Poetry
|
39
|
-
|
40
|
-
1. Add this plugin: `poetry self add keyrings.google-artifactregistry-auth`
|
41
|
-
2. Add these sections to `pyproject.toml`. Note that dayhoff-tools is in a separate group `pypirate` that installs separately from the others.
|
42
|
-
|
43
|
-
```toml
|
44
|
-
[tool.poetry.group.pypirate.dependencies]
|
45
|
-
dayhoff-tools = {version = "*", source = "pypirate"}
|
46
|
-
|
47
|
-
[[tool.poetry.source]]
|
48
|
-
name = "pypirate"
|
49
|
-
url = "https://us-central1-python.pkg.dev/enzyme-discovery/pypirate/simple/"
|
50
|
-
priority = "supplemental"
|
51
|
-
```
|
52
|
-
|
53
|
-
3. When building a dev container, or in other circumstances when you can't easily authenticate as above, run `poetry install --without pypirate`.
|
54
|
-
4. Otherwise, just `poetry install`.
|
55
|
-
5. To ensure you have the latest version, run `poetry update dayhoff-tools`.
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|