picsure 2.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- picsure-2.0.0/.devcontainer/devcontainer.json +25 -0
- picsure-2.0.0/.dockerignore +33 -0
- picsure-2.0.0/.env.example +16 -0
- picsure-2.0.0/.github/workflows/ci.yml +38 -0
- picsure-2.0.0/.github/workflows/docs.yml +31 -0
- picsure-2.0.0/.github/workflows/release.yml +111 -0
- picsure-2.0.0/.gitignore +50 -0
- picsure-2.0.0/.python-version +1 -0
- picsure-2.0.0/CHANGELOG.md +117 -0
- picsure-2.0.0/CONTRIBUTING.md +85 -0
- picsure-2.0.0/LICENSE +201 -0
- picsure-2.0.0/PKG-INFO +103 -0
- picsure-2.0.0/README.md +89 -0
- picsure-2.0.0/docker/Dockerfile +63 -0
- picsure-2.0.0/docker/entrypoint.sh +29 -0
- picsure-2.0.0/docker-compose.yml +63 -0
- picsure-2.0.0/docs/development/architecture.md +233 -0
- picsure-2.0.0/docs/development/docker.md +265 -0
- picsure-2.0.0/docs/development/releasing.md +132 -0
- picsure-2.0.0/docs/development/testing.md +214 -0
- picsure-2.0.0/docs/getting-started.md +138 -0
- picsure-2.0.0/docs/guides/building-queries.md +327 -0
- picsure-2.0.0/docs/guides/migrating-from-picsurehpdslib.md +174 -0
- picsure-2.0.0/docs/guides/running-and-exporting.md +162 -0
- picsure-2.0.0/docs/guides/search-and-facets.md +134 -0
- picsure-2.0.0/docs/index.md +61 -0
- picsure-2.0.0/docs/reference/api.md +142 -0
- picsure-2.0.0/mkdocs.yml +63 -0
- picsure-2.0.0/notebooks/0_Connect.ipynb +85 -0
- picsure-2.0.0/notebooks/10_Genomic_Filtering.ipynb +400 -0
- picsure-2.0.0/notebooks/1_Search.ipynb +109 -0
- picsure-2.0.0/notebooks/2_Query.ipynb +181 -0
- picsure-2.0.0/notebooks/3_Open_Query.ipynb +111 -0
- picsure-2.0.0/notebooks/4_Complex_Open_Queries.ipynb +255 -0
- picsure-2.0.0/notebooks/5_Load_Query_By_ID.ipynb +81 -0
- picsure-2.0.0/notebooks/6_Run_Query_By_ID.ipynb +76 -0
- picsure-2.0.0/notebooks/7_Export_Query_As_PFB.ipynb +138 -0
- picsure-2.0.0/notebooks/8_Remove_and_Replace_Query_Functions.ipynb +273 -0
- picsure-2.0.0/notebooks/9_Save_Query_By_Name.ipynb +200 -0
- picsure-2.0.0/notebooks/README.md +59 -0
- picsure-2.0.0/pyproject.toml +66 -0
- picsure-2.0.0/scripts/collect_env_metrics.py +405 -0
- picsure-2.0.0/src/picsure/__init__.py +78 -0
- picsure-2.0.0/src/picsure/_data/__init__.py +0 -0
- picsure-2.0.0/src/picsure/_data/variant_consequences.json +35 -0
- picsure-2.0.0/src/picsure/_dev/__init__.py +1 -0
- picsure-2.0.0/src/picsure/_dev/buffer.py +38 -0
- picsure-2.0.0/src/picsure/_dev/config.py +101 -0
- picsure-2.0.0/src/picsure/_dev/events.py +21 -0
- picsure-2.0.0/src/picsure/_dev/redaction.py +102 -0
- picsure-2.0.0/src/picsure/_dev/reporting.py +72 -0
- picsure-2.0.0/src/picsure/_dev/timing.py +86 -0
- picsure-2.0.0/src/picsure/_models/__init__.py +0 -0
- picsure-2.0.0/src/picsure/_models/clause.py +98 -0
- picsure-2.0.0/src/picsure/_models/clause_group.py +51 -0
- picsure-2.0.0/src/picsure/_models/count_result.py +28 -0
- picsure-2.0.0/src/picsure/_models/dictionary.py +93 -0
- picsure-2.0.0/src/picsure/_models/facet.py +208 -0
- picsure-2.0.0/src/picsure/_models/genomic_filter.py +134 -0
- picsure-2.0.0/src/picsure/_models/query.py +28 -0
- picsure-2.0.0/src/picsure/_models/query_type.py +43 -0
- picsure-2.0.0/src/picsure/_models/resource.py +20 -0
- picsure-2.0.0/src/picsure/_models/session.py +592 -0
- picsure-2.0.0/src/picsure/_services/__init__.py +0 -0
- picsure-2.0.0/src/picsure/_services/_errors.py +65 -0
- picsure-2.0.0/src/picsure/_services/connect.py +332 -0
- picsure-2.0.0/src/picsure/_services/consents.py +58 -0
- picsure-2.0.0/src/picsure/_services/export.py +253 -0
- picsure-2.0.0/src/picsure/_services/genomic_data.py +28 -0
- picsure-2.0.0/src/picsure/_services/genomic_search.py +95 -0
- picsure-2.0.0/src/picsure/_services/query_build.py +314 -0
- picsure-2.0.0/src/picsure/_services/query_edit.py +143 -0
- picsure-2.0.0/src/picsure/_services/query_load.py +277 -0
- picsure-2.0.0/src/picsure/_services/query_run.py +413 -0
- picsure-2.0.0/src/picsure/_services/query_save.py +188 -0
- picsure-2.0.0/src/picsure/_services/search.py +298 -0
- picsure-2.0.0/src/picsure/_transport/__init__.py +0 -0
- picsure-2.0.0/src/picsure/_transport/client.py +445 -0
- picsure-2.0.0/src/picsure/_transport/errors.py +70 -0
- picsure-2.0.0/src/picsure/_transport/platforms.py +202 -0
- picsure-2.0.0/src/picsure/errors.py +21 -0
- picsure-2.0.0/src/picsure/py.typed +0 -0
- picsure-2.0.0/tests/__init__.py +0 -0
- picsure-2.0.0/tests/conftest.py +36 -0
- picsure-2.0.0/tests/fixtures/dictionary_search.json +48 -0
- picsure-2.0.0/tests/fixtures/facets_response.json +84 -0
- picsure-2.0.0/tests/fixtures/profile.json +7 -0
- picsure-2.0.0/tests/fixtures/query_participant.csv +6 -0
- picsure-2.0.0/tests/fixtures/resources.json +4 -0
- picsure-2.0.0/tests/integration/__init__.py +0 -0
- picsure-2.0.0/tests/integration/conftest.py +103 -0
- picsure-2.0.0/tests/integration/test_connect_live.py +47 -0
- picsure-2.0.0/tests/integration/test_export_live.py +36 -0
- picsure-2.0.0/tests/integration/test_query_genomic_live.py +93 -0
- picsure-2.0.0/tests/integration/test_query_live.py +43 -0
- picsure-2.0.0/tests/integration/test_search_live.py +59 -0
- picsure-2.0.0/tests/unit/__init__.py +0 -0
- picsure-2.0.0/tests/unit/dev/__init__.py +0 -0
- picsure-2.0.0/tests/unit/dev/test_buffer.py +66 -0
- picsure-2.0.0/tests/unit/dev/test_client_events.py +177 -0
- picsure-2.0.0/tests/unit/dev/test_config.py +102 -0
- picsure-2.0.0/tests/unit/dev/test_connect_dev.py +121 -0
- picsure-2.0.0/tests/unit/dev/test_events.py +25 -0
- picsure-2.0.0/tests/unit/dev/test_off_path.py +51 -0
- picsure-2.0.0/tests/unit/dev/test_redaction.py +105 -0
- picsure-2.0.0/tests/unit/dev/test_reporting.py +111 -0
- picsure-2.0.0/tests/unit/dev/test_session_dev.py +113 -0
- picsure-2.0.0/tests/unit/dev/test_set_dev_mode.py +30 -0
- picsure-2.0.0/tests/unit/dev/test_timing.py +59 -0
- picsure-2.0.0/tests/unit/test_clause.py +182 -0
- picsure-2.0.0/tests/unit/test_clause_group.py +179 -0
- picsure-2.0.0/tests/unit/test_client.py +641 -0
- picsure-2.0.0/tests/unit/test_connect.py +498 -0
- picsure-2.0.0/tests/unit/test_consents.py +102 -0
- picsure-2.0.0/tests/unit/test_count_result.py +38 -0
- picsure-2.0.0/tests/unit/test_dictionary.py +118 -0
- picsure-2.0.0/tests/unit/test_errors.py +78 -0
- picsure-2.0.0/tests/unit/test_export.py +396 -0
- picsure-2.0.0/tests/unit/test_facade.py +75 -0
- picsure-2.0.0/tests/unit/test_facets.py +259 -0
- picsure-2.0.0/tests/unit/test_genomic_data.py +23 -0
- picsure-2.0.0/tests/unit/test_genomic_filter.py +109 -0
- picsure-2.0.0/tests/unit/test_genomic_search.py +53 -0
- picsure-2.0.0/tests/unit/test_platforms.py +146 -0
- picsure-2.0.0/tests/unit/test_query.py +40 -0
- picsure-2.0.0/tests/unit/test_query_build.py +443 -0
- picsure-2.0.0/tests/unit/test_query_edit.py +262 -0
- picsure-2.0.0/tests/unit/test_query_load.py +577 -0
- picsure-2.0.0/tests/unit/test_query_run.py +954 -0
- picsure-2.0.0/tests/unit/test_query_save.py +414 -0
- picsure-2.0.0/tests/unit/test_query_type.py +26 -0
- picsure-2.0.0/tests/unit/test_resource.py +41 -0
- picsure-2.0.0/tests/unit/test_search.py +509 -0
- picsure-2.0.0/tests/unit/test_session.py +801 -0
- picsure-2.0.0/uv.lock +4291 -0
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "picsure-dev",
|
|
3
|
+
"dockerComposeFile": ["../docker-compose.yml"],
|
|
4
|
+
"service": "dev",
|
|
5
|
+
"workspaceFolder": "/workspace",
|
|
6
|
+
"remoteUser": "dev",
|
|
7
|
+
"overrideCommand": true,
|
|
8
|
+
"customizations": {
|
|
9
|
+
"vscode": {
|
|
10
|
+
"settings": {
|
|
11
|
+
"python.defaultInterpreterPath": "/opt/venv/bin/python",
|
|
12
|
+
"python.terminal.activateEnvironment": false,
|
|
13
|
+
"python.testing.pytestEnabled": true,
|
|
14
|
+
"python.testing.pytestArgs": ["tests"],
|
|
15
|
+
"ruff.path": ["/opt/venv/bin/ruff"],
|
|
16
|
+
"mypy-type-checker.path": ["/opt/venv/bin/mypy"]
|
|
17
|
+
},
|
|
18
|
+
"extensions": [
|
|
19
|
+
"ms-python.python",
|
|
20
|
+
"ms-python.mypy-type-checker",
|
|
21
|
+
"charliermarsh.ruff"
|
|
22
|
+
]
|
|
23
|
+
}
|
|
24
|
+
}
|
|
25
|
+
}
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# The repo is bind-mounted at runtime, so the build context only needs
|
|
2
|
+
# what the Dockerfile actually COPYs (docker/entrypoint.sh). Keep this
|
|
3
|
+
# tight to make `docker compose build` fast.
|
|
4
|
+
|
|
5
|
+
# Build/cache artifacts
|
|
6
|
+
.venv/
|
|
7
|
+
__pycache__/
|
|
8
|
+
*.py[cod]
|
|
9
|
+
*.egg-info/
|
|
10
|
+
build/
|
|
11
|
+
dist/
|
|
12
|
+
.eggs/
|
|
13
|
+
*.egg
|
|
14
|
+
.mypy_cache/
|
|
15
|
+
.pytest_cache/
|
|
16
|
+
.ruff_cache/
|
|
17
|
+
htmlcov/
|
|
18
|
+
.coverage
|
|
19
|
+
site/
|
|
20
|
+
|
|
21
|
+
# Editor / IDE
|
|
22
|
+
.idea/
|
|
23
|
+
.vscode/
|
|
24
|
+
*.iml
|
|
25
|
+
|
|
26
|
+
# Local state
|
|
27
|
+
.worktrees/
|
|
28
|
+
.env
|
|
29
|
+
.git/
|
|
30
|
+
.github/
|
|
31
|
+
|
|
32
|
+
# Docs that aren't needed during build
|
|
33
|
+
docs/superpowers/
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# Copy to `.env` and fill in values to run integration tests without
|
|
2
|
+
# exporting env vars on every shell. Shell-exported vars override these.
|
|
3
|
+
|
|
4
|
+
PICSURE_INTEGRATION=1
|
|
5
|
+
PICSURE_TEST_PLATFORM=BDC_PREDEV_AUTHORIZED
|
|
6
|
+
PICSURE_TEST_TOKEN=
|
|
7
|
+
|
|
8
|
+
# Concept path used by query and export live tests. Must exist on the
|
|
9
|
+
# target deployment — this one works on BDC_DEV_OPEN / BDC_PREDEV_*.
|
|
10
|
+
# For a different platform, pick a path returned by a dictionary search
|
|
11
|
+
# (e.g. via `session.search("age")`).
|
|
12
|
+
PICSURE_TEST_CONCEPT_PATH=\open_access-1000Genomes\SIMULATED AGE\
|
|
13
|
+
|
|
14
|
+
# Search term used by search live tests. "age" works on most PIC-SURE
|
|
15
|
+
# deployments. Override if your target dataset uses different terms.
|
|
16
|
+
PICSURE_TEST_SEARCH_TERM=age
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
name: CI
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
branches: [main]
|
|
6
|
+
pull_request:
|
|
7
|
+
branches: [main]
|
|
8
|
+
|
|
9
|
+
jobs:
|
|
10
|
+
lint:
|
|
11
|
+
runs-on: ubuntu-latest
|
|
12
|
+
steps:
|
|
13
|
+
- uses: actions/checkout@v4
|
|
14
|
+
with:
|
|
15
|
+
fetch-depth: 0 # hatch-vcs reads git history/tags for the version
|
|
16
|
+
- uses: astral-sh/setup-uv@v4
|
|
17
|
+
with:
|
|
18
|
+
version: "latest"
|
|
19
|
+
- run: uv sync --frozen
|
|
20
|
+
- run: uv run ruff check src/ tests/
|
|
21
|
+
- run: uv run ruff format --check src/ tests/
|
|
22
|
+
- run: uv run mypy src/
|
|
23
|
+
|
|
24
|
+
test:
|
|
25
|
+
runs-on: ubuntu-latest
|
|
26
|
+
strategy:
|
|
27
|
+
matrix:
|
|
28
|
+
python-version: ["3.10", "3.11", "3.12"]
|
|
29
|
+
steps:
|
|
30
|
+
- uses: actions/checkout@v4
|
|
31
|
+
with:
|
|
32
|
+
fetch-depth: 0 # hatch-vcs reads git history/tags for the version
|
|
33
|
+
- uses: astral-sh/setup-uv@v4
|
|
34
|
+
with:
|
|
35
|
+
version: "latest"
|
|
36
|
+
- run: uv python install ${{ matrix.python-version }}
|
|
37
|
+
- run: uv sync --frozen --python ${{ matrix.python-version }}
|
|
38
|
+
- run: uv run pytest tests/unit/ -v --cov=picsure --cov-fail-under=80
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
name: Docs
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
branches: [main]
|
|
6
|
+
pull_request:
|
|
7
|
+
branches: [main]
|
|
8
|
+
|
|
9
|
+
jobs:
|
|
10
|
+
build:
|
|
11
|
+
runs-on: ubuntu-latest
|
|
12
|
+
steps:
|
|
13
|
+
- uses: actions/checkout@v4
|
|
14
|
+
with:
|
|
15
|
+
fetch-depth: 0 # hatch-vcs reads git history/tags for the version
|
|
16
|
+
|
|
17
|
+
- uses: astral-sh/setup-uv@v4
|
|
18
|
+
with:
|
|
19
|
+
version: "latest"
|
|
20
|
+
|
|
21
|
+
- run: uv sync --frozen --group docs
|
|
22
|
+
|
|
23
|
+
- name: Build docs (strict mode catches broken references)
|
|
24
|
+
run: uv run mkdocs build --strict
|
|
25
|
+
|
|
26
|
+
- name: Deploy to GitHub Pages
|
|
27
|
+
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
|
28
|
+
uses: peaceiris/actions-gh-pages@v4
|
|
29
|
+
with:
|
|
30
|
+
github_token: ${{ secrets.GITHUB_TOKEN }}
|
|
31
|
+
publish_dir: site
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
name: Release
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
tags: ["v*"]
|
|
6
|
+
|
|
7
|
+
# Least privilege by default; jobs opt into what they need.
|
|
8
|
+
permissions:
|
|
9
|
+
contents: read
|
|
10
|
+
|
|
11
|
+
jobs:
|
|
12
|
+
lint:
|
|
13
|
+
runs-on: ubuntu-latest
|
|
14
|
+
steps:
|
|
15
|
+
- uses: actions/checkout@v4
|
|
16
|
+
with:
|
|
17
|
+
fetch-depth: 0 # hatch-vcs reads git history/tags for the version
|
|
18
|
+
- uses: astral-sh/setup-uv@v4
|
|
19
|
+
with:
|
|
20
|
+
version: "latest"
|
|
21
|
+
- run: uv sync --frozen
|
|
22
|
+
- run: uv run ruff check src/ tests/
|
|
23
|
+
- run: uv run ruff format --check src/ tests/
|
|
24
|
+
- run: uv run mypy src/
|
|
25
|
+
|
|
26
|
+
test:
|
|
27
|
+
runs-on: ubuntu-latest
|
|
28
|
+
strategy:
|
|
29
|
+
matrix:
|
|
30
|
+
python-version: ["3.10", "3.11", "3.12"]
|
|
31
|
+
steps:
|
|
32
|
+
- uses: actions/checkout@v4
|
|
33
|
+
with:
|
|
34
|
+
fetch-depth: 0
|
|
35
|
+
- uses: astral-sh/setup-uv@v4
|
|
36
|
+
with:
|
|
37
|
+
version: "latest"
|
|
38
|
+
- run: uv python install ${{ matrix.python-version }}
|
|
39
|
+
- run: uv sync --frozen --python ${{ matrix.python-version }}
|
|
40
|
+
- run: uv run pytest tests/unit/ -v --cov=picsure --cov-fail-under=80
|
|
41
|
+
|
|
42
|
+
build:
|
|
43
|
+
needs: [lint, test]
|
|
44
|
+
runs-on: ubuntu-latest
|
|
45
|
+
outputs:
|
|
46
|
+
prerelease: ${{ steps.classify.outputs.prerelease }}
|
|
47
|
+
steps:
|
|
48
|
+
- uses: actions/checkout@v4
|
|
49
|
+
with:
|
|
50
|
+
fetch-depth: 0 # required: full history + tags so hatch-vcs derives the tagged version
|
|
51
|
+
|
|
52
|
+
- name: Classify tag (final vs pre-release)
|
|
53
|
+
id: classify
|
|
54
|
+
run: |
|
|
55
|
+
tag="${GITHUB_REF_NAME}"
|
|
56
|
+
if [[ "$tag" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
|
57
|
+
echo "prerelease=false" >> "$GITHUB_OUTPUT"
|
|
58
|
+
echo "Final release: $tag -> PyPI"
|
|
59
|
+
elif [[ "$tag" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(a|b|rc)[0-9]*$ ]]; then
|
|
60
|
+
echo "prerelease=true" >> "$GITHUB_OUTPUT"
|
|
61
|
+
echo "Pre-release: $tag -> TestPyPI"
|
|
62
|
+
else
|
|
63
|
+
echo "::error::Tag '$tag' is not vMAJOR.MINOR.PATCH or a PEP 440 pre-release (e.g. v2.0.0, v2.0.0rc1, v2.0.0rc)"
|
|
64
|
+
exit 1
|
|
65
|
+
fi
|
|
66
|
+
|
|
67
|
+
- uses: astral-sh/setup-uv@v4
|
|
68
|
+
with:
|
|
69
|
+
version: "latest"
|
|
70
|
+
|
|
71
|
+
- name: Build sdist + wheel
|
|
72
|
+
run: uv build
|
|
73
|
+
|
|
74
|
+
- name: Check distribution metadata
|
|
75
|
+
run: uvx twine check dist/*
|
|
76
|
+
|
|
77
|
+
- uses: actions/upload-artifact@v4
|
|
78
|
+
with:
|
|
79
|
+
name: dist
|
|
80
|
+
path: dist/
|
|
81
|
+
if-no-files-found: error
|
|
82
|
+
|
|
83
|
+
publish-testpypi:
|
|
84
|
+
needs: build
|
|
85
|
+
if: needs.build.outputs.prerelease == 'true'
|
|
86
|
+
runs-on: ubuntu-latest
|
|
87
|
+
environment: TestPyPi
|
|
88
|
+
permissions:
|
|
89
|
+
id-token: write # OIDC token for Trusted Publishing
|
|
90
|
+
steps:
|
|
91
|
+
- uses: actions/download-artifact@v4
|
|
92
|
+
with:
|
|
93
|
+
name: dist
|
|
94
|
+
path: dist/
|
|
95
|
+
- uses: pypa/gh-action-pypi-publish@release/v1
|
|
96
|
+
with:
|
|
97
|
+
repository-url: https://test.pypi.org/legacy/
|
|
98
|
+
|
|
99
|
+
publish-pypi:
|
|
100
|
+
needs: build
|
|
101
|
+
if: needs.build.outputs.prerelease == 'false'
|
|
102
|
+
runs-on: ubuntu-latest
|
|
103
|
+
environment: PyPi
|
|
104
|
+
permissions:
|
|
105
|
+
id-token: write # OIDC token for Trusted Publishing
|
|
106
|
+
steps:
|
|
107
|
+
- uses: actions/download-artifact@v4
|
|
108
|
+
with:
|
|
109
|
+
name: dist
|
|
110
|
+
path: dist/
|
|
111
|
+
- uses: pypa/gh-action-pypi-publish@release/v1
|
picsure-2.0.0/.gitignore
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
__pycache__/
|
|
2
|
+
*.py[cod]
|
|
3
|
+
*$py.class
|
|
4
|
+
*.egg-info/
|
|
5
|
+
dist/
|
|
6
|
+
build/
|
|
7
|
+
.eggs/
|
|
8
|
+
*.egg
|
|
9
|
+
.mypy_cache/
|
|
10
|
+
.pytest_cache/
|
|
11
|
+
.ruff_cache/
|
|
12
|
+
htmlcov/
|
|
13
|
+
.coverage
|
|
14
|
+
.env
|
|
15
|
+
.venv/
|
|
16
|
+
venv/
|
|
17
|
+
*.iml
|
|
18
|
+
.idea/
|
|
19
|
+
.vscode/
|
|
20
|
+
site/
|
|
21
|
+
.worktrees/
|
|
22
|
+
docs/superpowers
|
|
23
|
+
docs/superpowers/
|
|
24
|
+
|
|
25
|
+
# JupyterLab artifacts written into the source tree from the dev container
|
|
26
|
+
.ipynb_checkpoints/
|
|
27
|
+
|
|
28
|
+
# Only commit notebooks that live under notebooks/. Anything elsewhere
|
|
29
|
+
# (Untitled.ipynb in the repo root, scratch experiments inside src/,
|
|
30
|
+
# accidental notebooks from container sessions) stays out of git.
|
|
31
|
+
*.ipynb
|
|
32
|
+
!notebooks/**/*.ipynb
|
|
33
|
+
|
|
34
|
+
# Generated PFB / Avro export artifacts from the export notebooks (nb 7)
|
|
35
|
+
*.avro
|
|
36
|
+
|
|
37
|
+
# XDG trash directory created when a container process "deletes" a file
|
|
38
|
+
.Trash-*/
|
|
39
|
+
|
|
40
|
+
# Credentials sometimes pasted into the workspace
|
|
41
|
+
token.txt
|
|
42
|
+
token*.txt
|
|
43
|
+
*.token
|
|
44
|
+
|
|
45
|
+
# local scratch (HAR captures, ad-hoc temp files); never commit
|
|
46
|
+
docs/tmp/
|
|
47
|
+
*.har
|
|
48
|
+
|
|
49
|
+
# collect_env_metrics.py output (per-env latency baselines); never commit
|
|
50
|
+
metrics-results/
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
3.10
|
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
|
6
|
+
|
|
7
|
+
## [Unreleased]
|
|
8
|
+
|
|
9
|
+
## [2.0.0] - 2026-06-15
|
|
10
|
+
|
|
11
|
+
### Changed
|
|
12
|
+
- **BREAKING:** The query builders are renamed to a `build*` triplet whose names match their return types: `createSubQuery` → `buildClause` (returns `Clause`), the clause-combining `buildQuery` → `buildClauseGroup` (returns `ClauseGroup`). The name `buildQuery` is now the query *assembler* `buildQuery(phenotypicFilter=None, includeConcepts=())` and returns a `Query`.
|
|
13
|
+
- **BREAKING:** `Query` is now a dataclass (`phenotypicFilter`, `includeConcepts`) rather than a `Clause | ClauseGroup` type alias. `runQuery`/`exportAsPFB`/`saveQueryByName` accept a `Query` or a bare `Clause`/`ClauseGroup`. A saved query that selects output concepts now loads as a `Query` (previously a `ClauseGroup` of `SELECT` clauses).
|
|
14
|
+
|
|
15
|
+
### Removed
|
|
16
|
+
- **BREAKING:** `PhenotypicFilterType.SELECT` is removed. To include concept paths in the output without filtering, pass them to `buildQuery(includeConcepts=...)` instead of building a `SELECT` clause.
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
- `picsure.buildGenomicFilter(key, *, values)` builds a single categorical genomic filter. `values` is required (string or list of strings); `VariantFrequency` members are accepted and coerced. Variant-spec and SNP keys are rejected with an actionable error.
|
|
20
|
+
- `picsure.genomicConsequences()` returns an offline DataFrame of all variant consequences with a `severity` and `consequence` column. Available on any platform; no network call.
|
|
21
|
+
- `picsure.buildQuery(genomicFilters=None)` - `buildQuery` now accepts an optional `genomicFilters` list. Genomic filters are AND-combined with the phenotypic filter; a genomic-only query (no `phenotypicFilter`) is valid.
|
|
22
|
+
- `picsure.connect(supports_genomic=None)` - `connect` accepts an optional `supports_genomic` flag for platforms that do not auto-detect genomic capability. Genomic operations are available on BDC_AUTHORIZED, BDC_DEV_AUTHORIZED, BDC_PREDEV_AUTHORIZED, and NHANES_AUTHORIZED; all `*_OPEN` platforms are not genomic-capable.
|
|
23
|
+
- `Session.searchGenomicValues(genomicConceptPath, *, query="", page=1, size=50)` performs a paginated server lookup of valid values for any genomic key (genes, consequences). Returns a pandas DataFrame; pagination metadata is on `df.attrs`. Available on authorized platforms only.
|
|
24
|
+
- `GenomicFilter` dataclass representing a single categorical genomic filter (`key`, `values`).
|
|
25
|
+
- `VariantFrequency` enum with members `RARE`, `COMMON`, and `NOVEL` for use with the `Variant_frequency_as_text` genomic key.
|
|
26
|
+
- Variant result types for `runQuery`: `variant_count` (returns `CountResult`), `variant_list` (returns `list[str]`), `vcf_excerpt` (returns DataFrame), and `aggregate_vcf_excerpt` (returns DataFrame). These types are not served by BDC primary environments yet; the adapter raises `PicSureQueryError` with a clear message on an empty or 5xx response.
|
|
27
|
+
- `picsure.buildQuery(phenotypicFilter=None, includeConcepts=())` assembles a `Query` from a filter tree plus the concept paths to return as output columns. `includeConcepts` preserves order and de-duplicates.
|
|
28
|
+
- `picsure.removeSubQuery(query, target)` returns a copy of `query` with every structurally-equal occurrence of `target` removed, recursively through nested groups. Emptied `ClauseGroup`s are dropped. Raises `PicSureValidationError` if the whole tree would be removed.
|
|
29
|
+
- `picsure.replaceClause(query, target, replacement)` returns a copy of `query` with every structurally-equal occurrence of `target` swapped for `replacement`.
|
|
30
|
+
- `Session.saveQueryByName(query, name, *, overwrite=False)` submits the query via `POST /picsure/v3/query` and associates `name` with it via `POST /dataset/named/`. With `overwrite=True`, an existing record with the same name is `PUT`-updated to point at the freshly-submitted query. The backend's `NamedDataset` constraints (`name` max 255 chars, allowed: letters, digits, spaces, ``- _ \ / ? + = [ ] . ( ) : " '``) are validated client-side. Refused on open-access deployments. Returns the new query ID — pass it to `loadQueryByID` / `runQueryByID` later.
|
|
31
|
+
- `PicSureClient.put_json()` on the transport client (used by the `saveQueryByName` overwrite path).
|
|
32
|
+
- `picsure.PicSureAuthError`, `picsure.PicSureConnectionError`, `picsure.PicSureQueryError`, and `picsure.PicSureValidationError` are now re-exported from the top-level package, so `from picsure import PicSureQueryError` works (previously required `from picsure.errors import ...`).
|
|
33
|
+
- `Session.facets(term="", *, facets=None)` and `Session.showAllFacets(term="", *, facets=None)` now accept optional search term and facet selections. Counts returned are contextual to the provided search when term/facets are supplied; passing no arguments preserves the previous "global counts" behaviour.
|
|
34
|
+
- `DictionaryEntry` exposes `min`, `max`, `allow_filtering`, `meta`, and `study_acronym` fields. The corresponding columns (`min`, `max`, `allowFiltering`, `meta`, `studyAcronym`) are added to the `Session.searchDictionary` DataFrame result.
|
|
35
|
+
- `picsure.connect()` to authenticate and connect to a PIC-SURE instance.
|
|
36
|
+
- `Session.getResourceID()` to list available resources as a DataFrame.
|
|
37
|
+
- Platform name resolution for BDC Authorized, BDC Open, and Demo.
|
|
38
|
+
- Actionable error messages via `PicSureError`.
|
|
39
|
+
- Unit test suite with mocked HTTP via respx.
|
|
40
|
+
- Integration test scaffold gated by `PICSURE_INTEGRATION` env var.
|
|
41
|
+
- GitHub Actions CI with Python 3.10/3.11/3.12 matrix.
|
|
42
|
+
- `Session.searchDictionary()` to search the data dictionary with optional facet filtering.
|
|
43
|
+
- `Session.facets()` to retrieve available facet categories as a `FacetSet`.
|
|
44
|
+
- `Session.showAllFacets()` to display all facet categories and values as a DataFrame.
|
|
45
|
+
- `FacetSet` for building facet selections with validation.
|
|
46
|
+
- Search result deduplication by concept path.
|
|
47
|
+
- Zero-result searches return empty DataFrames with a stderr note.
|
|
48
|
+
- `picsure.createSubQuery()` to build individual filter clauses (FILTER, ANYRECORD, SELECT, REQUIRE).
|
|
49
|
+
- `picsure.buildQuery()` to combine clauses with AND/OR logic, supporting arbitrary nesting.
|
|
50
|
+
- `PhenotypicFilterType` and `GroupOperator` enums for type-safe clause and group construction.
|
|
51
|
+
- `Clause`, `ClauseGroup`, and `Query` types with `to_query_json()` serialization.
|
|
52
|
+
- `CountResult` dataclass exposing `value`, `margin`, `cap`, `raw`, and an `obfuscated` property for count query responses.
|
|
53
|
+
- Input validation with actionable error messages for invalid clause configurations.
|
|
54
|
+
- `Session.runQuery()` to execute queries and return a `CountResult`, a `dict[str, CountResult]` (cross-count), or a `DataFrame` (participant / timestamp).
|
|
55
|
+
- `Session.runQueryByID(query_id, type="count")` to load a saved query by ID and execute it in one call, returning the same result types as `runQuery`.
|
|
56
|
+
- `Session.exportAsPFB()` to export query results as PFB files.
|
|
57
|
+
- `Session.exportCSV()` and `Session.exportTSV()` to save DataFrames to disk.
|
|
58
|
+
- `PicSureClient.post_raw()` for non-JSON response handling.
|
|
59
|
+
- Query type validation with actionable error messages ("count", "participant", "timestamp").
|
|
60
|
+
- Documentation site with MkDocs + Material theme.
|
|
61
|
+
- Auto-generated API reference from docstrings via mkdocstrings.
|
|
62
|
+
- User guides: search, facets, query building, running, and exporting.
|
|
63
|
+
- Migration guide from PicSureHpdsLib with side-by-side examples.
|
|
64
|
+
- Docs CI workflow: build on PR, deploy to GitHub Pages on push to main.
|
|
65
|
+
|
|
66
|
+
### Changed
|
|
67
|
+
- `Session.runQuery` docstring now documents that ``participant`` / ``timestamp``
|
|
68
|
+
DataFrame cells may contain tab-joined multi-values; callers should use
|
|
69
|
+
``df[col].str.split("\t")`` when they need individual observations.
|
|
70
|
+
- Query endpoint changed from `/picsure/query/sync` to `/picsure/v3/query/sync`.
|
|
71
|
+
- `Clause.to_query_json()` / `ClauseGroup.to_query_json()` now emit the v3 `PhenotypicClause` / `PhenotypicSubquery` schema (`operator` / `phenotypicClauses` / `not` for groups; `phenotypicFilterType` / `conceptPath` / `values` / `min` / `max` / `not` for leaves). The previous wire format is no longer produced.
|
|
72
|
+
- `Clause.to_query_json()` now raises `PicSureValidationError` for `SELECT` clauses. Use `Clause.select_paths()` / `ClauseGroup.select_paths()` to retrieve output paths instead.
|
|
73
|
+
- `Session.runQuery(..., type="count")` now returns a `CountResult` dataclass instead of a plain `int`. Access the integer count via `result.value`; check `result.cap` for suppressed small-count responses (`result.value` is `None` in that case) and `result.margin` for noisy responses.
|
|
74
|
+
- `Session.runQuery(..., type="cross_count")` now returns a `dict[str, CountResult]` keyed by concept path instead of a DataFrame.
|
|
75
|
+
- `PicSureClient` now strips leading/trailing whitespace from the bearer token; a whitespace-only token is treated as anonymous (no `Authorization` header, `request-source: Open`).
|
|
76
|
+
- `Session.exportAsPFB()` / `picsure._services.export.export_pfb` now use the async flow (`POST /picsure/v3/query` → poll `/query/{id}/status` → `POST /query/{id}/result`) rather than `/query/sync`. Response bytes are streamed directly to disk. Polling uses exponential backoff (1s, 2s, 4s, … capped at 60s per poll) and fails with `PicSureConnectionError` after 10 minutes of cumulative waiting. The output file is written atomically (`.part` staging file + rename on success).
|
|
77
|
+
- `createSubQuery` now raises `PicSureValidationError` for additional invalid combinations:
|
|
78
|
+
FILTER clauses with both `categories` and `min`/`max`; REQUIRE or SELECT clauses
|
|
79
|
+
with any of `categories`/`min`/`max`; empty keys lists. These were previously
|
|
80
|
+
silently accepted and the extra arguments discarded (or rejected downstream by
|
|
81
|
+
the server with a less actionable error).
|
|
82
|
+
- `ClauseGroup.to_query_json()` now raises `PicSureValidationError` if any nested
|
|
83
|
+
child is a SELECT clause (previously it silently stripped them). This is
|
|
84
|
+
symmetric with `Clause.to_query_json()`, which has always raised on SELECT.
|
|
85
|
+
`ClauseGroup.select_paths()` and `build_query_body()` continue to handle SELECT
|
|
86
|
+
extraction at the top level; inline SELECTs inside a group are the error case.
|
|
87
|
+
- `createSubQuery` now defensively copies list arguments (`keys`, `categories`), so
|
|
88
|
+
mutating the caller's lists after construction does not affect the resulting
|
|
89
|
+
`Clause`.
|
|
90
|
+
- Dependency ranges tightened to upper-bounded majors (`httpx>=0.27,<1`, `pandas>=2,<3`) to avoid silent breakage on major releases.
|
|
91
|
+
- `CONTRIBUTING.md` lint instructions now cover both `src/` and `tests/`, matching the CI gate.
|
|
92
|
+
|
|
93
|
+
### Removed
|
|
94
|
+
- Wire-format docstrings on `Clause` and `ClauseGroup` no longer advertise the
|
|
95
|
+
`not` / negation field. The adapter still emits `"not": False` on the wire, but
|
|
96
|
+
negation is not supported by the public API; support will return in a later
|
|
97
|
+
release.
|
|
98
|
+
|
|
99
|
+
### Fixed
|
|
100
|
+
- `Session.runQuery(..., type="participant")` and `type="timestamp"` now surface
|
|
101
|
+
malformed-CSV responses as `PicSureQueryError` with a body preview (previously
|
|
102
|
+
leaked raw pandas `ParserError` / `EmptyDataError` / `UnicodeDecodeError`).
|
|
103
|
+
- `Session.runQuery(..., type="cross_count")` explicitly handles the direct-HPDS
|
|
104
|
+
response shape (`{concept_path: integer}`), not only the aggregate-obfuscation
|
|
105
|
+
response shape (`{concept_path: count_string}`).
|
|
106
|
+
- `Session.runQuery` now validates the query argument at body construction so
|
|
107
|
+
that passing a plain `dict` raises `PicSureValidationError` ("Query must be a
|
|
108
|
+
Clause or ClauseGroup") rather than a confusing `AttributeError` from an
|
|
109
|
+
internal accessor.
|
|
110
|
+
- `Session.searchDictionary` now raises `PicSureQueryError` when the server's paginated response indicates the result set was truncated (`last != True` or `content` length doesn't match `totalElements`). Previously the adapter silently returned the partial page.
|
|
111
|
+
- PFB export against v3 PIC-SURE was silently broken: the previous implementation posted `DATAFRAME_PFB` to `/query/sync`, which v3 HPDS has no handler for. Unit tests passed only because `respx` served a canned 200 body at the wrong URL.
|
|
112
|
+
- 4xx responses during PFB submission / status / result are now surfaced as `PicSureValidationError` / `PicSureQueryError` (previously the 4xx body bytes would be written to disk as if they were PFB).
|
|
113
|
+
- `OSError` / `PermissionError` during disk writes in `export_pfb` are now wrapped in `PicSureConnectionError` with the target path in the message (previously leaked raw).
|
|
114
|
+
- Getting-started and search-and-facets guide examples used ``"study_ids"`` as the facet category; the server returns ``"dataset_id"``. Guides updated to match. Users copy-pasting the earlier examples would have seen ``PicSureValidationError``.
|
|
115
|
+
- ``showAllFacets`` DataFrame column list in the user guide was outdated. Now correctly documents the six columns (``category``, ``Category Display``, ``display``, ``description``, ``value``, ``count``).
|
|
116
|
+
- Raw-string code snippets ending in a backslash (e.g. ``r"\phs1\sex\"``) were unterminated string literals and would ``SyntaxError`` on copy-paste. Replaced with doubled-backslash non-raw strings across ``README.md`` and the guides.
|
|
117
|
+
- API reference now documents `Platform`, `CountResult`, the `PicSureError` subclasses, and `Session.consents` / `Session.total_concepts` properties, which were exported but missing from the reference.
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Contributing
|
|
2
|
+
|
|
3
|
+
Welcome. This document is the entry point for working on `picsure`
|
|
4
|
+
itself — both new contributors getting set up and maintainers handling
|
|
5
|
+
day-to-day reviews, tests, and releases. User-facing usage docs live
|
|
6
|
+
under [`docs/`](docs/) and on the published site; this file and the
|
|
7
|
+
[`docs/development/`](docs/development/) folder are for people
|
|
8
|
+
**changing the library**.
|
|
9
|
+
|
|
10
|
+
## Where to find things
|
|
11
|
+
|
|
12
|
+
| Topic | Where |
|
|
13
|
+
|----------------------------------------|--------------------------------------------------------|
|
|
14
|
+
| Containerized dev environment + IDEs | [docs/development/docker.md](docs/development/docker.md) |
|
|
15
|
+
| Package layout and internals | [docs/development/architecture.md](docs/development/architecture.md) |
|
|
16
|
+
| Test layout, running, debugging CI | [docs/development/testing.md](docs/development/testing.md) |
|
|
17
|
+
| Versioning, tagging, publishing | [docs/development/releasing.md](docs/development/releasing.md) |
|
|
18
|
+
|
|
19
|
+
## Local development setup
|
|
20
|
+
|
|
21
|
+
Prefer a container? See
|
|
22
|
+
[docs/development/docker.md](docs/development/docker.md) for a
|
|
23
|
+
pre-configured Docker dev environment (no local `uv` or Python
|
|
24
|
+
required, JupyterLab included). That guide also covers
|
|
25
|
+
[IDE setup for VS Code and PyCharm](docs/development/docker.md#ide-setup--using-the-containers-python-from-your-editor).
|
|
26
|
+
|
|
27
|
+
For a host-native setup:
|
|
28
|
+
|
|
29
|
+
1. Install [uv](https://docs.astral.sh/uv/getting-started/installation/).
|
|
30
|
+
|
|
31
|
+
2. Clone the repo and install dependencies:
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
git clone https://github.com/hms-dbmi/pic-sure-python-adapter-hpds.git
|
|
35
|
+
cd pic-sure-python-adapter-hpds
|
|
36
|
+
uv sync
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
3. Run the checks:
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
uv run ruff check src/ tests/ # lint
|
|
43
|
+
uv run ruff format --check src/ tests/ # format check (both src and tests)
|
|
44
|
+
uv run mypy src/ # type check
|
|
45
|
+
uv run pytest tests/unit/ -v # unit tests
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
These four commands mirror the CI lint gate, which runs `ruff check`
|
|
49
|
+
and `ruff format --check` over **both** `src/` and `tests/`. Skipping
|
|
50
|
+
`tests/` locally will let contributor-written test files pass the
|
|
51
|
+
local check but fail CI.
|
|
52
|
+
|
|
53
|
+
## Branching and pull requests
|
|
54
|
+
|
|
55
|
+
- Branch from `main`. Use short, descriptive branch names.
|
|
56
|
+
- Write descriptive commit messages. The existing history follows a
|
|
57
|
+
loose conventional-commits style (`feat(query):`, `fix(client):`,
|
|
58
|
+
`chore(deps):`, `refactor(session):`, `docs(...)`). Match it when
|
|
59
|
+
it's natural; don't fight the linter if it isn't.
|
|
60
|
+
- Open the PR against `main`. CI must pass before merge:
|
|
61
|
+
- **Lint job** — `ruff check src/ tests/`, `ruff format --check
|
|
62
|
+
src/ tests/`, `mypy src/`.
|
|
63
|
+
- **Test matrix** — `pytest tests/unit/` on Python 3.10, 3.11, 3.12
|
|
64
|
+
with `--cov-fail-under=80`.
|
|
65
|
+
- **Docs** — `mkdocs build --strict` (broken cross-references fail
|
|
66
|
+
the build).
|
|
67
|
+
- If you touched the public API surface (anything re-exported from
|
|
68
|
+
`picsure/__init__.py`), update `CHANGELOG.md` under `[Unreleased]`.
|
|
69
|
+
|
|
70
|
+
## Code style
|
|
71
|
+
|
|
72
|
+
- `ruff` handles linting and formatting. Config lives in
|
|
73
|
+
`pyproject.toml` under `[tool.ruff]`.
|
|
74
|
+
- `mypy --strict` enforces type annotations on all code in `src/`.
|
|
75
|
+
- Google-style docstrings on all public functions.
|
|
76
|
+
- camelCase for public API method names (matches the product spec);
|
|
77
|
+
snake_case for everything internal.
|
|
78
|
+
|
|
79
|
+
## Where to file issues
|
|
80
|
+
|
|
81
|
+
GitHub Issues on the repo:
|
|
82
|
+
<https://github.com/hms-dbmi/pic-sure-python-adapter-hpds/issues>.
|
|
83
|
+
|
|
84
|
+
For security-sensitive reports, please coordinate with the
|
|
85
|
+
maintainers privately before opening a public issue.
|