scigantic-chembl 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,16 @@
1
+ MIT No Attribution
2
+
3
+ Copyright 2026 Scigantic
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy of this
6
+ software and associated documentation files (the "Software"), to deal in the Software
7
+ without restriction, including without limitation the rights to use, copy, modify,
8
+ merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
9
+ permit persons to whom the Software is furnished to do so.
10
+
11
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
12
+ INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
13
+ PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
14
+ HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF
15
+ CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE
16
+ OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,127 @@
1
+ Metadata-Version: 2.4
2
+ Name: scigantic-chembl
3
+ Version: 0.1.0
4
+ Summary: Query ChEMBL directly from a public S3 mirror with DuckDB. No download, no local database.
5
+ Author: Scigantic
6
+ License: MIT-0
7
+ Project-URL: Homepage, https://scigantic.com
8
+ Keywords: chembl,cheminformatics,duckdb,drug-discovery,bioactivity
9
+ Classifier: License :: OSI Approved :: MIT No Attribution License (MIT-0)
10
+ Classifier: Programming Language :: Python :: 3
11
+ Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
12
+ Classifier: Topic :: Scientific/Engineering :: Chemistry
13
+ Requires-Python: >=3.10
14
+ Description-Content-Type: text/markdown
15
+ License-File: LICENSE
16
+ Requires-Dist: duckdb>=0.10
17
+ Requires-Dist: pyarrow>=14
18
+ Requires-Dist: numpy>=2.0
19
+ Requires-Dist: pandas>=1.5
20
+ Provides-Extra: similarity
21
+ Requires-Dist: rdkit>=2023.9; extra == "similarity"
22
+ Provides-Extra: dev
23
+ Requires-Dist: pytest>=7; extra == "dev"
24
+ Dynamic: license-file
25
+
26
+ <h1 align="center">scigantic-chembl</h1>
27
+
28
+ <p align="center">
29
+ <a href="https://github.com/Scigantic/scigantic-chembl/actions/workflows/ci.yml">
30
+ <img alt="CI" src="https://github.com/Scigantic/scigantic-chembl/actions/workflows/ci.yml/badge.svg" /></a>
31
+ <a href="https://pypi.org/project/scigantic-chembl/">
32
+ <img alt="PyPI" src="https://img.shields.io/pypi/v/scigantic-chembl" /></a>
33
+ <a href="https://pypi.org/project/scigantic-chembl/">
34
+ <img alt="PyPI - Python Version" src="https://img.shields.io/pypi/pyversions/scigantic-chembl" /></a>
35
+ <a href="https://github.com/Scigantic/scigantic-chembl/blob/main/LICENSE">
36
+ <img alt="License" src="https://img.shields.io/github/license/Scigantic/scigantic-chembl" /></a>
37
+ </p>
38
+
39
+ Query ChEMBL directly from a public S3 mirror with DuckDB.
40
+
41
+ ```python
42
+ import scigantic_chembl as chembl
43
+
44
+ df = chembl.query("""
45
+ SELECT chembl_id, pref_name
46
+ FROM molecule_dictionary
47
+ WHERE pref_name IS NOT NULL
48
+ LIMIT 5
49
+ """)
50
+ ```
51
+
52
+ That query runs against `s3://scigantic-chembl` over DuckDB's httpfs extension. Nothing is downloaded first, and there's no local database file sitting on disk afterward.
53
+
54
+ ## Installation
55
+
56
+ ```console
57
+ $ pip install scigantic-chembl
58
+ ```
59
+
60
+ ## Compared to chembl-downloader
61
+
62
+ [chembl-downloader](https://github.com/cthoyt/chembl-downloader) is the standard way to work with ChEMBL in Python. It covers every release back to chembl_1, and once the SQLite dump is downloaded it works fully offline. This package gives up that range for less setup and two things chembl-downloader doesn't ship on its own: a pre-joined potency table, and similarity search with no separate index build. The mirror here only carries chembl_35 through chembl_37, and only chembl_37 has the pre-joined and similarity layers, so an older release or fully offline work is still a job for chembl-downloader.
63
+
64
+ ## Potency data, pre-joined
65
+
66
+ `activities` needs a five-table join and a few correctness filters before it's usable for structure-activity work. That join is already done, stored as `derived/activities_enriched.parquet`:
67
+
68
+ ```python
69
+ df = chembl.activities(target_chembl_id="CHEMBL203") # EGFR: 18,998 rows, 11,202 compounds
70
+ ```
71
+
72
+ The filters already applied are about correctness, not taste: `pchembl_value` present, `standard_relation = '='`, no `data_validity_comment`, not a `potential_duplicate`. `confidence_score` and `target_type` stay as columns rather than filters, since which rows count as usable SAR data is an analysis choice:
73
+
74
+ ```python
75
+ df = chembl.activities(target_chembl_id="CHEMBL203", min_confidence=8)
76
+ ```
77
+
78
+ `chembl.query()` still reaches the raw tables directly for anything the join leaves out.
79
+
80
+ ## Similarity search
81
+
82
+ ```python
83
+ gefitinib = "COC1=C(C=C2C(=C1)N=CN=C2NC3=CC(=C(C=C3)F)Cl)OCCCN4CCOCC4"
84
+ hits = chembl.similar_compounds(gefitinib, top_k=5)
85
+ ```
86
+
87
+ ```
88
+ chembl_id tanimoto
89
+ CHEMBL939 1.000000
90
+ CHEMBL14699 0.919355
91
+ CHEMBL4165375 0.916667
92
+ CHEMBL299672 0.857143
93
+ CHEMBL4448162 0.857143
94
+ ```
95
+
96
+ CHEMBL939 is gefitinib itself. Every compound with a comparable potency measurement (1.68M of ChEMBL's 2.9M structures) has a precomputed 2048-bit Morgan fingerprint, kept packed in memory and compared with numpy's `bitwise_count` rather than unpacked bit by bit. The corpus loads once per process, about 18 seconds on a typical home connection and faster from inside AWS; every call after that in the same process is under half a second. chembl-downloader can do similarity search too, through `chemfp`, but its own docs put building that index at tens of minutes.
97
+
98
+ Similarity search needs rdkit to encode the query molecule, so it's kept as an optional extra:
99
+
100
+ ```console
101
+ $ pip install "scigantic-chembl[similarity]"
102
+ ```
103
+
104
+ ## What's mirrored
105
+
106
+ ```python
107
+ chembl.releases()
108
+ ```
109
+
110
+ | release | raw tables | pre-joined activities | similarity search | CYP training set |
111
+ |---|---|---|---|---|
112
+ | chembl_37 | yes | yes | yes | yes |
113
+ | chembl_36 | yes | no | no | yes |
114
+ | chembl_35 | yes | no | no | yes |
115
+
116
+ chembl_36 and chembl_35 are raw-table access only: their `activities` table is missing a column chembl_37's has (`modality`), so calling `activities()` or `similar_compounds()` on either one raises `ReleaseCapabilityError` up front instead of failing partway through a join with a confusing error.
117
+
118
+ ## Command line
119
+
120
+ ```console
121
+ $ scigantic-chembl info
122
+ $ scigantic-chembl query "SELECT count(*) FROM activities" --release chembl_37
123
+ ```
124
+
125
+ ## License
126
+
127
+ MIT-0. See [LICENSE](LICENSE).
@@ -0,0 +1,102 @@
1
+ <h1 align="center">scigantic-chembl</h1>
2
+
3
+ <p align="center">
4
+ <a href="https://github.com/Scigantic/scigantic-chembl/actions/workflows/ci.yml">
5
+ <img alt="CI" src="https://github.com/Scigantic/scigantic-chembl/actions/workflows/ci.yml/badge.svg" /></a>
6
+ <a href="https://pypi.org/project/scigantic-chembl/">
7
+ <img alt="PyPI" src="https://img.shields.io/pypi/v/scigantic-chembl" /></a>
8
+ <a href="https://pypi.org/project/scigantic-chembl/">
9
+ <img alt="PyPI - Python Version" src="https://img.shields.io/pypi/pyversions/scigantic-chembl" /></a>
10
+ <a href="https://github.com/Scigantic/scigantic-chembl/blob/main/LICENSE">
11
+ <img alt="License" src="https://img.shields.io/github/license/Scigantic/scigantic-chembl" /></a>
12
+ </p>
13
+
14
+ Query ChEMBL directly from a public S3 mirror with DuckDB.
15
+
16
+ ```python
17
+ import scigantic_chembl as chembl
18
+
19
+ df = chembl.query("""
20
+ SELECT chembl_id, pref_name
21
+ FROM molecule_dictionary
22
+ WHERE pref_name IS NOT NULL
23
+ LIMIT 5
24
+ """)
25
+ ```
26
+
27
+ That query runs against `s3://scigantic-chembl` over DuckDB's httpfs extension. Nothing is downloaded first, and there's no local database file sitting on disk afterward.
28
+
29
+ ## Installation
30
+
31
+ ```console
32
+ $ pip install scigantic-chembl
33
+ ```
34
+
35
+ ## Compared to chembl-downloader
36
+
37
+ [chembl-downloader](https://github.com/cthoyt/chembl-downloader) is the standard way to work with ChEMBL in Python. It covers every release back to chembl_1, and once the SQLite dump is downloaded it works fully offline. This package gives up that range for less setup and two things chembl-downloader doesn't ship on its own: a pre-joined potency table, and similarity search with no separate index build. The mirror here only carries chembl_35 through chembl_37, and only chembl_37 has the pre-joined and similarity layers, so an older release or fully offline work is still a job for chembl-downloader.
38
+
39
+ ## Potency data, pre-joined
40
+
41
+ `activities` needs a five-table join and a few correctness filters before it's usable for structure-activity work. That join is already done, stored as `derived/activities_enriched.parquet`:
42
+
43
+ ```python
44
+ df = chembl.activities(target_chembl_id="CHEMBL203") # EGFR: 18,998 rows, 11,202 compounds
45
+ ```
46
+
47
+ The filters already applied are about correctness, not taste: `pchembl_value` present, `standard_relation = '='`, no `data_validity_comment`, not a `potential_duplicate`. `confidence_score` and `target_type` stay as columns rather than filters, since which rows count as usable SAR data is an analysis choice:
48
+
49
+ ```python
50
+ df = chembl.activities(target_chembl_id="CHEMBL203", min_confidence=8)
51
+ ```
52
+
53
+ `chembl.query()` still reaches the raw tables directly for anything the join leaves out.
54
+
55
+ ## Similarity search
56
+
57
+ ```python
58
+ gefitinib = "COC1=C(C=C2C(=C1)N=CN=C2NC3=CC(=C(C=C3)F)Cl)OCCCN4CCOCC4"
59
+ hits = chembl.similar_compounds(gefitinib, top_k=5)
60
+ ```
61
+
62
+ ```
63
+ chembl_id tanimoto
64
+ CHEMBL939 1.000000
65
+ CHEMBL14699 0.919355
66
+ CHEMBL4165375 0.916667
67
+ CHEMBL299672 0.857143
68
+ CHEMBL4448162 0.857143
69
+ ```
70
+
71
+ CHEMBL939 is gefitinib itself. Every compound with a comparable potency measurement (1.68M of ChEMBL's 2.9M structures) has a precomputed 2048-bit Morgan fingerprint, kept packed in memory and compared with numpy's `bitwise_count` rather than unpacked bit by bit. The corpus loads once per process, about 18 seconds on a typical home connection and faster from inside AWS; every call after that in the same process is under half a second. chembl-downloader can do similarity search too, through `chemfp`, but its own docs put building that index at tens of minutes.
72
+
73
+ Similarity search needs rdkit to encode the query molecule, so it's kept as an optional extra:
74
+
75
+ ```console
76
+ $ pip install "scigantic-chembl[similarity]"
77
+ ```
78
+
79
+ ## What's mirrored
80
+
81
+ ```python
82
+ chembl.releases()
83
+ ```
84
+
85
+ | release | raw tables | pre-joined activities | similarity search | CYP training set |
86
+ |---|---|---|---|---|
87
+ | chembl_37 | yes | yes | yes | yes |
88
+ | chembl_36 | yes | no | no | yes |
89
+ | chembl_35 | yes | no | no | yes |
90
+
91
+ chembl_36 and chembl_35 are raw-table access only: their `activities` table is missing a column chembl_37's has (`modality`), so calling `activities()` or `similar_compounds()` on either one raises `ReleaseCapabilityError` up front instead of failing partway through a join with a confusing error.
92
+
93
+ ## Command line
94
+
95
+ ```console
96
+ $ scigantic-chembl info
97
+ $ scigantic-chembl query "SELECT count(*) FROM activities" --release chembl_37
98
+ ```
99
+
100
+ ## License
101
+
102
+ MIT-0. See [LICENSE](LICENSE).
@@ -0,0 +1,44 @@
1
+ [build-system]
2
+ requires = ["setuptools>=61", "wheel"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "scigantic-chembl"
7
+ version = "0.1.0"
8
+ description = "Query ChEMBL directly from a public S3 mirror with DuckDB. No download, no local database."
9
+ readme = "README.md"
10
+ requires-python = ">=3.10"
11
+ license = { text = "MIT-0" }
12
+ authors = [{ name = "Scigantic" }]
13
+ keywords = ["chembl", "cheminformatics", "duckdb", "drug-discovery", "bioactivity"]
14
+ classifiers = [
15
+ "License :: OSI Approved :: MIT No Attribution License (MIT-0)",
16
+ "Programming Language :: Python :: 3",
17
+ "Topic :: Scientific/Engineering :: Bio-Informatics",
18
+ "Topic :: Scientific/Engineering :: Chemistry",
19
+ ]
20
+
21
+ dependencies = [
22
+ "duckdb>=0.10",
23
+ "pyarrow>=14",
24
+ "numpy>=2.0",
25
+ "pandas>=1.5",
26
+ ]
27
+
28
+ [project.optional-dependencies]
29
+ # rdkit is only needed to encode the query molecule in similar_compounds();
30
+ # everything else works without it.
31
+ similarity = ["rdkit>=2023.9"]
32
+ dev = ["pytest>=7"]
33
+
34
+ [project.urls]
35
+ Homepage = "https://scigantic.com"
36
+
37
+ [project.scripts]
38
+ scigantic-chembl = "scigantic_chembl.cli:main"
39
+
40
+ [tool.setuptools.packages.find]
41
+ where = ["src"]
42
+
43
+ [tool.pytest.ini_options]
44
+ testpaths = ["tests"]
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,26 @@
1
+ """Query ChEMBL directly from a public S3 mirror. No download, no local database."""
2
+
3
+ from .activities import activities
4
+ from .connection import connect, query
5
+ from .releases import (
6
+ ReleaseCapabilityError,
7
+ ReleaseInfo,
8
+ UnknownReleaseError,
9
+ latest,
10
+ releases,
11
+ )
12
+ from .similarity import similar_compounds
13
+
14
+ __version__ = "0.1.0"
15
+
16
+ __all__ = [
17
+ "activities",
18
+ "connect",
19
+ "query",
20
+ "similar_compounds",
21
+ "releases",
22
+ "latest",
23
+ "ReleaseInfo",
24
+ "ReleaseCapabilityError",
25
+ "UnknownReleaseError",
26
+ ]
@@ -0,0 +1,33 @@
1
+ """Shared constants: where the mirror lives, and what it currently carries."""
2
+
3
+ BUCKET = "scigantic-chembl"
4
+ REGION = "us-east-1"
5
+ DEFAULT_RELEASE = "chembl_37"
6
+
7
+ # What's actually mirrored today, and which derived artifacts each release
8
+ # carries. Bump this when the mirror gains a release or an artifact -- see
9
+ # infrastructure/kubernetes/chembl-release-check-cronjob.yaml in the main
10
+ # scigantic repo, which is what keeps the mirror itself current. chembl_36
11
+ # and chembl_35 are raw-table access only: their `activities` table is
12
+ # missing a column (`modality`) that chembl_37's has, so the pre-joined
13
+ # build fails on them rather than silently producing a subtly wrong join.
14
+ RELEASES = {
15
+ "chembl_37": {
16
+ "raw": True,
17
+ "activities_enriched": True,
18
+ "fingerprints": True,
19
+ "cyp_training": True,
20
+ },
21
+ "chembl_36": {
22
+ "raw": True,
23
+ "activities_enriched": False,
24
+ "fingerprints": False,
25
+ "cyp_training": True,
26
+ },
27
+ "chembl_35": {
28
+ "raw": True,
29
+ "activities_enriched": False,
30
+ "fingerprints": False,
31
+ "cyp_training": True,
32
+ },
33
+ }
@@ -0,0 +1,63 @@
1
+ """The pre-joined, pre-filtered potency table.
2
+
3
+ activities is a five-table join (activities, assays, target_dictionary,
4
+ molecule_dictionary, compound_structures, docs) with four correctness
5
+ filters most callers apply anyway: pchembl_value present, standard_relation
6
+ '=', no data_validity_comment, not a potential_duplicate. This is that join
7
+ done once and stored as derived/activities_enriched.parquet, not
8
+ recomputed on every call.
9
+ """
10
+
11
+ from __future__ import annotations
12
+
13
+ from typing import TYPE_CHECKING
14
+
15
+ from ._constants import BUCKET, DEFAULT_RELEASE
16
+ from .connection import connect
17
+ from .releases import _require
18
+
19
+ if TYPE_CHECKING:
20
+ import pandas as pd
21
+
22
+
23
+ def activities(
24
+ release: str = DEFAULT_RELEASE,
25
+ target_chembl_id: str | None = None,
26
+ min_confidence: int | None = None,
27
+ limit: int | None = None,
28
+ ) -> "pd.DataFrame":
29
+ """Potency measurements, already joined and filtered.
30
+
31
+ confidence_score and target_type ride along as columns rather than
32
+ filters -- which rows count as usable SAR data is an analysis choice,
33
+ not something to bake in silently. Pass min_confidence to apply the
34
+ common confidence_score >= 8 convention yourself.
35
+
36
+ Only chembl_37 carries this file. Raises ReleaseCapabilityError on
37
+ chembl_36 and chembl_35, whose `activities` table is missing a column
38
+ this join needs.
39
+ """
40
+ _require(release, "activities_enriched")
41
+ con = connect(release)
42
+ try:
43
+ path = f"s3://{BUCKET}/{release}/derived/activities_enriched.parquet"
44
+ con.execute(
45
+ "CREATE OR REPLACE VIEW activities_enriched AS "
46
+ f"SELECT * FROM read_parquet('{path}')"
47
+ )
48
+ where, params = [], []
49
+ if target_chembl_id is not None:
50
+ where.append("target_chembl_id = ?")
51
+ params.append(target_chembl_id)
52
+ if min_confidence is not None:
53
+ where.append("confidence_score >= ?")
54
+ params.append(min_confidence)
55
+ clause = f"WHERE {' AND '.join(where)}" if where else ""
56
+
57
+ sql = f"SELECT * FROM activities_enriched {clause} ORDER BY pchembl_value DESC"
58
+ if limit is not None:
59
+ sql += " LIMIT ?"
60
+ params.append(int(limit))
61
+ return con.execute(sql, params).df()
62
+ finally:
63
+ con.close()
@@ -0,0 +1,57 @@
1
+ """Command-line interface: `scigantic-chembl info` and `scigantic-chembl query`."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import argparse
6
+ import sys
7
+ from typing import Sequence
8
+
9
+ from ._constants import DEFAULT_RELEASE
10
+ from .connection import query as run_query
11
+ from .releases import releases
12
+
13
+
14
+ def _cmd_info(_args: argparse.Namespace) -> int:
15
+ for info in releases():
16
+ derived = [
17
+ name
18
+ for name, present in (
19
+ ("activities", info.activities_enriched),
20
+ ("fingerprints", info.fingerprints),
21
+ ("cyp_training", info.cyp_training),
22
+ )
23
+ if present
24
+ ]
25
+ suffix = f" + {', '.join(derived)}" if derived else " (raw tables only)"
26
+ print(f"{info.release}{suffix}")
27
+ return 0
28
+
29
+
30
+ def _cmd_query(args: argparse.Namespace) -> int:
31
+ df = run_query(args.sql, release=args.release)
32
+ print(df.to_csv(sep="\t", index=False), end="")
33
+ return 0
34
+
35
+
36
+ def main(argv: Sequence[str] | None = None) -> int:
37
+ parser = argparse.ArgumentParser(prog="scigantic-chembl")
38
+ subparsers = parser.add_subparsers(dest="command", required=True)
39
+
40
+ info_parser = subparsers.add_parser(
41
+ "info", help="list mirrored releases and what each one supports"
42
+ )
43
+ info_parser.set_defaults(func=_cmd_info)
44
+
45
+ query_parser = subparsers.add_parser(
46
+ "query", help="run SQL against a release and print tab-separated output"
47
+ )
48
+ query_parser.add_argument("sql")
49
+ query_parser.add_argument("--release", default=DEFAULT_RELEASE)
50
+ query_parser.set_defaults(func=_cmd_query)
51
+
52
+ args = parser.parse_args(argv)
53
+ return args.func(args)
54
+
55
+
56
+ if __name__ == "__main__":
57
+ sys.exit(main())
@@ -0,0 +1,75 @@
1
+ """DuckDB connection helpers. No download: queries run against the public
2
+ S3 mirror over httpfs.
3
+ """
4
+
5
+ from __future__ import annotations
6
+
7
+ from typing import TYPE_CHECKING
8
+
9
+ from ._constants import BUCKET, DEFAULT_RELEASE, REGION
10
+ from .releases import _validate_release
11
+
12
+ if TYPE_CHECKING:
13
+ import duckdb
14
+ import pandas as pd
15
+
16
+ # Registered as views on connect() so plain SQL can reference a table by
17
+ # name instead of a full read_parquet() path. Every table under
18
+ # <release>/parquet/ is still reachable that way even if it's not listed
19
+ # here -- this is just the set worth naming up front.
20
+ _CORE_TABLES = (
21
+ "activities",
22
+ "assays",
23
+ "target_dictionary",
24
+ "molecule_dictionary",
25
+ "compound_structures",
26
+ "compound_properties",
27
+ "molecule_synonyms",
28
+ "docs",
29
+ "drug_mechanism",
30
+ "drug_indication",
31
+ )
32
+
33
+
34
+ def connect(release: str = DEFAULT_RELEASE) -> "duckdb.DuckDBPyConnection":
35
+ """Open a DuckDB connection against s3://scigantic-chembl.
36
+
37
+ `SELECT * FROM activities` works directly; any other table under
38
+ `<release>/parquet/` is reachable with
39
+ `read_parquet('s3://scigantic-chembl/<release>/parquet/<table>.parquet')`.
40
+ """
41
+ import duckdb
42
+
43
+ _validate_release(release)
44
+
45
+ con = duckdb.connect()
46
+ con.execute("INSTALL httpfs")
47
+ con.execute("LOAD httpfs")
48
+ con.execute(f"SET s3_region='{REGION}'")
49
+ # The mirror is public-read. Without this, DuckDB looks for AWS
50
+ # credentials and fails on a machine that has none configured.
51
+ con.execute(
52
+ "CREATE OR REPLACE SECRET scigantic_chembl "
53
+ "(TYPE s3, PROVIDER config, KEY_ID '', SECRET '')"
54
+ )
55
+
56
+ base = f"s3://{BUCKET}/{release}/parquet"
57
+ for table in _CORE_TABLES:
58
+ con.execute(
59
+ f"CREATE OR REPLACE VIEW {table} AS "
60
+ f"SELECT * FROM read_parquet('{base}/{table}.parquet')"
61
+ )
62
+ return con
63
+
64
+
65
+ def query(sql: str, release: str = DEFAULT_RELEASE) -> "pd.DataFrame":
66
+ """Run SQL against a release and return a pandas DataFrame.
67
+
68
+ Opens a new connection per call. For several queries against the same
69
+ release, call connect() once and reuse it instead.
70
+ """
71
+ con = connect(release)
72
+ try:
73
+ return con.execute(sql).df()
74
+ finally:
75
+ con.close()
@@ -0,0 +1,51 @@
1
+ """Release metadata: what's mirrored, and what each release supports."""
2
+
3
+ from __future__ import annotations
4
+
5
+ from dataclasses import dataclass
6
+
7
+ from ._constants import DEFAULT_RELEASE, RELEASES
8
+
9
+
10
+ class UnknownReleaseError(LookupError):
11
+ """Raised when a release isn't mirrored at all."""
12
+
13
+
14
+ class ReleaseCapabilityError(LookupError):
15
+ """Raised when a release doesn't carry the artifact a call asked for."""
16
+
17
+
18
+ @dataclass(frozen=True)
19
+ class ReleaseInfo:
20
+ release: str
21
+ raw: bool
22
+ activities_enriched: bool
23
+ fingerprints: bool
24
+ cyp_training: bool
25
+
26
+
27
+ def releases() -> list[ReleaseInfo]:
28
+ """List every release the mirror carries, and what each one supports."""
29
+ return [ReleaseInfo(release=name, **caps) for name, caps in RELEASES.items()]
30
+
31
+
32
+ def latest() -> str:
33
+ """The release the archive treats as its current default."""
34
+ return DEFAULT_RELEASE
35
+
36
+
37
+ def _validate_release(release: str) -> None:
38
+ if release not in RELEASES:
39
+ known = ", ".join(RELEASES)
40
+ raise UnknownReleaseError(f"{release!r} is not mirrored. Known releases: {known}.")
41
+
42
+
43
+ def _require(release: str, capability: str) -> None:
44
+ _validate_release(release)
45
+ if not RELEASES[release][capability]:
46
+ raise ReleaseCapabilityError(
47
+ f"{release!r} has no {capability.replace('_', ' ')}. Only "
48
+ f"{DEFAULT_RELEASE} carries the derived layer today. Call "
49
+ "releases() to see what each release supports, or use query() "
50
+ "against the raw parquet tables instead."
51
+ )
@@ -0,0 +1,108 @@
1
+ """Structure similarity search over precomputed fingerprints.
2
+
3
+ Every compound carrying a comparable potency measurement (1.68M of ChEMBL's
4
+ 2.9M structures) has a precomputed 2048-bit Morgan fingerprint (radius 2),
5
+ packed to 256 bytes. Fingerprints are kept packed in memory and compared
6
+ with numpy's bitwise_count, so a search never unpacks to a 2048-column
7
+ array -- that difference is what keeps this to a fraction of a second
8
+ against a corpus that would otherwise be several gigabytes unpacked.
9
+
10
+ Requires rdkit to encode the query molecule: `pip install
11
+ scigantic-chembl[similarity]`.
12
+ """
13
+
14
+ from __future__ import annotations
15
+
16
+ from typing import TYPE_CHECKING
17
+
18
+ import numpy as np
19
+
20
+ from ._constants import BUCKET, DEFAULT_RELEASE
21
+ from .releases import _require
22
+
23
+ if TYPE_CHECKING:
24
+ import pandas as pd
25
+
26
+ _FP_BITS = 2048
27
+ _FP_BYTES = _FP_BITS // 8
28
+
29
+ # One corpus load per (release, process): reused across calls in the same
30
+ # session rather than re-fetched from S3 every time.
31
+ _corpus_cache: dict[str, tuple[list[str], "np.ndarray"]] = {}
32
+
33
+
34
+ def _fingerprint_packed(smiles: str) -> "np.ndarray":
35
+ """Encode a query SMILES the same way the stored corpus was encoded."""
36
+ try:
37
+ from rdkit import Chem
38
+ from rdkit.Chem import rdFingerprintGenerator
39
+ except ImportError as exc:
40
+ raise ImportError(
41
+ "similar_compounds() needs rdkit. Install with: "
42
+ "pip install 'scigantic-chembl[similarity]'"
43
+ ) from exc
44
+
45
+ mol = Chem.MolFromSmiles(smiles)
46
+ if mol is None:
47
+ raise ValueError(f"rdkit could not parse this SMILES: {smiles!r}")
48
+ generator = rdFingerprintGenerator.GetMorganGenerator(radius=2, fpSize=_FP_BITS)
49
+ bits = generator.GetFingerprintAsNumPy(mol).astype(np.uint8)
50
+ return np.packbits(bits)
51
+
52
+
53
+ def _load_corpus(release: str) -> tuple[list[str], "np.ndarray"]:
54
+ if release in _corpus_cache:
55
+ return _corpus_cache[release]
56
+
57
+ import pyarrow.fs as pafs
58
+ import pyarrow.parquet as pq
59
+
60
+ # pyarrow's own S3 filesystem, not DuckDB: reading the packed-byte
61
+ # column straight off its Arrow buffer is what keeps this fast. Going
62
+ # through DuckDB's dataframe conversion instead turns 1.68M individual
63
+ # 256-byte python bytes objects and a few seconds becomes minutes.
64
+ filesystem = pafs.S3FileSystem(region="us-east-1", anonymous=True)
65
+ key = f"{BUCKET}/{release}/derived/fingerprints.parquet"
66
+ table = pq.read_table(key, filesystem=filesystem)
67
+
68
+ ids = table.column("chembl_id").to_pylist()
69
+ raw = table.column("fp").combine_chunks().buffers()[1]
70
+ packed = np.frombuffer(raw, dtype=np.uint8).reshape(len(ids), _FP_BYTES)
71
+
72
+ _corpus_cache[release] = (ids, packed)
73
+ return ids, packed
74
+
75
+
76
+ def similar_compounds(
77
+ smiles: str,
78
+ release: str = DEFAULT_RELEASE,
79
+ top_k: int = 20,
80
+ ) -> "pd.DataFrame":
81
+ """Rank the corpus by Tanimoto similarity to a query SMILES.
82
+
83
+ Only chembl_37 has a fingerprints.parquet. The first call in a process
84
+ pays for loading the corpus from S3 (a few seconds); later calls in the
85
+ same process reuse it.
86
+ """
87
+ _require(release, "fingerprints")
88
+ query_fp = _fingerprint_packed(smiles)
89
+ ids, corpus = _load_corpus(release)
90
+
91
+ intersection = np.bitwise_count(corpus & query_fp).sum(axis=1, dtype=np.int32)
92
+ union = np.bitwise_count(corpus | query_fp).sum(axis=1, dtype=np.int32)
93
+ tanimoto = np.divide(
94
+ intersection,
95
+ union,
96
+ out=np.zeros(len(ids), dtype=np.float64),
97
+ where=union != 0,
98
+ )
99
+
100
+ order = np.argsort(-tanimoto, kind="stable")[:top_k]
101
+ import pandas as pd
102
+
103
+ return pd.DataFrame(
104
+ {
105
+ "chembl_id": [ids[i] for i in order],
106
+ "tanimoto": tanimoto[order],
107
+ }
108
+ )
@@ -0,0 +1,127 @@
1
+ Metadata-Version: 2.4
2
+ Name: scigantic-chembl
3
+ Version: 0.1.0
4
+ Summary: Query ChEMBL directly from a public S3 mirror with DuckDB. No download, no local database.
5
+ Author: Scigantic
6
+ License: MIT-0
7
+ Project-URL: Homepage, https://scigantic.com
8
+ Keywords: chembl,cheminformatics,duckdb,drug-discovery,bioactivity
9
+ Classifier: License :: OSI Approved :: MIT No Attribution License (MIT-0)
10
+ Classifier: Programming Language :: Python :: 3
11
+ Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
12
+ Classifier: Topic :: Scientific/Engineering :: Chemistry
13
+ Requires-Python: >=3.10
14
+ Description-Content-Type: text/markdown
15
+ License-File: LICENSE
16
+ Requires-Dist: duckdb>=0.10
17
+ Requires-Dist: pyarrow>=14
18
+ Requires-Dist: numpy>=2.0
19
+ Requires-Dist: pandas>=1.5
20
+ Provides-Extra: similarity
21
+ Requires-Dist: rdkit>=2023.9; extra == "similarity"
22
+ Provides-Extra: dev
23
+ Requires-Dist: pytest>=7; extra == "dev"
24
+ Dynamic: license-file
25
+
26
+ <h1 align="center">scigantic-chembl</h1>
27
+
28
+ <p align="center">
29
+ <a href="https://github.com/Scigantic/scigantic-chembl/actions/workflows/ci.yml">
30
+ <img alt="CI" src="https://github.com/Scigantic/scigantic-chembl/actions/workflows/ci.yml/badge.svg" /></a>
31
+ <a href="https://pypi.org/project/scigantic-chembl/">
32
+ <img alt="PyPI" src="https://img.shields.io/pypi/v/scigantic-chembl" /></a>
33
+ <a href="https://pypi.org/project/scigantic-chembl/">
34
+ <img alt="PyPI - Python Version" src="https://img.shields.io/pypi/pyversions/scigantic-chembl" /></a>
35
+ <a href="https://github.com/Scigantic/scigantic-chembl/blob/main/LICENSE">
36
+ <img alt="License" src="https://img.shields.io/github/license/Scigantic/scigantic-chembl" /></a>
37
+ </p>
38
+
39
+ Query ChEMBL directly from a public S3 mirror with DuckDB.
40
+
41
+ ```python
42
+ import scigantic_chembl as chembl
43
+
44
+ df = chembl.query("""
45
+ SELECT chembl_id, pref_name
46
+ FROM molecule_dictionary
47
+ WHERE pref_name IS NOT NULL
48
+ LIMIT 5
49
+ """)
50
+ ```
51
+
52
+ That query runs against `s3://scigantic-chembl` over DuckDB's httpfs extension. Nothing is downloaded first, and there's no local database file sitting on disk afterward.
53
+
54
+ ## Installation
55
+
56
+ ```console
57
+ $ pip install scigantic-chembl
58
+ ```
59
+
60
+ ## Compared to chembl-downloader
61
+
62
+ [chembl-downloader](https://github.com/cthoyt/chembl-downloader) is the standard way to work with ChEMBL in Python. It covers every release back to chembl_1, and once the SQLite dump is downloaded it works fully offline. This package gives up that range for less setup and two things chembl-downloader doesn't ship on its own: a pre-joined potency table, and similarity search with no separate index build. The mirror here only carries chembl_35 through chembl_37, and only chembl_37 has the pre-joined and similarity layers, so an older release or fully offline work is still a job for chembl-downloader.
63
+
64
+ ## Potency data, pre-joined
65
+
66
+ `activities` needs a five-table join and a few correctness filters before it's usable for structure-activity work. That join is already done, stored as `derived/activities_enriched.parquet`:
67
+
68
+ ```python
69
+ df = chembl.activities(target_chembl_id="CHEMBL203") # EGFR: 18,998 rows, 11,202 compounds
70
+ ```
71
+
72
+ The filters already applied are about correctness, not taste: `pchembl_value` present, `standard_relation = '='`, no `data_validity_comment`, not a `potential_duplicate`. `confidence_score` and `target_type` stay as columns rather than filters, since which rows count as usable SAR data is an analysis choice:
73
+
74
+ ```python
75
+ df = chembl.activities(target_chembl_id="CHEMBL203", min_confidence=8)
76
+ ```
77
+
78
+ `chembl.query()` still reaches the raw tables directly for anything the join leaves out.
79
+
80
+ ## Similarity search
81
+
82
+ ```python
83
+ gefitinib = "COC1=C(C=C2C(=C1)N=CN=C2NC3=CC(=C(C=C3)F)Cl)OCCCN4CCOCC4"
84
+ hits = chembl.similar_compounds(gefitinib, top_k=5)
85
+ ```
86
+
87
+ ```
88
+ chembl_id tanimoto
89
+ CHEMBL939 1.000000
90
+ CHEMBL14699 0.919355
91
+ CHEMBL4165375 0.916667
92
+ CHEMBL299672 0.857143
93
+ CHEMBL4448162 0.857143
94
+ ```
95
+
96
+ CHEMBL939 is gefitinib itself. Every compound with a comparable potency measurement (1.68M of ChEMBL's 2.9M structures) has a precomputed 2048-bit Morgan fingerprint, kept packed in memory and compared with numpy's `bitwise_count` rather than unpacked bit by bit. The corpus loads once per process, about 18 seconds on a typical home connection and faster from inside AWS; every call after that in the same process is under half a second. chembl-downloader can do similarity search too, through `chemfp`, but its own docs put building that index at tens of minutes.
97
+
98
+ Similarity search needs rdkit to encode the query molecule, so it's kept as an optional extra:
99
+
100
+ ```console
101
+ $ pip install "scigantic-chembl[similarity]"
102
+ ```
103
+
104
+ ## What's mirrored
105
+
106
+ ```python
107
+ chembl.releases()
108
+ ```
109
+
110
+ | release | raw tables | pre-joined activities | similarity search | CYP training set |
111
+ |---|---|---|---|---|
112
+ | chembl_37 | yes | yes | yes | yes |
113
+ | chembl_36 | yes | no | no | yes |
114
+ | chembl_35 | yes | no | no | yes |
115
+
116
+ chembl_36 and chembl_35 are raw-table access only: their `activities` table is missing a column chembl_37's has (`modality`), so calling `activities()` or `similar_compounds()` on either one raises `ReleaseCapabilityError` up front instead of failing partway through a join with a confusing error.
117
+
118
+ ## Command line
119
+
120
+ ```console
121
+ $ scigantic-chembl info
122
+ $ scigantic-chembl query "SELECT count(*) FROM activities" --release chembl_37
123
+ ```
124
+
125
+ ## License
126
+
127
+ MIT-0. See [LICENSE](LICENSE).
@@ -0,0 +1,21 @@
1
+ LICENSE
2
+ README.md
3
+ pyproject.toml
4
+ src/scigantic_chembl/__init__.py
5
+ src/scigantic_chembl/_constants.py
6
+ src/scigantic_chembl/activities.py
7
+ src/scigantic_chembl/cli.py
8
+ src/scigantic_chembl/connection.py
9
+ src/scigantic_chembl/releases.py
10
+ src/scigantic_chembl/similarity.py
11
+ src/scigantic_chembl.egg-info/PKG-INFO
12
+ src/scigantic_chembl.egg-info/SOURCES.txt
13
+ src/scigantic_chembl.egg-info/dependency_links.txt
14
+ src/scigantic_chembl.egg-info/entry_points.txt
15
+ src/scigantic_chembl.egg-info/requires.txt
16
+ src/scigantic_chembl.egg-info/top_level.txt
17
+ tests/test_activities.py
18
+ tests/test_cli.py
19
+ tests/test_connection.py
20
+ tests/test_releases.py
21
+ tests/test_similarity.py
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ scigantic-chembl = scigantic_chembl.cli:main
@@ -0,0 +1,10 @@
1
+ duckdb>=0.10
2
+ pyarrow>=14
3
+ numpy>=2.0
4
+ pandas>=1.5
5
+
6
+ [dev]
7
+ pytest>=7
8
+
9
+ [similarity]
10
+ rdkit>=2023.9
@@ -0,0 +1 @@
1
+ scigantic_chembl
@@ -0,0 +1,23 @@
1
+ import scigantic_chembl as chembl
2
+
3
+
4
+ def test_egfr_activities():
5
+ # CHEMBL203 is EGFR. Exact count verified against the live mirror; update
6
+ # this if chembl_37 is ever replaced by a different release under the
7
+ # same "current" pointer.
8
+ df = chembl.activities(target_chembl_id="CHEMBL203")
9
+ assert len(df) == 18998
10
+ assert df["compound_chembl_id"].nunique() == 11202
11
+ assert (df["target_chembl_id"] == "CHEMBL203").all()
12
+
13
+
14
+ def test_activities_sorted_by_potency_descending():
15
+ df = chembl.activities(target_chembl_id="CHEMBL203", limit=100)
16
+ assert df["pchembl_value"].is_monotonic_decreasing
17
+
18
+
19
+ def test_min_confidence_filter_narrows_results():
20
+ unfiltered = chembl.activities(target_chembl_id="CHEMBL203")
21
+ filtered = chembl.activities(target_chembl_id="CHEMBL203", min_confidence=8)
22
+ assert len(filtered) <= len(unfiltered)
23
+ assert (filtered["confidence_score"] >= 8).all()
@@ -0,0 +1,17 @@
1
+ from scigantic_chembl.cli import main
2
+
3
+
4
+ def test_info_command(capsys):
5
+ exit_code = main(["info"])
6
+ assert exit_code == 0
7
+ out = capsys.readouterr().out
8
+ assert "chembl_37" in out
9
+ assert "chembl_36" in out
10
+
11
+
12
+ def test_query_command(capsys):
13
+ exit_code = main(["query", "SELECT 1 AS n"])
14
+ assert exit_code == 0
15
+ out = capsys.readouterr().out
16
+ assert "n" in out
17
+ assert "1" in out
@@ -0,0 +1,33 @@
1
+ """Real queries against the live public mirror, no mocks.
2
+
3
+ Network-dependent by design: the whole point of this package is that it
4
+ answers real queries against real data with no local setup, so that is what
5
+ gets tested.
6
+ """
7
+
8
+ import scigantic_chembl as chembl
9
+
10
+
11
+ def test_query_returns_dataframe_with_expected_columns():
12
+ df = chembl.query(
13
+ "SELECT chembl_id, pref_name FROM molecule_dictionary "
14
+ "WHERE pref_name IS NOT NULL LIMIT 5"
15
+ )
16
+ assert list(df.columns) == ["chembl_id", "pref_name"]
17
+ assert len(df) == 5
18
+
19
+
20
+ def test_query_against_chembl_36_raw_table():
21
+ df = chembl.query("SELECT count(*) AS n FROM target_dictionary", release="chembl_36")
22
+ assert df["n"].iloc[0] > 0
23
+
24
+
25
+ def test_connect_reuse_across_two_queries():
26
+ con = chembl.connect()
27
+ try:
28
+ a = con.execute("SELECT count(*) FROM activities").fetchone()[0]
29
+ b = con.execute("SELECT count(*) FROM assays").fetchone()[0]
30
+ assert a > 0
31
+ assert b > 0
32
+ finally:
33
+ con.close()
@@ -0,0 +1,37 @@
1
+ import pytest
2
+
3
+ import scigantic_chembl as chembl
4
+ from scigantic_chembl.releases import ReleaseCapabilityError, UnknownReleaseError
5
+
6
+
7
+ def test_releases_lists_known_releases():
8
+ names = {r.release for r in chembl.releases()}
9
+ assert names == {"chembl_37", "chembl_36", "chembl_35"}
10
+
11
+
12
+ def test_latest_is_chembl_37():
13
+ assert chembl.latest() == "chembl_37"
14
+
15
+
16
+ def test_chembl_37_has_full_derived_layer():
17
+ info = {r.release: r for r in chembl.releases()}["chembl_37"]
18
+ assert info.activities_enriched
19
+ assert info.fingerprints
20
+ assert info.cyp_training
21
+
22
+
23
+ def test_chembl_36_is_raw_only():
24
+ info = {r.release: r for r in chembl.releases()}["chembl_36"]
25
+ assert info.raw
26
+ assert not info.activities_enriched
27
+ assert not info.fingerprints
28
+
29
+
30
+ def test_activities_on_chembl_36_raises_capability_error():
31
+ with pytest.raises(ReleaseCapabilityError):
32
+ chembl.activities(release="chembl_36")
33
+
34
+
35
+ def test_unknown_release_raises():
36
+ with pytest.raises(UnknownReleaseError):
37
+ chembl.query("SELECT 1", release="chembl_99")
@@ -0,0 +1,28 @@
1
+ import pytest
2
+
3
+ import scigantic_chembl as chembl
4
+
5
+ rdkit = pytest.importorskip("rdkit")
6
+
7
+ # PubChem CID 123631, canonical SMILES for gefitinib.
8
+ GEFITINIB_SMILES = "COC1=C(C=C2C(=C1)N=CN=C2NC3=CC(=C(C=C3)F)Cl)OCCCN4CCOCC4"
9
+
10
+
11
+ def test_similar_compounds_returns_query_itself_as_top_hit():
12
+ hits = chembl.similar_compounds(GEFITINIB_SMILES, top_k=5)
13
+ assert list(hits.columns) == ["chembl_id", "tanimoto"]
14
+ assert len(hits) == 5
15
+ assert hits["tanimoto"].iloc[0] == pytest.approx(1.0)
16
+ assert hits["tanimoto"].is_monotonic_decreasing
17
+
18
+
19
+ def test_similar_compounds_on_release_without_fingerprints_raises():
20
+ from scigantic_chembl.releases import ReleaseCapabilityError
21
+
22
+ with pytest.raises(ReleaseCapabilityError):
23
+ chembl.similar_compounds(GEFITINIB_SMILES, release="chembl_36")
24
+
25
+
26
+ def test_invalid_smiles_raises_value_error():
27
+ with pytest.raises(ValueError):
28
+ chembl.similar_compounds("not a smiles string")