tfbpshiny 1.0.0__tar.gz

tfbpshiny-1.0.0/LICENSE

@@ -0,0 +1,14 @@
+Copyright (c) 2024, Chase Mateusiak
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.

tfbpshiny-1.0.0/PKG-INFO

@@ -0,0 +1,159 @@
+Metadata-Version: 2.4
+Name: tfbpshiny
+Version: 1.0.0
+Summary: A shiny app which provides a dashboard to https://huggingface.co/collections/BrentLab/yeastresources
+License: GPL-3.0
+License-File: LICENSE
+Author: chasem
+Author-email: chasem@wustl.edu
+Requires-Python: >=3.11,<4.0
+Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Dist: faicons (>=0.2.2,<0.3.0)
+Requires-Dist: labretriever (>=1.1.2,<2.0.0)
+Requires-Dist: plotly (>=6.0.1,<7.0.0)
+Requires-Dist: python-dotenv (>=1.1.0,<2.0.0)
+Requires-Dist: scikit-learn (>=1.6.1,<2.0.0)
+Requires-Dist: shiny (>=1.4.0,<2.0.0)
+Requires-Dist: shinywidgets (>=0.7.1,<0.8.0)
+Requires-Dist: upsetjs-jupyter-widget (>=1.9.0,<2.0.0)
+Description-Content-Type: text/markdown
+
+# TFBPShiny
+
+A Shiny web application for exploring transcription factor binding and perturbation
+data from the [Brent Lab yeast collection](https://huggingface.co/collections/BrentLab/yeastresources).
+
+---
+
+## Resource Requirements
+
+This app requires the following minimum resources to run:
+
+- 4GB storage on disk
+- 8GB RAM (10GB or more is recommended for better performance)
+
+## Quick start
+
+If you wish to keep the app separated from your local environment, you should first
+create a virtual environment. You can do this with `venv`. `cd` to the directory
+where you want the virtual environment to be created, and run:
+
+```bash
+python -m venv tfbpshiny_env
+source tfbpshiny_env/bin/activate
+```
+
+### Install
+
+```bash
+python -m pip install tfbpshiny
+```
+
+### Run the app:
+
+This will download the necessary datasets from huggingface into a cache directory
+that is created in your current working directory. By default, it is called
+`./tfbpshiny_hf_cache`. When you run the app again, if you launch it from the same
+location it will verify that the cache is up to date, and use it, without
+re-downloading. You can also specify a custom cache directory with `--cache-dir`.
+
+```bash
+python -m tfbpshiny launch
+```
+
+To install the latest development version from GitHub, use:
+
+```bash
+python -m pip install git+https://github.com/BrentLab/tfbpshiny@dev
+```
+
+For production deployment (EC2/Docker) and shinyapps.io deployment instructions,
+see [docs/development.md](docs/development.md).
+
+---
+
+## Contributing
+
+### Setup
+
+```bash
+git clone https://github.com/BrentLab/tfbpshiny.git
+cd tfbpshiny
+poetry install
+pre-commit install
+# First-time Playwright setup (required for E2E tests)
+poetry run playwright install chromium
+```
+
+### Plotly JS bundle
+
+The app loads Plotly from a local bundle (`tfbpshiny/www/plotly-3.5.0.min.js`)
+rather than a CDN to avoid race conditions when multiple outputs initialize
+simultaneously. This file is gitignored due to its size (~4.8 MB). After
+cloning, download it once:
+
+```bash
+curl -fsSL https://cdn.plot.ly/plotly-3.5.0.min.js \
+    -o tfbpshiny/www/plotly-3.5.0.min.js
+```
+
+If the `plotly` Python package is upgraded, check the new JS version it expects:
+
+```bash
+python -c "
+import re, plotly.graph_objects as go
+from plotly.io import to_html
+m = re.search(r'plotly-([\d.]+)\.min\.js', to_html(go.Figure(), include_plotlyjs='cdn'))
+print(m.group(0))
+"
+```
+
+Then download the matching version and update the `src` in `tfbpshiny/app.py`.
+
+### Environment variables
+
+Create a `.env` file in the repo root to override defaults:
+
+```bash
+# Optional — only needed for private HuggingFace datasets
+HF_TOKEN=<your_huggingface_token>
+
+# Optional — override the VirtualDB config path
+VIRTUALDB_CONFIG=/path/to/custom_config.yaml
+```
+
+### Running the app
+
+```bash
+poetry run python -m tfbpshiny --log-level DEBUG launch \
+    --port 8010 --host 127.0.0.1 --debug
+```
+
+### Running tests
+
+```bash
+poetry run pytest tests/unit/      # unit tests
+poetry run pytest tests/e2e/       # end-to-end
+poetry run pytest                   # all tests
+```
+
+### Code quality
+
+```bash
+pre-commit run --all-files
+```
+
+### Branching
+
+1. Switch to `dev`: `git switch dev`
+1. Branch from `dev` — **not** `main`: `git switch -c my-feature`
+1. Keep branches small and focused to make review easier
+1. Rebase onto `dev` periodically: `git rebase dev`
+1. When ready, open a pull request targeting the BrentLab `dev`
+  branch — **not** `main`
+

tfbpshiny-1.0.0/README.md

@@ -0,0 +1,133 @@
+# TFBPShiny
+
+A Shiny web application for exploring transcription factor binding and perturbation
+data from the [Brent Lab yeast collection](https://huggingface.co/collections/BrentLab/yeastresources).
+
+---
+
+## Resource Requirements
+
+This app requires the following minimum resources to run:
+
+- 4GB storage on disk
+- 8GB RAM (10GB or more is recommended for better performance)
+
+## Quick start
+
+If you wish to keep the app separated from your local environment, you should first
+create a virtual environment. You can do this with `venv`. `cd` to the directory
+where you want the virtual environment to be created, and run:
+
+```bash
+python -m venv tfbpshiny_env
+source tfbpshiny_env/bin/activate
+```
+
+### Install
+
+```bash
+python -m pip install tfbpshiny
+```
+
+### Run the app:
+
+This will download the necessary datasets from huggingface into a cache directory
+that is created in your current working directory. By default, it is called
+`./tfbpshiny_hf_cache`. When you run the app again, if you launch it from the same
+location it will verify that the cache is up to date, and use it, without
+re-downloading. You can also specify a custom cache directory with `--cache-dir`.
+
+```bash
+python -m tfbpshiny launch
+```
+
+To install the latest development version from GitHub, use:
+
+```bash
+python -m pip install git+https://github.com/BrentLab/tfbpshiny@dev
+```
+
+For production deployment (EC2/Docker) and shinyapps.io deployment instructions,
+see [docs/development.md](docs/development.md).
+
+---
+
+## Contributing
+
+### Setup
+
+```bash
+git clone https://github.com/BrentLab/tfbpshiny.git
+cd tfbpshiny
+poetry install
+pre-commit install
+# First-time Playwright setup (required for E2E tests)
+poetry run playwright install chromium
+```
+
+### Plotly JS bundle
+
+The app loads Plotly from a local bundle (`tfbpshiny/www/plotly-3.5.0.min.js`)
+rather than a CDN to avoid race conditions when multiple outputs initialize
+simultaneously. This file is gitignored due to its size (~4.8 MB). After
+cloning, download it once:
+
+```bash
+curl -fsSL https://cdn.plot.ly/plotly-3.5.0.min.js \
+    -o tfbpshiny/www/plotly-3.5.0.min.js
+```
+
+If the `plotly` Python package is upgraded, check the new JS version it expects:
+
+```bash
+python -c "
+import re, plotly.graph_objects as go
+from plotly.io import to_html
+m = re.search(r'plotly-([\d.]+)\.min\.js', to_html(go.Figure(), include_plotlyjs='cdn'))
+print(m.group(0))
+"
+```
+
+Then download the matching version and update the `src` in `tfbpshiny/app.py`.
+
+### Environment variables
+
+Create a `.env` file in the repo root to override defaults:
+
+```bash
+# Optional — only needed for private HuggingFace datasets
+HF_TOKEN=<your_huggingface_token>
+
+# Optional — override the VirtualDB config path
+VIRTUALDB_CONFIG=/path/to/custom_config.yaml
+```
+
+### Running the app
+
+```bash
+poetry run python -m tfbpshiny --log-level DEBUG launch \
+    --port 8010 --host 127.0.0.1 --debug
+```
+
+### Running tests
+
+```bash
+poetry run pytest tests/unit/      # unit tests
+poetry run pytest tests/e2e/       # end-to-end
+poetry run pytest                   # all tests
+```
+
+### Code quality
+
+```bash
+pre-commit run --all-files
+```
+
+### Branching
+
+1. Switch to `dev`: `git switch dev`
+1. Branch from `dev` — **not** `main`: `git switch -c my-feature`
+1. Keep branches small and focused to make review easier
+1. Rebase onto `dev` periodically: `git rebase dev`
+1. When ready, open a pull request targeting the BrentLab `dev`
+  branch — **not** `main`

tfbpshiny-1.0.0/pyproject.toml

@@ -0,0 +1,66 @@
+[tool.poetry]
+name = "tfbpshiny"
+version = "1.0.0"
+description = "A shiny app which provides a dashboard to https://huggingface.co/collections/BrentLab/yeastresources"
+authors = ["chasem <chasem@wustl.edu>"]
+license = "GPL-3.0"
+readme = "README.md"
+
+[tool.poetry.dependencies]
+python = "^3.11"
+labretriever = "^1.1.2"
+shiny = "^1.4.0"
+shinywidgets = "^0.7.1"
+upsetjs-jupyter-widget = "^1.9.0"
+scikit-learn = "^1.6.1"
+plotly = "^6.0.1"
+python-dotenv = "^1.1.0"
+faicons = "^0.2.2"
+
+[tool.poetry.group.dev.dependencies]
+ipykernel = "^7.2.0"
+pytest = "^9.0.2"
+nbformat = "^5.10.4"
+playwright = "^1.58.0"
+pytest-playwright = "^0.7.2"
+
+[tool.pytest.ini_options]
+minversion = "6.0"
+python_files = [
+    "tests.py",
+    "test_*.py",
+]
+pythonpath = ['.']
+addopts = "--ignore=tmp"
+
+[tool.coverage.run]
+include = ["tfbpshiny/**"]
+omit = ["*/tests/*"]
+
+[tool.black]
+line-length = 88
+target-version = ['py311']
+include = '\.py$'
+
+[tool.isort]
+profile = "black"
+line_length = 88
+known_first_party = [
+    "tfbpshiny",
+]
+skip = ["venv/"]
+
+[tool.mypy]
+python_version = "3.11"
+check_untyped_defs = true
+ignore_missing_imports = true
+warn_unused_ignores = false
+warn_redundant_casts = true
+warn_unused_configs = true
+
+[tool.pylint.FORMAT]
+max-line-length = 88
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"

tfbpshiny-1.0.0/tfbpshiny/__main__.py

@@ -0,0 +1,215 @@
+from __future__ import annotations
+
+import argparse
+import logging
+import os
+import pathlib
+import sys
+from typing import Literal, cast
+
+from shiny import run_app
+
+from tfbpshiny.configure_logger import LogLevel, configure_logger
+
+_DEFAULT_VIRTUALDB_CONFIG = str(
+    __import__("pathlib").Path(__file__).parent / "brentlab_yeast_collection.yaml"
+)
+
+_DEFAULT_CACHE_DIR = "./tfbpshiny_hf_cache"
+
+
+def _apply_cache_dir(cache_dir: str) -> None:
+    """
+    Set ``HF_CACHE_DIR`` to the resolved absolute path before any HF imports.
+
+    Must be called before importing labretriever or huggingface_hub so that
+    ``snapshot_download`` and ``VirtualDB`` see the overridden path.
+
+    :param cache_dir: Path to the HuggingFace cache directory.
+
+    """
+    os.environ["HF_CACHE_DIR"] = str(pathlib.Path(cache_dir).resolve())
+
+
+def _run_initialize(
+    virtualdb_config: str,
+    hf_token: str | None,
+    log_level: int,
+    log_handler: Literal["console", "file"],
+) -> None:
+    """
+    Download all dataset files into the local HuggingFace cache and verify views.
+
+    Exits the process with code 1 if any download or view-verification step fails.
+
+    :param virtualdb_config: Path to the VirtualDB YAML config file.
+    :param hf_token: Optional HuggingFace token for private repo access.
+    :param log_level: Numeric logging level (e.g. ``logging.INFO``).
+    :param log_handler: Handler type passed to :func:`configure_logger`.
+
+    """
+    from tfbpshiny.utils.vdb_init import initialize_data
+
+    configure_logger("shiny", level=log_level, handler_type=log_handler)
+    logger = logging.getLogger("shiny")
+
+    cache_msg = os.environ.get("HF_CACHE_DIR", "(huggingface default)")
+    logger.info("Downloading all datasets into HuggingFace cache: %s", cache_msg)
+    try:
+        vdb, _ = initialize_data(virtualdb_config, hf_token, local_files_only=False)
+    except Exception:
+        logger.exception("Cache initialization failed.")
+        sys.exit(1)
+
+    logger.info("Verifying all dataset views are readable...")
+    views_df = vdb.query(
+        "SELECT view_name FROM duckdb_views()"
+        " WHERE schema_name = 'main'"
+        "   AND view_name NOT LIKE 'duckdb_%'"
+        "   AND view_name NOT LIKE 'sqlite_%'"
+        "   AND view_name NOT LIKE 'pragma_%'"
+        " ORDER BY view_name"
+    )
+    view_names = views_df["view_name"].tolist()
+    failed: list[str] = []
+    for view_name in view_names:
+        try:
+            df = vdb.query(f'SELECT * FROM "{view_name}" LIMIT 1')
+            logger.info("  OK  %-30s  (%d col(s))", view_name, len(df.columns))
+        except Exception:
+            logger.exception("  FAIL %s", view_name)
+            failed.append(view_name)
+
+    if failed:
+        logger.error("Verification failed for: %s", ", ".join(failed))
+        sys.exit(1)
+
+    logger.info("Cache initialization complete.")
+
+
+def run_launch(args: argparse.Namespace) -> None:
+    """
+    Download the dataset cache (unless ``--skip-initialize`` is set), then start the
+    app.
+
+    By default uses ``./tfbpshiny_hf_cache`` as the HuggingFace cache directory so
+    a plain ``python -m tfbpshiny launch`` is self-contained: it downloads data on
+    first run and serves it on subsequent runs from the same local directory.
+
+    """
+    cache_dir: str = args.cache_dir
+    _apply_cache_dir(cache_dir)
+
+    log_level = LogLevel.from_string(args.log_level)
+    hf_token: str | None = os.getenv("HF_TOKEN")
+
+    if not args.skip_initialize:
+        _run_initialize(
+            virtualdb_config=args.virtualdb_config,
+            hf_token=hf_token,
+            log_level=log_level.value,
+            log_handler=cast(Literal["console", "file"], args.log_handler),
+        )
+
+    # Env vars are the only reliable way to pass config to uvicorn reload workers,
+    # which re-import app.py in a subprocess and cannot see in-process mutations.
+    os.environ["TFBPSHINY_LOG_LEVEL"] = str(log_level.value)
+    os.environ["TFBPSHINY_LOG_HANDLER"] = args.log_handler
+    os.environ["VIRTUALDB_CONFIG"] = args.virtualdb_config
+    os.environ["TFBPSHINY_MATERIALIZE"] = "0" if args.no_materialize else "1"
+
+    kwargs: dict[str, object] = {"port": args.port, "host": args.host}
+    if args.debug:
+        kwargs.update({"reload": True, "reload_dirs": ["tfbpshiny/shiny_app"]})
+    run_app("tfbpshiny.app:app", **kwargs)  # type: ignore
+
+
+def make_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="tfbpshiny",
+        description=(
+            "tfbpshiny — TF Binding and Perturbation Explorer."
+            " Use --help after any command."
+        ),
+        epilog="Use 'tfbpshiny <command> --help' for more info on each command.",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+    parser.add_argument(
+        "--log-level",
+        type=str,
+        default="WARNING",
+        choices=["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"],
+        help="Set logging level.",
+    )
+    parser.add_argument(
+        "--log-handler",
+        type=str,
+        default="console",
+        choices=["console", "file"],
+        help="Set log handler type.",
+    )
+    parser.add_argument(
+        "--virtualdb-config",
+        type=str,
+        default=_DEFAULT_VIRTUALDB_CONFIG,
+        help="Path to the VirtualDB YAML configuration file.",
+    )
+
+    subparsers = parser.add_subparsers(dest="command", required=True)
+
+    launch_parser = subparsers.add_parser(
+        "launch",
+        help="Download the dataset cache (first run) and start the Shiny app.",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+    launch_parser.add_argument(
+        "--cache-dir",
+        type=str,
+        default=_DEFAULT_CACHE_DIR,
+        help=(
+            "HuggingFace cache directory. Datasets are downloaded here on first run "
+            "and read from here on subsequent runs. "
+            "Equivalent to setting HF_CACHE_DIR."
+        ),
+    )
+    launch_parser.add_argument(
+        "--skip-initialize",
+        action="store_true",
+        default=False,
+        help=(
+            "Skip the dataset download and verification step. "
+            "Use when the cache is already populated and you want a faster startup."
+        ),
+    )
+    launch_parser.add_argument(
+        "--no-materialize",
+        action="store_true",
+        default=False,
+        help=(
+            "Disable in-memory materialization of dataset views at startup. "
+            "Reduces startup memory at the cost of slower query performance. "
+            "Equivalent to setting TFBPSHINY_MATERIALIZE=0."
+        ),
+    )
+    launch_parser.add_argument(
+        "--port", type=int, default=8000, help="Port to serve the Shiny app on."
+    )
+    launch_parser.add_argument(
+        "--host", type=str, default="127.0.0.1", help="Host to bind the Shiny app."
+    )
+    launch_parser.add_argument(
+        "--debug", action="store_true", help="Enable debug mode with auto-reload."
+    )
+    launch_parser.set_defaults(func=run_launch)
+
+    return parser
+
+
+def main() -> None:
+    parser = make_parser()
+    args = parser.parse_args()
+    args.func(args)
+
+
+if __name__ == "__main__":
+    main()