opencloning 0.1.0__tar.gz

opencloning-0.1.0/LICENSE

@@ -0,0 +1,7 @@
+Copyright 2022 Manuel Lera Ramirez
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

opencloning-0.1.0/PKG-INFO

@@ -0,0 +1,168 @@
+Metadata-Version: 2.3
+Name: opencloning
+Version: 0.1.0
+Summary: Backend of OpenCloning, a web application to generate molecular cloning strategies in json format, and share them with others.
+License: MIT
+Author: Manuel Lera-Ramirez
+Author-email: manulera14@gmail.com
+Requires-Python: >=3.11,<4.0
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Dist: beautifulsoup4 (>=4.11.1,<5.0.0)
+Requires-Dist: fastapi
+Requires-Dist: httpx (>=0.25.0,<0.26.0)
+Requires-Dist: opencloning-linkml (>=0.2a0,<0.3)
+Requires-Dist: openpyxl (>=3.1.5,<4.0.0)
+Requires-Dist: pandas (>=2.2.3,<3.0.0)
+Requires-Dist: pydantic (>=2.7.1,<3.0.0)
+Requires-Dist: pydna (>=5.3,<6.0)
+Requires-Dist: python-multipart
+Requires-Dist: pyyaml (>=6.0.2,<7.0.0)
+Requires-Dist: regex (>=2023.10.3,<2024.0.0)
+Requires-Dist: requests (>=2.31.0,<3.0.0)
+Requires-Dist: uvicorn
+Project-URL: Repository, https://github.com/manulera/OpenCloning_backend
+Description-Content-Type: text/markdown
+
+[![Python tests](https://github.com/manulera/OpenCloning_backend/actions/workflows/ci.yml/badge.svg)](https://github.com/manulera/OpenCloning_backend/actions/workflows/ci.yml)
+[![codecov](https://codecov.io/gh/manulera/OpenCloning_backend/graph/badge.svg?token=CFIB2H6WMO)](https://codecov.io/gh/manulera/OpenCloning_backend)
+
+# OpenCloning Backend API
+
+This API is part of a bigger application, before going further, please go to the [main project readme](https://github.com/manulera/OpenCloning), where you can find an introduction.
+
+This python API is built with [FastAPI](https://fastapi.tiangolo.com/) and is for *in silico* cloning.
+
+## Summary
+
+Read [main project readme](https://github.com/manulera/OpenCloning) first.
+
+This API provides a series of entry points. The API documentation can be accessed [here](https://OpenCloning.api.genestorian.org/docs). You can use the documentation page to try some request directly on the browser. Otherwise, the API is open for you to make requests from a python script or command line at: [https://opencloning.api.genestorian.org/](https://opencloning.api.genestorian.org/).
+
+## Getting started
+
+If you want to quickly set up a local instance of the frontend and backend of the application, check [getting started in 5 minutes](https://github.com/manulera/OpenCloning#timer_clock-getting-started-in-5-minutes) in the main repository.
+
+### Running locally
+
+You can install this as a python package:
+
+```bash
+# Create a virtual environment
+python -m venv .venv
+# Activate the virtual environment
+source .venv/bin/activate
+# Install the package from github (will be in pypi at some point)
+pip install opencloning
+# Run the API (uvicorn should be installed in the virtual environment)
+uvicorn opencloning.main:app
+```
+
+### Running locally if you want to contribute
+
+For the management of the dependencies `poetry` is used, if you don't have it, visit https://python-poetry.org/.
+
+In the project directory:
+
+```bash
+# This should install the dependencies and create a virtual environment
+poetry install
+
+# Install the pre-commit hooks
+pre-commit install
+
+# Activate the virtual environment
+poetry shell
+
+```
+
+The virtual environment is installed in the project folder. This is convenient if you are using an IDE for development. For settings of vscode see the folder `.vscode`.
+
+Now you should be able to run the api by running:
+
+```bash
+# The --reload argument will reload the API if you make changes to the code
+uvicorn opencloning.main:app --reload --reload-exclude='.venv'
+```
+
+Then you should be able to open the API docs at [http://127.0.0.1:8000/docs](http://127.0.0.1:8000/docs) to know that your API is working.
+
+### Running locally with docker :whale:
+
+If you want to serve the full site (backend and frontend) with docker, check [getting started in 5 minutes](https://github.com/manulera/OpenCloning#timer_clock-getting-started-in-5-minutes) in the main repository.
+
+If you want to serve only the backend from a docker container, an image is available at [manulera/opencloningbackend](https://hub.docker.com/r/manulera/opencloningbackend). The image is built from the Dockerfile in the root of this repository and exposes the port 3000. To run it:
+
+```bash
+docker build -t manulera/opencloningbackend .
+docker run -d --name backendcontainer -p 8000:8000 manulera/opencloningbackend
+
+```
+
+If you don't want to download the repository and build the image, you can fetch the latest image from dockerhub (same image that is used in [https://opencloning.api.genestorian.org/](https://opencloning.api.genestorian.org/))
+
+```bash
+docker pull manulera/opencloningbackend
+docker run -d --name backendcontainer -p 8000:8000 manulera/opencloningbackend
+```
+
+The api will be running at `http://localhost:8000`, so you should be able to access the docs at [http://localhost:8000/docs](http://localhost:8000/docs).
+
+### Connecting to the frontend
+
+If you want to receive requests from the [frontend](https://github.com/manulera/OpenCloning_frontend), or from another web application you may have to include the url of the frontend application in the CORS exceptions. By default, if you run the dev server with `uvicorn opencloning.main:app --reload --reload-exclude='.venv'`, the backend will accept requests coming from `http://localhost:3000`, which is the default address of the frontend dev server (ran with `yarn start`).
+
+If you want to change the allowed origins, you can do so via env variables (comma-separated). e.g.:
+
+```
+ALLOWED_ORIGINS=http://localhost:3000,http://localhost:3001 uvicorn opencloning.main:app --reload --reload-exclude='.venv'
+```
+
+Similarly, the frontend should be configured to send requests to the backend address, [see here](https://github.com/manulera/OpenCloning_frontend#connecting-to-the-backend).
+
+#### Serving the frontend from the backend
+
+You may prefer to handle everything from a single server. You can do so by:
+* Build the [frontend](https://github.com/manulera/OpenCloning_frontend) with `yarn build`.
+* Copy the folder `build` from the frontend to the root directory of the backend, and rename it to `frontend`.
+* Set the environment variable `SERVE_FRONTEND=1` when running the backend. By default this will remove all allowed origins, but you can still set them with `ALLOWED_ORIGINS`.
+* Set the value of `backendUrl` in `frontend/config.js` to `/`.
+* Now, when you go to the root of the backend (e.g. `http://localhost:8000`), you should receive the frontend instead of the greeting page of the API.
+
+You can see how this is done in this [docker image](https://github.com/manulera/OpenCloning/blob/master/Dockerfile) and [docker-compose file](https://github.com/manulera/OpenCloning/blob/master/docker-compose.yml).
+
+## Contributing :hammer_and_wrench:
+
+Check [contribution guidelines in the main repository](https://github.com/manulera/OpenCloning/blob/master/CONTRIBUTING.md) for general guidelines.
+
+For more specific tasks:
+* Creating a new type of source: follow the [new source issue template](.github/ISSUE_TEMPLATE/new-source.md). You can create an issue like that [here](https://github.com/manulera/OpenCloning_backend/issues/new?assignees=&labels=new-source&projects=&template=new-source.md&title=New+source%3A+%3Cname-of-source%3E).
+
+## Running the tests locally
+
+```
+pytest -v -ks
+```
+
+## Notes
+
+### Ping a particular library version from github:
+
+```
+poetry add git+https://github.com/BjornFJohansson/pydna#4fd760d075f77cceeb27969e017e04b42f6d0aa3
+```
+
+### Generating API stubs
+
+For the frontend, it may be useful to produce stubs (I use them for writing the tests). See how this is implemented
+by looking at the `RecordStubRoute` class in `api_config_utils.py`. To run the dev server and record stubs:
+
+```bash
+RECORD_STUBS=1 uvicorn opencloning.main:app --reload --reload-exclude='.venv'
+```
+
+This will record the stubs (requests and responses) in the `stubs` folder.
+

opencloning-0.1.0/README.md

@@ -0,0 +1,138 @@
+[![Python tests](https://github.com/manulera/OpenCloning_backend/actions/workflows/ci.yml/badge.svg)](https://github.com/manulera/OpenCloning_backend/actions/workflows/ci.yml)
+[![codecov](https://codecov.io/gh/manulera/OpenCloning_backend/graph/badge.svg?token=CFIB2H6WMO)](https://codecov.io/gh/manulera/OpenCloning_backend)
+
+# OpenCloning Backend API
+
+This API is part of a bigger application, before going further, please go to the [main project readme](https://github.com/manulera/OpenCloning), where you can find an introduction.
+
+This python API is built with [FastAPI](https://fastapi.tiangolo.com/) and is for *in silico* cloning.
+
+## Summary
+
+Read [main project readme](https://github.com/manulera/OpenCloning) first.
+
+This API provides a series of entry points. The API documentation can be accessed [here](https://OpenCloning.api.genestorian.org/docs). You can use the documentation page to try some request directly on the browser. Otherwise, the API is open for you to make requests from a python script or command line at: [https://opencloning.api.genestorian.org/](https://opencloning.api.genestorian.org/).
+
+## Getting started
+
+If you want to quickly set up a local instance of the frontend and backend of the application, check [getting started in 5 minutes](https://github.com/manulera/OpenCloning#timer_clock-getting-started-in-5-minutes) in the main repository.
+
+### Running locally
+
+You can install this as a python package:
+
+```bash
+# Create a virtual environment
+python -m venv .venv
+# Activate the virtual environment
+source .venv/bin/activate
+# Install the package from github (will be in pypi at some point)
+pip install opencloning
+# Run the API (uvicorn should be installed in the virtual environment)
+uvicorn opencloning.main:app
+```
+
+### Running locally if you want to contribute
+
+For the management of the dependencies `poetry` is used, if you don't have it, visit https://python-poetry.org/.
+
+In the project directory:
+
+```bash
+# This should install the dependencies and create a virtual environment
+poetry install
+
+# Install the pre-commit hooks
+pre-commit install
+
+# Activate the virtual environment
+poetry shell
+
+```
+
+The virtual environment is installed in the project folder. This is convenient if you are using an IDE for development. For settings of vscode see the folder `.vscode`.
+
+Now you should be able to run the api by running:
+
+```bash
+# The --reload argument will reload the API if you make changes to the code
+uvicorn opencloning.main:app --reload --reload-exclude='.venv'
+```
+
+Then you should be able to open the API docs at [http://127.0.0.1:8000/docs](http://127.0.0.1:8000/docs) to know that your API is working.
+
+### Running locally with docker :whale:
+
+If you want to serve the full site (backend and frontend) with docker, check [getting started in 5 minutes](https://github.com/manulera/OpenCloning#timer_clock-getting-started-in-5-minutes) in the main repository.
+
+If you want to serve only the backend from a docker container, an image is available at [manulera/opencloningbackend](https://hub.docker.com/r/manulera/opencloningbackend). The image is built from the Dockerfile in the root of this repository and exposes the port 3000. To run it:
+
+```bash
+docker build -t manulera/opencloningbackend .
+docker run -d --name backendcontainer -p 8000:8000 manulera/opencloningbackend
+
+```
+
+If you don't want to download the repository and build the image, you can fetch the latest image from dockerhub (same image that is used in [https://opencloning.api.genestorian.org/](https://opencloning.api.genestorian.org/))
+
+```bash
+docker pull manulera/opencloningbackend
+docker run -d --name backendcontainer -p 8000:8000 manulera/opencloningbackend
+```
+
+The api will be running at `http://localhost:8000`, so you should be able to access the docs at [http://localhost:8000/docs](http://localhost:8000/docs).
+
+### Connecting to the frontend
+
+If you want to receive requests from the [frontend](https://github.com/manulera/OpenCloning_frontend), or from another web application you may have to include the url of the frontend application in the CORS exceptions. By default, if you run the dev server with `uvicorn opencloning.main:app --reload --reload-exclude='.venv'`, the backend will accept requests coming from `http://localhost:3000`, which is the default address of the frontend dev server (ran with `yarn start`).
+
+If you want to change the allowed origins, you can do so via env variables (comma-separated). e.g.:
+
+```
+ALLOWED_ORIGINS=http://localhost:3000,http://localhost:3001 uvicorn opencloning.main:app --reload --reload-exclude='.venv'
+```
+
+Similarly, the frontend should be configured to send requests to the backend address, [see here](https://github.com/manulera/OpenCloning_frontend#connecting-to-the-backend).
+
+#### Serving the frontend from the backend
+
+You may prefer to handle everything from a single server. You can do so by:
+* Build the [frontend](https://github.com/manulera/OpenCloning_frontend) with `yarn build`.
+* Copy the folder `build` from the frontend to the root directory of the backend, and rename it to `frontend`.
+* Set the environment variable `SERVE_FRONTEND=1` when running the backend. By default this will remove all allowed origins, but you can still set them with `ALLOWED_ORIGINS`.
+* Set the value of `backendUrl` in `frontend/config.js` to `/`.
+* Now, when you go to the root of the backend (e.g. `http://localhost:8000`), you should receive the frontend instead of the greeting page of the API.
+
+You can see how this is done in this [docker image](https://github.com/manulera/OpenCloning/blob/master/Dockerfile) and [docker-compose file](https://github.com/manulera/OpenCloning/blob/master/docker-compose.yml).
+
+## Contributing :hammer_and_wrench:
+
+Check [contribution guidelines in the main repository](https://github.com/manulera/OpenCloning/blob/master/CONTRIBUTING.md) for general guidelines.
+
+For more specific tasks:
+* Creating a new type of source: follow the [new source issue template](.github/ISSUE_TEMPLATE/new-source.md). You can create an issue like that [here](https://github.com/manulera/OpenCloning_backend/issues/new?assignees=&labels=new-source&projects=&template=new-source.md&title=New+source%3A+%3Cname-of-source%3E).
+
+## Running the tests locally
+
+```
+pytest -v -ks
+```
+
+## Notes
+
+### Ping a particular library version from github:
+
+```
+poetry add git+https://github.com/BjornFJohansson/pydna#4fd760d075f77cceeb27969e017e04b42f6d0aa3
+```
+
+### Generating API stubs
+
+For the frontend, it may be useful to produce stubs (I use them for writing the tests). See how this is implemented
+by looking at the `RecordStubRoute` class in `api_config_utils.py`. To run the dev server and record stubs:
+
+```bash
+RECORD_STUBS=1 uvicorn opencloning.main:app --reload --reload-exclude='.venv'
+```
+
+This will record the stubs (requests and responses) in the `stubs` folder.

opencloning-0.1.0/pyproject.toml

@@ -0,0 +1,52 @@
+[tool.poetry]
+authors = ["Manuel Lera-Ramirez <manulera14@gmail.com>"]
+description = "Backend of OpenCloning, a web application to generate molecular cloning strategies in json format, and share them with others."
+license = "MIT"
+name = "opencloning"
+version = "0.1.0"
+package-mode = true
+readme = "README.md"
+repository = "https://github.com/manulera/OpenCloning_backend"
+
+[tool.poetry.dependencies]
+beautifulsoup4 = "^4.11.1"
+fastapi = "*"
+httpx = "^0.25.0"
+python = "^3.11"
+python-multipart = "*"
+uvicorn = "*"
+pydna = "^5.3"
+requests = "^2.31.0"
+regex = "^2023.10.3"
+pydantic = "^2.7.1"
+pandas = "^2.2.3"
+openpyxl = "^3.1.5"
+pyyaml = "^6.0.2"
+opencloning-linkml = "^0.2a0"
+
+[tool.poetry.group.dev.dependencies]
+autopep8 = "^2.0.4"
+flake8-bugbear = "^24.2.6"
+black = "^24.2.0"
+pre-commit = "^3.6.2"
+watchfiles = "^0.21.0"
+
+[tool.poetry.group.test.dependencies]
+pytest = "8.3.4"
+pre-commit = "^3.6.2"
+pytest-cov = "^4.1.0"
+pytest-rerunfailures = "^14.0"
+respx = "^0.21.1"
+
+
+[tool.poetry.group.ipython.dependencies]
+ipython = "^8.20.0"
+ipykernel = "^6.28.0"
+
+[build-system]
+build-backend = "poetry.core.masonry.api"
+requires = ["poetry-core>=1.0.0"]
+
+[tool.black]
+skip-string-normalization = true
+line-length = 119

opencloning-0.1.0/src/opencloning/api_config_utils.py

@@ -0,0 +1,97 @@
+import json
+import datetime
+from fastapi.exceptions import RequestValidationError, HTTPException
+from typing import Callable
+from fastapi.routing import APIRoute
+from fastapi import Request, Response
+from fastapi.responses import JSONResponse
+import os
+import glob
+from fastapi.middleware.cors import CORSMiddleware
+
+
+class RecordStubRoute(APIRoute):
+    """Subclass of APIRoute that stores the request and response of a route in the folder `stubs`"""
+
+    def get_route_handler(self) -> Callable:
+        original_route_handler = super().get_route_handler()
+
+        async def custom_route_handler(request: Request) -> Response:
+            if request.method != 'POST':
+                return await original_route_handler(request)
+            formatted_request = {
+                'path': request.url.path,
+                'method': request.method,
+                'body': await request.json(),
+                'headers': dict(request.headers),
+            }
+            try:
+                response: JSONResponse = await original_route_handler(request)
+            except RequestValidationError as exc:
+                errors = exc.errors()
+                for e in errors:
+                    if 'ctx' in e:
+                        # This is a ValueError Object that cannot be serialized
+                        del e['ctx']
+                response = JSONResponse(content=errors, status_code=422)
+            except HTTPException as exc:
+                print('>>exception', exc)
+                detail = {'detail': exc.detail}
+                response = JSONResponse(content=detail, status_code=exc.status_code)
+
+            if type(response) is JSONResponse:
+                formatted_response = {
+                    'statusCode': response.status_code,
+                    'body': json.loads(response.body),
+                    'headers': dict(response.headers),
+                }
+
+                formatted_time = datetime.datetime.now().strftime('%Y_%m_%d-%H_%M_%S')
+                stub_folder = f'stubs{formatted_request["path"]}/{formatted_time}'
+                existing_folders = glob.glob(f'{stub_folder}_*')
+                stub_folder = f'{stub_folder}_{len(existing_folders)}'
+                if not os.path.exists(stub_folder):
+                    os.makedirs(stub_folder)
+                with open(f'{stub_folder}/request.json', 'w') as f:
+                    json.dump(formatted_request, f, indent=4)
+                with open(f'{stub_folder}/response.json', 'w') as f:
+                    json.dump(formatted_response, f, indent=4)
+                with open(f'{stub_folder}/response_body.json', 'w') as f:
+                    json.dump(formatted_response['body'], f, indent=4)
+
+            print(9 * ' ', '> stub written to', stub_folder)
+            return response
+
+        return custom_route_handler
+
+
+# Workaround for internal server errors: https://github.com/tiangolo/fastapi/discussions/7847#discussioncomment-5144709
+async def custom_http_exception_handler(request: Request, exc: Exception, app, allow_origins):
+
+    response = JSONResponse(content={'message': 'internal server error'}, status_code=500)
+
+    origin = request.headers.get('origin')
+
+    if origin:
+        # Have the middleware do the heavy lifting for us to parse
+        # all the config, then update our response headers
+        cors = CORSMiddleware(
+            app=app, allow_origins=allow_origins, allow_credentials=True, allow_methods=['*'], allow_headers=['*']
+        )
+        # Logic directly from Starlette's CORSMiddleware:
+        # https://github.com/encode/starlette/blob/master/starlette/middleware/cors.py#L152
+
+        response.headers.update(cors.simple_headers)
+        has_cookie = 'cookie' in request.headers
+        # If request includes any cookie headers, then we must respond
+        # with the specific origin instead of '*'.
+        if cors.allow_all_origins and has_cookie:
+            response.headers['Access-Control-Allow-Origin'] = origin
+
+        # If we only allow specific origins, then we have to mirror back
+        # the Origin header in the response.
+        elif not cors.allow_all_origins and cors.is_allowed_origin(origin=origin):
+            response.headers['Access-Control-Allow-Origin'] = origin
+            response.headers.add_vary_header('Origin')
+
+    return response

opencloning-0.1.0/src/opencloning/app_settings.py

@@ -0,0 +1,50 @@
+"""
+This module contains the settings for the app that can be set via environment variables.
+"""
+
+import os
+from pydantic import BaseModel
+
+
+def parse_bool(value: str) -> bool:
+    return value in {'1', 'TRUE', 'true', 'True'}
+
+
+# API settings ===============================================
+SERVE_FRONTEND = parse_bool(os.environ['SERVE_FRONTEND']) if 'SERVE_FRONTEND' in os.environ else False
+BATCH_CLONING = parse_bool(os.environ['BATCH_CLONING']) if 'BATCH_CLONING' in os.environ else True
+RECORD_STUBS = parse_bool(os.environ['RECORD_STUBS']) if 'RECORD_STUBS' in os.environ else False
+ALLOWED_ORIGINS = ['http://localhost:3000', 'http://localhost:5173']
+if os.environ.get('ALLOWED_ORIGINS') is not None:
+    # Remove trailing slash from each origin if ends with one
+    ALLOWED_ORIGINS = [origin.rstrip('/') for origin in os.environ['ALLOWED_ORIGINS'].split(',')]
+
+
+# External services settings =================================
+NCBI_API_KEY = os.environ.get('NCBI_API_KEY')
+PLANNOTATE_URL = os.environ['PLANNOTATE_URL'] if 'PLANNOTATE_URL' in os.environ else None
+PLANNOTATE_TIMEOUT = int(os.environ['PLANNOTATE_TIMEOUT']) if 'PLANNOTATE_TIMEOUT' in os.environ else 20
+# Handle trailing slash:
+if PLANNOTATE_URL is not None and not PLANNOTATE_URL.endswith('/'):
+    PLANNOTATE_URL += '/'
+
+
+class Settings(BaseModel):
+    SERVE_FRONTEND: bool
+    BATCH_CLONING: bool
+    RECORD_STUBS: bool
+    NCBI_API_KEY: str | None
+    ALLOWED_ORIGINS: list[str]
+    PLANNOTATE_URL: str | None
+    PLANNOTATE_TIMEOUT: int
+
+
+settings = Settings(
+    SERVE_FRONTEND=SERVE_FRONTEND,
+    BATCH_CLONING=BATCH_CLONING,
+    RECORD_STUBS=RECORD_STUBS,
+    NCBI_API_KEY=NCBI_API_KEY,
+    ALLOWED_ORIGINS=ALLOWED_ORIGINS,
+    PLANNOTATE_URL=PLANNOTATE_URL,
+    PLANNOTATE_TIMEOUT=PLANNOTATE_TIMEOUT,
+)