refscan 0.4.3__tar.gz → 0.5.0__tar.gz

refscan-0.4.3/README.md → refscan-0.5.0/PKG-INFO

@@ -1,3 +1,18 @@
+Metadata-Version: 2.3
+Name: refscan
+Version: 0.5.0
+Summary: Command-line program that scans the NMDC MongoDB database for referential integrity violations
+Keywords: mongodb,mongo,relationships,reference,database,data,referential integrity,scan
+Requires-Dist: linkml-runtime>=1.12.1,<2
+Requires-Dist: pymongo>=4.18.3,<5
+Requires-Dist: rich>=13.7.1,<14
+Requires-Dist: typer>=0.29.1,<0.30
+Requires-Python: >=3.10, <4.0
+Project-URL: Documentation, https://github.com/microbiomedata/refscan
+Project-URL: Homepage, https://github.com/microbiomedata/refscan
+Project-URL: Repository, https://github.com/microbiomedata/refscan
+Description-Content-Type: text/markdown
+
 # refscan
 
 `refscan` is a command-line tool people can use to **scan** the [NMDC](https://microbiomedata.org/) MongoDB database
@@ -59,7 +74,9 @@ whose definition `refscan` then uses to determine _which fields_ of that documen
 
 ## Usage
 
-### Install
+### Command-line application
+
+#### Install
 
 Assuming you have `pipx` installed, you can install the tool by running the following command:
 
@@ -72,7 +89,7 @@ pipx install refscan
 > Python scripts that are hosted on PyPI.
 > You can [install `pipx`](https://pipx.pypa.io/stable/installation/) by running `$ python -m pip install pipx`.
 
-### Run
+#### Run
 
 Once installed, you can display the tool's `--help` snippet by running:
 
@@ -99,7 +116,7 @@ At the time of this writing, the tool's `--help` snippet is:
 
 Each command has its own `--help` snippet.
 
-#### The `scan` command
+##### The `scan` command
 
 At the time of this writing, the `--help` snippet for the `scan` command is:
 
@@ -148,7 +165,7 @@ At the time of this writing, the `--help` snippet for the `scan` command is:
 
 <!-- Note: The above snippet was captured from a terminal window whose width was 90 characters. -->
 
-##### The MongoDB connection string (`--mongo-uri`)
+###### The MongoDB connection string (`--mongo-uri`)
 
 As documented in the `--help` snippet above, you can provide the MongoDB connection string to the tool via either
 (a) the `--mongo-uri` option; or (b) an environment variable named `MONGO_URI`. The latter can come in handy
@@ -161,7 +178,7 @@ Here's how you could create that environment variable:
 export MONGO_URI='mongodb://username:password@localhost:27017'
 ```
 
-##### The schema (`--schema`)
+###### The schema (`--schema`)
 
 As documented in the `--help` snippet above, you can provide the path to a YAML-formatted LinkML schema file to the tool
 via the `--schema` option.
@@ -195,7 +212,7 @@ curl -o schema.yaml https://raw.githubusercontent.com/microbiomedata/nmdc-schema
 ---
 </details>
 
-##### Output
+###### Output
 
 While `refscan` is running, it will display console output indicating what it's currently doing.
 
@@ -204,7 +221,7 @@ While `refscan` is running, it will display console output indicating what it's
 Once the scan is complete, the reference report (TSV file) and violation report (TSV file) will be available
 in the current directory (or in custom directories, if any were specified via CLI options).
 
-#### The `graph` command
+##### The `graph` command
 
 At the time of this writing, the `--help` snippet for the `graph` command is:
 
@@ -231,7 +248,7 @@ At the time of this writing, the `--help` snippet for the `graph` command is:
 
 <!-- Note: The above snippet was captured from a terminal window whose width was 90 characters. -->
 
-### Update
+#### Update
 
 You can update the tool to [the latest version available on PyPI](https://pypi.org/project/refscan/) by running:
 
@@ -239,7 +256,7 @@ You can update the tool to [the latest version available on PyPI](https://pypi.o
 pipx upgrade refscan
 ```
 
-### Uninstall
+#### Uninstall
 
 You can uninstall the tool from your computer by running:
 
@@ -249,15 +266,65 @@ pipx uninstall refscan
 
 ### Container-based usage
 
-You can also run `refscan` via a container image hosted by the [GitHub Container Registry](https://github.com/microbiomedata/refscan/pkgs/container/refscan).
+You can also run `refscan` via a container image hosted by the [GitHub Container Registry](https://github.com/microbiomedata/refscan/pkgs/container/refscan); for example:
 
 ```shell
-docker run --rm -it refscan --help
+docker run --rm -it ghcr.io/microbiomedata/refscan:sha-51b1bb9 --help
 ```
 
 > Note: When running `refscan` via a container image, you can reference your host machine via the [special hostname](https://docs.docker.com/desktop/troubleshoot-and-support/faqs/general/#how-do-i-connect-from-a-container-to-a-service-on-the-host), "`host.docker.internal`".
 > 
-> In other words, `$ docker run refscan --mongo-uri mongodb://host.docker.internal:27017` does the same thing as `$ refscan --mongo-uri mongodb://localhost:27017`, except the first command runs `refscan` within a container while the second one runs it directly on your hose machine.
+> In other words, `$ docker run refscan --mongo-uri mongodb://host.docker.internal:27017` does the same thing as `$ refscan --mongo-uri mongodb://localhost:27017`, except the first command runs `refscan` within a container while the second one runs it directly on your host machine.
+
+### Library usage
+
+`refscan` can also be used as a library. This allows application developers to use the various
+functions implemented within `refscan` without having to re-implement them.
+
+#### Installation
+
+```sh
+pip install refscan
+```
+
+#### Example usage
+
+```py
+from linkml_runtime import SchemaView
+from refscan.lib.helpers import (
+    get_collection_name_to_class_names_map,
+    get_collection_names_from_schema,
+    get_names_of_classes_eligible_for_collection,
+)
+
+
+# Instantiate a `SchemaView` bound to the schema.
+schema_view = SchemaView("path/to/schema.yaml")
+
+# Get the names of all collections the schema says can exist in the NMDC database.
+# i.e., "What collections can exist?"
+collection_names = get_collection_names_from_schema(schema_view=schema_view)
+
+# Get the names of the classes whose instances the schema says can be stored in each collection.
+# i.e., "What can exist in each collection?"
+collection_name_to_class_names_map = get_collection_name_to_class_names_map(
+    schema_view=schema_view
+)
+
+# Get the names of the classes whose instances the schema says can be stored in the "study_set" collection.
+# i.e., "What can exist in the 'study_set' collection?"
+class_names = get_names_of_classes_eligible_for_collection(
+    schema_view=schema_view,
+    collection_name="study_set",
+)
+```
+
+#### Note to NMDC developers
+
+If you think a given function currently implemented within the `refscan` repository would "fit better"
+in another NMDC repository, please create an issue or otherwise bring it to the attention of a
+`refscan` maintainer. For example, functions that depend upon only a `SchemaView` instance
+(and not upon a MongoDB connection) may "fit better" in the `nmdc-schema` repository.
 
 ## Development
 

refscan-0.4.3/PKG-INFO → refscan-0.5.0/README.md

@@ -1,18 +1,3 @@
-Metadata-Version: 2.3
-Name: refscan
-Version: 0.4.3
-Summary: Command-line program that scans the NMDC MongoDB database for referential integrity violations
-Keywords: mongodb,mongo,relationships,reference,database,data,referential integrity,scan
-Requires-Dist: linkml-runtime>=1.7.5,<2
-Requires-Dist: pymongo>=4.7.1,<5
-Requires-Dist: rich>=13.7.1,<14
-Requires-Dist: typer>=0.15.1,<0.16
-Requires-Python: >=3.9, <4.0
-Project-URL: Documentation, https://github.com/microbiomedata/refscan
-Project-URL: Homepage, https://github.com/microbiomedata/refscan
-Project-URL: Repository, https://github.com/microbiomedata/refscan
-Description-Content-Type: text/markdown
-
 # refscan
 
 `refscan` is a command-line tool people can use to **scan** the [NMDC](https://microbiomedata.org/) MongoDB database
@@ -74,7 +59,9 @@ whose definition `refscan` then uses to determine _which fields_ of that documen
 
 ## Usage
 
-### Install
+### Command-line application
+
+#### Install
 
 Assuming you have `pipx` installed, you can install the tool by running the following command:
 
@@ -87,7 +74,7 @@ pipx install refscan
 > Python scripts that are hosted on PyPI.
 > You can [install `pipx`](https://pipx.pypa.io/stable/installation/) by running `$ python -m pip install pipx`.
 
-### Run
+#### Run
 
 Once installed, you can display the tool's `--help` snippet by running:
 
@@ -114,7 +101,7 @@ At the time of this writing, the tool's `--help` snippet is:
 
 Each command has its own `--help` snippet.
 
-#### The `scan` command
+##### The `scan` command
 
 At the time of this writing, the `--help` snippet for the `scan` command is:
 
@@ -163,7 +150,7 @@ At the time of this writing, the `--help` snippet for the `scan` command is:
 
 <!-- Note: The above snippet was captured from a terminal window whose width was 90 characters. -->
 
-##### The MongoDB connection string (`--mongo-uri`)
+###### The MongoDB connection string (`--mongo-uri`)
 
 As documented in the `--help` snippet above, you can provide the MongoDB connection string to the tool via either
 (a) the `--mongo-uri` option; or (b) an environment variable named `MONGO_URI`. The latter can come in handy
@@ -176,7 +163,7 @@ Here's how you could create that environment variable:
 export MONGO_URI='mongodb://username:password@localhost:27017'
 ```
 
-##### The schema (`--schema`)
+###### The schema (`--schema`)
 
 As documented in the `--help` snippet above, you can provide the path to a YAML-formatted LinkML schema file to the tool
 via the `--schema` option.
@@ -210,7 +197,7 @@ curl -o schema.yaml https://raw.githubusercontent.com/microbiomedata/nmdc-schema
 ---
 </details>
 
-##### Output
+###### Output
 
 While `refscan` is running, it will display console output indicating what it's currently doing.
 
@@ -219,7 +206,7 @@ While `refscan` is running, it will display console output indicating what it's
 Once the scan is complete, the reference report (TSV file) and violation report (TSV file) will be available
 in the current directory (or in custom directories, if any were specified via CLI options).
 
-#### The `graph` command
+##### The `graph` command
 
 At the time of this writing, the `--help` snippet for the `graph` command is:
 
@@ -246,7 +233,7 @@ At the time of this writing, the `--help` snippet for the `graph` command is:
 
 <!-- Note: The above snippet was captured from a terminal window whose width was 90 characters. -->
 
-### Update
+#### Update
 
 You can update the tool to [the latest version available on PyPI](https://pypi.org/project/refscan/) by running:
 
@@ -254,7 +241,7 @@ You can update the tool to [the latest version available on PyPI](https://pypi.o
 pipx upgrade refscan
 ```
 
-### Uninstall
+#### Uninstall
 
 You can uninstall the tool from your computer by running:
 
@@ -264,15 +251,65 @@ pipx uninstall refscan
 
 ### Container-based usage
 
-You can also run `refscan` via a container image hosted by the [GitHub Container Registry](https://github.com/microbiomedata/refscan/pkgs/container/refscan).
+You can also run `refscan` via a container image hosted by the [GitHub Container Registry](https://github.com/microbiomedata/refscan/pkgs/container/refscan); for example:
 
 ```shell
-docker run --rm -it refscan --help
+docker run --rm -it ghcr.io/microbiomedata/refscan:sha-51b1bb9 --help
 ```
 
 > Note: When running `refscan` via a container image, you can reference your host machine via the [special hostname](https://docs.docker.com/desktop/troubleshoot-and-support/faqs/general/#how-do-i-connect-from-a-container-to-a-service-on-the-host), "`host.docker.internal`".
 > 
-> In other words, `$ docker run refscan --mongo-uri mongodb://host.docker.internal:27017` does the same thing as `$ refscan --mongo-uri mongodb://localhost:27017`, except the first command runs `refscan` within a container while the second one runs it directly on your hose machine.
+> In other words, `$ docker run refscan --mongo-uri mongodb://host.docker.internal:27017` does the same thing as `$ refscan --mongo-uri mongodb://localhost:27017`, except the first command runs `refscan` within a container while the second one runs it directly on your host machine.
+
+### Library usage
+
+`refscan` can also be used as a library. This allows application developers to use the various
+functions implemented within `refscan` without having to re-implement them.
+
+#### Installation
+
+```sh
+pip install refscan
+```
+
+#### Example usage
+
+```py
+from linkml_runtime import SchemaView
+from refscan.lib.helpers import (
+    get_collection_name_to_class_names_map,
+    get_collection_names_from_schema,
+    get_names_of_classes_eligible_for_collection,
+)
+
+
+# Instantiate a `SchemaView` bound to the schema.
+schema_view = SchemaView("path/to/schema.yaml")
+
+# Get the names of all collections the schema says can exist in the NMDC database.
+# i.e., "What collections can exist?"
+collection_names = get_collection_names_from_schema(schema_view=schema_view)
+
+# Get the names of the classes whose instances the schema says can be stored in each collection.
+# i.e., "What can exist in each collection?"
+collection_name_to_class_names_map = get_collection_name_to_class_names_map(
+    schema_view=schema_view
+)
+
+# Get the names of the classes whose instances the schema says can be stored in the "study_set" collection.
+# i.e., "What can exist in the 'study_set' collection?"
+class_names = get_names_of_classes_eligible_for_collection(
+    schema_view=schema_view,
+    collection_name="study_set",
+)
+```
+
+#### Note to NMDC developers
+
+If you think a given function currently implemented within the `refscan` repository would "fit better"
+in another NMDC repository, please create an issue or otherwise bring it to the attention of a
+`refscan` maintainer. For example, functions that depend upon only a `SchemaView` instance
+(and not upon a MongoDB connection) may "fit better" in the `nmdc-schema` repository.
 
 ## Development
 

{refscan-0.4.3 → refscan-0.5.0}/pyproject.toml

@@ -1,9 +1,9 @@
 [project]
 name = "refscan"
-version = "0.4.3"
+version = "0.5.0"
 description = "Command-line program that scans the NMDC MongoDB database for referential integrity violations"
 authors = []
-requires-python = ">= 3.9, < 4.0"
+requires-python = ">= 3.10, < 4.0"
 readme = "README.md"
 keywords = [
     "mongodb",
@@ -18,16 +18,16 @@ keywords = [
 dependencies = [
     # We use `linkml-runtime` to facilitate using `nmdc-schema`.
     # Docs: https://linkml.io/linkml/developers/schemaview.html#linkml_runtime.utils.schemaview.SchemaView
-    "linkml-runtime >= 1.7.5, < 2",
+    "linkml-runtime >= 1.12.1, < 2",
     # We use `pymongo` to communicate with MongoDB servers.
     # Docs: (https://pymongo.readthedocs.io/en/stable/
-    "pymongo >= 4.7.1, < 5",
+    "pymongo >= 4.18.3, < 5",
     # We use `rich` to format console output.
     # Docs: https://rich.readthedocs.io/en/stable/index.html
     "rich >= 13.7.1, < 14",
     # We use `typer` as a CLI framework.
     # Docs: https://typer.tiangolo.com/
-    "typer >= 0.15.1, < 0.16",
+    "typer >= 0.29.1, < 0.30",
 ]
 
 [project.urls]
@@ -47,7 +47,7 @@ dev = [
     # We use `pytest` to run tests.
     # Usage: $ uv run pytest
     # Docs: https://docs.pytest.org/en/stable/contents.html
-    "pytest >= 8.2.2, < 9",
+    "pytest>=8.2.2,<10",
     "pytest-cov >= 5.0.0, < 6",
     # We use `mongomock` to test MongoDB interaction without involving a real MongoDB instance.
     # Docs: https://github.com/mongomock/mongomock

refscan-0.5.0/refscan/lib/ReferringDocumentDescriptor.py

@@ -0,0 +1,24 @@
+from typing import TypedDict
+
+from bson import ObjectId
+
+
+class ReferringDocumentDescriptor(TypedDict):
+    """
+    A custom type that describes the items of the list returned by the function,
+    `refscan.scanner.identify_referring_documents`.
+
+    Docs: https://typing.python.org/en/latest/spec/typeddict.html
+    """
+
+    source_collection_name: str
+    """The name of the collection in which the referring document resides."""
+
+    source_class_name: str
+    """The name of the class of which the referring document represent an instance."""
+
+    source_document_object_id: ObjectId
+    """The value in the referring document's `_id` field."""
+
+    source_document_id: str
+    """The value in the referring document's `id` field."""

{refscan-0.4.3 → refscan-0.5.0}/refscan/scanner.py

@@ -14,6 +14,7 @@ from refscan.lib.helpers import (
     init_progress_bar,
 )
 from refscan.lib.ReferenceList import ReferenceList
+from refscan.lib.ReferringDocumentDescriptor import ReferringDocumentDescriptor
 from refscan.lib.Violation import Violation
 from refscan.lib.ViolationList import ViolationList
 from refscan.lib.constants import console as default_console
@@ -25,7 +26,7 @@ def identify_referring_documents(
     references: ReferenceList,
     finder: Finder,
     client_session: Optional[ClientSession] = None,
-) -> List[dict]:
+) -> List[ReferringDocumentDescriptor]:
     r"""
     Identifies documents that reference the specified one.
 
@@ -50,7 +51,7 @@ def identify_referring_documents(
     """
 
     # Initialize a list of descriptors of documents that reference the specified document.
-    referring_document_descriptors: List[Dict] = []
+    referring_document_descriptors: List[ReferringDocumentDescriptor] = []
 
     # If the specified document does not have an `id` field, we know it cannot be referenced.
     if "id" not in document:
@@ -111,7 +112,11 @@ def identify_referring_documents(
                 schema_view=schema_view,
                 class_uri=referring_document["type"],
             )
-            document_descriptor = dict(
+            if source_class_name is None:
+                raise ValueError(
+                    f"Failed to translate class_uri '{referring_document['type']}' into schema class name."
+                )
+            document_descriptor = ReferringDocumentDescriptor(
                 source_collection_name=collection_name,
                 source_class_name=source_class_name,
                 source_document_object_id=referring_document["_id"],
@@ -332,7 +337,9 @@ def scan(
             # Reference: https://rich.readthedocs.io/en/stable/reference/console.html#rich.console.Console.is_terminal
             #
             if not console.is_terminal:
-                console.print(f"Scanning collection '{source_collection_name}' ({num_relevant_documents} documents).")
+                console.print(
+                    f"Scanning collection '{source_collection_name}' ({num_relevant_documents} relevant documents)."
+                )
 
             # Advance the progress bar by 0 (this makes it so that, even if there are 0 relevant documents, the progress
             # bar does not continue incrementing its "elapsed time" even after a subsequent task has begun).