refscan 0.4.1__tar.gz → 0.4.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {refscan-0.4.1 → refscan-0.4.3}/PKG-INFO +15 -2
- {refscan-0.4.1 → refscan-0.4.3}/README.md +14 -1
- {refscan-0.4.1 → refscan-0.4.3}/pyproject.toml +1 -1
- {refscan-0.4.1 → refscan-0.4.3}/refscan/scanner.py +12 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/__init__.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/cli/cli.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/cli/graph.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/cli/scan.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/cli/version.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/grapher.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/lib/Finder.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/lib/Reference.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/lib/ReferenceList.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/lib/Violation.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/lib/ViolationList.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/lib/constants.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/lib/helpers.py +0 -0
- {refscan-0.4.1 → refscan-0.4.3}/refscan/templates/graph.template.html +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.3
|
|
2
2
|
Name: refscan
|
|
3
|
-
Version: 0.4.
|
|
3
|
+
Version: 0.4.3
|
|
4
4
|
Summary: Command-line program that scans the NMDC MongoDB database for referential integrity violations
|
|
5
5
|
Keywords: mongodb,mongo,relationships,reference,database,data,referential integrity,scan
|
|
6
6
|
Requires-Dist: linkml-runtime>=1.7.5,<2
|
|
@@ -36,7 +36,7 @@ graph LR
|
|
|
36
36
|
classDef dashed_border stroke-dasharray: 5 5
|
|
37
37
|
```
|
|
38
38
|
|
|
39
|
-
In addition to using refscan to scan the NMDC MongoDB database for referential integrity violations,
|
|
39
|
+
In addition to using `refscan` to scan the NMDC MongoDB database for referential integrity violations,
|
|
40
40
|
people can use `refscan` to generate **graphs** (diagrams) depicting which collections' documents (or which classes'
|
|
41
41
|
instances) can contain references to which _other_ collections' documents (or classes' instances) while still being
|
|
42
42
|
schema compliant.
|
|
@@ -60,6 +60,7 @@ Here is a summary of how each of `refscan`'s main functions works under the hood
|
|
|
60
60
|
### Graph
|
|
61
61
|
|
|
62
62
|
`refscan` does this in three stages:
|
|
63
|
+
|
|
63
64
|
1. It uses the LinkML schema to determine where references _can_ exist in a MongoDB database that conforms to the schema.
|
|
64
65
|
2. It formats that list of references into a data structure compatible with [`Cytoscape.js`](https://js.cytoscape.org/).
|
|
65
66
|
3. It outputs an HTML document that uses `Cytoscape.js` to visualize that data structure as a graph.
|
|
@@ -261,6 +262,18 @@ You can uninstall the tool from your computer by running:
|
|
|
261
262
|
pipx uninstall refscan
|
|
262
263
|
```
|
|
263
264
|
|
|
265
|
+
### Container-based usage
|
|
266
|
+
|
|
267
|
+
You can also run `refscan` via a container image hosted by the [GitHub Container Registry](https://github.com/microbiomedata/refscan/pkgs/container/refscan).
|
|
268
|
+
|
|
269
|
+
```shell
|
|
270
|
+
docker run --rm -it refscan --help
|
|
271
|
+
```
|
|
272
|
+
|
|
273
|
+
> Note: When running `refscan` via a container image, you can reference your host machine via the [special hostname](https://docs.docker.com/desktop/troubleshoot-and-support/faqs/general/#how-do-i-connect-from-a-container-to-a-service-on-the-host), "`host.docker.internal`".
|
|
274
|
+
>
|
|
275
|
+
> In other words, `$ docker run refscan --mongo-uri mongodb://host.docker.internal:27017` does the same thing as `$ refscan --mongo-uri mongodb://localhost:27017`, except the first command runs `refscan` within a container while the second one runs it directly on your hose machine.
|
|
276
|
+
|
|
264
277
|
## Development
|
|
265
278
|
|
|
266
279
|
We use [`uv`](https://docs.astral.sh/uv/) to both (a) manage dependencies and (b) build distributable packages that can be published to PyPI.
|
|
@@ -21,7 +21,7 @@ graph LR
|
|
|
21
21
|
classDef dashed_border stroke-dasharray: 5 5
|
|
22
22
|
```
|
|
23
23
|
|
|
24
|
-
In addition to using refscan to scan the NMDC MongoDB database for referential integrity violations,
|
|
24
|
+
In addition to using `refscan` to scan the NMDC MongoDB database for referential integrity violations,
|
|
25
25
|
people can use `refscan` to generate **graphs** (diagrams) depicting which collections' documents (or which classes'
|
|
26
26
|
instances) can contain references to which _other_ collections' documents (or classes' instances) while still being
|
|
27
27
|
schema compliant.
|
|
@@ -45,6 +45,7 @@ Here is a summary of how each of `refscan`'s main functions works under the hood
|
|
|
45
45
|
### Graph
|
|
46
46
|
|
|
47
47
|
`refscan` does this in three stages:
|
|
48
|
+
|
|
48
49
|
1. It uses the LinkML schema to determine where references _can_ exist in a MongoDB database that conforms to the schema.
|
|
49
50
|
2. It formats that list of references into a data structure compatible with [`Cytoscape.js`](https://js.cytoscape.org/).
|
|
50
51
|
3. It outputs an HTML document that uses `Cytoscape.js` to visualize that data structure as a graph.
|
|
@@ -246,6 +247,18 @@ You can uninstall the tool from your computer by running:
|
|
|
246
247
|
pipx uninstall refscan
|
|
247
248
|
```
|
|
248
249
|
|
|
250
|
+
### Container-based usage
|
|
251
|
+
|
|
252
|
+
You can also run `refscan` via a container image hosted by the [GitHub Container Registry](https://github.com/microbiomedata/refscan/pkgs/container/refscan).
|
|
253
|
+
|
|
254
|
+
```shell
|
|
255
|
+
docker run --rm -it refscan --help
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
> Note: When running `refscan` via a container image, you can reference your host machine via the [special hostname](https://docs.docker.com/desktop/troubleshoot-and-support/faqs/general/#how-do-i-connect-from-a-container-to-a-service-on-the-host), "`host.docker.internal`".
|
|
259
|
+
>
|
|
260
|
+
> In other words, `$ docker run refscan --mongo-uri mongodb://host.docker.internal:27017` does the same thing as `$ refscan --mongo-uri mongodb://localhost:27017`, except the first command runs `refscan` within a container while the second one runs it directly on your hose machine.
|
|
261
|
+
|
|
249
262
|
## Development
|
|
250
263
|
|
|
251
264
|
We use [`uv`](https://docs.astral.sh/uv/) to both (a) manage dependencies and (b) build distributable packages that can be published to PyPI.
|
|
@@ -322,6 +322,18 @@ def scan(
|
|
|
322
322
|
remaining_time_label="remaining",
|
|
323
323
|
)
|
|
324
324
|
|
|
325
|
+
# If the console is not connected to a device capable of understanding escape characters
|
|
326
|
+
# (in which case, Rich won't try to display the progress bar), print a message indicating
|
|
327
|
+
# that we are starting to scan this collection.
|
|
328
|
+
#
|
|
329
|
+
# Note: This output can serve as a "sign of life" for a user, when, for example, refscan is
|
|
330
|
+
# running in a Kubernetes CronJob where `tty` is `false` (i.e. `tty: false`).
|
|
331
|
+
#
|
|
332
|
+
# Reference: https://rich.readthedocs.io/en/stable/reference/console.html#rich.console.Console.is_terminal
|
|
333
|
+
#
|
|
334
|
+
if not console.is_terminal:
|
|
335
|
+
console.print(f"Scanning collection '{source_collection_name}' ({num_relevant_documents} documents).")
|
|
336
|
+
|
|
325
337
|
# Advance the progress bar by 0 (this makes it so that, even if there are 0 relevant documents, the progress
|
|
326
338
|
# bar does not continue incrementing its "elapsed time" even after a subsequent task has begun).
|
|
327
339
|
progress.update(task_id, advance=0)
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|