easylink 0.1.13__tar.gz → 0.1.14__tar.gz

{easylink-0.1.13 → easylink-0.1.14}/CHANGELOG.rst

@@ -1,3 +1,7 @@
+**0.1.14 - 5/1/25**
+
+ - Add support for EmbarrassinglyParallelSteps to accept sections (i.e. non-leaf steps)
+
 **0.1.13 - 4/21/25**
 
  - remove graphviz from doc build install

easylink-0.1.14/CONTRIBUTING.rst

@@ -0,0 +1,12 @@
+============
+Contributing
+============
+
+To run or develop easylink tests or documentation, you can install easylink with the ``[dev]`` extra::
+
+    $ git clone git@github.com:ihmeuw/easylink.git # or git clone https://github.com/ihmeuw/easylink.git
+    $ cd easylink
+    $ pip install -e .[dev]
+
+.. todo::
+    Show how to run the tests and build the documentation

{easylink-0.1.13 → easylink-0.1.14}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: easylink
-Version: 0.1.13
+Version: 0.1.14
 Summary: Research repository for the EasyLink ER ecosystem project.
 Home-page: https://github.com/ihmeuw/easylink
 Author: The EasyLink developers
@@ -19,11 +19,14 @@ Requires-Dist: pyarrow
 Requires-Dist: snakemake>=8.0.0
 Requires-Dist: snakemake-interface-executor-plugins<9.0.0
 Requires-Dist: snakemake-executor-plugin-slurm
+Requires-Dist: pandas-stubs
+Requires-Dist: pyarrow-stubs
 Provides-Extra: docs
 Requires-Dist: sphinx<8.2.0; extra == "docs"
 Requires-Dist: sphinx-rtd-theme; extra == "docs"
 Requires-Dist: sphinx-autodoc-typehints; extra == "docs"
 Requires-Dist: sphinx-click; extra == "docs"
+Requires-Dist: sphinx-autobuild; extra == "docs"
 Requires-Dist: typing_extensions; extra == "docs"
 Provides-Extra: test
 Requires-Dist: pytest; extra == "test"
@@ -34,6 +37,7 @@ Requires-Dist: sphinx<8.2.0; extra == "dev"
 Requires-Dist: sphinx-rtd-theme; extra == "dev"
 Requires-Dist: sphinx-autodoc-typehints; extra == "dev"
 Requires-Dist: sphinx-click; extra == "dev"
+Requires-Dist: sphinx-autobuild; extra == "dev"
 Requires-Dist: typing_extensions; extra == "dev"
 Requires-Dist: pytest; extra == "dev"
 Requires-Dist: pytest-cov; extra == "dev"
@@ -71,28 +75,32 @@ Installation
 
 There are a few things to install in order to use this package:
 
-- Install singularity. If this is not already installed on your system, you will 
-  likely need to request it from your system admin. 
-  Refer to https://docs.sylabs.io/guides/4.1/admin-guide/installation.html
+- Install singularity. 
 
+  You may need to request it from your system admin. 
+  Refer to https://docs.sylabs.io/guides/4.1/admin-guide/installation.html. 
+  You can check if you already have singularity installed by running the command ``singularity --version``. For an
+  existing installation, your singularity version number is printed.
 
-- Install graphviz via:
+- Install conda. 
+  
+  We recommend `miniforge <https://github.com/conda-forge/miniforge>`_. You can check if you already
+  have conda installed by running the command ``conda --version``. For an existing installation, a version 
+  will be displayed.
 
-  .. code-block:: console
-
-    $ conda install graphviz
-
-- Install EasyLink.
+- Install easylink, python and graphviz in a conda environment.
 
   Option 1 - Install from PyPI with pip::
 
+    $ conda create --name easylink -c conda-forge python=3.12 graphviz 'gcc<14' -y
+    $ conda activate easylink
     $ pip install easylink
 
   Option 2 - Build from source with pip::
-
-    $ git clone git@github.com:ihmeuw/easylink.git  # or git clone https://github.com/ihmeuw/easylink.git
-    $ cd easylink
-    $ pip install .
+    
+    $ conda create --name easylink -c conda-forge python=3.12 graphviz 'gcc<14' -y
+    $ conda activate easylink
+    $ pip install git+https://github.com/ihmeuw/easylink.git
 
 .. _end_installation:
 

easylink-0.1.14/README.rst

@@ -0,0 +1,53 @@
+========
+EasyLink
+========
+
+EasyLink is a framework that allows users to build and run highly configurable
+entity resolution (ER) pipelines.
+
+.. _python_support:
+
+**Supported Python versions: 3.11, 3.12**
+
+.. _end_python_support:
+
+Installation
+============
+
+.. _installation:
+
+There are a few things to install in order to use this package:
+
+- Install singularity. 
+
+  You may need to request it from your system admin. 
+  Refer to https://docs.sylabs.io/guides/4.1/admin-guide/installation.html. 
+  You can check if you already have singularity installed by running the command ``singularity --version``. For an
+  existing installation, your singularity version number is printed.
+
+- Install conda. 
+  
+  We recommend `miniforge <https://github.com/conda-forge/miniforge>`_. You can check if you already
+  have conda installed by running the command ``conda --version``. For an existing installation, a version 
+  will be displayed.
+
+- Install easylink, python and graphviz in a conda environment.
+
+  Option 1 - Install from PyPI with pip::
+
+    $ conda create --name easylink -c conda-forge python=3.12 graphviz 'gcc<14' -y
+    $ conda activate easylink
+    $ pip install easylink
+
+  Option 2 - Build from source with pip::
+    
+    $ conda create --name easylink -c conda-forge python=3.12 graphviz 'gcc<14' -y
+    $ conda activate easylink
+    $ pip install git+https://github.com/ihmeuw/easylink.git
+
+.. _end_installation:
+
+Documentation
+=============
+
+You can view documentation at https://easylink.readthedocs.io/en/latest/

easylink-0.1.14/docs/source/user_guide/tutorials/DAG-common-pipeline.svg

@@ -0,0 +1,175 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
+ "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by graphviz version 2.50.0 (0)
+ -->
+<!-- Title: snakemake_dag Pages: 1 -->
+<svg width="582pt" height="764pt"
+ viewBox="0.00 0.00 582.29 764.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
+<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 760)">
+<title>snakemake_dag</title>
+<polygon fill="white" stroke="transparent" points="-4,4 -4,-760 578.29,-760 578.29,4 -4,4"/>
+<!-- 0 -->
+<g id="node1" class="node">
+<title>0</title>
+<path fill="none" stroke="#d8a456" stroke-width="2" d="M206.09,-36C206.09,-36 176.09,-36 176.09,-36 170.09,-36 164.09,-30 164.09,-24 164.09,-24 164.09,-12 164.09,-12 164.09,-6 170.09,0 176.09,0 176.09,0 206.09,0 206.09,0 212.09,0 218.09,-6 218.09,-12 218.09,-12 218.09,-24 218.09,-24 218.09,-30 212.09,-36 206.09,-36"/>
+<text text-anchor="middle" x="191.09" y="-15" font-family="sans" font-size="10.00">all</text>
+</g>
+<!-- 1 -->
+<g id="node2" class="node">
+<title>1</title>
+<path fill="none" stroke="#b6d856" stroke-width="2" d="M232.74,-180C232.74,-180 149.43,-180 149.43,-180 143.43,-180 137.43,-174 137.43,-168 137.43,-168 137.43,-156 137.43,-156 137.43,-150 143.43,-144 149.43,-144 149.43,-144 232.74,-144 232.74,-144 238.74,-144 244.74,-150 244.74,-156 244.74,-156 244.74,-168 244.74,-168 244.74,-174 238.74,-180 232.74,-180"/>
+<text text-anchor="middle" x="191.09" y="-159" font-family="sans" font-size="10.00">step_4_python_pandas</text>
+</g>
+<!-- 1&#45;&gt;0 -->
+<g id="edge1" class="edge">
+<title>1&#45;&gt;0</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M179.2,-143.72C173.08,-133.76 166.24,-120.73 163.09,-108 159.24,-92.47 159.24,-87.53 163.09,-72 165.38,-62.75 169.61,-53.34 174.11,-45.07"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="177.22,-46.68 179.2,-36.28 171.16,-43.17 177.22,-46.68"/>
+</g>
+<!-- 11 -->
+<g id="node12" class="node">
+<title>11</title>
+<path fill="none" stroke="#d8cb56" stroke-width="2" d="M293.79,-108C293.79,-108 184.39,-108 184.39,-108 178.39,-108 172.39,-102 172.39,-96 172.39,-96 172.39,-84 172.39,-84 172.39,-78 178.39,-72 184.39,-72 184.39,-72 293.79,-72 293.79,-72 299.79,-72 305.79,-78 305.79,-84 305.79,-84 305.79,-96 305.79,-96 305.79,-102 299.79,-108 293.79,-108"/>
+<text text-anchor="middle" x="239.09" y="-87" font-family="sans" font-size="10.00">results_main_input_validator</text>
+</g>
+<!-- 1&#45;&gt;11 -->
+<g id="edge15" class="edge">
+<title>1&#45;&gt;11</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M202.95,-143.7C208.59,-135.47 215.44,-125.48 221.66,-116.42"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="224.59,-118.33 227.36,-108.1 218.82,-114.37 224.59,-118.33"/>
+</g>
+<!-- 2 -->
+<g id="node3" class="node">
+<title>2</title>
+<path fill="none" stroke="#567bd8" stroke-width="2" d="M226.27,-324C226.27,-324 11.91,-324 11.91,-324 5.91,-324 -0.09,-318 -0.09,-312 -0.09,-312 -0.09,-300 -0.09,-300 -0.09,-294 5.91,-288 11.91,-288 11.91,-288 226.27,-288 226.27,-288 232.27,-288 238.27,-294 238.27,-300 238.27,-300 238.27,-312 238.27,-312 238.27,-318 232.27,-324 226.27,-324"/>
+<text text-anchor="middle" x="119.09" y="-303" font-family="sans" font-size="10.00">aggregate_step_3_python_pandas_step_3_main_output</text>
+</g>
+<!-- 2&#45;&gt;1 -->
+<g id="edge3" class="edge">
+<title>2&#45;&gt;1</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M95.34,-287.91C84.54,-278.65 72.89,-266.2 67.09,-252 61.04,-237.19 58.27,-229.35 67.09,-216 80.8,-195.23 104.51,-182.57 127.55,-174.86"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="128.81,-178.14 137.35,-171.86 126.76,-171.44 128.81,-178.14"/>
+</g>
+<!-- 10 -->
+<g id="node11" class="node">
+<title>10</title>
+<path fill="none" stroke="#56d8b9" stroke-width="2" d="M293.95,-252C293.95,-252 88.23,-252 88.23,-252 82.23,-252 76.23,-246 76.23,-240 76.23,-240 76.23,-228 76.23,-228 76.23,-222 82.23,-216 88.23,-216 88.23,-216 293.95,-216 293.95,-216 299.95,-216 305.95,-222 305.95,-228 305.95,-228 305.95,-240 305.95,-240 305.95,-246 299.95,-252 293.95,-252"/>
+<text text-anchor="middle" x="191.09" y="-231" font-family="sans" font-size="10.00">step_4_python_pandas_step_4_main_input_validator</text>
+</g>
+<!-- 2&#45;&gt;10 -->
+<g id="edge14" class="edge">
+<title>2&#45;&gt;10</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M136.89,-287.7C145.69,-279.14 156.47,-268.66 166.09,-259.3"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="168.77,-261.58 173.5,-252.1 163.89,-256.57 168.77,-261.58"/>
+</g>
+<!-- 3 -->
+<g id="node4" class="node">
+<title>3</title>
+<path fill="none" stroke="#56d86b" stroke-width="2" d="M212.63,-396C212.63,-396 25.55,-396 25.55,-396 19.55,-396 13.55,-390 13.55,-384 13.55,-384 13.55,-372 13.55,-372 13.55,-366 19.55,-360 25.55,-360 25.55,-360 212.63,-360 212.63,-360 218.63,-360 224.63,-366 224.63,-372 224.63,-372 224.63,-384 224.63,-384 224.63,-390 218.63,-396 212.63,-396"/>
+<text text-anchor="middle" x="119.09" y="-375" font-family="sans" font-size="10.00">split_step_3_python_pandas_step_3_main_input</text>
+</g>
+<!-- 3&#45;&gt;2 -->
+<g id="edge6" class="edge">
+<title>3&#45;&gt;2</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M119.09,-359.7C119.09,-351.98 119.09,-342.71 119.09,-334.11"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="122.59,-334.1 119.09,-324.1 115.59,-334.1 122.59,-334.1"/>
+</g>
+<!-- 4 -->
+<g id="node5" class="node">
+<title>4</title>
+<path fill="none" stroke="#56a9d8" stroke-width="2" d="M160.74,-540C160.74,-540 77.43,-540 77.43,-540 71.43,-540 65.43,-534 65.43,-528 65.43,-528 65.43,-516 65.43,-516 65.43,-510 71.43,-504 77.43,-504 77.43,-504 160.74,-504 160.74,-504 166.74,-504 172.74,-510 172.74,-516 172.74,-516 172.74,-528 172.74,-528 172.74,-534 166.74,-540 160.74,-540"/>
+<text text-anchor="middle" x="119.09" y="-519" font-family="sans" font-size="10.00">step_2_python_pandas</text>
+</g>
+<!-- 4&#45;&gt;3 -->
+<g id="edge7" class="edge">
+<title>4&#45;&gt;3</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M95.34,-503.91C84.54,-494.65 72.89,-482.2 67.09,-468 61.04,-453.19 61.04,-446.81 67.09,-432 71.58,-421.02 79.56,-411.08 87.94,-402.86"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="90.32,-405.42 95.34,-396.09 85.6,-400.26 90.32,-405.42"/>
+</g>
+<!-- 8 -->
+<g id="node9" class="node">
+<title>8</title>
+<path fill="none" stroke="#8fd856" stroke-width="2" d="M293.95,-468C293.95,-468 88.23,-468 88.23,-468 82.23,-468 76.23,-462 76.23,-456 76.23,-456 76.23,-444 76.23,-444 76.23,-438 82.23,-432 88.23,-432 88.23,-432 293.95,-432 293.95,-432 299.95,-432 305.95,-438 305.95,-444 305.95,-444 305.95,-456 305.95,-456 305.95,-462 299.95,-468 293.95,-468"/>
+<text text-anchor="middle" x="191.09" y="-447" font-family="sans" font-size="10.00">step_3_python_pandas_step_3_main_input_validator</text>
+</g>
+<!-- 4&#45;&gt;8 -->
+<g id="edge13" class="edge">
+<title>4&#45;&gt;8</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M136.89,-503.7C145.69,-495.14 156.47,-484.66 166.09,-475.3"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="168.77,-477.58 173.5,-468.1 163.89,-472.57 168.77,-477.58"/>
+</g>
+<!-- 5 -->
+<g id="node6" class="node">
+<title>5</title>
+<path fill="none" stroke="#56d0d8" stroke-width="2" d="M160.74,-684C160.74,-684 77.43,-684 77.43,-684 71.43,-684 65.43,-678 65.43,-672 65.43,-672 65.43,-660 65.43,-660 65.43,-654 71.43,-648 77.43,-648 77.43,-648 160.74,-648 160.74,-648 166.74,-648 172.74,-654 172.74,-660 172.74,-660 172.74,-672 172.74,-672 172.74,-678 166.74,-684 160.74,-684"/>
+<text text-anchor="middle" x="119.09" y="-663" font-family="sans" font-size="10.00">step_1_python_pandas</text>
+</g>
+<!-- 5&#45;&gt;4 -->
+<g id="edge9" class="edge">
+<title>5&#45;&gt;4</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M95.34,-647.91C84.54,-638.65 72.89,-626.2 67.09,-612 61.04,-597.19 61.04,-590.81 67.09,-576 71.58,-565.02 79.56,-555.08 87.94,-546.86"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="90.32,-549.42 95.34,-540.09 85.6,-544.26 90.32,-549.42"/>
+</g>
+<!-- 7 -->
+<g id="node8" class="node">
+<title>7</title>
+<path fill="none" stroke="#68d856" stroke-width="2" d="M293.95,-612C293.95,-612 88.23,-612 88.23,-612 82.23,-612 76.23,-606 76.23,-600 76.23,-600 76.23,-588 76.23,-588 76.23,-582 82.23,-576 88.23,-576 88.23,-576 293.95,-576 293.95,-576 299.95,-576 305.95,-582 305.95,-588 305.95,-588 305.95,-600 305.95,-600 305.95,-606 299.95,-612 293.95,-612"/>
+<text text-anchor="middle" x="191.09" y="-591" font-family="sans" font-size="10.00">step_2_python_pandas_step_2_main_input_validator</text>
+</g>
+<!-- 5&#45;&gt;7 -->
+<g id="edge12" class="edge">
+<title>5&#45;&gt;7</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M136.89,-647.7C145.69,-639.14 156.47,-628.66 166.09,-619.3"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="168.77,-621.58 173.5,-612.1 163.89,-616.57 168.77,-621.58"/>
+</g>
+<!-- 6 -->
+<g id="node7" class="node">
+<title>6</title>
+<path fill="none" stroke="#d85656" stroke-width="2" d="M221.95,-756C221.95,-756 16.23,-756 16.23,-756 10.23,-756 4.23,-750 4.23,-744 4.23,-744 4.23,-732 4.23,-732 4.23,-726 10.23,-720 16.23,-720 16.23,-720 221.95,-720 221.95,-720 227.95,-720 233.95,-726 233.95,-732 233.95,-732 233.95,-744 233.95,-744 233.95,-750 227.95,-756 221.95,-756"/>
+<text text-anchor="middle" x="119.09" y="-735" font-family="sans" font-size="10.00">step_1_python_pandas_step_1_main_input_validator</text>
+</g>
+<!-- 6&#45;&gt;5 -->
+<g id="edge11" class="edge">
+<title>6&#45;&gt;5</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M119.09,-719.7C119.09,-711.98 119.09,-702.71 119.09,-694.11"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="122.59,-694.1 119.09,-684.1 115.59,-694.1 122.59,-694.1"/>
+</g>
+<!-- 7&#45;&gt;4 -->
+<g id="edge10" class="edge">
+<title>7&#45;&gt;4</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M173.29,-575.7C164.48,-567.14 153.71,-556.66 144.09,-547.3"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="146.29,-544.57 136.68,-540.1 141.41,-549.58 146.29,-544.57"/>
+</g>
+<!-- 8&#45;&gt;3 -->
+<g id="edge8" class="edge">
+<title>8&#45;&gt;3</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M173.29,-431.7C164.48,-423.14 153.71,-412.66 144.09,-403.3"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="146.29,-400.57 136.68,-396.1 141.41,-405.58 146.29,-400.57"/>
+</g>
+<!-- 9 -->
+<g id="node10" class="node">
+<title>9</title>
+<path fill="none" stroke="#d87d56" stroke-width="2" d="M562.49,-252C562.49,-252 335.69,-252 335.69,-252 329.69,-252 323.69,-246 323.69,-240 323.69,-240 323.69,-228 323.69,-228 323.69,-222 329.69,-216 335.69,-216 335.69,-216 562.49,-216 562.49,-216 568.49,-216 574.49,-222 574.49,-228 574.49,-228 574.49,-240 574.49,-240 574.49,-246 568.49,-252 562.49,-252"/>
+<text text-anchor="middle" x="449.09" y="-231" font-family="sans" font-size="10.00">step_4_python_pandas_step_4_secondary_input_validator</text>
+</g>
+<!-- 9&#45;&gt;1 -->
+<g id="edge4" class="edge">
+<title>9&#45;&gt;1</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M386.3,-215.97C346.34,-205.12 294.75,-191.13 254.64,-180.24"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="255.54,-176.86 244.97,-177.62 253.7,-183.62 255.54,-176.86"/>
+</g>
+<!-- 10&#45;&gt;1 -->
+<g id="edge5" class="edge">
+<title>10&#45;&gt;1</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M191.09,-215.7C191.09,-207.98 191.09,-198.71 191.09,-190.11"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="194.59,-190.1 191.09,-180.1 187.59,-190.1 194.59,-190.1"/>
+</g>
+<!-- 11&#45;&gt;0 -->
+<g id="edge2" class="edge">
+<title>11&#45;&gt;0</title>
+<path fill="none" stroke="grey" stroke-width="2" d="M227.22,-71.7C221.59,-63.47 214.73,-53.48 208.52,-44.42"/>
+<polygon fill="grey" stroke="grey" stroke-width="2" points="211.36,-42.37 202.82,-36.1 205.59,-46.33 211.36,-42.37"/>
+</g>
+</g>
+</svg>