IUExtract 1.0.0__tar.gz

iuextract-1.0.0/LICENSE

@@ -0,0 +1,33 @@
+The Clear BSD License
+
+Copyright (c) 2022 Marcello Gecchele, 
+ Tokunaga Laboratory of Computational Linguistics, Tokyo Institute of Technology
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted (subject to the limitations in the disclaimer
+below) provided that the following conditions are met:
+
+     * Redistributions of source code must retain the above copyright notice,
+     this list of conditions and the following disclaimer.
+
+     * Redistributions in binary form must reproduce the above copyright
+     notice, this list of conditions and the following disclaimer in the
+     documentation and/or other materials provided with the distribution.
+
+     * Neither the name of the copyright holder nor the names of its
+     contributors may be used to endorse or promote products derived from this
+     software without specific prior written permission.
+
+NO EXPRESS OR IMPLIED LICENSES TO ANY PARTY'S PATENT RIGHTS ARE GRANTED BY
+THIS LICENSE. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.

iuextract-1.0.0/MANIFEST.in

@@ -0,0 +1 @@
+include src/iuextract/transition_signals.txt

iuextract-1.0.0/PKG-INFO

@@ -0,0 +1,86 @@
+Metadata-Version: 2.1
+Name: IUExtract
+Version: 1.0.0
+Summary: Rule-based Idea Unit segmentation algorithm for the English language.
+Home-page: https://github.com/TT-CL/iuextract
+Author: Gecchele Marcello
+Author-email: Marcello Gecchele <linked.uno@pm.me>
+License: The Clear BSD License
+        
+        Copyright (c) 2022 Marcello Gecchele, 
+         Tokunaga Laboratory of Computational Linguistics, Tokyo Institute of Technology
+        All rights reserved.
+        
+        Redistribution and use in source and binary forms, with or without
+        modification, are permitted (subject to the limitations in the disclaimer
+        below) provided that the following conditions are met:
+        
+             * Redistributions of source code must retain the above copyright notice,
+             this list of conditions and the following disclaimer.
+        
+             * Redistributions in binary form must reproduce the above copyright
+             notice, this list of conditions and the following disclaimer in the
+             documentation and/or other materials provided with the distribution.
+        
+             * Neither the name of the copyright holder nor the names of its
+             contributors may be used to endorse or promote products derived from this
+             software without specific prior written permission.
+        
+        NO EXPRESS OR IMPLIED LICENSES TO ANY PARTY'S PATENT RIGHTS ARE GRANTED BY
+        THIS LICENSE. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+        CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+        LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+        PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+        CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+        EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+        PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+        BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+        IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+        ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+        POSSIBILITY OF SUCH DAMAGE.
+Project-URL: Homepage, https://tt-cl.github.io/iu-resources/
+Project-URL: Documentation, https://github.com/TT-CL/iuextract
+Project-URL: Repository, https://github.com/TT-CL/iuextract.git
+Project-URL: Issues, https://github.com/TT-CL/iuextract/issues
+Keywords: Idea Unit,textual segmentation,segmentation,linguistics
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: spacy
+
+# IUExtract
+Rule-based Idea Unit segmentation algorithm for the English language.
+
+## Installation
+First of all, you need to install the dependencies:
+```
+pip install spacy
+python -m spacy download en_core_web_lg
+```
+To install the package with the command line tool [install pipx](https://pipx.pypa.io/latest/installation/) and run the following command:
+```
+pipx install iuextract
+```
+
+If you only wish to use the package in your python projects you can install without executable via
+```
+pip install iuextract
+```
+
+## Command Line Interface (CLI) Usage
+Once installed via `pipx`, you can run
+```
+iuextract -i input_file.txt 
+```
+to segment `file.txt` into Idea Units. The segmented file will be printed on the console as standard output.
+You can specify an output file with the `-o` parameter.
+```
+iuextract -i input_file.txt -o output_file.txt
+```
+For more options you can call the help argument.
+```
+iuextract -h
+```

iuextract-1.0.0/README.md

@@ -0,0 +1,33 @@
+# IUExtract
+Rule-based Idea Unit segmentation algorithm for the English language.
+
+## Installation
+First of all, you need to install the dependencies:
+```
+pip install spacy
+python -m spacy download en_core_web_lg
+```
+To install the package with the command line tool [install pipx](https://pipx.pypa.io/latest/installation/) and run the following command:
+```
+pipx install iuextract
+```
+
+If you only wish to use the package in your python projects you can install without executable via
+```
+pip install iuextract
+```
+
+## Command Line Interface (CLI) Usage
+Once installed via `pipx`, you can run
+```
+iuextract -i input_file.txt 
+```
+to segment `file.txt` into Idea Units. The segmented file will be printed on the console as standard output.
+You can specify an output file with the `-o` parameter.
+```
+iuextract -i input_file.txt -o output_file.txt
+```
+For more options you can call the help argument.
+```
+iuextract -h
+```

iuextract-1.0.0/pyproject.toml

@@ -0,0 +1,35 @@
+[build-system]
+requires = ["setuptools>=42"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "IUExtract"
+version = "1.0.0"
+description = "Rule-based Idea Unit segmentation algorithm for the English language."
+readme = {file = "README.md", content-type = "text/markdown"}
+license = {file = "LICENSE"}
+keywords = ["Idea Unit", "textual segmentation", "segmentation", "linguistics"]
+
+requires-python = '>=3.8'
+
+dependencies = [
+  "spacy",
+]
+authors = [
+  {name = "Marcello Gecchele", email = "linked.uno@pm.me"},
+]
+
+classifiers = [
+  'Programming Language :: Python :: 3',
+  'License :: OSI Approved :: MIT License',
+  'Operating System :: OS Independent'
+  ]
+
+[project.scripts]
+iuextract = "iuextract:__main__"
+
+[project.urls]
+Homepage = "https://tt-cl.github.io/iu-resources/"
+Documentation = "https://github.com/TT-CL/iuextract"
+Repository = "https://github.com/TT-CL/iuextract.git"
+Issues = "https://github.com/TT-CL/iuextract/issues"

iuextract-1.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

iuextract-1.0.0/setup.py

@@ -0,0 +1,32 @@
+import setuptools
+
+with open("README.md", "r", encoding="utf-8") as fh:
+    long_description = fh.read()
+
+setuptools.setup(
+    name="iuextract",
+    version="0.0.10",
+    author="Gecchele Marcello",
+    author_email="git@gecchele.dev",
+    description="Extract Idea Units from strings and files",
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    url="https://github.com/TT-CL/iuextract",
+    
+    project_urls={
+        "Bug Tracker": "https://github.com/TT-CL/iuextract/issues",
+    },
+    classifiers=[
+        "Programming Language :: Python :: 3",
+        "License :: OSI Approved :: MIT License",
+        "Operating System :: OS Independent",
+    ],
+    package_dir={"": "src"},
+    packages=setuptools.find_packages(where="src"),
+    include_package_data=True,
+    python_requires=">=3.8",
+    install_requires = [
+        'setuptools',
+        'spacy>=3.0.0',
+    ]
+)

iuextract-1.0.0/src/IUExtract.egg-info/PKG-INFO

@@ -0,0 +1,86 @@
+Metadata-Version: 2.1
+Name: IUExtract
+Version: 1.0.0
+Summary: Rule-based Idea Unit segmentation algorithm for the English language.
+Home-page: https://github.com/TT-CL/iuextract
+Author: Gecchele Marcello
+Author-email: Marcello Gecchele <linked.uno@pm.me>
+License: The Clear BSD License
+        
+        Copyright (c) 2022 Marcello Gecchele, 
+         Tokunaga Laboratory of Computational Linguistics, Tokyo Institute of Technology
+        All rights reserved.
+        
+        Redistribution and use in source and binary forms, with or without
+        modification, are permitted (subject to the limitations in the disclaimer
+        below) provided that the following conditions are met:
+        
+             * Redistributions of source code must retain the above copyright notice,
+             this list of conditions and the following disclaimer.
+        
+             * Redistributions in binary form must reproduce the above copyright
+             notice, this list of conditions and the following disclaimer in the
+             documentation and/or other materials provided with the distribution.
+        
+             * Neither the name of the copyright holder nor the names of its
+             contributors may be used to endorse or promote products derived from this
+             software without specific prior written permission.
+        
+        NO EXPRESS OR IMPLIED LICENSES TO ANY PARTY'S PATENT RIGHTS ARE GRANTED BY
+        THIS LICENSE. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+        CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+        LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+        PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+        CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+        EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+        PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+        BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+        IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+        ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+        POSSIBILITY OF SUCH DAMAGE.
+Project-URL: Homepage, https://tt-cl.github.io/iu-resources/
+Project-URL: Documentation, https://github.com/TT-CL/iuextract
+Project-URL: Repository, https://github.com/TT-CL/iuextract.git
+Project-URL: Issues, https://github.com/TT-CL/iuextract/issues
+Keywords: Idea Unit,textual segmentation,segmentation,linguistics
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: spacy
+
+# IUExtract
+Rule-based Idea Unit segmentation algorithm for the English language.
+
+## Installation
+First of all, you need to install the dependencies:
+```
+pip install spacy
+python -m spacy download en_core_web_lg
+```
+To install the package with the command line tool [install pipx](https://pipx.pypa.io/latest/installation/) and run the following command:
+```
+pipx install iuextract
+```
+
+If you only wish to use the package in your python projects you can install without executable via
+```
+pip install iuextract
+```
+
+## Command Line Interface (CLI) Usage
+Once installed via `pipx`, you can run
+```
+iuextract -i input_file.txt 
+```
+to segment `file.txt` into Idea Units. The segmented file will be printed on the console as standard output.
+You can specify an output file with the `-o` parameter.
+```
+iuextract -i input_file.txt -o output_file.txt
+```
+For more options you can call the help argument.
+```
+iuextract -h
+```

iuextract-1.0.0/src/IUExtract.egg-info/SOURCES.txt

@@ -0,0 +1,23 @@
+LICENSE
+MANIFEST.in
+README.md
+pyproject.toml
+setup.py
+src/IUExtract.egg-info/PKG-INFO
+src/IUExtract.egg-info/SOURCES.txt
+src/IUExtract.egg-info/dependency_links.txt
+src/IUExtract.egg-info/entry_points.txt
+src/IUExtract.egg-info/requires.txt
+src/IUExtract.egg-info/top_level.txt
+src/iuextract/__init__.py
+src/iuextract/__main__.py
+src/iuextract/data.py
+src/iuextract/extract.py
+src/iuextract/gold.py
+src/iuextract/iu_utils.py
+src/iuextract/transition_signals.txt
+src/iuextract.egg-info/PKG-INFO
+src/iuextract.egg-info/SOURCES.txt
+src/iuextract.egg-info/dependency_links.txt
+src/iuextract.egg-info/requires.txt
+src/iuextract.egg-info/top_level.txt

iuextract-1.0.0/src/IUExtract.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

iuextract-1.0.0/src/IUExtract.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+iuextract = iuextract:__main__

iuextract-1.0.0/src/IUExtract.egg-info/requires.txt

@@ -0,0 +1 @@
+spacy

iuextract-1.0.0/src/IUExtract.egg-info/top_level.txt

@@ -0,0 +1 @@
+iuextract

iuextract-1.0.0/src/iuextract/__main__.py

@@ -0,0 +1,42 @@
+import argparse
+import sys
+import warnings
+from .data import import_file
+from .extract import label_ius
+from .iu_utils import iuDoc2str
+import spacy
+
+
+parser = argparse.ArgumentParser(prog='iuextract', description='Segment a raw text into Idea Units')
+parser.add_argument('-i', '--input', help='the {i}nput text. If it is not provided, the program will read the positional arguments input as a text.', type=argparse.FileType('r'), required=False, default=None)
+parser.add_argument('-o', '--output', help="the {o}utput file where to store the ius. By default the segmentation will be shown on the terminal and will not be stored on disk.", nargs='?', type=argparse.FileType('w'), default=sys.stdout)
+parser.add_argument('pos_input', help="the input text to analyze if no filename is provided.", nargs='*')
+#parser.add_argument('-o', '--output', help="the {o}utput file where to store the ius. By default the segmentation will be shown on the terminal and will not be stored on disk. \nAccepted filetype: .txt", required=False, default=None)
+parser.add_argument('-b', '--before', help="a sequence of text to place {b}efore each IU. By default, no prefix is set.", required=False, default="")
+parser.add_argument('-s', '--separator', help="a sequence of text to {s}eparate the index from the IU. By default, the separator is the character | .", required=False, default="|")
+parser.add_argument('-a', '--after', help="a sequence of text to place {a}fter each IU. By default, the suffix is the newline character.", required=False, default="\n")
+
+args = parser.parse_args()
+input_file = args.input
+text_input = args.pos_input
+text_input = ' '.join(text_input) if len(text_input) > 0 else None
+if input_file is None and text_input is None:
+    raise IOError("Please provide a valid input text. Run iuextract -h for more help.")
+if input_file is not None and text_input is not None:
+    input_warning = "WARNING: the program detected both an input file passed via the -i argument and input text via positional arguments. Ignoring the positional arguments and only processing the input file."
+    warnings.warn(input_warning)
+raw_input = text_input
+if input_file is not None:
+    raw_input = input_file.read()
+outputFile = args.output
+prefix = args.before
+suffix = args.after
+separator = args.separator
+
+nlp = spacy.load("en_core_web_lg")
+parsed = import_file(raw_input, nlp=nlp)
+label_ius(parsed)
+raw_output = iuDoc2str(parsed,gold=False, index_sep=separator, opener=prefix, closer=suffix)
+if not raw_output.endswith('\n'):
+    raw_output = f'{raw_output}\n'
+outputFile.write(raw_output)