pyconll 3.2.0__tar.gz → 3.3.0__tar.gz

{pyconll-3.2.0 → pyconll-3.3.0}/MANIFEST.in

@@ -1,4 +1,2 @@
 include LICENSE
 include README.md
-include README.rst
-include README

pyconll-3.3.0/PKG-INFO

@@ -0,0 +1,138 @@
+Metadata-Version: 2.4
+Name: pyconll
+Version: 3.3.0
+Summary: Read and manipulate CoNLL files
+Home-page: https://github.com/pyconll/pyconll
+Author: Matias Grioni
+Author-email: matgrioni@gmail.com
+License: MIT
+Keywords: nlp,conllu,conll,universal dependencies
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Education
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: Implementation :: CPython
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Utilities
+Requires-Python: ~=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license
+Dynamic: license-file
+Dynamic: requires-python
+Dynamic: summary
+
+[![Build Status](https://github.com/pyconll/pyconll/workflows/CI/badge.svg?branch=master)](https://github.com/pyconll/pyconll)
+[![Coverage Status](https://coveralls.io/repos/github/pyconll/pyconll/badge.svg?branch=master)](https://coveralls.io/github/pyconll/pyconll?branch=master)
+[![Documentation Status](https://readthedocs.org/projects/pyconll/badge/?version=stable)](https://pyconll.readthedocs.io/en/stable)
+[![Version](https://img.shields.io/github/v/release/pyconll/pyconll)](https://github.com/pyconll/pyconll/releases)
+[![gitter](https://badges.gitter.im/pyconll/pyconll.svg)](https://gitter.im/pyconll/pyconll?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
+
+## pyconll
+
+*Easily work with **CoNLL** files using the familiar syntax of **python**.*
+
+<img src="res/logo.svg" width="256px" height="256px">
+
+##### Links
+- [Homepage](https://pyconll.github.io)
+- [Documentation](https://pyconll.readthedocs.io/)
+
+
+### Installation
+
+As with most python packages, simply use `pip` to install from PyPi.
+
+```
+pip install pyconll
+```
+
+`pyconll` is also available as a conda package on the `pyconll` channel. Only packages 2.2.0 and newer are available on conda at the moment.
+
+```
+conda install -c pyconll pyconll
+```
+
+pyconll supports Python 3.10 or newer. In general, pyconll will focus development efforts on officially supported python versions.
+
+
+### Use
+
+This tool is intended to be a **minimal**, **low level**, **expressive** and **pragmatic** library in a widely used programming language. pyconll creates a thin API on top of raw CoNLL annotations that is simple and intuitive.
+
+It offers the following features:
+* Regular CI testing and validation against all UD v2.x versions.
+* A strong domain model that includes CoNLL sources, Sentences, Tokens, Trees, etc.
+* A typed API for better development experience and better semantics.
+* A focus on usability and simplicity in design (no dependencies)
+* Performance optimizations for a smooth development workflow no matter the dataset size (performs about 25%-35% faster than other comparable packages)
+
+See the following code example to understand the basics of the API.
+
+```python
+# This snippet finds sentences where a token marked with part of speech 'AUX' are
+# governed by a NOUN. For example, in French this is a less common construction
+# and we may want to validate these examples because we have previously found some
+# problematic examples of this construction.
+import pyconll
+
+train = pyconll.load_from_file('./ud/train.conllu')
+
+review_sentences = []
+
+# Conll objects are iterable over their sentences, and sentences are iterable
+# over their tokens. Sentences also de/serialize comment information.
+for sentence in train:
+    for token in sentence:
+
+        # Tokens have attributes such as upos, head, id, deprel, etc, and sentences
+        # can be indexed by a token's id. We must check that the token is not the
+        # root token, whose id, '0', cannot be looked up.
+        if token.upos == 'AUX' and (token.head != '0' and sentence[token.head].upos == 'NOUN'):
+            review_sentences.append(sentence)
+
+print('Review the following sentences:')
+for sent in review_sentences:
+    print(sent.id)
+```
+
+A full definition of the API can be found in the [documentation](https://pyconll.readthedocs.io/) or use the [quick start](https://pyconll.readthedocs.io/en/stable/starting.html) guide for a focused introduction.
+
+
+### Uses and Limitations
+
+This package edits CoNLL-U annotations. This does not include the annotated text itself. Word forms on Tokens are not editable and Sentence Tokens cannot be reassigned or reordered. `pyconll` focuses on editing CoNLL-U annotation rather than creating it or changing the underlying text that is annotated. If there is interest in this functionality area, please create a GitHub issue for more visibility.
+
+This package also is only validated against the CoNLL-U format. The CoNLL and CoNLL-X format are not supported, but are very similar. I originally intended to support these formats as well, but their format is not as well defined as CoNLL-U so they are not included. Please create an issue for visibility if this feature interests you.
+
+Lastly, linguistic data can often be very large and this package attempts to keep that in mind. pyconll provides methods for creating in memory conll objects along with an iterate only version in case a corpus is too large to store in memory (the size of the memory structure is several times larger than the actual corpus file). The iterate only version can parse upwards of 100,000 words per second on a 16gb ram machine, so for most datasets to be used on a local dev machine, this package will perform well. The 2.2.0 release also improves parse time and memory footprint by about 25%!
+
+
+### Contributing
+
+Contributions to this project are welcome and encouraged! If you are unsure how to contribute, here is a [guide](https://help.github.com/en/articles/creating-a-pull-request-from-a-fork) from Github explaining the basic workflow. After cloning this repo, please run `pip install -r requirements.txt` to properly setup locally. Some of these tools like yapf, pylint, and mypy do not have to be run locally, but CI builds will fail without their successful running. Some other release dependencies like twine and sphinx are also installed.
+
+For packaging new versions, use setuptools version 24.2.0 or greater for creating the appropriate packaging that recognizes the `python_requires` metadata. Final packaging and release is now done with Github actions so this is less of a concern.
+
+
+#### README and CHANGELOG
+
+When changing either of these files, please change the Markdown version and run ``make gendocs`` so that the other versions stay in sync.
+
+#### Release Checklist
+
+Below enumerates the general release process explicitly. This section is for internal use and most people do not have to worry about this. First note, that the dev branch is always a direct extension of master with the latest changes since the last release. That is, it is essentially a staging release branch.
+
+* Change the version in `pyconll/_version.py` appropriately.
+* Merge dev into master **locally**. Github does not offer a fast forward merge and explicitly uses --no-ff. So to keep the linear nature of changes, merge locally to fast forward. This is assuming that the dev branch looks good on CI tests which do not automatically run in this situation.
+* Push the master branch. This should start some CI tests specifically for master. After validating these results, create a tag corresponding to the next version number and push the tag.
+* Create a new release from this tag from the [Releases page](https://github.com/pyconll/pyconll/releases). On creating this release, two workflows will start. One releases to pypi, and the other releases to conda.
+* Validate these workflows pass, and the package is properly released on both platforms.

{pyconll-3.2.0 → pyconll-3.3.0}/README.md

@@ -29,7 +29,7 @@ pip install pyconll
 conda install -c pyconll pyconll
 ```
 
-pyconll supports Python 3.8 and greater. In general, pyconll will focus development efforts on officially supported python versions.
+pyconll supports Python 3.10 or newer. In general, pyconll will focus development efforts on officially supported python versions.
 
 
 ### Use

{pyconll-3.2.0 → pyconll-3.3.0}/pyconll/_version.py

@@ -5,4 +5,4 @@ This is surfaced publicly via __init__.py and keeps the version specification
 separate from other module logic to allow for easy parsing when packaging.
 """
 
-__version__ = '3.2.0'
+__version__ = '3.3.0'

{pyconll-3.2.0 → pyconll-3.3.0}/pyconll/conllable.py

@@ -12,6 +12,7 @@ class Conllable(metaclass=abc.ABCMeta):
     A Conllable mixin to indicate that the component can be converted into a
     CoNLL representation.
     """
+
     @abc.abstractmethod
     def conll(self) -> str:
         """

{pyconll-3.2.0 → pyconll-3.3.0}/pyconll/load.py

@@ -6,13 +6,13 @@ functionalities.
 """
 
 import os
-from typing import Iterable, Iterator, Union
+from typing import Iterable, Iterator
 
 from pyconll._parser import iter_sentences
 from pyconll.unit.conll import Conll
 from pyconll.unit.sentence import Sentence
 
-PathLike = Union[str, bytes, os.PathLike]
+PathLike = str | bytes | os.PathLike
 
 
 def load_from_string(source: str) -> Conll:
@@ -89,8 +89,7 @@ def iter_from_string(source: str) -> Iterator[Sentence]:
         ParseError: If there is an error parsing the input into a Conll object.
     """
     lines = source.splitlines()
-    for sentence in iter_sentences(lines):
-        yield sentence
+    yield from iter_sentences(lines)
 
 
 def iter_from_file(file_descriptor: PathLike) -> Iterator[Sentence]:
@@ -109,8 +108,7 @@ def iter_from_file(file_descriptor: PathLike) -> Iterator[Sentence]:
         ParseError: If there is an error parsing the input into a Conll object.
     """
     with open(file_descriptor, encoding='utf-8') as f:
-        for sentence in iter_sentences(f):
-            yield sentence
+        yield from iter_sentences(f)
 
 
 def iter_from_resource(resource: Iterable[str]) -> Iterator[Sentence]:
@@ -131,5 +129,4 @@ def iter_from_resource(resource: Iterable[str]) -> Iterator[Sentence]:
     Raises:
         ParseError: If there is an error parsing the input into a Conll object.
     """
-    for sentence in iter_sentences(resource):
-        yield sentence
+    yield from iter_sentences(resource)

{pyconll-3.2.0 → pyconll-3.3.0}/pyconll/tree/_treebuilder.py

@@ -23,6 +23,7 @@ class TreeBuilder(Generic[T]):
     created from the same TreeBuilder, Tree nodes will be unique, but data on
     the nodes will be shallow copies.
     """
+
     def __init__(self) -> None:
         """
         Creates a new empty TreeBuilder, with no internal data.

{pyconll-3.2.0 → pyconll-3.3.0}/pyconll/tree/tree.py

@@ -18,6 +18,7 @@ class Tree(Generic[T]):
     module which is a sort of friend class of Tree to maintain its immutable
     public contract.
     """
+
     def __init__(self, data: T) -> None:
         """
         Create a tree holding the value. Create a larger Tree, with TreeBuilder.
@@ -70,8 +71,7 @@ class Tree(Generic[T]):
         """
         Provides an iterator over the children.
         """
-        for child in self._children:
-            yield child
+        yield from self._children
 
     def __len__(self) -> int:
         """

{pyconll-3.2.0 → pyconll-3.3.0}/pyconll/unit/conll.py

@@ -2,7 +2,7 @@
 Defines the Conll type and the associated parsing and output logic.
 """
 
-from typing import Any, Iterable, Iterator, List, Union, MutableSequence, overload
+from typing import Any, Iterable, Iterator, MutableSequence, overload
 
 import pyconll._parser
 from pyconll.conllable import Conllable
@@ -17,6 +17,7 @@ class Conll(MutableSequence[Sentence], Conllable):
     specifies that the file must end in a new line but that requirement is
     relaxed here in parsing.
     """
+
     def __init__(self, it: Iterable[str]) -> None:
         """
         Create a CoNLL-U file collection of sentences.
@@ -28,7 +29,7 @@ class Conll(MutableSequence[Sentence], Conllable):
             ParseError: If there is an error constructing the sentences in the
                 iterator.
         """
-        self._sentences: List[Sentence] = []
+        self._sentences: list[Sentence] = []
 
         for sentence in pyconll._parser.iter_sentences(it):
             self._sentences.append(sentence)
@@ -99,8 +100,7 @@ class Conll(MutableSequence[Sentence], Conllable):
         Yields:
             An iterator over the sentences in this Conll object.
         """
-        for sentence in self._sentences:
-            yield sentence
+        yield from self._sentences
 
     @overload
     def __getitem__(self, key: int) -> Sentence:
@@ -156,7 +156,7 @@ class Conll(MutableSequence[Sentence], Conllable):
         """
         self._sentences[key] = item
 
-    def __delitem__(self, key: Union[int, slice]) -> None:
+    def __delitem__(self, key: int | slice) -> None:
         """
         Delete the Sentence corresponding with the given key.
 

{pyconll-3.2.0 → pyconll-3.3.0}/pyconll/unit/sentence.py

@@ -4,7 +4,7 @@ Defines the Sentence type and the associated parsing and output logic.
 
 from collections import OrderedDict
 import re
-from typing import ClassVar, Dict, Iterator, List, Optional, Sequence, overload
+from typing import ClassVar, Iterator, Optional, Sequence, overload
 
 from pyconll.conllable import Conllable
 from pyconll.exception import FormatError, ParseError
@@ -62,8 +62,8 @@ class Sentence(Sequence[Token], Conllable):
         lines = source.split('\n')
 
         self._meta: OrderedDict[str, Optional[str]] = OrderedDict()  # pylint: disable=E1136
-        self._tokens: List[Token] = []
-        self._ids_to_indexes: Dict[str, int] = {}
+        self._tokens: list[Token] = []
+        self._ids_to_indexes: dict[str, int] = {}
 
         for i, line in enumerate(lines):
             if line:
@@ -212,7 +212,7 @@ class Sentence(Sequence[Token], Conllable):
             ValueError: If the sentence can not be made into a tree because a
                 token has an empty head value or if there is no root token.
         """
-        children_tokens: Dict[str, List[Token]] = {}
+        children_tokens: dict[str, list[Token]] = {}
 
         for token in self:
             if token.head is not None:
@@ -244,7 +244,7 @@ class Sentence(Sequence[Token], Conllable):
     @staticmethod
     def _create_tree_helper(builder: TreeBuilder, sentence: 'Sentence',
                             root: Token,
-                            children_tokens: Dict[str, List[Token]]) -> None:
+                            children_tokens: dict[str, list[Token]]) -> None:
         """
         Method to create a tree from a sentence given the root token.
 
@@ -303,8 +303,7 @@ class Sentence(Sequence[Token], Conllable):
         Iterate through all the tokens in the Sentence including multiword
         tokens.
         """
-        for token in self._tokens:
-            yield token
+        yield from self._tokens
 
     @overload
     def __getitem__(self, key: str) -> Token:

{pyconll-3.2.0 → pyconll-3.3.0}/pyconll/unit/token.py

@@ -6,7 +6,7 @@ format.
 
 import functools
 import math
-from typing import Callable, ClassVar, Dict, Optional, Set, Tuple
+from typing import Callable, ClassVar, Optional
 
 from pyconll.conllable import Conllable
 from pyconll.exception import FormatError, ParseError
@@ -78,7 +78,7 @@ def _create_dict_tupled_empty_parse(size, strict):
 
     Args:
         size: The expected size of the tuple.
-        strict: Flag to signifiy if parsed values with less components than size
+        strict: Flag to signify if parsed values with less components than size
             will be accepted. In this case, missing values will be supplated
             with None.
 
@@ -89,6 +89,7 @@ def _create_dict_tupled_empty_parse(size, strict):
         ParseError: If the parsing is strict and there is a component size
             mismatch, or if there are too many components in general.
     """
+
     def _dict_tupled_empty_parser(v, v_delimiter):
         """
         Map a value into the appropriate form, for a tupled based column.
@@ -123,7 +124,7 @@ def _create_dict_tupled_empty_parse(size, strict):
     return _dict_tupled_empty_parser
 
 
-TUPLE_PARSER_MEMOIZE: Dict[int, Callable[[str, str], Tuple[Optional[str],
+TUPLE_PARSER_MEMOIZE: dict[int, Callable[[str, str], tuple[Optional[str],
                                                            ...]]] = {}
 
 
@@ -237,7 +238,7 @@ def _dict_empty_map_helper(values, empty, delim, av_separator, v_delimiter,
         if len(parts) == 1 or (len(parts) == 2 and parts[1] == ''):
             k = parts[0]
             v = None
-        elif len(parts) == 2:
+        else:
             k, v = parts
 
         parsed = parser(v, v_delimiter)
@@ -419,6 +420,7 @@ def _dict_conll_map_helper(values, empty, delim, av_separator, v_delimiter,
     Returns:
         The CoNLL-U formatted equivalent to the value.
     """
+
     def paramed(pair):
         f = formatter(pair[1], v_delimiter)
         if f is None:
@@ -445,6 +447,7 @@ class _TokenIdComparer:
     being compared by the start index and then by the end index, and decimal
     ids having the radix separated parts compared separately.
     """
+
     def __init__(self, token_id):
         """
         Create the comparer wrapping the given, assumed valid format, id.
@@ -619,9 +622,9 @@ class Token(Conllable):
     # Keys for sorting attribute-value columns. BY_ID converts the attribute
     # value pair to the integer value of the attribute, and BY_CASE_SENSITIVE
     # converts the pair to the lowercase version of the attribute.
-    BY_ID: ClassVar[Callable[[Tuple[
+    BY_ID: ClassVar[Callable[[tuple[
         str, str]], _TokenIdComparer]] = lambda pair: _TokenIdComparer(pair[0])
-    BY_CASE_INSENSITIVE: ClassVar[Callable[[Tuple[
+    BY_CASE_INSENSITIVE: ClassVar[Callable[[tuple[
         str, str]], str]] = lambda pair: pair[0].lower()
 
     def __init__(self, source: str, empty: bool = False) -> None:
@@ -678,18 +681,18 @@ class Token(Conllable):
 
         self.upos: Optional[str] = _unit_empty_map(fields[3], Token.EMPTY)
         self.xpos: Optional[str] = _unit_empty_map(fields[4], Token.EMPTY)
-        self.feats: Dict[str,
-                         Set[str]] = _dict_empty_map(fields[5], Token.EMPTY,
+        self.feats: dict[str,
+                         set[str]] = _dict_empty_map(fields[5], Token.EMPTY,
                                                      Token.COMPONENT_DELIMITER,
                                                      Token.AV_SEPARATOR,
                                                      Token.V_DELIMITER)
         self.head: Optional[str] = _unit_empty_map(fields[6], Token.EMPTY)
         self.deprel: Optional[str] = _unit_empty_map(fields[7], Token.EMPTY)
-        self.deps: Dict[str,
-                        Tuple[str, str, str, str]] = _dict_tupled_empty_map(
+        self.deps: dict[str,
+                        tuple[str, str, str, str]] = _dict_tupled_empty_map(
                             fields[8], Token.EMPTY, Token.COMPONENT_DELIMITER,
                             Token.AV_DEPS_SEPARATOR, Token.V_DEPS_DELIMITER, 4)
-        self.misc: Dict[str, Optional[Set[str]]] = _dict_mixed_empty_map(
+        self.misc: dict[str, Optional[set[str]]] = _dict_mixed_empty_map(
             fields[9], Token.EMPTY, Token.COMPONENT_DELIMITER,
             Token.AV_SEPARATOR, Token.V_DELIMITER)
 

{pyconll-3.2.0 → pyconll-3.3.0}/pyconll/util.py

@@ -5,7 +5,7 @@ collection of functions.
 
 import functools
 import itertools
-from typing import Iterable, Iterator, Sequence, List, Tuple
+from typing import Iterable, Iterator, Sequence
 
 from pyconll.unit.sentence import Sentence
 from pyconll.unit.token import Token
@@ -15,7 +15,7 @@ def find_ngrams(
     conll: Iterable[Sentence],
     ngram: Sequence[str],
     case_sensitive: bool = True
-) -> Iterator[Tuple[Sentence, int, List[Token]]]:
+) -> Iterator[tuple[Sentence, int, list[Token]]]:
     """
     Find the occurrences of the ngram in the provided Conll collection.
 
@@ -77,7 +77,7 @@ def find_ngrams(
             i += 1
 
 
-def find_nonprojective_deps(sentence: Sentence) -> List[Tuple[Token, Token]]:
+def find_nonprojective_deps(sentence: Sentence) -> list[tuple[Token, Token]]:
     """
     Find the nonprojective dependency pairs in the provided sentence.
 
@@ -95,7 +95,7 @@ def find_nonprojective_deps(sentence: Sentence) -> List[Tuple[Token, Token]]:
         dependency pair.
     """
     deps = _transform_tokens_to_sorted_dependency_arcs(sentence)
-    non_projective_deps: List[Tuple[int, int]] = []
+    non_projective_deps: list[tuple[int, int]] = []
 
     openings = [-1]
     closings = [len(sentence)]
@@ -169,6 +169,7 @@ class _DependencyComparer:
     """
     Wrapper to compare dependency arcs.
     """
+
     def __init__(self, dep):
         """
         Creates the wrapper for this dependency.

pyconll-3.3.0/pyconll.egg-info/PKG-INFO

@@ -0,0 +1,138 @@
+Metadata-Version: 2.4
+Name: pyconll
+Version: 3.3.0
+Summary: Read and manipulate CoNLL files
+Home-page: https://github.com/pyconll/pyconll
+Author: Matias Grioni
+Author-email: matgrioni@gmail.com
+License: MIT
+Keywords: nlp,conllu,conll,universal dependencies
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Education
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: Implementation :: CPython
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Utilities
+Requires-Python: ~=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license
+Dynamic: license-file
+Dynamic: requires-python
+Dynamic: summary
+
+[![Build Status](https://github.com/pyconll/pyconll/workflows/CI/badge.svg?branch=master)](https://github.com/pyconll/pyconll)
+[![Coverage Status](https://coveralls.io/repos/github/pyconll/pyconll/badge.svg?branch=master)](https://coveralls.io/github/pyconll/pyconll?branch=master)
+[![Documentation Status](https://readthedocs.org/projects/pyconll/badge/?version=stable)](https://pyconll.readthedocs.io/en/stable)
+[![Version](https://img.shields.io/github/v/release/pyconll/pyconll)](https://github.com/pyconll/pyconll/releases)
+[![gitter](https://badges.gitter.im/pyconll/pyconll.svg)](https://gitter.im/pyconll/pyconll?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
+
+## pyconll
+
+*Easily work with **CoNLL** files using the familiar syntax of **python**.*
+
+<img src="res/logo.svg" width="256px" height="256px">
+
+##### Links
+- [Homepage](https://pyconll.github.io)
+- [Documentation](https://pyconll.readthedocs.io/)
+
+
+### Installation
+
+As with most python packages, simply use `pip` to install from PyPi.
+
+```
+pip install pyconll
+```
+
+`pyconll` is also available as a conda package on the `pyconll` channel. Only packages 2.2.0 and newer are available on conda at the moment.
+
+```
+conda install -c pyconll pyconll
+```
+
+pyconll supports Python 3.10 or newer. In general, pyconll will focus development efforts on officially supported python versions.
+
+
+### Use
+
+This tool is intended to be a **minimal**, **low level**, **expressive** and **pragmatic** library in a widely used programming language. pyconll creates a thin API on top of raw CoNLL annotations that is simple and intuitive.
+
+It offers the following features:
+* Regular CI testing and validation against all UD v2.x versions.
+* A strong domain model that includes CoNLL sources, Sentences, Tokens, Trees, etc.
+* A typed API for better development experience and better semantics.
+* A focus on usability and simplicity in design (no dependencies)
+* Performance optimizations for a smooth development workflow no matter the dataset size (performs about 25%-35% faster than other comparable packages)
+
+See the following code example to understand the basics of the API.
+
+```python
+# This snippet finds sentences where a token marked with part of speech 'AUX' are
+# governed by a NOUN. For example, in French this is a less common construction
+# and we may want to validate these examples because we have previously found some
+# problematic examples of this construction.
+import pyconll
+
+train = pyconll.load_from_file('./ud/train.conllu')
+
+review_sentences = []
+
+# Conll objects are iterable over their sentences, and sentences are iterable
+# over their tokens. Sentences also de/serialize comment information.
+for sentence in train:
+    for token in sentence:
+
+        # Tokens have attributes such as upos, head, id, deprel, etc, and sentences
+        # can be indexed by a token's id. We must check that the token is not the
+        # root token, whose id, '0', cannot be looked up.
+        if token.upos == 'AUX' and (token.head != '0' and sentence[token.head].upos == 'NOUN'):
+            review_sentences.append(sentence)
+
+print('Review the following sentences:')
+for sent in review_sentences:
+    print(sent.id)
+```
+
+A full definition of the API can be found in the [documentation](https://pyconll.readthedocs.io/) or use the [quick start](https://pyconll.readthedocs.io/en/stable/starting.html) guide for a focused introduction.
+
+
+### Uses and Limitations
+
+This package edits CoNLL-U annotations. This does not include the annotated text itself. Word forms on Tokens are not editable and Sentence Tokens cannot be reassigned or reordered. `pyconll` focuses on editing CoNLL-U annotation rather than creating it or changing the underlying text that is annotated. If there is interest in this functionality area, please create a GitHub issue for more visibility.
+
+This package also is only validated against the CoNLL-U format. The CoNLL and CoNLL-X format are not supported, but are very similar. I originally intended to support these formats as well, but their format is not as well defined as CoNLL-U so they are not included. Please create an issue for visibility if this feature interests you.
+
+Lastly, linguistic data can often be very large and this package attempts to keep that in mind. pyconll provides methods for creating in memory conll objects along with an iterate only version in case a corpus is too large to store in memory (the size of the memory structure is several times larger than the actual corpus file). The iterate only version can parse upwards of 100,000 words per second on a 16gb ram machine, so for most datasets to be used on a local dev machine, this package will perform well. The 2.2.0 release also improves parse time and memory footprint by about 25%!
+
+
+### Contributing
+
+Contributions to this project are welcome and encouraged! If you are unsure how to contribute, here is a [guide](https://help.github.com/en/articles/creating-a-pull-request-from-a-fork) from Github explaining the basic workflow. After cloning this repo, please run `pip install -r requirements.txt` to properly setup locally. Some of these tools like yapf, pylint, and mypy do not have to be run locally, but CI builds will fail without their successful running. Some other release dependencies like twine and sphinx are also installed.
+
+For packaging new versions, use setuptools version 24.2.0 or greater for creating the appropriate packaging that recognizes the `python_requires` metadata. Final packaging and release is now done with Github actions so this is less of a concern.
+
+
+#### README and CHANGELOG
+
+When changing either of these files, please change the Markdown version and run ``make gendocs`` so that the other versions stay in sync.
+
+#### Release Checklist
+
+Below enumerates the general release process explicitly. This section is for internal use and most people do not have to worry about this. First note, that the dev branch is always a direct extension of master with the latest changes since the last release. That is, it is essentially a staging release branch.
+
+* Change the version in `pyconll/_version.py` appropriately.
+* Merge dev into master **locally**. Github does not offer a fast forward merge and explicitly uses --no-ff. So to keep the linear nature of changes, merge locally to fast forward. This is assuming that the dev branch looks good on CI tests which do not automatically run in this situation.
+* Push the master branch. This should start some CI tests specifically for master. After validating these results, create a tag corresponding to the next version number and push the tag.
+* Create a new release from this tag from the [Releases page](https://github.com/pyconll/pyconll/releases). On creating this release, two workflows will start. One releases to pypi, and the other releases to conda.
+* Validate these workflows pass, and the package is properly released on both platforms.

{pyconll-3.2.0 → pyconll-3.3.0}/pyconll.egg-info/SOURCES.txt

@@ -1,8 +1,6 @@
 LICENSE
 MANIFEST.in
-README
 README.md
-README.rst
 setup.py
 pyconll/__init__.py
 pyconll/_parser.py
@@ -22,4 +20,7 @@ pyconll/tree/tree.py
 pyconll/unit/__init__.py
 pyconll/unit/conll.py
 pyconll/unit/sentence.py
-pyconll/unit/token.py
+pyconll/unit/token.py
+tests/test_conllable.py
+tests/test_load.py
+tests/test_util.py

{pyconll-3.2.0 → pyconll-3.3.0}/setup.py

@@ -22,20 +22,20 @@ setup(
     packages = ['pyconll', 'pyconll.unit', 'pyconll.tree'],
     version = parse.package_version(make_relative('pyconll/_version.py')),
     description = 'Read and manipulate CoNLL files',
-    long_description = make_relative('README.rst').read_text(),
+    long_description = make_relative('README.md').read_text(),
+    long_description_content_type="text/markdown",
     author = 'Matias Grioni',
     author_email = 'matgrioni@gmail.com',
     url = 'https://github.com/pyconll/pyconll',
     license = 'MIT',
     keywords = ['nlp', 'conllu', 'conll', 'universal dependencies'],
-    python_requires = '~=3.8',
+    python_requires = '~=3.10',
     package_data = { 'pyconll': ['py.typed'] },
     classifiers = [
         'Development Status :: 5 - Production/Stable',
         'Intended Audience :: Developers',
         'Intended Audience :: Education',
         'Intended Audience :: Science/Research',
-        'License :: OSI Approved :: MIT License',
         'Programming Language :: Python :: 3 :: Only',
         'Programming Language :: Python :: Implementation :: CPython',
         'Topic :: Scientific/Engineering',