SinaTools 0.1.1__tar.gz

SinaTools-0.1.1/AUTHORS.rst

@@ -0,0 +1,13 @@
+=======
+Credits
+=======
+
+Development Lead
+----------------
+
+* SinaLab <sina.institute.bzu@gmail.com>
+
+Contributors
+------------
+
+None yet. Why not be the first?

SinaTools-0.1.1/CONTRIBUTING.rst

@@ -0,0 +1,128 @@
+.. highlight:: shell
+
+============
+Contributing
+============
+
+Contributions are welcome, and they are greatly appreciated! Every little bit
+helps, and credit will always be given.
+
+You can contribute in many ways:
+
+Types of Contributions
+----------------------
+
+Report Bugs
+~~~~~~~~~~~
+
+Report bugs at https://github.com/SinaLab/nlptools/issues.
+
+If you are reporting a bug, please include:
+
+* Your operating system name and version.
+* Any details about your local setup that might be helpful in troubleshooting.
+* Detailed steps to reproduce the bug.
+
+Fix Bugs
+~~~~~~~~
+
+Look through the GitHub issues for bugs. Anything tagged with "bug" and "help
+wanted" is open to whoever wants to implement it.
+
+Implement Features
+~~~~~~~~~~~~~~~~~~
+
+Look through the GitHub issues for features. Anything tagged with "enhancement"
+and "help wanted" is open to whoever wants to implement it.
+
+Write Documentation
+~~~~~~~~~~~~~~~~~~~
+
+nlptools could always use more documentation, whether as part of the
+official nlptools docs, in docstrings, or even on the web in blog posts,
+articles, and such.
+
+Submit Feedback
+~~~~~~~~~~~~~~~
+
+The best way to send feedback is to file an issue at https://github.com/SinaLab/nlptools/issues.
+
+If you are proposing a feature:
+
+* Explain in detail how it would work.
+* Keep the scope as narrow as possible, to make it easier to implement.
+* Remember that this is a volunteer-driven project, and that contributions
+  are welcome :)
+
+Get Started!
+------------
+
+Ready to contribute? Here's how to set up `nlptools` for local development.
+
+1. Fork the `nlptools` repo on GitHub.
+2. Clone your fork locally::
+
+    $ git clone git@github.com:your_name_here/nlptools.git
+
+3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development::
+
+    $ mkvirtualenv nlptools
+    $ cd nlptools/
+    $ python setup.py develop
+
+4. Create a branch for local development::
+
+    $ git checkout -b name-of-your-bugfix-or-feature
+
+   Now you can make your changes locally.
+
+5. When you're done making changes, check that your changes pass flake8 and the
+   tests, including testing other Python versions with tox::
+
+    $ flake8 nlptools tests
+    $ python setup.py test or pytest
+    $ tox
+
+   To get flake8 and tox, just pip install them into your virtualenv.
+
+6. Commit your changes and push your branch to GitHub::
+
+    $ git add .
+    $ git commit -m "Your detailed description of your changes."
+    $ git push origin name-of-your-bugfix-or-feature
+
+7. Submit a pull request through the GitHub website.
+
+Pull Request Guidelines
+-----------------------
+
+Before you submit a pull request, check that it meets these guidelines:
+
+1. The pull request should include tests.
+2. If the pull request adds functionality, the docs should be updated. Put
+   your new functionality into a function with a docstring, and add the
+   feature to the list in README.rst.
+3. The pull request should work for Python 3.5, 3.6, 3.7 and 3.8, and for PyPy. Check
+   https://travis-ci.com/sina_institute/nlptools/pull_requests
+   and make sure that the tests pass for all supported Python versions.
+
+Tips
+----
+
+To run a subset of tests::
+
+
+    $ python -m unittest tests.test_nlptools
+
+Deploying
+---------
+
+A reminder for the maintainers on how to deploy.
+Make sure all your changes are committed (including an entry in HISTORY.rst).
+Then run::
+
+$ bump2version patch # possible: major / minor / patch
+$ git push
+$ git push --tags
+
+Travis will then deploy to PyPI if tests pass.

SinaTools-0.1.1/HISTORY.rst

@@ -0,0 +1,8 @@
+=======
+History
+=======
+
+0.1.0 (2023-04-15)
+------------------
+
+* First release on PyPI.

SinaTools-0.1.1/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2023, SinaLab
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

SinaTools-0.1.1/MANIFEST.in

@@ -0,0 +1,28 @@
+include AUTHORS.rst
+include CONTRIBUTING.rst
+include HISTORY.rst
+include LICENSE
+include README.rst
+
+recursive-include tests *
+recursive-exclude * __pycache__
+recursive-exclude * *.py[co]
+
+recursive-include docs *.rst conf.py Makefile make.bat *.jpg *.png *.gif
+
+
+include setup.py
+include nlptools/VERSION
+include nlptools/utils/*.py
+include nlptools/arabiner/*.py
+include nlptools/arabert/*.py
+include nlptools/DataDownload/*.py
+include nlptools/morphology/*.py
+include nlptools/salma/*.py
+include nlptools/CLI/arabiner/bin/*.py
+include nlptools/CLI/morphology/*.py
+include nlptools/CLI/salma/*.py
+include nlptools/CLI/utils/*.py
+include nlptools/CLI/DataDownload/*.py
+include tests/*.py
+global-exclude *~

SinaTools-0.1.1/PKG-INFO

@@ -0,0 +1,58 @@
+Metadata-Version: 1.0
+Name: SinaTools
+Version: 0.1.1
+Summary: UNKNOWN
+Home-page: https://github.com/SinaLab/nlptools
+Author: UNKNOWN
+Author-email: UNKNOWN
+License: MIT license
+Description: ========
+        nlptools
+        ========
+        
+        
+        .. image:: https://img.shields.io/pypi/v/nlptools.svg
+                :target: https://pypi.python.org/pypi/SinaTools
+        
+        .. image:: https://img.shields.io/travis/sina_institute/nlptools.svg
+                :target: https://travis-ci.com/sina_institute/SinaTools
+        
+        .. image:: https://readthedocs.org/projects/nlptools/badge/?version=latest
+                :target: https://SinaTools.readthedocs.io/en/latest/?version=latest
+                :alt: Documentation Status
+        
+        
+        
+        
+        Python Boilerplate contains all the boilerplate you need to create a Python package.
+        
+        
+        * Free software: MIT license
+        * Documentation: https://SinaTools.readthedocs.io.
+        
+        
+        Features
+        --------
+        
+        * TODO
+        
+        Credits
+        -------
+        
+        This package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.
+        
+        .. _Cookiecutter: https://github.com/audreyr/cookiecutter
+        .. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage
+        
+        
+        =======
+        History
+        =======
+        
+        0.1.0 (2023-04-15)
+        ------------------
+        
+        * First release on PyPI.
+        
+Keywords: nlptools
+Platform: UNKNOWN

SinaTools-0.1.1/README.rst

@@ -0,0 +1,37 @@
+========
+nlptools
+========
+
+
+.. image:: https://img.shields.io/pypi/v/nlptools.svg
+        :target: https://pypi.python.org/pypi/SinaTools
+
+.. image:: https://img.shields.io/travis/sina_institute/nlptools.svg
+        :target: https://travis-ci.com/sina_institute/SinaTools
+
+.. image:: https://readthedocs.org/projects/nlptools/badge/?version=latest
+        :target: https://SinaTools.readthedocs.io/en/latest/?version=latest
+        :alt: Documentation Status
+
+
+
+
+Python Boilerplate contains all the boilerplate you need to create a Python package.
+
+
+* Free software: MIT license
+* Documentation: https://SinaTools.readthedocs.io.
+
+
+Features
+--------
+
+* TODO
+
+Credits
+-------
+
+This package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.
+
+.. _Cookiecutter: https://github.com/audreyr/cookiecutter
+.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage

SinaTools-0.1.1/SinaTools.egg-info/PKG-INFO

@@ -0,0 +1,58 @@
+Metadata-Version: 1.0
+Name: SinaTools
+Version: 0.1.1
+Summary: UNKNOWN
+Home-page: https://github.com/SinaLab/nlptools
+Author: UNKNOWN
+Author-email: UNKNOWN
+License: MIT license
+Description: ========
+        nlptools
+        ========
+        
+        
+        .. image:: https://img.shields.io/pypi/v/nlptools.svg
+                :target: https://pypi.python.org/pypi/SinaTools
+        
+        .. image:: https://img.shields.io/travis/sina_institute/nlptools.svg
+                :target: https://travis-ci.com/sina_institute/SinaTools
+        
+        .. image:: https://readthedocs.org/projects/nlptools/badge/?version=latest
+                :target: https://SinaTools.readthedocs.io/en/latest/?version=latest
+                :alt: Documentation Status
+        
+        
+        
+        
+        Python Boilerplate contains all the boilerplate you need to create a Python package.
+        
+        
+        * Free software: MIT license
+        * Documentation: https://SinaTools.readthedocs.io.
+        
+        
+        Features
+        --------
+        
+        * TODO
+        
+        Credits
+        -------
+        
+        This package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.
+        
+        .. _Cookiecutter: https://github.com/audreyr/cookiecutter
+        .. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage
+        
+        
+        =======
+        History
+        =======
+        
+        0.1.0 (2023-04-15)
+        ------------------
+        
+        * First release on PyPI.
+        
+Keywords: nlptools
+Platform: UNKNOWN

SinaTools-0.1.1/SinaTools.egg-info/SOURCES.txt

@@ -0,0 +1,191 @@
+AUTHORS.rst
+CONTRIBUTING.rst
+HISTORY.rst
+LICENSE
+MANIFEST.in
+README.rst
+setup.cfg
+setup.py
+SinaTools.egg-info/PKG-INFO
+SinaTools.egg-info/SOURCES.txt
+SinaTools.egg-info/dependency_links.txt
+SinaTools.egg-info/entry_points.txt
+SinaTools.egg-info/not-zip-safe
+SinaTools.egg-info/requires.txt
+SinaTools.egg-info/top_level.txt
+docs/Makefile
+docs/make.bat
+docs/build/_images/download.png
+docs/build/_static/download.png
+docs/build/_static/file.png
+docs/build/_static/minus.png
+docs/build/_static/plus.png
+docs/build/html/_images/SinaLogo.jpg
+docs/build/html/_images/download.png
+docs/build/html/_static/SinaLogo.jpg
+docs/build/html/_static/download.png
+docs/build/html/_static/file.png
+docs/build/html/_static/minus.png
+docs/build/html/_static/plus.png
+docs/source/License.rst
+docs/source/Overview.rst
+docs/source/about.rst
+docs/source/api.rst
+docs/source/authors.rst
+docs/source/cli_tools.rst
+docs/source/conf.py
+docs/source/index.rst
+docs/source/installation.rst
+docs/source/readme.rst
+docs/source/_static/SinaLogo.jpg
+docs/source/_static/download.png
+docs/source/api/DataDownload.rst
+docs/source/api/arabiner.rst
+docs/source/api/morphology.rst
+docs/source/api/salma.rst
+docs/source/api/utils.rst
+docs/source/api/DataDownload/downloader.rst
+docs/source/api/arabiner/bin/infer.rst
+docs/source/api/morphology/morph_analyzer.rst
+docs/source/api/salma/views.rst
+docs/source/api/utils/corpus_tokenizer.rst
+docs/source/api/utils/implication.rst
+docs/source/api/utils/jaccard.rst
+docs/source/api/utils/parser.rst
+docs/source/api/utils/sentence_tokenizer.rst
+docs/source/api/utils/text_transliteration.rst
+docs/source/cli_tools/DataDownload.rst
+docs/source/cli_tools/arabiner.rst
+docs/source/cli_tools/morphology.rst
+docs/source/cli_tools/salma.rst
+docs/source/cli_tools/utils.rst
+docs/source/cli_tools/DataDownload/download_files.rst
+docs/source/cli_tools/DataDownload/get_appdatadir.rst
+docs/source/cli_tools/arabiner/infer.rst
+docs/source/cli_tools/morphology/ALMA_multi_word.rst
+docs/source/cli_tools/morphology/morph_analyzer.rst
+docs/source/cli_tools/salma/salma_tools.rst
+docs/source/cli_tools/utils/arStrip.rst
+docs/source/cli_tools/utils/corpus_tokenizer.rst
+docs/source/cli_tools/utils/implication.rst
+docs/source/cli_tools/utils/jaccard.rst
+docs/source/cli_tools/utils/latin_remove.rst
+docs/source/cli_tools/utils/remove_punc.rst
+docs/source/cli_tools/utils/sentence_tokenizer.rst
+docs/source/cli_tools/utils/text_transliteration.rst
+nlptools/VERSION
+nlptools/__init__.py
+nlptools/environment.yml
+nlptools/install_env.py
+nlptools/nlptools.py
+nlptools/CLI/DataDownload/download_files.py
+nlptools/CLI/arabiner/bin/infer.py
+nlptools/CLI/arabiner/bin/infer2.py
+nlptools/CLI/morphology/ALMA_multi_word.py
+nlptools/CLI/morphology/morph_analyzer.py
+nlptools/CLI/salma/salma_tools.py
+nlptools/CLI/utils/__init__.py
+nlptools/CLI/utils/arStrip.py
+nlptools/CLI/utils/corpus_tokenizer.py
+nlptools/CLI/utils/implication.py
+nlptools/CLI/utils/jaccard.py
+nlptools/CLI/utils/latin_remove.py
+nlptools/CLI/utils/remove_Punc.py
+nlptools/CLI/utils/sentence_tokenizer.py
+nlptools/CLI/utils/text_transliteration.py
+nlptools/DataDownload/__init__.py
+nlptools/DataDownload/downloader.py
+nlptools/arabert/__init__.py
+nlptools/arabert/preprocess.py
+nlptools/arabert/arabert/__init__.py
+nlptools/arabert/arabert/create_classification_data.py
+nlptools/arabert/arabert/create_pretraining_data.py
+nlptools/arabert/arabert/extract_features.py
+nlptools/arabert/arabert/lamb_optimizer.py
+nlptools/arabert/arabert/modeling.py
+nlptools/arabert/arabert/optimization.py
+nlptools/arabert/arabert/run_classifier.py
+nlptools/arabert/arabert/run_pretraining.py
+nlptools/arabert/arabert/run_squad.py
+nlptools/arabert/arabert/tokenization.py
+nlptools/arabert/araelectra/__init__.py
+nlptools/arabert/araelectra/build_openwebtext_pretraining_dataset.py
+nlptools/arabert/araelectra/build_pretraining_dataset.py
+nlptools/arabert/araelectra/build_pretraining_dataset_single_file.py
+nlptools/arabert/araelectra/configure_finetuning.py
+nlptools/arabert/araelectra/configure_pretraining.py
+nlptools/arabert/araelectra/flops_computation.py
+nlptools/arabert/araelectra/run_finetuning.py
+nlptools/arabert/araelectra/run_pretraining.py
+nlptools/arabert/araelectra/finetune/__init__.py
+nlptools/arabert/araelectra/finetune/feature_spec.py
+nlptools/arabert/araelectra/finetune/preprocessing.py
+nlptools/arabert/araelectra/finetune/scorer.py
+nlptools/arabert/araelectra/finetune/task.py
+nlptools/arabert/araelectra/finetune/task_builder.py
+nlptools/arabert/araelectra/model/__init__.py
+nlptools/arabert/araelectra/model/modeling.py
+nlptools/arabert/araelectra/model/optimization.py
+nlptools/arabert/araelectra/model/tokenization.py
+nlptools/arabert/araelectra/pretrain/__init__.py
+nlptools/arabert/araelectra/pretrain/pretrain_data.py
+nlptools/arabert/araelectra/pretrain/pretrain_helpers.py
+nlptools/arabert/araelectra/util/__init__.py
+nlptools/arabert/araelectra/util/training_utils.py
+nlptools/arabert/araelectra/util/utils.py
+nlptools/arabert/aragpt2/__init__.py
+nlptools/arabert/aragpt2/create_pretraining_data.py
+nlptools/arabert/aragpt2/train_bpe_tokenizer.py
+nlptools/arabert/aragpt2/gpt2/__init__.py
+nlptools/arabert/aragpt2/gpt2/lamb_optimizer.py
+nlptools/arabert/aragpt2/gpt2/optimization.py
+nlptools/arabert/aragpt2/gpt2/run_pretraining.py
+nlptools/arabert/aragpt2/grover/__init__.py
+nlptools/arabert/aragpt2/grover/dataloader.py
+nlptools/arabert/aragpt2/grover/modeling.py
+nlptools/arabert/aragpt2/grover/modeling_gpt2.py
+nlptools/arabert/aragpt2/grover/optimization_adafactor.py
+nlptools/arabert/aragpt2/grover/train_tpu.py
+nlptools/arabert/aragpt2/grover/utils.py
+nlptools/arabiner/__init__.py
+nlptools/arabiner/bin/__init__.py
+nlptools/arabiner/bin/eval.py
+nlptools/arabiner/bin/infer.py
+nlptools/arabiner/bin/process.py
+nlptools/arabiner/bin/train.py
+nlptools/arabiner/data/__init__.py
+nlptools/arabiner/data/datasets.py
+nlptools/arabiner/data/transforms.py
+nlptools/arabiner/nn/BaseModel.py
+nlptools/arabiner/nn/BertNestedTagger.py
+nlptools/arabiner/nn/BertSeqTagger.py
+nlptools/arabiner/nn/__init__.py
+nlptools/arabiner/trainers/BaseTrainer.py
+nlptools/arabiner/trainers/BertNestedTrainer.py
+nlptools/arabiner/trainers/BertTrainer.py
+nlptools/arabiner/trainers/__init__.py
+nlptools/arabiner/utils/__init__.py
+nlptools/arabiner/utils/data.py
+nlptools/arabiner/utils/helpers.py
+nlptools/arabiner/utils/metrics.py
+nlptools/morphology/ALMA_multi_word.py
+nlptools/morphology/__init__.py
+nlptools/morphology/charsets.py
+nlptools/morphology/morph_analyzer.py
+nlptools/morphology/settings.py
+nlptools/morphology/tokenizers_words.py
+nlptools/salma/__init__.py
+nlptools/salma/settings.py
+nlptools/salma/views.py
+nlptools/salma/wsd.py
+nlptools/utils/__init__.py
+nlptools/utils/corpus_tokenizer.py
+nlptools/utils/implication.py
+nlptools/utils/jaccard.py
+nlptools/utils/parser.py
+nlptools/utils/readfile.py
+nlptools/utils/sentence_tokenizer.py
+nlptools/utils/text_transliteration.py
+nlptools/utils/utils.py
+tests/__init__.py
+tests/test_nlptools.py

SinaTools-0.1.1/SinaTools.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

SinaTools-0.1.1/SinaTools.egg-info/entry_points.txt

@@ -0,0 +1,18 @@
+[console_scripts]
+arabi_ner = nlptools.CLI.arabiner.bin.infer:main
+arabi_ner2 = nlptools.CLI.arabiner.bin.infer2:main
+install_env = nlptools.install_env:main
+sina_alma_multi_word = nlptools.CLI.morphology.ALMA_multi_word:main
+sina_appdatadir = nlptools.CLI.DataDownload.get_appdatadir:main
+sina_arStrip = nlptools.CLI.utils.arStrip:main
+sina_corpus_tokenizer = nlptools.CLI.utils.corpus_tokenizer:main
+sina_download_files = nlptools.CLI.DataDownload.download_files:main
+sina_implication = nlptools.CLI.utils.implication:main
+sina_jaccard_similarity = nlptools.CLI.utils.jaccard:main
+sina_morph_analyze = nlptools.CLI.morphology.morph_analyzer:main
+sina_remove_latin = nlptools.CLI.utils.latin_remove:main
+sina_remove_punctuation = nlptools.CLI.utils.remove_Punc:main
+sina_salma = nlptools.CLI.salma.salma_tools:main
+sina_sentence_tokenize = nlptools.CLI.utils.sentence_tokenizer:main
+sina_transliterate = nlptools.CLI.utils.text_transliteration:main
+

SinaTools-0.1.1/SinaTools.egg-info/not-zip-safe

@@ -0,0 +1 @@
+

SinaTools-0.1.1/SinaTools.egg-info/requires.txt

@@ -0,0 +1,12 @@
+six
+farasapy
+tqdm
+requests
+regex
+pathlib
+torch==1.13.0
+transformers==4.24.0
+torchtext==0.14.0
+torchvision==0.14.0
+seqeval==1.2.2
+natsort==7.1.1

SinaTools-0.1.1/SinaTools.egg-info/top_level.txt

@@ -0,0 +1 @@
+nlptools

SinaTools-0.1.1/docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SOURCEDIR     = source
+SPHINXOPTS    =
+SPHINXBUILD   = sphinx-build -c $(SOURCEDIR) -c $(SOURCEDIR)/config
+SPHINXPROJ    = nlptools
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+    @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+    @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

SinaTools-0.1.1/docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.http://sphinx-doc.org/
+	exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
+
+:end
+popd