PerSent 1.3.3__tar.gz → 1.5.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- persent-1.5.0/.gitignore +33 -0
- persent-1.5.0/CHANGELOG.md +77 -0
- persent-1.5.0/LICENSE +21 -0
- persent-1.5.0/PKG-INFO +235 -0
- persent-1.5.0/README.fa.md +194 -0
- persent-1.5.0/README.md +193 -0
- persent-1.5.0/pyproject.toml +82 -0
- persent-1.5.0/scripts/convert_legacy_models.py +65 -0
- persent-1.5.0/src/PerSent/CommentAnalyzer.py +178 -0
- persent-1.5.0/src/PerSent/SentimentAnalyzer.py +166 -0
- persent-1.5.0/src/PerSent/__init__.py +46 -0
- persent-1.5.0/src/PerSent/__main__.py +3 -0
- persent-1.5.0/src/PerSent/_models.py +170 -0
- persent-1.5.0/src/PerSent/_stemmer.py +25 -0
- persent-1.5.0/src/PerSent/_text.py +122 -0
- persent-1.5.0/src/PerSent/_utils.py +66 -0
- persent-1.5.0/src/PerSent/_version.py +1 -0
- persent-1.5.0/src/PerSent/cli.py +228 -0
- persent-1.5.0/src/PerSent/emotion.py +322 -0
- persent-1.5.0/src/PerSent/exceptions.py +26 -0
- persent-1.5.0/src/PerSent/models/comment_model.npz +0 -0
- persent-1.5.0/src/PerSent/models/emotion_lexicon.json +1 -0
- persent-1.5.0/src/PerSent/polarity.py +371 -0
- persent-1.5.0/src/PerSent/py.typed +0 -0
- persent-1.5.0/src/PerSent/results.py +146 -0
- persent-1.5.0/tests/conftest.py +59 -0
- persent-1.5.0/tests/test_cli.py +93 -0
- persent-1.5.0/tests/test_emotion.py +127 -0
- persent-1.5.0/tests/test_legacy_comment_analyzer.py +147 -0
- persent-1.5.0/tests/test_legacy_sentiment_analyzer.py +107 -0
- persent-1.5.0/tests/test_package.py +67 -0
- persent-1.5.0/tests/test_polarity.py +166 -0
- persent-1.3.3/MANIFEST.in +0 -2
- persent-1.3.3/PKG-INFO +0 -366
- persent-1.3.3/PerSent/CommentAnalyzer.py +0 -279
- persent-1.3.3/PerSent/SentimentAnalyzer.py +0 -346
- persent-1.3.3/PerSent/__init__.py +0 -10
- persent-1.3.3/PerSent/models/classifier.joblib +0 -0
- persent-1.3.3/PerSent/models/weighted_sentiment_model.joblib +0 -0
- persent-1.3.3/PerSent/models/word2vec.model +0 -0
- persent-1.3.3/PerSent.egg-info/PKG-INFO +0 -366
- persent-1.3.3/PerSent.egg-info/SOURCES.txt +0 -15
- persent-1.3.3/PerSent.egg-info/dependency_links.txt +0 -1
- persent-1.3.3/PerSent.egg-info/requires.txt +0 -8
- persent-1.3.3/PerSent.egg-info/top_level.txt +0 -1
- persent-1.3.3/README.md +0 -334
- persent-1.3.3/setup.cfg +0 -4
- persent-1.3.3/setup.py +0 -37
persent-1.5.0/.gitignore
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Python
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*.egg-info/
|
|
5
|
+
.eggs/
|
|
6
|
+
build/
|
|
7
|
+
dist/
|
|
8
|
+
*.egg
|
|
9
|
+
|
|
10
|
+
# Environments
|
|
11
|
+
.venv/
|
|
12
|
+
venv/
|
|
13
|
+
env/
|
|
14
|
+
|
|
15
|
+
# Tooling
|
|
16
|
+
.pytest_cache/
|
|
17
|
+
.ruff_cache/
|
|
18
|
+
.mypy_cache/
|
|
19
|
+
.coverage
|
|
20
|
+
coverage.xml
|
|
21
|
+
htmlcov/
|
|
22
|
+
.ipynb_checkpoints/
|
|
23
|
+
|
|
24
|
+
# Editors / OS
|
|
25
|
+
.idea/
|
|
26
|
+
.vscode/
|
|
27
|
+
.DS_Store
|
|
28
|
+
Thumbs.db
|
|
29
|
+
|
|
30
|
+
# Locally trained models
|
|
31
|
+
/model/
|
|
32
|
+
/models/
|
|
33
|
+
PerSent/model/
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here.
|
|
4
|
+
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses
|
|
5
|
+
[Semantic Versioning](https://semver.org/).
|
|
6
|
+
|
|
7
|
+
## [1.5.0] - Unreleased
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
- New API with snake_case methods and result objects:
|
|
11
|
+
- `PolarityClassifier`: `pretrained()`, `load()`, `save()`, `predict()`, `predict_many()`, `predict_proba()`,
|
|
12
|
+
`analyze()` / `analyze_many()` (→ `PolarityResult` with per-label probabilities), `fit()` / `fit_csv()`
|
|
13
|
+
(→ `TrainingReport` with accuracy, macro-F1 and per-label metrics), `evaluate()` / `evaluate_csv()`
|
|
14
|
+
(→ `EvaluationReport` with a confusion matrix), `analyze_csv()` (optional probability columns) and `summarize()`.
|
|
15
|
+
- `EmotionAnalyzer`: `pretrained()`, `load()`, `save()`, `analyze()` / `analyze_many()` (→ `EmotionResult` with
|
|
16
|
+
`dominant`, `matched_words`, English names), `scores()`, `dominant()`, `fit_csv()`, `add_lexicon()`, `add_word()`,
|
|
17
|
+
`analyze_csv()` (one column per emotion plus `dominant_emotion`).
|
|
18
|
+
- Training/analysis methods accept a CSV path or a `pandas.DataFrame`; emotions may be given in Persian or English.
|
|
19
|
+
- `persent` command-line tool (`polarity`, `emotion`, `csv`, `train`, `evaluate`), also as `python -m PerSent`.
|
|
20
|
+
- `PolarityResult`, `EmotionResult`, `TrainingReport`, `EvaluationReport`, `EMOTIONS`, `EMOTION_NAMES_EN`, `LABELS`
|
|
21
|
+
exported from the package.
|
|
22
|
+
|
|
23
|
+
### Changed
|
|
24
|
+
- Polarity predictions are computed in batches (faster for CSV files); results are unchanged.
|
|
25
|
+
- Emotion scores are always returned in the same order (شادی, غم, ترس, عصبانیت, تنفر, شگفتی, آرامش).
|
|
26
|
+
|
|
27
|
+
### Deprecated
|
|
28
|
+
- `CommentAnalyzer` and `SentimentAnalyzer` (and their camelCase methods) now emit a `DeprecationWarning`. They
|
|
29
|
+
keep their exact 1.4 behaviour and will be removed in PerSent 2.0. See the migration table in the README.
|
|
30
|
+
|
|
31
|
+
## [1.4.0] - 2026-11-10
|
|
32
|
+
|
|
33
|
+
### Added
|
|
34
|
+
- `pyproject.toml` packaging (hatchling) with a single version source (`src/PerSent/_version.py`).
|
|
35
|
+
- Support for Python 3.12 and 3.13 (requires Python ≥ 3.10).
|
|
36
|
+
- Test suite (pytest), linting (ruff) and GitHub Actions CI on Linux, Windows and macOS.
|
|
37
|
+
- Release workflow that publishes to PyPI with Trusted Publishing.
|
|
38
|
+
- `CommentAnalyzer.analyzeTexts()` for lists of texts.
|
|
39
|
+
- `CommentAnalyzer.train()` parameters `random_state`, `text_column`, `label_column`, `workers`; the split is now stratified.
|
|
40
|
+
- `SentimentAnalyzer.train(reset=...)` to start from an empty model.
|
|
41
|
+
- Exception hierarchy: `PerSentError`, `ModelNotLoadedError`, `ModelFileError`, `DataValidationError`
|
|
42
|
+
(all subclasses of `Exception`).
|
|
43
|
+
- Type hints (`py.typed`) and `scripts/convert_legacy_models.py`.
|
|
44
|
+
- Vendored hazm 0.9's suffix stemmer (MIT), so preprocessing — and therefore predictions — stay the same with
|
|
45
|
+
hazm 0.12 on Python 3.12+ (whose new stemmer changed ~10 % of predictions).
|
|
46
|
+
|
|
47
|
+
### Changed
|
|
48
|
+
- Pre-trained models are stored in pickle-free formats (`comment_model.npz`, `emotion_lexicon.json`).
|
|
49
|
+
Predictions are identical to 1.3.3; the wheel shrank from ~27 MB to ~14 MB.
|
|
50
|
+
- `model_dir` now defaults to `None`: `loadModel()` loads the bundled model and `train()` only saves when a
|
|
51
|
+
`model_dir` is set. Legacy 1.3.x model files in an explicit `model_dir` still load (with a `DeprecationWarning`).
|
|
52
|
+
- Status messages use `logging` (logger `"PerSent"`) instead of `print`.
|
|
53
|
+
- Removed the `numpy<2` / `scipy<1.14` pins that made the package uninstallable on Python 3.12+.
|
|
54
|
+
- Repository layout moved to `src/`.
|
|
55
|
+
|
|
56
|
+
### Fixed
|
|
57
|
+
- **Security:** `loadModel()` no longer loads (pickle) files from `./PerSent/model` in the current working
|
|
58
|
+
directory, and creating an analyzer no longer creates that folder.
|
|
59
|
+
- `SentimentAnalyzer.train()` failed with its default arguments on the bundled dataset (`emotion_col` default).
|
|
60
|
+
- `CommentAnalyzer` deleted all upper-case Latin letters ("iPhone" → "i hone") due to a regex range bug.
|
|
61
|
+
- `CommentAnalyzer.train(vector_size=...)` crashed for any size other than 100.
|
|
62
|
+
- Empty labels in the training CSV crashed training instead of being treated as `no_idea`.
|
|
63
|
+
- `SentimentAnalyzer.loadLex()` crashed when columns were given by name.
|
|
64
|
+
- Empty or digit-only texts were classified as `recommended`; they now return `no_idea`.
|
|
65
|
+
- Missing values (NaN) in CSV text columns were analyzed as the word "nan".
|
|
66
|
+
- With hazm >= 0.11, every text re-created hazm's `WordTokenizer` (~0.3 s per text); one instance is now reused.
|
|
67
|
+
|
|
68
|
+
### Removed
|
|
69
|
+
- `setup.py`, `setup.cfg`, `MANIFEST.in`, `requirements.txt`, committed `build/` output and `PKG-INFO`.
|
|
70
|
+
- Draft JOSS paper files (`paper.md`, `paper.bib`).
|
|
71
|
+
|
|
72
|
+
## [1.3.3] - 2025-08-22
|
|
73
|
+
- Last release with the original `setup.py` layout and pickle-based models.
|
|
74
|
+
|
|
75
|
+
[1.5.0]: https://github.com/RezaGooner/PerSent/compare/v1.4.0...HEAD
|
|
76
|
+
[1.4.0]: https://github.com/RezaGooner/PerSent/compare/088b919...v1.4.0
|
|
77
|
+
[1.3.3]: https://pypi.org/project/PerSent/1.3.3/
|
persent-1.5.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 𝐑𝐞𝐳𝐚 𝐀𝐬𝐚𝐝𝐢
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
persent-1.5.0/PKG-INFO
ADDED
|
@@ -0,0 +1,235 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: PerSent
|
|
3
|
+
Version: 1.5.0
|
|
4
|
+
Summary: Persian sentiment and emotion analysis toolkit
|
|
5
|
+
Project-URL: Homepage, https://github.com/RezaGooner/PerSent
|
|
6
|
+
Project-URL: Source, https://github.com/RezaGooner/PerSent
|
|
7
|
+
Project-URL: Issues, https://github.com/RezaGooner/PerSent/issues
|
|
8
|
+
Project-URL: Changelog, https://github.com/RezaGooner/PerSent/blob/main/CHANGELOG.md
|
|
9
|
+
Author-email: Reza Asadi <RezaAsadiProgrammer@Gmail.com>
|
|
10
|
+
License-Expression: MIT
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Keywords: emotion-detection,farsi,nlp,persian,sentiment-analysis,text-classification
|
|
13
|
+
Classifier: Development Status :: 4 - Beta
|
|
14
|
+
Classifier: Intended Audience :: Developers
|
|
15
|
+
Classifier: Intended Audience :: Science/Research
|
|
16
|
+
Classifier: Natural Language :: Persian
|
|
17
|
+
Classifier: Operating System :: OS Independent
|
|
18
|
+
Classifier: Programming Language :: Python :: 3
|
|
19
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
22
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
23
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
24
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
25
|
+
Classifier: Topic :: Text Processing :: Linguistic
|
|
26
|
+
Classifier: Typing :: Typed
|
|
27
|
+
Requires-Python: >=3.10
|
|
28
|
+
Requires-Dist: gensim>=4.3.2
|
|
29
|
+
Requires-Dist: hazm>=0.9.3
|
|
30
|
+
Requires-Dist: joblib>=1.1
|
|
31
|
+
Requires-Dist: numpy>=1.24
|
|
32
|
+
Requires-Dist: pandas>=1.5
|
|
33
|
+
Requires-Dist: scikit-learn>=1.2
|
|
34
|
+
Requires-Dist: tqdm>=4.62
|
|
35
|
+
Provides-Extra: dev
|
|
36
|
+
Requires-Dist: build; extra == 'dev'
|
|
37
|
+
Requires-Dist: pytest-cov; extra == 'dev'
|
|
38
|
+
Requires-Dist: pytest>=7; extra == 'dev'
|
|
39
|
+
Requires-Dist: ruff>=0.6; extra == 'dev'
|
|
40
|
+
Requires-Dist: twine; extra == 'dev'
|
|
41
|
+
Description-Content-Type: text/markdown
|
|
42
|
+
|
|
43
|
+
# PerSent (Persian Sentiment Analyzer)
|
|
44
|
+
|
|
45
|
+
[](https://pypi.org/project/PerSent/)
|
|
46
|
+
[](https://pypi.org/project/PerSent/)
|
|
47
|
+
[](https://github.com/RezaGooner/PerSent/actions/workflows/ci.yml)
|
|
48
|
+
[](LICENSE)
|
|
49
|
+
[](README.fa.md)
|
|
50
|
+
|
|
51
|
+

|
|
52
|
+
|
|
53
|
+
PerSent is a lightweight Python library and command-line tool for **sentiment and emotion analysis of Persian (Farsi) text**.
|
|
54
|
+
|
|
55
|
+
- **`PolarityClassifier`** labels product/service reviews as `recommended`, `not_recommended` or `no_idea`, with probabilities.
|
|
56
|
+
- **`EmotionAnalyzer`** scores seven emotions with a weighted lexicon:
|
|
57
|
+
شادی (happiness), غم (sadness), ترس (fear), عصبانیت (anger), تنفر (disgust), شگفتی (surprise), آرامش (calmness).
|
|
58
|
+
- Pre-trained models included (no download at runtime), CSV in/out, train and evaluate your own models.
|
|
59
|
+
|
|
60
|
+
## Installation
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
pip install PerSent
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Requires Python 3.10–3.13.
|
|
67
|
+
|
|
68
|
+
## Quick start
|
|
69
|
+
|
|
70
|
+
### Review polarity
|
|
71
|
+
|
|
72
|
+
```python
|
|
73
|
+
from PerSent import PolarityClassifier
|
|
74
|
+
|
|
75
|
+
clf = PolarityClassifier.pretrained()
|
|
76
|
+
|
|
77
|
+
clf.predict("کیفیت عالی داشت") # 'recommended'
|
|
78
|
+
clf.predict_proba("کیفیت عالی داشت") # {'not_recommended': 0.13, 'recommended': 0.74, 'no_idea': 0.13}
|
|
79
|
+
|
|
80
|
+
result = clf.analyze("افتضاح بود")
|
|
81
|
+
result.label, result.confidence # ('not_recommended', 0.9998)
|
|
82
|
+
|
|
83
|
+
clf.predict_many(["خوب بود", "پولم هدر رفت"]) # batch
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
### Emotions
|
|
87
|
+
|
|
88
|
+
```python
|
|
89
|
+
from PerSent import EmotionAnalyzer
|
|
90
|
+
|
|
91
|
+
emo = EmotionAnalyzer.pretrained()
|
|
92
|
+
result = emo.analyze("امتحانم رو خراب کردم. احساس میکنم یک شکست خوردهی تمام عیارم.")
|
|
93
|
+
|
|
94
|
+
result.scores # {'شادی': 0.0, 'غم': 36.0, 'ترس': 28.0, 'عصبانیت': 36.0, ...}
|
|
95
|
+
result.dominant # 'غم'
|
|
96
|
+
result.matched_words # lexicon words that produced the scores
|
|
97
|
+
result.scores_en() # {'happiness': 0.0, 'sadness': 36.0, 'fear': 28.0, 'anger': 36.0, ...}
|
|
98
|
+
result.to_dict(lang="en")
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
### CSV files
|
|
102
|
+
|
|
103
|
+
```python
|
|
104
|
+
clf.analyze_csv("comments.csv", "results.csv", text_column="نظرات", # name or 0-based / negative index
|
|
105
|
+
include_scores=True, summary_csv="summary.csv")
|
|
106
|
+
|
|
107
|
+
emo.analyze_csv("texts.csv", "emotions.csv", text_column="text", lang="en") # one column per emotion
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Both methods also accept a `pandas.DataFrame` and return the resulting DataFrame. Output files are UTF-8 with BOM, so
|
|
111
|
+
they open correctly in Excel.
|
|
112
|
+
|
|
113
|
+
### Command line
|
|
114
|
+
|
|
115
|
+
```bash
|
|
116
|
+
persent polarity "کیفیت عالی داشت" "افتضاح بود" --proba
|
|
117
|
+
echo "خیلی خوشحالم" | persent emotion --lang en
|
|
118
|
+
persent emotion --file texts.txt --json # one JSON object per line
|
|
119
|
+
persent csv comments.csv -o results.csv -c نظرات --scores --summary summary.csv
|
|
120
|
+
persent csv texts.csv -o emotions.csv --task emotion -c text
|
|
121
|
+
persent train polarity train.csv -o my_model/ --seed 42
|
|
122
|
+
persent evaluate test.csv -m my_model/
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
`python -m PerSent ...` works as well. Run `persent <command> --help` for all options.
|
|
126
|
+
|
|
127
|
+
## Training, evaluating and saving models
|
|
128
|
+
|
|
129
|
+
```python
|
|
130
|
+
from PerSent import PolarityClassifier, EmotionAnalyzer
|
|
131
|
+
|
|
132
|
+
clf = PolarityClassifier()
|
|
133
|
+
report = clf.fit_csv("dataset/train.csv", random_state=42) # columns: body, recommendation_status
|
|
134
|
+
print(report) # Trained on ... texts, tested on ...: accuracy=..., macro-F1=...
|
|
135
|
+
clf.save("my_model/") # -> my_model/comment_model.npz
|
|
136
|
+
clf = PolarityClassifier.load("my_model/")
|
|
137
|
+
print(clf.evaluate_csv("test.csv"))
|
|
138
|
+
|
|
139
|
+
emo = EmotionAnalyzer.pretrained() # or EmotionAnalyzer() to start empty
|
|
140
|
+
emo.fit_csv("dataset/emotion_dataset.csv") # columns: text, sentiment, weight
|
|
141
|
+
emo.add_lexicon("my_words.csv") # word, emotion, weight
|
|
142
|
+
emo.add_word("سرمست", "happiness", weight=1.5) # single words only
|
|
143
|
+
emo.save("my_emotions.json")
|
|
144
|
+
emo = EmotionAnalyzer.load("my_emotions.json")
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
Models are stored as plain `.npz` / `.json` files — loading them never executes code.
|
|
148
|
+
|
|
149
|
+
## API reference
|
|
150
|
+
|
|
151
|
+
### `PolarityClassifier`
|
|
152
|
+
|
|
153
|
+
| Member | Description |
|
|
154
|
+
|---|---|
|
|
155
|
+
| `PolarityClassifier.pretrained()` | Bundled model. |
|
|
156
|
+
| `PolarityClassifier.load(path)` / `save(path)` | Directory (`comment_model.npz`) or `.npz` file. 1.x pickle folders load with a warning. |
|
|
157
|
+
| `predict(text)` / `predict_many(texts)` | Label(s). Texts without letters return `no_idea`. |
|
|
158
|
+
| `predict_proba(text)` | `{label: probability}`. |
|
|
159
|
+
| `analyze(text)` / `analyze_many(texts)` | `PolarityResult(text, label, scores)` with `.confidence`, `.label_fa`, `.to_dict()`. |
|
|
160
|
+
| `fit(texts, labels, *, vector_size=100, window=5, min_count=1, epochs=5, test_size=0.2, random_state=None, C=1.0, ...)` | Train Word2Vec + logistic regression. Returns a `TrainingReport` (accuracy, macro-F1, per-label metrics on a stratified hold-out split). |
|
|
161
|
+
| `fit_csv(data, text_column="body", label_column="recommendation_status", **fit_kwargs)` | `fit` on a CSV path or DataFrame. |
|
|
162
|
+
| `evaluate(texts, labels)` / `evaluate_csv(data, ...)` | `EvaluationReport` with accuracy, macro-F1, per-label metrics and confusion matrix. |
|
|
163
|
+
| `analyze_csv(input, output=None, *, text_column=0, output_column="sentiment", include_scores=False, summary_csv=None)` | Analyze a column; returns a DataFrame. |
|
|
164
|
+
|
|
165
|
+
### `EmotionAnalyzer`
|
|
166
|
+
|
|
167
|
+
| Member | Description |
|
|
168
|
+
|---|---|
|
|
169
|
+
| `EmotionAnalyzer.pretrained()` | Bundled lexicon. |
|
|
170
|
+
| `EmotionAnalyzer.load(path)` / `save(path)` | `.json` file or directory. |
|
|
171
|
+
| `analyze(text)` / `analyze_many(texts)` | `EmotionResult(text, scores, matched_words)` with `.dominant`, `.dominant_en`, `.scores_en()`, `.to_dict(lang)`. |
|
|
172
|
+
| `scores(text)` / `dominant(text)` | Shortcuts. |
|
|
173
|
+
| `fit_csv(data, text_column="text", emotion_column=None, weight_column="weight", *, reset=False)` | Learn emotion words from labelled texts (merged into the current lexicon unless `reset=True`). |
|
|
174
|
+
| `add_lexicon(data, word_column=0, emotion_column=1, weight_column=2)` | Add a word/emotion/weight table. Emotions may be Persian or English names. |
|
|
175
|
+
| `add_word(word, emotion, weight=1.0)` | Add a single word. |
|
|
176
|
+
| `analyze_csv(input, output=None, *, text_column="text", expand=True, lang="fa")` | One column per emotion plus `dominant_emotion`. |
|
|
177
|
+
|
|
178
|
+
Errors derive from `PerSent.PerSentError`: `ModelNotLoadedError`, `ModelFileError`, `DataValidationError`
|
|
179
|
+
(which is also a `ValueError`). Messages are logged with the standard `logging` module (logger `"PerSent"`).
|
|
180
|
+
|
|
181
|
+
## Model quality and limitations
|
|
182
|
+
|
|
183
|
+
- The bundled polarity model reaches about **65 % accuracy** (macro-F1 ≈ 0.64) on a 10 000-review sample of
|
|
184
|
+
[`dataset/train.csv`](dataset/train.csv) — three balanced classes, so chance level is 33 %. Better models are planned.
|
|
185
|
+
- The emotion analyzer is lexicon-based and does **not handle negation** yet ("خوشحال نیستم" is scored as happiness).
|
|
186
|
+
- hazm's stop-word list contains opinion words such as "خوب", "بد" and "عالی", which are removed before
|
|
187
|
+
classification; short reviews made only of such words are therefore unreliable.
|
|
188
|
+
|
|
189
|
+
A larger training set is available [here](https://github.com/RezaGooner/Sentiment-Survey-Analyzer/tree/main/Dataset/big_train).
|
|
190
|
+
|
|
191
|
+
## Upgrading from 1.x
|
|
192
|
+
|
|
193
|
+
`CommentAnalyzer` and `SentimentAnalyzer` still work exactly as before but emit a `DeprecationWarning`; they will be
|
|
194
|
+
removed in PerSent 2.0.
|
|
195
|
+
|
|
196
|
+
| 1.x | 1.5+ |
|
|
197
|
+
|---|---|
|
|
198
|
+
| `a = CommentAnalyzer(); a.loadModel()` | `clf = PolarityClassifier.pretrained()` |
|
|
199
|
+
| `a.analyzeText(t)` | `clf.predict(t)` (or `clf.analyze(t)` for probabilities) |
|
|
200
|
+
| `a.analyzeTexts(ts)` | `clf.predict_many(ts)` |
|
|
201
|
+
| `a.train(csv)` → accuracy | `clf.fit_csv(csv)` → `TrainingReport` |
|
|
202
|
+
| `a.saveModel()` / `a.loadModel()` with `model_dir` | `clf.save(dir)` / `PolarityClassifier.load(dir)` |
|
|
203
|
+
| `a.analyzeCSV(i, o, s, col)` → DataFrame or `None` | `clf.analyze_csv(i, o, text_column=col, summary_csv=s)` (raises on error) |
|
|
204
|
+
| `s = SentimentAnalyzer(); s.loadModel()` | `emo = EmotionAnalyzer.pretrained()` |
|
|
205
|
+
| `s.analyzeText(t)` | `emo.analyze(t).scores` |
|
|
206
|
+
| `s.train(csv)` → `{"status": ...}` | `emo.fit_csv(csv)` (raises on error) |
|
|
207
|
+
| `s.loadLex(csv)` | `emo.add_lexicon(csv)` |
|
|
208
|
+
| `s.saveModel(name)` / `s.loadModel(name)` | `emo.save(path)` / `EmotionAnalyzer.load(path)` |
|
|
209
|
+
| `s.analyzeCSV(i, o)` → `True` / `False` | `emo.analyze_csv(i, o)` |
|
|
210
|
+
|
|
211
|
+
See [CHANGELOG.md](CHANGELOG.md) for all changes.
|
|
212
|
+
|
|
213
|
+
## Development
|
|
214
|
+
|
|
215
|
+
```bash
|
|
216
|
+
git clone https://github.com/RezaGooner/PerSent.git
|
|
217
|
+
cd PerSent
|
|
218
|
+
pip install -e ".[dev]"
|
|
219
|
+
pytest
|
|
220
|
+
ruff check . && ruff format --check .
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
## Contributing
|
|
224
|
+
|
|
225
|
+
Bug reports, ideas and pull requests are welcome — see [CONTRIBUTING](.github/CONTRIBUTING.md).
|
|
226
|
+
|
|
227
|
+
- [Open an issue](https://github.com/RezaGooner/PerSent/issues/new/choose)
|
|
228
|
+
- [Fork and send a pull request](https://github.com/RezaGooner/PerSent/fork)
|
|
229
|
+
- Email: `RezaAsadiProgrammer@gmail.com` · Telegram: `@RezaGooner`
|
|
230
|
+
|
|
231
|
+
The project grew out of [Sentiment-Survey-Analyzer](https://github.com/RezaGooner/Sentiment-Survey-Analyzer).
|
|
232
|
+
|
|
233
|
+
## License
|
|
234
|
+
|
|
235
|
+
[MIT](LICENSE)
|
|
@@ -0,0 +1,194 @@
|
|
|
1
|
+
<div dir="rtl">
|
|
2
|
+
|
|
3
|
+
# PerSent پرسنت
|
|
4
|
+
|
|
5
|
+
[](https://pypi.org/project/PerSent/)
|
|
6
|
+
[](https://pypi.org/project/PerSent/)
|
|
7
|
+
[](https://github.com/RezaGooner/PerSent/actions/workflows/ci.yml)
|
|
8
|
+
[](README.md)
|
|
9
|
+
|
|
10
|
+

|
|
11
|
+
|
|
12
|
+
پرسنت (Persian Sentiment Analyzer) یک کتابخانهٔ سبک پایتون و ابزار خط فرمان برای **تحلیل احساسات و هیجانهای متن فارسی** است.
|
|
13
|
+
|
|
14
|
+
- **`PolarityClassifier`**: نظر کاربران دربارهٔ کالا یا خدمات را همراه با احتمال هر دسته در یکی از سه دستهٔ `recommended` (پیشنهاد میکنم)، `not_recommended` (پیشنهاد نمیکنم) و `no_idea` (نظری ندارم) قرار میدهد.
|
|
15
|
+
- **`EmotionAnalyzer`**: درصد هفت حس شادی، غم، ترس، عصبانیت، تنفر، شگفتی و آرامش را با یک واژهنامهٔ وزندار محاسبه میکند.
|
|
16
|
+
- مدلهای آماده داخل خود بسته هستند و هنگام اجرا چیزی دانلود نمیشود. ورودی و خروجی CSV، و آموزش و ارزیابی مدل با دادههای خودتان هم پشتیبانی میشود.
|
|
17
|
+
|
|
18
|
+
## نصب
|
|
19
|
+
|
|
20
|
+
</div>
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
pip install PerSent
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
<div dir="rtl">
|
|
27
|
+
|
|
28
|
+
به پایتون ۳.۱۰ تا ۳.۱۳ نیاز دارد.
|
|
29
|
+
|
|
30
|
+
## شروع سریع
|
|
31
|
+
|
|
32
|
+
### تحلیل نظرات
|
|
33
|
+
|
|
34
|
+
</div>
|
|
35
|
+
|
|
36
|
+
```python
|
|
37
|
+
from PerSent import PolarityClassifier
|
|
38
|
+
|
|
39
|
+
clf = PolarityClassifier.pretrained()
|
|
40
|
+
|
|
41
|
+
clf.predict("کیفیت عالی داشت") # 'recommended'
|
|
42
|
+
clf.predict_proba("کیفیت عالی داشت") # {'not_recommended': 0.13, 'recommended': 0.74, 'no_idea': 0.13}
|
|
43
|
+
|
|
44
|
+
result = clf.analyze("افتضاح بود")
|
|
45
|
+
result.label, result.confidence # ('not_recommended', 0.9998)
|
|
46
|
+
result.label_fa # 'پیشنهاد نمیکنم'
|
|
47
|
+
|
|
48
|
+
clf.predict_many(["خوب بود", "پولم هدر رفت"]) # چند متن با هم
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
<div dir="rtl">
|
|
52
|
+
|
|
53
|
+
### تحلیل احساس
|
|
54
|
+
|
|
55
|
+
</div>
|
|
56
|
+
|
|
57
|
+
```python
|
|
58
|
+
from PerSent import EmotionAnalyzer
|
|
59
|
+
|
|
60
|
+
emo = EmotionAnalyzer.pretrained()
|
|
61
|
+
result = emo.analyze("امتحانم رو خراب کردم. احساس میکنم یک شکست خوردهی تمام عیارم.")
|
|
62
|
+
|
|
63
|
+
result.scores # {'شادی': 0.0, 'غم': 36.0, 'ترس': 28.0, 'عصبانیت': 36.0, ...}
|
|
64
|
+
result.dominant # 'غم'
|
|
65
|
+
result.matched_words # کلمههایی از واژهنامه که در متن پیدا شدند
|
|
66
|
+
result.scores_en() # همان امتیازها با نام انگلیسی احساسها
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
<div dir="rtl">
|
|
70
|
+
|
|
71
|
+
### فایل CSV
|
|
72
|
+
|
|
73
|
+
</div>
|
|
74
|
+
|
|
75
|
+
```python
|
|
76
|
+
clf.analyze_csv("comments.csv", "results.csv", text_column="نظرات", # نام ستون یا شمارهٔ آن (از ۰)
|
|
77
|
+
include_scores=True, summary_csv="summary.csv")
|
|
78
|
+
|
|
79
|
+
emo.analyze_csv("texts.csv", "emotions.csv", text_column="text") # یک ستون برای هر احساس
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
<div dir="rtl">
|
|
83
|
+
|
|
84
|
+
هر دو متد بهجای مسیر فایل، `DataFrame` پانداس هم میپذیرند و نتیجه را به شکل DataFrame برمیگردانند. فایلهای خروجی با کدگذاری UTF-8 همراه BOM ذخیره میشوند تا در Excel درست نمایش داده شوند.
|
|
85
|
+
|
|
86
|
+
### خط فرمان
|
|
87
|
+
|
|
88
|
+
</div>
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
persent polarity "کیفیت عالی داشت" "افتضاح بود" --proba
|
|
92
|
+
echo "خیلی خوشحالم" | persent emotion
|
|
93
|
+
persent emotion --file texts.txt --json
|
|
94
|
+
persent csv comments.csv -o results.csv -c نظرات --scores --summary summary.csv
|
|
95
|
+
persent csv texts.csv -o emotions.csv --task emotion -c text
|
|
96
|
+
persent train polarity train.csv -o my_model/ --seed 42
|
|
97
|
+
persent evaluate test.csv -m my_model/
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
<div dir="rtl">
|
|
101
|
+
|
|
102
|
+
دستور `python -m PerSent ...` هم همین کار را میکند. برای دیدن همهٔ گزینهها `persent <command> --help` را اجرا کنید.
|
|
103
|
+
|
|
104
|
+
## آموزش، ارزیابی و ذخیرهٔ مدل
|
|
105
|
+
|
|
106
|
+
</div>
|
|
107
|
+
|
|
108
|
+
```python
|
|
109
|
+
from PerSent import PolarityClassifier, EmotionAnalyzer
|
|
110
|
+
|
|
111
|
+
clf = PolarityClassifier()
|
|
112
|
+
report = clf.fit_csv("dataset/train.csv", random_state=42) # ستونها: body و recommendation_status
|
|
113
|
+
print(report) # دقت و macro-F1 روی دادهٔ آزمون
|
|
114
|
+
clf.save("my_model/")
|
|
115
|
+
clf = PolarityClassifier.load("my_model/")
|
|
116
|
+
print(clf.evaluate_csv("test.csv"))
|
|
117
|
+
|
|
118
|
+
emo = EmotionAnalyzer.pretrained() # یا EmotionAnalyzer() برای شروع از صفر
|
|
119
|
+
emo.fit_csv("dataset/emotion_dataset.csv") # ستونها: text، sentiment و weight
|
|
120
|
+
emo.add_lexicon("my_words.csv") # کلمه، حس، وزن
|
|
121
|
+
emo.add_word("سرمست", "شادی", weight=1.5) # فقط تککلمه
|
|
122
|
+
emo.save("my_emotions.json")
|
|
123
|
+
emo = EmotionAnalyzer.load("my_emotions.json")
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
<div dir="rtl">
|
|
127
|
+
|
|
128
|
+
مدلها به شکل فایلهای ساده `.npz` و `.json` ذخیره میشوند و بارگذاری آنها هیچ کدی اجرا نمیکند.
|
|
129
|
+
|
|
130
|
+
- برچسبهای آموزش `PolarityClassifier` یکی از `recommended`، `not_recommended` یا `no_idea` هستند و برچسب خالی `no_idea` در نظر گرفته میشود.
|
|
131
|
+
- نام احساسها در `EmotionAnalyzer` را میتوانید فارسی (`شادی`) یا انگلیسی (`happiness`) بنویسید.
|
|
132
|
+
|
|
133
|
+
جدول کامل متدها در [README انگلیسی](README.md#api-reference) آمده است.
|
|
134
|
+
|
|
135
|
+
## کیفیت مدل و محدودیتها
|
|
136
|
+
|
|
137
|
+
- دقت مدل آمادهٔ تحلیل نظرات روی نمونهای ۱۰٬۰۰۰تایی از [`dataset/train.csv`](dataset/train.csv) حدود **۶۵٪** است (macro-F1 ≈ ۰.۶۴). سه دسته متوازناند و دقت تصادفی ۳۳٪ است. بهبود مدل در برنامهٔ نسخههای بعدی است.
|
|
138
|
+
- تحلیلگر احساس بر پایهٔ واژهنامه کار میکند و **هنوز منفیسازی را تشخیص نمیدهد**. مثلاً «خوشحال نیستم» شادی حساب میشود.
|
|
139
|
+
- لیست ایستواژههای hazm کلمههایی مثل «خوب»، «بد» و «عالی» را هم شامل میشود و این کلمهها پیش از دستهبندی حذف میشوند. به همین دلیل نتیجه برای نظرهای خیلی کوتاه قابل اعتماد نیست.
|
|
140
|
+
|
|
141
|
+
مجموعهدادهٔ بزرگتر برای آموزش: [اینجا](https://github.com/RezaGooner/Sentiment-Survey-Analyzer/tree/main/Dataset/big_train)
|
|
142
|
+
|
|
143
|
+
## ارتقا از نسخهٔ 1.x
|
|
144
|
+
|
|
145
|
+
`CommentAnalyzer` و `SentimentAnalyzer` دقیقاً مثل قبل کار میکنند، ولی یک `DeprecationWarning` نشان میدهند و در نسخهٔ 2.0 حذف خواهند شد.
|
|
146
|
+
|
|
147
|
+
</div>
|
|
148
|
+
|
|
149
|
+
| 1.x | 1.5+ |
|
|
150
|
+
|---|---|
|
|
151
|
+
| `a = CommentAnalyzer(); a.loadModel()` | `clf = PolarityClassifier.pretrained()` |
|
|
152
|
+
| `a.analyzeText(t)` | `clf.predict(t)` / `clf.analyze(t)` |
|
|
153
|
+
| `a.train(csv)` | `clf.fit_csv(csv)` |
|
|
154
|
+
| `a.saveModel()` / `a.loadModel()` | `clf.save(dir)` / `PolarityClassifier.load(dir)` |
|
|
155
|
+
| `a.analyzeCSV(i, o, s, col)` | `clf.analyze_csv(i, o, text_column=col, summary_csv=s)` |
|
|
156
|
+
| `s = SentimentAnalyzer(); s.loadModel()` | `emo = EmotionAnalyzer.pretrained()` |
|
|
157
|
+
| `s.analyzeText(t)` | `emo.analyze(t).scores` |
|
|
158
|
+
| `s.train(csv)` / `s.loadLex(csv)` | `emo.fit_csv(csv)` / `emo.add_lexicon(csv)` |
|
|
159
|
+
| `s.saveModel(name)` / `s.loadModel(name)` | `emo.save(path)` / `EmotionAnalyzer.load(path)` |
|
|
160
|
+
| `s.analyzeCSV(i, o)` | `emo.analyze_csv(i, o)` |
|
|
161
|
+
|
|
162
|
+
<div dir="rtl">
|
|
163
|
+
|
|
164
|
+
متدهای جدید در صورت خطا exception میدهند، نه `None` یا `False`. فهرست کامل تغییرات در [CHANGELOG.md](CHANGELOG.md) است.
|
|
165
|
+
|
|
166
|
+
## توسعه
|
|
167
|
+
|
|
168
|
+
</div>
|
|
169
|
+
|
|
170
|
+
```bash
|
|
171
|
+
git clone https://github.com/RezaGooner/PerSent.git
|
|
172
|
+
cd PerSent
|
|
173
|
+
pip install -e ".[dev]"
|
|
174
|
+
pytest
|
|
175
|
+
ruff check . && ruff format --check .
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
<div dir="rtl">
|
|
179
|
+
|
|
180
|
+
## مشارکت
|
|
181
|
+
|
|
182
|
+
خوشحال میشوم پیشنهادها، گزارش باگ و Pull Requestهایتان را ببینم. راهنمای مشارکت در [CONTRIBUTING](.github/CONTRIBUTING.md) است.
|
|
183
|
+
|
|
184
|
+
- [ثبت Issue](https://github.com/RezaGooner/PerSent/issues/new/choose)
|
|
185
|
+
- [Fork و ارسال Pull Request](https://github.com/RezaGooner/PerSent/fork)
|
|
186
|
+
- ایمیل: `RezaAsadiProgrammer@gmail.com` · تلگرام: `@RezaGooner`
|
|
187
|
+
|
|
188
|
+
این پروژه از مخزن [Sentiment-Survey-Analyzer](https://github.com/RezaGooner/Sentiment-Survey-Analyzer) شکل گرفته است.
|
|
189
|
+
|
|
190
|
+
## مجوز
|
|
191
|
+
|
|
192
|
+
[MIT](LICENSE)
|
|
193
|
+
|
|
194
|
+
</div>
|