PerSent 1.3.3__tar.gz → 1.4.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- persent-1.4.0/.gitignore +33 -0
- persent-1.4.0/CHANGELOG.md +52 -0
- persent-1.4.0/LICENSE +21 -0
- persent-1.4.0/PKG-INFO +203 -0
- persent-1.4.0/README.fa.md +163 -0
- persent-1.4.0/README.md +161 -0
- persent-1.4.0/pyproject.toml +79 -0
- persent-1.4.0/scripts/convert_legacy_models.py +65 -0
- persent-1.4.0/src/PerSent/CommentAnalyzer.py +282 -0
- persent-1.4.0/src/PerSent/SentimentAnalyzer.py +306 -0
- persent-1.4.0/src/PerSent/__init__.py +30 -0
- persent-1.4.0/src/PerSent/_models.py +169 -0
- persent-1.4.0/src/PerSent/_stemmer.py +25 -0
- persent-1.4.0/src/PerSent/_text.py +52 -0
- persent-1.4.0/src/PerSent/_utils.py +51 -0
- persent-1.4.0/src/PerSent/_version.py +1 -0
- persent-1.4.0/src/PerSent/exceptions.py +26 -0
- persent-1.4.0/src/PerSent/models/comment_model.npz +0 -0
- persent-1.4.0/src/PerSent/models/emotion_lexicon.json +1 -0
- persent-1.4.0/src/PerSent/py.typed +0 -0
- persent-1.4.0/tests/conftest.py +43 -0
- persent-1.4.0/tests/test_comment_analyzer.py +143 -0
- persent-1.4.0/tests/test_package.py +67 -0
- persent-1.4.0/tests/test_sentiment_analyzer.py +103 -0
- persent-1.3.3/MANIFEST.in +0 -2
- persent-1.3.3/PKG-INFO +0 -366
- persent-1.3.3/PerSent/CommentAnalyzer.py +0 -279
- persent-1.3.3/PerSent/SentimentAnalyzer.py +0 -346
- persent-1.3.3/PerSent/__init__.py +0 -10
- persent-1.3.3/PerSent/models/classifier.joblib +0 -0
- persent-1.3.3/PerSent/models/weighted_sentiment_model.joblib +0 -0
- persent-1.3.3/PerSent/models/word2vec.model +0 -0
- persent-1.3.3/PerSent.egg-info/PKG-INFO +0 -366
- persent-1.3.3/PerSent.egg-info/SOURCES.txt +0 -15
- persent-1.3.3/PerSent.egg-info/dependency_links.txt +0 -1
- persent-1.3.3/PerSent.egg-info/requires.txt +0 -8
- persent-1.3.3/PerSent.egg-info/top_level.txt +0 -1
- persent-1.3.3/README.md +0 -334
- persent-1.3.3/setup.cfg +0 -4
- persent-1.3.3/setup.py +0 -37
persent-1.4.0/.gitignore
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Python
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*.egg-info/
|
|
5
|
+
.eggs/
|
|
6
|
+
build/
|
|
7
|
+
dist/
|
|
8
|
+
*.egg
|
|
9
|
+
|
|
10
|
+
# Environments
|
|
11
|
+
.venv/
|
|
12
|
+
venv/
|
|
13
|
+
env/
|
|
14
|
+
|
|
15
|
+
# Tooling
|
|
16
|
+
.pytest_cache/
|
|
17
|
+
.ruff_cache/
|
|
18
|
+
.mypy_cache/
|
|
19
|
+
.coverage
|
|
20
|
+
coverage.xml
|
|
21
|
+
htmlcov/
|
|
22
|
+
.ipynb_checkpoints/
|
|
23
|
+
|
|
24
|
+
# Editors / OS
|
|
25
|
+
.idea/
|
|
26
|
+
.vscode/
|
|
27
|
+
.DS_Store
|
|
28
|
+
Thumbs.db
|
|
29
|
+
|
|
30
|
+
# Locally trained models
|
|
31
|
+
/model/
|
|
32
|
+
/models/
|
|
33
|
+
PerSent/model/
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here.
|
|
4
|
+
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses
|
|
5
|
+
[Semantic Versioning](https://semver.org/).
|
|
6
|
+
|
|
7
|
+
## [1.4.0] - Unreleased
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
- `pyproject.toml` packaging (hatchling) with a single version source (`src/PerSent/_version.py`).
|
|
11
|
+
- Support for Python 3.12 and 3.13 (requires Python ≥ 3.10).
|
|
12
|
+
- Test suite (pytest), linting (ruff) and GitHub Actions CI on Linux, Windows and macOS.
|
|
13
|
+
- Release workflow that publishes to PyPI with Trusted Publishing.
|
|
14
|
+
- `CommentAnalyzer.analyzeTexts()` for lists of texts.
|
|
15
|
+
- `CommentAnalyzer.train()` parameters `random_state`, `text_column`, `label_column`, `workers`; the split is now stratified.
|
|
16
|
+
- `SentimentAnalyzer.train(reset=...)` to start from an empty model.
|
|
17
|
+
- Exception hierarchy: `PerSentError`, `ModelNotLoadedError`, `ModelFileError`, `DataValidationError`
|
|
18
|
+
(all subclasses of `Exception`).
|
|
19
|
+
- Type hints (`py.typed`) and `scripts/convert_legacy_models.py`.
|
|
20
|
+
- Vendored hazm 0.9's suffix stemmer (MIT), so preprocessing — and therefore predictions — stay the same with
|
|
21
|
+
hazm 0.12 on Python 3.12+ (whose new stemmer changed ~10 % of predictions).
|
|
22
|
+
|
|
23
|
+
### Changed
|
|
24
|
+
- Pre-trained models are stored in pickle-free formats (`comment_model.npz`, `emotion_lexicon.json`).
|
|
25
|
+
Predictions are identical to 1.3.3; the wheel shrank from ~27 MB to ~14 MB.
|
|
26
|
+
- `model_dir` now defaults to `None`: `loadModel()` loads the bundled model and `train()` only saves when a
|
|
27
|
+
`model_dir` is set. Legacy 1.3.x model files in an explicit `model_dir` still load (with a `DeprecationWarning`).
|
|
28
|
+
- Status messages use `logging` (logger `"PerSent"`) instead of `print`.
|
|
29
|
+
- Removed the `numpy<2` / `scipy<1.14` pins that made the package uninstallable on Python 3.12+.
|
|
30
|
+
- Repository layout moved to `src/`.
|
|
31
|
+
|
|
32
|
+
### Fixed
|
|
33
|
+
- **Security:** `loadModel()` no longer loads (pickle) files from `./PerSent/model` in the current working
|
|
34
|
+
directory, and creating an analyzer no longer creates that folder.
|
|
35
|
+
- `SentimentAnalyzer.train()` failed with its default arguments on the bundled dataset (`emotion_col` default).
|
|
36
|
+
- `CommentAnalyzer` deleted all upper-case Latin letters ("iPhone" → "i hone") due to a regex range bug.
|
|
37
|
+
- `CommentAnalyzer.train(vector_size=...)` crashed for any size other than 100.
|
|
38
|
+
- Empty labels in the training CSV crashed training instead of being treated as `no_idea`.
|
|
39
|
+
- `SentimentAnalyzer.loadLex()` crashed when columns were given by name.
|
|
40
|
+
- Empty or digit-only texts were classified as `recommended`; they now return `no_idea`.
|
|
41
|
+
- Missing values (NaN) in CSV text columns were analyzed as the word "nan".
|
|
42
|
+
- With hazm >= 0.11, every text re-created hazm's `WordTokenizer` (~0.3 s per text); one instance is now reused.
|
|
43
|
+
|
|
44
|
+
### Removed
|
|
45
|
+
- `setup.py`, `setup.cfg`, `MANIFEST.in`, `requirements.txt`, committed `build/` output and `PKG-INFO`.
|
|
46
|
+
- Draft JOSS paper files (`paper.md`, `paper.bib`).
|
|
47
|
+
|
|
48
|
+
## [1.3.3] - 2025-08-22
|
|
49
|
+
- Last release with the original `setup.py` layout and pickle-based models.
|
|
50
|
+
|
|
51
|
+
[1.4.0]: https://github.com/RezaGooner/PerSent/compare/088b919...HEAD
|
|
52
|
+
[1.3.3]: https://pypi.org/project/PerSent/1.3.3/
|
persent-1.4.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 𝐑𝐞𝐳𝐚 𝐀𝐬𝐚𝐝𝐢
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
persent-1.4.0/PKG-INFO
ADDED
|
@@ -0,0 +1,203 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: PerSent
|
|
3
|
+
Version: 1.4.0
|
|
4
|
+
Summary: Persian sentiment and emotion analysis toolkit
|
|
5
|
+
Project-URL: Homepage, https://github.com/RezaGooner/PerSent
|
|
6
|
+
Project-URL: Source, https://github.com/RezaGooner/PerSent
|
|
7
|
+
Project-URL: Issues, https://github.com/RezaGooner/PerSent/issues
|
|
8
|
+
Project-URL: Changelog, https://github.com/RezaGooner/PerSent/blob/main/CHANGELOG.md
|
|
9
|
+
Author-email: Reza Asadi <RezaAsadiProgrammer@Gmail.com>
|
|
10
|
+
License-Expression: MIT
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Keywords: emotion-detection,farsi,nlp,persian,sentiment-analysis,text-classification
|
|
13
|
+
Classifier: Development Status :: 4 - Beta
|
|
14
|
+
Classifier: Intended Audience :: Developers
|
|
15
|
+
Classifier: Intended Audience :: Science/Research
|
|
16
|
+
Classifier: Natural Language :: Persian
|
|
17
|
+
Classifier: Operating System :: OS Independent
|
|
18
|
+
Classifier: Programming Language :: Python :: 3
|
|
19
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
22
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
23
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
24
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
25
|
+
Classifier: Topic :: Text Processing :: Linguistic
|
|
26
|
+
Classifier: Typing :: Typed
|
|
27
|
+
Requires-Python: >=3.10
|
|
28
|
+
Requires-Dist: gensim>=4.3.2
|
|
29
|
+
Requires-Dist: hazm>=0.9.3
|
|
30
|
+
Requires-Dist: joblib>=1.1
|
|
31
|
+
Requires-Dist: numpy>=1.24
|
|
32
|
+
Requires-Dist: pandas>=1.5
|
|
33
|
+
Requires-Dist: scikit-learn>=1.2
|
|
34
|
+
Requires-Dist: tqdm>=4.62
|
|
35
|
+
Provides-Extra: dev
|
|
36
|
+
Requires-Dist: build; extra == 'dev'
|
|
37
|
+
Requires-Dist: pytest-cov; extra == 'dev'
|
|
38
|
+
Requires-Dist: pytest>=7; extra == 'dev'
|
|
39
|
+
Requires-Dist: ruff>=0.6; extra == 'dev'
|
|
40
|
+
Requires-Dist: twine; extra == 'dev'
|
|
41
|
+
Description-Content-Type: text/markdown
|
|
42
|
+
|
|
43
|
+
# PerSent (Persian Sentiment Analyzer)
|
|
44
|
+
|
|
45
|
+
[](https://pypi.org/project/PerSent/)
|
|
46
|
+
[](https://pypi.org/project/PerSent/)
|
|
47
|
+
[](https://github.com/RezaGooner/PerSent/actions/workflows/ci.yml)
|
|
48
|
+
[](LICENSE)
|
|
49
|
+
[](README.fa.md)
|
|
50
|
+
|
|
51
|
+

|
|
52
|
+
|
|
53
|
+
PerSent is a lightweight Python library for **sentiment and emotion analysis of Persian (Farsi) text**.
|
|
54
|
+
|
|
55
|
+
- **`CommentAnalyzer`** classifies product/service reviews as `recommended`, `not_recommended` or `no_idea`
|
|
56
|
+
(Word2Vec sentence embeddings + logistic regression).
|
|
57
|
+
- **`SentimentAnalyzer`** scores seven emotions with a weighted lexicon:
|
|
58
|
+
شادی (happiness), غم (sadness), ترس (fear), عصبانیت (anger), تنفر (disgust), شگفتی (surprise), آرامش (calmness).
|
|
59
|
+
- Single texts or whole CSV files, pre-trained models included, train your own models from CSV.
|
|
60
|
+
|
|
61
|
+
## Installation
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
pip install PerSent
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Requires Python 3.10–3.13. Pre-trained models are bundled with the package, so no download is needed at runtime.
|
|
68
|
+
|
|
69
|
+
## Quick start
|
|
70
|
+
|
|
71
|
+
### Review / comment polarity
|
|
72
|
+
|
|
73
|
+
```python
|
|
74
|
+
from PerSent import CommentAnalyzer
|
|
75
|
+
|
|
76
|
+
analyzer = CommentAnalyzer()
|
|
77
|
+
analyzer.loadModel() # bundled pre-trained model
|
|
78
|
+
|
|
79
|
+
analyzer.analyzeText("کیفیت عالی داشت") # 'recommended'
|
|
80
|
+
analyzer.analyzeText("افتضاح بود") # 'not_recommended'
|
|
81
|
+
analyzer.analyzeTexts(["خوب بود", "بد بود"]) # list of labels
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
Batch-process a CSV file (adds a `sentiment` column):
|
|
85
|
+
|
|
86
|
+
```python
|
|
87
|
+
analyzer.analyzeCSV("comments.csv", "results.csv") # first column
|
|
88
|
+
analyzer.analyzeCSV("comments.csv", "results.csv", None, -1) # last column
|
|
89
|
+
analyzer.analyzeCSV("comments.csv", "results.csv", None, "نظرات") # column by name
|
|
90
|
+
analyzer.analyzeCSV("comments.csv", "results.csv", "summary.csv") # + summary report
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
### Emotions
|
|
94
|
+
|
|
95
|
+
```python
|
|
96
|
+
from PerSent import SentimentAnalyzer
|
|
97
|
+
|
|
98
|
+
analyzer = SentimentAnalyzer()
|
|
99
|
+
analyzer.loadModel()
|
|
100
|
+
|
|
101
|
+
result = analyzer.analyzeText("امتحانم رو خراب کردم. احساس میکنم یک شکست خوردهی تمام عیارم.")
|
|
102
|
+
for emotion, score in sorted(result.items(), key=lambda x: x[1], reverse=True):
|
|
103
|
+
print(f"{emotion}: {score:.2f}%")
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
```text
|
|
107
|
+
غم: 36.00%
|
|
108
|
+
عصبانیت: 36.00%
|
|
109
|
+
ترس: 28.00%
|
|
110
|
+
شادی: 0.00%
|
|
111
|
+
...
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
```python
|
|
115
|
+
analyzer.analyzeCSV("input.csv", "output.csv") # text column 'text' -> 'sentiment_analysis'
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
## Training and saving your own models
|
|
119
|
+
|
|
120
|
+
Pass a `model_dir` to keep your own models. Without it, `loadModel()` always loads the bundled
|
|
121
|
+
pre-trained model and `train()` keeps the result in memory only.
|
|
122
|
+
|
|
123
|
+
```python
|
|
124
|
+
from PerSent import CommentAnalyzer, SentimentAnalyzer
|
|
125
|
+
|
|
126
|
+
comments = CommentAnalyzer(model_dir="my_models")
|
|
127
|
+
accuracy = comments.train("train.csv", random_state=42) # saved to my_models/comment_model.npz
|
|
128
|
+
comments.loadModel() # later: loads from my_models/
|
|
129
|
+
|
|
130
|
+
emotions = SentimentAnalyzer(model_dir="my_models")
|
|
131
|
+
emotions.train("dataset/emotion_dataset.csv") # saved to my_models/weighted_sentiment_model.json
|
|
132
|
+
emotions.saveModel("custom") # my_models/custom.json
|
|
133
|
+
emotions.loadModel("custom")
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
## API reference
|
|
137
|
+
|
|
138
|
+
### `CommentAnalyzer(model_dir=None)`
|
|
139
|
+
|
|
140
|
+
| Method | Description |
|
|
141
|
+
|---|---|
|
|
142
|
+
| `train(train_csv, test_size=0.2, vector_size=100, window=5, random_state=None, text_column="body", label_column="recommendation_status", workers=4)` | Train from a CSV. Labels must be `recommended`, `not_recommended` or `no_idea`; empty labels count as `no_idea`. Returns held-out accuracy. Saves to `model_dir` if set. |
|
|
143
|
+
| `analyzeText(text)` | Label for one text. Texts without letters (empty, digits only, …) return `no_idea`. |
|
|
144
|
+
| `analyzeTexts(texts, show_progress=False)` | Labels for many texts. |
|
|
145
|
+
| `analyzeCSV(input_csv, output_path, summary_path=None, text_column=0)` | Analyze a CSV column (name or 0-based / negative index). Returns a DataFrame, or `None` on error. |
|
|
146
|
+
| `saveModel(model_dir=None)` / `loadModel(model_dir=None)` | Save / load `comment_model.npz`. `loadModel` falls back to the bundled model. |
|
|
147
|
+
|
|
148
|
+
### `SentimentAnalyzer(model_dir=None)`
|
|
149
|
+
|
|
150
|
+
| Method | Description |
|
|
151
|
+
|---|---|
|
|
152
|
+
| `train(train_csv, text_col="text", emotion_col=None, weight_col="weight", reset=False)` | Learn weighted emotion words from a CSV. `emotion_col` defaults to `sentiment` (or `emotion`). Merges into the current model unless `reset=True`. Returns `{"status", "message"}`. |
|
|
153
|
+
| `loadLex(csv_file, word_col=0, emotion_col=1, weight_col=2)` | Add a word/emotion/weight lexicon. Columns by index, or by name for CSVs with a header. Invalid weights default to 1.0. |
|
|
154
|
+
| `analyzeText(text)` | `{emotion: percentage}` for the seven emotions (all 0.0 if no lexicon word is found). |
|
|
155
|
+
| `analyzeCSV(input_csv, output_csv, text_col="text", output_col="sentiment_analysis")` | Analyze a CSV. Returns `True` / `False`. |
|
|
156
|
+
| `saveModel(model_name="weighted_sentiment_model", model_dir=None)` / `loadModel(...)` | Save / load `<model_name>.json`. |
|
|
157
|
+
|
|
158
|
+
Errors derive from `PerSent.PerSentError` (`ModelNotLoadedError`, `ModelFileError`, `DataValidationError`).
|
|
159
|
+
Progress and status messages use the standard `logging` module (logger name `"PerSent"`).
|
|
160
|
+
|
|
161
|
+
## Model quality and limitations
|
|
162
|
+
|
|
163
|
+
- The bundled comment model reaches about **65 % accuracy** (macro-F1 ≈ 0.64) on a 10 000-review sample of
|
|
164
|
+
[`dataset/train.csv`](dataset/train.csv) (three balanced classes, so chance level is 33 %). Better models are planned.
|
|
165
|
+
- The emotion analyzer is lexicon-based and does **not handle negation** yet ("خوشحال نیستم" is scored as happiness).
|
|
166
|
+
- hazm's stop-word list contains common opinion words such as "خوب" and "عالی", which are removed before
|
|
167
|
+
classification.
|
|
168
|
+
|
|
169
|
+
A larger training set is available [here](https://github.com/RezaGooner/Sentiment-Survey-Analyzer/tree/main/Dataset/big_train).
|
|
170
|
+
|
|
171
|
+
## Upgrading from 1.3.x
|
|
172
|
+
|
|
173
|
+
- Models are stored in safe, pickle-free formats (`.npz` / `.json`). Models you saved with 1.3.x can still be loaded
|
|
174
|
+
with `CommentAnalyzer(model_dir=...)` / `SentimentAnalyzer(model_dir=...)` (a `DeprecationWarning` is shown);
|
|
175
|
+
call `saveModel()` once to convert them. `scripts/convert_legacy_models.py` converts a folder in one go.
|
|
176
|
+
- PerSent no longer creates or reads a `PerSent/model` folder in the current working directory. Pass `model_dir`
|
|
177
|
+
explicitly to save and load your own models.
|
|
178
|
+
|
|
179
|
+
See [CHANGELOG.md](CHANGELOG.md) for all changes.
|
|
180
|
+
|
|
181
|
+
## Development
|
|
182
|
+
|
|
183
|
+
```bash
|
|
184
|
+
git clone https://github.com/RezaGooner/PerSent.git
|
|
185
|
+
cd PerSent
|
|
186
|
+
pip install -e ".[dev]"
|
|
187
|
+
pytest
|
|
188
|
+
ruff check . && ruff format --check .
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
## Contributing
|
|
192
|
+
|
|
193
|
+
Bug reports, ideas and pull requests are welcome — see [CONTRIBUTING](.github/CONTRIBUTING.md).
|
|
194
|
+
|
|
195
|
+
- [Open an issue](https://github.com/RezaGooner/PerSent/issues/new/choose)
|
|
196
|
+
- [Fork and send a pull request](https://github.com/RezaGooner/PerSent/fork)
|
|
197
|
+
- Email: `RezaAsadiProgrammer@gmail.com` · Telegram: `@RezaGooner`
|
|
198
|
+
|
|
199
|
+
The project grew out of [Sentiment-Survey-Analyzer](https://github.com/RezaGooner/Sentiment-Survey-Analyzer).
|
|
200
|
+
|
|
201
|
+
## License
|
|
202
|
+
|
|
203
|
+
[MIT](LICENSE)
|
|
@@ -0,0 +1,163 @@
|
|
|
1
|
+
<div dir="rtl">
|
|
2
|
+
|
|
3
|
+
# PerSent پرسنت
|
|
4
|
+
|
|
5
|
+
[](https://pypi.org/project/PerSent/)
|
|
6
|
+
[](https://pypi.org/project/PerSent/)
|
|
7
|
+
[](https://github.com/RezaGooner/PerSent/actions/workflows/ci.yml)
|
|
8
|
+
[](README.md)
|
|
9
|
+
|
|
10
|
+

|
|
11
|
+
|
|
12
|
+
پرسنت (Persian Sentiment Analyzer) یک کتابخانهٔ سبک پایتون برای **تحلیل احساسات و هیجانهای متن فارسی** است.
|
|
13
|
+
|
|
14
|
+
- **`CommentAnalyzer`**: نظر کاربران دربارهٔ کالا یا خدمات را در یکی از سه دستهٔ `recommended` (پیشنهاد میکند)، `not_recommended` (پیشنهاد نمیکند) و `no_idea` (نظری ندارد) قرار میدهد.
|
|
15
|
+
- **`SentimentAnalyzer`**: درصد هفت حس شادی، غم، ترس، عصبانیت، تنفر، شگفتی و آرامش را در متن، با یک واژهنامهٔ وزندار محاسبه میکند.
|
|
16
|
+
- تحلیل تکمتن یا کل یک فایل CSV، همراه با مدلهای ازپیشآموزشدیده و امکان آموزش مدل با دادههای خودتان.
|
|
17
|
+
|
|
18
|
+
## نصب
|
|
19
|
+
|
|
20
|
+
</div>
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
pip install PerSent
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
<div dir="rtl">
|
|
27
|
+
|
|
28
|
+
به پایتون ۳.۱۰ تا ۳.۱۳ نیاز دارد. مدلهای ازپیشآموزشدیده داخل خود بسته هستند و هنگام اجرا چیزی دانلود نمیشود.
|
|
29
|
+
|
|
30
|
+
## شروع سریع
|
|
31
|
+
|
|
32
|
+
### تحلیل نظرات
|
|
33
|
+
|
|
34
|
+
</div>
|
|
35
|
+
|
|
36
|
+
```python
|
|
37
|
+
from PerSent import CommentAnalyzer
|
|
38
|
+
|
|
39
|
+
analyzer = CommentAnalyzer()
|
|
40
|
+
analyzer.loadModel() # مدل آمادهٔ همراه کتابخانه
|
|
41
|
+
|
|
42
|
+
analyzer.analyzeText("کیفیت عالی داشت") # 'recommended'
|
|
43
|
+
analyzer.analyzeText("افتضاح بود") # 'not_recommended'
|
|
44
|
+
analyzer.analyzeTexts(["خوب بود", "بد بود"]) # فهرست برچسبها
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
<div dir="rtl">
|
|
48
|
+
|
|
49
|
+
تحلیل یک فایل CSV (یک ستون `sentiment` به خروجی اضافه میشود):
|
|
50
|
+
|
|
51
|
+
</div>
|
|
52
|
+
|
|
53
|
+
```python
|
|
54
|
+
analyzer.analyzeCSV("comments.csv", "results.csv") # ستون اول
|
|
55
|
+
analyzer.analyzeCSV("comments.csv", "results.csv", None, -1) # ستون آخر
|
|
56
|
+
analyzer.analyzeCSV("comments.csv", "results.csv", None, "نظرات") # با نام ستون
|
|
57
|
+
analyzer.analyzeCSV("comments.csv", "results.csv", "summary.csv") # همراه با فایل خلاصه
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
<div dir="rtl">
|
|
61
|
+
|
|
62
|
+
فایل خلاصه تعداد و درصد نظرات هر دسته و تعداد کل نظرات را نشان میدهد.
|
|
63
|
+
|
|
64
|
+
### تحلیل احساس
|
|
65
|
+
|
|
66
|
+
</div>
|
|
67
|
+
|
|
68
|
+
```python
|
|
69
|
+
from PerSent import SentimentAnalyzer
|
|
70
|
+
|
|
71
|
+
analyzer = SentimentAnalyzer()
|
|
72
|
+
analyzer.loadModel()
|
|
73
|
+
|
|
74
|
+
result = analyzer.analyzeText("امتحانم رو خراب کردم. احساس میکنم یک شکست خوردهی تمام عیارم.")
|
|
75
|
+
for emotion, score in sorted(result.items(), key=lambda x: x[1], reverse=True):
|
|
76
|
+
print(f"{emotion}: {score:.2f}%")
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
```text
|
|
80
|
+
غم: 36.00%
|
|
81
|
+
عصبانیت: 36.00%
|
|
82
|
+
ترس: 28.00%
|
|
83
|
+
شادی: 0.00%
|
|
84
|
+
...
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
```python
|
|
88
|
+
analyzer.analyzeCSV("input.csv", "output.csv") # ستون متن: text ← ستون خروجی: sentiment_analysis
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
<div dir="rtl">
|
|
92
|
+
|
|
93
|
+
## آموزش و ذخیرهٔ مدلهای خودتان
|
|
94
|
+
|
|
95
|
+
برای نگه داشتن مدلهای خودتان، پارامتر `model_dir` را بدهید. اگر ندهید، `loadModel()` همیشه مدل آمادهٔ کتابخانه را بارگذاری میکند و `train()` چیزی روی دیسک نمینویسد.
|
|
96
|
+
|
|
97
|
+
</div>
|
|
98
|
+
|
|
99
|
+
```python
|
|
100
|
+
from PerSent import CommentAnalyzer, SentimentAnalyzer
|
|
101
|
+
|
|
102
|
+
comments = CommentAnalyzer(model_dir="my_models")
|
|
103
|
+
accuracy = comments.train("train.csv", random_state=42) # ذخیره در my_models/comment_model.npz
|
|
104
|
+
comments.loadModel() # بعداً: بارگذاری از my_models
|
|
105
|
+
|
|
106
|
+
emotions = SentimentAnalyzer(model_dir="my_models")
|
|
107
|
+
emotions.train("dataset/emotion_dataset.csv") # ذخیره در my_models/weighted_sentiment_model.json
|
|
108
|
+
emotions.saveModel("custom") # my_models/custom.json
|
|
109
|
+
emotions.loadModel("custom")
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
<div dir="rtl">
|
|
113
|
+
|
|
114
|
+
- فایل آموزش `CommentAnalyzer` باید ستونهای `body` (متن) و `recommendation_status` (برچسب) را داشته باشد. برچسبها یکی از `recommended`، `not_recommended` یا `no_idea` هستند و برچسب خالی `no_idea` در نظر گرفته میشود. خروجی `train` دقت مدل روی دادهٔ آزمون است.
|
|
115
|
+
- فایل آموزش `SentimentAnalyzer` ستونهای `text`، `sentiment` (یکی از هفت حس) و `weight` (وزن) دارد. نمونهاش در [`dataset/emotion_dataset.csv`](dataset/emotion_dataset.csv) است. با `reset=True` مدل قبلی پاک میشود. در غیر این صورت دادههای جدید به آن اضافه میشوند.
|
|
116
|
+
- با `loadLex(csv_file, word_col=0, emotion_col=1, weight_col=2)` میتوانید یک واژهنامهٔ «کلمه، حس، وزن» اضافه کنید. ستونها را با شماره یا، اگر فایل سرستون دارد، با نام مشخص کنید.
|
|
117
|
+
|
|
118
|
+
جدول کامل پارامترها در [README انگلیسی](README.md#api-reference) آمده است.
|
|
119
|
+
|
|
120
|
+
## کیفیت مدل و محدودیتها
|
|
121
|
+
|
|
122
|
+
- دقت مدل آمادهٔ تحلیل نظرات روی نمونهای ۱۰٬۰۰۰تایی از [`dataset/train.csv`](dataset/train.csv) حدود **۶۵٪** است (macro-F1 ≈ ۰.۶۴). سه دسته متوازناند و دقت تصادفی ۳۳٪ است. بهبود مدل در برنامهٔ نسخههای بعدی است.
|
|
123
|
+
- تحلیلگر احساس بر پایهٔ واژهنامه کار میکند و **هنوز منفیسازی را تشخیص نمیدهد**. مثلاً «خوشحال نیستم» شادی حساب میشود.
|
|
124
|
+
- لیست ایستواژههای hazm کلمههایی مثل «خوب» و «عالی» را هم شامل میشود و این کلمهها پیش از دستهبندی حذف میشوند.
|
|
125
|
+
|
|
126
|
+
مجموعهدادهٔ بزرگتر برای آموزش: [اینجا](https://github.com/RezaGooner/Sentiment-Survey-Analyzer/tree/main/Dataset/big_train)
|
|
127
|
+
|
|
128
|
+
## ارتقا از نسخهٔ 1.3
|
|
129
|
+
|
|
130
|
+
- مدلها حالا با قالبهای امن و بدون pickle (`.npz` و `.json`) ذخیره میشوند. مدلهایی را که با نسخهٔ 1.3 ذخیره کردهاید هنوز میتوانید با `model_dir` بارگذاری کنید (یک `DeprecationWarning` نمایش داده میشود). یک بار `saveModel()` را صدا بزنید تا به قالب جدید تبدیل شوند.
|
|
131
|
+
- کتابخانه دیگر پوشهٔ `PerSent/model` را در پوشهٔ جاری نمیسازد و از آنجا هم نمیخواند. برای مدلهای خودتان `model_dir` را صریحاً بدهید.
|
|
132
|
+
|
|
133
|
+
فهرست کامل تغییرات در [CHANGELOG.md](CHANGELOG.md) است.
|
|
134
|
+
|
|
135
|
+
## توسعه
|
|
136
|
+
|
|
137
|
+
</div>
|
|
138
|
+
|
|
139
|
+
```bash
|
|
140
|
+
git clone https://github.com/RezaGooner/PerSent.git
|
|
141
|
+
cd PerSent
|
|
142
|
+
pip install -e ".[dev]"
|
|
143
|
+
pytest
|
|
144
|
+
ruff check . && ruff format --check .
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
<div dir="rtl">
|
|
148
|
+
|
|
149
|
+
## مشارکت
|
|
150
|
+
|
|
151
|
+
خوشحال میشوم پیشنهادها، گزارش باگ و Pull Requestهایتان را ببینم. راهنمای مشارکت در [CONTRIBUTING](.github/CONTRIBUTING.md) است.
|
|
152
|
+
|
|
153
|
+
- [ثبت Issue](https://github.com/RezaGooner/PerSent/issues/new/choose)
|
|
154
|
+
- [Fork و ارسال Pull Request](https://github.com/RezaGooner/PerSent/fork)
|
|
155
|
+
- ایمیل: `RezaAsadiProgrammer@gmail.com` · تلگرام: `@RezaGooner`
|
|
156
|
+
|
|
157
|
+
این پروژه از مخزن [Sentiment-Survey-Analyzer](https://github.com/RezaGooner/Sentiment-Survey-Analyzer) شکل گرفته است.
|
|
158
|
+
|
|
159
|
+
## مجوز
|
|
160
|
+
|
|
161
|
+
[MIT](LICENSE)
|
|
162
|
+
|
|
163
|
+
</div>
|
persent-1.4.0/README.md
ADDED
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
# PerSent (Persian Sentiment Analyzer)
|
|
2
|
+
|
|
3
|
+
[](https://pypi.org/project/PerSent/)
|
|
4
|
+
[](https://pypi.org/project/PerSent/)
|
|
5
|
+
[](https://github.com/RezaGooner/PerSent/actions/workflows/ci.yml)
|
|
6
|
+
[](LICENSE)
|
|
7
|
+
[](README.fa.md)
|
|
8
|
+
|
|
9
|
+

|
|
10
|
+
|
|
11
|
+
PerSent is a lightweight Python library for **sentiment and emotion analysis of Persian (Farsi) text**.
|
|
12
|
+
|
|
13
|
+
- **`CommentAnalyzer`** classifies product/service reviews as `recommended`, `not_recommended` or `no_idea`
|
|
14
|
+
(Word2Vec sentence embeddings + logistic regression).
|
|
15
|
+
- **`SentimentAnalyzer`** scores seven emotions with a weighted lexicon:
|
|
16
|
+
شادی (happiness), غم (sadness), ترس (fear), عصبانیت (anger), تنفر (disgust), شگفتی (surprise), آرامش (calmness).
|
|
17
|
+
- Single texts or whole CSV files, pre-trained models included, train your own models from CSV.
|
|
18
|
+
|
|
19
|
+
## Installation
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
pip install PerSent
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Requires Python 3.10–3.13. Pre-trained models are bundled with the package, so no download is needed at runtime.
|
|
26
|
+
|
|
27
|
+
## Quick start
|
|
28
|
+
|
|
29
|
+
### Review / comment polarity
|
|
30
|
+
|
|
31
|
+
```python
|
|
32
|
+
from PerSent import CommentAnalyzer
|
|
33
|
+
|
|
34
|
+
analyzer = CommentAnalyzer()
|
|
35
|
+
analyzer.loadModel() # bundled pre-trained model
|
|
36
|
+
|
|
37
|
+
analyzer.analyzeText("کیفیت عالی داشت") # 'recommended'
|
|
38
|
+
analyzer.analyzeText("افتضاح بود") # 'not_recommended'
|
|
39
|
+
analyzer.analyzeTexts(["خوب بود", "بد بود"]) # list of labels
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Batch-process a CSV file (adds a `sentiment` column):
|
|
43
|
+
|
|
44
|
+
```python
|
|
45
|
+
analyzer.analyzeCSV("comments.csv", "results.csv") # first column
|
|
46
|
+
analyzer.analyzeCSV("comments.csv", "results.csv", None, -1) # last column
|
|
47
|
+
analyzer.analyzeCSV("comments.csv", "results.csv", None, "نظرات") # column by name
|
|
48
|
+
analyzer.analyzeCSV("comments.csv", "results.csv", "summary.csv") # + summary report
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
### Emotions
|
|
52
|
+
|
|
53
|
+
```python
|
|
54
|
+
from PerSent import SentimentAnalyzer
|
|
55
|
+
|
|
56
|
+
analyzer = SentimentAnalyzer()
|
|
57
|
+
analyzer.loadModel()
|
|
58
|
+
|
|
59
|
+
result = analyzer.analyzeText("امتحانم رو خراب کردم. احساس میکنم یک شکست خوردهی تمام عیارم.")
|
|
60
|
+
for emotion, score in sorted(result.items(), key=lambda x: x[1], reverse=True):
|
|
61
|
+
print(f"{emotion}: {score:.2f}%")
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
```text
|
|
65
|
+
غم: 36.00%
|
|
66
|
+
عصبانیت: 36.00%
|
|
67
|
+
ترس: 28.00%
|
|
68
|
+
شادی: 0.00%
|
|
69
|
+
...
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
```python
|
|
73
|
+
analyzer.analyzeCSV("input.csv", "output.csv") # text column 'text' -> 'sentiment_analysis'
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
## Training and saving your own models
|
|
77
|
+
|
|
78
|
+
Pass a `model_dir` to keep your own models. Without it, `loadModel()` always loads the bundled
|
|
79
|
+
pre-trained model and `train()` keeps the result in memory only.
|
|
80
|
+
|
|
81
|
+
```python
|
|
82
|
+
from PerSent import CommentAnalyzer, SentimentAnalyzer
|
|
83
|
+
|
|
84
|
+
comments = CommentAnalyzer(model_dir="my_models")
|
|
85
|
+
accuracy = comments.train("train.csv", random_state=42) # saved to my_models/comment_model.npz
|
|
86
|
+
comments.loadModel() # later: loads from my_models/
|
|
87
|
+
|
|
88
|
+
emotions = SentimentAnalyzer(model_dir="my_models")
|
|
89
|
+
emotions.train("dataset/emotion_dataset.csv") # saved to my_models/weighted_sentiment_model.json
|
|
90
|
+
emotions.saveModel("custom") # my_models/custom.json
|
|
91
|
+
emotions.loadModel("custom")
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
## API reference
|
|
95
|
+
|
|
96
|
+
### `CommentAnalyzer(model_dir=None)`
|
|
97
|
+
|
|
98
|
+
| Method | Description |
|
|
99
|
+
|---|---|
|
|
100
|
+
| `train(train_csv, test_size=0.2, vector_size=100, window=5, random_state=None, text_column="body", label_column="recommendation_status", workers=4)` | Train from a CSV. Labels must be `recommended`, `not_recommended` or `no_idea`; empty labels count as `no_idea`. Returns held-out accuracy. Saves to `model_dir` if set. |
|
|
101
|
+
| `analyzeText(text)` | Label for one text. Texts without letters (empty, digits only, …) return `no_idea`. |
|
|
102
|
+
| `analyzeTexts(texts, show_progress=False)` | Labels for many texts. |
|
|
103
|
+
| `analyzeCSV(input_csv, output_path, summary_path=None, text_column=0)` | Analyze a CSV column (name or 0-based / negative index). Returns a DataFrame, or `None` on error. |
|
|
104
|
+
| `saveModel(model_dir=None)` / `loadModel(model_dir=None)` | Save / load `comment_model.npz`. `loadModel` falls back to the bundled model. |
|
|
105
|
+
|
|
106
|
+
### `SentimentAnalyzer(model_dir=None)`
|
|
107
|
+
|
|
108
|
+
| Method | Description |
|
|
109
|
+
|---|---|
|
|
110
|
+
| `train(train_csv, text_col="text", emotion_col=None, weight_col="weight", reset=False)` | Learn weighted emotion words from a CSV. `emotion_col` defaults to `sentiment` (or `emotion`). Merges into the current model unless `reset=True`. Returns `{"status", "message"}`. |
|
|
111
|
+
| `loadLex(csv_file, word_col=0, emotion_col=1, weight_col=2)` | Add a word/emotion/weight lexicon. Columns by index, or by name for CSVs with a header. Invalid weights default to 1.0. |
|
|
112
|
+
| `analyzeText(text)` | `{emotion: percentage}` for the seven emotions (all 0.0 if no lexicon word is found). |
|
|
113
|
+
| `analyzeCSV(input_csv, output_csv, text_col="text", output_col="sentiment_analysis")` | Analyze a CSV. Returns `True` / `False`. |
|
|
114
|
+
| `saveModel(model_name="weighted_sentiment_model", model_dir=None)` / `loadModel(...)` | Save / load `<model_name>.json`. |
|
|
115
|
+
|
|
116
|
+
Errors derive from `PerSent.PerSentError` (`ModelNotLoadedError`, `ModelFileError`, `DataValidationError`).
|
|
117
|
+
Progress and status messages use the standard `logging` module (logger name `"PerSent"`).
|
|
118
|
+
|
|
119
|
+
## Model quality and limitations
|
|
120
|
+
|
|
121
|
+
- The bundled comment model reaches about **65 % accuracy** (macro-F1 ≈ 0.64) on a 10 000-review sample of
|
|
122
|
+
[`dataset/train.csv`](dataset/train.csv) (three balanced classes, so chance level is 33 %). Better models are planned.
|
|
123
|
+
- The emotion analyzer is lexicon-based and does **not handle negation** yet ("خوشحال نیستم" is scored as happiness).
|
|
124
|
+
- hazm's stop-word list contains common opinion words such as "خوب" and "عالی", which are removed before
|
|
125
|
+
classification.
|
|
126
|
+
|
|
127
|
+
A larger training set is available [here](https://github.com/RezaGooner/Sentiment-Survey-Analyzer/tree/main/Dataset/big_train).
|
|
128
|
+
|
|
129
|
+
## Upgrading from 1.3.x
|
|
130
|
+
|
|
131
|
+
- Models are stored in safe, pickle-free formats (`.npz` / `.json`). Models you saved with 1.3.x can still be loaded
|
|
132
|
+
with `CommentAnalyzer(model_dir=...)` / `SentimentAnalyzer(model_dir=...)` (a `DeprecationWarning` is shown);
|
|
133
|
+
call `saveModel()` once to convert them. `scripts/convert_legacy_models.py` converts a folder in one go.
|
|
134
|
+
- PerSent no longer creates or reads a `PerSent/model` folder in the current working directory. Pass `model_dir`
|
|
135
|
+
explicitly to save and load your own models.
|
|
136
|
+
|
|
137
|
+
See [CHANGELOG.md](CHANGELOG.md) for all changes.
|
|
138
|
+
|
|
139
|
+
## Development
|
|
140
|
+
|
|
141
|
+
```bash
|
|
142
|
+
git clone https://github.com/RezaGooner/PerSent.git
|
|
143
|
+
cd PerSent
|
|
144
|
+
pip install -e ".[dev]"
|
|
145
|
+
pytest
|
|
146
|
+
ruff check . && ruff format --check .
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
## Contributing
|
|
150
|
+
|
|
151
|
+
Bug reports, ideas and pull requests are welcome — see [CONTRIBUTING](.github/CONTRIBUTING.md).
|
|
152
|
+
|
|
153
|
+
- [Open an issue](https://github.com/RezaGooner/PerSent/issues/new/choose)
|
|
154
|
+
- [Fork and send a pull request](https://github.com/RezaGooner/PerSent/fork)
|
|
155
|
+
- Email: `RezaAsadiProgrammer@gmail.com` · Telegram: `@RezaGooner`
|
|
156
|
+
|
|
157
|
+
The project grew out of [Sentiment-Survey-Analyzer](https://github.com/RezaGooner/Sentiment-Survey-Analyzer).
|
|
158
|
+
|
|
159
|
+
## License
|
|
160
|
+
|
|
161
|
+
[MIT](LICENSE)
|