farsflow 0.1.0__tar.gz

farsflow-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mahdi Hosseini
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

farsflow-0.1.0/PKG-INFO

@@ -0,0 +1,126 @@
+Metadata-Version: 2.4
+Name: farsflow
+Version: 0.1.0
+Summary: Fast, modern, and modular Persian text preprocessing library.
+Author-email: Mahdi Hosseini <mhossza@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/mhhoss/farsflow
+Project-URL: Documentation, https://github.com/mhhoss/farsflow
+Project-URL: Source, https://github.com/mhhoss/farsflow
+Project-URL: Issues, https://github.com/mhhoss/farsflow/issues
+Keywords: farsi,Persian,preprocessing,nlp,text-cleaning,normalization,tokenization,machine-learning,llm
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest>=8.3.5; extra == "dev"
+Dynamic: license-file
+
+# farsflow
+
+farsflow is a small library for **Persian text preprocessing**, designed for practical NLP and LLM workflows.
+
+---
+
+## 🚀 Features
+
+- Deterministic pipeline (same input -> same output)
+- Safe character normalization
+- Joiner (ZWNJ) fixes
+- Whitespace & punctuation cleanup
+- Modular processors (use only the components you need)
+- Real‑world style sample tests
+- Zero dependencies
+
+---
+
+## 📦 Installation
+
+```bash
+pip install farsflow
+```
+
+## ✨ Quick Start
+
+```python
+import farsflow as ff
+
+text = "سلام  دنیا!  این یك   تست است  که می نویسم  ۴۵۶"
+cleaned = ff.clean(text)
+print(cleaned)
+```
+Expected output:  سلام دنیا! این یک تست است که می‌نویسم 456
+
+---
+
+## 🧩 Pipeline Components
+
+farsflow ships with a set of modular, composable components:
+
+- **Normalizer** — safe character normalization
+- **JoinerFixer** — fixes ZWNJ usage without over-correction
+- **SpaceCleaner** — trims redundant whitespace and punctuation spacing
+- **Pipeline** — orchestrates components in a deterministic order
+
+You can customize the pipeline:
+
+```python
+from farsflow import Pipeline, Normalizer, JoinerFixer
+
+pipeline = Pipeline(
+    Normalizer(),
+    JoinerFixer(),
+)
+
+text = "متن   تستی"
+pipeline(text)
+```
+
+---
+
+🧪 Testing
+
+```bash
+pytest
+# or:
+pytest path/to/test_file.py
+```
+
+---
+
+🗺 Roadmap (v0.2.0)
+
+- [ ] formalize the behavior of "ff.clean" as a safe and deterministic baseline
+- [ ] add optional normalization controls (e.g. digit normalization)
+- [ ] add optional noise-cleaning emoji and url processors
+- [ ] introduce simple profiles (ff.llm.clean, ff.embedding.query, ff.embedding.index) built on top of the same core
+- [ ] expand real-world test cases to ensure stable behavior across informal, mixed, and noisy Persian text
+
+---
+
+📄 License
+
+MIT License — see [LICENSE](LICENSE).
+
+---
+
+🤝 Contributing
+
+Contributions are welcome.  
+Please open an issue or submit a pull request on GitHub.
+
+📝 Changelog
+
+See [CHANGELOG](CHANGELOG) for version history.

farsflow-0.1.0/README.md

@@ -0,0 +1,96 @@
+# farsflow
+
+farsflow is a small library for **Persian text preprocessing**, designed for practical NLP and LLM workflows.
+
+---
+
+## 🚀 Features
+
+- Deterministic pipeline (same input -> same output)
+- Safe character normalization
+- Joiner (ZWNJ) fixes
+- Whitespace & punctuation cleanup
+- Modular processors (use only the components you need)
+- Real‑world style sample tests
+- Zero dependencies
+
+---
+
+## 📦 Installation
+
+```bash
+pip install farsflow
+```
+
+## ✨ Quick Start
+
+```python
+import farsflow as ff
+
+text = "سلام  دنیا!  این یك   تست است  که می نویسم  ۴۵۶"
+cleaned = ff.clean(text)
+print(cleaned)
+```
+Expected output:  سلام دنیا! این یک تست است که می‌نویسم 456
+
+---
+
+## 🧩 Pipeline Components
+
+farsflow ships with a set of modular, composable components:
+
+- **Normalizer** — safe character normalization
+- **JoinerFixer** — fixes ZWNJ usage without over-correction
+- **SpaceCleaner** — trims redundant whitespace and punctuation spacing
+- **Pipeline** — orchestrates components in a deterministic order
+
+You can customize the pipeline:
+
+```python
+from farsflow import Pipeline, Normalizer, JoinerFixer
+
+pipeline = Pipeline(
+    Normalizer(),
+    JoinerFixer(),
+)
+
+text = "متن   تستی"
+pipeline(text)
+```
+
+---
+
+🧪 Testing
+
+```bash
+pytest
+# or:
+pytest path/to/test_file.py
+```
+
+---
+
+🗺 Roadmap (v0.2.0)
+
+- [ ] formalize the behavior of "ff.clean" as a safe and deterministic baseline
+- [ ] add optional normalization controls (e.g. digit normalization)
+- [ ] add optional noise-cleaning emoji and url processors
+- [ ] introduce simple profiles (ff.llm.clean, ff.embedding.query, ff.embedding.index) built on top of the same core
+- [ ] expand real-world test cases to ensure stable behavior across informal, mixed, and noisy Persian text
+
+---
+
+📄 License
+
+MIT License — see [LICENSE](LICENSE).
+
+---
+
+🤝 Contributing
+
+Contributions are welcome.  
+Please open an issue or submit a pull request on GitHub.
+
+📝 Changelog
+
+See [CHANGELOG](CHANGELOG) for version history.

farsflow-0.1.0/pyproject.toml

@@ -0,0 +1,82 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "farsflow"
+version = "0.1.0"
+description = "Fast, modern, and modular Persian text preprocessing library."
+readme = "README.md"
+requires-python = ">=3.8"
+license = "MIT"
+authors = [
+    { name = "Mahdi Hosseini", email = "mhossza@gmail.com" }
+]
+
+keywords = [
+    "farsi",
+    "Persian",
+    "preprocessing",
+    "nlp",
+    "text-cleaning",
+    "normalization",
+    "tokenization",
+    "machine-learning",
+    "llm",
+]
+
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Programming Language :: Python",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Text Processing :: Linguistic",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+
+dependencies = [
+    # intentionally empty - farsflow is dependency-free
+]
+
+[project.urls]
+Homepage = "https://github.com/mhhoss/farsflow"
+Documentation = "https://github.com/mhhoss/farsflow"
+Source = "https://github.com/mhhoss/farsflow"
+Issues = "https://github.com/mhhoss/farsflow/issues"
+
+[tool.setuptools]
+package-dir = {"" = "src"}
+
+[tool.setuptools.packages.find]
+where = ["src"]
+include = ["farsflow*"]
+
+[tool.setuptools.package-data]
+"farsflow" = ["data/*.txt", "data/*.json"]
+
+[tool.black]
+line-length = 88
+target-version = ["py38"]
+
+[tool.isort]
+profile = "black"
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+
+[dependency-groups]
+dev = [
+    "build>=1.2.2.post1",
+    "twine>=6.1.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.3.5",
+]

farsflow-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

farsflow-0.1.0/src/farsflow/__init__.py

@@ -0,0 +1,25 @@
+from .pipeline.engine import Pipeline, create_pipeline
+from .processors.normalizer import NormalizerProcessor as Normalizer
+from .processors.joiner_fixer import JoinerFixerProcessor as JoinerFixer
+from .processors.space_cleaner import SpaceCleanerProcessor as SpaceCleaner
+
+
+def clean(text: str) -> str:
+    """
+    Run the default farsflow pipeline on the given text.
+
+    This is a convenience shortcut for simple use cases.
+    For more control, use `create_pipeline()` or `Pipeline` directly.
+    """
+    pipeline = create_pipeline()
+    return pipeline.run(text)
+
+
+__all__ = [
+    "Pipeline",
+    "create_pipeline",
+    "Normalizer",
+    "JoinerFixer",
+    "SpaceCleaner",
+    "clean",
+]

farsflow-0.1.0/src/farsflow/__version__.py

@@ -0,0 +1 @@
+__version__ = "0.1.0"

farsflow-0.1.0/src/farsflow/pipeline/engine.py

@@ -0,0 +1,54 @@
+from typing import List, Protocol
+
+from farsflow.processors.normalizer import NormalizerProcessor
+from farsflow.processors.joiner_fixer import JoinerFixerProcessor
+from farsflow.processors.space_cleaner import SpaceCleanerProcessor
+
+
+class Processor(Protocol):
+    """Interface for all processors in farsflow."""
+    def process(self, text: str) -> str:
+        ...
+
+
+class Pipeline:
+    """
+    A simple, modular pipeline engine for farsflow.
+    Executes a sequence of processors on input text.
+    """
+
+    def __init__(self, processors: List[Processor]):
+        self.processors = processors
+
+    def run(self, text: str) -> str:
+        """
+        Run the pipeline on the given text.
+        Each processor receives the output of the previous one.
+        """
+        for processor in self.processors:
+            text = processor.process(text)
+        return text
+    
+    def __call__(self, text: str) -> str:
+        """
+        Allow calling the pipeline instance directly: pipeline(text).
+        """
+        return self.run(text)
+
+
+def create_pipeline(processors: List[Processor] = None) -> Pipeline:
+    """
+    Creates a farsflow text-processing pipeline.
+
+    If no processors are provided, the default architecture is used:
+    Normalizer -> JoinerFixer -> SpaceCleaner
+    """
+    if processors is None:
+        processors = [
+            NormalizerProcessor(),
+            JoinerFixerProcessor(),
+            SpaceCleanerProcessor(),
+        ]
+
+    return Pipeline(processors)
+

farsflow-0.1.0/src/farsflow/processors/joiner_fixer.py

@@ -0,0 +1,86 @@
+import re
+
+ZWNJ = "\u200c"
+
+
+class JoinerFixerProcessor:
+    """
+    Restores correct Persian half-spaces (ZWNJ) based on safe, rule-based patterns.
+
+    - Only applies patterns that are highly reliable
+    - Does NOT try to be "smart" or guess semantics
+    - Keeps behavior deterministic and idempotent
+    """
+
+    def process(self, text: str) -> str:
+        # 0) Cleanup existing joiners/spaces to get a stable base
+        text = self._cleanup_wrong_joiners(text)
+
+        # 1) Prefix-based patterns (می / نمی)
+        text = self._apply_prefix_rules(text)
+
+        # 2) Suffix-based patterns (ها، فعل+ام/ای/اند)
+        text = self._apply_suffix_rules(text)
+
+        # 3) Verb conjugation patterns (رفته‌ام، گفته‌ای، دیده‌اند)
+        text = self._apply_verb_conjugation_rules(text)
+
+        # 4) Common, high-confidence compounds
+        text = self._apply_common_compounds(text)
+
+        # 5) Final cleanup (in case new patterns introduced extra spaces)
+        text = self._cleanup_wrong_joiners(text)
+
+        return text
+
+    def _apply_prefix_rules(self, text: str) -> str:
+        # می + فاصله + فعل/کلمه فارسی -> می‌فعل
+        text = re.sub(r"\bمی\s+(?=[آ-ی])", "می" + ZWNJ, text)
+
+        # نمی + فاصله + فعل/کلمه فارسی -> نمی‌فعل
+        text = re.sub(r"\bنمی\s+(?=[آ-ی])", "نمی" + ZWNJ, text)
+
+        return text
+
+    def _apply_suffix_rules(self, text: str) -> str:
+        # جمع "ها" — فقط اگر قبلش حرف/عدد باشد
+        text = re.sub(r"([آ-یA-Za-z0-9])\s+ها\b", r"\1" + ZWNJ + "ها", text)
+
+        # فعل/اسمِ ختم‌شده به «ه» + ام/ای/اند → ه‌ام/ه‌ای/ه‌اند
+        # (ضمیر ملکی عمومی مثل "ایران ام" را دست‌کاری نمی‌کنیم)
+        text = re.sub(r"([آ-ی]{2,}ه)\s+ام\b", r"\1" + ZWNJ + "ام", text)
+        text = re.sub(r"([آ-ی]{2,}ه)\s+ای\b", r"\1" + ZWNJ + "ای", text)
+        text = re.sub(r"([آ-ی]{2,}ه)\s+اند\b", r"\1" + ZWNJ + "اند", text)
+
+        return text
+
+    def _apply_verb_conjugation_rules(self, text: str) -> str:
+        # Note: این تابع فعلاً روی همان الگوهای بالا تکیه دارد؛
+        #       اگر در آینده الگوهای بیشتری اضافه شود، اینجا گسترش می‌یابد.
+        return text
+
+    def _apply_common_compounds(self, text: str) -> str:
+
+        # به کارگیری -> به‌کارگیری
+        text = re.sub(r"\bبه\s+کارگیری\b", "به" + ZWNJ + "کارگیری", text)
+
+        # دست خط -> دست‌خط
+        text = re.sub(r"\bدست\s+خط\b", "دست" + ZWNJ + "خط", text)
+
+        # Note: Only includes a few high‑confidence compounds in v0.1.0.
+        #       This list will expand gradually in future versions.
+
+        return text
+
+    def _cleanup_wrong_joiners(self, text: str) -> str:
+        # ۱) ZWNJهای تکراری -> یک ZWNJ
+        text = re.sub(ZWNJ + r"{2,}", ZWNJ, text)
+
+        # ۲) حذف فاصله بعد از ZWNJ
+        text = re.sub(ZWNJ + r"\s+", ZWNJ, text)
+
+        # ۳) حذف فاصله قبل از ZWNJ
+        text = re.sub(r"\s+" + ZWNJ, ZWNJ, text)
+
+        return text
+    

farsflow-0.1.0/src/farsflow/processors/normalizer.py

@@ -0,0 +1,62 @@
+class NormalizerProcessor:
+    """
+    Architecture-level Persian normalizer for farsflow.
+
+    - Minimal, deterministic, and semantic-safe
+    - Converts Arabic letters to Persian equivalents
+    - Removes unwanted invisible/control characters
+    - Optionally normalizes Persian/Arabic digits to ASCII digits
+    - Does NOT touch spacing, punctuation, or ZWNJ
+    - Suitable as a shared layer for ML/LLM/RAG/Embedding
+    """
+
+    CHAR_MAP = str.maketrans({
+        "ي": "ی",
+        "ى": "ی",
+        "ك": "ک",
+        "ؤ": "و",
+        "أ": "ا",
+        "إ": "ا",
+        "ة": "ه",
+        "ۀ": "ه",
+        # Note: "آ "هرگز تغییر نمی‌کند
+    })
+
+    # Remove or normalize only harmful/invisible characters
+    INVISIBLE_MAP = str.maketrans({
+        "\u200d": "",   # ZWJ
+        "\u200e": "",   # LRM
+        "\u200f": "",   # RLM
+        "\ufeff": "",   # BOM
+        "\u200b": "",   # zero-width space
+        "\u2060": "",   # word joiner
+        "\u00a0": " ",  # NBSP → normal space
+        # Note: ZWNJ (\u200c) حذف یا تبدیل نمیشود
+    })
+
+    # Persian + Arabic digits -> ASCII digits
+    DIGIT_MAP = str.maketrans({
+        "۰": "0", "۱": "1", "۲": "2", "۳": "3", "۴": "4",
+        "۵": "5", "۶": "6", "۷": "7", "۸": "8", "۹": "9",
+        "٠": "0", "١": "1", "٢": "2", "٣": "3", "٤": "4",
+        "٥": "5", "٦": "6", "٧": "7", "٨": "8", "٩": "9",
+    })
+
+    def __init__(self, normalize_digits: bool = True) -> None:
+        self.normalize_digits = normalize_digits
+
+    def process(self, text: str) -> str:
+        # 1) Arabic -> Persian letters
+        text = text.translate(self.CHAR_MAP)
+
+        # 2) Remove/normalize invisible/control characters (NOT ZWNJ)
+        text = text.translate(self.INVISIBLE_MAP)
+
+        # 3) Digit normalization (optional)
+        if self.normalize_digits:
+            text = text.translate(self.DIGIT_MAP)
+
+        # 4) SpaceCleanerProcessor handles spacing
+        # 5) JoinerFixerProcessor handles ZWNJ logic
+
+        return text

farsflow-0.1.0/src/farsflow/processors/space_cleaner.py

@@ -0,0 +1,54 @@
+import re
+
+
+class SpaceCleanerProcessor:
+    """
+    Cleans and standardizes spacing and punctuation for Persian text.
+    - Normalizes spaces
+    - Fixes spacing around punctuation
+    - Converts English punctuation to Persian equivalents
+    - Handles parentheses spacing safely
+    - Does NOT touch ZWNJ or semantic structure
+    """
+
+    PUNCT_MAP = str.maketrans({
+        ",": "،",
+        ";": "؛",
+        "?": "؟",
+    })
+
+    # Persian punctuation that must NOT have space before them
+    NO_SPACE_BEFORE = r"[،؛:؟!.,]"
+
+    # Punctuation that must have exactly one space after them
+    NEED_SPACE_AFTER = r"[،؛:؟!()]"
+
+    def process(self, text: str) -> str:
+
+        # 1) Convert English punctuation to Persian equivalents
+        text = text.translate(self.PUNCT_MAP)
+
+        # 2) Remove spaces before punctuation
+        text = re.sub(r"\s+(?=" + self.NO_SPACE_BEFORE + ")", "", text)
+
+        # 3) Ensure exactly one space after punctuation
+        text = re.sub(r"(" + self.NEED_SPACE_AFTER + r")(?=\S)", r"\1 ", text)
+
+        # 4) Fix spacing inside parentheses
+        text = re.sub(r"\(\s+", "(", text)
+        text = re.sub(r"\s+\)", ")", text)
+
+        # 5) Add missing space before "("
+        text = re.sub(r"([^\s(])\(", r"\1 (", text)
+
+        # 6) Add missing space after ")"
+        text = re.sub(r"\)([^\s)])", r") \1", text)
+
+        # 7) Remove space before punctuation after ")"
+        text = re.sub(r"\)\s+(?=" + self.NO_SPACE_BEFORE + ")", r")", text)
+
+        # 8) Collapse multiple spaces
+        text = " ".join(text.split())
+
+        return text
+    

farsflow-0.1.0/src/farsflow.egg-info/PKG-INFO

@@ -0,0 +1,126 @@
+Metadata-Version: 2.4
+Name: farsflow
+Version: 0.1.0
+Summary: Fast, modern, and modular Persian text preprocessing library.
+Author-email: Mahdi Hosseini <mhossza@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/mhhoss/farsflow
+Project-URL: Documentation, https://github.com/mhhoss/farsflow
+Project-URL: Source, https://github.com/mhhoss/farsflow
+Project-URL: Issues, https://github.com/mhhoss/farsflow/issues
+Keywords: farsi,Persian,preprocessing,nlp,text-cleaning,normalization,tokenization,machine-learning,llm
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest>=8.3.5; extra == "dev"
+Dynamic: license-file
+
+# farsflow
+
+farsflow is a small library for **Persian text preprocessing**, designed for practical NLP and LLM workflows.
+
+---
+
+## 🚀 Features
+
+- Deterministic pipeline (same input -> same output)
+- Safe character normalization
+- Joiner (ZWNJ) fixes
+- Whitespace & punctuation cleanup
+- Modular processors (use only the components you need)
+- Real‑world style sample tests
+- Zero dependencies
+
+---
+
+## 📦 Installation
+
+```bash
+pip install farsflow
+```
+
+## ✨ Quick Start
+
+```python
+import farsflow as ff
+
+text = "سلام  دنیا!  این یك   تست است  که می نویسم  ۴۵۶"
+cleaned = ff.clean(text)
+print(cleaned)
+```
+Expected output:  سلام دنیا! این یک تست است که می‌نویسم 456
+
+---
+
+## 🧩 Pipeline Components
+
+farsflow ships with a set of modular, composable components:
+
+- **Normalizer** — safe character normalization
+- **JoinerFixer** — fixes ZWNJ usage without over-correction
+- **SpaceCleaner** — trims redundant whitespace and punctuation spacing
+- **Pipeline** — orchestrates components in a deterministic order
+
+You can customize the pipeline:
+
+```python
+from farsflow import Pipeline, Normalizer, JoinerFixer
+
+pipeline = Pipeline(
+    Normalizer(),
+    JoinerFixer(),
+)
+
+text = "متن   تستی"
+pipeline(text)
+```
+
+---
+
+🧪 Testing
+
+```bash
+pytest
+# or:
+pytest path/to/test_file.py
+```
+
+---
+
+🗺 Roadmap (v0.2.0)
+
+- [ ] formalize the behavior of "ff.clean" as a safe and deterministic baseline
+- [ ] add optional normalization controls (e.g. digit normalization)
+- [ ] add optional noise-cleaning emoji and url processors
+- [ ] introduce simple profiles (ff.llm.clean, ff.embedding.query, ff.embedding.index) built on top of the same core
+- [ ] expand real-world test cases to ensure stable behavior across informal, mixed, and noisy Persian text
+
+---
+
+📄 License
+
+MIT License — see [LICENSE](LICENSE).
+
+---
+
+🤝 Contributing
+
+Contributions are welcome.  
+Please open an issue or submit a pull request on GitHub.
+
+📝 Changelog
+
+See [CHANGELOG](CHANGELOG) for version history.

farsflow-0.1.0/src/farsflow.egg-info/SOURCES.txt

@@ -0,0 +1,17 @@
+LICENSE
+README.md
+pyproject.toml
+src/farsflow/__init__.py
+src/farsflow/__version__.py
+src/farsflow.egg-info/PKG-INFO
+src/farsflow.egg-info/SOURCES.txt
+src/farsflow.egg-info/dependency_links.txt
+src/farsflow.egg-info/requires.txt
+src/farsflow.egg-info/top_level.txt
+src/farsflow/pipeline/__init__.py
+src/farsflow/pipeline/engine.py
+src/farsflow/processors/__init__.py
+src/farsflow/processors/joiner_fixer.py
+src/farsflow/processors/normalizer.py
+src/farsflow/processors/space_cleaner.py
+src/farsflow/profiles/__init__.py

farsflow-0.1.0/src/farsflow.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

farsflow-0.1.0/src/farsflow.egg-info/requires.txt

@@ -0,0 +1,3 @@
+
+[dev]
+pytest>=8.3.5

farsflow-0.1.0/src/farsflow.egg-info/top_level.txt

@@ -0,0 +1 @@
+farsflow