AbstractIntegratedModule 1.0.0__tar.gz → 1.0.1__tar.gz

{abstractintegratedmodule-1.0.0 → abstractintegratedmodule-1.0.1/AbstractIntegratedModule.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: AbstractIntegratedModule
-Version: 1.0.0
+Version: 1.0.1
 Summary: Library for Advanced Integrated Non-LLM AI Models - Optimized Backend Framework For Non-LLM AI Agent
 Author: Micro-Novelty
 Author-email: hernikpuspita5@gmail.com
@@ -33,31 +33,36 @@ Dynamic: summary
 # [=] IntegratedPipeline-Specialized-AI-Agent-library
 
 [~] Introduction:
-- IntegratedPipeline is a standalone Specialized AI Agent Library for Non-LLL memory - Augmented Agentic Framework orchestrator, Specifically designed to provide Agentic capability for any Autonomous Agentic Framework locally and Coordinatively that runs efficiently from consumer based machine to High-end embedded systems, where the AI Can directly and continously learn, with minimal and efficient compute, built-in augmented memory, Secure Peer-To-Peer (Multi-Agent) Coordination with security layers as an option, And Explainability capability based on proof from in it's internal metrics, reducing Black-Box condition necessary for reliability. 
-- AbstractIntegratedModule Contains specialized MLP using Its Own specialized geometric Weight shaping (AWE), Specialized efficient Transformer and LSTM (Long-short term memory) architecture for Scarce Data with Alpha-based computation, specifically designed for low-amount samples environment, or Messy environments.
+- IntegratedPipeline is a standalone Specialized AI Agent Library for Non-LLM memory Augmented Agentic Framework orchestrator - Specifically designed to provide Agentic capability for any Autonomous Agentic Framework locally and Coordinatively that runs efficiently from consumer based machine to High-end embedded systems, where the AI Can directly and continously learn, with minimal and efficient compute, built-in augmented memory, Secure Peer-To-Peer (Multi-Agent) Coordination with security layers as an option, And Explainability capability based on proof from in it's internal metrics, reducing Black-Box condition necessary for reliability. 
+- AbstractIntegratedModule Contains specialized MLP using Its Own specialized geometric Weight shaping (AWE), Specialized efficient Transformer and LSTM (Long-short term memory) architecture for Scarce Data with Alpha-based computation, specifically designed for low-amount samples environment or Messy environments.
 
-#### Github Repo (if you want to know or contribute)
-https://github.com/Micro-Novelty/IntegratedPipeline-Specialized-Non-LLM-AI-Agent-Framework
-
-#### Note: The README here you are reading is a direct copy from my README Repository, to download the necessary files, you can visit my Repository with the provided link above.
 
 ### Library Short Description:
-- Development Stage: 1.0.0 Official Release.
-- Maintainer: Micro-Novelty.
-- library Source-Code is Open-sourced on github.
+- Development Stage on PyPi: 1.0.1 Official Release.
+- Author and Maintainer: Micro-Novelty and EpsitronNet-bot.
+- library Source-Code is Open-sourced with MIT License.
 - Purpose: Specifically Designed for providing Non-LLM AI Agent Framework for edge Devices, Optimized for ARM64 architecture.
 - Library installation: 
   ```bash
   pip install AbstractIntegratedModule
   python -m install AbstractIntegratedModule
   ```
+
+### Github Link (for Visiting and cloning)
+- https://github.com/Micro-Novelty/IntegratedPipeline-Specialized-Non-LLM-AI-Agent-Framework
+
+- The library also includes precompiled binaries for:
+ - aarch64 manylinux (accepts version 2.17+) architecture, accepts python version 3.10, 3.11 only.
+ - aarch64 musllinux (accepts version 1.2+) architecture, accepts python version 3.10, 3.11 only.
+ - Windows 64 bit architecture (only python 3.13 only)
+
 - Library installation if you dont have aarch64 setup, you can download the correct wheel for your setup in this repository or by using pip:
   - ✨ use pip for downloading the correct wheels for your setup:
      - ```bash
        pip install abstractintegratedmodule --extra-index-url https://Micro-Novelty.github.io/abstract-modules/whl/ --break-system-packages
        # ensures proper installation by bypassing pip strict external download setup. 
        ```
-       - Note: By Using the above --extra-index installation method provides you with:
+       - Note: Using the above installation --extra-index method Provides you with:
        - x86_64 and aarch64 with manylinux (v. 2.17+) and musllinux (v. 1.2+) architecture only (No regular singular x86_64 arch wheel, tarball will be used for regular x86_64 and windows).
        - macOS with aarch64 and x86_64 architecture. (v. 10.9+)
        - All of this Wheels provided in the Repository Only Provides Wheels for python with version 3.10 3.11 and 3.12.
@@ -68,7 +73,7 @@ https://github.com/Micro-Novelty/IntegratedPipeline-Specialized-Non-LLM-AI-Agent
     - ```bash
       pip install abstract-weights-core
       ```
-    - Note: This Optimization would allow AbstractIntegratedModule faster JSON parsing and much more flexible database handling handled in Rust.
+    - Note: This Optimization would help AbstractIntegratedModule with faster JSON parsing and much more flexible database handling handled efficiently using Rust programming language.
    
 - Proven Capabilities:
    - The library has been thoroughly tested in Multiple Environments from Windows to ARM64 Environment. The library is now Robust for Wider use and Deployment.
@@ -78,12 +83,18 @@ https://github.com/Micro-Novelty/IntegratedPipeline-Specialized-Non-LLM-AI-Agent
    - LSTM is Optimized efficiently for scarce data with AWE method.
    - Robust Advanced prediction capabilities proven effective on ARM64 Using MLP + LSTM Architectures.
    - Transformer Optimized using Cython, to reduce Memory overhead and Reduce CPU Usage, With Reduced Training Time.
-______________
-  
-____________
+-----
+  - Changelog:
+     - v1.0.1:
+        - [=] New features:
+        - Applied clean up staled request after adding request for batch prediction to prevent memory leak.
+        - Fixed edge cases where Advanced prediction method reputation calibration function call receives final_probs=None in Multi-threading environment.
+        - Added parameters for users to directly set how much Epochs are needed for MLP and Transformer training.
+        - Added capability for the Model to save accurate answer to Database efficiently.
+        - Fixed bug that can cause shape and type errors in edge cases in calibration function inside MLP.
+       
 <img width="1280" height="600" alt="WhatsApp Image 2026-05-27 at 07 16 32" src="https://github.com/user-attachments/assets/4b58a556-45a3-419b-96fd-9c1b76cac574" />
 
-
 ## [+] MANN Intro
 [=] Memory augmented Neural network (MANN) is a neural network architecture coupled with an external, dynamic memory module, allowing it to store, retrieve, and update information similarly to a computer's RAM. Unlike traditional networks that store knowledge only in weight parameters, MANNs excel at fast learning, long-term dependency handling, and episodic recall, In IntegratedPipeline, Its memory is stored in a custom database inside your local machine, then later used for memory retrieval, transfered to the AI Dictionary where it can finnaly recall its memory when input condition matched with memory. 
 
@@ -119,7 +130,7 @@ Transformers are the modern standard for AI, introduced in 2017 with the famous
    - different from LLM that is static and cannot improve beyond its given training condition, AI Agent using IntegratedPipeline has a dynamic, flexible continously learning behavior with conditional training algorithms included in the library that has both supervised and unsupervised learning present, The learned input and predicted Output will be stored in the database, allowing it to recall its memory during processing and find matching known prediction given if input matched with the stored input inside the database. this Continous learning behavior is efficient because its not relying on weights for memory, allowing flexible and predictable behavior inside a given environment.
    
 3. Robust Specialized MLP, Transformer and LSTM Architecture with ensemble weighting architecture:
-  - IntegratedPipeline has 2 Different type's of AI Architecture stacked together, and one architecture to weight their confidence and probability fairly to get the final prediction, Specialized MLP for Noise robustness And Specialized Transformer that used Alpha-based Computing algorithm for contextual reasoning, LSTM architecture to provides proof-of-credibility over a certain output, acting as a support mechanism rather than Main orchestrator like MLP and Transformer. The reason why those Models complement and used together:
+  - IntegratedPipeline usually used 1 Model for regular prediction, but in some Conditions, it used 2 Different type's of AI Architecture stacked together, and one architecture to weight their confidence and probability fairly (ensemble method) to get the final prediction for the problem it faced, Specialized MLP for Noise robustness And Specialized Transformer that used Alpha-based Computing algorithm for contextual reasoning, LSTM architecture to provides proof-of-credibility over a certain output, acting as a support mechanism rather than Main orchestrator like MLP and Transformer. The reason why those Models complement and used together:
       - Specialized MLP Provides synchronous robust classification Against noise with its specialized Weight Encoder (AWE) to handle noise using eigenvalue based computing that is lightweight and efficient. This Method can't be replicated Inside Transformer FFN (Feed-forward-network) because of Transformer dynamic brute force computing where AWE-Based generated weight's get diluted over time by Transformer dynamic projection embedding, making AWE Generated weight causes inefficient inside Transformer dynamic FFN/QKV projection.
       - Specialized Transformer provides robust synchronous advanced contextual relationships, efficient data processing using Alpha based computing, The Transformer is tuned towards to be as flexible as possible to provide dynamic projection or fixed FFN projection training with minimal head's and dimension's to reduce computational power.
       - LSTM doesn't act as a Main orchestrator, instead it Provides coherent Short-term memory for the Ensemble architecture, acting as a support mechanism to provides proof-of-credibility of a given answer from past previous context input, this allows flexible and achievable Aggreement between Transformer and MLP over a short period of time.
@@ -161,35 +172,49 @@ Transformers are the modern standard for AI, introduced in 2017 with the famous
    2. - TF-IDF (Term Frequency-Inverse Document Frequency) is a numerical statistic used in machine learning and NLP to evaluate how important a word is to a document within a collection (corpus). It boosts rare words and penalizes common words (like "the", "and") by multiplying two metrics: how often a word appears in a document (TF) and the inverse frequency of the word across all documents (IDF). 
    3. - Explainability provides deeper transparency of why a model thought about a detail by showing its internal metrics like attention quality, from distributed peer memory or Ensemble prediction result's.
 
+
+### Important Note:
+- All of the files provided here can be found  in the github repository, if you are seeing this on PyPi and wanted to Try out the .py, .sh or Dockerfile scripts below, consider visiting the github link Above.
+
 ### Introduction and demo
-[![Introduction and demo:](https://youtube.com)](https://youtu.be/RmWvwDHU_QY?si=Lvl8mt8c_BnFypS_)
+- Video Documentation: [![Introduction and demo:](https://youtube.com)](https://youtu.be/RmWvwDHU_QY?si=Lvl8mt8c_BnFypS_)
+- Quick demo start: [main.py](main.py)
+  - purpose: let you demonstrate the advanced prediction method and asynchronous prediction directly.
+- Quick test of P2P:
+    - [multi_agent_client.py](P2P_Setups/multi_agent_client.py)
+    - [multi_agent_server.py](P2P_Setups/multi_agent_server.py)
+    - [P2PDirectTest.py](P2P_Setups/P2PDirectTest.py)
+    - Purpose: let you demonstrate simple P2P using AbstractIntegratedModule quickly, there may be bugs in this P2P setup so feel free to share it in issues.
 
 ## [=] Requirements
-[~] To run and execute IntegratedPipeline, Requirement's include Or see [Requirements](requirements.txt):
+[~] To run and execute IntegratedPipeline, Requirement's include Or see [Requirements](installation_guides/usage_needs.txt):
 - pip install for quick usage:
 - ```bash
   pip install AbstractIntegratedModule # or
   python -m pip install AbstractIntegratedModule
   ```
   
-- Download binaries manually in the repository (Choose one minimal, specified for your needs):
-- Binaries can be downloaded from repository release or code section.
-   - Windows Native OS - AbstractIntegratedModule.cp313-win_amd64 Supports Python 3.13 for Windows Only (Robust, Stable version)
-   - Linux x86_64 - AbstractIntegratedModule.cpython-312-x86_64-linux-gnu.so supports Python 3.12 only.
-   - Linux ARM64 - Raspberry Pi (Supports Raspberry pi 3 - 5) - AbstractIntegratedModule.cpython-310-aarch64-linux-gnu.so supports Python 3.10 only.
-   - Libraries needed (For binary usage):
+- (Optional) Download AbstractIntegratedModule Compiled Binary file extension manually in the repository for Users who can't use pip install (Choose one minimal, specified for your needs):
+   - AbstractOptimizedModules for Optimizing Transformer using Cython (Optional)
+   - AbstractIntegratedModule (Main library)
+   - Compiled Binaries can be downloaded from release section in the Main repo or You can use this link:
+     - Release section: https://github.com/Micro-Novelty/IntegratedPipeline-Specialized-Non-LLM-AI-Agent-Framework/releases
+     - Supported OS: Mac-Os, linux x86-64 and Windows.
+  
+- NOTE: for Mac-OS or linux x86-64 Users, consider downloading the correct python wheel for your setup in here for a much flexible installation:
+   - ```bash
+     pip install abstractintegratedmodule --extra-index-url https://Micro-Novelty.github.io/abstract-modules/whl/ --break-system-packages
+     ```
+     Note: using this installation method grants you a much safer, predictable behavior, and more secure AbstractIntegratedModule library rather than a pure binary file.
+  
+   - (Optional) Libraries needed (For using Compiled binary, not python wheels):
      - Pandas
      - aiohttp
      - scikit-learn
      - numpy
      - psutil
-
-   - Download AbstractIntegratedModule binaries from release section or in here, [AbstractIntegratedModule Package](AbstractIntegratedModule-Package.zip)
-     - This file contains:
-      - AbstractIntegratedModule.pyd (For windows machine).
-      - AbstractIntegratedModule.cpython-39-x86_64-linux-gnu.so (For linux x86_64).
-      - AbstractIntegratedModule.cpython-39-aarch64-linux-gnu.so (for Linux ARM64 - Raspberry Pi).
-
+   - (Optional) Install AbstractIntegratedModule Compiled binaries from release section:
+  
 [=] for labels assignation:
 - CSV file that contains training labels and titles used for training and prediction label map, example will be provided below, Go to Step's for in-depth Usage below.
 
@@ -213,20 +238,20 @@ Transformers are the modern standard for AI, introduced in 2017 with the famous
    - Consider using faster storage (USB SSD) for better performance
 
 ## [=] Docker Container Application
-0. See [Docker_installation_Section](Docker_installation_Section.md) for an in-depth explanation, or [Quick_Docker_start](Quick_Docker_start.sh) for a quick start.
+0. See [Docker_installation_Section](docker-setups/Docker_installation_Section.md) for an in-depth explanation, or [Quick_Docker_start](docker-setups/Quick_Docker_start.sh) for a quick start.
    - Note Consider checking:
-     - [Dockerfile](Dockerfile) contains all the instructions need to assemble a Docker container.
-     - [start.sh](start.sh) for Quick single agent start in Docker container.
-     - [entrypoint.sh](entrypoint.sh) for a Smart entrypoint used in Dockerfile (Must be downloaded too along with Dockerfile and put in the same folder with Dockerfile after its downloaded)
-     - [start-multi-agent-cluster.sh](start-multi-agent-cluster.sh) for Multi-agent cluster start in Docker container, What it does:
+     - [Dockerfile](docker-setups/Dockerfile) contains all the instructions need to assemble a Docker container.
+     - [start.sh](docker-setups/start.sh) for Quick single agent start in Docker container.
+     - [entrypoint.sh](docker-setups/entrypoint.sh) for a Smart entrypoint used in Dockerfile (Must be downloaded too along with Dockerfile and put in the same folder with Dockerfile after its downloaded)
+     - [start-multi-agent-cluster.sh](P2P_Setups/start-multi-agent-cluster.sh) for Multi-agent cluster start in Docker container, What it does:
        - Starts a multi-agent Docker cluster — runs docker-compose up -d in detached mode (background), scaling the agent-client service to 5 simultaneous instances, alongside whatever server is defined in the docker-compose.yml.
        - Tails the logs — runs docker-compose logs -f which streams live logs from all containers (server + all 5 clients) to your terminal until you hit Ctrl+C.
      - [main.py](main.py) for executing a python script in the Docker container that used main.py, like in this code:
         - ```bash
           docker run -it -v $(pwd):/app/data integrated-pipeline:latest python /app/data/main.py
           ```
-     - To use a [.dockerignore](.dockerignore) file, place it in your build context directory (the same location as your Dockerfile) to specify which files and folders should be excluded when building your image.
-     - [.env](.env) is used for environment setup in [entrypoint.sh](entrypoint.sh), this file must be in the same folder with Dockerfile and entrypoint.sh after its downloaded.
+     - To use a [.dockerignore](docker-setups/.dockerignore) file, place it in your build context directory (the same location as your Dockerfile) to specify which files and folders should be excluded when building your image.
+     - [.env](.env) is used for environment setup in [entrypoint.sh](docker-setups/entrypoint.sh), this file must be in the same folder with Dockerfile and entrypoint.sh after its downloaded.
 
 1. Build Image:
    - Clone repository:
@@ -235,9 +260,9 @@ Transformers are the modern standard for AI, introduced in 2017 with the famous
      cd IntegratedPipeline-Continous-Learning-AI-Agent-library-framework
      ```
    - Download:
-   - [Dockerfile](Dockerfile),
-   - [entrypoint.sh](entrypoint.sh)
-   - [.env](.env)
+   - [Dockerfile](docker-setups/Dockerfile),
+   - [entrypoint.sh](docker-setups/entrypoint.sh)
+   - [.env](docker-setups/.env)
    - In the code or release section.
      
      - If the downloaded Dockerfile or .dockerignore or .env has .txt extension, remove the extension:
@@ -256,7 +281,7 @@ Transformers are the modern standard for AI, introduced in 2017 with the famous
      sudo docker build -t integrated-agent.
      ```
 2. Download library dependencies for binaries usage:
-   [=] Required libraries (when using binary):
+   - [Optional]: Required libraries (when using binary, not python wheel provided in the library):
    - Numpy
    - Scikit-learn
    - pandas
@@ -264,7 +289,7 @@ Transformers are the modern standard for AI, introduced in 2017 with the famous
    - psutil
      
 4. Run IntegratedPipeline in a Container:
-    - Install AbstractIntegratedModule via PIP or binaries (for binaries, can  be downloaded in release section):
+    - Install AbstractIntegratedModule via PIP or Compiled binaries:
     - ```bash
       pip install AbstractIntegratedModule # or
       python -m pip install AbstractIntegratedModule
@@ -304,7 +329,7 @@ Transformers are the modern standard for AI, introduced in 2017 with the famous
    ```
    
    [=] Multi agent P2P (Consider docker-compose) :
-   - Note: Use the provided [docker-compose.yml](docker-compose.yml) for Quick multi-agent.
+   - Note: Use the provided [docker-compose.yml](docker-setups/docker-compose.yml) for Quick multi-agent.
    - Navigate to the folder: Use the cd command to enter the directory containing the docker-compose.yml file.
    - ```
      cd /path/to/your/folder
@@ -342,7 +367,7 @@ A. [=] Computational performance results with Transformer included during Advanc
 14:50:36.109 449.89% 116.8MiB / 3.71GiB
 14:50:38.123 423.56% 119MiB   / 3.71GiB
 ```
-- [=] Note: To read the full logs (For included Transformer performance): [performance_log_with_TF](performance_log_with_TF.txt)
+- [=] Note: To read the full logs (For included Transformer performance): [performance_log_with_TF](performance_logs/performance_log_with_TF.txt)
 - [=] Explanation: 
   - Average CPU Usage = 420% - 430% , Sustained roughlt 4.20 - 4.30 CPU Cores
     - Meaning = - Compute-Intensive workload
@@ -379,7 +404,7 @@ B. [=] Advanced Prediction without Transformer, Only Specialized MLP + LSTM usin
 10:02:16.873 98.84% 333.8MiB / 3.71GiB
 10:02:18.869 0.00%  132.7MiB / 3.71GiB # container stopped and round 2 finished
 ```
-- [=] Note: See full performance log in here: [performance_log_only_MLP](performance_log_only_MLP.txt)
+- [=] Note: See full performance log in here: [performance_log_only_MLP](performance_logs/performance_log_only_MLP.txt)
 - [=] Explanation:
      - Average CPU Usage = 85-90% Used, sustained roughly 1 CPU Core saturated on average
          - Meaning: - Very efficient computation
@@ -402,7 +427,9 @@ _______________________________________
 ## Source code of AbstractIntegratedModule 
 - Note: The source code is provided in the repository.
 - [=] Full Monolithic extensively-documented source code (12K+ Lines): [AbstractIntegratedPipeline-SourceCode.zip](AbstractIntegratedPipeline-SourceCode.zip)
-- [=] Separated Modules of AbstractIntegratedModule: [AbstractIntegratedModule-separated-modules](AbstractIntegratedModule-separated-modules)
+- [=] Separated Modules of AbstractIntegratedModule: [separated-modules](separated-modules)
+- [=] up-to-date Source code: [AbstractIntegratedModule.pyx](src/AbstractIntegratedModule.pyx) and [AbstractOptimizedModules.pyx](src/AbstractOptimizedModules.pyx) files for independent compilation or direct compiling on real ARM64 device or if you want to see the up-to-date, undocumented version of AbstractIntegratedModule library.
+- [lib.rs](rust_optimization_setups/lib.rs) that contains Rust optimization for Models weight handling.
 - [~] Note:
    - The source code is open and Free to anyone who:
    - Use it: Run the software for any personal, academic, or commercial purpose.
@@ -425,96 +452,70 @@ _______________________________________
      python -m pip install AbstractIntegratedModule
      ```
 
-1. Download for binaries:
-   - AbstractIntegratedModule.pyd (For Windows) (Python 3.13), 
-   - AbstractIntegratedModule.cpython-39-x86_64-linux-gnu.so (For linux x86_64)  (Python 3.12)
-   - AbstractIntegratedModule.cpython-39-aarch64-linux-gnu.so (for Linux ARM64 - Raspberry Pi) (Python 3.10)
-   -  ```
-      # Download from release
-      # AbstractIntegratedModule.pyd (windows) /
-      # Abstractcpython-39-x86_64-linux-gnu.so (x86_64) /
-      # AbstractIntegratedModule.cpython-39-aarch64-linux-gnu.so   
-      ```
-   
-   [=] Steps for installation:
-   Note: AbstractIntegratedModule has 5 library dependencies that must be installed on your computer if you use AbstractIntegratedModule binaries:
-   - [=] Required Libraries (For binaries usage):
-   - Numpy
-   - Scikit-learn
-   - pandas
-   - aiohttp
-   - psutil
-   
-   1. Clone repository:
-         - ```
-           # prerequisites (for Raspberry pi OS Only)
-           # Update system
-           sudo apt-get update
-           sudo apt-get upgrade -y
-
-           # Install Python 3.13 and development tools
-           sudo apt-get install python3.13 python3.13-dev python3.13-venv -y
-
-           # Install additional build tools
-           sudo apt-get install build-essential libatlas-base-dev libjasper-dev -y
-
-           # Clone immediately for Windows and x86_64 only without prerequisites          
-           git clone https://github.com/Micro-Novelty/IntegratedPipeline-Continous-Learning-AI-Agent-library-framework.git
-           cd IntegratedPipeline-Continous-Learning-AI-Agent-library-framework     
-           ```   
-   2. Install System Dependencies (for x86_64 installation):
-      ```
-      # Ubuntu/Debian
-      sudo apt-get update
-      sudo apt-get install python3.13 python3.13-dev python3.13-venv
+1. Clone repository:
+
+ - ```
+   # prerequisites (for Raspberry pi OS Only)
+   # Update system
+   sudo apt-get update
+   sudo apt-get upgrade -y
+
+   # Install Python 3.x and development tools
+   # Version 3.x means you can install python 3.10 to 3.13 only,
+   # choose one version specified for your needs (3.10 or 3.11 or 3.12 ...).
+   sudo apt-get install python3.x python3.x-dev python3.x-venv -y
+
+   # Install additional build tools
+   sudo apt-get install build-essential libatlas-base-dev libjasper-dev -y
 
-      # CentOS/RHEL
-      sudo yum install python313 python313-devel
+   # Clone immediately for Windows and x86_64 only without prerequisites          
+   git clone https://github.com/Micro-Novelty/IntegratedPipeline-Continous-Learning-AI-Agent-library-framework.git
+   cd IntegratedPipeline-Continous-Learning-AI-Agent-library-framework     
+   ```   
+ 2. Install System Dependencies (for x86_64 installation):
+    ```
+    # Ubuntu/Debian
+    # example with Version 3.10:
+    sudo apt-get update
+    sudo apt-get install python3.10 python3.10-dev python3.10-venv
+
+    # CentOS/RHEL
+    sudo yum install python310 python310-devel
 
-      # Fedora
-      sudo dnf install python3.13 python3.13-devel
+    # Fedora
+    sudo dnf install python3.10 python3.10-devel
+    ```
+    
+ 3. Create a virtual environment:
+     - ```
+       # Create virtual environment (windows)
+       # example with python 3.10
+       python -m venv venv
+       # Activate virtual environment
+       venv\Scripts\activate
+       
+       # Create virtual environment (x86_64) (ARM64 / raspberry pi)
+       python3.10 -m venv venv
+       # Activate virtual environment
+       source venv/bin/activate
+       ```
+         
+   
+ 4. Verify Installation:
+    - ```
+      python -c "from AbstractIntegratedModule import IntegratedPipeline; print('✓ Installation successful!')"
       ```
       
-   3. Create a virtual environment:
-       - ```
-         # Create virtual environment (windows)
-         python -m venv venv
-         # Activate virtual environment
-         venv\Scripts\activate
-         
-         # Create virtual environment (x86_64) (ARM64 / raspberry pi)
-         python3.13 -m venv venv
-         # Activate virtual environment
-         source venv/bin/activate
-         ```
-         
-   4. Copy AbstractIntegratedModule binary:
-      - ```
-        # For windows:
-        # Copy the .pyd file to your project root
-        # AbstractIntegratedModule.pyd
-        copy C:\path\to\AbstractIntegratedModule.pyd .\AbstractIntegratedModule.pyd
-        
-        # Copy the .so file to your project root (for x86_64)
-        cp /path/to/AbstractIntegratedModule.cpython-39-x86_64-linux-gnu.so ./AbstractIntegratedModule.cpython-39-x86_64-linux-gnu.so
-        
-        # Copy ARM64 / Raspberry pi binary
-        cp /path/to/AbstractIntegratedModule.cpython-39-aarch64-linux-gnu.so ./AbstractIntegratedModule.cpython-39-aarch64-linux-gnu.so
-
-   5. Verify Installation:
-      - ```
-        python -c "from AbstractIntegratedModule import IntegratedPipeline; print('✓ Installation successful!')"
-        ```
-        
-   6. Run main.py for quick test of successful imports:
-      - ```
-        # run this for quick import test.
-        python main.py
-        ```
+ 5. Run main.py for quick test of successful imports:
+    - Download main.py in our repository code section: [main.py](main.py)
+    - ```
+      # run this for quick import test.
+      python main.py
+      ```
            
          
      
-3. Create CSV file that contains training labels and titles:
+3. Create CSV file that contains training labels and titles (Optional for training using texts):
    -  Example format:
       ```txt
       window_title,label
@@ -532,6 +533,7 @@ _______________________________________
       ```
       Note = window_title is target_title and label is target_label, check step below to use it.
 
+
 4. Use IntegratedPipeline as in this example:
    ```python
    from AbstractIntegratedModule import IntegratedPipeline
@@ -545,11 +547,11 @@ _______________________________________
    # SSL Setup for users who used Lets encrypt / Public CA cert:
    # Server context
    server_ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
-   server_ctx.load_cert_chain('their.crt', 'their.key') # No load_verify_locations needed — OS trust store handles public CA
+   server_ctx.load_cert_chain('your_server.crt', 'your_server.key') # No load_verify_locations needed — OS trust store handles public CA
    
    # Client context  
    client_ctx = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
-   client_ctx.load_cert_chain('their.crt', 'their.key') # no load_verify_locations — OS trust store handles it
+   client_ctx.load_cert_chain('your_client.crt', 'your_client.key') # no load_verify_locations — OS trust store handles it
 
    # for enterprise and internal CA:
    '''
@@ -566,14 +568,14 @@ _______________________________________
    client_ctx.check_hostname = False
    ''' # uncomment to use
 
-   # Note: The Above setup is Important for Secure P2P In deployment case, for local P2P the above setup is optional and automatic self-signed CERT will be used for local P2P.
+   # Note: The Above setup is Important for Users who wants to do Secure P2P In deployment case, for local P2P the above setup is optional and automatic self-signed CERT will be used for local-device P2P.
 
    main_model = IntegratedPipeline(
       memory_name=memory_name,  # memory name for the AI you already initialized
       use_async=True, # local asynchronous prediction is permitted, if not PipelineAsyncManager wont start asynchronous prediction.
       agent_port=5001, # this port is used to set AgentDistributedInference server (optional)
       ssl_cert_file=cert_file, ssl_key_file=key_file,# provide your cert_file path or key_file path (optional)
-      ssl_context=server_ctx, # used by the Agent server (optional)
+      ssl_context=server_ctx, # used by the Agent server. (optional)
       client_ssl_context=client_ctx # used by the client. (optional)
       ) 
 
@@ -613,7 +615,15 @@ _______________________________________
    main_model.show_explainability_details = True
    main_model.distribution.predict_manager = main_prediction # set PipelinePredictionManager to AgentDistributedInference for asynchronous prediction later (Very important for asynchronous prediction)
    # main_model.use_transformer = True if you want to use transformer, this will notify all modules that used advanced_prediction_method will initiate prediction with both transformer and MLP.
-   
+
+   # set IntegratedPipeline Penalty rate when it output wrong answer:
+   main_model.error_decay = 0.75
+   # error_rate > 0.5 means old errors fade quickly — a class that was wrong 3 predictions ago matters less than one wrong just now, making the model less likely to output repetitive wrong answer.
+   # this a flexible tunable-knob for the model judgement regarding wrong answer, this will propagate through prediction layers to inform about the model repetitive answer and calibrate it immediately.
+
+   main_model.mlp_training_epochs = 2000
+   main_model.transformer_training_epochs = 200
+   # You can set how much epochs are needed to Train your MLP and Transformer for your Models.
    
    # test samples with more sophisticated rules and more complex titles for prediction
    # (title, intent)
@@ -637,14 +647,24 @@ _______________________________________
    # prevent the model from training and make weights unchanged for static prediction.
       
    results, chosen_label, confidence = main_prediction.advanced_prediction_method(
-      test_titles, label_map, example_rules,
+      titles=test_titles, label_map=label_map, rules=example_rules, # titles and rules can be set to None (Optional samples), but label_map must NOT be None.
          X=None, y=None # you could create your own X and y samples and put it here (Optional)
                show_proba=False, top_k=3, 
                use_transformer=True,
                return_attention=False,
                save_results=True,
                batch_size=2)
+   # Important Note: If you set titles and rules to None, you must provide X and y samples for prediction, otherwise the models cant predict anything.
    # batch size=2 is needed during transformer training for batching, if you have larger samples consider using batch_size > 8, for medium amount of samples (>10 -> <50 samples) consider using 2 or 4 batch_size.
+
+   # This setup below would allow you to save the Accurate answer (if the model guessed a specific problem correct) directly to the database,
+   # without initiating prediction over the sample repeatedly.
+   input_ids = main_model.model2.cache['input_ids'] # input indices the transformer used as input (model2 is Transformer).
+   index = results.get('index') # predicted index where the Model have chosen the label.
+   confidence = results.get('confidence') # the model confidence over the predicted Output from advanced prediction method.
+   main_model.accurate_cache_lookup.add_verified(X, input_ids, chosen_label, confidence, index,
+                     source='Correctly-Answered') # You can modify the the source with "Answered-Correctly" (fit to your needs)
+   
    # ... more features you can add
    ```
    
@@ -700,7 +720,10 @@ async_manager = PipelineAsyncManager(main_model,
 
 async_manager.start(method='Transformer_included', bootstrap_token=None) # boothstrap token is optional for better security
 
-texts = {'test_titles': test_titles, 'label_map': label_map, 'rules': example_rules, 'X': None, 'y':None, 'use_transformer': True} # all samples needed for advanced prediction method. (X and y are optional samples)
+texts = {'test_titles': test_titles, 'label_map': label_map, 'rules': example_rules, # test_titles and rules can be set to None here since they are Optional samples.
+                                                                                     # but label_map must not be None.
+          'X': None, 'y':None, 'use_transformer': True} # all samples needed for advanced prediction method. (X and y are optional samples)
+# Important Note: If you set titles and rules to None, you must provide X and y samples for prediction, otherwise the models cant predict anything.
 
 regular_predict = async_manager.predict(
    texts=texts,
@@ -710,11 +733,12 @@ regular_predict = async_manager.predict(
 
 # with retries: async_manager.predict(texts, timeout=60, retries=5, api_key=secret_key) # 5 times retry if failed
 
+# NOTE: This function below must require test_titles or titles, label_map and example rules:
 print('[==] Initiating advanced batch prediction')
          predicted_output = async_manager.advanced_batch_prediction(test_titles, label_map, example_rules, 
          X=None, y=None, # provide your initialized X and y samples (Also Optional, can be set to None)
          secret_key=secret_key, client_ip=None) # you can add client_ip to provide a robust authentication paired with secret_key
-# for better and faster advanced prediction, consider using advanced batch prediction like in the above example
+# for better and faster advanced prediction when using titles and rules, consider using advanced batch prediction like in the above example
 
 ```
 [=] Note:
@@ -819,7 +843,9 @@ try:
      api_key = agent1.get_api_key()
      print(f"\n🔑 Using API Key: {api_key[:20]}...")
 
-     texts = {"test_titles": test_titles, "label_map": label_map, "rules": rules, 'X': None, "y":None, "use_transformer": True, "agent_id": agent_id} # (X and y are optional samples here too)
+     texts = {"test_titles": test_titles, "label_map": label_map, "rules": rules, # test_titles and rules are Optional samples, can be set to None.
+'X': None, "y":None, "use_transformer": True, "agent_id": agent_id} # (X and y are optional samples here too)
+     # Important Note: If you set titles and rules to None, you must provide X and y samples for prediction, otherwise the models cant predict anything.
 
      # texts dictionary must contain test_titles, label_map, and rules that you can assign,
      # agent ID can be strings, int, or floats, recommendded to make it long for better security.
@@ -877,7 +903,7 @@ finally:
     - This setup used Hybrid feature in prediction handling, Asynchronous prediction request, and Synchronous prediction handling. Synchronous prediction does block code execution for a few seconds, it was used for a few reason here, such as:
        - allowing a more slower traffic between agents, preventing other agent to get the same peer prediction over time, making each interaction equals and each peer can receive different peer prediction output.
        
-[=] You can download this setup here for a direct test: [P2PDirectTest.py](P2PDirectTest.py)
+[=] You can download this setup here for a direct test: [P2PDirectTest.py](P2P_Setups/P2PDirectTest.py)
 
    - [=] for probability coordination, locally, get peers data from database or via socket.
 ```python
@@ -936,8 +962,8 @@ calibrated_probability = main_model._handle_distributed_connections(probs, attn_
 ```
 [~] Note: this calibrated_probability is later used to calculate confidence and chosen output based on given label_map.
    - Consider checking:
-     - [multi_agent_client.py](multi_agent_client.py) for a In-depth start for client testing.
-     - [multi_agent_server.py](multi_agent_server.py) for a In-depth start for server testing.
+     - [multi_agent_client.py](P2P_Setups/multi_agent_client.py) for a In-depth start for client testing.
+     - [multi_agent_server.py](P2P_Setups/multi_agent_server.py) for a In-depth start for server testing.
    - If you get undefined NoneType Behavior when using .accept(), consider see [Troubleshooting](#Troubleshooting) Issue 7 for a Quick fix.
         
 6. Cross-Session availability:
@@ -1191,7 +1217,7 @@ purpose: handles Proper LSTM Confidence and calibration layer.
   - [ROADMAP.md](ROADMAP.md)
   - [Contributing.md](Contributing.md)
   - [changelog.md](changelog.md)
-  - [requirements-dev.txt](requirements-dev.txt) for contributors requirements.
+  - [requirements-For-Dev](dev_needs.txt) for contributors requirements.
   - [architecture_diagram.js](architecture_diagram.js).
 
 ## License