AbstractIntegratedModule 0.3.1__tar.gz → 0.3.3__tar.gz

{abstractintegratedmodule-0.3.1 → abstractintegratedmodule-0.3.3}/AbstractIntegratedModule.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: AbstractIntegratedModule
-Version: 0.3.1
+Version: 0.3.3
 Summary: Framework for Advanced Integrated Non-LLM AI Module library - Backend Framework for Non-LLM AI Agent Framework
 Author: Micro-Novelty
 Author-email: hernikpuspita5@gmail.com
@@ -32,7 +32,9 @@ Dynamic: summary
 
 # [=] IntegratedPipeline-Specialized-AI-Agent-library
 
-[~] IntegratedPipeline is a standalone Specialized AI Agent Library for Non-LLL memory Augmented Agentic Framework orchestrator, Specifically designed to provide Agentic capability for any Autonomous Agentic Framework locally and Coordinatively that runs efficiently from consumer based machine to High-end embedded systems, where the AI Can directly and continously learn, with minimal and efficient compute, built-in augmented memory, Secure Peer-To-Peer (Multi-Agent) Coordination with security layers as an option, And Explainability capability based on proof from in it's internal metrics, reducing Black-Box condition necessary for reliability. Containing specialized MLP using Its Own specialized geometric Weight shaping (AWE) and Specialized efficient Transformer for Scarce Data with Alpha-based computation, specifically designed for low-amount samples environment, or Messy environments.
+[~] Introduction:
+- IntegratedPipeline is a standalone Specialized AI Agent Library for Non-LLL memory - Augmented Agentic Framework orchestrator, Specifically designed to provide Agentic capability for any Autonomous Agentic Framework locally and Coordinatively that runs efficiently from consumer based machine to High-end embedded systems, where the AI Can directly and continously learn, with minimal and efficient compute, built-in augmented memory, Secure Peer-To-Peer (Multi-Agent) Coordination with security layers as an option, And Explainability capability based on proof from in it's internal metrics, reducing Black-Box condition necessary for reliability. 
+- AbstractIntegratedModule Contains specialized MLP using Its Own specialized geometric Weight shaping (AWE), Specialized efficient Transformer and LSTM (Long-short term memory) architecture for Scarce Data with Alpha-based computation, specifically designed for low-amount samples environment, or Messy environments.
 
 #### Github Repo (if you want to know or contribute)
 https://github.com/Micro-Novelty/IntegratedPipeline-Specialized-Non-LLM-AI-Agent-Framework
@@ -40,7 +42,7 @@ https://github.com/Micro-Novelty/IntegratedPipeline-Specialized-Non-LLM-AI-Agent
 #### Note: The README here you are reading is a direct copy from my README Repository, to download the necessary files, you can visit my Repository with the provided link above.
 
 ### Library Short Description:
-- Development Stage: Beta, 0.3.1.
+- Development Stage: Beta, 0.3.3.
 - Maintainer: Micro-Novelty.
 - library Source-Code is Open-sourced on github.
 - Purpose: Specifically Designed for providing Non-LLM AI Agent Framework for edge Devices, Optimized for ARM64 architecture.
@@ -59,6 +61,7 @@ https://github.com/Micro-Novelty/IntegratedPipeline-Specialized-Non-LLM-AI-Agent
 - Proven Capabilities: 
    - Proven Works on ARM64 Environment, Training and Prediction works efficient on Docker ARM64 environment with QEMU, good parallelizing behavior is guaranteed.
    - P2P Works efficiently in ARM64 Docker + QEMU, No conflicting socket and all prediction works efficiently.
+   - AWE setup Proven Efficient on Hard-uncontrolled dataset such as Activity Recognition from the given Database.
 
 -----
 
@@ -66,11 +69,26 @@ https://github.com/Micro-Novelty/IntegratedPipeline-Specialized-Non-LLM-AI-Agent
 
 
 ## [+] MANN Intro
-[=] Memory augmented Neural network (MANN) is a neural network architecture coupled with an external, dynamic memory module, allowing it to store, retrieve, and update information similarly to a computer's RAM. Unlike traditional networks that store knowledge only in weight parameters, MANNs excel at fast learning, long-term dependency handling, and episodic recall, In IntegratedPipeline, Its memory is stored in a custom database inside your local machine, then later used for memory retrieval, transfered to the AI Dictionary where it can finnaly recall its memory when input condition matched with memory.
+[=] Memory augmented Neural network (MANN) is a neural network architecture coupled with an external, dynamic memory module, allowing it to store, retrieve, and update information similarly to a computer's RAM. Unlike traditional networks that store knowledge only in weight parameters, MANNs excel at fast learning, long-term dependency handling, and episodic recall, In IntegratedPipeline, Its memory is stored in a custom database inside your local machine, then later used for memory retrieval, transfered to the AI Dictionary where it can finnaly recall its memory when input condition matched with memory. 
 
 ## [+] Abstract Weight Encoder (AWE) Intro
 [=] AWE is a specialized custom weight shaping or encoding method Specifically designed for MLP Architecture, using eigenvalue and spectral methods as base equations to calculate covariance inside a given input data, and shape the correct Weight from the given eigenvalue, AWE Works by processing input and then captures the necessary eigenvalue to shape a properly initialized Weight that aligns with input data complexity, With this approach, MLP training will be much more consistent and robust against noise.
 
+## [+] LSTM And Transformer Intro:
+A. LSTMs (Long Short-Term Memory) and Transformers are foundational deep learning architectures built to process sequential data (like text or time series). While both handle the flow of time and context, they do so using completely different mechanisms.
+- LSTM (Long Short-Term Memory):
+  - LSTMs are an advanced class of Recurrent Neural Networks (RNNs) introduced to solve the problem of traditional RNNs forgetting earlier data.
+  - How it works: LSTMs process data sequentially—one word or time-step at a time. They regulate information using "gates" (forget, input, and output) that determine what information from the sequence to keep or discard.
+  - Use Cases: Ideal for tasks with strictly continuous chronological data like time-series forecasting (e.g., stock market or weather prediction) or speech recognition.
+  - Limitations: Because they process data in a strict chain, it is difficult to parallelize training, making them slow and prone to forgetting long contexts.
+
+B. Transformer:
+Transformers are the modern standard for AI, introduced in 2017 with the famous "Attention Is All You Need" paper.
+- How it works: Instead of reading sequences linearly, Transformers read the entire sequence all at once. They utilize a self-attention mechanism, which calculates how much "attention" or weight every part of the sequence should give to every other part, understanding the global context instantly.
+- Use Cases: The backbone of Large Language Models (LLMs) like ChatGPT or BERT, making them perfect for machine translation, text generation, and summarization.
+- Limitations: Transformers require massive amounts of training data and computing power to work effectively.
+
+
 [~] For a much In-Depth Explanation You can visit This repository to learn more about AWE and its performance results:
 - Link: https://github.com/Micro-Novelty/Specialized-MLP-for-noise-robustness
 
@@ -84,12 +102,14 @@ https://github.com/Micro-Novelty/IntegratedPipeline-Specialized-Non-LLM-AI-Agent
 2. Continously Learning behavior for an Agent:
    - different from LLM that is static and cannot improve beyond its given training condition, AI Agent using IntegratedPipeline has a dynamic, flexible continously learning behavior with conditional training algorithms included in the library that has both supervised and unsupervised learning present, The learned input and predicted Output will be stored in the database, allowing it to recall its memory during processing and find matching known prediction given if input matched with the stored input inside the database. this Continous learning behavior is efficient because its not relying on weights for memory, allowing flexible and predictable behavior inside a given environment.
    
-3. Robust Specialized MLP and Transformer Architecture with ensemble weighting:
-  - IntegratedPipeline has 2 Different type's of AI Architecture stacked together, and one architecture to weight their confidence and probability fairly to get the final prediction, Specialized MLP for Noise robustness And Specialized Transformer that used Alpha-based Computing algorithm for contextual reasoning, The reason why those Models complement and used together :
+3. Robust Specialized MLP, Transformer and LSTM Architecture with ensemble weighting architecture:
+  - IntegratedPipeline has 2 Different type's of AI Architecture stacked together, and one architecture to weight their confidence and probability fairly to get the final prediction, Specialized MLP for Noise robustness And Specialized Transformer that used Alpha-based Computing algorithm for contextual reasoning, LSTM architecture to provides proof-of-credibility over a certain output, acting as a support mechanism rather than Main orchestrator like MLP and Transformer. The reason why those Models complement and used together:
       - Specialized MLP Provides synchronous robust classification Against noise with its specialized Weight Encoder (AWE) to handle noise using eigenvalue based computing that is lightweight and efficient. This Method can't be replicated Inside Transformer FFN (Feed-forward-network) because of Transformer dynamic brute force computing where AWE-Based generated weight's get diluted over time by Transformer dynamic projection embedding, making AWE Generated weight causes inefficient inside Transformer dynamic FFN/QKV projection.
       - Specialized Transformer provides robust synchronous advanced contextual relationships, efficient data processing using Alpha based computing, The Transformer is tuned towards to be as flexible as possible to provide dynamic projection or fixed FFN projection training with minimal head's and dimension's to reduce computational power.
+      - LSTM doesn't act as a Main orchestrator, instead it Provides coherent Short-term memory for the Ensemble architecture, acting as a support mechanism to provides proof-of-credibility of a given answer from past previous context input, this allows flexible and achievable Aggreement between Transformer and MLP over a short period of time.
       - Ensemble weighting provides the model a much more robust classification best from both worlds perspective, weighting both MLP and Transformer confidence and probability, combined with Attention quality from the transformer to get the final prediction of an input if transformer is allowed and permitted to be in use.
    
+
 4. flexible and secure Peer-to-Peer Coordination (Multi-Agent):
    - IntegratedPipeline offers Peer to Peer communication capabilities asynchronously, Where IntegratedPipeline directly checks for other Peer presence directly to the local database present in the local computer or system (Synchronous prediction from peer previous data in the database), or externally, by using asynchronous request for initiating prediction, P2P is secured Using:
    - secure socket using user provided SSL CERT. on both client and server,
@@ -118,7 +138,8 @@ https://github.com/Micro-Novelty/IntegratedPipeline-Specialized-Non-LLM-AI-Agent
 [3.] Non-Representative data (Undersampled) : IntegratedPipeline Support's ambiguous data that come's from file with format such as CSV Format to extract title's and label's necessary to create automatic Dataset for Later use in Training, making it optimized for specific task's and easier dataset creation with lower overfitting rate for reliability.
 
 [=] Architectural-Overview
-<img width="1600" height="860" alt="WhatsApp Image 2026-05-09 at 16 13 41" src="https://github.com/user-attachments/assets/580722aa-bbd8-4148-a425-4bff01c06c47" />
+<img width="1600" height="859" alt="New Arch" src="https://github.com/user-attachments/assets/a5db1ed2-9149-4635-8d95-d022ad6e8608" />
+---
 [=] Contextual meaning:
    1. - Sequence encoding is a machine learning technique that transforms a sequential input (like text, time-series data, or audio) into a compact, fixed-length numerical vector, often called a context vector
    2. - TF-IDF (Term Frequency-Inverse Document Frequency) is a numerical statistic used in machine learning and NLP to evaluate how important a word is to a document within a collection (corpus). It boosts rare words and penalizes common words (like "the", "and") by multiplying two metrics: how often a word appears in a document (TF) and the inverse frequency of the word across all documents (IDF). 
@@ -316,7 +337,7 @@ A. [=] Computational performance results with Transformer included during Advanc
                 - Repeated 269 MiB suggests allocator settled, ndarray pools stabilized, workload reached steady state
                 - gradual, safe and steady batch accumulation, reduced possibility of leak explosion.
 
-B. [=] Advanced Prediction without Transformer, Only Specialized MLP using AWE.
+B. [=] Advanced Prediction without Transformer, Only Specialized MLP + LSTM using AWE.
 ```txt
 == TIME == | CPU %  | RAM \ RAM LIMIT |
 10:01:37.456 100.44% 269.8MiB / 3.71GiB # initial round 1 training started
@@ -498,11 +519,13 @@ B. [=] Advanced Prediction without Transformer, Only Specialized MLP using AWE.
    import numpy as np
 
    memory_name = 'agent_memory'
+   cert_file = <your_cert_file_dir> # your .crt file
+   key_file = <your_key_file_dir> # your .key file
+
    main_model = IntegratedPipeline(
       memory_name=memory_name,  # memory name for the AI you already initialized
       use_async=True, # local asynchronous prediction is permitted, if not PipelineAsyncManager wont start asynchronous prediction.
       agent_port=5001, # this port is used to set AgentDistributedInference server (optional)
-      singleton_permitted=False, # set IntegratedPipeline to singleton if you don't use it for P2P, False if you use P2P.
       ssl_cert_file=cert_file, ssl_key_file=key_file) # provide your cert_file path or key_file path (optional)
 
    main_prediction = PipelinePredictionManager(
@@ -656,7 +679,9 @@ import traceback
 
 prediction_manager = PipelinePredictionManager(main_model, label_csv=<your_training_labels.txt>, target_title=<target_title>, label=<target_label>)
 
-secondary_model = IntegratedPipeline(memory_name=memory_name, use_async=True, agent_port=8080, singleton_permitted=True, ssl_cert_file=cert_file, ssl_key_file=key_file) # provide cert_file path or key_file path (optional)
+secondary_model = IntegratedPipeline(memory_name=memory_name, 
+                use_async=True, agent_port=8080, 
+                ssl_cert_file=cert_file, ssl_key_file=key_file) # provide cert_file path or key_file path (optional)
 # secondary model of integrated pipeline is critical for ARM64 environment to prevent socket conflict during P2P with the first Integrated pipeline instance.
 # make sure that the agent_port in secondary_model must be different from agent_port in first IntegratedPipeline instance you initialized.
 
@@ -713,13 +738,12 @@ agent2 = CohesiveAgentDeployment(
       # you must put "known_peers" in the config so python can identify the list of IPs and Ports 
       #  "known_peers": [ 
         #    ["127.0.0.1", 5555], can be modified using real IP or local IP.
-        #    ["127.0.0.1", 5556]
+        #    ["127.0.0.1", 5556],
+             # more ip and port lists...
      #   ]
     # }
 
-agent1.pipeline.use_transformer = False # set agent1 pipeline to not use transformer for efficient processing in ARM64 env.
-agent2.pipeline.use_transformer = False
- 
+
 try:
      # Start both agents
      print("\n🚀 Starting Agent 1...")
@@ -741,7 +765,9 @@ try:
 
      # texts dictionary must contain test_titles, label_map, and rules that you can assign,
      # agent ID can be strings, int, or floats, recommendded to make it long for better security.
-
+     agent1.pipeline.use_transformer = False # set agent1 pipeline to not use transformer for efficient processing in ARM64 env.
+     agent2.pipeline.use_transformer = False
+      
       # Make prediction with peer ensemble
       # Connection will be guaranteed successfull during discovery.
      result = await agent1.multi_modal_peer_ensemble_prediction(
@@ -1017,7 +1043,7 @@ When gradients flow backward:
 [=] 1. - Consider checking and run: [IntegratedPipeline_Flow.html](IntegratedPipeline_Flow.html) regarding each function of the whole components and deep-dive mechanism.
     2. - consider checking [ARCHITECTURE.md](ARCHITECTURE.md) for more explanation about the main components.
        
-[=] With 20 total architectures working together as a standalone library that is efficient and robust, Main components include:
+[=] With 23 total architectures working together as a standalone library that is efficient and robust, Main components include:
     
 1. GeometricWeightShaping
 Purpose: Analyzes the geometric structure of data (anisotropy, spectral properties, complexity) and generates optimal weight matrices based on that geometry. Essentially teaches the model how to "understand" the shape of data before processing it, Highly robust to noise, making it an excellent fit for messy environment.
@@ -1078,12 +1104,21 @@ purpose: Handles both synchronous prediction, and P2P Asynchronous prediction re
 ---
 20. ConsecutivePeerAgent
 purpose: handles ensemble weighting from received output from peer, managed server and acts as a listener simultaneously.
-
+---
+21. LSTMCell
+purpose: handles LSTM Core equations In Numpy.
+---
+22. LSTMNetwork
+purpose: handling LSTM Feed-forward layer, Training and Samples handling.
+---
+23. LSTMEngine
+purpose: handles Proper LSTM Confidence and calibration layer.
 
 ## Full Documentation Features
 - [Go to IntegratedPipeline-Specialized-AI-Agent-library](#IntegratedPipeline-Specialized-AI-Agent-library)
 - [Go to MANN Intro](#MANN-Intro)
 - [Go to Abstract Weight Encoder (AWE) Intro](#Abstract-Weight-Encoder-(AWE)-Intro)
+- [Go to LSTM And Transformer Into](#-lstm-and-transformer-intro)
 - [Go to Why IntegratedPipeline?](#Why-IntegratedPipeline?)
 - [Go to Requirements](#Requirements)
 - [Go to System-Specific-Notes](#System-Specific-Notes)