edip 1.1.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of edip might be problematic. Click here for more details.

edip-1.1.1/PKG-INFO ADDED
@@ -0,0 +1,247 @@
1
+ Metadata-Version: 2.4
2
+ Name: edip
3
+ Version: 1.1.1
4
+ Summary: Edge Deterministic Ingestion Protocol using dynamic SQLite constraint reflection
5
+ Author: Uditha Manohara Prasad
6
+ License: Proprietary
7
+ Project-URL: Homepage, https://neziora.com
8
+ Project-URL: Documentation, https://neziora.com
9
+ Keywords: edge-ai,slm,sqlite,deterministic-ai,document-ingestion,data-engineering,rag
10
+ Requires-Python: >=3.9
11
+ Description-Content-Type: text/markdown
12
+ Requires-Dist: requests>=2.28.0
13
+ Requires-Dist: watchdog>=3.0.0
14
+
15
+ # Edge Deterministic Ingestion Protocol (EDIP)
16
+
17
+ [![Release]([suspicious link removed])]([suspicious link removed])
18
+ [![License: Proprietary / Closed Source]([suspicious link removed])](LICENSE)
19
+ [![Python]([suspicious link removed])]([suspicious link removed])
20
+
21
+ > **Autonomous, air-gapped document ingestion middleware that disciplines probabilistic Small Language Models (SLMs) using relational database constraint reflection and tri-folder isolation.**
22
+
23
+ ---
24
+
25
+ ## Executive Summary
26
+
27
+ Enterprise document extraction pipelines face severe reliability bottlenecks when deploying generative AI at the edge. Small Language Models (1B–3B parameters) operating locally frequently hallucinate identifiers, fail structural types, or introduce relational inconsistencies. Traditional systems either crash upon database insertion or silently corrupt financial ledgers.
28
+
29
+ The **Edge Deterministic Ingestion Protocol (EDIP)** eliminates this failure mode by converting **relational database integrity constraints (Primary Keys, Foreign Keys, CHECK constraints, and NOT NULL rules)** into a real-time, closed-loop error-reflection firewall. Operating 100% offline without cloud API overhead, EDIP enforces strict **Third Normal Form (3NF)** compliance before transactions commit.
30
+
31
+ ---
32
+
33
+ ## Architectural Highlights
34
+
35
+ ```text
36
+ ┌──────────────────────────────┐
37
+ │ ./Inbound_Invoices │
38
+ │ │
39
+ │ Monitored Entry Directory │
40
+ │ (Watchdog Service) │
41
+ └──────────────┬───────────────┘
42
+ │
43
+ ▼
44
+ ┌───────────────────┐
45
+ │ EDIP Engine │
46
+ │ Loop │
47
+ └─────────┬─────────┘
48
+ │
49
+ ├── Dynamic Schema Introspection
50
+ │ (PRAGMA inspection)
51
+ │
52
+ ├── Structured Extraction
53
+ │ (Local SLM via Ollama / GGUF)
54
+ │
55
+ ├── Transactional Validation
56
+ │ (Atomic SQLite WAL Engine)
57
+ │
58
+ └── Error-Reflection Loop
59
+ (Runtime Constraint Feedback)
60
+ │
61
+ ┌─────────────┴─────────────┐
62
+ │ │
63
+ ▼ ▼
64
+ ┌──────────────────┐ ┌──────────────────────┐
65
+ │ SUCCESS / DEDUP │ │ CONSTRAINT FAILURE / │
66
+ │ │ │ RETRIES EXHAUSTED │
67
+ └────────┬─────────┘ └──────────┬───────────┘
68
+ │ │
69
+ ▼ ▼
70
+ ┌──────────────────┐ ┌──────────────────────┐
71
+ │ ./Processed_ │ │ ./Quarantine_ │
72
+ │ Archive │ │ Documents │
73
+ └──────────────────┘ │ │
74
+ │ Human Review Queue │
75
+ └──────────┬───────────┘
76
+ │
77
+ ▼
78
+ ┌──────────────────────┐
79
+ │ Dead-Letter Queue │
80
+ │ (DLQ) │
81
+ │ │
82
+ │ audit_status: │
83
+ │ PENDING_HUMAN_AUDIT │
84
+ └──────────────────────┘
85
+ ```
86
+
87
+ ### Key Capabilities
88
+
89
+ - **Zero-Cloud Air-Gapped Operation**: Optimized for edge deployment on commodity CPU/NPU hardware running quantized local models (`llama3.2:1b`, `qwen2.5:1.5b`).
90
+
91
+ - **Dynamic Schema Introspection**: Discovers arbitrary tables, relational hierarchies, column types, and foreign key references at runtime using native SQLite metadata commands.
92
+
93
+ - **Tri-Folder Enterprise Routing**:
94
+ - `Inbound`: Monitored entry directory.
95
+ - `Processed Archive`: Automatically archives successfully ingested or deduplicated files.
96
+ - `Quarantine Documents`: Physically isolates invalid, injected, or unresolvable documents for manual review.
97
+
98
+ - **Dead-Letter Queue (DLQ) Auditing**: Preserves offending payloads, stack traces, and raw text in a dedicated quarantine ledger for administrative sign-off.
99
+
100
+ - **Atomic Concurrency (WAL Mode)**: Leverages SQLite Write-Ahead Logging (`PRAGMA journal_mode=WAL`) to maintain concurrent multi-process reads with zero database locking conflicts.
101
+
102
+ ---
103
+
104
+ ## Installation & Environment Setup
105
+
106
+ ### Clone the Repository
107
+
108
+ ```bash
109
+ git clone [suspicious link removed]
110
+ cd edip-engine
111
+ ```
112
+
113
+ ### Install Dependencies
114
+
115
+ ```bash
116
+ pip install -r requirements.txt
117
+ ```
118
+
119
+ ### Install EDIP Client SDK
120
+
121
+ ```bash
122
+ pip install -e .
123
+ ```
124
+
125
+ ---
126
+
127
+ ## Quickstart & Usage
128
+
129
+ ### 1. Initializing the Engine
130
+
131
+ ```python
132
+ from edip import EDIPEngine, AmbientWatcher
133
+
134
+ # Configure the engine targeting your database
135
+ engine = EDIPEngine(
136
+ db_path="enterprise_erp.db",
137
+ target_tables=["Invoices"],
138
+ model="llama3.2:1b",
139
+ api_url="http://localhost:11434/api/generate",
140
+ max_retries=3,
141
+ archive_dir="./Processed_Archive",
142
+ quarantine_dir="./Quarantine_Documents"
143
+ )
144
+ ```
145
+
146
+ ### 2. Processing Inbound Documents Headlessly
147
+
148
+ ```python
149
+ # Ingest single documents directly
150
+ success, result = engine.ingest_document("path/to/invoice.pdf")
151
+
152
+ if success:
153
+ print(f"Ingested successfully: {result['payload']}")
154
+ print(f"Archived to: {result['archived_path']}")
155
+ else:
156
+ print(f"Quarantined for audit: {result['quarantined_path']}")
157
+ print(f"DLQ Reference ID: #{result['quarantine_id']}")
158
+ ```
159
+
160
+ ### 3. Running Continuous Directory Monitoring
161
+
162
+ ```python
163
+ def audit_callback(file_path, success, info):
164
+ status = "SUCCESS" if success else "QUARANTINED"
165
+ print(f"[{status}] File: {file_path} | Details: {info}")
166
+
167
+
168
+ # Launch continuous ambient background monitoring
169
+ watcher = AmbientWatcher(
170
+ engine=engine,
171
+ watch_dir="./Inbound_Invoices",
172
+ callback=audit_callback
173
+ )
174
+
175
+ watcher.start(blocking=True)
176
+ ```
177
+
178
+ ---
179
+
180
+ ## Adversarial Stress-Test Benchmarks
181
+
182
+ The EDIP architecture has been rigorously validated across adversarial document conditions:
183
+
184
+ | **Scenario** | **Adversarial Challenge** | **EDIP Behavior** | **Final State** |
185
+ |---|---|---|---|
186
+ | **Legacy ID Migration** | Invoice lists deprecated cloud vendor ID (`9999`) while noting legacy ID (`101`). | Relational FK violation triggers error-reflection; model self-corrects to registered ID. | **Archived** (`supplier_id: 101`) |
187
+ | **Messy Layout / Unstructured** | Amount written in descriptive prose without tabular columns. | Schema enforcement parses clean `REAL` float format. | **Archived** (`total_amount: 5250.5`) |
188
+ | **Prompt Injection Defense** | Embedded malicious prompt instructions (`"DROP TABLE INVOICES"`). | Parameterized execution and constraint enforcement block injection payload. | **Quarantined** (`PENDING_HUMAN_AUDIT`) |
189
+ | **Duplicate Submission** | Re-submission of previously processed transaction. | `UNIQUE NOT NULL` constraint prevents duplicate insertion; handles idempotently. | **Archived** (No duplicate rows) |
190
+ | **Unregistered Vendor** | Invoice from unregistered supplier not present in master table. | Foreign key constraint fails; transaction halts safely without ledger contamination. | **Quarantined** (`PENDING_HUMAN_AUDIT`) |
191
+
192
+ ---
193
+
194
+ ## Intellectual Property & Authorship
195
+
196
+ The architectural design, mathematical schema constraints reflection loop, and tri-folder isolation protocols are proprietary intellectual property developed by **Uditha Manohara Prasad** and **Neziora**.
197
+
198
+ ### Lead Architect
199
+
200
+ **Uditha Manohara Prasad**
201
+
202
+ *Software Engineer (UG) | AI Systems Researcher*
203
+
204
+ - **LinkedIn**: [linkedin.com/in/uditha-manohara](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fuditha-manohara%2F)
205
+ - **X / Twitter**: [@uditha_manohara](https://www.google.com/url?sa=E&q=https%3A%2F%2Fx.com%2Fuditha%5Fmanohara)
206
+ - **Instagram**: [@uditha.manohara](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.instagram.com%2Fuditha.manohara%2F)
207
+ - **Facebook**: [facebook.com/udithamprasad](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.facebook.com%2Fudithamprasad%2F)
208
+ - **TikTok**: [@udithamprasad](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.tiktok.com%2F%40udithamprasad)
209
+ - **WhatsApp**: [+94 76 182 3815]([suspicious link removed])
210
+
211
+ ### Organization
212
+
213
+ **Neziora**
214
+
215
+ *Next-Generation AI Software & Research*
216
+
217
+ - **Website**: [neziora.com](https://www.google.com/url?sa=E&q=https%3A%2F%2Fneziora.com)
218
+ - **LinkedIn**: [linkedin.com/company/neziora](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fneziora)
219
+ - **X / Twitter**: [@Neziora](https://www.google.com/url?sa=E&q=https%3A%2F%2Fx.com%2FNeziora)
220
+ - **Instagram**: [@nezioraofficial](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.instagram.com%2Fnezioraofficial%2F)
221
+ - **Facebook**: [facebook.com/neziora](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.facebook.com/neziora/)
222
+ - **TikTok**: [@neziora](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.tiktok.com%2F%40neziora)
223
+ - **WhatsApp**: [+94 72 654 5635]([suspicious link removed])
224
+
225
+ ---
226
+
227
+ ## Citation
228
+
229
+ If you reference the EDIP methodology or architecture in technical reports, academic publications, or open science repositories, please cite:
230
+
231
+ ```bibtex
232
+ @software{prasad2026edip,
233
+ author = {Uditha Manohara Prasad},
234
+ title = {Edge Deterministic Ingestion Protocol (EDIP): Closed-Loop Relational Constraint Reflection for Edge AI},
235
+ year = {2026},
236
+ publisher = {Neziora},
237
+ url = {https://neziora.com}
238
+ }
239
+ ```
240
+
241
+ ---
242
+
243
+ ## License
244
+
245
+ Copyright © 2026 Neziora & Uditha Manohara Prasad. All rights reserved.
246
+
247
+ Source code distribution, binary compilation, and commercial deployment are subject to proprietary licensing terms.
edip-1.1.1/README.md ADDED
@@ -0,0 +1,233 @@
1
+ # Edge Deterministic Ingestion Protocol (EDIP)
2
+
3
+ [![Release]([suspicious link removed])]([suspicious link removed])
4
+ [![License: Proprietary / Closed Source]([suspicious link removed])](LICENSE)
5
+ [![Python]([suspicious link removed])]([suspicious link removed])
6
+
7
+ > **Autonomous, air-gapped document ingestion middleware that disciplines probabilistic Small Language Models (SLMs) using relational database constraint reflection and tri-folder isolation.**
8
+
9
+ ---
10
+
11
+ ## Executive Summary
12
+
13
+ Enterprise document extraction pipelines face severe reliability bottlenecks when deploying generative AI at the edge. Small Language Models (1B–3B parameters) operating locally frequently hallucinate identifiers, fail structural types, or introduce relational inconsistencies. Traditional systems either crash upon database insertion or silently corrupt financial ledgers.
14
+
15
+ The **Edge Deterministic Ingestion Protocol (EDIP)** eliminates this failure mode by converting **relational database integrity constraints (Primary Keys, Foreign Keys, CHECK constraints, and NOT NULL rules)** into a real-time, closed-loop error-reflection firewall. Operating 100% offline without cloud API overhead, EDIP enforces strict **Third Normal Form (3NF)** compliance before transactions commit.
16
+
17
+ ---
18
+
19
+ ## Architectural Highlights
20
+
21
+ ```text
22
+ ┌──────────────────────────────┐
23
+ │ ./Inbound_Invoices │
24
+ │ │
25
+ │ Monitored Entry Directory │
26
+ │ (Watchdog Service) │
27
+ └──────────────┬───────────────┘
28
+ │
29
+ ▼
30
+ ┌───────────────────┐
31
+ │ EDIP Engine │
32
+ │ Loop │
33
+ └─────────┬─────────┘
34
+ │
35
+ ├── Dynamic Schema Introspection
36
+ │ (PRAGMA inspection)
37
+ │
38
+ ├── Structured Extraction
39
+ │ (Local SLM via Ollama / GGUF)
40
+ │
41
+ ├── Transactional Validation
42
+ │ (Atomic SQLite WAL Engine)
43
+ │
44
+ └── Error-Reflection Loop
45
+ (Runtime Constraint Feedback)
46
+ │
47
+ ┌─────────────┴─────────────┐
48
+ │ │
49
+ ▼ ▼
50
+ ┌──────────────────┐ ┌──────────────────────┐
51
+ │ SUCCESS / DEDUP │ │ CONSTRAINT FAILURE / │
52
+ │ │ │ RETRIES EXHAUSTED │
53
+ └────────┬─────────┘ └──────────┬───────────┘
54
+ │ │
55
+ ▼ ▼
56
+ ┌──────────────────┐ ┌──────────────────────┐
57
+ │ ./Processed_ │ │ ./Quarantine_ │
58
+ │ Archive │ │ Documents │
59
+ └──────────────────┘ │ │
60
+ │ Human Review Queue │
61
+ └──────────┬───────────┘
62
+ │
63
+ ▼
64
+ ┌──────────────────────┐
65
+ │ Dead-Letter Queue │
66
+ │ (DLQ) │
67
+ │ │
68
+ │ audit_status: │
69
+ │ PENDING_HUMAN_AUDIT │
70
+ └──────────────────────┘
71
+ ```
72
+
73
+ ### Key Capabilities
74
+
75
+ - **Zero-Cloud Air-Gapped Operation**: Optimized for edge deployment on commodity CPU/NPU hardware running quantized local models (`llama3.2:1b`, `qwen2.5:1.5b`).
76
+
77
+ - **Dynamic Schema Introspection**: Discovers arbitrary tables, relational hierarchies, column types, and foreign key references at runtime using native SQLite metadata commands.
78
+
79
+ - **Tri-Folder Enterprise Routing**:
80
+ - `Inbound`: Monitored entry directory.
81
+ - `Processed Archive`: Automatically archives successfully ingested or deduplicated files.
82
+ - `Quarantine Documents`: Physically isolates invalid, injected, or unresolvable documents for manual review.
83
+
84
+ - **Dead-Letter Queue (DLQ) Auditing**: Preserves offending payloads, stack traces, and raw text in a dedicated quarantine ledger for administrative sign-off.
85
+
86
+ - **Atomic Concurrency (WAL Mode)**: Leverages SQLite Write-Ahead Logging (`PRAGMA journal_mode=WAL`) to maintain concurrent multi-process reads with zero database locking conflicts.
87
+
88
+ ---
89
+
90
+ ## Installation & Environment Setup
91
+
92
+ ### Clone the Repository
93
+
94
+ ```bash
95
+ git clone [suspicious link removed]
96
+ cd edip-engine
97
+ ```
98
+
99
+ ### Install Dependencies
100
+
101
+ ```bash
102
+ pip install -r requirements.txt
103
+ ```
104
+
105
+ ### Install EDIP Client SDK
106
+
107
+ ```bash
108
+ pip install -e .
109
+ ```
110
+
111
+ ---
112
+
113
+ ## Quickstart & Usage
114
+
115
+ ### 1. Initializing the Engine
116
+
117
+ ```python
118
+ from edip import EDIPEngine, AmbientWatcher
119
+
120
+ # Configure the engine targeting your database
121
+ engine = EDIPEngine(
122
+ db_path="enterprise_erp.db",
123
+ target_tables=["Invoices"],
124
+ model="llama3.2:1b",
125
+ api_url="http://localhost:11434/api/generate",
126
+ max_retries=3,
127
+ archive_dir="./Processed_Archive",
128
+ quarantine_dir="./Quarantine_Documents"
129
+ )
130
+ ```
131
+
132
+ ### 2. Processing Inbound Documents Headlessly
133
+
134
+ ```python
135
+ # Ingest single documents directly
136
+ success, result = engine.ingest_document("path/to/invoice.pdf")
137
+
138
+ if success:
139
+ print(f"Ingested successfully: {result['payload']}")
140
+ print(f"Archived to: {result['archived_path']}")
141
+ else:
142
+ print(f"Quarantined for audit: {result['quarantined_path']}")
143
+ print(f"DLQ Reference ID: #{result['quarantine_id']}")
144
+ ```
145
+
146
+ ### 3. Running Continuous Directory Monitoring
147
+
148
+ ```python
149
+ def audit_callback(file_path, success, info):
150
+ status = "SUCCESS" if success else "QUARANTINED"
151
+ print(f"[{status}] File: {file_path} | Details: {info}")
152
+
153
+
154
+ # Launch continuous ambient background monitoring
155
+ watcher = AmbientWatcher(
156
+ engine=engine,
157
+ watch_dir="./Inbound_Invoices",
158
+ callback=audit_callback
159
+ )
160
+
161
+ watcher.start(blocking=True)
162
+ ```
163
+
164
+ ---
165
+
166
+ ## Adversarial Stress-Test Benchmarks
167
+
168
+ The EDIP architecture has been rigorously validated across adversarial document conditions:
169
+
170
+ | **Scenario** | **Adversarial Challenge** | **EDIP Behavior** | **Final State** |
171
+ |---|---|---|---|
172
+ | **Legacy ID Migration** | Invoice lists deprecated cloud vendor ID (`9999`) while noting legacy ID (`101`). | Relational FK violation triggers error-reflection; model self-corrects to registered ID. | **Archived** (`supplier_id: 101`) |
173
+ | **Messy Layout / Unstructured** | Amount written in descriptive prose without tabular columns. | Schema enforcement parses clean `REAL` float format. | **Archived** (`total_amount: 5250.5`) |
174
+ | **Prompt Injection Defense** | Embedded malicious prompt instructions (`"DROP TABLE INVOICES"`). | Parameterized execution and constraint enforcement block injection payload. | **Quarantined** (`PENDING_HUMAN_AUDIT`) |
175
+ | **Duplicate Submission** | Re-submission of previously processed transaction. | `UNIQUE NOT NULL` constraint prevents duplicate insertion; handles idempotently. | **Archived** (No duplicate rows) |
176
+ | **Unregistered Vendor** | Invoice from unregistered supplier not present in master table. | Foreign key constraint fails; transaction halts safely without ledger contamination. | **Quarantined** (`PENDING_HUMAN_AUDIT`) |
177
+
178
+ ---
179
+
180
+ ## Intellectual Property & Authorship
181
+
182
+ The architectural design, mathematical schema constraints reflection loop, and tri-folder isolation protocols are proprietary intellectual property developed by **Uditha Manohara Prasad** and **Neziora**.
183
+
184
+ ### Lead Architect
185
+
186
+ **Uditha Manohara Prasad**
187
+
188
+ *Software Engineer (UG) | AI Systems Researcher*
189
+
190
+ - **LinkedIn**: [linkedin.com/in/uditha-manohara](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fuditha-manohara%2F)
191
+ - **X / Twitter**: [@uditha_manohara](https://www.google.com/url?sa=E&q=https%3A%2F%2Fx.com%2Fuditha%5Fmanohara)
192
+ - **Instagram**: [@uditha.manohara](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.instagram.com%2Fuditha.manohara%2F)
193
+ - **Facebook**: [facebook.com/udithamprasad](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.facebook.com%2Fudithamprasad%2F)
194
+ - **TikTok**: [@udithamprasad](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.tiktok.com%2F%40udithamprasad)
195
+ - **WhatsApp**: [+94 76 182 3815]([suspicious link removed])
196
+
197
+ ### Organization
198
+
199
+ **Neziora**
200
+
201
+ *Next-Generation AI Software & Research*
202
+
203
+ - **Website**: [neziora.com](https://www.google.com/url?sa=E&q=https%3A%2F%2Fneziora.com)
204
+ - **LinkedIn**: [linkedin.com/company/neziora](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fneziora)
205
+ - **X / Twitter**: [@Neziora](https://www.google.com/url?sa=E&q=https%3A%2F%2Fx.com%2FNeziora)
206
+ - **Instagram**: [@nezioraofficial](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.instagram.com%2Fnezioraofficial%2F)
207
+ - **Facebook**: [facebook.com/neziora](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.facebook.com/neziora/)
208
+ - **TikTok**: [@neziora](https://www.google.com/url?sa=E&q=https%3A%2F%2Fwww.tiktok.com%2F%40neziora)
209
+ - **WhatsApp**: [+94 72 654 5635]([suspicious link removed])
210
+
211
+ ---
212
+
213
+ ## Citation
214
+
215
+ If you reference the EDIP methodology or architecture in technical reports, academic publications, or open science repositories, please cite:
216
+
217
+ ```bibtex
218
+ @software{prasad2026edip,
219
+ author = {Uditha Manohara Prasad},
220
+ title = {Edge Deterministic Ingestion Protocol (EDIP): Closed-Loop Relational Constraint Reflection for Edge AI},
221
+ year = {2026},
222
+ publisher = {Neziora},
223
+ url = {https://neziora.com}
224
+ }
225
+ ```
226
+
227
+ ---
228
+
229
+ ## License
230
+
231
+ Copyright © 2026 Neziora & Uditha Manohara Prasad. All rights reserved.
232
+
233
+ Source code distribution, binary compilation, and commercial deployment are subject to proprietary licensing terms.
@@ -0,0 +1,13 @@
1
+
2
+ from .inspector import SchemaInspector
3
+ from .engine import EDIPEngine
4
+ from .watcher import AmbientWatcher
5
+
6
+
7
+ __all__ = [
8
+ "SchemaInspector",
9
+ "EDIPEngine",
10
+ "AmbientWatcher"
11
+ ]
12
+
13
+ __version__ = "1.0.0"