leai 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- leai-0.1.0/PKG-INFO +346 -0
- leai-0.1.0/README.md +319 -0
- leai-0.1.0/leai/__init__.py +4 -0
- leai-0.1.0/leai/__main__.py +4 -0
- leai-0.1.0/leai/ai/__init__.py +13 -0
- leai-0.1.0/leai/ai/anthropic_client.py +131 -0
- leai-0.1.0/leai/ai/base.py +29 -0
- leai-0.1.0/leai/ai/factory.py +71 -0
- leai-0.1.0/leai/ai/gemini_client.py +140 -0
- leai-0.1.0/leai/ai/openai_client.py +102 -0
- leai-0.1.0/leai/ai/prompts.py +54 -0
- leai-0.1.0/leai/annotations.py +59 -0
- leai-0.1.0/leai/ask_rag.py +114 -0
- leai-0.1.0/leai/chat_session.py +88 -0
- leai-0.1.0/leai/cli.py +789 -0
- leai-0.1.0/leai/compression.py +111 -0
- leai-0.1.0/leai/config.py +108 -0
- leai-0.1.0/leai/docs.py +921 -0
- leai-0.1.0/leai/enrich.py +194 -0
- leai-0.1.0/leai/models.py +161 -0
- leai-0.1.0/leai/oracle.py +930 -0
- leai-0.1.0/leai/raw.py +423 -0
- leai-0.1.0/leai.egg-info/PKG-INFO +346 -0
- leai-0.1.0/leai.egg-info/SOURCES.txt +34 -0
- leai-0.1.0/leai.egg-info/dependency_links.txt +1 -0
- leai-0.1.0/leai.egg-info/entry_points.txt +2 -0
- leai-0.1.0/leai.egg-info/requires.txt +5 -0
- leai-0.1.0/leai.egg-info/top_level.txt +1 -0
- leai-0.1.0/pyproject.toml +54 -0
- leai-0.1.0/setup.cfg +4 -0
- leai-0.1.0/tests/test_ai_integration.py +175 -0
- leai-0.1.0/tests/test_ask_rag.py +106 -0
- leai-0.1.0/tests/test_chat_session.py +86 -0
- leai-0.1.0/tests/test_compression.py +106 -0
- leai-0.1.0/tests/test_config_and_docs.py +552 -0
- leai-0.1.0/tests/test_synonyms.py +65 -0
leai-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,346 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: leai
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Oracle Database reverse engineering, lineage tracing, impact analysis, and RAG/LLM documentation engine
|
|
5
|
+
Author: Lucas Bral
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/lucasbral/leai
|
|
8
|
+
Project-URL: Repository, https://github.com/lucasbral/leai
|
|
9
|
+
Project-URL: Issues, https://github.com/lucasbral/leai/issues
|
|
10
|
+
Keywords: oracle,database,rag,llm,plsql,lineage,documentation,copilot,vector-db
|
|
11
|
+
Classifier: Development Status :: 4 - Beta
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Topic :: Database
|
|
14
|
+
Classifier: Topic :: Software Development :: Documentation
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
20
|
+
Requires-Python: >=3.10
|
|
21
|
+
Description-Content-Type: text/markdown
|
|
22
|
+
Requires-Dist: oracledb>=2.2.0
|
|
23
|
+
Requires-Dist: typer>=0.12.3
|
|
24
|
+
Requires-Dist: rich>=13.7.1
|
|
25
|
+
Requires-Dist: PyYAML>=6.0.1
|
|
26
|
+
Requires-Dist: pydantic>=2.7.0
|
|
27
|
+
|
|
28
|
+
# LEAI โ Oracle Database Intelligence & Documentation Engine
|
|
29
|
+
|
|
30
|
+
**LEAI** is a reverse engineering, impact analysis, and documentation engine for **Oracle Database**, specifically designed to power **Retrieval-Augmented Generation (RAG)**, **LLMs**, and software engineers maintaining complex enterprise database ecosystems.
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## ๐ What Is It?
|
|
35
|
+
|
|
36
|
+
Enterprise Oracle databases accumulate years of business rules scattered across hundreds of tables, views, triggers, and massive PL/SQL packages (3,000 to 10,000+ lines of code).
|
|
37
|
+
|
|
38
|
+
Enabling developers or AI assistants to reliably understand such environments is challenging due to three main issues:
|
|
39
|
+
1. **Token Inefficiency & Hallucinations:** Sending entire monolithic packages into an LLM context is expensive, slow, and triggers the "Lost in the Middle" attention degradation.
|
|
40
|
+
2. **Hidden Dependencies:** Altering a single column can silently break triggers, views, and procedures across multiple schemas.
|
|
41
|
+
3. **Synonyms and Aliases:** Stored procedures frequently access tables via private or public synonyms (`PUBLIC SYNONYM`), creating the false impression that referenced objects do not exist or belong elsewhere.
|
|
42
|
+
|
|
43
|
+
LEAI solves this by extracting the Oracle data dictionary, constructing a cross-schema dependency graph, and formatting the technical context specifically for humans and LLMs.
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## โ๏ธ How It Works
|
|
48
|
+
|
|
49
|
+
LEAI operates via a **3-stage decoupled pipeline**:
|
|
50
|
+
|
|
51
|
+
```
|
|
52
|
+
[Oracle Database]
|
|
53
|
+
โ
|
|
54
|
+
โผ (leai extract)
|
|
55
|
+
โโโโโโโโโโโโโโโ
|
|
56
|
+
โ 1. RAW JSON โ โโ> Pure technical dictionary snapshot (DDL, columns, types, PKs, FKs, Synonyms).
|
|
57
|
+
โโโโโโโโโโโโโโโ
|
|
58
|
+
โ
|
|
59
|
+
โผ (leai annotate / leai enrich)
|
|
60
|
+
โโโโโโโโโโโโโโโ
|
|
61
|
+
โ 2. YAML โ โโ> Editable business annotations (descriptions, rules, tags). Preserves human
|
|
62
|
+
โโโโโโโโโโโโโโโ documentation and allows AI to fill missing stubs without overwriting.
|
|
63
|
+
โ
|
|
64
|
+
โผ (leai compile / leai trace)
|
|
65
|
+
โโโโโโโโโโโโโโโ
|
|
66
|
+
โ 3. DOCS โ โโ> Markdown with YAML Frontmatter + Mermaid.js lineage diagrams + structured
|
|
67
|
+
โโโโโโโโโโโโโโโ chunks for Vector DBs (pgvector, Chroma, Qdrant).
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
### Core Technologies & Internal Mechanics:
|
|
71
|
+
|
|
72
|
+
- **Multi-Level Lineage Tracing (`trace`):**
|
|
73
|
+
Identifies upstream dependencies and downstream consumers with configurable depth (`--depth N`), automatically computing change risk levels (`LOW`, `MEDIUM`, `HIGH`, `CRITICAL`).
|
|
74
|
+
- **Transparent Synonym & Dblink Resolution:**
|
|
75
|
+
Resolves `ALL_SYNONYMS` and `PUBLIC SYNONYMS` directly to their underlying physical target objects, including remote database links (`@dblink`).
|
|
76
|
+
- **PL/SQL Semantic Compression:**
|
|
77
|
+
When querying a specific procedure (`TEST_PROC`) inside a 10,000-line package, LEAI surgically extracts only the requested subprogram body and produces a lightweight signature skeleton of the rest of the package, **reducing token consumption by up to 95%**.
|
|
78
|
+
- **Dynamic Contextual RAG (`ask` & `chat`):**
|
|
79
|
+
Automatically detects database entities mentioned in user prompts, executes on-the-fly dependency tracing, and delivers a surgical, noise-free context payload to the LLM.
|
|
80
|
+
- **Native Multi-Provider AI Support:**
|
|
81
|
+
Direct HTTPS REST integration with **OpenAI (ChatGPT)**, **Google Gemini**, **Anthropic Claude**, **DeepSeek**, **Qwen**, **Kimi**, and **Ollama (local & free)** without heavy external dependencies.
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
## ๐ Getting Started
|
|
86
|
+
|
|
87
|
+
### 1. Installation
|
|
88
|
+
|
|
89
|
+
We recommend using **`uv`** for fast and isolated execution:
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
# Clone the repository and navigate to the directory
|
|
93
|
+
cd leai
|
|
94
|
+
|
|
95
|
+
# Synchronize dependencies and virtual environment
|
|
96
|
+
uv sync
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
*(Or using standard pip: `pip install -e .`)*
|
|
100
|
+
|
|
101
|
+
---
|
|
102
|
+
|
|
103
|
+
### 2. Configuration (`leai.yml`)
|
|
104
|
+
|
|
105
|
+
Create a `leai.yml` file in the root of your project:
|
|
106
|
+
|
|
107
|
+
```yaml
|
|
108
|
+
# Oracle connection string (supports environment variables ${VAR})
|
|
109
|
+
dsn: "oracle://${DB_USER}:${DB_PASS}@${DB_HOST}:1521/${DB_SERVICE}"
|
|
110
|
+
|
|
111
|
+
# Schemas integrated into your ecosystem graph
|
|
112
|
+
schemas:
|
|
113
|
+
- HR
|
|
114
|
+
- FINANCE
|
|
115
|
+
- CORE
|
|
116
|
+
|
|
117
|
+
# Pipeline output directories
|
|
118
|
+
rawPath: "./raw" # Raw technical snapshots (JSON)
|
|
119
|
+
annotationsPath: "./annotations" # Business annotations (YAML)
|
|
120
|
+
docPath: "./docs" # Final compiled documentation (Markdown)
|
|
121
|
+
|
|
122
|
+
# AI Provider Configuration for enrich, ask, and chat
|
|
123
|
+
ai:
|
|
124
|
+
default_provider: "openai" # openai, gemini, anthropic, deepseek, qwen, kimi, ollama
|
|
125
|
+
temperature: 0.2
|
|
126
|
+
providers:
|
|
127
|
+
openai:
|
|
128
|
+
api_key: "${OPENAI_API_KEY}"
|
|
129
|
+
model: "gpt-4o-mini"
|
|
130
|
+
gemini:
|
|
131
|
+
api_key: "${GEMINI_API_KEY}"
|
|
132
|
+
model: "gemini-1.5-flash"
|
|
133
|
+
anthropic:
|
|
134
|
+
api_key: "${ANTHROPIC_API_KEY}"
|
|
135
|
+
model: "claude-3-5-sonnet-20241022"
|
|
136
|
+
ollama:
|
|
137
|
+
base_url: "http://localhost:11434/v1"
|
|
138
|
+
model: "llama3.1"
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
---
|
|
142
|
+
|
|
143
|
+
## ๐ CLI Command Reference
|
|
144
|
+
|
|
145
|
+
### 1. `uv run leai` (or `leai generate`)
|
|
146
|
+
Executes the full pipeline: extracts technical snapshots from Oracle, synchronizes business annotation stubs, and compiles final Markdown docs.
|
|
147
|
+
|
|
148
|
+
| Parameter / Flag | Type | Description |
|
|
149
|
+
| :--- | :--- | :--- |
|
|
150
|
+
| `-c`, `--config PATH` | Option | Path to the configuration file (Default: `leai.yml`). |
|
|
151
|
+
| `-t`, `--object-type TEXT` | Option | Filter specific object types (e.g., `-t tables -t views -t packages`). |
|
|
152
|
+
|
|
153
|
+
```bash
|
|
154
|
+
uv run leai
|
|
155
|
+
uv run leai generate -t tables -t packages --config prod.yml
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
---
|
|
159
|
+
|
|
160
|
+
### 2. `uv run leai extract`
|
|
161
|
+
Connects to Oracle and extracts raw JSON technical snapshots into the `raw/` directory.
|
|
162
|
+
|
|
163
|
+
| Parameter / Flag | Type | Description |
|
|
164
|
+
| :--- | :--- | :--- |
|
|
165
|
+
| `-s`, `--schema TEXT` | Option | Extract only a specific schema. |
|
|
166
|
+
| `-t`, `--object-type TEXT` | Option | Filter object types to extract. |
|
|
167
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
168
|
+
|
|
169
|
+
```bash
|
|
170
|
+
uv run leai extract
|
|
171
|
+
uv run leai extract -s HR -t tables -t views
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
---
|
|
175
|
+
|
|
176
|
+
### 3. `uv run leai annotate`
|
|
177
|
+
Reads JSON snapshots from `raw/` and generates/synchronizes YAML stubs in `annotations/`, preserving existing manual documentation (Offline Mode).
|
|
178
|
+
|
|
179
|
+
| Parameter / Flag | Type | Description |
|
|
180
|
+
| :--- | :--- | :--- |
|
|
181
|
+
| `-t`, `--object-type TEXT` | Option | Synchronize only specific object types. |
|
|
182
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
183
|
+
|
|
184
|
+
```bash
|
|
185
|
+
uv run leai annotate
|
|
186
|
+
uv run leai annotate -t tables
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
---
|
|
190
|
+
|
|
191
|
+
### 4. `uv run leai compile`
|
|
192
|
+
Recompiles the entire Markdown documentation in `docs/` by merging `raw/` and `annotations/` without connecting to the database.
|
|
193
|
+
|
|
194
|
+
| Parameter / Flag | Type | Description |
|
|
195
|
+
| :--- | :--- | :--- |
|
|
196
|
+
| `-t`, `--object-type TEXT` | Option | Compile only specific object types. |
|
|
197
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
198
|
+
|
|
199
|
+
```bash
|
|
200
|
+
uv run leai compile
|
|
201
|
+
uv run leai compile -t views
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
---
|
|
205
|
+
|
|
206
|
+
### 5. `uv run leai trace <OBJECT>`
|
|
207
|
+
Generates deep impact analysis, terminal hierarchical trees, change risk calculations, and Mermaid.js lineage dossiers.
|
|
208
|
+
|
|
209
|
+
| Parameter / Flag | Type | Description |
|
|
210
|
+
| :--- | :--- | :--- |
|
|
211
|
+
| `OBJECT` | **Required Argument** | Name of the table, view, procedure, or synonym to trace (e.g., `EMPLOYEES`). |
|
|
212
|
+
| `-d`, `--depth INT` | Option | Max graph traversal depth (Default: `1` for direct, `2+` for multi-level). |
|
|
213
|
+
| `--rag-json`, `--rag` | Flag | Also exports structured JSON chunks for Vector DB ingestion. |
|
|
214
|
+
| `--offline` | Flag | Resolves dependencies locally from `raw/` snapshots without connecting to Oracle. |
|
|
215
|
+
| `-s`, `--schema TEXT` | Option | Schema of target object (searches all configured schemas if omitted). |
|
|
216
|
+
| `-o`, `--output PATH` | Option | Custom file path for the generated Markdown dossier. |
|
|
217
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
218
|
+
|
|
219
|
+
```bash
|
|
220
|
+
# Multi-level lineage trace (Depth 2)
|
|
221
|
+
uv run leai trace EMPLOYEES --depth 2
|
|
222
|
+
|
|
223
|
+
# Offline mode with RAG JSON chunk export
|
|
224
|
+
uv run leai trace EMPLOYEES --offline --depth 2 --rag-json
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
---
|
|
228
|
+
|
|
229
|
+
### 6. `uv run leai enrich`
|
|
230
|
+
Uses AI (LLMs) to analyze DDLs and PL/SQL code, automatically generating business rules and column descriptions in `annotations/` with real-time progress bars.
|
|
231
|
+
|
|
232
|
+
| Parameter / Flag | Type | Description |
|
|
233
|
+
| :--- | :--- | :--- |
|
|
234
|
+
| `-o`, `--object-name TEXT` | Option | Specific object name to enrich (e.g., `-o EMPLOYEES`). |
|
|
235
|
+
| `-p`, `--provider TEXT` | Option | AI provider (`openai`, `gemini`, `anthropic`, `deepseek`, `qwen`, `kimi`, `ollama`). |
|
|
236
|
+
| `-m`, `--model TEXT` | Option | Model identifier (e.g., `gpt-4o-mini`, `gemini-1.5-flash`, `claude-3-5-sonnet-20241022`). |
|
|
237
|
+
| `--overwrite` | Flag | Forces regeneration of existing descriptions and comments. |
|
|
238
|
+
| `-t`, `--object-type TEXT` | Option | Filter object types to enrich (e.g., `-t tables -t packages`). |
|
|
239
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
240
|
+
|
|
241
|
+
```bash
|
|
242
|
+
# Enrich using default provider
|
|
243
|
+
uv run leai enrich
|
|
244
|
+
|
|
245
|
+
# Enrich using Google Gemini or Anthropic Claude
|
|
246
|
+
uv run leai enrich --provider gemini --model gemini-1.5-flash
|
|
247
|
+
uv run leai enrich --provider anthropic --model claude-3-5-sonnet-20241022
|
|
248
|
+
|
|
249
|
+
# Enrich a single table with forced overwrite
|
|
250
|
+
uv run leai enrich -o EMPLOYEES --overwrite
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
---
|
|
254
|
+
|
|
255
|
+
### 7. `uv run leai ask <QUESTION>`
|
|
256
|
+
Asks one-off natural language questions answered with dynamic RAG context directly in your terminal.
|
|
257
|
+
|
|
258
|
+
| Parameter / Flag | Type | Description |
|
|
259
|
+
| :--- | :--- | :--- |
|
|
260
|
+
| `QUESTION` | **Required Argument** | The question regarding database structure, dependencies, or business rules. |
|
|
261
|
+
| `-p`, `--provider TEXT` | Option | AI provider to use. |
|
|
262
|
+
| `-m`, `--model TEXT` | Option | Model identifier to use. |
|
|
263
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
264
|
+
|
|
265
|
+
```bash
|
|
266
|
+
uv run leai ask "Which views or stored procedures query the EMPLOYEES table?"
|
|
267
|
+
uv run leai ask "How does the payroll calculation workflow operate?" --provider gemini
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
---
|
|
271
|
+
|
|
272
|
+
### 8. `uv run leai chat`
|
|
273
|
+
Launches an interactive multi-turn terminal chat session with persistent conversation memory and cumulative graph context.
|
|
274
|
+
|
|
275
|
+
| Parameter / Flag | Type | Description |
|
|
276
|
+
| :--- | :--- | :--- |
|
|
277
|
+
| `-p`, `--provider TEXT` | Option | AI provider to use. |
|
|
278
|
+
| `-m`, `--model TEXT` | Option | Model identifier to use. |
|
|
279
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
280
|
+
|
|
281
|
+
```bash
|
|
282
|
+
uv run leai chat
|
|
283
|
+
uv run leai chat --provider anthropic --model claude-3-5-sonnet-20241022
|
|
284
|
+
uv run leai chat --provider ollama --model llama3.1
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
#### ๐ฎ Interactive In-Session Commands:
|
|
288
|
+
- `/clear`: Clears conversation history and active entity memory.
|
|
289
|
+
- `/save [file.md]`: Exports the complete transcript and generated scripts into a Markdown file.
|
|
290
|
+
- `/help`: Displays available commands.
|
|
291
|
+
- `/exit` or `/quit`: Closes the chat session.
|
|
292
|
+
|
|
293
|
+
---
|
|
294
|
+
|
|
295
|
+
### 9. `uv run leai changes`
|
|
296
|
+
Audits and lists recently created or modified database objects (via Oracle's `LAST_DDL_TIME`).
|
|
297
|
+
|
|
298
|
+
| Parameter / Flag | Type | Description |
|
|
299
|
+
| :--- | :--- | :--- |
|
|
300
|
+
| `-d`, `--days INT` | Option | Number of trailing days to audit (Default: `7`). |
|
|
301
|
+
| `-u`, `--user TEXT` | Option | Filter by modifying user / schema (e.g., `-u HR`). |
|
|
302
|
+
| `-s`, `--schema TEXT` | Option | Target schema. |
|
|
303
|
+
| `-t`, `--object-type TEXT` | Option | Filter object types. |
|
|
304
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
305
|
+
|
|
306
|
+
```bash
|
|
307
|
+
# Objects altered in the last 15 days
|
|
308
|
+
uv run leai changes -d 15
|
|
309
|
+
|
|
310
|
+
# Filter by schema
|
|
311
|
+
uv run leai changes -d 30 -u HR
|
|
312
|
+
```
|
|
313
|
+
|
|
314
|
+
---
|
|
315
|
+
|
|
316
|
+
## ๐ Directory Structure
|
|
317
|
+
|
|
318
|
+
```text
|
|
319
|
+
my_project/
|
|
320
|
+
โโโ leai.yml
|
|
321
|
+
โโโ raw/ <-- Raw JSON snapshots extracted from Oracle
|
|
322
|
+
โ โโโ HR/
|
|
323
|
+
โ โโโ tables/
|
|
324
|
+
โ โโโ views/
|
|
325
|
+
โ โโโ synonyms/
|
|
326
|
+
โ โโโ code_objects/
|
|
327
|
+
โโโ annotations/ <-- YAML business rules & annotations (editable)
|
|
328
|
+
โ โโโ HR/
|
|
329
|
+
โ โโโ tables/
|
|
330
|
+
โ โโโ code_objects/
|
|
331
|
+
โโโ docs/ <-- Final compiled Markdown for LLMs, RAG, and humans
|
|
332
|
+
โโโ HR/
|
|
333
|
+
โโโ tables/
|
|
334
|
+
โโโ dossiers/ <-- Impact dossiers generated by leai trace
|
|
335
|
+
โโโ code_objects/
|
|
336
|
+
```
|
|
337
|
+
|
|
338
|
+
---
|
|
339
|
+
|
|
340
|
+
## ๐งช Automated Testing
|
|
341
|
+
|
|
342
|
+
To run the complete automated test suite:
|
|
343
|
+
|
|
344
|
+
```bash
|
|
345
|
+
uv run python -m unittest discover tests
|
|
346
|
+
```
|
leai-0.1.0/README.md
ADDED
|
@@ -0,0 +1,319 @@
|
|
|
1
|
+
# LEAI โ Oracle Database Intelligence & Documentation Engine
|
|
2
|
+
|
|
3
|
+
**LEAI** is a reverse engineering, impact analysis, and documentation engine for **Oracle Database**, specifically designed to power **Retrieval-Augmented Generation (RAG)**, **LLMs**, and software engineers maintaining complex enterprise database ecosystems.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## ๐ What Is It?
|
|
8
|
+
|
|
9
|
+
Enterprise Oracle databases accumulate years of business rules scattered across hundreds of tables, views, triggers, and massive PL/SQL packages (3,000 to 10,000+ lines of code).
|
|
10
|
+
|
|
11
|
+
Enabling developers or AI assistants to reliably understand such environments is challenging due to three main issues:
|
|
12
|
+
1. **Token Inefficiency & Hallucinations:** Sending entire monolithic packages into an LLM context is expensive, slow, and triggers the "Lost in the Middle" attention degradation.
|
|
13
|
+
2. **Hidden Dependencies:** Altering a single column can silently break triggers, views, and procedures across multiple schemas.
|
|
14
|
+
3. **Synonyms and Aliases:** Stored procedures frequently access tables via private or public synonyms (`PUBLIC SYNONYM`), creating the false impression that referenced objects do not exist or belong elsewhere.
|
|
15
|
+
|
|
16
|
+
LEAI solves this by extracting the Oracle data dictionary, constructing a cross-schema dependency graph, and formatting the technical context specifically for humans and LLMs.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## โ๏ธ How It Works
|
|
21
|
+
|
|
22
|
+
LEAI operates via a **3-stage decoupled pipeline**:
|
|
23
|
+
|
|
24
|
+
```
|
|
25
|
+
[Oracle Database]
|
|
26
|
+
โ
|
|
27
|
+
โผ (leai extract)
|
|
28
|
+
โโโโโโโโโโโโโโโ
|
|
29
|
+
โ 1. RAW JSON โ โโ> Pure technical dictionary snapshot (DDL, columns, types, PKs, FKs, Synonyms).
|
|
30
|
+
โโโโโโโโโโโโโโโ
|
|
31
|
+
โ
|
|
32
|
+
โผ (leai annotate / leai enrich)
|
|
33
|
+
โโโโโโโโโโโโโโโ
|
|
34
|
+
โ 2. YAML โ โโ> Editable business annotations (descriptions, rules, tags). Preserves human
|
|
35
|
+
โโโโโโโโโโโโโโโ documentation and allows AI to fill missing stubs without overwriting.
|
|
36
|
+
โ
|
|
37
|
+
โผ (leai compile / leai trace)
|
|
38
|
+
โโโโโโโโโโโโโโโ
|
|
39
|
+
โ 3. DOCS โ โโ> Markdown with YAML Frontmatter + Mermaid.js lineage diagrams + structured
|
|
40
|
+
โโโโโโโโโโโโโโโ chunks for Vector DBs (pgvector, Chroma, Qdrant).
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
### Core Technologies & Internal Mechanics:
|
|
44
|
+
|
|
45
|
+
- **Multi-Level Lineage Tracing (`trace`):**
|
|
46
|
+
Identifies upstream dependencies and downstream consumers with configurable depth (`--depth N`), automatically computing change risk levels (`LOW`, `MEDIUM`, `HIGH`, `CRITICAL`).
|
|
47
|
+
- **Transparent Synonym & Dblink Resolution:**
|
|
48
|
+
Resolves `ALL_SYNONYMS` and `PUBLIC SYNONYMS` directly to their underlying physical target objects, including remote database links (`@dblink`).
|
|
49
|
+
- **PL/SQL Semantic Compression:**
|
|
50
|
+
When querying a specific procedure (`TEST_PROC`) inside a 10,000-line package, LEAI surgically extracts only the requested subprogram body and produces a lightweight signature skeleton of the rest of the package, **reducing token consumption by up to 95%**.
|
|
51
|
+
- **Dynamic Contextual RAG (`ask` & `chat`):**
|
|
52
|
+
Automatically detects database entities mentioned in user prompts, executes on-the-fly dependency tracing, and delivers a surgical, noise-free context payload to the LLM.
|
|
53
|
+
- **Native Multi-Provider AI Support:**
|
|
54
|
+
Direct HTTPS REST integration with **OpenAI (ChatGPT)**, **Google Gemini**, **Anthropic Claude**, **DeepSeek**, **Qwen**, **Kimi**, and **Ollama (local & free)** without heavy external dependencies.
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## ๐ Getting Started
|
|
59
|
+
|
|
60
|
+
### 1. Installation
|
|
61
|
+
|
|
62
|
+
We recommend using **`uv`** for fast and isolated execution:
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
# Clone the repository and navigate to the directory
|
|
66
|
+
cd leai
|
|
67
|
+
|
|
68
|
+
# Synchronize dependencies and virtual environment
|
|
69
|
+
uv sync
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
*(Or using standard pip: `pip install -e .`)*
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
### 2. Configuration (`leai.yml`)
|
|
77
|
+
|
|
78
|
+
Create a `leai.yml` file in the root of your project:
|
|
79
|
+
|
|
80
|
+
```yaml
|
|
81
|
+
# Oracle connection string (supports environment variables ${VAR})
|
|
82
|
+
dsn: "oracle://${DB_USER}:${DB_PASS}@${DB_HOST}:1521/${DB_SERVICE}"
|
|
83
|
+
|
|
84
|
+
# Schemas integrated into your ecosystem graph
|
|
85
|
+
schemas:
|
|
86
|
+
- HR
|
|
87
|
+
- FINANCE
|
|
88
|
+
- CORE
|
|
89
|
+
|
|
90
|
+
# Pipeline output directories
|
|
91
|
+
rawPath: "./raw" # Raw technical snapshots (JSON)
|
|
92
|
+
annotationsPath: "./annotations" # Business annotations (YAML)
|
|
93
|
+
docPath: "./docs" # Final compiled documentation (Markdown)
|
|
94
|
+
|
|
95
|
+
# AI Provider Configuration for enrich, ask, and chat
|
|
96
|
+
ai:
|
|
97
|
+
default_provider: "openai" # openai, gemini, anthropic, deepseek, qwen, kimi, ollama
|
|
98
|
+
temperature: 0.2
|
|
99
|
+
providers:
|
|
100
|
+
openai:
|
|
101
|
+
api_key: "${OPENAI_API_KEY}"
|
|
102
|
+
model: "gpt-4o-mini"
|
|
103
|
+
gemini:
|
|
104
|
+
api_key: "${GEMINI_API_KEY}"
|
|
105
|
+
model: "gemini-1.5-flash"
|
|
106
|
+
anthropic:
|
|
107
|
+
api_key: "${ANTHROPIC_API_KEY}"
|
|
108
|
+
model: "claude-3-5-sonnet-20241022"
|
|
109
|
+
ollama:
|
|
110
|
+
base_url: "http://localhost:11434/v1"
|
|
111
|
+
model: "llama3.1"
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## ๐ CLI Command Reference
|
|
117
|
+
|
|
118
|
+
### 1. `uv run leai` (or `leai generate`)
|
|
119
|
+
Executes the full pipeline: extracts technical snapshots from Oracle, synchronizes business annotation stubs, and compiles final Markdown docs.
|
|
120
|
+
|
|
121
|
+
| Parameter / Flag | Type | Description |
|
|
122
|
+
| :--- | :--- | :--- |
|
|
123
|
+
| `-c`, `--config PATH` | Option | Path to the configuration file (Default: `leai.yml`). |
|
|
124
|
+
| `-t`, `--object-type TEXT` | Option | Filter specific object types (e.g., `-t tables -t views -t packages`). |
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
uv run leai
|
|
128
|
+
uv run leai generate -t tables -t packages --config prod.yml
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
---
|
|
132
|
+
|
|
133
|
+
### 2. `uv run leai extract`
|
|
134
|
+
Connects to Oracle and extracts raw JSON technical snapshots into the `raw/` directory.
|
|
135
|
+
|
|
136
|
+
| Parameter / Flag | Type | Description |
|
|
137
|
+
| :--- | :--- | :--- |
|
|
138
|
+
| `-s`, `--schema TEXT` | Option | Extract only a specific schema. |
|
|
139
|
+
| `-t`, `--object-type TEXT` | Option | Filter object types to extract. |
|
|
140
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
141
|
+
|
|
142
|
+
```bash
|
|
143
|
+
uv run leai extract
|
|
144
|
+
uv run leai extract -s HR -t tables -t views
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
### 3. `uv run leai annotate`
|
|
150
|
+
Reads JSON snapshots from `raw/` and generates/synchronizes YAML stubs in `annotations/`, preserving existing manual documentation (Offline Mode).
|
|
151
|
+
|
|
152
|
+
| Parameter / Flag | Type | Description |
|
|
153
|
+
| :--- | :--- | :--- |
|
|
154
|
+
| `-t`, `--object-type TEXT` | Option | Synchronize only specific object types. |
|
|
155
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
156
|
+
|
|
157
|
+
```bash
|
|
158
|
+
uv run leai annotate
|
|
159
|
+
uv run leai annotate -t tables
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
---
|
|
163
|
+
|
|
164
|
+
### 4. `uv run leai compile`
|
|
165
|
+
Recompiles the entire Markdown documentation in `docs/` by merging `raw/` and `annotations/` without connecting to the database.
|
|
166
|
+
|
|
167
|
+
| Parameter / Flag | Type | Description |
|
|
168
|
+
| :--- | :--- | :--- |
|
|
169
|
+
| `-t`, `--object-type TEXT` | Option | Compile only specific object types. |
|
|
170
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
171
|
+
|
|
172
|
+
```bash
|
|
173
|
+
uv run leai compile
|
|
174
|
+
uv run leai compile -t views
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
---
|
|
178
|
+
|
|
179
|
+
### 5. `uv run leai trace <OBJECT>`
|
|
180
|
+
Generates deep impact analysis, terminal hierarchical trees, change risk calculations, and Mermaid.js lineage dossiers.
|
|
181
|
+
|
|
182
|
+
| Parameter / Flag | Type | Description |
|
|
183
|
+
| :--- | :--- | :--- |
|
|
184
|
+
| `OBJECT` | **Required Argument** | Name of the table, view, procedure, or synonym to trace (e.g., `EMPLOYEES`). |
|
|
185
|
+
| `-d`, `--depth INT` | Option | Max graph traversal depth (Default: `1` for direct, `2+` for multi-level). |
|
|
186
|
+
| `--rag-json`, `--rag` | Flag | Also exports structured JSON chunks for Vector DB ingestion. |
|
|
187
|
+
| `--offline` | Flag | Resolves dependencies locally from `raw/` snapshots without connecting to Oracle. |
|
|
188
|
+
| `-s`, `--schema TEXT` | Option | Schema of target object (searches all configured schemas if omitted). |
|
|
189
|
+
| `-o`, `--output PATH` | Option | Custom file path for the generated Markdown dossier. |
|
|
190
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
191
|
+
|
|
192
|
+
```bash
|
|
193
|
+
# Multi-level lineage trace (Depth 2)
|
|
194
|
+
uv run leai trace EMPLOYEES --depth 2
|
|
195
|
+
|
|
196
|
+
# Offline mode with RAG JSON chunk export
|
|
197
|
+
uv run leai trace EMPLOYEES --offline --depth 2 --rag-json
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
---
|
|
201
|
+
|
|
202
|
+
### 6. `uv run leai enrich`
|
|
203
|
+
Uses AI (LLMs) to analyze DDLs and PL/SQL code, automatically generating business rules and column descriptions in `annotations/` with real-time progress bars.
|
|
204
|
+
|
|
205
|
+
| Parameter / Flag | Type | Description |
|
|
206
|
+
| :--- | :--- | :--- |
|
|
207
|
+
| `-o`, `--object-name TEXT` | Option | Specific object name to enrich (e.g., `-o EMPLOYEES`). |
|
|
208
|
+
| `-p`, `--provider TEXT` | Option | AI provider (`openai`, `gemini`, `anthropic`, `deepseek`, `qwen`, `kimi`, `ollama`). |
|
|
209
|
+
| `-m`, `--model TEXT` | Option | Model identifier (e.g., `gpt-4o-mini`, `gemini-1.5-flash`, `claude-3-5-sonnet-20241022`). |
|
|
210
|
+
| `--overwrite` | Flag | Forces regeneration of existing descriptions and comments. |
|
|
211
|
+
| `-t`, `--object-type TEXT` | Option | Filter object types to enrich (e.g., `-t tables -t packages`). |
|
|
212
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
213
|
+
|
|
214
|
+
```bash
|
|
215
|
+
# Enrich using default provider
|
|
216
|
+
uv run leai enrich
|
|
217
|
+
|
|
218
|
+
# Enrich using Google Gemini or Anthropic Claude
|
|
219
|
+
uv run leai enrich --provider gemini --model gemini-1.5-flash
|
|
220
|
+
uv run leai enrich --provider anthropic --model claude-3-5-sonnet-20241022
|
|
221
|
+
|
|
222
|
+
# Enrich a single table with forced overwrite
|
|
223
|
+
uv run leai enrich -o EMPLOYEES --overwrite
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
---
|
|
227
|
+
|
|
228
|
+
### 7. `uv run leai ask <QUESTION>`
|
|
229
|
+
Asks one-off natural language questions answered with dynamic RAG context directly in your terminal.
|
|
230
|
+
|
|
231
|
+
| Parameter / Flag | Type | Description |
|
|
232
|
+
| :--- | :--- | :--- |
|
|
233
|
+
| `QUESTION` | **Required Argument** | The question regarding database structure, dependencies, or business rules. |
|
|
234
|
+
| `-p`, `--provider TEXT` | Option | AI provider to use. |
|
|
235
|
+
| `-m`, `--model TEXT` | Option | Model identifier to use. |
|
|
236
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
237
|
+
|
|
238
|
+
```bash
|
|
239
|
+
uv run leai ask "Which views or stored procedures query the EMPLOYEES table?"
|
|
240
|
+
uv run leai ask "How does the payroll calculation workflow operate?" --provider gemini
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
---
|
|
244
|
+
|
|
245
|
+
### 8. `uv run leai chat`
|
|
246
|
+
Launches an interactive multi-turn terminal chat session with persistent conversation memory and cumulative graph context.
|
|
247
|
+
|
|
248
|
+
| Parameter / Flag | Type | Description |
|
|
249
|
+
| :--- | :--- | :--- |
|
|
250
|
+
| `-p`, `--provider TEXT` | Option | AI provider to use. |
|
|
251
|
+
| `-m`, `--model TEXT` | Option | Model identifier to use. |
|
|
252
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
253
|
+
|
|
254
|
+
```bash
|
|
255
|
+
uv run leai chat
|
|
256
|
+
uv run leai chat --provider anthropic --model claude-3-5-sonnet-20241022
|
|
257
|
+
uv run leai chat --provider ollama --model llama3.1
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
#### ๐ฎ Interactive In-Session Commands:
|
|
261
|
+
- `/clear`: Clears conversation history and active entity memory.
|
|
262
|
+
- `/save [file.md]`: Exports the complete transcript and generated scripts into a Markdown file.
|
|
263
|
+
- `/help`: Displays available commands.
|
|
264
|
+
- `/exit` or `/quit`: Closes the chat session.
|
|
265
|
+
|
|
266
|
+
---
|
|
267
|
+
|
|
268
|
+
### 9. `uv run leai changes`
|
|
269
|
+
Audits and lists recently created or modified database objects (via Oracle's `LAST_DDL_TIME`).
|
|
270
|
+
|
|
271
|
+
| Parameter / Flag | Type | Description |
|
|
272
|
+
| :--- | :--- | :--- |
|
|
273
|
+
| `-d`, `--days INT` | Option | Number of trailing days to audit (Default: `7`). |
|
|
274
|
+
| `-u`, `--user TEXT` | Option | Filter by modifying user / schema (e.g., `-u HR`). |
|
|
275
|
+
| `-s`, `--schema TEXT` | Option | Target schema. |
|
|
276
|
+
| `-t`, `--object-type TEXT` | Option | Filter object types. |
|
|
277
|
+
| `-c`, `--config PATH` | Option | Path to `leai.yml`. |
|
|
278
|
+
|
|
279
|
+
```bash
|
|
280
|
+
# Objects altered in the last 15 days
|
|
281
|
+
uv run leai changes -d 15
|
|
282
|
+
|
|
283
|
+
# Filter by schema
|
|
284
|
+
uv run leai changes -d 30 -u HR
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
---
|
|
288
|
+
|
|
289
|
+
## ๐ Directory Structure
|
|
290
|
+
|
|
291
|
+
```text
|
|
292
|
+
my_project/
|
|
293
|
+
โโโ leai.yml
|
|
294
|
+
โโโ raw/ <-- Raw JSON snapshots extracted from Oracle
|
|
295
|
+
โ โโโ HR/
|
|
296
|
+
โ โโโ tables/
|
|
297
|
+
โ โโโ views/
|
|
298
|
+
โ โโโ synonyms/
|
|
299
|
+
โ โโโ code_objects/
|
|
300
|
+
โโโ annotations/ <-- YAML business rules & annotations (editable)
|
|
301
|
+
โ โโโ HR/
|
|
302
|
+
โ โโโ tables/
|
|
303
|
+
โ โโโ code_objects/
|
|
304
|
+
โโโ docs/ <-- Final compiled Markdown for LLMs, RAG, and humans
|
|
305
|
+
โโโ HR/
|
|
306
|
+
โโโ tables/
|
|
307
|
+
โโโ dossiers/ <-- Impact dossiers generated by leai trace
|
|
308
|
+
โโโ code_objects/
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
---
|
|
312
|
+
|
|
313
|
+
## ๐งช Automated Testing
|
|
314
|
+
|
|
315
|
+
To run the complete automated test suite:
|
|
316
|
+
|
|
317
|
+
```bash
|
|
318
|
+
uv run python -m unittest discover tests
|
|
319
|
+
```
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
from leai.ai.base import BaseLLMClient
|
|
2
|
+
from leai.ai.factory import get_llm_client
|
|
3
|
+
from leai.ai.openai_client import OpenAICompatibleClient
|
|
4
|
+
from leai.ai.gemini_client import GeminiClient
|
|
5
|
+
from leai.ai.anthropic_client import AnthropicClient
|
|
6
|
+
|
|
7
|
+
__all__ = [
|
|
8
|
+
"BaseLLMClient",
|
|
9
|
+
"get_llm_client",
|
|
10
|
+
"OpenAICompatibleClient",
|
|
11
|
+
"GeminiClient",
|
|
12
|
+
"AnthropicClient",
|
|
13
|
+
]
|