tracemind-regression 0.1.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- tracemind_regression-0.1.1/LICENSE +21 -0
- tracemind_regression-0.1.1/PKG-INFO +327 -0
- tracemind_regression-0.1.1/README.md +318 -0
- tracemind_regression-0.1.1/pyproject.toml +21 -0
- tracemind_regression-0.1.1/sdk/tracemind/__init__.py +2 -0
- tracemind_regression-0.1.1/sdk/tracemind/collector.py +54 -0
- tracemind_regression-0.1.1/sdk/tracemind/config.py +0 -0
- tracemind_regression-0.1.1/sdk/tracemind/models.py +111 -0
- tracemind_regression-0.1.1/sdk/tracemind/serialize.py +109 -0
- tracemind_regression-0.1.1/sdk/tracemind/tracker.py +81 -0
- tracemind_regression-0.1.1/sdk/tracemind_regression.egg-info/PKG-INFO +327 -0
- tracemind_regression-0.1.1/sdk/tracemind_regression.egg-info/SOURCES.txt +14 -0
- tracemind_regression-0.1.1/sdk/tracemind_regression.egg-info/dependency_links.txt +1 -0
- tracemind_regression-0.1.1/sdk/tracemind_regression.egg-info/top_level.txt +1 -0
- tracemind_regression-0.1.1/setup.cfg +4 -0
- tracemind_regression-0.1.1/tests/test_tracker.py +0 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Tejas Rajesh Dharmale
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,327 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: tracemind-regression
|
|
3
|
+
Version: 0.1.1
|
|
4
|
+
Summary: Python SDK for tracking function execution and detecting behavioural regressions
|
|
5
|
+
Requires-Python: >=3.10
|
|
6
|
+
Description-Content-Type: text/markdown
|
|
7
|
+
License-File: LICENSE
|
|
8
|
+
Dynamic: license-file
|
|
9
|
+
|
|
10
|
+
# TraceMind
|
|
11
|
+
|
|
12
|
+
### AI-Powered Software Behaviour Intelligence & Regression Detection
|
|
13
|
+
|
|
14
|
+
> **Understand how your software behaves. Detect what changes. Discover what your tests missed.**
|
|
15
|
+
|
|
16
|
+
TraceMind is an AI-powered developer tool designed to observe software execution, learn behavioural patterns, and identify unexpected changes between software versions.
|
|
17
|
+
|
|
18
|
+
It aims to help developers detect behavioural regressions, discover missing test cases, and understand the impact of software updates before they reach production.
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## The Problem
|
|
23
|
+
|
|
24
|
+
Software applications evolve continuously. Developers update functions, APIs, business logic, dependencies, and models.
|
|
25
|
+
|
|
26
|
+
Even when an application passes its existing tests, a new release can introduce unexpected behavioural changes.
|
|
27
|
+
|
|
28
|
+
For example:
|
|
29
|
+
|
|
30
|
+
- An API starts returning a different response structure.
|
|
31
|
+
- A function produces unexpected outputs for certain inputs.
|
|
32
|
+
- An update increases execution latency.
|
|
33
|
+
- An AI model starts producing different predictions.
|
|
34
|
+
- Existing tests fail to cover unusual input conditions.
|
|
35
|
+
|
|
36
|
+
Traditional testing checks predefined expectations. Monitoring tools track system health and performance.
|
|
37
|
+
|
|
38
|
+
However, developers also need a way to understand how software behaviour evolves across versions.
|
|
39
|
+
|
|
40
|
+
**TraceMind aims to bridge this gap.**
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
## The Proposed Solution
|
|
45
|
+
|
|
46
|
+
TraceMind observes selected software executions and builds a historical record of their behaviour.
|
|
47
|
+
|
|
48
|
+
When a new software version is introduced, it compares the previous and updated behaviour using representative inputs.
|
|
49
|
+
|
|
50
|
+
Its analysis engine identifies differences, detects unusual patterns, and helps developers investigate potential regressions.
|
|
51
|
+
|
|
52
|
+
AI is used selectively to explain meaningful changes and suggest additional test cases.
|
|
53
|
+
|
|
54
|
+
### How It Works
|
|
55
|
+
|
|
56
|
+
1. **Observe:** Capture selected function inputs, outputs, exceptions, and execution times through a Python SDK.
|
|
57
|
+
2. **Store:** Send execution records to a backend and maintain historical trace data.
|
|
58
|
+
3. **Compare:** Evaluate behavioural differences between software versions.
|
|
59
|
+
4. **Detect:** Use statistical methods and machine learning to identify unusual patterns.
|
|
60
|
+
5. **Generate:** Suggest additional regression tests for uncovered input conditions.
|
|
61
|
+
6. **Explain:** Use an LLM to provide understandable explanations and test recommendations.
|
|
62
|
+
7. **Visualise:** Display behavioural changes and analysis results through a developer dashboard.
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## System Architecture
|
|
67
|
+
|
|
68
|
+
```text
|
|
69
|
+
Developer Application
|
|
70
|
+
|
|
|
71
|
+
v
|
|
72
|
+
TraceMind Python SDK
|
|
73
|
+
|
|
|
74
|
+
v
|
|
75
|
+
FastAPI Backend
|
|
76
|
+
|
|
|
77
|
+
Validation Layer
|
|
78
|
+
/ \
|
|
79
|
+
v v
|
|
80
|
+
PostgreSQL Redis Queue
|
|
81
|
+
|
|
|
82
|
+
v
|
|
83
|
+
Celery Workers
|
|
84
|
+
|
|
|
85
|
+
v
|
|
86
|
+
Behaviour Analysis
|
|
87
|
+
/ \
|
|
88
|
+
v v
|
|
89
|
+
ML Engine Test Generator
|
|
90
|
+
\ /
|
|
91
|
+
v v
|
|
92
|
+
AI Analysis
|
|
93
|
+
|
|
|
94
|
+
v
|
|
95
|
+
Analysis Results
|
|
96
|
+
|
|
|
97
|
+
+--------+--------+
|
|
98
|
+
| |
|
|
99
|
+
v v
|
|
100
|
+
React Dashboard CI/CD
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
### Core Components
|
|
104
|
+
|
|
105
|
+
| Component | Responsibility |
|
|
106
|
+
|---|---|
|
|
107
|
+
| Python SDK | Capture execution traces |
|
|
108
|
+
| FastAPI | API and data ingestion |
|
|
109
|
+
| PostgreSQL | Store projects, versions and traces |
|
|
110
|
+
| Redis | Background job queue |
|
|
111
|
+
| Celery | Asynchronous processing |
|
|
112
|
+
| Comparison Engine | Identify behavioural differences |
|
|
113
|
+
| ML Engine | Detect unusual execution patterns |
|
|
114
|
+
| AI Service | Explain changes and suggest tests |
|
|
115
|
+
| React | Developer dashboard |
|
|
116
|
+
|
|
117
|
+
---
|
|
118
|
+
|
|
119
|
+
## 🧠 AI and Machine Learning Strategy
|
|
120
|
+
|
|
121
|
+
TraceMind follows a hybrid architecture.
|
|
122
|
+
|
|
123
|
+
Not every operation requires an LLM.
|
|
124
|
+
|
|
125
|
+
| Operation | Approach |
|
|
126
|
+
|---|---|
|
|
127
|
+
| Trace collection | Python |
|
|
128
|
+
| Duplicate detection | Hashing |
|
|
129
|
+
| Output comparison | Deterministic algorithms |
|
|
130
|
+
| Feature extraction | NumPy / pandas |
|
|
131
|
+
| Anomaly detection | Scikit-learn |
|
|
132
|
+
| Test generation | LLM |
|
|
133
|
+
| Behaviour explanations | LLM |
|
|
134
|
+
| Result caching | Fingerprinting |
|
|
135
|
+
|
|
136
|
+
### Token Optimisation
|
|
137
|
+
|
|
138
|
+
The project aims to minimise unnecessary LLM API consumption through:
|
|
139
|
+
|
|
140
|
+
- Selective LLM invocation.
|
|
141
|
+
- Hash-based deduplication.
|
|
142
|
+
- Caching previously generated explanations.
|
|
143
|
+
- Sending compact, relevant execution records.
|
|
144
|
+
- Structured JSON responses.
|
|
145
|
+
- Keeping deterministic analysis outside the LLM.
|
|
146
|
+
|
|
147
|
+
The objective is to use AI where reasoning adds value rather than treating an LLM as the entire backend.
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
151
|
+
## 🛠️ Technology Stack
|
|
152
|
+
|
|
153
|
+
| Area | Technology |
|
|
154
|
+
|---|---|
|
|
155
|
+
| Language | Python |
|
|
156
|
+
| SDK | Python |
|
|
157
|
+
| Backend | FastAPI |
|
|
158
|
+
| Database | PostgreSQL |
|
|
159
|
+
| Queue | Redis |
|
|
160
|
+
| Background Jobs | Celery |
|
|
161
|
+
| Data Processing | NumPy, pandas |
|
|
162
|
+
| Machine Learning | Scikit-learn |
|
|
163
|
+
| AI Integration | Gemini API |
|
|
164
|
+
| Frontend | React |
|
|
165
|
+
| Testing | pytest, Hypothesis |
|
|
166
|
+
| Containerisation | Docker |
|
|
167
|
+
| Version Control | Git, GitHub |
|
|
168
|
+
|
|
169
|
+
---
|
|
170
|
+
|
|
171
|
+
## Project Structure
|
|
172
|
+
|
|
173
|
+
```text
|
|
174
|
+
tracemind/
|
|
175
|
+
│
|
|
176
|
+
├── sdk/
|
|
177
|
+
│ ├── tracemind/
|
|
178
|
+
│ │ ├── tracker.py
|
|
179
|
+
│ │ ├── collector.py
|
|
180
|
+
│ │ ├── serializer.py
|
|
181
|
+
│ │ └── config.py
|
|
182
|
+
│ └── tests/
|
|
183
|
+
│
|
|
184
|
+
├── backend/
|
|
185
|
+
│ └── app/
|
|
186
|
+
│ ├── api/
|
|
187
|
+
│ ├── models/
|
|
188
|
+
│ ├── schemas/
|
|
189
|
+
│ ├── services/
|
|
190
|
+
│ ├── database/
|
|
191
|
+
│ └── main.py
|
|
192
|
+
│
|
|
193
|
+
├── workers/
|
|
194
|
+
│ ├── tasks.py
|
|
195
|
+
│ └── processor.py
|
|
196
|
+
│
|
|
197
|
+
├── engine/
|
|
198
|
+
│ ├── comparator.py
|
|
199
|
+
│ ├── features.py
|
|
200
|
+
│ ├── anomaly.py
|
|
201
|
+
│ ├── test_generator.py
|
|
202
|
+
│ └── evaluator.py
|
|
203
|
+
│
|
|
204
|
+
├── ai/
|
|
205
|
+
│ ├── analyzer.py
|
|
206
|
+
│ ├── prompts.py
|
|
207
|
+
│ └── cache.py
|
|
208
|
+
│
|
|
209
|
+
├── frontend/
|
|
210
|
+
│ └── src/
|
|
211
|
+
│
|
|
212
|
+
├── infrastructure/
|
|
213
|
+
│ └── docker-compose.yml
|
|
214
|
+
│
|
|
215
|
+
├── docs/
|
|
216
|
+
├── README.md
|
|
217
|
+
└── requirements.txt
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
---
|
|
221
|
+
|
|
222
|
+
## Development Roadmap
|
|
223
|
+
|
|
224
|
+
The project is planned as an eight-week MVP development cycle.
|
|
225
|
+
|
|
226
|
+
| Week | Milestone | Expected Deliverable |
|
|
227
|
+
|---|---|---|
|
|
228
|
+
| 01 | Python SDK | Function execution recorder |
|
|
229
|
+
| 02 | Backend | FastAPI + PostgreSQL |
|
|
230
|
+
| 03 | Comparison Engine | Version behaviour comparison |
|
|
231
|
+
| 04 | ML Engine | Anomaly detection |
|
|
232
|
+
| 05 | AI Integration | Test suggestions and explanations |
|
|
233
|
+
| 06 | Frontend | React dashboard |
|
|
234
|
+
| 07 | Infrastructure | Redis, Celery and Docker |
|
|
235
|
+
| 08 | Finalisation | Testing, documentation and demo |
|
|
236
|
+
|
|
237
|
+
### Development Status
|
|
238
|
+
|
|
239
|
+
- [ ] Python execution tracking SDK
|
|
240
|
+
- [ ] Trace ingestion API
|
|
241
|
+
- [ ] PostgreSQL integration
|
|
242
|
+
- [ ] Behaviour comparison engine
|
|
243
|
+
- [ ] ML anomaly detection
|
|
244
|
+
- [ ] AI-powered test suggestions
|
|
245
|
+
- [ ] AI-generated explanations
|
|
246
|
+
- [ ] React dashboard
|
|
247
|
+
- [ ] Background processing
|
|
248
|
+
- [ ] Docker deployment
|
|
249
|
+
- [ ] End-to-end testing
|
|
250
|
+
|
|
251
|
+
*Note: These are planned milestones, not claims of completed functionality.*
|
|
252
|
+
|
|
253
|
+
---
|
|
254
|
+
|
|
255
|
+
## Example Use Case
|
|
256
|
+
|
|
257
|
+
Consider a company maintaining a Python-based backend service.
|
|
258
|
+
|
|
259
|
+
A developer updates an existing function. The function still executes successfully, but its output changes for certain inputs.
|
|
260
|
+
|
|
261
|
+
TraceMind aims to:
|
|
262
|
+
|
|
263
|
+
1. Retrieve representative execution inputs from the previous version.
|
|
264
|
+
2. Execute those inputs against the updated version in a controlled environment.
|
|
265
|
+
3. Compare the old and new outputs.
|
|
266
|
+
4. Identify unexpected behavioural changes.
|
|
267
|
+
5. Suggest additional test cases.
|
|
268
|
+
6. Generate an explanation for the developer.
|
|
269
|
+
|
|
270
|
+
This allows engineering teams to investigate potential regressions before deploying a new release.
|
|
271
|
+
|
|
272
|
+
---
|
|
273
|
+
|
|
274
|
+
## Security Considerations
|
|
275
|
+
|
|
276
|
+
TraceMind is designed with security considerations from the beginning:
|
|
277
|
+
|
|
278
|
+
- Sensitive-data redaction before trace storage.
|
|
279
|
+
- Configurable trace collection.
|
|
280
|
+
- Environment-based API credentials.
|
|
281
|
+
- Project-level access control.
|
|
282
|
+
- Controlled execution for replayed test inputs.
|
|
283
|
+
- No automatic modification of application source code.
|
|
284
|
+
|
|
285
|
+
Production deployment will require additional security and isolation controls.
|
|
286
|
+
|
|
287
|
+
---
|
|
288
|
+
|
|
289
|
+
## Long-Term Vision
|
|
290
|
+
|
|
291
|
+
The initial MVP focuses on Python applications.
|
|
292
|
+
|
|
293
|
+
Future possibilities include:
|
|
294
|
+
|
|
295
|
+
- API behavioural intelligence.
|
|
296
|
+
- Distributed system regression analysis.
|
|
297
|
+
- AI model behaviour comparison.
|
|
298
|
+
- CI/CD pipeline integration.
|
|
299
|
+
- Automated test coverage discovery.
|
|
300
|
+
- Multi-language SDK support.
|
|
301
|
+
- Enterprise deployment and access controls.
|
|
302
|
+
|
|
303
|
+
The long-term goal is to make TraceMind a behavioural intelligence layer that helps engineering teams understand how their software changes over time.
|
|
304
|
+
|
|
305
|
+
---
|
|
306
|
+
|
|
307
|
+
## Project Motivation
|
|
308
|
+
|
|
309
|
+
TraceMind is an independent engineering project exploring the intersection of:
|
|
310
|
+
|
|
311
|
+
- Artificial Intelligence
|
|
312
|
+
- Machine Learning
|
|
313
|
+
- Backend Engineering
|
|
314
|
+
- Software Testing
|
|
315
|
+
- System Design
|
|
316
|
+
- Developer Infrastructure
|
|
317
|
+
|
|
318
|
+
The objective is to build a practical AI-enabled engineering product while developing deeper expertise in scalable backend systems, ML engineering, and efficient LLM integration.
|
|
319
|
+
|
|
320
|
+
---
|
|
321
|
+
|
|
322
|
+
**Project:** TraceMind
|
|
323
|
+
**Category:** AI / ML Engineering + Developer Tools
|
|
324
|
+
**Initial Language Support:** Python
|
|
325
|
+
**Development Target:** 8-week MVP
|
|
326
|
+
|
|
327
|
+
⭐ If you find this project interesting, consider following its development.
|
|
@@ -0,0 +1,318 @@
|
|
|
1
|
+
# TraceMind
|
|
2
|
+
|
|
3
|
+
### AI-Powered Software Behaviour Intelligence & Regression Detection
|
|
4
|
+
|
|
5
|
+
> **Understand how your software behaves. Detect what changes. Discover what your tests missed.**
|
|
6
|
+
|
|
7
|
+
TraceMind is an AI-powered developer tool designed to observe software execution, learn behavioural patterns, and identify unexpected changes between software versions.
|
|
8
|
+
|
|
9
|
+
It aims to help developers detect behavioural regressions, discover missing test cases, and understand the impact of software updates before they reach production.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## The Problem
|
|
14
|
+
|
|
15
|
+
Software applications evolve continuously. Developers update functions, APIs, business logic, dependencies, and models.
|
|
16
|
+
|
|
17
|
+
Even when an application passes its existing tests, a new release can introduce unexpected behavioural changes.
|
|
18
|
+
|
|
19
|
+
For example:
|
|
20
|
+
|
|
21
|
+
- An API starts returning a different response structure.
|
|
22
|
+
- A function produces unexpected outputs for certain inputs.
|
|
23
|
+
- An update increases execution latency.
|
|
24
|
+
- An AI model starts producing different predictions.
|
|
25
|
+
- Existing tests fail to cover unusual input conditions.
|
|
26
|
+
|
|
27
|
+
Traditional testing checks predefined expectations. Monitoring tools track system health and performance.
|
|
28
|
+
|
|
29
|
+
However, developers also need a way to understand how software behaviour evolves across versions.
|
|
30
|
+
|
|
31
|
+
**TraceMind aims to bridge this gap.**
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## The Proposed Solution
|
|
36
|
+
|
|
37
|
+
TraceMind observes selected software executions and builds a historical record of their behaviour.
|
|
38
|
+
|
|
39
|
+
When a new software version is introduced, it compares the previous and updated behaviour using representative inputs.
|
|
40
|
+
|
|
41
|
+
Its analysis engine identifies differences, detects unusual patterns, and helps developers investigate potential regressions.
|
|
42
|
+
|
|
43
|
+
AI is used selectively to explain meaningful changes and suggest additional test cases.
|
|
44
|
+
|
|
45
|
+
### How It Works
|
|
46
|
+
|
|
47
|
+
1. **Observe:** Capture selected function inputs, outputs, exceptions, and execution times through a Python SDK.
|
|
48
|
+
2. **Store:** Send execution records to a backend and maintain historical trace data.
|
|
49
|
+
3. **Compare:** Evaluate behavioural differences between software versions.
|
|
50
|
+
4. **Detect:** Use statistical methods and machine learning to identify unusual patterns.
|
|
51
|
+
5. **Generate:** Suggest additional regression tests for uncovered input conditions.
|
|
52
|
+
6. **Explain:** Use an LLM to provide understandable explanations and test recommendations.
|
|
53
|
+
7. **Visualise:** Display behavioural changes and analysis results through a developer dashboard.
|
|
54
|
+
|
|
55
|
+
---
|
|
56
|
+
|
|
57
|
+
## System Architecture
|
|
58
|
+
|
|
59
|
+
```text
|
|
60
|
+
Developer Application
|
|
61
|
+
|
|
|
62
|
+
v
|
|
63
|
+
TraceMind Python SDK
|
|
64
|
+
|
|
|
65
|
+
v
|
|
66
|
+
FastAPI Backend
|
|
67
|
+
|
|
|
68
|
+
Validation Layer
|
|
69
|
+
/ \
|
|
70
|
+
v v
|
|
71
|
+
PostgreSQL Redis Queue
|
|
72
|
+
|
|
|
73
|
+
v
|
|
74
|
+
Celery Workers
|
|
75
|
+
|
|
|
76
|
+
v
|
|
77
|
+
Behaviour Analysis
|
|
78
|
+
/ \
|
|
79
|
+
v v
|
|
80
|
+
ML Engine Test Generator
|
|
81
|
+
\ /
|
|
82
|
+
v v
|
|
83
|
+
AI Analysis
|
|
84
|
+
|
|
|
85
|
+
v
|
|
86
|
+
Analysis Results
|
|
87
|
+
|
|
|
88
|
+
+--------+--------+
|
|
89
|
+
| |
|
|
90
|
+
v v
|
|
91
|
+
React Dashboard CI/CD
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
### Core Components
|
|
95
|
+
|
|
96
|
+
| Component | Responsibility |
|
|
97
|
+
|---|---|
|
|
98
|
+
| Python SDK | Capture execution traces |
|
|
99
|
+
| FastAPI | API and data ingestion |
|
|
100
|
+
| PostgreSQL | Store projects, versions and traces |
|
|
101
|
+
| Redis | Background job queue |
|
|
102
|
+
| Celery | Asynchronous processing |
|
|
103
|
+
| Comparison Engine | Identify behavioural differences |
|
|
104
|
+
| ML Engine | Detect unusual execution patterns |
|
|
105
|
+
| AI Service | Explain changes and suggest tests |
|
|
106
|
+
| React | Developer dashboard |
|
|
107
|
+
|
|
108
|
+
---
|
|
109
|
+
|
|
110
|
+
## 🧠 AI and Machine Learning Strategy
|
|
111
|
+
|
|
112
|
+
TraceMind follows a hybrid architecture.
|
|
113
|
+
|
|
114
|
+
Not every operation requires an LLM.
|
|
115
|
+
|
|
116
|
+
| Operation | Approach |
|
|
117
|
+
|---|---|
|
|
118
|
+
| Trace collection | Python |
|
|
119
|
+
| Duplicate detection | Hashing |
|
|
120
|
+
| Output comparison | Deterministic algorithms |
|
|
121
|
+
| Feature extraction | NumPy / pandas |
|
|
122
|
+
| Anomaly detection | Scikit-learn |
|
|
123
|
+
| Test generation | LLM |
|
|
124
|
+
| Behaviour explanations | LLM |
|
|
125
|
+
| Result caching | Fingerprinting |
|
|
126
|
+
|
|
127
|
+
### Token Optimisation
|
|
128
|
+
|
|
129
|
+
The project aims to minimise unnecessary LLM API consumption through:
|
|
130
|
+
|
|
131
|
+
- Selective LLM invocation.
|
|
132
|
+
- Hash-based deduplication.
|
|
133
|
+
- Caching previously generated explanations.
|
|
134
|
+
- Sending compact, relevant execution records.
|
|
135
|
+
- Structured JSON responses.
|
|
136
|
+
- Keeping deterministic analysis outside the LLM.
|
|
137
|
+
|
|
138
|
+
The objective is to use AI where reasoning adds value rather than treating an LLM as the entire backend.
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## 🛠️ Technology Stack
|
|
143
|
+
|
|
144
|
+
| Area | Technology |
|
|
145
|
+
|---|---|
|
|
146
|
+
| Language | Python |
|
|
147
|
+
| SDK | Python |
|
|
148
|
+
| Backend | FastAPI |
|
|
149
|
+
| Database | PostgreSQL |
|
|
150
|
+
| Queue | Redis |
|
|
151
|
+
| Background Jobs | Celery |
|
|
152
|
+
| Data Processing | NumPy, pandas |
|
|
153
|
+
| Machine Learning | Scikit-learn |
|
|
154
|
+
| AI Integration | Gemini API |
|
|
155
|
+
| Frontend | React |
|
|
156
|
+
| Testing | pytest, Hypothesis |
|
|
157
|
+
| Containerisation | Docker |
|
|
158
|
+
| Version Control | Git, GitHub |
|
|
159
|
+
|
|
160
|
+
---
|
|
161
|
+
|
|
162
|
+
## Project Structure
|
|
163
|
+
|
|
164
|
+
```text
|
|
165
|
+
tracemind/
|
|
166
|
+
│
|
|
167
|
+
├── sdk/
|
|
168
|
+
│ ├── tracemind/
|
|
169
|
+
│ │ ├── tracker.py
|
|
170
|
+
│ │ ├── collector.py
|
|
171
|
+
│ │ ├── serializer.py
|
|
172
|
+
│ │ └── config.py
|
|
173
|
+
│ └── tests/
|
|
174
|
+
│
|
|
175
|
+
├── backend/
|
|
176
|
+
│ └── app/
|
|
177
|
+
│ ├── api/
|
|
178
|
+
│ ├── models/
|
|
179
|
+
│ ├── schemas/
|
|
180
|
+
│ ├── services/
|
|
181
|
+
│ ├── database/
|
|
182
|
+
│ └── main.py
|
|
183
|
+
│
|
|
184
|
+
├── workers/
|
|
185
|
+
│ ├── tasks.py
|
|
186
|
+
│ └── processor.py
|
|
187
|
+
│
|
|
188
|
+
├── engine/
|
|
189
|
+
│ ├── comparator.py
|
|
190
|
+
│ ├── features.py
|
|
191
|
+
│ ├── anomaly.py
|
|
192
|
+
│ ├── test_generator.py
|
|
193
|
+
│ └── evaluator.py
|
|
194
|
+
│
|
|
195
|
+
├── ai/
|
|
196
|
+
│ ├── analyzer.py
|
|
197
|
+
│ ├── prompts.py
|
|
198
|
+
│ └── cache.py
|
|
199
|
+
│
|
|
200
|
+
├── frontend/
|
|
201
|
+
│ └── src/
|
|
202
|
+
│
|
|
203
|
+
├── infrastructure/
|
|
204
|
+
│ └── docker-compose.yml
|
|
205
|
+
│
|
|
206
|
+
├── docs/
|
|
207
|
+
├── README.md
|
|
208
|
+
└── requirements.txt
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
---
|
|
212
|
+
|
|
213
|
+
## Development Roadmap
|
|
214
|
+
|
|
215
|
+
The project is planned as an eight-week MVP development cycle.
|
|
216
|
+
|
|
217
|
+
| Week | Milestone | Expected Deliverable |
|
|
218
|
+
|---|---|---|
|
|
219
|
+
| 01 | Python SDK | Function execution recorder |
|
|
220
|
+
| 02 | Backend | FastAPI + PostgreSQL |
|
|
221
|
+
| 03 | Comparison Engine | Version behaviour comparison |
|
|
222
|
+
| 04 | ML Engine | Anomaly detection |
|
|
223
|
+
| 05 | AI Integration | Test suggestions and explanations |
|
|
224
|
+
| 06 | Frontend | React dashboard |
|
|
225
|
+
| 07 | Infrastructure | Redis, Celery and Docker |
|
|
226
|
+
| 08 | Finalisation | Testing, documentation and demo |
|
|
227
|
+
|
|
228
|
+
### Development Status
|
|
229
|
+
|
|
230
|
+
- [ ] Python execution tracking SDK
|
|
231
|
+
- [ ] Trace ingestion API
|
|
232
|
+
- [ ] PostgreSQL integration
|
|
233
|
+
- [ ] Behaviour comparison engine
|
|
234
|
+
- [ ] ML anomaly detection
|
|
235
|
+
- [ ] AI-powered test suggestions
|
|
236
|
+
- [ ] AI-generated explanations
|
|
237
|
+
- [ ] React dashboard
|
|
238
|
+
- [ ] Background processing
|
|
239
|
+
- [ ] Docker deployment
|
|
240
|
+
- [ ] End-to-end testing
|
|
241
|
+
|
|
242
|
+
*Note: These are planned milestones, not claims of completed functionality.*
|
|
243
|
+
|
|
244
|
+
---
|
|
245
|
+
|
|
246
|
+
## Example Use Case
|
|
247
|
+
|
|
248
|
+
Consider a company maintaining a Python-based backend service.
|
|
249
|
+
|
|
250
|
+
A developer updates an existing function. The function still executes successfully, but its output changes for certain inputs.
|
|
251
|
+
|
|
252
|
+
TraceMind aims to:
|
|
253
|
+
|
|
254
|
+
1. Retrieve representative execution inputs from the previous version.
|
|
255
|
+
2. Execute those inputs against the updated version in a controlled environment.
|
|
256
|
+
3. Compare the old and new outputs.
|
|
257
|
+
4. Identify unexpected behavioural changes.
|
|
258
|
+
5. Suggest additional test cases.
|
|
259
|
+
6. Generate an explanation for the developer.
|
|
260
|
+
|
|
261
|
+
This allows engineering teams to investigate potential regressions before deploying a new release.
|
|
262
|
+
|
|
263
|
+
---
|
|
264
|
+
|
|
265
|
+
## Security Considerations
|
|
266
|
+
|
|
267
|
+
TraceMind is designed with security considerations from the beginning:
|
|
268
|
+
|
|
269
|
+
- Sensitive-data redaction before trace storage.
|
|
270
|
+
- Configurable trace collection.
|
|
271
|
+
- Environment-based API credentials.
|
|
272
|
+
- Project-level access control.
|
|
273
|
+
- Controlled execution for replayed test inputs.
|
|
274
|
+
- No automatic modification of application source code.
|
|
275
|
+
|
|
276
|
+
Production deployment will require additional security and isolation controls.
|
|
277
|
+
|
|
278
|
+
---
|
|
279
|
+
|
|
280
|
+
## Long-Term Vision
|
|
281
|
+
|
|
282
|
+
The initial MVP focuses on Python applications.
|
|
283
|
+
|
|
284
|
+
Future possibilities include:
|
|
285
|
+
|
|
286
|
+
- API behavioural intelligence.
|
|
287
|
+
- Distributed system regression analysis.
|
|
288
|
+
- AI model behaviour comparison.
|
|
289
|
+
- CI/CD pipeline integration.
|
|
290
|
+
- Automated test coverage discovery.
|
|
291
|
+
- Multi-language SDK support.
|
|
292
|
+
- Enterprise deployment and access controls.
|
|
293
|
+
|
|
294
|
+
The long-term goal is to make TraceMind a behavioural intelligence layer that helps engineering teams understand how their software changes over time.
|
|
295
|
+
|
|
296
|
+
---
|
|
297
|
+
|
|
298
|
+
## Project Motivation
|
|
299
|
+
|
|
300
|
+
TraceMind is an independent engineering project exploring the intersection of:
|
|
301
|
+
|
|
302
|
+
- Artificial Intelligence
|
|
303
|
+
- Machine Learning
|
|
304
|
+
- Backend Engineering
|
|
305
|
+
- Software Testing
|
|
306
|
+
- System Design
|
|
307
|
+
- Developer Infrastructure
|
|
308
|
+
|
|
309
|
+
The objective is to build a practical AI-enabled engineering product while developing deeper expertise in scalable backend systems, ML engineering, and efficient LLM integration.
|
|
310
|
+
|
|
311
|
+
---
|
|
312
|
+
|
|
313
|
+
**Project:** TraceMind
|
|
314
|
+
**Category:** AI / ML Engineering + Developer Tools
|
|
315
|
+
**Initial Language Support:** Python
|
|
316
|
+
**Development Target:** 8-week MVP
|
|
317
|
+
|
|
318
|
+
⭐ If you find this project interesting, consider following its development.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=68"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "tracemind-regression"
|
|
7
|
+
version = "0.1.1"
|
|
8
|
+
description = "Python SDK for tracking function execution and detecting behavioural regressions"
|
|
9
|
+
readme = "README.md"
|
|
10
|
+
requires-python = ">=3.10"
|
|
11
|
+
dependencies = []
|
|
12
|
+
|
|
13
|
+
[tool.setuptools]
|
|
14
|
+
package-dir = {"" = "sdk"}
|
|
15
|
+
|
|
16
|
+
[tool.setuptools.packages.find]
|
|
17
|
+
where = ["sdk"]
|
|
18
|
+
|
|
19
|
+
[tool.pytest.ini_options]
|
|
20
|
+
pythonpath = ["sdk"]
|
|
21
|
+
testpaths = ["tests"]
|