parsely-dip 0.0.0__tar.gz → 0.0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (29) hide show
  1. parsely_dip-0.0.1/PKG-INFO +411 -0
  2. parsely_dip-0.0.1/README.md +393 -0
  3. {parsely_dip-0.0.0 → parsely_dip-0.0.1}/pyproject.toml +10 -1
  4. parsely_dip-0.0.1/src/parsely_dip/__init__.py +64 -0
  5. parsely_dip-0.0.1/src/parsely_dip/cli/main.py +23 -0
  6. parsely_dip-0.0.1/src/parsely_dip/engine/nlp.py +143 -0
  7. parsely_dip-0.0.1/src/parsely_dip/engine/regex.py +62 -0
  8. parsely_dip-0.0.1/src/parsely_dip/engine/registry.py +34 -0
  9. parsely_dip-0.0.1/src/parsely_dip/engine/splitter.py +19 -0
  10. parsely_dip-0.0.1/src/parsely_dip/engine/stanza_service.py +421 -0
  11. parsely_dip-0.0.1/src/parsely_dip/intents/__init__.py +1 -0
  12. parsely_dip-0.0.1/src/parsely_dip/intents/scrum.py +77 -0
  13. parsely_dip-0.0.1/src/parsely_dip/intents/time.py +19 -0
  14. parsely_dip-0.0.1/src/parsely_dip/intents/weather.py +37 -0
  15. parsely_dip-0.0.1/src/parsely_dip.egg-info/PKG-INFO +411 -0
  16. parsely_dip-0.0.1/src/parsely_dip.egg-info/SOURCES.txt +22 -0
  17. parsely_dip-0.0.1/src/parsely_dip.egg-info/entry_points.txt +2 -0
  18. parsely_dip-0.0.1/src/parsely_dip.egg-info/requires.txt +4 -0
  19. parsely_dip-0.0.0/PKG-INFO +0 -80
  20. parsely_dip-0.0.0/README.md +0 -66
  21. parsely_dip-0.0.0/src/parsely_dip/__init__.py +0 -3
  22. parsely_dip-0.0.0/src/parsely_dip.egg-info/PKG-INFO +0 -80
  23. parsely_dip-0.0.0/src/parsely_dip.egg-info/SOURCES.txt +0 -10
  24. {parsely_dip-0.0.0 → parsely_dip-0.0.1}/setup.cfg +0 -0
  25. {parsely_dip-0.0.0 → parsely_dip-0.0.1}/src/parsely_dip/cli/__init__.py +0 -0
  26. {parsely_dip-0.0.0 → parsely_dip-0.0.1}/src/parsely_dip/engine/__init__.py +0 -0
  27. {parsely_dip-0.0.0/src/parsely_dip/intents → parsely_dip-0.0.1/src/parsely_dip/patterns}/__init__.py +0 -0
  28. {parsely_dip-0.0.0 → parsely_dip-0.0.1}/src/parsely_dip.egg-info/dependency_links.txt +0 -0
  29. {parsely_dip-0.0.0 → parsely_dip-0.0.1}/src/parsely_dip.egg-info/top_level.txt +0 -0
@@ -0,0 +1,411 @@
1
+ Metadata-Version: 2.4
2
+ Name: parsely-dip
3
+ Version: 0.0.1
4
+ Summary: PARSELY-DIP: Deterministic Intent Parser — RegEx and NLP pipeline for intent recognition
5
+ Author-email: George Butiri <george@iseestudios.com>
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/gbutiri/parsely-dip
8
+ Keywords: nlp,intent,parser,deterministic,regex
9
+ Classifier: Development Status :: 1 - Planning
10
+ Classifier: Programming Language :: Python :: 3
11
+ Classifier: Topic :: Text Processing :: Linguistic
12
+ Requires-Python: >=3.9
13
+ Description-Content-Type: text/markdown
14
+ Requires-Dist: stanza>=1.5
15
+ Requires-Dist: requests>=2.28
16
+ Requires-Dist: python-dotenv>=1.0
17
+ Requires-Dist: flask>=3.0
18
+
19
+ # PARSELY-DIP
20
+
21
+ **Parsing And RegEx Syntactic Engine with Linguistic Yield — Deterministic Intent Parser**
22
+
23
+ *Parsely dip for silicon chips.*
24
+
25
+ A deterministic intent recognition engine that processes natural language through a cascading pipeline — RegEx first, then constituency and dependency parsing via Stanza, then LLM fallback. Each layer only fires if the one above didn't match. The cheapest, fastest layer runs first. The LLM is the last resort, not the default.
26
+
27
+ Your LLM is expensive, slow, and unpredictable. When a user says "what time is it" or "move the card to done," there is zero ambiguity. A regex handles it in microseconds. An LLM spends tokens guessing what you already know. PARSELY-DIP intercepts deterministic commands before they reach the LLM, executes them directly, and returns the result.
28
+
29
+ ## What It Does
30
+
31
+ ```python
32
+ from parsely_dip import parse
33
+
34
+ result = parse("what time is it")
35
+ # result = "14:32"
36
+
37
+ result = parse("what is the weather like")
38
+ # result = "It's 36°F and broken clouds in Cleveland."
39
+
40
+ result = parse("tell me about quantum physics")
41
+ # result = None (no match — pass to LLM)
42
+ ```
43
+
44
+ One call. One input. Response string or None.
45
+
46
+ ## Install
47
+
48
+ ```bash
49
+ pip install parsely-dip
50
+ ```
51
+
52
+ From source:
53
+
54
+ ```bash
55
+ git clone https://github.com/gbutiri/parsely-dip.git
56
+ cd parsely-dip
57
+ pip install -e .
58
+ ```
59
+
60
+ ### NLP Layer Setup (Optional)
61
+
62
+ The RegEx layer works out of the box. The NLP layer requires Stanza and a running parse service.
63
+
64
+ **1. Download the Stanza English model (~526MB):**
65
+
66
+ ```bash
67
+ python -c "import stanza; stanza.download('en')"
68
+ ```
69
+
70
+ **2. (Recommended) Download the accurate model with transformer support:**
71
+
72
+ ```bash
73
+ python -c "import stanza; stanza.download('en', package='default_accurate')"
74
+ pip install transformers sentencepiece
75
+ ```
76
+
77
+ The `default_accurate` model uses PEFT fine-tuned transformers (Google Electra Large). The biggest accuracy improvement is in constituency parsing — the core of NLP intent matching. Requires ~1-2GB extra VRAM on a dedicated GPU.
78
+
79
+ **3. (Recommended) Install PyTorch with GPU support:**
80
+
81
+ ```bash
82
+ pip install torch torchvision --index-url https://download.pytorch.org/whl/cu130
83
+ ```
84
+
85
+ Without this, Stanza runs on CPU. With a dedicated GPU (RTX 3060+), parsing is near-instant.
86
+
87
+ **4. Start the NLP service:**
88
+
89
+ ```bash
90
+ python -m parsely_dip.engine.stanza_service
91
+ ```
92
+
93
+ The service loads once and stays running. PARSELY-DIP calls it via HTTP on port 5013 for each query that passes the RegEx layer. The service auto-detects the best available model (`default_accurate` > `default`) and reports GPU status on startup.
94
+
95
+ ---
96
+
97
+ ## Three-Tier Pipeline
98
+
99
+ ```
100
+ User Input
101
+ |
102
+ v
103
+ [RegEx Layer] — Pattern matching, microseconds, zero dependencies
104
+ | match? --> handler executes, returns response
105
+ | no match? --> continue
106
+ v
107
+ [NLP Layer] — Stanza constituency + dependency parsing via HTTP service
108
+ | match? --> handler executes, returns response
109
+ | no match? --> continue
110
+ v
111
+ [LLM Fallback] — parse() returns None, caller decides what to do
112
+ ```
113
+
114
+ ### Layer 1: RegEx
115
+
116
+ Patterns stored in flat `.patterns` text files. One pattern per line. No JSON escaping nightmares.
117
+
118
+ ```
119
+ # Format: (regex) => intent_name
120
+ # intents/time.py
121
+ (what('s|\s+is)\s+the\s+time|what\s+time\s+is\s+it)\?? => tell_time
122
+
123
+ # intents/weather.py
124
+ ((what|how)('s|\s+is)\s+the\s+weather(\s+like)?)\?? => tell_weather
125
+
126
+ # intents/scrum.py
127
+ show(\s+me|\s+us)?\s+the\s+(current|active)(\s+scrum)?\s+cards?[.!]? => show_current_card
128
+ ```
129
+
130
+ **Pattern convention:** `\s+` goes BEFORE the word it separates, not after.
131
+
132
+ ```
133
+ CORRECT: (what('s|\s+is)\s+the\s+time)
134
+ WRONG: (what('s|is\s+)the\s+time\s+)
135
+ ```
136
+
137
+ The space belongs to the approach of the next word, not trailing from the previous.
138
+
139
+ ### Layer 2: NLP
140
+
141
+ Patterns stored in `.json` files. Each pattern defines a grammatical structure using sentence type, POS tags, dependency relations, and head words. Matches on linguistic features, not exact strings — so "what time is it, please?" and "hey, what's the time right now?" both match without needing separate regex patterns.
142
+
143
+ ```json
144
+ [
145
+ {
146
+ "intent": "tell_time",
147
+ "nlp": {
148
+ "sentence_type": ["SBARQ", "SQ", "WHNP"],
149
+ "words": [
150
+ {"word": "what", "pos": "DET", "dep": "det", "required": true},
151
+ {"lemma": "time", "pos": "NOUN", "required": true},
152
+ {"lemma": "be", "pos": "AUX", "dep": "cop", "required": true},
153
+ {"word": "it", "pos": "PRON", "dep": "nsubj", "required": true}
154
+ ]
155
+ }
156
+ }
157
+ ]
158
+ ```
159
+
160
+ The NLP layer requires the Stanza service running on port 5013. If the service is not running, the NLP layer is silently skipped and the pipeline falls through to LLM.
161
+
162
+ ### Layer 3: LLM Fallback
163
+
164
+ `parse()` returns `None`. The caller decides what to do — send to an LLM, show an error, or ignore. PARSELY-DIP does not call any LLM itself.
165
+
166
+ ---
167
+
168
+ ## Intent Handlers
169
+
170
+ Self-registering via the `@intent` decorator. Import the module, the decorator registers the handler. No config files, no setup step.
171
+
172
+ ```python
173
+ from parsely_dip.engine.registry import intent
174
+
175
+ @intent('tell_time')
176
+ def tell_time():
177
+ from datetime import datetime
178
+ now = datetime.now()
179
+ return f"{now.hour:02d}:{now.minute:02d}"
180
+ ```
181
+
182
+ ### Built-in Intents
183
+
184
+ | Intent | File | What It Does |
185
+ |--------|------|-------------|
186
+ | `tell_time` | `intents/time.py` | Returns current time in 24-hour format |
187
+ | `tell_weather` | `intents/weather.py` | Returns weather via OpenWeatherMap API (requires `WEATHER_API_KEY` in `.env`) |
188
+ | `show_current_card` | `intents/scrum.py` | Shows active scrum cards from SQLite database |
189
+ | `read_current_card` | `intents/scrum.py` | Same data as show, but intended for LLM to summarize |
190
+
191
+ ### Adding New Intents
192
+
193
+ 1. Create a new file in `intents/` (e.g., `intents/greeting.py`)
194
+ 2. Write a handler function with the `@intent` decorator
195
+ 3. Add regex patterns to `patterns/base.patterns`
196
+ 4. (Optional) Add NLP patterns to `patterns/base_nlp.json`
197
+ 5. Import the module in `__init__.py`
198
+
199
+ ---
200
+
201
+ ## Project Structure
202
+
203
+ ```
204
+ parsely-dip/
205
+ pyproject.toml — Package config, dependencies
206
+ README.md — This file
207
+ env_parselydip/ — Virtual environment
208
+ db/ — Database files (if needed by intents)
209
+ logs/ — Log files
210
+ tests/ — Test suite
211
+ src/parsely_dip/
212
+ __init__.py — parse(prompt) single entry point
213
+ engine/
214
+ registry.py — @intent decorator, handler registry, dispatch()
215
+ regex.py — load_patterns(), check_regex()
216
+ nlp.py — load_nlp_patterns(), check_nlp(), match_nlp_pattern()
217
+ splitter.py — Sentence splitting (future expansion)
218
+ stanza_service.py — Stanza NLP Flask service (port 5013)
219
+ intents/
220
+ __init__.py — Auto-imports all intent modules
221
+ time.py — tell_time handler
222
+ weather.py — tell_weather handler (OpenWeatherMap API)
223
+ scrum.py — show_current_card, read_current_card handlers
224
+ patterns/
225
+ base.patterns — RegEx patterns (flat text, one per line)
226
+ base_nlp.json — NLP patterns (structured JSON)
227
+ cli/
228
+ __init__.py — CLI entry point (future)
229
+ ```
230
+
231
+ ---
232
+
233
+ ## Hook Integration
234
+
235
+ PARSELY-DIP is designed to run as a Claude Code `UserPromptSubmit` hook. The hook intercepts the user's message, runs it through the pipeline, and either handles it deterministically or lets the LLM process it.
236
+
237
+ ### Hook Script
238
+
239
+ ```bash
240
+ #!/bin/bash
241
+ PROJECT_DIR="${CLAUDE_PROJECT_DIR:-.}"
242
+ VENV_PY="$PROJECT_DIR/env_bibliotech/Scripts/python.exe"
243
+ [ ! -f "$VENV_PY" ] && exit 0
244
+
245
+ "$VENV_PY" -c "
246
+ import sys, json
247
+ from parsely_dip import parse
248
+ data = json.load(sys.stdin)
249
+ prompt = data.get('prompt', '')
250
+ if prompt:
251
+ r = parse(prompt)
252
+ if r:
253
+ print('=== PARSELY-DIP ===')
254
+ print('Relay this to the user EXACTLY as written, nothing else:')
255
+ print(r)
256
+ print('=== END PARSELY-DIP ===')
257
+ " 2>/dev/null
258
+ exit 0
259
+ ```
260
+
261
+ ### How It Works
262
+
263
+ 1. Hook reads the user's prompt from stdin (JSON with `prompt` field)
264
+ 2. Calls `parsely_dip.parse(prompt)`
265
+ 3. If result: prints it to stdout (shown to LLM as context, LLM relays verbatim)
266
+ 4. If None: no output, LLM processes the prompt normally
267
+
268
+ ### Known Limitation
269
+
270
+ Claude Code's `UserPromptSubmit` hooks cannot display text directly to the user without the LLM firing. The documented `decision: "block"` + `reason` field blocks the prompt but does not render the reason in the VS Code extension (confirmed bug). The current approach uses plain text stdout with exit 0 — the LLM sees the result and relays it.
271
+
272
+ ---
273
+
274
+ ## Stanza NLP Service
275
+
276
+ The NLP service is a Flask app that wraps Stanford's Stanza NLP library. It runs as a background service on port 5013, loads the model once at startup, and handles parse requests via HTTP.
277
+
278
+ ### Starting the Service
279
+
280
+ ```bash
281
+ python -m parsely_dip.engine.stanza_service
282
+ ```
283
+
284
+ ### What Happens at Startup
285
+
286
+ 1. Tries to load `default_accurate` (transformer-based, best accuracy)
287
+ 2. If that fails (missing packages), prompts the user to install or continue with standard
288
+ 3. Falls back to `default` (CharLM-based, solid accuracy)
289
+ 4. If no model found, prints install instructions and exits
290
+ 5. Reports GPU status (name of GPU if available, install command if not)
291
+
292
+ ### Service Endpoints
293
+
294
+ | Endpoint | Method | Description |
295
+ |----------|--------|-------------|
296
+ | `/process_syntactic_parsing` | POST | Parse text, return words with POS/dependency/constituency |
297
+ | `/debug_parse` | POST | Raw parse data for debugging sentence structure |
298
+
299
+ ### Interactive Mode
300
+
301
+ ```bash
302
+ python -m parsely_dip.engine.stanza_service --chat
303
+ ```
304
+
305
+ Opens an interactive prompt where you can type sentences and see their constituency trees and dependency relations. Useful for building new NLP patterns.
306
+
307
+ ### Security
308
+
309
+ - Localhost only (127.0.0.1) — rejects non-local requests
310
+ - Optional token auth via `STANZA_API_TOKEN` environment variable — enforced if set, skipped if not
311
+
312
+ ---
313
+
314
+ ## NLP Pattern Specification
315
+
316
+ NLP patterns define grammatical structures that map to intents. Unlike regex (exact string matching), NLP patterns match on linguistic features extracted by Stanza.
317
+
318
+ ### Pattern Structure
319
+
320
+ ```json
321
+ {
322
+ "intent": "intent_name",
323
+ "nlp": {
324
+ "sentence_type": "SBARQ",
325
+ "words": [
326
+ {
327
+ "word": "exact_word",
328
+ "lemma": "base_form",
329
+ "pos": "NOUN",
330
+ "dep": "nsubj",
331
+ "head_lemma": "parent_word",
332
+ "required": true
333
+ }
334
+ ]
335
+ }
336
+ }
337
+ ```
338
+
339
+ ### Matching Modes
340
+
341
+ - **Exact Word Match** — `word` specified: match that exact word in that grammatical position
342
+ - **Structural Match (Slot)** — `word` empty: match ANY word with specified POS + dependency features
343
+ - **Optional Words** — `required: false`: pattern matches with or without this word
344
+
345
+ ### Supported Values
346
+
347
+ **Sentence Types:** S, SBARQ, SQ, SINV, FRAG (+ 20 more constituency labels)
348
+
349
+ **POS Tags (17 Universal):** NOUN, VERB, AUX, ADJ, ADV, PRON, DET, ADP, NUM, PART, CCONJ, SCONJ, INTJ, PROPN, PUNCT, SYM, X
350
+
351
+ **Dependency Relations (37+):** nsubj, obj, root, det, cop, aux, mark, case, advmod, amod, compound, conj, cc, xcomp, ccomp, advcl, acl, nmod, obl, nummod, appos, dep, fixed, flat, list, parataxis, orphan, goeswith, reparandum, punct, clf, discourse, dislocated, expl, iobj, vocative, csubj
352
+
353
+ ### Specificity Rule
354
+
355
+ **A loose pattern that matches incorrectly is WORSE than no pattern (LLM fallback).**
356
+
357
+ Every NLP pattern must be maximally specific. Include all words that disambiguate the intent — articles, pronouns, structural words. If removing a word would cause false positives, that word is required.
358
+
359
+ ---
360
+
361
+ ## Configuration
362
+
363
+ ### .env
364
+
365
+ ```
366
+ WEATHER_API_KEY=your_openweathermap_key
367
+ STANZA_API_TOKEN=optional_security_token
368
+ ```
369
+
370
+ ### pyproject.toml Dependencies
371
+
372
+ ```toml
373
+ dependencies = [
374
+ "stanza>=1.5",
375
+ "requests>=2.28",
376
+ "python-dotenv>=1.0",
377
+ "flask>=3.0",
378
+ ]
379
+ ```
380
+
381
+ Optional (for `default_accurate` model):
382
+ ```
383
+ pip install transformers sentencepiece
384
+ pip install torch torchvision --index-url https://download.pytorch.org/whl/cu130
385
+ ```
386
+
387
+ ---
388
+
389
+ ## Requirements
390
+
391
+ - Python 3.9+
392
+ - Stanza 1.5+ (for NLP layer)
393
+ - Flask 3.0+ (for NLP service)
394
+ - A dedicated GPU is recommended but not required (RTX 3060+ for transformer models)
395
+ - The RegEx layer works with zero dependencies beyond the base package
396
+
397
+ ## Target Audience
398
+
399
+ Linguists and NLP researchers who understand constituency trees, dependency relations, and POS tags. You can run commands and follow instructions, but you should not have to debug import errors or port conflicts. PARSELY-DIP tells you what's wrong and how to fix it.
400
+
401
+ ## Status
402
+
403
+ v0.0.1 — Core engine built. RegEx pipeline working with time, weather, and scrum card intents. NLP layer ported from Uni with Stanza service (default_accurate with Electra Large transformer, GPU accelerated). Hook integration tested with Claude Code. CLI available via `parsely` command.
404
+
405
+ ## License
406
+
407
+ Source-available. Personal and development use permitted.
408
+
409
+ ## Author
410
+
411
+ George Butiri — [george@iseestudios.com](mailto:george@iseestudios.com)