live-vector-space 0.1.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,171 @@
1
+ Metadata-Version: 2.4
2
+ Name: live-vector-space
3
+ Version: 0.1.0
4
+ Summary: A whole codebase in a few KB, every byte one lookup away: a small map plus exact, numbered pieces for LLMs.
5
+ License-Expression: AGPL-3.0-or-later
6
+ Project-URL: Homepage, https://github.com/devkancheti4-design/live-vector-space
7
+ Keywords: llm,codebase map,context window,compression,mcp,code search
8
+ Classifier: Development Status :: 3 - Alpha
9
+ Classifier: Programming Language :: Python :: 3
10
+ Classifier: Programming Language :: Python :: 3.14
11
+ Classifier: Environment :: Console
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: Topic :: Software Development
14
+ Requires-Python: >=3.14
15
+ Description-Content-Type: text/markdown
16
+ License-File: LICENSE
17
+ Requires-Dist: numpy>=1.24
18
+ Dynamic: license-file
19
+
20
+ # lvs: a whole codebase in a few KB, every byte one lookup away
21
+
22
+ `lvs` gives an LLM a small map of a whole codebase (a few KB, however big the code) plus exact,
23
+ numbered pieces of the code to look up. The store keeps every byte, compressed about as small as zip,
24
+ and hands over any single piece on its own. So a model can know a codebase many times bigger than its
25
+ context window without reading it whole, and without ever decompressing anything itself.
26
+
27
+ Measured on two big Python codebases that no model had seen (66 blind runs, details in
28
+ [MILESTONE.md](https://github.com/devkancheti4-design/live-vector-space/blob/main/MILESTONE.md)):
29
+
30
+ | What was measured | Result |
31
+ |---|---|
32
+ | The map view alone (`map` plus a zoom into the main package, about 4.6 KB), 10 architecture questions per run | **100/100** right (raw code of the same size: 34/100) |
33
+ | Map + lookups, exact questions about codebases of 5-6 million tokens | **97.5%** right (468/480), for **2-4%** of the tokens a full read would take |
34
+ | A 6.2-million-token codebase, one session in a 1M-token window | **46/48** right in 111,501 tokens; **88.7%** of the code stayed compressed |
35
+ | Store size (reef, 14.5 MB of Python) | **30.7%** of the code (zip -9: 30.3%), any of 17,660 pieces readable alone, byte-exact |
36
+
37
+ ## Install
38
+
39
+ Needs Python 3.14+ (for the built-in `compression.zstd`) and numpy.
40
+
41
+ ```bash
42
+ pip install live-vector-space
43
+ ```
44
+
45
+ The command it installs is `lvs`. For the latest code from GitHub instead:
46
+ `pip install git+https://github.com/devkancheti4-design/live-vector-space`
47
+
48
+ ## Use
49
+
50
+ ```bash
51
+ cd your-project
52
+ lvs build # the map and the compressed store, in .lvs/
53
+ lvs map # the whole codebase in a few KB: read this first
54
+ lvs files src/server # zoom into a folder: its files, intents and piece numbers
55
+ lvs find timeout retry # the numbered pieces that mention each word, with the line that matches
56
+ lvs get 812 813 # the exact code of those pieces
57
+ lvs rebuild /tmp/copy # every file back, byte-exact
58
+ ```
59
+
60
+ Lookups work from any subfolder (`lvs` finds `.lvs/` in a parent folder, the way git finds `.git`).
61
+ Run `lvs build` again after the code changes. `--store PREFIX` keeps several maps apart. Folders that
62
+ are never the project's own code (hidden folders, `venv`, `site-packages`, `node_modules`, `build`,
63
+ `dist`) are skipped; `--exclude NAME ...` skips more.
64
+
65
+ ## Give it to an AI agent (MCP)
66
+
67
+ `lvs mcp` is a Model Context Protocol server over stdio, with five tools: `map`, `files`, `find`,
68
+ `get` (read-only lookups in the compressed store) and `build` (make or refresh the store). It tells
69
+ the agent to read the map first, then find and get only the pieces it needs.
70
+
71
+ With Claude Code, from your project folder:
72
+
73
+ ```bash
74
+ claude mcp add lvs -- lvs mcp --root "$PWD"
75
+ ```
76
+
77
+ Any other MCP client takes the same command in its config:
78
+
79
+ ```json
80
+ {"mcpServers": {"lvs": {"command": "lvs", "args": ["mcp", "--root", "/path/to/your-project"]}}}
81
+ ```
82
+
83
+ ## Watch code live
84
+
85
+ ```bash
86
+ lvs live path/to/file.py # every statement as a vector in 3-D space, redrawn on each save
87
+ ```
88
+
89
+ ## How it works
90
+
91
+ - **Numbered exact pieces.** Each file is split at its natural joints (functions, classes, sections)
92
+ into pieces numbered across the whole codebase. Joined in order, they rebuild every file byte for byte.
93
+ - **The store.** Every piece is compressed with zstd and a dictionary shared by all pieces, so the whole
94
+ store is about as small as zip, yet any one piece unpacks on its own in an instant.
95
+ - **The map.** Folders and files with sizes and one-line intents, plus the code's shape: every statement
96
+ becomes a point in a 3-D cube, and the points gather into cores of similar code, each with its words
97
+ and its pieces. The map stays a few KB at any size (2.1 KB for reef's 14.5 MB of code), and `files`
98
+ zooms in, page by page.
99
+ - **The word index.** `find` goes from words to piece numbers and shows the line that matched, so the
100
+ model reads only the pieces it needs.
101
+
102
+ ## Tests
103
+
104
+ ```bash
105
+ python3 -m unittest discover -s tests
106
+ ```
107
+
108
+ ## License
109
+
110
+ GNU Affero General Public License v3.0 or later (AGPL-3.0-or-later); see [LICENSE](https://github.com/devkancheti4-design/live-vector-space/blob/main/LICENSE).
111
+
112
+ # The research behind it
113
+
114
+ Watch a Python file live, as vectors in 3-D space, plus a set of experiments on
115
+ squeezing code into shapes and maps that an LLM can read cheaply. The scripts below are the
116
+ research versions; the `lvs` package in `src/lvs/` is the product built from them.
117
+
118
+ ![The square minus the code path](https://raw.githubusercontent.com/devkancheti4-design/live-vector-space/main/code_square.svg)
119
+
120
+ ## What's here
121
+
122
+ | File | What it does |
123
+ |---|---|
124
+ | `live_vector_space.py` | Turns every statement of a Python file into a 1024-d token vector, projects them to 3-D with PCA, and serves a live browser view: a cube filled with 14,000 particles of random "fluid" that settle into the code's shape. It updates every time the file is saved. |
125
+ | `skeleton.py` | Squeezes a file into 8 labeled cores of similar statements (a position in the cube plus a few words each). |
126
+ | `shape_codec.py` | Stores the 3-D points on a 12-bit grid in the cube, tied to anchor shapes, and decodes a perfect copy. |
127
+ | `optimal_shapes.py` | Lets the points choose their own shapes (blob, line, curve, sheet) by bit count (minimum description length). |
128
+ | `code_map.py` | A cheap map of a file plus exact, numbered pieces an LLM can look up; rebuilds the file byte-exact. |
129
+
130
+ Generated outputs: `code_square.svg`, `square_vs_scrambled.svg`, `shape_map.svg`,
131
+ `optimal_shapes.html`, `optimal_shapes_control.html` (open locally in a browser),
132
+ `code_map.json` and `code_store.bin`.
133
+
134
+ ## Run it
135
+
136
+ Needs Python 3 with numpy; `code_map.py` needs Python 3.14+ for the built-in `compression.zstd`.
137
+
138
+ ```bash
139
+ python3 live_vector_space.py # live 3-D view of itself at http://localhost:8765
140
+ python3 live_vector_space.py path/to/other.py # any other Python file
141
+ python3 live_vector_space.py --square square.svg # save the square-minus-the-path picture
142
+ python3 skeleton.py --stats
143
+ python3 shape_codec.py --svg shape_map.svg
144
+ python3 optimal_shapes.py --html optimal_shapes.html
145
+ python3 code_map.py build
146
+ python3 code_map.py find fluid svg
147
+ python3 code_map.py get 50
148
+ ```
149
+
150
+ Viewer controls: drag to rotate, scroll to zoom, hover or click a point. Keys: `H` shake the fluid,
151
+ `F` fluid on/off, `V` vectors, `P` code path, `L` labels, `T` table, `S` square, `Space` spin.
152
+
153
+ ## What the experiments found
154
+
155
+ Measured on `live_vector_space.py` itself (52,552 bytes, 357 statements).
156
+
157
+ - **The shape alone doesn't carry meaning.** Swapping all 964 names and words for gibberish leaves the
158
+ fingerprint identical (exactly, with unhashed word counts). Fresh models given only the shape's numbers
159
+ scored 7/14 on a quiz about the program, the same as answering "no" to everything.
160
+ - **Words carry the gist cheaply.** The file's one-line intent plus the skeleton (about 750 bytes) scored
161
+ 12/14 on what the program does, but 0/4 on details.
162
+ - **A map plus numbered lookups is the sweet spot.** With the map and exact lookups by number
163
+ (`code_map.py`), a fresh model scored 14/14 and 4/4, the same as reading the whole file, using about
164
+ 9,200 extra tokens instead of about 30,400, and reading 8 of the 80 pieces.
165
+ - **Compression.** The cube grid stores the 357 points in 1,609 bytes; shapes the points chose themselves
166
+ (8 blobs) take 1,471. The cloud is a nearly round ball, so lines and sheets don't help here (on control
167
+ data that really forms lines and a sheet they save 32%). An exact copy of the code needs about 16 KB
168
+ (brotli); numbered pieces with a shared zstd dictionary take 24 KB, the price of reading any piece on
169
+ its own.
170
+
171
+ Each LLM test was a single run on one file, so small score differences may be chance.
@@ -0,0 +1,13 @@
1
+ live_vector_space-0.1.0.dist-info/licenses/LICENSE,sha256=jVa0BUaKrRH4erV2P5AeJ24I2WRv9chIGxditreJ6e0,34524
2
+ lvs/__init__.py,sha256=CLykR8n8YFE55lDDfaT_5lJ0qtlJtQWdFfNdsCRmXl4,411
3
+ lvs/__main__.py,sha256=E6Gls0DNz8GQK2K-kOUIx8cYhgANW_CH54VKrfCfs14,52
4
+ lvs/cli.py,sha256=W0CchnUJj0p5H_BpnPBXm9XjbUNwBxKfiltK6WgvCgs,606
5
+ lvs/codemap.py,sha256=O9qCInQUYUorX5eO3ZH35tafwoeXeZSBs6ym3F-s-U8,29307
6
+ lvs/live.py,sha256=0L79jKtNblBYlbgXyUV4GneTkuWuphpHT1Wh4s_8jIs,52497
7
+ lvs/mcp_server.py,sha256=AbVSmEmO2LiMj9mpRjW2EjHAj4kvVWfPebiULv9hArY,11268
8
+ lvs/skeleton.py,sha256=NnIFtQp9DWfkHP1tgEQJJsAY1OVoDiF8nQerL2IqL2Q,6596
9
+ live_vector_space-0.1.0.dist-info/METADATA,sha256=3-LVsTxDMj0icDDXtM3pRyt6jsckSXJJCbfg9XUfeF0,8610
10
+ live_vector_space-0.1.0.dist-info/WHEEL,sha256=mOdXR9C3wMVov7cSY8PY-XCzo3SmmWCHvGQr5csJoao,91
11
+ live_vector_space-0.1.0.dist-info/entry_points.txt,sha256=vT_86H3DkCY3pca5jHqsBY4LI6hiaMMSC3-qW-HO_Y4,37
12
+ live_vector_space-0.1.0.dist-info/top_level.txt,sha256=K_kQa17eb9jRWXKIqY2S_8TPrD9ubHp1gDq7OFvIhiY,4
13
+ live_vector_space-0.1.0.dist-info/RECORD,,
@@ -0,0 +1,5 @@
1
+ Wheel-Version: 1.0
2
+ Generator: setuptools (84.2.1)
3
+ Root-Is-Purelib: true
4
+ Tag: py3-none-any
5
+
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ lvs = lvs.cli:main