1bcoder 0.1.9__tar.gz → 0.1.10__tar.gz

{1bcoder-0.1.9 → 1bcoder-0.1.10}/1bcoder.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: 1bcoder
-Version: 0.1.9
+Version: 0.1.10
 Summary: AI coding assistant agent for 1B–7B local models (Ollama, LMStudio, llama.cpp). Terminal REPL with file editing, project map, agents, scripts, and parallel multi-model queries.
 Project-URL: Homepage, https://github.com/szholobetsky/1bcoder
 Project-URL: Repository, https://github.com/szholobetsky/1bcoder
@@ -59,7 +59,7 @@ Key design decisions:
 - **Short agent system prompts, at most 5 tools per agent, one function per agent** — `ask`, `edit`, `fill`, `scan`, `compact`. Not universal agents with bloated skill sets.
 - **Tolerant of long and malformed output** — post-processing is automatic; the programmer does not teach the model JSON syntax.
 - **`/parallel`** — send the same context to several models simultaneously and combine results; a 0.5b and a 1b model working together often outperform either alone; designed to coordinate small models running on multiple machines or phones.
-- **`/map`** — project structure index with structural diff; lets the model navigate a codebase without loading it into context.
+- **`/map`** — project structure index with structural diff and keyword grounding; lets the model navigate a codebase without loading it into context. Keyword extract supports three modes: exact, like (`%token%`), and fuzzy (subword); `-o` resolves each keyword to the files it appears in.
 - **`/ctx`** — surgical context management: savepoints, selective compaction, named context library, multi-turn rollback. Small models cannot afford wasted tokens.
 - **`/scan`** — reads any large file chunk by chunk and builds a themed summary without overflowing context.
 - **`/proc`** — parameterized command scripts for repeatable preparation workflows.
@@ -538,8 +538,8 @@ The map command scans your project with language-agnostic regex, extracts defini
 /map trace <start> <end> [-y]                  — shortest dependency path between two points
 /map idiff [path] [depth]                      — re-index then show diff vs previous snapshot
 /map diff                                      — show diff without re-indexing (safe to repeat)
-/map keyword index                             — build keyword vocabulary from map.txt
-/map keyword extract <text> [-f] [-a] [-n] [-c] — extract real identifiers from keyword.txt matching text/file
+/map keyword index                                  — build keyword vocabulary from map.txt
+/map keyword extract <text> [-f] [-l] [-a] [-s] [-n] [-c] [-o] — extract real identifiers from keyword.txt matching text/file
 ```
 
 **Partial / incremental indexing** — for large codebases where a full re-scan is slow:
@@ -583,6 +583,33 @@ This lets you re-index a changed module in seconds instead of hours.
 /map find models -d 2                    — filenames + defines/vars only
 ```
 
+**`/map keyword`** — two-step identifier grounding:
+
+```
+/map keyword index                   — scan map.txt → .1bcoder/keyword.txt (word, count, line refs)
+/map keyword extract "fix rule search" -l -o   — find identifiers, show which files they're in
+```
+
+Extract flags:
+
+| Flag | Mode | Effect |
+|---|---|---|
+| *(none)* | exact | query word must exactly match a keyword |
+| `-f` | fuzzy | splits camelCase/snake_case into subwords; matches if all query subwords (≥5 chars) appear in the identifier |
+| `-l` | like | substring match: any keyword containing the token (`%token%`) |
+| `-n` | — | show frequency count: `RuleIndex(25)` |
+| `-s` | — | sort by frequency descending |
+| `-a` | — | sort alphabetically |
+| `-c` | — | comma-separated output (default: one per line) |
+| `-o` | — | show origin files: `parameter_rule -> lib/cop/rule.rb, config/default.yml` |
+
+```
+/map keyword extract "add validation rule" -l -o
+# parameter_rule -> lib/rubocop/cop/style/rule.rb, config/default.yml
+# cop_rule       -> lib/rubocop/cop/base.rb
+# load_rules     -> lib/rubocop/config_loader.rb
+```
+
 **`/map trace`** — three modes:
 
 **1. Backwards BFS** (who depends on this?):

{1bcoder-0.1.9 → 1bcoder-0.1.10}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: 1bcoder
-Version: 0.1.9
+Version: 0.1.10
 Summary: AI coding assistant agent for 1B–7B local models (Ollama, LMStudio, llama.cpp). Terminal REPL with file editing, project map, agents, scripts, and parallel multi-model queries.
 Project-URL: Homepage, https://github.com/szholobetsky/1bcoder
 Project-URL: Repository, https://github.com/szholobetsky/1bcoder
@@ -59,7 +59,7 @@ Key design decisions:
 - **Short agent system prompts, at most 5 tools per agent, one function per agent** — `ask`, `edit`, `fill`, `scan`, `compact`. Not universal agents with bloated skill sets.
 - **Tolerant of long and malformed output** — post-processing is automatic; the programmer does not teach the model JSON syntax.
 - **`/parallel`** — send the same context to several models simultaneously and combine results; a 0.5b and a 1b model working together often outperform either alone; designed to coordinate small models running on multiple machines or phones.
-- **`/map`** — project structure index with structural diff; lets the model navigate a codebase without loading it into context.
+- **`/map`** — project structure index with structural diff and keyword grounding; lets the model navigate a codebase without loading it into context. Keyword extract supports three modes: exact, like (`%token%`), and fuzzy (subword); `-o` resolves each keyword to the files it appears in.
 - **`/ctx`** — surgical context management: savepoints, selective compaction, named context library, multi-turn rollback. Small models cannot afford wasted tokens.
 - **`/scan`** — reads any large file chunk by chunk and builds a themed summary without overflowing context.
 - **`/proc`** — parameterized command scripts for repeatable preparation workflows.
@@ -538,8 +538,8 @@ The map command scans your project with language-agnostic regex, extracts defini
 /map trace <start> <end> [-y]                  — shortest dependency path between two points
 /map idiff [path] [depth]                      — re-index then show diff vs previous snapshot
 /map diff                                      — show diff without re-indexing (safe to repeat)
-/map keyword index                             — build keyword vocabulary from map.txt
-/map keyword extract <text> [-f] [-a] [-n] [-c] — extract real identifiers from keyword.txt matching text/file
+/map keyword index                                  — build keyword vocabulary from map.txt
+/map keyword extract <text> [-f] [-l] [-a] [-s] [-n] [-c] [-o] — extract real identifiers from keyword.txt matching text/file
 ```
 
 **Partial / incremental indexing** — for large codebases where a full re-scan is slow:
@@ -583,6 +583,33 @@ This lets you re-index a changed module in seconds instead of hours.
 /map find models -d 2                    — filenames + defines/vars only
 ```
 
+**`/map keyword`** — two-step identifier grounding:
+
+```
+/map keyword index                   — scan map.txt → .1bcoder/keyword.txt (word, count, line refs)
+/map keyword extract "fix rule search" -l -o   — find identifiers, show which files they're in
+```
+
+Extract flags:
+
+| Flag | Mode | Effect |
+|---|---|---|
+| *(none)* | exact | query word must exactly match a keyword |
+| `-f` | fuzzy | splits camelCase/snake_case into subwords; matches if all query subwords (≥5 chars) appear in the identifier |
+| `-l` | like | substring match: any keyword containing the token (`%token%`) |
+| `-n` | — | show frequency count: `RuleIndex(25)` |
+| `-s` | — | sort by frequency descending |
+| `-a` | — | sort alphabetically |
+| `-c` | — | comma-separated output (default: one per line) |
+| `-o` | — | show origin files: `parameter_rule -> lib/cop/rule.rb, config/default.yml` |
+
+```
+/map keyword extract "add validation rule" -l -o
+# parameter_rule -> lib/rubocop/cop/style/rule.rb, config/default.yml
+# cop_rule       -> lib/rubocop/cop/base.rb
+# load_rules     -> lib/rubocop/config_loader.rb
+```
+
 **`/map trace`** — three modes:
 
 **1. Backwards BFS** (who depends on this?):

{1bcoder-0.1.9 → 1bcoder-0.1.10}/README.md

@@ -44,7 +44,7 @@ Key design decisions:
 - **Short agent system prompts, at most 5 tools per agent, one function per agent** — `ask`, `edit`, `fill`, `scan`, `compact`. Not universal agents with bloated skill sets.
 - **Tolerant of long and malformed output** — post-processing is automatic; the programmer does not teach the model JSON syntax.
 - **`/parallel`** — send the same context to several models simultaneously and combine results; a 0.5b and a 1b model working together often outperform either alone; designed to coordinate small models running on multiple machines or phones.
-- **`/map`** — project structure index with structural diff; lets the model navigate a codebase without loading it into context.
+- **`/map`** — project structure index with structural diff and keyword grounding; lets the model navigate a codebase without loading it into context. Keyword extract supports three modes: exact, like (`%token%`), and fuzzy (subword); `-o` resolves each keyword to the files it appears in.
 - **`/ctx`** — surgical context management: savepoints, selective compaction, named context library, multi-turn rollback. Small models cannot afford wasted tokens.
 - **`/scan`** — reads any large file chunk by chunk and builds a themed summary without overflowing context.
 - **`/proc`** — parameterized command scripts for repeatable preparation workflows.
@@ -523,8 +523,8 @@ The map command scans your project with language-agnostic regex, extracts defini
 /map trace <start> <end> [-y]                  — shortest dependency path between two points
 /map idiff [path] [depth]                      — re-index then show diff vs previous snapshot
 /map diff                                      — show diff without re-indexing (safe to repeat)
-/map keyword index                             — build keyword vocabulary from map.txt
-/map keyword extract <text> [-f] [-a] [-n] [-c] — extract real identifiers from keyword.txt matching text/file
+/map keyword index                                  — build keyword vocabulary from map.txt
+/map keyword extract <text> [-f] [-l] [-a] [-s] [-n] [-c] [-o] — extract real identifiers from keyword.txt matching text/file
 ```
 
 **Partial / incremental indexing** — for large codebases where a full re-scan is slow:
@@ -568,6 +568,33 @@ This lets you re-index a changed module in seconds instead of hours.
 /map find models -d 2                    — filenames + defines/vars only
 ```
 
+**`/map keyword`** — two-step identifier grounding:
+
+```
+/map keyword index                   — scan map.txt → .1bcoder/keyword.txt (word, count, line refs)
+/map keyword extract "fix rule search" -l -o   — find identifiers, show which files they're in
+```
+
+Extract flags:
+
+| Flag | Mode | Effect |
+|---|---|---|
+| *(none)* | exact | query word must exactly match a keyword |
+| `-f` | fuzzy | splits camelCase/snake_case into subwords; matches if all query subwords (≥5 chars) appear in the identifier |
+| `-l` | like | substring match: any keyword containing the token (`%token%`) |
+| `-n` | — | show frequency count: `RuleIndex(25)` |
+| `-s` | — | sort by frequency descending |
+| `-a` | — | sort alphabetically |
+| `-c` | — | comma-separated output (default: one per line) |
+| `-o` | — | show origin files: `parameter_rule -> lib/cop/rule.rb, config/default.yml` |
+
+```
+/map keyword extract "add validation rule" -l -o
+# parameter_rule -> lib/rubocop/cop/style/rule.rb, config/default.yml
+# cop_rule       -> lib/rubocop/cop/base.rb
+# load_rules     -> lib/rubocop/config_loader.rb
+```
+
 **`/map trace`** — three modes:
 
 **1. Backwards BFS** (who depends on this?):

{1bcoder-0.1.9 → 1bcoder-0.1.10}/chat.py

@@ -916,7 +916,7 @@ Output capture operators (work with any command — LLM reply, tool, proc):
     CSV format: word, count, semicolon-separated list of line numbers in map.txt.
     Sorted alphabetically. Run once after /map index (or whenever map changes).
     e.g.  /map keyword index
-/map keyword extract <text or file> [-a] [-f] [-n] [-c]
+/map keyword extract <text or file> [-a] [-s] [-f] [-l] [-n] [-c] [-o]
     Extract real identifiers from keyword.txt matching words in the given text or file.
     Output is always real identifiers from keyword.txt — never synthetic splits.
     Default (exact): query word must exactly match a keyword.txt entry.
@@ -928,16 +928,20 @@ Output capture operators (work with any command — LLM reply, tool, proc):
         "RuleIndex" → matches RuleIndex only (needs both 'rule' AND 'index')
         "coverage"  → matches CoverageMetric, LineCoverage, BranchCoverage
         "RuleIndex" → does NOT match Rule (missing 'index') or Index (missing 'rule')
+    -l  like match (%token%): any keyword containing the query token as substring
+        "rule" → RuleIndex, rule_validator, parseRule, getRuleSet
     -a  alphabetical order
     -s  sort by codebase count descending (most frequent first)
     -n  show codebase count next to each word: RuleIndex(25) RuleName(12)
         (-n implies -s)
     -c  comma-separated output instead of one per line
+    -o  show origin: which files each keyword appears in: keyword -> file1, file2
     e.g.  /map keyword extract notes.txt
           /map keyword extract notes.txt -f
+          /map keyword extract notes.txt -l
           /map keyword extract notes.txt -f -n -c
           /map keyword extract "add isbn field to the Book class" -f -a
-          /map keyword extract "fix rule search" -f -c
+          /map keyword extract "fix rule search" -l -o
 
 /hook before|after <cmd> <script>
     Run a script before or after a command (edit, patch, fix, insert).
@@ -1175,8 +1179,11 @@ def _fmt_ctx(n: int) -> str:
 
 
 def read_file(path, start=None, end=None, line_numbers=True):
-    with open(path, "r", encoding="utf-8") as f:
-        lines = f.readlines()
+    try:
+        with open(path, "r", encoding="utf-8") as f:
+            lines = f.readlines()
+    except UnicodeDecodeError:
+        raise ValueError(f"[read] {path}: binary file, cannot read as text")
     total = len(lines)
     if start is not None:
         start = max(1, start)
@@ -3828,7 +3835,7 @@ advanced_tools =
                     _ok(f"context: injected {label}")
             except FileNotFoundError:
                 print(f"file not found: {path}")
-            except OSError as e:
+            except (OSError, ValueError) as e:
                 _err(e)
 
     def _cmd_edit(self, user_input: str):
@@ -7731,11 +7738,13 @@ Config stored in ~/.1bcoder/translate.json
         sort_alpha  = "-a" in args
         sort_count  = "-s" in args
         fuzzy       = "-f" in args
+        like        = "-l" in args
         show_counts = "-n" in args
         csv_out     = "-c" in args
-        src_tokens  = [a for a in args if a not in ("-a", "-s", "-f", "-n", "-c")]
+        show_origin = "-o" in args
+        src_tokens  = [a for a in args if a not in ("-a", "-s", "-f", "-l", "-n", "-c", "-o")]
         if not src_tokens:
-            print("usage: /map keyword extract <text or file> [-a] [-s] [-f] [-n] [-c]")
+            print("usage: /map keyword extract <text or file> [-a] [-s] [-f] [-l] [-n] [-c] [-o]")
             return
         # load keyword vocab: word → count
         _csv.field_size_limit(10_000_000)  # lines field can be large for common words
@@ -7777,6 +7786,12 @@ Config stored in ~/.1bcoder/translate.json
                     # keyword matches if ALL query subwords are present in keyword's subwords
                     if query_parts <= kp and kw not in seen:
                         seen[kw] = i
+        elif like:
+            for i, m in enumerate(token_re.finditer(text)):
+                token = m.group().lower()
+                for j, kw in enumerate(kw_freq):
+                    if token in kw.lower() and kw not in seen:
+                        seen[kw] = i * 100000 + j
         else:
             # default: exact identifier match
             kw_set = set(kw_freq)
@@ -7793,6 +7808,30 @@ Config stored in ~/.1bcoder/translate.json
             result = sorted(seen, key=lambda w: (-kw_freq[w], w.lower()))
         else:
             result = sorted(seen, key=lambda w: (seen[w], w.lower()))
+        if show_origin:
+            map_path = os.path.join(BCODER_DIR, "map.txt")
+            line_to_file = map_query._build_line_to_file(map_path)
+            # load line numbers column from keyword.txt
+            kw_lines: dict = {}
+            with open(kw_path, encoding="utf-8", newline="") as f:
+                reader2 = _csv.reader(f)
+                next(reader2, None)
+                for row in reader2:
+                    if len(row) >= 3 and row[0] in seen:
+                        kw_lines[row[0]] = [int(x) for x in row[2].split(";") if x]
+            lines_out = []
+            for w in result:
+                label = f"{w}({kw_freq[w]})" if show_counts else w
+                files: list = []
+                seen_files: set = set()
+                for ln in kw_lines.get(w, []):
+                    f = line_to_file.get(ln)
+                    if f and f not in seen_files:
+                        seen_files.add(f)
+                        files.append(f)
+                lines_out.append(f"{label} -> {', '.join(files)}" if files else label)
+            print("\n".join(lines_out))
+            return
         if show_counts:
             items = [f"{w}({kw_freq[w]})" for w in result]
         else:

{1bcoder-0.1.9 → 1bcoder-0.1.10}/map_index.py

@@ -292,15 +292,18 @@ def build_map(root: str, depth: int = 2, map_path: str = None) -> str:
                 global_index[name] = rel
 
     # ── link phase (changed files only) ──────────────────────────────────────────
+    global_set = set(global_index.keys())
     file_links = {}   # rel → { target_rel → { name → kind } }
     for rel, text in tqdm(file_content.items(), desc="linking", unit="file", file=sys.stderr):
+        tokens_in_file = set(_WORD_RE.findall(text))
+        candidates = tokens_in_file & global_set
         by_target = {}
-        for name, target_rel in global_index.items():
+        for name in candidates:
+            target_rel = global_index[name]
             if target_rel == rel:
                 continue
-            if re.search(r'\b' + re.escape(name) + r'\b', text):
-                kind = classify_ref(name, text)
-                by_target.setdefault(target_rel, {})[name] = kind
+            kind = classify_ref(name, text)
+            by_target.setdefault(target_rel, {})[name] = kind
         file_links[rel] = by_target
 
     # ── format output ────────────────────────────────────────────────────────────

{1bcoder-0.1.9 → 1bcoder-0.1.10}/map_query.py

@@ -538,6 +538,74 @@ def idiff_report(map_prev: str, map_curr: str) -> str:
     return '\n'.join(lines)
 
 
+# ── keywords ────────────────────────────────────────────────────────────────────
+
+def _split_identifier(name: str) -> list:
+    """Split camelCase / snake_case / PascalCase into lowercase subwords."""
+    parts = re.split(r'[_\-]+', name)
+    result = []
+    for part in parts:
+        if not part:
+            continue
+        s = re.sub(r'([A-Z]+)([A-Z][a-z])', r'\1_\2', part)
+        s = re.sub(r'([a-z\d])([A-Z])', r'\1_\2', s)
+        result.extend(w.lower() for w in s.split('_') if len(w) >= 2)
+    seen: dict = {}
+    for w in result:
+        seen.setdefault(w, None)
+    return list(seen)
+
+
+def _build_line_to_file(map_path: str) -> dict:
+    """Return {line_number: file_path} for every line in map.txt.
+
+    File ownership: nearest non-indented, non-comment line above.
+    """
+    with open(map_path, encoding='utf-8', errors='replace') as f:
+        lines = f.readlines()
+    result = {}
+    current_file = None
+    for i, line in enumerate(lines, 1):
+        stripped = line.rstrip('\n')
+        if stripped and not stripped[0].isspace() and not stripped.startswith('#'):
+            current_file = stripped.strip()
+        if current_file:
+            result[i] = current_file
+    return result
+
+
+def keyword_to_files(map_path: str, word: str) -> list:
+    """Return list of files where *word* appears, in order of first occurrence."""
+    import csv as _csv
+
+    kw_path = os.path.join(os.path.dirname(os.path.abspath(map_path)), 'keyword.txt')
+    if not os.path.exists(kw_path):
+        return []
+
+    line_nums = []
+    _csv.field_size_limit(10_000_000)
+    with open(kw_path, encoding='utf-8', newline='') as f:
+        reader = _csv.reader(f)
+        next(reader, None)
+        for row in reader:
+            if len(row) >= 3 and row[0] == word:
+                line_nums = [int(x) for x in row[2].split(';') if x]
+                break
+
+    if not line_nums:
+        return []
+
+    line_to_file = _build_line_to_file(map_path)
+    seen: set = set()
+    files = []
+    for ln in line_nums:
+        f = line_to_file.get(ln)
+        if f and f not in seen:
+            seen.add(f)
+            files.append(f)
+    return files
+
+
 # ── CLI entry point ─────────────────────────────────────────────────────────────
 
 def main():

{1bcoder-0.1.9 → 1bcoder-0.1.10}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "1bcoder"
-version = "0.1.9"
+version = "0.1.10"
 description = "AI coding assistant agent for 1B–7B local models (Ollama, LMStudio, llama.cpp). Terminal REPL with file editing, project map, agents, scripts, and parallel multi-model queries."
 requires-python = ">=3.10"
 readme = {file = "README.md", content-type = "text/markdown"}