appbinary-inspector 0.2.0__tar.gz → 0.6.0__tar.gz

appbinary_inspector-0.6.0/PKG-INFO

@@ -0,0 +1,120 @@
+Metadata-Version: 2.5
+Name: appbinary-inspector
+Version: 0.6.0
+Summary: Open-source lazy-card character model plus static binary inspection tools.
+Author: William Laurento
+License: MIT
+License-File: LICENSE
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.10
+Provides-Extra: torch
+Requires-Dist: torch>=2.1; extra == 'torch'
+Description-Content-Type: text/markdown
+
+# appbinary-inspector
+
+Static inspection tools plus an open-source, disk-paged character-model architecture.
+
+## Features
+
+- Detects PE, ELF, Mach-O, ZIP, 7z, and RAR headers
+- Calculates SHA-256 and SHA-1 hashes
+- Extracts readable strings and entropy
+- Reads basic PE header metadata
+- Extracts embedded ZIP archives and carves recognizable binary signatures
+- Exports PE headers and raw binary sections for authorized analysis
+- Builds a compact tokenizer for source code, logs, or extracted text
+- Provides editable PyTorch classifier, autoencoder, and transformer templates
+- Keeps CardStack model cards as separately stored 4-bit SSD files
+- Loads only router-selected cards into a small LRU cache during generation
+- Generates one next byte at a time for true character-level output
+
+## Install
+
+```bash
+pip install appbinary-inspector
+```
+
+## Use
+
+```bash
+appbin inspect program.exe
+appbin extract program.exe --output extracted
+appbin binary-extract program.exe --output pe_parts
+appbin tokenize source.py --vocab source-tokenizer.json
+appbin model-template classifier
+```
+
+## CardStack: low-VRAM model experiments
+
+CardStack is a small trainable byte-level language model. Its expert cards are
+stored as separate 4-bit files. At inference, inactive cards remain on disk and
+the cache retains only a limited number of recently selected cards.
+
+```python
+from appbinary_inspector import CardStackConfig, CardStackLM, train_next_character
+
+model = CardStackLM(CardStackConfig(width=128, layers=4, cards_per_layer=8, active_cards=2, cache_cards=4))
+losses = train_next_character(model, open("training.txt", encoding="utf-8").read(), steps=500)
+model.save("my-card-model")  # writes a base file plus separate 4-bit card files
+
+paged = CardStackLM.load("my-card-model")
+print(paged.generate("Hello", new_characters=80))
+```
+
+CLI version:
+
+```bash
+appbin card-create --output my-card-model --width 128 --layers 4 --cards 8 --active-cards 2
+appbin card-train my-card-model training.txt --steps 500
+appbin card-generate my-card-model "Hello" --characters 80
+```
+
+## Text chat interface
+
+Open a local chat box after training a model. It binds only to your computer,
+opens in your browser, and never sends prompts to a remote service.
+
+```bash
+appbin chat my-card-model
+```
+
+## Python import API
+
+Use the model from normal Python code without a terminal command interface:
+
+```python
+from appbinary_inspector import appbincardcr
+
+mycard = appbincardcr("my-card-model")
+reply = mycard("Explain gravity", output=True)
+
+# Or call it in one line:
+reply = appbincardcr(mycard, "Explain gravity", output=True)
+```
+
+This reduces active model-card memory. It does not make a dense, unmodified
+70B model fit instantly on a low-VRAM GPU: disk bandwidth and CPU/GPU transfer
+time become the tradeoff, and models need to be trained for sparse routing.
+
+## PyTorch models
+
+Install optional PyTorch support:
+
+```bash
+pip install "appbinary-inspector[torch]"
+```
+
+```python
+from appbinary_inspector.torch_builder import create_model, preset
+
+config = preset("classifier")
+config["layers"][0]["out_features"] = 512
+model = create_model(config)
+```
+
+## Safety
+
+Targets are read as bytes only and never executed. The package does not defeat encryption, DRM, licensing controls, passwords, or code protections. It can analyze plaintext, export raw PE sections, and extract recognizable embedded content from files you are authorized to inspect.

appbinary_inspector-0.6.0/README.md

@@ -0,0 +1,105 @@
+# appbinary-inspector
+
+Static inspection tools plus an open-source, disk-paged character-model architecture.
+
+## Features
+
+- Detects PE, ELF, Mach-O, ZIP, 7z, and RAR headers
+- Calculates SHA-256 and SHA-1 hashes
+- Extracts readable strings and entropy
+- Reads basic PE header metadata
+- Extracts embedded ZIP archives and carves recognizable binary signatures
+- Exports PE headers and raw binary sections for authorized analysis
+- Builds a compact tokenizer for source code, logs, or extracted text
+- Provides editable PyTorch classifier, autoencoder, and transformer templates
+- Keeps CardStack model cards as separately stored 4-bit SSD files
+- Loads only router-selected cards into a small LRU cache during generation
+- Generates one next byte at a time for true character-level output
+
+## Install
+
+```bash
+pip install appbinary-inspector
+```
+
+## Use
+
+```bash
+appbin inspect program.exe
+appbin extract program.exe --output extracted
+appbin binary-extract program.exe --output pe_parts
+appbin tokenize source.py --vocab source-tokenizer.json
+appbin model-template classifier
+```
+
+## CardStack: low-VRAM model experiments
+
+CardStack is a small trainable byte-level language model. Its expert cards are
+stored as separate 4-bit files. At inference, inactive cards remain on disk and
+the cache retains only a limited number of recently selected cards.
+
+```python
+from appbinary_inspector import CardStackConfig, CardStackLM, train_next_character
+
+model = CardStackLM(CardStackConfig(width=128, layers=4, cards_per_layer=8, active_cards=2, cache_cards=4))
+losses = train_next_character(model, open("training.txt", encoding="utf-8").read(), steps=500)
+model.save("my-card-model")  # writes a base file plus separate 4-bit card files
+
+paged = CardStackLM.load("my-card-model")
+print(paged.generate("Hello", new_characters=80))
+```
+
+CLI version:
+
+```bash
+appbin card-create --output my-card-model --width 128 --layers 4 --cards 8 --active-cards 2
+appbin card-train my-card-model training.txt --steps 500
+appbin card-generate my-card-model "Hello" --characters 80
+```
+
+## Text chat interface
+
+Open a local chat box after training a model. It binds only to your computer,
+opens in your browser, and never sends prompts to a remote service.
+
+```bash
+appbin chat my-card-model
+```
+
+## Python import API
+
+Use the model from normal Python code without a terminal command interface:
+
+```python
+from appbinary_inspector import appbincardcr
+
+mycard = appbincardcr("my-card-model")
+reply = mycard("Explain gravity", output=True)
+
+# Or call it in one line:
+reply = appbincardcr(mycard, "Explain gravity", output=True)
+```
+
+This reduces active model-card memory. It does not make a dense, unmodified
+70B model fit instantly on a low-VRAM GPU: disk bandwidth and CPU/GPU transfer
+time become the tradeoff, and models need to be trained for sparse routing.
+
+## PyTorch models
+
+Install optional PyTorch support:
+
+```bash
+pip install "appbinary-inspector[torch]"
+```
+
+```python
+from appbinary_inspector.torch_builder import create_model, preset
+
+config = preset("classifier")
+config["layers"][0]["out_features"] = 512
+model = create_model(config)
+```
+
+## Safety
+
+Targets are read as bytes only and never executed. The package does not defeat encryption, DRM, licensing controls, passwords, or code protections. It can analyze plaintext, export raw PE sections, and extract recognizable embedded content from files you are authorized to inspect.

{appbinary_inspector-0.2.0 → appbinary_inspector-0.6.0}/pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "hatchling.build"
 
 [project]
 name = "appbinary-inspector"
-version = "0.2.0"
-description = "Safe static inspection and embedded-file extraction for application binaries."
+version = "0.6.0"
+description = "Open-source lazy-card character model plus static binary inspection tools."
 readme = "README.md"
 requires-python = ">=3.10"
 license = {text = "MIT"}

{appbinary_inspector-0.2.0 → appbinary_inspector-0.6.0}/src/appbinary_inspector/__init__.py

@@ -3,6 +3,8 @@
 from .analyzer import inspect_file
 from .extract import extract_binary_parts, extract_embedded
 from .tokenizer import WordTokenizer
+from .cardstack import CardStackConfig, CardStackLM, train_next_character
+from .card_api import Card, appbincardcr
 
-__all__ = ["inspect_file", "extract_embedded", "extract_binary_parts", "WordTokenizer"]
-__version__ = "0.2.0"
+__all__ = ["inspect_file", "extract_embedded", "extract_binary_parts", "WordTokenizer", "CardStackConfig", "CardStackLM", "train_next_character", "Card", "appbincardcr"]
+__version__ = "0.6.0"

appbinary_inspector-0.6.0/src/appbinary_inspector/card_api.py

@@ -0,0 +1,37 @@
+"""Small Python API for using a disk-paged CardStack model in normal code."""
+from __future__ import annotations
+
+from pathlib import Path
+
+from .cardstack import CardStackLM
+
+
+class Card:
+    """A callable local text model.
+
+    Example:
+        mycard = appbincardcr("my-card-model")
+        reply = mycard("Explain gravity", output=True)
+    """
+
+    def __init__(self, model: str | Path | CardStackLM) -> None:
+        self.model = CardStackLM.load(model) if isinstance(model, (str, Path)) else model
+
+    def __call__(self, text: str, output: bool = False, characters: int = 180, temperature: float = 0.8) -> str:
+        reply = self.model.generate(text, new_characters=characters, temperature=temperature)
+        if output:
+            print(reply)
+        return reply
+
+    def reply(self, text: str, **options) -> str:
+        return self(text, **options)
+
+
+def appbincardcr(mycard: str | Path | CardStackLM | Card, text: str | None = None, output: bool = False, **options):
+    """Create or call a CardStack text model with one short function.
+
+    ``appbincardcr("my-card-model")`` returns a reusable card.
+    ``appbincardcr(mycard, "Hello", output=True)`` generates and prints text.
+    """
+    card = mycard if isinstance(mycard, Card) else Card(mycard)
+    return card if text is None else card(text, output=output, **options)

appbinary_inspector-0.6.0/src/appbinary_inspector/cardstack.py

@@ -0,0 +1,246 @@
+"""A compact, character-by-character language model with on-demand expert cards.
+
+This is a research-sized model architecture, not a pre-trained replacement for
+large commercial models. Each layer routes work to only ``active_cards`` expert
+cards. Serialized cards live in separate files and are loaded into an LRU cache
+only when the router selects them.
+"""
+from __future__ import annotations
+
+import json
+import math
+from collections import OrderedDict
+from dataclasses import asdict, dataclass
+from pathlib import Path
+
+
+def _torch():
+    try:
+        import torch
+        import torch.nn as nn
+        import torch.nn.functional as functional
+        return torch, nn, functional
+    except ImportError as error:
+        raise RuntimeError("Install PyTorch first: pip install appbinary-inspector[torch]") from error
+
+
+@dataclass
+class CardStackConfig:
+    width: int = 128
+    layers: int = 4
+    cards_per_layer: int = 8
+    active_cards: int = 2
+    context_size: int = 128
+    cache_cards: int = 4
+    card_bits: int = 4
+
+
+def _pack_int4(values):
+    """Pack signed 4-bit values two per byte for SSD storage."""
+    torch, _, _ = _torch()
+    unsigned = (values.to(torch.int16) + 8).to(torch.uint8).flatten()
+    if unsigned.numel() % 2:
+        unsigned = torch.cat([unsigned, torch.zeros(1, dtype=torch.uint8)])
+    return unsigned[0::2] | (unsigned[1::2] << 4)
+
+
+def _unpack_int4(packed, count: int):
+    torch, _, _ = _torch()
+    low = packed & 0x0F
+    high = packed >> 4
+    return (torch.stack((low, high), dim=1).flatten()[:count].to(torch.int16) - 8).to(torch.float32)
+
+
+def quantize_card(state: dict) -> dict:
+    """Convert a card state dict to independently scaled 4-bit tensors."""
+    torch, _, _ = _torch()
+    tensors = {}
+    for name, tensor in state.items():
+        source = tensor.detach().cpu().float()
+        scale = max(float(source.abs().max()) / 7.0, 1e-8)
+        quantized = torch.round(source / scale).clamp(-8, 7).to("int8")
+        tensors[name] = {"shape": list(source.shape), "scale": scale, "count": source.numel(), "packed": _pack_int4(quantized)}
+    return {"format": "cardstack-int4-v1", "tensors": tensors}
+
+
+def dequantize_card(payload: dict) -> dict:
+    """Restore a quantized card state dict just before the card is needed."""
+    if payload.get("format") != "cardstack-int4-v1":
+        return payload
+    torch, _, _ = _torch()
+    result = {}
+    for name, item in payload["tensors"].items():
+        values = _unpack_int4(item["packed"], item["count"]) * item["scale"]
+        result[name] = values.reshape(item["shape"])
+    return result
+
+
+class CardStackLM:
+    """Byte-level LM. A generation step emits exactly one next byte/character."""
+
+    def __init__(self, config: CardStackConfig, card_directory: str | Path | None = None) -> None:
+        torch, nn, _ = _torch()
+        if not 1 <= config.active_cards <= config.cards_per_layer:
+            raise ValueError("active_cards must be between 1 and cards_per_layer")
+        self.torch, self.nn, self.config = torch, nn, config
+        self.embedding = nn.Embedding(256, config.width)
+        self.position = nn.Embedding(config.context_size, config.width)
+        self.routers = nn.ModuleList([nn.Linear(config.width, config.cards_per_layer) for _ in range(config.layers)])
+        self.norms = nn.ModuleList([nn.LayerNorm(config.width) for _ in range(config.layers)])
+        self.head = nn.Linear(config.width, 256)
+        self.card_directory = Path(card_directory) if card_directory else None
+        self.experts = nn.ModuleList() if not card_directory else None
+        if self.experts is not None:
+            for _ in range(config.layers):
+                self.experts.append(nn.ModuleList([self._new_card() for _ in range(config.cards_per_layer)]))
+        self.cache: OrderedDict[tuple[int, int], object] = OrderedDict()
+
+    def _new_card(self):
+        nn = self.nn
+        return nn.Sequential(nn.Linear(self.config.width, self.config.width * 2), nn.GELU(), nn.Linear(self.config.width * 2, self.config.width))
+
+    def _card(self, layer: int, index: int):
+        if self.experts is not None:
+            return self.experts[layer][index]
+        key = (layer, index)
+        if key in self.cache:
+            self.cache.move_to_end(key)
+            return self.cache[key]
+        if self.card_directory is None:
+            raise RuntimeError("No card storage configured")
+        card = self._new_card()
+        state = self.torch.load(self.card_directory / f"layer_{layer:02d}_card_{index:02d}.pt", map_location="cpu", weights_only=True)
+        state = dequantize_card(state)
+        card.load_state_dict(state)
+        card.eval()
+        self.cache[key] = card
+        while len(self.cache) > self.config.cache_cards:
+            self.cache.popitem(last=False)
+        return card
+
+    def prefetch(self, layer: int, card_indices: list[int]) -> None:
+        """Load likely next cards before the caller reaches that layer."""
+        if self.experts is None:
+            for index in card_indices:
+                self._card(layer, int(index))
+
+    def parameters(self):
+        modules = [self.embedding, self.position, self.routers, self.norms, self.head]
+        if self.experts is not None:
+            modules.append(self.experts)
+        for module in modules:
+            yield from module.parameters()
+
+    def train(self) -> None:
+        for module in (self.embedding, self.position, self.routers, self.norms, self.head):
+            module.train()
+        if self.experts is not None:
+            self.experts.train()
+
+    def eval(self) -> None:
+        for module in (self.embedding, self.position, self.routers, self.norms, self.head):
+            module.eval()
+        if self.experts is not None:
+            self.experts.eval()
+
+    def forward(self, token_ids):
+        torch = self.torch
+        _, sequence = token_ids.shape
+        if sequence > self.config.context_size:
+            token_ids = token_ids[:, -self.config.context_size:]
+            sequence = token_ids.shape[1]
+        positions = torch.arange(sequence, device=token_ids.device)
+        hidden = self.embedding(token_ids) + self.position(positions)[None, :, :]
+        for layer, (router, norm) in enumerate(zip(self.routers, self.norms)):
+            normalized = norm(hidden)
+            scores = router(normalized)
+            values, indices = scores.topk(self.config.active_cards, dim=-1)
+            # After layer 0 the current representation is a useful cheap hint for
+            # the next router, so prefetch its likely cards from SSD into the LRU.
+            if self.experts is None and layer + 1 < self.config.layers:
+                next_scores = self.routers[layer + 1](normalized)
+                next_indices = next_scores.topk(self.config.active_cards, dim=-1).indices.unique().tolist()
+                self.prefetch(layer + 1, next_indices)
+            weights = values.softmax(dim=-1)
+            routed = torch.zeros_like(hidden)
+            # Run only cards selected anywhere in this batch, then apply their gate.
+            for card_index in indices.unique().tolist():
+                card_output = self._card(layer, card_index)(normalized)
+                gate = (weights * (indices == card_index)).sum(dim=-1, keepdim=True)
+                routed = routed + card_output * gate
+            hidden = hidden + routed
+        return self.head(hidden)
+
+    def save(self, directory: str | Path, quantize_cards: bool = True) -> None:
+        if self.experts is None:
+            raise RuntimeError("A paged inference model cannot be saved as a new training bundle")
+        destination = Path(directory)
+        destination.mkdir(parents=True, exist_ok=True)
+        (destination / "config.json").write_text(json.dumps(asdict(self.config), indent=2), encoding="utf-8")
+        self.torch.save({"embedding": self.embedding.state_dict(), "position": self.position.state_dict(), "routers": self.routers.state_dict(), "norms": self.norms.state_dict(), "head": self.head.state_dict()}, destination / "base.pt")
+        for layer, cards in enumerate(self.experts):
+            for index, card in enumerate(cards):
+                payload = quantize_card(card.state_dict()) if quantize_cards else card.state_dict()
+                self.torch.save(payload, destination / f"layer_{layer:02d}_card_{index:02d}.pt")
+
+    @classmethod
+    def load(cls, directory: str | Path) -> "CardStackLM":
+        folder = Path(directory)
+        config = CardStackConfig(**json.loads((folder / "config.json").read_text(encoding="utf-8")))
+        model = cls(config, card_directory=folder)
+        state = model.torch.load(folder / "base.pt", map_location="cpu", weights_only=True)
+        for name in ("embedding", "position", "routers", "norms", "head"):
+            getattr(model, name).load_state_dict(state[name])
+        model.eval()
+        return model
+
+    @classmethod
+    def load_for_training(cls, directory: str | Path) -> "CardStackLM":
+        """Load every card into memory for training. Use ``load`` for low-memory inference."""
+        folder = Path(directory)
+        config = CardStackConfig(**json.loads((folder / "config.json").read_text(encoding="utf-8")))
+        model = cls(config)
+        state = model.torch.load(folder / "base.pt", map_location="cpu", weights_only=True)
+        for name in ("embedding", "position", "routers", "norms", "head"):
+            getattr(model, name).load_state_dict(state[name])
+        for layer, cards in enumerate(model.experts):
+            for index, card in enumerate(cards):
+                payload = model.torch.load(folder / f"layer_{layer:02d}_card_{index:02d}.pt", map_location="cpu", weights_only=True)
+                card.load_state_dict(dequantize_card(payload))
+        return model
+
+    def generate(self, prompt: str, new_characters: int = 80, temperature: float = 0.8) -> str:
+        torch = self.torch
+        if temperature <= 0:
+            raise ValueError("temperature must be positive")
+        values = list(prompt.encode("utf-8", "replace")) or [32]
+        self.eval()
+        with torch.no_grad():
+            for _ in range(new_characters):
+                input_ids = torch.tensor([values[-self.config.context_size:]], dtype=torch.long)
+                logits = self.forward(input_ids)[0, -1] / temperature
+                next_value = torch.multinomial(torch.softmax(logits, dim=-1), 1).item()
+                values.append(next_value)
+        return bytes(values).decode("utf-8", "replace")
+
+
+def train_next_character(model: CardStackLM, text: str, steps: int = 500, learning_rate: float = 3e-4) -> list[float]:
+    """Train a small model from user-supplied text. Returns sampled loss values."""
+    torch, _, functional = _torch()
+    data = torch.tensor(list(text.encode("utf-8", "replace")), dtype=torch.long)
+    if len(data) <= model.config.context_size:
+        raise ValueError("Training text must exceed context_size bytes")
+    model.train()
+    optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)
+    losses = []
+    for step in range(steps):
+        start = torch.randint(0, len(data) - model.config.context_size - 1, (1,)).item()
+        inputs = data[start:start + model.config.context_size].unsqueeze(0)
+        targets = data[start + 1:start + model.config.context_size + 1].unsqueeze(0)
+        optimizer.zero_grad()
+        loss = functional.cross_entropy(model.forward(inputs).reshape(-1, 256), targets.reshape(-1))
+        loss.backward()
+        optimizer.step()
+        if step % max(1, steps // 10) == 0 or step == steps - 1:
+            losses.append(round(float(loss.detach()), 4))
+    return losses

appbinary_inspector-0.6.0/src/appbinary_inspector/chat_server.py

@@ -0,0 +1,67 @@
+"""Local browser chat UI for a trained CardStack model."""
+from __future__ import annotations
+
+import json
+import threading
+import webbrowser
+from http import HTTPStatus
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+
+from .cardstack import CardStackLM
+
+PAGE = """<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>CardStack Chat</title><style>
+:root{--bg:#0a0d13;--panel:#121823;--line:#263142;--text:#f1f5f9;--muted:#9dacbf;--accent:#67e8f9;--user:#193352}*{box-sizing:border-box}body{margin:0;background:var(--bg);color:var(--text);font:16px system-ui,-apple-system,Segoe UI,sans-serif}.shell{max-width:920px;min-height:100vh;margin:auto;padding:28px 18px;display:flex;flex-direction:column;gap:20px}header{display:flex;justify-content:space-between;align-items:flex-start;border-bottom:1px solid var(--line);padding-bottom:18px}h1{font-size:22px;margin:0 0 5px}p{margin:0;color:var(--muted)}.badge{font-size:12px;color:var(--accent);border:1px solid #1d6976;border-radius:999px;padding:6px 9px}.chat{flex:1;display:flex;flex-direction:column;gap:14px}.empty{margin:auto;text-align:center;max-width:470px}.empty h2{font-size:24px;margin-bottom:8px}.message{max-width:84%;padding:13px 15px;border:1px solid var(--line);border-radius:14px;white-space:pre-wrap;line-height:1.45}.assistant{align-self:flex-start;background:var(--panel)}.user{align-self:flex-end;background:var(--user)}form{display:flex;gap:10px;position:sticky;bottom:0;background:var(--bg);padding-top:12px}textarea{resize:none;min-height:54px;max-height:180px;flex:1;border:1px solid var(--line);background:var(--panel);color:var(--text);border-radius:12px;padding:15px;font:inherit}button{border:0;border-radius:12px;padding:0 18px;background:var(--accent);color:#062027;font-weight:700;cursor:pointer}button:disabled{opacity:.55;cursor:wait}.hint{font-size:12px;color:var(--muted);text-align:center}@media(max-width:560px){.shell{padding:18px 12px}.badge{display:none}.message{max-width:94%}button{padding:0 14px}}</style></head><body><main class="shell"><header><div><h1>CardStack Chat</h1><p>Disk-paged character model · cards load only when routed</p></div><span class="badge">LOCAL ONLY</span></header><section class="chat" id="chat"><div class="empty" id="empty"><h2>Start a conversation</h2><p>This model produces one next character at a time. Quality depends on the text used to train its cards.</p></div></section><form id="form"><textarea id="prompt" aria-label="Message" placeholder="Type a message" required></textarea><button id="send" type="submit">Send</button></form><div class="hint">Enter to send · Shift+Enter for a new line</div></main><script>const chat=document.querySelector('#chat'),empty=document.querySelector('#empty'),form=document.querySelector('#form'),prompt=document.querySelector('#prompt'),send=document.querySelector('#send');function add(kind,text){empty?.remove();const el=document.createElement('div');el.className='message '+kind;el.textContent=text;chat.append(el);el.scrollIntoView({block:'end',behavior:'smooth'});return el}form.addEventListener('submit',async e=>{e.preventDefault();const text=prompt.value.trim();if(!text)return;add('user',text);prompt.value='';send.disabled=true;const reply=add('assistant','Thinking…');try{const r=await fetch('/api/generate',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({prompt:text,characters:180})});const body=await r.json();reply.textContent=body.text||body.error||'No response.'}catch(err){reply.textContent='Could not reach the local model: '+err.message}finally{send.disabled=false;prompt.focus()}});prompt.addEventListener('keydown',e=>{if(e.key==='Enter'&&!e.shiftKey){e.preventDefault();form.requestSubmit()}});prompt.focus();</script></body></html>"""
+
+
+def serve_chat(model_directory: str | Path, host: str = "127.0.0.1", port: int = 8765, open_browser: bool = True) -> None:
+    """Serve a local-only chat box for a disk-paged CardStack model."""
+    model = CardStackLM.load(model_directory)
+    model_lock = threading.Lock()
+
+    class Handler(BaseHTTPRequestHandler):
+        def _send(self, status: int, content_type: str, body: str) -> None:
+            encoded = body.encode("utf-8")
+            self.send_response(status)
+            self.send_header("Content-Type", f"{content_type}; charset=utf-8")
+            self.send_header("Content-Length", str(len(encoded)))
+            self.end_headers()
+            self.wfile.write(encoded)
+
+        def do_GET(self) -> None:
+            if self.path != "/":
+                self._send(HTTPStatus.NOT_FOUND, "text/plain", "Not found")
+                return
+            self._send(HTTPStatus.OK, "text/html", PAGE)
+
+        def do_POST(self) -> None:
+            if self.path != "/api/generate":
+                self._send(HTTPStatus.NOT_FOUND, "application/json", '{"error":"Not found"}')
+                return
+            try:
+                size = int(self.headers.get("Content-Length", "0"))
+                request = json.loads(self.rfile.read(min(size, 20_000)))
+                prompt = str(request.get("prompt", "")).strip()
+                characters = max(1, min(int(request.get("characters", 180)), 500))
+                if not prompt:
+                    raise ValueError("Write a message first.")
+                with model_lock:
+                    text = model.generate(prompt, new_characters=characters)
+                self._send(HTTPStatus.OK, "application/json", json.dumps({"text": text}))
+            except Exception as error:
+                self._send(HTTPStatus.BAD_REQUEST, "application/json", json.dumps({"error": str(error)}))
+
+        def log_message(self, *_args) -> None:
+            return
+
+    server = ThreadingHTTPServer((host, port), Handler)
+    address = f"http://{host}:{port}"
+    if open_browser:
+        webbrowser.open(address)
+    print(f"CardStack Chat is running at {address}. Press Ctrl+C to stop.")
+    try:
+        server.serve_forever()
+    except KeyboardInterrupt:
+        pass
+    finally:
+        server.server_close()

appbinary_inspector-0.6.0/src/appbinary_inspector/cli.py

@@ -0,0 +1,83 @@
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+from .analyzer import inspect_file
+from .extract import extract_embedded
+from .extract import extract_binary_parts
+from .tokenizer import WordTokenizer
+from .torch_builder import PRESETS
+from .cardstack import CardStackConfig, CardStackLM, train_next_character
+from .chat_server import serve_chat
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Static application-binary inspector. Never executes targets.")
+    commands = parser.add_subparsers(dest="command", required=True)
+    inspect_cmd = commands.add_parser("inspect", help="inspect a binary")
+    inspect_cmd.add_argument("file")
+    inspect_cmd.add_argument("--strings", type=int, default=200)
+    extract_cmd = commands.add_parser("extract", help="extract recognizable embedded files")
+    extract_cmd.add_argument("file")
+    extract_cmd.add_argument("--output", required=True)
+    parts_cmd = commands.add_parser("binary-extract", help="export PE headers and raw sections")
+    parts_cmd.add_argument("file")
+    parts_cmd.add_argument("--output", required=True)
+    tokenize_cmd = commands.add_parser("tokenize", help="train a tokenizer and encode a text/source file")
+    tokenize_cmd.add_argument("file")
+    tokenize_cmd.add_argument("--vocab", default="tokenizer.json")
+    tokenize_cmd.add_argument("--max-vocab", type=int, default=20_000)
+    preset_cmd = commands.add_parser("model-template", help="print a PyTorch model configuration preset")
+    preset_cmd.add_argument("name", choices=sorted(PRESETS))
+    card_create = commands.add_parser("card-create", help="create a disk-paged character model")
+    card_create.add_argument("--output", required=True)
+    card_create.add_argument("--width", type=int, default=128)
+    card_create.add_argument("--layers", type=int, default=4)
+    card_create.add_argument("--cards", type=int, default=8)
+    card_create.add_argument("--active-cards", type=int, default=2)
+    card_create.add_argument("--cache-cards", type=int, default=4)
+    card_train = commands.add_parser("card-train", help="train a CardStack model from text")
+    card_train.add_argument("model")
+    card_train.add_argument("text_file")
+    card_train.add_argument("--steps", type=int, default=500)
+    card_generate = commands.add_parser("card-generate", help="generate one character at a time from disk-paged cards")
+    card_generate.add_argument("model")
+    card_generate.add_argument("prompt")
+    card_generate.add_argument("--characters", type=int, default=80)
+    chat_cmd = commands.add_parser("chat", help="open a local browser chat box for a trained CardStack model")
+    chat_cmd.add_argument("model")
+    chat_cmd.add_argument("--port", type=int, default=8765)
+    chat_cmd.add_argument("--no-browser", action="store_true")
+    args = parser.parse_args()
+    if args.command == "inspect":
+        result = inspect_file(args.file, args.strings)
+    elif args.command == "extract":
+        result = extract_embedded(args.file, args.output)
+    elif args.command == "binary-extract":
+        result = extract_binary_parts(args.file, args.output)
+    elif args.command == "tokenize":
+        text = Path(args.file).read_text(encoding="utf-8", errors="replace")
+        tokenizer = WordTokenizer().fit([text], max_vocab=args.max_vocab)
+        tokenizer.save(args.vocab)
+        result = {"vocabulary_path": args.vocab, "vocabulary_size": len(tokenizer.vocabulary), "tokens": tokenizer.tokenize(text), "ids": tokenizer.encode(text)}
+    elif args.command == "card-create":
+        config = CardStackConfig(width=args.width, layers=args.layers, cards_per_layer=args.cards, active_cards=args.active_cards, cache_cards=args.cache_cards)
+        CardStackLM(config).save(args.output)
+        result = {"model": args.output, "config": config.__dict__, "note": "Cards were saved in 4-bit files. Train before expecting useful output."}
+    elif args.command == "card-train":
+        model = CardStackLM.load_for_training(args.model)
+        losses = train_next_character(model, Path(args.text_file).read_text(encoding="utf-8", errors="replace"), steps=args.steps)
+        model.save(args.model)
+        result = {"model": args.model, "steps": args.steps, "loss_samples": losses}
+    elif args.command == "card-generate":
+        result = {"text": CardStackLM.load(args.model).generate(args.prompt, new_characters=args.characters)}
+    elif args.command == "chat":
+        serve_chat(args.model, port=args.port, open_browser=not args.no_browser)
+        return
+    else:
+        result = PRESETS[args.name]
+    print(json.dumps(result, indent=2))
+
+if __name__ == "__main__":
+    main()

appbinary_inspector-0.2.0/PKG-INFO

@@ -1,65 +0,0 @@
-Metadata-Version: 2.5
-Name: appbinary-inspector
-Version: 0.2.0
-Summary: Safe static inspection and embedded-file extraction for application binaries.
-Author: William Laurento
-License: MIT
-License-File: LICENSE
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Operating System :: OS Independent
-Classifier: Programming Language :: Python :: 3
-Requires-Python: >=3.10
-Provides-Extra: torch
-Requires-Dist: torch>=2.1; extra == 'torch'
-Description-Content-Type: text/markdown
-
-# appbinary-inspector
-
-Static inspection of application binaries, text tokenization, and config-first PyTorch model building without executing analyzed files.
-
-## Features
-
-- Detects PE, ELF, Mach-O, ZIP, 7z, and RAR headers
-- Calculates SHA-256 and SHA-1 hashes
-- Extracts readable strings and entropy
-- Reads basic PE header metadata
-- Extracts embedded ZIP archives and carves recognizable binary signatures
-- Exports PE headers and raw binary sections for authorized analysis
-- Builds a compact tokenizer for source code, logs, or extracted text
-- Provides editable PyTorch classifier, autoencoder, and transformer templates
-
-## Install
-
-```bash
-pip install appbinary-inspector
-```
-
-## Use
-
-```bash
-appbin inspect program.exe
-appbin extract program.exe --output extracted
-appbin binary-extract program.exe --output pe_parts
-appbin tokenize source.py --vocab source-tokenizer.json
-appbin model-template classifier
-```
-
-## PyTorch models
-
-Install optional PyTorch support:
-
-```bash
-pip install "appbinary-inspector[torch]"
-```
-
-```python
-from appbinary_inspector.torch_builder import create_model, preset
-
-config = preset("classifier")
-config["layers"][0]["out_features"] = 512
-model = create_model(config)
-```
-
-## Safety
-
-Targets are read as bytes only and never executed. The package does not defeat encryption, DRM, licensing controls, passwords, or code protections. It can analyze plaintext, export raw PE sections, and extract recognizable embedded content from files you are authorized to inspect.

appbinary_inspector-0.2.0/README.md

@@ -1,50 +0,0 @@
-# appbinary-inspector
-
-Static inspection of application binaries, text tokenization, and config-first PyTorch model building without executing analyzed files.
-
-## Features
-
-- Detects PE, ELF, Mach-O, ZIP, 7z, and RAR headers
-- Calculates SHA-256 and SHA-1 hashes
-- Extracts readable strings and entropy
-- Reads basic PE header metadata
-- Extracts embedded ZIP archives and carves recognizable binary signatures
-- Exports PE headers and raw binary sections for authorized analysis
-- Builds a compact tokenizer for source code, logs, or extracted text
-- Provides editable PyTorch classifier, autoencoder, and transformer templates
-
-## Install
-
-```bash
-pip install appbinary-inspector
-```
-
-## Use
-
-```bash
-appbin inspect program.exe
-appbin extract program.exe --output extracted
-appbin binary-extract program.exe --output pe_parts
-appbin tokenize source.py --vocab source-tokenizer.json
-appbin model-template classifier
-```
-
-## PyTorch models
-
-Install optional PyTorch support:
-
-```bash
-pip install "appbinary-inspector[torch]"
-```
-
-```python
-from appbinary_inspector.torch_builder import create_model, preset
-
-config = preset("classifier")
-config["layers"][0]["out_features"] = 512
-model = create_model(config)
-```
-
-## Safety
-
-Targets are read as bytes only and never executed. The package does not defeat encryption, DRM, licensing controls, passwords, or code protections. It can analyze plaintext, export raw PE sections, and extract recognizable embedded content from files you are authorized to inspect.

appbinary_inspector-0.2.0/src/appbinary_inspector/cli.py

@@ -1,48 +0,0 @@
-from __future__ import annotations
-
-import argparse
-import json
-from pathlib import Path
-
-from .analyzer import inspect_file
-from .extract import extract_embedded
-from .extract import extract_binary_parts
-from .tokenizer import WordTokenizer
-from .torch_builder import PRESETS
-
-def main() -> None:
-    parser = argparse.ArgumentParser(description="Static application-binary inspector. Never executes targets.")
-    commands = parser.add_subparsers(dest="command", required=True)
-    inspect_cmd = commands.add_parser("inspect", help="inspect a binary")
-    inspect_cmd.add_argument("file")
-    inspect_cmd.add_argument("--strings", type=int, default=200)
-    extract_cmd = commands.add_parser("extract", help="extract recognizable embedded files")
-    extract_cmd.add_argument("file")
-    extract_cmd.add_argument("--output", required=True)
-    parts_cmd = commands.add_parser("binary-extract", help="export PE headers and raw sections")
-    parts_cmd.add_argument("file")
-    parts_cmd.add_argument("--output", required=True)
-    tokenize_cmd = commands.add_parser("tokenize", help="train a tokenizer and encode a text/source file")
-    tokenize_cmd.add_argument("file")
-    tokenize_cmd.add_argument("--vocab", default="tokenizer.json")
-    tokenize_cmd.add_argument("--max-vocab", type=int, default=20_000)
-    preset_cmd = commands.add_parser("model-template", help="print a PyTorch model configuration preset")
-    preset_cmd.add_argument("name", choices=sorted(PRESETS))
-    args = parser.parse_args()
-    if args.command == "inspect":
-        result = inspect_file(args.file, args.strings)
-    elif args.command == "extract":
-        result = extract_embedded(args.file, args.output)
-    elif args.command == "binary-extract":
-        result = extract_binary_parts(args.file, args.output)
-    elif args.command == "tokenize":
-        text = Path(args.file).read_text(encoding="utf-8", errors="replace")
-        tokenizer = WordTokenizer().fit([text], max_vocab=args.max_vocab)
-        tokenizer.save(args.vocab)
-        result = {"vocabulary_path": args.vocab, "vocabulary_size": len(tokenizer.vocabulary), "tokens": tokenizer.tokenize(text), "ids": tokenizer.encode(text)}
-    else:
-        result = PRESETS[args.name]
-    print(json.dumps(result, indent=2))
-
-if __name__ == "__main__":
-    main()