strands-slm 0.1.0__tar.gz

strands_slm-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 cagataycali
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

strands_slm-0.1.0/PKG-INFO

@@ -0,0 +1,164 @@
+Metadata-Version: 2.4
+Name: strands-slm
+Version: 0.1.0
+Summary: Self-learning model: a Strands-Agents-expert Qwen3-VL-2B whose weights keep changing at inference — surprise-gated, bounded, with a provable off-switch.
+Author: cagataycali
+License: MIT
+Project-URL: Homepage, https://github.com/cagataycali/slm
+Project-URL: Repository, https://github.com/cagataycali/slm
+Keywords: test-time-learning,continual-learning,fast-weights,lora,qwen,strands-agents,self-learning
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch>=2.1
+Requires-Dist: transformers>=4.45
+Requires-Dist: huggingface_hub>=0.20
+Provides-Extra: train
+Requires-Dist: peft>=0.10; extra == "train"
+Requires-Dist: accelerate; extra == "train"
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Dynamic: license-file
+
+# slm
+
+**A model whose weights change while it runs.**
+Predict, get surprised, rewrite a small bounded part of yourself, never forget the base.
+
+Every LLM you have used is frozen at deployment. `slm` wraps a frozen
+[Qwen3-VL-2B post-tuned on the strands-agents codebase](https://huggingface.co/cagataydev/strands-qwen3-vl-2b)
+with a plastic layer that keeps learning at inference — with a provable off-switch.
+
+```bash
+pip install strands-slm
+```
+
+## Quickstart
+
+As a [Strands Agents](https://github.com/strands-agents) model provider — every turn can change the weights:
+
+```python
+from strands import Agent
+from strands_tools import shell
+from slm import SLM
+
+model = SLM("cagataydev/strands-qwen3-vl-2b")
+agent = Agent(tools=[shell], model=model)
+
+agent("use the shell tool to run: echo hello")   # this turn updated the weights
+```
+
+Or drive the learning loop directly:
+
+```python
+from slm import StrandsPlasticQwen
+
+m = StrandsPlasticQwen.from_pretrained()
+print(m.chat("How do I create a custom tool in Strands Agents?"))
+
+for doc in your_stream:
+    m.observe(doc, learn=True)   # predicts; if surprised, rewrites its fast weights
+m.reset()                        # bit-exact back to the base
+```
+
+**See it happen: [demo.ipynb](demo.ipynb)** — ask the model a question it cannot
+know, let it read documents (pure inference), ask again — it knows. Then reset,
+and it forgets. Executed outputs and plots embedded; validated on an L40S:
+P(correct) 0.09 → 0.74, greedy answers 3/3, reset Δlogits = 0.
+
+## How it works
+
+```
+frozen Qwen3-VL-2B            instinct — never updated, cannot forget
+  + strands LoRA (merged)     slow: post-tuned strands-agents expertise
+  + plastic LoRA              fast: ~1.6M params over the frozen 2.13B,
+                              updated on every observation at inference
+  + surprise gate             learn only when prediction error spikes
+  + EMA decay                 bounded plasticity — learns AND retains
+
+loss = next-observation prediction error   (the free label from reality)
+```
+
+## Results
+
+Measured on a single GPU, seed-replicated. The base model is never updated.
+
+| claim | evidence |
+|---|---|
+| Domain expert | strands probe NLL 4.85 → 2.22, 8/8 probes improved |
+| Learns while running | continual OOD stream NLL 6.18 → 5.37, pure inference |
+| Does not forget | strands expertise after OOD learning: Δ −0.01 |
+| Agent competence grows | held-out tasks 0/4 → 4/4 after 18 curated lessons, 5/5 seeds |
+| Fact memory | 15/15 facts at 100% verbatim recall |
+| Fleet learning | two agents' experience files summed losslessly |
+| Persistence | experience survives process death bit-exact |
+| Provable off-switch | `reset()` is bit-identical to the base, Δlogits = 0 |
+| Cost | +0.11–0.25 s/turn learning overhead |
+
+The stability–plasticity dial, measured (OOD baseline NLL 4.23):
+
+| lr | EMA decay | OOD gain | retention Δ | |
+|---|---|---|---|---|
+| 2e-3 | 0.98 | +0.05 | +0.00 | too timid |
+| 8e-3 | 0.98 | +0.89 | +0.03 | the sweet spot |
+| 1e-2 | none | +3.30 | +7.09 | forgets the base |
+
+## API
+
+| method | what it does |
+|---|---|
+| `SLM(model_id, plasticity="high", placement="deep")` | Strands provider; agent turns learn automatically |
+| `.teach(prompt, response)` | curated lesson: bind a future query to a desired response |
+| `.observe(text, learn=True)` | free-form learning; returns pre-update surprise (NLL) |
+| `.consolidate(epochs=5)` | sleep phase: replay the lesson buffer, harden weak memories |
+| `.revise(prompt, old, new)` | targeted unlearning: flip a consolidated belief |
+| `.save_fast_weights(path)` / `.load_fast_weights(path)` | persist or restore acquired experience |
+| `.merge_experience(paths)` | fleet learning: compose multiple agents' experience files |
+| `.reset()` | the off-switch — exactly the base model again |
+| `.surprise_log` | (turn, NLL) history — watch it learn |
+
+## What we learned building it
+
+1. Placement determines what can be learned: attention q/v LoRA stores bindings
+   about 4x more sample-efficiently than the LM head.
+2. There is a free-learning regime (deep placement, lr 2e-2, decay 0.999):
+   skill acquisition at zero retention cost.
+3. You retrieve in the format you learned — render lessons through the real
+   chat template or the knowledge is invisible at inference.
+4. Curation is the difference between experience and learning: raw feedback
+   transcripts teach nothing; distilled (task → corrected response) pairs
+   take held-out competence from 0/4 to 4/4.
+5. Interleave or lose it: sequential lessons evict each other; replay makes
+   them coexist. Sleep-style consolidation hardens weak memories.
+6. Belief revision is a terminal operation: whatever is learned last in a
+   semantic neighborhood wins — order lessons before the revision.
+
+## Honest limitations
+
+- A bolt-on linear memory degrades single-prompt in-context recall — softmax
+  attention is already the better mechanism there. The win is persistent
+  cross-sequence adaptation, which the context window cannot retain.
+- About a third of naive test-time-training gains in the literature are pure
+  calibration (even zero-information targets help an over-confident head).
+  Our evals control for this with an information-ladder baseline.
+- Composition (learned schema x unseen entity) plateaus near 67% at 2B.
+- All findings are at 2B scale; scaling behavior is unknown.
+
+## Reproduce the post-tune
+
+```bash
+pip install "strands-slm[train]"
+python scripts/build_corpus.py       # strands-agents repos -> corpus.jsonl
+python scripts/train_lora.py --steps 1200 --bs 2 --accum 4 --lr 1e-4
+python scripts/eval_strands.py       # base vs tuned probes
+```
+
+Private HF repos need `HF_TOKEN` in the environment, or pass `token=`.
+
+## License
+
+MIT

strands_slm-0.1.0/README.md

@@ -0,0 +1,138 @@
+# slm
+
+**A model whose weights change while it runs.**
+Predict, get surprised, rewrite a small bounded part of yourself, never forget the base.
+
+Every LLM you have used is frozen at deployment. `slm` wraps a frozen
+[Qwen3-VL-2B post-tuned on the strands-agents codebase](https://huggingface.co/cagataydev/strands-qwen3-vl-2b)
+with a plastic layer that keeps learning at inference — with a provable off-switch.
+
+```bash
+pip install strands-slm
+```
+
+## Quickstart
+
+As a [Strands Agents](https://github.com/strands-agents) model provider — every turn can change the weights:
+
+```python
+from strands import Agent
+from strands_tools import shell
+from slm import SLM
+
+model = SLM("cagataydev/strands-qwen3-vl-2b")
+agent = Agent(tools=[shell], model=model)
+
+agent("use the shell tool to run: echo hello")   # this turn updated the weights
+```
+
+Or drive the learning loop directly:
+
+```python
+from slm import StrandsPlasticQwen
+
+m = StrandsPlasticQwen.from_pretrained()
+print(m.chat("How do I create a custom tool in Strands Agents?"))
+
+for doc in your_stream:
+    m.observe(doc, learn=True)   # predicts; if surprised, rewrites its fast weights
+m.reset()                        # bit-exact back to the base
+```
+
+**See it happen: [demo.ipynb](demo.ipynb)** — ask the model a question it cannot
+know, let it read documents (pure inference), ask again — it knows. Then reset,
+and it forgets. Executed outputs and plots embedded; validated on an L40S:
+P(correct) 0.09 → 0.74, greedy answers 3/3, reset Δlogits = 0.
+
+## How it works
+
+```
+frozen Qwen3-VL-2B            instinct — never updated, cannot forget
+  + strands LoRA (merged)     slow: post-tuned strands-agents expertise
+  + plastic LoRA              fast: ~1.6M params over the frozen 2.13B,
+                              updated on every observation at inference
+  + surprise gate             learn only when prediction error spikes
+  + EMA decay                 bounded plasticity — learns AND retains
+
+loss = next-observation prediction error   (the free label from reality)
+```
+
+## Results
+
+Measured on a single GPU, seed-replicated. The base model is never updated.
+
+| claim | evidence |
+|---|---|
+| Domain expert | strands probe NLL 4.85 → 2.22, 8/8 probes improved |
+| Learns while running | continual OOD stream NLL 6.18 → 5.37, pure inference |
+| Does not forget | strands expertise after OOD learning: Δ −0.01 |
+| Agent competence grows | held-out tasks 0/4 → 4/4 after 18 curated lessons, 5/5 seeds |
+| Fact memory | 15/15 facts at 100% verbatim recall |
+| Fleet learning | two agents' experience files summed losslessly |
+| Persistence | experience survives process death bit-exact |
+| Provable off-switch | `reset()` is bit-identical to the base, Δlogits = 0 |
+| Cost | +0.11–0.25 s/turn learning overhead |
+
+The stability–plasticity dial, measured (OOD baseline NLL 4.23):
+
+| lr | EMA decay | OOD gain | retention Δ | |
+|---|---|---|---|---|
+| 2e-3 | 0.98 | +0.05 | +0.00 | too timid |
+| 8e-3 | 0.98 | +0.89 | +0.03 | the sweet spot |
+| 1e-2 | none | +3.30 | +7.09 | forgets the base |
+
+## API
+
+| method | what it does |
+|---|---|
+| `SLM(model_id, plasticity="high", placement="deep")` | Strands provider; agent turns learn automatically |
+| `.teach(prompt, response)` | curated lesson: bind a future query to a desired response |
+| `.observe(text, learn=True)` | free-form learning; returns pre-update surprise (NLL) |
+| `.consolidate(epochs=5)` | sleep phase: replay the lesson buffer, harden weak memories |
+| `.revise(prompt, old, new)` | targeted unlearning: flip a consolidated belief |
+| `.save_fast_weights(path)` / `.load_fast_weights(path)` | persist or restore acquired experience |
+| `.merge_experience(paths)` | fleet learning: compose multiple agents' experience files |
+| `.reset()` | the off-switch — exactly the base model again |
+| `.surprise_log` | (turn, NLL) history — watch it learn |
+
+## What we learned building it
+
+1. Placement determines what can be learned: attention q/v LoRA stores bindings
+   about 4x more sample-efficiently than the LM head.
+2. There is a free-learning regime (deep placement, lr 2e-2, decay 0.999):
+   skill acquisition at zero retention cost.
+3. You retrieve in the format you learned — render lessons through the real
+   chat template or the knowledge is invisible at inference.
+4. Curation is the difference between experience and learning: raw feedback
+   transcripts teach nothing; distilled (task → corrected response) pairs
+   take held-out competence from 0/4 to 4/4.
+5. Interleave or lose it: sequential lessons evict each other; replay makes
+   them coexist. Sleep-style consolidation hardens weak memories.
+6. Belief revision is a terminal operation: whatever is learned last in a
+   semantic neighborhood wins — order lessons before the revision.
+
+## Honest limitations
+
+- A bolt-on linear memory degrades single-prompt in-context recall — softmax
+  attention is already the better mechanism there. The win is persistent
+  cross-sequence adaptation, which the context window cannot retain.
+- About a third of naive test-time-training gains in the literature are pure
+  calibration (even zero-information targets help an over-confident head).
+  Our evals control for this with an information-ladder baseline.
+- Composition (learned schema x unseen entity) plateaus near 67% at 2B.
+- All findings are at 2B scale; scaling behavior is unknown.
+
+## Reproduce the post-tune
+
+```bash
+pip install "strands-slm[train]"
+python scripts/build_corpus.py       # strands-agents repos -> corpus.jsonl
+python scripts/train_lora.py --steps 1200 --bs 2 --accum 4 --lr 1e-4
+python scripts/eval_strands.py       # base vs tuned probes
+```
+
+Private HF repos need `HF_TOKEN` in the environment, or pass `token=`.
+
+## License
+
+MIT

strands_slm-0.1.0/pyproject.toml

@@ -0,0 +1,35 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "strands-slm"
+version = "0.1.0"
+description = "Self-learning model: a Strands-Agents-expert Qwen3-VL-2B whose weights keep changing at inference — surprise-gated, bounded, with a provable off-switch."
+readme = "README.md"
+requires-python = ">=3.9"
+license = {text = "MIT"}
+authors = [{name = "cagataycali"}]
+keywords = ["test-time-learning", "continual-learning", "fast-weights", "lora", "qwen", "strands-agents", "self-learning"]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Intended Audience :: Science/Research",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+dependencies = [
+    "torch>=2.1",
+    "transformers>=4.45",
+    "huggingface_hub>=0.20",
+]
+
+[project.optional-dependencies]
+train = ["peft>=0.10", "accelerate"]
+dev = ["pytest"]
+
+[project.urls]
+Homepage = "https://github.com/cagataycali/slm"
+Repository = "https://github.com/cagataycali/slm"
+
+[tool.setuptools.packages.find]
+include = ["slm*"]

strands_slm-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

strands_slm-0.1.0/slm/__init__.py

@@ -0,0 +1,32 @@
+"""
+slm — self-learning model.
+
+A Strands-Agents-expert Qwen3-VL-2B that keeps learning after deployment:
+
+  frozen Qwen3-VL-2B            (instinct — never updated, can't forget)
+    + strands LoRA (merged)     (SLOW: post-tuned strands-agents expertise)
+    + plastic LoRA on lm_head   (FAST: surprise-gated, EMA-decayed,
+                                 updated at inference — with a provable off-switch)
+
+Quick start:
+    from slm import StrandsPlasticQwen
+
+    m = StrandsPlasticQwen.from_pretrained()        # cagataydev/strands-qwen3-vl-2b
+    print(m.chat("How do I create a custom tool in Strands Agents?"))
+
+    for doc in your_stream:
+        m.observe(doc, learn=True)   # predicts; if surprised, rewrites fast weights
+    m.reset()                        # exactly back to the strands-expert base
+"""
+from .qwen import StrandsPlasticQwen, DEFAULT_MODEL
+
+
+def __getattr__(name):
+    # Lazy import: SLM needs strands-agents installed
+    if name == "SLM":
+        from .strands_model import SLM
+        return SLM
+    raise AttributeError(f"module 'slm' has no attribute {name!r}")
+
+__version__ = "0.1.0"
+__all__ = ["StrandsPlasticQwen", "SLM", "DEFAULT_MODEL", "__version__"]

strands_slm-0.1.0/slm/qwen.py

@@ -0,0 +1,158 @@
+"""
+slm.qwen — self-learning Qwen3-VL runtimes (optional extra: pip install self-learning-model[qwen]).
+
+Two runtimes over a frozen (or merged strands-expert) Qwen3-VL-2B:
+
+  StrandsPlasticQwen — the Strands-Agents-expert Qwen (post-tuned, merged) with a
+      plastic LoRA head on lm_head that keeps learning at inference:
+      surprise-gated SGD + EMA decay (bounded plasticity), provable off-switch.
+
+Validated (see README.md / PROOF.md in the repo):
+  * continual OOD stream: NLL drops online while base knowledge is retained
+  * reset() restores the base exactly (Δlogits = 0)
+
+Usage:
+    from slm.qwen import StrandsPlasticQwen
+    m = StrandsPlasticQwen.from_pretrained()            # default: strands-expert model
+    print(m.chat("How do I create a custom tool in Strands Agents?"))
+    m.observe(new_docs, learn=True)                     # self-learn after deployment
+    m.reset()                                           # off-switch
+
+Requires: torch, transformers (installed via the [qwen] extra). Private HF repos
+need HF_TOKEN in the environment.
+"""
+import os
+
+DEFAULT_MODEL = "cagataydev/strands-qwen3-vl-2b"
+
+
+def _require_torch():
+    try:
+        import torch  # noqa
+        import transformers  # noqa
+    except ImportError as e:
+        raise ImportError(
+            "slm.qwen needs the optional deps: pip install 'self-learning-model[qwen]'"
+        ) from e
+
+
+class StrandsPlasticQwen:
+    """Strands-expert Qwen3-VL-2B + fast plastic LoRA head (self-learning at inference)."""
+
+    def __init__(self, model, tok, head, lr=8e-3, decay=0.98, k_gate=0.0):
+        import torch
+        self.model, self.tok, self.head = model, tok, head
+        self.opt = torch.optim.SGD([head.A, head.B], lr=lr)
+        self.decay, self.k_gate = decay, k_gate
+        self.mean, self.beta = None, 0.9
+        self.device = next(model.parameters()).device
+
+    # ---------------- constructors ----------------
+    @classmethod
+    def from_pretrained(cls, model_id=DEFAULT_MODEL, device="cuda", r_fast=16,
+                        token=None, **kw):
+        """Load the merged strands-expert model (or any Qwen3-VL id) + attach
+        the fast plastic head. Private repos: pass token= or set HF_TOKEN."""
+        _require_torch()
+        import torch
+        import torch.nn as nn
+        from transformers import AutoModelForImageTextToText, AutoProcessor
+        token = token or os.getenv("HF_TOKEN") or os.getenv("HUGGING_FACE_HUB_TOKEN")
+        dtype = torch.bfloat16 if str(device).startswith("cuda") else torch.float32
+        proc = AutoProcessor.from_pretrained(model_id, token=token)
+        model = AutoModelForImageTextToText.from_pretrained(
+            model_id, dtype=dtype, device_map=device, token=token)
+        model.eval()
+        for p in model.parameters():
+            p.requires_grad_(False)
+
+        # attach fast plastic LoRA on lm_head
+        head_name, head_mod = None, None
+        for name, mod in model.named_modules():
+            if name.endswith("lm_head") and isinstance(mod, nn.Linear):
+                head_name, head_mod = name, mod
+        head = _PlasticHead(head_mod, r=r_fast)
+        parent = model
+        *pth, last = head_name.split(".")
+        for pp in pth:
+            parent = getattr(parent, pp)
+        setattr(parent, last, head)
+        return cls(model, proc.tokenizer, head, **kw)
+
+    # ---------------- fast self-learning ----------------
+    def reset(self):
+        """Wipe fast adaptation -> exactly the strands-expert base again."""
+        import torch
+        import torch.nn as nn
+        with torch.no_grad():
+            self.head.B.zero_()
+            nn.init.normal_(self.head.A, std=0.01)
+        self.mean = None
+
+    def _nll(self, ids):
+        import torch
+        o = self.model(input_ids=ids)
+        lg = o.logits[:, :-1, :]
+        return torch.nn.functional.cross_entropy(
+            lg.reshape(-1, lg.size(-1)).float(), ids[:, 1:].reshape(-1))
+
+    def observe(self, text, learn=True, max_length=2048):
+        """Predict `text`; if surprised, rewrite the fast weights (bounded).
+        Returns the pre-update NLL (the surprise)."""
+        import torch
+        ids = self.tok(text, return_tensors="pt", truncation=True,
+                       max_length=max_length).input_ids.to(self.device)
+        if ids.shape[1] < 2:
+            return None
+        with torch.no_grad():
+            e = self._nll(ids).item()
+        fire = (self.mean is None) or (e > self.mean + self.k_gate * abs(self.mean))
+        self.mean = e if self.mean is None else self.beta * self.mean + (1 - self.beta) * e
+        if learn and fire:
+            loss = self._nll(ids)
+            self.opt.zero_grad()
+            loss.backward()
+            torch.nn.utils.clip_grad_norm_([self.head.A, self.head.B], 1.0)
+            self.opt.step()
+            with torch.no_grad():
+                self.head.B.mul_(self.decay)      # EMA decay = bounded plasticity
+        return e
+
+    # ---------------- chat ----------------
+    def chat(self, user_msg, max_new_tokens=512, temperature=0.7):
+        import torch
+        msgs = [{"role": "user", "content": user_msg}]
+        ids = self.tok.apply_chat_template(
+            msgs, add_generation_prompt=True, return_tensors="pt").to(self.device)
+        with torch.no_grad():
+            out = self.model.generate(
+                input_ids=ids, max_new_tokens=max_new_tokens,
+                do_sample=temperature > 0, temperature=max(temperature, 1e-5),
+                pad_token_id=self.tok.eos_token_id)
+        return self.tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True)
+
+
+def _plastic_head_cls():
+    import torch
+    import torch.nn as nn
+
+    class PlasticHead(nn.Module):
+        """Fast LoRA on lm_head: y = base(x) + scale*(x A)B. Only A,B change."""
+        def __init__(self, base, r=16, scale=2.0):
+            super().__init__()
+            self.base = base
+            for p in base.parameters():
+                p.requires_grad_(False)
+            dev, dt = base.weight.device, base.weight.dtype
+            self.A = nn.Parameter(torch.randn(base.in_features, r, device=dev, dtype=dt) * 0.01)
+            self.B = nn.Parameter(torch.zeros(r, base.out_features, device=dev, dtype=dt))
+            self.scale = scale
+
+        def forward(self, x):
+            return self.base(x) + self.scale * ((x @ self.A) @ self.B)
+
+    return PlasticHead
+
+
+def _PlasticHead(base, r=16, scale=2.0):
+    return _plastic_head_cls()(base, r=r, scale=scale)