nock-cli 0.2.0__tar.gz → 0.3.0__tar.gz

{nock_cli-0.2.0 → nock_cli-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nock-cli
-Version: 0.2.0
+Version: 0.3.0
 Summary: Open models that live on your machine: code, video and a coding agent, offline.
 Author: NockAI
 License-Expression: Apache-2.0
@@ -101,6 +101,23 @@ and edited a test file 0 times. It works on your own files, so **every edit is s
 before it runs, and each one needs your yes**. `--yes` skips the questions; use it only in a copy or a git branch.
 It refuses to start on GPUs under about 20 GB (`--force` to try anyway, very slowly).
 
+## Use it from your apps: `nock serve`
+
+```bash
+nock serve
+```
+
+One local, OpenAI-compatible address for every skill you have installed. In your editor or tool, set the base URL to
+`http://127.0.0.1:8080/v1`, any API key, and the model to `nock-coder`, `nock-video` or `nock-agent`.
+
+```bash
+curl http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json"   -d '{"model": "nock-coder", "messages": [{"role": "user", "content": "regex for an EVM address"}]}'
+```
+
+The model starts on first use, and one model is in memory at a time, so asking for another swaps them (it can take
+a while). Streaming works. It listens on this machine only, refuses requests from websites, and logs no questions or
+answers. `--port` changes the port.
+
 ## Speed and GPUs
 
 `nock doctor` shows which device the engine uses. If it says **software device** or **no GPU device**, the

{nock_cli-0.2.0 → nock_cli-0.3.0}/README.md

@@ -1,97 +1,114 @@
-# Nock CLI
-
-Open models that live on your machine. One tool for the NockAI family: code, video, and a coding agent, offline.
-
-```bash
-nock add coder video
-nock code "regex for an EVM address"
-nock video find meeting.mp4 "when do they talk about the budget"
-nock fix ./my-project "the total is off by one"     # experimental, big GPU
-```
-
-Everything runs on your computer. The models run in a local llama.cpp server on `127.0.0.1`. Nothing is uploaded.
-
-## Install
-
-You need Python 3.9 or newer.
-
-```bash
-pipx install nock-cli
-nock add coder        # or: video, agent
-nock doctor
-```
-
-`nock add` downloads, once:
-- the engine: an official [llama.cpp](https://github.com/ggml-org/llama.cpp) build (about 35 MB; Vulkan on Windows
-  and Linux, Metal on Apple Silicon). Already have `llama-server`? Nock uses it.
-- the skill's model from Hugging Face.
-
-| Skill | Model | Download | Needs |
-|---|---|---|---|
-| `coder` | [Nock Coder 1.5B](https://huggingface.co/Nock-AI/nock-coder-1.5b-GGUF) | 1 GB | any laptop |
-| `video` | [Nock Video 4B](https://huggingface.co/Nock-AI/nock-video-4b-GGUF) (preview) | 3.4 GB | a GPU with 6 GB for 64 frames; [ffmpeg](https://ffmpeg.org/download.html) |
-| `agent` | [Nock Agent 31B](https://huggingface.co/Nock-AI/nock-agent-31b-GGUF) (experimental) | 18.7 GB | a GPU with about 20 GB (a Mac with 32 GB or more may work; not tested) |
-
-`nock list` shows what is installed, `nock remove <skill>` deletes a skill's files. Files go to `~/.nock`
-(set `NOCK_HOME` to use another folder).
-
-## Nock Coder: `nock code`
-
-```bash
-nock code "write a Solidity ERC-20 with a cap"
-cat error.log | nock code "why does this fail?"
-nock code                                  # a chat; type exit to quit
-```
-
-Code first, one short line of explanation. The same model and prompt as `ollama run nockai/nock-coder`. It is a
-1.5B model, so review code before you run it. On HumanEval it passes 118 of 164.
-
-## Nock Video: `nock video`
-
-```bash
-nock video summarize meeting.mp4
-nock video ask clip.mp4 "what error message shows on screen?"
-nock video find clip.mp4 "when the cake comes out"
-```
-
-Files or links. Links are downloaded to your own machine with [yt-dlp](https://github.com/yt-dlp/yt-dlp), which
-you install yourself (`pipx inject nock-cli yt-dlp`). Only use videos you have the right to use.
-
-Nock Video 4B is a **preview**. On our test of 20 videos, through llama.cpp at 64 frames, it found the moment in
-43 of 56 questions, answered 59 of 60 questions about what is on screen, and gave 6 of 7 good summaries. Full
-results, including the misses, are on the [model card](https://huggingface.co/Nock-AI/nock-video-4b-GGUF).
-
-- It reads frames only, not sound.
-- A long video is read as at most 64 frames, so short events in long videos can be missed.
-- 64 frames used about 5.5 GB of GPU memory in our test. On GPUs under 6 GB, Nock uses 32 frames (the first
-  preview dropped from 42 to 35 of 56 moments at 32 frames; the current model was not measured at 32).
-  `--frames` overrides it.
-
-## Nock Agent: `nock fix` (experimental)
-
-```bash
-nock fix ./my-project "the total is off by one"
-```
-
-Nock Agent reads the code, makes the smallest edit and runs the tests. On our agent test it solved 145 of 150 runs
-and edited a test file 0 times. It works on your own files, so **every edit is shown as a diff and every command
-before it runs, and each one needs your yes**. `--yes` skips the questions; use it only in a copy or a git branch.
-It refuses to start on GPUs under about 20 GB (`--force` to try anyway, very slowly).
-
-## Speed and GPUs
-
-`nock doctor` shows which device the engine uses. If it says **software device** or **no GPU device**, the
-engine runs on the CPU and answers take many minutes. This happens inside most containers and VMs. On a machine
-with an NVIDIA GPU, `nock setup --backend cuda` installs the CUDA build instead (a bigger download: about 630 MB on Windows, 740 MB on Linux).
-
-| Option | Meaning |
-|---|---|
-| `--backend vulkan\|cuda\|cpu\|metal` | which llama.cpp build `nock add` / `nock setup` installs |
-| `NOCK_LLAMA_SERVER=/path/llama-server` | use your own llama.cpp build |
-| `NOCK_HOME=/path` | where models and the engine are stored |
-
-Server logs are in `~/.nock/logs/` if something goes wrong.
-
-## License
-
-Apache-2.0. Each model has its own license on its model card (all Apache-2.0).
+# Nock CLI
+
+Open models that live on your machine. One tool for the NockAI family: code, video, and a coding agent, offline.
+
+```bash
+nock add coder video
+nock code "regex for an EVM address"
+nock video find meeting.mp4 "when do they talk about the budget"
+nock fix ./my-project "the total is off by one"     # experimental, big GPU
+```
+
+Everything runs on your computer. The models run in a local llama.cpp server on `127.0.0.1`. Nothing is uploaded.
+
+## Install
+
+You need Python 3.9 or newer.
+
+```bash
+pipx install nock-cli
+nock add coder        # or: video, agent
+nock doctor
+```
+
+`nock add` downloads, once:
+- the engine: an official [llama.cpp](https://github.com/ggml-org/llama.cpp) build (about 35 MB; Vulkan on Windows
+  and Linux, Metal on Apple Silicon). Already have `llama-server`? Nock uses it.
+- the skill's model from Hugging Face.
+
+| Skill | Model | Download | Needs |
+|---|---|---|---|
+| `coder` | [Nock Coder 1.5B](https://huggingface.co/Nock-AI/nock-coder-1.5b-GGUF) | 1 GB | any laptop |
+| `video` | [Nock Video 4B](https://huggingface.co/Nock-AI/nock-video-4b-GGUF) (preview) | 3.4 GB | a GPU with 6 GB for 64 frames; [ffmpeg](https://ffmpeg.org/download.html) |
+| `agent` | [Nock Agent 31B](https://huggingface.co/Nock-AI/nock-agent-31b-GGUF) (experimental) | 18.7 GB | a GPU with about 20 GB (a Mac with 32 GB or more may work; not tested) |
+
+`nock list` shows what is installed, `nock remove <skill>` deletes a skill's files. Files go to `~/.nock`
+(set `NOCK_HOME` to use another folder).
+
+## Nock Coder: `nock code`
+
+```bash
+nock code "write a Solidity ERC-20 with a cap"
+cat error.log | nock code "why does this fail?"
+nock code                                  # a chat; type exit to quit
+```
+
+Code first, one short line of explanation. The same model and prompt as `ollama run nockai/nock-coder`. It is a
+1.5B model, so review code before you run it. On HumanEval it passes 118 of 164.
+
+## Nock Video: `nock video`
+
+```bash
+nock video summarize meeting.mp4
+nock video ask clip.mp4 "what error message shows on screen?"
+nock video find clip.mp4 "when the cake comes out"
+```
+
+Files or links. Links are downloaded to your own machine with [yt-dlp](https://github.com/yt-dlp/yt-dlp), which
+you install yourself (`pipx inject nock-cli yt-dlp`). Only use videos you have the right to use.
+
+Nock Video 4B is a **preview**. On our test of 20 videos, through llama.cpp at 64 frames, it found the moment in
+43 of 56 questions, answered 59 of 60 questions about what is on screen, and gave 6 of 7 good summaries. Full
+results, including the misses, are on the [model card](https://huggingface.co/Nock-AI/nock-video-4b-GGUF).
+
+- It reads frames only, not sound.
+- A long video is read as at most 64 frames, so short events in long videos can be missed.
+- 64 frames used about 5.5 GB of GPU memory in our test. On GPUs under 6 GB, Nock uses 32 frames (the first
+  preview dropped from 42 to 35 of 56 moments at 32 frames; the current model was not measured at 32).
+  `--frames` overrides it.
+
+## Nock Agent: `nock fix` (experimental)
+
+```bash
+nock fix ./my-project "the total is off by one"
+```
+
+Nock Agent reads the code, makes the smallest edit and runs the tests. On our agent test it solved 145 of 150 runs
+and edited a test file 0 times. It works on your own files, so **every edit is shown as a diff and every command
+before it runs, and each one needs your yes**. `--yes` skips the questions; use it only in a copy or a git branch.
+It refuses to start on GPUs under about 20 GB (`--force` to try anyway, very slowly).
+
+## Use it from your apps: `nock serve`
+
+```bash
+nock serve
+```
+
+One local, OpenAI-compatible address for every skill you have installed. In your editor or tool, set the base URL to
+`http://127.0.0.1:8080/v1`, any API key, and the model to `nock-coder`, `nock-video` or `nock-agent`.
+
+```bash
+curl http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json"   -d '{"model": "nock-coder", "messages": [{"role": "user", "content": "regex for an EVM address"}]}'
+```
+
+The model starts on first use, and one model is in memory at a time, so asking for another swaps them (it can take
+a while). Streaming works. It listens on this machine only, refuses requests from websites, and logs no questions or
+answers. `--port` changes the port.
+
+## Speed and GPUs
+
+`nock doctor` shows which device the engine uses. If it says **software device** or **no GPU device**, the
+engine runs on the CPU and answers take many minutes. This happens inside most containers and VMs. On a machine
+with an NVIDIA GPU, `nock setup --backend cuda` installs the CUDA build instead (a bigger download: about 630 MB on Windows, 740 MB on Linux).
+
+| Option | Meaning |
+|---|---|
+| `--backend vulkan\|cuda\|cpu\|metal` | which llama.cpp build `nock add` / `nock setup` installs |
+| `NOCK_LLAMA_SERVER=/path/llama-server` | use your own llama.cpp build |
+| `NOCK_HOME=/path` | where models and the engine are stored |
+
+Server logs are in `~/.nock/logs/` if something goes wrong.
+
+## License
+
+Apache-2.0. Each model has its own license on its model card (all Apache-2.0).

{nock_cli-0.2.0 → nock_cli-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "nock-cli"
-version = "0.2.0"
+version = "0.3.0"
 description = "Open models that live on your machine: code, video and a coding agent, offline."
 readme = "README.md"
 requires-python = ">=3.9"

{nock_cli-0.2.0 → nock_cli-0.3.0}/src/nock_cli/__init__.py

@@ -1,3 +1,3 @@
 """Nock CLI: open models that live on your machine."""
 
-__version__ = "0.2.0"
+__version__ = "0.3.0"

{nock_cli-0.2.0 → nock_cli-0.3.0}/src/nock_cli/main.py

@@ -1,222 +1,251 @@
-"""Command line entry point: `nock add`, `nock code`, `nock video ...`, `nock fix`, `nock doctor`."""
-
-import argparse
-import shutil
-import subprocess
-import sys
-import urllib.error
-
-from nock_cli import __version__, agent, code, engine, models, paths, video
-
-SKILLS = list(models.MODELS)
-
-
-def ensure_engine(settings, backend=None):
-  """Installs Nock's llama.cpp build when there is none yet (or when a backend is asked for), and records whether
-  it can use a GPU, so every later command can warn without asking the engine again."""
-  existing = engine.locate_server(settings)
-  if existing and not backend:
-    if "uses_gpu" not in settings:
-      record_devices(settings, existing)
-    return
-  print("Installing the engine (llama.cpp, from github.com/ggml-org/llama.cpp)...")
-  server, chosen = engine.install(backend)
-  settings.update(llama_server=str(server), llama_cpp_build=engine.LLAMA_CPP_BUILD, backend=chosen)
-  record_devices(settings, server)
-
-
-def record_devices(settings, server):
-  settings["uses_gpu"], settings["device"], settings["device_id"] = engine.gpu_check(engine.list_devices(server))
-  if not settings["uses_gpu"] and settings.get("backend") != "cpu":
-    warn_no_gpu(settings)
-
-
-def warn_no_gpu(settings):
-  print(f"\nWarning: {settings.get('device')}.", file=sys.stderr)
-  if video.gpu_memory_mb() is not None:
-    print("  This machine has an NVIDIA GPU. Try: nock setup --backend cuda", file=sys.stderr)
-  print("", file=sys.stderr)
-
-
-def install_skills(skills, backend=None):
-  settings = paths.load_settings()
-  print(f"Nock home: {paths.nock_home()}")
-  ensure_engine(settings, backend)
-  for skill in skills:
-    spec = models.MODELS[skill]
-    print(f"Adding {spec['title']} ({spec['size']}) from huggingface.co/{spec['repo']}...")
-    settings.setdefault("models", {})[skill] = models.install(skill)
-    paths.save_settings(settings)
-    print(f"Ready. Try: {spec['try']}")
-  paths.save_settings(settings)
-
-
-def command_add(args):
-  install_skills(args.skills, args.backend)
-
-
-def command_setup(args):
-  # Kept from 0.1: `nock setup` installs the engine and Nock Video, or the skills named after it.
-  install_skills(args.skills or ["video"], args.backend)
-
-
-def command_remove(args):
-  settings = paths.load_settings()
-  for skill in args.skills:
-    models.remove(skill)
-    settings.get("models", {}).pop(skill, None)
-    print(f"Removed {models.MODELS[skill]['title']}.")
-  paths.save_settings(settings)
-
-
-def command_list(args):
-  installed = paths.load_settings().get("models", {})
-  for skill, spec in models.MODELS.items():
-    if skill in installed:
-      state = f"installed ({models.folder_size_gb(skill):.1f} GB, {installed[skill]['revision'][:7]})"
-    else:
-      state = f"not installed ({spec['size']}): nock add {skill}"
-    print(f"  {skill:<6} {spec['title']:<30} {state}")
-
-
-def check(label, ok, detail):
-  print(f"  [{'ok' if ok else '--'}] {label}: {detail}")
-  return ok
-
-
-def command_doctor(args):
-  settings = paths.load_settings()
-  print(f"nock {__version__}, home {paths.nock_home()}")
-  server = engine.locate_server(settings)
-  ready = check("engine", bool(server), f"{server} ({settings.get('backend', 'own build')})" if server else "missing: run `nock add video`")
-  if server:
-    uses_gpu, device, _ = engine.gpu_check(engine.list_devices(server))
-    check("GPU in the engine", uses_gpu, device)
-    if not uses_gpu and video.gpu_memory_mb() is not None:
-      print("       This machine has an NVIDIA GPU. Try: nock setup --backend cuda")
-  ready &= check("ffmpeg (for video)", bool(shutil.which("ffmpeg") and shutil.which("ffprobe")),
-                 shutil.which("ffmpeg") or "missing: install ffmpeg (https://ffmpeg.org/download.html)")
-  check("yt-dlp (optional, for links)", bool(shutil.which("yt-dlp")), shutil.which("yt-dlp") or "not installed: pipx inject nock-cli yt-dlp")
-  installed = settings.get("models", {})
-  for skill in SKILLS:
-    item = installed.get(skill)
-    check(f"skill {skill}", bool(item), f"{item['repo']}@{item['revision'][:7]}" if item else f"not installed: nock add {skill}")
-  memory = video.gpu_memory_mb()
-  check("NVIDIA GPU", memory is not None,
-        f"{memory} MB" if memory is not None else "not found (other GPUs work through Vulkan or Metal; CPU only is slow)")
-  free_gb = shutil.disk_usage(paths.nock_home().anchor or "/").free / 1e9
-  check("free disk", free_gb > 5, f"{free_gb:.0f} GB")
-  return 0 if ready else 1
-
-
-def start_server_factory(skill):
-  settings = paths.load_settings()
-  server_binary = engine.locate_server(settings)
-  installed = settings.get("models", {}).get(skill)
-  if not server_binary or not installed:
-    sys.exit(f"{models.MODELS[skill]['title']} is not installed yet. Run: nock add {skill}")
-  if settings.get("uses_gpu") is False and settings.get("backend") != "cpu":
-    warn_no_gpu(settings)
-  log_path = paths.nock_home() / "logs" / f"{skill}-server.log"
-  default_context = models.MODELS[skill].get("context")
-
-  def start(context=None):
-    return engine.LocalServer(server_binary, installed["model"], installed.get("mmproj"), context or default_context,
-                              99, log_path, settings.get("device_id"))
-
-  return start
-
-
-def command_video(args):
-  kind = {"summarize": "summary", "ask": "ask", "find": "moment"}[args.action]
-  question = video.SUMMARY_QUESTION if kind == "summary" else args.question
-  frames = video.choose_frames(args.frames)
-  print(video.run(args.source, question, kind, frames, start_server_factory("video")))
-
-
-def command_code(args):
-  code.run(" ".join(args.prompt).strip(), start_server_factory("coder"))
-
-
-def command_fix(args):
-  need = models.MODELS["agent"]["min_gpu_mb"]
-  memory = video.gpu_memory_mb()
-  if memory is not None and memory < need and not args.force:
-    sys.exit(f"Nock Agent 31B needs a GPU with about {need // 1000} GB; this one has {memory // 1000} GB. "
-             "Use --force to try anyway (it will be very slow).")
-  if memory is None:
-    print(f"Note: Nock Agent 31B needs about {need // 1000} GB of GPU memory (a Mac with 32 GB or more may work; not tested).", file=sys.stderr)
-  approve = agent.Approver(always=args.yes)
-  print(agent.run(args.project, args.task, approve, start_server_factory("agent")))
-
-
-def build_parser():
-  parser = argparse.ArgumentParser(prog="nock", description="Open models that live on your machine.")
-  parser.add_argument("--version", action="version", version=f"nock {__version__}")
-  commands = parser.add_subparsers(dest="command", required=True)
-
-  add = commands.add_parser("add", help="add a skill: coder (1 GB), video (3.4 GB), agent (18.7 GB)")
-  add.add_argument("skills", nargs="+", choices=SKILLS)
-  add.add_argument("--backend", choices=engine.BACKENDS, help="llama.cpp build: vulkan (default), cuda, cpu, metal (Mac)")
-  add.set_defaults(handler=command_add)
-
-  remove = commands.add_parser("remove", help="delete a skill's files")
-  remove.add_argument("skills", nargs="+", choices=SKILLS)
-  remove.set_defaults(handler=command_remove)
-
-  listing = commands.add_parser("list", help="show the skills and what is installed")
-  listing.set_defaults(handler=command_list)
-
-  setup = commands.add_parser("setup", help="install the engine and Nock Video (or the skills you name)")
-  setup.add_argument("skills", nargs="*", choices=SKILLS)
-  setup.add_argument("--backend", choices=engine.BACKENDS, help="llama.cpp build: vulkan (default), cuda, cpu, metal (Mac)")
-  setup.set_defaults(handler=command_setup)
-
-  doctor = commands.add_parser("doctor", help="check that everything is in place")
-  doctor.set_defaults(handler=command_doctor)
-
-  code_parser = commands.add_parser("code", help="ask Nock Coder; no question starts a chat; pipe a file or log in")
-  code_parser.add_argument("prompt", nargs="*")
-  code_parser.set_defaults(handler=command_code)
-
-  video_parser = commands.add_parser("video", help="ask about a video file or link, offline")
-  actions = video_parser.add_subparsers(dest="action", required=True)
-  summarize = actions.add_parser("summarize", help="a few short points about the video")
-  summarize.add_argument("source", help="video file or link")
-  ask = actions.add_parser("ask", help="a question about what the video shows")
-  ask.add_argument("source", help="video file or link")
-  ask.add_argument("question")
-  find = actions.add_parser("find", help="when does something happen")
-  find.add_argument("source", help="video file or link")
-  find.add_argument("question", help='what to find, for example "the error message shows up"')
-  for action in (summarize, ask, find):
-    action.add_argument("--frames", type=int, help=f"frames to read (default {video.MAX_FRAMES}, "
-                                                   f"{video.LOW_MEMORY_FRAMES} on GPUs under 6 GB)")
-  video_parser.set_defaults(handler=command_video)
-
-  fix = commands.add_parser("fix", help="Nock Agent (experimental, big GPU): fix an issue in a project folder")
-  fix.add_argument("project", help="the project folder")
-  fix.add_argument("task", help='what to fix, for example "the total is off by one"')
-  fix.add_argument("--yes", action="store_true", help="approve every edit and command without asking")
-  fix.add_argument("--force", action="store_true", help="start even on a GPU that is too small")
-  fix.set_defaults(handler=command_fix)
-  return parser
-
-
-def main(argv=None):
-  args = build_parser().parse_args(argv)
-  try:
-    result = args.handler(args)
-  except KeyboardInterrupt:
-    sys.exit("Stopped.")
-  except (engine.EngineError, models.ModelError, video.VideoError, ValueError) as error:
-    sys.exit(f"nock: {error}")
-  except subprocess.CalledProcessError as error:
-    sys.exit(f"nock: {error.cmd[0]} failed (exit code {error.returncode})")
-  except urllib.error.URLError as error:
-    sys.exit(f"nock: network error: {error.reason}")
-  sys.exit(result or 0)
-
-
-if __name__ == "__main__":
-  main()
+"""Command line entry point: `nock add`, `nock code`, `nock video ...`, `nock fix`, `nock doctor`."""
+
+import argparse
+import shutil
+import subprocess
+import sys
+import urllib.error
+
+from nock_cli import __version__, agent, code, engine, models, paths, serve, video
+
+SKILLS = list(models.MODELS)
+
+
+def ensure_engine(settings, backend=None):
+  """Installs Nock's llama.cpp build when there is none yet (or when a backend is asked for), and records whether
+  it can use a GPU, so every later command can warn without asking the engine again."""
+  existing = engine.locate_server(settings)
+  if existing and not backend:
+    if "uses_gpu" not in settings:
+      record_devices(settings, existing)
+    return
+  print("Installing the engine (llama.cpp, from github.com/ggml-org/llama.cpp)...")
+  server, chosen = engine.install(backend)
+  settings.update(llama_server=str(server), llama_cpp_build=engine.LLAMA_CPP_BUILD, backend=chosen)
+  record_devices(settings, server)
+
+
+def record_devices(settings, server):
+  settings["uses_gpu"], settings["device"], settings["device_id"] = engine.gpu_check(engine.list_devices(server))
+  if not settings["uses_gpu"] and settings.get("backend") != "cpu":
+    warn_no_gpu(settings)
+
+
+def warn_no_gpu(settings):
+  print(f"\nWarning: {settings.get('device')}.", file=sys.stderr)
+  if video.gpu_memory_mb() is not None:
+    print("  This machine has an NVIDIA GPU. Try: nock setup --backend cuda", file=sys.stderr)
+  print("", file=sys.stderr)
+
+
+def install_skills(skills, backend=None):
+  settings = paths.load_settings()
+  print(f"Nock home: {paths.nock_home()}")
+  ensure_engine(settings, backend)
+  for skill in skills:
+    spec = models.MODELS[skill]
+    print(f"Adding {spec['title']} ({spec['size']}) from huggingface.co/{spec['repo']}...")
+    settings.setdefault("models", {})[skill] = models.install(skill)
+    paths.save_settings(settings)
+    print(f"Ready. Try: {spec['try']}")
+  paths.save_settings(settings)
+
+
+def command_add(args):
+  install_skills(args.skills, args.backend)
+
+
+def command_setup(args):
+  # Kept from 0.1: `nock setup` installs the engine and Nock Video, or the skills named after it.
+  install_skills(args.skills or ["video"], args.backend)
+
+
+def command_remove(args):
+  settings = paths.load_settings()
+  for skill in args.skills:
+    models.remove(skill)
+    settings.get("models", {}).pop(skill, None)
+    print(f"Removed {models.MODELS[skill]['title']}.")
+  paths.save_settings(settings)
+
+
+def command_list(args):
+  installed = paths.load_settings().get("models", {})
+  for skill, spec in models.MODELS.items():
+    if skill in installed:
+      state = f"installed ({models.folder_size_gb(skill):.1f} GB, {installed[skill]['revision'][:7]})"
+    else:
+      state = f"not installed ({spec['size']}): nock add {skill}"
+    print(f"  {skill:<6} {spec['title']:<30} {state}")
+
+
+def check(label, ok, detail):
+  print(f"  [{'ok' if ok else '--'}] {label}: {detail}")
+  return ok
+
+
+def command_doctor(args):
+  settings = paths.load_settings()
+  print(f"nock {__version__}, home {paths.nock_home()}")
+  server = engine.locate_server(settings)
+  ready = check("engine", bool(server), f"{server} ({settings.get('backend', 'own build')})" if server else "missing: run `nock add video`")
+  if server:
+    uses_gpu, device, _ = engine.gpu_check(engine.list_devices(server))
+    check("GPU in the engine", uses_gpu, device)
+    if not uses_gpu and video.gpu_memory_mb() is not None:
+      print("       This machine has an NVIDIA GPU. Try: nock setup --backend cuda")
+  ready &= check("ffmpeg (for video)", bool(shutil.which("ffmpeg") and shutil.which("ffprobe")),
+                 shutil.which("ffmpeg") or "missing: install ffmpeg (https://ffmpeg.org/download.html)")
+  check("yt-dlp (optional, for links)", bool(shutil.which("yt-dlp")), shutil.which("yt-dlp") or "not installed: pipx inject nock-cli yt-dlp")
+  installed = settings.get("models", {})
+  for skill in SKILLS:
+    item = installed.get(skill)
+    check(f"skill {skill}", bool(item), f"{item['repo']}@{item['revision'][:7]}" if item else f"not installed: nock add {skill}")
+  memory = video.gpu_memory_mb()
+  check("NVIDIA GPU", memory is not None,
+        f"{memory} MB" if memory is not None else "not found (other GPUs work through Vulkan or Metal; CPU only is slow)")
+  free_gb = shutil.disk_usage(paths.nock_home().anchor or "/").free / 1e9
+  check("free disk", free_gb > 5, f"{free_gb:.0f} GB")
+  return 0 if ready else 1
+
+
+def start_server_factory(skill):
+  settings = paths.load_settings()
+  server_binary = engine.locate_server(settings)
+  installed = settings.get("models", {}).get(skill)
+  if not server_binary or not installed:
+    sys.exit(f"{models.MODELS[skill]['title']} is not installed yet. Run: nock add {skill}")
+  if settings.get("uses_gpu") is False and settings.get("backend") != "cpu":
+    warn_no_gpu(settings)
+  log_path = paths.nock_home() / "logs" / f"{skill}-server.log"
+  default_context = models.MODELS[skill].get("context")
+
+  def start(context=None):
+    return engine.LocalServer(server_binary, installed["model"], installed.get("mmproj"), context or default_context,
+                              99, log_path, settings.get("device_id"))
+
+  return start
+
+
+def command_video(args):
+  kind = {"summarize": "summary", "ask": "ask", "find": "moment"}[args.action]
+  question = video.SUMMARY_QUESTION if kind == "summary" else args.question
+  frames = video.choose_frames(args.frames)
+  print(video.run(args.source, question, kind, frames, start_server_factory("video")))
+
+
+def command_code(args):
+  code.run(" ".join(args.prompt).strip(), start_server_factory("coder"))
+
+
+def command_fix(args):
+  need = models.MODELS["agent"]["min_gpu_mb"]
+  memory = video.gpu_memory_mb()
+  if memory is not None and memory < need and not args.force:
+    sys.exit(f"Nock Agent 31B needs a GPU with about {need // 1000} GB; this one has {memory // 1000} GB. "
+             "Use --force to try anyway (it will be very slow).")
+  if memory is None:
+    print(f"Note: Nock Agent 31B needs about {need // 1000} GB of GPU memory (a Mac with 32 GB or more may work; not tested).", file=sys.stderr)
+  approve = agent.Approver(always=args.yes)
+  print(agent.run(args.project, args.task, approve, start_server_factory("agent")))
+
+
+def command_serve(args):
+  installed = paths.load_settings().get("models", {})
+  if not installed:
+    sys.exit("No skills installed yet. Run: nock add coder")
+  memory = video.gpu_memory_mb()
+  starters = {}
+  for skill in installed:
+    need = models.MODELS.get(skill, {}).get("min_gpu_mb")
+    if need and memory is not None and memory < need and not args.force:
+      print(f"Leaving out {skill}: it needs about {need // 1000} GB of GPU memory (--force to include it).", file=sys.stderr)
+      continue
+    start = start_server_factory(skill)
+    starters[skill] = make_starter(skill, start)
+  if not starters:
+    sys.exit("None of the installed skills fits this GPU. Run: nock add coder")
+  serve.run(starters, args.port)
+
+
+def make_starter(skill, start):
+  # Video has no fixed context in MODELS: its size follows the frame count, and an app can send up to 64 frames.
+  context = models.MODELS[skill].get("context") or video.context_for(video.MAX_FRAMES)
+  return lambda: start(context)
+
+
+def build_parser():
+  parser = argparse.ArgumentParser(prog="nock", description="Open models that live on your machine.")
+  parser.add_argument("--version", action="version", version=f"nock {__version__}")
+  commands = parser.add_subparsers(dest="command", required=True)
+
+  add = commands.add_parser("add", help="add a skill: coder (1 GB), video (3.4 GB), agent (18.7 GB)")
+  add.add_argument("skills", nargs="+", choices=SKILLS)
+  add.add_argument("--backend", choices=engine.BACKENDS, help="llama.cpp build: vulkan (default), cuda, cpu, metal (Mac)")
+  add.set_defaults(handler=command_add)
+
+  remove = commands.add_parser("remove", help="delete a skill's files")
+  remove.add_argument("skills", nargs="+", choices=SKILLS)
+  remove.set_defaults(handler=command_remove)
+
+  listing = commands.add_parser("list", help="show the skills and what is installed")
+  listing.set_defaults(handler=command_list)
+
+  setup = commands.add_parser("setup", help="install the engine and Nock Video (or the skills you name)")
+  setup.add_argument("skills", nargs="*", choices=SKILLS)
+  setup.add_argument("--backend", choices=engine.BACKENDS, help="llama.cpp build: vulkan (default), cuda, cpu, metal (Mac)")
+  setup.set_defaults(handler=command_setup)
+
+  doctor = commands.add_parser("doctor", help="check that everything is in place")
+  doctor.set_defaults(handler=command_doctor)
+
+  code_parser = commands.add_parser("code", help="ask Nock Coder; no question starts a chat; pipe a file or log in")
+  code_parser.add_argument("prompt", nargs="*")
+  code_parser.set_defaults(handler=command_code)
+
+  video_parser = commands.add_parser("video", help="ask about a video file or link, offline")
+  actions = video_parser.add_subparsers(dest="action", required=True)
+  summarize = actions.add_parser("summarize", help="a few short points about the video")
+  summarize.add_argument("source", help="video file or link")
+  ask = actions.add_parser("ask", help="a question about what the video shows")
+  ask.add_argument("source", help="video file or link")
+  ask.add_argument("question")
+  find = actions.add_parser("find", help="when does something happen")
+  find.add_argument("source", help="video file or link")
+  find.add_argument("question", help='what to find, for example "the error message shows up"')
+  for action in (summarize, ask, find):
+    action.add_argument("--frames", type=int, help=f"frames to read (default {video.MAX_FRAMES}, "
+                                                   f"{video.LOW_MEMORY_FRAMES} on GPUs under 6 GB)")
+  video_parser.set_defaults(handler=command_video)
+
+  serve_parser = commands.add_parser("serve", help="one local OpenAI-style address for your apps (this machine only)")
+  serve_parser.add_argument("--port", type=int, default=serve.DEFAULT_PORT)
+  serve_parser.add_argument("--force", action="store_true", help="include skills that may not fit this GPU")
+  serve_parser.set_defaults(handler=command_serve)
+
+  fix = commands.add_parser("fix", help="Nock Agent (experimental, big GPU): fix an issue in a project folder")
+  fix.add_argument("project", help="the project folder")
+  fix.add_argument("task", help='what to fix, for example "the total is off by one"')
+  fix.add_argument("--yes", action="store_true", help="approve every edit and command without asking")
+  fix.add_argument("--force", action="store_true", help="start even on a GPU that is too small")
+  fix.set_defaults(handler=command_fix)
+  return parser
+
+
+def main(argv=None):
+  args = build_parser().parse_args(argv)
+  try:
+    result = args.handler(args)
+  except KeyboardInterrupt:
+    sys.exit("Stopped.")
+  except (engine.EngineError, models.ModelError, video.VideoError, ValueError) as error:
+    sys.exit(f"nock: {error}")
+  except subprocess.CalledProcessError as error:
+    sys.exit(f"nock: {error.cmd[0]} failed (exit code {error.returncode})")
+  except urllib.error.URLError as error:
+    sys.exit(f"nock: network error: {error.reason}")
+  sys.exit(result or 0)
+
+
+if __name__ == "__main__":
+  main()

nock_cli-0.3.0/src/nock_cli/serve.py

@@ -0,0 +1,181 @@
+"""`nock serve`: one local, OpenAI-compatible address for every installed skill.
+
+Editors and tools point at http://127.0.0.1:8080/v1 and choose a model by name. The right llama-server starts on
+first use; only one model is in memory at a time, so asking for another one swaps them. Everything stays on this
+machine: it binds to 127.0.0.1, and refuses requests from websites (a page you visit could otherwise call it).
+"""
+
+import json
+import threading
+import urllib.error
+import urllib.request
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+from nock_cli import __version__
+from nock_cli.code import SYSTEM_PROMPT as CODER_SYSTEM_PROMPT
+
+DEFAULT_PORT = 8080
+MODEL_NAMES = {"nock-coder": "coder", "nock-video": "video", "nock-agent": "agent"}
+SYSTEM_PROMPTS = {"coder": CODER_SYSTEM_PROMPT}  # same answers as `nock code`, unless the app sends its own
+MAX_BODY_BYTES = 64 * 1024 * 1024  # room for many video frames as images
+UPSTREAM_TIMEOUT = 1800
+LOCAL_HOSTS = ("127.0.0.1", "localhost", "[::1]")
+
+
+def is_local_host(value):
+  """True for 127.0.0.1, localhost and [::1], with or without a port."""
+  if not value:
+    return False
+  value = value.split("://", 1)[-1].split("/", 1)[0].lower()
+  host = value if value.startswith("[") and value.endswith("]") else value.rsplit(":", 1)[0] if ":" in value else value
+  return host in LOCAL_HOSTS
+
+
+def allowed_request(headers):
+  """Blocks other websites (Origin) and DNS rebinding (Host). Apps and scripts send no Origin and pass."""
+  if not is_local_host(headers.get("Host")):
+    return False
+  origin = headers.get("Origin")
+  return origin is None or is_local_host(origin)
+
+
+def prepare_body(body, skill):
+  """Adds the skill's system prompt when the app sent none. The app's other settings pass through unchanged."""
+  prompt = SYSTEM_PROMPTS.get(skill)
+  messages = body.get("messages")
+  if prompt and isinstance(messages, list) and not any(isinstance(m, dict) and m.get("role") == "system" for m in messages):
+    body = {**body, "messages": [{"role": "system", "content": prompt}, *messages]}
+  return body
+
+
+class ModelRouter:
+  """Keeps at most one llama-server running. `starters` maps a skill to a function returning an unstarted server."""
+
+  def __init__(self, starters):
+    self.starters = starters
+    self.lock = threading.Lock()  # held for a whole request: one answer at a time, and no swap in the middle
+    self.current_skill = None
+    self.current_server = None
+
+  def load(self, skill):
+    """Call with the lock held. Returns the running server for this skill."""
+    if self.current_skill != skill:
+      self.stop()
+      server = self.starters[skill]()
+      server.__enter__()
+      self.current_skill, self.current_server = skill, server
+    return self.current_server
+
+  def stop(self):
+    if self.current_server:
+      self.current_server.__exit__(None, None, None)
+    self.current_skill, self.current_server = None, None
+
+
+def make_handler(router):
+  class Handler(BaseHTTPRequestHandler):
+    server_version = f"nock/{__version__}"
+    protocol_version = "HTTP/1.1"
+
+    def log_message(self, *args):
+      pass  # questions and answers are never logged
+
+    def send_json(self, status, payload):
+      data = json.dumps(payload).encode()
+      self.send_response(status)
+      self.send_header("Content-Type", "application/json")
+      self.send_header("Content-Length", str(len(data)))
+      self.end_headers()
+      self.wfile.write(data)
+
+    def send_error_json(self, status, message):
+      self.send_json(status, {"error": {"message": message, "type": "nock_error"}})
+
+    def do_OPTIONS(self):
+      # No CORS headers: a browser's preflight from a website fails here.
+      self.send_response(403)
+      self.send_header("Content-Length", "0")
+      self.end_headers()
+
+    def do_GET(self):
+      if not allowed_request(self.headers):
+        return self.send_error_json(403, "Requests from websites are not allowed.")
+      if self.path == "/health":
+        return self.send_json(200, {"status": "ok", "loaded": router.current_skill})
+      if self.path == "/v1/models":
+        names = [name for name, skill in MODEL_NAMES.items() if skill in router.starters]
+        return self.send_json(200, {"object": "list", "data": [{"id": name, "object": "model", "owned_by": "nockai"} for name in names]})
+      self.send_error_json(404, "Not found. Try /v1/models or /v1/chat/completions.")
+
+    def do_POST(self):
+      if not allowed_request(self.headers):
+        return self.send_error_json(403, "Requests from websites are not allowed.")
+      if self.path != "/v1/chat/completions":
+        return self.send_error_json(404, "Only /v1/chat/completions is available.")
+      length = int(self.headers.get("Content-Length") or 0)
+      if length > MAX_BODY_BYTES:
+        return self.send_error_json(413, "Request is too large.")
+      try:
+        body = json.loads(self.rfile.read(length) or b"{}")
+      except ValueError:
+        return self.send_error_json(400, "The request body is not valid JSON.")
+      skill = MODEL_NAMES.get(body.get("model"))
+      if not skill:
+        return self.send_error_json(400, f"Unknown model. Use one of: {', '.join(MODEL_NAMES)}.")
+      if skill not in router.starters:
+        return self.send_error_json(404, f"{body['model']} is not installed. Run: nock add {skill}")
+      with router.lock:
+        try:
+          server = router.load(skill)
+        except Exception as error:
+          router.stop()
+          return self.send_error_json(503, f"Could not start {body['model']}: {error}")
+        self.forward(server.url, prepare_body(body, skill))
+
+    def forward(self, upstream_url, body):
+      request = urllib.request.Request(f"{upstream_url}/v1/chat/completions", data=json.dumps(body).encode(),
+                                       headers={"Content-Type": "application/json"})
+      try:
+        upstream = urllib.request.urlopen(request, timeout=UPSTREAM_TIMEOUT)
+      except urllib.error.HTTPError as error:
+        upstream = error  # the engine's own error, passed on with its status
+      except OSError as error:
+        return self.send_error_json(502, f"The engine did not answer: {error}")
+      with upstream:
+        self.send_response(upstream.status)
+        self.send_header("Content-Type", upstream.headers.get("Content-Type", "application/json"))
+        # Chunked so streamed answers reach the app word by word.
+        self.send_header("Transfer-Encoding", "chunked")
+        self.end_headers()
+        while True:
+          chunk = upstream.read1(65536)
+          if not chunk:
+            break
+          self.wfile.write(f"{len(chunk):X}\r\n".encode() + chunk + b"\r\n")
+          self.wfile.flush()
+        self.wfile.write(b"0\r\n\r\n")
+
+  return Handler
+
+
+def create_server(starters, port=DEFAULT_PORT):
+  router = ModelRouter(starters)
+  httpd = ThreadingHTTPServer(("127.0.0.1", port), make_handler(router))
+  return httpd, router
+
+
+def run(starters, port=DEFAULT_PORT):
+  try:
+    httpd, router = create_server(starters, port)
+  except OSError:
+    raise ValueError(f"Port {port} is already in use. Try: nock serve --port {port + 1}")
+  print(f"nock serve: http://127.0.0.1:{port}/v1  (this machine only)")
+  print(f"Models: {', '.join(name for name, skill in MODEL_NAMES.items() if skill in starters)}")
+  print("In your app: base URL as above, any API key, model as listed. Ctrl+C to stop.")
+  try:
+    httpd.serve_forever()
+  except KeyboardInterrupt:
+    pass
+  finally:
+    router.stop()
+    httpd.server_close()

{nock_cli-0.2.0 → nock_cli-0.3.0}/src/nock_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nock-cli
-Version: 0.2.0
+Version: 0.3.0
 Summary: Open models that live on your machine: code, video and a coding agent, offline.
 Author: NockAI
 License-Expression: Apache-2.0
@@ -101,6 +101,23 @@ and edited a test file 0 times. It works on your own files, so **every edit is s
 before it runs, and each one needs your yes**. `--yes` skips the questions; use it only in a copy or a git branch.
 It refuses to start on GPUs under about 20 GB (`--force` to try anyway, very slowly).
 
+## Use it from your apps: `nock serve`
+
+```bash
+nock serve
+```
+
+One local, OpenAI-compatible address for every skill you have installed. In your editor or tool, set the base URL to
+`http://127.0.0.1:8080/v1`, any API key, and the model to `nock-coder`, `nock-video` or `nock-agent`.
+
+```bash
+curl http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json"   -d '{"model": "nock-coder", "messages": [{"role": "user", "content": "regex for an EVM address"}]}'
+```
+
+The model starts on first use, and one model is in memory at a time, so asking for another swaps them (it can take
+a while). Streaming works. It listens on this machine only, refuses requests from websites, and logs no questions or
+answers. `--port` changes the port.
+
 ## Speed and GPUs
 
 `nock doctor` shows which device the engine uses. If it says **software device** or **no GPU device**, the

{nock_cli-0.2.0 → nock_cli-0.3.0}/src/nock_cli.egg-info/SOURCES.txt

@@ -11,6 +11,7 @@ src/nock_cli/engine.py
 src/nock_cli/main.py
 src/nock_cli/models.py
 src/nock_cli/paths.py
+src/nock_cli/serve.py
 src/nock_cli/video.py
 src/nock_cli.egg-info/PKG-INFO
 src/nock_cli.egg-info/SOURCES.txt

{nock_cli-0.2.0 → nock_cli-0.3.0}/tests/test_cli.py

@@ -6,12 +6,14 @@ import tarfile
 import tempfile
 import threading
 import unittest
+import urllib.error
+import urllib.request
 import zipfile
 from contextlib import contextmanager
 from http.server import BaseHTTPRequestHandler, HTTPServer
 from pathlib import Path
 
-from nock_cli import agent, code, engine, main, models, video
+from nock_cli import agent, code, engine, main, models, serve, video
 
 
 class FrameTimesTest(unittest.TestCase):
@@ -203,6 +205,86 @@ class AgentTest(unittest.TestCase):
     self.assertIn("outside the project", output)
 
 
+class FakeEngine:
+  """Stands in for one llama-server: records its requests and whether it is running."""
+
+  def __init__(self, name, log):
+    self.name, self.log = name, log
+    self.fake = None
+
+  def __enter__(self):
+    self.log.append(f"start {self.name}")
+    self.fake = FakeModel([{"role": "assistant", "content": f"answer from {self.name}"}] * 5)
+    self.url = self.fake.url
+    return self
+
+  def __exit__(self, *args):
+    self.log.append(f"stop {self.name}")
+    self.fake.close()
+
+
+class ServeTest(unittest.TestCase):
+  def setUp(self):
+    self.log = []
+    self.engines = {}
+
+    def starter(name):
+      def start():
+        self.engines[name] = FakeEngine(name, self.log)
+        return self.engines[name]
+      return start
+
+    self.httpd, self.router = serve.create_server({"coder": starter("coder"), "video": starter("video")}, 0)
+    self.port = self.httpd.server_address[1]
+    threading.Thread(target=self.httpd.serve_forever, daemon=True).start()
+
+  def tearDown(self):
+    self.router.stop()
+    self.httpd.shutdown()
+    self.httpd.server_close()
+
+  def call(self, path, body=None, headers=None):
+    data = json.dumps(body).encode() if body is not None else None
+    request = urllib.request.Request(f"http://127.0.0.1:{self.port}{path}", data=data, headers=headers or {})
+    try:
+      with urllib.request.urlopen(request, timeout=10) as response:
+        return response.status, json.load(response)
+    except urllib.error.HTTPError as error:
+      return error.code, json.load(error)
+
+  def ask(self, model, **extra):
+    return self.call("/v1/chat/completions", {"model": model, "messages": [{"role": "user", "content": "hi"}], **extra})
+
+  def test_lists_installed_models_only(self):
+    _, listing = self.call("/v1/models")
+    self.assertEqual([m["id"] for m in listing["data"]], ["nock-coder", "nock-video"])
+
+  def test_routes_and_swaps_one_model_at_a_time(self):
+    self.assertEqual(self.ask("nock-coder")[1]["choices"][0]["message"]["content"], "answer from coder")
+    self.ask("nock-coder")
+    self.assertEqual(self.ask("nock-video")[1]["choices"][0]["message"]["content"], "answer from video")
+    self.assertEqual(self.log, ["start coder", "stop coder", "start video"])
+
+  def test_adds_coder_prompt_unless_the_app_sends_one(self):
+    self.ask("nock-coder")
+    sent = self.engines["coder"].fake.requests[0]["messages"]
+    self.assertEqual(sent[0]["content"], code.SYSTEM_PROMPT)
+    own = serve.prepare_body({"messages": [{"role": "system", "content": "mine"}]}, "coder")
+    self.assertEqual(len(own["messages"]), 1)
+
+  def test_errors_are_clear(self):
+    self.assertEqual(self.ask("gpt-5")[0], 400)
+    status, reply = self.ask("nock-agent")
+    self.assertEqual(status, 404)
+    self.assertIn("nock add agent", reply["error"]["message"])
+
+  def test_websites_are_refused(self):
+    self.assertEqual(self.call("/v1/models", headers={"Origin": "https://evil.example"})[0], 403)
+    self.assertEqual(self.call("/v1/models", headers={"Host": "evil.example"})[0], 403)
+    self.assertEqual(self.call("/v1/models", headers={"Origin": "http://localhost:3000"})[0], 200)
+    self.assertEqual(self.log, [])
+
+
 class ParserTest(unittest.TestCase):
   def test_commands_parse(self):
     parser = main.build_parser()