llmsetup 0.1.0__tar.gz

llmsetup-0.1.0/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 The llmsetup Authors
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

llmsetup-0.1.0/NOTICE

@@ -0,0 +1,7 @@
+llmsetup - catfish-1234/llmsetup
+Copyright 2026 The llmsetup Authors
+
+Licensed under the Apache License, Version 2.0. See LICENSE.
+
+Model weights downloaded or registered by llmsetup retain their publishers'
+separate licenses. The llmsetup software license does not license those weights.

llmsetup-0.1.0/PKG-INFO

@@ -0,0 +1,291 @@
+Metadata-Version: 2.4
+Name: llmsetup
+Version: 0.1.0
+Summary: Interview-driven setup assistant for local open-weight LLMs: picks the model, tunes the runtime, proves it works, and registers it with every local LLM app you use.
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/catfish-1234/llmsetup
+Project-URL: Source, https://github.com/catfish-1234/llmsetup
+Project-URL: Issues, https://github.com/catfish-1234/llmsetup/issues
+Project-URL: Documentation, https://github.com/catfish-1234/llmsetup#readme
+Keywords: llm,local-llm,gguf,ollama,llama.cpp,lm-studio,vllm,mlx
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: NOTICE
+Requires-Dist: PyYAML>=6.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Requires-Dist: pytest-cov>=4; extra == "dev"
+Requires-Dist: ruff==0.17.3; extra == "dev"
+Dynamic: license-file
+
+# llmsetup
+
+llmsetup sets up an open-weight LLM on your machine. It scans your hardware and
+installed runtimes first, then asks what you want to do, such as coding, chat,
+or running agents. You never need to enter hardware specs. It picks a model,
+downloads the weights, starts a server, tests the capabilities you need, and
+registers the working setup with your local apps.
+
+When something fails, it tries repairs and fallback options automatically.
+It only gives you a manual step after those options have failed.
+
+This is the Python project at
+[catfish-1234/llmsetup](https://github.com/catfish-1234/llmsetup),
+licensed under Apache-2.0. It was renamed from llmfit to avoid confusion with the
+unrelated [AlexsJones/llmfit](https://github.com/AlexsJones/llmfit) Rust project.
+See [the name change and migration notes](docs/POSITIONING.md).
+
+## Install
+
+Use Python 3.9 or newer, preferably a version that still receives security
+updates. PyYAML is the only runtime Python dependency. Install from this
+repository; the renamed package has not been published to PyPI. You need
+repository access to clone it.
+
+```bash
+git clone https://github.com/catfish-1234/llmsetup.git
+cd llmsetup
+python -m venv .venv
+source .venv/bin/activate
+python -m pip install .
+llmsetup --version
+llmsetup --help
+```
+
+On Windows, activate with `.venv\Scripts\Activate.ps1` in PowerShell. For
+source development, use `python -m pip install -e .`.
+
+You can make plans without a GPU or inference engine. Adapters cover Ollama,
+llama.cpp, LM Studio, vLLM, MLX, and TabbyAPI. Setup prefers an installed engine.
+If that engine is missing or fails, it tries compatible installed alternatives,
+including llama.cpp on CPU. It can install a pinned, checksum-verified llama.cpp
+build in its own data directory. Windows x86-64 machines with a detected GPU
+get the Vulkan build; CPU-only machines get the CPU build. CPU fallback is
+available when GPU attempts fail. It does not need administrator access.
+
+## Quickstart
+
+```bash
+# Inspect hardware, current memory use, and available runtimes.
+llmsetup detect
+
+# Scan the machine and ask about your goals.
+llmsetup setup
+
+# Skip the interview when you already know what you want.
+llmsetup plan --task coding --task agent --context 32768 --commercial --yes
+
+# Inspect tuning without installing an engine, downloading weights, or starting a server.
+llmsetup setup --dry-run --task coding --context 32768 --yes
+
+# Download, start, verify, and repair automatically.
+llmsetup setup --task coding --context 8192 --yes
+
+# Read the saved settings, checks, repair history, and launch command.
+llmsetup report
+```
+
+`--task` is repeatable, for example `--task coding --task agent`. `--context`
+sets the desired context in tokens; setup can shorten it to fit memory.
+Without an explicit context, coding and agent profiles aim for 32K with a
+16K floor, and warn if memory forces them below that floor.
+`--yes` skips goal questions. A dry run writes a local report but runs no
+verification, so it cannot tell you whether a model works on your hardware.
+Use `llmsetup detect` to inspect the scan, or `llmsetup runtimes` to inspect
+runtime availability. Neither command installs anything.
+
+Detection covers OS, CPU and core counts, RAM, GPUs and VRAM, and installed
+runtimes where they can be detected. A failed probe does not ask you for specs.
+It uses conservative assumptions, reports them, and proceeds. If the requested
+context cannot fit, planning tries shorter windows while keeping the required
+capabilities. Dedicated GPU plans use total VRAM with a 10% reserve. Current
+usage by other apps produces a warning and does not downgrade the model.
+If no GPU configuration fits, compatible runtimes try system RAM on CPU.
+
+You can also ask for a specific model by name during setup. llmsetup explains
+how that model fits your hardware, including the quantization and context it
+can hold and the speed to expect, and points out better-fitting alternatives
+when the fit is poor. If you keep the model, planning and recovery tune
+everything else around it instead of swapping it out. See
+[asking for a specific model](docs/AUTOMATIC_SETUP.md#asking-for-a-specific-model).
+For a scripted choice, use `llmsetup setup --model qwen3-8b --task chat --yes`.
+If that model cannot work after recovery, setup exits with a failure and keeps
+the requested model in the report. Alternatives require an explicit choice.
+
+Setup retries transient provisioning failures and server starts, replaces
+corrupt managed weights, tries cached weights when downloads are blocked, and
+can reduce context, change quantization, offload
+to CPU, swap models, or switch runtimes. Recovery has a fixed budget. The report
+shows the final settings and repair history. A blocked network, an unsupported
+portable runtime, or a requirement no available model can satisfy may still
+need your help. The final error gives the next step. See
+[automatic setup and recovery](docs/AUTOMATIC_SETUP.md) for the limits.
+
+Setup can download many gigabytes, install the local fallback engine, start a
+server, write reports, and register verified models with detected apps.
+`--no-register` skips app registration. `--no-repair` disables repairs, extra
+provisioning attempts, installation, and runtime fallback. Download transport
+retries still apply. A preferred `--runtime` may change during recovery.
+
+After verification, llmsetup stops servers it started. Shared Ollama and
+LM Studio daemons keep running. The report includes a command to start the model
+again; generated client snippets need a running endpoint.
+
+## Other commands
+
+```bash
+# Reuse a profile with automatic catalog updates disabled.
+llmsetup plan --answers examples/coding.yaml --json
+llmsetup setup --answers examples/coding.yaml --dry-run --report setup.md
+
+# Plan for CPU inference.
+llmsetup plan --cpu-only --task chat --context 8192 --yes
+
+# Check an existing OpenAI-compatible server. Give its root URL, without /v1.
+llmsetup verify --base-url http://127.0.0.1:11434 --model qwen3:8b \
+  --tools --checks health,completion,tool_calling --yes
+
+# Print the last report. Review it before sharing.
+llmsetup report
+
+# Inspect the catalog or add architecture details from a local GGUF.
+llmsetup catalog list
+llmsetup catalog show qwen3-8b
+llmsetup catalog add my-finetune --gguf ./my-finetune-Q4_K_M.gguf --publisher me
+
+# Open the local planning UI. Downloads are handed off to the CLI.
+llmsetup ui --no-browser
+```
+
+Profiles contain `tasks`, `clients`, `context_tokens`, `priority`,
+`concurrent_users`, capability flags such as `needs_tools`, and `offline_only`.
+Save interview answers with `llmsetup setup --save-profile profile.yaml`.
+Command-line options override matching profile values. Use
+`llmsetup <command> --help` for the available flags.
+
+JSON results go to stdout; progress and diagnostics go to stderr. Setup and
+verification return a nonzero exit code when a blocking check fails. A dry run
+can exit successfully without verifying a model.
+
+## Troubleshooting
+
+Start with `llmsetup detect`, `llmsetup runtimes`, and `llmsetup report`.
+Failures print the rerun command with your original options. Runtime failures
+also include the engine command or server log when available.
+
+| Symptom | What to do |
+| --- | --- |
+| Windows NVIDIA setup uses CPU unexpectedly | Run `llmsetup detect` to confirm the GPU. With recovery enabled, setup upgrades a managed CPU build to Vulkan before replanning. A CPU-only system build on PATH takes precedence; install a GPU-capable llama-server or remove that PATH entry. Review GPU layers in the report. |
+| VRAM is busy or setup runs out of memory | Close GPU-heavy apps and rerun the printed command. Recovery can reduce context, quantization, or GPU layers, then try CPU. Use `--context 8192` to request a smaller window explicitly. |
+| A reasoning model returns an empty first answer | Verification retries with more output-token room for thinking. Streaming recognizes `reasoning_content`, `reasoning`, and `thinking`. If the retry stays empty, the check fails and the report records it. |
+| Port is in use or a server exits | llama.cpp selects a free nearby port. A conflicting service on Ollama's port is reported. Stop the conflicting service, inspect the named log, then rerun the printed command. Failed owned servers are stopped before retrying. |
+| Download is partial, blocked, or times out | Valid `.part` files remain available for resume. Rerun with the same model and options. Check disk space, network access, model licence, and `HF_TOKEN` for gated models. A trusted mirror can be selected with `HF_ENDPOINT`. |
+| Runtime is missing or cannot load a native library | Recovery tries compatible installed engines and the managed fallback. If those fail, install a compatible runtime on PATH and rerun. `--no-repair` disables automatic installation and runtime fallback. |
+| Windows console uses cp1252 | Status, interview, and saved-report output replace unsupported characters; subprocess output is decoded as UTF-8 with replacement. Changing the code page is optional. |
+
+External operations are bounded. Ollama import has a 60-minute timeout and
+daemon startup has 45 seconds. llama.cpp startup allows 90 seconds plus 12
+seconds per GiB of weights. Model downloads use a 60-second socket timeout,
+a two-hour total budget per file, and at most four transport retries. Runtime
+archive downloads use a 30-second socket timeout and a five-minute total budget.
+An in-flight read can take up to the socket timeout before the total budget is
+checked again. A timed-out operation fails with a resume or rerun instruction.
+These limits apply per attempt; setup recovery can make further bounded attempts.
+
+## Catalogs and downloads
+
+The bundled YAML catalog is extended by files in `$LLMSETUP_HOME/catalog.d/`,
+then `LLMSETUP_CATALOG` and explicit `--catalog` files, in that order. Later
+entries with the same ID patch individual fields. Architecture values are
+marked `measured` or `estimated`; metadata in a downloaded GGUF can correct
+estimates.
+
+Automatic catalog refreshes are enabled by default and checked at most daily.
+They use this repository's raw catalog URL. If the source is unreachable, the
+bundled catalog stays available. A private repository needs an accessible
+mirror or local catalog source. Do not put access tokens in URLs or catalog files.
+
+```bash
+llmsetup catalog auto-update             # show source, cadence, and status
+llmsetup catalog auto-update --disable   # persistent opt-out
+llmsetup catalog auto-update --enable    # allow automatic checks again
+llmsetup catalog update                 # explicitly fetch now
+llmsetup catalog refresh --offline      # compare only local GGUFs
+```
+
+`LLMSETUP_AUTO_UPDATE=0` disables automatic catalog refreshes for one process.
+`LLMSETUP_CATALOG_URL` selects a trusted HTTP(S) or `file://` source. Disabling
+catalog refreshes, including through `offline_only`, does not stop a real
+`setup` from downloading weights. Use `plan` or `setup --dry-run` to inspect a
+configuration without provisioning.
+
+Model downloads resolve and save a Hugging Face commit revision. You can also
+pin revisions in a catalog overlay. Ollama imports the pinned GGUF and keeps
+its own copy. See [reproducibility notes](docs/REPRODUCIBILITY.md) for details.
+
+## Local files
+
+Default data lives in `$XDG_DATA_HOME/llmsetup` or `~/.local/share/llmsetup` on
+Unix, and `%LOCALAPPDATA%\llmsetup` on Windows. `LLMSETUP_HOME` overrides the
+location of profiles, catalog overlays, reports, and generated configs.
+`LLMSETUP_MODEL_STORE` overrides the shared model store.
+
+If you used the old name, you can point these variables at your existing
+directories. The rename does not move or delete data. Follow the
+[migration notes](docs/POSITIONING.md#migrating-an-existing-installation) before
+changing an existing installation.
+
+Weights can be large. App registration can replace colliding model links or
+generated files, so check the destinations and back up app configuration you
+maintain yourself.
+
+## Limits
+
+- Memory and speed estimates depend on the catalog, engine and driver versions,
+  free memory, and other running work. A predicted fit can still run out of memory.
+- Verification runs targeted smoke tests, not a general model-quality benchmark.
+  Passing a tool or JSON check does not prove reliability on every workload.
+- Runtime adapters depend on external command-line interfaces that can change.
+  Unit tests use fakes; hardware compatibility needs tests with the actual engine.
+- Repairs may shorten context, offload work to CPU, or change the model. Read the
+  repair journal and final settings before using the result.
+- `doctor` detects hardware again and replans for the saved model and profile.
+  It does not replay every previous low-level setting.
+- Byte counts and trusted transport do not prove model provenance. Review
+  publishers' licenses and sources. Model and software licenses are separate.
+- The UI is a local loopback development server without authentication. Keep it
+  local. Generated shell examples use POSIX syntax unless stated otherwise.
+
+## Development
+
+```bash
+python -m pip install -c requirements-ci.txt -e ".[dev]"
+python -m pytest
+python -m pytest --cov=llmsetup --cov-report=term-missing
+ruff check src/ tests/ scripts/
+```
+
+Tests use temporary app directories and fake runtimes. UI tests need local
+loopback sockets. No GPU or downloaded model is required.
+
+- [Architecture](docs/ARCHITECTURE.md) and [design notes](docs/DESIGN.md)
+- [Contributing](CONTRIBUTING.md) and [security reporting](SECURITY.md)
+- [Repository review and priorities](docs/REPOSITORY_REVIEW.md)
+- [Project name and migration](docs/POSITIONING.md)
+
+## License
+
+Apache-2.0; see [LICENSE](LICENSE). Downloaded model weights and external
+inference engines retain their own licenses.