logit-classifier 0.1.0__tar.gz → 0.2.1__tar.gz

{logit_classifier-0.1.0 → logit_classifier-0.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: logit-classifier
-Version: 0.1.0
+Version: 0.2.1
 Summary: Local zero-shot classifier for text and images. Declares options, returns a calibrated probability for each, generates no text. Accepts the TypeSafe System One request format.
 Project-URL: Repository, https://github.com/Blakeem/logit-classifier
 Project-URL: Bug Tracker, https://github.com/Blakeem/logit-classifier/issues
@@ -88,8 +88,8 @@ beside your project instead, which is what the examples do.
 Config(models_dir=Path("models"))
 ```
 
-`LOGIT_MODELS_DIR` sets the same folder for every process, and `HF_HOME` moves the
-Hugging Face cache itself.
+`LOGIT_MODELS_DIR` sets the same folder for the service and for `Config.from_env()`.
+`HF_HOME` moves the Hugging Face cache itself.
 
 ## Models
 
@@ -98,8 +98,9 @@ Hugging Face cache itself.
 | reads images | yes | no |
 | better at | choice and score | noul |
 
-`Qwen3-VL-4B-Instruct` is the default. Set `LOGIT_MODEL_ID` to use the other one. Any Qwen
-chat model loads, and a model with no fitted temperature gets 2.5.
+`Qwen3-VL-4B-Instruct` is the default. Set `LOGIT_MODEL_ID` to use the other one in the
+service or through `Config.from_env()`. Any Qwen chat model loads, and a model with no
+fitted temperature gets 2.5.
 
 ## Python API
 
@@ -169,7 +170,7 @@ the model.
 
 ```json
 {
-  "model": "logit-classifier-0.1.0",
+  "model": "logit-classifier-0.2.1",
   "answers": {
     "department": {
       "type": "choice",
@@ -210,7 +211,7 @@ only in process.
   "questions": { "sky": { "type": "noul", "instructions": "This crop shows the night sky" } } }
 ```
 
-The image is encoded once for the entire request, so asking several questions about one
+The image is encoded once for the entire request. So asking several questions about one
 picture costs little more than asking one.
 
 A model with no vision tower rejects the request rather than ignoring the picture.
@@ -230,8 +231,9 @@ Each script runs on its own.
 
 ## Configuration
 
-Every setting reads from the environment at startup. `logit-classifier config` prints what
-they produce.
+The service and `logit-classifier config` read these variables at startup through
+`Config.from_env()`. Library code gets them by calling `Config.from_env()`, since `Config()`
+reads none of them. `logit-classifier config` prints what they produce.
 
 | Variable | Default | Effect |
 |---|---|---|
@@ -242,6 +244,7 @@ they produce.
 | `LOGIT_PRIOR_DEBIAS` | `1` | set to `0` to skip the label prior |
 | `LOGIT_BATCH_BRANCHES` | `1` | set to `0` for one forward pass per branch |
 | `LOGIT_SCORE_METHOD` | `joint` | set to `independent` to judge each level alone |
+| `LOGIT_PERMUTATIONS` | `1` | letterings averaged per question, each one adds branches |
 | `LOGIT_ABSTAIN` | `1` | set to `0` to drop the `none of these` label below 52 options |
 | `LOGIT_CALIBRATION_PATH` | `calibration.json` | where the service stores the running prior |
 
@@ -269,9 +272,10 @@ Nothing is sampled, so the same request returns bitwise identical logits.
 
 Batch composition and padding length both change the low bits of a bfloat16 forward pass,
 and both are pure functions of the request. So the same question asked inside two
-different requests can differ slightly. Set `LOGIT_BATCH_BRANCHES=0` to score each branch
-alone, which makes a question independent of the questions sent with it and costs one
-forward pass per branch.
+different requests can differ slightly. Scoring each branch alone makes a question
+independent of the questions sent with it and costs one forward pass per branch. The service
+turns it on with `LOGIT_BATCH_BRANCHES=0`. Library code passes `Config(batch_branches=False)`,
+and `ComfyClipBackend` takes `batch_branches=False` as a keyword.
 
 A forward pass needs several process-global torch settings held at known values. Torch
 exposes none of them as a call argument, so the backend sets them around each pass and
@@ -320,6 +324,24 @@ the render and label contracts and returns one token id per label.
 `logit_classifier.backends.hf.HFBackend` is the transformers implementation to read
 against.
 
+`ComfyClipBackend` is the backend over a Qwen3-VL text encoder that the workflow already
+loaded, such as the one Krea 2 uses. So no second model is loaded into VRAM.
+
+```python
+from logit_classifier import Classifier, Config, NoulQuestion, SystemOneRequest
+from logit_classifier.backends.comfy_clip import ComfyClipBackend
+
+classifier = Classifier(Config(), backend=ComfyClipBackend(clip))
+question = NoulQuestion(instructions="This image visibly contains a dragon")
+request = SystemOneRequest(state="", questions={"dragon": question})
+response, _ = classifier.classify(request, image=image)
+```
+
+The `image` keyword takes an image the host already decoded, such as a ComfyUI IMAGE of
+shape `[1, H, W, 3]`. An empty state asks about the image alone. Questions share one forward
+pass of up to 4,096 tokens, and the image counts toward that limit. Any text encoder other
+than Qwen3-VL raises `UnsupportedModelError`.
+
 ## Differences From Jev
 
 This project matches the System One request and response format and the documented limits.
@@ -329,8 +351,8 @@ Jev is trained for calibrated probabilities. This project reads them from a gene
 so the choices agree more often than the confidences do.
 
 Jev judges a score level without its number or its neighbours. This project judges all
-levels together by default. Set `LOGIT_SCORE_METHOD=independent` for the documented
-behavior.
+levels together by default. The service follows the documented behavior with
+`LOGIT_SCORE_METHOD=independent`, and library code with `Config(score_method="independent")`.
 
 Jev publishes status codes but no error body. The error shape here is our own.
 

{logit_classifier-0.1.0 → logit_classifier-0.2.1}/README.md

@@ -55,8 +55,8 @@ beside your project instead, which is what the examples do.
 Config(models_dir=Path("models"))
 ```
 
-`LOGIT_MODELS_DIR` sets the same folder for every process, and `HF_HOME` moves the
-Hugging Face cache itself.
+`LOGIT_MODELS_DIR` sets the same folder for the service and for `Config.from_env()`.
+`HF_HOME` moves the Hugging Face cache itself.
 
 ## Models
 
@@ -65,8 +65,9 @@ Hugging Face cache itself.
 | reads images | yes | no |
 | better at | choice and score | noul |
 
-`Qwen3-VL-4B-Instruct` is the default. Set `LOGIT_MODEL_ID` to use the other one. Any Qwen
-chat model loads, and a model with no fitted temperature gets 2.5.
+`Qwen3-VL-4B-Instruct` is the default. Set `LOGIT_MODEL_ID` to use the other one in the
+service or through `Config.from_env()`. Any Qwen chat model loads, and a model with no
+fitted temperature gets 2.5.
 
 ## Python API
 
@@ -136,7 +137,7 @@ the model.
 
 ```json
 {
-  "model": "logit-classifier-0.1.0",
+  "model": "logit-classifier-0.2.1",
   "answers": {
     "department": {
       "type": "choice",
@@ -177,7 +178,7 @@ only in process.
   "questions": { "sky": { "type": "noul", "instructions": "This crop shows the night sky" } } }
 ```
 
-The image is encoded once for the entire request, so asking several questions about one
+The image is encoded once for the entire request. So asking several questions about one
 picture costs little more than asking one.
 
 A model with no vision tower rejects the request rather than ignoring the picture.
@@ -197,8 +198,9 @@ Each script runs on its own.
 
 ## Configuration
 
-Every setting reads from the environment at startup. `logit-classifier config` prints what
-they produce.
+The service and `logit-classifier config` read these variables at startup through
+`Config.from_env()`. Library code gets them by calling `Config.from_env()`, since `Config()`
+reads none of them. `logit-classifier config` prints what they produce.
 
 | Variable | Default | Effect |
 |---|---|---|
@@ -209,6 +211,7 @@ they produce.
 | `LOGIT_PRIOR_DEBIAS` | `1` | set to `0` to skip the label prior |
 | `LOGIT_BATCH_BRANCHES` | `1` | set to `0` for one forward pass per branch |
 | `LOGIT_SCORE_METHOD` | `joint` | set to `independent` to judge each level alone |
+| `LOGIT_PERMUTATIONS` | `1` | letterings averaged per question, each one adds branches |
 | `LOGIT_ABSTAIN` | `1` | set to `0` to drop the `none of these` label below 52 options |
 | `LOGIT_CALIBRATION_PATH` | `calibration.json` | where the service stores the running prior |
 
@@ -236,9 +239,10 @@ Nothing is sampled, so the same request returns bitwise identical logits.
 
 Batch composition and padding length both change the low bits of a bfloat16 forward pass,
 and both are pure functions of the request. So the same question asked inside two
-different requests can differ slightly. Set `LOGIT_BATCH_BRANCHES=0` to score each branch
-alone, which makes a question independent of the questions sent with it and costs one
-forward pass per branch.
+different requests can differ slightly. Scoring each branch alone makes a question
+independent of the questions sent with it and costs one forward pass per branch. The service
+turns it on with `LOGIT_BATCH_BRANCHES=0`. Library code passes `Config(batch_branches=False)`,
+and `ComfyClipBackend` takes `batch_branches=False` as a keyword.
 
 A forward pass needs several process-global torch settings held at known values. Torch
 exposes none of them as a call argument, so the backend sets them around each pass and
@@ -287,6 +291,24 @@ the render and label contracts and returns one token id per label.
 `logit_classifier.backends.hf.HFBackend` is the transformers implementation to read
 against.
 
+`ComfyClipBackend` is the backend over a Qwen3-VL text encoder that the workflow already
+loaded, such as the one Krea 2 uses. So no second model is loaded into VRAM.
+
+```python
+from logit_classifier import Classifier, Config, NoulQuestion, SystemOneRequest
+from logit_classifier.backends.comfy_clip import ComfyClipBackend
+
+classifier = Classifier(Config(), backend=ComfyClipBackend(clip))
+question = NoulQuestion(instructions="This image visibly contains a dragon")
+request = SystemOneRequest(state="", questions={"dragon": question})
+response, _ = classifier.classify(request, image=image)
+```
+
+The `image` keyword takes an image the host already decoded, such as a ComfyUI IMAGE of
+shape `[1, H, W, 3]`. An empty state asks about the image alone. Questions share one forward
+pass of up to 4,096 tokens, and the image counts toward that limit. Any text encoder other
+than Qwen3-VL raises `UnsupportedModelError`.
+
 ## Differences From Jev
 
 This project matches the System One request and response format and the documented limits.
@@ -296,8 +318,8 @@ Jev is trained for calibrated probabilities. This project reads them from a gene
 so the choices agree more often than the confidences do.
 
 Jev judges a score level without its number or its neighbours. This project judges all
-levels together by default. Set `LOGIT_SCORE_METHOD=independent` for the documented
-behavior.
+levels together by default. The service follows the documented behavior with
+`LOGIT_SCORE_METHOD=independent`, and library code with `Config(score_method="independent")`.
 
 Jev publishes status codes but no error body. The error shape here is our own.
 

{logit_classifier-0.1.0 → logit_classifier-0.2.1}/examples/compare_models.py

@@ -19,7 +19,8 @@ from pathlib import Path
 from logit_classifier import Classifier, Config, load_model, parse_request
 
 # Weights land beside the project instead of in the global Hugging Face cache.
-# Set LOGIT_MODELS_DIR, or HF_HOME, to keep them somewhere shared across projects.
+# Edit MODELS_DIR to share one folder across projects.
+# Neither LOGIT_MODELS_DIR nor HF_HOME reaches this script.
 MODELS_DIR = Path(__file__).resolve().parent.parent / "models"
 
 MODELS = ["Qwen/Qwen3-VL-4B-Instruct", "Qwen/Qwen3-4B-Instruct-2507"]

{logit_classifier-0.1.0 → logit_classifier-0.2.1}/examples/images.py

@@ -23,7 +23,8 @@ from logit_classifier import (
 )
 
 # Weights land beside the project instead of in the global Hugging Face cache.
-# Set LOGIT_MODELS_DIR, or HF_HOME, to keep them somewhere shared across projects.
+# Edit MODELS_DIR to share one folder across projects.
+# Neither LOGIT_MODELS_DIR nor HF_HOME reaches this script.
 MODELS_DIR = Path(__file__).resolve().parent.parent / "models"
 QUESTIONS = {
     "subject": {

{logit_classifier-0.1.0 → logit_classifier-0.2.1}/examples/question_types.py

@@ -14,7 +14,8 @@ from pathlib import Path
 from logit_classifier import Classifier, Config, load_model, parse_request
 
 # Weights land beside the project instead of in the global Hugging Face cache.
-# Set LOGIT_MODELS_DIR, or HF_HOME, to keep them somewhere shared across projects.
+# Edit MODELS_DIR to share one folder across projects.
+# Neither LOGIT_MODELS_DIR nor HF_HOME reaches this script.
 MODELS_DIR = Path(__file__).resolve().parent.parent / "models"
 
 REVIEW = "Shipped two days late and the box was crushed, but the product itself works fine."

{logit_classifier-0.1.0 → logit_classifier-0.2.1}/examples/quickstart.py

@@ -16,7 +16,8 @@ from pathlib import Path
 from logit_classifier import Classifier, Config, load_model, parse_request
 
 # Weights land beside the project instead of in the global Hugging Face cache.
-# Set LOGIT_MODELS_DIR, or HF_HOME, to keep them somewhere shared across projects.
+# Edit MODELS_DIR to share one folder across projects.
+# Neither LOGIT_MODELS_DIR nor HF_HOME reaches this script.
 MODELS_DIR = Path(__file__).resolve().parent.parent / "models"
 
 SAMPLE_TEXT = "I have been trying to connect my Stripe account for 3 days and it keeps failing."

{logit_classifier-0.1.0 → logit_classifier-0.2.1}/pyproject.toml

@@ -136,7 +136,8 @@ show_error_codes = true
 [[tool.mypy.overrides]]
 # transformers publishes no complete stubs, and accelerate none at all. torch is
 # listed so the type gate still runs where the hf extra is absent, such as CI.
-module = ["transformers.*", "accelerate.*", "torch.*"]
+# The ComfyUI host supplies comfy, and neither this venv nor CI has it.
+module = ["transformers.*", "accelerate.*", "torch.*", "comfy.*"]
 ignore_missing_imports = true
 
 [[tool.mypy.overrides]]
@@ -148,6 +149,12 @@ module = ["logit_classifier.backends.hf"]
 disallow_untyped_calls = false
 disallow_untyped_decorators = false
 
+[[tool.mypy.overrides]]
+# torch leaves fp16_bf16_reduction_math_sdp_allowed unannotated, and a line ignore
+# would go unused, and so fail, where the hf extra is absent and torch is Any.
+module = ["logit_classifier.backends._torch_window"]
+disallow_untyped_calls = false
+
 [tool.pytest.ini_options]
 pythonpath = ["src"]
 testpaths = ["tests"]
@@ -159,7 +166,7 @@ requires = ["hatchling"]
 build-backend = "hatchling.build"
 
 [tool.hatch.version]
-path = "src/logit_classifier/__init__.py"
+path = "src/logit_classifier/_version.py"
 
 [tool.hatch.build.targets.wheel]
 packages = ["src/logit_classifier"]

{logit_classifier-0.1.0 → logit_classifier-0.2.1}/src/logit_classifier/__init__.py

@@ -9,10 +9,12 @@ with the `[hf]` extra and the HTTP service with `[service]`.
 
 from __future__ import annotations
 
+from ._version import __version__
 from .backends.base import (
     Backend,
     BackendContractError,
     BranchLogits,
+    UnsupportedModelError,
     VisionUnsupportedError,
     verify_backend,
 )
@@ -45,8 +47,6 @@ from .schema import (
 )
 from .vision import ImageError
 
-__version__ = "0.1.0"
-
 # The ComfyUI socket type a node pack declares for a loaded classifier. It lives
 # here so the library and the packs cannot drift apart on the spelling.
 COMFY_SOCKET_TYPE = "LOGIT_CLASSIFIER"
@@ -79,6 +79,7 @@ __all__ = [
     "ScoreQuestion",
     "SystemOneRequest",
     "SystemOneResponse",
+    "UnsupportedModelError",
     "Usage",
     "VisionUnsupportedError",
     "__version__",

logit_classifier-0.2.1/src/logit_classifier/_version.py

@@ -0,0 +1,3 @@
+"""The package version, apart from __init__ so config.py can read it without a cycle."""
+
+__version__ = "0.2.1"

logit_classifier-0.2.1/src/logit_classifier/backends/_torch_window.py

@@ -0,0 +1,96 @@
+"""The determinism window every backend's forward pass runs inside.
+
+torch is imported inside the window rather than at module scope, so a backend over a
+host's own model can import this without pulling torch into a core-only process.
+"""
+
+from __future__ import annotations
+
+import threading
+from collections.abc import Iterator
+from contextlib import contextmanager
+from typing import Any
+
+# Two windows open on two threads would interleave their saves, and the second to
+# exit would restore the pinned values rather than the host's. The service already
+# serialises on its own GPU lock, which is always taken before this one.
+_WINDOW = threading.RLock()
+
+
+@contextmanager
+def _determinism() -> Iterator[None]:
+    """Hold the torch globals that decide bit-exact reductions, for one forward pass.
+
+    Every alternative value buys speed by giving up bit-exactness, so none is tunable.
+    torch exposes none of them as a call argument, so scoping them means setting them
+    here and putting the host's values back after. A ComfyUI host sharing this process
+    keeps its own settings everywhere outside the block.
+
+    On the CUDA attention path only allow_bf16_reduced_precision_reduction moves a logit
+    on either shipped model. The rest are kept because cudnn picks convolution algorithms
+    by timing, which is specific to the card, and `ab_determinism_scope.py` measured one.
+
+    The sdp and fp16 accumulation settings are held for a third reason, that ComfyUI turns
+    each of them on and neither is reachable by that sweep. `ab_math_sdp_reduction.py`
+    measures the sdp one on the math backend.
+
+    Where torch has per-backend matmul slots, the coarse getter raises once a host has
+    set a slot directly, so only the raw slots are saved, pinned and put back there.
+    """
+    import torch
+
+    with _WINDOW:
+        saved_benchmark = torch.backends.cudnn.benchmark
+        saved_deterministic = torch.backends.cudnn.deterministic
+        saved_bf16 = torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction
+        # torch ships no stub for the mkldnn matmul slot, so it is reached by name.
+        mkldnn: Any = getattr(torch.backends.mkldnn, "matmul", None)
+        has_slots = hasattr(torch.backends.cuda.matmul, "fp32_precision") and hasattr(
+            mkldnn, "fp32_precision"
+        )
+        saved_matmul = None if has_slots else torch.get_float32_matmul_precision()
+        saved_cuda = torch.backends.cuda.matmul.fp32_precision if has_slots else None
+        saved_mkldnn = mkldnn.fp32_precision if has_slots else None
+        # ComfyUI turns this on at import, at comfy/model_management.py:569. It governs
+        # the math attention backend, which runs only when attention_backends is empty.
+        has_sdp = hasattr(torch.backends.cuda, "allow_fp16_bf16_reduction_math_sdp") and hasattr(
+            torch.backends.cuda, "fp16_bf16_reduction_math_sdp_allowed"
+        )
+        saved_sdp = bool(
+            has_sdp and torch.backends.cuda.fp16_bf16_reduction_math_sdp_allowed()
+        )
+        # A bare --fast turns this on, since comfy/cli_args.py then enables every
+        # PerformanceFeature. It is the fp16 sibling of the bf16 reduction above.
+        has_fp16_acc = hasattr(torch.backends.cuda.matmul, "allow_fp16_accumulation")
+        saved_fp16_acc = bool(
+            has_fp16_acc and torch.backends.cuda.matmul.allow_fp16_accumulation
+        )
+
+        # Pinning sits inside the try, so a setter that raises part way still restores the host.
+        try:
+            torch.backends.cudnn.benchmark = False
+            torch.backends.cudnn.deterministic = True
+            torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = False
+            if has_slots:
+                torch.backends.cuda.matmul.fp32_precision = "ieee"
+                mkldnn.fp32_precision = "ieee"
+            else:
+                torch.set_float32_matmul_precision("highest")
+            if has_sdp:
+                torch.backends.cuda.allow_fp16_bf16_reduction_math_sdp(False)
+            if has_fp16_acc:
+                torch.backends.cuda.matmul.allow_fp16_accumulation = False
+            yield
+        finally:
+            torch.backends.cudnn.benchmark = saved_benchmark
+            torch.backends.cudnn.deterministic = saved_deterministic
+            torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = saved_bf16
+            if has_slots:
+                torch.backends.cuda.matmul.fp32_precision = saved_cuda
+                mkldnn.fp32_precision = saved_mkldnn
+            elif saved_matmul is not None:
+                torch.set_float32_matmul_precision(saved_matmul)
+            if has_sdp:
+                torch.backends.cuda.allow_fp16_bf16_reduction_math_sdp(saved_sdp)
+            if has_fp16_acc:
+                torch.backends.cuda.matmul.allow_fp16_accumulation = saved_fp16_acc

{logit_classifier-0.1.0 → logit_classifier-0.2.1}/src/logit_classifier/backends/base.py

@@ -32,6 +32,10 @@ class BackendContractError(LogitClassifierError, RuntimeError):
     """A backend returned rows the port does not allow."""
 
 
+class UnsupportedModelError(LogitClassifierError, ValueError):
+    """A host passed a loaded model this backend cannot read a logit row from."""
+
+
 @dataclass(frozen=True)
 class BranchLogits:
     """Raw label logits for one branch, before any calibration."""