foretoken 0.0.2__tar.gz → 0.0.3__tar.gz

{foretoken-0.0.2/foretoken.egg-info → foretoken-0.0.3}/PKG-INFO

@@ -1,17 +1,17 @@
 Metadata-Version: 2.4
 Name: foretoken
-Version: 0.0.2
+Version: 0.0.3
 Summary: Command-line tools for installing and operating Foretoken
 Author: Foretoken contributors
 License-Expression: Apache-2.0
-Requires-Python: >=3.10
+Requires-Python: >=3.11
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: packaging>=24.0
 Requires-Dist: PyYAML>=6.0
 Provides-Extra: bench
 Requires-Dist: datasets>=2.14; extra == "bench"
-Requires-Dist: evalscope>=1.10; extra == "bench"
+Requires-Dist: evalscope[perf]==1.11.1; extra == "bench"
 Requires-Dist: httpx>=0.27; extra == "bench"
 Requires-Dist: huggingface_hub>=0.23; extra == "bench"
 Requires-Dist: matplotlib>=3.7; extra == "bench"
@@ -19,9 +19,9 @@ Requires-Dist: numpy>=1.24; extra == "bench"
 Requires-Dist: openai>=1.40; extra == "bench"
 Requires-Dist: tqdm>=4.65; extra == "bench"
 Requires-Dist: transformers>=4.40; extra == "bench"
-Requires-Dist: vllm>=0.18; extra == "bench"
-Requires-Dist: wandb>=0.16; extra == "bench"
+Requires-Dist: wandb>=0.19.10; extra == "bench"
 Provides-Extra: dev
+Requires-Dist: grafana-foundation-sdk==1769699379!11.6.0; extra == "dev"
 Requires-Dist: pytest>=7.0; extra == "dev"
 Dynamic: license-file
 
@@ -34,13 +34,11 @@ SPDX-FileCopyrightText: Copyright contributors to the Foretoken project
 
 English | [简体中文](README_zh.md)
 
-The Foretoken command-line tool installs the shared Kubernetes platform, deploys model services from Kustomize configurations, reports serving readiness, resolves frontend endpoints, and runs benchmarks through one `foretoken` entry point.
-
-For a new cluster, start by installing the command-line tool. If `foretoken --version` already works, go straight to platform installation. If the cluster already has the Foretoken platform, start with model deployment.
+The Foretoken command-line tool installs the shared Kubernetes platform, deploys model services from Kustomize configurations, reports serving readiness, resolves frontend URLs, and runs benchmarks through one `foretoken` entry point.
 
 ## Before you start
 
-You need Python 3.10 or later, an active Kubernetes context, `kubectl`, and Helm. GPU nodes must already have their vendor driver and Kubernetes device plugin. Source installation also requires Docker and Make, plus either a local kind/k3d cluster or an OCI registry reachable by every target node.
+You need Python 3.11 or later, an active Kubernetes context, `kubectl`, and Helm. GPU nodes must already have their vendor driver and Kubernetes device plugin.
 
 ## Install the command-line tool
 
@@ -61,7 +59,7 @@ source .venv/bin/activate
 uv pip install foretoken
 ```
 
-This step only installs the `foretoken` command in the current Python environment; it does not change the Kubernetes cluster. Run `foretoken --version` to see the command-line tool and corresponding platform version.
+Run `foretoken --version` to check the installed command-line tool version.
 
 ## Install the Kubernetes platform
 
@@ -75,11 +73,13 @@ The default uses release images and local access through a `LoadBalancer` Servic
 foretoken install
 ```
 
-During installation, the command-line tool discovers Prometheus and accelerator metric exporters. It reuses compatible shared instances, installs managed Prometheus and NVIDIA DCGM Exporter releases when needed, and connects to the mxExporter already provided by a MetaX cluster. See [Observability](../observability/README.md) for monitoring selection and configuration.
+Installation selects the NVIDIA or MetaX runtime from the cluster's GPU resources. Explicit runtime settings in `--values` take precedence; in a mixed-GPU cluster, select a resource with `runtime.vllm.gpu.resourceName` or restrict the nodes with `runtime.vllm.gpu.nodeSelector`.
+
+Installation also sets up monitoring, reusing a Prometheus and GPU metrics exporter already in the cluster when they exist. See [Observability](../observability/README.md).
 
 ### Gateway mode
 
-The command-line tool creates a dedicated `GatewayClass` and `Gateway` only when the cluster runs Envoy Gateway:
+Gateway mode creates a dedicated `GatewayClass` and `Gateway`, installing Envoy Gateway if no compatible controller is available:
 
 ```bash
 foretoken install --frontend-mode gateway
@@ -98,7 +98,7 @@ Add `--gateway-section-name LISTENER` only when more than one listener matches.
 
 ### Current source
 
-Build Foretoken images from the current source tree and configure the platform to use them:
+Prepare the build tools listed in the [source deployment guide](../docs/custom-deployment.md), then build and install from the repository root:
 
 ```bash
 foretoken install -e .
@@ -115,21 +115,31 @@ Registry login authorizes the local image push. Private registries also need `im
 
 ### Installation options
 
-Repeatable `--values` files provide platform image, runtime, and hardware settings. Release and source installs record their mode in Helm metadata and cannot switch silently. Releases originally installed directly with Helm remain under their existing Helm lifecycle and are not adopted automatically.
+Repeatable `--values` files provide platform image, runtime, and hardware settings.
 
-### Persistent runtime cache
+Model services are reached through an IP address outside the cluster. k3d, k3s, and cloud clusters assign one automatically. Clusters built with kubeadm, RKE2, or kubespray have no address assignment by default, so installation there ends with `LoadBalancer support Not verified`. Give Foretoken a range of unused addresses in the nodes' subnet, confirmed with the cluster administrator, and it assigns them to services:
 
-Create one `RuntimeCache` in a workload namespace to let Foretoken provision and manage a shared cache PVC. Existing PVCs remain supported through `workload.cache.claimName`. See [Persistent Runtime Cache](../docs/development/runtime-cache.md).
+```yaml
+loadBalancer:
+  managedAddresses:
+    - 192.168.1.240-192.168.1.250
+```
+
+```bash
+foretoken install --values platform-values.yaml
+```
 
 ## Deploy and operate model services
 
-Deploy one frontend and all models rendered by a Kustomize root. The multi-model example needs up to four GPUs, 12 CPU cores, 100 GiB memory, and a default `ReadWriteMany` StorageClass with online expansion; use `examples/quickstart` for the smallest path.
+Run from the repository checkout prepared in the [Quick Start](../README.md). Deploy one frontend and all models rendered by a Kustomize root.
+
+See the [multi-model example](../examples/multi-model-quickstart/README.md) for resources and [model storage](../docs/model-storage.md) for directory or PVC configuration. Use `examples/quickstart` for a single model.
 
 ```bash
-foretoken deploy examples/multi-model-quickstart
+foretoken deploy examples/multi-model-quickstart --timeout 20m
 ```
 
-The command applies the configuration, reports each `FrontendService` and `ModelService` state when it changes, and exits when every service reports Ready for its current configuration. Change the default ten-minute deadline with `--timeout`.
+The command applies the configuration, reports service state changes, and exits when every service is Ready. Without `--timeout`, it waits up to ten minutes.
 
 Inspect the same deployment without applying it:
 
@@ -158,44 +168,17 @@ FORETOKEN_REQUEST_HOST="$(foretoken endpoint examples/multi-model-quickstart --h
 
 `--host` returns the host and optional port for direct access, or the configured routing hostname for an HTTP Gateway. `foretoken endpoint` waits for the LoadBalancer or Gateway address; use `foretoken deploy` to wait for the services to become ready.
 
-## Run benchmarks
-
-Install the optional benchmark dependencies with pip:
-
-```bash
-pip install 'foretoken[bench]'
-
-# For source installation from the repository:
-# pip install -e .
-# pip install -e '.[bench]'
-```
-
-Or install the benchmark dependencies in the activated uv environment:
+## Benchmark model services
 
-```bash
-uv pip install 'foretoken[bench]'
-```
-
-Then run the benchmark:
-
-```bash
-foretoken bench examples/multi-model-quickstart --model Qwen/Qwen3-0.6B
-```
-
-The command-line tool uses the active `kubectl` context and honors standard Kubernetes configuration such as `KUBECONFIG`.
+Use `foretoken bench` to measure model-service performance. Commands and examples are in [Model Service Benchmarks](../benchmarks/README.md).
 
 ## Clean up
 
-Delete the resources rendered by the same configuration:
+Delete the deployed services before uninstalling the platform:
 
 ```bash
 foretoken delete examples/multi-model-quickstart
-```
-
-The command waits for deletion and ignores resources that are already absent. After deleting all Foretoken services, remove the platform release:
-
-```bash
 foretoken uninstall
 ```
 
-The command preserves Foretoken CRDs and refuses to uninstall while user-owned services remain. It removes monitoring and Gateway resources managed by the command-line tool with the platform, while reused cluster components remain unchanged.
+Foretoken CRDs and reused cluster components are retained. Managed MetalLB is also retained while other services depend on it.

{foretoken-0.0.2 → foretoken-0.0.3}/README.md

@@ -21,50 +21,50 @@ If you only need to serve a single model on one GPU, using an inference engine s
 
 | Feature | Description | Status |
 |---|---|---|
-| Benchmarking | Performance benchmarks and parameter sweeps, correctness evaluation, and SLO simulation | In development |
+| [Benchmarking](benchmarks/README.md) | Measure model-service performance | In development |
 | Profiling | Use PyTorch Profiler and Nsight to identify compute, communication, and CPU/GPU bottlenecks | Planned |
 | Hardware support | Common interfaces for device capabilities, runtimes, communication, and metrics; see [MetaX deployment](docs/metax-deployment.md) | In development |
 | Request routing | Select instances based on load, queues, KV reuse, and service levels | Research |
 | Distributed inference | Aggregated serving, Prefill/Decode disaggregation, and WideEP parallelism | Research |
 | Control plane | Model services, replica management, autoscaling, updates, and failure recovery | In development |
-| [Observability](observability/README.md) | Collect runtime metrics, evaluate alerts, and profile CPU/GPU bottlenecks | In development |
+| [Observability](observability/README.md) | Collect service and accelerator metrics, evaluate alerts, and inspect the system Dashboard | In development |
 
 ## Quick Start
 
-This Quick Start requires Python 3.10+, Kubernetes with an expandable default `StorageClass`, `kubectl`, Helm, one GPU, and a working `LoadBalancer` (k3s ServiceLB is sufficient for k3d). See the [k3d guide](docs/k3d-deployment.md) for a single-machine test cluster.
+Start with a GPU-enabled Kubernetes cluster and Python 3.11+, `kubectl`, and Helm installed locally.
 
-### 1. Install the command-line tool
-
-Install the published command-line tool package:
+### 1. Get the examples and install the command-line tool
 
 ```bash
+git clone https://github.com/shiweijiezero/foretoken.git
+cd foretoken
 pip install foretoken
 
-# For source installation from the repository:
+# From a source checkout:
 # pip install -e .
 ```
 
 ### 2. Install the Kubernetes platform
 
-By default, installation uses the Foretoken images published on GHCR:
-
 ```bash
-# Release images:
+# Use release images from GHCR:
 foretoken install
 
-# Source installation from the repository:
+# Build and install from a source checkout:
 # foretoken install -e .
 ```
 
-This installs the Foretoken CRDs and controller in the `foretoken-platform` namespace and waits for the controller to become ready. The default mode exposes the frontend through a `LoadBalancer` Service. Source installation rebuilds the images and updates the cluster; to deploy the current source to a remote cluster, see the [source deployment guide](docs/custom-deployment.md).
+For deployment on MetaX GPUs, follow the [MetaX deployment guide](docs/metax-deployment.md).
+
+See the [source deployment guide](docs/custom-deployment.md) for build tools and remote clusters.
 
 ### 3. Deploy the Quick Start
 
 ```bash
-foretoken deploy examples/quickstart
+foretoken deploy examples/quickstart --timeout 20m
 ```
 
-This example deploys one frontend service, one `Qwen/Qwen3-0.6B` model replica, and an automatically expanding runtime cache PVC starting at 10 GiB. The workload requests one GPU, 8 CPU, and 52 GiB memory; allow additional capacity for the platform. See the [single-model example](examples/quickstart/README.md) for its resource configuration and [`examples/`](examples/) for more deployments.
+This example deploys one frontend service and one `Qwen/Qwen3-0.6B` model replica, requesting one GPU, 8 CPU, and 52 GiB memory. More deployments are available in [`examples/`](examples/).
 
 ### 4. Send a test request
 
@@ -77,24 +77,24 @@ curl --fail-with-body --no-buffer \
   -d '{"model":"Qwen/Qwen3-0.6B","messages":[{"role":"user","content":"Hello"}],"stream":true}'
 ```
 
-### 5. Run a benchmark
+### 5. Measure the model service
 
 ```bash
 pip install 'foretoken[bench]'
 
-# For source installation from the repository:
-# pip install -e .
+# From a source checkout:
 # pip install -e '.[bench]'
-foretoken bench examples/quickstart
+
+foretoken bench examples/quickstart --output local,wandb
 ```
 
-See [Benchmarking](benchmarks/README.md) for datasets, remote endpoints, result storage, and parameter sweeps.
+Run `wandb login` before first using W&B. See [Model Service Benchmarks](benchmarks/README.md) for more examples.
 
 ## Gateway Mode
 
 Gateway mode provides a shared entry point through Kubernetes Gateway and a hostname. It suits clusters that already use Gateway or manage external traffic centrally.
 
-Foretoken creates its default Gateway for Envoy Gateway. Install Envoy Gateway, then add the public hostname under `spec` in `examples/quickstart/frontend.yaml`:
+Add the public hostname under `spec` in `examples/quickstart/frontend.yaml`:
 
 ```yaml
 spec:
@@ -104,18 +104,13 @@ spec:
 Then run:
 
 ```bash
-# Install Envoy Gateway
-helm upgrade --install envoy-gateway \
-  oci://docker.io/envoyproxy/gateway-helm \
-  --namespace envoy-gateway-system \
-  --create-namespace \
-  --wait
-
 # Install the platform in Gateway mode
 foretoken install --frontend-mode gateway
+# For a source-installed platform:
+# foretoken install -e . --frontend-mode gateway
 
 # Deploy the Quick Start
-foretoken deploy examples/quickstart
+foretoken deploy examples/quickstart --timeout 20m
 
 # Resolve the Gateway address and request hostname
 FORETOKEN_FRONTEND_URL="$(foretoken endpoint examples/quickstart)"
@@ -129,7 +124,7 @@ curl --fail-with-body --no-buffer \
   -d '{"model":"Qwen/Qwen3-0.6B","messages":[{"role":"user","content":"Hello"}],"stream":true}'
 ```
 
-See the [command-line tool guide](cli/README.md) to reuse a Gateway from another controller, select a listener, or configure TLS.
+The command installs Envoy Gateway when needed. See the [command-line tool guide](cli/README.md) to reuse an existing Gateway or select a listener.
 
 ## Stop and Uninstall
 
@@ -143,6 +138,12 @@ foretoken uninstall
 
 The uninstall command preserves Foretoken CRDs and reused cluster components. It removes the platform and the monitoring or Gateway resources managed by the command-line tool.
 
+## Deployment Guides
+
+- [Source builds and private registries](docs/custom-deployment.md)
+- [Single-machine GPU clusters with k3d](docs/k3d-deployment.md)
+- [MetaX GPUs](docs/metax-deployment.md)
+
 ## Related Projects
 
 - [vLLM](https://github.com/vllm-project/vllm)
@@ -160,7 +161,7 @@ See [Contributing to Foretoken](CONTRIBUTING.md) for development principles, col
 Thank you to everyone who has contributed to Foretoken.
 
 <a href="https://github.com/shiweijiezero/foretoken/graphs/contributors">
-  <img src="https://contrib.rocks/image?repo=shiweijiezero/foretoken" alt="Foretoken contributors" />
+  <img src="https://contrib.rocks/image?repo=shiweijiezero/foretoken" width="256" alt="Foretoken contributors" />
 </a>
 
 ## License

foretoken-0.0.3/benchmarks/__init__.py

@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the Foretoken project
+
+"""Foretoken benchmark command family; each domain owns an independent lifecycle."""

foretoken-0.0.3/benchmarks/config/__init__.py

@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the Foretoken project
+
+"""Benchmark configuration types and command-line parsing."""