horde-model-reference 0.9.2__tar.gz → 2.0.0__tar.gz

horde_model_reference-2.0.0/.CONTRIBUTING.md

@@ -0,0 +1,64 @@
+# Contributing
+
+All pull requests, large or small, from anyone are welcome!
+
+## Table of Contents
+
+- [Contributing](#contributing)
+    - [Table of Contents](#table-of-contents)
+    - [Environment Management](#environment-management)
+        - [First time setup](#first-time-setup)
+    - [Code Quality Tools](#code-quality-tools)
+    - [Code Style and System Design](#code-style-and-system-design)
+
+## Environment Management
+
+[uv](https://github.com/astral-sh/uv/) is the suggested python environment management tool.
+
+### First time setup
+
+- Install uv, as described [in the uv installation guide](https://github.com/astral-sh/uv/#installation).
+- `uv python install 3.10 3.11 3.12 3.13` # We currently support these versions.
+- `uv set 3.12` # Set to your desired default version.
+- `uv self update`
+- `uv sync --all-groups`
+- The `.venv/` directory will now be created with all project, development and documentation dependencies installed.
+    - Be sure to point your IDE to the python binary appropriate for your OS in this directory.
+
+## Code Quality Tools
+
+- [**tox**](https://tox.wiki/)
+    - Creates virtual environments for CI or local pytest runs.
+        - Note that the CI does not current execute calls to the production API by default.
+    - Run `tox list` or see `tox.ini` for more info
+- [**pre-commit**](https://pre-commit.com/)
+    - Creates virtual environments for formatting and linting tools
+    - Run `pre-commit run --all-files` or see `.pre-commit-config.yaml` for more info.
+
+> Note: Many of the tools below are run by `pre-commit` automatically, but can also be run manually if desired.
+
+- [**ruff**](https://github.com/astral-sh/ruff)
+    -  Also provides formatting (replacing `black`) with `ruff format . --fix`
+    - Linting rules from a wide variety of selectable rule sets
+    - See `pyproject.toml` for the rules used.
+    - See all rules (but not necessarily used in the project) [available in ruff here](https://beta.ruff.rs/docs/rules/).
+    - Run with `ruff check .`
+        - Note: When using autofixing (`ruff check . --fix`), changes may be made that require running black, which can then result in needing to run `ruff check . --fix` again.
+        - Consider running `black . && ruff check . --fix && black . && ruff check . --fix` to avoid this.
+- [**mypy**](https://mypy-lang.org/)
+    - Static type safety
+    - I recommending using the [mypy daemon](https://mypy.readthedocs.io/en/stable/mypy_daemon.html) instead of periodically running `pre-commit` (or `mypy` directly.).
+- [**pyright**](https://github.com/microsoft/pyright)
+    - Shipped with vscode by default (via the python extension `ms-python.vscode-pylance`)
+    - Suggested settings:
+        - `"python.analysis.typeCheckingMode": "off"`
+            - The pylance extension has certain opinionated type checking assertions which are clash with mypy.
+            - For example, overriding an optional field to be non-optional is considered by pylance to be a type error due to the field being invariant and the parent class potentially settings it to `None`. However, by convention in the SDK, this is a forbidden pattern.
+        - `"python.analysis.languageServerMode": "full"`
+        - `"python.testing.pytestEnabled": true`
+- [**tach**](https://github.com/gauge-sh/tach)
+    - Enforces internal namespace dependency constraints. This helps avoid circular dependencies and helps ensure implementations are in a logical place.
+
+## Code Style and System Design
+
+- See the [python haidra style guide](docs/haidra-assets/docs/meta/python.md) for standards on code style, system design, testing, and documentation.

horde_model_reference-2.0.0/.dockerignore

@@ -0,0 +1,78 @@
+# Git
+.git
+.gitignore
+.gitmodules
+.github
+
+# Python
+__pycache__
+*.py[cod]
+*$py.class
+*.so
+.Python
+*.egg-info
+dist
+.eggs
+.pytest_cache
+.mypy_cache
+.ruff_cache
+.coverage
+.tox
+.nox
+
+# Virtual environments
+.venv
+venv
+ENV
+env
+
+# IDE
+.vscode
+.idea
+*.swp
+*.swo
+*~
+.DS_Store
+
+# Documentation
+docs
+*.md
+README.md
+CONTRIBUTING.md
+CLAUDE.md
+DEPLOYMENT.md
+LICENSE
+mkdocs.yml
+
+# Test files
+tests
+*.test.py
+test_*.py
+
+# Development files
+.env
+.env.*
+!.env.example
+docker-compose*.yml
+!docker-compose.yml
+.pre-commit-config.yaml
+
+# Logs
+logs
+*.log
+
+# Models (exclude from build context)
+models
+
+# Temporary files
+*.tmp
+*.bak
+key.pem
+
+# CI/CD
+.travis.yml
+.gitlab-ci.yml
+
+# Other
+schemas/*.json
+!schemas/*.schema.json

horde_model_reference-2.0.0/.env.example

@@ -0,0 +1,181 @@
+# Auto-generated example environment file for Horde Model Reference
+# This file is generated by the settings_doc package.
+# Do not edit this file manually.
+
+# Generated on (UTC): 2025-10-25 12:29:16
+
+# DEPLOYMENT QUICK START GUIDE
+
+# Choose your deployment scenario and uncomment the relevant settings below:
+
+# 1. REPLICA MODE (Default) - Client/Worker nodes
+#    - Uses GitHub or PRIMARY API to fetch model references
+#    - Set: HORDE_MODEL_REFERENCE_REPLICATE_MODE=REPLICA
+#    - Optionally set: HORDE_MODEL_REFERENCE_PRIMARY_API_URL=<your-primary-url>
+
+# 2. PRIMARY MODE - Single Worker (No Redis)
+#    - Authoritative source for model references
+#    - Set: HORDE_MODEL_REFERENCE_REPLICATE_MODE=PRIMARY
+#    - No Redis configuration needed
+#   - Ideal for: Development, testing, low-traffic deployments
+
+# 3. PRIMARY MODE - Multi-Worker (With Redis)
+#   - Authoritative source with distributed caching
+#   - Set: HORDE_MODEL_REFERENCE_REPLICATE_MODE=PRIMARY
+#   - Configure all HORDE_MODEL_REFERENCE_REDIS_* settings
+#   - Ideal for: Production, high-traffic deployments
+
+# For detailed deployment instructions, see:
+#   - Docker: docker-compose.yml (single-worker)
+#   - Docker + Redis: docker-compose.redis.yml (multi-worker)
+#   - Non-Docker: DEPLOYMENT.md
+
+
+################################################################################
+# Horde Model Reference Settings
+################################################################################
+
+# Indicates if copies of the model reference are canonical or replicated. Clients should always be replicas.
+# Possible values:
+#   `PRIMARY`, `REPLICA`
+# HORDE_MODEL_REFERENCE_REPLICATE_MODE=REPLICA
+
+# Whether to create the default model reference folders on initialization.
+# HORDE_MODEL_REFERENCE_MAKE_FOLDERS=False
+
+# The GitHub owner of the repository.
+# HORDE_MODEL_REFERENCE_IMAGE_GITHUB_REPO_OWNER=Haidra-Org
+
+# The name of the GitHub repository used for image model references.
+# HORDE_MODEL_REFERENCE_IMAGE_GITHUB_REPO_NAME=AI-Horde-image-model-reference
+
+# The GitHub branch of the repository.
+# HORDE_MODEL_REFERENCE_IMAGE_GITHUB_REPO_BRANCH=main
+
+# Settings for the GitHub proxy, if any.
+# HORDE_MODEL_REFERENCE_IMAGE_GITHUB_REPO_PROXY_SETTINGS=
+
+# The GitHub owner of the repository.
+# HORDE_MODEL_REFERENCE_TEXT_GITHUB_REPO_OWNER=Haidra-Org
+
+# The name of the GitHub repository used for text model references.
+# HORDE_MODEL_REFERENCE_TEXT_GITHUB_REPO_NAME=AI-Horde-text-model-reference
+
+# The GitHub branch of the repository.
+# HORDE_MODEL_REFERENCE_TEXT_GITHUB_REPO_BRANCH=main
+
+# Settings for the GitHub proxy, if any.
+# HORDE_MODEL_REFERENCE_TEXT_GITHUB_REPO_PROXY_SETTINGS=
+
+# The time-to-live for in memory caches of model reference files, in seconds.
+# HORDE_MODEL_REFERENCE_CACHE_TTL_SECONDS=60
+
+# The maximum number of attempts to retry downloading a legacy model reference file.
+# HORDE_MODEL_REFERENCE_LEGACY_DOWNLOAD_RETRY_MAX_ATTEMPTS=3
+
+# The backoff time in seconds between retry attempts when downloading a legacy model reference file.
+# HORDE_MODEL_REFERENCE_LEGACY_DOWNLOAD_RETRY_BACKOFF_SECONDS=2
+
+# Whether to use Redis for distributed caching. Only should be used in PRIMARY mode.
+# HORDE_MODEL_REFERENCE_REDIS_USE_REDIS=False
+
+# Redis connection URL. Format: redis://[:password]@host:port/db
+# HORDE_MODEL_REFERENCE_REDIS_URL=redis://localhost:6379/0
+
+# Connection pool size for Redis connections.
+# HORDE_MODEL_REFERENCE_REDIS_POOL_SIZE=10
+
+# Socket timeout in seconds for Redis operations.
+# HORDE_MODEL_REFERENCE_REDIS_SOCKET_TIMEOUT=5
+
+# Connection timeout in seconds when establishing Redis connection.
+# HORDE_MODEL_REFERENCE_REDIS_SOCKET_CONNECT_TIMEOUT=5
+
+# Maximum number of retry attempts for failed Redis operations.
+# HORDE_MODEL_REFERENCE_REDIS_RETRY_MAX_ATTEMPTS=3
+
+# Backoff time in seconds between retry attempts for Redis operations.
+# HORDE_MODEL_REFERENCE_REDIS_RETRY_BACKOFF_SECONDS=0.5
+
+# Prefix for all Redis keys to namespace model reference data.
+# HORDE_MODEL_REFERENCE_REDIS_KEY_PREFIX=horde:model_ref
+
+# TTL for cached entries in seconds. If None, uses cache_ttl_seconds from main settings.
+# HORDE_MODEL_REFERENCE_REDIS_TTL_SECONDS=
+
+# Enable pub/sub for cache invalidation across multiple PRIMARY workers.
+# HORDE_MODEL_REFERENCE_REDIS_USE_PUBSUB=True
+
+# URL of PRIMARY server API for REPLICA clients to fetch model references from. If None, REPLICA clients will only use GitHub. Example: https://stablehorde.net/api/model_references/
+# HORDE_MODEL_REFERENCE_PRIMARY_API_URL=https://stablehorde.net/api/model_references/
+
+# Timeout in seconds for HTTP requests to PRIMARY API.
+# HORDE_MODEL_REFERENCE_PRIMARY_API_TIMEOUT=10
+
+# Whether REPLICA clients should fallback to GitHub if PRIMARY API is unavailable.
+# HORDE_MODEL_REFERENCE_ENABLE_GITHUB_FALLBACK=True
+
+# Whether PRIMARY mode should seed from GitHub on first initialization if local files don't exist. Only used in PRIMARY mode. If True, will download and convert legacy references once on startup.
+# HORDE_MODEL_REFERENCE_GITHUB_SEED_ENABLED=False
+
+# Which format is the canonical source of truth. Controls which API has write access. 'v2' (default): v2 API has CRUD, v1 API is read-only (converts from v2 to legacy). 'legacy': v1 API has CRUD, v2 API is read-only (converts from legacy to v2).
+# Possible values:
+#   `legacy`, `v2`
+# HORDE_MODEL_REFERENCE_CANONICAL_FORMAT=v2
+
+
+
+################################################################################
+# AI Horde Client+Worker Settings
+################################################################################
+
+# The URL for this AI Horde instance. If more than one, additional URLs are in the field `alt_horde_urls`.
+# HORDE_URL=https://aihorde.net/api/
+
+# Alternative API endpoints for the AI Horde. These should all lead to the same logical AI Horde.
+# ALT_HORDE_URLS=[HttpUrl('https://stablehorde.net/api/')]
+
+# The API key used for authenticating requests to the AI Horde.
+# API_KEY=**********
+
+# The API endpoint for AI Horde ratings.
+# RATINGS_URL=https://ratings.aihorde.net/api/
+
+# The folder where application logs are stored.
+# LOGS_FOLDER=./logs
+
+# The folder where AI worker (or client) files are stored, most notably models and checkpoints.
+# AIWORKER_CACHE_HOME=./models
+
+# The hugging face home directory.
+# HF_HOME=~/.cache/huggingface
+
+# The standard XDG cache directory.
+# XDG_CACHE_HOME=~/.cache/
+
+
+
+################################################################################
+# Github Proxying Settings
+################################################################################
+
+# The base URL for a http(s) GitHub proxy. If None, no proxy is used. This is intended for users where github        may be blocked.
+# GITHUB_PROXY_URL_BASE=
+
+
+
+################################################################################
+# AI Horde CI Settings
+################################################################################
+
+# Indicates if any CI/CD pipeline is ongoing.
+# TESTS_ONGOING=False
+
+# Indicates if the hordelib CI/CD pipeline is ongoing.
+# HORDELIB_CI_ONGOING=False
+
+# Indicates if the AI Horde SDK is currently being tested.
+# HORDE_SDK_TESTING=False
+
+# Indicates if the AI Horde is currently being tested.
+# AI_HORDE_TESTING=False

horde_model_reference-2.0.0/.env.primary.example

@@ -0,0 +1,72 @@
+# Example environment configuration for PRIMARY mode deployment
+# Copy this to .env.primary and configure as needed
+
+# ============================================================================
+# PRIMARY Mode Configuration
+# ============================================================================
+
+# REQUIRED: Set to PRIMARY mode
+HORDE_MODEL_REFERENCE_REPLICATE_MODE=PRIMARY
+
+# Create folders automatically if they don't exist
+HORDE_MODEL_REFERENCE_MAKE_FOLDERS=true
+
+# Cache TTL in seconds (default: 60)
+HORDE_MODEL_REFERENCE_CACHE_TTL_SECONDS=60
+
+# ============================================================================
+# Data Format Configuration
+# ============================================================================
+
+# Which format is canonical (controls which API has write access)
+# - 'legacy': v1 API has CRUD, v2 API is read-only (converts from legacy to v2)
+# - 'v2': v2 API has CRUD, v1 API is read-only (converts from v2 to legacy)
+# Default is 'v2' but set to 'legacy' during transition period
+HORDE_MODEL_REFERENCE_CANONICAL_FORMAT=legacy
+
+# ============================================================================
+# GitHub Seeding (First-Time Setup)
+# ============================================================================
+
+# Enable GitHub seeding on first startup if local files don't exist
+# Set to 'true' for initial deployment, then change to 'false'
+HORDE_MODEL_REFERENCE_GITHUB_SEED_ENABLED=false
+
+# ============================================================================
+# Redis Configuration (Multi-Worker Deployments Only)
+# ============================================================================
+
+# Enable Redis for distributed caching (required for multi-worker setups)
+# HORDE_MODEL_REFERENCE_REDIS_USE_REDIS=true
+
+# Redis connection URL
+# HORDE_MODEL_REFERENCE_REDIS_URL=redis://redis:6379/0
+
+# Redis connection pool size
+# HORDE_MODEL_REFERENCE_REDIS_POOL_SIZE=10
+
+# Redis TTL (uses cache_ttl_seconds if not specified)
+# HORDE_MODEL_REFERENCE_REDIS_TTL_SECONDS=60
+
+# Enable pub/sub for cache invalidation across workers
+# HORDE_MODEL_REFERENCE_REDIS_USE_PUBSUB=true
+
+# ============================================================================
+# Data Directory
+# ============================================================================
+
+# Cache home directory (set by Docker, but can be overridden)
+# AIWORKER_CACHE_HOME=/data
+
+# ============================================================================
+# Optional: GitHub Repository Overrides (for testing/forks)
+# ============================================================================
+
+# Override default GitHub repositories (uncomment to use)
+# HORDE_MODEL_REFERENCE_IMAGE_GITHUB_REPO_OWNER=Haidra-Org
+# HORDE_MODEL_REFERENCE_IMAGE_GITHUB_REPO_NAME=AI-Horde-image-model-reference
+# HORDE_MODEL_REFERENCE_IMAGE_GITHUB_REPO_BRANCH=main
+
+# HORDE_MODEL_REFERENCE_TEXT_GITHUB_REPO_OWNER=Haidra-Org
+# HORDE_MODEL_REFERENCE_TEXT_GITHUB_REPO_NAME=AI-Horde-text-model-reference
+# HORDE_MODEL_REFERENCE_TEXT_GITHUB_REPO_BRANCH=main

horde_model_reference-2.0.0/.env.sync.example

@@ -0,0 +1,105 @@
+# ============================================================================
+# GitHub Sync Service Configuration
+# ============================================================================
+# Copy this file to .env.sync and configure for your environment
+# Load with: docker-compose --env-file .env.sync --profile sync up -d
+
+# ============================================================================
+# GitHub Authentication - Choose ONE method
+# ============================================================================
+
+# METHOD 1: Personal Access Token (simpler, for individual use)
+# Create at: https://github.com/settings/tokens
+# Required scopes: Contents (Read & Write), Pull Requests (Read & Write)
+# GITHUB_TOKEN=ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+
+# METHOD 2: GitHub App (recommended for production)
+# Required scopes: Contents (Read & Write), Pull Requests (Read & Write)
+# Setup instructions in SYNC_README.md
+GITHUB_APP_ID=1234567
+GITHUB_APP_INSTALLATION_ID=12345678
+
+# Private Key: Choose ONE option
+# Option A: Path to mounted PEM file (recommended for Docker)
+GITHUB_APP_PRIVATE_KEY_PATH=/app/github-app-key.pem
+
+# Option B: Inline key content (escape newlines as \n)
+# GITHUB_APP_PRIVATE_KEY=-----BEGIN RSA PRIVATE KEY-----\nMIIE...\n-----END RSA PRIVATE KEY-----
+
+# ============================================================================
+# PRIMARY API Configuration
+# ============================================================================
+
+# URL of your PRIMARY instance (required)
+# For Docker: use service name, e.g., http://horde-model-reference:19800
+# For local: use localhost, e.g., http://localhost:19800
+HORDE_GITHUB_SYNC_PRIMARY_API_URL=http://horde-model-reference:19800
+
+# API request timeout in seconds
+# HORDE_GITHUB_SYNC_PRIMARY_API_TIMEOUT=30
+
+# ============================================================================
+# GitHub Repository Configuration (Optional Overrides)
+# ============================================================================
+
+# Override default repositories (useful for testing with forks)
+# HORDE_MODEL_REFERENCE_IMAGE_GITHUB_REPO_OWNER=Haidra-Org
+# HORDE_MODEL_REFERENCE_IMAGE_GITHUB_REPO_NAME=AI-Horde-image-model-reference
+# HORDE_MODEL_REFERENCE_IMAGE_GITHUB_REPO_BRANCH=main
+
+# HORDE_MODEL_REFERENCE_TEXT_GITHUB_REPO_OWNER=Haidra-Org
+# HORDE_MODEL_REFERENCE_TEXT_GITHUB_REPO_NAME=AI-Horde-text-model-reference
+# HORDE_MODEL_REFERENCE_TEXT_GITHUB_REPO_BRANCH=main
+
+# ============================================================================
+# Sync Behavior
+# ============================================================================
+
+# Sync interval in seconds (default: 21600 = 6 hours)
+SYNC_INTERVAL=21600
+
+# Minimum number of changes required to create a PR
+# HORDE_GITHUB_SYNC_MIN_CHANGES_THRESHOLD=1
+
+# Comma-separated list of categories to sync (default: all)
+# HORDE_GITHUB_SYNC_CATEGORIES_TO_SYNC=image_generation,text_generation
+
+# Dry run mode - preview changes without creating PRs (true/false)
+# HORDE_GITHUB_SYNC_DRY_RUN=false
+
+# Enable verbose logging (true/false)
+# HORDE_GITHUB_SYNC_VERBOSE_LOGGING=false
+
+# ============================================================================
+# Pull Request Configuration
+# ============================================================================
+
+# Comma-separated list of GitHub usernames to assign as reviewers
+# HORDE_GITHUB_SYNC_PR_REVIEWERS=username1,username2
+
+# Comma-separated list of labels to apply to PRs
+# HORDE_GITHUB_SYNC_PR_LABELS=automated,sync,ready-for-review
+
+# GitHub team to auto-assign for review (format: org-name/team-name)
+# HORDE_GITHUB_SYNC_PR_AUTO_ASSIGN_TEAM=Haidra-Org/model-maintainers
+
+# ============================================================================
+# Git Commit Configuration
+# ============================================================================
+
+# Author/committer information for automated commits
+GIT_AUTHOR_NAME=Horde Model Reference Bot
+GIT_AUTHOR_EMAIL=bot@horde-model-reference.local
+GIT_COMMITTER_NAME=Horde Model Reference Bot
+GIT_COMMITTER_EMAIL=bot@horde-model-reference.local
+
+# ============================================================================
+# Advanced: Repository Clone Configuration
+# ============================================================================
+
+# Base directory for persistent repository clones (optional)
+# If set, repos will be cloned once and reused across sync runs
+# HORDE_GITHUB_SYNC_TARGET_CLONE_DIR=/data/github-repos
+
+# Temporary directory for git operations (optional)
+# HORDE_GITHUB_SYNC_SYNC_TEMP_DIR=/tmp/github-sync

{horde_model_reference-0.9.2 → horde_model_reference-2.0.0}/.github/workflows/codeql.yml

@@ -53,11 +53,11 @@ jobs:
 
     steps:
     - name: Checkout repository
-      uses: actions/checkout@v3
+      uses: actions/checkout@v4
 
     # Initializes the CodeQL tools for scanning.
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@v2
+      uses: github/codeql-action/init@v3
       with:
         languages: ${{ matrix.language }}
         # If you wish to specify custom queries, you can do so here or in a config file.
@@ -71,7 +71,7 @@ jobs:
     # Autobuild attempts to build any compiled languages (C/C++, C#, Go, Java, or Swift).
     # If this step fails, then you should remove it and run the build manually (see below)
     - name: Autobuild
-      uses: github/codeql-action/autobuild@v2
+      uses: github/codeql-action/autobuild@v3
 
     # ℹ️ Command-line programs to run using the OS shell.
     # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
@@ -84,6 +84,6 @@ jobs:
     #     ./location_of_script_within_repo/buildscript.sh
 
     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@v2
+      uses: github/codeql-action/analyze@v3
       with:
         category: "/language:${{matrix.language}}"