programasweights 0.1.0__tar.gz → 0.1.1__tar.gz

programasweights-0.1.1/.gitignore

@@ -0,0 +1,34 @@
+# Dependencies
+node_modules/
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+
+# Build outputs  
+dist/
+build/
+
+# Cache directories
+.vite/
+.cache/
+
+# Python
+__pycache__/
+*.pyc
+*.pyo
+.venv
+.env
+
+# System files
+.DS_Store
+Thumbs.db
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+
+# Compiled models (optional - comment out if you want to track them)
+web-app/backend/compiled_models/
+web-app/backend/temp/

programasweights-0.1.1/.readthedocs.yaml

@@ -0,0 +1,13 @@
+version: 2
+
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.11"   # avoid 3.13 for now; many packages lag support
+
+sphinx:
+  configuration: docs/conf.py
+
+python:
+  install:
+    - requirements: docs/requirements.txt

programasweights-0.1.1/PKG-INFO

@@ -0,0 +1,88 @@
+Metadata-Version: 2.4
+Name: programasweights
+Version: 0.1.1
+Summary: ProgramAsWeights: a minimal runtime where programs are weight blobs run by a fixed interpreter.
+Author-email: ProgramAsWeights <support@programasweights.com>
+License: MIT
+Keywords: neural-programs,nlp,prefix-tuning,runtime,transformers
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.8
+Requires-Dist: torch
+Requires-Dist: transformers
+Provides-Extra: data
+Requires-Dist: openai>=1.0.0; extra == 'data'
+Requires-Dist: tqdm>=4.66.0; extra == 'data'
+Provides-Extra: test
+Requires-Dist: pytest; extra == 'test'
+Provides-Extra: train
+Requires-Dist: accelerate>=0.33.0; extra == 'train'
+Requires-Dist: datasets>=2.19.0; extra == 'train'
+Requires-Dist: evaluate>=0.4.2; extra == 'train'
+Requires-Dist: rich>=13.7.0; extra == 'train'
+Requires-Dist: safetensors>=0.4.0; extra == 'train'
+Description-Content-Type: text/markdown
+
+# ProgramAsWeights
+
+A minimal runtime where programs are weight blobs run by a fixed interpreter.
+
+## Installation
+
+```bash
+pip install programasweights
+```
+
+## Quick Start
+
+```python
+import programasweights as paw
+
+# Load a compiled program
+f = paw.function("path/to/program", interpreter_name="yuntian-deng/paw-interpreter")
+
+# Use the program function
+result = f("Your input string here")
+print(result)
+```
+
+## Example Usage
+
+```python
+import programasweights as paw
+
+# Load a program that processes key-value mappings
+f = paw.function("outputs_1spec/prefix_kv/eval_program", interpreter_name="yuntian-deng/paw-interpreter")
+
+# Process input
+string = "A: b b:a, d:a"
+result = f(string)
+print(result)  # Output: {"A":"b":"a","D":"d":"a"}
+```
+
+## Features
+
+- **Simple API**: Load and run neural programs with just a few lines of code
+- **Flexible Interpreters**: Support for different transformer models as interpreters
+- **Efficient Execution**: Optimized runtime for prefix-tuned neural programs
+- **Easy Integration**: Seamlessly integrate neural programs into your Python applications
+
+## Requirements
+
+- Python >= 3.8
+- PyTorch
+- Transformers
+
+## License
+
+MIT License

programasweights-0.1.1/PYPI_README.md

@@ -0,0 +1,53 @@
+# ProgramAsWeights
+
+A minimal runtime where programs are weight blobs run by a fixed interpreter.
+
+## Installation
+
+```bash
+pip install programasweights
+```
+
+## Quick Start
+
+```python
+import programasweights as paw
+
+# Load a compiled program
+f = paw.function("path/to/program", interpreter_name="yuntian-deng/paw-interpreter")
+
+# Use the program function
+result = f("Your input string here")
+print(result)
+```
+
+## Example Usage
+
+```python
+import programasweights as paw
+
+# Load a program that processes key-value mappings
+f = paw.function("outputs_1spec/prefix_kv/eval_program", interpreter_name="yuntian-deng/paw-interpreter")
+
+# Process input
+string = "A: b b:a, d:a"
+result = f(string)
+print(result)  # Output: {"A":"b":"a","D":"d":"a"}
+```
+
+## Features
+
+- **Simple API**: Load and run neural programs with just a few lines of code
+- **Flexible Interpreters**: Support for different transformer models as interpreters
+- **Efficient Execution**: Optimized runtime for prefix-tuned neural programs
+- **Easy Integration**: Seamlessly integrate neural programs into your Python applications
+
+## Requirements
+
+- Python >= 3.8
+- PyTorch
+- Transformers
+
+## License
+
+MIT License

{programasweights-0.1.0 → programasweights-0.1.1}/README.md

@@ -1,13 +1,5 @@
 # ProgramAsWeights
 
-**⚠️ PLACEHOLDER PACKAGE - NOT YET FUNCTIONAL ⚠️**
-
-This package is currently a placeholder to reserve the name "programasweights" on PyPI. 
-
-The functionality is not yet available for public use. Please check back later for updates.
-
----
-
 Programs as small weight blobs; a fixed interpreter runs them.
 
 ## Development
@@ -40,6 +32,74 @@ print(f("Hello"))
 - weights = programs; base model = interpreter.
 - Deterministic by default (greedy generation).
 
+## 🌐 Web Interface
+
+ProgramAsWeights includes a modern web interface for easy interaction with the system. The web app provides a user-friendly way to compile specifications, test neural programs, and manage models without writing code.
+
+### Features
+
+- **Interactive Model Selection**: Choose from available compiler and interpreter models
+- **Natural Language Specifications**: Describe your program in plain English
+- **GPT-Powered Examples**: Auto-generate test data using OpenAI's API
+- **Real-time Compilation**: Compile specs into neural programs instantly
+- **Interactive Testing**: Test compiled programs with custom inputs
+- **Model Downloads**: Download compiled models as `.tgz` files
+- **Community Sharing**: Publish and discover programs on the leaderboard
+
+### Quick Start
+
+1. **Prerequisites**: Ensure you have Python 3.8+, Node.js 16+, and npm installed
+
+2. **Start the application**:
+   ```bash
+   cd web-app
+   ./start.sh
+   ```
+
+3. **Access the interface**:
+   - **Web App**: http://localhost:5173
+   - **API Docs**: http://localhost:8000/docs
+
+The startup script automatically:
+- Installs Python and Node.js dependencies
+- Starts the FastAPI backend server
+- Starts the React frontend development server
+
+### Manual Setup (Alternative)
+
+If you prefer manual setup:
+
+```bash
+# Backend
+cd web-app/backend
+pip install -r requirements.txt
+python run_server.py
+
+# Frontend (in another terminal)
+cd web-app/frontend  
+npm install
+npm run dev
+```
+
+### Configuration
+
+Create `web-app/backend/.env` with:
+```bash
+OPENAI_API_KEY=your_openai_api_key_here  # Optional: for GPT test generation
+CHECKPOINT_DIR=../../outputs_1spec/prefix_kv  # Path to trained models
+```
+
+### Example Workflow
+
+1. Select "Qwen 2.5 Coder 0.5B" as compiler and "PAW Interpreter" as interpreter
+2. Enter specification: *"Parse a string like '(A) cat (B) dog' into a JSON list"*
+3. Generate examples with GPT or add manually
+4. Click "Compile" to create your neural program
+5. Test with input: *"(A) red apple (B) green banana"*
+6. Download or publish your compiled model
+
+For detailed documentation, see [`web-app/README.md`](web-app/README.md).
+
 ## Train a compiler→KV prefix→interpreter (defaults)
 
 ```bash

programasweights-0.1.1/docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

programasweights-0.1.1/docs/api-reference.rst

@@ -0,0 +1,194 @@
+API Reference
+=============
+
+This page documents the main ProgramAsWeights API for end users.
+
+Core Functions
+--------------
+
+paw.function()
+~~~~~~~~~~~~~~
+
+Load and execute a neural program.
+
+.. code-block:: python
+
+   paw.function(path, *, name=None, interpreter_name="google/flan-t5-small", 
+                tokenizer_name=None, max_new_tokens=128)
+
+**Parameters:**
+
+- **path** (*str*) - HuggingFace model ID (e.g., "yuntian-deng/paw-parser") or local directory path
+- **name** (*str, optional*) - Custom name for the program (defaults to basename of path)
+- **interpreter_name** (*str*) - Interpreter model to use (defaults to "google/flan-t5-small")
+- **tokenizer_name** (*str, optional*) - Custom tokenizer path (defaults to interpreter's tokenizer)
+- **max_new_tokens** (*int*) - Maximum tokens to generate (default: 128)
+
+**Returns:**
+``Callable[[Union[str, List[str]]], Union[str, List[str]]]`` - Function that accepts string or list of strings
+
+**Examples:**
+
+.. code-block:: python
+
+   # Basic usage
+   fn = paw.function("yuntian-deng/paw-sentiment")
+   result = fn("This is great!")
+   
+   # Batch processing
+   results = fn(["Good product", "Bad service", "Okay experience"])
+   
+   # Custom interpreter
+   fn = paw.function("yuntian-deng/paw-parser", 
+                     interpreter_name="yuntian-deng/paw-interpreter")
+
+paw.compile()
+~~~~~~~~~~~~~
+
+Compile a natural language spec into a neural program (requires trained models).
+
+.. code-block:: python
+
+   paw.compile(out_dir, *, spec, checkpoint_dir="outputs/prefix_kv/checkpoint", 
+               prefix_steps=None)
+
+**Parameters:**
+
+- **out_dir** (*str*) - Directory to save the compiled program
+- **spec** (*str*) - Natural language description of the desired function
+- **checkpoint_dir** (*str*) - Path to trained compiler/interpreter checkpoints
+- **prefix_steps** (*int, optional*) - Override prefix length (defaults to trained value)
+
+**Returns:**
+*str* - Path to the compiled program directory
+
+**Example:**
+
+.. code-block:: python
+
+   # Compile a custom program
+   program_dir = paw.compile(
+       out_dir="my_custom_parser",
+       spec="Extract email addresses from text and return as JSON array"
+   )
+   
+   # Use the compiled program
+   fn = paw.function(program_dir)
+   emails = fn("Contact: alice@example.com or bob@test.org")
+
+Utility Functions
+-----------------
+
+paw.compile_dummy()
+~~~~~~~~~~~~~~~~~~~
+
+Create a dummy program for testing (legacy).
+
+.. code-block:: python
+
+   paw.compile_dummy(out_path, spec=None, *, seed=None, num_tokens=32)
+
+**Parameters:**
+
+- **out_path** (*str*) - Output file path
+- **spec** (*str, optional*) - Ignored (for API compatibility)
+- **seed** (*int, optional*) - Random seed for deterministic output
+- **num_tokens** (*int*) - Number of dummy tokens to generate
+
+**Returns:**
+*str* - Path to the created dummy program
+
+**Example:**
+
+.. code-block:: python
+
+   # Create deterministic dummy for testing
+   paw.compile_dummy("test.weights", seed=42)
+   fn = paw.function("test.weights")
+   print(fn("test input"))
+
+Configuration
+-------------
+
+Environment Variables
+~~~~~~~~~~~~~~~~~~~~~
+
+**PROGRAMASWEIGHTS_DEVICE**
+   Force device selection:
+   
+   - ``"cpu"`` - Use CPU only
+   - ``"cuda"`` - Use GPU if available, fallback to CPU
+   - Not set - Auto-detect (GPU if available)
+
+   .. code-block:: bash
+   
+      export PROGRAMASWEIGHTS_DEVICE=cpu
+      python my_script.py
+
+**HF_TOKEN**
+   HuggingFace authentication token for accessing private models:
+   
+   .. code-block:: bash
+   
+      export HF_TOKEN=your_token_here
+
+Error Handling
+--------------
+
+Common Exceptions
+~~~~~~~~~~~~~~~~~
+
+**ValueError**
+   - Invalid program artifact type
+   - Missing KV prefix in program
+   - Malformed program directory
+
+**FileNotFoundError**
+   - Program path doesn't exist
+   - Missing program.json manifest
+   - Missing kv_prefix.pt file
+
+**RuntimeError**
+   - Model loading failures
+   - GPU out of memory
+   - Generation errors
+
+**Example error handling:**
+
+.. code-block:: python
+
+   import programasweights as paw
+   
+   try:
+       fn = paw.function("yuntian-deng/paw-nonexistent")
+   except FileNotFoundError:
+       print("Program not found - check the model ID")
+   except ValueError as e:
+       print(f"Invalid program: {e}")
+   
+   try:
+       result = fn("input text")
+   except RuntimeError as e:
+       print(f"Execution failed: {e}")
+
+Type Hints
+----------
+
+For better IDE support:
+
+.. code-block:: python
+
+   from typing import Union, List, Callable
+   import programasweights as paw
+   
+   # Type hints for program functions
+   ProgramFunction = Callable[[Union[str, List[str]]], Union[str, List[str]]]
+   
+   def load_parser() -> ProgramFunction:
+       return paw.function("yuntian-deng/paw-parser")
+   
+   def process_texts(texts: List[str]) -> List[str]:
+       parser = load_parser()
+       return parser(texts)
+
+Next: :doc:`examples/text-parsing` 

programasweights-0.1.1/docs/conf.py

@@ -0,0 +1,27 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = 'programasweights'
+copyright = '2025, PAW'
+author = 'PAW'
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = ['myst_parser']
+
+templates_path = ['_templates']
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = 'sphinx_rtd_theme'
+html_static_path = ['_static']

programasweights-0.1.1/docs/index.rst

@@ -0,0 +1,84 @@
+ProgramAsWeights Documentation
+==============================
+
+**Programs as small weight blobs; a fixed interpreter runs them.**
+
+ProgramAsWeights (PAW) is a new programming paradigm where fuzzy functions—like "determine if a review is positive" or "extract the final answer from reasoning text"—are represented as compact neural weight blobs that execute locally without external API calls.
+
+Quick Start
+-----------
+
+Install and use a pre-trained program in 3 lines:
+
+.. code-block:: python
+
+   import programasweights as paw
+   
+   # Load a pre-trained program for parsing options
+   fn = paw.function("yuntian-deng/paw-option-parser")
+   
+   # Use it like any Python function
+   result = fn("(A) cat (B) dog (C) both (A) and (B)")
+   print(result)  # → ["(A) cat", "(B) dog", "(C) both (A) and (B)"]
+
+No internet connection required after download. No API keys. No external dependencies.
+
+Why ProgramAsWeights?
+--------------------
+
+**Replace expensive LLM API calls** with lightweight local execution:
+
+- ❌ **Before:** ``openai.chat.completions.create(...)`` - $$$, internet-dependent, fragile
+- ✅ **After:** ``paw.function("task")("input")`` - local, fast, reproducible
+
+**Perfect for fuzzy functions** that are conceptually simple but hard to code precisely:
+
+- Sentiment analysis, bias detection, answer extraction
+- HTML cleaning, format conversion, typo-tolerant parsing  
+- Evaluation tasks like checking answer equivalence
+- Linguistic analysis (counting nouns/verbs, readability scoring)
+
+Documentation Sections
+======================
+
+.. toctree::
+   :maxdepth: 2
+   :caption: User Guide:
+
+   installation
+   quickstart
+   using-pretrained
+   api-reference
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Examples:
+
+   examples/text-parsing
+   examples/evaluation-tasks
+   examples/bias-analysis
+   examples/html-processing
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Advanced (Researchers):
+
+   training/overview
+   training/custom-datasets
+   training/model-architecture
+   
+.. toctree::
+   :maxdepth: 1
+   :caption: Reference:
+
+   api
+   troubleshooting
+   contributing
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
+

programasweights-0.1.1/docs/installation.rst

@@ -0,0 +1,95 @@
+Installation
+============
+
+ProgramAsWeights requires Python 3.8+ and PyTorch.
+
+Quick Install
+-------------
+
+.. code-block:: bash
+
+   pip install programasweights
+
+That's it! The package automatically handles model downloads when you first use a program.
+
+Requirements
+------------
+
+**Minimal requirements** (automatically installed):
+
+- ``torch`` - Neural network runtime
+- ``transformers`` - Model loading and inference
+- ``safetensors`` - Efficient tensor storage
+
+**Optional dependencies** for advanced users:
+
+.. code-block:: bash
+
+   # For training your own models (researchers only)
+   pip install programasweights[train]
+   
+   # For generating new datasets
+   pip install programasweights[data]
+
+System Requirements
+-------------------
+
+**Memory:**
+- **CPU:** 4GB+ RAM recommended
+- **GPU:** 8GB+ VRAM for larger models (optional, falls back to CPU)
+
+**Storage:**
+- ~2-4GB per interpreter model (downloaded once, cached locally)
+- ~100MB per compiled program
+
+**Network:**
+- Internet connection for initial model download
+- Offline execution after download
+
+Verify Installation
+-------------------
+
+.. code-block:: python
+
+   import programasweights as paw
+   print(paw.__version__)  # Should print version number
+
+**Test with a dummy program:**
+
+.. code-block:: python
+
+   # Create a test program (uses dummy compiler for demo)
+   paw.compile_dummy("test.weights", seed=42)
+   
+   # Load and run it
+   fn = paw.function("test.weights")
+   print(fn("hello world"))
+
+GPU Setup (Optional)
+--------------------
+
+ProgramAsWeights automatically uses GPU if available. To force CPU-only:
+
+.. code-block:: bash
+
+   export PROGRAMASWEIGHTS_DEVICE=cpu
+
+To verify GPU usage:
+
+.. code-block:: python
+
+   import torch
+   print(f"CUDA available: {torch.cuda.is_available()}")
+   print(f"Device: {torch.cuda.get_device_name() if torch.cuda.is_available() else 'CPU'}")
+
+Troubleshooting
+---------------
+
+**Common issues:**
+
+- **"No module named 'programasweights'"** → Run ``pip install programasweights``
+- **Model download fails** → Check internet connection and disk space
+- **Out of memory** → Set ``PROGRAMASWEIGHTS_DEVICE=cpu`` or use smaller models
+- **Slow execution** → Enable GPU or use smaller input batches
+
+See :doc:`troubleshooting` for detailed solutions.