entropy-predict 0.1.0__tar.gz

entropy_predict-0.1.0/.env.example

@@ -0,0 +1,9 @@
+# Entropy API Keys
+# These are the only settings that belong in .env (secrets only).
+# For provider/model config, use: entropy config set <key> <value>
+
+# OpenAI (required if using openai as provider)
+OPENAI_API_KEY=sk-...
+
+# Anthropic (from https://console.anthropic.com/settings/keys)
+ANTHROPIC_API_KEY=sk-ant-...

entropy_predict-0.1.0/.github/workflows/publish.yml

@@ -0,0 +1,26 @@
+name: Publish to PyPI
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Build package
+        run: |
+          pip install build
+          python -m build
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

entropy_predict-0.1.0/.gitignore

@@ -0,0 +1,55 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual environments
+env/
+venv/
+.venv/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# Environment
+.env
+
+# Storage (database and populations)
+storage/
+
+# Simulation results
+results/
+
+# Cache
+data/cache/
+
+# Logs
+logs/
+
+# OS
+.DS_Store
+Thumbs.db
+
+
+austin/
+examples/

entropy_predict-0.1.0/CLAUDE.md

@@ -0,0 +1,176 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## What Entropy Is
+
+Entropy is a predictive intelligence framework that simulates how real human populations respond to scenarios. It creates synthetic populations grounded in real-world statistical data, enriches them with LLM-extrapolated psychographic attributes, connects them via social networks, and runs agent-based simulations where each agent reasons individually via LLM calls. The output is not a poll — it's a simulation of emergent collective behavior.
+
+Competitor reference: [Aaru](https://aaru.com) operates in the same space (multi-agent population simulation for predictive intelligence). Entropy differentiates through its grounding pipeline — every attribute distribution is researched from real-world sources with citations, not just LLM-generated.
+
+## Commands
+
+```bash
+pip install -e ".[dev]"      # Install with dev deps
+
+# Set API keys (secrets only — in .env or env vars)
+export ANTHROPIC_API_KEY=sk-ant-...
+export OPENAI_API_KEY=sk-...
+
+# Configure providers and models
+entropy config set pipeline.provider claude        # Use Claude for population/scenario building
+entropy config set simulation.provider openai      # Use OpenAI for agent reasoning
+entropy config set simulation.model gpt-5-mini     # Override simulation model
+entropy config set simulation.routine_model gpt-5-mini  # Cheap model for Pass 2 classification
+entropy config set simulation.rate_tier 2          # Rate limit tier (1-4)
+entropy config show                                # View current config
+
+pytest                       # Run all tests
+pytest tests/test_sampler.py # Single test file
+pytest -k "test_name"        # Single test by name
+
+ruff check .                 # Lint
+ruff format .                # Format
+```
+
+CLI entry point: `entropy` (defined in `pyproject.toml` → `entropy.cli:app`). Python >=3.11.
+
+## Pipeline (7 sequential commands)
+
+```
+entropy spec → entropy extend → entropy sample → entropy network → entropy persona → entropy scenario → entropy simulate
+ │
+ entropy results
+```
+
+Each command produces a file consumed by the next. `entropy validate` is a utility runnable at any point. `entropy results` is a viewer for simulation output.
+
+## Architecture
+
+Three phases, each mapping to a package under `entropy/`:
+
+### Phase 1: Population Creation (`entropy/population/`)
+
+**The validity pipeline.** This is where predictive accuracy is won or lost.
+
+1. **Sufficiency check** (`spec_builder/sufficiency.py`) — LLM validates the description has enough context (who, how many, where).
+
+2. **Attribute selection** (`spec_builder/selector.py`) — LLM discovers 25-40 attributes across 4 categories: `universal` (age, gender), `population_specific` (specialty, seniority), `context_specific` (scenario-relevant), `personality` (Big Five). Each attribute gets a type (`int`/`float`/`categorical`/`boolean`) and sampling strategy (`independent`/`derived`/`conditional`).
+
+3. **Hydration** (`spec_builder/hydrator.py` → `hydrators/`) — The most important step. Four sub-steps, each using different LLM tiers:
+   - **2a: Independent** (`hydrators/independent.py`) — `agentic_research()` with web search finds real-world distributions with source URLs. This is the grounding layer.
+   - **2b: Derived** (`hydrators/derived.py`) — `reasoning_call()` specifies deterministic formulas (e.g., `years_experience = age - 26`).
+   - **2c: Conditional base** (`hydrators/conditional.py`) — `agentic_research()` finds base distributions for attributes that depend on others.
+   - **2d: Conditional modifiers** (`hydrators/conditional.py`) — `reasoning_call()` specifies how attribute values shift based on other attributes. Type-specific: numeric gets `multiply`/`add`, categorical gets `weight_overrides`, boolean gets `probability_override`.
+
+4. **Constraint binding** (`spec_builder/binder.py`) — Topological sort (Kahn's algorithm, `utils/graphs.py`) resolves attribute dependencies into a valid sampling order. Raises `CircularDependencyError` with cycle path.
+
+5. **Sampling** (`sampler/core.py`) — Iterates through `sampling_order`, routing each attribute by strategy. Supports 6 distribution types: normal, lognormal, uniform, beta, categorical, boolean. Hard constraints (min/max) are clamped post-sampling. Formula parameters evaluated via `utils/eval_safe.py` (restricted Python eval, whitelisted builtins only).
+
+6. **Network generation** (`network/generator.py`) — Hybrid algorithm: similarity-based edge probability with degree correction, calibrated via binary search to hit target avg_degree, then Watts-Strogatz rewiring (5%) for small-world properties. Edge probability: `base_rate * sigmoid(similarity) * degree_factor_a * degree_factor_b`.
+
+### Phase 2: Scenario Compilation (`entropy/scenario/`)
+
+**Compiler** (`compiler.py`) orchestrates 5 steps: parse event → generate exposure rules → determine interaction model → define outcomes → assemble spec.
+
+- **Event types**: product_launch, policy_change, pricing_change, technology_release, organizational_change, market_event, crisis_event
+- **Exposure channels**: broadcast, targeted, organic — with per-timestep rules containing conditions and probabilities
+- **Outcomes**: categorical (enum options), boolean, float (with range), open_ended
+- Auto-configures simulation parameters based on population size (<500: 50 timesteps, ≤5000: 100, >5000: 168)
+
+### Phase 3: Simulation (`entropy/simulation/`)
+
+**Engine** (`engine.py`) runs per-timestep loop:
+1. Apply seed exposures from scenario rules (`propagation.py`)
+2. Propagate through network — conviction-gated sharing (very_uncertain agents don't share)
+3. Select agents to reason — first exposure OR multi-touch threshold exceeded (default: 3 new exposures since last reasoning)
+4. **Two-pass async LLM reasoning** (`reasoning.py`) — Rate-limiter-controlled (token bucket per provider/model):
+   - **Pass 1 (role-play)**: Agent reasons in first person with no categorical enums. Produces reasoning, public_statement, sentiment, conviction level, will_share. Memory trace (last 3 reasoning summaries) is fed back for re-reasoning agents.
+   - **Pass 2 (classification)**: A cheap model classifies the free-text reasoning into scenario-defined categorical/boolean/float outcomes. Position is extracted here — it is output-only, never used in peer influence.
+5. **Conviction-based flip resistance**: Firm+ conviction agents reject position flips unless new conviction is moderate+
+6. **Semantic peer influence**: Agents see peers' public_statement + sentiment tone, NOT position labels
+7. Update state (`state.py`) — SQLite-backed with indexed tables for agent_states, exposures, memory_traces, timeline
+8. Check stopping conditions (`stopping.py`) — Compound conditions like `"exposure_rate > 0.95 and no_state_changes_for > 10"`, convergence detection via sentiment variance
+
+**Two-pass reasoning rationale**: Single-pass reasoning caused 83% of agents to pick safe middle options (central tendency bias). Splitting role-play from classification fixes this — agents reason naturally in Pass 1, then a cheap model maps to categories in Pass 2.
+
+**Conviction system**: Agents pick from categorical levels (`very_uncertain` / `leaning` / `moderate` / `firm` / `absolute`), mapped to floats (0.1/0.3/0.5/0.7/0.9) only for storage and threshold math. Agents never see numeric values.
+
+**Rate limiter** (`core/rate_limiter.py`): Token bucket with dual RPM + TPM buckets. Provider-aware defaults from `core/rate_limits.py` (Anthropic/OpenAI, tiers 1-4). Replaces the old hardcoded `Semaphore(50)`. CLI flags: `--rate-tier`, config: `simulation.rate_tier`, `simulation.rpm_override`, `simulation.tpm_override`.
+
+**Persona system** (`population/persona/` + `simulation/persona.py`): The `entropy persona` command generates a `PersonaConfig` via 5-step LLM pipeline (structure → boolean → categorical → relative → concrete phrasings). At simulation time, agents are rendered computationally using this config — no per-agent LLM calls. Relative attributes (personality, attitudes) are positioned against population stats via z-scores ("I'm much more price-sensitive than most people"). Concrete attributes use format specs for proper number/time rendering. **Trait salience**: If `decision_relevant_attributes` is set on `OutcomeConfig`, those attributes are grouped first under "Most Relevant to This Decision" in the persona.
+
+## LLM Integration (`entropy/core/llm.py`)
+
+All LLM calls go through this file — never call providers directly elsewhere. Two-zone routing:
+
+**Pipeline zone** (phases 1-2: spec, extend, persona, scenario) — configured via `entropy config set pipeline.*`:
+
+| Function | Default Model | Tools | Use |
+|----------|--------------|-------|-----|
+| `simple_call()` | provider default (haiku/gpt-5-mini) | none | Sufficiency checks, simple extractions |
+| `reasoning_call()` | provider default (sonnet/gpt-5) | none | Attribute selection, hydration, scenario compilation. Supports validator callback + retry |
+| `agentic_research()` | provider default (sonnet/gpt-5) | web_search | Distribution hydration with real-world data. Extracts source URLs |
+
+**Simulation zone** (phase 3: agent reasoning) — configured via `entropy config set simulation.*`:
+
+| Function | Default Model | Tools | Use |
+|----------|--------------|-------|-----|
+| `simple_call_async()` | provider default | none | Pass 1 role-play reasoning + Pass 2 classification (async) |
+
+Two-pass model routing: Pass 1 uses `simulation.model` (pivotal reasoning). Pass 2 uses `simulation.routine_model` (cheap classification). Both default to provider default if not set. CLI: `--model`, `--pivotal-model`, `--routine-model`. Standard inference only — no thinking/extended models (no o1, o3, extended thinking).
+
+**Provider abstraction** (`entropy/core/providers/`): `LLMProvider` base class with `OpenAIProvider` and `ClaudeProvider` implementations. Factory functions `get_pipeline_provider()` and `get_simulation_provider()` read from `EntropyConfig`.
+
+**Config** (`entropy/config.py`): `EntropyConfig` with `PipelineConfig` and `SimZoneConfig` zones. Resolution order: env vars > config file (`~/.config/entropy/config.json`) > defaults. API keys always from env vars (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`). For package use: `from entropy.config import configure, EntropyConfig`.
+
+**Default zones**: Pipeline = Claude (population/scenario building). Simulation = OpenAI (agent reasoning). `SimZoneConfig` fields: `provider`, `model`, `pivotal_model`, `routine_model`, `max_concurrent`, `rate_tier`, `rpm_override`, `tpm_override`.
+
+All calls use structured output (`response_format: json_schema`). Failed validations are fed back as "PREVIOUS ATTEMPT FAILED" prompts for self-correction.
+
+## Data Models (`entropy/core/models/`)
+
+All Pydantic v2. Key hierarchy:
+
+- `population.py`: `PopulationSpec` → `AttributeSpec` → `SamplingConfig` → `Distribution` / `Modifier` / `Constraint`
+- `scenario.py`: `ScenarioSpec` → `Event`, `SeedExposure` (channels + rules), `InteractionConfig`, `SpreadConfig`, `OutcomeConfig`
+- `simulation.py`: `ConvictionLevel`, `MemoryEntry`, `AgentState` (conviction, public_statement), `PeerOpinion` (public_statement, credibility), `ReasoningContext` (memory_trace), `ReasoningResponse` (conviction, public_statement, reasoning_summary), `SimulationRunConfig` (pivotal_model, routine_model), `TimestepSummary` (average_conviction, sentiment_variance)
+- `network.py`: `Edge`, `NodeMetrics`, `NetworkMetrics`
+- `validation.py`: `ValidationIssue`, `ValidationResult`
+
+YAML serialization via `to_yaml()`/`from_yaml()` on `PopulationSpec` and `ScenarioSpec`.
+
+## Validation (`entropy/population/validator/`)
+
+Two layers for population specs:
+- **Structural** (`structural.py`): ERROR-level — type/modifier compatibility, range violations, distribution params, dependency cycles, condition syntax, formula references, duplicates, strategy consistency
+- **Semantic** (`semantic.py`): WARNING-level — no-op detection, modifier stacking, categorical option reference validity
+
+Scenario validation (`entropy/scenario/validator.py`): attribute reference validity, edge type references, probability ranges.
+
+## Key Conventions
+
+- Conditions and formulas use restricted Python syntax via `eval_safe()` — whitelisted builtins only (abs, min, max, round, int, float, str, len, sum, all, any, bool)
+- Agent IDs use the `_id` field from agent JSON, falling back to string index
+- Network edges are bidirectional (stored as source/target, traversed both ways)
+- Exposure credibility: `event_credibility * channel_credibility` for seed, fixed 0.85 for peer
+- "Position" = first required categorical outcome (extracted in Pass 2, used for aggregation/output only — never used in peer influence)
+- Peer influence is semantic: agents see neighbors' `public_statement` + sentiment tone, not position labels
+- Conviction is categorical for agents (`very_uncertain`/`leaning`/`moderate`/`firm`/`absolute`), mapped to floats (0.1–0.9) only for storage/thresholds
+- Memory traces: 3-entry sliding window per agent, fed back into reasoning prompts for re-reasoning
+- The `persona` command generates detailed persona configs; `extend` still generates a simpler `persona_template` for backwards compatibility
+- Simulation auto-detects `{population_stem}.persona.yaml` and uses the new rendering if present
+- Network config defaults in `network/config.py` are currently hardcoded for the German surgeons example and need generalization
+
+## Tests
+
+pytest + pytest-asyncio. Fixtures in `tests/conftest.py` include seeded RNG (`Random(42)`), minimal/complex population specs, and sample agents. Six test files covering models, network, sampler, scenario, simulation, validator.
+
+## File Formats
+
+- Population/scenario specs: YAML
+- Agents: JSON (array of objects with `_id`)
+- Network: JSON (`{meta, nodes, edges}`)
+- Simulation state: SQLite (tables: agent_states, exposures, memory_traces, timeline, timestep_summaries)
+- Timeline: JSONL (streaming, crash-safe)
+- Results: JSON files in output directory (agent_states.json, by_timestep.json, outcome_distributions.json, meta.json)

entropy_predict-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025-2026 Devesh Paragiri
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

entropy_predict-0.1.0/PKG-INFO

@@ -0,0 +1,191 @@
+Metadata-Version: 2.4
+Name: entropy-predict
+Version: 0.1.0
+Summary: Predictive intelligence through agent-based population simulation
+Project-URL: Homepage, https://github.com/exaforge/entropy
+Project-URL: Repository, https://github.com/exaforge/entropy
+Project-URL: Issues, https://github.com/exaforge/entropy/issues
+Author: Devesh Paragiri
+License-Expression: MIT
+License-File: LICENSE
+Keywords: agent-based,llm,population,predictive-intelligence,simulation
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.11
+Requires-Dist: anthropic>=0.77.0
+Requires-Dist: fastapi>=0.115.0
+Requires-Dist: networkx>=3.4.0
+Requires-Dist: numpy>=2.0.0
+Requires-Dist: openai>=1.50.0
+Requires-Dist: pydantic>=2.9.0
+Requires-Dist: python-dotenv>=1.0.0
+Requires-Dist: pyyaml>=6.0.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: scipy>=1.14.0
+Requires-Dist: typer>=0.12.0
+Requires-Dist: uvicorn>=0.32.0
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.24.0; extra == 'dev'
+Requires-Dist: pytest>=8.0.0; extra == 'dev'
+Requires-Dist: ruff>=0.14.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# Entropy
+
+Predictive intelligence through agent-based population simulation. Create synthetic populations grounded in real-world data, simulate how they respond to events, and watch opinions emerge through social networks.
+
+Not a survey. Not a poll. A simulation of collective human behavior.
+
+## What It Does
+
+You describe a population and a scenario. Entropy builds statistically grounded synthetic agents, connects them in a social network, and has each one reason individually about the event using an LLM. Opinions form, spread through the network, and evolve — producing distributional predictions you can segment and analyze.
+
+```
+entropy spec → entropy extend → entropy sample → entropy network → entropy persona → entropy scenario → entropy simulate
+                                                                                                               │
+                                                                                                        entropy results
+```
+
+## Install
+
+```bash
+pip install entropy-predict
+```
+
+Or from source:
+
+```bash
+git clone https://github.com/exaforge/entropy.git
+cd entropy
+pip install -e ".[dev]"
+```
+
+## Setup
+
+```bash
+# API keys (in .env or exported)
+export OPENAI_API_KEY=sk-...
+export ANTHROPIC_API_KEY=sk-ant-...
+
+# Configure providers
+entropy config set pipeline.provider claude      # Claude for population/scenario building
+entropy config set simulation.provider openai    # OpenAI for agent reasoning
+entropy config show
+```
+
+## Quick Start
+
+```bash
+# Build a population
+entropy spec "500 Austin TX commuters who drive into downtown for work" -o austin/base.yaml
+entropy extend austin/base.yaml -s "Response to a $15/day downtown congestion tax" -o austin/population.yaml
+entropy sample austin/population.yaml -o austin/agents.json --seed 42
+entropy network austin/agents.json -o austin/network.json --seed 42
+entropy persona austin/population.yaml --agents austin/agents.json
+
+# Compile and run a scenario
+entropy scenario -p austin/population.yaml -a austin/agents.json -n austin/network.json -o austin/scenario.yaml
+entropy simulate austin/scenario.yaml -o austin/results/ --seed 42
+
+# View results
+entropy results austin/results/
+entropy results austin/results/ --segment income
+```
+
+### What Comes Out
+
+Outcomes are defined per-scenario — categorical, float, boolean, or open-ended. You choose what to measure.
+
+```
+═══════════════════════════════════════════════════════════
+SIMULATION RESULTS: austin_congestion_tax
+═══════════════════════════════════════════════════════════
+
+Population: 500 agents | Duration: 47 timesteps | Model: gpt-5
+Stopped: exposure_rate > 0.95 and no_state_changes_for > 5
+
+EXPOSURE
+────────────────────────────────────────
+Final exposure rate: 96.8%
+Reasoning calls: 1,847
+Average conviction: 0.64 (moderate-to-firm)
+
+OUTCOMES
+────────────────────────────────────────
+commute_response (categorical):
+  drive_and_pay          38%  ███████████████░░░░░
+  switch_to_transit      24%  █████████░░░░░░░░░░░
+  shift_schedule         19%  ███████░░░░░░░░░░░░░
+  telework_more          12%  ████░░░░░░░░░░░░░░░░
+  undecided               7%  ██░░░░░░░░░░░░░░░░░░
+
+sentiment (float, -1 to 1):
+  mean: -0.18  std: 0.41  min: -0.9  max: 0.7
+
+willingness_to_pay (boolean):
+  yes: 42%  no: 58%
+
+protest_likelihood (float, 0 to 1):
+  mean: 0.31  std: 0.28
+
+SEGMENT: income
+────────────────────────────────────────
+< $50k:   drive_and_pay 22% | switch_to_transit 14% | protest 41%
+$50-100k: drive_and_pay 40% | switch_to_transit 28% | shift_schedule 21%
+> $100k:  drive_and_pay 51% | switch_to_transit 31% | telework_more 14%
+```
+
+Each agent reasoned individually. A low-income commuter with no transit access reacts differently than a tech worker near a rail stop — not because we scripted it, but because their attributes, persona, and social context led them there.
+
+The scenario YAML controls what gets tracked:
+
+```yaml
+outcomes:
+  suggested_outcomes:
+  - name: commute_response
+    type: categorical
+    options: [drive_and_pay, switch_to_transit, shift_schedule, telework_more, undecided]
+  - name: sentiment
+    type: float
+    range: [-1.0, 1.0]
+  - name: willingness_to_pay
+    type: boolean
+  - name: protest_likelihood
+    type: float
+    range: [0.0, 1.0]
+```
+
+## How It Works
+
+**Population creation** — An LLM discovers relevant attributes (demographics, psychographics, scenario-specific), then researches real-world distributions with citations. Agents are sampled from these distributions respecting all dependencies. A social network connects them based on attribute similarity with small-world properties.
+
+**Persona rendering** — Each agent gets a first-person narrative built from their attributes. Relative traits are positioned against population statistics ("I'm much more price-sensitive than most people"). Generated once per population, applied computationally per agent.
+
+**Two-pass reasoning** — Pass 1: the agent role-plays their reaction in natural language (no enum labels, no anchoring). Pass 2: a cheap model classifies the freeform response into outcome categories. This eliminates the central tendency bias that plagues single-pass structured extraction.
+
+**Network propagation** — Agents share information through social connections. Edge types, spread modifiers, and decay control how opinions travel. Multi-touch re-reasoning lets agents update their position after hearing from multiple peers.
+
+## Documentation
+
+- **[CLI Reference](docs/commands.md)** — Every command with arguments, options, and examples
+- **[Architecture](docs/architecture.md)** — How the system works under the hood
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest                    # Run tests
+ruff check .              # Lint
+ruff format .             # Format
+```
+
+## License
+
+MIT

entropy_predict-0.1.0/README.md

@@ -0,0 +1,152 @@
+# Entropy
+
+Predictive intelligence through agent-based population simulation. Create synthetic populations grounded in real-world data, simulate how they respond to events, and watch opinions emerge through social networks.
+
+Not a survey. Not a poll. A simulation of collective human behavior.
+
+## What It Does
+
+You describe a population and a scenario. Entropy builds statistically grounded synthetic agents, connects them in a social network, and has each one reason individually about the event using an LLM. Opinions form, spread through the network, and evolve — producing distributional predictions you can segment and analyze.
+
+```
+entropy spec → entropy extend → entropy sample → entropy network → entropy persona → entropy scenario → entropy simulate
+                                                                                                               │
+                                                                                                        entropy results
+```
+
+## Install
+
+```bash
+pip install entropy-predict
+```
+
+Or from source:
+
+```bash
+git clone https://github.com/exaforge/entropy.git
+cd entropy
+pip install -e ".[dev]"
+```
+
+## Setup
+
+```bash
+# API keys (in .env or exported)
+export OPENAI_API_KEY=sk-...
+export ANTHROPIC_API_KEY=sk-ant-...
+
+# Configure providers
+entropy config set pipeline.provider claude      # Claude for population/scenario building
+entropy config set simulation.provider openai    # OpenAI for agent reasoning
+entropy config show
+```
+
+## Quick Start
+
+```bash
+# Build a population
+entropy spec "500 Austin TX commuters who drive into downtown for work" -o austin/base.yaml
+entropy extend austin/base.yaml -s "Response to a $15/day downtown congestion tax" -o austin/population.yaml
+entropy sample austin/population.yaml -o austin/agents.json --seed 42
+entropy network austin/agents.json -o austin/network.json --seed 42
+entropy persona austin/population.yaml --agents austin/agents.json
+
+# Compile and run a scenario
+entropy scenario -p austin/population.yaml -a austin/agents.json -n austin/network.json -o austin/scenario.yaml
+entropy simulate austin/scenario.yaml -o austin/results/ --seed 42
+
+# View results
+entropy results austin/results/
+entropy results austin/results/ --segment income
+```
+
+### What Comes Out
+
+Outcomes are defined per-scenario — categorical, float, boolean, or open-ended. You choose what to measure.
+
+```
+═══════════════════════════════════════════════════════════
+SIMULATION RESULTS: austin_congestion_tax
+═══════════════════════════════════════════════════════════
+
+Population: 500 agents | Duration: 47 timesteps | Model: gpt-5
+Stopped: exposure_rate > 0.95 and no_state_changes_for > 5
+
+EXPOSURE
+────────────────────────────────────────
+Final exposure rate: 96.8%
+Reasoning calls: 1,847
+Average conviction: 0.64 (moderate-to-firm)
+
+OUTCOMES
+────────────────────────────────────────
+commute_response (categorical):
+  drive_and_pay          38%  ███████████████░░░░░
+  switch_to_transit      24%  █████████░░░░░░░░░░░
+  shift_schedule         19%  ███████░░░░░░░░░░░░░
+  telework_more          12%  ████░░░░░░░░░░░░░░░░
+  undecided               7%  ██░░░░░░░░░░░░░░░░░░
+
+sentiment (float, -1 to 1):
+  mean: -0.18  std: 0.41  min: -0.9  max: 0.7
+
+willingness_to_pay (boolean):
+  yes: 42%  no: 58%
+
+protest_likelihood (float, 0 to 1):
+  mean: 0.31  std: 0.28
+
+SEGMENT: income
+────────────────────────────────────────
+< $50k:   drive_and_pay 22% | switch_to_transit 14% | protest 41%
+$50-100k: drive_and_pay 40% | switch_to_transit 28% | shift_schedule 21%
+> $100k:  drive_and_pay 51% | switch_to_transit 31% | telework_more 14%
+```
+
+Each agent reasoned individually. A low-income commuter with no transit access reacts differently than a tech worker near a rail stop — not because we scripted it, but because their attributes, persona, and social context led them there.
+
+The scenario YAML controls what gets tracked:
+
+```yaml
+outcomes:
+  suggested_outcomes:
+  - name: commute_response
+    type: categorical
+    options: [drive_and_pay, switch_to_transit, shift_schedule, telework_more, undecided]
+  - name: sentiment
+    type: float
+    range: [-1.0, 1.0]
+  - name: willingness_to_pay
+    type: boolean
+  - name: protest_likelihood
+    type: float
+    range: [0.0, 1.0]
+```
+
+## How It Works
+
+**Population creation** — An LLM discovers relevant attributes (demographics, psychographics, scenario-specific), then researches real-world distributions with citations. Agents are sampled from these distributions respecting all dependencies. A social network connects them based on attribute similarity with small-world properties.
+
+**Persona rendering** — Each agent gets a first-person narrative built from their attributes. Relative traits are positioned against population statistics ("I'm much more price-sensitive than most people"). Generated once per population, applied computationally per agent.
+
+**Two-pass reasoning** — Pass 1: the agent role-plays their reaction in natural language (no enum labels, no anchoring). Pass 2: a cheap model classifies the freeform response into outcome categories. This eliminates the central tendency bias that plagues single-pass structured extraction.
+
+**Network propagation** — Agents share information through social connections. Edge types, spread modifiers, and decay control how opinions travel. Multi-touch re-reasoning lets agents update their position after hearing from multiple peers.
+
+## Documentation
+
+- **[CLI Reference](docs/commands.md)** — Every command with arguments, options, and examples
+- **[Architecture](docs/architecture.md)** — How the system works under the hood
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest                    # Run tests
+ruff check .              # Lint
+ruff format .             # Format
+```
+
+## License
+
+MIT