@pikaa-ai/pikaa 0.3.22 → 0.3.24

package/skills/cellxgene-census/SKILL.md

@@ -1,283 +0,0 @@
----
-name: cellxgene-census
-description: Query the CZ CELLxGENE Census programmatically for versioned public single-cell and spatial transcriptomics data. Use when you need population-scale cell metadata, gene expression slices, Census summary counts, source H5AD URIs/downloads, embeddings, spatial Census data, or reference atlas comparisons across organisms, tissues, diseases, assays, and cell types. For analyzing your own local single-cell data use scanpy, anndata, or scvi-tools.
-allowed-tools: Read Write Edit Bash
-license: MIT
-compatibility: Requires Python >=3.10,<3.13. Examples target cellxgene-census 1.17.x and the 2025-11-08 stable LTS Census; spatial workflows need the spatial extra and TileDB-SOMA >=1.15.5. No authentication is required for public Census data.
-metadata:
-  version: "1.2"
-  skill-author: K-Dense Inc.
----
-
-# CZ CELLxGENE Census
-
-## Overview
-
-The CZ CELLxGENE Census provides programmatic access to a comprehensive, versioned collection of standardized single-cell and spatial transcriptomics data from CZ CELLxGENE Discover. This skill enables efficient querying and analysis of public Census releases without downloading whole datasets first.
-
-The Census includes:
-- **217+ million total cells** and **125+ million unique cells** in the 2025-11-08 stable LTS release
-- **1,845 datasets** in the 2025-11-08 stable LTS release
-- **Human, mouse, marmoset, rhesus macaque, and chimpanzee** data in the current schema
-- **Standardized metadata** (cell types, tissues, diseases, donors)
-- **Raw gene expression** matrices and source H5AD lookup/download helpers
-- **Pre-calculated summary counts, embeddings, and spatial data**
-- **Integration with AnnData, Scanpy, TileDB-SOMA, TileDB-SOMA-ML, and other analysis tools**
-
-## When to Use This Skill
-
-This skill should be used when:
-- Querying single-cell expression data by cell type, tissue, or disease
-- Exploring available single-cell datasets and metadata
-- Training machine learning models on single-cell data
-- Performing large-scale cross-dataset analyses
-- Integrating Census data with scanpy or other analysis frameworks
-- Computing statistics across millions of cells
-- Accessing pre-calculated embeddings or model predictions
-
-## Installation and Setup
-
-Install the Census API:
-```bash
-uv pip install "cellxgene-census==1.17.*"
-```
-
-For spatial workflows:
-```bash
-uv pip install "cellxgene-census[spatial]==1.17.*" "spatialdata[extra]>=0.2.5"
-```
-
-For PyTorch model training, use TileDB-SOMA-ML. The old `cellxgene_census.experimental.ml` loaders are deprecated:
-
-```bash
-uv pip install "cellxgene-census==1.17.*" tiledbsoma-ml
-```
-
-## Core Workflow Patterns
-
-Eight patterns, each with code, are in
-[references/core_workflow_patterns.md](references/core_workflow_patterns.md):
-
-1. **Opening the Census** — always pin `census_version` so an analysis stays reproducible.
-2. **Exploring Census information** — available datasets, cell counts, and summary tables.
-3. **Querying expression data** — small to medium scale into an `AnnData`.
-4. **Large-scale queries** — out-of-core processing when the slice will not fit in memory.
-5. **Machine learning with PyTorch** — the Census data loaders.
-6. **Spatial Census data** — accessing spatial assays.
-7. **Integration with Scanpy** — handing a Census slice to a standard Scanpy workflow.
-8. **Multi-dataset integration** — combining datasets and handling batch effects.
-
-## Key Concepts and Best Practices
-
-### Always Filter for Primary Data
-Unless analyzing duplicates, always include `is_primary_data == True` in queries to avoid counting cells multiple times:
-```python
-obs_value_filter="cell_type == 'B cell' and is_primary_data == True"
-```
-
-### Specify Census Version for Reproducibility
-Always specify the Census version in production analyses:
-```python
-census = cellxgene_census.open_soma(census_version="2025-11-08")
-```
-
-### Estimate Query Size Before Loading
-For large queries, first check the number of cells to avoid memory issues:
-```python
-# Get cell count
-metadata = cellxgene_census.get_obs(
-    census, "homo_sapiens",
-    value_filter="tissue_general == 'brain' and is_primary_data == True",
-    column_names=["soma_joinid"]
-)
-n_cells = len(metadata)
-print(f"Query will return {n_cells:,} cells")
-
-# If too large (>100k), use out-of-core processing
-```
-
-### Use tissue_general for Broader Groupings
-The `tissue_general` field provides coarser categories than `tissue`, useful for cross-tissue analyses:
-```python
-# Broader grouping
-obs_value_filter="tissue_general == 'immune system'"
-
-# Specific tissue
-obs_value_filter="tissue == 'peripheral blood mononuclear cell'"
-```
-
-### Select Only Needed Columns
-Minimize data transfer by specifying only required metadata columns:
-```python
-obs_column_names=["cell_type", "tissue_general", "disease"]  # Not all columns
-```
-
-### Check Dataset Presence for Gene-Specific Queries
-When analyzing specific genes, verify which datasets measured them:
-```python
-presence = cellxgene_census.get_presence_matrix(
-    census,
-    "homo_sapiens",
-    var_value_filter="feature_name in ['CD4', 'CD8A']"
-)
-```
-
-### Two-Step Workflow: Explore Then Query
-First explore metadata to understand available data, then query expression:
-```python
-# Step 1: Explore what's available
-metadata = cellxgene_census.get_obs(
-    census, "homo_sapiens",
-    value_filter="disease == 'COVID-19' and is_primary_data == True",
-    column_names=["cell_type", "tissue_general"]
-)
-print(metadata.value_counts())
-
-# Step 2: Query based on findings
-adata = cellxgene_census.get_anndata(
-    census=census,
-    organism="Homo sapiens",
-    obs_value_filter="disease == 'COVID-19' and cell_type == 'T cell' and is_primary_data == True",
-)
-```
-
-## Available Metadata Fields
-
-### Cell Metadata (obs)
-Key fields for filtering:
-- `cell_type`, `cell_type_ontology_term_id`
-- `tissue`, `tissue_general`, `tissue_ontology_term_id`
-- `disease`, `disease_ontology_term_id`
-- `assay`, `assay_ontology_term_id`
-- `donor_id`, `sex`, `self_reported_ethnicity`
-- `development_stage`, `development_stage_ontology_term_id`
-- `dataset_id`
-- `is_primary_data` (Boolean: True = unique cell)
-
-The current schema includes organism collections beyond human and mouse. Confirm available organisms for the selected release with `list(census["census_data"].keys())`.
-
-### Gene Metadata (var)
-- `feature_id` (Ensembl gene ID, e.g., "ENSG00000161798")
-- `feature_name` (Gene symbol, e.g., "FOXP2")
-- `feature_type`
-- `feature_length` (Gene length in base pairs)
-- `nnz`, `n_measured_obs` (availability summaries useful for checking sparsity and coverage)
-
-## Reference Documentation
-
-This skill includes detailed reference documentation:
-
-### references/census_schema.md
-Comprehensive documentation of:
-- Census data structure and organization
-- All available metadata fields
-- Value filter syntax and operators
-- SOMA object types
-- Data inclusion criteria
-
-**When to read:** When you need detailed schema information, full list of metadata fields, or complex filter syntax.
-
-### references/common_patterns.md
-Examples and patterns for:
-- Exploratory queries (metadata only)
-- Small-to-medium queries (AnnData)
-- Large queries (out-of-core processing)
-- PyTorch integration
-- Spatial Census access patterns
-- Scanpy integration workflows
-- Multi-dataset integration
-- Best practices and common pitfalls
-
-**When to read:** When implementing specific query patterns, looking for code examples, or troubleshooting common issues.
-
-## Common Use Cases
-
-### Use Case 1: Explore Cell Types in a Tissue
-```python
-with cellxgene_census.open_soma() as census:
-    cells = cellxgene_census.get_obs(
-        census, "homo_sapiens",
-        value_filter="tissue_general == 'lung' and is_primary_data == True",
-        column_names=["cell_type"]
-    )
-    print(cells["cell_type"].value_counts())
-```
-
-### Use Case 2: Query Marker Gene Expression
-```python
-with cellxgene_census.open_soma() as census:
-    adata = cellxgene_census.get_anndata(
-        census=census,
-        organism="Homo sapiens",
-        var_value_filter="feature_name in ['CD4', 'CD8A', 'CD19']",
-        obs_value_filter="cell_type in ['T cell', 'B cell'] and is_primary_data == True",
-    )
-```
-
-### Use Case 3: Train Cell Type Classifier
-```python
-import tiledbsoma as soma
-from tiledbsoma_ml import ExperimentDataset, experiment_dataloader
-
-with cellxgene_census.open_soma() as census:
-    experiment = census["census_data"]["homo_sapiens"]
-    with experiment.axis_query(
-        measurement_name="RNA",
-        obs_query=soma.AxisQuery(value_filter="is_primary_data == True"),
-    ) as query:
-        dataset = ExperimentDataset(
-            query=query,
-            layer_name="raw",
-            obs_column_names=["cell_type"],
-            batch_size=128,
-            shuffle=True,
-        )
-        dataloader = experiment_dataloader(dataset)
-
-        for X, obs in dataloader:
-            labels = obs["cell_type"]
-            # Training logic
-            pass
-```
-
-### Use Case 4: Cross-Tissue Analysis
-```python
-with cellxgene_census.open_soma() as census:
-    adata = cellxgene_census.get_anndata(
-        census=census,
-        organism="Homo sapiens",
-        obs_value_filter="cell_type == 'macrophage' and tissue_general in ['lung', 'liver', 'brain'] and is_primary_data == True",
-    )
-
-    # Analyze macrophage differences across tissues
-    sc.tl.rank_genes_groups(adata, groupby="tissue_general")
-```
-
-## Troubleshooting
-
-### Query Returns Too Many Cells
-- Add more specific filters to reduce scope
-- Use `tissue` instead of `tissue_general` for finer granularity
-- Filter by specific `dataset_id` if known
-- Switch to out-of-core processing for large queries
-
-### Memory Errors
-- Reduce query scope with more restrictive filters
-- Select fewer genes with `var_value_filter`
-- Use out-of-core processing with `axis_query()`
-- Process data in batches
-
-### Duplicate Cells in Results
-- Always include `is_primary_data == True` in filters
-- Check if intentionally querying across multiple datasets
-
-### Gene Not Found
-- Verify gene name spelling (case-sensitive)
-- Try Ensembl ID with `feature_id` instead of `feature_name`
-- Check dataset presence matrix to see if gene was measured
-- Some genes may have been filtered during Census construction
-
-### Version Inconsistencies
-- Always specify `census_version` explicitly
-- Use same version across all analyses
-- Check release notes for version-specific changes

package/skills/cirq/SKILL.md

@@ -1,370 +0,0 @@
----
-name: cirq
-description: Google quantum computing framework. Use when targeting Google Quantum AI hardware, designing noise-aware circuits, or running quantum characterization experiments. Best for Google hardware, noise modeling, and low-level circuit design. For IBM hardware use qiskit; for quantum ML with autodiff use pennylane; for physics simulations use qutip.
-license: Apache-2.0 license
-allowed-tools: Read Write Edit Bash
-metadata:
-  version: "1.0"
-  skill-author: K-Dense Inc.
----
-
-# Cirq - Quantum Computing with Python
-
-Cirq is Google Quantum AI's open-source framework for designing, simulating, and running quantum circuits on quantum computers and simulators.
-
-## When to Use This Skill
-
-Use this skill when:
-- Building, simulating, or optimizing NISQ circuits in Python
-- Running jobs on Google Quantum AI processors (via `cirq-google`) or partner backends (IonQ, Azure Quantum, AQT, Pasqal)
-- Modeling noise, compiling to hardware gatesets, or designing characterization experiments
-- Using parameter sweeps, transformers, or the ReCirq experiment patterns
-
-For IBM hardware use **qiskit**; for quantum ML with autodiff use **pennylane**; for physics simulations use **qutip**.
-
-## Installation
-
-Requires Python 3.11+. Current stable release: **1.6.1** (August 2025). Vendor packages share the same version number.
-
-```bash
-uv pip install "cirq==1.6.1"
-```
-
-For hardware integration (pin matching versions for reproducibility):
-```bash
-# Google Quantum Engine (requires approved GCP project access)
-uv pip install "cirq-google==1.6.1"
-
-# IonQ
-uv pip install "cirq-ionq==1.6.1"
-
-# AQT (Alpine Quantum Technologies)
-uv pip install "cirq-aqt==1.6.1"
-
-# Pasqal
-uv pip install "cirq-pasqal==1.6.1"
-
-# Azure Quantum (IonQ, Honeywell/Quantinuum backends)
-uv pip install "azure-quantum[cirq]"
-```
-
-For latest features during development, omit version pins; for production or hardware runs, pin all packages to the same Cirq release.
-
-## Quick Start
-
-### Basic Circuit
-
-```python
-import cirq
-import numpy as np
-
-# Create qubits
-q0, q1 = cirq.LineQubit.range(2)
-
-# Build circuit
-circuit = cirq.Circuit(
-    cirq.H(q0),              # Hadamard on q0
-    cirq.CNOT(q0, q1),       # CNOT with q0 control, q1 target
-    cirq.measure(q0, q1, key='result')
-)
-
-print(circuit)
-
-# Simulate
-simulator = cirq.Simulator()
-result = simulator.run(circuit, repetitions=1000)
-
-# Display results
-print(result.histogram(key='result'))
-```
-
-### Parameterized Circuit
-
-```python
-import sympy
-
-# Define symbolic parameter
-theta = sympy.Symbol('theta')
-
-# Create parameterized circuit
-circuit = cirq.Circuit(
-    cirq.ry(theta)(q0),
-    cirq.measure(q0, key='m')
-)
-
-# Sweep over parameter values
-sweep = cirq.Linspace('theta', start=0, stop=2*np.pi, length=20)
-results = simulator.run_sweep(circuit, params=sweep, repetitions=1000)
-
-# Process results
-for params, result in zip(sweep, results):
-    theta_val = params['theta']
-    counts = result.histogram(key='m')
-    print(f"θ={theta_val:.2f}: {counts}")
-```
-
-## Core Capabilities
-
-### Circuit Building
-For comprehensive information about building quantum circuits, including qubits, gates, operations, custom gates, and circuit patterns, see:
-- **[references/building.md](references/building.md)** - Complete guide to circuit construction
-
-Common topics:
-- Qubit types (GridQubit, LineQubit, NamedQubit)
-- Single and two-qubit gates
-- Parameterized gates and operations
-- Custom gate decomposition
-- Circuit organization with moments
-- Standard circuit patterns (Bell states, GHZ, QFT)
-- Import/export (OpenQASM, JSON)
-- Working with qudits and observables
-
-### Simulation
-For detailed information about simulating quantum circuits, including exact simulation, noisy simulation, parameter sweeps, and the Quantum Virtual Machine, see:
-- **[references/simulation.md](references/simulation.md)** - Complete guide to quantum simulation
-
-Common topics:
-- Exact simulation (state vector, density matrix)
-- Sampling and measurements
-- Parameter sweeps (single and multiple parameters)
-- Noisy simulation
-- State histograms and visualization
-- Quantum Virtual Machine (QVM)
-- Expectation values and observables
-- Performance optimization
-
-### Circuit Transformation
-For information about optimizing, compiling, and manipulating quantum circuits, see:
-- **[references/transformation.md](references/transformation.md)** - Complete guide to circuit transformations
-
-Common topics:
-- Transformer framework
-- Gate decomposition
-- Circuit optimization (merge gates, eject Z gates, drop negligible operations)
-- Circuit compilation for hardware
-- Qubit routing and SWAP insertion
-- Custom transformers
-- Transformation pipelines
-
-### Hardware Integration
-For information about running circuits on real quantum hardware from various providers, see:
-- **[references/hardware.md](references/hardware.md)** - Complete guide to hardware integration
-
-Supported providers:
-- **Google Quantum AI** (`cirq-google`) — Sycamore, Weber, Willow processors via Quantum Engine (restricted access; requires approved GCP project)
-- **IonQ** (`cirq-ionq`) — trapped-ion QPUs and simulators
-- **Azure Quantum** (`azure-quantum[cirq]`) — IonQ and Honeywell/Quantinuum backends
-- **AQT** (`cirq-aqt`) — Alpine Quantum Technologies
-- **Pasqal** (`cirq-pasqal`) — neutral-atom devices
-
-Topics include device representation, qubit selection, authentication, job management, and circuit optimization for hardware. See [Access and authentication](https://quantumai.google/cirq/google/access) for Google Cloud setup.
-
-### Noise Modeling
-For information about modeling noise, noisy simulation, characterization, and error mitigation, see:
-- **[references/noise.md](references/noise.md)** - Complete guide to noise modeling
-
-Common topics:
-- Noise channels (depolarizing, amplitude damping, phase damping)
-- Noise models (constant, gate-specific, qubit-specific, thermal)
-- Adding noise to circuits
-- Readout noise
-- Noise characterization (randomized benchmarking, XEB)
-- Noise visualization (heatmaps)
-- Error mitigation techniques
-
-### Quantum Experiments
-For information about designing experiments, parameter sweeps, data collection, and using the ReCirq framework, see:
-- **[references/experiments.md](references/experiments.md)** - Complete guide to quantum experiments
-
-Common topics:
-- Experiment design patterns
-- Parameter sweeps and data collection
-- ReCirq framework structure
-- Common algorithms (VQE, QAOA, QPE)
-- Data analysis and visualization
-- Statistical analysis and fidelity estimation
-- Parallel data collection
-
-## Common Patterns
-
-### Variational Algorithm Template
-
-```python
-import scipy.optimize
-
-def variational_algorithm(ansatz, cost_function, initial_params):
-    """Template for variational quantum algorithms."""
-
-    def objective(params):
-        circuit = ansatz(params)
-        simulator = cirq.Simulator()
-        result = simulator.simulate(circuit)
-        return cost_function(result)
-
-    # Optimize
-    result = scipy.optimize.minimize(
-        objective,
-        initial_params,
-        method='COBYLA'
-    )
-
-    return result
-
-# Define ansatz
-def my_ansatz(params):
-    q = cirq.LineQubit(0)
-    return cirq.Circuit(
-        cirq.ry(params[0])(q),
-        cirq.rz(params[1])(q)
-    )
-
-# Define cost function
-def my_cost(result):
-    state = result.final_state_vector
-    # Calculate cost based on state
-    return np.real(state[0])
-
-# Run optimization
-result = variational_algorithm(my_ansatz, my_cost, [0.0, 0.0])
-```
-
-### Hardware Execution Template
-
-```python
-import os
-
-def run_on_hardware(circuit, provider='google', processor_id=None, repetitions=1000):
-    """Template for running on quantum hardware."""
-
-    if provider == 'google':
-        import cirq_google as cg
-
-        project_id = os.environ['GOOGLE_CLOUD_PROJECT']
-        engine = cg.Engine(project_id=project_id)
-
-        # List available processors: engine.list_processors()
-        processor_id = processor_id or 'weber'  # use your assigned processor_id
-        sampler = engine.get_sampler(processor_id=processor_id)
-        return sampler.run(circuit, repetitions=repetitions)
-
-    elif provider == 'ionq':
-        import cirq_ionq as ionq
-
-        # Requires IONQ_API_KEY in environment
-        service = ionq.Service()
-        return service.run(circuit, repetitions=repetitions, target='qpu')
-
-    elif provider == 'azure':
-        from azure.quantum.cirq import AzureQuantumService
-
-        service = AzureQuantumService(
-            resource_id=os.environ['AZURE_QUANTUM_RESOURCE_ID'],
-            location=os.environ['AZURE_QUANTUM_LOCATION'],
-        )
-        return service.run(circuit, repetitions=repetitions, target='ionq.qpu')
-
-    else:
-        raise ValueError(f"Unknown provider: {provider}")
-```
-
-### Noise Study Template
-
-```python
-def noise_comparison_study(circuit, noise_levels):
-    """Compare circuit performance at different noise levels."""
-
-    results = {}
-
-    for noise_level in noise_levels:
-        # Create noisy circuit
-        noisy_circuit = circuit.with_noise(cirq.depolarize(p=noise_level))
-
-        # Simulate
-        simulator = cirq.DensityMatrixSimulator()
-        result = simulator.run(noisy_circuit, repetitions=1000)
-
-        # Analyze
-        results[noise_level] = {
-            'histogram': result.histogram(key='result'),
-            'dominant_state': max(
-                result.histogram(key='result').items(),
-                key=lambda x: x[1]
-            )
-        }
-
-    return results
-
-# Run study
-noise_levels = [0.0, 0.001, 0.01, 0.05, 0.1]
-results = noise_comparison_study(circuit, noise_levels)
-```
-
-## Best Practices
-
-1. **Circuit Design**
-   - Use appropriate qubit types for your topology
-   - Keep circuits modular and reusable
-   - Label measurements with descriptive keys
-   - Validate circuits against device constraints before execution
-
-2. **Simulation**
-   - Use state vector simulation for pure states (more efficient)
-   - Use density matrix simulation only when needed (mixed states, noise)
-   - Leverage parameter sweeps instead of individual runs
-   - Monitor memory usage for large systems (2^n grows quickly)
-
-3. **Hardware Execution**
-   - Always test on simulators first
-   - Select best qubits using calibration data
-   - Optimize circuits for target hardware gateset
-   - Implement error mitigation for production runs
-   - Store expensive hardware results immediately
-
-4. **Circuit Optimization**
-   - Start with high-level built-in transformers
-   - Chain multiple optimizations in sequence
-   - Track depth and gate count reduction
-   - Validate correctness after transformation
-
-5. **Noise Modeling**
-   - Use realistic noise models from calibration data
-   - Include all error sources (gate, decoherence, readout)
-   - Characterize before mitigating
-   - Keep circuits shallow to minimize noise accumulation
-
-6. **Experiments**
-   - Structure experiments with clear separation (data generation, collection, analysis)
-   - Use ReCirq patterns for reproducibility
-   - Save intermediate results frequently
-   - Parallelize independent tasks
-   - Document thoroughly with metadata
-
-## Additional Resources
-
-- **Official Documentation**: https://quantumai.google/cirq
-- **API Reference**: https://quantumai.google/reference/python/cirq
-- **Tutorials**: https://quantumai.google/cirq/tutorials
-- **Examples**: https://github.com/quantumlib/Cirq/tree/main/examples
-- **Version policy**: https://quantumai.google/cirq/dev/versions
-- **ReCirq**: https://github.com/quantumlib/ReCirq
-
-## Common Issues
-
-**Circuit too deep for hardware:**
-- Use circuit optimization transformers to reduce depth
-- See `transformation.md` for optimization techniques
-
-**Memory issues with simulation:**
-- Switch from density matrix to state vector simulator
-- Reduce number of qubits or use stabilizer simulator for Clifford circuits
-
-**Device validation errors:**
-- Check qubit connectivity with device.metadata.nx_graph
-- Decompose gates to device-native gateset
-- See `hardware.md` for device-specific compilation
-
-**Noisy simulation too slow:**
-- Density matrix simulation is O(2^2n) - consider reducing qubits
-- Use noise models selectively on critical operations only
-- See `simulation.md` for performance optimization
-