isage-tooluse-sias 0.1.0__py3-none-any.whl

isage_tooluse_sias-0.1.0.dist-info/METADATA

@@ -0,0 +1,186 @@
+Metadata-Version: 2.4
+Name: isage-tooluse-sias
+Version: 0.1.0
+Summary: SAGE Tool Use SIAS - Sample-Importance-Aware Selection for tool selection and agent training
+Author-email: IntelliStream Team <shuhao_zhang@hust.edu.cn>
+License: MIT
+Project-URL: Homepage, https://github.com/intellistream/sage-sias
+Project-URL: Repository, https://github.com/intellistream/sage-sias
+Project-URL: Documentation, https://github.com/intellistream/sage-sias#readme
+Project-URL: Bug Tracker, https://github.com/intellistream/sage-sias/issues
+Keywords: sias,tool-selection,tool-use,sample-selection,importance-aware,continual-learning,coreset,active-learning,agent-training
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.20.0
+Requires-Dist: typing-extensions>=4.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: ruff>=0.8.4; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Provides-Extra: torch
+Requires-Dist: torch>=2.0.0; extra == "torch"
+Provides-Extra: all
+Requires-Dist: isage-sias[dev,torch]; extra == "all"
+Dynamic: license-file
+
+# SAGE Tool Use SIAS (Sample-Importance-Aware Selection)
+
+**Tool selection algorithm using sample-importance-aware selection for agent training and tool curation**
+
+[![PyPI version](https://badge.fury.io/py/isage-tooluse-sias.svg)](https://badge.fury.io/py/isage-tooluse-sias)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+## 🎯 Overview
+
+`sage-tooluse-sias` provides Sample-Importance-Aware Selection algorithms specifically designed for:
+
+- **Tool Selection**: Select important tools for agent use
+- **Agent Training**: Select important trajectories for fine-tuning
+- **Continual Learning**: Efficient sample selection for continual/lifelong learning
+- **Tool/Trajectory Curation**: Curate representative samples for agent development
+
+## 📦 Installation
+
+```bash
+# Basic installation
+pip install isage-tooluse-sias
+
+# With PyTorch support
+pip install isage-tooluse-sias[torch]
+
+# Development installation
+pip install isage-tooluse-sias[dev]
+```
+
+## 🚀 Quick Start
+
+### Continual Learning
+
+```python
+from sage_sias import ContinualLearner
+
+# Create continual learner
+learner = ContinualLearner(
+    buffer_size=1000,
+    selection_strategy="importance"
+)
+
+# Add samples
+for data, label in stream:
+    learner.add_sample(data, label)
+
+# Get selected samples
+important_samples = learner.get_buffer()
+```
+
+### Coreset Selection
+
+```python
+from sage_sias import CoresetSelector
+
+# Create coreset selector
+selector = CoresetSelector(
+    target_size=100,
+    method="kmeans++"
+)
+
+# Select representative samples
+coreset = selector.select(dataset, features)
+```
+
+## 📚 Key Components
+
+### 1. **Continual Learner** (`continual_learner.py`)
+
+Manages sample selection for continual learning:
+- Buffer management with importance-based eviction
+- Multiple selection strategies (random, importance, diversity)
+- Support for experience replay
+
+### 2. **Coreset Selector** (`coreset_selector.py`)
+
+Selects representative subsets:
+- K-means++ based selection
+- Diversity-aware sampling
+- Importance scoring
+- Support for large-scale datasets
+
+### 3. **Types** (`types.py`)
+
+Common data types and protocols:
+- Sample representation
+- Importance scoring interfaces
+- Selection strategies
+
+## 🔧 Architecture
+
+```
+sage_sias/
+├── continual_learner.py    # Continual learning with buffer management
+├── coreset_selector.py      # Coreset selection algorithms
+├── types.py                 # Common types and protocols
+└── __init__.py             # Public API exports
+```
+
+## 🎓 Use Cases
+
+1. **Agent Training**: Select important trajectories for fine-tuning
+2. **Data Pruning**: Reduce dataset size while maintaining performance
+3. **Active Learning**: Query most informative samples
+4. **Memory Management**: Maintain representative samples in limited buffers
+5. **Transfer Learning**: Select relevant samples for adaptation
+
+## 🔗 Integration with SAGE
+
+This package is part of the SAGE ecosystem but can be used independently:
+
+```python
+# Standalone usage
+from sage_sias import ContinualLearner, CoresetSelector
+
+# With SAGE agentic (optional)
+from sage_agentic import AgentTrainer
+from sage_sias import CoresetSelector
+
+trainer = AgentTrainer()
+selector = CoresetSelector(target_size=100)
+important_trajectories = selector.select(all_trajectories)
+trainer.train(important_trajectories)
+```
+
+## 📖 Documentation
+
+- **Repository**: https://github.com/intellistream/sage-tooluse-sias
+- **SAGE Documentation**: https://intellistream.github.io/SAGE-Pub/
+- **Issues**: https://github.com/intellistream/sage-tooluse-sias/issues
+
+## 🤝 Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.
+
+## 📄 License
+
+MIT License - see [LICENSE](LICENSE) file for details.
+
+## 🙏 Acknowledgments
+
+Originally part of the [SAGE](https://github.com/intellistream/SAGE) framework, now maintained as an independent package for broader community use.
+
+## 📧 Contact
+
+- **Team**: IntelliStream Team
+- **Email**: shuhao_zhang@hust.edu.cn
+- **GitHub**: https://github.com/intellistream

isage_tooluse_sias-0.1.0.dist-info/RECORD

@@ -0,0 +1,10 @@
+isage_tooluse_sias-0.1.0.dist-info/licenses/LICENSE,sha256=FviS_SBgJ3MCvV517X_20AfytowsiItMOu_l3GJWFnI,1080
+sage_sias/__init__.py,sha256=WG9xoaAvrYdyFFvC1g34mVL_IBapR2ix0VN9aazs--o,962
+sage_sias/continual_learner.py,sha256=LA_o2k81VO287jLvt7HmFvAhNtpJv-Zz7xjXqi0YPAo,6240
+sage_sias/coreset_selector.py,sha256=L8JJfUvFwQl0GQOSWVZ2bl9uxSB7q-WCuoQpiawU9TU,10756
+sage_sias/setup.py,sha256=GEYqJMCRQHES0U_92Uo9gUY9H-XnqiHX1UJ28fDVDJk,122
+sage_sias/types.py,sha256=iAlEU2Obxp5F0iTxpwGXLoOA3LyiOKqY-Oqm133tfQw,2418
+isage_tooluse_sias-0.1.0.dist-info/METADATA,sha256=um7pu6_rSVvQ2WTxE4x-o8DfGmO0QPrSq1fa-lgsNV0,5897
+isage_tooluse_sias-0.1.0.dist-info/WHEEL,sha256=wUyA8OaulRlbfwMtmQsvNngGrxQHAvkKcvRmdizlJi0,92
+isage_tooluse_sias-0.1.0.dist-info/top_level.txt,sha256=OCb1-BhPYZaVCErrA9_R5Eln0UgS6kq2KiEER_c87vY,10
+isage_tooluse_sias-0.1.0.dist-info/RECORD,,

isage_tooluse_sias-0.1.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (80.10.2)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

isage_tooluse_sias-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025-2026 IntelliStream Team
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

isage_tooluse_sias-0.1.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+sage_sias

sage_sias/__init__.py

@@ -0,0 +1,30 @@
+"""SIAS - Streaming Importance-Aware Agent System.
+
+Core components for agent tool selection and sample importance:
+- CoresetSelector: Importance-aware sample selection
+- OnlineContinualLearner: Experience replay with importance weighting
+- SelectionSummary: Summary statistics for selection operations
+
+Usage:
+    from sage_agentic.sias import CoresetSelector, OnlineContinualLearner
+
+    selector = CoresetSelector(strategy="hybrid")
+    selected = selector.select(samples, target_size=1000)
+
+    learner = OnlineContinualLearner(buffer_size=2048, replay_ratio=0.25)
+    batch = learner.update_buffer(new_samples)
+"""
+
+from .coreset_selector import CoresetSelector, SelectionSummary
+from .continual_learner import OnlineContinualLearner
+from .types import ImportanceScore, SampleWithImportance
+
+__all__ = [
+    "CoresetSelector",
+    "OnlineContinualLearner",
+    "SelectionSummary",
+    "ImportanceScore",
+    "SampleWithImportance",
+]
+
+__version__ = "0.1.0"

sage_sias/continual_learner.py

@@ -0,0 +1,184 @@
+"""
+Online Continual Learning with Experience Replay
+
+Implements an experience replay buffer for online/incremental training that
+prevents catastrophic forgetting. The buffer is managed using coreset selection
+to retain the most valuable samples.
+
+This is a core component of SIAS (Streaming Importance-Aware Agent System).
+"""
+
+from __future__ import annotations
+
+import random
+from typing import Iterable, Optional, Sequence
+
+from .coreset_selector import CoresetSelector, SampleT, SelectionSummary
+
+
+class OnlineContinualLearner:
+    """
+    Maintain a replay buffer for online continual learning.
+
+    Implements experience replay to prevent catastrophic forgetting during
+    incremental/online training. The buffer is managed using coreset selection
+    to keep the most valuable samples.
+
+    Attributes:
+        buffer_size: Maximum number of samples to keep in buffer
+        replay_ratio: Ratio of replay samples to add per batch (e.g., 0.25 = 25%)
+        selector: CoresetSelector for buffer management
+
+    Example:
+        >>> learner = OnlineContinualLearner(buffer_size=2048, replay_ratio=0.25)
+        >>> for new_batch in data_stream:
+        ...     training_batch = learner.update_buffer(new_batch)
+        ...     train_step(training_batch)
+    """
+
+    def __init__(
+        self,
+        buffer_size: int = 2048,
+        replay_ratio: float = 0.3,
+        selector: Optional[CoresetSelector] = None,
+        random_seed: int = 17,
+    ) -> None:
+        """
+        Initialize OnlineContinualLearner.
+
+        Args:
+            buffer_size: Maximum samples to keep in replay buffer
+            replay_ratio: Fraction of batch size to sample from buffer
+            selector: CoresetSelector for buffer management (default: hybrid)
+            random_seed: Random seed for reproducibility
+        """
+        self.buffer_size = buffer_size
+        self.replay_ratio = replay_ratio
+        self.selector = selector or CoresetSelector(strategy="hybrid")
+        self._buffer: list[SampleT] = []
+        self._metrics: dict[str, float] = {}
+        self._rng = random.Random(random_seed)
+
+    @property
+    def buffer(self) -> list[SampleT]:
+        """Access the current buffer (read-only view)."""
+        return list(self._buffer)
+
+    @property
+    def buffer_len(self) -> int:
+        """Current number of samples in buffer."""
+        return len(self._buffer)
+
+    def update_buffer(
+        self,
+        new_samples: Sequence[SampleT],
+        metrics: Optional[dict[str, float]] = None,
+    ) -> list[SampleT]:
+        """
+        Update buffer with new samples and return training batch.
+
+        This method:
+        1. Adds new samples to the buffer
+        2. If buffer exceeds size limit, uses coreset selection to prune
+        3. Returns new samples + replay samples for training
+
+        Args:
+            new_samples: New samples to add to buffer
+            metrics: Optional metrics dict mapping sample_id to importance score
+
+        Returns:
+            Training batch combining new samples with replay samples
+        """
+        if not new_samples:
+            return list(self._buffer)
+
+        if metrics:
+            self._metrics.update(metrics)
+
+        # Combine buffer with new samples
+        combined = list(self._buffer) + list(new_samples)
+
+        # Prune if over capacity
+        if len(combined) > self.buffer_size:
+            combined = self.selector.select(
+                combined,
+                target_size=self.buffer_size,
+                metrics=self._metrics,
+            )
+            # Clean up metrics for removed samples
+            combined_ids = {self._get_sample_id(sample) for sample in combined}
+            self._metrics = {k: v for k, v in self._metrics.items() if k in combined_ids}
+
+        self._buffer = combined
+        return self._assemble_training_batch(new_samples)
+
+    def _assemble_training_batch(
+        self,
+        new_samples: Sequence[SampleT],
+    ) -> list[SampleT]:
+        """Combine new samples with replay samples."""
+        new_ids = {self._get_sample_id(s) for s in new_samples}
+        replay = self.sample_replay(len(new_samples), exclude=new_ids)
+        return list(new_samples) + replay
+
+    def sample_replay(
+        self,
+        new_batch_size: int,
+        *,
+        exclude: Optional[Iterable[str]] = None,
+    ) -> list[SampleT]:
+        """
+        Sample from replay buffer.
+
+        Args:
+            new_batch_size: Size of new batch (replay size = batch_size * ratio)
+            exclude: Sample IDs to exclude from replay
+
+        Returns:
+            List of replay samples
+        """
+        if not self._buffer or self.replay_ratio <= 0:
+            return []
+
+        exclude = set(exclude or [])
+        available = [
+            sample for sample in self._buffer if self._get_sample_id(sample) not in exclude
+        ]
+        if not available:
+            return []
+
+        replay_size = max(1, int(new_batch_size * self.replay_ratio))
+        replay_size = min(replay_size, len(available))
+        return self._rng.sample(available, replay_size)
+
+    def buffer_snapshot(self) -> list[SampleT]:
+        """Return a copy of the current buffer."""
+        return list(self._buffer)
+
+    def buffer_summary(self) -> SelectionSummary:
+        """Get summary statistics for the buffer."""
+        return SelectionSummary(
+            total_samples=len(self._buffer),
+            selected_samples=len(self._buffer),
+            strategy=f"buffer:{self.selector.strategy}",
+        )
+
+    def clear(self) -> None:
+        """Clear the buffer and metrics."""
+        self._buffer = []
+        self._metrics = {}
+
+    def update_metrics(self, metrics: dict[str, float]) -> None:
+        """
+        Update importance metrics for samples in buffer.
+
+        Args:
+            metrics: Dict mapping sample_id to importance score
+        """
+        self._metrics.update(metrics)
+
+    def _get_sample_id(self, sample: SampleT) -> str:
+        """Get sample_id from sample (supports dict or object)."""
+        if isinstance(sample, dict):
+            return sample.get("sample_id", sample.get("dialog_id", str(id(sample))))
+        return getattr(sample, "sample_id", getattr(sample, "dialog_id", str(id(sample))))

sage_sias/coreset_selector.py

@@ -0,0 +1,302 @@
+"""
+Coreset Selection for Efficient Training
+
+Implements lightweight coreset selection strategies that identify the most
+valuable samples for training, reducing computational cost while maintaining
+model quality.
+
+Strategies:
+    - loss_topk: Select samples with highest loss (most informative)
+    - diversity: Select samples maximizing coverage of feature space
+    - hybrid: Combination of loss-based and diversity-based selection
+    - random: Uniform random sampling (baseline)
+
+This is a core component of SIAS (Streaming Importance-Aware Agent System).
+"""
+
+from __future__ import annotations
+
+import math
+import random
+import re
+from collections import Counter
+from dataclasses import dataclass
+from typing import Any, Optional, Protocol, Sequence, runtime_checkable
+
+
+@dataclass(slots=True)
+class SelectionSummary:
+    """Summary statistics for a selection operation."""
+
+    total_samples: int
+    selected_samples: int
+    strategy: str
+
+
+@runtime_checkable
+class SampleProtocol(Protocol):
+    """Protocol for samples that can be used with CoresetSelector."""
+
+    @property
+    def sample_id(self) -> str:
+        """Unique identifier for the sample."""
+        ...
+
+    @property
+    def text(self) -> str:
+        """Text content of the sample."""
+        ...
+
+    @property
+    def metadata(self) -> dict[str, Any]:
+        """Metadata dictionary."""
+        ...
+
+
+# Type alias for any sample that implements the protocol
+SampleT = Any  # Should implement SampleProtocol
+
+
+class CoresetSelector:
+    """
+    Implements lightweight coreset selection strategies.
+
+    This class provides several strategies for selecting a representative
+    subset of samples from a larger dataset, optimizing for training efficiency.
+
+    Attributes:
+        strategy: Selection strategy ("loss_topk", "diversity", "hybrid", "random")
+        metric_key: Key in metadata to use for loss-based selection
+        diversity_temperature: Temperature for diversity scoring
+        random_seed: Seed for reproducibility
+
+    Example:
+        >>> selector = CoresetSelector(strategy="hybrid")
+        >>> selected = selector.select(samples, target_size=1000)
+        >>> print(f"Selected {len(selected)} from {len(samples)} samples")
+    """
+
+    STRATEGIES = ("loss_topk", "diversity", "hybrid", "random")
+
+    def __init__(
+        self,
+        strategy: str = "loss_topk",
+        metric_key: str = "loss",
+        diversity_temperature: float = 0.7,
+        random_seed: int = 13,
+    ) -> None:
+        """
+        Initialize CoresetSelector.
+
+        Args:
+            strategy: Selection strategy to use
+            metric_key: Metadata key for loss-based selection
+            diversity_temperature: Temperature for diversity scoring
+            random_seed: Random seed for reproducibility
+        """
+        if strategy not in self.STRATEGIES:
+            raise ValueError(f"Unknown strategy: {strategy}. Choose from {self.STRATEGIES}")
+
+        self.strategy = strategy
+        self.metric_key = metric_key
+        self.diversity_temperature = diversity_temperature
+        self._rng = random.Random(random_seed)
+
+    # ------------------------------------------------------------------
+    # Public API
+    # ------------------------------------------------------------------
+    def select(
+        self,
+        samples: Sequence[SampleT],
+        *,
+        target_size: Optional[int],
+        metrics: Optional[dict[str, float]] = None,
+    ) -> list[SampleT]:
+        """
+        Select a subset of samples using the configured strategy.
+
+        Args:
+            samples: Input samples to select from
+            target_size: Number of samples to select (None = keep all)
+            metrics: Optional external metrics dict mapping sample_id to score
+
+        Returns:
+            List of selected samples
+        """
+        if target_size is None or target_size <= 0 or target_size >= len(samples):
+            return list(samples)
+
+        if self.strategy == "loss_topk":
+            return self._select_loss(samples, target_size, metrics)
+        if self.strategy == "diversity":
+            return self._select_diversity(samples, target_size)
+        if self.strategy == "hybrid":
+            return self._select_hybrid(samples, target_size, metrics)
+        return self._select_random(samples, target_size)
+
+    def summary(self, original_size: int, selected_size: int) -> SelectionSummary:
+        """Create a summary of the selection operation."""
+        return SelectionSummary(
+            total_samples=original_size,
+            selected_samples=selected_size,
+            strategy=self.strategy,
+        )
+
+    # ------------------------------------------------------------------
+    # Selection Strategies
+    # ------------------------------------------------------------------
+    def _select_loss(
+        self,
+        samples: Sequence[SampleT],
+        target_size: int,
+        metrics: Optional[dict[str, float]],
+    ) -> list[SampleT]:
+        """Select samples with highest loss/importance scores."""
+
+        def score(sample: SampleT) -> float:
+            sample_id = self._get_sample_id(sample)
+            if metrics and sample_id in metrics:
+                return metrics[sample_id]
+            meta = self._get_metadata(sample)
+            meta_val = meta.get(self.metric_key)
+            if isinstance(meta_val, (int, float)):
+                return float(meta_val)
+            return 0.0
+
+        ranked = sorted(samples, key=score, reverse=True)
+        return list(ranked[:target_size])
+
+    def _select_random(
+        self,
+        samples: Sequence[SampleT],
+        target_size: int,
+    ) -> list[SampleT]:
+        """Uniform random sampling."""
+        return self._rng.sample(list(samples), target_size)
+
+    def _select_hybrid(
+        self,
+        samples: Sequence[SampleT],
+        target_size: int,
+        metrics: Optional[dict[str, float]],
+    ) -> list[SampleT]:
+        """Hybrid selection: 60% loss-based + 40% diversity-based."""
+        loss_portion = int(target_size * 0.6)
+        div_portion = target_size - loss_portion
+
+        # First select high-loss samples
+        top_loss = self._select_loss(samples, loss_portion or 1, metrics)
+        top_loss_ids = {self._get_sample_id(s) for s in top_loss}
+
+        # Then select diverse samples from remaining
+        remaining = [s for s in samples if self._get_sample_id(s) not in top_loss_ids]
+        if not remaining:
+            return top_loss
+
+        diversity = self._select_diversity(remaining, max(div_portion, 1))
+        merged = (top_loss + diversity)[:target_size]
+        return merged
+
+    def _select_diversity(
+        self,
+        samples: Sequence[SampleT],
+        target_size: int,
+    ) -> list[SampleT]:
+        """Select samples maximizing feature space coverage."""
+        if not samples:
+            return []
+
+        # Extract features for all samples
+        features = {
+            self._get_sample_id(sample): self._text_features(self._get_text(sample))
+            for sample in samples
+        }
+
+        selected: list[SampleT] = []
+        candidates = list(samples)
+
+        # Start with the sample that has the highest token variance
+        scores = {
+            self._get_sample_id(sample): self._feature_norm(features[self._get_sample_id(sample)])
+            for sample in samples
+        }
+        first = max(candidates, key=lambda s: scores.get(self._get_sample_id(s), 0.0))
+        selected.append(first)
+        candidates = [s for s in candidates if self._get_sample_id(s) != self._get_sample_id(first)]
+
+        # Iteratively select most diverse samples
+        while candidates and len(selected) < target_size:
+            best_candidate = max(
+                candidates,
+                key=lambda sample: self._min_distance(sample, selected, features),
+            )
+            selected.append(best_candidate)
+            candidates = [
+                s
+                for s in candidates
+                if self._get_sample_id(s) != self._get_sample_id(best_candidate)
+            ]
+
+        return selected
+
+    # ------------------------------------------------------------------
+    # Feature Extraction Helpers
+    # ------------------------------------------------------------------
+    def _text_features(self, text: str) -> Counter:
+        """Extract normalized token frequency features from text."""
+        tokens = re.findall(r"[a-zA-Z0-9_]+", text.lower())
+        filtered = [token for token in tokens if len(token) > 2]
+        counts = Counter(filtered)
+        total = sum(counts.values()) or 1.0
+        for key in counts:
+            counts[key] /= total
+        return counts
+
+    def _feature_norm(self, features: Counter) -> float:
+        """Compute L2 norm of feature vector."""
+        return math.sqrt(sum(value * value for value in features.values()))
+
+    def _cosine_similarity(self, left: Counter, right: Counter) -> float:
+        """Compute cosine similarity between two feature vectors."""
+        keys = left.keys() & right.keys()
+        if not keys:
+            return 0.0
+        return sum(left[key] * right[key] for key in keys)
+
+    def _min_distance(
+        self,
+        candidate: SampleT,
+        selected: Sequence[SampleT],
+        features: dict[str, Counter],
+    ) -> float:
+        """Compute minimum distance from candidate to selected set."""
+        cand_feat = features[self._get_sample_id(candidate)]
+        if not selected:
+            return 1.0
+        sims = [
+            self._cosine_similarity(cand_feat, features[self._get_sample_id(item)])
+            for item in selected
+        ]
+        similarity = max(sims) if sims else 0.0
+        return 1.0 - similarity
+
+    # ------------------------------------------------------------------
+    # Sample Access Helpers (support both dict and object access)
+    # ------------------------------------------------------------------
+    def _get_sample_id(self, sample: SampleT) -> str:
+        """Get sample_id from sample (supports dict or object)."""
+        if isinstance(sample, dict):
+            return sample.get("sample_id", sample.get("dialog_id", str(id(sample))))
+        return getattr(sample, "sample_id", getattr(sample, "dialog_id", str(id(sample))))
+
+    def _get_text(self, sample: SampleT) -> str:
+        """Get text from sample (supports dict or object)."""
+        if isinstance(sample, dict):
+            return sample.get("text", "")
+        return getattr(sample, "text", "")
+
+    def _get_metadata(self, sample: SampleT) -> dict[str, Any]:
+        """Get metadata from sample (supports dict or object)."""
+        if isinstance(sample, dict):
+            return sample.get("metadata", {})
+        return getattr(sample, "metadata", {})

sage_sias/setup.py

@@ -0,0 +1,6 @@
+"""Setup script for sage-sias package."""
+
+from setuptools import setup
+
+# All configuration is in pyproject.toml
+setup()

sage_sias/types.py

@@ -0,0 +1,94 @@
+"""
+SIAS Core Data Types
+
+Defines the core data structures used across SIAS components.
+These are designed to be independent of specific data sources.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Protocol, runtime_checkable
+
+
+@dataclass(slots=True)
+class SIASSample:
+    """
+    Generic sample container for SIAS algorithms.
+
+    This is a lightweight data class that can wrap samples from various sources.
+    The only required fields are sample_id and text; everything else is optional.
+
+    Attributes:
+        sample_id: Unique identifier for this sample
+        text: The text content (or serialized representation)
+        metadata: Arbitrary metadata dictionary
+        importance_score: SSIS-computed importance score (set during training)
+    """
+
+    sample_id: str
+    text: str
+    metadata: dict[str, Any] = field(default_factory=dict)
+    importance_score: float = 0.0
+
+    def __hash__(self) -> int:
+        return hash(self.sample_id)
+
+    def __eq__(self, other: object) -> bool:
+        if isinstance(other, SIASSample):
+            return self.sample_id == other.sample_id
+        return False
+
+
+@runtime_checkable
+class SampleProtocol(Protocol):
+    """
+    Protocol for samples that can be used with SIAS algorithms.
+
+    Any class with these attributes can be used with CoresetSelector
+    and OnlineContinualLearner without modification.
+    """
+
+    @property
+    def sample_id(self) -> str:
+        """Unique identifier for the sample."""
+        ...
+
+    @property
+    def text(self) -> str:
+        """Text content of the sample."""
+        ...
+
+    @property
+    def metadata(self) -> dict[str, Any]:
+        """Metadata dictionary."""
+        ...
+
+
+# Backward compatibility alias
+# This allows existing code using ProcessedDialog to work with SIAS
+# by implementing the SampleProtocol
+Sample = SIASSample
+
+
+def wrap_sample(
+    sample_id: str,
+    text: str,
+    metadata: dict[str, Any] | None = None,
+    **kwargs: Any,
+) -> SIASSample:
+    """
+    Factory function to create a SIASSample.
+
+    Args:
+        sample_id: Unique identifier
+        text: Text content
+        metadata: Optional metadata dict
+        **kwargs: Additional metadata fields
+
+    Returns:
+        A new SIASSample instance
+    """
+    meta = metadata or {}
+    meta.update(kwargs)
+    return SIASSample(sample_id=sample_id, text=text, metadata=meta)