calt-x 0.1.0__tar.gz

calt_x-0.1.0/PKG-INFO

@@ -0,0 +1,168 @@
+Metadata-Version: 2.4
+Name: calt-x
+Version: 0.1.0
+Summary: A library for computational algebra using Transformers
+Author-email: Yuta Sato <sato.yuta@gmail.com>
+Project-URL: Source, https://github.com/HiroshiKERA/calt
+Project-URL: Issues, https://github.com/HiroshiKERA/calt/issues
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+Requires-Dist: transformers>=4.49.0
+Requires-Dist: omegaconf>=2.3.0
+Requires-Dist: torch>=2.6.0
+Requires-Dist: wandb>=0.15.11
+Requires-Dist: accelerate>=0.29.0
+Requires-Dist: joblib>=1.5.0
+Requires-Dist: sympy>=1.12
+
+# CALT: Computer ALgebra with Transformer
+This project is currently in its initial development phase. The file structure and content are subject to significant changes. Please ensure you are referring to the latest version when using it.
+
+# Environment Setup using Docker
+
+This guide explains how to set up the development environment using Docker.
+
+## Prerequisites
+
+- Docker installed on your system.
+- NVIDIA GPU drivers installed if you plan to use GPU acceleration (`--gpus all` option).
+
+## Build the Docker Image
+
+To build the Docker image, navigate to the `calt` directory and run the following command:
+
+```bash
+make build
+```
+
+Alternatively, you can use the direct Docker command:
+
+```bash
+docker build -t ta-sage .
+```
+
+## Run the Docker Container
+
+To run the Docker container in detached mode with GPU support, execute:
+
+```bash
+make run
+```
+
+The direct Docker command is:
+
+```bash
+docker run --gpus all -d --name ta-sage-container -v "$(pwd)":/app ta-sage tail -f /dev/null
+```
+*Note: When running this command directly in your terminal, `$(pwd)` will resolve to your current working directory. The Makefile uses `$(CURDIR)` which serves the same purpose within the Makefile context.*
+
+## Access the Container
+
+Once the container is running, you can access it using:
+
+```bash
+docker exec -it ta-sage-container bash
+```
+
+## Stop and Remove the Container
+
+To stop and remove the container, you can use:
+
+```bash
+make stop
+```
+
+Or manually:
+
+```bash
+docker stop ta-sage-container
+docker rm ta-sage-container
+```
+
+## Local Setup (without Docker)
+
+This section describes how to set up the environment locally without using Docker. This assumes you have SageMath installed on your system.
+
+### 1. Install SageMath
+
+You can install SageMath using `apt` on Debian/Ubuntu-based systems. It's not necessary to have the absolute latest version.
+
+Install SageMath:
+
+```bash
+sudo apt-get install -y sagemath
+```
+
+### 2. Install Dependencies
+
+Once SageMath is installed, you can install the required Python packages using `sage -pip`.
+
+First, upgrade pip:
+
+```bash
+sage -pip install --upgrade pip
+```
+
+Next, install the Python dependencies:
+
+```bash
+sage -pip install --break-system-packages \
+    "torch==2.6.0" \
+    "transformers>=4.49.0" \
+    "omegaconf>=2.3.0" \
+    "wandb>=0.15.11" \
+    "accelerate>=0.29.0" \
+    "joblib>=1.5.0"
+```
+
+**For GPU support with PyTorch:**
+If you need GPU support, replace the `torch` installation line with the one that specifies the CUDA version compatible with your system. For example, for CUDA 12.4:
+
+```bash
+sage -pip install --break-system-packages \
+    --extra-index-url https://download.pytorch.org/whl/cu124 \
+    "torch==2.6.0"
+```
+
+### 3. Install `transformer_algebra` (Editable)
+
+Finally, install the `transformer_algebra` package in editable mode. Navigate to the root of the `calt` project directory (where this `README.md` and the  `pyproject.toml` for `transformer_algebra` are located) and run:
+
+```bash
+sage -pip install -e .
+```
+This command assumes that the necessary setup files for `transformer_algebra` are in the current directory (`.`). If `transformer_algebra` is a subdirectory (e.g., `/app` as in the Dockerfile context), you would run `sage -pip install -e /path/to/transformer_algebra_directory`.
+
+## Generating Datasets
+
+To generate the default dataset, run the following command from the project root:
+
+```bash
+sage scripts/generate.py
+```
+
+To generate datasets using a different `ProblemGenerator` class, you will need to modify `scripts/generate.py` by uncommenting the desired `ProblemGenerator` class and commenting out others.
+
+## Running Training
+
+To start training with the default configuration, execute the following command from the project root:
+
+```bash
+sage scripts/train.py
+```
+
+### Weights & Biases (wandb) Setup
+
+If you are using Weights & Biases (wandb) for the first time to log training progress, you will need to create an account on their website and set up your API key. When you run the training script for the first time, you will be prompted to enter your API key.
+
+https://wandb.ai/site/
+
+### Configuration
+
+Training parameters can be modified by editing the configuration file located at `config/train_example.yaml`.
+
+## Demonstrations
+
+Simple demonstrations for data generation and training are available as Jupyter Notebook files. You can find them in the `notebook` directory (please create this directory and add your notebooks if it doesn't exist yet).
+
+To run these notebooks, you need to start SageMath's Jupyter server using the command `sage -n` and then select the SageMath kernel in the notebook interface.

calt_x-0.1.0/README.md

@@ -0,0 +1,151 @@
+# CALT: Computer ALgebra with Transformer
+This project is currently in its initial development phase. The file structure and content are subject to significant changes. Please ensure you are referring to the latest version when using it.
+
+# Environment Setup using Docker
+
+This guide explains how to set up the development environment using Docker.
+
+## Prerequisites
+
+- Docker installed on your system.
+- NVIDIA GPU drivers installed if you plan to use GPU acceleration (`--gpus all` option).
+
+## Build the Docker Image
+
+To build the Docker image, navigate to the `calt` directory and run the following command:
+
+```bash
+make build
+```
+
+Alternatively, you can use the direct Docker command:
+
+```bash
+docker build -t ta-sage .
+```
+
+## Run the Docker Container
+
+To run the Docker container in detached mode with GPU support, execute:
+
+```bash
+make run
+```
+
+The direct Docker command is:
+
+```bash
+docker run --gpus all -d --name ta-sage-container -v "$(pwd)":/app ta-sage tail -f /dev/null
+```
+*Note: When running this command directly in your terminal, `$(pwd)` will resolve to your current working directory. The Makefile uses `$(CURDIR)` which serves the same purpose within the Makefile context.*
+
+## Access the Container
+
+Once the container is running, you can access it using:
+
+```bash
+docker exec -it ta-sage-container bash
+```
+
+## Stop and Remove the Container
+
+To stop and remove the container, you can use:
+
+```bash
+make stop
+```
+
+Or manually:
+
+```bash
+docker stop ta-sage-container
+docker rm ta-sage-container
+```
+
+## Local Setup (without Docker)
+
+This section describes how to set up the environment locally without using Docker. This assumes you have SageMath installed on your system.
+
+### 1. Install SageMath
+
+You can install SageMath using `apt` on Debian/Ubuntu-based systems. It's not necessary to have the absolute latest version.
+
+Install SageMath:
+
+```bash
+sudo apt-get install -y sagemath
+```
+
+### 2. Install Dependencies
+
+Once SageMath is installed, you can install the required Python packages using `sage -pip`.
+
+First, upgrade pip:
+
+```bash
+sage -pip install --upgrade pip
+```
+
+Next, install the Python dependencies:
+
+```bash
+sage -pip install --break-system-packages \
+    "torch==2.6.0" \
+    "transformers>=4.49.0" \
+    "omegaconf>=2.3.0" \
+    "wandb>=0.15.11" \
+    "accelerate>=0.29.0" \
+    "joblib>=1.5.0"
+```
+
+**For GPU support with PyTorch:**
+If you need GPU support, replace the `torch` installation line with the one that specifies the CUDA version compatible with your system. For example, for CUDA 12.4:
+
+```bash
+sage -pip install --break-system-packages \
+    --extra-index-url https://download.pytorch.org/whl/cu124 \
+    "torch==2.6.0"
+```
+
+### 3. Install `transformer_algebra` (Editable)
+
+Finally, install the `transformer_algebra` package in editable mode. Navigate to the root of the `calt` project directory (where this `README.md` and the  `pyproject.toml` for `transformer_algebra` are located) and run:
+
+```bash
+sage -pip install -e .
+```
+This command assumes that the necessary setup files for `transformer_algebra` are in the current directory (`.`). If `transformer_algebra` is a subdirectory (e.g., `/app` as in the Dockerfile context), you would run `sage -pip install -e /path/to/transformer_algebra_directory`.
+
+## Generating Datasets
+
+To generate the default dataset, run the following command from the project root:
+
+```bash
+sage scripts/generate.py
+```
+
+To generate datasets using a different `ProblemGenerator` class, you will need to modify `scripts/generate.py` by uncommenting the desired `ProblemGenerator` class and commenting out others.
+
+## Running Training
+
+To start training with the default configuration, execute the following command from the project root:
+
+```bash
+sage scripts/train.py
+```
+
+### Weights & Biases (wandb) Setup
+
+If you are using Weights & Biases (wandb) for the first time to log training progress, you will need to create an account on their website and set up your API key. When you run the training script for the first time, you will be prompted to enter your API key.
+
+https://wandb.ai/site/
+
+### Configuration
+
+Training parameters can be modified by editing the configuration file located at `config/train_example.yaml`.
+
+## Demonstrations
+
+Simple demonstrations for data generation and training are available as Jupyter Notebook files. You can find them in the `notebook` directory (please create this directory and add your notebooks if it doesn't exist yet).
+
+To run these notebooks, you need to start SageMath's Jupyter server using the command `sage -n` and then select the SageMath kernel in the notebook interface.

calt_x-0.1.0/pyproject.toml

@@ -0,0 +1,47 @@
+[build-system]
+requires = ["setuptools>=65", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "calt-x"
+version = "0.1.0"
+description = "A library for computational algebra using Transformers"
+readme = "README.md"
+requires-python = ">=3.12"
+authors = [
+  {name = "Yuta Sato", email = "sato.yuta@gmail.com"}
+]
+
+dependencies = [
+    "transformers>=4.49.0",
+    "omegaconf>=2.3.0",
+    "torch>=2.6.0",
+    "wandb>=0.15.11",
+    "accelerate>=0.29.0",
+    "joblib>=1.5.0",
+    "sympy>=1.12",
+]
+
+[dependency-groups]
+dev = [
+    "mypy>=1.15.0",
+    "ruff>=0.9.9",
+    "pydantic>=2.10.6",
+    "pytest>=8.3.5",
+]
+
+[tool.ruff]
+lint.per-file-ignores = {"src/calt/__init__.py" = ["F401"]}
+exclude = [
+    "sage/**",
+]
+
+[tool.setuptools]
+package-dir = {"" = "src"}
+
+[tool.setuptools.packages.find]
+where = ["src"] 
+
+[project.urls]
+Source        = "https://github.com/HiroshiKERA/calt"
+Issues        = "https://github.com/HiroshiKERA/calt/issues"

calt_x-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

calt_x-0.1.0/src/calt/__init__.py

@@ -0,0 +1,9 @@
+from .trainer.trainer import PolynomialTrainer
+from .trainer.utils import count_cuda_devices
+from .data_loader.data_loader import data_loader
+from .data_loader.utils.data_collator import StandardDataset, StandardDataCollator
+from .data_loader.utils.tokenizer import set_tokenizer
+from .data_loader.utils.preprocessor import SymbolicToInternalProcessor, IntegerToInternalProcessor
+# from .generate.dataset_generator import DatasetGenerator
+# from .generate.utils.polynomial_sampler import PolynomialSampler
+from .generate.utils.dataset_writer import DatasetWriter

calt_x-0.1.0/src/calt/data_loader/data_loader.py

@@ -0,0 +1,86 @@
+"""Data loading utilities for the Transformer Algebra project.
+
+This module defines helper functions that build the training and evaluation
+`Dataset`, `Tokenizer`, and `DataCollator` objects used throughout the
+library.  In particular, the `data_loader` factory translates symbolic
+polynomial expressions into the internal token representation expected by the
+Transformer models.
+"""
+
+from .utils.data_collator import StandardDataset, StandardDataCollator
+from .utils.preprocessor import SymbolicToInternalProcessor, IntegerToInternalProcessor
+from .utils.tokenizer import set_tokenizer
+from transformers import PreTrainedTokenizerFast as StandardTokenizer
+from typing import Tuple
+
+
+def data_loader(
+    train_dataset_path: str,
+    test_dataset_path: str,
+    field: str,
+    num_variables: int,
+    max_degree: int,
+    max_coeff: int,
+    max_length: int = 512,
+    processor_name: str = "polynomial",
+) -> Tuple[StandardDataset, StandardTokenizer, StandardDataCollator]:
+    """Create dataset, tokenizer and data-collator objects.
+
+    Parameters
+    ----------
+    train_dataset_path : str
+        Path to the file that stores the *training* samples.
+    test_dataset_path : str
+        Path to the file that stores the *evaluation* samples.
+    field : str
+        Finite-field identifier (e.g. ``"Q"`` for the rationals or ``"Zp"``
+        for a prime field) used to generate the vocabulary.
+    num_variables : int
+        Maximum number of symbolic variables (\(x_1, \dots, x_n\)) that can
+        appear in a polynomial.
+    max_degree : int
+        Maximum total degree allowed for any monomial term.
+    max_coeff : int
+        Maximum absolute value of the coefficients appearing in the data.
+    max_length : int, default ``512``
+        Hard upper bound on the token sequence length.  Longer sequences will
+        be *right-truncated*.
+    processor_name : str, default ``"polynomial"``
+        Name of the processor to use for converting symbolic expressions into
+        internal token IDs.  The default processor is ``"polynomial"``, which
+        handles polynomial expressions.  The alternative processor is
+        ``"integer"``, which handles integer expressions.
+
+    Returns
+    -------
+    Tuple[StandardDataset, StandardTokenizer, StandardDataCollator]
+        1. ``dataset``  – a ``dict`` with ``"train"`` and ``"test"`` splits
+           containing :class:`StandardDataset` instances.
+        2. ``tokenizer`` – a :class:`PreTrainedTokenizerFast` capable of
+           encoding symbolic expressions into token IDs and vice versa.
+        3. ``data_collator`` – a callable that assembles batches and applies
+           dynamic padding so they can be fed to a HuggingFace ``Trainer``.
+    """
+    if processor_name == "polynomial":
+        preprocessor = SymbolicToInternalProcessor(
+            num_variables=num_variables,
+            max_degree=max_degree,
+            max_coeff=max_coeff,
+        )
+    elif processor_name == "numeric":
+        preprocessor = IntegerToInternalProcessor(max_coeff=max_coeff)
+    else:
+        raise ValueError(f"Unknown processor: {processor_name}")
+
+    train_dataset = StandardDataset(train_dataset_path, preprocessor)
+    test_dataset = StandardDataset(test_dataset_path, preprocessor)
+    tokenizer = set_tokenizer(
+        num_vars=num_variables,
+        field=field,
+        max_degree=max_degree,
+        max_coeff=max_coeff,
+        max_length=max_length,
+    )
+    data_collator = StandardDataCollator(tokenizer)
+    dataset = {"train": train_dataset, "test": test_dataset}
+    return dataset, tokenizer, data_collator

calt_x-0.1.0/src/calt/data_loader/utils/data_collator.py

@@ -0,0 +1,129 @@
+from transformers import PreTrainedTokenizerFast as Tokenizer
+from .preprocessor import AbstractPreprocessor
+from typing import Dict
+from torch.utils.data import Dataset
+import torch
+
+
+class StandardDataset(Dataset):
+    def __init__(self, data_path: str, preprocessor: AbstractPreprocessor) -> None:
+        self.data_path = data_path
+        self.input_texts = []
+        self.targets_texts = []
+        self.preprocessor = preprocessor
+
+        # Load and parse the data file
+        with open(self.data_path, "r", encoding="utf-8") as f:
+            for raw_line in f:
+                line = raw_line.strip()
+                if not line:
+                    continue  # Skip empty lines
+
+                # Split input and target expressions using "#" delimiter
+                if "#" not in line:
+                    continue  # Skip lines with unexpected format (no delimiter)
+
+                input_part, target_part = line.split("#", 1)
+                self.input_texts.append(input_part.strip())
+                self.targets_texts.append(target_part.strip())
+
+    def __getitem__(self, idx: int) -> Dict[str, str]:
+        """Get dataset item and convert to internal representation.
+
+        Parameters
+        ----------
+        idx : int
+            Index of the item to retrieve
+
+        Returns
+        -------
+        tuple
+            A pair (src, tgt) of preprocessed source and target
+        """
+        src = self.preprocessor(self.input_texts[idx])
+        tgt = self.preprocessor(self.targets_texts[idx])
+        return {"input": src, "target": tgt}
+
+    def __len__(self) -> int:
+        return len(self.input_texts)
+
+
+class StandardDataCollator:
+    def __init__(self, tokenizer: Tokenizer = None) -> None:
+        self.tokenizer = tokenizer
+
+    def _pad_sequences(self, sequences, padding_value=0):
+        """Pads a list of sequences and converts them to a tensor."""
+        # Calculate the maximum length of the sequences.
+        max_length = max(len(seq) for seq in sequences)
+
+        # Apply padding.
+        padded_sequences = []
+        for seq in sequences:
+            padding_length = max_length - len(seq)
+            # Pad the sequence with the specified padding value.
+            padded_seq = seq + [padding_value] * padding_length
+            padded_sequences.append(padded_seq)
+
+        # '+2' for bos/eos tokens.
+        # Initialize a tensor of zeros with the appropriate shape.
+        padded = torch.zeros(len(sequences), max_length + 2, dtype=torch.long)
+        # Fill the tensor with the padded sequences, leaving space for BOS/EOS tokens.
+        padded[:, 1 : max_length + 1] = torch.tensor(padded_sequences)
+
+        return padded
+
+    def __call__(self, batch):
+        """
+        Collates a batch of data samples.
+        If a tokenizer is provided, it tokenizes 'input' and 'target' attributes.
+        Other attributes starting with 'target_' are prefixed with 'decoder_' and padded.
+        """
+        batch_dict = {}
+
+        # Get the attributes from the first item in the batch.
+        attributes = batch[0].keys()
+
+        if self.tokenizer is None:
+            # If no tokenizer is provided, return the batch as is.
+            for attribute in attributes:
+                attribute_batch = [item[attribute] for item in batch]
+                batch_dict[attribute] = attribute_batch
+
+            return batch_dict
+
+        for attribute in attributes:
+            attribute_batch = [item[attribute] for item in batch]
+
+            if attribute == "input":
+                # Tokenize the input sequences.
+                inputs = self.tokenizer(attribute_batch, padding="longest", return_tensors="pt")
+                batch_dict["input_ids"] = inputs["input_ids"]
+                batch_dict["attention_mask"] = inputs["attention_mask"]
+
+            elif attribute == "target":
+                # Tokenize the target sequences.
+                targets = self.tokenizer(attribute_batch, padding="longest", return_tensors="pt")
+                # Prepare decoder input ids (remove the last token, usually EOS).
+                batch_dict["decoder_input_ids"] = targets["input_ids"][:, :-1].contiguous()
+                # Prepare decoder attention mask accordingly.
+                batch_dict["decoder_attention_mask"] = targets["attention_mask"][:, :-1].contiguous()
+
+                # Prepare labels for the loss calculation (shift by one, usually remove BOS).
+                labels = targets["input_ids"][:, 1:].contiguous()
+                label_attention_mask = targets["attention_mask"][:, 1:].contiguous().bool()
+                # Set padding tokens in labels to -100 to be ignored by the loss function.
+                labels[~label_attention_mask] = -100
+                batch_dict["labels"] = labels
+
+            else:
+                # For other attributes, if they start with 'target_',
+                # prefix them with 'decoder_' (e.g., 'target_aux' becomes 'decoder_aux').
+                if attribute.startswith("target_"):
+                    attribute_key = "decoder_" + attribute[7:]  #  Corrected key for batch_dict
+                else:
+                    attribute_key = attribute  # Use original attribute name if no prefix
+                # Pad the sequences for these attributes.
+                batch_dict[attribute_key] = self._pad_sequences(attribute_batch, padding_value=0)
+
+        return batch_dict