libthx 0.3.0__tar.gz → 0.4.0__tar.gz

libthx-0.4.0/PKG-INFO

@@ -0,0 +1,180 @@
+Metadata-Version: 2.4
+Name: libthx
+Version: 0.4.0
+Summary: Architecture experimentation and training infrastructure.
+Requires-Python: <3.14,>=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: click>=8.2.1
+Requires-Dist: cloudpickle>=3.1.1
+Requires-Dist: flax>=0.12.2
+Requires-Dist: flatbuffers>=25.12.0
+Requires-Dist: jsonlines>=4.0.0
+Requires-Dist: loguru>=0.7.3
+Requires-Dist: numpy>=2.4.1
+Requires-Dist: omegaconf>=2.3.0
+Requires-Dist: orbax>=0.1.9
+Requires-Dist: pydantic>=2.12.5
+Requires-Dist: python-dotenv>=1.2.1
+Requires-Dist: rich>=13.5.2
+Requires-Dist: seaborn>=0.13.2
+Requires-Dist: tiktoken>=0.12.0
+Requires-Dist: torchax>=0.0.11
+Requires-Dist: wandb>=0.24.1
+Requires-Dist: datasets>=4.5.0
+Requires-Dist: huggingface-hub>=1.4.1
+Requires-Dist: flatbuffers>=25.12.19
+Requires-Dist: pyarrow>=23.0.1
+Requires-Dist: chdb>=4.3.0
+Requires-Dist: deltalake>=1.6.3
+Requires-Dist: typer>=0.24.1
+Requires-Dist: nicegui<4,>=3.16.0
+Requires-Dist: pygwalker>=0.5.0.1
+Requires-Dist: pandas>=2
+Requires-Dist: comet-ml>=3.58.6
+Requires-Dist: bottle>=0.13.4
+Requires-Dist: cheroot>=11.1.2
+Provides-Extra: fever
+Requires-Dist: wikipedia>=1.4.0; extra == "fever"
+Provides-Extra: huggingface
+Requires-Dist: tokenizers>=0.22.2; extra == "huggingface"
+Requires-Dist: transformers>=5.1.0; extra == "huggingface"
+Requires-Dist: accelerate>=1.13.0; extra == "huggingface"
+Provides-Extra: cuda13
+Requires-Dist: jax[cuda13]>=0.4.23; extra == "cuda13"
+Requires-Dist: torch>=2.9.1; extra == "cuda13"
+Requires-Dist: torchax>=0.0.11; extra == "cuda13"
+Provides-Extra: cuda12
+Requires-Dist: jax[cuda12]>=0.4.23; extra == "cuda12"
+Requires-Dist: torch>=2.9.1; extra == "cuda12"
+Requires-Dist: torchax>=0.0.11; extra == "cuda12"
+Provides-Extra: tpu
+Requires-Dist: jax[tpu]>=0.4.23; extra == "tpu"
+Requires-Dist: torch>=2.9.1; extra == "tpu"
+Requires-Dist: torchax>=0.0.11; extra == "tpu"
+Provides-Extra: cpu
+Requires-Dist: jax>=0.4.23; extra == "cpu"
+Requires-Dist: torch>=2.9.1; extra == "cpu"
+Requires-Dist: torchax>=0.0.11; extra == "cpu"
+Provides-Extra: dev
+Requires-Dist: coverage>=7.9.1; extra == "dev"
+Requires-Dist: coveralls>=4.0.1; extra == "dev"
+Requires-Dist: pytest>=8.4.1; extra == "dev"
+Requires-Dist: playwright>=1.55.0; extra == "dev"
+Requires-Dist: pytest-cov>=6.2.1; extra == "dev"
+Requires-Dist: ruff>=0.12.1; extra == "dev"
+Requires-Dist: pre-commit>=4.2.0; extra == "dev"
+Requires-Dist: mypy>=1.16.1; extra == "dev"
+Requires-Dist: ipdb>=0.13.13; extra == "dev"
+Requires-Dist: xprof~=2.21.5; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: zensical<0.1,>=0.0.60; extra == "docs"
+Requires-Dist: mkdocstrings-python>=2.0.3; extra == "docs"
+Requires-Dist: pymdown-extensions>=10.15; extra == "docs"
+Provides-Extra: all
+Requires-Dist: tokenizers>=0.22.2; extra == "all"
+Requires-Dist: transformers>=5.1.0; extra == "all"
+Requires-Dist: accelerate>=1.13.0; extra == "all"
+Requires-Dist: coverage>=7.9.1; extra == "all"
+Requires-Dist: coveralls>=4.0.1; extra == "all"
+Requires-Dist: pytest>=8.4.1; extra == "all"
+Requires-Dist: playwright>=1.55.0; extra == "all"
+Requires-Dist: pytest-cov>=6.2.1; extra == "all"
+Requires-Dist: ruff>=0.12.1; extra == "all"
+Requires-Dist: pre-commit>=4.2.0; extra == "all"
+Requires-Dist: mypy>=1.16.1; extra == "all"
+Requires-Dist: ipdb>=0.13.13; extra == "all"
+Requires-Dist: xprof~=2.21.5; extra == "all"
+Requires-Dist: zensical<0.1,>=0.0.60; extra == "all"
+Requires-Dist: mkdocstrings-python>=2.0.3; extra == "all"
+Requires-Dist: pymdown-extensions>=10.15; extra == "all"
+Requires-Dist: wikipedia>=1.4.0; extra == "all"
+Dynamic: license-file
+
+# hi, welcome to theseus
+
+this is theseus. it makes GPUs and TPUs warm and fuzzy inside by harnessing the power neural architecture research™® on Human Languages℗. its fine. not too fast, not too good, but it does the thing and you can use it. to be clear, it gets about mid-20s MFU on a mid day and high-30s MFU on a good day, and it doesn't do anything fancy. i'm here to tell you how to use it, and i'd love to [know about it](mailto:hi@jemoka.com) if you ended up doing that.
+
+## Installation
+
+theseus is a template repo. You use it by having it. So, please, clone the repo:
+
+```bash
+git clone https://github.com/Jemoka/thx.git theseus
+cd theseus
+```
+
+We use the [uv package manager](https://docs.astral.sh/uv/getting-started/), so make sure you have that. Now, depending on from whomst your computers must be warm, please choose your adventure:
+
+- cuda13: `uv sync --group all --group cuda13`
+- cuda12: `uv sync --group all --group cuda12`
+- TPUs: `uv sync --group all --group tpu`
+- CPU: `uv sync --group all --group cpu`
+
+<!-- ## Running -->
+
+<!-- You have two options: -->
+
+<!-- 1. **Run theseus here**: [running experiments locally](docs/How-to/running-local.md) -->
+<!-- 2. **Run theseus in chonky remote**:  [run on a remote cluster](docs/How-to/running-remote.md); just the `cpu` variant of theseus for your laptop is fine for that even if remote needs CUDA -->
+
+## Start Quickly
+We are going to invoke stuff that theseus already has built in, like tokenizing fineweb and training a GPT model. You refer to these by what's called *job keys*, like `data/tokenize/fineweb` and `gpt/train/pretrain`.
+
+First, make a folder where things will go
+
+```bash
+mkdir /folder/where/things/go
+```
+
+Then, tokenize some data:
+
+
+```python
+from theseus.quick import quick
+
+with quick("/folder/where/things/go") as q:
+    q.build("data/tokenize/fineweb", "my-tokenize-job")
+    job = q.create()
+    job()
+```
+
+Finally, train a model:
+
+```python
+with quick("/folder/where/things/go") as q:
+    q.build("gpt/train/pretrain", "my-training-job")
+
+    # add salt (configurate) to taste
+    # hint: q.config is an omegaconf, so you can print it
+    q.config.training.per_device_batch_size = 8 # required! idk how much hbm u have
+
+    # and then make it go brr
+    job = q.create()
+    job()
+```
+
+Your dataset job needs to share the same root folder as your training job. This is how theseus knows to load the dataset you've tokenized instead of blowing up and complaining you didn't have the dataset tokenized.
+
+## What's next?
+- Read the [docs](https://theseus.jemoka.com/quickstart).
+- Read the 80% funnier [tutorial](https://theseus.jemoka.com/Tutorials/).
+- Near bedtime? Read the [design](https://theseus.jemoka.com/Design).
+
+## What's inside?
+Code. It occasionally runs, and when it does you get features!
+
+1. **models that go brrr**: mid-20s MFU on a mid day and high-30s MFU on a good day
+2. **determinism** up to float: every step and every batch will be replayed exactly upon restoration
+3. **time travel debugging**: because of ^ you can inspect, replay or recompute activations of *any layer, at any checkpoint, on any device, at any time* (or your money back)
+4. **composition**: inheritance based module-level composition 
+5. **sharding**: one-click FSDP, ZERO level 1, and Tensor Parallelism
+6. **configurationing**: OmegaConf compatible configuration + discovery API with strict type resolution
+7. **dispatch**: remote dispatch infrastructure and resource solving with support for SSH, SLURM, K8s Volcano, and GCP TPU hosts
+8. **dataset and evaluations**: a buncha datasets already built-in, and adding one is just telling us how to manipulate strings and we'll make it embarrassingly parallel
+
+...and probably more things I forgot. I'll try hard to at least have me or Codex write How-To guides covering each of these aspects, and link them here as they show up. 
+
+<p align="center">
+  <img src="docs/assets/colophon.png" alt="" width="32">
+</p>

libthx-0.4.0/README.md

@@ -0,0 +1,87 @@
+# hi, welcome to theseus
+
+this is theseus. it makes GPUs and TPUs warm and fuzzy inside by harnessing the power neural architecture research™® on Human Languages℗. its fine. not too fast, not too good, but it does the thing and you can use it. to be clear, it gets about mid-20s MFU on a mid day and high-30s MFU on a good day, and it doesn't do anything fancy. i'm here to tell you how to use it, and i'd love to [know about it](mailto:hi@jemoka.com) if you ended up doing that.
+
+## Installation
+
+theseus is a template repo. You use it by having it. So, please, clone the repo:
+
+```bash
+git clone https://github.com/Jemoka/thx.git theseus
+cd theseus
+```
+
+We use the [uv package manager](https://docs.astral.sh/uv/getting-started/), so make sure you have that. Now, depending on from whomst your computers must be warm, please choose your adventure:
+
+- cuda13: `uv sync --group all --group cuda13`
+- cuda12: `uv sync --group all --group cuda12`
+- TPUs: `uv sync --group all --group tpu`
+- CPU: `uv sync --group all --group cpu`
+
+<!-- ## Running -->
+
+<!-- You have two options: -->
+
+<!-- 1. **Run theseus here**: [running experiments locally](docs/How-to/running-local.md) -->
+<!-- 2. **Run theseus in chonky remote**:  [run on a remote cluster](docs/How-to/running-remote.md); just the `cpu` variant of theseus for your laptop is fine for that even if remote needs CUDA -->
+
+## Start Quickly
+We are going to invoke stuff that theseus already has built in, like tokenizing fineweb and training a GPT model. You refer to these by what's called *job keys*, like `data/tokenize/fineweb` and `gpt/train/pretrain`.
+
+First, make a folder where things will go
+
+```bash
+mkdir /folder/where/things/go
+```
+
+Then, tokenize some data:
+
+
+```python
+from theseus.quick import quick
+
+with quick("/folder/where/things/go") as q:
+    q.build("data/tokenize/fineweb", "my-tokenize-job")
+    job = q.create()
+    job()
+```
+
+Finally, train a model:
+
+```python
+with quick("/folder/where/things/go") as q:
+    q.build("gpt/train/pretrain", "my-training-job")
+
+    # add salt (configurate) to taste
+    # hint: q.config is an omegaconf, so you can print it
+    q.config.training.per_device_batch_size = 8 # required! idk how much hbm u have
+
+    # and then make it go brr
+    job = q.create()
+    job()
+```
+
+Your dataset job needs to share the same root folder as your training job. This is how theseus knows to load the dataset you've tokenized instead of blowing up and complaining you didn't have the dataset tokenized.
+
+## What's next?
+- Read the [docs](https://theseus.jemoka.com/quickstart).
+- Read the 80% funnier [tutorial](https://theseus.jemoka.com/Tutorials/).
+- Near bedtime? Read the [design](https://theseus.jemoka.com/Design).
+
+## What's inside?
+Code. It occasionally runs, and when it does you get features!
+
+1. **models that go brrr**: mid-20s MFU on a mid day and high-30s MFU on a good day
+2. **determinism** up to float: every step and every batch will be replayed exactly upon restoration
+3. **time travel debugging**: because of ^ you can inspect, replay or recompute activations of *any layer, at any checkpoint, on any device, at any time* (or your money back)
+4. **composition**: inheritance based module-level composition 
+5. **sharding**: one-click FSDP, ZERO level 1, and Tensor Parallelism
+6. **configurationing**: OmegaConf compatible configuration + discovery API with strict type resolution
+7. **dispatch**: remote dispatch infrastructure and resource solving with support for SSH, SLURM, K8s Volcano, and GCP TPU hosts
+8. **dataset and evaluations**: a buncha datasets already built-in, and adding one is just telling us how to manipulate strings and we'll make it embarrassingly parallel
+
+...and probably more things I forgot. I'll try hard to at least have me or Codex write How-To guides covering each of these aspects, and link them here as they show up. 
+
+<p align="center">
+  <img src="docs/assets/colophon.png" alt="" width="32">
+</p>

libthx-0.4.0/libthx.egg-info/PKG-INFO

@@ -0,0 +1,180 @@
+Metadata-Version: 2.4
+Name: libthx
+Version: 0.4.0
+Summary: Architecture experimentation and training infrastructure.
+Requires-Python: <3.14,>=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: click>=8.2.1
+Requires-Dist: cloudpickle>=3.1.1
+Requires-Dist: flax>=0.12.2
+Requires-Dist: flatbuffers>=25.12.0
+Requires-Dist: jsonlines>=4.0.0
+Requires-Dist: loguru>=0.7.3
+Requires-Dist: numpy>=2.4.1
+Requires-Dist: omegaconf>=2.3.0
+Requires-Dist: orbax>=0.1.9
+Requires-Dist: pydantic>=2.12.5
+Requires-Dist: python-dotenv>=1.2.1
+Requires-Dist: rich>=13.5.2
+Requires-Dist: seaborn>=0.13.2
+Requires-Dist: tiktoken>=0.12.0
+Requires-Dist: torchax>=0.0.11
+Requires-Dist: wandb>=0.24.1
+Requires-Dist: datasets>=4.5.0
+Requires-Dist: huggingface-hub>=1.4.1
+Requires-Dist: flatbuffers>=25.12.19
+Requires-Dist: pyarrow>=23.0.1
+Requires-Dist: chdb>=4.3.0
+Requires-Dist: deltalake>=1.6.3
+Requires-Dist: typer>=0.24.1
+Requires-Dist: nicegui<4,>=3.16.0
+Requires-Dist: pygwalker>=0.5.0.1
+Requires-Dist: pandas>=2
+Requires-Dist: comet-ml>=3.58.6
+Requires-Dist: bottle>=0.13.4
+Requires-Dist: cheroot>=11.1.2
+Provides-Extra: fever
+Requires-Dist: wikipedia>=1.4.0; extra == "fever"
+Provides-Extra: huggingface
+Requires-Dist: tokenizers>=0.22.2; extra == "huggingface"
+Requires-Dist: transformers>=5.1.0; extra == "huggingface"
+Requires-Dist: accelerate>=1.13.0; extra == "huggingface"
+Provides-Extra: cuda13
+Requires-Dist: jax[cuda13]>=0.4.23; extra == "cuda13"
+Requires-Dist: torch>=2.9.1; extra == "cuda13"
+Requires-Dist: torchax>=0.0.11; extra == "cuda13"
+Provides-Extra: cuda12
+Requires-Dist: jax[cuda12]>=0.4.23; extra == "cuda12"
+Requires-Dist: torch>=2.9.1; extra == "cuda12"
+Requires-Dist: torchax>=0.0.11; extra == "cuda12"
+Provides-Extra: tpu
+Requires-Dist: jax[tpu]>=0.4.23; extra == "tpu"
+Requires-Dist: torch>=2.9.1; extra == "tpu"
+Requires-Dist: torchax>=0.0.11; extra == "tpu"
+Provides-Extra: cpu
+Requires-Dist: jax>=0.4.23; extra == "cpu"
+Requires-Dist: torch>=2.9.1; extra == "cpu"
+Requires-Dist: torchax>=0.0.11; extra == "cpu"
+Provides-Extra: dev
+Requires-Dist: coverage>=7.9.1; extra == "dev"
+Requires-Dist: coveralls>=4.0.1; extra == "dev"
+Requires-Dist: pytest>=8.4.1; extra == "dev"
+Requires-Dist: playwright>=1.55.0; extra == "dev"
+Requires-Dist: pytest-cov>=6.2.1; extra == "dev"
+Requires-Dist: ruff>=0.12.1; extra == "dev"
+Requires-Dist: pre-commit>=4.2.0; extra == "dev"
+Requires-Dist: mypy>=1.16.1; extra == "dev"
+Requires-Dist: ipdb>=0.13.13; extra == "dev"
+Requires-Dist: xprof~=2.21.5; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: zensical<0.1,>=0.0.60; extra == "docs"
+Requires-Dist: mkdocstrings-python>=2.0.3; extra == "docs"
+Requires-Dist: pymdown-extensions>=10.15; extra == "docs"
+Provides-Extra: all
+Requires-Dist: tokenizers>=0.22.2; extra == "all"
+Requires-Dist: transformers>=5.1.0; extra == "all"
+Requires-Dist: accelerate>=1.13.0; extra == "all"
+Requires-Dist: coverage>=7.9.1; extra == "all"
+Requires-Dist: coveralls>=4.0.1; extra == "all"
+Requires-Dist: pytest>=8.4.1; extra == "all"
+Requires-Dist: playwright>=1.55.0; extra == "all"
+Requires-Dist: pytest-cov>=6.2.1; extra == "all"
+Requires-Dist: ruff>=0.12.1; extra == "all"
+Requires-Dist: pre-commit>=4.2.0; extra == "all"
+Requires-Dist: mypy>=1.16.1; extra == "all"
+Requires-Dist: ipdb>=0.13.13; extra == "all"
+Requires-Dist: xprof~=2.21.5; extra == "all"
+Requires-Dist: zensical<0.1,>=0.0.60; extra == "all"
+Requires-Dist: mkdocstrings-python>=2.0.3; extra == "all"
+Requires-Dist: pymdown-extensions>=10.15; extra == "all"
+Requires-Dist: wikipedia>=1.4.0; extra == "all"
+Dynamic: license-file
+
+# hi, welcome to theseus
+
+this is theseus. it makes GPUs and TPUs warm and fuzzy inside by harnessing the power neural architecture research™® on Human Languages℗. its fine. not too fast, not too good, but it does the thing and you can use it. to be clear, it gets about mid-20s MFU on a mid day and high-30s MFU on a good day, and it doesn't do anything fancy. i'm here to tell you how to use it, and i'd love to [know about it](mailto:hi@jemoka.com) if you ended up doing that.
+
+## Installation
+
+theseus is a template repo. You use it by having it. So, please, clone the repo:
+
+```bash
+git clone https://github.com/Jemoka/thx.git theseus
+cd theseus
+```
+
+We use the [uv package manager](https://docs.astral.sh/uv/getting-started/), so make sure you have that. Now, depending on from whomst your computers must be warm, please choose your adventure:
+
+- cuda13: `uv sync --group all --group cuda13`
+- cuda12: `uv sync --group all --group cuda12`
+- TPUs: `uv sync --group all --group tpu`
+- CPU: `uv sync --group all --group cpu`
+
+<!-- ## Running -->
+
+<!-- You have two options: -->
+
+<!-- 1. **Run theseus here**: [running experiments locally](docs/How-to/running-local.md) -->
+<!-- 2. **Run theseus in chonky remote**:  [run on a remote cluster](docs/How-to/running-remote.md); just the `cpu` variant of theseus for your laptop is fine for that even if remote needs CUDA -->
+
+## Start Quickly
+We are going to invoke stuff that theseus already has built in, like tokenizing fineweb and training a GPT model. You refer to these by what's called *job keys*, like `data/tokenize/fineweb` and `gpt/train/pretrain`.
+
+First, make a folder where things will go
+
+```bash
+mkdir /folder/where/things/go
+```
+
+Then, tokenize some data:
+
+
+```python
+from theseus.quick import quick
+
+with quick("/folder/where/things/go") as q:
+    q.build("data/tokenize/fineweb", "my-tokenize-job")
+    job = q.create()
+    job()
+```
+
+Finally, train a model:
+
+```python
+with quick("/folder/where/things/go") as q:
+    q.build("gpt/train/pretrain", "my-training-job")
+
+    # add salt (configurate) to taste
+    # hint: q.config is an omegaconf, so you can print it
+    q.config.training.per_device_batch_size = 8 # required! idk how much hbm u have
+
+    # and then make it go brr
+    job = q.create()
+    job()
+```
+
+Your dataset job needs to share the same root folder as your training job. This is how theseus knows to load the dataset you've tokenized instead of blowing up and complaining you didn't have the dataset tokenized.
+
+## What's next?
+- Read the [docs](https://theseus.jemoka.com/quickstart).
+- Read the 80% funnier [tutorial](https://theseus.jemoka.com/Tutorials/).
+- Near bedtime? Read the [design](https://theseus.jemoka.com/Design).
+
+## What's inside?
+Code. It occasionally runs, and when it does you get features!
+
+1. **models that go brrr**: mid-20s MFU on a mid day and high-30s MFU on a good day
+2. **determinism** up to float: every step and every batch will be replayed exactly upon restoration
+3. **time travel debugging**: because of ^ you can inspect, replay or recompute activations of *any layer, at any checkpoint, on any device, at any time* (or your money back)
+4. **composition**: inheritance based module-level composition 
+5. **sharding**: one-click FSDP, ZERO level 1, and Tensor Parallelism
+6. **configurationing**: OmegaConf compatible configuration + discovery API with strict type resolution
+7. **dispatch**: remote dispatch infrastructure and resource solving with support for SSH, SLURM, K8s Volcano, and GCP TPU hosts
+8. **dataset and evaluations**: a buncha datasets already built-in, and adding one is just telling us how to manipulate strings and we'll make it embarrassingly parallel
+
+...and probably more things I forgot. I'll try hard to at least have me or Codex write How-To guides covering each of these aspects, and link them here as they show up. 
+
+<p align="center">
+  <img src="docs/assets/colophon.png" alt="" width="32">
+</p>