pixeltable 0.2.6__tar.gz → 0.2.8__tar.gz

pixeltable-0.2.8/PKG-INFO

@@ -0,0 +1,137 @@
+Metadata-Version: 2.1
+Name: pixeltable
+Version: 0.2.8
+Summary: Pixeltable: The Multimodal AI Data Plane
+Author: Marcel Kornacker
+Author-email: marcelk@gmail.com
+Requires-Python: >=3.9,<4.0
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Dist: av (>=10.0.0)
+Requires-Dist: beautifulsoup4 (>=4.0.0,<5.0.0)
+Requires-Dist: cloudpickle (>=2.2.1,<3.0.0)
+Requires-Dist: ftfy (>=6.2.0,<7.0.0)
+Requires-Dist: jinja2 (>=3.1.3,<4.0.0)
+Requires-Dist: jmespath (>=1.0.1,<2.0.0)
+Requires-Dist: mistune (>=3.0.2,<4.0.0)
+Requires-Dist: more-itertools (>=10.2,<11.0)
+Requires-Dist: numpy (>=1.25)
+Requires-Dist: opencv-python-headless (>=4.7.0.68,<5.0.0.0)
+Requires-Dist: pandas (>=2.0,<3.0)
+Requires-Dist: pgserver (==0.1.4)
+Requires-Dist: pgvector (>=0.2.1,<0.3.0)
+Requires-Dist: pillow (>=9.3.0)
+Requires-Dist: psutil (>=5.9.5,<6.0.0)
+Requires-Dist: psycopg2-binary (>=2.9.5,<3.0.0)
+Requires-Dist: pymupdf (>=1.24.1,<2.0.0)
+Requires-Dist: pyyaml (>=6.0.1,<7.0.0)
+Requires-Dist: requests (>=2.31.0,<3.0.0)
+Requires-Dist: setuptools (==69.1.1)
+Requires-Dist: sqlalchemy[mypy] (>=2.0.23,<3.0.0)
+Requires-Dist: tenacity (>=8.2,<9.0)
+Requires-Dist: tqdm (>=4.64)
+Description-Content-Type: text/markdown
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/pixeltable/pixeltable/master/docs/release/pixeltable-banner.png" alt="Pixeltable" width="45%" />
+
+# Unifying Data, Models, and Orchestration for AI Products
+
+[![License](https://img.shields.io/badge/License-Apache%202.0-darkblue.svg)](https://opensource.org/licenses/Apache-2.0)
+![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pixeltable?logo=python&logoColor=white)
+  [![Platform Support](https://img.shields.io/badge/platform-Linux%20%7C%20macOS%20%7C%20Windows-8A2BE2)]()
+[![pytest status](https://github.com/pixeltable/pixeltable/actions/workflows/pytest.yml/badge.svg)](https://github.com/pixeltable/pixeltable/actions)
+[![PyPI Package](https://img.shields.io/pypi/v/pixeltable?color=darkorange)](https://pypi.org/project/pixeltable/)
+
+[Installation](https://pixeltable.github.io/pixeltable/getting-started/) | [Documentation](https://pixeltable.readme.io/) | [API Reference](https://pixeltable.github.io/pixeltable/) | [Code Samples](https://pixeltable.readme.io/recipes) | [Examples](https://github.com/pixeltable/pixeltable/tree/master/docs/release/tutorials)
+</div>
+
+Pixeltable is a Python library that lets AI engineers and data scientists focus on exploration, modeling, and app development without dealing with the customary data plumbing.
+
+## What problems does Pixeltable solve?
+
+Today’s solutions for AI app development require extensive custom coding and infrastructure plumbing. Tracking lineage and versions between and across data transformations, models, and deployment is cumbersome. With Pixeltable you can store, transform, index, and iterate on your data within the same table interface, whether it's text, images, embeddings, or even video. Built-in lineage and versioning ensure transparency and reproducibility, while the development-to-production mirror streamlines deployment.
+
+## 💾 Installation
+
+```python
+%pip install pixeltable
+```
+
+To verify that it's working:
+
+```python
+import pixeltable as pxt
+pxt.init()
+```
+> [!NOTE]
+> Check out the [Pixeltable Basics](https://pixeltable.readme.io/docs/pixeltable-basics) tutorial for a tour of its most important features.
+
+## 💡 Get Started
+Learn how to create tables, populate them with data, and enhance them with built-in or user-defined transformations and AI operations.
+
+| Topic      |                                                                                Notebook                                                                                |                                                                                API                                                                                |
+|:--------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------:|
+| Get Started    | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/master/docs/tutorials/pixeltable-basics.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> |                 [![API](https://img.shields.io/badge/api-reference-blue.svg)](https://pixeltable.github.io/pixeltable/api/pixeltable/)                 |
+| User-Defined Functions (UDFs)    | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/master/docs/release/howto/udfs-in-pixeltable.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> |                 [![API](https://img.shields.io/badge/api-reference-blue.svg)](https://pixeltable.github.io/pixeltable/api/iterators/document-splitter/)                 |
+| Comparing Object Detection Models | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/master/docs/release/tutorials/object-detection-in-videos.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a>            | [![API](https://img.shields.io/badge/api-reference-blue.svg)](https://pixeltable.github.io/pixeltable/api-cheat-sheet/#frame-extraction-for-video-data) |
+| Experimenting with Chunking (RAG) | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/master/docs/release/tutorials/rag-operations.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>            | [![API](https://img.shields.io/badge/api-reference-blue.svg)](https://pixeltable.github.io/pixeltable/api/iterators/document-splitter/) |
+| Working with External Files    | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/master/docs/release/howto/working-with-external-files.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> |                 [![API](https://img.shields.io/badge/api-reference-blue.svg)](https://pixeltable.github.io/pixeltable/api-cheat-sheet/#inserting-data-into-a-table)                 |
+
+## ❓ FAQ
+
+### What does Pixeltable provide me with? Pixeltable provides:
+
+- Data storage and versioning
+- Combined Data and Model Lineage
+- Indexing (e.g. embedding vectors) and Data Retrieval
+- Orchestration of multimodal workloads
+- Incremental updates
+- Code is automatically production-ready
+
+### Why should you use Pixeltable?
+
+- **It gives you transparency and reproducibility**
+  - All generated data is automatically recorded and versioned
+  - You will never need to re-run a workload because you lost track of the input data
+- **It saves you money**
+  - All data changes are automatically incremental
+  - You never need to re-run pipelines from scratch because you’re adding data
+- **It integrates with any existing Python code or libraries**
+  - Bring your ever-changing code and workloads
+  - You choose the models, tools, and AI practices (e.g., your embedding model for a vector index); Pixeltable orchestrates the data
+ 
+### What is Pixeltable not providing?
+
+- Pixeltable is not a low-code, prescriptive AI solution. We empower you to use the best frameworks and techniques for your specific needs.
+- We do not aim to replace your existing AI toolkit, but rather enhance it by streamlining the underlying data infrastructure and orchestration.
+
+> [!TIP]
+> Check out the [Integrations](https://pixeltable.readme.io/docs/working-with-openai) section, and feel free to submit a request for additional ones.
+
+## 📙 Example of Use Cases
+
+- **Interact with video data at the frame level** without having to think about frame extraction, intermediate file storage, or storage space explosion.
+- **Augment your data incrementally and interactively with built-in functions and UDFs**, such as image transformations, model inference, and visualizations, without having to think about data pipelines, incremental updates, or capturing function output.
+- **Interact with all the data relevant to your AI application** (video, images, documents, audio, structured data, JSON) through a simple dataframe-style API directly in Python. This includes:
+  - similarity search on embeddings, supported by high-dimensional vector indexing;
+  - path expressions and transformations on JSON data;
+  - PIL and OpenCV image operations;
+  - assembling frames into videos.
+- **Perform keyword and image similarity search at the video frame level** without having to worry about frame storage.
+- **Access all Pixeltable-resident data directly as a PyTorch dataset** in your training scripts.
+- **Understand the compute and storage costs of your data at the granularity** of individual augmentations and get cost projections before adding new data and new augmentations.
+- **Rely on Pixeltable's automatic versioning and snapshot functionality** to protect against regressions and to ensure reproducibility.
+
+## 🐛 Contributions & Feedback
+
+Are you experiencing issues or bugs with Pixeltable? File an [Issue](https://github.com/pixeltable/pixeltable/issues).
+</br>Do you want to contribute? Feel free to open a [PR](https://github.com/pixeltable/pixeltable/pulls).
+
+## :classical_building: License
+
+This library is licensed under the Apache 2.0 License.
+

pixeltable-0.2.8/README.md

@@ -0,0 +1,99 @@
+<div align="center">
+<img src="https://raw.githubusercontent.com/pixeltable/pixeltable/master/docs/release/pixeltable-banner.png" alt="Pixeltable" width="45%" />
+
+# Unifying Data, Models, and Orchestration for AI Products
+
+[![License](https://img.shields.io/badge/License-Apache%202.0-darkblue.svg)](https://opensource.org/licenses/Apache-2.0)
+![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pixeltable?logo=python&logoColor=white)
+  [![Platform Support](https://img.shields.io/badge/platform-Linux%20%7C%20macOS%20%7C%20Windows-8A2BE2)]()
+[![pytest status](https://github.com/pixeltable/pixeltable/actions/workflows/pytest.yml/badge.svg)](https://github.com/pixeltable/pixeltable/actions)
+[![PyPI Package](https://img.shields.io/pypi/v/pixeltable?color=darkorange)](https://pypi.org/project/pixeltable/)
+
+[Installation](https://pixeltable.github.io/pixeltable/getting-started/) | [Documentation](https://pixeltable.readme.io/) | [API Reference](https://pixeltable.github.io/pixeltable/) | [Code Samples](https://pixeltable.readme.io/recipes) | [Examples](https://github.com/pixeltable/pixeltable/tree/master/docs/release/tutorials)
+</div>
+
+Pixeltable is a Python library that lets AI engineers and data scientists focus on exploration, modeling, and app development without dealing with the customary data plumbing.
+
+## What problems does Pixeltable solve?
+
+Today’s solutions for AI app development require extensive custom coding and infrastructure plumbing. Tracking lineage and versions between and across data transformations, models, and deployment is cumbersome. With Pixeltable you can store, transform, index, and iterate on your data within the same table interface, whether it's text, images, embeddings, or even video. Built-in lineage and versioning ensure transparency and reproducibility, while the development-to-production mirror streamlines deployment.
+
+## 💾 Installation
+
+```python
+%pip install pixeltable
+```
+
+To verify that it's working:
+
+```python
+import pixeltable as pxt
+pxt.init()
+```
+> [!NOTE]
+> Check out the [Pixeltable Basics](https://pixeltable.readme.io/docs/pixeltable-basics) tutorial for a tour of its most important features.
+
+## 💡 Get Started
+Learn how to create tables, populate them with data, and enhance them with built-in or user-defined transformations and AI operations.
+
+| Topic      |                                                                                Notebook                                                                                |                                                                                API                                                                                |
+|:--------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------:|
+| Get Started    | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/master/docs/tutorials/pixeltable-basics.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> |                 [![API](https://img.shields.io/badge/api-reference-blue.svg)](https://pixeltable.github.io/pixeltable/api/pixeltable/)                 |
+| User-Defined Functions (UDFs)    | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/master/docs/release/howto/udfs-in-pixeltable.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> |                 [![API](https://img.shields.io/badge/api-reference-blue.svg)](https://pixeltable.github.io/pixeltable/api/iterators/document-splitter/)                 |
+| Comparing Object Detection Models | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/master/docs/release/tutorials/object-detection-in-videos.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a>            | [![API](https://img.shields.io/badge/api-reference-blue.svg)](https://pixeltable.github.io/pixeltable/api-cheat-sheet/#frame-extraction-for-video-data) |
+| Experimenting with Chunking (RAG) | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/master/docs/release/tutorials/rag-operations.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>            | [![API](https://img.shields.io/badge/api-reference-blue.svg)](https://pixeltable.github.io/pixeltable/api/iterators/document-splitter/) |
+| Working with External Files    | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/master/docs/release/howto/working-with-external-files.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> |                 [![API](https://img.shields.io/badge/api-reference-blue.svg)](https://pixeltable.github.io/pixeltable/api-cheat-sheet/#inserting-data-into-a-table)                 |
+
+## ❓ FAQ
+
+### What does Pixeltable provide me with? Pixeltable provides:
+
+- Data storage and versioning
+- Combined Data and Model Lineage
+- Indexing (e.g. embedding vectors) and Data Retrieval
+- Orchestration of multimodal workloads
+- Incremental updates
+- Code is automatically production-ready
+
+### Why should you use Pixeltable?
+
+- **It gives you transparency and reproducibility**
+  - All generated data is automatically recorded and versioned
+  - You will never need to re-run a workload because you lost track of the input data
+- **It saves you money**
+  - All data changes are automatically incremental
+  - You never need to re-run pipelines from scratch because you’re adding data
+- **It integrates with any existing Python code or libraries**
+  - Bring your ever-changing code and workloads
+  - You choose the models, tools, and AI practices (e.g., your embedding model for a vector index); Pixeltable orchestrates the data
+ 
+### What is Pixeltable not providing?
+
+- Pixeltable is not a low-code, prescriptive AI solution. We empower you to use the best frameworks and techniques for your specific needs.
+- We do not aim to replace your existing AI toolkit, but rather enhance it by streamlining the underlying data infrastructure and orchestration.
+
+> [!TIP]
+> Check out the [Integrations](https://pixeltable.readme.io/docs/working-with-openai) section, and feel free to submit a request for additional ones.
+
+## 📙 Example of Use Cases
+
+- **Interact with video data at the frame level** without having to think about frame extraction, intermediate file storage, or storage space explosion.
+- **Augment your data incrementally and interactively with built-in functions and UDFs**, such as image transformations, model inference, and visualizations, without having to think about data pipelines, incremental updates, or capturing function output.
+- **Interact with all the data relevant to your AI application** (video, images, documents, audio, structured data, JSON) through a simple dataframe-style API directly in Python. This includes:
+  - similarity search on embeddings, supported by high-dimensional vector indexing;
+  - path expressions and transformations on JSON data;
+  - PIL and OpenCV image operations;
+  - assembling frames into videos.
+- **Perform keyword and image similarity search at the video frame level** without having to worry about frame storage.
+- **Access all Pixeltable-resident data directly as a PyTorch dataset** in your training scripts.
+- **Understand the compute and storage costs of your data at the granularity** of individual augmentations and get cost projections before adding new data and new augmentations.
+- **Rely on Pixeltable's automatic versioning and snapshot functionality** to protect against regressions and to ensure reproducibility.
+
+## 🐛 Contributions & Feedback
+
+Are you experiencing issues or bugs with Pixeltable? File an [Issue](https://github.com/pixeltable/pixeltable/issues).
+</br>Do you want to contribute? Feel free to open a [PR](https://github.com/pixeltable/pixeltable/pulls).
+
+## :classical_building: License
+
+This library is licensed under the Apache 2.0 License.

{pixeltable-0.2.6 → pixeltable-0.2.8}/pixeltable/__init__.py

@@ -1,5 +1,7 @@
 from .catalog import Column, Table, InsertableTable, View
 from .dataframe import DataFrame
+from .datatransfer import Remote
+from .catalog import Column, Table, InsertableTable, View
 from .exceptions import Error, Error
 from .exprs import RELATIVE_PATH_ROOT
 from .func import Function, udf, uda, Aggregator, expr_udf
@@ -21,7 +23,7 @@ from .type_system import (
 from .utils.help import help
 
 # noinspection PyUnresolvedReferences
-from . import functions, io
+from . import functions, io, iterators
 from .__version__ import __version__, __version_tuple__
 
 __all__ = [

{pixeltable-0.2.6 → pixeltable-0.2.8}/pixeltable/__version__.py

@@ -1,3 +1,3 @@
 # These version placeholders will be replaced during build.
-__version__ = "0.2.6"
-__version_tuple__ = (0, 2, 6)
+__version__ = "0.2.8"
+__version_tuple__ = (0, 2, 8)

{pixeltable-0.2.6 → pixeltable-0.2.8}/pixeltable/catalog/column.py

@@ -22,7 +22,8 @@ class Column:
             computed_with: Optional[Union['Expr', Callable]] = None,
             is_pk: bool = False, stored: Optional[bool] = None,
             col_id: Optional[int] = None, schema_version_add: Optional[int] = None,
-            schema_version_drop: Optional[int] = None, sa_col_type: Optional[sql.sqltypes.TypeEngine] = None
+            schema_version_drop: Optional[int] = None, sa_col_type: Optional[sql.sqltypes.TypeEngine] = None,
+            records_errors: Optional[bool] = None
     ):
         """Column constructor.
 
@@ -80,12 +81,14 @@ class Column:
         assert self.col_type is not None
 
         self.stored = stored
-        self.dependent_cols: Set[Column] = set()  # cols with value_exprs that reference us; set by TableVersion
+        self.dependent_cols: set[Column] = set()  # cols with value_exprs that reference us; set by TableVersion
         self.id = col_id
         self.is_pk = is_pk
         self.schema_version_add = schema_version_add
         self.schema_version_drop = schema_version_drop
 
+        self._records_errors = records_errors
+
         # column in the stored table for the values of this Column
         self.sa_col: Optional[sql.schema.Column] = None
         self.sa_col_type = sa_col_type
@@ -131,6 +134,9 @@ class Column:
     @property
     def records_errors(self) -> bool:
         """True if this column also stores error information."""
+        # default: record errors for computed and media columns
+        if self._records_errors is not None:
+            return self._records_errors
         return self.is_stored and (self.is_computed or self.col_type.is_media_type())
 
     def source(self) -> None:

{pixeltable-0.2.6 → pixeltable-0.2.8}/pixeltable/catalog/insertable_table.py

@@ -60,25 +60,29 @@ class InsertableTable(Table):
             return tbl
 
     @overload
-    def insert(self, rows: Iterable[Dict[str, Any]], /, print_stats: bool = False, fail_on_exception: bool = True): ...
+    def insert(
+            self, rows: Iterable[Dict[str, Any]], /, *, print_stats: bool = False, fail_on_exception: bool = True
+    ) -> UpdateStatus: ...
 
     @overload
-    def insert(self, print_stats: bool = False, fail_on_exception: bool = True, **kwargs: Any): ...
+    def insert(self, *, print_stats: bool = False, fail_on_exception: bool = True, **kwargs: Any) -> UpdateStatus: ...
 
-    def insert(self, *args, **kwargs) -> UpdateStatus:
-        """Insert rows into table.
+    def insert(
+            self, rows: Optional[Iterable[dict[str, Any]]] = None, /, *, print_stats: bool = False,
+            fail_on_exception: bool = True, **kwargs: Any
+    ) -> UpdateStatus:
+        """Inserts rows into this table. There are two mutually exclusive call patterns:
 
         To insert multiple rows at a time:
-
-        ``insert(rows: List[Dict[str, Any]], print_stats: bool = False, fail_on_exception: bool = True)``
+        ``insert(rows: Iterable[dict[str, Any]], /, *, print_stats: bool = False, fail_on_exception: bool = True)``
 
         To insert just a single row, you can use the more convenient syntax:
-        ``insert(print_stats: bool = False, fail_on_exception: bool = True, **kwargs: Any)``
+        ``insert(*, print_stats: bool = False, fail_on_exception: bool = True, **kwargs: Any)``
 
         Args:
             rows: (if inserting multiple rows) A list of rows to insert, each of which is a dictionary mapping column
                 names to values.
-            kwargs: (if inserting a single row) keyword-argument pairs representing column names and values.
+            kwargs: (if inserting a single row) Keyword-argument pairs representing column names and values.
             print_stats: If ``True``, print statistics about the cost of computed columns.
             fail_on_exception:
                 Determines how exceptions in computed columns and invalid media files (e.g., corrupt images)
@@ -102,16 +106,27 @@ class InsertableTable(Table):
 
             >>> tbl.insert(a=1, b=1, c=1)
         """
-        print_stats = kwargs.pop('print_stats', False)
-        fail_on_exception = kwargs.pop('fail_on_exception', True)
-        if len(args) > 0:
-            # There's a positional argument; this means `rows` is expressed as a
-            # list of dicts (multi-insert)
-            rows = list(args[0])
-        else:
-            # No positional argument; this means we're inserting a single row
-            # using kwargs syntax
+        # The commented code is the intended implementation, with signature (*args, **kwargs).
+        # That signature cannot be used currently, due to a present limitation in mkdocs.
+        # See: https://github.com/mkdocstrings/mkdocstrings/issues/669
+
+        # print_stats = kwargs.pop('print_stats', False)
+        # fail_on_exception = kwargs.pop('fail_on_exception', True)
+        # if len(args) > 0:
+        #     # There's a positional argument; this means `rows` is expressed as a
+        #     # list of dicts (multi-insert)
+        #     rows = list(args[0])
+        # else:
+        #     # No positional argument; this means we're inserting a single row
+        #     # using kwargs syntax
+        #     rows = [kwargs]
+
+        if rows is None:
             rows = [kwargs]
+        else:
+            rows = list(rows)
+            if len(kwargs) > 0:
+                raise excs.Error('`kwargs` cannot be specified unless `rows is None`.')
 
         if not isinstance(rows, list):
             raise excs.Error('rows must be a list of dictionaries')

{pixeltable-0.2.6 → pixeltable-0.2.8}/pixeltable/catalog/table.py

@@ -1,9 +1,10 @@
 from __future__ import annotations
 
+import itertools
 import json
 import logging
 from pathlib import Path
-from typing import Union, Any, List, Dict, Optional, Callable, Set, Tuple, Iterable
+from typing import Union, Any, List, Dict, Optional, Callable, Set, Tuple, Iterable, Type
 from uuid import UUID
 
 import pandas as pd
@@ -16,6 +17,7 @@ import pixeltable.exceptions as excs
 import pixeltable.exprs as exprs
 import pixeltable.metadata.schema as schema
 import pixeltable.type_system as ts
+import pixeltable.index as index
 from .column import Column
 from .globals import is_valid_identifier, is_system_column_name, UpdateStatus
 from .schema_object import SchemaObject
@@ -102,27 +104,26 @@ class Table(SchemaObject):
         from pixeltable.dataframe import DataFrame
         return DataFrame(self.tbl_version_path).group_by(*items)
 
-    def collect(self) -> 'pixeltable.dataframe.DataFrameResultSet':  # type: ignore[name-defined, no-untyped-def]
-        """Return rows from this table.
-        """
+    def collect(self) -> 'pixeltable.dataframe.DataFrameResultSet':
+        """Return rows from this table."""
         return self.df().collect()
 
     def show(
             self, *args, **kwargs
-    ) -> 'pixeltable.dataframe.DataFrameResultSet':  # type: ignore[name-defined, no-untyped-def]
+    ) -> 'pixeltable.dataframe.DataFrameResultSet':
         """Return rows from this table.
         """
         return self.df().show(*args, **kwargs)
 
     def head(
             self, *args, **kwargs
-    ) -> 'pixeltable.dataframe.DataFrameResultSet':  # type: ignore[name-defined, no-untyped-def]
+    ) -> 'pixeltable.dataframe.DataFrameResultSet':
         """Return the first n rows inserted into this table."""
         return self.df().head(*args, **kwargs)
 
     def tail(
             self, *args, **kwargs
-    ) -> 'pixeltable.dataframe.DataFrameResultSet':  # type: ignore[name-defined, no-untyped-def]
+    ) -> 'pixeltable.dataframe.DataFrameResultSet':
         """Return the last n rows inserted into this table."""
         return self.df().tail(*args, **kwargs)
 
@@ -514,6 +515,24 @@ class Table(SchemaObject):
         status = self.tbl_version_path.tbl_version.add_index(col, idx_name=idx_name, idx=idx)
         # TODO: how to deal with exceptions here? drop the index and raise?
 
+    def drop_embedding_index(self, *, column_name: Optional[str] = None, idx_name: Optional[str] = None) -> None:
+        """Drop an embedding index from the table.
+
+        Args:
+            column_name: The name of the column whose embedding index to drop. Invalid if the column has multiple
+              embedding indices.
+            idx_name: The name of the index to drop.
+
+        Raises:
+            Error: If the index does not exist.
+
+        Examples:
+            Drop embedding index on the ``img`` column:
+
+            >>> tbl.drop_embedding_index(column_name='img')
+        """
+        self._drop_index(column_name=column_name, idx_name=idx_name, _idx_class=index.EmbeddingIndex)
+
     def drop_index(self, *, column_name: Optional[str] = None, idx_name: Optional[str] = None) -> None:
         """Drop an index from the table.
 
@@ -529,6 +548,12 @@ class Table(SchemaObject):
 
             >>> tbl.drop_index(column_name='img')
         """
+        self._drop_index(column_name=column_name, idx_name=idx_name)
+
+    def _drop_index(
+            self, *, column_name: Optional[str] = None, idx_name: Optional[str] = None,
+            _idx_class: Optional[Type[index.IndexBase]] = None
+    ) -> None:
         if self.tbl_version_path.is_snapshot():
             raise excs.Error('Cannot drop an index from a snapshot')
         self._check_is_dropped()
@@ -547,12 +572,14 @@ class Table(SchemaObject):
             if col.tbl.id != tbl_version.id:
                 raise excs.Error(
                     f'Column {column_name}: cannot drop index from column that belongs to base ({col.tbl.name})')
-            idx_ids = [info.id for info in tbl_version.idxs_by_name.values() if info.col.id == col.id]
-            if len(idx_ids) == 0:
+            idx_info = [info for info in tbl_version.idxs_by_name.values() if info.col.id == col.id]
+            if _idx_class is not None:
+                idx_info = [info for info in idx_info if isinstance(info.idx, _idx_class)]
+            if len(idx_info) == 0:
                 raise excs.Error(f'Column {column_name} does not have an index')
-            if len(idx_ids) > 1:
+            if len(idx_info) > 1:
                 raise excs.Error(f'Column {column_name} has multiple indices; specify idx_name instead')
-            idx_id = idx_ids[0]
+            idx_id = idx_info[0].id
         self.tbl_version_path.tbl_version.drop_index(idx_id)
 
     def update(
@@ -682,7 +709,6 @@ class Table(SchemaObject):
 
         return update_targets
 
-
     def revert(self) -> None:
         """Reverts the table to the previous version.
 
@@ -693,3 +719,132 @@ class Table(SchemaObject):
             raise excs.Error('Cannot revert a snapshot')
         self._check_is_dropped()
         self.tbl_version_path.tbl_version.revert()
+
+    def _link(
+            self,
+            remote: 'pixeltable.datatransfer.Remote',
+            col_mapping: Optional[dict[str, str]] = None
+    ) -> None:
+        """
+        Links the specified `Remote` to this table. Once a remote is linked, it can be synchronized with
+        this `Table` by calling [`Table.sync()`]. A record of the link
+        is stored in table metadata and will persist across sessions.
+
+        Args:
+            remote (pixeltable.datatransfer.Remote): The `Remote` to link to this table.
+            col_mapping: An optional mapping of columns from this `Table` to columns in the `Remote`.
+        """
+        # TODO(aaron-siegel): Refactor `col_mapping`
+        if len(self._get_remotes()) > 0:
+            raise excs.Error('Linking more than one `Remote` to a table is not currently supported.')
+        self._check_is_dropped()
+        export_cols = remote.get_export_columns()
+        import_cols = remote.get_import_columns()
+        is_col_mapping_user_specified = col_mapping is not None
+        if col_mapping is None:
+            # Use the identity mapping by default if `col_mapping` is not specified
+            col_mapping = {col: col for col in itertools.chain(export_cols.keys(), import_cols.keys())}
+        self._validate_remote(export_cols, import_cols, col_mapping, is_col_mapping_user_specified)
+        self.tbl_version_path.tbl_version.link(remote, col_mapping)
+        print(f'Linked remote {remote} to table `{self.get_name()}`.')
+
+    def unlink(self) -> None:
+        """
+        Unlinks this table's `Remote`s.
+        """
+        self._check_is_dropped()
+        remotes = self._get_remotes()
+        assert len(remotes) <= 1
+
+        remote = next(iter(remotes.keys()))
+        self.tbl_version_path.tbl_version.unlink(remote)
+        # TODO: Provide an option to auto-delete the project
+        print(f'Unlinked remote {remote} from table `{self.get_name()}`.')
+
+    def _validate_remote(
+            self,
+            export_cols: dict[str, ts.ColumnType],
+            import_cols: dict[str, ts.ColumnType],
+            col_mapping: Optional[dict[str, str]],
+            is_col_mapping_user_specified: bool
+    ):
+        # Validate names
+        t_cols = self.column_names()
+        for t_col, r_col in col_mapping.items():
+            if t_col not in t_cols:
+                if is_col_mapping_user_specified:
+                    raise excs.Error(
+                        f'Column name `{t_col}` appears as a key in `col_mapping`, but Table `{self.get_name()}` '
+                        'contains no such column.'
+                    )
+                else:
+                    raise excs.Error(
+                        f'Column `{t_col}` does not exist in Table `{self.get_name()}`. Either add a column `{t_col}`, '
+                        f'or specify a `col_mapping` to associate a different column with the remote field `{r_col}`.'
+                    )
+            if r_col not in export_cols and r_col not in import_cols:
+                raise excs.Error(
+                    f'Column name `{r_col}` appears as a value in `col_mapping`, but the remote '
+                    f'configuration has no column `{r_col}`.'
+                )
+        # Validate column specs
+        t_col_types = self.column_types()
+        for t_col, r_col in col_mapping.items():
+            t_col_type = t_col_types[t_col]
+            if r_col in export_cols:
+                # Validate that the table column can be assigned to the remote column
+                r_col_type = export_cols[r_col]
+                if not r_col_type.is_supertype_of(t_col_type):
+                    raise excs.Error(
+                        f'Column `{t_col}` cannot be exported to remote column `{r_col}` (incompatible types)'
+                    )
+            if r_col in import_cols:
+                # Validate that the remote column can be assigned to the table column
+                if self.tbl_version_path.get_column(t_col).is_computed:
+                    raise excs.Error(
+                        f'Column `{t_col}` is a computed column, which cannot be populated from a remote column'
+                    )
+                r_col_type = import_cols[r_col]
+                if not t_col_type.is_supertype_of(r_col_type):
+                    raise excs.Error(
+                        f'Column `{t_col}` cannot be imported from remote column `{r_col}` (incompatible types)'
+                    )
+
+    def _get_remotes(self) -> dict[pixeltable.datatransfer.Remote, dict[str, str]]:
+        """
+        Gets a `dict` of all `Remote`s linked to this table.
+        """
+        return self.tbl_version_path.tbl_version.get_remotes()
+
+    def sync(
+            self,
+            *,
+            export_data: bool = True,
+            import_data: bool = True
+    ):
+        """
+        Synchronizes this table with its linked `Remote`s.
+
+        Args:
+            export_data: If `True`, data from this table will be exported to the external store during synchronization.
+            import_data: If `True`, data from the external store will be imported to this table during synchronization.
+        """
+        remotes = self._get_remotes()
+        assert len(remotes) <= 1
+
+        # Validation
+        for remote in remotes:
+            col_mapping = remotes[remote]
+            r_cols = set(col_mapping.values())
+            # Validate export/import
+            if export_data and not any(col in r_cols for col in remote.get_export_columns()):
+                raise excs.Error(
+                    f'Attempted to sync with export_data=True, but there are no columns to export: {remote}'
+                )
+            if import_data and not any(col in r_cols for col in remote.get_import_columns()):
+                raise excs.Error(
+                    f'Attempted to sync with import_data=True, but there are no columns to import: {remote}'
+                )
+
+        for remote in remotes:
+            remote.sync(self, remotes[remote], export_data=export_data, import_data=import_data)