pixeltable 0.2.22__tar.gz → 0.2.24__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of pixeltable might be problematic. Click here for more details.
- {pixeltable-0.2.22 → pixeltable-0.2.24}/PKG-INFO +119 -46
- {pixeltable-0.2.22 → pixeltable-0.2.24}/README.md +118 -45
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/__init__.py +2 -2
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/__version__.py +2 -2
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/column.py +8 -22
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/insertable_table.py +26 -8
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/table.py +179 -83
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/table_version.py +13 -39
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/table_version_path.py +2 -2
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/view.py +2 -2
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/dataframe.py +20 -28
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/env.py +2 -0
- pixeltable-0.2.24/pixeltable/exec/cache_prefetch_node.py +262 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exec/data_row_batch.py +3 -3
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exec/exec_context.py +2 -2
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exec/exec_node.py +2 -2
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exec/expr_eval_node.py +8 -8
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/arithmetic_expr.py +9 -4
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/column_ref.py +4 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/comparison.py +5 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/json_path.py +1 -1
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/func/aggregate_function.py +8 -8
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/func/expr_template_function.py +6 -5
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/func/udf.py +6 -11
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/huggingface.py +145 -25
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/llama_cpp.py +3 -2
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/mistralai.py +1 -1
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/openai.py +1 -1
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/together.py +1 -1
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/util.py +5 -2
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/globals.py +55 -6
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/plan.py +1 -1
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/tool/create_test_db_dump.py +1 -1
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/type_system.py +83 -35
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/utils/coco.py +5 -5
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/utils/formatter.py +3 -3
- pixeltable-0.2.24/pixeltable/utils/s3.py +16 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pyproject.toml +4 -1
- pixeltable-0.2.22/pixeltable/exec/cache_prefetch_node.py +0 -116
- pixeltable-0.2.22/pixeltable/utils/s3.py +0 -13
- {pixeltable-0.2.22 → pixeltable-0.2.24}/LICENSE +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/__init__.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/catalog.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/dir.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/globals.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/named_function.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/path.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/path_dict.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/catalog/schema_object.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exceptions.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exec/__init__.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exec/aggregation_node.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exec/component_iteration_node.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exec/in_memory_data_node.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exec/row_update_node.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exec/sql_node.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/__init__.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/array_slice.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/column_property_ref.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/compound_predicate.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/data_row.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/expr.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/expr_dict.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/expr_set.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/function_call.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/globals.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/in_predicate.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/inline_expr.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/is_null.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/json_mapper.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/literal.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/method_ref.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/object_ref.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/row_builder.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/rowid_ref.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/similarity_expr.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/sql_element_cache.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/type_cast.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/exprs/variable.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/ext/__init__.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/ext/functions/__init__.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/ext/functions/whisperx.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/ext/functions/yolox.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/func/__init__.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/func/callable_function.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/func/function.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/func/function_registry.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/func/globals.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/func/query_template_function.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/func/signature.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/__init__.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/anthropic.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/audio.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/fireworks.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/globals.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/image.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/json.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/ollama.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/replicate.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/string.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/timestamp.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/video.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/vision.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/functions/whisper.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/index/__init__.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/index/base.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/index/btree.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/index/embedding_index.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/io/__init__.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/io/external_store.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/io/fiftyone.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/io/globals.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/io/hf_datasets.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/io/label_studio.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/io/pandas.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/io/parquet.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/iterators/__init__.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/iterators/base.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/iterators/document.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/iterators/string.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/iterators/video.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/__init__.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/converters/convert_10.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/converters/convert_12.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/converters/convert_13.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/converters/convert_14.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/converters/convert_15.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/converters/convert_16.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/converters/convert_17.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/converters/convert_18.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/converters/convert_19.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/converters/convert_20.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/converters/convert_21.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/converters/util.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/notes.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/metadata/schema.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/py.typed +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/store.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/tool/create_test_video.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/tool/doc_plugins/griffe.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/tool/doc_plugins/mkdocstrings.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/tool/doc_plugins/templates/material/udf.html.jinja +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/tool/embed_udf.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/tool/mypy_plugin.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/utils/__init__.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/utils/arrow.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/utils/code.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/utils/documents.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/utils/filecache.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/utils/http_server.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/utils/media_store.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/utils/pytorch.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/utils/sql.py +0 -0
- {pixeltable-0.2.22 → pixeltable-0.2.24}/pixeltable/utils/transactional_directory.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: pixeltable
|
|
3
|
-
Version: 0.2.
|
|
3
|
+
Version: 0.2.24
|
|
4
4
|
Summary: Pixeltable: The Multimodal AI Data Plane
|
|
5
5
|
Author: Pixeltable, Inc.
|
|
6
6
|
Author-email: contact@pixeltable.com
|
|
@@ -36,7 +36,8 @@ Requires-Dist: tqdm (>=4.64)
|
|
|
36
36
|
Description-Content-Type: text/markdown
|
|
37
37
|
|
|
38
38
|
<div align="center">
|
|
39
|
-
<img src="https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/source/data/pixeltable-logo-large.png"
|
|
39
|
+
<img src="https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/source/data/pixeltable-logo-large.png"
|
|
40
|
+
alt="Pixeltable" width="50%" />
|
|
40
41
|
<br></br>
|
|
41
42
|
|
|
42
43
|
<h2>AI Data Insfrastructure — Declarative, Multimodal, and Incremental</h2>
|
|
@@ -47,43 +48,65 @@ Description-Content-Type: text/markdown
|
|
|
47
48
|
<br>
|
|
48
49
|
[](https://github.com/pixeltable/pixeltable/actions/workflows/pytest.yml)
|
|
49
50
|
[](https://github.com/pixeltable/pixeltable/actions/workflows/nightly.yml)
|
|
50
|
-
[](https://pypi.org/project/pixeltable/)
|
|
51
51
|
[](https://app.fossa.com/projects/git%2Bgithub.com%2Fpixeltable%2Fpixeltable?ref=badge_shield&issueType=security)
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
52
|
+
[](https://pypi.org/project/pixeltable/)
|
|
53
|
+
<a target="_blank" href="https://huggingface.co/Pixeltable">
|
|
54
|
+
<img src="https://img.shields.io/badge/🤗-HF Space-FF7D04" alt="Visit our Hugging Face space"/>
|
|
55
|
+
</a>
|
|
56
|
+
|
|
57
|
+
[Installation](https://docs.pixeltable.com/docs/installation) |
|
|
58
|
+
[Documentation](https://pixeltable.readme.io/) |
|
|
59
|
+
[API Reference](https://pixeltable.github.io/pixeltable/) |
|
|
60
|
+
[Code Samples](https://github.com/pixeltable/pixeltable?tab=readme-ov-file#-code-samples) |
|
|
61
|
+
[Computer Vision](https://docs.pixeltable.com/docs/object-detection-in-videos) |
|
|
62
|
+
[LLM](https://docs.pixeltable.com/docs/document-indexing-and-rag)
|
|
55
63
|
</div>
|
|
56
64
|
|
|
57
|
-
Pixeltable is a Python library providing a declarative interface for multimodal data (text, images, audio, video).
|
|
65
|
+
Pixeltable is a Python library providing a declarative interface for multimodal data (text, images, audio, video).
|
|
66
|
+
It features built-in versioning, lineage tracking, and incremental updates, enabling users to **store**, **transform**,
|
|
67
|
+
**index**, and **iterate** on data for their ML workflows.
|
|
58
68
|
|
|
59
69
|
Data transformations, model inference, and custom logic are embedded as **computed columns**.
|
|
60
|
-
|
|
61
|
-
- **
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
70
|
+
|
|
71
|
+
- **Load/Query all data types**: Interact with
|
|
72
|
+
[video data](https://github.com/pixeltable/pixeltable?tab=readme-ov-file#import-media-data-into-pixeltable-videos-images-audio)
|
|
73
|
+
at the [frame level](https://github.com/pixeltable/pixeltable?tab=readme-ov-file#text-and-image-similarity-search-on-video-frames-with-embedding-indexes)
|
|
74
|
+
and documents at the [chunk level](https://github.com/pixeltable/pixeltable?tab=readme-ov-file#automate-data-operations-with-views-eg-split-documents-into-chunks)
|
|
75
|
+
- **Incremental updates for data transformation**: Maintain an
|
|
76
|
+
[embedding index](https://docs.pixeltable.com/docs/embedding-vector-indexes) colocated with your data
|
|
77
|
+
- **Lazy evaluation and cache management**: Eliminates the need for
|
|
78
|
+
[manual frame extraction](https://docs.pixeltable.com/docs/object-detection-in-videos)
|
|
79
|
+
- **Integrates with any Python libraries**: Use
|
|
80
|
+
[built-in and custom functions (UDFs)](https://docs.pixeltable.com/docs/user-defined-functions-udfs)
|
|
81
|
+
without complex pipelines
|
|
82
|
+
- **Data format agnostic and extensibility**: Access tables as Parquet files,
|
|
83
|
+
[PyTorch datasets](https://pixeltable.github.io/pixeltable/api/data-frame/#pixeltable.DataFrame.to_pytorch_dataset),
|
|
84
|
+
or [COCO annotations](https://pixeltable.github.io/pixeltable/api/table/#pixeltable.Table.to_coco_dataset)
|
|
65
85
|
|
|
66
86
|
## 💾 Installation
|
|
67
87
|
|
|
68
88
|
```python
|
|
69
89
|
pip install pixeltable
|
|
70
90
|
```
|
|
91
|
+
|
|
71
92
|
**Pixeltable is persistent. Unlike in-memory Python libraries such as Pandas, Pixeltable is a database.**
|
|
72
93
|
|
|
73
94
|
## 💡 Getting Started
|
|
95
|
+
|
|
74
96
|
Learn how to create tables, populate them with data, and enhance them with built-in or user-defined transformations.
|
|
75
97
|
|
|
76
98
|
| Topic | Notebook | Topic | Notebook |
|
|
77
99
|
|:----------|:-----------------|:-------------------------|:---------------------------------:|
|
|
78
|
-
| 10-Minute Tour of Pixeltable | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/pixeltable-basics.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> | Tables and Data Operations | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/fundamentals/tables-and-data-operations.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a>
|
|
79
|
-
| User-Defined Functions (UDFs) | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/howto/udfs-in-pixeltable.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> | Object Detection Models | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/object-detection-in-videos.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a>
|
|
80
|
-
| Incremental Prompt Engineering | <a target="_blank" href="https://colab.research.google.com/github/mistralai/cookbook/blob/main/third_party/Pixeltable/incremental_prompt_engineering_and_model_comparison.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Github"/> | Working with External Files | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/howto/working-with-external-files.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a>
|
|
81
|
-
| Integrating with Label Studio | <a target="_blank" href="https://pixeltable.readme.io/docs/label-studio"> <img src="https://img.shields.io/badge/Documentation-013056" alt="Visit our documentation"/></a> | Audio/Video Transcript Indexing | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/audio-transcriptions.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
|
|
82
|
-
| Multimodal Application | <a target="_blank" href="https://huggingface.co/spaces/Pixeltable/Multimodal-Powerhouse"> <img src="https://img.shields.io/badge/Hugging Face-FF7D04" alt="Visit our documentation"/></a> | Document Indexing and RAG | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/rag-demo.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
|
|
100
|
+
| 10-Minute Tour of Pixeltable | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/pixeltable-basics.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> | Tables and Data Operations | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/fundamentals/tables-and-data-operations.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> |
|
|
101
|
+
| User-Defined Functions (UDFs) | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/howto/udfs-in-pixeltable.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> | Object Detection Models | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/object-detection-in-videos.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> |
|
|
102
|
+
| Incremental Prompt Engineering | <a target="_blank" href="https://colab.research.google.com/github/mistralai/cookbook/blob/main/third_party/Pixeltable/incremental_prompt_engineering_and_model_comparison.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Github"/> | Working with External Files | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/howto/working-with-external-files.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> |
|
|
103
|
+
| Integrating with Label Studio | <a target="_blank" href="https://pixeltable.readme.io/docs/label-studio"> <img src="https://img.shields.io/badge/Documentation-013056" alt="Visit our documentation"/></a> | Audio/Video Transcript Indexing | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/audio-transcriptions.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> |
|
|
104
|
+
| Multimodal Application | <a target="_blank" href="https://huggingface.co/spaces/Pixeltable/Multimodal-Powerhouse"> <img src="https://img.shields.io/badge/Hugging Face-FF7D04" alt="Visit our documentation"/></a> | Document Indexing and RAG | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/rag-demo.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> |
|
|
83
105
|
|
|
84
106
|
## 🧱 Code Samples
|
|
85
107
|
|
|
86
108
|
### Import media data into Pixeltable (videos, images, audio...)
|
|
109
|
+
|
|
87
110
|
```python
|
|
88
111
|
import pixeltable as pxt
|
|
89
112
|
|
|
@@ -97,9 +120,11 @@ paths = [
|
|
|
97
120
|
]
|
|
98
121
|
v.insert({'video': prefix + p} for p in paths)
|
|
99
122
|
```
|
|
123
|
+
|
|
100
124
|
Learn how to [work with data in Pixeltable](https://pixeltable.readme.io/docs/working-with-external-files).
|
|
101
125
|
|
|
102
126
|
### Object detection in images using DETR model
|
|
127
|
+
|
|
103
128
|
```python
|
|
104
129
|
import pixeltable as pxt
|
|
105
130
|
from pixeltable.functions import huggingface
|
|
@@ -117,18 +142,22 @@ paths = [
|
|
|
117
142
|
t.insert({'image': prefix + p} for p in paths)
|
|
118
143
|
|
|
119
144
|
# Add a computed column for image classification
|
|
120
|
-
t
|
|
121
|
-
|
|
122
|
-
|
|
145
|
+
t.add_computed_column(classification=huggingface.detr_for_object_detection(
|
|
146
|
+
t.image,
|
|
147
|
+
model_id='facebook/detr-resnet-50'
|
|
148
|
+
))
|
|
123
149
|
|
|
124
150
|
# Retrieve the rows where cats have been identified
|
|
125
151
|
t.select(animal = t.image,
|
|
126
152
|
classification = t.classification.label_text[0]) \
|
|
127
153
|
.where(t.classification.label_text[0]=='cat').head()
|
|
128
154
|
```
|
|
129
|
-
|
|
155
|
+
|
|
156
|
+
Learn about computed columns and object detection:
|
|
157
|
+
[Comparing object detection models](https://pixeltable.readme.io/docs/object-detection-in-videos).
|
|
130
158
|
|
|
131
159
|
### Extend Pixeltable's capabilities with user-defined functions
|
|
160
|
+
|
|
132
161
|
```python
|
|
133
162
|
@pxt.udf
|
|
134
163
|
def draw_boxes(img: PIL.Image.Image, boxes: list[list[float]]) -> PIL.Image.Image:
|
|
@@ -138,9 +167,12 @@ def draw_boxes(img: PIL.Image.Image, boxes: list[list[float]]) -> PIL.Image.Imag
|
|
|
138
167
|
d.rectangle(box, width=3) # Draw bounding box rectangles on the copied image
|
|
139
168
|
return result
|
|
140
169
|
```
|
|
141
|
-
|
|
170
|
+
|
|
171
|
+
Learn more about user-defined functions:
|
|
172
|
+
[UDFs in Pixeltable](https://pixeltable.readme.io/docs/user-defined-functions-udfs).
|
|
142
173
|
|
|
143
174
|
### Automate data operations with views, e.g., split documents into chunks
|
|
175
|
+
|
|
144
176
|
```python
|
|
145
177
|
# In this example, the view is defined by iteration over the chunks of a DocumentSplitter
|
|
146
178
|
chunks_table = pxt.create_view(
|
|
@@ -151,16 +183,21 @@ chunks_table = pxt.create_view(
|
|
|
151
183
|
separators='token_limit', limit=300)
|
|
152
184
|
)
|
|
153
185
|
```
|
|
154
|
-
|
|
186
|
+
|
|
187
|
+
Learn how to leverage views to build your
|
|
188
|
+
[RAG workflow](https://pixeltable.readme.io/docs/document-indexing-and-rag).
|
|
155
189
|
|
|
156
190
|
### Evaluate model performance
|
|
191
|
+
|
|
157
192
|
```python
|
|
158
193
|
# The computation of the mAP metric can become a query over the evaluation output
|
|
159
194
|
frames_view.select(mean_ap(frames_view.eval_yolox_tiny), mean_ap(frames_view.eval_yolox_m)).show()
|
|
160
195
|
```
|
|
196
|
+
|
|
161
197
|
Learn how to leverage Pixeltable for [Model analytics](https://pixeltable.readme.io/docs/object-detection-in-videos).
|
|
162
198
|
|
|
163
199
|
### Working with inference services
|
|
200
|
+
|
|
164
201
|
```python
|
|
165
202
|
chat_table = pxt.create_table('together_demo.chat', {'input': pxt.String})
|
|
166
203
|
|
|
@@ -168,19 +205,23 @@ chat_table = pxt.create_table('together_demo.chat', {'input': pxt.String})
|
|
|
168
205
|
messages = [{'role': 'user', 'content': chat_table.input}]
|
|
169
206
|
|
|
170
207
|
# This example shows how additional parameters from the Together API can be used in Pixeltable
|
|
171
|
-
chat_table
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
208
|
+
chat_table.add_computed_column(
|
|
209
|
+
output=chat_completions(
|
|
210
|
+
messages=messages,
|
|
211
|
+
model='mistralai/Mixtral-8x7B-Instruct-v0.1',
|
|
212
|
+
max_tokens=300,
|
|
213
|
+
stop=['\n'],
|
|
214
|
+
temperature=0.7,
|
|
215
|
+
top_p=0.9,
|
|
216
|
+
top_k=40,
|
|
217
|
+
repetition_penalty=1.1,
|
|
218
|
+
logprobs=1,
|
|
219
|
+
echo=True
|
|
220
|
+
)
|
|
221
|
+
)
|
|
222
|
+
chat_table.add_computed_column(
|
|
223
|
+
response=chat_table.output.choices[0].message.content
|
|
182
224
|
)
|
|
183
|
-
chat_table['response'] = chat_table.output.choices[0].message.content
|
|
184
225
|
|
|
185
226
|
# Start a conversation
|
|
186
227
|
chat_table.insert([
|
|
@@ -189,9 +230,11 @@ chat_table.insert([
|
|
|
189
230
|
])
|
|
190
231
|
chat_table.select(chat_table.input, chat_table.response).head()
|
|
191
232
|
```
|
|
233
|
+
|
|
192
234
|
Learn how to interact with inference services such as [Together AI](https://pixeltable.readme.io/docs/together-ai) in Pixeltable.
|
|
193
235
|
|
|
194
236
|
### Text and image similarity search on video frames with embedding indexes
|
|
237
|
+
|
|
195
238
|
```python
|
|
196
239
|
import pixeltable as pxt
|
|
197
240
|
from pixeltable.functions.huggingface import clip_image, clip_text
|
|
@@ -225,8 +268,8 @@ frames_view.order_by(sim, asc=False).limit(5).select(frames_view.frame, sim=sim)
|
|
|
225
268
|
sample_text = 'red truck'
|
|
226
269
|
sim = frames_view.frame.similarity(sample_text)
|
|
227
270
|
frames_view.order_by(sim, asc=False).limit(5).select(frames_view.frame, sim=sim).collect()
|
|
228
|
-
|
|
229
271
|
```
|
|
272
|
+
|
|
230
273
|
Learn how to work with [Embedding and Vector Indexes](https://docs.pixeltable.com/docs/embedding-vector-indexes).
|
|
231
274
|
|
|
232
275
|
## 🔄 AI Stack Comparison
|
|
@@ -265,11 +308,15 @@ Learn how to work with [Embedding and Vector Indexes](https://docs.pixeltable.co
|
|
|
265
308
|
|
|
266
309
|
### What is Pixeltable?
|
|
267
310
|
|
|
268
|
-
Pixeltable unifies data storage, versioning, and indexing with orchestration and model versioning under a declarative
|
|
311
|
+
Pixeltable unifies data storage, versioning, and indexing with orchestration and model versioning under a declarative
|
|
312
|
+
table interface, with transformations, model inference, and custom logic represented as computed columns.
|
|
269
313
|
|
|
270
314
|
### What problems does Pixeltable solve?
|
|
271
315
|
|
|
272
|
-
Today's solutions for AI app development require extensive custom coding and infrastructure plumbing.
|
|
316
|
+
Today's solutions for AI app development require extensive custom coding and infrastructure plumbing.
|
|
317
|
+
Tracking lineage and versions between and across data transformations, models, and deployments is cumbersome.
|
|
318
|
+
Pixeltable lets ML Engineers and Data Scientists focus on exploration, modeling, and app development without
|
|
319
|
+
dealing with the customary data plumbing.
|
|
273
320
|
|
|
274
321
|
### What does Pixeltable provide me with? Pixeltable provides:
|
|
275
322
|
|
|
@@ -290,22 +337,48 @@ Today's solutions for AI app development require extensive custom coding and inf
|
|
|
290
337
|
- You never need to re-run pipelines from scratch because you’re adding data
|
|
291
338
|
- **It integrates with any existing Python code or libraries**
|
|
292
339
|
- Bring your ever-changing code and workloads
|
|
293
|
-
- You choose the models, tools, and AI practices (e.g., your embedding model for a vector index);
|
|
340
|
+
- You choose the models, tools, and AI practices (e.g., your embedding model for a vector index);
|
|
341
|
+
Pixeltable orchestrates the data
|
|
294
342
|
|
|
295
343
|
### What is Pixeltable not providing?
|
|
296
344
|
|
|
297
|
-
- Pixeltable is not a low-code, prescriptive AI solution. We empower you to use the best frameworks and techniques for
|
|
298
|
-
|
|
345
|
+
- Pixeltable is not a low-code, prescriptive AI solution. We empower you to use the best frameworks and techniques for
|
|
346
|
+
your specific needs.
|
|
347
|
+
- We do not aim to replace your existing AI toolkit, but rather enhance it by streamlining the underlying data
|
|
348
|
+
infrastructure and orchestration.
|
|
299
349
|
|
|
300
350
|
> [!TIP]
|
|
301
|
-
> Check out the [Integrations](https://pixeltable.readme.io/docs/working-with-openai) section, and feel free to submit
|
|
351
|
+
> Check out the [Integrations](https://pixeltable.readme.io/docs/working-with-openai) section, and feel free to submit
|
|
352
|
+
> a request for additional ones.
|
|
353
|
+
|
|
354
|
+
## 🤝 Contributing to Pixeltable
|
|
355
|
+
|
|
356
|
+
We're excited to welcome contributions from the community! Here's how you can get involved:
|
|
357
|
+
|
|
358
|
+
### 🐛 Report Issues
|
|
359
|
+
|
|
360
|
+
- Found a bug? [Open an issue](https://github.com/pixeltable/pixeltable/issues)
|
|
361
|
+
- Include steps to reproduce and environment details
|
|
362
|
+
|
|
363
|
+
### 💡 Submit Changes
|
|
364
|
+
|
|
365
|
+
- Fork the repository
|
|
366
|
+
- Create a feature branch
|
|
367
|
+
- Submit a [pull request](https://github.com/pixeltable/pixeltable/pulls)
|
|
368
|
+
- See our [Contributing Guide](CONTRIBUTING.md) for detailed instructions
|
|
369
|
+
|
|
370
|
+
### 💬 Join the Discussion
|
|
371
|
+
|
|
372
|
+
- Have questions? Start a [Discussion](https://github.com/pixeltable/pixeltable/discussions)
|
|
373
|
+
- Share your Pixeltable projects and use cases
|
|
374
|
+
- Help others in the community
|
|
302
375
|
|
|
303
|
-
|
|
376
|
+
### 📝 Improve Documentation
|
|
304
377
|
|
|
305
|
-
|
|
306
|
-
|
|
378
|
+
- Suggest examples and tutorials
|
|
379
|
+
- Propose improvements
|
|
307
380
|
|
|
308
|
-
##
|
|
381
|
+
## 🏢 License
|
|
309
382
|
|
|
310
383
|
This library is licensed under the Apache 2.0 License.
|
|
311
384
|
|
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
<div align="center">
|
|
2
|
-
<img src="https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/source/data/pixeltable-logo-large.png"
|
|
2
|
+
<img src="https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/source/data/pixeltable-logo-large.png"
|
|
3
|
+
alt="Pixeltable" width="50%" />
|
|
3
4
|
<br></br>
|
|
4
5
|
|
|
5
6
|
<h2>AI Data Insfrastructure — Declarative, Multimodal, and Incremental</h2>
|
|
@@ -10,43 +11,65 @@
|
|
|
10
11
|
<br>
|
|
11
12
|
[](https://github.com/pixeltable/pixeltable/actions/workflows/pytest.yml)
|
|
12
13
|
[](https://github.com/pixeltable/pixeltable/actions/workflows/nightly.yml)
|
|
13
|
-
[](https://pypi.org/project/pixeltable/)
|
|
14
14
|
[](https://app.fossa.com/projects/git%2Bgithub.com%2Fpixeltable%2Fpixeltable?ref=badge_shield&issueType=security)
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
15
|
+
[](https://pypi.org/project/pixeltable/)
|
|
16
|
+
<a target="_blank" href="https://huggingface.co/Pixeltable">
|
|
17
|
+
<img src="https://img.shields.io/badge/🤗-HF Space-FF7D04" alt="Visit our Hugging Face space"/>
|
|
18
|
+
</a>
|
|
19
|
+
|
|
20
|
+
[Installation](https://docs.pixeltable.com/docs/installation) |
|
|
21
|
+
[Documentation](https://pixeltable.readme.io/) |
|
|
22
|
+
[API Reference](https://pixeltable.github.io/pixeltable/) |
|
|
23
|
+
[Code Samples](https://github.com/pixeltable/pixeltable?tab=readme-ov-file#-code-samples) |
|
|
24
|
+
[Computer Vision](https://docs.pixeltable.com/docs/object-detection-in-videos) |
|
|
25
|
+
[LLM](https://docs.pixeltable.com/docs/document-indexing-and-rag)
|
|
18
26
|
</div>
|
|
19
27
|
|
|
20
|
-
Pixeltable is a Python library providing a declarative interface for multimodal data (text, images, audio, video).
|
|
28
|
+
Pixeltable is a Python library providing a declarative interface for multimodal data (text, images, audio, video).
|
|
29
|
+
It features built-in versioning, lineage tracking, and incremental updates, enabling users to **store**, **transform**,
|
|
30
|
+
**index**, and **iterate** on data for their ML workflows.
|
|
21
31
|
|
|
22
32
|
Data transformations, model inference, and custom logic are embedded as **computed columns**.
|
|
23
|
-
|
|
24
|
-
- **
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
33
|
+
|
|
34
|
+
- **Load/Query all data types**: Interact with
|
|
35
|
+
[video data](https://github.com/pixeltable/pixeltable?tab=readme-ov-file#import-media-data-into-pixeltable-videos-images-audio)
|
|
36
|
+
at the [frame level](https://github.com/pixeltable/pixeltable?tab=readme-ov-file#text-and-image-similarity-search-on-video-frames-with-embedding-indexes)
|
|
37
|
+
and documents at the [chunk level](https://github.com/pixeltable/pixeltable?tab=readme-ov-file#automate-data-operations-with-views-eg-split-documents-into-chunks)
|
|
38
|
+
- **Incremental updates for data transformation**: Maintain an
|
|
39
|
+
[embedding index](https://docs.pixeltable.com/docs/embedding-vector-indexes) colocated with your data
|
|
40
|
+
- **Lazy evaluation and cache management**: Eliminates the need for
|
|
41
|
+
[manual frame extraction](https://docs.pixeltable.com/docs/object-detection-in-videos)
|
|
42
|
+
- **Integrates with any Python libraries**: Use
|
|
43
|
+
[built-in and custom functions (UDFs)](https://docs.pixeltable.com/docs/user-defined-functions-udfs)
|
|
44
|
+
without complex pipelines
|
|
45
|
+
- **Data format agnostic and extensibility**: Access tables as Parquet files,
|
|
46
|
+
[PyTorch datasets](https://pixeltable.github.io/pixeltable/api/data-frame/#pixeltable.DataFrame.to_pytorch_dataset),
|
|
47
|
+
or [COCO annotations](https://pixeltable.github.io/pixeltable/api/table/#pixeltable.Table.to_coco_dataset)
|
|
28
48
|
|
|
29
49
|
## 💾 Installation
|
|
30
50
|
|
|
31
51
|
```python
|
|
32
52
|
pip install pixeltable
|
|
33
53
|
```
|
|
54
|
+
|
|
34
55
|
**Pixeltable is persistent. Unlike in-memory Python libraries such as Pandas, Pixeltable is a database.**
|
|
35
56
|
|
|
36
57
|
## 💡 Getting Started
|
|
58
|
+
|
|
37
59
|
Learn how to create tables, populate them with data, and enhance them with built-in or user-defined transformations.
|
|
38
60
|
|
|
39
61
|
| Topic | Notebook | Topic | Notebook |
|
|
40
62
|
|:----------|:-----------------|:-------------------------|:---------------------------------:|
|
|
41
|
-
| 10-Minute Tour of Pixeltable | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/pixeltable-basics.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> | Tables and Data Operations | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/fundamentals/tables-and-data-operations.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a>
|
|
42
|
-
| User-Defined Functions (UDFs) | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/howto/udfs-in-pixeltable.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> | Object Detection Models | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/object-detection-in-videos.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a>
|
|
43
|
-
| Incremental Prompt Engineering | <a target="_blank" href="https://colab.research.google.com/github/mistralai/cookbook/blob/main/third_party/Pixeltable/incremental_prompt_engineering_and_model_comparison.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Github"/> | Working with External Files | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/howto/working-with-external-files.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a>
|
|
44
|
-
| Integrating with Label Studio | <a target="_blank" href="https://pixeltable.readme.io/docs/label-studio"> <img src="https://img.shields.io/badge/Documentation-013056" alt="Visit our documentation"/></a> | Audio/Video Transcript Indexing | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/audio-transcriptions.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
|
|
45
|
-
| Multimodal Application | <a target="_blank" href="https://huggingface.co/spaces/Pixeltable/Multimodal-Powerhouse"> <img src="https://img.shields.io/badge/Hugging Face-FF7D04" alt="Visit our documentation"/></a> | Document Indexing and RAG | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/rag-demo.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
|
|
63
|
+
| 10-Minute Tour of Pixeltable | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/pixeltable-basics.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> | Tables and Data Operations | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/fundamentals/tables-and-data-operations.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> |
|
|
64
|
+
| User-Defined Functions (UDFs) | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/howto/udfs-in-pixeltable.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> | Object Detection Models | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/object-detection-in-videos.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> |
|
|
65
|
+
| Incremental Prompt Engineering | <a target="_blank" href="https://colab.research.google.com/github/mistralai/cookbook/blob/main/third_party/Pixeltable/incremental_prompt_engineering_and_model_comparison.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Github"/> | Working with External Files | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/howto/working-with-external-files.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a> |
|
|
66
|
+
| Integrating with Label Studio | <a target="_blank" href="https://pixeltable.readme.io/docs/label-studio"> <img src="https://img.shields.io/badge/Documentation-013056" alt="Visit our documentation"/></a> | Audio/Video Transcript Indexing | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/audio-transcriptions.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> |
|
|
67
|
+
| Multimodal Application | <a target="_blank" href="https://huggingface.co/spaces/Pixeltable/Multimodal-Powerhouse"> <img src="https://img.shields.io/badge/Hugging Face-FF7D04" alt="Visit our documentation"/></a> | Document Indexing and RAG | <a target="_blank" href="https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/tutorials/rag-demo.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> |
|
|
46
68
|
|
|
47
69
|
## 🧱 Code Samples
|
|
48
70
|
|
|
49
71
|
### Import media data into Pixeltable (videos, images, audio...)
|
|
72
|
+
|
|
50
73
|
```python
|
|
51
74
|
import pixeltable as pxt
|
|
52
75
|
|
|
@@ -60,9 +83,11 @@ paths = [
|
|
|
60
83
|
]
|
|
61
84
|
v.insert({'video': prefix + p} for p in paths)
|
|
62
85
|
```
|
|
86
|
+
|
|
63
87
|
Learn how to [work with data in Pixeltable](https://pixeltable.readme.io/docs/working-with-external-files).
|
|
64
88
|
|
|
65
89
|
### Object detection in images using DETR model
|
|
90
|
+
|
|
66
91
|
```python
|
|
67
92
|
import pixeltable as pxt
|
|
68
93
|
from pixeltable.functions import huggingface
|
|
@@ -80,18 +105,22 @@ paths = [
|
|
|
80
105
|
t.insert({'image': prefix + p} for p in paths)
|
|
81
106
|
|
|
82
107
|
# Add a computed column for image classification
|
|
83
|
-
t
|
|
84
|
-
|
|
85
|
-
|
|
108
|
+
t.add_computed_column(classification=huggingface.detr_for_object_detection(
|
|
109
|
+
t.image,
|
|
110
|
+
model_id='facebook/detr-resnet-50'
|
|
111
|
+
))
|
|
86
112
|
|
|
87
113
|
# Retrieve the rows where cats have been identified
|
|
88
114
|
t.select(animal = t.image,
|
|
89
115
|
classification = t.classification.label_text[0]) \
|
|
90
116
|
.where(t.classification.label_text[0]=='cat').head()
|
|
91
117
|
```
|
|
92
|
-
|
|
118
|
+
|
|
119
|
+
Learn about computed columns and object detection:
|
|
120
|
+
[Comparing object detection models](https://pixeltable.readme.io/docs/object-detection-in-videos).
|
|
93
121
|
|
|
94
122
|
### Extend Pixeltable's capabilities with user-defined functions
|
|
123
|
+
|
|
95
124
|
```python
|
|
96
125
|
@pxt.udf
|
|
97
126
|
def draw_boxes(img: PIL.Image.Image, boxes: list[list[float]]) -> PIL.Image.Image:
|
|
@@ -101,9 +130,12 @@ def draw_boxes(img: PIL.Image.Image, boxes: list[list[float]]) -> PIL.Image.Imag
|
|
|
101
130
|
d.rectangle(box, width=3) # Draw bounding box rectangles on the copied image
|
|
102
131
|
return result
|
|
103
132
|
```
|
|
104
|
-
|
|
133
|
+
|
|
134
|
+
Learn more about user-defined functions:
|
|
135
|
+
[UDFs in Pixeltable](https://pixeltable.readme.io/docs/user-defined-functions-udfs).
|
|
105
136
|
|
|
106
137
|
### Automate data operations with views, e.g., split documents into chunks
|
|
138
|
+
|
|
107
139
|
```python
|
|
108
140
|
# In this example, the view is defined by iteration over the chunks of a DocumentSplitter
|
|
109
141
|
chunks_table = pxt.create_view(
|
|
@@ -114,16 +146,21 @@ chunks_table = pxt.create_view(
|
|
|
114
146
|
separators='token_limit', limit=300)
|
|
115
147
|
)
|
|
116
148
|
```
|
|
117
|
-
|
|
149
|
+
|
|
150
|
+
Learn how to leverage views to build your
|
|
151
|
+
[RAG workflow](https://pixeltable.readme.io/docs/document-indexing-and-rag).
|
|
118
152
|
|
|
119
153
|
### Evaluate model performance
|
|
154
|
+
|
|
120
155
|
```python
|
|
121
156
|
# The computation of the mAP metric can become a query over the evaluation output
|
|
122
157
|
frames_view.select(mean_ap(frames_view.eval_yolox_tiny), mean_ap(frames_view.eval_yolox_m)).show()
|
|
123
158
|
```
|
|
159
|
+
|
|
124
160
|
Learn how to leverage Pixeltable for [Model analytics](https://pixeltable.readme.io/docs/object-detection-in-videos).
|
|
125
161
|
|
|
126
162
|
### Working with inference services
|
|
163
|
+
|
|
127
164
|
```python
|
|
128
165
|
chat_table = pxt.create_table('together_demo.chat', {'input': pxt.String})
|
|
129
166
|
|
|
@@ -131,19 +168,23 @@ chat_table = pxt.create_table('together_demo.chat', {'input': pxt.String})
|
|
|
131
168
|
messages = [{'role': 'user', 'content': chat_table.input}]
|
|
132
169
|
|
|
133
170
|
# This example shows how additional parameters from the Together API can be used in Pixeltable
|
|
134
|
-
chat_table
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
171
|
+
chat_table.add_computed_column(
|
|
172
|
+
output=chat_completions(
|
|
173
|
+
messages=messages,
|
|
174
|
+
model='mistralai/Mixtral-8x7B-Instruct-v0.1',
|
|
175
|
+
max_tokens=300,
|
|
176
|
+
stop=['\n'],
|
|
177
|
+
temperature=0.7,
|
|
178
|
+
top_p=0.9,
|
|
179
|
+
top_k=40,
|
|
180
|
+
repetition_penalty=1.1,
|
|
181
|
+
logprobs=1,
|
|
182
|
+
echo=True
|
|
183
|
+
)
|
|
184
|
+
)
|
|
185
|
+
chat_table.add_computed_column(
|
|
186
|
+
response=chat_table.output.choices[0].message.content
|
|
145
187
|
)
|
|
146
|
-
chat_table['response'] = chat_table.output.choices[0].message.content
|
|
147
188
|
|
|
148
189
|
# Start a conversation
|
|
149
190
|
chat_table.insert([
|
|
@@ -152,9 +193,11 @@ chat_table.insert([
|
|
|
152
193
|
])
|
|
153
194
|
chat_table.select(chat_table.input, chat_table.response).head()
|
|
154
195
|
```
|
|
196
|
+
|
|
155
197
|
Learn how to interact with inference services such as [Together AI](https://pixeltable.readme.io/docs/together-ai) in Pixeltable.
|
|
156
198
|
|
|
157
199
|
### Text and image similarity search on video frames with embedding indexes
|
|
200
|
+
|
|
158
201
|
```python
|
|
159
202
|
import pixeltable as pxt
|
|
160
203
|
from pixeltable.functions.huggingface import clip_image, clip_text
|
|
@@ -188,8 +231,8 @@ frames_view.order_by(sim, asc=False).limit(5).select(frames_view.frame, sim=sim)
|
|
|
188
231
|
sample_text = 'red truck'
|
|
189
232
|
sim = frames_view.frame.similarity(sample_text)
|
|
190
233
|
frames_view.order_by(sim, asc=False).limit(5).select(frames_view.frame, sim=sim).collect()
|
|
191
|
-
|
|
192
234
|
```
|
|
235
|
+
|
|
193
236
|
Learn how to work with [Embedding and Vector Indexes](https://docs.pixeltable.com/docs/embedding-vector-indexes).
|
|
194
237
|
|
|
195
238
|
## 🔄 AI Stack Comparison
|
|
@@ -228,11 +271,15 @@ Learn how to work with [Embedding and Vector Indexes](https://docs.pixeltable.co
|
|
|
228
271
|
|
|
229
272
|
### What is Pixeltable?
|
|
230
273
|
|
|
231
|
-
Pixeltable unifies data storage, versioning, and indexing with orchestration and model versioning under a declarative
|
|
274
|
+
Pixeltable unifies data storage, versioning, and indexing with orchestration and model versioning under a declarative
|
|
275
|
+
table interface, with transformations, model inference, and custom logic represented as computed columns.
|
|
232
276
|
|
|
233
277
|
### What problems does Pixeltable solve?
|
|
234
278
|
|
|
235
|
-
Today's solutions for AI app development require extensive custom coding and infrastructure plumbing.
|
|
279
|
+
Today's solutions for AI app development require extensive custom coding and infrastructure plumbing.
|
|
280
|
+
Tracking lineage and versions between and across data transformations, models, and deployments is cumbersome.
|
|
281
|
+
Pixeltable lets ML Engineers and Data Scientists focus on exploration, modeling, and app development without
|
|
282
|
+
dealing with the customary data plumbing.
|
|
236
283
|
|
|
237
284
|
### What does Pixeltable provide me with? Pixeltable provides:
|
|
238
285
|
|
|
@@ -253,21 +300,47 @@ Today's solutions for AI app development require extensive custom coding and inf
|
|
|
253
300
|
- You never need to re-run pipelines from scratch because you’re adding data
|
|
254
301
|
- **It integrates with any existing Python code or libraries**
|
|
255
302
|
- Bring your ever-changing code and workloads
|
|
256
|
-
- You choose the models, tools, and AI practices (e.g., your embedding model for a vector index);
|
|
303
|
+
- You choose the models, tools, and AI practices (e.g., your embedding model for a vector index);
|
|
304
|
+
Pixeltable orchestrates the data
|
|
257
305
|
|
|
258
306
|
### What is Pixeltable not providing?
|
|
259
307
|
|
|
260
|
-
- Pixeltable is not a low-code, prescriptive AI solution. We empower you to use the best frameworks and techniques for
|
|
261
|
-
|
|
308
|
+
- Pixeltable is not a low-code, prescriptive AI solution. We empower you to use the best frameworks and techniques for
|
|
309
|
+
your specific needs.
|
|
310
|
+
- We do not aim to replace your existing AI toolkit, but rather enhance it by streamlining the underlying data
|
|
311
|
+
infrastructure and orchestration.
|
|
262
312
|
|
|
263
313
|
> [!TIP]
|
|
264
|
-
> Check out the [Integrations](https://pixeltable.readme.io/docs/working-with-openai) section, and feel free to submit
|
|
314
|
+
> Check out the [Integrations](https://pixeltable.readme.io/docs/working-with-openai) section, and feel free to submit
|
|
315
|
+
> a request for additional ones.
|
|
316
|
+
|
|
317
|
+
## 🤝 Contributing to Pixeltable
|
|
318
|
+
|
|
319
|
+
We're excited to welcome contributions from the community! Here's how you can get involved:
|
|
320
|
+
|
|
321
|
+
### 🐛 Report Issues
|
|
322
|
+
|
|
323
|
+
- Found a bug? [Open an issue](https://github.com/pixeltable/pixeltable/issues)
|
|
324
|
+
- Include steps to reproduce and environment details
|
|
325
|
+
|
|
326
|
+
### 💡 Submit Changes
|
|
327
|
+
|
|
328
|
+
- Fork the repository
|
|
329
|
+
- Create a feature branch
|
|
330
|
+
- Submit a [pull request](https://github.com/pixeltable/pixeltable/pulls)
|
|
331
|
+
- See our [Contributing Guide](CONTRIBUTING.md) for detailed instructions
|
|
332
|
+
|
|
333
|
+
### 💬 Join the Discussion
|
|
334
|
+
|
|
335
|
+
- Have questions? Start a [Discussion](https://github.com/pixeltable/pixeltable/discussions)
|
|
336
|
+
- Share your Pixeltable projects and use cases
|
|
337
|
+
- Help others in the community
|
|
265
338
|
|
|
266
|
-
|
|
339
|
+
### 📝 Improve Documentation
|
|
267
340
|
|
|
268
|
-
|
|
269
|
-
|
|
341
|
+
- Suggest examples and tutorials
|
|
342
|
+
- Propose improvements
|
|
270
343
|
|
|
271
|
-
##
|
|
344
|
+
## 🏢 License
|
|
272
345
|
|
|
273
346
|
This library is licensed under the Apache 2.0 License.
|