event2vector 0.3.0__tar.gz → 0.4.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- event2vector-0.4.0/CHANGELOG.md +64 -0
- event2vector-0.4.0/CITATION.cff +26 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/MANIFEST.in +2 -0
- {event2vector-0.3.0/event2vector.egg-info → event2vector-0.4.0}/PKG-INFO +40 -5
- event2vector-0.3.0/PKG-INFO → event2vector-0.4.0/README.md +35 -35
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/__init__.py +3 -1
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/geometries.py +167 -1
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/models.py +26 -2
- event2vector-0.3.0/README.md → event2vector-0.4.0/event2vector.egg-info/PKG-INFO +70 -1
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector.egg-info/SOURCES.txt +4 -2
- {event2vector-0.3.0 → event2vector-0.4.0}/pyproject.toml +5 -4
- event2vector-0.4.0/tests/test_affine.py +97 -0
- event2vector-0.4.0/tests/test_device.py +75 -0
- event2vector-0.3.0/tests/test_datasets.py +0 -82
- event2vector-0.3.0/tests/test_models.py +0 -360
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/baselines.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/data.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/datasets/__init__.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/datasets/arithmetic.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/datasets/brown.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/datasets/decay.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/datasets/groups.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/datasets/lifepath.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/datasets/realdata.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/eval/__init__.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/eval/figures.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/eval/harness.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector/metrics.py +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector.egg-info/dependency_links.txt +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector.egg-info/requires.txt +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/event2vector.egg-info/top_level.txt +0 -0
- {event2vector-0.3.0 → event2vector-0.4.0}/setup.cfg +0 -0
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to `event2vector` are documented here. The format follows
|
|
4
|
+
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project uses
|
|
5
|
+
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
|
+
|
|
7
|
+
## [0.4.0] - 2026-10-15
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- `geometry="affine"` (`AffineModel`): each event is an invertible affine map
|
|
12
|
+
`x -> exp(M_e) x + v_e` applied on the left of an affine state, so an event can
|
|
13
|
+
act on the history (shrink, stretch or rotate it) instead of only adding to it.
|
|
14
|
+
With `M_e = 0`, the initialisation, the point is exactly the Euclidean sum. The
|
|
15
|
+
step is recovered exactly by the reconstruction loss, and subsequences compose
|
|
16
|
+
into one affine map:
|
|
17
|
+
- `AffineModel.point(state)` returns the plotted point `b` from a state;
|
|
18
|
+
- `AffineModel.compose(event_ids)` returns one `(A, v)` for a subsequence;
|
|
19
|
+
- `AffineModel.override_strength(event_ids)` returns the singular values of its
|
|
20
|
+
linear part.
|
|
21
|
+
- Estimator options `affine_readout` (`"point"` or `"full"`) and
|
|
22
|
+
`affine_max_log_scale` (bounds how much one event can stretch or shrink the state).
|
|
23
|
+
- Tests for the affine geometry (`tests/test_affine.py`).
|
|
24
|
+
|
|
25
|
+
### Changed
|
|
26
|
+
|
|
27
|
+
- The MPS-safe `matrix_exp` is now a shared helper used by both the `rotation` and
|
|
28
|
+
`affine` geometries, and automatic device selection falls back to the CPU for
|
|
29
|
+
`affine` as it already did for `rotation`.
|
|
30
|
+
|
|
31
|
+
## [0.3.1] - 2026-10-11
|
|
32
|
+
|
|
33
|
+
### Fixed
|
|
34
|
+
|
|
35
|
+
- `Event2Vec(geometry="rotation")` no longer crashes on Apple Silicon with
|
|
36
|
+
`NotImplementedError: The operator 'aten::linalg_matrix_exp' is not currently
|
|
37
|
+
implemented for the MPS device`. `RotationModel` now computes the matrix
|
|
38
|
+
exponential on the CPU when its input is on MPS and moves the result back, so
|
|
39
|
+
`device="mps"`, a bare `RotationModel` and a `ProductModel` with a rotation
|
|
40
|
+
factor all work.
|
|
41
|
+
- The project links on PyPI pointed to `github.com/sulcantonin/event2vector`,
|
|
42
|
+
which does not exist; they now point to
|
|
43
|
+
[`github.com/sulcantonin/event2vec_public`](https://github.com/sulcantonin/event2vec_public).
|
|
44
|
+
|
|
45
|
+
### Changed
|
|
46
|
+
|
|
47
|
+
- When the device is chosen automatically and MPS would be picked, the
|
|
48
|
+
`rotation` geometry runs on the CPU instead and emits a `RuntimeWarning`. On
|
|
49
|
+
MPS every step round-trips through the CPU, which made training roughly 8–14×
|
|
50
|
+
slower than on the CPU alone for this model. Pass `device="mps"` to use MPS
|
|
51
|
+
anyway. CUDA and the other geometries are unaffected.
|
|
52
|
+
|
|
53
|
+
### Added
|
|
54
|
+
|
|
55
|
+
- Tests for the automatic device selection (`tests/test_device.py`).
|
|
56
|
+
- This changelog, linked from the PyPI project page.
|
|
57
|
+
|
|
58
|
+
## [0.3.0] - 2026-07-21
|
|
59
|
+
|
|
60
|
+
Earlier releases are listed on [PyPI](https://pypi.org/project/event2vector/#history).
|
|
61
|
+
|
|
62
|
+
[0.4.0]: https://pypi.org/project/event2vector/0.4.0/
|
|
63
|
+
[0.3.1]: https://pypi.org/project/event2vector/0.3.1/
|
|
64
|
+
[0.3.0]: https://pypi.org/project/event2vector/0.3.0/
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
cff-version: 1.2.0
|
|
2
|
+
message: "If you use event2vector, please cite the paper below."
|
|
3
|
+
title: "event2vector: geometric embeddings for event sequences"
|
|
4
|
+
type: software
|
|
5
|
+
authors:
|
|
6
|
+
- family-names: Sulc
|
|
7
|
+
given-names: Antonin
|
|
8
|
+
version: 0.4.0
|
|
9
|
+
date-released: 2026-10-15
|
|
10
|
+
license: MIT
|
|
11
|
+
repository-code: "https://github.com/sulcantonin/event2vec_public"
|
|
12
|
+
url: "https://event2vec.com"
|
|
13
|
+
keywords:
|
|
14
|
+
- sequence-embedding
|
|
15
|
+
- event-sequences
|
|
16
|
+
- representation-learning
|
|
17
|
+
- geometric deep learning
|
|
18
|
+
preferred-citation:
|
|
19
|
+
type: article
|
|
20
|
+
title: "Event2Vec: A Geometric Approach to Learning Composable Representations of Event Sequences"
|
|
21
|
+
authors:
|
|
22
|
+
- family-names: Sulc
|
|
23
|
+
given-names: Antonin
|
|
24
|
+
journal: "arXiv preprint arXiv:2509.12188"
|
|
25
|
+
year: 2025
|
|
26
|
+
url: "https://arxiv.org/abs/2509.12188"
|
|
@@ -1,12 +1,13 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: event2vector
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.4.0
|
|
4
4
|
Summary: Scikit-learn-style geometric embeddings for event sequences.
|
|
5
5
|
Author: Antonin Sulc
|
|
6
6
|
License-Expression: MIT
|
|
7
|
-
Project-URL: Homepage, https://github.com/sulcantonin/
|
|
8
|
-
Project-URL: Repository, https://github.com/sulcantonin/
|
|
9
|
-
Project-URL: Issues, https://github.com/sulcantonin/
|
|
7
|
+
Project-URL: Homepage, https://github.com/sulcantonin/event2vec_public
|
|
8
|
+
Project-URL: Repository, https://github.com/sulcantonin/event2vec_public
|
|
9
|
+
Project-URL: Issues, https://github.com/sulcantonin/event2vec_public/issues
|
|
10
|
+
Project-URL: Changelog, https://github.com/sulcantonin/event2vec_public/blob/main/CHANGELOG.md
|
|
10
11
|
Keywords: sequence-embedding,event-sequences,representation-learning,word2vec,scikit-learn,time-series
|
|
11
12
|
Classifier: Programming Language :: Python :: 3
|
|
12
13
|
Classifier: Operating System :: OS Independent
|
|
@@ -132,7 +133,7 @@ Key methods:
|
|
|
132
133
|
- `most_similar`: gensim-style nearest-neighbor lookup over learned event embeddings using tokens or full sequences as queries.
|
|
133
134
|
- `pad_sequences=True`: enables fully vectorized batches with masking for substantial throughput gains on large corpora.
|
|
134
135
|
|
|
135
|
-
Device control: set `use_gpu=False` to force CPU even if CUDA/MPS is present, or pass an explicit `device` (e.g., `"cuda:0"` or `"cpu"`).
|
|
136
|
+
Device control: set `use_gpu=False` to force CPU even if CUDA/MPS is present, or pass an explicit `device` (e.g., `"cuda:0"` or `"cpu"`). On Apple Silicon the `rotation` and `affine` geometries default to the CPU, because MPS has no `matrix_exp` kernel and the CPU is faster for them; pass `device="mps"` to override.
|
|
136
137
|
|
|
137
138
|
## Geometries (v0.2)
|
|
138
139
|
|
|
@@ -146,6 +147,7 @@ Beyond `euclidean` and `hyperbolic`, the estimator exposes the paper's pilot geo
|
|
|
146
147
|
| `spherical` | κ-gyroaddition (`kappa>0`) | cyclic / modular (periodic) structure |
|
|
147
148
|
| `rotation` | left `SO(n)` product | non-commutative orientation (order matters) |
|
|
148
149
|
| `heisenberg` | Heisenberg group product | order-dependent signed (Lévy) area |
|
|
150
|
+
| `affine` | `x -> exp(M_e) x + v_e` on the left | events that act on the history (shrink, stretch, rotate it), not only add to it |
|
|
149
151
|
| `product` | per-factor update | two structure types at once (e.g. hyperbolic × spherical) |
|
|
150
152
|
|
|
151
153
|
```python
|
|
@@ -161,6 +163,39 @@ model = Event2Vec(num_event_types=len(vocab), geometry="rotation", rotation_n=3)
|
|
|
161
163
|
model = Event2Vec(num_event_types=len(vocab), geometry="product", product_kappas=(-1.0, 1.0))
|
|
162
164
|
```
|
|
163
165
|
|
|
166
|
+
### Affine geometry (v0.4)
|
|
167
|
+
|
|
168
|
+
In the additive geometries an event always pushes the same way. With
|
|
169
|
+
`geometry="affine"` each event is an invertible affine map
|
|
170
|
+
`x -> exp(M_e) x + v_e`, so the same event contributes differently depending on
|
|
171
|
+
what came before, while the model stays a group:
|
|
172
|
+
|
|
173
|
+
- with every `M_e = 0` (the initialisation) the point is exactly the Euclidean sum,
|
|
174
|
+
so training starts from the additive model;
|
|
175
|
+
- every step is invertible and the reconstruction loss recovers it exactly;
|
|
176
|
+
- any subsequence composes into a single affine map, so its effect on *any*
|
|
177
|
+
history is one `A x + v`.
|
|
178
|
+
|
|
179
|
+
The state is the whole map `[vec(A), b]`; the point to plot is `b`, the image of
|
|
180
|
+
the origin (`model.model.point(...)`). The singular values of a subsequence's `A`
|
|
181
|
+
say how much it overrides what came before: `1` passes the history through,
|
|
182
|
+
`< 1` erases it along that direction.
|
|
183
|
+
|
|
184
|
+
```python
|
|
185
|
+
model = Event2Vec(num_event_types=len(vocab), geometry="affine", embedding_dim=2)
|
|
186
|
+
model.fit(sequences)
|
|
187
|
+
|
|
188
|
+
states = model.transform(sequences, as_numpy=False) # [vec(A), b] per sequence
|
|
189
|
+
points = model.model.point(states) # (n, 2) points to plot
|
|
190
|
+
|
|
191
|
+
A, v = model.model.compose([3, 7, 1]) # one map for a subsequence
|
|
192
|
+
print(model.model.override_strength([3, 7, 1])) # its singular values
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Options: `affine_readout="point"` (default; the decoder reads `b`) or `"full"`
|
|
196
|
+
(it also reads `A`), and `affine_max_log_scale` to bound how strongly a single
|
|
197
|
+
event can stretch or shrink the state.
|
|
198
|
+
|
|
164
199
|
Reproduce the curvature sweep on a real knowledge graph (auto-downloads WN18RR):
|
|
165
200
|
|
|
166
201
|
```bash
|
|
@@ -1,37 +1,3 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: event2vector
|
|
3
|
-
Version: 0.3.0
|
|
4
|
-
Summary: Scikit-learn-style geometric embeddings for event sequences.
|
|
5
|
-
Author: Antonin Sulc
|
|
6
|
-
License-Expression: MIT
|
|
7
|
-
Project-URL: Homepage, https://github.com/sulcantonin/event2vector
|
|
8
|
-
Project-URL: Repository, https://github.com/sulcantonin/event2vector
|
|
9
|
-
Project-URL: Issues, https://github.com/sulcantonin/event2vector/issues
|
|
10
|
-
Keywords: sequence-embedding,event-sequences,representation-learning,word2vec,scikit-learn,time-series
|
|
11
|
-
Classifier: Programming Language :: Python :: 3
|
|
12
|
-
Classifier: Operating System :: OS Independent
|
|
13
|
-
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
14
|
-
Classifier: Intended Audience :: Science/Research
|
|
15
|
-
Requires-Python: >=3.9
|
|
16
|
-
Description-Content-Type: text/markdown
|
|
17
|
-
Requires-Dist: numpy>=1.21
|
|
18
|
-
Requires-Dist: torch>=1.13
|
|
19
|
-
Requires-Dist: scikit-learn>=1.0
|
|
20
|
-
Requires-Dist: tqdm>=4.0
|
|
21
|
-
Provides-Extra: viz
|
|
22
|
-
Requires-Dist: matplotlib; extra == "viz"
|
|
23
|
-
Requires-Dist: seaborn; extra == "viz"
|
|
24
|
-
Provides-Extra: manifold
|
|
25
|
-
Requires-Dist: openTSNE; extra == "manifold"
|
|
26
|
-
Provides-Extra: nlp
|
|
27
|
-
Requires-Dist: gensim; extra == "nlp"
|
|
28
|
-
Provides-Extra: hyperbolic
|
|
29
|
-
Requires-Dist: geoopt>=0.5; extra == "hyperbolic"
|
|
30
|
-
Provides-Extra: dev
|
|
31
|
-
Requires-Dist: pytest; extra == "dev"
|
|
32
|
-
Requires-Dist: black; extra == "dev"
|
|
33
|
-
Requires-Dist: ruff; extra == "dev"
|
|
34
|
-
|
|
35
1
|
<div align="center">
|
|
36
2
|
|
|
37
3
|
# Event2Vector (event2vec)
|
|
@@ -132,7 +98,7 @@ Key methods:
|
|
|
132
98
|
- `most_similar`: gensim-style nearest-neighbor lookup over learned event embeddings using tokens or full sequences as queries.
|
|
133
99
|
- `pad_sequences=True`: enables fully vectorized batches with masking for substantial throughput gains on large corpora.
|
|
134
100
|
|
|
135
|
-
Device control: set `use_gpu=False` to force CPU even if CUDA/MPS is present, or pass an explicit `device` (e.g., `"cuda:0"` or `"cpu"`).
|
|
101
|
+
Device control: set `use_gpu=False` to force CPU even if CUDA/MPS is present, or pass an explicit `device` (e.g., `"cuda:0"` or `"cpu"`). On Apple Silicon the `rotation` and `affine` geometries default to the CPU, because MPS has no `matrix_exp` kernel and the CPU is faster for them; pass `device="mps"` to override.
|
|
136
102
|
|
|
137
103
|
## Geometries (v0.2)
|
|
138
104
|
|
|
@@ -146,6 +112,7 @@ Beyond `euclidean` and `hyperbolic`, the estimator exposes the paper's pilot geo
|
|
|
146
112
|
| `spherical` | κ-gyroaddition (`kappa>0`) | cyclic / modular (periodic) structure |
|
|
147
113
|
| `rotation` | left `SO(n)` product | non-commutative orientation (order matters) |
|
|
148
114
|
| `heisenberg` | Heisenberg group product | order-dependent signed (Lévy) area |
|
|
115
|
+
| `affine` | `x -> exp(M_e) x + v_e` on the left | events that act on the history (shrink, stretch, rotate it), not only add to it |
|
|
149
116
|
| `product` | per-factor update | two structure types at once (e.g. hyperbolic × spherical) |
|
|
150
117
|
|
|
151
118
|
```python
|
|
@@ -161,6 +128,39 @@ model = Event2Vec(num_event_types=len(vocab), geometry="rotation", rotation_n=3)
|
|
|
161
128
|
model = Event2Vec(num_event_types=len(vocab), geometry="product", product_kappas=(-1.0, 1.0))
|
|
162
129
|
```
|
|
163
130
|
|
|
131
|
+
### Affine geometry (v0.4)
|
|
132
|
+
|
|
133
|
+
In the additive geometries an event always pushes the same way. With
|
|
134
|
+
`geometry="affine"` each event is an invertible affine map
|
|
135
|
+
`x -> exp(M_e) x + v_e`, so the same event contributes differently depending on
|
|
136
|
+
what came before, while the model stays a group:
|
|
137
|
+
|
|
138
|
+
- with every `M_e = 0` (the initialisation) the point is exactly the Euclidean sum,
|
|
139
|
+
so training starts from the additive model;
|
|
140
|
+
- every step is invertible and the reconstruction loss recovers it exactly;
|
|
141
|
+
- any subsequence composes into a single affine map, so its effect on *any*
|
|
142
|
+
history is one `A x + v`.
|
|
143
|
+
|
|
144
|
+
The state is the whole map `[vec(A), b]`; the point to plot is `b`, the image of
|
|
145
|
+
the origin (`model.model.point(...)`). The singular values of a subsequence's `A`
|
|
146
|
+
say how much it overrides what came before: `1` passes the history through,
|
|
147
|
+
`< 1` erases it along that direction.
|
|
148
|
+
|
|
149
|
+
```python
|
|
150
|
+
model = Event2Vec(num_event_types=len(vocab), geometry="affine", embedding_dim=2)
|
|
151
|
+
model.fit(sequences)
|
|
152
|
+
|
|
153
|
+
states = model.transform(sequences, as_numpy=False) # [vec(A), b] per sequence
|
|
154
|
+
points = model.model.point(states) # (n, 2) points to plot
|
|
155
|
+
|
|
156
|
+
A, v = model.model.compose([3, 7, 1]) # one map for a subsequence
|
|
157
|
+
print(model.model.override_strength([3, 7, 1])) # its singular values
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
Options: `affine_readout="point"` (default; the decoder reads `b`) or `"full"`
|
|
161
|
+
(it also reads `A`), and `affine_max_log_scale` to bound how strongly a single
|
|
162
|
+
event can stretch or shrink the state.
|
|
163
|
+
|
|
164
164
|
Reproduce the curvature sweep on a real knowledge graph (auto-downloads WN18RR):
|
|
165
165
|
|
|
166
166
|
```bash
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
__version__ = "0.
|
|
1
|
+
__version__ = "0.4.0"
|
|
2
2
|
|
|
3
3
|
from .models import EuclideanModel, HyperbolicModel, HyperbolicUtils, Event2Vec
|
|
4
4
|
from .geometries import (
|
|
@@ -6,6 +6,7 @@ from .geometries import (
|
|
|
6
6
|
StereographicModel,
|
|
7
7
|
RotationModel,
|
|
8
8
|
HeisenbergModel,
|
|
9
|
+
AffineModel,
|
|
9
10
|
ProductModel,
|
|
10
11
|
)
|
|
11
12
|
from .data import generate_sequences, get_sequences
|
|
@@ -20,6 +21,7 @@ __all__ = [
|
|
|
20
21
|
"StereographicModel",
|
|
21
22
|
"RotationModel",
|
|
22
23
|
"HeisenbergModel",
|
|
24
|
+
"AffineModel",
|
|
23
25
|
"ProductModel",
|
|
24
26
|
"generate_sequences",
|
|
25
27
|
"get_sequences",
|
|
@@ -17,6 +17,10 @@ rotation (SO(n)) left matrix product non-commutative orientation;
|
|
|
17
17
|
abelianization
|
|
18
18
|
heisenberg nilpotent group product the signed (Levy) area / commutator,
|
|
19
19
|
a purely order-dependent quantity
|
|
20
|
+
affine (Aff+(d)) x -> exp(M_e) x + v_e events that act on the history
|
|
21
|
+
on the left (shrink / stretch / rotate it) as
|
|
22
|
+
well as add to it; M_e = 0 is
|
|
23
|
+
exactly the Euclidean sum
|
|
20
24
|
product per-factor update two structure types at once
|
|
21
25
|
(e.g. hierarchy H x cycle S)
|
|
22
26
|
================ =========================== ===================================
|
|
@@ -35,6 +39,22 @@ import torch.nn as nn
|
|
|
35
39
|
EPS = 1e-5
|
|
36
40
|
|
|
37
41
|
|
|
42
|
+
def _matrix_exp(a):
|
|
43
|
+
"""torch.matrix_exp that also works for MPS tensors: MPS has no matrix_exp
|
|
44
|
+
kernel (pytorch/pytorch#141287), so it is computed on the CPU and moved back.
|
|
45
|
+
Both transfers are differentiable."""
|
|
46
|
+
if a.device.type == "mps":
|
|
47
|
+
return torch.matrix_exp(a.cpu()).to(a.device)
|
|
48
|
+
return torch.matrix_exp(a)
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
def _solve(a, b):
|
|
52
|
+
"""torch.linalg.solve, computed on the CPU for MPS tensors (as above)."""
|
|
53
|
+
if a.device.type == "mps":
|
|
54
|
+
return torch.linalg.solve(a.cpu(), b.cpu()).to(a.device)
|
|
55
|
+
return torch.linalg.solve(a, b)
|
|
56
|
+
|
|
57
|
+
|
|
38
58
|
# --------------------------------------------------------------------------- #
|
|
39
59
|
# kappa-stereographic model: one signed curvature for E / S / H
|
|
40
60
|
# --------------------------------------------------------------------------- #
|
|
@@ -216,7 +236,7 @@ class RotationModel(nn.Module):
|
|
|
216
236
|
a = torch.zeros(b, self.n, self.n, device=gen.device, dtype=gen.dtype)
|
|
217
237
|
a[:, self.triu_i, self.triu_j] = gen
|
|
218
238
|
a = a - a.transpose(1, 2)
|
|
219
|
-
return
|
|
239
|
+
return _matrix_exp(a)
|
|
220
240
|
|
|
221
241
|
def zero_state(self, batch, device):
|
|
222
242
|
return torch.eye(self.n, device=device).reshape(1, -1).repeat(batch, 1)
|
|
@@ -329,6 +349,152 @@ class HeisenbergModel(nn.Module):
|
|
|
329
349
|
return ((a - b) ** 2).sum(-1, keepdim=True)
|
|
330
350
|
|
|
331
351
|
|
|
352
|
+
# --------------------------------------------------------------------------- #
|
|
353
|
+
# Affine group model: events that act on the history, not only add to it
|
|
354
|
+
# --------------------------------------------------------------------------- #
|
|
355
|
+
class AffineModel(nn.Module):
|
|
356
|
+
"""State is an orientation-preserving affine map ``x -> A x + b`` of R^d,
|
|
357
|
+
stored flat as ``[vec(A), b]`` and starting at the identity. Each event is the
|
|
358
|
+
affine map ``x -> exp(M_e) x + v_e`` applied on the left::
|
|
359
|
+
|
|
360
|
+
A_t = exp(M_e) A_{t-1} b_t = exp(M_e) b_{t-1} + v_e
|
|
361
|
+
|
|
362
|
+
``b`` is the sequence's *point* (the image of the origin). With every
|
|
363
|
+
``M_e = 0`` it is exactly the Euclidean sum of the ``v_e``, so this geometry
|
|
364
|
+
contains event2vec's additive model and starts training from it.
|
|
365
|
+
|
|
366
|
+
The linear part lets an event act on what came before -- shrink, stretch or
|
|
367
|
+
rotate it -- so the same event contributes differently after different
|
|
368
|
+
histories, while the model stays a group: every step is invertible (``exp``
|
|
369
|
+
of any matrix is), the reconstruction loss recovers it exactly, and any
|
|
370
|
+
subsequence composes into one affine map (:meth:`compose`). Its singular
|
|
371
|
+
values (:meth:`override_strength`) say how strongly that subsequence
|
|
372
|
+
overrides the history: values below 1 erase the history along a direction.
|
|
373
|
+
|
|
374
|
+
Args:
|
|
375
|
+
embedding_dim: dimension ``d`` of the point ``b`` (2 for a planar map).
|
|
376
|
+
Each event has ``d + d*d`` parameters (``v_e`` and ``M_e``).
|
|
377
|
+
readout: ``"point"`` decodes the next event from ``b`` only (the plotted
|
|
378
|
+
point); ``"full"`` also reads ``A``.
|
|
379
|
+
max_log_scale: if set, ``M_e`` entries are squashed to
|
|
380
|
+
``(-max_log_scale, max_log_scale)``, bounding how much one event can
|
|
381
|
+
stretch or shrink the state (each step's condition number is at most
|
|
382
|
+
``exp(2 * d * max_log_scale)``). ``None`` leaves ``M_e`` free.
|
|
383
|
+
"""
|
|
384
|
+
|
|
385
|
+
geometry = "affine"
|
|
386
|
+
|
|
387
|
+
def __init__(self, num_event_types, embedding_dim=2, dropout_p=0.0,
|
|
388
|
+
readout="point", max_log_scale=None):
|
|
389
|
+
super().__init__()
|
|
390
|
+
if readout not in ("point", "full"):
|
|
391
|
+
raise ValueError("readout must be 'point' or 'full'.")
|
|
392
|
+
d = embedding_dim
|
|
393
|
+
self.d = d
|
|
394
|
+
self.point_dim = d
|
|
395
|
+
self.embedding_dim = d + d * d # generator: [v_e, vec(M_e)]
|
|
396
|
+
self.state_dim = d * d + d # state: [vec(A), b]
|
|
397
|
+
self.readout = readout
|
|
398
|
+
self.max_log_scale = max_log_scale
|
|
399
|
+
self.embedding = nn.Embedding(num_event_types, self.embedding_dim)
|
|
400
|
+
self.dropout = nn.Dropout(p=dropout_p)
|
|
401
|
+
self.linear_dec = nn.Linear(d if readout == "point" else self.state_dim,
|
|
402
|
+
num_event_types)
|
|
403
|
+
self.reset_parameters()
|
|
404
|
+
|
|
405
|
+
def reset_parameters(self):
|
|
406
|
+
if self.linear_dec is not None: # None when used as a ProductModel factor
|
|
407
|
+
nn.init.xavier_uniform_(self.linear_dec.weight)
|
|
408
|
+
if self.linear_dec.bias is not None:
|
|
409
|
+
nn.init.zeros_(self.linear_dec.bias)
|
|
410
|
+
d = self.d
|
|
411
|
+
with torch.no_grad():
|
|
412
|
+
nn.init.normal_(self.embedding.weight[:, :d], 0.0, 0.02) # v_e as Euclidean
|
|
413
|
+
self.embedding.weight[:, d:].zero_() # M_e = 0: A_e = I
|
|
414
|
+
|
|
415
|
+
# -- group elements ----------------------------------------------------- #
|
|
416
|
+
def _maps(self, gen):
|
|
417
|
+
"""(B, d + d*d) generators -> (A_e (B, d, d), v_e (B, d))."""
|
|
418
|
+
d = self.d
|
|
419
|
+
v, m = gen[:, :d], gen[:, d:].reshape(-1, d, d)
|
|
420
|
+
if self.max_log_scale is not None:
|
|
421
|
+
c = self.max_log_scale
|
|
422
|
+
m = c * torch.tanh(m / c)
|
|
423
|
+
return _matrix_exp(m), v
|
|
424
|
+
|
|
425
|
+
def _split(self, h):
|
|
426
|
+
d = self.d
|
|
427
|
+
return h[:, : d * d].reshape(-1, d, d), h[:, d * d:]
|
|
428
|
+
|
|
429
|
+
@staticmethod
|
|
430
|
+
def _join(a, b):
|
|
431
|
+
return torch.cat([a.reshape(a.shape[0], -1), b], dim=-1)
|
|
432
|
+
|
|
433
|
+
def zero_state(self, batch, device):
|
|
434
|
+
a = torch.eye(self.d, device=device).expand(batch, -1, -1)
|
|
435
|
+
return self._join(a, torch.zeros(batch, self.d, device=device))
|
|
436
|
+
|
|
437
|
+
def point(self, h):
|
|
438
|
+
"""The plotted point ``b`` of a state (or of ``transform``'s output)."""
|
|
439
|
+
return h[..., self.d * self.d:]
|
|
440
|
+
|
|
441
|
+
def to_tangent(self, h):
|
|
442
|
+
return self.point(h) if self.readout == "point" else h
|
|
443
|
+
|
|
444
|
+
# -- recurrence ---------------------------------------------------------- #
|
|
445
|
+
def encoder(self, x, h):
|
|
446
|
+
a_e, v_e = self._maps(self.dropout(self.embedding(x)))
|
|
447
|
+
a_old, b_old = self._split(h)
|
|
448
|
+
a_new = torch.bmm(a_e, a_old)
|
|
449
|
+
b_new = torch.bmm(a_e, b_old.unsqueeze(-1)).squeeze(-1) + v_e
|
|
450
|
+
return self._join(a_new, b_new), self._join(a_e, v_e)
|
|
451
|
+
|
|
452
|
+
def decoder(self, h):
|
|
453
|
+
if self.linear_dec is None:
|
|
454
|
+
raise RuntimeError(
|
|
455
|
+
"this model is a ProductModel factor; decode through the "
|
|
456
|
+
"product's shared decoder, not the factor's")
|
|
457
|
+
return self.linear_dec(self.to_tangent(h))
|
|
458
|
+
|
|
459
|
+
def forward(self, x, h=None):
|
|
460
|
+
if h is None:
|
|
461
|
+
h = self.zero_state(x.shape[0], x.device)
|
|
462
|
+
h_next, e = self.encoder(x, h)
|
|
463
|
+
return self.decoder(h_next), h_next, e
|
|
464
|
+
|
|
465
|
+
def recover_increment(self, h_old, h_new):
|
|
466
|
+
"""g_e = g_new . g_old^-1: A_e = A_new A_old^-1, v_e = b_new - A_e b_old."""
|
|
467
|
+
a_old, b_old = self._split(h_old)
|
|
468
|
+
a_new, b_new = self._split(h_new)
|
|
469
|
+
# A_e A_old = A_new <=> A_old^T A_e^T = A_new^T (solve, don't invert)
|
|
470
|
+
a_e = _solve(a_old.transpose(1, 2), a_new.transpose(1, 2)).transpose(1, 2)
|
|
471
|
+
v_e = b_new - torch.bmm(a_e, b_old.unsqueeze(-1)).squeeze(-1)
|
|
472
|
+
return self._join(a_e, v_e)
|
|
473
|
+
|
|
474
|
+
def dist2(self, a, b):
|
|
475
|
+
return ((a - b) ** 2).sum(-1, keepdim=True) # Frobenius^2 on [A, b]
|
|
476
|
+
|
|
477
|
+
# -- interpretation -------------------------------------------------------- #
|
|
478
|
+
@torch.no_grad()
|
|
479
|
+
def compose(self, event_ids):
|
|
480
|
+
"""One affine map (A, v) for a whole subsequence: applying it to any state
|
|
481
|
+
equals feeding the events one by one (exact; it is a group product)."""
|
|
482
|
+
dev = self.embedding.weight.device
|
|
483
|
+
a, v = torch.eye(self.d, device=dev), torch.zeros(self.d, device=dev)
|
|
484
|
+
for i in event_ids:
|
|
485
|
+
a_e, v_e = self._maps(self.embedding.weight[int(i)].unsqueeze(0))
|
|
486
|
+
a, v = a_e[0] @ a, a_e[0] @ v + v_e[0]
|
|
487
|
+
return a, v
|
|
488
|
+
|
|
489
|
+
@torch.no_grad()
|
|
490
|
+
def override_strength(self, event_ids):
|
|
491
|
+
"""Singular values of the subsequence's linear part, largest first:
|
|
492
|
+
1 = the history passes through unchanged, < 1 = erased along that
|
|
493
|
+
direction, > 1 = amplified."""
|
|
494
|
+
a, _ = self.compose(event_ids)
|
|
495
|
+
return torch.linalg.svdvals(a)
|
|
496
|
+
|
|
497
|
+
|
|
332
498
|
# --------------------------------------------------------------------------- #
|
|
333
499
|
# Product manifold: two structure types at once (e.g. hierarchy H x cycle S)
|
|
334
500
|
# --------------------------------------------------------------------------- #
|
|
@@ -8,6 +8,7 @@ import torch.nn as nn
|
|
|
8
8
|
from torch.nn.utils.rnn import pad_sequence
|
|
9
9
|
|
|
10
10
|
from .geometries import (
|
|
11
|
+
AffineModel,
|
|
11
12
|
HeisenbergModel,
|
|
12
13
|
ProductModel,
|
|
13
14
|
RotationModel,
|
|
@@ -196,6 +197,8 @@ class Event2Vec:
|
|
|
196
197
|
kappa: Optional[float] = None,
|
|
197
198
|
rotation_n: int = 3,
|
|
198
199
|
heisenberg_m: int = 1,
|
|
200
|
+
affine_readout: str = "point",
|
|
201
|
+
affine_max_log_scale: Optional[float] = None,
|
|
199
202
|
product_kappas: Sequence[float] = (-1.0, 1.0),
|
|
200
203
|
lambda_reconstruction: float = 0.2,
|
|
201
204
|
lambda_consistency: float = 0.2,
|
|
@@ -232,6 +235,8 @@ class Event2Vec:
|
|
|
232
235
|
self.kappa = kappa
|
|
233
236
|
self.rotation_n = rotation_n
|
|
234
237
|
self.heisenberg_m = heisenberg_m
|
|
238
|
+
self.affine_readout = affine_readout
|
|
239
|
+
self.affine_max_log_scale = affine_max_log_scale
|
|
235
240
|
self.product_kappas = tuple(product_kappas)
|
|
236
241
|
self.lambda_reconstruction = lambda_reconstruction
|
|
237
242
|
self.lambda_consistency = lambda_consistency
|
|
@@ -249,7 +254,18 @@ class Event2Vec:
|
|
|
249
254
|
if use_gpu and torch.cuda.is_available():
|
|
250
255
|
device = torch.device("cuda")
|
|
251
256
|
elif use_gpu and hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
|
|
252
|
-
|
|
257
|
+
if self.geometry in ("rotation", "affine"):
|
|
258
|
+
# MPS has no matrix_exp, so every step would round-trip through
|
|
259
|
+
# the CPU; for a model this small the CPU is much faster.
|
|
260
|
+
warnings.warn(
|
|
261
|
+
f"geometry='{self.geometry}' needs torch.matrix_exp, which MPS does not "
|
|
262
|
+
"implement; using device='cpu'. Pass device='mps' to run on MPS "
|
|
263
|
+
"anyway (matrix_exp is then computed on the CPU).",
|
|
264
|
+
RuntimeWarning,
|
|
265
|
+
)
|
|
266
|
+
device = torch.device("cpu")
|
|
267
|
+
else:
|
|
268
|
+
device = torch.device("mps")
|
|
253
269
|
else:
|
|
254
270
|
device = torch.device("cpu")
|
|
255
271
|
else:
|
|
@@ -307,6 +323,14 @@ class Event2Vec:
|
|
|
307
323
|
m=self.heisenberg_m,
|
|
308
324
|
dropout_p=self.dropout_p,
|
|
309
325
|
)
|
|
326
|
+
elif self.geometry == "affine":
|
|
327
|
+
model = AffineModel(
|
|
328
|
+
num_event_types=self.num_event_types,
|
|
329
|
+
embedding_dim=self.embedding_dim,
|
|
330
|
+
dropout_p=self.dropout_p,
|
|
331
|
+
readout=self.affine_readout,
|
|
332
|
+
max_log_scale=self.affine_max_log_scale,
|
|
333
|
+
)
|
|
310
334
|
elif self.geometry == "product":
|
|
311
335
|
factors = [
|
|
312
336
|
StereographicModel(
|
|
@@ -321,7 +345,7 @@ class Event2Vec:
|
|
|
321
345
|
else:
|
|
322
346
|
raise ValueError(
|
|
323
347
|
"geometry must be one of: 'euclidean', 'hyperbolic', 'spherical', "
|
|
324
|
-
"'stereographic', 'rotation', 'heisenberg', 'product'."
|
|
348
|
+
"'stereographic', 'rotation', 'heisenberg', 'affine', 'product'."
|
|
325
349
|
)
|
|
326
350
|
return model.to(self.device)
|
|
327
351
|
|
|
@@ -1,3 +1,38 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: event2vector
|
|
3
|
+
Version: 0.4.0
|
|
4
|
+
Summary: Scikit-learn-style geometric embeddings for event sequences.
|
|
5
|
+
Author: Antonin Sulc
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/sulcantonin/event2vec_public
|
|
8
|
+
Project-URL: Repository, https://github.com/sulcantonin/event2vec_public
|
|
9
|
+
Project-URL: Issues, https://github.com/sulcantonin/event2vec_public/issues
|
|
10
|
+
Project-URL: Changelog, https://github.com/sulcantonin/event2vec_public/blob/main/CHANGELOG.md
|
|
11
|
+
Keywords: sequence-embedding,event-sequences,representation-learning,word2vec,scikit-learn,time-series
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Operating System :: OS Independent
|
|
14
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
15
|
+
Classifier: Intended Audience :: Science/Research
|
|
16
|
+
Requires-Python: >=3.9
|
|
17
|
+
Description-Content-Type: text/markdown
|
|
18
|
+
Requires-Dist: numpy>=1.21
|
|
19
|
+
Requires-Dist: torch>=1.13
|
|
20
|
+
Requires-Dist: scikit-learn>=1.0
|
|
21
|
+
Requires-Dist: tqdm>=4.0
|
|
22
|
+
Provides-Extra: viz
|
|
23
|
+
Requires-Dist: matplotlib; extra == "viz"
|
|
24
|
+
Requires-Dist: seaborn; extra == "viz"
|
|
25
|
+
Provides-Extra: manifold
|
|
26
|
+
Requires-Dist: openTSNE; extra == "manifold"
|
|
27
|
+
Provides-Extra: nlp
|
|
28
|
+
Requires-Dist: gensim; extra == "nlp"
|
|
29
|
+
Provides-Extra: hyperbolic
|
|
30
|
+
Requires-Dist: geoopt>=0.5; extra == "hyperbolic"
|
|
31
|
+
Provides-Extra: dev
|
|
32
|
+
Requires-Dist: pytest; extra == "dev"
|
|
33
|
+
Requires-Dist: black; extra == "dev"
|
|
34
|
+
Requires-Dist: ruff; extra == "dev"
|
|
35
|
+
|
|
1
36
|
<div align="center">
|
|
2
37
|
|
|
3
38
|
# Event2Vector (event2vec)
|
|
@@ -98,7 +133,7 @@ Key methods:
|
|
|
98
133
|
- `most_similar`: gensim-style nearest-neighbor lookup over learned event embeddings using tokens or full sequences as queries.
|
|
99
134
|
- `pad_sequences=True`: enables fully vectorized batches with masking for substantial throughput gains on large corpora.
|
|
100
135
|
|
|
101
|
-
Device control: set `use_gpu=False` to force CPU even if CUDA/MPS is present, or pass an explicit `device` (e.g., `"cuda:0"` or `"cpu"`).
|
|
136
|
+
Device control: set `use_gpu=False` to force CPU even if CUDA/MPS is present, or pass an explicit `device` (e.g., `"cuda:0"` or `"cpu"`). On Apple Silicon the `rotation` and `affine` geometries default to the CPU, because MPS has no `matrix_exp` kernel and the CPU is faster for them; pass `device="mps"` to override.
|
|
102
137
|
|
|
103
138
|
## Geometries (v0.2)
|
|
104
139
|
|
|
@@ -112,6 +147,7 @@ Beyond `euclidean` and `hyperbolic`, the estimator exposes the paper's pilot geo
|
|
|
112
147
|
| `spherical` | κ-gyroaddition (`kappa>0`) | cyclic / modular (periodic) structure |
|
|
113
148
|
| `rotation` | left `SO(n)` product | non-commutative orientation (order matters) |
|
|
114
149
|
| `heisenberg` | Heisenberg group product | order-dependent signed (Lévy) area |
|
|
150
|
+
| `affine` | `x -> exp(M_e) x + v_e` on the left | events that act on the history (shrink, stretch, rotate it), not only add to it |
|
|
115
151
|
| `product` | per-factor update | two structure types at once (e.g. hyperbolic × spherical) |
|
|
116
152
|
|
|
117
153
|
```python
|
|
@@ -127,6 +163,39 @@ model = Event2Vec(num_event_types=len(vocab), geometry="rotation", rotation_n=3)
|
|
|
127
163
|
model = Event2Vec(num_event_types=len(vocab), geometry="product", product_kappas=(-1.0, 1.0))
|
|
128
164
|
```
|
|
129
165
|
|
|
166
|
+
### Affine geometry (v0.4)
|
|
167
|
+
|
|
168
|
+
In the additive geometries an event always pushes the same way. With
|
|
169
|
+
`geometry="affine"` each event is an invertible affine map
|
|
170
|
+
`x -> exp(M_e) x + v_e`, so the same event contributes differently depending on
|
|
171
|
+
what came before, while the model stays a group:
|
|
172
|
+
|
|
173
|
+
- with every `M_e = 0` (the initialisation) the point is exactly the Euclidean sum,
|
|
174
|
+
so training starts from the additive model;
|
|
175
|
+
- every step is invertible and the reconstruction loss recovers it exactly;
|
|
176
|
+
- any subsequence composes into a single affine map, so its effect on *any*
|
|
177
|
+
history is one `A x + v`.
|
|
178
|
+
|
|
179
|
+
The state is the whole map `[vec(A), b]`; the point to plot is `b`, the image of
|
|
180
|
+
the origin (`model.model.point(...)`). The singular values of a subsequence's `A`
|
|
181
|
+
say how much it overrides what came before: `1` passes the history through,
|
|
182
|
+
`< 1` erases it along that direction.
|
|
183
|
+
|
|
184
|
+
```python
|
|
185
|
+
model = Event2Vec(num_event_types=len(vocab), geometry="affine", embedding_dim=2)
|
|
186
|
+
model.fit(sequences)
|
|
187
|
+
|
|
188
|
+
states = model.transform(sequences, as_numpy=False) # [vec(A), b] per sequence
|
|
189
|
+
points = model.model.point(states) # (n, 2) points to plot
|
|
190
|
+
|
|
191
|
+
A, v = model.model.compose([3, 7, 1]) # one map for a subsequence
|
|
192
|
+
print(model.model.override_strength([3, 7, 1])) # its singular values
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Options: `affine_readout="point"` (default; the decoder reads `b`) or `"full"`
|
|
196
|
+
(it also reads `A`), and `affine_max_log_scale` to bound how strongly a single
|
|
197
|
+
event can stretch or shrink the state.
|
|
198
|
+
|
|
130
199
|
Reproduce the curvature sweep on a real knowledge graph (auto-downloads WN18RR):
|
|
131
200
|
|
|
132
201
|
```bash
|
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
CHANGELOG.md
|
|
2
|
+
CITATION.cff
|
|
1
3
|
MANIFEST.in
|
|
2
4
|
README.md
|
|
3
5
|
pyproject.toml
|
|
@@ -22,5 +24,5 @@ event2vector/datasets/realdata.py
|
|
|
22
24
|
event2vector/eval/__init__.py
|
|
23
25
|
event2vector/eval/figures.py
|
|
24
26
|
event2vector/eval/harness.py
|
|
25
|
-
tests/
|
|
26
|
-
tests/
|
|
27
|
+
tests/test_affine.py
|
|
28
|
+
tests/test_device.py
|