highwater 0.0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,217 @@
1
+ Metadata-Version: 2.4
2
+ Name: highwater
3
+ Version: 0.0.1
4
+ Summary: Durable execution for streaming applications
5
+ License-Expression: Apache-2.0
6
+ Project-URL: Homepage, https://highwater.cloud
7
+ Project-URL: Documentation, https://highwater.cloud
8
+ Project-URL: Repository, https://github.com/henneberger/highwater
9
+ Requires-Python: >=3.11
10
+ Description-Content-Type: text/markdown
11
+
12
+ # Highwater
13
+
14
+ Durable execution for streaming applications.
15
+
16
+ Highwater lets you write stateful stream processing as ordinary Python. It keeps each key ordered, persists every accepted event and state transition, tracks event-time progress, retries failed invocations, and scales execution with load.
17
+
18
+ ```python
19
+ from dataclasses import dataclass
20
+ from highwater import process
21
+
22
+ @dataclass(frozen=True)
23
+ class Deposit:
24
+ account_id: str
25
+ amount: int
26
+
27
+ @process.defn(key="account_id")
28
+ @dataclass
29
+ class Balance:
30
+ total: int = 0
31
+
32
+ @process.event
33
+ async def apply(self, event: Deposit):
34
+ self.total += event.amount
35
+ return {"account_id": event.account_id, "balance": self.total}
36
+ ```
37
+
38
+ No topology builder. No separate state database. No recovery code in your application.
39
+
40
+ ## Install Highwater
41
+
42
+ ```bash
43
+ pip install highwater
44
+ ```
45
+
46
+ ## Run locally
47
+
48
+ ```bash
49
+ highwater dev app.py
50
+ ```
51
+
52
+ `highwater dev` starts a complete local environment, discovers the Processes in `app.py`, and prints the local ingestion endpoint. Storage, partitions, leases, and execution pools use development defaults.
53
+
54
+ Send an event from Python:
55
+
56
+ ```python
57
+ from highwater import Client
58
+
59
+ client = Client()
60
+ balances = client.process("Balance")
61
+ await balances.send(
62
+ Deposit("account-a", 5),
63
+ event_id="deposit-1001",
64
+ )
65
+ ```
66
+
67
+ Or send JSON to the generated event endpoint:
68
+
69
+ ```bash
70
+ curl -X POST http://localhost:7233/v1/processes/Balance/events \
71
+ -H 'content-type: application/json' \
72
+ -H 'idempotency-key: deposit-1001' \
73
+ -d '{"account_id":"account-a","amount":5}'
74
+ ```
75
+
76
+ ## Deploy
77
+
78
+ ```bash
79
+ highwater deploy app.py
80
+ ```
81
+
82
+ Highwater packages the application, creates a versioned deployment, provisions event ingestion, and scales execution independently for each state partition. The same Process API runs continuously, on demand, or on a schedule.
83
+
84
+ ```bash
85
+ highwater deploy app.py --schedule '0 * * * *'
86
+ ```
87
+
88
+ A schedule controls when compute drains available events. It does not turn off ingestion or weaken durability.
89
+
90
+ ## Why Highwater
91
+
92
+ Traditional stream processors are good at dataflow graphs. Durable execution systems are good at long-running application code. Highwater combines their strongest ideas around one abstraction: a durable Process keyed by the entity your application already understands.
93
+
94
+ ```text
95
+ events ──► durable inbox ──► Python Process ──► state + output
96
+ per key retryable atomic
97
+ ```
98
+
99
+ - One key runs one state transition at a time.
100
+ - Different keys scale independently.
101
+ - Accepted events survive executor and host failures.
102
+ - State and output commit together.
103
+ - Stable event identifiers make uncertain retries safe.
104
+ - Watermarks let code wait for event-time completeness.
105
+ - Backpressure reaches ingestion before queues become unbounded.
106
+
107
+ ## Streaming that can be batchy
108
+
109
+ Highwater continuously batches transport and durable commits. Application code stays event-oriented unless it opts into vectorized execution:
110
+
111
+ ```python
112
+ @process.defn(key="document_id")
113
+ class Embeddings:
114
+ @process.batch(max_size=128, max_delay=0.025)
115
+ async def embed(self, documents: list[Document]):
116
+ vectors = await model.embed([doc.text for doc in documents])
117
+ return [
118
+ {"document_id": doc.document_id, "embedding": vector}
119
+ for doc, vector in zip(documents, vectors, strict=True)
120
+ ]
121
+ ```
122
+
123
+ The batch runs when it reaches 128 documents or its oldest document waits 25 milliseconds. Scheduled deployments use the same mechanism to drain finite bursts and scale back to zero.
124
+
125
+ ## Event time without a dataflow language
126
+
127
+ ```python
128
+ @process.defn(
129
+ key="account_id",
130
+ event_time="occurred_at",
131
+ wait_until=process.complete,
132
+ )
133
+ @dataclass
134
+ class DailyBalance:
135
+ total: int = 0
136
+ ```
137
+
138
+ `process.complete` runs an event only after Highwater knows the input is complete through that timestamp. The platform owns source progress, idleness, late-data policy, and watermark coordination.
139
+
140
+ Highwater also provides native incremental filters, windows, deduplication, interval joins, and temporal as-of joins. Use them for common state machines and keep application-specific decisions in Python.
141
+
142
+ ## Event ingestion
143
+
144
+ Every deployment receives managed HTTPS and SDK ingestion. Highwater assigns durable source positions, validates idempotency keys, partitions by Process key, and applies admission backpressure.
145
+
146
+ Customers publish events to Highwater. Connectors, brokers, storage tiers, and partition movement are platform concerns rather than application configuration.
147
+
148
+ ## Execution model
149
+
150
+ Highwater groups keyed Processes into movable partitions. Each partition pipelines and group-commits state transitions, keeps hot state close to execution, and snapshots durable progress asynchronously. Execution containers are cached according to observed traffic and can scale to zero when idle.
151
+
152
+ Cross-partition messages carry causal commit dependencies. A receiver can begin speculative work, but it cannot commit a result before the sender's dependency is durable. This avoids a distributed transaction on every message while preserving recovery order.
153
+
154
+ Leases control where an invocation may run. A lease token is a renewable capability tied to a durable partition generation. Expiration only makes a lease eligible for revocation; a durable generation change fences the old executor. Late completions from a prior generation cannot commit.
155
+
156
+ These choices follow the partitioned, pipelined execution model described by Microsoft Research's [Netherite](https://www.microsoft.com/en-us/research/publication/netherite-efficient-execution-of-serverless-workflows/) and the workload-aware warm execution findings from [Serverless in the Wild](https://www.microsoft.com/en-us/research/publication/serverless-in-the-wild-characterizing-and-optimizing-the-serverless-workload-at-a-large-cloud-provider/).
157
+
158
+ ## Delivery guarantees
159
+
160
+ | Boundary | Guarantee |
161
+ | --- | --- |
162
+ | Event admission | acknowledged after a durable append |
163
+ | Per-key execution | ordered, one committed transition at a time |
164
+ | Invocation | at least once across failures |
165
+ | State and output | atomic within one Process transition |
166
+ | Event retry | idempotent with a stable event identifier |
167
+ | Output delivery | at least once with a stable message identifier |
168
+ | Event-time progress | monotonic per input partition |
169
+
170
+ Direct, non-idempotent external side effects can still occur more than once when an invocation fails after the effect. Use a destination idempotency key or Highwater's transactional output delivery.
171
+
172
+ ## Performance
173
+
174
+ The partition execution path completes 100,000 distinct durable keyed transitions at a median **99,951 events per second** with five execution instances on one development machine. The ordinary per-event handler reaches a median **89,029 events per second**. Every admission and completion is acknowledged only after its authoritative WAL append.
175
+
176
+ Execution instances receive disjoint partition sets, so application compute can run on separate hosts without coordinating each event. Durable owner epochs and activation sequences fence delayed work after a service restart. Moving the partition state-machine owners themselves between service hosts remains part of the multi-host durability work described in [Scaling architecture](docs/SCALING.md).
177
+
178
+ See [Performance](docs/PERFORMANCE.md) for the reproducible benchmark, scaling results, and measurement boundary. One hot key remains serial by design; split the key or use a commutative aggregation when one entity needs internal parallelism.
179
+
180
+ ## Documentation
181
+
182
+ The Docusaurus site lives in [`website`](website/README.md).
183
+
184
+ ```bash
185
+ cd website
186
+ yarn install
187
+ yarn start
188
+ ```
189
+
190
+ The public documentation covers durable Processes, managed event ingestion, event time, batching, scheduled deployments, joins, scaling, backpressure, upgrades, recovery, delivery guarantees, and lease fencing.
191
+
192
+ The standalone product landing page lives in [`landing`](landing/README.md). Its static files can be previewed directly and deployed to an S3 origin without an application server.
193
+
194
+ ## Repository
195
+
196
+ The repository contains the execution engine, Python SDK, examples, benchmarks, and documentation source. Internal crate and module names remain implementation details behind the packaged CLI.
197
+
198
+ ```text
199
+ crates/ execution and protocol implementation
200
+ src/temporal_code/ Python SDK implementation
201
+ examples/ streaming applications
202
+ benchmarks/ durable throughput benchmark
203
+ docs/ implementation design notes
204
+ website/ public documentation
205
+ landing/ product landing page
206
+ ```
207
+
208
+ ## Build the implementation
209
+
210
+ Contributors working on the engine can build and test from source. End users install `highwater` and use `highwater dev`; these commands are not part of the customer setup path.
211
+
212
+ ```bash
213
+ cargo test --workspace
214
+ cargo clippy --workspace --all-targets -- -D warnings
215
+ PYTHONPATH=src python3 -m unittest discover -s tests -v
216
+ cd website && yarn build
217
+ ```
@@ -0,0 +1,206 @@
1
+ # Highwater
2
+
3
+ Durable execution for streaming applications.
4
+
5
+ Highwater lets you write stateful stream processing as ordinary Python. It keeps each key ordered, persists every accepted event and state transition, tracks event-time progress, retries failed invocations, and scales execution with load.
6
+
7
+ ```python
8
+ from dataclasses import dataclass
9
+ from highwater import process
10
+
11
+ @dataclass(frozen=True)
12
+ class Deposit:
13
+ account_id: str
14
+ amount: int
15
+
16
+ @process.defn(key="account_id")
17
+ @dataclass
18
+ class Balance:
19
+ total: int = 0
20
+
21
+ @process.event
22
+ async def apply(self, event: Deposit):
23
+ self.total += event.amount
24
+ return {"account_id": event.account_id, "balance": self.total}
25
+ ```
26
+
27
+ No topology builder. No separate state database. No recovery code in your application.
28
+
29
+ ## Install Highwater
30
+
31
+ ```bash
32
+ pip install highwater
33
+ ```
34
+
35
+ ## Run locally
36
+
37
+ ```bash
38
+ highwater dev app.py
39
+ ```
40
+
41
+ `highwater dev` starts a complete local environment, discovers the Processes in `app.py`, and prints the local ingestion endpoint. Storage, partitions, leases, and execution pools use development defaults.
42
+
43
+ Send an event from Python:
44
+
45
+ ```python
46
+ from highwater import Client
47
+
48
+ client = Client()
49
+ balances = client.process("Balance")
50
+ await balances.send(
51
+ Deposit("account-a", 5),
52
+ event_id="deposit-1001",
53
+ )
54
+ ```
55
+
56
+ Or send JSON to the generated event endpoint:
57
+
58
+ ```bash
59
+ curl -X POST http://localhost:7233/v1/processes/Balance/events \
60
+ -H 'content-type: application/json' \
61
+ -H 'idempotency-key: deposit-1001' \
62
+ -d '{"account_id":"account-a","amount":5}'
63
+ ```
64
+
65
+ ## Deploy
66
+
67
+ ```bash
68
+ highwater deploy app.py
69
+ ```
70
+
71
+ Highwater packages the application, creates a versioned deployment, provisions event ingestion, and scales execution independently for each state partition. The same Process API runs continuously, on demand, or on a schedule.
72
+
73
+ ```bash
74
+ highwater deploy app.py --schedule '0 * * * *'
75
+ ```
76
+
77
+ A schedule controls when compute drains available events. It does not turn off ingestion or weaken durability.
78
+
79
+ ## Why Highwater
80
+
81
+ Traditional stream processors are good at dataflow graphs. Durable execution systems are good at long-running application code. Highwater combines their strongest ideas around one abstraction: a durable Process keyed by the entity your application already understands.
82
+
83
+ ```text
84
+ events ──► durable inbox ──► Python Process ──► state + output
85
+ per key retryable atomic
86
+ ```
87
+
88
+ - One key runs one state transition at a time.
89
+ - Different keys scale independently.
90
+ - Accepted events survive executor and host failures.
91
+ - State and output commit together.
92
+ - Stable event identifiers make uncertain retries safe.
93
+ - Watermarks let code wait for event-time completeness.
94
+ - Backpressure reaches ingestion before queues become unbounded.
95
+
96
+ ## Streaming that can be batchy
97
+
98
+ Highwater continuously batches transport and durable commits. Application code stays event-oriented unless it opts into vectorized execution:
99
+
100
+ ```python
101
+ @process.defn(key="document_id")
102
+ class Embeddings:
103
+ @process.batch(max_size=128, max_delay=0.025)
104
+ async def embed(self, documents: list[Document]):
105
+ vectors = await model.embed([doc.text for doc in documents])
106
+ return [
107
+ {"document_id": doc.document_id, "embedding": vector}
108
+ for doc, vector in zip(documents, vectors, strict=True)
109
+ ]
110
+ ```
111
+
112
+ The batch runs when it reaches 128 documents or its oldest document waits 25 milliseconds. Scheduled deployments use the same mechanism to drain finite bursts and scale back to zero.
113
+
114
+ ## Event time without a dataflow language
115
+
116
+ ```python
117
+ @process.defn(
118
+ key="account_id",
119
+ event_time="occurred_at",
120
+ wait_until=process.complete,
121
+ )
122
+ @dataclass
123
+ class DailyBalance:
124
+ total: int = 0
125
+ ```
126
+
127
+ `process.complete` runs an event only after Highwater knows the input is complete through that timestamp. The platform owns source progress, idleness, late-data policy, and watermark coordination.
128
+
129
+ Highwater also provides native incremental filters, windows, deduplication, interval joins, and temporal as-of joins. Use them for common state machines and keep application-specific decisions in Python.
130
+
131
+ ## Event ingestion
132
+
133
+ Every deployment receives managed HTTPS and SDK ingestion. Highwater assigns durable source positions, validates idempotency keys, partitions by Process key, and applies admission backpressure.
134
+
135
+ Customers publish events to Highwater. Connectors, brokers, storage tiers, and partition movement are platform concerns rather than application configuration.
136
+
137
+ ## Execution model
138
+
139
+ Highwater groups keyed Processes into movable partitions. Each partition pipelines and group-commits state transitions, keeps hot state close to execution, and snapshots durable progress asynchronously. Execution containers are cached according to observed traffic and can scale to zero when idle.
140
+
141
+ Cross-partition messages carry causal commit dependencies. A receiver can begin speculative work, but it cannot commit a result before the sender's dependency is durable. This avoids a distributed transaction on every message while preserving recovery order.
142
+
143
+ Leases control where an invocation may run. A lease token is a renewable capability tied to a durable partition generation. Expiration only makes a lease eligible for revocation; a durable generation change fences the old executor. Late completions from a prior generation cannot commit.
144
+
145
+ These choices follow the partitioned, pipelined execution model described by Microsoft Research's [Netherite](https://www.microsoft.com/en-us/research/publication/netherite-efficient-execution-of-serverless-workflows/) and the workload-aware warm execution findings from [Serverless in the Wild](https://www.microsoft.com/en-us/research/publication/serverless-in-the-wild-characterizing-and-optimizing-the-serverless-workload-at-a-large-cloud-provider/).
146
+
147
+ ## Delivery guarantees
148
+
149
+ | Boundary | Guarantee |
150
+ | --- | --- |
151
+ | Event admission | acknowledged after a durable append |
152
+ | Per-key execution | ordered, one committed transition at a time |
153
+ | Invocation | at least once across failures |
154
+ | State and output | atomic within one Process transition |
155
+ | Event retry | idempotent with a stable event identifier |
156
+ | Output delivery | at least once with a stable message identifier |
157
+ | Event-time progress | monotonic per input partition |
158
+
159
+ Direct, non-idempotent external side effects can still occur more than once when an invocation fails after the effect. Use a destination idempotency key or Highwater's transactional output delivery.
160
+
161
+ ## Performance
162
+
163
+ The partition execution path completes 100,000 distinct durable keyed transitions at a median **99,951 events per second** with five execution instances on one development machine. The ordinary per-event handler reaches a median **89,029 events per second**. Every admission and completion is acknowledged only after its authoritative WAL append.
164
+
165
+ Execution instances receive disjoint partition sets, so application compute can run on separate hosts without coordinating each event. Durable owner epochs and activation sequences fence delayed work after a service restart. Moving the partition state-machine owners themselves between service hosts remains part of the multi-host durability work described in [Scaling architecture](docs/SCALING.md).
166
+
167
+ See [Performance](docs/PERFORMANCE.md) for the reproducible benchmark, scaling results, and measurement boundary. One hot key remains serial by design; split the key or use a commutative aggregation when one entity needs internal parallelism.
168
+
169
+ ## Documentation
170
+
171
+ The Docusaurus site lives in [`website`](website/README.md).
172
+
173
+ ```bash
174
+ cd website
175
+ yarn install
176
+ yarn start
177
+ ```
178
+
179
+ The public documentation covers durable Processes, managed event ingestion, event time, batching, scheduled deployments, joins, scaling, backpressure, upgrades, recovery, delivery guarantees, and lease fencing.
180
+
181
+ The standalone product landing page lives in [`landing`](landing/README.md). Its static files can be previewed directly and deployed to an S3 origin without an application server.
182
+
183
+ ## Repository
184
+
185
+ The repository contains the execution engine, Python SDK, examples, benchmarks, and documentation source. Internal crate and module names remain implementation details behind the packaged CLI.
186
+
187
+ ```text
188
+ crates/ execution and protocol implementation
189
+ src/temporal_code/ Python SDK implementation
190
+ examples/ streaming applications
191
+ benchmarks/ durable throughput benchmark
192
+ docs/ implementation design notes
193
+ website/ public documentation
194
+ landing/ product landing page
195
+ ```
196
+
197
+ ## Build the implementation
198
+
199
+ Contributors working on the engine can build and test from source. End users install `highwater` and use `highwater dev`; these commands are not part of the customer setup path.
200
+
201
+ ```bash
202
+ cargo test --workspace
203
+ cargo clippy --workspace --all-targets -- -D warnings
204
+ PYTHONPATH=src python3 -m unittest discover -s tests -v
205
+ cd website && yarn build
206
+ ```
@@ -0,0 +1,28 @@
1
+ [build-system]
2
+ requires = ["setuptools>=68"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "highwater"
7
+ version = "0.0.1"
8
+ description = "Durable execution for streaming applications"
9
+ readme = "README.md"
10
+ license = "Apache-2.0"
11
+ requires-python = ">=3.11"
12
+ dependencies = []
13
+
14
+ [project.urls]
15
+ Homepage = "https://highwater.cloud"
16
+ Documentation = "https://highwater.cloud"
17
+ Repository = "https://github.com/henneberger/highwater"
18
+
19
+ [project.scripts]
20
+ highwater-worker = "temporal_code.rust_worker:main"
21
+ temporal-code-worker = "temporal_code.rust_worker:main"
22
+
23
+ [tool.setuptools.packages.find]
24
+ where = ["src"]
25
+
26
+ [tool.pytest.ini_options]
27
+ pythonpath = ["src"]
28
+ testpaths = ["tests"]
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,2 @@
1
+ from temporal_code import *
2
+ from temporal_code import __all__
@@ -0,0 +1,217 @@
1
+ Metadata-Version: 2.4
2
+ Name: highwater
3
+ Version: 0.0.1
4
+ Summary: Durable execution for streaming applications
5
+ License-Expression: Apache-2.0
6
+ Project-URL: Homepage, https://highwater.cloud
7
+ Project-URL: Documentation, https://highwater.cloud
8
+ Project-URL: Repository, https://github.com/henneberger/highwater
9
+ Requires-Python: >=3.11
10
+ Description-Content-Type: text/markdown
11
+
12
+ # Highwater
13
+
14
+ Durable execution for streaming applications.
15
+
16
+ Highwater lets you write stateful stream processing as ordinary Python. It keeps each key ordered, persists every accepted event and state transition, tracks event-time progress, retries failed invocations, and scales execution with load.
17
+
18
+ ```python
19
+ from dataclasses import dataclass
20
+ from highwater import process
21
+
22
+ @dataclass(frozen=True)
23
+ class Deposit:
24
+ account_id: str
25
+ amount: int
26
+
27
+ @process.defn(key="account_id")
28
+ @dataclass
29
+ class Balance:
30
+ total: int = 0
31
+
32
+ @process.event
33
+ async def apply(self, event: Deposit):
34
+ self.total += event.amount
35
+ return {"account_id": event.account_id, "balance": self.total}
36
+ ```
37
+
38
+ No topology builder. No separate state database. No recovery code in your application.
39
+
40
+ ## Install Highwater
41
+
42
+ ```bash
43
+ pip install highwater
44
+ ```
45
+
46
+ ## Run locally
47
+
48
+ ```bash
49
+ highwater dev app.py
50
+ ```
51
+
52
+ `highwater dev` starts a complete local environment, discovers the Processes in `app.py`, and prints the local ingestion endpoint. Storage, partitions, leases, and execution pools use development defaults.
53
+
54
+ Send an event from Python:
55
+
56
+ ```python
57
+ from highwater import Client
58
+
59
+ client = Client()
60
+ balances = client.process("Balance")
61
+ await balances.send(
62
+ Deposit("account-a", 5),
63
+ event_id="deposit-1001",
64
+ )
65
+ ```
66
+
67
+ Or send JSON to the generated event endpoint:
68
+
69
+ ```bash
70
+ curl -X POST http://localhost:7233/v1/processes/Balance/events \
71
+ -H 'content-type: application/json' \
72
+ -H 'idempotency-key: deposit-1001' \
73
+ -d '{"account_id":"account-a","amount":5}'
74
+ ```
75
+
76
+ ## Deploy
77
+
78
+ ```bash
79
+ highwater deploy app.py
80
+ ```
81
+
82
+ Highwater packages the application, creates a versioned deployment, provisions event ingestion, and scales execution independently for each state partition. The same Process API runs continuously, on demand, or on a schedule.
83
+
84
+ ```bash
85
+ highwater deploy app.py --schedule '0 * * * *'
86
+ ```
87
+
88
+ A schedule controls when compute drains available events. It does not turn off ingestion or weaken durability.
89
+
90
+ ## Why Highwater
91
+
92
+ Traditional stream processors are good at dataflow graphs. Durable execution systems are good at long-running application code. Highwater combines their strongest ideas around one abstraction: a durable Process keyed by the entity your application already understands.
93
+
94
+ ```text
95
+ events ──► durable inbox ──► Python Process ──► state + output
96
+ per key retryable atomic
97
+ ```
98
+
99
+ - One key runs one state transition at a time.
100
+ - Different keys scale independently.
101
+ - Accepted events survive executor and host failures.
102
+ - State and output commit together.
103
+ - Stable event identifiers make uncertain retries safe.
104
+ - Watermarks let code wait for event-time completeness.
105
+ - Backpressure reaches ingestion before queues become unbounded.
106
+
107
+ ## Streaming that can be batchy
108
+
109
+ Highwater continuously batches transport and durable commits. Application code stays event-oriented unless it opts into vectorized execution:
110
+
111
+ ```python
112
+ @process.defn(key="document_id")
113
+ class Embeddings:
114
+ @process.batch(max_size=128, max_delay=0.025)
115
+ async def embed(self, documents: list[Document]):
116
+ vectors = await model.embed([doc.text for doc in documents])
117
+ return [
118
+ {"document_id": doc.document_id, "embedding": vector}
119
+ for doc, vector in zip(documents, vectors, strict=True)
120
+ ]
121
+ ```
122
+
123
+ The batch runs when it reaches 128 documents or its oldest document waits 25 milliseconds. Scheduled deployments use the same mechanism to drain finite bursts and scale back to zero.
124
+
125
+ ## Event time without a dataflow language
126
+
127
+ ```python
128
+ @process.defn(
129
+ key="account_id",
130
+ event_time="occurred_at",
131
+ wait_until=process.complete,
132
+ )
133
+ @dataclass
134
+ class DailyBalance:
135
+ total: int = 0
136
+ ```
137
+
138
+ `process.complete` runs an event only after Highwater knows the input is complete through that timestamp. The platform owns source progress, idleness, late-data policy, and watermark coordination.
139
+
140
+ Highwater also provides native incremental filters, windows, deduplication, interval joins, and temporal as-of joins. Use them for common state machines and keep application-specific decisions in Python.
141
+
142
+ ## Event ingestion
143
+
144
+ Every deployment receives managed HTTPS and SDK ingestion. Highwater assigns durable source positions, validates idempotency keys, partitions by Process key, and applies admission backpressure.
145
+
146
+ Customers publish events to Highwater. Connectors, brokers, storage tiers, and partition movement are platform concerns rather than application configuration.
147
+
148
+ ## Execution model
149
+
150
+ Highwater groups keyed Processes into movable partitions. Each partition pipelines and group-commits state transitions, keeps hot state close to execution, and snapshots durable progress asynchronously. Execution containers are cached according to observed traffic and can scale to zero when idle.
151
+
152
+ Cross-partition messages carry causal commit dependencies. A receiver can begin speculative work, but it cannot commit a result before the sender's dependency is durable. This avoids a distributed transaction on every message while preserving recovery order.
153
+
154
+ Leases control where an invocation may run. A lease token is a renewable capability tied to a durable partition generation. Expiration only makes a lease eligible for revocation; a durable generation change fences the old executor. Late completions from a prior generation cannot commit.
155
+
156
+ These choices follow the partitioned, pipelined execution model described by Microsoft Research's [Netherite](https://www.microsoft.com/en-us/research/publication/netherite-efficient-execution-of-serverless-workflows/) and the workload-aware warm execution findings from [Serverless in the Wild](https://www.microsoft.com/en-us/research/publication/serverless-in-the-wild-characterizing-and-optimizing-the-serverless-workload-at-a-large-cloud-provider/).
157
+
158
+ ## Delivery guarantees
159
+
160
+ | Boundary | Guarantee |
161
+ | --- | --- |
162
+ | Event admission | acknowledged after a durable append |
163
+ | Per-key execution | ordered, one committed transition at a time |
164
+ | Invocation | at least once across failures |
165
+ | State and output | atomic within one Process transition |
166
+ | Event retry | idempotent with a stable event identifier |
167
+ | Output delivery | at least once with a stable message identifier |
168
+ | Event-time progress | monotonic per input partition |
169
+
170
+ Direct, non-idempotent external side effects can still occur more than once when an invocation fails after the effect. Use a destination idempotency key or Highwater's transactional output delivery.
171
+
172
+ ## Performance
173
+
174
+ The partition execution path completes 100,000 distinct durable keyed transitions at a median **99,951 events per second** with five execution instances on one development machine. The ordinary per-event handler reaches a median **89,029 events per second**. Every admission and completion is acknowledged only after its authoritative WAL append.
175
+
176
+ Execution instances receive disjoint partition sets, so application compute can run on separate hosts without coordinating each event. Durable owner epochs and activation sequences fence delayed work after a service restart. Moving the partition state-machine owners themselves between service hosts remains part of the multi-host durability work described in [Scaling architecture](docs/SCALING.md).
177
+
178
+ See [Performance](docs/PERFORMANCE.md) for the reproducible benchmark, scaling results, and measurement boundary. One hot key remains serial by design; split the key or use a commutative aggregation when one entity needs internal parallelism.
179
+
180
+ ## Documentation
181
+
182
+ The Docusaurus site lives in [`website`](website/README.md).
183
+
184
+ ```bash
185
+ cd website
186
+ yarn install
187
+ yarn start
188
+ ```
189
+
190
+ The public documentation covers durable Processes, managed event ingestion, event time, batching, scheduled deployments, joins, scaling, backpressure, upgrades, recovery, delivery guarantees, and lease fencing.
191
+
192
+ The standalone product landing page lives in [`landing`](landing/README.md). Its static files can be previewed directly and deployed to an S3 origin without an application server.
193
+
194
+ ## Repository
195
+
196
+ The repository contains the execution engine, Python SDK, examples, benchmarks, and documentation source. Internal crate and module names remain implementation details behind the packaged CLI.
197
+
198
+ ```text
199
+ crates/ execution and protocol implementation
200
+ src/temporal_code/ Python SDK implementation
201
+ examples/ streaming applications
202
+ benchmarks/ durable throughput benchmark
203
+ docs/ implementation design notes
204
+ website/ public documentation
205
+ landing/ product landing page
206
+ ```
207
+
208
+ ## Build the implementation
209
+
210
+ Contributors working on the engine can build and test from source. End users install `highwater` and use `highwater dev`; these commands are not part of the customer setup path.
211
+
212
+ ```bash
213
+ cargo test --workspace
214
+ cargo clippy --workspace --all-targets -- -D warnings
215
+ PYTHONPATH=src python3 -m unittest discover -s tests -v
216
+ cd website && yarn build
217
+ ```