picopyn 1.0.0__tar.gz → 2.0.0.dev1__tar.gz

{picopyn-1.0.0/picopyn.egg-info → picopyn-2.0.0.dev1}/PKG-INFO

@@ -1,16 +1,16 @@
 Metadata-Version: 2.4
 Name: picopyn
-Version: 1.0.0
+Version: 2.0.0.dev1
 Summary: Python connector for working with the distributed Picodata database.
-Author-email: Asya Lomakina <a.lomakina@picodata.io>, Igor Kuznetsov <i.kuznetsov@picodata.io>, Dmitriy Кoltsov <dkoltsov@picodata.io>, Daniil Gorlyakov <d.gorlyakov@picodata.io>
+Author-email: Asya Lomakina <a.lomakina@picodata.io>, Igor Kuznetsov <i.kuznetsov@picodata.io>, Dmitriy Koltsov <dkoltsov@picodata.io>, Daniil Gorlyakov <d.gorlyakov@picodata.io>
 License-Expression: BSD-2-Clause
-Project-URL: Source, https://git.picodata.io/solution/picopyn
+Project-URL: Source, https://git.picodata.io/core/drivers/picopyn
 Project-URL: Documentation, https://picopyn.readthedocs.io/en/stable/
 Project-URL: Changelog, https://git.picodata.io/core/drivers/picopyn/-/blob/main/CHANGELOG.md
 Keywords: database,picodata
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Operating System :: OS Independent
-Classifier: Development Status :: 3 - Alpha
+Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Topic :: Database
 Classifier: Framework :: AsyncIO
@@ -18,10 +18,11 @@ Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
 License-File: AUTHORS.md
-Requires-Dist: async-timeout==5.0.1
 Requires-Dist: asyncpg==0.30.0
-Requires-Dist: typing_extensions==4.12.2
+Requires-Dist: typing_extensions<5,>=4.12.2
 Requires-Dist: psycopg[binary]==3.3.4
+Requires-Dist: msgpack<2,>=1.0.0
+Requires-Dist: mmh3<6,>=5.2.1
 Provides-Extra: test
 Requires-Dist: pytest==8.3.5; extra == "test"
 Requires-Dist: pytest-asyncio==0.25.3; extra == "test"
@@ -29,7 +30,7 @@ Requires-Dist: ruff==0.11.13; extra == "test"
 Requires-Dist: mypy==1.16.0; extra == "test"
 Requires-Dist: black==25.1.0; extra == "test"
 Provides-Extra: docs
-Requires-Dist: mkdocs<2.0,>=1.0; extra == "docs"
+Requires-Dist: mkdocs<2.0,>=1.6; extra == "docs"
 Requires-Dist: mkdocs-material; extra == "docs"
 Requires-Dist: mkdocstrings[python]; extra == "docs"
 Provides-Extra: benchmark
@@ -49,16 +50,22 @@ Dynamic: license-file
 **Picopyn** is a Python package for working with the distributed [Picodata](https://picodata.io/en/) database.
 
 **Version compatibility**:
-| Picodata version | Status |
-| ---------------- | ------ |
-| >=25.4.x, < 26.2.x | ✅ Fully supported |
-| 26.2.x | ⚠️ Not yet tested |
+| Picodata version  | Picopyn version   |
+| ----------------- | ----------------- |
+| >=25.2.1, <25.4.4 | 0.1.1             |
+| >=25.4.4, <25.5.1 | 0.2.0             |
+| >=25.5.1, <26.1.x | 1.0.0             |
+| >=26.1.1, <26.2.x | 2.0.0             |
+| 26.2.x            | ⚠️ Unreleased yet |
 
 ## Features
 * Connection pooling with configurable pool size
 * Optional automatic node discovery
 * Pluggable load-balancing strategies
-* Provides both asynchronous API (based on [asyncpg](https://github.com/MagicStack/asyncpg)) and synchronous [DPAPI-compatible](https://peps.python.org/pep-0249/) API (based on [psycopg](https://www.psycopg.org/))
+* Asynchronous API based on [asyncpg](https://github.com/MagicStack/asyncpg)
+    * (async only) Shard-aware routing
+    * (async only) Automatic pool membership reconciliation
+* Synchronous [DPAPI-compatible](https://peps.python.org/pep-0249/) API based on [psycopg](https://www.psycopg.org/)
 
 ## Navigation
 * [Installation](#installation)
@@ -79,7 +86,7 @@ pip install picopyn
 Or from source:
 
 ```sh
-git clone https://git.picodata.io/solution/picopyn.git
+git clone https://git.picodata.io/core/drivers/picopyn.git
 cd picopyn
 pip install -e .
 ```
@@ -118,20 +125,18 @@ Sync:
 from picopyn.synchronous import connect
 
 # create and connect to the picodata cluster
-conn = connect("postgresql://admin:pass@localhost:5432")
-cur = conn.cursor()
+with connect("postgresql://admin:pass@localhost:5432") as conn:
+    cur = conn.cursor()
 
-# execute DDL operations
-cur.execute('''
-    CREATE TABLE "warehouse" (id INTEGER NOT NULL, item TEXT NOT NULL, PRIMARY KEY (id)) USING memtx DISTRIBUTED BY (id) OPTION (TIMEOUT = 3.0);
-''')
-
-# execute DML/DQL operations
-cur.execute('INSERT INTO "warehouse" VALUES (%s, %s)', (1, "test"))
-cur.execute('SELECT * FROM "warehouse"')
-print(cur.fetchall())
+    # execute DDL operations
+    cur.execute('''
+        CREATE TABLE "warehouse" (id INTEGER NOT NULL, item TEXT NOT NULL, PRIMARY KEY (id)) USING memtx DISTRIBUTED BY (id) OPTION (TIMEOUT = 3.0);
+    ''')
 
-conn.close()
+    # execute DML/DQL operations
+    cur.execute('INSERT INTO "warehouse" VALUES (%s, %s)', (1, "test"))
+    cur.execute('SELECT * FROM "warehouse"')
+    print(cur.fetchall())
 ```
 
 ## Development

{picopyn-1.0.0 → picopyn-2.0.0.dev1}/README.md

@@ -7,16 +7,22 @@
 **Picopyn** is a Python package for working with the distributed [Picodata](https://picodata.io/en/) database.
 
 **Version compatibility**:
-| Picodata version | Status |
-| ---------------- | ------ |
-| >=25.4.x, < 26.2.x | ✅ Fully supported |
-| 26.2.x | ⚠️ Not yet tested |
+| Picodata version  | Picopyn version   |
+| ----------------- | ----------------- |
+| >=25.2.1, <25.4.4 | 0.1.1             |
+| >=25.4.4, <25.5.1 | 0.2.0             |
+| >=25.5.1, <26.1.x | 1.0.0             |
+| >=26.1.1, <26.2.x | 2.0.0             |
+| 26.2.x            | ⚠️ Unreleased yet |
 
 ## Features
 * Connection pooling with configurable pool size
 * Optional automatic node discovery
 * Pluggable load-balancing strategies
-* Provides both asynchronous API (based on [asyncpg](https://github.com/MagicStack/asyncpg)) and synchronous [DPAPI-compatible](https://peps.python.org/pep-0249/) API (based on [psycopg](https://www.psycopg.org/))
+* Asynchronous API based on [asyncpg](https://github.com/MagicStack/asyncpg)
+    * (async only) Shard-aware routing
+    * (async only) Automatic pool membership reconciliation
+* Synchronous [DPAPI-compatible](https://peps.python.org/pep-0249/) API based on [psycopg](https://www.psycopg.org/)
 
 ## Navigation
 * [Installation](#installation)
@@ -37,7 +43,7 @@ pip install picopyn
 Or from source:
 
 ```sh
-git clone https://git.picodata.io/solution/picopyn.git
+git clone https://git.picodata.io/core/drivers/picopyn.git
 cd picopyn
 pip install -e .
 ```
@@ -76,20 +82,18 @@ Sync:
 from picopyn.synchronous import connect
 
 # create and connect to the picodata cluster
-conn = connect("postgresql://admin:pass@localhost:5432")
-cur = conn.cursor()
+with connect("postgresql://admin:pass@localhost:5432") as conn:
+    cur = conn.cursor()
 
-# execute DDL operations
-cur.execute('''
-    CREATE TABLE "warehouse" (id INTEGER NOT NULL, item TEXT NOT NULL, PRIMARY KEY (id)) USING memtx DISTRIBUTED BY (id) OPTION (TIMEOUT = 3.0);
-''')
-
-# execute DML/DQL operations
-cur.execute('INSERT INTO "warehouse" VALUES (%s, %s)', (1, "test"))
-cur.execute('SELECT * FROM "warehouse"')
-print(cur.fetchall())
+    # execute DDL operations
+    cur.execute('''
+        CREATE TABLE "warehouse" (id INTEGER NOT NULL, item TEXT NOT NULL, PRIMARY KEY (id)) USING memtx DISTRIBUTED BY (id) OPTION (TIMEOUT = 3.0);
+    ''')
 
-conn.close()
+    # execute DML/DQL operations
+    cur.execute('INSERT INTO "warehouse" VALUES (%s, %s)', (1, "test"))
+    cur.execute('SELECT * FROM "warehouse"')
+    print(cur.fetchall())
 ```
 
 ## Development

picopyn-2.0.0.dev1/docs/async.md

@@ -0,0 +1,6 @@
+# Async
+
+::: picopyn.asynchronous
+    options:
+      filters:
+        - "!^_"

picopyn-2.0.0.dev1/docs/async_internal.md

@@ -0,0 +1,18 @@
+# Async: Internal
+
+!!! warning
+    Not public API. Driver wires this up and owns it; nothing here is meant
+    to be constructed or called directly. Documented anyway so you can see
+    how it actually works
+
+::: picopyn.asynchronous.topology_tracker
+    options:
+      show_root_heading: true
+      filters:
+        - "!^_"
+
+::: picopyn.asynchronous.metadata_service
+    options:
+      show_root_heading: true
+      filters:
+        - "!^_"

picopyn-2.0.0.dev1/docs/bucket.md

@@ -0,0 +1,7 @@
+# Bucket ID
+
+::: picopyn.bucket
+    options:
+      filters:
+        - "!^_"
+        - "^__init__$"

picopyn-2.0.0.dev1/docs/changelog.md

@@ -0,0 +1,3 @@
+# Changelog
+
+--8<-- "CHANGELOG.md"

{picopyn-1.0.0 → picopyn-2.0.0.dev1}/docs/contributing.md

@@ -2,6 +2,6 @@
 
 Contributions are welcome! The project is hosted on GitLab:
 
-**[git.picodata.io/solution/picopyn](https://git.picodata.io/solution/picopyn)**
+**[git.picodata.io/core/drivers/picopyn](https://git.picodata.io/core/drivers/picopyn)**
 
-To get started with the development environment, running tests, and code style requirements, see the [Development](https://git.picodata.io/solution/picopyn#development) section in the project README.
+To get started with the development environment, running tests, and code style requirements, see the [Development](https://git.picodata.io/core/drivers/picopyn#development) section in the project README.

picopyn-2.0.0.dev1/docs/diagrams/_pool_init.md

@@ -0,0 +1,64 @@
+```mermaid
+---
+title: Pool initialization
+---
+
+flowchart TD
+%% === Initialization ===
+classDef success fill:#e8f5e9,stroke:#4caf50,color:#1b5e20
+classDef error fill:#ffebee,stroke:#f44336,color:#b71c1c
+classDef neutral fill:#f5f5f5,stroke:#9e9e9e,color:#424242
+classDef note fill:#fffde7,stroke:#f9a825,stroke-dasharray:4,color:#555
+
+START([Start]):::neutral
+DONE([Pool ready]):::success
+ERROR([Initialization failed]):::error
+CFG[/"Pool settings<br>· DSN string<br>· pool size<br>· forbidden tiers<br>· cluster discovery mode"/]
+
+USAGE_DETAILS_NOTE>"Return to pool lifecycle details"]:::note
+click USAGE_DETAILS_NOTE "#picopyn.asynchronous.Pool" "Return to pool lifecycle details"
+
+%% === Data objects ===
+POOL[[Connection pool]]
+B_DSN[[DSN node list]]
+B_SYS[(Picodata system tables)]
+D_DSN[[DSN node list]]
+D_SYS[(Picodata system tables)]
+CAND[[Candidate node list]]
+
+%% === Actions ===
+
+START --> CFG --> PARSE[Parse DSN string to DSN node list] --> DISCOVERY{Cluster discovery enabled?}
+
+DISCOVERY -->|yes| D_FILTER[Use DSN nodes to connect to<br>Picodata cluster and retrieve<br>online nodes with allowed tiers<br>as a candidate node list]
+DISCOVERY -->|no| B_FILTER[Filter DSN nodes: use only online<br>nodes with allowed tiers]
+
+subgraph DiscoverMode["Cluster discovery mode"]
+    D_FILTER --> D_FILL_POOL[Try to fill pool with connections<br>to candidates up to pool size]
+
+    D_DETAILS_NOTE>"Discovery mode details"]:::note
+    click D_DETAILS_NOTE "#pool-init-discovery-enabled" "Discovery mode details"
+
+    D_FILTER <-. write .-> CAND
+    CAND <-. read .-> D_FILL_POOL
+end
+
+subgraph BootstrapMode["Bootstrap mode"]
+    B_FILTER --> B_FILL_POOL[Try to fill pool with connections<br>to DSN nodes up to pool size]
+
+    B_DETAILS_NOTE>"Bootstrap mode details"]:::note
+    click B_DETAILS_NOTE "#pool-init-discovery-disabled" "Bootstrap mode details"
+end
+
+B_DSN <-. read .-> B_FILTER
+B_SYS <-. read .-> B_FILTER
+B_FILL_POOL <-. write .-> POOL
+
+D_DSN <-. read .-> D_FILTER
+D_SYS <-. read .-> D_FILTER
+D_FILL_POOL <-. write .-> POOL
+
+POOL <-.-> POOL_CHECK{Pool full?}
+POOL_CHECK -->|no| ERROR
+POOL_CHECK -->|yes| DONE
+```

picopyn-2.0.0.dev1/docs/diagrams/_pool_init_discovery_disabled.md

@@ -0,0 +1,46 @@
+```mermaid
+---
+title: Pool initialization with cluster discovery disabled
+---
+
+flowchart TD
+
+%% === Initialization ===
+classDef success fill:#e8f5e9,stroke:#4caf50,color:#1b5e20
+classDef error fill:#ffebee,stroke:#f44336,color:#b71c1c
+classDef neutral fill:#f5f5f5,stroke:#9e9e9e,color:#424242
+classDef note fill:#fffde7,stroke:#f9a825,stroke-dasharray:4,color:#555
+
+START([Start]):::neutral
+DONE([Pool ready]):::success
+ERROR([Initialization failed]):::error
+CFG[/"Pool settings<br>· DSN string<br>· pool size<br>· forbidden tiers<br>· cluster discovery disabled"/]
+
+GENERAL_DETAILS_NOTE>"Return to general algorithm"]:::note
+click GENERAL_DETAILS_NOTE "#picopyn.asynchronous.Pool.connect" "Return to general algorithm"
+
+%% === Data objects ===
+B_DSN[[DSN node list]]
+BOOTSTRAP_POOL[[Connection pool]]
+SYS[(Picodata system tables)]
+
+%% === Bootstrap mode ===
+START --> CFG --> PARSE[Parse DSN string to DSN node list] --> B_PoolFilling
+PARSE <-. write .-> B_DSN
+B_DSN <-. read .-> B_CONNECT_NODE[Connect to DSN node]
+
+B_PoolFilling_NOTE>"iterate over DSN nodes until either:<br>- the list is empty<br>- pool_size connections<br>have been established"]:::note
+B_PoolFilling_NOTE -.-> B_PoolFilling
+subgraph B_PoolFilling["Pool filling"]
+    direction TB
+    SYS <-. read .-> B_CHECK_NODE_TIER[Check node tier]
+    B_CONNECT_NODE --> B_CHECK_NODE_TIER
+    B_CHECK_NODE_TIER -->|node offline or tier forbidden| B_EXCLUDE[Exclude DSN node]
+    B_CHECK_NODE_TIER -->|node online and tier allowed| B_SAVE[Save connection to pool]
+    B_SAVE <-. write .-> BOOTSTRAP_POOL
+end
+
+B_PoolFilling --> POOL_CHECK{Pool full?}
+POOL_CHECK -->|no| ERROR
+POOL_CHECK -->|yes| DONE
+```

picopyn-2.0.0.dev1/docs/diagrams/_pool_init_discovery_enabled.md

@@ -0,0 +1,55 @@
+```mermaid
+---
+title: Pool initialization with cluster discovery enabled
+---
+
+flowchart TD
+
+%% === Initialization ===
+classDef success fill:#e8f5e9,stroke:#4caf50,color:#1b5e20
+classDef error fill:#ffebee,stroke:#f44336,color:#b71c1c
+classDef neutral fill:#f5f5f5,stroke:#9e9e9e,color:#424242
+classDef note fill:#fffde7,stroke:#f9a825,stroke-dasharray:4,color:#555
+
+START([Start]):::neutral
+DONE([Pool ready]):::success
+D_ERROR_CAND([Initialization failed]):::error
+ERROR([Initialization failed]):::error
+CFG[/"Pool settings<br>· DSN string<br>· pool_size<br>· forbidden tiers<br>· cluster discovery enabled"/]
+
+GENERAL_DETAILS_NOTE>"Return to general algorithm"]:::note
+click GENERAL_DETAILS_NOTE "#picopyn.asynchronous.Pool.connect" "Return to general algorithm"
+
+%% === Data objects ===
+D_DSN[[DSN node list]]
+CAND[[Candidate node list]]
+DISCOVERY_POOL[[Connection pool]]
+SYS[(Picodata system tables)]
+
+%% === Discovery mode ===
+START --> CFG --> PARSE[Parse DSN string to<br>DSN node list] --> DiscoverNodes
+PARSE <-. write .-> D_DSN
+
+D_DiscoverNodes_NOTE>"iterate over DSN nodes<br>to establish a temporary<br>connection and retrieve the<br>list of online nodes with<br>allowed tiers"]:::note
+D_DiscoverNodes_NOTE -.-> DiscoverNodes
+D_DSN <-. read .-> D_RECEIVE[Try to connect and receive cluster nodes]
+subgraph DiscoverNodes["Discover cluster nodes"]
+    SYS <-. read .-> D_RECEIVE
+    D_RECEIVE -->|could not select candidates| D_ERROR_CAND
+    D_RECEIVE <-. write .-> CAND
+end
+DiscoverNodes --> D_PoolFilling
+
+D_PoolFilling_NOTE>"iterate over candidate nodes<br>until either:<br>- the list is empty<br>- pool_size connections<br>have been established"]:::note
+D_PoolFilling_NOTE -.-> D_PoolFilling
+CAND <-. read .-> D_CONNECT_NODE[Connect to candidate node]
+subgraph D_PoolFilling["Pool filling"]
+    D_CONNECT_NODE -->|connection error| D_EXCLUDE[Exclude candidate node]
+    D_CONNECT_NODE -->|connected| D_SAVE[Save opened connection to pool]
+    D_SAVE <-. write .-> DISCOVERY_POOL
+end
+
+D_PoolFilling --> POOL_CHECK{Pool full?}
+POOL_CHECK -->|no| ERROR
+POOL_CHECK -->|yes| DONE
+```

picopyn-2.0.0.dev1/docs/diagrams/_pool_lifecycle.md

@@ -0,0 +1,44 @@
+```mermaid
+---
+title: Pool lifecycle
+---
+
+flowchart TD
+%% === Style ===
+classDef neutral fill:#f5f5f5,stroke:#9e9e9e,color:#424242
+classDef note fill:#fffde7,stroke:#f9a825,stroke-dasharray:4,color:#555
+
+%% === Data objects ===
+POOL[[Connection pool]]
+
+%% === Lifecycle ===
+START([Start]):::neutral
+DONE([Pool closed]):::neutral
+
+CFG[/"Pool settings<br>· DSN<br>· pool size<br>· forbidden tiers<br>· cluster discovery mode"/]
+
+START --> CFG --> CREATE["Pool.__init__"] --> CONNECT["pool.connect()"] --> UsePool
+CONNECT <-. write .-> POOL
+
+CONNECT_DETAILS_NOTE>"Connect algorithm details"]:::note
+click CONNECT_DETAILS_NOTE "#picopyn.asynchronous.Pool.connect" "Connect algorithm details"
+CONNECT_DETAILS_NOTE -.-> CONNECT
+
+RECONCILE_NOTE>"(discovery mode only):<br>pool membership changes<br>in the background:<br>connections are opened/closed<br>to match topology changes"]:::note
+click RECONCILE_NOTE "#picopyn.asynchronous.Pool.topology" "Reconcile algorithm details"
+RECONCILE_NOTE -.-> POOL
+
+CYCLE_NOTE>"repeated for each<br>database operation"]:::note
+CYCLE_NOTE -.-> UsePool
+
+subgraph UsePool["Pool usage"]
+    direction TB
+    ACQUIRE["pool.acquire()"] --> USE["Execute queries<br>· pool.fetchrow()<br>· pool.fetch()<br>· pool.execute()"] --> RELEASE["pool.release(conn)"]
+end
+
+ACQUIRE <-. delete .-> POOL
+RELEASE <-. write .-> POOL
+CLOSE <-. delete .-> POOL
+UsePool --> CLOSE
+CLOSE --> DONE
+```

picopyn-2.0.0.dev1/docs/diagrams/_query_metadata_pool_init.md

@@ -0,0 +1,55 @@
+```mermaid
+---
+title: Query metadata -- pool initialization in cache meta context
+---
+
+flowchart TD
+classDef neutral fill:#f5f5f5,stroke:#9e9e9e,color:#424242
+classDef note fill:#fffde7,stroke:#f9a825,stroke-dasharray:4,color:#555
+
+
+INIT(["Pool init"]):::neutral
+
+INIT --> CACHE_EMPTY["Create empty query meta cache"] --> CONNECT["Fill the pool with ordinary<br>connections to picodata"] --> Pool1
+
+NOTE_CONN>"connection options require<br>statement invalidation signal only"]:::note
+NOTE_CONN -.-> CONNECT
+
+CONNECT_DETAILS_NOTE>"Connect algorithm details"]:::note
+click CONNECT_DETAILS_NOTE "#picopyn.asynchronous.Pool.connect" "Connect algorithm details"
+CONNECT_DETAILS_NOTE -.-> CONNECT
+
+subgraph Pool1["Pool"]
+    direction TB
+
+    CONN11["conn to inst1"] --> PICO11[(Picodata)]
+    CONN12["..."] --> PICO12[(Picodata)]
+    CONN1N["conn to instN"] --> PICO1N[(Picodata)]
+
+    CACHE1[/"Query meta cache"/]
+end
+
+Pool1 --> CONNECT_SERV["create service connection<br>for meta listener"]
+
+NOTE_CONN_SERV>"connection options require statement invalidation signal<br>and distribution key metadata"]:::note
+NOTE_CONN_SERV -.-> CONNECT_SERV
+
+CONNECT_SERV --> LIST["add picodata notice listener"]
+
+LIST --> Pool
+
+subgraph Pool["Pool"]
+    direction TB
+
+    CONN1["conn1 for<br>user usage"] --> PICO1[(Picodata1)]
+    CONN2["..."] --> PICO2[(...)]
+    CONNN["connN for<br>user usage"] --> PICON[(PicodataN)]
+
+    PIC(["can be any of"]) -.- CONNSERV["service connection with Notice listener"]
+    PICO1 -.- PIC
+    PICO2 -.- PIC
+    PICON -.- PIC
+
+    CACHE[/"Query meta cache"/]
+end
+```

picopyn-2.0.0.dev1/docs/diagrams/_query_metadata_usage_lookup.md

@@ -0,0 +1,44 @@
+```mermaid
+---
+title: Query metadata -- usage lookup
+---
+
+flowchart TD
+
+classDef neutral fill:#f5f5f5,stroke:#9e9e9e,color:#424242
+classDef note fill:#fffde7,stroke:#f9a825,stroke-dasharray:4,color:#555
+
+
+QUERY(["parametrized query"]):::neutral --> CALL(["Pool.execute"]) --> HASH
+
+HASH["hash the query text"] --> SEARCH_CACHE{"cached meta by hash?"}
+
+CACHE1[/"Pool query meta cache"/] -. read .- SEARCH_CACHE
+SEARCH_CACHE -->|yes| USE_CACHE["metadata found in cache"]
+SEARCH_CACHE -->|no| CACHE_MISS["no metadata yet --<br>schedule background prepare"] --> ACQUIRE_CONN
+
+subgraph Routing["BUCKET ROUTING"]
+    direction TB
+
+    CALC["calculate `bucket_id`<br>from dk_meta"]
+
+    CALC --> MAP_CONN["find bucket master<br>for this tier"]
+
+    MAP_CONN -->|routed| CALL_THE_CONN["run the query on<br>the chosen connection"]
+    MAP_CONN -->|cannot route| ACQUIRE_CONN
+
+    CALL_THE_CONN --> RESULT(["give result to the user"]):::neutral
+end
+
+USE_CACHE --> CALC
+
+subgraph Fallback["FALLBACK"]
+    direction TB
+
+    ACQUIRE_CONN["acquire pool connection<br>via the configured balance strategy"]
+
+    ACQUIRE_CONN --> CALL_CONN["run the query on<br>the chosen connection"]
+
+    CALL_CONN --> RESULT2(["give result to the user"]):::neutral
+end
+```

picopyn-2.0.0.dev1/docs/diagrams/_reconcile_node_outside_pool_died.md

@@ -0,0 +1,50 @@
+```mermaid
+---
+title: A node outside the pool died
+---
+
+flowchart TD
+
+classDef offline fill:#ffebee,stroke:#f44336,color:#b71c1c
+classDef online fill:#e8f5e9,stroke:#4caf50,color:#1b5e20
+classDef neutral fill:#f5f5f5,stroke:#9e9e9e,color:#424242
+
+classDef note fill:#fffde7,stroke:#f9a825,stroke-dasharray:4,color:#555
+
+subgraph before["Before"]
+    subgraph Picodata_before["Picodata"]
+        p1[("ins 1")]:::online
+        p2[("ins 2")]:::online
+        p3[("ins 3")]:::online
+        p4[("ins 4")]:::online
+    end
+
+    subgraph Pool_before["Pool: max_size=3"]
+        c1["conn"] --> p1
+        c2["conn"] --> p2
+        c3["conn"] --> p3
+    end
+end
+EVENT1[/"Event<br>Instance 4 dead"/]:::neutral -.-> REFILL{{"reconcile pool"}}
+before -.-> REFILL
+
+NOTE>"Node outside the pool died<br>=> pool stays unchanged"]:::note
+NOTE-.->REFILL
+
+REFILL --> after
+
+subgraph after["After"]
+    subgraph Picodata_after["Picodata"]
+        p21[("ins 1")]:::online
+        p22[("ins 2")]:::online
+        p23[("ins 3")]:::online
+        p24[("ins 4")]:::offline
+    end
+
+    subgraph Pool_after["Pool: max_size=3"]
+        c21["conn"] --> p21
+        c22["conn"] --> p22
+        c23["conn"] --> p23
+    end
+end
+```

picopyn-2.0.0.dev1/docs/diagrams/_reconcile_pool_with_topology.md

@@ -0,0 +1,46 @@
+```mermaid
+---
+title: Reconcile pool according to topology
+---
+flowchart TD
+
+classDef neutral fill:#f5f5f5,stroke:#9e9e9e,color:#424242
+classDef note fill:#fffde7,stroke:#f9a825,stroke-dasharray:4,color:#555
+classDef success fill:#e8f5e9,stroke:#4caf50,color:#1b5e20
+classDef error fill:#ffebee,stroke:#f44336,color:#b71c1c
+
+subgraph Pool["Pool"]
+    Settings[/"forbidden tiers=[...]<br>max size=N"/]
+    FreeConns["free conns=[...]"]
+    UsedConns["used conns=[...]"]
+    Topology[/"Topology"/]
+end
+
+TOPOLOGY_APPLY["Apply change to topology"] -.-> Topology
+
+EVENT_REMOVE[/"Event<br>delete instance A_uuid"/]:::neutral -.-> TOPOLOGY_APPLY
+EVENT_REPLACE[/"Event<br>replace instance B_uuid"/]:::neutral -.-> TOPOLOGY_APPLY
+
+TOPOLOGY_DETAILS_NOTE>"Topology details"]:::note
+click TOPOLOGY_DETAILS_NOTE "../topology/#picopyn.topology.Topology" "Topology details"
+TOPOLOGY_DETAILS_NOTE -.-> TOPOLOGY_APPLY
+
+TOPOLOGY_APPLY --> GET_LIST["build candidate_list from Topology:<br>- uuids with allowed tiers<br>- online uuids"]
+GET_LIST --> REMOVE["close all connections whose<br>uuid not in candidate_list"] --> CHECK_SIZE_START
+
+REMOVE -.-> FreeConns
+REMOVE -.-> UsedConns
+
+CHECK_SIZE_START{"current pool size < max size?"}
+CHECK_SIZE_START -->|no| POOL_OK(["Pool okay<br>(exit)"]):::success
+CHECK_SIZE_START -->|yes| MIN["choose uuid from candidate_list<br>with min connections count"]
+
+MIN -->|got candidate| ADD_CONN["try to connect to candidate<br>and add connection to pool..."]
+MIN -->|no candidate| NO_CANDIDATE2(["pool is temporarily not<br>full until next event<br>(exit)"]):::error
+
+ADD_CONN -->|connection failed| EXCLUDE["exclude the candidate uuid<br/>from list in this loop"] --> CHECK_SIZE_LOOP
+ADD_CONN-->|connected| CHECK_SIZE_LOOP{"current size < max size?"}
+
+CHECK_SIZE_LOOP -->|no| POOL_OK2(["Pool okay<br>(exit)"]):::success
+CHECK_SIZE_LOOP -->|yes| MIN
+```