qql-cli 1.2.0__tar.gz → 1.4.0__tar.gz

{qql_cli-1.2.0 → qql_cli-1.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: qql-cli
-Version: 1.2.0
+Version: 1.4.0
 Summary: A SQL-like query language CLI wrapper for Qdrant vector database
 Project-URL: Homepage, https://github.com/pavanjava/qql
 Project-URL: Repository, https://github.com/pavanjava/qql
@@ -83,7 +83,9 @@ qql> SEARCH notes SIMILAR TO 'vector databases' LIMIT 5 USING HYBRID RERANK
 - [The QQL Shell](#the-qql-shell)
 - [All QQL Operations](#all-qql-operations)
   - [INSERT — add a point](#insert--add-a-point)
+  - [INSERT BULK — batch insert](#insert-bulk--batch-insert-multiple-points)
   - [SEARCH — find similar points](#search--find-similar-points)
+  - [RECOMMEND — retrieve by example IDs](#recommend--retrieve-by-example-ids)
   - [Query-Time Search Params (`EXACT`, `WITH`)](#query-time-search-params-exact-with)
   - [WHERE Clause Filters](#where-clause-filters)
   - [Hybrid Search (USING HYBRID)](#hybrid-search-using-hybrid)
@@ -92,6 +94,9 @@ qql> SEARCH notes SIMILAR TO 'vector databases' LIMIT 5 USING HYBRID RERANK
   - [CREATE COLLECTION — create a collection](#create-collection--create-a-collection)
   - [DROP COLLECTION — delete a collection](#drop-collection--delete-a-collection)
   - [DELETE — remove a point](#delete--remove-a-point)
+- [Script Files](#script-files)
+  - [EXECUTE — run a script file](#execute--run-a-qql-script-file)
+  - [DUMP COLLECTION — export to script](#dump-collection--export-collection-to-a-qql-script-file)
 - [Embedding Models](#embedding-models)
 - [Value Types in Dictionaries](#value-types-in-dictionaries)
 - [Configuration File](#configuration-file)
@@ -230,6 +235,8 @@ Inserts a new document into a collection. The `text` field is **mandatory** —
 
 If the collection does not exist yet, it is **created automatically** with the correct vector dimensions.
 
+If you include an `id` field in `VALUES`, QQL uses it as the Qdrant point ID. Supported explicit IDs are unsigned integers or UUID strings. If you omit `id`, QQL generates a UUID automatically.
+
 **Syntax:**
 ```
 INSERT INTO COLLECTION <collection_name> VALUES {<dict>}
@@ -248,6 +255,7 @@ INSERT INTO COLLECTION articles VALUES {'text': 'Qdrant supports cosine similari
 Insert with metadata:
 ```sql
 INSERT INTO COLLECTION articles VALUES {
+  'id': 1001,
   'text': 'Neural networks learn representations from data',
   'author': 'alice',
   'category': 'ml',
@@ -275,13 +283,13 @@ INSERT INTO COLLECTION articles VALUES {'text': 'hello world'}
 **What happens internally:**
 1. The `text` value is embedded into a dense vector using the configured model.
 2. In hybrid mode, a sparse BM25 vector is also generated.
-3. A UUID is auto-generated as the point ID.
-4. All fields (including `text`) are stored in the payload.
+3. If `id` is provided, it is used as the point ID; otherwise a UUID is auto-generated.
+4. All fields except `id` are stored in the payload.
 5. The point is upserted into Qdrant.
 
 **Rules:**
 - `text` is always required. Omitting it raises an error.
-- A point ID (UUID) is generated automatically — you do not provide one.
+- `id`, when provided, must be an unsigned integer or UUID string.
 - If the collection already exists with a different vector size (from a different model), an error is raised with a clear message.
 - Hybrid inserts require a hybrid collection (created with `CREATE COLLECTION ... HYBRID` or auto-created on first `USING HYBRID` insert).
 
@@ -293,6 +301,8 @@ Inserts multiple documents in a single statement. Each item in the array must co
 
 If the collection does not exist yet, it is **created automatically** on the first bulk insert.
 
+Each record may optionally include an `id` field. This is the preferred way to keep seed data deterministic and to make follow-up operations like `RECOMMEND` or `DELETE` reproducible.
+
 **Syntax:**
 ```
 INSERT BULK INTO COLLECTION <collection_name> VALUES [<dict>, <dict>, ...]
@@ -315,9 +325,9 @@ INSERT BULK INTO COLLECTION articles VALUES [
 Bulk insert with metadata:
 ```sql
 INSERT BULK INTO COLLECTION articles VALUES [
-  {'text': 'Attention is all you need', 'author': 'vaswani', 'year': 2017},
-  {'text': 'BERT: Pre-training of deep bidirectional transformers', 'author': 'devlin', 'year': 2018},
-  {'text': 'Language models are few-shot learners', 'author': 'brown', 'year': 2020}
+  {'id': 1001, 'text': 'Attention is all you need', 'author': 'vaswani', 'year': 2017},
+  {'id': 1002, 'text': 'BERT: Pre-training of deep bidirectional transformers', 'author': 'devlin', 'year': 2018},
+  {'id': 1003, 'text': 'Language models are few-shot learners', 'author': 'brown', 'year': 2020}
 ]
 ```
 
@@ -332,7 +342,7 @@ INSERT BULK INTO COLLECTION articles VALUES [
 **Rules:**
 - Every dict in the array must contain a `"text"` key. Missing `text` on any item raises an error with the offending index.
 - An empty array `[]` raises an error.
-- A UUID is auto-generated for each point — you do not provide IDs.
+- `id`, when provided, must be an unsigned integer or UUID string.
 - Supports all the same `USING` clauses as single `INSERT`.
 
 ---
@@ -415,13 +425,111 @@ Results are displayed as a table with three columns:
 ```
 
 - **Score** — similarity score. Higher is more relevant.
-- **ID** — the UUID of the matching point.
+- **ID** — the point ID returned by Qdrant. This may be an integer or a UUID string.
 - **Payload** — all fields stored alongside the vector.
 
 **Important:** Use the same model for SEARCH as you used for INSERT. Mixing models produces meaningless scores because the vectors live in different spaces.
 
 ---
 
+### RECOMMEND — retrieve by example IDs
+
+Performs a Qdrant recommendation query using existing point IDs as positive and optional negative examples.
+
+This is useful when you already know which stored points represent the kind of result you want. Qdrant uses those examples to retrieve nearby points, and QQL automatically excludes the seed IDs from the results.
+
+**Syntax:**
+```sql
+RECOMMEND FROM <collection_name> POSITIVE IDS (<id>, ...) LIMIT <n>
+RECOMMEND FROM <collection_name> POSITIVE IDS (<id>, ...) NEGATIVE IDS (<id>, ...) LIMIT <n>
+RECOMMEND FROM <collection_name> POSITIVE IDS (<id>, ...) STRATEGY '<strategy>' LIMIT <n>
+RECOMMEND FROM <collection_name> POSITIVE IDS (<id>, ...) LIMIT <n> WHERE <filter>
+RECOMMEND FROM <collection_name> POSITIVE IDS (<id>, ...) LIMIT <n> OFFSET <n>
+RECOMMEND FROM <collection_name> POSITIVE IDS (<id>, ...) LIMIT <n> SCORE THRESHOLD <f>
+RECOMMEND FROM <collection_name> POSITIVE IDS (<id>, ...) LIMIT <n> WITH { exact: true, hnsw_ef: <n> }
+RECOMMEND FROM <collection_name> POSITIVE IDS (<id>, ...) LIMIT <n> LOOKUP FROM <collection>
+RECOMMEND FROM <collection_name> POSITIVE IDS (<id>, ...) LIMIT <n> LOOKUP FROM <collection> VECTOR '<name>'
+RECOMMEND FROM <collection_name> POSITIVE IDS (<id>, ...) LIMIT <n> USING '<vector_name>'
+```
+
+**Examples:**
+
+Recommend more results like two known articles:
+```sql
+RECOMMEND FROM articles POSITIVE IDS (1001, 1002) LIMIT 5
+```
+
+Recommend similar results while steering away from one bad example:
+```sql
+RECOMMEND FROM articles POSITIVE IDS (1001, 1002) NEGATIVE IDS (1009) LIMIT 5
+```
+
+Use Qdrant's `best_score` recommendation strategy:
+```sql
+RECOMMEND FROM articles POSITIVE IDS (1001) STRATEGY 'best_score' LIMIT 10
+```
+
+Recommend only within a filtered subset:
+```sql
+RECOMMEND FROM articles POSITIVE IDS (1001) LIMIT 5 WHERE year >= 2020 AND status = 'published'
+```
+
+Paginate recommendations (skip first 5, return next 10):
+```sql
+RECOMMEND FROM articles POSITIVE IDS (1001) LIMIT 10 OFFSET 5
+```
+
+Filter out low-confidence recommendations:
+```sql
+RECOMMEND FROM articles POSITIVE IDS (1001) LIMIT 10 SCORE THRESHOLD 0.5
+```
+
+Exact KNN baseline for recommendations:
+```sql
+RECOMMEND FROM articles POSITIVE IDS (1001) LIMIT 5 WITH { exact: true }
+```
+
+Cross-collection recommend (look up example IDs from another collection):
+```sql
+RECOMMEND FROM target_collection
+  POSITIVE IDS ('a')
+  LOOKUP FROM source_collection VECTOR 'dense'
+  LIMIT 5
+```
+
+Recommend using a specific named vector in the target collection:
+```sql
+RECOMMEND FROM articles
+  POSITIVE IDS (1001)
+  USING 'sparse'
+  LIMIT 5
+```
+
+Full-featured recommend:
+```sql
+RECOMMEND FROM articles
+  POSITIVE IDS (1001, 1002)
+  NEGATIVE IDS (1009)
+  STRATEGY 'best_score'
+  LOOKUP FROM other_collection VECTOR 'dense'
+  USING 'dense'
+  LIMIT 10
+  OFFSET 5
+  SCORE THRESHOLD 0.5
+  WHERE year >= 2020
+  WITH { exact: true }
+```
+
+**Supported strategies:**
+
+- `average_vector`
+- `best_score`
+- `sum_scores`
+
+**Clause order:** `POSITIVE IDS` → `NEGATIVE IDS` → `STRATEGY` → `LOOKUP FROM` → `USING` → `LIMIT` → `OFFSET` → `SCORE THRESHOLD` → `WHERE` → `WITH`
+
+---
+
 ### Query-Time Search Params (`EXACT`, `WITH`)
 
 QQL supports a small set of Qdrant query-time search parameters on `SEARCH` statements.
@@ -862,7 +970,7 @@ Raises an error if the collection does not exist.
 
 ### DELETE — remove a point
 
-Deletes a single point from a collection by its ID. The point ID is the UUID returned by INSERT.
+Deletes a single point from a collection by its ID. The ID may be an integer or a UUID string, either generated by QQL or supplied explicitly on INSERT.
 
 **Syntax:**
 ```
@@ -886,6 +994,163 @@ To find a point's ID, run a SEARCH first and copy the ID from the results table.
 
 ---
 
+## Script Files
+
+QQL supports reading from and writing to `.qql` script files, making it easy to automate bulk operations, seed databases, and back up collections.
+
+---
+
+### EXECUTE — run a .qql script file
+
+Execute a file containing multiple QQL statements in sequence. Each statement is parsed and executed in order. `--` comments are stripped before parsing.
+
+**CLI usage:**
+```bash
+qql execute /path/to/script.qql
+
+# Stop on first error instead of continuing through all statements
+qql execute /path/to/script.qql --stop-on-error
+```
+
+**In-shell usage (inside the QQL REPL):**
+```
+qql> EXECUTE /path/to/script.qql
+qql> \e /path/to/script.qql
+```
+
+**Script format:**
+
+```sql
+-- This is a comment — the entire line is ignored
+-- ============================================================
+--  QQL Script — populate articles collection
+-- ============================================================
+
+-- Step 1: create the collection
+CREATE COLLECTION articles
+
+-- Step 2: bulk insert records
+INSERT BULK INTO COLLECTION articles VALUES [
+  {'text': 'Neural networks learn representations', 'year': 2023},
+  {'text': 'Attention mechanisms in transformers',  'year': 2024}
+]
+
+-- Step 3: verify
+SHOW COLLECTIONS
+```
+
+**Rules:**
+- `--` to end-of-line is a comment and is ignored (inline or full-line)
+- Statements can span multiple lines (e.g. `INSERT BULK ... VALUES [...]`)
+- `RECOMMEND` statements work in `.qql` files the same way they do in the REPL
+- Blank lines between statements are ignored
+- By default all statements run even if one fails; use `--stop-on-error` to halt early
+
+**Included examples:**
+- [`resources/sample.qql`](resources/sample.qql) seeds the demo medical dataset
+- [`resources/sample_v2.qql`](resources/sample_v2.qql) is a compact end-to-end example with explicit IDs and runnable `RECOMMEND` statements
+
+**Example output:**
+```
+Executing: /path/to/script.qql
+
+[1/3] CREATE COLLECTION articles
+  ✓ Collection 'articles' created (384-dimensional vectors, cosine distance)
+[2/3] INSERT BULK INTO COLLECTION articles VALUES [ …
+  ✓ Inserted 2 points
+[3/3] SHOW COLLECTIONS
+  ✓ 1 collection(s) found
+
+Done. 3/3 statement(s) succeeded.
+```
+
+---
+
+### DUMP COLLECTION — export collection to a .qql script file
+
+Export every point in a collection to a `.qql` script file. The generated file is valid QQL — it can be re-imported with `qql execute` to restore or migrate the collection. Points are written in batches of 50 as `INSERT BULK` statements.
+
+**CLI usage:**
+```bash
+qql dump <collection_name> <output.qql>
+```
+
+**In-shell usage (inside the QQL REPL):**
+```
+qql> DUMP COLLECTION <name> <output.qql>
+```
+
+**Example:**
+```bash
+qql dump medical_records /tmp/medical_records.qql
+```
+
+```
+Dumping: 'medical_records'  →  /tmp/medical_records.qql
+
+  Collection type : hybrid (dense + sparse)
+  Points          : 41
+  Batches         : 1  (50 points/batch)
+
+  [1/1] wrote 41 point(s)
+
+Done. 41 point(s) written.
+```
+
+**Generated file structure:**
+```sql
+-- ============================================================
+-- QQL Dump — collection: medical_records
+-- Generated : 2026-04-19 14:32:11
+-- Points    : 41
+-- Type      : hybrid (dense + sparse)
+-- Note      : Re-importing re-embeds all text using the
+--             configured model (see: qql connect).
+-- ============================================================
+
+CREATE COLLECTION medical_records HYBRID
+
+-- Batch 1 / 1  (records 1–41)
+INSERT BULK INTO COLLECTION medical_records VALUES [
+  {
+    'text': 'Alzheimers disease is characterized by...',
+    'title': 'Alzheimers Disease Overview',
+    'department': 'neurology',
+    'year': 2023,
+    'peer_reviewed': true
+  },
+  ...
+] USING HYBRID
+
+-- ============================================================
+-- End of dump
+-- Written : 41
+-- Skipped : 0  (no 'text' field)
+-- ============================================================
+```
+
+**Round-trip workflow — backup and restore:**
+```bash
+# 1. Dump the collection
+qql dump medical_records backup.qql
+
+# 2. Drop it
+qql> DROP COLLECTION medical_records
+
+# 3. Restore from the dump
+qql execute backup.qql
+```
+
+**Rules and notes:**
+- Points without a `'text'` payload field are **skipped** (counted in the footer comment).
+- Hybrid collections produce `CREATE COLLECTION <name> HYBRID` and `INSERT BULK ... USING HYBRID` statements.
+- Dense collections produce plain `CREATE COLLECTION <name>` and `INSERT BULK` statements.
+- All payload value types are preserved: strings, integers, floats, booleans (`true`/`false`), `null`, lists, and nested dicts.
+- Re-importing re-embeds all text using your currently configured model — use the same model as the original collection to preserve semantic accuracy.
+- Parent directories of the output path are created automatically.
+
+---
+
 ## Embedding Models
 
 QQL uses [Fastembed](https://github.com/qdrant/fastembed) to convert text into vectors locally — no external API call is needed.
@@ -1057,15 +1322,15 @@ result = run_query(
     "INSERT INTO COLLECTION notes VALUES {'text': 'hello world', 'author': 'alice', 'year': 2024}",
     url="http://localhost:6333",
 )
-print(result.message)   # "Inserted 1 point [<uuid>]"
-print(result.data)      # {"id": "...", "collection": "notes"}
+print(result.message)   # "Inserted 1 point [<id>]"
+print(result.data)      # {"id": 1001 or "<uuid>", "collection": "notes"}
 
 # Insert with hybrid vectors
 result = run_query(
     "INSERT INTO COLLECTION notes VALUES {'text': 'hello world'} USING HYBRID",
     url="http://localhost:6333",
 )
-print(result.message)   # "Inserted 1 point [<uuid>] (hybrid)"
+print(result.message)   # "Inserted 1 point [<id>] (hybrid)"
 
 # Dense search with WHERE filter
 result = run_query(
@@ -1120,9 +1385,10 @@ class ExecutionResult:
 
 | Operation | `result.data` type |
 |---|---|
-| INSERT (dense) | `{"id": "<uuid>", "collection": "<name>"}` |
-| INSERT (hybrid) | `{"id": "<uuid>", "collection": "<name>"}` |
+| INSERT (dense) | `{"id": int | "<uuid>", "collection": "<name>"}` |
+| INSERT (hybrid) | `{"id": int | "<uuid>", "collection": "<name>"}` |
 | SEARCH | `[{"id": str, "score": float, "payload": dict}, ...]` |
+| RECOMMEND | `[{"id": str, "score": float, "payload": dict}, ...]` |
 | SHOW COLLECTIONS | `["name1", "name2", ...]` |
 | CREATE COLLECTION | `None` |
 | DROP COLLECTION | `None` |