chub-dev 0.2.0-beta.3 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,1395 +0,0 @@
1
- ---
2
- name: sdk
3
- description: "Vector database for AI applications with semantic search, hybrid search, reranking, and integrated embeddings"
4
- metadata:
5
- languages: "python"
6
- versions: "7.3.0"
7
- updated-on: "2025-10-26"
8
- source: maintainer
9
- tags: "pinecone,sdk,vector-db,ai,search"
10
- ---
11
- # Pinecone Python SDK Coding Guidelines
12
-
13
- You are a Pinecone vector database coding expert. Help me with writing code using the Pinecone SDK for Python.
14
-
15
- You can find the official SDK documentation and code samples here:
16
- https://docs.pinecone.io/
17
-
18
- ## Golden Rule: Use the Correct and Current SDK
19
-
20
- Always use the official Pinecone Python SDK for all Pinecone vector database interactions. Do not use legacy libraries or unofficial packages.
21
-
22
- - **Library Name:** Pinecone Python SDK
23
- - **PyPI Package:** `pinecone`
24
- - **Legacy Libraries**: `pinecone-client` (deprecated) and other unofficial packages are not recommended
25
-
26
- **Installation:**
27
-
28
- - **Correct:** `pip install pinecone`
29
- - **Correct (with optional dependencies):** `pip install "pinecone[asyncio,grpc]"`
30
-
31
- **APIs and Usage:**
32
-
33
- - **Correct:** `from pinecone import Pinecone`
34
- - **Correct:** `pc = Pinecone(api_key='...')`
35
- - **Correct:** `pc.create_index(...)`
36
- - **Correct:** `pc.Index(host=...)`
37
- - **Incorrect:** `import pinecone` (legacy pattern)
38
- - **Incorrect:** `pinecone.init()` (deprecated)
39
- - **Incorrect:** `pinecone.Index('name')` (legacy pattern)
40
-
41
- ## Installation
42
-
43
- Install the Pinecone SDK using pip:
44
-
45
- ```bash
46
- pip install pinecone
47
- ```
48
-
49
- ### With Optional Dependencies
50
-
51
- For enhanced functionality, install with optional dependencies:
52
-
53
- ```bash
54
- pip install "pinecone[asyncio,grpc]"
55
- ```
56
-
57
- - `asyncio`: Enables async/await support
58
- - `grpc`: Enables gRPC for better performance with high-concurrency scenarios
59
-
60
- ### Alternative Package Managers
61
-
62
- **Using uv:**
63
- ```bash
64
- uv add pinecone
65
- ```
66
-
67
- **Using Poetry:**
68
- ```bash
69
- poetry add pinecone
70
- ```
71
-
72
- **Requirements:**
73
- - Python 3.9 or higher
74
- - Tested with CPython 3.9 to 3.13
75
-
76
- ## Initialization and API Key
77
-
78
- The SDK requires creating a `Pinecone` instance for all API calls.
79
-
80
- ### Using Environment Variable
81
-
82
- Set your API key as an environment variable:
83
-
84
- ```bash
85
- export PINECONE_API_KEY="your_api_key_here"
86
- ```
87
-
88
- Then initialize:
89
-
90
- ```python
91
- from pinecone import Pinecone
92
-
93
- pc = Pinecone()
94
- ```
95
-
96
- ### Using Configuration Parameter
97
-
98
- Pass the API key directly:
99
-
100
- ```python
101
- from pinecone import Pinecone
102
-
103
- pc = Pinecone(api_key='your_api_key_here')
104
- ```
105
-
106
- ### gRPC Client (Recommended for High Performance)
107
-
108
- For better performance with high-concurrency operations:
109
-
110
- ```python
111
- from pinecone.grpc import PineconeGRPC as Pinecone
112
-
113
- pc = Pinecone(api_key='your_api_key_here')
114
- ```
115
-
116
- The gRPC client offers multiplexing advantages over REST for parallel processing.
117
-
118
- ## Creating Indexes
119
-
120
- ### Serverless Index (Basic)
121
-
122
- ```python
123
- from pinecone import Pinecone, ServerlessSpec
124
-
125
- pc = Pinecone(api_key='your_api_key_here')
126
-
127
- pc.create_index(
128
- name='my-index',
129
- dimension=1536,
130
- metric='cosine',
131
- spec=ServerlessSpec(
132
- cloud='aws',
133
- region='us-west-2'
134
- )
135
- )
136
- ```
137
-
138
- ### Serverless Index with Deletion Protection
139
-
140
- ```python
141
- from pinecone import Pinecone, ServerlessSpec
142
-
143
- pc = Pinecone(api_key='your_api_key_here')
144
-
145
- pc.create_index(
146
- name='protected-index',
147
- dimension=1536,
148
- metric='cosine',
149
- spec=ServerlessSpec(
150
- cloud='aws',
151
- region='us-east-1'
152
- ),
153
- deletion_protection='enabled'
154
- )
155
- ```
156
-
157
- ### Pod-Based Index
158
-
159
- ```python
160
- from pinecone import Pinecone, PodSpec
161
-
162
- pc = Pinecone(api_key='your_api_key_here')
163
-
164
- pc.create_index(
165
- name='pod-index',
166
- dimension=1536,
167
- metric='cosine',
168
- spec=PodSpec(
169
- environment='us-west-2',
170
- pod_type='p1.x1',
171
- pods=1,
172
- replicas=1
173
- )
174
- )
175
- ```
176
-
177
- ### Hybrid Search Index (Sparse-Dense)
178
-
179
- For hybrid search supporting both dense and sparse vectors, use `dotproduct` metric:
180
-
181
- ```python
182
- from pinecone import Pinecone, ServerlessSpec
183
-
184
- pc = Pinecone(api_key='your_api_key_here')
185
-
186
- pc.create_index(
187
- name='hybrid-index',
188
- dimension=1024,
189
- metric='dotproduct',
190
- spec=ServerlessSpec(
191
- cloud='aws',
192
- region='us-east-1'
193
- )
194
- )
195
- ```
196
-
197
- **Note:** The `dotproduct` metric is the only metric that supports sparse-dense hybrid search.
198
-
199
- ### Using Enums for Type Safety
200
-
201
- ```python
202
- from pinecone import Pinecone, ServerlessSpec, CloudProvider, AwsRegion, VectorType
203
-
204
- pc = Pinecone(api_key='your_api_key_here')
205
-
206
- index_config = pc.create_index(
207
- name='typed-index',
208
- dimension=1536,
209
- metric='cosine',
210
- spec=ServerlessSpec(
211
- cloud=CloudProvider.AWS,
212
- region=AwsRegion.US_EAST_1
213
- ),
214
- vector_type=VectorType.DENSE
215
- )
216
-
217
- print(index_config.host)
218
- ```
219
-
220
- ## Index Management
221
-
222
- ### Describe Index
223
-
224
- Get index details including status, dimension, and host:
225
-
226
- ```python
227
- from pinecone import Pinecone
228
-
229
- pc = Pinecone(api_key='your_api_key_here')
230
-
231
- index_description = pc.describe_index('my-index')
232
- print(index_description)
233
- print(f"Host: {index_description.host}")
234
- ```
235
-
236
- ### List All Indexes
237
-
238
- ```python
239
- from pinecone import Pinecone
240
-
241
- pc = Pinecone(api_key='your_api_key_here')
242
-
243
- index_list = pc.list_indexes()
244
- for index in index_list:
245
- print(index.name)
246
- ```
247
-
248
- ### Delete Index
249
-
250
- ```python
251
- from pinecone import Pinecone
252
-
253
- pc = Pinecone(api_key='your_api_key_here')
254
-
255
- pc.delete_index('my-index')
256
- ```
257
-
258
- ### Configure Index (Scale)
259
-
260
- For pod-based indexes, adjust replicas and pod type:
261
-
262
- ```python
263
- from pinecone import Pinecone
264
-
265
- pc = Pinecone(api_key='your_api_key_here')
266
-
267
- pc.configure_index(
268
- name='pod-index',
269
- replicas=2,
270
- pod_type='p1.x2'
271
- )
272
- ```
273
-
274
- ### Check if Index Exists
275
-
276
- ```python
277
- from pinecone import Pinecone
278
-
279
- pc = Pinecone(api_key='your_api_key_here')
280
-
281
- if pc.has_index('my-index'):
282
- print('Index exists')
283
- else:
284
- print('Index does not exist')
285
- ```
286
-
287
- ## Connecting to an Index
288
-
289
- ### Basic Connection
290
-
291
- ```python
292
- from pinecone import Pinecone
293
-
294
- pc = Pinecone(api_key='your_api_key_here')
295
-
296
- # Get index host from description
297
- index_config = pc.describe_index('my-index')
298
- index = pc.Index(host=index_config.host)
299
- ```
300
-
301
- ### Using gRPC Client
302
-
303
- ```python
304
- from pinecone.grpc import PineconeGRPC as Pinecone
305
-
306
- pc = Pinecone(api_key='your_api_key_here')
307
-
308
- index_config = pc.describe_index('my-index')
309
- index = pc.Index(host=index_config.host)
310
- ```
311
-
312
- ## Upserting Vectors
313
-
314
- ### Dense Vectors (Basic)
315
-
316
- ```python
317
- from pinecone import Pinecone
318
-
319
- pc = Pinecone(api_key='your_api_key_here')
320
- index_config = pc.describe_index('my-index')
321
- index = pc.Index(host=index_config.host)
322
-
323
- index.upsert(
324
- vectors=[
325
- ("vec1", [0.1, 0.2, 0.3, 0.4, 0.5]),
326
- ("vec2", [0.2, 0.3, 0.4, 0.5, 0.6]),
327
- ]
328
- )
329
- ```
330
-
331
- ### Dense Vectors with Metadata
332
-
333
- ```python
334
- index.upsert(
335
- vectors=[
336
- ("vec1", [0.1, 0.2, 0.3, 0.4, 0.5], {"genre": "comedy", "year": 2020}),
337
- ("vec2", [0.2, 0.3, 0.4, 0.5, 0.6], {"genre": "action", "year": 2021}),
338
- ]
339
- )
340
- ```
341
-
342
- ### Using Dictionary Format
343
-
344
- ```python
345
- index.upsert(
346
- vectors=[
347
- {
348
- "id": "vec1",
349
- "values": [0.1, 0.2, 0.3, 0.4, 0.5],
350
- "metadata": {"genre": "comedy", "year": 2020}
351
- },
352
- {
353
- "id": "vec2",
354
- "values": [0.2, 0.3, 0.4, 0.5, 0.6],
355
- "metadata": {"genre": "action", "year": 2021}
356
- }
357
- ]
358
- )
359
- ```
360
-
361
- ### Namespaced Upsert
362
-
363
- ```python
364
- index.upsert(
365
- vectors=[
366
- ("vec1", [0.1, 0.2, 0.3], {"category": "A"}),
367
- ],
368
- namespace='example-namespace'
369
- )
370
- ```
371
-
372
- ### Batch Upsert (Large Datasets)
373
-
374
- For upserting many vectors, batch them in groups of up to 1,000 records (max 2 MB per batch):
375
-
376
- ```python
377
- from pinecone import Pinecone
378
-
379
- pc = Pinecone(api_key='your_api_key_here')
380
- index_config = pc.describe_index('my-index')
381
- index = pc.Index(host=index_config.host)
382
-
383
- batch_size = 1000
384
- vectors = [] # Your vector list
385
-
386
- for i in range(0, len(vectors), batch_size):
387
- batch = vectors[i:i + batch_size]
388
- index.upsert(vectors=batch)
389
- ```
390
-
391
- ### Sparse-Dense Vectors (Hybrid Search)
392
-
393
- ```python
394
- index.upsert(
395
- vectors=[
396
- {
397
- "id": "vec1",
398
- "values": [0.1, 0.2, 0.3], # Dense vector
399
- "sparse_values": {
400
- "indices": [10, 45, 16],
401
- "values": [0.5, 0.5, 0.2]
402
- },
403
- "metadata": {"text": "original text content"}
404
- }
405
- ]
406
- )
407
- ```
408
-
409
- ### Parallel Upsert with Threading
410
-
411
- For better throughput, use parallel upserts:
412
-
413
- ```python
414
- from concurrent.futures import ThreadPoolExecutor
415
- from pinecone import Pinecone
416
-
417
- pc = Pinecone(api_key='your_api_key_here')
418
- index_config = pc.describe_index('my-index')
419
- index = pc.Index(host=index_config.host)
420
-
421
- def upsert_batch(batch):
422
- return index.upsert(vectors=batch)
423
-
424
- batch_size = 1000
425
- batches = [vectors[i:i + batch_size] for i in range(0, len(vectors), batch_size)]
426
-
427
- with ThreadPoolExecutor(max_workers=10) as executor:
428
- results = list(executor.map(upsert_batch, batches))
429
- ```
430
-
431
- ## Querying Vectors
432
-
433
- ### Basic Query (Dense)
434
-
435
- ```python
436
- from pinecone import Pinecone
437
-
438
- pc = Pinecone(api_key='your_api_key_here')
439
- index_config = pc.describe_index('my-index')
440
- index = pc.Index(host=index_config.host)
441
-
442
- query_response = index.query(
443
- vector=[0.1, 0.2, 0.3, 0.4, 0.5],
444
- top_k=10,
445
- include_metadata=True,
446
- include_values=False
447
- )
448
-
449
- for match in query_response.matches:
450
- print(f"ID: {match.id}, Score: {match.score}, Metadata: {match.metadata}")
451
- ```
452
-
453
- ### Query with Metadata Filter
454
-
455
- ```python
456
- query_response = index.query(
457
- vector=[0.1, 0.2, 0.3, 0.4, 0.5],
458
- top_k=5,
459
- filter={"genre": {"$eq": "comedy"}},
460
- include_metadata=True
461
- )
462
- ```
463
-
464
- ### Complex Metadata Filters
465
-
466
- ```python
467
- # Multiple conditions with $and
468
- response = index.query(
469
- vector=[0.1, 0.2, 0.3],
470
- top_k=10,
471
- filter={
472
- "$and": [
473
- {"genre": {"$eq": "comedy"}},
474
- {"year": {"$gte": 2020}}
475
- ]
476
- },
477
- include_metadata=True
478
- )
479
-
480
- # Using $or operator
481
- response2 = index.query(
482
- vector=[0.1, 0.2, 0.3],
483
- top_k=10,
484
- filter={
485
- "$or": [
486
- {"genre": {"$eq": "comedy"}},
487
- {"genre": {"$eq": "drama"}}
488
- ]
489
- },
490
- include_metadata=True
491
- )
492
-
493
- # Using $in for multiple values
494
- response3 = index.query(
495
- vector=[0.1, 0.2, 0.3],
496
- top_k=10,
497
- filter={"genre": {"$in": ["comedy", "action", "drama"]}},
498
- include_metadata=True
499
- )
500
- ```
501
-
502
- ### Query by ID
503
-
504
- ```python
505
- query_response = index.query(
506
- id='vec1',
507
- top_k=10,
508
- include_metadata=True
509
- )
510
- ```
511
-
512
- ### Namespaced Query
513
-
514
- ```python
515
- query_response = index.query(
516
- vector=[0.1, 0.2, 0.3],
517
- top_k=5,
518
- namespace='example-namespace',
519
- include_metadata=True
520
- )
521
- ```
522
-
523
- ### Hybrid Query (Sparse-Dense)
524
-
525
- ```python
526
- query_response = index.query(
527
- vector=[0.1, 0.2, 0.3], # Dense query vector
528
- sparse_vector={
529
- "indices": [10, 45, 16],
530
- "values": [0.5, 0.5, 0.2]
531
- },
532
- top_k=10,
533
- include_metadata=True
534
- )
535
- ```
536
-
537
- ## Fetching Vectors
538
-
539
- ### Fetch by IDs
540
-
541
- ```python
542
- from pinecone import Pinecone
543
-
544
- pc = Pinecone(api_key='your_api_key_here')
545
- index_config = pc.describe_index('my-index')
546
- index = pc.Index(host=index_config.host)
547
-
548
- fetch_response = index.fetch(ids=['vec1', 'vec2', 'vec3'])
549
- print(fetch_response.vectors)
550
- ```
551
-
552
- ### Fetch from Namespace
553
-
554
- ```python
555
- fetch_response = index.fetch(
556
- ids=['vec1', 'vec2'],
557
- namespace='example-namespace'
558
- )
559
-
560
- for id, vector in fetch_response.vectors.items():
561
- print(f"ID: {id}, Values: {vector.values}, Metadata: {vector.metadata}")
562
- ```
563
-
564
- ## Updating Vectors
565
-
566
- ### Update Values
567
-
568
- ```python
569
- from pinecone import Pinecone
570
-
571
- pc = Pinecone(api_key='your_api_key_here')
572
- index_config = pc.describe_index('my-index')
573
- index = pc.Index(host=index_config.host)
574
-
575
- index.update(
576
- id='vec1',
577
- values=[0.9, 0.8, 0.7, 0.6, 0.5]
578
- )
579
- ```
580
-
581
- ### Update Metadata
582
-
583
- ```python
584
- index.update(
585
- id='vec1',
586
- set_metadata={"genre": "documentary", "year": 2023}
587
- )
588
- ```
589
-
590
- ### Update with Namespace
591
-
592
- ```python
593
- index.update(
594
- id='vec1',
595
- values=[0.1, 0.2, 0.3],
596
- set_metadata={"updated": True},
597
- namespace='example-namespace'
598
- )
599
- ```
600
-
601
- ### Update Both Values and Metadata
602
-
603
- ```python
604
- index.update(
605
- id='vec1',
606
- values=[0.9, 0.8, 0.7, 0.6, 0.5],
607
- set_metadata={"genre": "thriller", "updated": True}
608
- )
609
- ```
610
-
611
- ## Deleting Vectors
612
-
613
- ### Delete by IDs
614
-
615
- ```python
616
- from pinecone import Pinecone
617
-
618
- pc = Pinecone(api_key='your_api_key_here')
619
- index_config = pc.describe_index('my-index')
620
- index = pc.Index(host=index_config.host)
621
-
622
- index.delete(ids=['vec1', 'vec2', 'vec3'])
623
- ```
624
-
625
- ### Delete with Metadata Filter
626
-
627
- ```python
628
- index.delete(
629
- filter={"genre": {"$eq": "comedy"}}
630
- )
631
- ```
632
-
633
- ### Delete All in Namespace
634
-
635
- ```python
636
- index.delete(delete_all=True, namespace='example-namespace')
637
- ```
638
-
639
- **Warning:** Deleting all records from a namespace will also delete the namespace itself. This operation is irreversible.
640
-
641
- ### Delete from Specific Namespace
642
-
643
- ```python
644
- index.delete(
645
- ids=['vec1', 'vec2'],
646
- namespace='example-namespace'
647
- )
648
- ```
649
-
650
- ## Index Statistics
651
-
652
- ### Get Index Stats
653
-
654
- ```python
655
- from pinecone import Pinecone
656
-
657
- pc = Pinecone(api_key='your_api_key_here')
658
- index_config = pc.describe_index('my-index')
659
- index = pc.Index(host=index_config.host)
660
-
661
- stats = index.describe_index_stats()
662
- print(f"Total vectors: {stats.total_vector_count}")
663
- print(f"Dimension: {stats.dimension}")
664
- print(f"Namespaces: {stats.namespaces}")
665
- ```
666
-
667
- ### Get Stats with Filter
668
-
669
- ```python
670
- stats = index.describe_index_stats(
671
- filter={"genre": {"$eq": "comedy"}}
672
- )
673
- print(f"Filtered vector count: {stats.total_vector_count}")
674
- ```
675
-
676
- ## Integrated Inference (Embeddings)
677
-
678
- Pinecone provides hosted embedding models through the Inference API.
679
-
680
- ### Generate Embeddings
681
-
682
- ```python
683
- from pinecone import Pinecone
684
-
685
- pc = Pinecone(api_key='your_api_key_here')
686
-
687
- # Embed documents
688
- embeddings = pc.inference.embed(
689
- model="multilingual-e5-large",
690
- inputs=[
691
- "Turkey is a classic meat to eat at American Thanksgiving.",
692
- "Many people enjoy the beautiful mosques in Turkey."
693
- ],
694
- parameters={
695
- "input_type": "passage",
696
- "truncate": "END"
697
- }
698
- )
699
-
700
- for embedding in embeddings:
701
- print(f"Values: {embedding['values'][:5]}...") # Show first 5 dimensions
702
- ```
703
-
704
- ### Embed Queries
705
-
706
- ```python
707
- from pinecone import Pinecone
708
-
709
- pc = Pinecone(api_key='your_api_key_here')
710
-
711
- query_embedding = pc.inference.embed(
712
- model="multilingual-e5-large",
713
- inputs=["How should I prepare my turkey?"],
714
- parameters={
715
- "input_type": "query",
716
- "truncate": "END"
717
- }
718
- )
719
-
720
- # Use the embedding for querying
721
- index_config = pc.describe_index('my-index')
722
- index = pc.Index(host=index_config.host)
723
-
724
- results = index.query(
725
- vector=query_embedding[0]['values'],
726
- top_k=10,
727
- include_metadata=True
728
- )
729
- ```
730
-
731
- ### Using Enum for Models
732
-
733
- ```python
734
- from pinecone import Pinecone, EmbedModel
735
-
736
- pc = Pinecone(api_key='your_api_key_here')
737
-
738
- embeddings = pc.inference.embed(
739
- model=EmbedModel.Multilingual_E5_Large,
740
- inputs=["Sample text"],
741
- parameters={"input_type": "passage"}
742
- )
743
- ```
744
-
745
- ### Available Embedding Models
746
-
747
- - `multilingual-e5-large`: Efficient dense embedding model trained on multilingual datasets
748
- - `pinecone-sparse-english-v0`: Sparse embedding model for keyword or hybrid semantic/keyword search
749
-
750
- ## Sparse Embeddings for Hybrid Search
751
-
752
- ### Using Pinecone Sparse Embedding
753
-
754
- ```python
755
- from pinecone import Pinecone
756
-
757
- pc = Pinecone(api_key='your_api_key_here')
758
-
759
- # Generate sparse embeddings
760
- sparse_embeddings = pc.inference.embed(
761
- model="pinecone-sparse-english-v0",
762
- inputs=[
763
- "Apple's first product was the Apple I computer.",
764
- "The company was founded in 1976."
765
- ],
766
- parameters={
767
- "input_type": "passage"
768
- }
769
- )
770
-
771
- # Generate dense embeddings
772
- dense_embeddings = pc.inference.embed(
773
- model="multilingual-e5-large",
774
- inputs=[
775
- "Apple's first product was the Apple I computer.",
776
- "The company was founded in 1976."
777
- ],
778
- parameters={
779
- "input_type": "passage"
780
- }
781
- )
782
-
783
- # Upsert both
784
- index_config = pc.describe_index('hybrid-index')
785
- index = pc.Index(host=index_config.host)
786
-
787
- for i, (dense, sparse) in enumerate(zip(dense_embeddings, sparse_embeddings)):
788
- index.upsert(
789
- vectors=[{
790
- "id": f"doc{i}",
791
- "values": dense['values'],
792
- "sparse_values": {
793
- "indices": sparse['indices'],
794
- "values": sparse['values']
795
- }
796
- }]
797
- )
798
- ```
799
-
800
- ## Namespaces
801
-
802
- Namespaces partition records within an index and enable multitenancy.
803
-
804
- ### Create Namespace (Implicit)
805
-
806
- Namespaces are created automatically when you upsert to them:
807
-
808
- ```python
809
- from pinecone import Pinecone
810
-
811
- pc = Pinecone(api_key='your_api_key_here')
812
- index_config = pc.describe_index('my-index')
813
- index = pc.Index(host=index_config.host)
814
-
815
- # Create namespace for customer 1
816
- index.upsert(
817
- vectors=[("vec1", [0.1, 0.2, 0.3])],
818
- namespace='customer-1'
819
- )
820
-
821
- # Create namespace for customer 2
822
- index.upsert(
823
- vectors=[("vec1", [0.4, 0.5, 0.6])],
824
- namespace='customer-2'
825
- )
826
- ```
827
-
828
- ### List Namespaces
829
-
830
- ```python
831
- stats = index.describe_index_stats()
832
- print(list(stats.namespaces.keys()))
833
- ```
834
-
835
- ### Default Namespace
836
-
837
- If no namespace is specified, vectors are stored in the `""` (empty string) namespace:
838
-
839
- ```python
840
- # This uses the default namespace
841
- index.upsert(vectors=[("vec1", [0.1, 0.2, 0.3])])
842
-
843
- # Equivalent to
844
- index.upsert(vectors=[("vec1", [0.1, 0.2, 0.3])], namespace='')
845
- ```
846
-
847
- ## Reranking
848
-
849
- Use Pinecone's reranking models to reorder results by relevance.
850
-
851
- ### Rerank Results
852
-
853
- ```python
854
- from pinecone import Pinecone
855
-
856
- pc = Pinecone(api_key='your_api_key_here')
857
-
858
- reranked = pc.inference.rerank(
859
- model="bge-reranker-v2-m3",
860
- query="What is the capital of France?",
861
- documents=[
862
- {"id": "doc1", "text": "Paris is the capital of France."},
863
- {"id": "doc2", "text": "London is the capital of England."},
864
- {"id": "doc3", "text": "Berlin is the capital of Germany."}
865
- ],
866
- top_n=2,
867
- return_documents=True
868
- )
869
-
870
- for result in reranked.data:
871
- print(f"ID: {result['id']}, Score: {result['score']}, Text: {result.get('document', {}).get('text')}")
872
- ```
873
-
874
- ### Rerank Query Results
875
-
876
- ```python
877
- from pinecone import Pinecone
878
-
879
- pc = Pinecone(api_key='your_api_key_here')
880
- index_config = pc.describe_index('my-index')
881
- index = pc.Index(host=index_config.host)
882
-
883
- # First, query for candidates
884
- query_response = index.query(
885
- vector=[0.1, 0.2, 0.3],
886
- top_k=50,
887
- include_metadata=True
888
- )
889
-
890
- # Prepare documents for reranking
891
- documents = [
892
- {"id": match.id, "text": match.metadata.get('text', '')}
893
- for match in query_response.matches
894
- ]
895
-
896
- # Rerank
897
- reranked = pc.inference.rerank(
898
- model="bge-reranker-v2-m3",
899
- query="User's search query",
900
- documents=documents,
901
- top_n=10
902
- )
903
-
904
- print(reranked)
905
- ```
906
-
907
- ## Collections (Backups)
908
-
909
- Collections are static snapshots of an index that can be used to create new indexes.
910
-
911
- ### Create Collection
912
-
913
- ```python
914
- from pinecone import Pinecone
915
-
916
- pc = Pinecone(api_key='your_api_key_here')
917
-
918
- pc.create_collection(
919
- name='my-collection',
920
- source='my-index'
921
- )
922
- ```
923
-
924
- ### List Collections
925
-
926
- ```python
927
- collections = pc.list_collections()
928
- for collection in collections:
929
- print(collection.name)
930
- ```
931
-
932
- ### Describe Collection
933
-
934
- ```python
935
- collection_info = pc.describe_collection('my-collection')
936
- print(collection_info)
937
- ```
938
-
939
- ### Create Index from Collection
940
-
941
- ```python
942
- from pinecone import Pinecone, ServerlessSpec
943
-
944
- pc = Pinecone(api_key='your_api_key_here')
945
-
946
- pc.create_index(
947
- name='new-index-from-backup',
948
- dimension=1536,
949
- metric='cosine',
950
- spec=ServerlessSpec(
951
- cloud='aws',
952
- region='us-west-2'
953
- ),
954
- source_collection='my-collection'
955
- )
956
- ```
957
-
958
- ### Delete Collection
959
-
960
- ```python
961
- pc.delete_collection('my-collection')
962
- ```
963
-
964
- ## Async Support
965
-
966
- The SDK supports async/await for non-blocking operations.
967
-
968
- ### Installation with Async Support
969
-
970
- ```bash
971
- pip install "pinecone[asyncio]"
972
- ```
973
-
974
- ### Async Initialization
975
-
976
- ```python
977
- from pinecone import Pinecone
978
-
979
- pc = Pinecone(api_key='your_api_key_here')
980
- index_config = pc.describe_index('my-index')
981
- index = pc.Index(host=index_config.host, async_req=True)
982
- ```
983
-
984
- ### Async Upsert
985
-
986
- ```python
987
- import asyncio
988
- from pinecone import Pinecone
989
-
990
- async def async_upsert():
991
- pc = Pinecone(api_key='your_api_key_here')
992
- index_config = pc.describe_index('my-index')
993
- index = pc.Index(host=index_config.host, async_req=True)
994
-
995
- await index.upsert(
996
- vectors=[("vec1", [0.1, 0.2, 0.3])]
997
- )
998
-
999
- asyncio.run(async_upsert())
1000
- ```
1001
-
1002
- ### Async Query
1003
-
1004
- ```python
1005
- import asyncio
1006
- from pinecone import Pinecone
1007
-
1008
- async def async_query():
1009
- pc = Pinecone(api_key='your_api_key_here')
1010
- index_config = pc.describe_index('my-index')
1011
- index = pc.Index(host=index_config.host, async_req=True)
1012
-
1013
- results = await index.query(
1014
- vector=[0.1, 0.2, 0.3],
1015
- top_k=10,
1016
- include_metadata=True
1017
- )
1018
- return results
1019
-
1020
- results = asyncio.run(async_query())
1021
- ```
1022
-
1023
- ## Error Handling
1024
-
1025
- ### Basic Error Handling
1026
-
1027
- ```python
1028
- from pinecone import Pinecone
1029
- from pinecone.exceptions import PineconeException
1030
-
1031
- pc = Pinecone(api_key='your_api_key_here')
1032
-
1033
- try:
1034
- index_config = pc.describe_index('my-index')
1035
- index = pc.Index(host=index_config.host)
1036
-
1037
- results = index.query(
1038
- vector=[0.1, 0.2, 0.3],
1039
- top_k=10
1040
- )
1041
- except PineconeException as e:
1042
- print(f"Pinecone error: {e}")
1043
- except Exception as e:
1044
- print(f"Unexpected error: {e}")
1045
- ```
1046
-
1047
- ### Handling Specific Errors
1048
-
1049
- ```python
1050
- from pinecone import Pinecone
1051
- from pinecone.exceptions import NotFoundException, UnauthorizedException
1052
-
1053
- pc = Pinecone(api_key='your_api_key_here')
1054
-
1055
- try:
1056
- index_config = pc.describe_index('nonexistent-index')
1057
- except NotFoundException:
1058
- print("Index not found")
1059
- except UnauthorizedException:
1060
- print("Invalid API key")
1061
- ```
1062
-
1063
- ## Performance Best Practices
1064
-
1065
- ### Use gRPC Client for High Throughput
1066
-
1067
- ```python
1068
- from pinecone.grpc import PineconeGRPC as Pinecone
1069
-
1070
- pc = Pinecone(api_key='your_api_key_here')
1071
- index_config = pc.describe_index('my-index')
1072
- index = pc.Index(host=index_config.host)
1073
- ```
1074
-
1075
- The gRPC client offers better performance for high-concurrency scenarios.
1076
-
1077
- ### Batch Operations
1078
-
1079
- Always batch upsert operations for better throughput:
1080
-
1081
- ```python
1082
- # Good: Batch upsert
1083
- vectors = generate_vectors(1000)
1084
- index.upsert(vectors=vectors)
1085
-
1086
- # Bad: Individual upserts
1087
- for vector in vectors:
1088
- index.upsert(vectors=[vector]) # Don't do this
1089
- ```
1090
-
1091
- ### Parallel Requests
1092
-
1093
- Use ThreadPoolExecutor for independent parallel operations:
1094
-
1095
- ```python
1096
- from concurrent.futures import ThreadPoolExecutor
1097
- from pinecone import Pinecone
1098
-
1099
- pc = Pinecone(api_key='your_api_key_here')
1100
- index_config = pc.describe_index('my-index')
1101
- index = pc.Index(host=index_config.host)
1102
-
1103
- def query_batch(query_vector):
1104
- return index.query(vector=query_vector, top_k=5)
1105
-
1106
- query_vectors = [[0.1, 0.2, 0.3], [0.4, 0.5, 0.6], [0.7, 0.8, 0.9]]
1107
-
1108
- with ThreadPoolExecutor(max_workers=10) as executor:
1109
- results = list(executor.map(query_batch, query_vectors))
1110
- ```
1111
-
1112
- ### Connection Reuse
1113
-
1114
- Reuse the index connection instead of recreating it:
1115
-
1116
- ```python
1117
- # Good: Create once, reuse
1118
- from pinecone import Pinecone
1119
-
1120
- pc = Pinecone(api_key='your_api_key_here')
1121
- index_config = pc.describe_index('my-index')
1122
- index = pc.Index(host=index_config.host)
1123
-
1124
- for i in range(100):
1125
- index.query(vector=get_vector(i), top_k=5)
1126
-
1127
- # Bad: Recreate each time
1128
- for i in range(100):
1129
- index_config = pc.describe_index('my-index')
1130
- index = pc.Index(host=index_config.host) # Don't do this
1131
- index.query(vector=get_vector(i), top_k=5)
1132
- ```
1133
-
1134
- ## Metadata Filtering Operators
1135
-
1136
- Pinecone supports MongoDB-style query operators for metadata filtering:
1137
-
1138
- - `$eq`: Equal to
1139
- - `$ne`: Not equal to
1140
- - `$gt`: Greater than
1141
- - `$gte`: Greater than or equal to
1142
- - `$lt`: Less than
1143
- - `$lte`: Less than or equal to
1144
- - `$in`: In array
1145
- - `$nin`: Not in array
1146
- - `$exists`: Field exists
1147
- - `$and`: Logical AND
1148
- - `$or`: Logical OR
1149
-
1150
- ### Example: Complex Filter
1151
-
1152
- ```python
1153
- results = index.query(
1154
- vector=[0.1, 0.2, 0.3],
1155
- top_k=10,
1156
- filter={
1157
- "$and": [
1158
- {"year": {"$gte": 2020, "$lte": 2023}},
1159
- {
1160
- "$or": [
1161
- {"genre": {"$eq": "comedy"}},
1162
- {"genre": {"$eq": "drama"}}
1163
- ]
1164
- },
1165
- {"rating": {"$exists": True}}
1166
- ]
1167
- },
1168
- include_metadata=True
1169
- )
1170
- ```
1171
-
1172
- ## Limits and Constraints
1173
-
1174
- - **Max vectors per upsert**: 1,000 records (or 2 MB per batch)
1175
- - **Max vectors per upsert (with text)**: 96 records
1176
- - **Metadata size per vector**: 40 KB max
1177
- - **Record ID length**: 512 characters max
1178
- - **Dense vector dimensions**: Up to 20,000
1179
- - **Sparse vector non-zero values**: 2,048 max
1180
- - **Top-K query limit**: 10,000
1181
-
1182
- ## Common Patterns
1183
-
1184
- ### Pattern: Complete Workflow
1185
-
1186
- ```python
1187
- from pinecone import Pinecone, ServerlessSpec
1188
-
1189
- # Initialize
1190
- pc = Pinecone(api_key='your_api_key_here')
1191
-
1192
- # Create index
1193
- if not pc.has_index('example-index'):
1194
- pc.create_index(
1195
- name='example-index',
1196
- dimension=1536,
1197
- metric='cosine',
1198
- spec=ServerlessSpec(cloud='aws', region='us-west-2')
1199
- )
1200
-
1201
- # Connect to index
1202
- index_config = pc.describe_index('example-index')
1203
- index = pc.Index(host=index_config.host)
1204
-
1205
- # Upsert vectors
1206
- index.upsert(
1207
- vectors=[
1208
- ("doc1", [0.1] * 1536, {"title": "Document 1", "category": "A"}),
1209
- ("doc2", [0.2] * 1536, {"title": "Document 2", "category": "B"}),
1210
- ]
1211
- )
1212
-
1213
- # Query
1214
- results = index.query(
1215
- vector=[0.1] * 1536,
1216
- top_k=5,
1217
- filter={"category": {"$eq": "A"}},
1218
- include_metadata=True
1219
- )
1220
-
1221
- for match in results.matches:
1222
- print(f"ID: {match.id}, Score: {match.score}, Metadata: {match.metadata}")
1223
- ```
1224
-
1225
- ### Pattern: Multi-Tenant with Namespaces
1226
-
1227
- ```python
1228
- from pinecone import Pinecone
1229
-
1230
- pc = Pinecone(api_key='your_api_key_here')
1231
- index_config = pc.describe_index('multi-tenant-index')
1232
- index = pc.Index(host=index_config.host)
1233
-
1234
- # Tenant 1
1235
- index.upsert(
1236
- vectors=[("vec1", [0.1, 0.2, 0.3], {"user": "user1"})],
1237
- namespace='tenant-1'
1238
- )
1239
-
1240
- # Tenant 2
1241
- index.upsert(
1242
- vectors=[("vec1", [0.4, 0.5, 0.6], {"user": "user2"})],
1243
- namespace='tenant-2'
1244
- )
1245
-
1246
- # Query tenant-specific data
1247
- tenant1_results = index.query(
1248
- vector=[0.1, 0.2, 0.3],
1249
- top_k=5,
1250
- namespace='tenant-1'
1251
- )
1252
-
1253
- tenant2_results = index.query(
1254
- vector=[0.1, 0.2, 0.3],
1255
- top_k=5,
1256
- namespace='tenant-2'
1257
- )
1258
- ```
1259
-
1260
- ### Pattern: Hybrid Search with Integrated Inference
1261
-
1262
- ```python
1263
- from pinecone import Pinecone, ServerlessSpec
1264
-
1265
- pc = Pinecone(api_key='your_api_key_here')
1266
-
1267
- # Create hybrid index
1268
- if not pc.has_index('hybrid-index'):
1269
- pc.create_index(
1270
- name='hybrid-index',
1271
- dimension=1024,
1272
- metric='dotproduct',
1273
- spec=ServerlessSpec(cloud='aws', region='us-east-1')
1274
- )
1275
-
1276
- # Connect to index
1277
- index_config = pc.describe_index('hybrid-index')
1278
- index = pc.Index(host=index_config.host)
1279
-
1280
- # Prepare data
1281
- data = [
1282
- "Turkey is a classic meat to eat at American Thanksgiving.",
1283
- "Many people enjoy the beautiful mosques in Turkey."
1284
- ]
1285
-
1286
- # Generate dense embeddings
1287
- dense_embeddings = pc.inference.embed(
1288
- model="multilingual-e5-large",
1289
- inputs=data,
1290
- parameters={"input_type": "passage"}
1291
- )
1292
-
1293
- # Generate sparse embeddings
1294
- sparse_embeddings = pc.inference.embed(
1295
- model="pinecone-sparse-english-v0",
1296
- inputs=data,
1297
- parameters={"input_type": "passage"}
1298
- )
1299
-
1300
- # Upsert combined vectors
1301
- records = []
1302
- for i, (text, dense, sparse) in enumerate(zip(data, dense_embeddings, sparse_embeddings)):
1303
- records.append({
1304
- "id": f"doc{i}",
1305
- "values": dense['values'],
1306
- "sparse_values": {
1307
- "indices": sparse['indices'],
1308
- "values": sparse['values']
1309
- },
1310
- "metadata": {"text": text}
1311
- })
1312
-
1313
- index.upsert(vectors=records)
1314
-
1315
- # Query with hybrid search
1316
- query = "How should I prepare my turkey?"
1317
-
1318
- query_dense = pc.inference.embed(
1319
- model="multilingual-e5-large",
1320
- inputs=[query],
1321
- parameters={"input_type": "query"}
1322
- )
1323
-
1324
- query_sparse = pc.inference.embed(
1325
- model="pinecone-sparse-english-v0",
1326
- inputs=[query],
1327
- parameters={"input_type": "query"}
1328
- )
1329
-
1330
- # Apply alpha weighting (0.7 = 70% dense, 30% sparse)
1331
- alpha = 0.7
1332
-
1333
- results = index.query(
1334
- vector=[v * alpha for v in query_dense[0]['values']],
1335
- sparse_vector={
1336
- "indices": query_sparse[0]['indices'],
1337
- "values": [v * (1 - alpha) for v in query_sparse[0]['values']]
1338
- },
1339
- top_k=10,
1340
- include_metadata=True
1341
- )
1342
-
1343
- for match in results.matches:
1344
- print(f"Score: {match.score}, Text: {match.metadata['text']}")
1345
- ```
1346
-
1347
- ### Pattern: RAG Pipeline
1348
-
1349
- ```python
1350
- from pinecone import Pinecone
1351
-
1352
- pc = Pinecone(api_key='your_api_key_here')
1353
-
1354
- # Embed and upsert documents
1355
- documents = [
1356
- "Document 1 text content...",
1357
- "Document 2 text content...",
1358
- "Document 3 text content..."
1359
- ]
1360
-
1361
- embeddings = pc.inference.embed(
1362
- model="multilingual-e5-large",
1363
- inputs=documents,
1364
- parameters={"input_type": "passage"}
1365
- )
1366
-
1367
- index_config = pc.describe_index('rag-index')
1368
- index = pc.Index(host=index_config.host)
1369
-
1370
- vectors = [
1371
- (f"doc{i}", emb['values'], {"text": doc})
1372
- for i, (doc, emb) in enumerate(zip(documents, embeddings))
1373
- ]
1374
-
1375
- index.upsert(vectors=vectors)
1376
-
1377
- # Query and retrieve context
1378
- user_query = "What is the content about?"
1379
-
1380
- query_embedding = pc.inference.embed(
1381
- model="multilingual-e5-large",
1382
- inputs=[user_query],
1383
- parameters={"input_type": "query"}
1384
- )
1385
-
1386
- results = index.query(
1387
- vector=query_embedding[0]['values'],
1388
- top_k=3,
1389
- include_metadata=True
1390
- )
1391
-
1392
- # Extract context for LLM
1393
- context = "\n\n".join([match.metadata['text'] for match in results.matches])
1394
- print(f"Context for LLM:\n{context}")
1395
- ```