llm-gemini 0.3a0__tar.gz → 0.4.1__tar.gz

{llm_gemini-0.3a0 → llm_gemini-0.4.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: llm-gemini
-Version: 0.3a0
+Version: 0.4.1
 Summary: LLM plugin to access Google's Gemini family of models
 Author: Simon Willison
 License: Apache-2.0
@@ -11,11 +11,12 @@ Project-URL: CI, https://github.com/simonw/llm-gemini/actions
 Classifier: License :: OSI Approved :: Apache Software License
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: llm>=0.17a0
+Requires-Dist: llm>=0.18
 Requires-Dist: httpx
 Requires-Dist: ijson
 Provides-Extra: test
 Requires-Dist: pytest; extra == "test"
+Requires-Dist: pytest-recording; extra == "test"
 
 # llm-gemini
 
@@ -35,13 +36,13 @@ llm install llm-gemini
 ## Usage
 
 Configure the model by setting a key called "gemini" to your [API key](https://aistudio.google.com/app/apikey):
-
 ```bash
 llm keys set gemini
 ```
 ```
 <paste key here>
 ```
+You can also set the API key by assigning it to the environment variable `LLM_GEMINI_KEY`.
 
 Now run the model using `-m gemini-1.5-pro-latest`, for example:
 
@@ -55,16 +56,13 @@ llm -m gemini-1.5-pro-latest "A joke about a pelican and a walrus"
 >
 > The pelican taps its beak thoughtfully. "I believe," it says, "it's a billfish."
 
-To chat interactively with the model, run `llm chat`:
-
-```bash
-llm chat -m gemini-1.5-pro-latest
-```
-
 Other models are:
 
 - `gemini-1.5-flash-latest`
-- gemini-1.5-flash-8b-latest` - the least expensive
+- `gemini-1.5-flash-8b-latest` - the least expensive
+- `gemini-exp-1114` - recent experimental
+
+### Images, audio and video
 
 Gemini models are multi-modal. You can provide images, audio or video files as input like this:
 
@@ -76,8 +74,52 @@ Or with a URL:
 llm -m gemini-1.5-flash-8b-latest 'describe image' \
   -a https://static.simonwillison.net/static/2024/pelicans.jpg
 ```
+Audio works too:
+
+```bash
+llm -m gemini-1.5-pro-latest 'transcribe audio' -a audio.mp3
+```
+
+And video:
+
+```bash
+llm -m gemini-1.5-pro-latest 'describe what happens' -a video.mp4
+```
+The Gemini prompting guide includes [extensive advice](https://ai.google.dev/gemini-api/docs/file-prompting-strategies) on multi-modal prompting.
+
+### JSON output
+
+Use `-o json_object 1` to force the output to be JSON:
+
+```bash
+llm -m gemini-1.5-flash-latest -o json_object 1 \
+  '3 largest cities in California, list of {"name": "..."}'
+```
+Outputs:
+```json
+{"cities": [{"name": "Los Angeles"}, {"name": "San Diego"}, {"name": "San Jose"}]}
+```
+
+### Code execution
+
+Gemini models can [write and execute code](https://ai.google.dev/gemini-api/docs/code-execution) - they can decide to write Python code, execute it in a secure sandbox and use the result as part of their response.
+
+To enable this feature, use `-o code_execution 1`:
+
+```bash
+llm -m gemini-1.5-pro-latest -o code_execution 1 \
+'use python to calculate (factorial of 13) * 3'
+```
+
+### Chat
+
+To chat interactively with the model, run `llm chat`:
+
+```bash
+llm chat -m gemini-1.5-pro-latest
+```
 
-### Embeddings
+## Embeddings
 
 The plugin also adds support for the `text-embedding-004` embedding model.
 

{llm_gemini-0.3a0 → llm_gemini-0.4.1}/README.md

@@ -16,13 +16,13 @@ llm install llm-gemini
 ## Usage
 
 Configure the model by setting a key called "gemini" to your [API key](https://aistudio.google.com/app/apikey):
-
 ```bash
 llm keys set gemini
 ```
 ```
 <paste key here>
 ```
+You can also set the API key by assigning it to the environment variable `LLM_GEMINI_KEY`.
 
 Now run the model using `-m gemini-1.5-pro-latest`, for example:
 
@@ -36,16 +36,13 @@ llm -m gemini-1.5-pro-latest "A joke about a pelican and a walrus"
 >
 > The pelican taps its beak thoughtfully. "I believe," it says, "it's a billfish."
 
-To chat interactively with the model, run `llm chat`:
-
-```bash
-llm chat -m gemini-1.5-pro-latest
-```
-
 Other models are:
 
 - `gemini-1.5-flash-latest`
-- gemini-1.5-flash-8b-latest` - the least expensive
+- `gemini-1.5-flash-8b-latest` - the least expensive
+- `gemini-exp-1114` - recent experimental
+
+### Images, audio and video
 
 Gemini models are multi-modal. You can provide images, audio or video files as input like this:
 
@@ -57,8 +54,52 @@ Or with a URL:
 llm -m gemini-1.5-flash-8b-latest 'describe image' \
   -a https://static.simonwillison.net/static/2024/pelicans.jpg
 ```
+Audio works too:
+
+```bash
+llm -m gemini-1.5-pro-latest 'transcribe audio' -a audio.mp3
+```
+
+And video:
+
+```bash
+llm -m gemini-1.5-pro-latest 'describe what happens' -a video.mp4
+```
+The Gemini prompting guide includes [extensive advice](https://ai.google.dev/gemini-api/docs/file-prompting-strategies) on multi-modal prompting.
+
+### JSON output
+
+Use `-o json_object 1` to force the output to be JSON:
+
+```bash
+llm -m gemini-1.5-flash-latest -o json_object 1 \
+  '3 largest cities in California, list of {"name": "..."}'
+```
+Outputs:
+```json
+{"cities": [{"name": "Los Angeles"}, {"name": "San Diego"}, {"name": "San Jose"}]}
+```
+
+### Code execution
+
+Gemini models can [write and execute code](https://ai.google.dev/gemini-api/docs/code-execution) - they can decide to write Python code, execute it in a secure sandbox and use the result as part of their response.
+
+To enable this feature, use `-o code_execution 1`:
+
+```bash
+llm -m gemini-1.5-pro-latest -o code_execution 1 \
+'use python to calculate (factorial of 13) * 3'
+```
+
+### Chat
+
+To chat interactively with the model, run `llm chat`:
+
+```bash
+llm chat -m gemini-1.5-pro-latest
+```
 
-### Embeddings
+## Embeddings
 
 The plugin also adds support for the `text-embedding-004` embedding model.
 

{llm_gemini-0.3a0 → llm_gemini-0.4.1}/llm_gemini.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: llm-gemini
-Version: 0.3a0
+Version: 0.4.1
 Summary: LLM plugin to access Google's Gemini family of models
 Author: Simon Willison
 License: Apache-2.0
@@ -11,11 +11,12 @@ Project-URL: CI, https://github.com/simonw/llm-gemini/actions
 Classifier: License :: OSI Approved :: Apache Software License
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: llm>=0.17a0
+Requires-Dist: llm>=0.18
 Requires-Dist: httpx
 Requires-Dist: ijson
 Provides-Extra: test
 Requires-Dist: pytest; extra == "test"
+Requires-Dist: pytest-recording; extra == "test"
 
 # llm-gemini
 
@@ -35,13 +36,13 @@ llm install llm-gemini
 ## Usage
 
 Configure the model by setting a key called "gemini" to your [API key](https://aistudio.google.com/app/apikey):
-
 ```bash
 llm keys set gemini
 ```
 ```
 <paste key here>
 ```
+You can also set the API key by assigning it to the environment variable `LLM_GEMINI_KEY`.
 
 Now run the model using `-m gemini-1.5-pro-latest`, for example:
 
@@ -55,16 +56,13 @@ llm -m gemini-1.5-pro-latest "A joke about a pelican and a walrus"
 >
 > The pelican taps its beak thoughtfully. "I believe," it says, "it's a billfish."
 
-To chat interactively with the model, run `llm chat`:
-
-```bash
-llm chat -m gemini-1.5-pro-latest
-```
-
 Other models are:
 
 - `gemini-1.5-flash-latest`
-- gemini-1.5-flash-8b-latest` - the least expensive
+- `gemini-1.5-flash-8b-latest` - the least expensive
+- `gemini-exp-1114` - recent experimental
+
+### Images, audio and video
 
 Gemini models are multi-modal. You can provide images, audio or video files as input like this:
 
@@ -76,8 +74,52 @@ Or with a URL:
 llm -m gemini-1.5-flash-8b-latest 'describe image' \
   -a https://static.simonwillison.net/static/2024/pelicans.jpg
 ```
+Audio works too:
+
+```bash
+llm -m gemini-1.5-pro-latest 'transcribe audio' -a audio.mp3
+```
+
+And video:
+
+```bash
+llm -m gemini-1.5-pro-latest 'describe what happens' -a video.mp4
+```
+The Gemini prompting guide includes [extensive advice](https://ai.google.dev/gemini-api/docs/file-prompting-strategies) on multi-modal prompting.
+
+### JSON output
+
+Use `-o json_object 1` to force the output to be JSON:
+
+```bash
+llm -m gemini-1.5-flash-latest -o json_object 1 \
+  '3 largest cities in California, list of {"name": "..."}'
+```
+Outputs:
+```json
+{"cities": [{"name": "Los Angeles"}, {"name": "San Diego"}, {"name": "San Jose"}]}
+```
+
+### Code execution
+
+Gemini models can [write and execute code](https://ai.google.dev/gemini-api/docs/code-execution) - they can decide to write Python code, execute it in a secure sandbox and use the result as part of their response.
+
+To enable this feature, use `-o code_execution 1`:
+
+```bash
+llm -m gemini-1.5-pro-latest -o code_execution 1 \
+'use python to calculate (factorial of 13) * 3'
+```
+
+### Chat
+
+To chat interactively with the model, run `llm chat`:
+
+```bash
+llm chat -m gemini-1.5-pro-latest
+```
 
-### Embeddings
+## Embeddings
 
 The plugin also adds support for the `text-embedding-004` embedding model.
 

{llm_gemini-0.3a0 → llm_gemini-0.4.1}/llm_gemini.egg-info/requires.txt

@@ -1,6 +1,7 @@
-llm>=0.17a0
+llm>=0.18
 httpx
 ijson
 
 [test]
 pytest
+pytest-recording

llm_gemini-0.4.1/llm_gemini.py

@@ -0,0 +1,319 @@
+import httpx
+import ijson
+import llm
+from pydantic import Field
+from typing import Optional
+
+SAFETY_SETTINGS = [
+    {
+        "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
+        "threshold": "BLOCK_NONE",
+    },
+    {
+        "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
+        "threshold": "BLOCK_NONE",
+    },
+    {
+        "category": "HARM_CATEGORY_HATE_SPEECH",
+        "threshold": "BLOCK_NONE",
+    },
+    {
+        "category": "HARM_CATEGORY_HARASSMENT",
+        "threshold": "BLOCK_NONE",
+    },
+]
+
+
+@llm.hookimpl
+def register_models(register):
+    # Register both sync and async versions of each model
+    for model_id in [
+        "gemini-pro",
+        "gemini-1.5-pro-latest",
+        "gemini-1.5-flash-latest",
+        "gemini-1.5-pro-001",
+        "gemini-1.5-flash-001",
+        "gemini-1.5-pro-002",
+        "gemini-1.5-flash-002",
+        "gemini-1.5-flash-8b-latest",
+        "gemini-1.5-flash-8b-001",
+        "gemini-exp-1114",
+    ]:
+        register(GeminiPro(model_id), AsyncGeminiPro(model_id))
+
+
+def resolve_type(attachment):
+    mime_type = attachment.resolve_type()
+    # https://github.com/simonw/llm/issues/587#issuecomment-2439785140
+    if mime_type == "audio/mpeg":
+        mime_type = "audio/mp3"
+    return mime_type
+
+
+class _SharedGemini:
+    needs_key = "gemini"
+    key_env_var = "LLM_GEMINI_KEY"
+    can_stream = True
+
+    attachment_types = (
+        # PDF
+        "application/pdf",
+        # Images
+        "image/png",
+        "image/jpeg",
+        "image/webp",
+        "image/heic",
+        "image/heif",
+        # Audio
+        "audio/wav",
+        "audio/mp3",
+        "audio/aiff",
+        "audio/aac",
+        "audio/ogg",
+        "audio/flac",
+        "audio/mpeg",  # Treated as audio/mp3
+        # Video
+        "video/mp4",
+        "video/mpeg",
+        "video/mov",
+        "video/avi",
+        "video/x-flv",
+        "video/mpg",
+        "video/webm",
+        "video/wmv",
+        "video/3gpp",
+        "video/quicktime",
+    )
+
+    class Options(llm.Options):
+        code_execution: Optional[bool] = Field(
+            description="Enables the model to generate and run Python code",
+            default=None,
+        )
+        temperature: Optional[float] = Field(
+            description=(
+                "Controls the randomness of the output. Use higher values for "
+                "more creative responses, and lower values for more "
+                "deterministic responses."
+            ),
+            default=None,
+            ge=0.0,
+            le=2.0,
+        )
+        max_output_tokens: Optional[int] = Field(
+            description="Sets the maximum number of tokens to include in a candidate.",
+            default=None,
+        )
+        top_p: Optional[float] = Field(
+            description=(
+                "Changes how the model selects tokens for output. Tokens are "
+                "selected from the most to least probable until the sum of "
+                "their probabilities equals the topP value."
+            ),
+            default=None,
+            ge=0.0,
+            le=1.0,
+        )
+        top_k: Optional[int] = Field(
+            description=(
+                "Changes how the model selects tokens for output. A topK of 1 "
+                "means the selected token is the most probable among all the "
+                "tokens in the model's vocabulary, while a topK of 3 means "
+                "that the next token is selected from among the 3 most "
+                "probable using the temperature."
+            ),
+            default=None,
+            ge=1,
+        )
+        json_object: Optional[bool] = Field(
+            description="Output a valid JSON object {...}",
+            default=None,
+        )
+
+    def __init__(self, model_id):
+        self.model_id = model_id
+
+    def build_messages(self, prompt, conversation):
+        messages = []
+        if conversation:
+            for response in conversation.responses:
+                parts = []
+                for attachment in response.attachments:
+                    mime_type = resolve_type(attachment)
+                    parts.append(
+                        {
+                            "inlineData": {
+                                "data": attachment.base64_content(),
+                                "mimeType": mime_type,
+                            }
+                        }
+                    )
+                if response.prompt.prompt:
+                    parts.append({"text": response.prompt.prompt})
+                messages.append({"role": "user", "parts": parts})
+                messages.append({"role": "model", "parts": [{"text": response.text()}]})
+
+        parts = []
+        if prompt.prompt:
+            parts.append({"text": prompt.prompt})
+        for attachment in prompt.attachments:
+            mime_type = resolve_type(attachment)
+            parts.append(
+                {
+                    "inlineData": {
+                        "data": attachment.base64_content(),
+                        "mimeType": mime_type,
+                    }
+                }
+            )
+
+        messages.append({"role": "user", "parts": parts})
+        return messages
+
+    def build_request_body(self, prompt, conversation):
+        body = {
+            "contents": self.build_messages(prompt, conversation),
+            "safetySettings": SAFETY_SETTINGS,
+        }
+        if prompt.options and prompt.options.code_execution:
+            body["tools"] = [{"codeExecution": {}}]
+        if prompt.system:
+            body["systemInstruction"] = {"parts": [{"text": prompt.system}]}
+
+        config_map = {
+            "temperature": "temperature",
+            "max_output_tokens": "maxOutputTokens",
+            "top_p": "topP",
+            "top_k": "topK",
+        }
+        if prompt.options and prompt.options.json_object:
+            body["generationConfig"] = {"response_mime_type": "application/json"}
+
+        if any(
+            getattr(prompt.options, key, None) is not None for key in config_map.keys()
+        ):
+            generation_config = {}
+            for key, other_key in config_map.items():
+                config_value = getattr(prompt.options, key, None)
+                if config_value is not None:
+                    generation_config[other_key] = config_value
+            body["generationConfig"] = generation_config
+
+        return body
+
+    def process_part(self, part):
+        if "text" in part:
+            return part["text"]
+        elif "executableCode" in part:
+            return f'```{part["executableCode"]["language"].lower()}\n{part["executableCode"]["code"].strip()}\n```\n'
+        elif "codeExecutionResult" in part:
+            return f'```\n{part["codeExecutionResult"]["output"].strip()}\n```\n'
+        return ""
+
+
+class GeminiPro(_SharedGemini, llm.Model):
+    def execute(self, prompt, stream, response, conversation):
+        key = self.get_key()
+        url = f"https://generativelanguage.googleapis.com/v1beta/models/{self.model_id}:streamGenerateContent"
+        gathered = []
+        body = self.build_request_body(prompt, conversation)
+
+        with httpx.stream(
+            "POST",
+            url,
+            timeout=None,
+            headers={"x-goog-api-key": key},
+            json=body,
+        ) as http_response:
+            events = ijson.sendable_list()
+            coro = ijson.items_coro(events, "item")
+            for chunk in http_response.iter_bytes():
+                coro.send(chunk)
+                if events:
+                    event = events[0]
+                    if isinstance(event, dict) and "error" in event:
+                        raise llm.ModelError(event["error"]["message"])
+                    try:
+                        part = event["candidates"][0]["content"]["parts"][0]
+                        yield self.process_part(part)
+                    except KeyError:
+                        yield ""
+                    gathered.append(event)
+                    events.clear()
+        response.response_json = gathered
+
+
+class AsyncGeminiPro(_SharedGemini, llm.AsyncModel):
+    async def execute(self, prompt, stream, response, conversation):
+        key = self.get_key()
+        url = f"https://generativelanguage.googleapis.com/v1beta/models/{self.model_id}:streamGenerateContent"
+        gathered = []
+        body = self.build_request_body(prompt, conversation)
+
+        async with httpx.AsyncClient() as client:
+            async with client.stream(
+                "POST",
+                url,
+                timeout=None,
+                headers={"x-goog-api-key": key},
+                json=body,
+            ) as http_response:
+                events = ijson.sendable_list()
+                coro = ijson.items_coro(events, "item")
+                async for chunk in http_response.aiter_bytes():
+                    coro.send(chunk)
+                    if events:
+                        event = events[0]
+                        if isinstance(event, dict) and "error" in event:
+                            raise llm.ModelError(event["error"]["message"])
+                        try:
+                            part = event["candidates"][0]["content"]["parts"][0]
+                            yield self.process_part(part)
+                        except KeyError:
+                            yield ""
+                        gathered.append(event)
+                        events.clear()
+        response.response_json = gathered
+
+
+@llm.hookimpl
+def register_embedding_models(register):
+    register(
+        GeminiEmbeddingModel("text-embedding-004", "text-embedding-004"),
+    )
+
+
+class GeminiEmbeddingModel(llm.EmbeddingModel):
+    needs_key = "gemini"
+    key_env_var = "LLM_GEMINI_KEY"
+    batch_size = 20
+
+    def __init__(self, model_id, gemini_model_id):
+        self.model_id = model_id
+        self.gemini_model_id = gemini_model_id
+
+    def embed_batch(self, items):
+        headers = {
+            "Content-Type": "application/json",
+            "x-goog-api-key": self.get_key(),
+        }
+        data = {
+            "requests": [
+                {
+                    "model": "models/" + self.gemini_model_id,
+                    "content": {"parts": [{"text": item}]},
+                }
+                for item in items
+            ]
+        }
+
+        with httpx.Client() as client:
+            response = client.post(
+                f"https://generativelanguage.googleapis.com/v1beta/models/{self.gemini_model_id}:batchEmbedContents",
+                headers=headers,
+                json=data,
+                timeout=None,
+            )
+
+        response.raise_for_status()
+        return [item["values"] for item in response.json()["embeddings"]]

{llm_gemini-0.3a0 → llm_gemini-0.4.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "llm-gemini"
-version = "0.3a0"
+version = "0.4.1"
 description = "LLM plugin to access Google's Gemini family of models"
 readme = "README.md"
 authors = [{name = "Simon Willison"}]
@@ -9,7 +9,7 @@ classifiers = [
     "License :: OSI Approved :: Apache Software License"
 ]
 dependencies = [
-    "llm>=0.17a0",
+    "llm>=0.18",
     "httpx",
     "ijson"
 ]
@@ -24,4 +24,4 @@ CI = "https://github.com/simonw/llm-gemini/actions"
 gemini = "llm_gemini"
 
 [project.optional-dependencies]
-test = ["pytest"]
+test = ["pytest", "pytest-recording"]

llm_gemini-0.4.1/tests/test_gemini.py

@@ -0,0 +1,29 @@
+import llm
+import os
+import pytest
+
+GEMINI_API_KEY = os.environ.get("PYTEST_GEMINI_API_KEY", None) or "gm-..."
+
+
+@pytest.mark.vcr
+@pytest.mark.asyncio
+async def test_prompt():
+    model = llm.get_model("gemini-1.5-flash-latest")
+    model.key = model.key or GEMINI_API_KEY
+    response = model.prompt("Name for a pet pelican, just the name")
+    assert str(response) == "Percy"
+    assert response.response_json == [
+        {
+            "candidates": [
+                {"content": {"parts": [{"text": "Percy"}], "role": "model"}}
+            ],
+            "usageMetadata": {"promptTokenCount": 10, "totalTokenCount": 10},
+            "modelVersion": "gemini-1.5-flash-002",
+        }
+    ]
+    # And try it async too
+    async_model = llm.get_async_model("gemini-1.5-flash-latest")
+    async_model.key = async_model.key or GEMINI_API_KEY
+    response = await async_model.prompt("Name for a pet pelican, just the name")
+    text = await response.text()
+    assert text == "Percy"

llm_gemini-0.3a0/llm_gemini.py

@@ -1,193 +0,0 @@
-import httpx
-import ijson
-import llm
-import urllib.parse
-
-# We disable all of these to avoid random unexpected errors
-SAFETY_SETTINGS = [
-    {
-        "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
-        "threshold": "BLOCK_NONE",
-    },
-    {
-        "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
-        "threshold": "BLOCK_NONE",
-    },
-    {
-        "category": "HARM_CATEGORY_HATE_SPEECH",
-        "threshold": "BLOCK_NONE",
-    },
-    {
-        "category": "HARM_CATEGORY_HARASSMENT",
-        "threshold": "BLOCK_NONE",
-    },
-]
-
-
-@llm.hookimpl
-def register_models(register):
-    register(GeminiPro("gemini-pro"))
-    register(GeminiPro("gemini-1.5-pro-latest"))
-    register(GeminiPro("gemini-1.5-flash-latest"))
-    register(GeminiPro("gemini-1.5-pro-001"))
-    register(GeminiPro("gemini-1.5-flash-001"))
-    register(GeminiPro("gemini-1.5-pro-002"))
-    register(GeminiPro("gemini-1.5-flash-002"))
-    register(GeminiPro("gemini-1.5-flash-8b-latest"))
-    register(GeminiPro("gemini-1.5-flash-8b-001"))
-
-
-def resolve_type(attachment):
-    mime_type = attachment.resolve_type()
-    # https://github.com/simonw/llm/issues/587#issuecomment-2439785140
-    if mime_type == "audio/mpeg":
-        mime_type = "audio/mp3"
-    return mime_type
-
-
-class GeminiPro(llm.Model):
-    can_stream = True
-
-    attachment_types = (
-        # PDF
-        "application/pdf",
-        # Images
-        "image/png",
-        "image/jpeg",
-        "image/webp",
-        "image/heic",
-        "image/heif",
-        # Audio
-        "audio/wav",
-        "audio/mp3",
-        "audio/aiff",
-        "audio/aac",
-        "audio/ogg",
-        "audio/flac",
-        "audio/mpeg",  # Treated as audio/mp3
-        # Video
-        "video/mp4",
-        "video/mpeg",
-        "video/mov",
-        "video/avi",
-        "video/x-flv",
-        "video/mpg",
-        "video/webm",
-        "video/wmv",
-        "video/3gpp",
-    )
-
-    def __init__(self, model_id):
-        self.model_id = model_id
-
-    def build_messages(self, prompt, conversation):
-        messages = []
-        if conversation:
-            for response in conversation.responses:
-                parts = []
-                for attachment in response.attachments:
-                    mime_type = resolve_type(attachment)
-                    parts.append(
-                        {
-                            "inlineData": {
-                                "data": attachment.base64_content(),
-                                "mimeType": mime_type,
-                            }
-                        }
-                    )
-                parts.append({"text": response.prompt.prompt})
-                messages.append({"role": "user", "parts": parts})
-                messages.append({"role": "model", "parts": [{"text": response.text()}]})
-
-        parts = [{"text": prompt.prompt}]
-        for attachment in prompt.attachments:
-            mime_type = resolve_type(attachment)
-            parts.append(
-                {
-                    "inlineData": {
-                        "data": attachment.base64_content(),
-                        "mimeType": mime_type,
-                    }
-                }
-            )
-
-        messages.append({"role": "user", "parts": parts})
-        return messages
-
-    def execute(self, prompt, stream, response, conversation):
-        key = llm.get_key("", "gemini", "LLM_GEMINI_KEY")
-        url = "https://generativelanguage.googleapis.com/v1beta/models/{}:streamGenerateContent?".format(
-            self.model_id
-        ) + urllib.parse.urlencode(
-            {"key": key}
-        )
-        gathered = []
-        body = {
-            "contents": self.build_messages(prompt, conversation),
-            "safetySettings": SAFETY_SETTINGS,
-        }
-        if prompt.system:
-            body["systemInstruction"] = {"parts": [{"text": prompt.system}]}
-        with httpx.stream(
-            "POST",
-            url,
-            timeout=None,
-            json=body,
-        ) as http_response:
-            events = ijson.sendable_list()
-            coro = ijson.items_coro(events, "item")
-            for chunk in http_response.iter_bytes():
-                coro.send(chunk)
-                if events:
-                    event = events[0]
-                    if isinstance(event, dict) and "error" in event:
-                        raise llm.ModelError(event["error"]["message"])
-                    try:
-                        yield event["candidates"][0]["content"]["parts"][0]["text"]
-                    except KeyError:
-                        yield ""
-                    gathered.append(event)
-                    events.clear()
-        response.response_json = gathered
-
-
-@llm.hookimpl
-def register_embedding_models(register):
-    register(
-        GeminiEmbeddingModel("text-embedding-004", "text-embedding-004"),
-    )
-
-
-class GeminiEmbeddingModel(llm.EmbeddingModel):
-    needs_key = "gemini"
-    key_env_var = "LLM_GEMINI_KEY"
-    batch_size = 20
-
-    def __init__(self, model_id, gemini_model_id):
-        self.model_id = model_id
-        self.gemini_model_id = gemini_model_id
-
-    def embed_batch(self, items):
-        headers = {
-            "Content-Type": "application/json",
-        }
-        data = {
-            "requests": [
-                {
-                    "model": "models/" + self.gemini_model_id,
-                    "content": {"parts": [{"text": item}]},
-                }
-                for item in items
-            ]
-        }
-
-        with httpx.Client() as client:
-            response = client.post(
-                f"https://generativelanguage.googleapis.com/v1beta/models/{self.gemini_model_id}:batchEmbedContents?key={self.get_key()}",
-                headers=headers,
-                json=data,
-                timeout=None,
-            )
-
-        response.raise_for_status()
-        return [item["values"] for item in response.json()["embeddings"]]

llm_gemini-0.3a0/tests/test_gemini.py

@@ -1,6 +0,0 @@
-from llm.plugins import pm
-
-
-def test_plugin_is_installed():
-    names = [mod.__name__ for mod in pm.get_plugins()]
-    assert "llm_gemini" in names