llm-gemini 0.2__tar.gz → 0.3__tar.gz

{llm_gemini-0.2 → llm_gemini-0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: llm-gemini
-Version: 0.2
+Version: 0.3
 Summary: LLM plugin to access Google's Gemini family of models
 Author: Simon Willison
 License: Apache-2.0
@@ -11,7 +11,7 @@ Project-URL: CI, https://github.com/simonw/llm-gemini/actions
 Classifier: License :: OSI Approved :: Apache Software License
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: llm
+Requires-Dist: llm>=0.17
 Requires-Dist: httpx
 Requires-Dist: ijson
 Provides-Extra: test
@@ -43,23 +43,66 @@ llm keys set gemini
 <paste key here>
 ```
 
-Now run the model using `-m gemini-pro`, for example:
+Now run the model using `-m gemini-1.5-pro-latest`, for example:
 
 ```bash
-llm -m gemini-pro "A joke about a pelican and a walrus"
+llm -m gemini-1.5-pro-latest "A joke about a pelican and a walrus"
 ```
 
-> Why did the pelican get mad at the walrus?
+> A pelican walks into a seafood restaurant with a huge fish hanging out of its beak.  The walrus, sitting at the bar, eyes it enviously.
 >
-> Because he called him a hippo-crit.
+> "Hey," the walrus says, "That looks delicious! What kind of fish is that?"
+>
+> The pelican taps its beak thoughtfully. "I believe," it says, "it's a billfish."
+
+### Images, audio and video
+
+Gemini models are multi-modal. You can provide images, audio or video files as input like this:
+
+```bash
+llm -m gemini-1.5-flash-latest 'extract text' -a image.jpg
+```
+Or with a URL:
+```bash
+llm -m gemini-1.5-flash-8b-latest 'describe image' \
+  -a https://static.simonwillison.net/static/2024/pelicans.jpg
+```
+Audio works too:
+
+```bash
+llm -m gemini-1.5-pro-latest 'transcribe audio' -a audio.mp3
+```
+
+And video:
+
+```bash
+llm -m gemini-1.5-pro-latest 'describe what happens' -a video.mp4
+```
+
+## Code execution
+
+Gemini models can [write and execute code](https://ai.google.dev/gemini-api/docs/code-execution) - they can decide to write Python code, execute it in a secure sandbox and use the result as part of their response.
+
+To enable this feature, use `-o code_execution 1`:
+
+```bash
+llm -m gemini-1.5-pro-latest -o code_execution 1 \
+'use python to calculate (factorial of 13) * 3'
+```
+
+### Chat
 
 To chat interactively with the model, run `llm chat`:
 
 ```bash
-llm chat -m gemini-pro
+llm chat -m gemini-1.5-pro-latest
 ```
 
-If you have access to the Gemini 1.5 Pro preview you can use `-m gemini-1.5-pro-latest` to work with that model.
+Other models are:
+
+- `gemini-1.5-flash-latest`
+- gemini-1.5-flash-8b-latest` - the least expensive
+
 
 ### Embeddings
 

{llm_gemini-0.2 → llm_gemini-0.3}/README.md

@@ -24,23 +24,66 @@ llm keys set gemini
 <paste key here>
 ```
 
-Now run the model using `-m gemini-pro`, for example:
+Now run the model using `-m gemini-1.5-pro-latest`, for example:
 
 ```bash
-llm -m gemini-pro "A joke about a pelican and a walrus"
+llm -m gemini-1.5-pro-latest "A joke about a pelican and a walrus"
 ```
 
-> Why did the pelican get mad at the walrus?
+> A pelican walks into a seafood restaurant with a huge fish hanging out of its beak.  The walrus, sitting at the bar, eyes it enviously.
 >
-> Because he called him a hippo-crit.
+> "Hey," the walrus says, "That looks delicious! What kind of fish is that?"
+>
+> The pelican taps its beak thoughtfully. "I believe," it says, "it's a billfish."
+
+### Images, audio and video
+
+Gemini models are multi-modal. You can provide images, audio or video files as input like this:
+
+```bash
+llm -m gemini-1.5-flash-latest 'extract text' -a image.jpg
+```
+Or with a URL:
+```bash
+llm -m gemini-1.5-flash-8b-latest 'describe image' \
+  -a https://static.simonwillison.net/static/2024/pelicans.jpg
+```
+Audio works too:
+
+```bash
+llm -m gemini-1.5-pro-latest 'transcribe audio' -a audio.mp3
+```
+
+And video:
+
+```bash
+llm -m gemini-1.5-pro-latest 'describe what happens' -a video.mp4
+```
+
+## Code execution
+
+Gemini models can [write and execute code](https://ai.google.dev/gemini-api/docs/code-execution) - they can decide to write Python code, execute it in a secure sandbox and use the result as part of their response.
+
+To enable this feature, use `-o code_execution 1`:
+
+```bash
+llm -m gemini-1.5-pro-latest -o code_execution 1 \
+'use python to calculate (factorial of 13) * 3'
+```
+
+### Chat
 
 To chat interactively with the model, run `llm chat`:
 
 ```bash
-llm chat -m gemini-pro
+llm chat -m gemini-1.5-pro-latest
 ```
 
-If you have access to the Gemini 1.5 Pro preview you can use `-m gemini-1.5-pro-latest` to work with that model.
+Other models are:
+
+- `gemini-1.5-flash-latest`
+- gemini-1.5-flash-8b-latest` - the least expensive
+
 
 ### Embeddings
 

{llm_gemini-0.2 → llm_gemini-0.3}/llm_gemini.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: llm-gemini
-Version: 0.2
+Version: 0.3
 Summary: LLM plugin to access Google's Gemini family of models
 Author: Simon Willison
 License: Apache-2.0
@@ -11,7 +11,7 @@ Project-URL: CI, https://github.com/simonw/llm-gemini/actions
 Classifier: License :: OSI Approved :: Apache Software License
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: llm
+Requires-Dist: llm>=0.17
 Requires-Dist: httpx
 Requires-Dist: ijson
 Provides-Extra: test
@@ -43,23 +43,66 @@ llm keys set gemini
 <paste key here>
 ```
 
-Now run the model using `-m gemini-pro`, for example:
+Now run the model using `-m gemini-1.5-pro-latest`, for example:
 
 ```bash
-llm -m gemini-pro "A joke about a pelican and a walrus"
+llm -m gemini-1.5-pro-latest "A joke about a pelican and a walrus"
 ```
 
-> Why did the pelican get mad at the walrus?
+> A pelican walks into a seafood restaurant with a huge fish hanging out of its beak.  The walrus, sitting at the bar, eyes it enviously.
 >
-> Because he called him a hippo-crit.
+> "Hey," the walrus says, "That looks delicious! What kind of fish is that?"
+>
+> The pelican taps its beak thoughtfully. "I believe," it says, "it's a billfish."
+
+### Images, audio and video
+
+Gemini models are multi-modal. You can provide images, audio or video files as input like this:
+
+```bash
+llm -m gemini-1.5-flash-latest 'extract text' -a image.jpg
+```
+Or with a URL:
+```bash
+llm -m gemini-1.5-flash-8b-latest 'describe image' \
+  -a https://static.simonwillison.net/static/2024/pelicans.jpg
+```
+Audio works too:
+
+```bash
+llm -m gemini-1.5-pro-latest 'transcribe audio' -a audio.mp3
+```
+
+And video:
+
+```bash
+llm -m gemini-1.5-pro-latest 'describe what happens' -a video.mp4
+```
+
+## Code execution
+
+Gemini models can [write and execute code](https://ai.google.dev/gemini-api/docs/code-execution) - they can decide to write Python code, execute it in a secure sandbox and use the result as part of their response.
+
+To enable this feature, use `-o code_execution 1`:
+
+```bash
+llm -m gemini-1.5-pro-latest -o code_execution 1 \
+'use python to calculate (factorial of 13) * 3'
+```
+
+### Chat
 
 To chat interactively with the model, run `llm chat`:
 
 ```bash
-llm chat -m gemini-pro
+llm chat -m gemini-1.5-pro-latest
 ```
 
-If you have access to the Gemini 1.5 Pro preview you can use `-m gemini-1.5-pro-latest` to work with that model.
+Other models are:
+
+- `gemini-1.5-flash-latest`
+- gemini-1.5-flash-8b-latest` - the least expensive
+
 
 ### Embeddings
 

{llm_gemini-0.2 → llm_gemini-0.3}/llm_gemini.egg-info/requires.txt

@@ -1,4 +1,4 @@
-llm
+llm>=0.17
 httpx
 ijson
 

llm_gemini-0.3/llm_gemini.py

@@ -0,0 +1,253 @@
+import httpx
+import ijson
+import llm
+from pydantic import Field
+from typing import Optional
+
+import urllib.parse
+
+# We disable all of these to avoid random unexpected errors
+SAFETY_SETTINGS = [
+    {
+        "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
+        "threshold": "BLOCK_NONE",
+    },
+    {
+        "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
+        "threshold": "BLOCK_NONE",
+    },
+    {
+        "category": "HARM_CATEGORY_HATE_SPEECH",
+        "threshold": "BLOCK_NONE",
+    },
+    {
+        "category": "HARM_CATEGORY_HARASSMENT",
+        "threshold": "BLOCK_NONE",
+    },
+]
+
+
+@llm.hookimpl
+def register_models(register):
+    register(GeminiPro("gemini-pro"))
+    register(GeminiPro("gemini-1.5-pro-latest"))
+    register(GeminiPro("gemini-1.5-flash-latest"))
+    register(GeminiPro("gemini-1.5-pro-001"))
+    register(GeminiPro("gemini-1.5-flash-001"))
+    register(GeminiPro("gemini-1.5-pro-002"))
+    register(GeminiPro("gemini-1.5-flash-002"))
+    register(GeminiPro("gemini-1.5-flash-8b-latest"))
+    register(GeminiPro("gemini-1.5-flash-8b-001"))
+
+
+def resolve_type(attachment):
+    mime_type = attachment.resolve_type()
+    # https://github.com/simonw/llm/issues/587#issuecomment-2439785140
+    if mime_type == "audio/mpeg":
+        mime_type = "audio/mp3"
+    return mime_type
+
+
+class GeminiPro(llm.Model):
+    needs_key = "gemini"
+    key_env_var = "LLM_GEMINI_KEY"
+    can_stream = True
+
+    attachment_types = (
+        # PDF
+        "application/pdf",
+        # Images
+        "image/png",
+        "image/jpeg",
+        "image/webp",
+        "image/heic",
+        "image/heif",
+        # Audio
+        "audio/wav",
+        "audio/mp3",
+        "audio/aiff",
+        "audio/aac",
+        "audio/ogg",
+        "audio/flac",
+        "audio/mpeg",  # Treated as audio/mp3
+        # Video
+        "video/mp4",
+        "video/mpeg",
+        "video/mov",
+        "video/avi",
+        "video/x-flv",
+        "video/mpg",
+        "video/webm",
+        "video/wmv",
+        "video/3gpp",
+    )
+
+    class Options(llm.Options):
+        code_execution: Optional[bool] = Field(
+            description="Enables the model to generate and run Python code",
+            default=None,
+        )
+        temperature: Optional[float] = Field(
+            description="Controls the randomness of the output. Use higher values for more creative responses, and lower values for more deterministic responses.",
+            default=None,
+            ge=0.0,
+            le=2.0,
+        )
+        max_output_tokens: Optional[int] = Field(
+            description="Sets the maximum number of tokens to include in a candidate.",
+            default=None,
+        )
+        top_p: Optional[float] = Field(
+            description="Changes how the model selects tokens for output. Tokens are selected from the most to least probable until the sum of their probabilities equals the topP value.",
+            default=None,
+            ge=0.0,
+            le=1.0,
+        )
+        top_k: Optional[int] = Field(
+            description="Changes how the model selects tokens for output. A topK of 1 means the selected token is the most probable among all the tokens in the model's vocabulary, while a topK of 3 means that the next token is selected from among the 3 most probable using the temperature.",
+            default=None,
+            ge=1,
+        )
+
+    def __init__(self, model_id):
+        self.model_id = model_id
+
+    def build_messages(self, prompt, conversation):
+        messages = []
+        if conversation:
+            for response in conversation.responses:
+                parts = []
+                for attachment in response.attachments:
+                    mime_type = resolve_type(attachment)
+                    parts.append(
+                        {
+                            "inlineData": {
+                                "data": attachment.base64_content(),
+                                "mimeType": mime_type,
+                            }
+                        }
+                    )
+                parts.append({"text": response.prompt.prompt})
+                messages.append({"role": "user", "parts": parts})
+                messages.append({"role": "model", "parts": [{"text": response.text()}]})
+
+        parts = [{"text": prompt.prompt}]
+        for attachment in prompt.attachments:
+            mime_type = resolve_type(attachment)
+            parts.append(
+                {
+                    "inlineData": {
+                        "data": attachment.base64_content(),
+                        "mimeType": mime_type,
+                    }
+                }
+            )
+
+        messages.append({"role": "user", "parts": parts})
+        return messages
+
+    def execute(self, prompt, stream, response, conversation):
+        key = self.get_key()
+        url = "https://generativelanguage.googleapis.com/v1beta/models/{}:streamGenerateContent?".format(
+            self.model_id
+        ) + urllib.parse.urlencode(
+            {"key": key}
+        )
+        gathered = []
+        body = {
+            "contents": self.build_messages(prompt, conversation),
+            "safetySettings": SAFETY_SETTINGS,
+        }
+        if prompt.options and prompt.options.code_execution:
+            body["tools"] = [{"codeExecution": {}}]
+        if prompt.system:
+            body["systemInstruction"] = {"parts": [{"text": prompt.system}]}
+
+        config_map = {
+            "temperature": "temperature",
+            "max_output_tokens": "maxOutputTokens",
+            "top_p": "topP",
+            "top_k": "topK",
+        }
+        # If any of those are set in prompt.options...
+        if any(
+            getattr(prompt.options, key, None) is not None for key in config_map.keys()
+        ):
+            generation_config = {}
+            for key, other_key in config_map.items():
+                config_value = getattr(prompt.options, key, None)
+                if config_value is not None:
+                    generation_config[other_key] = config_value
+            body["generationConfig"] = generation_config
+
+        with httpx.stream(
+            "POST",
+            url,
+            timeout=None,
+            json=body,
+        ) as http_response:
+            events = ijson.sendable_list()
+            coro = ijson.items_coro(events, "item")
+            for chunk in http_response.iter_bytes():
+                coro.send(chunk)
+                if events:
+                    event = events[0]
+                    if isinstance(event, dict) and "error" in event:
+                        raise llm.ModelError(event["error"]["message"])
+                    try:
+                        part = event["candidates"][0]["content"]["parts"][0]
+                        if "text" in part:
+                            yield part["text"]
+                        elif "executableCode" in part:
+                            # For code_execution
+                            yield f'```{part["executableCode"]["language"].lower()}\n{part["executableCode"]["code"].strip()}\n```\n'
+                        elif "codeExecutionResult" in part:
+                            # For code_execution
+                            yield f'```\n{part["codeExecutionResult"]["output"].strip()}\n```\n'
+                    except KeyError:
+                        yield ""
+                    gathered.append(event)
+                    events.clear()
+        response.response_json = gathered
+
+
+@llm.hookimpl
+def register_embedding_models(register):
+    register(
+        GeminiEmbeddingModel("text-embedding-004", "text-embedding-004"),
+    )
+
+
+class GeminiEmbeddingModel(llm.EmbeddingModel):
+    needs_key = "gemini"
+    key_env_var = "LLM_GEMINI_KEY"
+    batch_size = 20
+
+    def __init__(self, model_id, gemini_model_id):
+        self.model_id = model_id
+        self.gemini_model_id = gemini_model_id
+
+    def embed_batch(self, items):
+        headers = {
+            "Content-Type": "application/json",
+        }
+        data = {
+            "requests": [
+                {
+                    "model": "models/" + self.gemini_model_id,
+                    "content": {"parts": [{"text": item}]},
+                }
+                for item in items
+            ]
+        }
+
+        with httpx.Client() as client:
+            response = client.post(
+                f"https://generativelanguage.googleapis.com/v1beta/models/{self.gemini_model_id}:batchEmbedContents?key={self.get_key()}",
+                headers=headers,
+                json=data,
+                timeout=None,
+            )
+
+        response.raise_for_status()
+        return [item["values"] for item in response.json()["embeddings"]]

{llm_gemini-0.2 → llm_gemini-0.3}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "llm-gemini"
-version = "0.2"
+version = "0.3"
 description = "LLM plugin to access Google's Gemini family of models"
 readme = "README.md"
 authors = [{name = "Simon Willison"}]
@@ -9,7 +9,7 @@ classifiers = [
     "License :: OSI Approved :: Apache Software License"
 ]
 dependencies = [
-    "llm",
+    "llm>=0.17",
     "httpx",
     "ijson"
 ]

llm_gemini-0.2/llm_gemini.py

@@ -1,134 +0,0 @@
-import httpx
-import ijson
-import llm
-import urllib.parse
-
-# We disable all of these to avoid random unexpected errors
-SAFETY_SETTINGS = [
-    {
-        "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
-        "threshold": "BLOCK_NONE",
-    },
-    {
-        "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
-        "threshold": "BLOCK_NONE",
-    },
-    {
-        "category": "HARM_CATEGORY_HATE_SPEECH",
-        "threshold": "BLOCK_NONE",
-    },
-    {
-        "category": "HARM_CATEGORY_HARASSMENT",
-        "threshold": "BLOCK_NONE",
-    },
-]
-
-
-@llm.hookimpl
-def register_models(register):
-    register(GeminiPro("gemini-pro"))
-    register(GeminiPro("gemini-1.5-pro-latest"))
-    register(GeminiPro("gemini-1.5-flash-latest"))
-    register(GeminiPro("gemini-1.5-pro-001"))
-    register(GeminiPro("gemini-1.5-flash-001"))
-    register(GeminiPro("gemini-1.5-pro-002"))
-    register(GeminiPro("gemini-1.5-flash-002"))
-    register(GeminiPro("gemini-1.5-flash-8b-latest"))
-    register(GeminiPro("gemini-1.5-flash-8b-001"))
-
-
-class GeminiPro(llm.Model):
-    can_stream = True
-
-    def __init__(self, model_id):
-        self.model_id = model_id
-
-    def build_messages(self, prompt, conversation):
-        if not conversation:
-            return [{"role": "user", "parts": [{"text": prompt.prompt}]}]
-        messages = []
-        for response in conversation.responses:
-            messages.append(
-                {"role": "user", "parts": [{"text": response.prompt.prompt}]}
-            )
-            messages.append({"role": "model", "parts": [{"text": response.text()}]})
-        messages.append({"role": "user", "parts": [{"text": prompt.prompt}]})
-        return messages
-
-    def execute(self, prompt, stream, response, conversation):
-        key = llm.get_key("", "gemini", "LLM_GEMINI_KEY")
-        url = "https://generativelanguage.googleapis.com/v1beta/models/{}:streamGenerateContent?".format(
-            self.model_id
-        ) + urllib.parse.urlencode(
-            {"key": key}
-        )
-        gathered = []
-        body = {
-            "contents": self.build_messages(prompt, conversation),
-            "safetySettings": SAFETY_SETTINGS,
-        }
-        if prompt.system:
-            body["systemInstruction"] = {"parts": [{"text": prompt.system}]}
-        with httpx.stream(
-            "POST",
-            url,
-            timeout=None,
-            json=body,
-        ) as http_response:
-            events = ijson.sendable_list()
-            coro = ijson.items_coro(events, "item")
-            for chunk in http_response.iter_bytes():
-                coro.send(chunk)
-                if events:
-                    event = events[0]
-                    if isinstance(event, dict) and "error" in event:
-                        raise llm.ModelError(event["error"]["message"])
-                    try:
-                        yield event["candidates"][0]["content"]["parts"][0]["text"]
-                    except KeyError:
-                        yield ""
-                    gathered.append(event)
-                    events.clear()
-        response.response_json = gathered
-
-
-@llm.hookimpl
-def register_embedding_models(register):
-    register(
-        GeminiEmbeddingModel("text-embedding-004", "text-embedding-004"),
-    )
-
-
-class GeminiEmbeddingModel(llm.EmbeddingModel):
-    needs_key = "gemini"
-    key_env_var = "LLM_GEMINI_KEY"
-    batch_size = 20
-
-    def __init__(self, model_id, gemini_model_id):
-        self.model_id = model_id
-        self.gemini_model_id = gemini_model_id
-
-    def embed_batch(self, items):
-        headers = {
-            "Content-Type": "application/json",
-        }
-        data = {
-            "requests": [
-                {
-                    "model": "models/" + self.gemini_model_id,
-                    "content": {"parts": [{"text": item}]},
-                }
-                for item in items
-            ]
-        }
-
-        with httpx.Client() as client:
-            response = client.post(
-                f"https://generativelanguage.googleapis.com/v1beta/models/{self.gemini_model_id}:batchEmbedContents?key={self.get_key()}",
-                headers=headers,
-                json=data,
-                timeout=None,
-            )
-
-        response.raise_for_status()
-        return [item["values"] for item in response.json()["embeddings"]]