gemini-calo 0.1.0__tar.gz

gemini_calo-0.1.0/PKG-INFO

@@ -0,0 +1,190 @@
+Metadata-Version: 2.1
+Name: gemini-calo
+Version: 0.1.0
+Summary: A Fastapi Based Proxy for Gemini API
+Home-page: https://github.com/state-alchemists/gemini-calo
+License: AGPL-3.0-or-later
+Author: Go Frendi Gunawan
+Author-email: gofrendiasgard@gmail.com
+Requires-Python: >=3.10,<4.0.0
+Classifier: License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Dist: fastapi (>=0.116.1,<0.117.0)
+Requires-Dist: uvicorn (>=0.35.0,<0.36.0)
+Project-URL: Documentation, https://github.com/state-alchemists/gemini-calo
+Project-URL: Repository, https://github.com/state-alchemists/gemini-calo
+Description-Content-Type: text/markdown
+
+# Gemini Calo
+
+**Gemini Calo** is a powerful, yet simple, FastAPI-based proxy server for Google's Gemini API. It provides a seamless way to add a layer of authentication, logging, and monitoring to your Gemini API requests. It's designed to be run as a standalone server or integrated into your existing FastAPI applications.
+
+One of its key features is providing an OpenAI-compatible endpoint, allowing you to use Gemini models with tools and libraries that are built for the OpenAI API.
+
+## Key Features
+
+*   **Authentication:** Secure your Gemini API access with an additional layer of API key authentication.
+*   **Request Logging:** Detailed logging of all incoming requests and outgoing responses.
+*   **OpenAI Compatibility:** Use Gemini models through an OpenAI-compatible `/v1/chat/completions` endpoint.
+*   **Round-Robin API Keys:** Distribute your requests across multiple Gemini API keys.
+*   **Easy Integration:** Use it as a standalone server or mount it into your existing FastAPI project.
+*   **Extensible:** Easily add your own custom middleware to suit your needs.
+
+## How It's Useful
+
+-   **Centralized API Key Management:** Instead of hardcoding your Gemini API keys in various clients, you can manage them in one place.
+-   **Security:** Protect your expensive Gemini API keys by exposing only a proxy key to your users or client applications.
+-   **Monitoring & Observability:** The logging middleware gives you insight into how your API is being used, helping you debug issues and monitor usage patterns.
+-   **Seamless Migration:** If you have existing tools that use the OpenAI API, you can switch to using Google's Gemini models without significant code changes.
+
+## Running the Built-in Server
+
+You can quickly get the proxy server up and running with just a few steps.
+
+### 1. Installation
+
+Install the package using pip:
+
+```bash
+pip install gemini-calo
+```
+
+### 2. Configuration
+
+The server is configured through environment variables. You can create a `.env` file in your working directory to store them.
+
+*   `GEMINI_CALO_API_KEYS`: A comma-separated list of your Google Gemini API keys. The proxy will rotate through these keys for outgoing requests.
+*   `GEMINI_CALO_PROXY_API_KEYS`: (Optional) A comma-separated list of API keys that clients must provide to use the proxy. If not set, the proxy will be open to anyone.
+*   `GEMINI_CALO_HTTP_PORT`: (Optional) The port on which the server will run. Defaults to `8000`.
+
+**Example `.env` file:**
+
+```
+GEMINI_CALO_API_KEYS=your_gemini_key_1,your_gemini_key_2
+GEMINI_CALO_PROXY_API_KEYS=my_secret_proxy_key_1,my_secret_proxy_key_2
+GEMINI_CALO_HTTP_PORT=8080
+```
+
+### 3. Running the Server
+
+Once configured, you can start the server with the `gemini-calo` command:
+
+```bash
+gemini-calo
+```
+
+The server will start on the configured port (e.g., `http://0.0.0.0:8080`).
+
+## Integrating with an Existing FastAPI Application
+
+If you have an existing FastAPI application, you can easily integrate Gemini Calo's proxy functionality into it.
+
+```python
+from fastapi import FastAPI
+from gemini_calo.proxy import GeminiProxyService
+from gemini_calo.middlewares.auth import auth_middleware
+from gemini_calo.middlewares.logging import logging_middleware
+from functools import partial
+import os
+
+# Your existing FastAPI app
+app = FastAPI()
+
+# 1. Initialize the GeminiProxyService
+gemini_api_keys = os.getenv("GEMINI_CALO_API_KEYS", "").split(",")
+proxy_service = GeminiProxyService(gemini_api_keys=gemini_api_keys)
+
+# 2. (Optional) Add Authentication Middleware
+proxy_api_keys = os.getenv("GEMINI_CALO_PROXY_API_KEYS", "").split(",")
+if proxy_api_keys:
+    auth_middleware_with_keys = partial(auth_middleware, user_api_key_checker=proxy_api_keys)
+    app.middleware("http")(auth_middleware_with_keys)
+
+# 3. (Optional) Add Logging Middleware
+app.middleware("http")(logging_middleware)
+
+# 4. Mount the Gemini and OpenAI routers
+app.include_router(proxy_service.gemini_router)
+app.include_router(proxy_service.openai_router)
+
+@app.get("/health")
+def health_check():
+    return {"status": "ok"}
+
+# Now you can run your app as usual with uvicorn
+# uvicorn your_app_file:app --reload
+```
+
+## How the Middleware Works
+
+Middleware in FastAPI are functions that process every request before it reaches the specific path operation and every response before it is sent back to the client. Gemini Calo includes two useful middlewares by default.
+
+### Logging Middleware (`logging_middleware`)
+
+This middleware logs the details of every incoming request and outgoing response, including headers and body content. This is invaluable for debugging and monitoring. It's designed to handle both standard and streaming responses correctly.
+
+### Authentication Middleware (`auth_middleware`)
+
+This middleware protects your proxy endpoints. It checks for an API key in the `Authorization` header (as a Bearer token) or the `x-goog-api-key` header. It then validates this key against the list of keys you provided in the `GEMINI_CALO_PROXY_API_KEYS` environment variable. If the key is missing or invalid, it returns a `401 Unauthorized` error.
+
+### Adding Your Own Middleware
+
+Because Gemini Calo is built on FastAPI, you can easily add your own custom middleware. For example, you could add a middleware for rate limiting, CORS, or custom header injection.
+
+#### Advanced Middleware: Modifying Request Body and Headers
+
+Here is a more advanced example that intercepts a request, modifies its JSON body, adds a new header, and then forwards it to the actual endpoint. This can be useful for injecting default values, adding metadata, or transforming request payloads.
+
+**Important:** Reading the request body consumes it. To allow the endpoint to read the body again, we must reconstruct the request with the modified body.
+
+```python
+from fastapi import FastAPI, Request
+from starlette.datastructures import MutableHeaders
+import json
+
+app = FastAPI()
+
+# This middleware will add a 'user_id' to the request body
+# and a 'X-Request-ID' to the headers.
+async def modify_request_middleware(request: Request, call_next):
+    # Get the original request body
+    body = await request.body()
+    
+    # Modify headers
+    request_headers = MutableHeaders(request.headers)
+    request_headers["X-Request-ID"] = "some-unique-id"
+    
+    # Modify body (if it's JSON)
+    new_body = body
+    if body and request.headers.get("content-type") == "application/json":
+        try:
+            json_body = json.loads(body)
+            # Add or modify a key
+            json_body["user_id"] = "injected-user-123"
+            new_body = json.dumps(json_body).encode()
+        except json.JSONDecodeError:
+            # Body is not valid JSON, pass it through
+            pass
+
+    # To pass the modified body and headers, we need to create a new Request object.
+    # We do this by defining a new 'receive' channel.
+    async def receive():
+        return {"type": "http.request", "body": new_body, "more_body": False}
+
+    # We replace the original request's scope with the modified headers
+    request.scope["headers"] = request_headers.raw
+
+    # Create the new request object and pass it to the next middleware/endpoint
+    new_request = Request(request.scope, receive)
+    response = await call_next(new_request)
+    
+    return response
+
+app.middleware("http")(modify_request_middleware)
+
+# ... then add the Gemini Calo proxy and routers as shown above
+```
+

gemini_calo-0.1.0/README.md

@@ -0,0 +1,169 @@
+# Gemini Calo
+
+**Gemini Calo** is a powerful, yet simple, FastAPI-based proxy server for Google's Gemini API. It provides a seamless way to add a layer of authentication, logging, and monitoring to your Gemini API requests. It's designed to be run as a standalone server or integrated into your existing FastAPI applications.
+
+One of its key features is providing an OpenAI-compatible endpoint, allowing you to use Gemini models with tools and libraries that are built for the OpenAI API.
+
+## Key Features
+
+*   **Authentication:** Secure your Gemini API access with an additional layer of API key authentication.
+*   **Request Logging:** Detailed logging of all incoming requests and outgoing responses.
+*   **OpenAI Compatibility:** Use Gemini models through an OpenAI-compatible `/v1/chat/completions` endpoint.
+*   **Round-Robin API Keys:** Distribute your requests across multiple Gemini API keys.
+*   **Easy Integration:** Use it as a standalone server or mount it into your existing FastAPI project.
+*   **Extensible:** Easily add your own custom middleware to suit your needs.
+
+## How It's Useful
+
+-   **Centralized API Key Management:** Instead of hardcoding your Gemini API keys in various clients, you can manage them in one place.
+-   **Security:** Protect your expensive Gemini API keys by exposing only a proxy key to your users or client applications.
+-   **Monitoring & Observability:** The logging middleware gives you insight into how your API is being used, helping you debug issues and monitor usage patterns.
+-   **Seamless Migration:** If you have existing tools that use the OpenAI API, you can switch to using Google's Gemini models without significant code changes.
+
+## Running the Built-in Server
+
+You can quickly get the proxy server up and running with just a few steps.
+
+### 1. Installation
+
+Install the package using pip:
+
+```bash
+pip install gemini-calo
+```
+
+### 2. Configuration
+
+The server is configured through environment variables. You can create a `.env` file in your working directory to store them.
+
+*   `GEMINI_CALO_API_KEYS`: A comma-separated list of your Google Gemini API keys. The proxy will rotate through these keys for outgoing requests.
+*   `GEMINI_CALO_PROXY_API_KEYS`: (Optional) A comma-separated list of API keys that clients must provide to use the proxy. If not set, the proxy will be open to anyone.
+*   `GEMINI_CALO_HTTP_PORT`: (Optional) The port on which the server will run. Defaults to `8000`.
+
+**Example `.env` file:**
+
+```
+GEMINI_CALO_API_KEYS=your_gemini_key_1,your_gemini_key_2
+GEMINI_CALO_PROXY_API_KEYS=my_secret_proxy_key_1,my_secret_proxy_key_2
+GEMINI_CALO_HTTP_PORT=8080
+```
+
+### 3. Running the Server
+
+Once configured, you can start the server with the `gemini-calo` command:
+
+```bash
+gemini-calo
+```
+
+The server will start on the configured port (e.g., `http://0.0.0.0:8080`).
+
+## Integrating with an Existing FastAPI Application
+
+If you have an existing FastAPI application, you can easily integrate Gemini Calo's proxy functionality into it.
+
+```python
+from fastapi import FastAPI
+from gemini_calo.proxy import GeminiProxyService
+from gemini_calo.middlewares.auth import auth_middleware
+from gemini_calo.middlewares.logging import logging_middleware
+from functools import partial
+import os
+
+# Your existing FastAPI app
+app = FastAPI()
+
+# 1. Initialize the GeminiProxyService
+gemini_api_keys = os.getenv("GEMINI_CALO_API_KEYS", "").split(",")
+proxy_service = GeminiProxyService(gemini_api_keys=gemini_api_keys)
+
+# 2. (Optional) Add Authentication Middleware
+proxy_api_keys = os.getenv("GEMINI_CALO_PROXY_API_KEYS", "").split(",")
+if proxy_api_keys:
+    auth_middleware_with_keys = partial(auth_middleware, user_api_key_checker=proxy_api_keys)
+    app.middleware("http")(auth_middleware_with_keys)
+
+# 3. (Optional) Add Logging Middleware
+app.middleware("http")(logging_middleware)
+
+# 4. Mount the Gemini and OpenAI routers
+app.include_router(proxy_service.gemini_router)
+app.include_router(proxy_service.openai_router)
+
+@app.get("/health")
+def health_check():
+    return {"status": "ok"}
+
+# Now you can run your app as usual with uvicorn
+# uvicorn your_app_file:app --reload
+```
+
+## How the Middleware Works
+
+Middleware in FastAPI are functions that process every request before it reaches the specific path operation and every response before it is sent back to the client. Gemini Calo includes two useful middlewares by default.
+
+### Logging Middleware (`logging_middleware`)
+
+This middleware logs the details of every incoming request and outgoing response, including headers and body content. This is invaluable for debugging and monitoring. It's designed to handle both standard and streaming responses correctly.
+
+### Authentication Middleware (`auth_middleware`)
+
+This middleware protects your proxy endpoints. It checks for an API key in the `Authorization` header (as a Bearer token) or the `x-goog-api-key` header. It then validates this key against the list of keys you provided in the `GEMINI_CALO_PROXY_API_KEYS` environment variable. If the key is missing or invalid, it returns a `401 Unauthorized` error.
+
+### Adding Your Own Middleware
+
+Because Gemini Calo is built on FastAPI, you can easily add your own custom middleware. For example, you could add a middleware for rate limiting, CORS, or custom header injection.
+
+#### Advanced Middleware: Modifying Request Body and Headers
+
+Here is a more advanced example that intercepts a request, modifies its JSON body, adds a new header, and then forwards it to the actual endpoint. This can be useful for injecting default values, adding metadata, or transforming request payloads.
+
+**Important:** Reading the request body consumes it. To allow the endpoint to read the body again, we must reconstruct the request with the modified body.
+
+```python
+from fastapi import FastAPI, Request
+from starlette.datastructures import MutableHeaders
+import json
+
+app = FastAPI()
+
+# This middleware will add a 'user_id' to the request body
+# and a 'X-Request-ID' to the headers.
+async def modify_request_middleware(request: Request, call_next):
+    # Get the original request body
+    body = await request.body()
+    
+    # Modify headers
+    request_headers = MutableHeaders(request.headers)
+    request_headers["X-Request-ID"] = "some-unique-id"
+    
+    # Modify body (if it's JSON)
+    new_body = body
+    if body and request.headers.get("content-type") == "application/json":
+        try:
+            json_body = json.loads(body)
+            # Add or modify a key
+            json_body["user_id"] = "injected-user-123"
+            new_body = json.dumps(json_body).encode()
+        except json.JSONDecodeError:
+            # Body is not valid JSON, pass it through
+            pass
+
+    # To pass the modified body and headers, we need to create a new Request object.
+    # We do this by defining a new 'receive' channel.
+    async def receive():
+        return {"type": "http.request", "body": new_body, "more_body": False}
+
+    # We replace the original request's scope with the modified headers
+    request.scope["headers"] = request_headers.raw
+
+    # Create the new request object and pass it to the next middleware/endpoint
+    new_request = Request(request.scope, receive)
+    response = await call_next(new_request)
+    
+    return response
+
+app.middleware("http")(modify_request_middleware)
+
+# ... then add the Gemini Calo proxy and routers as shown above
+```

gemini_calo-0.1.0/gemini_calo/__init__.py

@@ -0,0 +1,7 @@
+from gemini_calo.proxy import GeminiProxyService
+from gemini_calo.middlewares.auth import auth_middleware
+from gemini_calo.middlewares.logging import logging_middleware
+
+assert GeminiProxyService
+assert auth_middleware
+assert logging_middleware

gemini_calo-0.1.0/gemini_calo/__main__.py

@@ -0,0 +1,34 @@
+import os
+import uvicorn
+from fastapi import FastAPI
+from gemini_calo.proxy import GeminiProxyService
+from gemini_calo.middlewares.logging import logging_middleware
+from gemini_calo.middlewares.auth import auth_middleware
+from functools import partial
+
+
+def start_server():
+    api_keys = os.getenv("GEMINI_CALO_API_KEYS", "").split(",")
+    if not api_keys or api_keys == [""]:
+        raise ValueError("GEMINI_CALO_API_KEYS not found or empty in .env file")
+
+    app = FastAPI()
+    proxy = GeminiProxyService(gemini_api_keys=api_keys)
+
+    proxy_api_key = os.getenv("GEMINI_CALO_PROXY_API_KEYS", "")
+    if proxy_api_key != "":
+        proxy_api_keys = proxy_api_key.split(",")
+        auth_middleware_with_key = partial(auth_middleware, user_api_key_checker=proxy_api_keys)
+        app.middleware("http")(auth_middleware_with_key)
+
+    app.middleware("http")(logging_middleware)
+    app.include_router(proxy.gemini_router)
+    app.include_router(proxy.openai_router)
+
+    @app.get("/")
+    def read_root():
+        return {"Hello": "Proxy"}
+
+    uvicorn.run(
+        app, host="0.0.0.0", port=int(os.getenv("GEMINI_CALO_HTTP_PORT", "8000"))
+    )

gemini_calo-0.1.0/gemini_calo/logger.py

@@ -0,0 +1,12 @@
+import logging
+
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s [%(levelname)s] %(message)s",
+    handlers=[
+        logging.FileHandler("app.log"),
+        logging.StreamHandler()
+    ]
+)
+
+logger = logging.getLogger(__name__)

gemini_calo-0.1.0/gemini_calo/middlewares/auth.py

@@ -0,0 +1,66 @@
+import inspect
+from typing import Any, Callable, Coroutine
+from fastapi import Request, Response
+from fastapi.responses import JSONResponse
+from gemini_calo.proxy import GeminiProxyService, REQUEST_TYPE
+
+
+async def auth_middleware(
+    request: Request,
+    call_next: Callable[[Request], Coroutine[Any, Any, Response]],
+    user_api_key_checker: str | list[str] | Callable[[str], bool | Coroutine[Any, Any, bool]] | None = None
+) -> Response:
+    # If no checker is set, skip this
+    if user_api_key_checker is None:
+        return await call_next(request)
+    # Only apply check for specifics URLs
+    request_type = GeminiProxyService.get_request_type(request)
+    if request_type == REQUEST_TYPE.OTHER:
+        return await call_next(request)
+    # Get User API Key from request header
+    api_key = _get_request_api_key(request)
+    if not api_key:
+        return JSONResponse(
+            status_code=401,
+            content={
+                "error": (
+                    "API key is missing. Provide the API key in "
+                    "'Authorization: Bearer <key>' or 'x-goog-api-key' header."
+                )
+            },
+        )
+    # Check whether User API Key is valid
+    is_authorized = await _check_is_authorized(user_api_key_checker, api_key)
+    if not is_authorized:
+        return JSONResponse(
+            status_code=401,
+            content={"error": "The provided API Key is not valid."},
+        )
+    return await call_next(request)
+
+
+def _get_request_api_key(request: Request) -> str | None:
+    api_key = None
+    auth_header = request.headers.get("Authorization")
+    if auth_header:
+        try:
+            scheme, _, api_key = auth_header.partition(" ")
+            if scheme.lower() == "bearer":
+                return api_key
+            return None
+        except ValueError:
+            return None
+    return request.headers.get("x-goog-api-key")
+
+
+async def _check_is_authorized(
+    user_api_key_checker: str | list[str] | Callable[[str], bool | Coroutine[Any, Any, bool]] | None,
+    api_key: str
+) -> bool:
+    if inspect.iscoroutinefunction(user_api_key_checker):
+        return await user_api_key_checker(api_key)
+    if callable(user_api_key_checker):
+        return user_api_key_checker(api_key)
+    if isinstance(user_api_key_checker, str):
+        return api_key == user_api_key_checker
+    return api_key in user_api_key_checker

gemini_calo-0.1.0/gemini_calo/middlewares/logging.py

@@ -0,0 +1,65 @@
+from fastapi import Request, Response
+from fastapi.responses import StreamingResponse
+from gemini_calo.logger import logger
+import json
+
+
+async def logging_middleware(request: Request, call_next):
+    """
+    Logs incoming request and outgoing response details.
+    Handles both regular and streaming responses.
+    """
+    logger.info(f"Incoming request: {request.method} {request.url}")
+    logger.debug(f"Request headers: {request.headers}")
+
+    # Read the body and re-create the request to make the body available again
+    body = await request.body()
+    if body:
+        try:
+            logger.debug(f"Request body: {json.loads(body)}")
+        except json.JSONDecodeError:
+            logger.debug(f"Request body (non-JSON): {body.decode(errors='ignore')}")
+
+    async def receive():
+        return {"type": "http.request", "body": body}
+
+    request = Request(request.scope, receive)
+
+    response = await call_next(request)
+
+    logger.info(f"Outgoing response: Status {response.status_code}")
+    logger.debug(f"Response headers: {response.headers}")
+
+    if isinstance(response, StreamingResponse):
+        # For streaming responses, we need to wrap the iterator to log chunks
+        original_iterator = response.body_iterator
+
+        async def logging_iterator():
+            full_body = b""
+            async for chunk in original_iterator:
+                full_body += chunk
+                yield chunk
+            try:
+                logger.debug(f"Response body (stream): {json.loads(full_body)}")
+            except json.JSONDecodeError:
+                logger.debug(f"Response body (stream, non-JSON): {full_body.decode(errors='ignore')}")
+
+        return StreamingResponse(
+            logging_iterator(),
+            status_code=response.status_code,
+            headers=dict(response.headers),
+            media_type=response.media_type,
+        )
+    
+    # For non-streaming responses
+    response_body = b""
+    if hasattr(response, "body"):
+        response_body = response.body
+    
+    if response_body:
+        try:
+            logger.debug(f"Response body: {json.loads(response_body)}")
+        except json.JSONDecodeError:
+            logger.debug(f"Response body (non-JSON): {response_body.decode(errors='ignore')}")
+
+    return response

gemini_calo-0.1.0/gemini_calo/proxy.py

@@ -0,0 +1,156 @@
+from typing import Any
+from fastapi import Request, Response, APIRouter
+from fastapi.responses import StreamingResponse
+from gemini_calo.logger import logger
+from enum import Enum
+import httpx
+
+
+class REQUEST_TYPE(Enum):
+    OPENAI_COMPLETION: str = "openai-completion"
+    OPENAI_EMBEDDING: str = "openai-embedding"
+    GEMINI_COMPLETION: str = "gemini-completion"
+    GEMINI_STREAMING_COMPLETION: str = "gemini-streaming-completion"
+    GEMINI_EMBEDDING: str = "gemini-embedding"
+    OTHER: str = "other"
+
+
+class GeminiProxyService:
+    def __init__(
+        self,
+        base_url: str = "https://generativelanguage.googleapis.com",
+        gemini_api_keys: list[str] = [],
+    ):
+        self._gemini_api_keys = gemini_api_keys
+        self._current_gemini_api_key_index = 0
+        self._base_url = base_url
+        self.openai_router = APIRouter()
+        self.gemini_router = APIRouter()
+        self._add_routes()
+
+    def _add_routes(self):
+        self.openai_router.add_api_route(
+            "/v1/chat/completions",
+            self.forward_openai_request,
+            methods=["POST"],
+            response_model=Any,
+            # response_class=Response,
+        )
+        self.openai_router.add_api_route(
+            "/v1/embeddings",
+            self.forward_openai_request,
+            methods=["POST"],
+            response_model=Any,
+            # response_class=Response,
+        )
+        self.gemini_router.add_api_route(
+            "/v1beta/models/{model_name:path}:generateContent",
+            self.forward_gemini_request,
+            methods=["POST"],
+            response_model=Any,
+            # response_class=Response,
+        )
+        self.gemini_router.add_api_route(
+            "/v1beta/models/{model_name:path}:streamGenerateContent",
+            self.forward_gemini_request,
+            methods=["POST"],
+            response_model=Any,
+            # response_class=Response,
+        )
+        self.gemini_router.add_api_route(
+            "/v1beta/models/{model_name:path}:embedContent",
+            self.forward_gemini_request,
+            methods=["POST"],
+            response_model=Any,
+            # response_class=Response,
+        )
+
+    @classmethod
+    def get_request_type(cls, request: Request) -> str:
+        if request.url.path == "/v1/chat/completions":
+            return REQUEST_TYPE.OPENAI_COMPLETION
+        if request.url.path == "/v1/embeddings":
+            return REQUEST_TYPE.OPENAI_EMBEDDING
+        if request.url.path.startswith("/v1beta/models/"):
+            if request.url.path.endswith(":generateContent"):
+                return REQUEST_TYPE.GEMINI_COMPLETION
+            if request.url.path.endswith(":streamGenerateContent"):
+                return REQUEST_TYPE.GEMINI_STREAMING_COMPLETION
+            if request.url.path.endswith(":embedContent"):
+                return REQUEST_TYPE.GEMINI_EMBEDDING
+        return REQUEST_TYPE.OTHER
+
+    def get_gemini_api_key(self):
+        key = self._gemini_api_keys[self._current_gemini_api_key_index]
+        self._current_gemini_api_key_index += 1
+        self._current_gemini_api_key_index %= len(self._gemini_api_keys)
+        return key
+
+    def get_httpx_client(self):
+        return httpx.AsyncClient(base_url=self._base_url)
+
+    async def forward_openai_request(self, request: Request) -> Response:
+        """Forward openai request"""
+        api_key = self.get_gemini_api_key()
+        client = self.get_httpx_client()
+        url = httpx.URL(
+            scheme="https",
+            path=request.url.path,
+            query=request.url.query.encode("utf-8"),
+        )
+        headers = {
+            "Content-Type": "application/json",
+            "Authorization": f"Bearer {api_key}",
+        }
+        logger.info(f"Forwarding OpenAI request to: {url}")
+        req = client.build_request(
+            request.method,
+            url,
+            headers=headers,
+            content=request.stream(),
+            timeout=300.0,
+        )
+        response = await client.send(req, stream=True)
+        if hasattr(response, "aiter_raw"):
+            return StreamingResponse(
+                response.aiter_raw(),
+                status_code=response.status_code,
+                headers=dict(response.headers),
+            )
+        response_body = await response.aread()
+        return Response(
+            content=response_body,
+            status_code=response.status_code,
+            headers=dict(response.headers),
+        )
+
+    async def forward_gemini_request(self, request: Request) -> Response:
+        """Forward gemini request"""
+        api_key = self.get_gemini_api_key()
+        client = self.get_httpx_client()
+        url = httpx.URL(path=request.url.path, query=request.url.query.encode("utf-8"))
+        headers = {
+            "Content-Type": "application/json",
+            "x-goog-api-key": api_key,
+        }
+        logger.info(f"Forwarding request to: {url}")
+        req = client.build_request(
+            request.method,
+            url,
+            headers=headers,
+            content=request.stream(),
+            timeout=300.0,
+        )
+        response = await client.send(req, stream=True)
+        if hasattr(response, "aiter_raw"):
+            return StreamingResponse(
+                response.aiter_raw(),
+                status_code=response.status_code,
+                headers=dict(response.headers),
+            )
+        response_body = await response.aread()
+        return Response(
+            content=response_body,
+            status_code=response.status_code,
+            headers=dict(response.headers),
+        )

gemini_calo-0.1.0/pyproject.toml

@@ -0,0 +1,30 @@
+[tool.poetry]
+name = "gemini-calo"
+version = "0.1.0"
+description = "A Fastapi Based Proxy for Gemini API"
+authors = ["Go Frendi Gunawan <gofrendiasgard@gmail.com>"]
+license = "AGPL-3.0-or-later"
+readme = "README.md"
+homepage = "https://github.com/state-alchemists/gemini-calo"
+repository = "https://github.com/state-alchemists/gemini-calo"
+documentation = "https://github.com/state-alchemists/gemini-calo"
+
+[tool.poetry.dependencies]
+python = ">=3.10,<4.0.0"
+fastapi = "^0.116.1"
+uvicorn = "^0.35.0"
+
+[tool.poetry.group.dev.dependencies]
+pytest = "~8.3.5"
+httpx = "~0.28.1"
+pytest-httpx = "~0.35.0"
+
+[tool.pytest.ini_options]
+asyncio_default_fixture_loop_scope = "function"
+
+[tool.poetry.scripts]
+gemini-calo = "gemini_calo.__main__:start_server"
+
+[build-system]
+requires = ["poetry-core>=1.0.0"]
+build-backend = "poetry.core.masonry.api"