scrapegraphai 2.2.2__tar.gz → 2.2.4__tar.gz

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/CHANGELOG.md

@@ -1,3 +1,50 @@
+## [2.2.4](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.3...v2.2.4) (2026-09-07)
+
+
+### Bug Fixes
+
+* 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not the config file ([8769c3b](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/8769c3bddd7c865963cc7e245eefb496f55dc519))
+* **models:** add Gemini 2.5 token limits so they are not truncated to 8192 ([c21af20](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/c21af206862c13be1848eac75b4c04250718c8d9))
+* **fetch:** surface HTTP errors and missing content instead of answering NA ([f91478e](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/f91478eacf86485f6b9efcf843fc0c815dde1ec5)), closes [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102)
+
+
+### CI
+
+* **release:** 2.2.0-beta.10 [skip ci] ([0bb8bc9](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/0bb8bc935028b4f0a91444db2866ec0142f97199))
+* **release:** 2.2.0-beta.7 [skip ci] ([decfc6b](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/decfc6bb6eb10a29ed6aaabb07244b8915042604))
+* **release:** 2.2.0-beta.8 [skip ci] ([d59c3df](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/d59c3dfceecdacbba4e17f237b017117cf7f1cee)), closes [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102)
+* **release:** 2.2.0-beta.9 [skip ci] ([3047ef8](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/3047ef8eda694d19c6fe4654777ea6343744acba))
+* **release:** 2.2.4-beta.1 [skip ci] ([8b3a97c](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/8b3a97c3b41aec29df0512e71f186a98ad747aa1)), closes [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102)
+
+## [2.2.4-beta.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.3...v2.2.4-beta.1) (2026-09-07)
+
+
+### Bug Fixes
+
+* 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not the config file ([8769c3b](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/8769c3bddd7c865963cc7e245eefb496f55dc519))
+* **models:** add Gemini 2.5 token limits so they are not truncated to 8192 ([c21af20](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/c21af206862c13be1848eac75b4c04250718c8d9))
+* **fetch:** surface HTTP errors and missing content instead of answering NA ([f91478e](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/f91478eacf86485f6b9efcf843fc0c815dde1ec5)), closes [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102)
+
+
+### CI
+
+* **release:** 2.2.0-beta.10 [skip ci] ([0bb8bc9](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/0bb8bc935028b4f0a91444db2866ec0142f97199))
+* **release:** 2.2.0-beta.7 [skip ci] ([decfc6b](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/decfc6bb6eb10a29ed6aaabb07244b8915042604))
+* **release:** 2.2.0-beta.8 [skip ci] ([d59c3df](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/d59c3dfceecdacbba4e17f237b017117cf7f1cee)), closes [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102)
+* **release:** 2.2.0-beta.9 [skip ci] ([3047ef8](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/3047ef8eda694d19c6fe4654777ea6343744acba))
+
+## [2.2.3](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.2...v2.2.3) (2026-09-07)
+
+
+### Bug Fixes
+
+* **proxy:** handle schemeless proxy server format and broker routing ([70dbd2d](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/70dbd2d26d63bdc7473293371a275a4d856737f9))
+
+
+### Docs
+
+* add NodeMaven sponsor to all README languages ([cc8568c](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/cc8568cc76f765f3d2bd2db0bdf5018d347af101))
+
 ## [2.2.2](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.1...v2.2.2) (2026-08-23)
 
 

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: scrapegraphai
-Version: 2.2.2
+Version: 2.2.4
 Summary: A web scraping library based on LangChain which uses LLM and direct graph logic to create scraping pipelines.
 Author-email: Marco Vinciguerra <mvincig11@gmail.com>, Lorenzo Padoan <lorenzo.padoan977@gmail.com>
 Keywords: ai,artificial intelligence,gpt,graph,langchain,llm,machine learning,natural language processing,nlp,openai,rag,scrapegraph,scrapegraphai,scraping,web scraping,web scraping library,web scraping tool,webscraping

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/docs/chinese.md

@@ -158,6 +158,12 @@ print(json.dumps(result, indent=4))
 
 ScrapeGraphAI 的文档可以在[这里](https://docs.scrapegraphai.com/introduction)找到。
 
+## 🏆 赞助商
+
+[![NodeMaven](assets/nodemaven-banner.png)](https://go.nodemaven.com/scrapegraphyai)
+
+ScrapeGraphAI 用户优惠码：`SCRAPEGRAPH35`（移动和住宅代理 35% 折扣），`SCRAPEGRAPH40`（ISP / 静态代理 40% 折扣）。
+
 ## 🤝 贡献
 
 欢迎贡献并加入我们的 Discord 服务器与我们讨论改进和提出建议！

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/docs/french.md

@@ -160,6 +160,12 @@ N'oubliez pas d'installer [Ollama](https://ollama.com/) et de télécharger les
 
 La documentation pour ScrapeGraphAI se trouve [ici](https://docs.scrapegraphai.com/introduction).
 
+## 🏆 Sponsors
+
+[![NodeMaven](assets/nodemaven-banner.png)](https://go.nodemaven.com/scrapegraphyai)
+
+Codes pour les utilisateurs de ScrapeGraphAI : `SCRAPEGRAPH35` (35 % sur les proxies mobile et résidentiels), `SCRAPEGRAPH40` (40 % sur les proxies ISP / statiques).
+
 ## 🤝 Contribuer
 
 N'hésitez pas à contribuer et à rejoindre notre serveur Discord pour discuter avec nous des améliorations et nous donner des suggestions !

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/docs/german.md

@@ -160,6 +160,12 @@ Denken Sie daran, [Ollama](https://ollama.com/) installiert zu haben und die Mod
 
 Die Dokumentation zu ScrapeGraphAI finden Sie [hier](https://docs.scrapegraphai.com/introduction).
 
+## 🏆 Sponsoren
+
+[![NodeMaven](assets/nodemaven-banner.png)](https://go.nodemaven.com/scrapegraphyai)
+
+Codes für ScrapeGraphAI-Nutzer: `SCRAPEGRAPH35` (35 % auf Mobile- und Residential-Proxys), `SCRAPEGRAPH40` (40 % auf ISP- / Static-Proxys).
+
 ## 🤝 Mitwirken
 
 Fühlen Sie sich frei, beizutragen und treten Sie unserem Discord-Server bei, um mit uns über Verbesserungen zu diskutieren und uns Vorschläge zu machen!

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/docs/italian.md

@@ -162,6 +162,12 @@ Ricordati di avere [Ollama](https://ollama.com/) installato e di scaricare i mod
 La documentazione di ScrapeGraphAI è disponibile [qui](https://scrapegraph-ai.readthedocs.io/en/latest/).
 Consulta anche il Docusaurus [qui](https://docs-oss.scrapegraphai.com/).
 
+## 🏆 Sponsor
+
+[![NodeMaven](assets/nodemaven-banner.png)](https://go.nodemaven.com/scrapegraphyai)
+
+Codici per gli utenti ScrapeGraphAI: `SCRAPEGRAPH35` (35% su proxy mobile e residenziali), `SCRAPEGRAPH40` (40% su proxy ISP / statici).
+
 ## 🤝 Vuoi contribuire?
 
 Sentiti libero di contribuire e unisciti al nostro server Discord per discutere con noi su cosa migliorare e darci suggerimenti!

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/docs/japanese.md

@@ -158,6 +158,12 @@ print(json.dumps(result, indent=4))
 
 ScrapeGraphAIのドキュメントは[こちら](https://docs.scrapegraphai.com/introduction)で見ることができます。
 
+## 🏆 スポンサー
+
+[![NodeMaven](assets/nodemaven-banner.png)](https://go.nodemaven.com/scrapegraphyai)
+
+ScrapeGraphAI ユーザー向けコード：`SCRAPEGRAPH35`（モバイル / 住宅用プロキシ 35% オフ）、`SCRAPEGRAPH40`（ISP / 静的プロキシ 40% オフ）。
+
 ## 🤝 貢献
 
 貢献を歓迎し、Discordサーバーで改善や提案について話し合います！

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/docs/korean.md

@@ -158,6 +158,12 @@ OpenAI, Groq, Azure, Gemini와 같은 API를 통해 다양한 LLM을 사용할
 
 ScrapeGraphAI 관련 문서는 [여기](https://docs.scrapegraphai.com/introduction)에서 확인하실 수 있습니다.
 
+## 🏆 스폰서
+
+[![NodeMaven](assets/nodemaven-banner.png)](https://go.nodemaven.com/scrapegraphyai)
+
+ScrapeGraphAI 사용자 코드: `SCRAPEGRAPH35` (모바일 및 주거용 프록시 35% 할인), `SCRAPEGRAPH40` (ISP / 고정 프록시 40% 할인).
+
 ## 🤝 기여
 
 자유롭게 기여하고 Discord 서버에 참여하여 개선 사항을 논의하고 제안해 주세요!

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/docs/portuguese.md

@@ -158,6 +158,12 @@ Lembre-se de ter o [Ollama](https://ollama.com/) instalado e baixar os modelos u
 
 A documentação do ScrapeGraphAI pode ser encontrada [aqui](https://docs.scrapegraphai.com/introduction).
 
+## 🏆 Patrocinadores
+
+[![NodeMaven](assets/nodemaven-banner.png)](https://go.nodemaven.com/scrapegraphyai)
+
+Códigos para usuários do ScrapeGraphAI: `SCRAPEGRAPH35` (35% em proxies móveis e residenciais), `SCRAPEGRAPH40` (40% em proxies ISP / estáticos).
+
 ## 🤝 Contribuindo
 
 Sinta-se à vontade para contribuir e junte-se ao nosso servidor Discord para discutir melhorias e nos dar sugestões!

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/docs/russian.md

@@ -158,6 +158,12 @@ print(json.dumps(result, indent=4))
 
 Документация для ScrapeGraphAI доступна [здесь](https://docs.scrapegraphai.com/introduction).
 
+## 🏆 Спонсоры
+
+[![NodeMaven](assets/nodemaven-banner.png)](https://go.nodemaven.com/scrapegraphyai)
+
+Коды для пользователей ScrapeGraphAI: `SCRAPEGRAPH35` (скидка 35% на mobile и residential-прокси), `SCRAPEGRAPH40` (скидка 40% на ISP / static-прокси).
+
 ## 🤝 Участие
 
 Не стесняйтесь вносить свой вклад и присоединяйтесь к нашему серверу Discord, чтобы обсудить с нами улучшения и дать нам предложения!

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/docs/spanish.md

@@ -165,6 +165,12 @@ Recuerda tener [Ollama](https://ollama.com/) instalado y descargar los modelos c
 
 La documentación de ScrapeGraphAI está disponible [aquí](https://docs.scrapegraphai.com/introduction).
 
+## 🏆 Patrocinadores
+
+[![NodeMaven](assets/nodemaven-banner.png)](https://go.nodemaven.com/scrapegraphyai)
+
+Códigos para usuarios de ScrapeGraphAI: `SCRAPEGRAPH35` (35% en proxies móviles y residenciales), `SCRAPEGRAPH40` (40% en proxies ISP / estáticos).
+
 ## 🤝 Contribuciones
 
 ¡Siéntete libre de contribuir y únete a nuestro servidor de Discord para discutir mejoras y compartir sugerencias!

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/docs/turkish.md

@@ -159,6 +159,12 @@ Yerel modelleri kullanmak istiyorsanız, [Ollama](https://ollama.com/) kurulu ol
 
 ScrapeGraphAI dokümantasyonuna [buradan](https://docs.scrapegraphai.com/introduction) ulaşabilirsiniz.
 
+## 🏆 Sponsorlar
+
+[![NodeMaven](assets/nodemaven-banner.png)](https://go.nodemaven.com/scrapegraphyai)
+
+ScrapeGraphAI kullanıcıları için kodlar: `SCRAPEGRAPH35` (mobil ve residential proxy'lerde %35 indirim), `SCRAPEGRAPH40` (ISP / static proxy'lerde %40 indirim).
+
 ## 🤝 Katkıda Bulunun
 
 Projeye katkıda bulunmaktan çekinmeyin ve geliştirmeleri tartışmak ve bize önerilerde bulunmak için Discord sunucumuza katılın!

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "scrapegraphai"
 
-version = "2.2.2"
+version = "2.2.4"
 
 description = "A web scraping library based on LangChain which uses LLM and direct graph logic to create scraping pipelines."
 authors = [

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/scrapegraphai/helpers/models_tokens.py

@@ -140,6 +140,11 @@ models_tokens = {
         "gemini-2.0-flash-latest": 1000000,
         "gemini-2.0-flash-exp": 1000000,
         "gemini-2.0-pro-exp": 2000000,
+        "gemini-2.5-flash": 1000000,
+        "gemini-2.5-flash-latest": 1000000,
+        "gemini-2.5-flash-lite": 1000000,
+        "gemini-2.5-pro": 1000000,
+        "gemini-flash-latest": 1000000,
         "models/embedding-001": 2048,
     },
     "google_vertexai": {
@@ -150,6 +155,10 @@ models_tokens = {
         "gemini-2.0-flash-exp": 1048576,
         "gemini-2.0-pro": 2000000,
         "gemini-2.0-pro-exp": 2000000,
+        "gemini-2.5-flash": 1048576,
+        "gemini-2.5-flash-lite": 1048576,
+        "gemini-2.5-pro": 1048576,
+        "gemini-flash-latest": 1048576,
     },
     "ollama": {
         "command-r": 12800,

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/scrapegraphai/telemetry/telemetry.py

@@ -36,6 +36,19 @@ def _load_config(config_location: str) -> configparser.ConfigParser:
     return config
 
 
+def _parse_bool(value: str) -> bool:
+    """Parse a boolean from a string using configparser's accepted spellings.
+
+    Accepts the same values as the config file does, so
+    ``SCRAPEGRAPHAI_TELEMETRY_ENABLED=false`` and ``telemetry_enabled = false``
+    behave identically. Raises ValueError on anything unrecognised.
+    """
+    try:
+        return configparser.ConfigParser.BOOLEAN_STATES[value.strip().lower()]
+    except KeyError:
+        raise ValueError(f"invalid boolean value: {value!r}")
+
+
 def _check_config_and_environ_for_telemetry_flag(default_value: bool, config_obj):
     telemetry_enabled = default_value
     if "telemetry_enabled" in config_obj["DEFAULT"]:
@@ -44,13 +57,18 @@ def _check_config_and_environ_for_telemetry_flag(default_value: bool, config_obj
         except Exception:
             pass
 
-    if os.environ.get("SCRAPEGRAPHAI_TELEMETRY_ENABLED") is not None:
+    env_value = os.environ.get("SCRAPEGRAPHAI_TELEMETRY_ENABLED")
+    if env_value is not None:
         try:
-            telemetry_enabled = config_obj.getboolean(
-                "DEFAULT", "telemetry_enabled"
+            telemetry_enabled = _parse_bool(env_value)
+        except ValueError:
+            logger.warning(
+                "SCRAPEGRAPHAI_TELEMETRY_ENABLED is set to %r, which is not a "
+                "recognised boolean. Telemetry is left at %s. Use one of: "
+                "true/false, yes/no, on/off, 1/0.",
+                env_value,
+                telemetry_enabled,
             )
-        except Exception:
-            pass
 
     return telemetry_enabled
 

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/scrapegraphai/utils/proxy_rotation.py

@@ -142,18 +142,18 @@ def _parse_proxy(proxy: ProxySettings) -> ProxySettings:
     """
     assert "server" in proxy, "missing server in the proxy configuration"
 
-    auhtorization = [x in proxy for x in ("username", "password")]
+    authorization = [x in proxy for x in ("username", "password")]
 
     message = "username and password must be provided in pairs or not at all"
 
-    assert all(auhtorization) or not any(auhtorization), message
+    assert all(authorization) or not any(authorization), message
 
     parsed = {"server": proxy["server"]}
 
     if proxy.get("bypass"):
         parsed["bypass"] = proxy["bypass"]
 
-    if all(auhtorization):
+    if all(authorization):
         parsed["username"] = proxy["username"]
         parsed["password"] = proxy["password"]
 
@@ -192,9 +192,14 @@ def parse_or_search_proxy(proxy: Proxy) -> ProxySettings:
     """
     Parses a proxy configuration or searches for a matching one via broker.
     """
-    assert "server" in proxy, "Missing 'server' field in the proxy configuration."
+    assert "server" in proxy, "missing server in the proxy configuration"
+
+    server = proxy["server"]
+    if server == "broker":
+        return _search_proxy(proxy)
 
-    parsed_url = urlparse(proxy["server"])
+    server_with_scheme = server if "://" in server else f"http://{server}"
+    parsed_url = urlparse(server_with_scheme)
     server_address = parsed_url.hostname
 
     if server_address is None:
@@ -206,6 +211,6 @@ def parse_or_search_proxy(proxy: Proxy) -> ProxySettings:
     ):
         return _parse_proxy(proxy)
 
-    assert proxy["server"] == "broker", f"Unknown proxy server type: {proxy['server']}"
+    assert proxy["server"] == "broker", f"unknown proxy server type: {proxy['server']}"
 
     return _search_proxy(proxy)

{scrapegraphai-2.2.2 → scrapegraphai-2.2.4}/tests/test_models_tokens.py

@@ -7,9 +7,9 @@ class TestModelsTokens:
     def test_openai_tokens(self):
         """Test that the 'openai' provider exists and its tokens are valid positive integers."""
         openai_models = models_tokens.get("openai")
-        assert openai_models is not None, (
-            "'openai' key should be present in models_tokens"
-        )
+        assert (
+            openai_models is not None
+        ), "'openai' key should be present in models_tokens"
         for model, token in openai_models.items():
             assert isinstance(model, str), "Model name should be a string"
             assert isinstance(token, int), "Token limit should be an integer"
@@ -30,19 +30,50 @@ class TestModelsTokens:
         assert google_genai is not None, "'google_genai' key should be present"
         assert google_vertexai is not None, "'google_vertexai' key should be present"
         # Check a specific key from google_genai
-        assert "gemini-pro" in google_genai, (
-            "'gemini-pro' should be in google_genai models"
-        )
+        assert (
+            "gemini-pro" in google_genai
+        ), "'gemini-pro' should be in google_genai models"
         # Validate token values types
         for provider in [google_genai, google_vertexai]:
             for token in provider.values():
                 assert isinstance(token, int), "Token limit must be an integer"
 
+    def test_gemini_2_5_models_are_registered(self):
+        """Gemini 2.5 / flash-latest must be in the table so they are not truncated to 8192.
+
+        #1121: an unknown model silently falls back to an 8192-token window.
+        google_genai/gemini-2.5-flash is the reported case; gemini-flash-latest
+        is the current flash alias. Both have a 1M input context.
+        """
+        google_genai = models_tokens["google_genai"]
+        google_vertexai = models_tokens["google_vertexai"]
+
+        for model in (
+            "gemini-2.5-flash",
+            "gemini-2.5-flash-latest",
+            "gemini-2.5-flash-lite",
+            "gemini-2.5-pro",
+            "gemini-flash-latest",
+        ):
+            assert (
+                google_genai.get(model) == 1000000
+            ), f"Expected 1M context for {model} in google_genai"
+
+        for model in (
+            "gemini-2.5-flash",
+            "gemini-2.5-flash-lite",
+            "gemini-2.5-pro",
+            "gemini-flash-latest",
+        ):
+            assert (
+                google_vertexai.get(model) == 1048576
+            ), f"Expected 1M context for {model} in google_vertexai"
+
     def test_non_existent_provider(self):
         """Test that a non-existent provider returns None."""
-        assert models_tokens.get("non_existent") is None, (
-            "Non-existent provider should return None"
-        )
+        assert (
+            models_tokens.get("non_existent") is None
+        ), "Non-existent provider should return None"
 
     def test_total_model_keys(self):
         """Test that the total number of models across all providers is above an expected count."""
@@ -59,120 +90,120 @@ class TestModelsTokens:
         """Ensure that model token names are non-empty strings."""
         for provider, model_dict in models_tokens.items():
             for model in model_dict.keys():
-                assert model != "", (
-                    f"Model name in provider '{provider}' should not be empty."
-                )
+                assert (
+                    model != ""
+                ), f"Model name in provider '{provider}' should not be empty."
 
     def test_token_limits_range(self):
         """Test that token limits for all models fall within a plausible range (e.g., 1 to 300000)."""
         for provider, model_dict in models_tokens.items():
             for model, token in model_dict.items():
-                assert 1 <= token <= 1100000, (
-                    f"Token limit for {model} in provider {provider} is out of plausible range."
-                )
+                assert (
+                    1 <= token <= 1100000
+                ), f"Token limit for {model} in provider {provider} is out of plausible range."
 
     def test_provider_structure(self):
         """Test that every provider in models_tokens has a dictionary as its value."""
         for provider, models in models_tokens.items():
-            assert isinstance(models, dict), (
-                f"Provider {provider} should map to a dictionary, got {type(models).__name__}"
-            )
+            assert isinstance(
+                models, dict
+            ), f"Provider {provider} should map to a dictionary, got {type(models).__name__}"
 
     def test_non_empty_provider(self):
         """Test that each provider dictionary is not empty."""
         for provider, models in models_tokens.items():
-            assert len(models) > 0, (
-                f"Provider {provider} should contain at least one model."
-            )
+            assert (
+                len(models) > 0
+            ), f"Provider {provider} should contain at least one model."
 
     def test_specific_model_token_values(self):
         """Test specific expected token values for selected models from various providers."""
         # Verify a token for a selected model from the 'openai' provider
         openai = models_tokens.get("openai")
-        assert openai.get("gpt-3.5-turbo-0125") == 16385, (
-            "Expected token limit for gpt-3.5-turbo-0125 in openai to be 16385"
-        )
+        assert (
+            openai.get("gpt-3.5-turbo-0125") == 16385
+        ), "Expected token limit for gpt-3.5-turbo-0125 in openai to be 16385"
 
         # Verify a token for a selected model from the 'azure_openai' provider
         azure = models_tokens.get("azure_openai")
-        assert azure.get("gpt-3.5") == 4096, (
-            "Expected token limit for gpt-3.5 in azure_openai to be 4096"
-        )
+        assert (
+            azure.get("gpt-3.5") == 4096
+        ), "Expected token limit for gpt-3.5 in azure_openai to be 4096"
 
         # Verify a token for a selected model from the 'anthropic' provider
         anthropic = models_tokens.get("anthropic")
-        assert anthropic.get("claude_instant") == 100000, (
-            "Expected token limit for claude_instant in anthropic to be 100000"
-        )
+        assert (
+            anthropic.get("claude_instant") == 100000
+        ), "Expected token limit for claude_instant in anthropic to be 100000"
 
     def test_providers_count(self):
         """Test that the total number of providers is as expected (at least 15)."""
-        assert len(models_tokens) >= 15, (
-            "Expected at least 15 providers in models_tokens"
-        )
+        assert (
+            len(models_tokens) >= 15
+        ), "Expected at least 15 providers in models_tokens"
 
     def test_non_existent_model(self):
         """Test that a non-existent model within a valid provider returns None."""
         openai = models_tokens.get("openai")
-        assert openai.get("non_existent_model") is None, (
-            "Non-existent model should return None from a valid provider."
-        )
+        assert (
+            openai.get("non_existent_model") is None
+        ), "Non-existent model should return None from a valid provider."
 
     def test_no_whitespace_in_model_names(self):
         """Test that model names do not contain leading or trailing whitespace."""
         for provider, model_dict in models_tokens.items():
             for model in model_dict.keys():
                 # Assert that stripping whitespace does not change the model name
-                assert model == model.strip(), (
-                    f"Model name '{model}' in provider '{provider}' contains leading or trailing whitespace."
-                )
+                assert (
+                    model == model.strip()
+                ), f"Model name '{model}' in provider '{provider}' contains leading or trailing whitespace."
 
     def test_specific_models_additional(self):
         """Test specific token values for additional models across various providers."""
         # Check some models in the 'ollama' provider
         ollama = models_tokens.get("ollama")
-        assert ollama.get("llama2") == 4096, (
-            "Expected token limit for 'llama2' in ollama to be 4096"
-        )
-        assert ollama.get("llama2:70b") == 4096, (
-            "Expected token limit for 'llama2:70b' in ollama to be 4096"
-        )
+        assert (
+            ollama.get("llama2") == 4096
+        ), "Expected token limit for 'llama2' in ollama to be 4096"
+        assert (
+            ollama.get("llama2:70b") == 4096
+        ), "Expected token limit for 'llama2:70b' in ollama to be 4096"
 
         # Check a specific model from the 'mistralai' provider
         mistralai = models_tokens.get("mistralai")
-        assert mistralai.get("open-codestral-mamba") == 256000, (
-            "Expected token limit for 'open-codestral-mamba' in mistralai to be 256000"
-        )
+        assert (
+            mistralai.get("open-codestral-mamba") == 256000
+        ), "Expected token limit for 'open-codestral-mamba' in mistralai to be 256000"
 
         # Check a specific model from the 'deepseek' provider
         deepseek = models_tokens.get("deepseek")
-        assert deepseek.get("deepseek-chat") == 28672, (
-            "Expected token limit for 'deepseek-chat' in deepseek to be 28672"
-        )
+        assert (
+            deepseek.get("deepseek-chat") == 28672
+        ), "Expected token limit for 'deepseek-chat' in deepseek to be 28672"
 
         # Check a model from the 'ernie' provider
         ernie = models_tokens.get("ernie")
-        assert ernie.get("ernie-bot") == 4096, (
-            "Expected token limit for 'ernie-bot' in ernie to be 4096"
-        )
+        assert (
+            ernie.get("ernie-bot") == 4096
+        ), "Expected token limit for 'ernie-bot' in ernie to be 4096"
 
     def test_nvidia_specific(self):
         """Test specific token value for 'meta/codellama-70b' in the nvidia provider."""
         nvidia = models_tokens.get("nvidia")
         assert nvidia is not None, "'nvidia' provider should exist"
         # Verify token for 'meta/codellama-70b' equals 16384 as defined in the nvidia dictionary
-        assert nvidia.get("meta/codellama-70b") == 16384, (
-            "Expected token limit for 'meta/codellama-70b' in nvidia to be 16384"
-        )
+        assert (
+            nvidia.get("meta/codellama-70b") == 16384
+        ), "Expected token limit for 'meta/codellama-70b' in nvidia to be 16384"
 
     def test_groq_specific(self):
         """Test specific token value for 'claude-3-haiku-20240307\'' in the groq provider."""
         groq = models_tokens.get("groq")
         assert groq is not None, "'groq' provider should exist"
         # Note: The model name has an embedded apostrophe at the end in its name.
-        assert groq.get("claude-3-haiku-20240307'") == 8192, (
-            "Expected token limit for 'claude-3-haiku-20240307\\'' in groq to be 8192"
-        )
+        assert (
+            groq.get("claude-3-haiku-20240307'") == 8192
+        ), "Expected token limit for 'claude-3-haiku-20240307\\'' in groq to be 8192"
 
     def test_togetherai_specific(self):
         """Test specific token value for 'meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo' in the toghetherai provider."""
@@ -180,15 +211,15 @@ class TestModelsTokens:
         assert togetherai is not None, "'toghetherai' provider should exist"
         expected = 128000
         model_name = "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo"
-        assert togetherai.get(model_name) == expected, (
-            f"Expected token limit for '{model_name}' in toghetherai to be {expected}"
-        )
+        assert (
+            togetherai.get(model_name) == expected
+        ), f"Expected token limit for '{model_name}' in toghetherai to be {expected}"
 
     def test_ernie_all_values(self):
         """Test that all models in the 'ernie' provider have token values exactly 4096."""
         ernie = models_tokens.get("ernie")
         assert ernie is not None, "'ernie' provider should exist"
         for model, token in ernie.items():
-            assert token == 4096, (
-                f"Expected token limit for '{model}' in ernie to be 4096, got {token}"
-            )
+            assert (
+                token == 4096
+            ), f"Expected token limit for '{model}' in ernie to be 4096, got {token}"