skillware 0.2.9__tar.gz → 0.3.2__tar.gz

{skillware-0.2.9/skillware.egg-info → skillware-0.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: skillware
-Version: 0.2.9
+Version: 0.3.2
 Summary: A framework for modular, self-contained AI skills.
 Author-email: ARPA Hellenic Logical Systems <skillware-os@arpacorp.net>
 License: MIT License
@@ -45,7 +45,7 @@ Requires-Dist: black; extra == "dev"
 Provides-Extra: cli
 Requires-Dist: rich>=13.0; extra == "cli"
 Provides-Extra: gemini
-Requires-Dist: google-generativeai; extra == "gemini"
+Requires-Dist: google-genai; extra == "gemini"
 Provides-Extra: claude
 Requires-Dist: anthropic; extra == "claude"
 Provides-Extra: openai
@@ -54,7 +54,7 @@ Provides-Extra: office
 Requires-Dist: pymupdf; extra == "office"
 Provides-Extra: all
 Requires-Dist: rich>=13.0; extra == "all"
-Requires-Dist: google-generativeai; extra == "all"
+Requires-Dist: google-genai; extra == "all"
 Requires-Dist: anthropic; extra == "all"
 Requires-Dist: openai; extra == "all"
 Requires-Dist: pymupdf; extra == "all"
@@ -81,6 +81,7 @@ Dynamic: license-file
   <a href="#architecture">Architecture</a> •
   <a href="#quick-start">Quick Start</a> •
   <a href="#documentation">Documentation</a> •
+  <a href="CHANGELOG.md">Changelog</a> •
   <a href="#contributing">Contributing</a> •
   <a href="#comparison">Comparison</a> •
   <a href="#contact">Contact</a>
@@ -103,28 +104,34 @@ A **Skill** in this framework provides everything an Agent needs to master a dom
 3.  **Governance**: Constitution and safety boundaries.
 4.  **Interface**: Standardized schemas for LLM tool calling.
 
+### Skill library
+
+Browse capabilities by category in the [Skill library](docs/skills/README.md).
+
 ## Architecture
 
-This repository is organized into a core framework, a registry of skills, and documentation.
+This repository is organized into a core framework, a registry of skills, and
+documentation. Runnable provider scripts are indexed in
+[examples/README.md](examples/README.md).
 
 ```text
 Skillware/
-├── docs/                       # Comprehensive Documentation & Usage Guides
-├── examples/                   # Reference Implementations
-│   └── basic_agent.py          # Example showing SkillLoader integration
+├── docs/                       # Introduction, testing, skill catalog, usage guides (docs/usage/)
+├── examples/                   # Provider reference scripts (Gemini, Claude, OpenAI, Ollama, ...)
 ├── skills/                     # Skill Registry
 │   └── category/               # Domain boundaries (e.g., finance)
 │       └── skill_name/         # The Skill bundle
 │           ├── manifest.yaml   # Definition, schema, and constitution
 │           ├── skill.py        # Executable Python logic
 │           ├── instructions.md # Cognitive map for the LLM
-│           └── test_skill.py   # Unit tests & schema validation
+│           ├── card.json       # Optional UI presentation metadata
+│           └── test_skill.py   # Unit tests and schema validation
 ├── skillware/                  # Core Framework Package
 │   ├── cli.py                  # Command-line interface
 │   └── core/
 │       ├── base_skill.py       # Abstract Base Class for skills
 │       ├── env.py              # Environment Management
-│       └── loader.py           # Universal Skill Loader & Model Adapter
+│       └── loader.py           # Universal Skill Loader and Model Adapter
 ├── templates/                  # Boilerplate templates for new skills
 │   └── python_skill/           # Standard template with required files
 └── tests/                      # Automated test suite
@@ -157,7 +164,7 @@ pip install "skillware[cli]"
 skillware list
 ```
 
-This prints a table of all locally available skills and confirms the install and path resolution are working.
+This prints a table of all locally available skills and confirms the install and path resolution are working. Running `skillware` with no arguments opens the interactive menu.
 
 ### 3. Configuration
 
@@ -169,8 +176,12 @@ GOOGLE_API_KEY="your_key"
 
 ### 4. Usage Example (Gemini)
 
+This example requires the Google SDK optional extra: `pip install "skillware[gemini]"` (local dev: `pip install -e ".[gemini]"`). See the [Gemini usage guide](docs/usage/gemini.md) for setup details.
+
 ```python
-import google.generativeai as genai
+import os
+import google.genai as genai
+from google.genai import types
 from skillware.core.loader import SkillLoader
 from skillware.core.env import load_env_file
 
@@ -179,36 +190,64 @@ load_env_file()
 
 # 1. Load the Skill from the Registry
 # The loader reads the code, manifest, and instructions automatically
-skill_bundle = SkillLoader.load_skill("category/skill_name")  # see docs/usage/README.md for path search order
-
-# 2. Model & Chat Setup
-model = genai.GenerativeModel(
-    'gemini-2.5-flash',
-    tools=[SkillLoader.to_gemini_tool(skill_bundle)], # The "Adapter"
-    system_instruction=skill_bundle['instructions']   # The "Mind"
+skill_bundle = SkillLoader.load_skill("finance/wallet_screening")
+skill = skill_bundle["module"].WalletScreeningSkill(
+    config={"ETHERSCAN_API_KEY": os.environ.get("ETHERSCAN_API_KEY")}
 )
-chat = model.start_chat(enable_automatic_function_calling=True)
+
+# 2. Client & Tool Setup
+client = genai.Client()
+tool = SkillLoader.to_gemini_tool(skill_bundle)       # The "Adapter"
+system_instruction = skill_bundle['instructions']     # The "Mind"
 
 # 3. Agent Loop
-# The SDK handles the loop: model -> tool call -> execution -> result -> model reply.
-response = chat.send_message("Screen wallet 0xd8dA... for risks.")
-print(response.text)
+response = client.models.generate_content(
+    model="gemini-2.5-flash",
+    contents="Screen wallet 0xd8dA... for risks.",
+    config=types.GenerateContentConfig(
+        tools=[tool],
+        system_instruction=system_instruction,
+    ),
+)
+
+for part in response.candidates[0].content.parts:
+    if part.function_call:
+        result = skill.execute(dict(part.function_call.args))
+        follow_up = client.models.generate_content(
+            model="gemini-2.5-flash",
+            contents=[
+                "Use this tool result to answer the original request.",
+                {
+                    "function_response": {
+                        "name": part.function_call.name,
+                        "response": {"result": result},
+                    }
+                },
+            ],
+            config=types.GenerateContentConfig(
+                tools=[tool],
+                system_instruction=system_instruction,
+            ),
+        )
+        print(follow_up.text)
+    else:
+        print(part.text)
 ```
 
+For other providers and shared integration patterns, see the [usage guides index](docs/usage/README.md), [agent loops](docs/usage/agent_loops.md), [Gemini](docs/usage/gemini.md), [Claude](docs/usage/claude.md), [OpenAI](docs/usage/openai.md), [DeepSeek](docs/usage/deepseek.md), [Ollama](docs/usage/ollama.md), [API keys for skills](docs/usage/api_keys.md), and the [skill usage template](docs/usage/skill_usage_template.md) for contributors.
+
 ## Documentation
 
-*   **[Core Logic & Philosophy](docs/introduction.md)**: Details on how Skillware decouples Logic, Cognition, and Governance.
-*   **[Testing Guidelines](docs/TESTING.md)**: Instructions for validating skills and checking local coverage.
-*   **[Usage guides](docs/usage/README.md)**: Provider adapters and navigation to integration docs.
-*   **[Agent loops](docs/usage/agent_loops.md)**: Shared load, tool-call, and `execute` pattern across providers.
-*   **[Usage Guide: Gemini](docs/usage/gemini.md)**: Integration with Google's GenAI SDK.
-*   **[Usage Guide: Claude](docs/usage/claude.md)**: Integration with Anthropic's SDK.
-*   **[Usage Guide: OpenAI](docs/usage/openai.md)**: Integration with OpenAI Chat Completions tool calling.
-*   **[Usage Guide: DeepSeek](docs/usage/deepseek.md)**: Integration with the DeepSeek API via `to_deepseek_tool()`.
-*   **[Usage Guide: Ollama](docs/usage/ollama.md)**: Native integration for local models via Ollama.
-*   **[API Keys for Skills](docs/usage/api_keys.md)**: Environment variables, cloud/CI setup, and security for skills that call external APIs.
-*   **[Skill Library](docs/skills/README.md)**: Available capabilities.
-*   **[CLI Reference](docs/usage/cli.md)**: Discover and inspect locally installed skills from the terminal.
+| Topic | Link |
+| :--- | :--- |
+| Introduction | [docs/introduction.md](docs/introduction.md) |
+| Changelog | [CHANGELOG.md](CHANGELOG.md) |
+| Testing | [docs/TESTING.md](docs/TESTING.md) |
+| Skill library | [docs/skills/README.md](docs/skills/README.md) |
+| Usage guides | [index](docs/usage/README.md) — [Gemini](docs/usage/gemini.md) · [Claude](docs/usage/claude.md) · [OpenAI](docs/usage/openai.md) · [DeepSeek](docs/usage/deepseek.md) · [Ollama](docs/usage/ollama.md); [agent loops](docs/usage/agent_loops.md); [API keys](docs/usage/api_keys.md) |
+| Examples index | [examples/README.md](examples/README.md) |
+| CLI | [docs/usage/cli.md](docs/usage/cli.md) |
+| Contributing | [CONTRIBUTING.md](CONTRIBUTING.md) · [agent workflow](docs/contributing/ai_native_workflow.md) |
 
 ## Contributing
 

{skillware-0.2.9 → skillware-0.3.2}/README.md

@@ -19,6 +19,7 @@
   <a href="#architecture">Architecture</a> •
   <a href="#quick-start">Quick Start</a> •
   <a href="#documentation">Documentation</a> •
+  <a href="CHANGELOG.md">Changelog</a> •
   <a href="#contributing">Contributing</a> •
   <a href="#comparison">Comparison</a> •
   <a href="#contact">Contact</a>
@@ -41,28 +42,34 @@ A **Skill** in this framework provides everything an Agent needs to master a dom
 3.  **Governance**: Constitution and safety boundaries.
 4.  **Interface**: Standardized schemas for LLM tool calling.
 
+### Skill library
+
+Browse capabilities by category in the [Skill library](docs/skills/README.md).
+
 ## Architecture
 
-This repository is organized into a core framework, a registry of skills, and documentation.
+This repository is organized into a core framework, a registry of skills, and
+documentation. Runnable provider scripts are indexed in
+[examples/README.md](examples/README.md).
 
 ```text
 Skillware/
-├── docs/                       # Comprehensive Documentation & Usage Guides
-├── examples/                   # Reference Implementations
-│   └── basic_agent.py          # Example showing SkillLoader integration
+├── docs/                       # Introduction, testing, skill catalog, usage guides (docs/usage/)
+├── examples/                   # Provider reference scripts (Gemini, Claude, OpenAI, Ollama, ...)
 ├── skills/                     # Skill Registry
 │   └── category/               # Domain boundaries (e.g., finance)
 │       └── skill_name/         # The Skill bundle
 │           ├── manifest.yaml   # Definition, schema, and constitution
 │           ├── skill.py        # Executable Python logic
 │           ├── instructions.md # Cognitive map for the LLM
-│           └── test_skill.py   # Unit tests & schema validation
+│           ├── card.json       # Optional UI presentation metadata
+│           └── test_skill.py   # Unit tests and schema validation
 ├── skillware/                  # Core Framework Package
 │   ├── cli.py                  # Command-line interface
 │   └── core/
 │       ├── base_skill.py       # Abstract Base Class for skills
 │       ├── env.py              # Environment Management
-│       └── loader.py           # Universal Skill Loader & Model Adapter
+│       └── loader.py           # Universal Skill Loader and Model Adapter
 ├── templates/                  # Boilerplate templates for new skills
 │   └── python_skill/           # Standard template with required files
 └── tests/                      # Automated test suite
@@ -95,7 +102,7 @@ pip install "skillware[cli]"
 skillware list
 ```
 
-This prints a table of all locally available skills and confirms the install and path resolution are working.
+This prints a table of all locally available skills and confirms the install and path resolution are working. Running `skillware` with no arguments opens the interactive menu.
 
 ### 3. Configuration
 
@@ -107,8 +114,12 @@ GOOGLE_API_KEY="your_key"
 
 ### 4. Usage Example (Gemini)
 
+This example requires the Google SDK optional extra: `pip install "skillware[gemini]"` (local dev: `pip install -e ".[gemini]"`). See the [Gemini usage guide](docs/usage/gemini.md) for setup details.
+
 ```python
-import google.generativeai as genai
+import os
+import google.genai as genai
+from google.genai import types
 from skillware.core.loader import SkillLoader
 from skillware.core.env import load_env_file
 
@@ -117,36 +128,64 @@ load_env_file()
 
 # 1. Load the Skill from the Registry
 # The loader reads the code, manifest, and instructions automatically
-skill_bundle = SkillLoader.load_skill("category/skill_name")  # see docs/usage/README.md for path search order
-
-# 2. Model & Chat Setup
-model = genai.GenerativeModel(
-    'gemini-2.5-flash',
-    tools=[SkillLoader.to_gemini_tool(skill_bundle)], # The "Adapter"
-    system_instruction=skill_bundle['instructions']   # The "Mind"
+skill_bundle = SkillLoader.load_skill("finance/wallet_screening")
+skill = skill_bundle["module"].WalletScreeningSkill(
+    config={"ETHERSCAN_API_KEY": os.environ.get("ETHERSCAN_API_KEY")}
 )
-chat = model.start_chat(enable_automatic_function_calling=True)
+
+# 2. Client & Tool Setup
+client = genai.Client()
+tool = SkillLoader.to_gemini_tool(skill_bundle)       # The "Adapter"
+system_instruction = skill_bundle['instructions']     # The "Mind"
 
 # 3. Agent Loop
-# The SDK handles the loop: model -> tool call -> execution -> result -> model reply.
-response = chat.send_message("Screen wallet 0xd8dA... for risks.")
-print(response.text)
+response = client.models.generate_content(
+    model="gemini-2.5-flash",
+    contents="Screen wallet 0xd8dA... for risks.",
+    config=types.GenerateContentConfig(
+        tools=[tool],
+        system_instruction=system_instruction,
+    ),
+)
+
+for part in response.candidates[0].content.parts:
+    if part.function_call:
+        result = skill.execute(dict(part.function_call.args))
+        follow_up = client.models.generate_content(
+            model="gemini-2.5-flash",
+            contents=[
+                "Use this tool result to answer the original request.",
+                {
+                    "function_response": {
+                        "name": part.function_call.name,
+                        "response": {"result": result},
+                    }
+                },
+            ],
+            config=types.GenerateContentConfig(
+                tools=[tool],
+                system_instruction=system_instruction,
+            ),
+        )
+        print(follow_up.text)
+    else:
+        print(part.text)
 ```
 
+For other providers and shared integration patterns, see the [usage guides index](docs/usage/README.md), [agent loops](docs/usage/agent_loops.md), [Gemini](docs/usage/gemini.md), [Claude](docs/usage/claude.md), [OpenAI](docs/usage/openai.md), [DeepSeek](docs/usage/deepseek.md), [Ollama](docs/usage/ollama.md), [API keys for skills](docs/usage/api_keys.md), and the [skill usage template](docs/usage/skill_usage_template.md) for contributors.
+
 ## Documentation
 
-*   **[Core Logic & Philosophy](docs/introduction.md)**: Details on how Skillware decouples Logic, Cognition, and Governance.
-*   **[Testing Guidelines](docs/TESTING.md)**: Instructions for validating skills and checking local coverage.
-*   **[Usage guides](docs/usage/README.md)**: Provider adapters and navigation to integration docs.
-*   **[Agent loops](docs/usage/agent_loops.md)**: Shared load, tool-call, and `execute` pattern across providers.
-*   **[Usage Guide: Gemini](docs/usage/gemini.md)**: Integration with Google's GenAI SDK.
-*   **[Usage Guide: Claude](docs/usage/claude.md)**: Integration with Anthropic's SDK.
-*   **[Usage Guide: OpenAI](docs/usage/openai.md)**: Integration with OpenAI Chat Completions tool calling.
-*   **[Usage Guide: DeepSeek](docs/usage/deepseek.md)**: Integration with the DeepSeek API via `to_deepseek_tool()`.
-*   **[Usage Guide: Ollama](docs/usage/ollama.md)**: Native integration for local models via Ollama.
-*   **[API Keys for Skills](docs/usage/api_keys.md)**: Environment variables, cloud/CI setup, and security for skills that call external APIs.
-*   **[Skill Library](docs/skills/README.md)**: Available capabilities.
-*   **[CLI Reference](docs/usage/cli.md)**: Discover and inspect locally installed skills from the terminal.
+| Topic | Link |
+| :--- | :--- |
+| Introduction | [docs/introduction.md](docs/introduction.md) |
+| Changelog | [CHANGELOG.md](CHANGELOG.md) |
+| Testing | [docs/TESTING.md](docs/TESTING.md) |
+| Skill library | [docs/skills/README.md](docs/skills/README.md) |
+| Usage guides | [index](docs/usage/README.md) — [Gemini](docs/usage/gemini.md) · [Claude](docs/usage/claude.md) · [OpenAI](docs/usage/openai.md) · [DeepSeek](docs/usage/deepseek.md) · [Ollama](docs/usage/ollama.md); [agent loops](docs/usage/agent_loops.md); [API keys](docs/usage/api_keys.md) |
+| Examples index | [examples/README.md](examples/README.md) |
+| CLI | [docs/usage/cli.md](docs/usage/cli.md) |
+| Contributing | [CONTRIBUTING.md](CONTRIBUTING.md) · [agent workflow](docs/contributing/ai_native_workflow.md) |
 
 ## Contributing
 

{skillware-0.2.9 → skillware-0.3.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "skillware"
-version = "0.2.9"
+version = "0.3.2"
 description = "A framework for modular, self-contained AI skills."
 readme = "README.md"
 authors = [
@@ -39,7 +39,7 @@ cli = [
     "rich>=13.0",
 ]
 gemini = [
-    "google-generativeai",
+    "google-genai",
 ]
 claude = [
     "anthropic",
@@ -52,7 +52,7 @@ office = [
 ]
 all = [
     "rich>=13.0",
-    "google-generativeai",
+    "google-genai",
     "anthropic",
     "openai",
     "pymupdf",

{skillware-0.2.9 → skillware-0.3.2}/skills/compliance/mica_module/manifest.yaml

@@ -1,6 +1,7 @@
 name: compliance/mica_module
 version: 0.1.0
 description: A highly specialized, localized RAG and policy enforcement engine for the Markets in Crypto-Assets (MiCA) regulation.
+short_description: "Local RAG and policy enforcement engine for MiCA regulation."
 issuer:
   name: Ross Peili
   email: vpeilivanidis@gmail.com
@@ -21,7 +22,7 @@ parameters:
   required:
     - user_prompt
 requirements:
-  - google.generativeai
+  - google-genai
 constitution: |
   1. COMPLIANCE: Do not invent or hallucinate financial regulations. Base all enforcement strictly on the provided MiCA corpus blocks.
   2. FOCUS: Act exclusively as a compliance firewall and RAG engine; retrieve relevant chunks incrementally.

{skillware-0.2.9 → skillware-0.3.2}/skills/compliance/mica_module/skill.py

@@ -1,7 +1,7 @@
 import json
 import os
 from typing import Any, Dict, List
-import google.generativeai as genai
+
 from skillware.core.base_skill import BaseSkill
 
 
@@ -207,11 +207,43 @@ class MiCAModuleSkill(BaseSkill):
         """
 
         try:
-            model = genai.GenerativeModel(model_name)
-            resp = model.generate_content(
-                prompt_payload,
-                generation_config=genai.GenerationConfig(
-                    response_mime_type="application/json", temperature=0.0
+            import google.genai as genai
+            from google.genai import types
+        except ImportError:
+            return {
+                "policy_status": "CAUTION",
+                "gemini_evaluator_feedback": {
+                    "grade": "N/A",
+                    "holes_found": "google-genai is not installed.",
+                    "suggestion": "Proceed manually integrating the extracted logic.",
+                },
+                "final_context_for_agent": (
+                    f"Output your final answer seamlessly integrating and adhering to these MiCA rules:\n{context}"
+                ),
+            }
+
+        api_key = os.environ.get("GOOGLE_API_KEY")
+        if not api_key:
+            return {
+                "policy_status": "CAUTION",
+                "gemini_evaluator_feedback": {
+                    "grade": "N/A",
+                    "holes_found": "GOOGLE_API_KEY is not configured.",
+                    "suggestion": "Proceed manually integrating the extracted logic.",
+                },
+                "final_context_for_agent": (
+                    f"Output your final answer seamlessly integrating and adhering to these MiCA rules:\n{context}"
+                ),
+            }
+
+        try:
+            client = genai.Client(api_key=api_key)
+            resp = client.models.generate_content(
+                model=model_name,
+                contents=prompt_payload,
+                config=types.GenerateContentConfig(
+                    response_mime_type="application/json",
+                    temperature=0.0,
                 ),
             )
             return json.loads(resp.text)

{skillware-0.2.9 → skillware-0.3.2}/skills/compliance/pii_masker/manifest.yaml

@@ -1,6 +1,7 @@
 name: compliance/pii_masker
 version: 0.1.0
 description: High-precision, local PII detection and redaction using the micro-f1-mask model via Ollama.
+short_description: "Detects and redacts PII locally using the micro-f1-mask model."
 issuer:
   name: Ross Peili
   email: vpeilivanidis@gmail.com

{skillware-0.2.9 → skillware-0.3.2}/skills/compliance/tos_evaluator/manifest.yaml

@@ -1,6 +1,7 @@
 name: "compliance/tos_evaluator"
 version: "0.1.0"
 description: "Evaluates whether an intended automated website action appears permissible by checking robots.txt, discovered legal pages, and optional low-cost LLM review for ambiguous policy language."
+short_description: "Evaluates whether an automated web action is permitted by site policy."
 issuer:
   name: Ross Peili
   email: vpeilivanidis@gmail.com

{skillware-0.2.9 → skillware-0.3.2}/skills/compliance/tos_evaluator/skill.py

@@ -11,9 +11,11 @@ from bs4 import BeautifulSoup
 from skillware.core.base_skill import BaseSkill
 
 try:
-    import google.generativeai as genai
+    import google.genai as genai
+    from google.genai import types
 except ImportError:  # pragma: no cover - dependency is optional at runtime
     genai = None
+    types = None
 
 
 class TOSEvaluatorSkill(BaseSkill):
@@ -458,14 +460,13 @@ class TOSEvaluatorSkill(BaseSkill):
                 "status": "skipped",
                 "reason": f"Unsupported llm_provider '{normalized['llm_provider']}'.",
             }
-        if genai is None:
-            return {"status": "skipped", "reason": "google-generativeai is not installed."}
+        if genai is None or types is None:
+            return {"status": "skipped", "reason": "google-genai is not installed."}
 
         api_key = os.environ.get("GOOGLE_API_KEY")
         if not api_key:
             return {"status": "skipped", "reason": "GOOGLE_API_KEY is not configured."}
 
-        genai.configure(api_key=api_key)
         prompt = {
             "target_url": normalized["target_url"],
             "intended_action": normalized["intended_action"],
@@ -479,11 +480,13 @@ class TOSEvaluatorSkill(BaseSkill):
         }
 
         try:
-            model = genai.GenerativeModel(normalized["llm_model"])
-            response = model.generate_content(
-                json.dumps(prompt, ensure_ascii=True),
-                generation_config=genai.GenerationConfig(
-                    response_mime_type="application/json", temperature=0.0
+            client = genai.Client(api_key=api_key)
+            response = client.models.generate_content(
+                model=normalized["llm_model"],
+                contents=json.dumps(prompt, ensure_ascii=True),
+                config=types.GenerateContentConfig(
+                    response_mime_type="application/json",
+                    temperature=0.0,
                 ),
             )
             parsed = json.loads(response.text)

skillware-0.3.2/skills/data_engineering/novelty_extractor/__init__.py

@@ -0,0 +1,3 @@
+from .skill import NoveltyExtractor
+
+__all__ = ["NoveltyExtractor"]

skillware-0.3.2/skills/data_engineering/novelty_extractor/card.json

@@ -0,0 +1,37 @@
+{
+  "name": "Novelty Extractor",
+  "description": "Filters a text dataset by semantic novelty, retaining only chunks that carry new information above a configurable threshold.",
+  "icon": "filter",
+  "ui_schema": {
+    "dataset_chunk": {
+      "ui:widget": "textarea",
+      "ui:placeholder": "Paste your raw text dataset here..."
+    },
+    "novelty_threshold": {
+      "ui:widget": "range",
+      "ui:options": {
+        "min": 0.0,
+        "max": 1.0,
+        "step": 0.05
+      }
+    },
+    "baseline_chunks": {
+      "ui:widget": "textarea",
+      "ui:placeholder": "Optional. Paste distilled_content from a previous call here..."
+    },
+    "chunk_strategy": {
+      "ui:widget": "select",
+      "ui:options": {
+        "enumOptions": [
+          {"value": "paragraph", "label": "Paragraph"},
+          {"value": "sentence", "label": "Sentence"}
+        ]
+      }
+    }
+  },
+  "issuer": {
+    "name": "Martin Rizzo Lozano",
+    "email": "rizzolozanomartin@gmail.com",
+    "github": "rizzoMartin"
+  }
+}

skillware-0.3.2/skills/data_engineering/novelty_extractor/instructions.md

@@ -0,0 +1,60 @@
+# Novelty Extractor
+
+You are an agent equipped with the Novelty Extractor skill. This skill filters
+large text datasets by semantic similarity, retaining only chunks that carry
+genuinely new information.
+
+## When to invoke this skill
+
+Invoke `data_engineering/novelty_extractor` when:
+
+- You need to distill a large corpus into high-signal content before passing
+  it to a model or storing it.
+- You are processing a dataset in multiple turns and want to avoid redundancy
+  across turns by passing prior output as `baseline_chunks`.
+- A user asks you to remove repetitive or redundant content from a document
+  or dataset.
+
+## Parameters
+
+- `dataset_chunk` (required): The raw text to filter.
+- `novelty_threshold` (required): A float between 0.0 and 1.0. Chunks with
+  maximum cosine similarity to already-seen content below this value are kept.
+  Lower values mean stricter filtering. A value of 0.85 is a sensible default
+  for most corpora.
+- `baseline_chunks` (optional): Text from a previous call's `distilled_content`.
+  Pass this to filter consistently across multiple turns.
+- `chunk_strategy` (optional): How to split the input. Use `"paragraph"`
+  (default) for documents with clear paragraph breaks, or `"sentence"` for
+  denser text.
+
+## Interpreting the output
+
+- `distilled_content`: The filtered text. Pass this to downstream tasks or
+  store it as the new baseline for the next turn.
+- `compression_ratio`: The percentage of input chunks that were dropped. A
+  high ratio (above 80%) may indicate the threshold is too strict or the
+  input is highly repetitive.
+- `redundant_chunks_dropped`: The raw count of discarded chunks. Use this
+  to audit filtering behavior.
+
+## Handling edge cases
+
+- If `distilled_content` is empty and no error is returned, all input chunks
+  were too similar to the baseline. Consider lowering `novelty_threshold`.
+- If an `error` key is present in the output, the skill encountered an
+  unexpected failure. Inspect the value and retry with sanitized input.
+- The first call without a baseline always keeps the first chunk regardless
+  of threshold, since there is nothing to compare against.
+
+## Multi-turn example
+
+Turn 1: pass the first batch without baseline.
+Turn 2: pass the next batch with `baseline_chunks` set to the previous
+`distilled_content`.
+Repeat until the full corpus is processed.
+
+Note: the skill only filters each batch against the provided baseline. It does
+not accumulate results across turns automatically. The caller is responsible
+for concatenating `distilled_content` from each turn to build the full
+distilled dataset.