dynamic-skill-compiler 0.1.0__tar.gz

dynamic_skill_compiler-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 ZJUNLP
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

dynamic_skill_compiler-0.1.0/MANIFEST.in

@@ -0,0 +1,12 @@
+include README.md
+include LICENSE
+include pyproject.toml
+recursive-include src/dynamic_skill_compiler *.py py.typed
+recursive-include tests *.py
+recursive-include reports *.md
+prune experiments/results
+prune results
+prune .cache
+prune .venv-experiments
+prune skillnet-ai
+global-exclude __pycache__ *.py[cod] .DS_Store

dynamic_skill_compiler-0.1.0/PKG-INFO

@@ -0,0 +1,155 @@
+Metadata-Version: 2.4
+Name: dynamic-skill-compiler
+Version: 0.1.0
+Summary: Task-driven compiler for agent skill libraries.
+Author: ZJUNLP
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/taomiao/DynamicSkillCompiler
+Project-URL: Repository, https://github.com/taomiao/DynamicSkillCompiler
+Project-URL: Issues, https://github.com/taomiao/DynamicSkillCompiler/issues
+Keywords: agent,skills,compiler,retrieval,planning
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: semantic
+Requires-Dist: numpy>=1.24; extra == "semantic"
+Requires-Dist: openai>=1.109.1; extra == "semantic"
+Provides-Extra: dev
+Requires-Dist: build>=1.2.1; extra == "dev"
+Requires-Dist: twine>=5.1.1; extra == "dev"
+Provides-Extra: all
+Requires-Dist: numpy>=1.24; extra == "all"
+Requires-Dist: openai>=1.109.1; extra == "all"
+Dynamic: license-file
+
+# Dynamic Skill Compiler
+
+Dynamic Skill Compiler (DSC) is a task-driven compiler for agent skills. It takes a natural-language task, retrieves candidate skills from a local skill library, builds a skill graph, and emits a compact, dependency-aware skill package for execution.
+
+## What DSC Does
+
+- Decomposes a task into subgoals and required capabilities.
+- Retrieves local skill assets and their declared relationships.
+- Extracts relevant skill fragments instead of passing whole skill files.
+- Builds a graph over `similar_to`, `belong_to`, `compose_with`, and `depend_on` relationships.
+- Runs compiler passes to select, repair, augment, and prune the skill set.
+- Produces a `CompiledSkillPackage` with selected skills, execution order, coverage metrics, and pass traces.
+
+## Repository Layout
+
+```text
+src/dynamic_skill_compiler/   Core DSC compiler package
+tests/                        Compiler and experiment integration tests
+experiments/                  Benchmark runners and local skill libraries
+reports/                      DSC design notes
+```
+
+The benchmark runners under `experiments/` are kept as evaluation harnesses for ScienceWorld, ALFWorld, and WebShop. They are intentionally separate from the core compiler package.
+
+## Quick Start
+
+Install from a local checkout:
+
+```bash
+python -m pip install -e .
+```
+
+After the package is published to PyPI, users can install it with:
+
+```bash
+python -m pip install dynamic-skill-compiler
+```
+
+Semantic embedding support is optional:
+
+```bash
+python -m pip install "dynamic-skill-compiler[semantic]"
+```
+
+```python
+from dynamic_skill_compiler import (
+    CompilerConfig,
+    DynamicSkillCompiler,
+    LocalEnvironment,
+    LocalSkillLibraryRetriever,
+)
+
+retriever = LocalSkillLibraryRetriever("experiments/src/skills/scienceworld")
+compiler = DynamicSkillCompiler(
+    retriever=retriever,
+    config=CompilerConfig(),
+)
+
+compiled = compiler.compile(
+    "Your task is to measure the temperature of an unknown substance.",
+    environment=LocalEnvironment(benchmark="scienceworld"),
+)
+
+print([skill.asset.name for skill in compiled.compiled_skills])
+print(compiled.execution_order)
+print(compiled.metrics.coverage_score)
+```
+
+## CLI
+
+The package installs a `dsc` command for quick local compilation:
+
+```bash
+dsc "Measure the temperature of the unknown object." \
+  --skills-dir experiments/src/skills/scienceworld \
+  --pretty
+```
+
+The command prints a JSON summary containing selected skills, execution order,
+coverage metrics, compiler pass traces, and dropped-skill reasons.
+
+## Build And Publish
+
+Build source and wheel distributions:
+
+```bash
+python -m pip install ".[dev]"
+python -m build
+python -m twine check dist/*
+```
+
+Publish to TestPyPI first:
+
+```bash
+python -m twine upload --repository testpypi dist/*
+```
+
+Publish to PyPI:
+
+```bash
+python -m twine upload dist/*
+```
+
+Use a PyPI API token via `TWINE_USERNAME=__token__` and
+`TWINE_PASSWORD=pypi-...` or through your local `.pypirc`.
+
+## Tests
+
+```bash
+PYTHONPATH=src python -m unittest discover -s tests -p "test_*.py"
+
+PYTHONPATH=src:experiments/src .venv-experiments/bin/python \
+  experiments/test_runtime_execution_guard.py
+```
+
+## Branches
+
+- `main`: standalone DSC codebase.
+- `codex/v0321`: latest DSC development branch mirrored into `main`.

dynamic_skill_compiler-0.1.0/README.md

@@ -0,0 +1,119 @@
+# Dynamic Skill Compiler
+
+Dynamic Skill Compiler (DSC) is a task-driven compiler for agent skills. It takes a natural-language task, retrieves candidate skills from a local skill library, builds a skill graph, and emits a compact, dependency-aware skill package for execution.
+
+## What DSC Does
+
+- Decomposes a task into subgoals and required capabilities.
+- Retrieves local skill assets and their declared relationships.
+- Extracts relevant skill fragments instead of passing whole skill files.
+- Builds a graph over `similar_to`, `belong_to`, `compose_with`, and `depend_on` relationships.
+- Runs compiler passes to select, repair, augment, and prune the skill set.
+- Produces a `CompiledSkillPackage` with selected skills, execution order, coverage metrics, and pass traces.
+
+## Repository Layout
+
+```text
+src/dynamic_skill_compiler/   Core DSC compiler package
+tests/                        Compiler and experiment integration tests
+experiments/                  Benchmark runners and local skill libraries
+reports/                      DSC design notes
+```
+
+The benchmark runners under `experiments/` are kept as evaluation harnesses for ScienceWorld, ALFWorld, and WebShop. They are intentionally separate from the core compiler package.
+
+## Quick Start
+
+Install from a local checkout:
+
+```bash
+python -m pip install -e .
+```
+
+After the package is published to PyPI, users can install it with:
+
+```bash
+python -m pip install dynamic-skill-compiler
+```
+
+Semantic embedding support is optional:
+
+```bash
+python -m pip install "dynamic-skill-compiler[semantic]"
+```
+
+```python
+from dynamic_skill_compiler import (
+    CompilerConfig,
+    DynamicSkillCompiler,
+    LocalEnvironment,
+    LocalSkillLibraryRetriever,
+)
+
+retriever = LocalSkillLibraryRetriever("experiments/src/skills/scienceworld")
+compiler = DynamicSkillCompiler(
+    retriever=retriever,
+    config=CompilerConfig(),
+)
+
+compiled = compiler.compile(
+    "Your task is to measure the temperature of an unknown substance.",
+    environment=LocalEnvironment(benchmark="scienceworld"),
+)
+
+print([skill.asset.name for skill in compiled.compiled_skills])
+print(compiled.execution_order)
+print(compiled.metrics.coverage_score)
+```
+
+## CLI
+
+The package installs a `dsc` command for quick local compilation:
+
+```bash
+dsc "Measure the temperature of the unknown object." \
+  --skills-dir experiments/src/skills/scienceworld \
+  --pretty
+```
+
+The command prints a JSON summary containing selected skills, execution order,
+coverage metrics, compiler pass traces, and dropped-skill reasons.
+
+## Build And Publish
+
+Build source and wheel distributions:
+
+```bash
+python -m pip install ".[dev]"
+python -m build
+python -m twine check dist/*
+```
+
+Publish to TestPyPI first:
+
+```bash
+python -m twine upload --repository testpypi dist/*
+```
+
+Publish to PyPI:
+
+```bash
+python -m twine upload dist/*
+```
+
+Use a PyPI API token via `TWINE_USERNAME=__token__` and
+`TWINE_PASSWORD=pypi-...` or through your local `.pypirc`.
+
+## Tests
+
+```bash
+PYTHONPATH=src python -m unittest discover -s tests -p "test_*.py"
+
+PYTHONPATH=src:experiments/src .venv-experiments/bin/python \
+  experiments/test_runtime_execution_guard.py
+```
+
+## Branches
+
+- `main`: standalone DSC codebase.
+- `codex/v0321`: latest DSC development branch mirrored into `main`.

dynamic_skill_compiler-0.1.0/pyproject.toml

@@ -0,0 +1,63 @@
+[build-system]
+requires = ["setuptools>=68.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "dynamic-skill-compiler"
+version = "0.1.0"
+description = "Task-driven compiler for agent skill libraries."
+readme = "README.md"
+authors = [{ name = "ZJUNLP" }]
+license = "MIT"
+license-files = ["LICENSE"]
+requires-python = ">=3.9"
+keywords = [
+    "agent",
+    "skills",
+    "compiler",
+    "retrieval",
+    "planning",
+]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Science/Research",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+dependencies = []
+
+[project.optional-dependencies]
+semantic = [
+    "numpy>=1.24",
+    "openai>=1.109.1",
+]
+dev = [
+    "build>=1.2.1",
+    "twine>=5.1.1",
+]
+all = [
+    "numpy>=1.24",
+    "openai>=1.109.1",
+]
+
+[project.urls]
+Homepage = "https://github.com/taomiao/DynamicSkillCompiler"
+Repository = "https://github.com/taomiao/DynamicSkillCompiler"
+Issues = "https://github.com/taomiao/DynamicSkillCompiler/issues"
+
+[project.scripts]
+dsc = "dynamic_skill_compiler.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+dynamic_skill_compiler = ["py.typed"]

dynamic_skill_compiler-0.1.0/reports/dynamic_skill_compiler_design.md

@@ -0,0 +1,99 @@
+# Dynamic Skill Compiler Design
+
+## Goal
+
+Dynamic Skill Compiler (DSC) is a task-driven compiler for local agent skill libraries. Instead of passing a raw top-k skill list into the agent, DSC builds a task-specific skill graph and emits a compact, dependency-aware package that is easier for the executor to use.
+
+## Compiler Pipeline
+
+1. Query optimization
+   - Normalize the raw task.
+   - Extract compact retrieval queries.
+   - Generate semantic queries for broader recall.
+   - Infer constraints such as low-token, local-only, or evaluation-heavy tasks.
+
+2. Query understanding
+   - Convert the task into a `QueryPlan`.
+   - Separate required capabilities from optional capabilities.
+   - Preserve benchmark and environment constraints for downstream scoring.
+
+3. Skill retrieval
+   - Retrieve candidates from local skill libraries.
+   - Merge and deduplicate candidates into a unified pool.
+   - Use semantic and lexical signals when embeddings are available.
+
+4. Skill graph construction
+   - Preserve declared `depend_on`, `belong_to`, `compose_with`, and `similar_to` edges.
+   - Infer extra `similar_to` edges from capability overlap.
+   - Build a multi-relational graph over all candidates.
+
+5. Task-aware compilation
+   - Score each candidate by task coverage, quality, cost, and latency.
+   - Remove low-relevance nodes.
+   - Collapse redundant `similar_to` clusters while keeping the best utility path.
+   - Prune broad parent skills when more specific children satisfy the task.
+   - Reintroduce required dependencies to maintain executability.
+   - Trim isolated low-value nodes.
+
+6. Localization and compression
+   - Rewrite generic commands and placeholders against the active workspace.
+   - Resolve `{cwd}`, `{workspace_root}`, and Python binary selection.
+   - Select relevant fragments instead of always passing whole skill files.
+   - Preserve authoritative reference guidance when quality-first mode is enabled.
+
+7. Compilation output
+   - A compiled subgraph.
+   - An execution order.
+   - A dropped-skill audit trail.
+   - Coverage, redundancy, token-cost, and pass-trace metrics.
+
+## Intended Gains Over Direct Retrieval
+
+DSC targets gains in:
+
+- Safety and executability
+  - Dependency closure after pruning prevents broken packages.
+- Completeness
+  - Task coverage is explicitly measured before finalizing the package.
+- Maintainability
+  - Redundant overlapping skills are removed, reducing graph sprawl.
+- Cost awareness
+  - Candidate selection prefers lower token and lower execution cost skills when coverage is similar.
+- Token efficiency
+  - The compiler removes unrelated, overlapping, and overly broad skills before packaging.
+
+## Current Implementation Scope
+
+The current implementation provides:
+
+- Query optimizer
+- In-memory and local-library retrievers
+- Optional semantic retrieval cache
+- Multi-relational graph builder
+- Heuristic compiler with redundancy pruning and dependency repair
+- Environment localization
+- Fragment-level skill compression
+- Unit tests for the core compiler logic
+
+## Experimental Upgrades
+
+The benchmark harness can evaluate DSC on ScienceWorld, ALFWorld, and WebShop with these ablations:
+
+- no query optimization
+- no graph pruning
+- no localization
+- no dependency repair
+- no fragment compression
+- direct retrieval baseline
+
+## Suggested Paper Framing
+
+- Baseline: direct retrieval plus raw top-k skill use
+- Method: Dynamic Skill Compiler
+- Main claim: compile-time optimization of skill graphs improves task success and token efficiency
+- Key measured outcomes:
+  - task success
+  - prompt token usage
+  - selected skill count
+  - graph density after compilation
+  - execution latency

dynamic_skill_compiler-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

dynamic_skill_compiler-0.1.0/src/dynamic_skill_compiler/__init__.py

@@ -0,0 +1,66 @@
+from dynamic_skill_compiler.models import (
+    CompilationMetrics,
+    CompiledSkill,
+    CompiledSkillPackage,
+    CompilerPassTrace,
+    LocalEnvironment,
+    QueryPlan,
+    SkillAsset,
+    SkillFragment,
+    SkillGraph,
+    SkillRelation,
+    Subgoal,
+)
+from dynamic_skill_compiler.decompose import TaskDecomposer
+from dynamic_skill_compiler.fragments import FragmentMatcher, SkillFragmentExtractor
+from dynamic_skill_compiler.graph import (
+    DEFAULT_GRAPH_PASSES,
+    GRAPH_PASS_PRESETS,
+    LEGACY_DEFAULT_GRAPH_PASSES,
+    SLIM_GRAPH_PASSES,
+    SUPPORTED_GRAPH_PASSES,
+)
+from dynamic_skill_compiler.grounding import EnvironmentGrounder
+from dynamic_skill_compiler.pipeline import CompilerConfig, DynamicSkillCompiler
+from dynamic_skill_compiler.semantic import SemanticSoftMatcher
+from dynamic_skill_compiler.query import QueryOptimizer
+from dynamic_skill_compiler.retriever import (
+    CompositeSkillRetriever,
+    InMemorySkillRetriever,
+    LocalSkillLibraryRetriever,
+    SkillRetriever,
+)
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "CompilationMetrics",
+    "CompiledSkill",
+    "CompiledSkillPackage",
+    "CompilerPassTrace",
+    "CompilerConfig",
+    "CompositeSkillRetriever",
+    "DEFAULT_GRAPH_PASSES",
+    "DynamicSkillCompiler",
+    "EnvironmentGrounder",
+    "FragmentMatcher",
+    "GRAPH_PASS_PRESETS",
+    "InMemorySkillRetriever",
+    "LEGACY_DEFAULT_GRAPH_PASSES",
+    "LocalSkillLibraryRetriever",
+    "LocalEnvironment",
+    "QueryOptimizer",
+    "QueryPlan",
+    "SkillAsset",
+    "SkillFragment",
+    "SkillFragmentExtractor",
+    "SkillGraph",
+    "SkillRelation",
+    "SkillRetriever",
+    "SLIM_GRAPH_PASSES",
+    "Subgoal",
+    "SemanticSoftMatcher",
+    "SUPPORTED_GRAPH_PASSES",
+    "TaskDecomposer",
+    "__version__",
+]

dynamic_skill_compiler-0.1.0/src/dynamic_skill_compiler/cli.py

@@ -0,0 +1,141 @@
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from dataclasses import asdict, is_dataclass
+from typing import Any
+
+from dynamic_skill_compiler.models import LocalEnvironment
+from dynamic_skill_compiler.pipeline import CompilerConfig, DynamicSkillCompiler
+from dynamic_skill_compiler.retriever import LocalSkillLibraryRetriever
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="dsc",
+        description="Compile a local skill library into a task-specific DSC package.",
+    )
+    parser.add_argument(
+        "query",
+        nargs="?",
+        help="Task query to compile. If omitted, the query is read from stdin.",
+    )
+    parser.add_argument(
+        "--skills-dir",
+        required=True,
+        help="Directory containing local skill folders with SKILL.md files.",
+    )
+    parser.add_argument(
+        "--benchmark",
+        default="generic",
+        help="Optional environment/benchmark label used by adaptive compiler profiles.",
+    )
+    parser.add_argument(
+        "--cwd",
+        default=".",
+        help="Execution working directory used for localization.",
+    )
+    parser.add_argument(
+        "--workspace-root",
+        default=".",
+        help="Workspace root used for localization.",
+    )
+    parser.add_argument(
+        "--python-bin",
+        default=sys.executable or "python",
+        help="Python executable name/path used for localization.",
+    )
+    parser.add_argument(
+        "--min-relevance",
+        type=float,
+        default=CompilerConfig.min_relevance,
+        help="Minimum utility score required for skill selection.",
+    )
+    parser.add_argument(
+        "--preserve-top-k",
+        type=int,
+        default=CompilerConfig.preserve_top_k,
+        help="Always preserve at least this many top-scored skills.",
+    )
+    parser.add_argument(
+        "--max-selected-skills",
+        type=int,
+        default=CompilerConfig.max_selected_skills,
+        help="Hard cap for selected skills. Use 0 for no explicit cap.",
+    )
+    parser.add_argument(
+        "--pretty",
+        action="store_true",
+        help="Pretty-print JSON output.",
+    )
+    parser.add_argument(
+        "--include-instructions",
+        action="store_true",
+        help="Include localized selected instructions in the JSON output.",
+    )
+    return parser
+
+
+def compile_from_args(args: argparse.Namespace) -> dict[str, Any]:
+    query = args.query if args.query is not None else sys.stdin.read().strip()
+    if not query:
+        raise SystemExit("A task query is required, either as an argument or on stdin.")
+
+    compiler = DynamicSkillCompiler(
+        retriever=LocalSkillLibraryRetriever(args.skills_dir),
+        config=CompilerConfig(
+            min_relevance=args.min_relevance,
+            preserve_top_k=args.preserve_top_k,
+            max_selected_skills=args.max_selected_skills,
+        ),
+    )
+    compiled = compiler.compile(
+        query,
+        environment=LocalEnvironment(
+            cwd=args.cwd,
+            workspace_root=args.workspace_root,
+            python_bin=args.python_bin,
+            benchmark=args.benchmark,
+        ),
+    )
+    summary = compiler.summarize(compiled)
+    summary["metrics"] = _to_jsonable(compiled.metrics)
+    if args.include_instructions:
+        summary["compiled_skills"] = [
+            {
+                "name": item.asset.name,
+                "skill_id": item.asset.skill_id,
+                "selected_reason": item.selected_reason,
+                "utility_score": item.utility_score,
+                "localized_instructions": item.localized_instructions,
+                "selected_fragments": [_to_jsonable(fragment) for fragment in item.selected_fragments],
+            }
+            for item in compiled.compiled_skills
+        ]
+    return summary
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = build_parser()
+    args = parser.parse_args(argv)
+    summary = compile_from_args(args)
+    indent = 2 if args.pretty else None
+    print(json.dumps(summary, ensure_ascii=False, indent=indent))
+    return 0
+
+
+def _to_jsonable(value: Any) -> Any:
+    if is_dataclass(value):
+        return _to_jsonable(asdict(value))
+    if isinstance(value, dict):
+        return {str(key): _to_jsonable(item) for key, item in value.items()}
+    if isinstance(value, (list, tuple)):
+        return [_to_jsonable(item) for item in value]
+    if isinstance(value, set):
+        return sorted(_to_jsonable(item) for item in value)
+    return value
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())