aicl-lang 0.1.0__tar.gz

aicl_lang-0.1.0/LICENSE

@@ -0,0 +1,27 @@
+MIT License
+
+Copyright (c) 2026 AICL contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+---
+
+The specification (SPEC.md) is additionally licensed under
+Creative Commons Attribution 4.0 International (CC BY 4.0):
+https://creativecommons.org/licenses/by/4.0/

aicl_lang-0.1.0/PKG-INFO

@@ -0,0 +1,352 @@
+Metadata-Version: 2.4
+Name: aicl-lang
+Version: 0.1.0
+Summary: AICL: a compact, unambiguous language for agent-to-agent communication (spec + reference implementation)
+Author: AICL contributors
+License: MIT
+Project-URL: Homepage, https://github.com/shotaros2/aicl
+Project-URL: Issues, https://github.com/shotaros2/aicl/issues
+Project-URL: Specification, https://github.com/shotaros2/aicl/blob/main/SPEC.md
+Keywords: agents,llm,protocol,interlingua,multi-agent,speech-acts
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: tokens
+Requires-Dist: tiktoken>=0.7; extra == "tokens"
+Provides-Extra: llm
+Requires-Dist: anthropic>=1.0; extra == "llm"
+Requires-Dist: google-genai>=1.0; extra == "llm"
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Dynamic: license-file
+
+# AICL — AI Common Language
+
+**A common language for AI agents, the way English is for people: one tongue any agent
+can learn, with no ambiguity.**
+Specification + zero-dependency reference implementation. Draft v0.1.0.
+[日本語の README はこちら](README.ja.md)
+
+```
+(ask  #a.1 @a>@b (temp tokyo $t :unit C :at 2026-11-05T09:00+09:00))
+(tell #b.1 @b>@a ^a.1 {$t 18.5C} ~0.9 :src jma)
+```
+
+## The problem
+
+When two AI agents cooperate today they write each other English (or Japanese) prose:
+
+> "Could you check whether **it** finished and, if so, roll it out to **EU**, but only if the error rate is **fine**? **Should take about 5 minutes.**"
+
+Every bold span is a guess the receiver has to make: which object, which region id, what
+threshold, is that an ETA or a duration, measured when. Natural language is also padded with
+greetings, hedges and thanks that cost tokens and carry no information, and nothing about the
+exchange is machine-checkable: no one can verify that every request got an answer.
+
+## What AICL does
+
+AICL is a small formal message language with four interchangeable encodings
+(readable text, dense text for LLM-to-LLM exchange, JSON, deterministic CBOR) and one
+canonical form that can be hashed and signed.
+Its design rules are the whole point:
+
+| Rule | In practice |
+|---|---|
+| Intent is explicit | Every message opens with a speech act: `ask tell do ok no done fail prop acc rej def clar why sub unsub end` |
+| No pronouns | Everything is referred to by id: `#a.3`, `@bob`, `(ref #b.2)` |
+| Numbers carry units | `18.5C`, `100USD`, `0.5%`. A bare `42` is dimensionless |
+| Time is absolute | `2026-11-05T09:00+09:00`. "Tomorrow" does not exist |
+| Uncertainty is numeric | `~0.8` on any form. No "probably" |
+| Scope is structural | `(not (and a b))` cannot be misread |
+| Unknown words are illegal | A receiver that meets a term it does not know **must** answer `clar`, never guess. Terms are introduced in-band with `def` |
+| Natural language is quarantined | Free text only inside `(nl en "…")`, so the ambiguous part of any message is visible and measurable |
+| Every request is closed | An `ask` ends with `tell`/`no`/`fail`; a `do` with `ok` then `done`/`fail`. The validator checks this |
+
+The full specification is in [SPEC.md](SPEC.md) (16 pages, CC BY 4.0); the grammar in
+[grammar/aicl.ebnf](grammar/aicl.ebnf).
+
+### A complete exchange
+
+```
+(ask  #a.1 @a>@b (deploy-status build-4821 $s :finished-at $at))
+(tell #b.1 @b>@a ^a.1 {$s finished $at 2026-11-04T08:50Z} :src ci-api)
+(do   #a.2 @a>@b (rollout build-4821 :region eu-west-1)
+      :if (< (error-rate build-4821 :window PT10M) 0.5%))
+(ok   #b.2 @b>@a ^a.2 :eta PT5M)
+(tell #b.3 @b>@a (error-rate build-4821 0.3% :window PT10M) :src metrics)
+(done #b.4 @b>@a ^a.2 (rollout build-4821 :region eu-west-1 :at 2026-11-04T09:04:12Z))
+```
+
+More in [examples/](examples/): negotiation, in-band term definition and clarification,
+subscriptions, broadcast discovery, coded failures.
+
+### The dense form: what agents actually send
+
+LLMs consume tokens, not bytes, so "binary" would make LLM-to-LLM traffic *more*
+expensive (it would be base64 in the prompt). Instead, AICL has a **dense text encoding**
+(SPEC §8): the same structure with one-letter speech acts, merged headers, one-letter
+core keys and ISO-basic timestamps. It is bijective with the readable form, so
+`aicl fmt` always recovers the readable version for audit and debugging.
+
+```
+(P a.1>b (meeting :a 20261106T1000+9 :dur PT30M :with [@a @b]))
+(R b.1>a ^a.1 :w (busy @b :from 20261106T0900+9 :to 20261106T1100+9))
+(P b.2>a (meeting :a 20261106T1400+9 :dur PT30M :with [@a @b]))
+(C a.2>b ^b.2)
+```
+
+### Codebooks: cheaper words without ambiguity
+
+What a tokenizer finds cheap is accidental: `Tokyo` is one cl100k token, `tokyo` two,
+`東京` three. A **codebook** (SPEC §8.1) is a shared table from concept ids to allowed
+surface forms. The sender picks, per concept, the form that costs the fewest tokens on
+the *receiver's* tokenizer; the receiver maps every form back to the concept id before
+it reads anything. Four rules keep this exactly as unambiguous as the dense form's
+`A` = `ask`:
+
+1. Only concept ids are canonical: validation, hashing and `def` never see a surface form.
+2. Each form belongs to exactly one concept (the loader rejects any table that breaks this).
+3. A form not in the codebook is an unknown term → `clar`, never a guess. Agents must not
+   shorten or translate vocabulary on their own.
+4. The receiver's tokenizer and codebook id are agreed in-band (`(tokenizer $t)`, `(codebook $id)`), never assumed.
+
+```bash
+aicl codebook codebooks/imagegen.json --tokenizer cl100k_base   # validate, show chosen forms
+aicl pack   msg.kd --codebook codebooks/imagegen.json --tokenizer cl100k_base
+aicl unpack msg.kd --codebook codebooks/imagegen.json           # back to concept ids
+aicl check  msg.kd --codebook codebooks/imagegen.json           # concepts count as known terms
+```
+
+## Honest numbers: precision is not free
+
+`bench/compare.py` compares realistic natural-language agent dialogues with the same
+information in AICL (cl100k_base tokenizer, which GPT-4-class and many open models use).
+All columns are tokens; "packed" uses `codebooks/bench.json`:
+
+| case | NL | readable | ratio | dense | ratio | packed | ratio |
+|---|---|---|---|---|---|---|---|
+| weather | 138 | 128 | 0.93 | 98 | 0.71 | 94 | 0.68 |
+| clarify | 188 | 239 | 1.27 | 190 | 1.01 | 186 | 0.99 |
+| deploy | 150 | 201 | 1.34 | 164 | 1.09 | 162 | 1.08 |
+| scheduling | 124 | 211 | 1.70 | 158 | 1.27 | 158 | 1.27 |
+| **total** | **600** | **779** | **1.30** | **610** | **1.02** | **600** | **1.00** |
+
+The readable form costs about 30 % more tokens than polite English, because BPE
+vocabularies are tuned for prose. The dense form removes 22 % of that; a codebook of
+*safe* forms (see the eval below) takes a little more and lands **at parity with natural
+language while carrying strictly more information** (exact instants with time zones,
+ids, thresholds, provenance) and zero ambiguity. Truncation forms would save another
+1–2 %, but small models misread them, so the example codebook no longer uses any. In characters, every AICL form is far shorter. The scheduling case does not
+move because its vocabulary was already one token per word; the remaining cost there is
+the four time-zoned instants, which natural language simply leaves out.
+
+Re-run the benchmark with your own model's tokenizer (`--tokenizer o200k_base`, or any
+`tiktoken` encoding) before drawing conclusions for it.
+
+### Does the model still read it correctly?
+
+The library decodes every encoding perfectly; the open question is whether a *language
+model* does, especially for packed messages that mix scripts. `evals/readback.py` asks a
+model to write the canonical form of every example message in each encoding, parses and
+canonicalises the answer, and reports accuracy and tokens per encoding:
+
+```bash
+python evals/readback.py --dry-run                      # 69 messages x 4 encodings, no API calls
+python evals/readback.py --provider anthropic --model claude-opus-5-5 --out evals/results/opus-5-5.json
+python evals/readback.py --provider gemini --model gemini-3.8-flash --out evals/results/gemini-3.8-flash.json
+```
+
+Providers: `anthropic` (`pip install anthropic`, `ANTHROPIC_API_KEY`) and `gemini`
+(`pip install google-genai`, `GEMINI_API_KEY`). Any other model needs one function of
+the form `ask(system, user) -> str` added to `PROVIDERS`.
+
+A token saving that lowers read-back accuracy is a loss; adopt a codebook for a model only
+after this eval passes for it. Results so far (`evals/results/`, first 10 messages,
+`gemini-3.5-flash-lite`, the smallest model available to us):
+
+| encoding | accuracy | what failed |
+|---|---|---|
+| readable | 90 % | one term "corrected" to a codebook form |
+| dense | **100 %** | — |
+| packed | 60 % → 80 % → 90 % | English truncations (`deploy`, `roll`, `err`) left unexpanded; last failure is one key code (`:u`) |
+| packed-chars | 80 % → 90 % → 90 % | same |
+
+The first arrow is the effect of presenting the codebook form→concept and adding worked
+examples to the eval prompt; the second is removing truncation forms from the codebook.
+With forms that look like forms, the only remaining miss is a dense key code, and the
+vocabulary errors are gone. Two conclusions: **the dense form is safe even for a small
+model**, and **codebook forms must look like forms** (CamelCase, all-caps, non-Latin):
+truncations that read as real words are the failure mode, so the example codebooks no
+longer contain any and the loader warns about them. Results for larger models are welcome.
+
+## Install and use
+
+```bash
+pip install -e .            # or: pip install aicl-lang  (once published)
+```
+
+```bash
+aicl fmt    examples/02_deploy.aicl        # canonical single-line form (also decodes dense input)
+aicl dense  examples/02_deploy.aicl        # dense text encoding for LLM-to-LLM exchange
+aicl pretty examples/02_deploy.aicl        # readable layout
+aicl json   examples/02_deploy.aicl        # JSON encoding
+aicl cbor   examples/01_weather.aicl -o m.cbor
+aicl hash   examples/02_deploy.aicl        # sha256 of canonical form, per message
+aicl check  examples/02_deploy.aicl --known deploy-status --known rollout --known error-rate
+aicl stats  examples/05_subscription.aicl  # size, tokens, nl ratio
+```
+
+```python
+import aicl
+
+msgs = aicl.parse_messages(open("examples/02_deploy.aicl").read())
+for issue in aicl.validate_conversation(msgs, known_terms=["deploy-status", "rollout", "error-rate"]):
+    print(issue)                             # protocol violations, open requests, unresolved terms
+
+m = msgs[1]
+print(m.act, m.sender, m.receiver, m.reply)  # tell b a a.1
+print(aicl.to_text(m.to_form()))            # canonical text
+print(aicl.to_dense(m.to_form()))           # dense text
+print(aicl.to_json_text(m.to_form()))       # canonical JSON
+print(aicl.digest(m.to_form()))             # sha256
+
+from aicl.build import msg, form, q
+reply = msg("tell", "b.9", "b", "a", aicl.KMap([(aicl.Var("t"), q(18.5, "C"))]), reply="a.1", conf=0.9)
+```
+
+## Cooperation: specialists working together
+
+AIs have different strengths. The cooperation layer (`aicl/coop.py`,
+[schemas/cooperation.md](schemas/cooperation.md)) lets an image agent, a video agent and a
+code agent work on one goal, speaking only AICL:
+
+- **Specialists** announce `(coop:capability image-gen :agent @img)`, accept
+  `(coop:task …)` with `ok`, and return `(coop:artifact …)` with `done`.
+- A **hub** keeps the registry, turns the requester's goal into a `(coop:plan …)`, delegates
+  tasks in dependency order and passes each artifact to the next task in full, never as
+  "the previous result".
+- The **bus** validates every message; a specialist's `fail` reaches the requester as `fail`
+  with `:why (ref #…)`, never as a silent retry.
+
+```
+(do   #user.1 @user>@hub (coop:goal (nl ja "東京の夜景の10秒動画を作って、それを埋め込んだWebページを作って")))
+(tell #hub.2 @hub>@user (coop:plan [(coop:task t1 image-gen …) (coop:task t2 video-gen :in {keyframe (coop:from t1) …}) (coop:task t3 code-gen …)]))
+(do   #hub.3 @hub>@img (coop:task t1 image-gen :in {prompt "Tokyo skyline at night" width 1920px height 1080px}))
+(done #img.3 @img>@hub ^hub.3 (coop:artifact img-974186 :type image :task t1 :uri "file://out/img-974186.png"))
+(do   #hub.4 @hub>@vid (coop:task t2 video-gen :in {keyframe (coop:artifact img-974186 …) duration 10s}))
+…
+(done #hub.6 @hub>@user ^user.1 [(coop:artifact img-974186 …) (coop:artifact vid-… :type video …) (coop:artifact page-1 :type code :content "<!doctype html>…")])
+```
+
+`python examples/cooperation_demo.py` runs this offline with stub specialists: 16 messages,
+0 protocol errors, natural language only inside the user's goal (nl ratio 2 %). Swap the
+stub functions for real APIs, or use `LLMAgent` for a model-backed specialist and
+`LLMPlanner` for a model-backed hub, and nothing else changes.
+
+`python examples/cooperation_gemini.py` does exactly that with Gemini in three roles
+(`GEMINI_API_KEY` required): the hub's planner writes the plan in AICL, the image
+specialist calls a Gemini image model and returns the PNG as an artifact, and the code
+specialist *is* Gemini speaking dense AICL, answering the task with `K` and `Z` messages
+that the bus validates before accepting. Output lands in `out/` (transcript, image, page).
+On the free tier Gemini's image models are not available (a consumer Gemini subscription
+does not change API quotas), so by default the text model draws the image as SVG;
+`--image-model gemini-nano-banana-2.1` uses a real image model once API billing is on. Live runs with `gemini-3.5-flash-lite` as planner
+and code agent: the plan, two dependent tasks and both artifacts reach the requester in
+12 messages with 0 protocol errors. Getting there taught three things, all now in the code:
+the harness, not the model, owns message ids (it tells the model its next id and reply
+header as an AICL comment, and renumbers if needed); a malformed reply gets one correction
+turn with the validator's verdict before it becomes `fail :code parse`; and two redundant-sigil
+slips a small model makes (`a.1>@b`, `^#a.1`) are accepted on input because they cannot
+change meaning, while anything ambiguous is still rejected.
+
+## Make an LLM speak AICL
+
+[prompts/system_prompt.md](prompts/system_prompt.md) is a ~700-token system prompt that
+teaches any instruction-following model the readable language;
+[prompts/system_prompt_dense.md](prompts/system_prompt_dense.md) teaches the dense form,
+which is what two LLM agents should use with each other. `examples/llm_dialogue.py` runs two
+Claude instances negotiating a meeting in AICL, with every message parsed and validated
+at the boundary before it is delivered:
+
+```bash
+pip install anthropic
+python examples/llm_dialogue.py
+```
+
+`examples/two_agents_demo.py` does the same with rule-based agents and no network.
+
+### Hand the folder to your own agent
+
+The repository is self-describing for agents that read a project folder:
+
+- `AGENTS.md` (Codex, Cursor and most general agents) and `CLAUDE.md` (Claude Code)
+  tell the agent to load the system prompt and validate every message with `aicl check`.
+- `SKILL.md` makes the folder an [Agent Skill](https://github.com/anthropics/skills):
+  copy it to `.claude/skills/aicl/` (or your framework's skills directory) and the
+  agent loads AICL only when it talks to another agent.
+
+Two agents that have both read this folder can talk to each other in AICL with no
+further setup; the validator is the referee.
+
+If you only want to hand over the language (no code), give the agent the
+[rulebook/](rulebook/) folder: both language references, the spec, the example codebooks
+and the worked conversations, with a bilingual index saying which file is for what. It is
+generated by `python tools/build_rulebook.py` and checked by the test suite, so it never
+drifts from the sources.
+
+## Repository layout
+
+```
+SPEC.md                 the language (normative)
+grammar/aicl.ebnf      text encoding grammar
+aicl/                  reference implementation (Python ≥ 3.11, no dependencies)
+  lexer.py parser.py    text → terms
+  printer.py            canonical + pretty text, sha256 digest
+  dense.py              dense text encoding (tokenizer-tuned, bijective with canonical)
+  codebook.py           tokenizer-aware codebooks: pack / unpack with the ambiguity rules
+  coop.py               cooperation layer: bus, specialists, hub, planners
+schemas/                published vocabularies (cooperation)
+codebooks/              example codebooks (bench, imagegen)
+  json_codec.py         JSON encoding
+  cbor_codec.py         deterministic CBOR (RFC 8949 §4.2)
+  validate.py           message + conversation validator, nl ratio
+  cli.py                `aicl` command
+examples/               worked conversations + demos
+prompts/                system prompts for LLM agents (readable and dense)
+rulebook/               generated hand-off folder: language references, spec, codebooks, examples
+tools/                  build_rulebook.py
+bench/                  NL vs AICL comparison
+evals/                  LLM read-back accuracy evaluation
+tests/                  unittest suite (also the conformance tests)
+```
+
+## Relation to prior work
+
+- **KQML / FIPA-ACL** (1990s agent communication languages) introduced speech-act
+  performatives; AICL keeps that idea and adds what LLM agents need: quarantined natural
+  language, numeric confidence, in-band term definition and a canonical, hashable form.
+- **JSON-RPC, MCP, A2A** are transports and tool-call envelopes. They say nothing about the
+  *content* agents exchange; a AICL message can travel inside any of them (use the JSON encoding).
+- **Lojban, Ithkuil** are unambiguous languages for humans. AICL is for machines and is
+  deliberately tiny.
+
+## Roadmap
+
+- v0.2: standard published codebooks per domain, shared by id; typed `def` signatures;
+  standard schemas for scheduling, data query and tool invocation.
+- Ports: the implementation is ~1700 lines including docstrings and meant to be re-implemented in an afternoon.
+  Conformance criteria are in SPEC §12; open an issue to be listed here.
+- Identity and signatures (`@agent` → public key).
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for how language changes are proposed.
+
+## License
+
+Code: MIT. Specification: CC BY 4.0. AICL stands for *AI Common Language*; the project was
+briefly called Koine before 0.1.0.