langfx.js 0.1.0-alpha.0

package/docs/PYTHON_PROTOCOL_PARITY.md

@@ -0,0 +1,964 @@
+# Python-protocol parity
+
+Captured Python commit: `504233a5c4a10e45684530ff58d7e2bd9d8ae58e` (clean); Python 3.14.0.
+
+The capture executes default Python lf.query with fake model responses and independently checks LfQuery.render/parse_result. The JS check uses the same responses with equivalent explicitly registered classes. No inference or credentials are used.
+
+| Case | Parsed values/class identity | Roles + request | Exact full prompt |
+| --- | --- | --- | --- |
+| integer | PASS | PASS | PASS |
+| class | PASS | PASS | PASS |
+| union-foo | PASS | PASS | PASS |
+| union-bar | PASS | PASS | PASS |
+| nested | PASS | PASS | PASS |
+| few-shot | PASS | PASS | PASS |
+| nullable-list | PASS | PASS | PASS |
+| nested-few-shot | PASS | PASS | PASS |
+| few-shot-system | PASS | PASS | PASS |
+
+## Scope and remaining limits
+
+- For these captured cases, JS matches Python python:2.0 labels, built-in demonstration, schema declarations, dependency ordering, fences, few-shot formatting, and whitespace exactly.
+- The check asserts complete role/text arrays with no normalization or substitutions. A whitespace mutation is checked to fail equality. Parsed results, class identity, and wrapper behavior are checked independently.
+- This Python checkout rejects bare top-level union annotations in Schema.from_value. Union cases use the equivalent explicit PyGlove ValueSpec, recorded as schemaAdaptation in the fixture. This corrects the earlier assumption that a bare Foo | Bar works on every Python prompt-query path.
+- JS requires explicit pure factories and supports only a bounded data grammar. This comparison does not establish parity for defaults, inheritance, tuples, recursive classes, arbitrary Python code, errors, streaming, or live model quality.
+
+## Captures
+
+### integer
+
+Response: `42`
+
+**Python**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  int
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+**TypeScript**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  int
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+### class
+
+Response: `Foo(text="hello")`
+
+**Python**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Foo
+
+  ```python
+  class Foo:
+    text: str
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+**TypeScript**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Foo
+
+  ```python
+  class Foo:
+    text: str
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+### union-foo
+
+Response: `Foo(text="hello")`
+
+**Python**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Union[Foo, Bar]
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+**TypeScript**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Union[Foo, Bar]
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+### union-bar
+
+Response: `output = Bar(count=7)`
+
+**Python**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Union[Foo, Bar]
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+**TypeScript**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Union[Foo, Bar]
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+### nested
+
+Response: `Group(items=[Foo(text="hi"), Bar(count=2)])`
+
+**Python**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Group
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+
+  class Group:
+    items: list[Union[Foo, Bar]]
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+**TypeScript**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Group
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+
+  class Group:
+    items: list[Union[Foo, Bar]]
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+### few-shot
+
+Response: `Bar(count=7)`
+
+**Python**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Union[Foo, Bar]
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+  ```
+
+REQUEST:
+  Say hello.
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Foo(
+    text='hello'
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Union[Foo, Bar]
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+**TypeScript**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Union[Foo, Bar]
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+  ```
+
+REQUEST:
+  Say hello.
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Foo(
+    text='hello'
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Union[Foo, Bar]
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+### nullable-list
+
+Response: `[Foo(text="hello"), None]`
+
+**Python**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  list[Foo | None]
+
+  ```python
+  class Foo:
+    text: str
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+**TypeScript**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  list[Foo | None]
+
+  ```python
+  class Foo:
+    text: str
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+### nested-few-shot
+
+Response: `Group(items=[Bar(count=2)])`
+
+**Python**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Group
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+
+  class Group:
+    items: list[Union[Foo, Bar]]
+  ```
+
+REQUEST:
+  A group.
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Group(
+    items=[Foo(
+        text='hi'
+      ), Bar(
+        count=2
+      )]
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Group
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+
+  class Group:
+    items: list[Union[Foo, Bar]]
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+**TypeScript**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Group
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+
+  class Group:
+    items: list[Union[Foo, Bar]]
+  ```
+
+REQUEST:
+  A group.
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Group(
+    items=[Foo(
+        text='hi'
+      ), Bar(
+        count=2
+      )]
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Group
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+
+  class Group:
+    items: list[Union[Foo, Bar]]
+  ```
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+### few-shot-system
+
+Response: `Foo(text="done")`
+
+**Python**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Union[Foo, Bar]
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+  ```
+
+REQUEST:
+  Say hello.
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Foo(
+    text='hello'
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Union[Foo, Bar]
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+  ```
+
+Be concise.
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````
+
+**TypeScript**
+
+````text
+[system]
+Please respond to the last REQUEST with OUTPUT PYTHON OBJECT only according to OUTPUT PYTHON TYPE.
+
+OUTPUT PYTHON TYPE:
+  Answer
+
+  ```python
+  class Answer:
+    final_answer: int
+  ```
+
+REQUEST:
+  1 + 1 =
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Answer(
+    final_answer=2
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Union[Foo, Bar]
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+  ```
+
+REQUEST:
+  Say hello.
+
+OUTPUT PYTHON OBJECT:
+  ```python
+  output = Foo(
+    text='hello'
+  )
+  ```
+
+OUTPUT PYTHON TYPE:
+  Union[Foo, Bar]
+
+  ```python
+  class Foo:
+    text: str
+
+  class Bar:
+    count: int
+  ```
+
+Be concise.
+
+[user]
+REQUEST:
+  Return the requested value.
+
+OUTPUT PYTHON OBJECT:
+
+````