faf-python-sdk 1.3.1__py3-none-any.whl → 2.0.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {faf_python_sdk-1.3.1.dist-info → faf_python_sdk-2.0.0.dist-info}/METADATA +33 -22
- faf_python_sdk-2.0.0.dist-info/RECORD +16 -0
- {faf_python_sdk-1.3.1.dist-info → faf_python_sdk-2.0.0.dist-info}/WHEEL +1 -1
- faf_sdk/__init__.py +13 -5
- faf_sdk/_kernel_yaml.py +493 -0
- faf_sdk/_libyaml_scanner.py +874 -0
- faf_sdk/interop.py +50 -5
- faf_sdk/mk4.py +146 -106
- faf_python_sdk-1.3.1.dist-info/RECORD +0 -14
- {faf_python_sdk-1.3.1.dist-info → faf_python_sdk-2.0.0.dist-info}/licenses/LICENSE +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: faf-python-sdk
|
|
3
|
-
Version:
|
|
3
|
+
Version: 2.0.0
|
|
4
4
|
Summary: Persistent project context for Python — parse, validate, and score `.faf` files. The foundation other Python FAF tools (gemini-faf-mcp, custom MCP servers, CI validators) build on. IANA-registered application/vnd.faf+yaml.
|
|
5
5
|
Project-URL: Homepage, https://faf.one
|
|
6
6
|
Project-URL: Documentation, https://github.com/Wolfe-Jam/faf-python-sdk
|
|
@@ -41,29 +41,42 @@ The foundation other Python FAF tools build on. If you're building MCP servers,
|
|
|
41
41
|
[](https://builder.faf.one)
|
|
42
42
|
[](https://pypi.org/project/faf-python-sdk/)
|
|
43
43
|
[](https://pypi.org/project/faf-python-sdk/)
|
|
44
|
-
[](https://github.com/Wolfe-Jam/faf-python-sdk)
|
|
45
45
|
[](https://www.iana.org/assignments/media-types/application/vnd.faf+yaml)
|
|
46
46
|
|
|
47
47
|
**Media Type:** `application/vnd.faf+yaml` (IANA registered)
|
|
48
48
|
|
|
49
|
-
## What's New in
|
|
49
|
+
## What's New in v2.0.0 — The Always33 Edition
|
|
50
50
|
|
|
51
|
-
|
|
51
|
+
One engine, one number: faf-python-sdk scores all 33 slots exactly like faf-kernel — the same score faf-cli 8, claude-faf-mcp 7, faf-mcp 4 and grok-faf-mcp 2 give.
|
|
52
52
|
|
|
53
|
-
|
|
53
|
+
- **Always 33 slots.** The 12 enterprise slots count unless marked `slotignored`. 21 base slots filled with no markers: 64% (21/33). The same file plus the 12 markers: 100% (21/21). `faf auto` (faf-cli) writes the markers.
|
|
54
|
+
- `tbd` / `todo` are placeholders; short keys (`framework`, `css`, `state`, `api`, `db`, `pkg_manager`) are read.
|
|
55
|
+
- YAML is read the way the kernel reads it; unreadable YAML scores 0 without raising.
|
|
56
|
+
- `score_faf(yaml, tier=...)` still accepts `tier`; it no longer changes the slot count.
|
|
57
|
+
- **Upgrading:** `result.slots` lists all 33 slots under the kernel's names (`stack.framework`, `css`, `state`, `api`, `db`, `pkg_manager`; were `frontend`, `css_framework`, `state_management`, `api_type`, `database`, `package_manager`). Code that reads slots by name needs the new names. YAML the kernel can't read now scores 0 (a duplicate key, a second document, nesting deeper than 128), and rounding is half away from zero (12.5% → 13), as the kernel does.
|
|
58
|
+
- Parity harness: 845/845 fixtures match faf-kernel (`faf-scoring-kernel@3.0.0`), including the `project.faf` of 58 public repos.
|
|
54
59
|
|
|
55
|
-
|
|
60
|
+
See [CHANGELOG.md](CHANGELOG.md) for the full list of changes.
|
|
61
|
+
|
|
62
|
+
## v1.4.0 — The Interop Edition
|
|
63
|
+
|
|
64
|
+
The interop functions get their real names: `author_agents_md` / `author_gemini_md` are public, `render_*` is the impl, `generate_*` is deprecated (removed in 2.0).
|
|
65
|
+
|
|
66
|
+
Output is byte-identical — a naming change, not a behaviour change. Existing `from faf_sdk import generate_agents_md` keeps working, now with a `DeprecationWarning`.
|
|
67
|
+
|
|
68
|
+
`faf_sdk.interop` — `author_agents_md(faf)` and `author_gemini_md(faf)`, Python ports of faf-cli's `src/interop/agents.ts` + `gemini.ts`, in parity with the canonical TypeScript. Deterministic BETTER-shaped projection: setup (install→build→dev ordered) · tests · layout · conventions · three-tier guardrails · definition of done · security · commit · stack. Human Context (who/why marketing) is intentionally omitted from AGENTS.md — it belongs in the README / .faf DNA, not agent ops.
|
|
56
69
|
|
|
57
70
|
```python
|
|
58
|
-
from faf_sdk import parse_file,
|
|
71
|
+
from faf_sdk import parse_file, author_agents_md
|
|
59
72
|
|
|
60
73
|
faf = parse_file("project.faf")
|
|
61
|
-
print(
|
|
74
|
+
print(author_agents_md(faf.data.raw)) # takes the raw dict — carries top-level commands / key_files / security
|
|
62
75
|
```
|
|
63
76
|
|
|
64
77
|
Any Python FAF tool that authors an AI-context file wraps this now — never hand-roll one. `gemini-faf-mcp` 2.7.0's `faf_agents` / `faf_gemini` are the reference wrappers.
|
|
65
78
|
|
|
66
|
-
##
|
|
79
|
+
## v1.2.0 — The Dart Edition
|
|
67
80
|
|
|
68
81
|
Adds `detect_dart_project()`: content-aware Dart/Flutter detection from a `pubspec.yaml` (Flutter app vs package · Dart MCP / backend / CLI / library), reproducing faf-cli's engine byte-for-byte — 20 shared fixtures, parity-tested.
|
|
69
82
|
|
|
@@ -74,7 +87,7 @@ d = detect_dart_project(".")
|
|
|
74
87
|
print(d.app_type, d.framework) # e.g. "mobile" "Flutter"
|
|
75
88
|
```
|
|
76
89
|
|
|
77
|
-
##
|
|
90
|
+
## v1.1.0
|
|
78
91
|
|
|
79
92
|
**Mk4 Championship Scoring Engine** — the same 33-slot scoring algorithm used by the Rust compiler and TypeScript CLI, now in Python. Same slots, same formula, same scores. Every FAF tool in every language now agrees on what 100% means.
|
|
80
93
|
|
|
@@ -114,26 +127,24 @@ print(f"Slots: {result.populated}/{result.total} populated")
|
|
|
114
127
|
|
|
115
128
|
## Mk4 Scoring
|
|
116
129
|
|
|
117
|
-
The Mk4 engine scores `.faf` files
|
|
130
|
+
The Mk4 engine scores `.faf` files against 33 slots (project metadata, human context, tech stack, and 12 enterprise slots), exactly as faf-kernel does. Each slot is **Populated**, **Empty**, or **Slotignored**. The score is populated ÷ active, where active = 33 − slotignored.
|
|
118
131
|
|
|
119
132
|
```python
|
|
120
|
-
from faf_sdk import score_faf
|
|
133
|
+
from faf_sdk import score_faf
|
|
121
134
|
|
|
122
|
-
# Base scoring (21 slots)
|
|
123
135
|
result = score_faf(yaml_content)
|
|
124
136
|
print(result.score) # 0-100
|
|
125
|
-
print(result.tier) #
|
|
137
|
+
print(result.tier) # TROPHY / GOLD / SILVER / BRONZE / GREEN / YELLOW / RED / WHITE
|
|
126
138
|
print(result.populated) # slots with real data
|
|
127
|
-
print(result.active) #
|
|
128
|
-
print(result.slots) # per-slot breakdown
|
|
129
|
-
|
|
130
|
-
# Enterprise scoring (33 slots — adds monorepo/infra)
|
|
131
|
-
result = score_faf(yaml_content, LicenseTier.ENTERPRISE)
|
|
139
|
+
print(result.active) # 33 minus slotignored
|
|
140
|
+
print(result.slots) # per-slot breakdown, 33 entries in kernel order
|
|
132
141
|
```
|
|
133
142
|
|
|
134
|
-
**Placeholder rejection:** Values like `"null"`, `"unknown"`, `"n/a"`, `"Describe your project goal"` are
|
|
143
|
+
**Placeholder rejection:** Values like `"null"`, `"none"`, `"unknown"`, `"n/a"`, `"tbd"`, `"todo"`, `"Describe your project goal"` (case-insensitive) are scored as Empty — not Populated.
|
|
144
|
+
|
|
145
|
+
**Slotignored:** Set any slot to `slotignored` to exclude it from scoring. A project that does not use the 12 enterprise slots marks them `slotignored` (`faf auto` writes them) and can still reach 100%.
|
|
135
146
|
|
|
136
|
-
**
|
|
147
|
+
**Short keys:** `stack.framework`, `css`, `state`, `api`, `db`, `pkg_manager` are the canonical names; the legacy `frontend`, `css_framework`, `state_management`, `api_type`, `database`, `package_manager` are read when the short key is empty.
|
|
137
148
|
|
|
138
149
|
## Parsing
|
|
139
150
|
|
|
@@ -188,7 +199,7 @@ root = find_project_root()
|
|
|
188
199
|
|
|
189
200
|
| Function | Returns | Description |
|
|
190
201
|
|----------|---------|-------------|
|
|
191
|
-
| `score_faf(yaml
|
|
202
|
+
| `score_faf(yaml)` | `Mk4Result` | Mk4 score, always 33 slots (faf-kernel parity) |
|
|
192
203
|
| `parse(content)` | `FafFile` | Parse YAML string |
|
|
193
204
|
| `parse_file(path)` | `FafFile` | Parse from file path |
|
|
194
205
|
| `validate(faf)` | `ValidationResult` | Structure validation + warnings |
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
faf_sdk/__init__.py,sha256=KEbv-KJADTuc0yp8SZAIn15I-wCG33cFzPy48_SSJts,2048
|
|
2
|
+
faf_sdk/_kernel_yaml.py,sha256=Ytr_GaFid7MqRbyl_2mnrGWY_Oiec8T3EeHCMnkYrWA,18965
|
|
3
|
+
faf_sdk/_libyaml_scanner.py,sha256=wDqTdOOZAsZ8c89N0jgRS4vu9TSZhCQgylq4cKfn1DU,34591
|
|
4
|
+
faf_sdk/dart_detection.json,sha256=jFtKnyyMVVFsunizteFZ2pLhYC1qC2nCIWQB9SdOqKE,1195
|
|
5
|
+
faf_sdk/detect.py,sha256=GothQNKChKSEwke74MRy63VyBYLqOzN1j3yzjsSVpJA,6310
|
|
6
|
+
faf_sdk/discovery.py,sha256=3Jj3lSzPAj9l0_VjJneLAwo1dMXvhneFhcVN4m2ejOw,8643
|
|
7
|
+
faf_sdk/interop.py,sha256=6sjj3Zk9bca0Pt7KSwZ3Gopb-eaclKtRYyeZvluYZLw,14819
|
|
8
|
+
faf_sdk/mk4.py,sha256=Xi_Oe67naXF6-9oLQdtUjvTO6x9PAzUJPct7VPjQpZQ,7687
|
|
9
|
+
faf_sdk/parser.py,sha256=lJJysj52X0Q_aGhl4PMXY2PuMqASFHdVRiSZFDdampk,4925
|
|
10
|
+
faf_sdk/types.py,sha256=sm1ezSzCc-93bszr4ite31_xZRKjwqf2utlqN6_0QuM,6615
|
|
11
|
+
faf_sdk/validator.py,sha256=6uneOwar4GYUF52BnAQTu159kE4mh4RWRgG6onVbiG4,5730
|
|
12
|
+
faf_sdk/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
13
|
+
faf_python_sdk-2.0.0.dist-info/METADATA,sha256=EJQ_IG0g7Gkf0K0EqS5j2aS33CcTyVH0-3HZhjb772c,10646
|
|
14
|
+
faf_python_sdk-2.0.0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
|
|
15
|
+
faf_python_sdk-2.0.0.dist-info/licenses/LICENSE,sha256=ARScF5tFhbQnYO2V5QAuCwhDHcxKdOWTOV81Pxx_j7U,1065
|
|
16
|
+
faf_python_sdk-2.0.0.dist-info/RECORD,,
|
faf_sdk/__init__.py
CHANGED
|
@@ -25,7 +25,15 @@ from .validator import validate, ValidationResult
|
|
|
25
25
|
from .mk4 import score_faf, Mk4Result, SlotState, LicenseTier
|
|
26
26
|
from .discovery import find_faf_file, find_project_root, load_fafignore
|
|
27
27
|
from .detect import detect_dart_project, DartProject
|
|
28
|
-
from .interop import
|
|
28
|
+
from .interop import (
|
|
29
|
+
author_agents_md,
|
|
30
|
+
author_gemini_md,
|
|
31
|
+
render_agents_md,
|
|
32
|
+
render_gemini_md,
|
|
33
|
+
generate_agents_md, # deprecated alias — kept in 2.0, removal planned
|
|
34
|
+
generate_gemini_md, # deprecated alias — kept in 2.0, removal planned
|
|
35
|
+
faf_meta_tag,
|
|
36
|
+
)
|
|
29
37
|
from .types import (
|
|
30
38
|
FafData,
|
|
31
39
|
ProjectInfo,
|
|
@@ -36,7 +44,7 @@ from .types import (
|
|
|
36
44
|
AIScoring
|
|
37
45
|
)
|
|
38
46
|
|
|
39
|
-
__version__ = "
|
|
47
|
+
__version__ = "2.0.0"
|
|
40
48
|
__all__ = [
|
|
41
49
|
# Parser
|
|
42
50
|
"parse",
|
|
@@ -58,9 +66,9 @@ __all__ = [
|
|
|
58
66
|
# Detection (Dart/Flutter — A+B hybrid, parity with faf-cli)
|
|
59
67
|
"detect_dart_project",
|
|
60
68
|
"DartProject",
|
|
61
|
-
# Interop — AGENTS.md / GEMINI.md
|
|
62
|
-
"
|
|
63
|
-
"
|
|
69
|
+
# Interop — AGENTS.md / GEMINI.md authoring (parity with faf-cli src/interop)
|
|
70
|
+
"author_agents_md",
|
|
71
|
+
"author_gemini_md",
|
|
64
72
|
"faf_meta_tag",
|
|
65
73
|
# Types
|
|
66
74
|
"FafData",
|
faf_sdk/_kernel_yaml.py
ADDED
|
@@ -0,0 +1,493 @@
|
|
|
1
|
+
"""
|
|
2
|
+
Kernel YAML loader — reads YAML the way faf-kernel does.
|
|
3
|
+
|
|
4
|
+
faf-kernel (Rust) parses with serde_yaml_ng 0.10 into a ``serde_yaml_ng::Value``.
|
|
5
|
+
PyYAML's ``safe_load`` differs from that in ways that move scores (YAML 1.1
|
|
6
|
+
merge keys, silent duplicate keys, YAML 1.1 scalar types, unknown-tag errors),
|
|
7
|
+
so the scorer does not use it. This module scans with a port of libyaml's
|
|
8
|
+
scanner (``_libyaml_scanner``), parses with PyYAML's parser on top of it, and
|
|
9
|
+
rebuilds the value from the event stream exactly as serde_yaml_ng's
|
|
10
|
+
``DeserializerFromEvents`` + ``Value`` visitor do:
|
|
11
|
+
|
|
12
|
+
- first document only; a second document is an error
|
|
13
|
+
- no ``<<`` merge — it is an ordinary key
|
|
14
|
+
- a duplicate mapping key is an error
|
|
15
|
+
- plain scalars resolve with the YAML 1.2 core rules serde_yaml_ng uses
|
|
16
|
+
- ``!!bool`` / ``!!int`` / ``!!float`` / ``!!null`` must parse or it is an error;
|
|
17
|
+
any other global tag (``!!str``, ``!!binary``, ``tag:...``) reads as a string
|
|
18
|
+
- a local tag (``!foo``, or the bare ``!``) wraps the value in :class:`Tagged`
|
|
19
|
+
- nesting deeper than 128 collections is an error (the scanner stops at
|
|
20
|
+
flow level 129, so deep ``[``/``{`` input fails in linear time)
|
|
21
|
+
- alias expansion past ``100 × events`` jumps is an error ("repetition limit")
|
|
22
|
+
- anchor ids follow serde_yaml_ng: a redefined name can hand its id to a later
|
|
23
|
+
anchor, and an alias resolves to the last node registered under its id
|
|
24
|
+
- an empty ``?`` key inside ``[...]`` also consumes the next token, as libyaml
|
|
25
|
+
does, so ``[?, a]`` / ``[?]`` are errors
|
|
26
|
+
|
|
27
|
+
Any error raises :class:`KernelYamlError`.
|
|
28
|
+
"""
|
|
29
|
+
|
|
30
|
+
import math
|
|
31
|
+
import re
|
|
32
|
+
from typing import Any, Dict, List, Optional, Set, Tuple
|
|
33
|
+
|
|
34
|
+
import yaml
|
|
35
|
+
import yaml.parser
|
|
36
|
+
from yaml import events as ev
|
|
37
|
+
|
|
38
|
+
from ._libyaml_scanner import LibyamlScanner
|
|
39
|
+
|
|
40
|
+
_TAG_BOOL = "tag:yaml.org,2002:bool"
|
|
41
|
+
_TAG_INT = "tag:yaml.org,2002:int"
|
|
42
|
+
_TAG_FLOAT = "tag:yaml.org,2002:float"
|
|
43
|
+
_TAG_NULL = "tag:yaml.org,2002:null"
|
|
44
|
+
|
|
45
|
+
_MAX_DEPTH = 128 # serde_yaml_ng remaining_depth
|
|
46
|
+
_JUMP_FACTOR = 100 # serde_yaml_ng: jumpcount > events.len() * 100
|
|
47
|
+
|
|
48
|
+
|
|
49
|
+
class KernelYamlError(ValueError):
|
|
50
|
+
"""The kernel cannot read this YAML (it would return a parse error)."""
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
class Tagged:
|
|
54
|
+
"""A value carrying a local YAML tag (serde_yaml_ng ``Value::Tagged``)."""
|
|
55
|
+
|
|
56
|
+
__slots__ = ("tag", "value")
|
|
57
|
+
|
|
58
|
+
def __init__(self, tag: str, value: Any) -> None:
|
|
59
|
+
self.tag = tag
|
|
60
|
+
self.value = value
|
|
61
|
+
|
|
62
|
+
|
|
63
|
+
class Mapping(dict): # type: ignore[type-arg]
|
|
64
|
+
"""A YAML mapping. Keys are :func:`_key` forms so equality follows
|
|
65
|
+
serde_yaml_ng ``Value`` (``1`` != ``1.0`` != ``true`` != ``"1"``)."""
|
|
66
|
+
|
|
67
|
+
|
|
68
|
+
def untag(value: Any) -> Any:
|
|
69
|
+
"""serde_yaml_ng ``Value::untag_ref``."""
|
|
70
|
+
while isinstance(value, Tagged):
|
|
71
|
+
value = value.value
|
|
72
|
+
return value
|
|
73
|
+
|
|
74
|
+
|
|
75
|
+
def str_key(name: str) -> Tuple[str, str]:
|
|
76
|
+
"""Mapping key form of a string (for slot lookups)."""
|
|
77
|
+
return ("s", name)
|
|
78
|
+
|
|
79
|
+
|
|
80
|
+
def _key(value: Any) -> Any:
|
|
81
|
+
"""Hashable form of a value with serde_yaml_ng ``Value`` equality."""
|
|
82
|
+
if value is None:
|
|
83
|
+
return ("n",)
|
|
84
|
+
if isinstance(value, bool):
|
|
85
|
+
return ("b", value)
|
|
86
|
+
if isinstance(value, int):
|
|
87
|
+
return ("i", value)
|
|
88
|
+
if isinstance(value, float):
|
|
89
|
+
return ("f", "nan" if math.isnan(value) else value)
|
|
90
|
+
if isinstance(value, str):
|
|
91
|
+
return ("s", value)
|
|
92
|
+
if isinstance(value, list):
|
|
93
|
+
return ("l", tuple(_key(v) for v in value))
|
|
94
|
+
if isinstance(value, Tagged):
|
|
95
|
+
# serde_yaml_ng Tag equality ignores one leading "!"
|
|
96
|
+
tag = value.tag[1:] if value.tag.startswith("!") else value.tag
|
|
97
|
+
return ("t", tag, _key(value.value))
|
|
98
|
+
if isinstance(value, dict):
|
|
99
|
+
return ("m", frozenset((k, _key(v)) for k, v in value.items()))
|
|
100
|
+
raise KernelYamlError("unhashable key") # pragma: no cover
|
|
101
|
+
|
|
102
|
+
|
|
103
|
+
# --- scalar resolution (serde_yaml_ng de.rs) --------------------------------
|
|
104
|
+
|
|
105
|
+
_DIGITS = {2: frozenset("01"), 8: frozenset("01234567"),
|
|
106
|
+
10: frozenset("0123456789"), 16: frozenset("0123456789abcdefABCDEF")}
|
|
107
|
+
_U64_MAX = 2**64 - 1
|
|
108
|
+
_I64_MIN = -(2**63)
|
|
109
|
+
_U128_MAX = 2**128 - 1
|
|
110
|
+
_I128_MIN = -(2**127)
|
|
111
|
+
_RUST_F64 = re.compile(r"[+-]?(?:[0-9]+\.?[0-9]*|\.[0-9]+)(?:[eE][+-]?[0-9]+)?\Z")
|
|
112
|
+
|
|
113
|
+
|
|
114
|
+
def _from_str_radix(s: str, radix: int, signed: bool, lo: int, hi: int) -> Optional[int]:
|
|
115
|
+
"""Rust ``{u,i}N::from_str_radix``: optional sign, digits only, in range."""
|
|
116
|
+
body = s
|
|
117
|
+
neg = False
|
|
118
|
+
if body[:1] == "+":
|
|
119
|
+
body = body[1:]
|
|
120
|
+
elif body[:1] == "-" and signed:
|
|
121
|
+
body = body[1:]
|
|
122
|
+
neg = True
|
|
123
|
+
if not body or any(c not in _DIGITS[radix] for c in body):
|
|
124
|
+
return None
|
|
125
|
+
n = int(body, radix)
|
|
126
|
+
if neg:
|
|
127
|
+
n = -n
|
|
128
|
+
return n if lo <= n <= hi else None
|
|
129
|
+
|
|
130
|
+
|
|
131
|
+
def _digits_but_not_number(s: str) -> bool:
|
|
132
|
+
t = s[1:] if s[:1] in ("-", "+") else s
|
|
133
|
+
return len(t) > 1 and t[0] == "0" and all(c in "0123456789" for c in t[1:])
|
|
134
|
+
|
|
135
|
+
|
|
136
|
+
def _parse_unsigned(s: str, hi: int) -> Optional[int]:
|
|
137
|
+
unpositive = s[1:] if s[:1] == "+" else s
|
|
138
|
+
for prefix, radix in (("0x", 16), ("0o", 8), ("0b", 2)):
|
|
139
|
+
if unpositive.startswith(prefix):
|
|
140
|
+
rest = unpositive[2:]
|
|
141
|
+
if rest[:1] in ("+", "-"):
|
|
142
|
+
return None
|
|
143
|
+
n = _from_str_radix(rest, radix, False, 0, hi)
|
|
144
|
+
if n is not None:
|
|
145
|
+
return n
|
|
146
|
+
if unpositive[:1] in ("+", "-"):
|
|
147
|
+
return None
|
|
148
|
+
if _digits_but_not_number(s):
|
|
149
|
+
return None
|
|
150
|
+
return _from_str_radix(unpositive, 10, False, 0, hi)
|
|
151
|
+
|
|
152
|
+
|
|
153
|
+
def _parse_negative(s: str, lo: int, hi: int) -> Optional[int]:
|
|
154
|
+
for prefix, radix in (("-0x", 16), ("-0o", 8), ("-0b", 2)):
|
|
155
|
+
if s.startswith(prefix):
|
|
156
|
+
n = _from_str_radix("-" + s[3:], radix, True, lo, hi)
|
|
157
|
+
if n is not None:
|
|
158
|
+
return n
|
|
159
|
+
if _digits_but_not_number(s):
|
|
160
|
+
return None
|
|
161
|
+
return _from_str_radix(s, 10, True, lo, hi)
|
|
162
|
+
|
|
163
|
+
|
|
164
|
+
def _parse_int(s: str) -> Optional[int]:
|
|
165
|
+
"""serde_yaml_ng ``visit_int``: u64, i64, then u128, i128.
|
|
166
|
+
|
|
167
|
+
``Value`` has no 128-bit numbers, so an integer that only fits u128/i128
|
|
168
|
+
is an error ("invalid type: integer ..."), not a float or a string.
|
|
169
|
+
"""
|
|
170
|
+
n = _parse_unsigned(s, _U64_MAX)
|
|
171
|
+
if n is None:
|
|
172
|
+
n = _parse_negative(s, _I64_MIN, 2**63 - 1)
|
|
173
|
+
if n is not None:
|
|
174
|
+
return n
|
|
175
|
+
if _parse_unsigned(s, _U128_MAX) is not None or \
|
|
176
|
+
_parse_negative(s, _I128_MIN, 2**127 - 1) is not None:
|
|
177
|
+
raise KernelYamlError("invalid type: 128-bit integer")
|
|
178
|
+
return None
|
|
179
|
+
|
|
180
|
+
|
|
181
|
+
def _parse_f64(s: str) -> Optional[float]:
|
|
182
|
+
"""serde_yaml_ng ``parse_f64`` (Rust ``f64::from_str`` grammar, finite only)."""
|
|
183
|
+
if s[:1] == "+":
|
|
184
|
+
unpositive = s[1:]
|
|
185
|
+
if unpositive[:1] in ("+", "-"):
|
|
186
|
+
return None
|
|
187
|
+
else:
|
|
188
|
+
unpositive = s
|
|
189
|
+
if unpositive in (".inf", ".Inf", ".INF"):
|
|
190
|
+
return math.inf
|
|
191
|
+
if s in ("-.inf", "-.Inf", "-.INF"):
|
|
192
|
+
return -math.inf
|
|
193
|
+
if s in (".nan", ".NaN", ".NAN"):
|
|
194
|
+
return math.nan
|
|
195
|
+
if _RUST_F64.match(unpositive):
|
|
196
|
+
f = float(unpositive)
|
|
197
|
+
return f if math.isfinite(f) else None
|
|
198
|
+
return None # Rust parses inf/nan words, but they are not finite → None
|
|
199
|
+
|
|
200
|
+
|
|
201
|
+
def _parse_null(s: str) -> bool:
|
|
202
|
+
return s in ("null", "Null", "NULL", "~")
|
|
203
|
+
|
|
204
|
+
|
|
205
|
+
def _parse_bool(s: str) -> Optional[bool]:
|
|
206
|
+
if s in ("true", "True", "TRUE"):
|
|
207
|
+
return True
|
|
208
|
+
if s in ("false", "False", "FALSE"):
|
|
209
|
+
return False
|
|
210
|
+
return None
|
|
211
|
+
|
|
212
|
+
|
|
213
|
+
def _untagged_scalar(s: str) -> Any:
|
|
214
|
+
"""serde_yaml_ng ``visit_untagged_scalar`` (plain scalars)."""
|
|
215
|
+
if s == "" or _parse_null(s):
|
|
216
|
+
return None
|
|
217
|
+
b = _parse_bool(s)
|
|
218
|
+
if b is not None:
|
|
219
|
+
return b
|
|
220
|
+
n = _parse_int(s)
|
|
221
|
+
if n is not None:
|
|
222
|
+
return n
|
|
223
|
+
if not _digits_but_not_number(s):
|
|
224
|
+
f = _parse_f64(s)
|
|
225
|
+
if f is not None:
|
|
226
|
+
return f
|
|
227
|
+
return s
|
|
228
|
+
|
|
229
|
+
|
|
230
|
+
def _scalar(event: ev.ScalarEvent, tagged_already: bool) -> Any:
|
|
231
|
+
"""serde_yaml_ng ``visit_scalar``."""
|
|
232
|
+
v: str = event.value
|
|
233
|
+
tag = event.tag
|
|
234
|
+
plain = event.style is None
|
|
235
|
+
if tag is not None and not tagged_already:
|
|
236
|
+
if tag == _TAG_BOOL:
|
|
237
|
+
b = _parse_bool(v)
|
|
238
|
+
if b is None:
|
|
239
|
+
raise KernelYamlError("invalid value: expected a boolean")
|
|
240
|
+
return b
|
|
241
|
+
if tag == _TAG_INT:
|
|
242
|
+
n = _parse_int(v)
|
|
243
|
+
if n is None:
|
|
244
|
+
raise KernelYamlError("invalid value: expected an integer")
|
|
245
|
+
return n
|
|
246
|
+
if tag == _TAG_FLOAT:
|
|
247
|
+
f = _parse_f64(v)
|
|
248
|
+
if f is None:
|
|
249
|
+
raise KernelYamlError("invalid value: expected a float")
|
|
250
|
+
return f
|
|
251
|
+
if tag == _TAG_NULL:
|
|
252
|
+
if not _parse_null(v):
|
|
253
|
+
raise KernelYamlError("invalid value: expected null")
|
|
254
|
+
return None
|
|
255
|
+
if tag.startswith("!") and plain:
|
|
256
|
+
return _untagged_scalar(v)
|
|
257
|
+
elif plain:
|
|
258
|
+
return _untagged_scalar(v)
|
|
259
|
+
return v
|
|
260
|
+
|
|
261
|
+
|
|
262
|
+
def _enum_tag(tag: Optional[str]) -> Optional[str]:
|
|
263
|
+
"""serde_yaml_ng ``parse_tag``: local tags (``!...``) become ``Tagged``."""
|
|
264
|
+
if not tag or tag[0] != "!":
|
|
265
|
+
return None
|
|
266
|
+
try:
|
|
267
|
+
tag.encode("utf-8")
|
|
268
|
+
except UnicodeEncodeError:
|
|
269
|
+
return None # not valid UTF-8 (from %-escapes): serde_yaml_ng parse_tag → None
|
|
270
|
+
return tag[1:] or tag
|
|
271
|
+
|
|
272
|
+
|
|
273
|
+
# --- document loading --------------------------------------------------------
|
|
274
|
+
|
|
275
|
+
class _Parser(LibyamlScanner, yaml.parser.Parser):
|
|
276
|
+
"""PyYAML's parser over the libyaml scanner port, with libyaml's
|
|
277
|
+
directive rules (``%YAML`` must be 1.1 or 1.2, no duplicates)."""
|
|
278
|
+
|
|
279
|
+
def __init__(self, text: str) -> None:
|
|
280
|
+
LibyamlScanner.__init__(self, text)
|
|
281
|
+
yaml.parser.Parser.__init__(self)
|
|
282
|
+
|
|
283
|
+
def parse_flow_sequence_entry_mapping_key(self) -> Any:
|
|
284
|
+
# libyaml parse_flow_sequence_entry_mapping_key: the KEY token is
|
|
285
|
+
# already consumed by parse_flow_sequence_entry; when the key is empty
|
|
286
|
+
# it also consumes the next token (`:`, `,` or `]`). PyYAML does not,
|
|
287
|
+
# so `[? , a]` / `[?]` parse in PyYAML and fail in the kernel.
|
|
288
|
+
self.get_token() # KEY
|
|
289
|
+
if not self.check_token(yaml.tokens.ValueToken, yaml.tokens.FlowEntryToken,
|
|
290
|
+
yaml.tokens.FlowSequenceEndToken):
|
|
291
|
+
self.states.append(self.parse_flow_sequence_entry_mapping_value)
|
|
292
|
+
return self.parse_flow_node()
|
|
293
|
+
token = self.get_token()
|
|
294
|
+
self.state = self.parse_flow_sequence_entry_mapping_value
|
|
295
|
+
return self.process_empty_scalar(token.end_mark)
|
|
296
|
+
|
|
297
|
+
def process_directives(self) -> Any:
|
|
298
|
+
self.yaml_version = None
|
|
299
|
+
self.tag_handles = {}
|
|
300
|
+
while self.check_token(yaml.tokens.DirectiveToken):
|
|
301
|
+
token = self.get_token()
|
|
302
|
+
if token.name == "YAML":
|
|
303
|
+
if self.yaml_version is not None:
|
|
304
|
+
raise yaml.parser.ParserError(
|
|
305
|
+
None, None, "found duplicate %YAML directive", token.start_mark)
|
|
306
|
+
if token.value not in ((1, 1), (1, 2)):
|
|
307
|
+
raise yaml.parser.ParserError(
|
|
308
|
+
None, None, "found incompatible YAML document", token.start_mark)
|
|
309
|
+
self.yaml_version = token.value
|
|
310
|
+
elif token.name == "TAG":
|
|
311
|
+
handle, prefix = token.value
|
|
312
|
+
if handle in self.tag_handles:
|
|
313
|
+
raise yaml.parser.ParserError(
|
|
314
|
+
None, None, "found duplicate %TAG directive", token.start_mark)
|
|
315
|
+
self.tag_handles[handle] = prefix
|
|
316
|
+
value = self.yaml_version, (self.tag_handles.copy() if self.tag_handles else None)
|
|
317
|
+
for key in self.DEFAULT_TAGS:
|
|
318
|
+
if key not in self.tag_handles:
|
|
319
|
+
self.tag_handles[key] = self.DEFAULT_TAGS[key]
|
|
320
|
+
return value
|
|
321
|
+
|
|
322
|
+
|
|
323
|
+
def _events(text: str) -> List[Any]:
|
|
324
|
+
"""Events of the single document (serde_yaml_ng ``Loader``)."""
|
|
325
|
+
try:
|
|
326
|
+
parser = _Parser(text)
|
|
327
|
+
stream: List[Any] = []
|
|
328
|
+
while parser.check_event():
|
|
329
|
+
stream.append(parser.get_event())
|
|
330
|
+
except yaml.YAMLError as e:
|
|
331
|
+
raise KernelYamlError(str(e)) from None
|
|
332
|
+
docs: List[List[Any]] = []
|
|
333
|
+
current: Optional[List[Any]] = None
|
|
334
|
+
for event in stream:
|
|
335
|
+
if isinstance(event, ev.DocumentStartEvent):
|
|
336
|
+
current = []
|
|
337
|
+
elif isinstance(event, ev.DocumentEndEvent):
|
|
338
|
+
if current is not None:
|
|
339
|
+
docs.append(current)
|
|
340
|
+
current = None
|
|
341
|
+
elif current is not None:
|
|
342
|
+
current.append(event)
|
|
343
|
+
if len(docs) > 1:
|
|
344
|
+
raise KernelYamlError(
|
|
345
|
+
"deserializing from YAML containing more than one document is not supported")
|
|
346
|
+
return docs[0] if docs else []
|
|
347
|
+
|
|
348
|
+
|
|
349
|
+
class _Builder:
|
|
350
|
+
"""serde_yaml_ng ``DeserializerFromEvents`` → ``Value``.
|
|
351
|
+
|
|
352
|
+
Anchored nodes are built once and memoised with the number of alias jumps
|
|
353
|
+
and the collection depth they contain, so an alias costs O(1) while the
|
|
354
|
+
jump count and depth checks come out exactly as the kernel's re-walk.
|
|
355
|
+
"""
|
|
356
|
+
|
|
357
|
+
def __init__(self, events: List[Any]) -> None:
|
|
358
|
+
self.events = events
|
|
359
|
+
self.limit = max(len(events), 1) * _JUMP_FACTOR
|
|
360
|
+
self.jumps = 0
|
|
361
|
+
# alias event index -> anchored node event index (resolved at load)
|
|
362
|
+
self.alias_target: Dict[int, int] = {}
|
|
363
|
+
# anchored node index -> (value, jumps inside, depth inside, end index)
|
|
364
|
+
self.memo: Dict[int, Tuple[Any, int, int, int]] = {}
|
|
365
|
+
self.in_progress: Set[int] = set()
|
|
366
|
+
self._resolve_aliases()
|
|
367
|
+
|
|
368
|
+
def _resolve_aliases(self) -> None:
|
|
369
|
+
# serde_yaml_ng Loader: an anchor gets id = len(anchors) *after* any
|
|
370
|
+
# earlier definition of the same name, so a redefined name reuses a
|
|
371
|
+
# slot and a later new anchor can take an id already in use. An alias
|
|
372
|
+
# takes the id its name has when the alias is read; the id's target is
|
|
373
|
+
# the last node registered under it anywhere in the document.
|
|
374
|
+
anchors: Dict[str, int] = {}
|
|
375
|
+
id_pos: Dict[int, int] = {}
|
|
376
|
+
alias_id: Dict[int, int] = {}
|
|
377
|
+
for i, e in enumerate(self.events):
|
|
378
|
+
if isinstance(e, ev.AliasEvent):
|
|
379
|
+
if e.anchor not in anchors:
|
|
380
|
+
raise KernelYamlError("unknown anchor")
|
|
381
|
+
alias_id[i] = anchors[e.anchor]
|
|
382
|
+
elif isinstance(e, (ev.ScalarEvent, ev.SequenceStartEvent,
|
|
383
|
+
ev.MappingStartEvent)) and e.anchor is not None:
|
|
384
|
+
new_id = len(anchors)
|
|
385
|
+
anchors[e.anchor] = new_id
|
|
386
|
+
id_pos[new_id] = i
|
|
387
|
+
self.alias_target = {i: id_pos[a] for i, a in alias_id.items()}
|
|
388
|
+
|
|
389
|
+
def build(self) -> Any:
|
|
390
|
+
if not self.events:
|
|
391
|
+
return None # Event::Void → Value::Null
|
|
392
|
+
# Structural nesting past the limit fails before any recursion.
|
|
393
|
+
level = 0
|
|
394
|
+
for e in self.events:
|
|
395
|
+
if isinstance(e, (ev.SequenceStartEvent, ev.MappingStartEvent)):
|
|
396
|
+
level += 1
|
|
397
|
+
if level > _MAX_DEPTH:
|
|
398
|
+
raise KernelYamlError("recursion limit exceeded")
|
|
399
|
+
elif isinstance(e, (ev.SequenceEndEvent, ev.MappingEndEvent)):
|
|
400
|
+
level -= 1
|
|
401
|
+
value, _jumps, depth, _end = self._node(0)
|
|
402
|
+
if depth > _MAX_DEPTH:
|
|
403
|
+
raise KernelYamlError("recursion limit exceeded")
|
|
404
|
+
if self.jumps > self.limit:
|
|
405
|
+
raise KernelYamlError("repetition limit exceeded")
|
|
406
|
+
return value
|
|
407
|
+
|
|
408
|
+
def _node(self, i: int) -> Tuple[Any, int, int, int]:
|
|
409
|
+
"""Build the node at event ``i``.
|
|
410
|
+
|
|
411
|
+
Returns (value, alias jumps inside, collection depth, next index).
|
|
412
|
+
"""
|
|
413
|
+
e = self.events[i]
|
|
414
|
+
if isinstance(e, ev.AliasEvent):
|
|
415
|
+
value, inner_jumps, depth, _end = self._anchored(self.alias_target[i])
|
|
416
|
+
self._jump(1 + inner_jumps)
|
|
417
|
+
return value, 1 + inner_jumps, depth, i + 1
|
|
418
|
+
if getattr(e, "anchor", None) is not None:
|
|
419
|
+
value, inner_jumps, depth, end = self._anchored(i)
|
|
420
|
+
self._jump(inner_jumps) # the walk at the definition site
|
|
421
|
+
return value, inner_jumps, depth, end
|
|
422
|
+
return self._build(i)
|
|
423
|
+
|
|
424
|
+
def _anchored(self, i: int) -> Tuple[Any, int, int, int]:
|
|
425
|
+
"""Value of the anchored node at ``i``, built once. The jump count it
|
|
426
|
+
returns is charged by the caller, once per walk (definition or alias)."""
|
|
427
|
+
if i in self.memo:
|
|
428
|
+
return self.memo[i]
|
|
429
|
+
if i in self.in_progress:
|
|
430
|
+
# An alias inside its own anchor: the kernel recurses until the
|
|
431
|
+
# depth or repetition limit trips.
|
|
432
|
+
raise KernelYamlError("recursion limit exceeded")
|
|
433
|
+
self.in_progress.add(i)
|
|
434
|
+
before = self.jumps
|
|
435
|
+
value, _j, depth, end = self._build(i)
|
|
436
|
+
inner = self.jumps - before
|
|
437
|
+
self.jumps = before
|
|
438
|
+
self.in_progress.discard(i)
|
|
439
|
+
result = (value, inner, depth, end)
|
|
440
|
+
self.memo[i] = result
|
|
441
|
+
return result
|
|
442
|
+
|
|
443
|
+
def _jump(self, n: int) -> None:
|
|
444
|
+
self.jumps += n
|
|
445
|
+
if self.jumps > self.limit:
|
|
446
|
+
raise KernelYamlError("repetition limit exceeded")
|
|
447
|
+
|
|
448
|
+
def _build(self, i: int) -> Tuple[Any, int, int, int]:
|
|
449
|
+
e = self.events[i]
|
|
450
|
+
start_jumps = self.jumps
|
|
451
|
+
if isinstance(e, ev.ScalarEvent):
|
|
452
|
+
tag = _enum_tag(e.tag)
|
|
453
|
+
if tag is not None:
|
|
454
|
+
return Tagged(tag, _scalar(e, True)), 0, 0, i + 1
|
|
455
|
+
return _scalar(e, False), 0, 0, i + 1
|
|
456
|
+
if isinstance(e, ev.SequenceStartEvent):
|
|
457
|
+
items: List[Any] = []
|
|
458
|
+
depth = 0
|
|
459
|
+
j = i + 1
|
|
460
|
+
while not isinstance(self.events[j], ev.SequenceEndEvent):
|
|
461
|
+
value, _jumps, d, j = self._node(j)
|
|
462
|
+
items.append(value)
|
|
463
|
+
depth = max(depth, d)
|
|
464
|
+
depth += 1
|
|
465
|
+
if depth > _MAX_DEPTH:
|
|
466
|
+
raise KernelYamlError("recursion limit exceeded")
|
|
467
|
+
tag = _enum_tag(e.tag)
|
|
468
|
+
result: Any = Tagged(tag, items) if tag is not None else items
|
|
469
|
+
return result, self.jumps - start_jumps, depth, j + 1
|
|
470
|
+
if isinstance(e, ev.MappingStartEvent):
|
|
471
|
+
mapping = Mapping()
|
|
472
|
+
depth = 0
|
|
473
|
+
j = i + 1
|
|
474
|
+
while not isinstance(self.events[j], ev.MappingEndEvent):
|
|
475
|
+
key, _jumps, dk, j = self._node(j)
|
|
476
|
+
k = _key(key)
|
|
477
|
+
if k in mapping:
|
|
478
|
+
raise KernelYamlError("duplicate entry with key")
|
|
479
|
+
value, _jumps, dv, j = self._node(j)
|
|
480
|
+
mapping[k] = value
|
|
481
|
+
depth = max(depth, dk, dv)
|
|
482
|
+
depth += 1
|
|
483
|
+
if depth > _MAX_DEPTH:
|
|
484
|
+
raise KernelYamlError("recursion limit exceeded")
|
|
485
|
+
tag = _enum_tag(e.tag)
|
|
486
|
+
result = Tagged(tag, mapping) if tag is not None else mapping
|
|
487
|
+
return result, self.jumps - start_jumps, depth, j + 1
|
|
488
|
+
raise KernelYamlError("unexpected event") # pragma: no cover
|
|
489
|
+
|
|
490
|
+
|
|
491
|
+
def load(text: str) -> Any:
|
|
492
|
+
"""Load YAML exactly as faf-kernel reads it, or raise :class:`KernelYamlError`."""
|
|
493
|
+
return _Builder(_events(text)).build()
|