semantifold 0.14.0 → 0.15.0

package/docs/goals.md

@@ -2,7 +2,7 @@
 
 Semantifold's long-term goal is a language-neutral toolkit for understanding, transforming, generating, and eventually executing programs through a shared high-level semantic representation. The representation should make types, control flow, calls, declarations, diagnostics, and source provenance understandable without exposing a language parser's private tree.
 
-The current source tree proves one minimal Tasks 001–004 vertical slice across fifteen registered source languages: PHP, Ruby, JavaScript, TypeScript, Java, Kotlin/JVM, strictly annotated Python, canonical C#, bounded Go, C17, C++20, Rust edition 2021, Swift, Dart, and Zig. Dart and Zig additionally implement the bounded Task 005 required-function/call profile; the original five retain the delivered Task 013 acceptance gate for the combined Tasks 001–008 profile, Task 009 nominal closed records, Task 010 explicit acyclic multi-source semantic programs, Task 011 exact nominal typed unchecked errors, and Task 012 invariant unbounded function/record type parameters with closed applications and argument-only first-order call inference. Task 014 is delivered through PR #41, Task 032 through PR #42, Task 033 through PR #43, and Task 034 through PR #44. Task 035 is delivered through PR #45 and merge commit `a2aad9d4932bfc6087adc4ff8bca43678c92413c`, with versioned canonical standard-library contracts, target host providers and protected native bindings, capability negotiation, and program-mode provider linking. Task 036 is delivered through PR #46 and merge `9aba48f9d48bcef4fd4c9275aec14bfcc4f0904c`; it adds bounded versioned executable qualification facades and exact parser-backed native identity resolution for the original five. Task 037 adds only the first concrete Ruby-source-to-PHP-target blocking TCP/line-output slice on those foundations. The source tree includes versioned source provenance, deterministic semantic node/symbol identity, range-based generated mappings, Source Map v3 interop, composition, and diagnostic remapping across those bounded profiles. The published `semantifold@0.6.0` and annotated `v0.6.0` target exact commit `f76e6d64b388d65959e77f747ae2d60a999d61f5`, including Task 028 from PR #51 and Task 031 from PR #52. Neither the source tree nor that release provides a general analyzer, transformation API, interpreter, optimizer, implicit project/package discovery, debugger integration, coverage remapping, or lossless formatter.
+The current source tree proves one minimal Tasks 001–004 vertical slice across fifteen registered source languages: PHP, Ruby, JavaScript, TypeScript, Java, Kotlin/JVM, strictly annotated Python, canonical C#, bounded Go, C17, C++20, Rust edition 2021, Swift, Dart, and Zig. C, Dart, Zig, and Python additionally implement the bounded Task 005 required-function/call profile; the original five retain the delivered Task 013 acceptance gate for the combined Tasks 001–008 profile, Task 009 nominal closed records, Task 010 explicit acyclic multi-source semantic programs, Task 011 exact nominal typed unchecked errors, and Task 012 invariant unbounded function/record type parameters with closed applications and argument-only first-order call inference. Task 014 is delivered through PR #41, Task 032 through PR #42, Task 033 through PR #43, and Task 034 through PR #44. Task 035 is delivered through PR #45 and merge commit `a2aad9d4932bfc6087adc4ff8bca43678c92413c`, with versioned canonical standard-library contracts, target host providers and protected native bindings, capability negotiation, and program-mode provider linking. Task 036 is delivered through PR #46 and merge `9aba48f9d48bcef4fd4c9275aec14bfcc4f0904c`; it adds bounded versioned executable qualification facades and exact parser-backed native identity resolution for the original five. Task 037 adds only the first concrete Ruby-source-to-PHP-target blocking TCP/line-output slice on those foundations. The source tree includes versioned source provenance, deterministic semantic node/symbol identity, range-based generated mappings, Source Map v3 interop, composition, and diagnostic remapping across those bounded profiles. The published `semantifold@0.6.0` and annotated `v0.6.0` target exact commit `f76e6d64b388d65959e77f747ae2d60a999d61f5`, including Task 028 from PR #51 and Task 031 from PR #52. Neither the source tree nor that release provides a general analyzer, transformation API, interpreter, optimizer, implicit project/package discovery, debugger integration, coverage remapping, or lossless formatter.
 
 The long-term product direction includes bounded standard-library portability through [language compatibility stdlib/facades, canonical Semantifold stdlib contracts/capabilities, and target host providers/native bindings](standard-library-portability.md). That direction is incremental and versioned; it does not promise arbitrary standard-library coverage or arbitrary-application translation. The `0.3.0` release has no imports, classes or methods, loops, optional values, typed errors, resources, host APIs, or stdlib portability.
 

package/docs/language-support.md

@@ -12,7 +12,7 @@ The immutable registry is authoritative for both dispatch and discovery:
 | `typescript` | yes | yes | no | no | no | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes | single | yes | rich + v3 | parse, generate, compile, execute (`tsc`, `node`) |
 | `java` | yes | yes | no | no | no | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes | single | yes | rich + v3 | parse, generate, compile, execute (`javac`, `java`) |
 | `kotlin` | yes | yes | no | no | no | no | no | no | no | no | no | no | no | no | no | no | single | yes | rich + v3 | parse, generate, compile, execute (`kotlinc`, `java25`) |
-| `python` | yes | yes | no | no | no | no | no | no | no | no | no | no | no | no | no | no | single | yes | rich + v3 | parse, generate, compile, execute (`python3`) |
+| `python` | yes | yes | no | no | no | yes | no | no | no | no | no | no | no | no | no | no | single | yes | rich + v3 | parse, generate, compile, execute (`python3`) |
 | `csharp` | yes | yes | no | no | no | no | no | no | no | no | no | no | no | no | no | no | multiple | yes | rich + v3 | parse, generate, restore, compile, execute (`dotnet`) |
 | `go` | yes | yes | no | no | no | no | no | no | no | no | no | no | no | no | no | no | multiple | yes | rich + v3 | parse, generate, compile, validate, execute (`go`) |
 | `cpp` | yes | yes | no | no | no | no | no | no | no | no | no | no | no | no | no | no | single | yes | rich + v3 | parse, generate, compile, link, execute (`clangpp`, native executable) |
@@ -84,7 +84,7 @@ The shared negative corpus covers parser recovery, missing types, truthiness, mi
 
 ## Task 005 general functions and calls
 
-PHP, Ruby, JavaScript with JSDoc, TypeScript, Java, C, Dart, and Zig accept any number of required positional parameters, including zero. The module resolver collects every unique function signature before validating bodies, assigns declarations deterministic `function:N` identities, and binds each direct call to that identity plus its ordered parameter and return types. This permits forward calls, nested calls, direct recursion, and unambiguous mutual recursion without overload or dynamic dispatch. Arity and every positional scalar type must match exactly.
+PHP, Ruby, JavaScript with JSDoc, TypeScript, Java, C, Dart, Zig, and Python accept any number of required positional parameters, including zero. The module resolver collects every unique function signature before validating bodies, assigns declarations deterministic `function:N` identities, and binds each direct call to that identity plus its ordered parameter and return types. This permits forward calls, nested calls, direct recursion, and unambiguous mutual recursion without overload or dynamic dispatch. Arity and every positional scalar type must match exactly.
 
 Function returns are `integer`, `boolean`, `string`, or `void`. A non-void function has an expression on every reachable return and returns on every reachable path. A void function may fall through or use a bare return, and its call is legal only as an `ExpressionStatement`. Void is never a value: it cannot type a parameter or local, initialize or assign a binding, become an argument or operand, be printed, or be returned as a value. Generation revalidates declaration identities and complete resolved signatures in externally supplied IR. Function names, parameters, return annotations, callees, arguments, and resolution references retain deterministic parser-backed provenance and mappings.
 
@@ -97,8 +97,9 @@ The source profiles remain deliberately exact:
 - C accepts file-scope `static` semantic functions with any number of explicitly typed named parameters — spelled `void` for an empty list — exact scalar or `void` returns, and unqualified direct calls. A `static` prototype may precede a definition for forward or mutual recursion and must agree exactly with its unique definition; a prototype without a definition, a conflicting prototype, an implicit declaration, a bare return in a non-void function, or `void` used as a parameter, local, argument, operand or printed value is rejected. Generated zero-parameter signatures spell `(void)` and generated prototypes make forward calls and recursion compilable under `-Wstrict-prototypes`/`-Wmissing-prototypes`. See [C source and target](c.md).
 - Dart accepts synchronous top-level functions with exact explicit `int`, `bool`, `String`, or `void` returns, explicitly typed required positional parameters, initialized typed locals, direct calls, and exact `void main()`/`print`. Inference, `var`, inferred `final`, optional/named/default arguments, methods/classes, generics, imports, dynamic/nullability, collections, exceptions, async, and isolates are rejected. Generated Dart alone may contain formatter-added trailing commas and the complete canonical private runtime prefix; neither grants those forms to ordinary source. See [Dart source and target](dart.md).
 - Zig accepts top-level functions with exact explicit `i64`, `bool`, `[]const u8`, or `void` returns, explicitly typed required positional parameters, initialized typed `const`/`var` locals, direct calls, and the exact generated support/`pub fn main()` shell. Inference, optional/default/rest parameters, methods, containers, user slices/pointers/allocators, imports beyond the canonical standard-library support, error unions, `comptime`, generics, reflection, FFI, async, and concurrency are rejected. See [Zig source and target](zig.md).
+- Python accepts synchronous top-level `def` functions with any number of explicitly `int`/`bool`/`str`-annotated required positional parameters and an exact `int`/`bool`/`str` or `None` return annotation, where `None` is the only void spelling. A void function may fall through or use a bare `return`, and its call is legal only as a top-level or block statement expression. Direct calls are identifier callees with positional arguments only; forward calls, direct recursion, and unambiguous mutual recursion resolve exactly. Default, positional-only, keyword-only, variadic, starred, or unannotated parameters, keyword or starred call arguments, nested function declarations, and every dynamic callee form remain rejected; a value `return` inside a `-> None` function fails with `VOID_RETURN_VALUE`.
 
-Python, C#, C++, Rust, Swift, Kotlin/JVM, Go, and browser Wasm remain on their Task 025 Tasks 001–004 profile. Their registry feature flag is false, their existing exact-two/scalar-return behavior is unchanged, and a Task-005-only arity, void return, or void call statement is a located `UNSUPPORTED_CAPABILITY` before source, bytes, or a partial artifact set is exposed. This boundary does not broaden the future Task 013 aggregate acceptance.
+C#, C++, Rust, Swift, Kotlin/JVM, Go, and browser Wasm remain on their Task 025 Tasks 001–004 profile. Their registry feature flag is false, their existing exact-two/scalar-return behavior is unchanged, and a Task-005-only arity, void return, or void call statement is a located `UNSUPPORTED_CAPABILITY` before source, bytes, or a partial artifact set is exposed. This boundary does not broaden the future Task 013 aggregate acceptance.
 
 ## Task 006 immutable lists and maps
 

package/docs/testing.md

@@ -36,7 +36,7 @@ These tests generate and publish source artifacts only. They do not run `javac`,
 
 ## Task 005 general function signatures and calls
 
-Task 005 stays bounded to PHP, Ruby, JavaScript with JSDoc, TypeScript, Java, Dart, Zig, and C. `general-function-signatures-and-calls.spec.js` proves equivalent zero-, one-, and three-parameter signatures, complete signature collection, nested calls, terminating recursion, scalar/void returns, and the stable semantic failures. `general-function-frontend-validation.spec.js` owns the excluded parameter, call, method, receiver, dispatch, annotation, overload, generic, and higher-order source forms. `general-function-backend-validation.spec.js` proves generation/reparse, exact external-IR signature revalidation, target conflicts, the public feature flag, and transactional rejection from every deferred Task 025/Wasm target. `general-function-provenance-and-mapping.spec.js` proves deterministic declaration/resolution identities and exact signature/call mappings. `general-function-runtime-execution.spec.js` invokes real PHP, Ruby, Node, local `tsc` followed by Node, `javac`, and `java`, requiring exact `ready\n6\n` output; a missing command fails. `c-task005-acceptance.spec.js` proves the C cohort extension: generated-C reparse with identical meaning, the C fixture through every original-five text target, and real Clang execution of each original-five Task-005 fixture and the C fixture at `-O0`/`-O2` with and without sanitizers, including prototype-mediated mutual recursion and bare void returns.
+Task 005 stays bounded to PHP, Ruby, JavaScript with JSDoc, TypeScript, Java, Dart, Zig, C, and Python. `general-function-signatures-and-calls.spec.js` proves equivalent zero-, one-, and three-parameter signatures, complete signature collection, nested calls, terminating recursion, scalar/void returns, and the stable semantic failures across the six source fixtures, including `fixtures/functions/program.py`. `general-function-frontend-validation.spec.js` owns the excluded parameter, call, method, receiver, dispatch, annotation, overload, generic, and higher-order source forms. `general-function-backend-validation.spec.js` proves generation/reparse, exact external-IR signature revalidation, target conflicts, the public feature flag, and transactional rejection from every deferred Task 025/Wasm target. `general-function-provenance-and-mapping.spec.js` proves deterministic declaration/resolution identities and exact signature/call mappings. `general-function-runtime-execution.spec.js` invokes real PHP, Ruby, Node, local `tsc` followed by Node, `javac`, `java`, and `python3`, requiring exact `ready\n6\n` output; a missing command fails. `c-task005-acceptance.spec.js` proves the C cohort extension: generated-C reparse with identical meaning, the C fixture through every original-five text target, and real Clang execution of each original-five Task-005 fixture and the C fixture at `-O0`/`-O2` with and without sanitizers, including prototype-mediated mutual recursion and bare void returns.
 
 Run the Task 005 files individually and sequentially:
 
@@ -49,7 +49,7 @@ npx velocious-test spec/general-function-runtime-execution.spec.js
 npx velocious-test spec/c-task005-acceptance.spec.js
 ```
 
-Task 013 is the separate terminal original-five aggregate gate. Task 005 does not expand that matrix or adopt Python, C#, C++, Rust, Swift, Kotlin/JVM, Go, or browser Wasm beyond focused capability rejection.
+Task 013 is the separate terminal original-five aggregate gate. Task 013 is the separate terminal original-five aggregate gate. Task 005 does not expand that matrix or adopt C#, C++, Rust, Swift, Kotlin/JVM, Go, or browser Wasm beyond focused capability rejection.
 
 ## Task 011 typed errors and handling
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "semantifold",
-  "version": "0.14.0",
+  "version": "0.15.0",
   "description": "Language-neutral semantic code analysis, transformation, generation, and execution toolkit",
   "type": "module",
   "main": "build/index.js",

package/src/backends/python.js

@@ -81,14 +81,20 @@ function emitStatement(writer, statement, indent, path) {
     emitIf(writer, statement, indent, path, "if")
     return
   }
-  if (statement.kind == "ExpressionStatement") throw new TypeError("Python Task 005 expression statement reached emission.")
+  if (statement.kind == "ExpressionStatement") {
+    writer.synthetic(indent, "indentation", [statement], [path])
+    emitExpression(writer, statement.expression, `${path}/expression`, "python", identity)
+    writer.synthetic("\n", "line break", [statement], [path])
+    return
+  }
 
   writer.synthetic(indent, "indentation", [statement], [path])
   if (statement.kind == "ReturnStatement") {
-    if (!statement.expression) throw new TypeError("Python bare return reached emission.")
     writer.mapped("return", {mappingKind: "anchor", node: statement, path})
-    writer.synthetic(" ", "return spacing", [statement], [path])
-    emitExpression(writer, statement.expression, `${path}/expression`, "python", identity)
+    if (statement.expression) {
+      writer.synthetic(" ", "return spacing", [statement], [path])
+      emitExpression(writer, statement.expression, `${path}/expression`, "python", identity)
+    }
     writer.synthetic("\n", "line break", [statement], [path])
     return
   }

package/src/backends/shared.js

@@ -27,7 +27,7 @@ const binaryOperationSyntax = Object.freeze({
   StringEqual: Object.freeze({default: "==", strict: "==="}),
   StringNotEqual: Object.freeze({default: "!=", strict: "!=="})
 })
-const task005Languages = new Set(["php", "ruby", "javascript", "typescript", "java", "c", "dart", "zig"])
+const task005Languages = new Set(["php", "ruby", "javascript", "typescript", "java", "c", "dart", "zig", "python"])
 const task006Languages = new Set(["php", "ruby", "javascript", "typescript", "java"])
 const task007Languages = new Set(["php", "ruby", "javascript", "typescript", "java"])
 const task008Languages = new Set(["php", "ruby", "javascript", "typescript", "java"])

package/src/frontends/python.js

@@ -5,8 +5,8 @@ import PythonLanguage from "tree-sitter-python"
 import {missingType, parseFailure, SemantifoldDiagnostic, unsupportedSyntax} from "../diagnostic.js"
 import {locationFromOffsets, moduleLocation, utf8ByteOffsetToUtf16Offset} from "../semantic/location.js"
 import {withAdaptedOperation} from "../semantic/operators.js"
-import {withParserRanges} from "../semantic/provenance.js"
-import {hasOnlyUnicodeScalars} from "../semantic/scalars.js"
+import {setParserRanges, withParserRanges} from "../semantic/provenance.js"
+import {hasOnlyUnicodeScalars, voidType} from "../semantic/scalars.js"
 import {requireSourceScalarType} from "./scalars.js"
 
 const parser = new Parser()
@@ -171,6 +171,31 @@ function convertType(node, subject, ownerLocation, filename, source) {
   return requireSourceScalarType("python", children[0].text, subject, ownerLocation, typeLocation)
 }
 
+/**
+ * Converts one exact scalar or Task 005 `None` function-return annotation.
+ * @param {import("tree-sitter").SyntaxNode | null} node - Return type node, if present.
+ * @param {string} subject - Typed function return.
+ * @param {import("../semantic/types.js").SourceLocation} ownerLocation - Owning function.
+ * @param {string} filename - Source filename.
+ * @param {string} source - Complete source.
+ * @returns {import("../semantic/types.js").SemanticFunctionReturnType} Semantic return type.
+ */
+function convertReturnType(node, subject, ownerLocation, filename, source) {
+  if (!node) return missingType("python", subject, ownerLocation)
+  const typeLocation = nodeLocation(node, filename, source)
+  const children = node.namedChildren.filter((child) => child.type != "comment")
+
+  if (node.type == "type" && children.length == 1 && children[0].type == "none" && node.text == "None") {
+    const type = voidType()
+
+    setParserRanges(type, {type: typeLocation})
+
+    return type
+  }
+
+  return convertType(node, subject, ownerLocation, filename, source)
+}
+
 /**
  * Finds exactly one anonymous operator token directly owned by a parser node.
  * @param {import("tree-sitter").SyntaxNode} node - Operation node.
@@ -434,7 +459,8 @@ function convertStatement(node, context, visible, filename, source) {
   if (node.type == "return_statement") {
     const expressions = node.namedChildren.filter((child) => child.type != "comment")
 
-    if (expressions.length != 1) return unsupportedSyntax("python", "return without one expression", location)
+    if (expressions.length > 1) return unsupportedSyntax("python", "return with more than one expression", location)
+    if (expressions.length == 0) return {kind: "ReturnStatement", location}
 
     return {expression: convertExpression(expressions[0], filename, source), kind: "ReturnStatement", location}
   }
@@ -444,8 +470,16 @@ function convertStatement(node, context, visible, filename, source) {
 
     if (expressions.length != 1) return unsupportedSyntax("python", node.type, location)
     if (expressions[0].type == "assignment") return convertAssignment(expressions[0], context, visible, filename, source)
+    if (expressions[0].type == "call") {
+      const calleeNode = requiredField(expressions[0], "function", filename, source)
+
+      if (calleeNode.type == "identifier" && calleeNode.text == "print") return convertPrint(expressions[0], filename, source, location)
+      const call = /** @type {import("../semantic/types.js").CallExpression} */ (convertExpression(expressions[0], filename, source))
+
+      return {expression: call, kind: /** @type {const} */ ("ExpressionStatement"), location}
+    }
 
-    return convertPrint(expressions[0], filename, source, location)
+    return unsupportedSyntax("python", "expression statement other than assignment, print, or direct call", location)
   }
 
   return unsupportedSyntax("python", node.type, location)
@@ -611,7 +645,7 @@ function convertFunction(node, context, filename, source) {
     location,
     name,
     parameters,
-    returnType: convertType(returnNode, `Function '${name}' return`, location, filename, source)
+    returnType: convertReturnType(returnNode, `Function '${name}' return`, location, filename, source)
   }, {name: nodeLocation(nameNode, filename, source)})
 }
 

package/src/language-registry.js

@@ -545,6 +545,22 @@ const records = [
     acceptance: {stages: ["parse", "generate", "compile", "execute"], toolchains: ["python"]},
     check: {factory: createPythonCheckPlan, stages: ["compile"], toolchains: ["python"]},
     defaultFilename: "program.py",
+    features: {
+      boundedSubtyping: false,
+      closedRecords: false,
+      conditionControlledLoops: false,
+      conditionalExpressions: false,
+      effectfulCapabilitiesAndResources: false,
+      generalFunctionsAndCalls: true,
+      immutableCollections: false,
+      optionalValues: false,
+      orderedListIteration: false,
+      orderedMapIteration: false,
+      referenceClasses: false,
+      stringInterpolation: false,
+      typedErrors: false,
+      typeParametersAndGenerics: false
+    },
     frontend: pythonFrontend,
     id: "python",
     mediaType: "text/x-python",

package/src/semantic/validate.js

@@ -8,7 +8,7 @@ import {parserRangeFor} from "./provenance.js"
 import {validateEffectGraph} from "./effects.js"
 import {validateResourceLifetimes} from "./lifetime.js"
 
-const task005Languages = new Set(["php", "ruby", "javascript", "typescript", "java", "c", "dart", "zig"])
+const task005Languages = new Set(["php", "ruby", "javascript", "typescript", "java", "c", "dart", "zig", "python"])
 
 /** @typedef {Map<string, import("./types.js").RecordDeclaration>} RecordRegistry */
 /** @typedef {Map<string, import("./types.js").ClassDeclaration>} ClassRegistry */