eyelang 1.7.2 → 1.7.4

package/README.md

@@ -62,6 +62,7 @@ python3 -m http.server
 ```bash
 npm test
 npm run test:conformance
+npm run conformance:report
 npm run test:examples
 npm run test:regression
 ```

package/docs/guide.md

@@ -497,7 +497,13 @@ node test/run-regression.mjs
 node test/run-examples.mjs
 ```
 
-The conformance suite lives in [`test/conformance/`](../test/conformance/) as one flat eyelang corpus. Each case is a small `.eye` program with an exact expected stdout `.eye` file, and some cases also include a goal file for testing the embeddable solver, so other implementations can reuse the same cases. The suite covers the standard language surface from the language reference, including reusable built-ins. The regression suite lives in [`test/run-regression.mjs`](../test/run-regression.mjs) and covers CLI regressions, the public JavaScript API, and white-box invariants for parser, unification, and indexing behavior.
+Summarize the conformance corpus by category:
+
+```sh
+npm run conformance:report
+```
+
+The conformance suite lives in [`test/conformance/`](../test/conformance/) as a file-based eyelang corpus. Positive cases pair `cases/<name>.eye` with exact expected stdout under `expected/<name>.eye`; negative cases pair `errors/<name>.eye` with exact expected error text under `expected-errors/<name>.txt`; warning cases pair `warnings/<name>.eye` with exact `--warnings` stdout and stderr files under `expected-warnings/`. Cases may be grouped in category directories such as `atoms/`, `variables/`, `negation/`, and `syntax/`, so another implementation can reuse the same corpus as an executable language contract. The suite covers the standard language surface from the language reference, including reusable built-ins, standard errors, and standard warnings. The regression suite lives in [`test/run-regression.mjs`](../test/run-regression.mjs) and covers CLI regressions, the public JavaScript API, and white-box invariants for parser, unification, and indexing behavior.
 
 ## Development and release
 
@@ -506,6 +512,7 @@ Common commands:
 ```sh
 npm run test:eyelang        # alias for npm test
 npm test                    # full conformance, regression/API/white-box, examples, and proof examples
+npm run conformance:report  # conformance coverage summary by category
 node test/run-conformance.mjs
 node test/run-regression.mjs
 node test/run-examples.mjs

package/docs/language-reference.md

@@ -702,7 +702,7 @@ A conforming eyelang implementation supports the standard language described abo
 
 Browser execution, package layout, CLI URL loading, and any implementation-specific built-ins described in host documentation are outside this conformance surface unless separately standardized.
 
-Conformance cases live in the repository under `test/conformance/`. They are run by `npm test` before the example suite, and can be run alone with `node test/run-conformance.mjs`. Each case has an input program under `conformance/cases/` and an exact expected standard-output file under `conformance/expected/`; both use `.eye` so expected output remains eyelang-readable.
+Conformance cases live in the repository under `test/conformance/`. They are run by `npm test` before the example suite, and can be run alone with `node test/run-conformance.mjs`. Positive cases have input programs under `conformance/cases/` and exact expected standard-output files under `conformance/expected/`; both use `.eye` so expected output remains eyelang-readable. Expected-error cases live under `conformance/errors/` with exact messages under `conformance/expected-errors/`. Expected-warning cases live under `conformance/warnings/` with exact `--warnings` stdout and stderr files under `conformance/expected-warnings/`.
 
 ## 15. Relationship to ISO Prolog
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "eyelang",
-  "version": "1.7.2",
+  "version": "1.7.4",
   "description": "A small Prolog-like logic programming language for rules, goals, answers, and proofs.",
   "type": "module",
   "main": "./index.js",
@@ -44,6 +44,7 @@
     "test:examples": "node test/run-examples.mjs",
     "test:regression": "node test/run-regression.mjs",
     "preversion": "npm test",
-    "postversion": "git push origin HEAD --follow-tags"
+    "postversion": "git push origin HEAD --follow-tags",
+    "conformance:report": "node test/run-conformance-report.mjs"
   }
 }

package/test/conformance/README.md

@@ -2,12 +2,25 @@
 
 This directory contains the executable conformance cases for the Eyelang language and reference engine. The normative language description is in the [Eyelang language reference](../../../docs/language-reference.md).
 
-The suite is intentionally file-based so another implementation can run the same programs and compare exact standard output. A case consists of:
+The suite is intentionally file-based so another implementation can run the same programs and compare exact standard output, expected errors, and expected warnings. The conformance corpus is part of the public language contract, not just an implementation smoke test.
+
+A normal positive case consists of:
 
 - `conformance/cases/<name>.eye` — input program;
 - `conformance/expected/<name>.eye` — exact expected standard output, stored as eyelang-readable facts.
 
-The current runner compares standard output from normal execution. Proof explanations are opt-in in the CLI and are not part of these conformance goldens. Standard error, performance, and resource limits are outside this suite.
+Expected-error cases consist of:
+
+- `conformance/errors/<name>.eye` — input program that must fail during parsing or execution;
+- `conformance/expected-errors/<name>.txt` — exact expected error message followed by a newline.
+
+Expected-warning cases consist of:
+
+- `conformance/warnings/<name>.eye` — input program run through the CLI with `--warnings`;
+- `conformance/expected-warnings/<name>.eye` — exact expected standard output;
+- `conformance/expected-warnings/<name>.txt` — exact expected standard error.
+
+Case names may be nested in category directories such as `atoms/`, `variables/`, `negation/`, or `syntax/`. Expected files mirror the same relative path.
 
 ## Running the suite
 
@@ -23,21 +36,29 @@ Run only the conformance suite:
 node test/run-conformance.mjs
 ```
 
-Run matching conformance cases by passing a filename fragment:
+Summarize conformance coverage by category:
+
+```sh
+npm run conformance:report
+```
+
+Run matching conformance cases by passing a filename or directory fragment:
 
 ```sh
 node test/run-conformance.mjs reusable
 node test/run-conformance.mjs 092_scalar_string_conversions
+node test/run-conformance.mjs variables/
+node test/run-conformance.mjs error/variables
 ```
 
-The runner executes materialized programs in-process through the public JavaScript API so small conformance cases avoid measuring Node startup overhead.
+The runner executes normal materialized programs in-process through the public JavaScript API so small conformance cases avoid measuring Node startup overhead. Warning cases intentionally use the CLI because warning output is a host-interface contract.
 
 ## Scope
 
-The conformance corpus is a single eyelang suite. It covers the standard language described by the language reference: lexical syntax, facts, definite clauses, first-order terms, lists, conjunction, structured unification, left-to-right goal-directed proof search, materialized output, read-back printing, standard built-ins, declarations, and standard host behavior.
+The conformance corpus is a single eyelang suite. It covers the standard language described by the language reference: lexical syntax, facts, definite clauses, first-order terms, lists, conjunction, structured unification, left-to-right goal-directed proof search, materialized output, read-back printing, standard built-ins, declarations, warnings, errors, and standard host behavior.
 
 The suite deliberately does not separate `core` and `extension` profiles. Reusable built-ins such as arithmetic, strings, lists, aggregation, context terms, term inspection, and search control are part of the standard eyelang conformance surface. Implementation-specific built-ins may still exist in downstream hosts, but they should have their own tests outside this corpus unless they are standardized.
 
 ## Updating expected output
 
-There is no committed auto-accept mode. To update an expected file, run the matching case with the conformance runner, inspect the result, and replace the corresponding file under `conformance/expected/` deliberately.
+There is no committed auto-accept mode. To update an expected file, run the matching case with the conformance runner, inspect the result, and replace the corresponding file under `conformance/expected/`, `conformance/expected-errors/`, or `conformance/expected-warnings/` deliberately.

package/test/conformance/cases/atoms/graphic_angle_atom_stays_graphic.eye

@@ -0,0 +1,4 @@
+% Graphic atoms that are not absolute IRIs remain graphic atoms.
+materialize(answer, 1).
+operator(<=>).
+answer(?op) :- operator(?op).

package/test/conformance/cases/atoms/graphic_less_than_atom.eye

@@ -0,0 +1,4 @@
+% A lone graphic < remains a graphic atom, not an IRI opener.
+materialize(answer, 1).
+seed(<).
+answer(?x) :- seed(?x).

package/test/conformance/cases/atoms/iri_atoms_in_lists.eye

@@ -0,0 +1,4 @@
+% IRI atoms can appear anywhere ordinary atoms can appear, including lists.
+materialize(answer, 1).
+route([<urn:example:a>, <urn:example:b>, <urn:example:c>]).
+answer(?route) :- route(?route).

package/test/conformance/cases/atoms/iri_mailto_readback.eye

@@ -0,0 +1,4 @@
+% Non-http absolute IRI atoms are ordinary atoms.
+materialize(answer, 1).
+seed(<mailto:alice@example.org>).
+answer(?x) :- seed(?x).

package/test/conformance/cases/atoms/iri_query_fragment_readback.eye

@@ -0,0 +1,4 @@
+% Query strings and fragments are part of the IRI atom text.
+materialize(answer, 1).
+seed(<https://example.org/path?x=1#frag>).
+answer(?x) :- seed(?x).

package/test/conformance/cases/atoms/iri_quoted_absolute_readback.eye

@@ -0,0 +1,5 @@
+% Quoted absolute IRI atoms read back in angle-bracket form.
+materialize(answer, 1).
+item('https://example.org/alice').
+item('urn:example:bob').
+answer(?iri) :- item(?iri).

package/test/conformance/cases/atoms/iri_quoted_and_angle_unify.eye

@@ -0,0 +1,3 @@
+% Angle-bracket and quoted spellings denote the same absolute IRI atom.
+materialize(answer, 1).
+answer(ok) :- eq(<urn:example:a>, 'urn:example:a').

package/test/conformance/cases/atoms/iri_urn_readback.eye

@@ -0,0 +1,4 @@
+% URN IRI atoms read back in angle-bracket form.
+materialize(answer, 1).
+seed(<urn:example:alpha>).
+answer(?x) :- seed(?x).

package/test/conformance/cases/atoms/quoted_urn_iri_readback.eye

@@ -0,0 +1,4 @@
+% Quoted absolute IRI atoms use canonical angle-bracket read-back.
+materialize(answer, 1).
+seed('urn:example:quoted').
+answer(?x) :- seed(?x).

package/test/conformance/cases/atoms/zero_arity_atom_roundtrip.eye

@@ -0,0 +1,5 @@
+% Arity-zero data is represented as atoms, never zero-arity compounds.
+materialize(answer, 2).
+answer(name, ?name) :- compound_name_arguments(nil, ?name, ?args).
+answer(args, ?args) :- compound_name_arguments(nil, ?name, ?args).
+answer(term, ?term) :- compound_name_arguments(?term, nil, []).

package/test/conformance/cases/builtins/atom_string_iri_atom.eye

@@ -0,0 +1,3 @@
+% atom_string/2 converts an IRI atom to its lexical string.
+materialize(answer, 1).
+answer(?text) :- atom_string(<urn:example:a>, ?text).

package/test/conformance/cases/builtins/eq_unifies_iri_atoms.eye

@@ -0,0 +1,3 @@
+% eq/2 unifies angle and quoted spellings of the same absolute IRI atom.
+materialize(answer, 1).
+answer(ok) :- eq(<urn:example:a>, 'urn:example:a').

package/test/conformance/cases/declarations/memoize_is_ordinary_fact.eye

@@ -0,0 +1,4 @@
+% `memoize/2` is no longer a declaration, but it remains an ordinary fact.
+materialize(answer, 2).
+memoize(path, 2).
+answer(?name, ?arity) :- memoize(?name, ?arity).

package/test/conformance/cases/declarations/mode_determinism_are_facts.eye

@@ -0,0 +1,8 @@
+% Advisory declarations are also ordinary facts that programs can inspect.
+mode(path, 2, [in, out]).
+semidet(edge, 2).
+det(root, 1).
+materialize(answer, 2).
+answer(mode, ?modes) :- mode(path, 2, ?modes).
+answer(semidet, edge) :- semidet(edge, 2).
+answer(det, root) :- det(root, 1).

package/test/conformance/cases/declarations/mode_ignored_when_arity_mismatch.eye

@@ -0,0 +1,5 @@
+% Invalid advisory mode length is ignored as metadata but remains a fact.
+materialize(answer, 1).
+mode(edge, 2, [in]).
+edge(a, b).
+answer(ok) :- mode(edge, 2, [in]), edge(a, b).

package/test/conformance/cases/declarations/table_is_also_fact.eye

@@ -0,0 +1,4 @@
+% Declarations remain ordinary facts unless materialized output excludes them.
+materialize(answer, 2).
+table(path, 2).
+answer(?name, ?arity) :- table(?name, ?arity).

package/test/conformance/cases/lists/empty_list_term.eye

@@ -0,0 +1,4 @@
+% The empty list is a first-class term.
+materialize(answer, 1).
+seed([]).
+answer(?x) :- seed(?x).

package/test/conformance/cases/lists/improper_list_readback.eye

@@ -0,0 +1,4 @@
+% Improper lists preserve their tail in read-back.
+materialize(answer, 1).
+seed([a, b | tail]).
+answer(?x) :- seed(?x).

package/test/conformance/cases/lists/list_with_iri_atoms.eye

@@ -0,0 +1,4 @@
+% Lists can contain IRI atoms directly.
+materialize(answer, 1).
+seed([<urn:example:a>, <urn:example:b>]).
+answer(?x) :- seed(?x).

package/test/conformance/cases/materialize/source_facts_suppressed.eye

@@ -0,0 +1,5 @@
+% materialize/2 prints derived answers, not source facts for the same predicate.
+materialize(answer, 1).
+seed(a).
+answer(source).
+answer(?x) :- seed(?x).

package/test/conformance/cases/negation/not_known_goal_fails.eye

@@ -0,0 +1,4 @@
+% Negation fails when its inner goal succeeds.
+materialize(answer, 1).
+seen(a).
+answer(ok) :- not(seen(a)).

package/test/conformance/cases/negation/not_unknown_goal_succeeds.eye

@@ -0,0 +1,3 @@
+% Negation succeeds when its inner goal has no solution.
+materialize(answer, 1).
+answer(ok) :- not(missing(fact)).

package/test/conformance/cases/negation/stratified_negation_answers.eye

@@ -0,0 +1,6 @@
+% Stratified negation is portable and produces ordinary answers.
+materialize(open, 1).
+place(a).
+place(b).
+closed(b).
+open(?x) :- place(?x), not(closed(?x)).

package/test/conformance/cases/syntax/parenthesized_query_style_body.eye

@@ -0,0 +1,5 @@
+% Parentheses may group a body conjunction without changing meaning.
+materialize(answer, 1).
+a(ok).
+b(ok).
+answer(?x) :- (a(?x), b(?x)).

package/test/conformance/cases/table/table_does_not_change_answers.eye

@@ -0,0 +1,7 @@
+% `table/2` is a search-control declaration and does not change answers.
+materialize(path, 2).
+table(path, 2).
+edge(a, b).
+edge(b, c).
+path(?x, ?y) :- edge(?x, ?y).
+path(?x, ?z) :- edge(?x, ?y), path(?y, ?z).

package/test/conformance/cases/terms/compound_name_arguments_atom_zero_args.eye

@@ -0,0 +1,3 @@
+% compound_name_arguments/3 observes atoms as name plus empty argument list.
+materialize(answer, 2).
+answer(?name, ?args) :- compound_name_arguments(nil, ?name, ?args).

package/test/conformance/cases/terms/compound_name_arguments_builds_atom_from_empty_args.eye

@@ -0,0 +1,3 @@
+% Building a term with an empty argument list yields an atom, not nil().
+materialize(answer, 1).
+answer(?term) :- compound_name_arguments(?term, nil, []).

package/test/conformance/cases/terms/functor_atom_arity_zero.eye

@@ -0,0 +1,3 @@
+% Atoms are zero-arity terms for functor/3.
+materialize(answer, 2).
+answer(?name, ?arity) :- functor(nil, ?name, ?arity).

package/test/conformance/cases/variables/anonymous_in_two_goals.eye

@@ -0,0 +1,4 @@
+% Each ?_ occurrence is fresh, so these two goals do not have to agree.
+materialize(answer, 1).
+pair(a, b).
+answer(ok) :- pair(?_, ?_).

package/test/conformance/cases/variables/question_anonymous_not_reused.eye

@@ -0,0 +1,6 @@
+% Each `?_` occurrence is anonymous and independent.
+materialize(answer, 1).
+pair(a, b).
+pair(c, d).
+answer(left(?x)) :- pair(?x, ?_).
+answer(right(?y)) :- pair(?_, ?y).

package/test/conformance/cases/variables/question_underscore_named_reuse.eye

@@ -0,0 +1,5 @@
+% `?_name` is a named variable and must be reused within a clause.
+materialize(answer, 1).
+pair(a, a).
+pair(a, b).
+answer(?_value) :- pair(?_value, ?_value).

package/test/conformance/cases/variables/question_uppercase_variable.eye

@@ -0,0 +1,4 @@
+% The question mark, not letter case, marks variables.
+materialize(answer, 1).
+item(ok).
+answer(?X) :- item(?X).

package/test/conformance/cases/variables/question_variable_digits_and_underscore.eye

@@ -0,0 +1,4 @@
+% Variable names may continue with digits and underscores after the leading name character.
+materialize(answer, 2).
+pair(a, b).
+answer(?x_1, ?y2) :- pair(?x_1, ?y2).

package/test/conformance/cases/variables/variable_scope_per_clause.eye

@@ -0,0 +1,6 @@
+% Reusing a variable name in separate clauses does not connect the clauses.
+materialize(answer, 1).
+left(a).
+right(b).
+answer(?x) :- left(?x).
+answer(?x) :- right(?x).

package/test/conformance/errors/atoms/relative_angle_name_rejected.eye

@@ -0,0 +1 @@
+answer(<relative>).

package/test/conformance/errors/atoms/space_in_angle_iri_rejected.eye

@@ -0,0 +1 @@
+answer(<urn:example:a b>).

package/test/conformance/errors/atoms/unclosed_angle_iri_rejected.eye

@@ -0,0 +1 @@
+answer(<urn:example:open).

package/test/conformance/errors/atoms/zero_arity_compound_rejected.eye

@@ -0,0 +1 @@
+p(nil()).

package/test/conformance/errors/syntax/colon_name_rejected.eye

@@ -0,0 +1 @@
+ex:alice.

package/test/conformance/errors/syntax/missing_final_dot_rejected.eye

@@ -0,0 +1 @@
+answer(ok)

package/test/conformance/errors/syntax/unclosed_list_rejected.eye

@@ -0,0 +1 @@
+answer([a, b).

package/test/conformance/errors/syntax/unclosed_quoted_atom_rejected.eye

@@ -0,0 +1 @@
+p('unterminated).

package/test/conformance/errors/syntax/uppercase_predicate_rejected.eye

@@ -0,0 +1 @@
+Answer(ok).

package/test/conformance/errors/terms/zero_arity_compound_nested_rejected.eye

@@ -0,0 +1 @@
+answer(f(nil())).

package/test/conformance/errors/variables/bare_underscore_rejected.eye

@@ -0,0 +1 @@
+p(_).

package/test/conformance/errors/variables/question_digit_rejected.eye

@@ -0,0 +1 @@
+answer(?1).

package/test/conformance/errors/variables/question_dot_rejected.eye

@@ -0,0 +1 @@
+answer(?.).

package/test/conformance/errors/variables/underscore_named_rejected.eye

@@ -0,0 +1 @@
+p(_name).

package/test/conformance/errors/variables/uppercase_variable_rejected.eye

@@ -0,0 +1 @@
+p(X).

package/test/conformance/expected/atoms/graphic_angle_atom_stays_graphic.eye

@@ -0,0 +1 @@
+answer(<=>).

package/test/conformance/expected/atoms/graphic_less_than_atom.eye

@@ -0,0 +1 @@
+answer(<).

package/test/conformance/expected/atoms/iri_atoms_in_lists.eye

@@ -0,0 +1 @@
+answer([<urn:example:a>, <urn:example:b>, <urn:example:c>]).

package/test/conformance/expected/atoms/iri_mailto_readback.eye

@@ -0,0 +1 @@
+answer(<mailto:alice@example.org>).

package/test/conformance/expected/atoms/iri_query_fragment_readback.eye

@@ -0,0 +1 @@
+answer(<https://example.org/path?x=1#frag>).

package/test/conformance/expected/atoms/iri_quoted_absolute_readback.eye

@@ -0,0 +1,2 @@
+answer(<https://example.org/alice>).
+answer(<urn:example:bob>).

package/test/conformance/expected/atoms/iri_quoted_and_angle_unify.eye

@@ -0,0 +1 @@
+answer(ok).

package/test/conformance/expected/atoms/iri_urn_readback.eye

@@ -0,0 +1 @@
+answer(<urn:example:alpha>).

package/test/conformance/expected/atoms/quoted_urn_iri_readback.eye

@@ -0,0 +1 @@
+answer(<urn:example:quoted>).

package/test/conformance/expected/atoms/zero_arity_atom_roundtrip.eye

@@ -0,0 +1,3 @@
+answer(name, nil).
+answer(args, []).
+answer(term, nil).

package/test/conformance/expected/builtins/atom_string_iri_atom.eye

@@ -0,0 +1 @@
+answer("urn:example:a").

package/test/conformance/expected/builtins/eq_unifies_iri_atoms.eye

@@ -0,0 +1 @@
+answer(ok).

package/test/conformance/expected/declarations/memoize_is_ordinary_fact.eye

@@ -0,0 +1 @@
+answer(path, 2).

package/test/conformance/expected/declarations/mode_determinism_are_facts.eye

@@ -0,0 +1,3 @@
+answer(mode, [in, out]).
+answer(semidet, edge).
+answer(det, root).

package/test/conformance/expected/declarations/mode_ignored_when_arity_mismatch.eye

@@ -0,0 +1 @@
+answer(ok).

package/test/conformance/expected/declarations/table_is_also_fact.eye

@@ -0,0 +1 @@
+answer(path, 2).

package/test/conformance/expected/lists/empty_list_term.eye

@@ -0,0 +1 @@
+answer([]).

package/test/conformance/expected/lists/improper_list_readback.eye

@@ -0,0 +1 @@
+answer([a, b | tail]).

package/test/conformance/expected/lists/list_with_iri_atoms.eye

@@ -0,0 +1 @@
+answer([<urn:example:a>, <urn:example:b>]).

package/test/conformance/expected/materialize/source_facts_suppressed.eye

@@ -0,0 +1 @@
+answer(a).

package/test/conformance/expected/negation/not_unknown_goal_succeeds.eye

@@ -0,0 +1 @@
+answer(ok).

package/test/conformance/expected/negation/stratified_negation_answers.eye

@@ -0,0 +1 @@
+open(a).

package/test/conformance/expected/syntax/parenthesized_query_style_body.eye

@@ -0,0 +1 @@
+answer(ok).

package/test/conformance/expected/table/table_does_not_change_answers.eye

@@ -0,0 +1,3 @@
+path(a, b).
+path(b, c).
+path(a, c).

package/test/conformance/expected/terms/compound_name_arguments_atom_zero_args.eye

@@ -0,0 +1 @@
+answer(nil, []).

package/test/conformance/expected/terms/compound_name_arguments_builds_atom_from_empty_args.eye

@@ -0,0 +1 @@
+answer(nil).

package/test/conformance/expected/terms/functor_atom_arity_zero.eye

@@ -0,0 +1 @@
+answer(nil, 0).

package/test/conformance/expected/variables/anonymous_in_two_goals.eye

@@ -0,0 +1 @@
+answer(ok).

package/test/conformance/expected/variables/question_anonymous_not_reused.eye

@@ -0,0 +1,4 @@
+answer(left(a)).
+answer(left(c)).
+answer(right(b)).
+answer(right(d)).

package/test/conformance/expected/variables/question_underscore_named_reuse.eye

@@ -0,0 +1 @@
+answer(a).

package/test/conformance/expected/variables/question_uppercase_variable.eye

@@ -0,0 +1 @@
+answer(ok).

package/test/conformance/expected/variables/question_variable_digits_and_underscore.eye

@@ -0,0 +1 @@
+answer(a, b).

package/test/conformance/expected/variables/variable_scope_per_clause.eye

@@ -0,0 +1,2 @@
+answer(a).
+answer(b).

package/test/conformance/expected-errors/atoms/relative_angle_name_rejected.txt

@@ -0,0 +1 @@
+parse line 1: expected ), got relative

package/test/conformance/expected-errors/atoms/space_in_angle_iri_rejected.txt

@@ -0,0 +1 @@
+parse line 1: expected ), got urn

package/test/conformance/expected-errors/atoms/unclosed_angle_iri_rejected.txt

@@ -0,0 +1 @@
+parse line 1: expected ), got urn

package/test/conformance/expected-errors/atoms/zero_arity_compound_rejected.txt

@@ -0,0 +1 @@
+parse line 1: zero-arity compound syntax is not supported; use atom "nil" for arity zero data

package/test/conformance/expected-errors/syntax/colon_name_rejected.txt

@@ -0,0 +1 @@
+colon names are not supported; use name or prefix_name

package/test/conformance/expected-errors/syntax/missing_final_dot_rejected.txt

@@ -0,0 +1 @@
+parse line 2: expected ., got 

package/test/conformance/expected-errors/syntax/unclosed_list_rejected.txt

@@ -0,0 +1 @@
+parse line 1: expected ], got )

package/test/conformance/expected-errors/syntax/unclosed_quoted_atom_rejected.txt

@@ -0,0 +1 @@
+parse line 1: unterminated quoted term

package/test/conformance/expected-errors/syntax/uppercase_predicate_rejected.txt

@@ -0,0 +1 @@
+parse line 1: bad character "A"

package/test/conformance/expected-errors/terms/zero_arity_compound_nested_rejected.txt

@@ -0,0 +1 @@
+parse line 1: zero-arity compound syntax is not supported; use atom "nil" for arity zero data

package/test/conformance/expected-errors/variables/bare_underscore_rejected.txt

@@ -0,0 +1 @@
+parse line 1: bad character "_"

package/test/conformance/expected-errors/variables/question_digit_rejected.txt

@@ -0,0 +1 @@
+parse line 1: expected ), got 1

package/test/conformance/expected-errors/variables/question_dot_rejected.txt

@@ -0,0 +1 @@
+parse line 1: expected ), got .

package/test/conformance/expected-errors/variables/underscore_named_rejected.txt

@@ -0,0 +1 @@
+parse line 1: bad character "_"

package/test/conformance/expected-errors/variables/uppercase_variable_rejected.txt

@@ -0,0 +1 @@
+parse line 1: bad character "X"

package/test/conformance/expected-warnings/negation/negative_unknown_group_quiet.eye

@@ -0,0 +1 @@
+answer(ok).

package/test/conformance/expected-warnings/negation/no_negation_quiet.eye

@@ -0,0 +1 @@
+answer(ok).

package/test/conformance/expected-warnings/negation/stratified_quiet.eye

@@ -0,0 +1 @@
+answer(ok).

package/test/conformance/expected-warnings/negation/unstratified_mutual.txt

@@ -0,0 +1,3 @@
+eyelang warning: unstratified negation
+  p/1 depends negatively on q/1
+  q/1 depends negatively on p/1

package/test/conformance/expected-warnings/negation/unstratified_self.txt

@@ -0,0 +1,2 @@
+eyelang warning: unstratified negation
+  p/1 depends negatively on p/1

package/test/conformance/expected-warnings/negation/unstratified_three_step.eye

@@ -0,0 +1 @@
+answer(ok).

package/test/conformance/expected-warnings/negation/unstratified_three_step.txt

@@ -0,0 +1,2 @@
+eyelang warning: unstratified negation
+  r/1 depends negatively on p/1

package/test/conformance/warnings/negation/negative_unknown_group_quiet.eye

@@ -0,0 +1,2 @@
+materialize(answer, 1).
+answer(ok) :- not(missing(ok)).

package/test/conformance/warnings/negation/no_negation_quiet.eye

@@ -0,0 +1,3 @@
+materialize(answer, 1).
+seed(ok).
+answer(?x) :- seed(?x).

package/test/conformance/warnings/negation/stratified_quiet.eye

@@ -0,0 +1,4 @@
+% Stratified negation emits no portability warning.
+materialize(answer, 1).
+candidate(a).
+answer(ok) :- candidate(a), not(blocked(a)).

package/test/conformance/warnings/negation/unstratified_mutual.eye

@@ -0,0 +1,5 @@
+% Warnings report unstratified negation without changing normal execution.
+materialize(answer, 1).
+p(a) :- not(q(a)).
+q(a) :- not(p(a)).
+answer(ok).

package/test/conformance/warnings/negation/unstratified_self.eye

@@ -0,0 +1,4 @@
+% A direct negative self-dependency is reported as unstratified.
+materialize(answer, 1).
+p(a) :- not(p(a)).
+answer(ok).

package/test/conformance/warnings/negation/unstratified_three_step.eye

@@ -0,0 +1,6 @@
+materialize(answer, 1).
+p(a) :- q(a).
+q(a) :- r(a).
+r(a) :- not(p(a)).
+seed(ok).
+answer(?x) :- seed(?x).

package/test/run-conformance-report.mjs

@@ -0,0 +1,108 @@
+#!/usr/bin/env node
+// Static conformance corpus report.
+// This complements the executable runner with a category summary that makes
+// coverage growth visible without running every case.
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)));
+const conformanceRoot = path.join(root, 'conformance');
+
+const KINDS = [
+  { kind: 'cases', expectedKind: 'expected', expectedExt: '.eye', column: 'positive' },
+  { kind: 'errors', expectedKind: 'expected-errors', expectedExt: '.txt', column: 'errors' },
+  { kind: 'warnings', expectedKind: 'expected-warnings', expectedExt: '.eye', column: 'warnings' },
+];
+
+export function buildConformanceReport() {
+  const categories = new Map();
+  const issues = [];
+
+  for (const { kind, expectedKind, expectedExt, column } of KINDS) {
+    const base = path.join(conformanceRoot, kind);
+    if (!fs.existsSync(base)) continue;
+    for (const file of listEyeFiles(base)) {
+      const category = categoryOf(file);
+      const counts = ensureCategory(categories, category);
+      counts[column]++;
+      counts.total++;
+
+      const stem = file.slice(0, -4);
+      const expected = path.join(conformanceRoot, expectedKind, `${stem}${expectedExt}`);
+      if (!fs.existsSync(expected)) issues.push(`missing ${expectedKind}/${stem}${expectedExt}`);
+      if (kind === 'warnings') {
+        const expectedStderr = path.join(conformanceRoot, expectedKind, `${stem}.txt`);
+        if (!fs.existsSync(expectedStderr)) issues.push(`missing ${expectedKind}/${stem}.txt`);
+      }
+    }
+  }
+
+  const rows = [...categories.entries()]
+    .sort(([a], [b]) => a.localeCompare(b))
+    .map(([category, counts]) => ({ category, ...counts }));
+  const total = rows.reduce((acc, row) => ({
+    positive: acc.positive + row.positive,
+    errors: acc.errors + row.errors,
+    warnings: acc.warnings + row.warnings,
+    total: acc.total + row.total,
+  }), { positive: 0, errors: 0, warnings: 0, total: 0 });
+
+  return { rows, total, issues: issues.sort() };
+}
+
+export function formatConformanceReport(report = buildConformanceReport()) {
+  const lines = [
+    '# Conformance Eyelang report',
+    '',
+    'This report summarizes the file-based conformance corpus under `test/conformance/`.',
+    '',
+    '| Category | Positive | Errors | Warnings | Total |',
+    '|---|---:|---:|---:|---:|',
+  ];
+
+  for (const row of report.rows) {
+    lines.push(`| ${row.category} | ${row.positive} | ${row.errors} | ${row.warnings} | ${row.total} |`);
+  }
+  lines.push(`| **Total** | **${report.total.positive}** | **${report.total.errors}** | **${report.total.warnings}** | **${report.total.total}** |`);
+
+  if (report.issues.length > 0) {
+    lines.push('', '## Corpus issues', '');
+    for (const issue of report.issues) lines.push(`- ${issue}`);
+  }
+
+  return `${lines.join('\n')}\n`;
+}
+
+function listEyeFiles(base, dir = base) {
+  const files = [];
+  for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
+    const full = path.join(dir, entry.name);
+    if (entry.isDirectory()) {
+      files.push(...listEyeFiles(base, full));
+    } else if (entry.isFile() && entry.name.endsWith('.eye')) {
+      files.push(path.relative(base, full).split(path.sep).join('/'));
+    }
+  }
+  return files.sort();
+}
+
+function categoryOf(file) {
+  const parts = file.split('/');
+  return parts.length > 1 ? parts[0] : 'legacy-numbered';
+}
+
+function ensureCategory(categories, category) {
+  let counts = categories.get(category);
+  if (!counts) {
+    counts = { positive: 0, errors: 0, warnings: 0, total: 0 };
+    categories.set(category, counts);
+  }
+  return counts;
+}
+
+if (process.argv[1] != null && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
+  const report = buildConformanceReport();
+  process.stdout.write(formatConformanceReport(report));
+  if (report.issues.length > 0) process.exit(1);
+}

package/test/run-conformance.mjs

@@ -1,6 +1,6 @@
 #!/usr/bin/env node
 // Conformance test runner.
-// It executes cases in-process so the conformance corpus measures engine behavior instead of Node process startup.
+// It executes normal cases in-process so the conformance corpus measures engine behavior instead of Node process startup.
 import fs from 'node:fs';
 import path from 'node:path';
 import { spawnSync } from 'node:child_process';
@@ -9,29 +9,67 @@ import { fileURLToPath } from 'node:url';
 import { TestReporter, isMainModule } from './test-style.mjs';
 
 const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)));
+const packageRoot = path.resolve(root, '..');
+const cliBin = path.join(packageRoot, 'src', 'bin.js');
 const filterArg = process.argv[2] ?? null;
 
 export function runConformance(reporter = new TestReporter(), requestedFilter = null) {
   const filter = requestedFilter ?? filterArg;
   const label = filter == null ? 'eyelang' : `eyelang ${filter}`;
   reporter.section(`Conformance ${label}`);
-  for (const file of listCaseFiles(filter)) runCaseFile(reporter, file);
+  for (const file of listCaseFiles('cases', filter)) runCaseFile(reporter, file);
+  for (const file of listCaseFiles('errors', filter)) runErrorFile(reporter, file);
+  for (const file of listCaseFiles('warnings', filter)) runWarningFile(reporter, file);
   reporter.sectionTotal(`conformance ${label}`);
 }
 
-function listCaseFiles(filter = null) {
-  const casesDir = path.join(root, 'conformance', 'cases');
-  return fs.readdirSync(casesDir)
-    .filter((name) => name.endsWith('.eye'))
-    .filter((name) => filter == null || name.includes(filter) || name.slice(0, -4) === filter)
+function listCaseFiles(kind, filter = null) {
+  const base = path.join(root, 'conformance', kind);
+  if (!fs.existsSync(base)) return [];
+  return listEyeFiles(base)
+    .filter((name) => matchesFilter(kind, name, filter))
     .sort();
 }
 
+function matchesFilter(kind, name, filter) {
+  if (filter == null) return true;
+  const stem = name.slice(0, -4);
+  const label = kind === 'errors' ? 'error' : kind === 'warnings' ? 'warning' : kind;
+  return name.includes(filter)
+    || stem === filter
+    || stem.includes(filter)
+    || `${kind}/${stem}`.includes(filter)
+    || `${label}/${stem}`.includes(filter);
+}
+
+function listEyeFiles(base, dir = base) {
+  const files = [];
+  for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
+    const full = path.join(dir, entry.name);
+    if (entry.isDirectory()) {
+      files.push(...listEyeFiles(base, full));
+    } else if (entry.isFile() && entry.name.endsWith('.eye')) {
+      files.push(path.relative(base, full).split(path.sep).join('/'));
+    }
+  }
+  return files;
+}
+
 function runCaseFile(reporter, file) {
   const name = file.slice(0, -4);
   reporter.test(name, () => runCase(name, file));
 }
 
+function runErrorFile(reporter, file) {
+  const name = file.slice(0, -4);
+  reporter.test(`error/${name}`, () => runErrorCase(name, file));
+}
+
+function runWarningFile(reporter, file) {
+  const name = file.slice(0, -4);
+  reporter.test(`warning/${name}`, () => runWarningCase(name, file));
+}
+
 function runCase(name, file) {
   const casesDir = path.join(root, 'conformance', 'cases');
   const expectedDir = path.join(root, 'conformance', 'expected');
@@ -41,13 +79,57 @@ function runCase(name, file) {
   const program = Program.parseSources([{ text, filename: file }], { sourceMetadata: false, markRecursive: false });
   const actual = run(program).stdout;
 
+  compareExpectedFile(expected, actual, name, 'output');
+}
+
+function runErrorCase(name, file) {
+  const casesDir = path.join(root, 'conformance', 'errors');
+  const expectedDir = path.join(root, 'conformance', 'expected-errors');
+  const programFile = path.join(casesDir, file);
+  const expected = path.join(expectedDir, `${name}.txt`);
+  const text = fs.readFileSync(programFile, 'utf8');
+  let actual = null;
+
+  try {
+    const program = Program.parseSources([{ text, filename: file }], { sourceMetadata: false, markRecursive: false });
+    run(program);
+  } catch (error) {
+    actual = `${error?.message ?? String(error)}\n`;
+  }
+
+  if (actual == null) throw new Error(`expected error for ${name}, but program succeeded`);
+  compareExpectedFile(expected, actual, name, 'error');
+}
+
+function runWarningCase(name, file) {
+  const warningsDir = path.join(root, 'conformance', 'warnings');
+  const expectedDir = path.join(root, 'conformance', 'expected-warnings');
+  const programFile = path.join(warningsDir, file);
+  const expectedStdout = path.join(expectedDir, `${name}.eye`);
+  const expectedStderr = path.join(expectedDir, `${name}.txt`);
+  const text = fs.readFileSync(programFile, 'utf8');
+  const result = spawnSync(process.execPath, [cliBin, '--warnings', '-'], {
+    cwd: packageRoot,
+    input: text,
+    encoding: 'utf8',
+  });
+
+  if (result.status !== 0) {
+    throw new Error(`warning case ${name} exited with ${result.status}\n${result.stderr}`.trimEnd());
+  }
+
+  compareExpectedFile(expectedStdout, result.stdout, name, 'warning stdout');
+  compareExpectedFile(expectedStderr, result.stderr, name, 'warning stderr');
+}
+
+function compareExpectedFile(expected, actual, name, kind) {
   if (!fs.existsSync(expected)) {
-    throw new Error(`missing expected file: ${path.relative(root, expected)}`);
+    throw new Error(`missing expected ${kind} file: ${path.relative(root, expected)}`);
   }
 
   const expectedText = fs.readFileSync(expected, 'utf8');
   if (expectedText !== actual) {
-    throw new Error(`output mismatch for ${name}\n${diffText(expected, actual)}`.trimEnd());
+    throw new Error(`${kind} mismatch for ${name}\n${diffText(expected, actual)}`.trimEnd());
   }
 }
 

package/test/run-regression.mjs

@@ -35,6 +35,7 @@ import {
 import { parseGoalText } from '../src/parser.js';
 import { selectClauseCandidates } from '../src/program.js';
 import { TestReporter, isMainModule } from './test-style.mjs';
+import { buildConformanceReport, formatConformanceReport } from './run-conformance-report.mjs';
 import { hashHex } from '../src/hash.js';
 
 const testRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)));
@@ -365,6 +366,18 @@ function documentationSyncCases() {
       name: 'documented npm scripts exist in package.json',
       run: () => assertArrayEqual(missingDocumentedPackageScripts(), [], 'missing documented npm scripts'),
     },
+    {
+      name: 'conformance report summarizes public corpus',
+      run: () => {
+        const report = buildConformanceReport();
+        assertArrayEqual(report.issues, [], 'conformance report issues');
+        assertEqual(report.total.total >= 150, true, 'conformance case count');
+        assertEqual(report.total.positive + report.total.errors + report.total.warnings, report.total.total, 'conformance total');
+        const text = formatConformanceReport(report);
+        assertIncludes(text, '| variables |', 'report');
+        assertIncludes(text, '| **Total** |', 'report');
+      },
+    },
     {
       name: 'source-checkout setup docs match package bin',
       run: () => {