lograder 0.0.2__tar.gz

lograder-0.0.2/LICENSE

@@ -0,0 +1,7 @@
+Copyright 2025 Logan Dapp
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

lograder-0.0.2/PKG-INFO

@@ -0,0 +1,342 @@
+Metadata-Version: 2.4
+Name: lograder
+Version: 0.0.2
+Summary: An API for easy Gradescope Autograder assignment creation.
+Author-email: lognd <logan@logand.app>
+License: Copyright 2025 Logan Dapp
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+Project-URL: Homepage, https://github.com/lognd/lograder
+Project-URL: Documentation, https://github.com/lognd/lograder
+Project-URL: Source, https://github.com/lognd/lograder
+Project-URL: Issues, https://github.com/lognd/lograder/issues
+Keywords: gradescope,autograder
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pydantic>=2.11.7
+Requires-Dist: colorama>=0.4.6
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: black>=24.0; extra == "dev"
+Requires-Dist: ruff>=0.2; extra == "dev"
+Requires-Dist: isort>=5.13; extra == "dev"
+Requires-Dist: mypy>=1.0; extra == "dev"
+Requires-Dist: pytest>=8.4.1; extra == "dev"
+Requires-Dist: types-colorama>=0.4.15.20250801; extra == "dev"
+Dynamic: license-file
+
+# `lograder`: A Gradescope Autograder API
+
+----
+This project just serves to standard different kinds of tests
+that can be run on student code for the Gradescope autograder.
+Additionally, this project was developed for the **University
+of Florida's Fall 2025 COP3504C** (*Advanced Programming 
+Fundamentals*), taught by Michael Link. However, you are
+completely free to use, remix, refactor, and abuse this code
+as much as you like.
+
+----
+# Project Builders
+
+----
+
+### C++ Complete Project with [I/O Comparison](#output-comparison)
+
+#### Build from C++ Source (*WIP*)
+To build from source, you will need to import the C++
+`CxxSourceBuilder`. The executable will be randomly
+named and put in either a build directory, if the student
+has one (`./build`) or the project root directory (`./`).
+
+```py
+from lograder.dispatch import CxxSourceDispatcher
+from lograder.output import AssignmentSummary
+
+# Note that when you make a test, it's automatically
+# registered with the `lograder.tests.registry.TestRegistry`
+
+assignment = CxxSourceDispatcher(project_root="/autograder/submission")
+preprocessor_results = assignment.preprocess()
+build_results = assignment.build()
+runtime_results = assignment.run_tests()
+
+summary = AssignmentSummary(
+    preprocessor_output=preprocessor_results.get_output(),
+    build_output=build_results.get_output(),
+    runtime_summary=runtime_results.get_summary(),
+    test_cases=runtime_results.get_test_cases()
+)
+```
+
+#### Build using CMake (*WIP*)
+To build from a `CMakeLists.txt`, you will need to import the C++
+`CMakeBuilder`. This method will automatically run a breadth-first
+search starting in the project root directory (`./`) and "lock on"
+the first (i.e. the file in the highest-level) `CMakeLists.txt` that
+it finds. If it can't find a `CMakeLists.txt`, it will raise an error.
+
+Additionally, the program will look for the following targets first:
+`main`, `build`, and `demo`. Afterward, it will search for any target
+that doesn't match: `all`, `install`, `test`, `package`, `package_source`,
+`edit_cache`, `rebuild_cache`, `clean`, `help`, `ALL_BUILD`, `ZERO_CHECK`,
+`INSTALL`, `RUN_TESTS`, and `PACKAGE`, and run the first target that it
+finds. If it can't find a valid target, it will raise an error.
+
+```py
+from lograder.dispatch import CMakeDispatcher
+from lograder.output import AssignmentSummary
+
+# Note that when you make a test, it's automatically
+# registered with the `lograder.tests.registry.TestRegistry`
+
+assignment = CMakeDispatcher(project_root="/autograder/submission")
+preprocessor_results = assignment.preprocess()
+build_results = assignment.build()
+runtime_results = assignment.run_tests()
+
+summary = AssignmentSummary(
+    preprocessor_output=preprocessor_results.get_output(),
+    build_output=build_results.get_output(),
+    runtime_summary=runtime_results.get_summary(),
+    test_cases=runtime_results.get_test_cases()
+)
+```
+----
+
+### C++ Catch2 Unit Testing (*WIP*)
+
+----
+
+### Python Complete Project with [I/O Comparison](#output-comparison)
+
+#### Run project from `main.py` (*WIP*)
+
+#### Run project from `pyproject.toml` (*WIP*)
+
+----
+
+### Python pytest Unit Testing (*WIP*)
+
+----
+
+### Makefile Complete Project with [I/O Comparison](#output-comparison) (*WIP*)
+
+To build from a `Makefile`, you will need a `MakefileBuilder`. It follows
+the same general idea as the `CMakeBuilder` except that it searches for
+`Makefile` instead of `CMakeLists.txt`. Additionally, `MakefileBuilder`
+will just run the default `make`.
+
+```py
+from lograder.dispatch import MakefileDispatcher
+from lograder.output import AssignmentSummary
+
+# Note that when you make a test, it's automatically
+# registered with the `lograder.tests.registry.TestRegistry`
+
+assignment = MakefileDispatcher(project_root="/autograder/submission")
+preprocessor_results = assignment.preprocess()
+build_results = assignment.build()
+runtime_results = assignment.run_tests()
+
+summary = AssignmentSummary(
+    preprocessor_output=preprocessor_results.get_output(),
+    build_output=build_results.get_output(),
+    runtime_summary=runtime_results.get_summary(),
+    test_cases=runtime_results.get_test_cases()
+)
+```
+
+----
+# Test Generation
+
+----
+
+## Output Comparison
+
+### Compare Simple Strings
+
+For the smallest number of tiny test cases, there's no reason
+to have an over-bloated mess. You can just use:
+
+```py
+from typing import Sequence, Optional, List
+from pathlib import Path
+from lograder.tests import make_tests_from_strs, ExecutableOutputComparisonTest
+
+
+def make_test_from_strs(
+        *,  # kwargs-only; to avoid confusion with argument sequence.
+        names: Sequence[str],
+        inputs: Sequence[str],
+        expected_outputs: Sequence[str],
+        flag_sets: Optional[Sequence[List[str | Path]]] = None,
+        # Pass flags like ["--option-1", "--option-2"] to student programs
+        weights: Optional[Sequence[float]] = None,  # Defaults to equal-weight.
+) -> List[ExecutableOutputComparisonTest]: ...
+
+
+# Here's an example of how you'd use the above method:
+make_tests_from_strs(
+    names=["Test Case 1", "Test Case 2"],
+    inputs=["stdin-1", "stdin-2"],
+    expected_outputs=["stdout-1", "stdout-2"]
+)
+```
+
+### Compare from Files
+
+If you have a larger test, it would be very convenient to
+read files for input and output. Luckily, there's just the
+method to do so:
+
+```py
+from typing import Sequence, Optional, List
+from pathlib import Path
+from lograder.tests import make_tests_from_files, FilePath, ExecutableOutputComparisonTest
+
+
+# `make_tests_from_files` has the following signature.
+def make_tests_from_files(
+        *,  # kwargs-only; to avoid confusion with argument sequence.
+        names: Sequence[str],
+        input_files: Optional[Sequence[FilePath]] = None,  # `input_files` and `input_strs` mutually exclusive.
+        input_strs: Optional[Sequence[str]] = None,
+        expected_output_files: Optional[Sequence[FilePath]] = None,
+        # same with `expected_output_files` and `expected_output_strs`
+        expected_output_strs: Optional[Sequence[str]] = None,
+        flag_sets: Optional[Sequence[List[str | Path]]] = None,
+        # Pass flags like ["--option-1", "--option-2"] to student programs
+        weights: Optional[Sequence[float]] = None,  # Defaults to equal-weight.
+) -> List[ExecutableOutputComparisonTest]: ...
+
+
+# Here's an example of how you'd use the above method:
+make_tests_from_files(
+    names=["Test Case 1", "Test Case 2"],
+    input_files=["test/inputs/input1.txt", "test/inputs/input2.txt"],
+    expected_output_files=["test/inputs/output1.txt", "test/inputs/output2.txt"]
+)
+```
+
+### Compare from Template
+
+Finally, sometimes the test-cases might be very long but 
+very repetitive. You can use `make_tests_from_template` 
+and pass a `TestCaseTemplate` object and ...
+
+```py
+from typing import Sequence, Optional, List
+from pathlib import Path
+from lograder.tests import make_tests_from_template, TestCaseTemplate, FilePath
+
+
+# Here's the signature of a `TemplateSubstitution`
+class TemplateSubstitution:
+    def __init__(self, *args, **kwargs):
+        # Stores args and kwargs to pass to str.format(...) later.
+        ...
+
+
+TSub = TemplateSubstitution  # Here's an alias that's quicker to type.
+
+
+# Here's the signature of a `TestCaseTemplate`
+class TestCaseTemplate:
+    def __init__(self, *,
+                 inputs: Optional[Sequence[str]] = None,
+                 input_template_file: Optional[FilePath] = None,
+                 input_template_str: Optional[str] = None,
+                 input_substitutions: Optional[Sequence[TemplateSubstitution]] = None,
+                 expected_outputs: Optional[Sequence[str]] = None,
+                 expected_output_template_file: Optional[FilePath] = None,
+                 expected_output_template_str: Optional[str] = None,
+                 expected_output_substitutions: Optional[Sequence[TemplateSubstitution]] = None,
+                 flag_sets: Optional[Sequence[List[str | Path]]] = None,  # Pass flags like ["--option-1", "--option-2"] to student programs
+                 ):
+        # +=====================================================================================+
+        # | Validation Rules                                                                    |
+        # +=====================================================================================+
+        #   * If `inputs` is specified, all other `input_*` parameters must be left unspecified.
+        #   * Same thing with `expected_outputs`.
+        #   * If `inputs` is not specified, you must specify either (mutually exclusive) 
+        #     `input_template_file` or `input_template_str` that follows a typical python
+        #     format string, and you must specify `input_substitutions`.
+        #   * Same thing with `expected_output_template_file`, `expected_output_template_str`, 
+        #     and `expected_output_substitutions`
+        ...
+
+
+# Here's an example of how you would use TestCaseTemplate
+test_suite_1 = TestCaseTemplate(
+    inputs=["A", "B", "C"],  # Three (3) Total Cases
+    expected_output_template_str="{}, {kwarged}, {}",
+    expected_output_substitutions=[
+        TSub(1.0, 2.0, kwarged="middle-arg-1"),  # Case 1 Substitutions
+        TSub(2.0, 5.0, kwarged="middle-arg-2"),  # Case 2 Substitutions
+        TSub(7.0, 6.0, kwarged="middle-arg-3"),  # Case 3 Substitutions
+    ]
+)
+make_tests_from_template(
+    ["Test 1", "Test 2", "Test 3"],
+    test_suite_1
+)  # remember to construct the tests!
+
+```
+
+### Compare from Python Generator/Iterable
+
+Sometimes, you want to generate a ton of test-cases (especially
+small test-cases), and it would be incredibly waste to have thousands
+of single-line files. You can create a python generator function that
+follows either the following `Protocol` or `TypedDict`.
+
+```py
+from typing import Protocol, TypedDict, Generator, NotRequired, List
+from pathlib import Path
+from lograder.tests import make_tests_from_generator
+
+
+# Your generator may return objects following the protocol...
+class TestCaseProtocol(Protocol):
+    def get_name(self): ...
+
+    def get_input(self): ...
+
+    def get_expected_output(self): ...
+
+class FlaggedTestCaseProtocol(TestCaseProtocol, Protocol):
+    def get_flags(self) -> List[str | Path]: ...
+    
+# Notice that TestCaseProtocol defaults to equal-weights
+class WeightedTestCaseProtocol(TestCaseProtocol, Protocol):
+    def get_weight(self): ...
+
+# ... or you can directly return a dict with the following keys.
+class TestCaseDict(TypedDict):
+    name: str
+    input: str
+    expected_output: str
+    weight: NotRequired[float]  # Defaults to 1.0, a.k.a. equal-weight.
+    flags: NotRequired[List[str | Path]]
+
+
+# Here's an example of the syntax as well as the required 
+# signature of such a method:
+@make_tests_from_generator
+def test_suite_1() -> Generator[TestCaseProtocol | WeightedTestCaseProtocol | TestCaseDict, None, None]:
+    pass
+
+# You'll have to query the `TestRegistry` from `lograder.tests` to access these tests directly, though.
+```
+
+
+

lograder-0.0.2/README.md

@@ -0,0 +1,307 @@
+# `lograder`: A Gradescope Autograder API
+
+----
+This project just serves to standard different kinds of tests
+that can be run on student code for the Gradescope autograder.
+Additionally, this project was developed for the **University
+of Florida's Fall 2025 COP3504C** (*Advanced Programming 
+Fundamentals*), taught by Michael Link. However, you are
+completely free to use, remix, refactor, and abuse this code
+as much as you like.
+
+----
+# Project Builders
+
+----
+
+### C++ Complete Project with [I/O Comparison](#output-comparison)
+
+#### Build from C++ Source (*WIP*)
+To build from source, you will need to import the C++
+`CxxSourceBuilder`. The executable will be randomly
+named and put in either a build directory, if the student
+has one (`./build`) or the project root directory (`./`).
+
+```py
+from lograder.dispatch import CxxSourceDispatcher
+from lograder.output import AssignmentSummary
+
+# Note that when you make a test, it's automatically
+# registered with the `lograder.tests.registry.TestRegistry`
+
+assignment = CxxSourceDispatcher(project_root="/autograder/submission")
+preprocessor_results = assignment.preprocess()
+build_results = assignment.build()
+runtime_results = assignment.run_tests()
+
+summary = AssignmentSummary(
+    preprocessor_output=preprocessor_results.get_output(),
+    build_output=build_results.get_output(),
+    runtime_summary=runtime_results.get_summary(),
+    test_cases=runtime_results.get_test_cases()
+)
+```
+
+#### Build using CMake (*WIP*)
+To build from a `CMakeLists.txt`, you will need to import the C++
+`CMakeBuilder`. This method will automatically run a breadth-first
+search starting in the project root directory (`./`) and "lock on"
+the first (i.e. the file in the highest-level) `CMakeLists.txt` that
+it finds. If it can't find a `CMakeLists.txt`, it will raise an error.
+
+Additionally, the program will look for the following targets first:
+`main`, `build`, and `demo`. Afterward, it will search for any target
+that doesn't match: `all`, `install`, `test`, `package`, `package_source`,
+`edit_cache`, `rebuild_cache`, `clean`, `help`, `ALL_BUILD`, `ZERO_CHECK`,
+`INSTALL`, `RUN_TESTS`, and `PACKAGE`, and run the first target that it
+finds. If it can't find a valid target, it will raise an error.
+
+```py
+from lograder.dispatch import CMakeDispatcher
+from lograder.output import AssignmentSummary
+
+# Note that when you make a test, it's automatically
+# registered with the `lograder.tests.registry.TestRegistry`
+
+assignment = CMakeDispatcher(project_root="/autograder/submission")
+preprocessor_results = assignment.preprocess()
+build_results = assignment.build()
+runtime_results = assignment.run_tests()
+
+summary = AssignmentSummary(
+    preprocessor_output=preprocessor_results.get_output(),
+    build_output=build_results.get_output(),
+    runtime_summary=runtime_results.get_summary(),
+    test_cases=runtime_results.get_test_cases()
+)
+```
+----
+
+### C++ Catch2 Unit Testing (*WIP*)
+
+----
+
+### Python Complete Project with [I/O Comparison](#output-comparison)
+
+#### Run project from `main.py` (*WIP*)
+
+#### Run project from `pyproject.toml` (*WIP*)
+
+----
+
+### Python pytest Unit Testing (*WIP*)
+
+----
+
+### Makefile Complete Project with [I/O Comparison](#output-comparison) (*WIP*)
+
+To build from a `Makefile`, you will need a `MakefileBuilder`. It follows
+the same general idea as the `CMakeBuilder` except that it searches for
+`Makefile` instead of `CMakeLists.txt`. Additionally, `MakefileBuilder`
+will just run the default `make`.
+
+```py
+from lograder.dispatch import MakefileDispatcher
+from lograder.output import AssignmentSummary
+
+# Note that when you make a test, it's automatically
+# registered with the `lograder.tests.registry.TestRegistry`
+
+assignment = MakefileDispatcher(project_root="/autograder/submission")
+preprocessor_results = assignment.preprocess()
+build_results = assignment.build()
+runtime_results = assignment.run_tests()
+
+summary = AssignmentSummary(
+    preprocessor_output=preprocessor_results.get_output(),
+    build_output=build_results.get_output(),
+    runtime_summary=runtime_results.get_summary(),
+    test_cases=runtime_results.get_test_cases()
+)
+```
+
+----
+# Test Generation
+
+----
+
+## Output Comparison
+
+### Compare Simple Strings
+
+For the smallest number of tiny test cases, there's no reason
+to have an over-bloated mess. You can just use:
+
+```py
+from typing import Sequence, Optional, List
+from pathlib import Path
+from lograder.tests import make_tests_from_strs, ExecutableOutputComparisonTest
+
+
+def make_test_from_strs(
+        *,  # kwargs-only; to avoid confusion with argument sequence.
+        names: Sequence[str],
+        inputs: Sequence[str],
+        expected_outputs: Sequence[str],
+        flag_sets: Optional[Sequence[List[str | Path]]] = None,
+        # Pass flags like ["--option-1", "--option-2"] to student programs
+        weights: Optional[Sequence[float]] = None,  # Defaults to equal-weight.
+) -> List[ExecutableOutputComparisonTest]: ...
+
+
+# Here's an example of how you'd use the above method:
+make_tests_from_strs(
+    names=["Test Case 1", "Test Case 2"],
+    inputs=["stdin-1", "stdin-2"],
+    expected_outputs=["stdout-1", "stdout-2"]
+)
+```
+
+### Compare from Files
+
+If you have a larger test, it would be very convenient to
+read files for input and output. Luckily, there's just the
+method to do so:
+
+```py
+from typing import Sequence, Optional, List
+from pathlib import Path
+from lograder.tests import make_tests_from_files, FilePath, ExecutableOutputComparisonTest
+
+
+# `make_tests_from_files` has the following signature.
+def make_tests_from_files(
+        *,  # kwargs-only; to avoid confusion with argument sequence.
+        names: Sequence[str],
+        input_files: Optional[Sequence[FilePath]] = None,  # `input_files` and `input_strs` mutually exclusive.
+        input_strs: Optional[Sequence[str]] = None,
+        expected_output_files: Optional[Sequence[FilePath]] = None,
+        # same with `expected_output_files` and `expected_output_strs`
+        expected_output_strs: Optional[Sequence[str]] = None,
+        flag_sets: Optional[Sequence[List[str | Path]]] = None,
+        # Pass flags like ["--option-1", "--option-2"] to student programs
+        weights: Optional[Sequence[float]] = None,  # Defaults to equal-weight.
+) -> List[ExecutableOutputComparisonTest]: ...
+
+
+# Here's an example of how you'd use the above method:
+make_tests_from_files(
+    names=["Test Case 1", "Test Case 2"],
+    input_files=["test/inputs/input1.txt", "test/inputs/input2.txt"],
+    expected_output_files=["test/inputs/output1.txt", "test/inputs/output2.txt"]
+)
+```
+
+### Compare from Template
+
+Finally, sometimes the test-cases might be very long but 
+very repetitive. You can use `make_tests_from_template` 
+and pass a `TestCaseTemplate` object and ...
+
+```py
+from typing import Sequence, Optional, List
+from pathlib import Path
+from lograder.tests import make_tests_from_template, TestCaseTemplate, FilePath
+
+
+# Here's the signature of a `TemplateSubstitution`
+class TemplateSubstitution:
+    def __init__(self, *args, **kwargs):
+        # Stores args and kwargs to pass to str.format(...) later.
+        ...
+
+
+TSub = TemplateSubstitution  # Here's an alias that's quicker to type.
+
+
+# Here's the signature of a `TestCaseTemplate`
+class TestCaseTemplate:
+    def __init__(self, *,
+                 inputs: Optional[Sequence[str]] = None,
+                 input_template_file: Optional[FilePath] = None,
+                 input_template_str: Optional[str] = None,
+                 input_substitutions: Optional[Sequence[TemplateSubstitution]] = None,
+                 expected_outputs: Optional[Sequence[str]] = None,
+                 expected_output_template_file: Optional[FilePath] = None,
+                 expected_output_template_str: Optional[str] = None,
+                 expected_output_substitutions: Optional[Sequence[TemplateSubstitution]] = None,
+                 flag_sets: Optional[Sequence[List[str | Path]]] = None,  # Pass flags like ["--option-1", "--option-2"] to student programs
+                 ):
+        # +=====================================================================================+
+        # | Validation Rules                                                                    |
+        # +=====================================================================================+
+        #   * If `inputs` is specified, all other `input_*` parameters must be left unspecified.
+        #   * Same thing with `expected_outputs`.
+        #   * If `inputs` is not specified, you must specify either (mutually exclusive) 
+        #     `input_template_file` or `input_template_str` that follows a typical python
+        #     format string, and you must specify `input_substitutions`.
+        #   * Same thing with `expected_output_template_file`, `expected_output_template_str`, 
+        #     and `expected_output_substitutions`
+        ...
+
+
+# Here's an example of how you would use TestCaseTemplate
+test_suite_1 = TestCaseTemplate(
+    inputs=["A", "B", "C"],  # Three (3) Total Cases
+    expected_output_template_str="{}, {kwarged}, {}",
+    expected_output_substitutions=[
+        TSub(1.0, 2.0, kwarged="middle-arg-1"),  # Case 1 Substitutions
+        TSub(2.0, 5.0, kwarged="middle-arg-2"),  # Case 2 Substitutions
+        TSub(7.0, 6.0, kwarged="middle-arg-3"),  # Case 3 Substitutions
+    ]
+)
+make_tests_from_template(
+    ["Test 1", "Test 2", "Test 3"],
+    test_suite_1
+)  # remember to construct the tests!
+
+```
+
+### Compare from Python Generator/Iterable
+
+Sometimes, you want to generate a ton of test-cases (especially
+small test-cases), and it would be incredibly waste to have thousands
+of single-line files. You can create a python generator function that
+follows either the following `Protocol` or `TypedDict`.
+
+```py
+from typing import Protocol, TypedDict, Generator, NotRequired, List
+from pathlib import Path
+from lograder.tests import make_tests_from_generator
+
+
+# Your generator may return objects following the protocol...
+class TestCaseProtocol(Protocol):
+    def get_name(self): ...
+
+    def get_input(self): ...
+
+    def get_expected_output(self): ...
+
+class FlaggedTestCaseProtocol(TestCaseProtocol, Protocol):
+    def get_flags(self) -> List[str | Path]: ...
+    
+# Notice that TestCaseProtocol defaults to equal-weights
+class WeightedTestCaseProtocol(TestCaseProtocol, Protocol):
+    def get_weight(self): ...
+
+# ... or you can directly return a dict with the following keys.
+class TestCaseDict(TypedDict):
+    name: str
+    input: str
+    expected_output: str
+    weight: NotRequired[float]  # Defaults to 1.0, a.k.a. equal-weight.
+    flags: NotRequired[List[str | Path]]
+
+
+# Here's an example of the syntax as well as the required 
+# signature of such a method:
+@make_tests_from_generator
+def test_suite_1() -> Generator[TestCaseProtocol | WeightedTestCaseProtocol | TestCaseDict, None, None]:
+    pass
+
+# You'll have to query the `TestRegistry` from `lograder.tests` to access these tests directly, though.
+```
+
+
+