pdd-cli 0.0.90__py3-none-any.whl → 0.0.118__py3-none-any.whl

pdd/prompts/generate_test_LLM.prompt

@@ -9,6 +9,16 @@
  - The example file will be saved at: <test_file_path>{test_file_path}</test_file_path>
  - The module name (without extension) is: <module_name>{module_name}</module_name>
 
+% EXISTING TESTS (if provided - your output will be APPENDED to this file):
+<existing_tests>{existing_tests}</existing_tests>
+
+% If existing tests are provided above:
+    - Generate ONLY NEW test functions (your output will be appended to the existing file)
+    - Do NOT include import statements (they already exist in the file)
+    - Do NOT duplicate any existing test function names
+    - Maintain consistent style with existing tests (fixtures, naming conventions)
+    - Focus on testing functionality NOT already covered by existing tests
+
 % Follow these rules:
     - CRITICAL: You MUST analyze the actual code provided in code_under_test and generate tests for the EXACT functions defined in that code
     - CRITICAL: Import statements must use the ACTUAL module name from the code file path, not generic names. Include the necessary code/path info to import the code under test.
@@ -22,7 +32,16 @@
     - Setup and teardown methods should only use public APIs and environment variables, never reset internal module state directly.
     - Design tests to be independent of implementation details that might change when code is regenerated.
     - For test isolation, use fixtures and mocking of external dependencies rather than manipulating internal module state. In general minimize the amount of mocking needed so that the tests are more robust to changes in the code under test and more code is tested.
-<include>./context/test.prompt</include>
+
+% TEST ISOLATION PRINCIPLE:
+% CRITICAL: Each test MUST be isolated and not pollute state for other tests.
+    - Tests must clean up any state they modify (environment variables, global state, files, mocks)
+    - Use the testing framework's built-in isolation mechanisms (fixtures, setup/teardown, context managers)
+    - Avoid modifying global or module-level state directly; if unavoidable, always restore original state
+    - Prefer function-scoped test resources over shared/module-scoped ones to ensure isolation
+
+<include>context/test.prompt</include>
+<include>context/pytest_isolation_example.py</include>
 
 <instructions>
     1. FIRST: Carefully analyze the ACTUAL code provided in code_under_test:

pdd/prompts/generate_test_from_example_LLM.prompt

@@ -0,0 +1,115 @@
+% You are an expert Software Test Engineer. Your goal is to generate tests based on the intended behavior described in a prompt and demonstrated in an example file.
+
+% Here a description of what the code is supposed to do and was the prompt that generated the code: <prompt_that_generated_code>{prompt_that_generated_code}</prompt_that_generated_code>
+
+% Here is an example showing how the module should be used: <example_usage>{example}</example_usage>
+
+% File path information:
+ - The example file is located at: <example_file_path>{source_file_path}</example_file_path>
+ - The test file will be saved at: <test_file_path>{test_file_path}</test_file_path>
+ - The module name (without extension) is: <module_name>{module_name}</module_name>
+
+% EXISTING TESTS (if provided - your output will be APPENDED to this file):
+<existing_tests>{existing_tests}</existing_tests>
+
+% If existing tests are provided above:
+    - Generate ONLY NEW test functions (your output will be appended to the existing file)
+    - Do NOT include import statements (they already exist in the file)
+    - Do NOT duplicate any existing test function names
+    - Maintain consistent style with existing tests (fixtures, naming conventions)
+    - Focus on testing functionality NOT already covered by existing tests
+
+% Follow these rules:
+    - CRITICAL: Analyze the EXAMPLE to understand the API (function names, parameters, return values)
+    - CRITICAL: Import statements must match the module structure shown in the example
+    - CRITICAL: Test the intended function names and behavior based on the prompt
+    - The module name for the code under test will have the same name as the function name
+    - The unit test should be in {language}. If Python, use pytest.
+    - Use individual test functions for each case to make it easier to identify which specific cases pass or fail.
+    - Use the description of the functionality in the prompt to generate tests with useful tests with good code coverage.
+    - Focus on testing the INTENDED FUNCTIONALITY, not implementation details.
+    - NEVER access internal implementation details (variables/functions starting with underscore) in your tests.
+    - Setup and teardown methods should only use public APIs and environment variables, never reset internal module state directly.
+    - Design tests to be independent of implementation details.
+    - For test isolation, use fixtures and mocking of external dependencies rather than manipulating internal module state. In general minimize the amount of mocking needed so that the tests are more robust to changes in the code under test and more code is tested.
+    - Know that the generated test will be in a different directory (`tests`) than the module (in directory `pdd`) it is calling and will need an absolute reference. The module file name will be same as the function name.
+    - Created files should be in the `output` directory.
+    - Data files (language_format.csv and llm_model.csv) already exist in the PDD_PATH/`data` directory. Do not write over them. It already contains data for popular languages and LLM models and can be used for tests.
+    - The PDD_PATH environment variable is already set.
+
+% PYTEST TEST ISOLATION AND ANTI-POLLUTION RULES:
+% CRITICAL: Generated tests MUST be isolated and not pollute state for other tests. Follow these rules strictly:
+
+% 1. ENVIRONMENT VARIABLES:
+    - ALWAYS use monkeypatch.setenv() or monkeypatch.delenv() instead of os.environ["VAR"] = "value"
+    - NEVER use direct os.environ manipulation - it persists beyond the test and pollutes other tests
+    - BAD:  os.environ["API_KEY"] = "test_key"  # POLLUTION: persists after test ends
+    - GOOD: monkeypatch.setenv("API_KEY", "test_key")  # Auto-cleaned by pytest
+
+% 2. MOCKING EXTERNAL DEPENDENCIES:
+    - Use context managers or monkeypatch for mocks - they auto-cleanup after the test
+    - Prefer monkeypatch.setattr() over unittest.mock.patch() decorators at module level
+    - BAD:  @patch('module.func') at module/class level  # Can leak if exception occurs
+    - GOOD: monkeypatch.setattr('module.func', mock_func)  # Always cleaned up
+    - GOOD: with patch('module.func') as mock:  # Context manager ensures cleanup
+
+% 3. FIXTURE CLEANUP WITH YIELD:
+    - Use yield-based fixtures with cleanup code after yield for any resources
+    - Prefer function-scoped fixtures over module or session scope to ensure isolation
+    - BAD:  @pytest.fixture(scope="module") without cleanup  # State leaks between tests
+    - GOOD: @pytest.fixture with yield and cleanup after yield  # Always cleans up
+    - Example of proper fixture:
+        @pytest.fixture
+        def temp_resource():
+            resource = setup_resource()
+            yield resource
+            resource.cleanup()  # Always runs after test, even on failure
+
+% 4. SYS.MODULES MANIPULATION:
+    - AVOID manipulating sys.modules directly whenever possible
+    - If unavoidable, ALWAYS save and restore in try/finally or fixture with yield
+    - BAD:  sys.modules["module"] = mock_module  # Pollutes all subsequent tests
+    - GOOD: Use a fixture that saves, mocks, and restores:
+        @pytest.fixture
+        def mock_module():
+            saved = sys.modules.get("module")
+            sys.modules["module"] = MagicMock()
+            yield
+            if saved is not None:
+                sys.modules["module"] = saved
+            elif "module" in sys.modules:
+                del sys.modules["module"]
+
+% 5. FILE SYSTEM OPERATIONS:
+    - ALWAYS use the tmp_path fixture for creating temporary files and directories
+    - NEVER create files in the working directory or fixed paths
+    - BAD:  with open("test_output.txt", "w") as f: ...  # Leaves file behind
+    - GOOD: def test_file(tmp_path): (tmp_path / "test_output.txt").write_text(...)
+
+% 6. GLOBAL/MODULE STATE:
+    - Never modify global variables or module-level state directly in tests
+    - Use monkeypatch.setattr() for any module-level variables that need changing
+    - Reset any singleton instances using fixtures with proper teardown
+
+% SUMMARY OF GOOD PATTERNS:
+    - Use tmp_path fixture for file operations
+    - Use monkeypatch fixture for environment variables and attributes
+    - Use pytest.raises() as context manager for exception testing
+    - Prefer function-scoped fixtures over module or session scope
+    - Use yield in fixtures to ensure cleanup runs even on test failure
+
+
+<instructions>
+    1. FIRST: Carefully analyze the EXAMPLE to understand:
+        - How to import the module (exact import statements)
+        - What functions/classes are exposed
+        - How they are called (parameters, return values)
+    2. SECOND: Analyze the prompt that describes the intended functionality and edge cases.
+    3. THIRD: For each edge case explain whether it is better to do the test using Z3 formal verification or unit tests.
+    4. FOURTH: Develop a detailed test plan that will ensure the intended functionality is correct. This should involve both Z3 formal verification and unit tests.
+    5. FIFTH: Write the test file with:
+        a) The first part of the test file should be the detailed test plan from step 4 above in comments.
+        b) Import statements matching the module structure from the example
+        c) Tests for the intended function names and behavior from the prompt
+        d) Z3 formal verification tests that are runnable as unit tests.
+</instructions>