pdd-cli 0.0.45__py3-none-any.whl → 0.0.90__py3-none-any.whl

pdd/prompts/detect_change_LLM.prompt

@@ -16,8 +16,19 @@
 <example>
     <input_example>
         <prompt_list_example>
-        [{"PROMPT_NAME": "change_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python Software Engineer. Your goal is to write a Python function, \"change\", that will modify an input_prompt into a modified_prompt per the change_prompt. All output to the console will be pretty printed using the Python Rich library. Ensure that the module imports are done using relative imports.\n\n% Here are the inputs and outputs of the function:\n    Inputs:\n        - 'input_prompt' - A string that contains the prompt that will be modified by the change_prompt.\n        - 'input_code' - A string that contains the code that was generated from the input_prompt.\n        - 'change_prompt' - A string that contains the instructions of how to modify the input_prompt.\n        - 'strength': A float value representing the strength parameter for the LLM model, used to influence the model's behavior.\n        - 'temperature': A float value representing the temperature parameter for the LLM model, used to control the randomness of the model's output.\n    Outputs:\n        - 'modified_prompt' - A string that contains the modified prompt that was changed based on the change_prompt.\n        - 'total_cost': A float value representing the total cost of running the function.\n        - 'model_name': A string representing the name of the selected LLM model.\n\n% Here is an example how to preprocess the prompt from a file: ```<./context/preprocess_example.py>```\n\n% Example usage of the Langchain LCEL program: ```<./context/langchain_lcel_example.py>```\n\n% Example of selecting a Langchain LLM and counting tokens using llm_selector: ```<./context/llm_selector_example.py>```\n\n% Steps to be followed by the function:\n    1. Load the '$PDD_PATH/prompts/xml/change_LLM.prompt' and '$PDD_PATH/prompts/extract_prompt_change_LLM.prompt' files.\n    2. Preprocess the change_LLM prompt using the preprocess function from the preprocess module and set double_curly_brackets to false.\n    3. Create a Langchain LCEL template from the processed change_LLM prompt to return a string output.    \n    4. Use the llm_selector function for the LLM model and token counting.\n    5. Run the input_prompt through the model using Langchain LCEL:\n        - a. Pass the following string parameters to the prompt during invocation:             \n            * 'input_prompt'\n            * 'input_code'\n            * 'change_prompt' (preprocess this with double_curly_brackets set to false)\n        - b. Calculate the input and output token count using token_counter from llm_selector and pretty print the output of 4a, including the token count and estimated cost. The cost from llm_selector is in dollars per million tokens.\n    6. Create a Langchain LCEL template with strength .9 from the extract_prompt_change_LLM prompt that outputs JSON:\n        - a. Pass the following string parameters to the prompt during invocation: 'llm_output' (this string is from Step 4).\n        - b. Calculate input and output token count using token_counter from llm_selector and pretty print the running message with the token count and cost.\n        - c. Use 'get' function to extract 'modified_prompt' key values using from the dictionary output.\n    7. Pretty print the extracted modified_prompt using Rich Markdown function. Include token counts and costs.\n    8. Return the 'modified_prompt' string, the total_cost of both invokes and model_name use for the change_LLM prompt.\n\n% Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages."}, {"PROMPT_NAME": "preprocess_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python engineer. Your goal is to write a Python function, 'preprocess_prompt', that will preprocess the prompt from a prompt string for a LLM. This will use regular expressions to preprocess specific XML-like tags, if any, in the prompt. All output to the console will be pretty print using the Python rich library.\n\n% Here are the inputs and outputs of the function:\n    Input: \n        'prompt' - A string that is the prompt to preprocess\n        'recursive' - A boolean that is True if the program needs to recursively process the includes in the prompt and False if it does not need to recursively process the prompt. Default is True.\n        'double_curly_brackets' - A boolean that is True if the curly brackets need to be doubled and False if they do not need to be doubled. Default is True.\n        'exclude_keys' - An optional list of strings that are excluded from the curly bracket doubling.\n    Output: returns a string that is the preprocessed prompt, with any leading or trailing whitespace removed.\n\n% Here are the XML-like tags to preprocess, other tags will remain unmodified:\n    'include' - This tag will include the content of the file indicated in the include tag. The 'include tag' will be directly replaced with the content of the file in the prompt, without wrapping it in a new tag.\n    'pdd' - This tag indicates a comment and anything in this XML will be deleted from the string including the 'pdd' tags themselves.\n    'shell' - This tag indicates that there are shell commands to run. Capture all output of the shell commands and include it in the prompt but remove the shell tags.\n\n% Includes can be nested, that is there can be includes inside of the files of the includes and 'preprocess' should be called recursively on these include files if recursive is True. There are two ways of having includes in the prompt:\n    1. Will check to see if the file has any angle brackets in triple backticks. If so, it will read the included file indicated in the angle brackets and replace the angle brackets with the content of the included file. This will be done recursively until there are no more angle brackets in triple backticks. The program will then remove the angle brackets but leave the contents in the triple backticks.\n    2. The XML 'include' mentioned above.\n\n% If double_curly_brackets is True, the program will check to see if the file has any single curly brackets and if it does and the string in the curly brackets are not in the exclude_keys list, it will check to see if the curly brackets are already doubled before doubling the curly brackets.\n\n% The program should resolve file paths using the PDD_PATH environment variable. Implement a function 'get_file_path' that takes a file name and returns the full path using this environment variable.\n\n% Keep the user informed of the progress of the program by pretty printing messages."}, {"PROMPT_NAME": "unfinished_prompt_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python engineer. Your goal is to write a python function called 'unfinished_prompt' that will determine if a given prompt is complete or needs to continue.\n\n% Here are the inputs and outputs of the function:\n    Inputs:\n        'prompt_text' - A string containing the prompt text to analyze.\n        'strength' - A float that is the strength of the LLM model to use for the analysis. Default is 0.5.\n        'temperature' - A float that is the temperature of the LLM model to use for the analysis. Default is 0.\n    Outputs:\n        'reasoning' - A string containing the structured reasoning for the completeness assessment.\n        'is_finished' - A boolean indicating whether the prompt is complete (True) or incomplete (False).\n        'total_cost' - A float that is the total cost of the analysis function. This is an optional output.\n        'model_name' - A string that is the name of the LLM model used for the analysis. This is an optional output.\n\n% Here is an example of a Langchain LCEL program: ```<./context/langchain_lcel_example.py>```\n\n% Here is an example how to select the Langchain llm and count tokens: ```<./context/llm_selector_example.py>```\n\n% Note: Use relative import for 'llm_selector' to ensure compatibility within the package structure (i.e. 'from .llm_selector') instead of 'from pdd.llm_selector'.\n\n% This function will do the following:\n    Step 1. Use $PDD_PATH environment variable to get the path to the project. Load the '$PDD_PATH/prompts/unfinished_prompt_LLM.prompt' file.\n    Step 2. Create a Langchain LCEL template from unfinished_prompt_LLM prompt so that it returns a JSON output.\n    Step 3. Use the llm_selector function for the LLM model.\n    Step 4. Run the prompt text through the model using Langchain LCEL.\n        4a. Pass the following string parameters to the prompt during invoke:\n            - 'PROMPT_TEXT'\n        4b. Pretty print a message letting the user know it is running and how many tokens (using token_counter function from llm_selector) are in the prompt and the cost. The cost from llm_selector is in dollars per million tokens.\n        4c. The dictionary output of the LCEL will have the keys 'reasoning' and 'is_finished'. Be sure to access these keys using the get method with default error messages.\n        4d. Pretty print the reasoning and completion status using the rich library. Also, print the number of tokens in the result, the output token cost and the total_cost.\n    Step 5. Return the 'reasoning' string and 'is_finished' boolean from the JSON output using 'get', and the 'total_cost' float, and 'model_name' string.\n\n% Ensure that the function handles potential errors gracefully, such as missing input parameters or issues with the LLM model responses.\n\n"}, {"PROMPT_NAME": "xml_tagger_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python engineer. Your goal is to write a Python function, \"xml_tagger\", that will enhance a given LLM prompt by adding XML tags to improve its structure and readability.\n\n% The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name'). All output to the console will be pretty printed using the Python Rich library. Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.
-% The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```\n\n% Here are the inputs and outputs of the function:\n    Input: \n        'raw_prompt' - A string containing the prompt that needs XML tagging to improve its organization and clarity.\n        'strength' - A float value representing the strength parameter for the LLM model.\n        'temperature' - A float value representing the temperature parameter for the LLM model.\n    Output: \n        'xml_tagged' - A string containing the prompt with properly added XML tags.\n        'total_cost' - A float representing the total cost of running the LCELs.\n        'model_name' - A string representing the name of the selected LLM model.\n\n% Here is an example of a LangChain Expression Language (LCEL) program: <lcel_example>import os
+        [{"PROMPT_NAME": "change_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python Software Engineer. Your goal is to write a Python function, \"change\", that will modify an input_prompt into a modified_prompt per the change_prompt. All output to the console will be pretty printed using the Python Rich library. Ensure that the module imports are done using relative imports.\n\n% Here are the inputs and outputs of the function:\n    Inputs:\n        - 'input_prompt' - A string that contains the prompt that will be modified by the change_prompt.\n        - 'input_code' - A string that contains the code that was generated from the input_prompt.\n        - 'change_prompt' - A string that contains the instructions of how to modify the input_prompt.\n        - 'strength': A float value representing the strength parameter for the LLM model, used to influence the model's behavior.\n        - 'temperature': A float value representing the temperature parameter for the LLM model, used to control the randomness of the model's output.\n    Outputs:\n        - 'modified_prompt' - A string that contains the modified prompt that was changed based on the change_prompt.\n        - 'total_cost': A float value representing the total cost of running the function.\n        - 'model_name': A string representing the name of the selected LLM model.\n\n% Here is an example how to preprocess the prompt from a file: ```<./context/preprocess_example.py>```\n\n% Example usage of the Langchain LCEL program: ```<./context/langchain_lcel_example.py>```\n\n% Example of selecting a Langchain LLM and counting tokens using llm_selector: ```<./context/llm_selector_example.py>```\n\n% Steps to be followed by the function:\n    1. Load the '$PDD_PATH/prompts/xml/change_LLM.prompt' and '$PDD_PATH/prompts/extract_prompt_change_LLM.prompt' files.\n    2. Preprocess the change_LLM prompt using the preprocess function from the preprocess module and set double_curly_brackets to false.\n    3. Create a Langchain LCEL template from the processed change_LLM prompt to return a string output.    \n    4. Use the llm_selector function for the LLM model and token counting.\n    5. Run the input_prompt through the model using Langchain LCEL:\n        - a. Pass the following string parameters to the prompt during invocation:             \n            * 'input_prompt'\n            * 'input_code'\n            * 'change_prompt' (preprocess this with double_curly_brackets set to false)\n        - b. Calculate the input and output token count using token_counter from llm_selector and pretty print the output of 4a, including the token count and estimated cost. The cost from llm_selector is in dollars per million tokens.\n    6. Create a Langchain LCEL template with strength .9 from the extract_prompt_change_LLM prompt that outputs JSON:\n        - a. Pass the following string parameters to the prompt during invocation: 'llm_output' (this string is from Step 4).\n        - b. Calculate input and output token count using token_counter from llm_selector and pretty print the running message with the token count and cost.\n        - c. Use 'get' function to extract 'modified_prompt' key values using from the dictionary output.\n    7. Pretty print the extracted modified_prompt using Rich Markdown function. Include token counts and costs.\n    8. Return the 'modified_prompt' string, the total_cost of both invokes and model_name use for the change_LLM prompt.\n\n% Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages."}, {"PROMPT_NAME": "preprocess_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python engineer. Your goal is to write a Python function, 'preprocess_prompt', that will preprocess the prompt from a prompt string for a LLM. This will use regular expressions to preprocess specific XML-like tags, if any, in the prompt. All output to the console will be pretty print using the Python rich library.\n\n% Here are the inputs and outputs of the function:\n    Input: \n        'prompt' - A string that is the prompt to preprocess\n        'recursive' - A boolean that is True if the program needs to recursively process the includes in the prompt and False if it does not need to recursively process the prompt. Default is True.\n        'double_curly_brackets' - A boolean that is True if the curly brackets need to be doubled and False if they do not need to be doubled. Default is True.\n        'exclude_keys' - An optional list of strings that are excluded from the curly bracket doubling.\n    Output: returns a string that is the preprocessed prompt, with any leading or trailing whitespace removed.\n\n% Here are the XML-like tags to preprocess, other tags will remain unmodified:\n    'include' - This tag will include the content of the file indicated in the include tag. The 'include tag' will be directly replaced with the content of the file in the prompt, without wrapping it in a new tag.\n    'pdd' - This tag indicates a comment and anything in this XML will be deleted from the string including the 'pdd' tags themselves.\n    'shell' - This tag indicates that there are shell commands to run. Capture all output of the shell commands and include it in the prompt but remove the shell tags.\n\n% Includes can be nested, that is there can be includes inside of the files of the includes and 'preprocess' should be called recursively on these include files if recursive is True. There are two ways of having includes in the prompt:\n    1. Will check to see if the file has any angle brackets in triple backticks. If so, it will read the included file indicated in the angle brackets and replace the angle brackets with the content of the included file. This will be done recursively until there are no more angle brackets in triple backticks. The program will then remove the angle brackets but leave the contents in the triple backticks.\n    2. The XML 'include' mentioned above.\n\n% If double_curly_brackets is True, the program will check to see if the file has any single curly brackets and if it does and the string in the curly brackets are not in the exclude_keys list, it will check to see if the curly brackets are already doubled before doubling the curly brackets.\n\n% The program should resolve file paths using the PDD_PATH environment variable. Implement a function 'get_file_path' that takes a file name and returns the full path using this environment variable.\n\n% Keep the user informed of the progress of the program by pretty printing messages."}, {"PROMPT_NAME": "unfinished_prompt_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python engineer. Your goal is to write a python function called 'unfinished_prompt' that will determine if a given prompt is complete or needs to continue.\n\n% Here are the inputs and outputs of the function:\n    Inputs:\n        'prompt_text' - A string containing the prompt text to analyze.\n        'strength' - A float that is the strength of the LLM model to use for the analysis. Default is 0.5.\n        'temperature' - A float that is the temperature of the LLM model to use for the analysis. Default is 0.\n    Outputs:\n        'reasoning' - A string containing the structured reasoning for the completeness assessment.\n        'is_finished' - A boolean indicating whether the prompt is complete (True) or incomplete (False).\n        'total_cost' - A float that is the total cost of the analysis function. This is an optional output.\n        'model_name' - A string that is the name of the LLM model used for the analysis. This is an optional output.\n\n% Here is an example of a Langchain LCEL program: ```<./context/langchain_lcel_example.py>```\n\n% Here is an example how to select the Langchain llm and count tokens: ```<./context/llm_selector_example.py>```\n\n% Note: Use relative import for 'llm_selector' to ensure compatibility within the package structure (i.e. 'from .llm_selector') instead of 'from pdd.llm_selector'.\n\n% This function will do the following:\n    Step 1. Use $PDD_PATH environment variable to get the path to the project. Load the '$PDD_PATH/prompts/unfinished_prompt_LLM.prompt' file.\n    Step 2. Create a Langchain LCEL template from unfinished_prompt_LLM prompt so that it returns a JSON output.\n    Step 3. Use the llm_selector function for the LLM model.\n    Step 4. Run the prompt text through the model using Langchain LCEL.\n        4a. Pass the following string parameters to the prompt during invoke:\n            - 'PROMPT_TEXT'\n        4b. Pretty print a message letting the user know it is running and how many tokens (using token_counter function from llm_selector) are in the prompt and the cost. The cost from llm_selector is in dollars per million tokens.\n        4c. The dictionary output of the LCEL will have the keys 'reasoning' and 'is_finished'. Be sure to access these keys using the get method with default error messages.\n        4d. Pretty print the reasoning and completion status using the rich library. Also, print the number of tokens in the result, the output token cost and the total_cost.\n    Step 5. Return the 'reasoning' string and 'is_finished' boolean from the JSON output using 'get', and the 'total_cost' float, and 'model_name' string.\n\n% Ensure that the function handles potential errors gracefully, such as missing input parameters or issues with the LLM model responses.\n\n"}, {"PROMPT_NAME": "xml_tagger_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python engineer. Your goal is to write a Python function, \"xml_tagger\", that will enhance a given LLM prompt by adding XML tags to improve its structure and readability.\n\n% You are an expert Python engineer.
+
+% Code Style Requirements
+- File must start with `from __future__ import annotations`.
+- All functions must be fully type-hinted.
+- Use `rich.console.Console` for all printing.
+
+% Package Structure
+- The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name').
+- The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```
+
+% Error Handling
+- Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.\n\n% Here are the inputs and outputs of the function:\n    Input: \n        'raw_prompt' - A string containing the prompt that needs XML tagging to improve its organization and clarity.\n        'strength' - A float value representing the strength parameter for the LLM model.\n        'temperature' - A float value representing the temperature parameter for the LLM model.\n    Output: \n        'xml_tagged' - A string containing the prompt with properly added XML tags.\n        'total_cost' - A float representing the total cost of running the LCELs.\n        'model_name' - A string representing the name of the selected LLM model.\n\n% Here is an example of a LangChain Expression Language (LCEL) program: <lcel_example>import os
 from langchain_core.prompts import PromptTemplate
 from langchain_community.cache import SQLiteCache
 from langchain_community.llms.mlx_pipeline import MLXPipeline
@@ -367,12 +378,59 @@ if __name__ == "__main__":
         <change_description_example>
         % Use context/python_preamble.prompt to make prompts more compact. Some prompts might already have this.
 
-% Here is what is inside context/python_preamble.prompt:<preamble>% The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name'). All output to the console will be pretty printed using the Python Rich library. Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.
-% The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```</preamble>
+% Here is what is inside context/python_preamble.prompt:<preamble>% You are an expert Python engineer.
+
+% Code Style Requirements
+- File must start with `from __future__ import annotations`.
+- All functions must be fully type-hinted.
+- Use `rich.console.Console` for all printing.
+
+% Package Structure
+- The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name').
+- The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```
+
+% Error Handling
+- Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.</preamble>
 
 % Here is an example of this being done: <example>% You are an expert Python engineer. Your goal is to write a python function, "code_generator", that will compile a prompt into a code file. 
 
-[File not found: ../context/python_preamble.prompt]
+You are an expert Python engineer working on the PDD Cloud project.
+
+-------------------------------------------------------------------------------
+Python Coding Standards (PDD Cloud)
+-------------------------------------------------------------------------------
+
+## Style
+- Python 3.12, PEP 8 compliant
+- Type hints required (from typing import Optional, List, Dict, Any, Tuple)
+- Imports: standard lib -> third-party -> local (alphabetical within groups)
+
+## Critical First Import
+import function_import_setup  # MUST be first import
+
+This enables subprocess module resolution for Firebase Functions Framework.
+All endpoint files must have this as their very first import statement.
+
+## Standard Library Imports
+import os
+import logging
+
+## Error Handling Pattern
+Map exceptions to HTTP status codes:
+- AuthenticationError -> 401 Unauthorized
+- AuthorizationError -> 403 Forbidden
+- ValidationError -> 400 Bad Request
+- ResourceNotFoundError -> 404 Not Found
+- DatabaseError / Exception -> 500 Internal Server Error
+
+All error responses: {"error": "descriptive message"}
+
+## Logging
+logger = logging.getLogger(__name__)
+
+## Response Format
+Return tuple: (response_dict, status_code)
+
 
 % Here are the inputs and outputs of the function:
     Inputs: 
@@ -380,13 +438,13 @@ if __name__ == "__main__":
         'language' - A string that is the language type (e.g. python, bash) of file that will be outputed by the LLM.
         'strength' - A float between 0 and 1 that is the strength of the LLM model to use.
         'temperature' - A float that is the temperature of the LLM model to use. Default is 0.
-        'time' - A float between 0 and 1 that controls the thinking effort for the LLM model, passed to llm_invoke. Default is DEFAULT_TIME.
+        'time' - A float in [0,1] or None that controls the thinking effort for the LLM model, passed to llm_invoke. Default is DEFAULT_TIME. If None, treat as DEFAULT_TIME before invoking llm_invoke.
         'verbose' - A boolean that indicates whether to print out the details of the function. Default is False.
         'preprocess_prompt' - A boolean that indicates whether to preprocess the prompt. Default is True.
     Outputs:
         'runnable_code' - A string that is runnable code
-        'total_cost' - A float that is the total cost of the model run
-        'model_name' - A string that is the name of the selected LLM model
+        'total_cost' - A float that is the total cost of all LLM calls within this function (initial generation, unfinished check, continuation if used, and postprocess)
+        'model_name' - A string that is the name of the selected LLM model used for the main generation (or continuation). Postprocess may use a different model internally and does not change this value.
 
 % Here is how to use the internal modules:
     <internal_modules>
@@ -417,14 +475,25 @@ if __name__ == "__main__":
     </internal_modules>
 
 % This program will do the following:
-    Step 1. Conditionally preprocess the raw prompt using the preprocess function from the preprocess module based on the value of 'preprocess_prompt'. If 'preprocess_prompt' is True, preprocess the prompt; otherwise, use the raw prompt directly.
+    Step 1. Conditionally preprocess the raw prompt using the preprocess function from the preprocess module based on the value of 'preprocess_prompt'. If 'preprocess_prompt' is True, preprocess the prompt; otherwise, use the raw prompt directly. When preprocessing, it is acceptable to enable options such as double_curly_brackets=True and recursive=False to preserve placeholders and avoid over-expansion.
+
+    Step 2. Generate the initial response as follows:
+        - If the prompt contains embedded data URLs (e.g., 'data:image/...;base64,...'), split the prompt into alternating text and image parts (preserving order) and call llm_invoke with messages=[{role: 'user', content: [{type: 'image_url', image_url: {url: ...}}, {type: 'text', text: ...}, ...]}] and the provided strength, temperature, time, and verbose.
+        - Otherwise, call llm_invoke with the (preprocessed or raw) prompt, input_json={}, and the provided strength, temperature, time, and verbose.
+
+    Step 3. Detect if the generation is incomplete using the unfinished_prompt function (use strength=0.5, temperature=0.0) by passing in the last 600 characters of the output of Step 2. Pass through language, time, and verbose.
+        - a. If incomplete, call the continue_generation function to complete the generation and set final_output to that result.
+        - b. Else, set final_output to the initial model output.
+
+    Step 4. Postprocess the final_output using the postprocess function from the postprocess module with the EXTRACTION_STRENGTH constant. Use temperature=0.0 and pass through language, time, and verbose.
 
-    Step 2. Run the prompt (either preprocessed or raw) through llm_invoke with an empty dictionary ('{}') and the provided strength, temperature, and time.
+    Step 5. Return the runnable_code, total_cost and model_name.
 
-    Step 3. Detect if the generation is incomplete using the unfinished_prompt function (strength .5) by passing in the last 600 characters of the output of Step 3.
-        - a. If incomplete, call the continue_generation function to complete the generation.
-        - b. Else, if complete, postprocess the model output result using the postprocess function from the postprocess module with the EXTRACTION_STRENGTH constant. Be sure to pass in all parameters like verbose to postprocess.
-    Step 4. Return the runnable_code, total_cost and model_name.</example>
+% Validation and defaults:
+    - Validate non-empty 'prompt' and 'language'.
+    - Enforce 0 ≤ strength ≤ 1 and 0 ≤ temperature ≤ 2.
+    - Enforce 0 ≤ time ≤ 1 when provided; if time is None, substitute DEFAULT_TIME before llm_invoke.
+</example>
         </change_description_example>
     </input_example>
 
@@ -465,24 +534,57 @@ if __name__ == "__main__":
        - Insert the contents of the file `./context/python_preamble.prompt` immediately after the role and goal statement using 'include' XML tags.
        - Remove any redundant instructions that are covered by the preamble, such as those related to pretty printing and handling edge cases.
        - Ensure that the logical flow of the prompt is maintained and that any unique instructions specific to this prompt are retained.
-       - Here is what is inside context/python_preamble.prompt:<preamble>% The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name'). All output to the console will be pretty printed using the Python Rich library. Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.
-% The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```</preamble>
+       - Here is what is inside context/python_preamble.prompt:<preamble>% You are an expert Python engineer.
+
+% Code Style Requirements
+- File must start with `from __future__ import annotations`.
+- All functions must be fully type-hinted.
+- Use `rich.console.Console` for all printing.
+
+% Package Structure
+- The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name').
+- The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```
+
+% Error Handling
+- Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.</preamble>
 
    - **preprocess_python.prompt**
      - **Instructions:**
        - Insert the contents of the file `./context/python_preamble.prompt` immediately after the role and goal statement using 'include' XML tags.
        - Remove any redundant instructions that are covered by the preamble, such as those related to pretty printing and handling edge cases.
        - Ensure that the logical flow of the prompt is maintained and that any unique instructions specific to this prompt are retained.
-       - Here is what is inside context/python_preamble.prompt:<preamble>% The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name'). All output to the console will be pretty printed using the Python Rich library. Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.
-% The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```</preamble>
+       - Here is what is inside context/python_preamble.prompt:<preamble>% You are an expert Python engineer.
+
+% Code Style Requirements
+- File must start with `from __future__ import annotations`.
+- All functions must be fully type-hinted.
+- Use `rich.console.Console` for all printing.
+
+% Package Structure
+- The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name').
+- The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```
+
+% Error Handling
+- Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.</preamble>
 
    - **unfinished_prompt_python.prompt**
      - **Instructions:**
        - Insert the contents of the file `./context/python_preamble.prompt` immediately after the role and goal statement using 'include' XML tags.
        - Remove any redundant instructions that are covered by the preamble, such as those related to pretty printing and handling edge cases.
        - Ensure that the logical flow of the prompt is maintained and that any unique instructions specific to this prompt are retained.
-       - Here is what is inside context/python_preamble.prompt:<preamble>% The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name'). All output to the console will be pretty printed using the Python Rich library. Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.
-% The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```</preamble>
+       - Here is what is inside context/python_preamble.prompt:<preamble>% You are an expert Python engineer.
+
+% Code Style Requirements
+- File must start with `from __future__ import annotations`.
+- All functions must be fully type-hinted.
+- Use `rich.console.Console` for all printing.
+
+% Package Structure
+- The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name').
+- The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```
+
+% Error Handling
+- Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.</preamble>
     </output_example>
 </example>
 

pdd/prompts/example_generator_LLM.prompt

@@ -4,7 +4,28 @@
 
 % The language of the example should be in: <language_for_example>{language}</language_for_example>
 
+% File path information:
+ - The code module file is located at: <code_module_file_path>{source_file_path}</code_module_file_path>
+ - The example file will be saved at: <example_file_path>{example_file_path}</example_file_path>
+ - The module name (without extension) is: <module_name>{module_name}</module_name>
+
+% IMPORT INSTRUCTIONS: Use the appropriate import mechanism for the target language
+ - CRITICAL: Use the exact module_name provided in the file path information above - DO NOT use generic names like "greetings" or "module"
+ - Avoid package-style imports unless the file is actually in a package structure
+ - Import the specific functions/classes that are defined in the code module
+ - CRITICAL: Do not assume module names - use the exact module_name provided
+ - CRITICAL: Import the appropriate functions/classes from the module (e.g., if module_name is "simple_math", import "add", "subtract", etc.)
+ - CRITICAL: Never include hardcoded absolute paths like "/Users/username/project/examples/" in comments or code
+    - Use only relative path descriptions in comments (e.g., "relative to project root" or "relative to this script")
+    - Make the example portable across different development environments
+    - Use dynamic path resolution with os.path.dirname(__file__) and relative path construction
+    - In comments, describe file structure using relative terms, not absolute paths
+ - CRITICAL: The 'extracted_code' field must preserve all newlines using proper JSON escaping (\\n). The output must be valid JSON where multi-line code is represented with \\n escape sequences for each line break.
+
 % Make sure the following happens:
     - Document in detail what the input and output parameters in the doc strings
-    - Someone needs to be able to fully understand how to use the module from the example.
+    - Someone needs to be able to fully understand how to use the module from the example.        
+    - Use correct import statements based on the actual file structure
+    - The example should be a complete, runnable script that imports from the actual module
+    - Include proper file path handling and module discovery if needed
 <include>./context/example.prompt</include>

pdd/prompts/extract_code_LLM.prompt

@@ -19,4 +19,8 @@
 % Output a JSON object with the following keys:
     - 'focus': String containing the focus of the generation.
     - 'explanation': String explanation of why this block_type was the focus of the generation and explain any errors detected in the code, if a code type of block.
-    - 'extracted_code': String containing the entire generated and corrected block_type of focus.
+    - 'extracted_code': String containing the entire generated and corrected block_type of focus.
+
+% CRITICAL: The 'extracted_code' field MUST preserve all newlines and indentation from the original code. In JSON, newlines must be represented as the escape sequence \n (backslash-n). Each line break in the code must appear as \n in the JSON string value. Do NOT concatenate lines together or remove line breaks. The extracted code must be properly formatted and runnable.
+
+% IMPORTANT: Output ONLY the JSON object. Do not add any trailing whitespace or newlines after the closing brace.

pdd/prompts/extract_program_code_fix_LLM.prompt

@@ -20,4 +20,10 @@
     - 'update_program': Boolean indicating whether the program needs to be updated.
     - 'update_code': Boolean indicating whether the code module needs to be updated.
     - 'fixed_program': The entire updated program code or empty String if no update is needed.
-    - 'fixed_code': The entire updated code module or empty String if no update is needed.
+    - 'fixed_code': The entire updated code module or empty String if no update is needed.
+
+% IMPORTANT JSON formatting rules for code strings:
+    - Use standard JSON escaping for newlines: a single backslash-n (\n) represents an actual newline
+    - Do NOT double-escape newlines. Use \n not \\n for line breaks in code
+    - String literals in code that need to print newlines should use \n (which appears as \\n in JSON)
+    - Example: for code with two lines "def foo():\n    pass", the JSON should be: "def foo():\n    pass"

pdd/prompts/extract_prompt_update_LLM.prompt

@@ -1,14 +1,13 @@
-% You are an expert Software Engineer. Your goal is to extract the updated prompt from the LLM output.
+% You are an expert Software Engineer. Your goal is to extract the updated prompt from the LLM output in JSON format.
 
 % Here is the generated llm_output: <llm_output>{llm_output}</llm_output>
 
-% The LLM output contains the modified prompt that will generate the modified code, possibly with some additional commentary or explanation.
-% Your task is to identify and extract ONLY the modified prompt itself, without adding any JSON structure or additional formatting.
+% The LLM output contains the modified prompt that will generate the modified code, possibly with some additional commentary or explanation. Your task is to identify and extract ONLY the modified prompt itself, without adding any additional formatting.
 
 % Ensure you:
-% 1. Remove any "# Modified Prompt" headers or similar text that isn't part of the actual prompt
-% 2. Preserve all markdown, code blocks, and formatting within the actual prompt
-% 3. Don't add any explanatory text, JSON wrappers, or your own commentary
-% 4. Return only the text that constitutes the actual prompt
+  1. Remove any "# Modified Prompt" headers or similar text that isn't part of the actual prompt
+  2. Preserve all markdown, code blocks, and formatting within the actual prompt
+  3. Don't add any explanatory text, JSON wrappers, or your own commentary
+  4. Return only the text that constitutes the actual prompt
 
-% The "modified_prompt" should be the complete, standalone prompt that could be used directly to generate the modified code.
+% The "modified_prompt" JSON key should be the complete, standalone prompt that could be used directly to generate the modified code.

pdd/prompts/extract_promptline_LLM.prompt

@@ -1,11 +1,17 @@
-% You are an expert Software Engineer. Your goal is to extract the closest matching substring described by the prompt's output to be outputed in JSON format.
-
-% Here is the llm_output to parse: <llm_output>{llm_output}</llm_output>
-
-% When extracting the closest matching substring from llm_output, consider and correct the following for the extracted code:
-    - Should be a substring of prompt_file.
-    - Should be a a substring that closely matches code_str in content.
-
-% Output a JSON object with the following keys:
-    - 'explanation': String explanation of why this prompt_line matches the code_str and explain any errors detected in the code.
-    - 'prompt_line': String containing the closest matching verbatim substring of the prompt_file that matches code_str in content. This is not the line number.
+% You are an extremely literal parser. The LLM output below follows this structure:
+%   <analysis> ... </analysis>
+%   <verbatim_prompt_line>
+%   <<EXACT SUBSTRING FROM PROMPT_FILE>>
+%   </verbatim_prompt_line>
+%
+% Task
+%   • Extract the text between <verbatim_prompt_line> and </verbatim_prompt_line> (if present).
+%   • Output ONLY JSON with the keys:
+%       - "prompt_line": the exact substring between the tags. Do not alter whitespace or characters except for JSON escaping.
+%       - "explanation": short confirmation (<=120 characters) that the substring was copied verbatim, or describe why extraction failed.
+%   • If the tags are missing or empty, set "prompt_line" to "" and explain the issue.
+%   • Do not wrap the JSON in Markdown. No commentary, no additional keys.
+%
+<llm_output>
+{llm_output}
+</llm_output>

pdd/prompts/find_verification_errors_LLM.prompt

@@ -21,6 +21,12 @@
     Step 4. Identify any potential edge cases, error handling issues, or performance concerns that could cause problems in the future.
     Step 5. Check the code for potential bugs that haven't manifested yet.
     Step 6. If any issues are found, explain in detail the root cause of each issue and how it could impact the program's functioning.
+    Step 6.5. When analyzing errors involving mocked dependencies:
+       - Identify if 'program' uses MagicMock, unittest.mock, or similar mocking
+       - Trace the mock configuration to verify it matches expected real API behavior
+       - For AttributeError on mock objects: check if mock.return_value or mock.__getitem__.return_value has correct type
+       - Flag errors as "Mock Configuration Error" when the mock setup doesn't match real API return types
+       - Flag errors as "Production Code Error" only when the API usage is clearly incorrect
     Step 7. Carefully distinguish between:
        a. Incompatibilities (functions called by program but missing from code_module) - these are critical issues
        b. Prompt adherence issues (code doesn't match prompt requirements) - these are important but secondary to compatibility

pdd/prompts/fix_code_module_errors_LLM.prompt

@@ -2,11 +2,13 @@
 
 % IMPORTANT: The crash command should fix whatever needs to be fixed to make the program run successfully:
 - If the code module has bugs, fix the code module
-- If the calling program has bugs, fix the calling program  
+- If the calling program has bugs, fix the calling program
 - If both have issues that contribute to the crash, fix BOTH
 - The goal is to ensure the program runs without errors after all fixes are applied
+- If you are not able to fix the calling program, or if you cannot access it, you shouldn't guess at fixes to the calling program. Do not add the functionality of the calling program to the code_module.
+- You must ensure that the code_module strictly adheres to the prompt requirements
 
-% Here is the program that is running the code_module that crashed and/or has errors: <program>{program}</program>
+% Here is the calling program that is running the code_module that crashed and/or has errors: <program>{program}</program>
 
 % Here is the prompt that generated the code_module below: <prompt>{prompt}</prompt>
 

pdd/prompts/fix_errors_from_unit_tests_LLM.prompt

@@ -10,6 +10,14 @@
 
 % If the verfication program fails to run, the code_under_test and unit_test are unchanged from the previous iteration.
 
+% IMPORTANT: The original prompt is the authoritative specification for what the code should do.
+% When analyzing errors:
+%   - If the code doesn't match the prompt specification, fix the CODE
+%   - If the unit_test expects behavior not specified in the prompt, fix the TEST
+%   - Never add functionality to the code that isn't specified in the prompt
+%   - Tests should verify prompt-specified behavior, not arbitrary expectations
+
+
 
 <instructions>
 % Follow these steps to solve these errors:

pdd/prompts/fix_verification_errors_LLM.prompt

@@ -21,6 +21,28 @@
     5. Prefer making additive changes (adding new functions, improving existing ones) rather than removing functionality, even if that means going beyond the minimal requirements of the prompt.
     6. If your previous fixes resulted in verification failures related to missing functions, ensure those functions are included in your solution.
 
+% MOCK VS PRODUCTION CODE GUIDANCE:
+    1. IDENTIFY THE TEST FILE: The 'program' file may be a TEST FILE that uses mocks (MagicMock, unittest.mock, patch) to simulate external dependencies.
+       - Look for imports: `from unittest.mock import MagicMock, patch`
+       - Look for mock setup patterns: `mock_obj.return_value`, `mock_obj.__getitem__.return_value`
+
+    2. WHEN ERRORS OCCUR IN MOCK INTERACTIONS:
+       - FIRST check if the mock setup is incorrect (wrong return_value structure, missing __getitem__ configuration)
+       - Mock return types must match the REAL API return types exactly
+       - Common mock bugs:
+         * `__getitem__.return_value = [item]` when it should be `= item` (for APIs where indexing returns single item)
+         * Missing chained mock configuration (e.g., `mock.method().other_method()`)
+
+    3. PRESERVE PRODUCTION CODE API USAGE:
+       - The 'code_module' implements PRODUCTION code that calls real APIs
+       - Assume production code uses CORRECT API patterns unless you have documentation proving otherwise
+       - Do NOT change production code indexing patterns (like `[0][0]`) without external API documentation
+
+    4. DIAGNOSIS PRIORITY for "AttributeError" or type mismatch:
+       a. First: Check mock.return_value / mock.__getitem__.return_value structure
+       b. Second: Check if mock chaining matches expected API call pattern
+       c. Third: Only then consider if production code has wrong API usage
+
 % Follow these steps to fix the program or code_module:
     Step 1. Analyze and understand each identified issue in the context of the code_module and program.
     Step 2. Analyze how the program uses the code_module to determine all functions that must be preserved.

pdd/prompts/generate_test_LLM.prompt

@@ -4,7 +4,15 @@
 
 % Here is the code under test: <code_under_test>{code}</code_under_test>
 
+% File path information:
+ - The code under test module file is located at: <code_under_test_file_path>{source_file_path}</code_under_test_file_path>
+ - The example file will be saved at: <test_file_path>{test_file_path}</test_file_path>
+ - The module name (without extension) is: <module_name>{module_name}</module_name>
+
 % Follow these rules:
+    - CRITICAL: You MUST analyze the actual code provided in code_under_test and generate tests for the EXACT functions defined in that code
+    - CRITICAL: Import statements must use the ACTUAL module name from the code file path, not generic names. Include the necessary code/path info to import the code under test.
+    - CRITICAL: Test the ACTUAL function names, parameters, and behavior shown in the provided code
     - The module name for the code under test will have the same name as the function name
     - The unit test should be in {language}. If Python, use pytest.
     - Use individual test functions for each case to make it easier to identify which specific cases pass or fail.
@@ -17,10 +25,17 @@
 <include>./context/test.prompt</include>
 
 <instructions>
-    1. Carefully read the prompt that generated the code under test and determine what might be possible edge cases.
-    2. For each edge case explain whether it is better to do the test using Z3 formal verification or unit tests.
-    3. Develop a detailed test plan that will ensure the code under test is correct. This should involve both Z3 formal verification and unit tests.
-    4. Now write the test file. 
-        a) The first part of the test file should be the detailed test plan from step 3 above in comments. 
-        b) Then write the tests and Z3 formal verification tests that are runnable as unit tests.
+    1. FIRST: Carefully analyze the ACTUAL code provided in code_under_test:
+        - Identify the EXACT function names defined in the code
+        - Identify the EXACT parameters and their types
+        - Identify the EXACT return values and behavior
+        - Identify any error conditions or edge cases
+    2. SECOND: Analyze the prompt that generated the code to understand the intended functionality and edge cases.
+    3. THIRD: For each edge case explain whether it is better to do the test using Z3 formal verification or unit tests.
+    4. FOURTH: Develop a detailed test plan that will ensure the code under test is correct. This should involve both Z3 formal verification and unit tests.
+    5. FIFTH: Write the test file with:
+        a) The first part of the test file should be the detailed test plan from step 4 above in comments. 
+        b) Import statements using the ACTUAL module name from the code file path (e.g., if code is in "my_function.py", use "from my_function import function_name")
+        c) Tests for the ACTUAL function names and behavior shown in the provided code
+        d) Z3 formal verification tests that are runnable as unit tests.
 </instructions>

pdd/prompts/increase_tests_LLM.prompt

@@ -12,8 +12,4 @@ Here is the coverage report: ```{coverage_report}```
     - The module name for the code under test will have the same name as the function name
     - The unit test should be in {language}. If Python, use pytest.
     - Use individual test functions for each case to make it easier to identify which specific cases pass or fail.
-    - Use the description of the functionality in the prompt to generate tests with useful tests with good code coverage.
-    - Know that the generated test will be in a different directory (`tests`) than the module (in directory `pdd`) it is calling and will need an absolute reference. The module file name will be same as the function name.
-    - Created files should be in the `output` directory.
-    - Data files (language_format.csv and llm_model.csv) already exist in the PDD_PATH/`data` directory. Do not write over them. It already contains data for popular languages and LLM models and can be used for tests.
-    - The PDD_PATH environment variable is already set.
+    - Use the description of the functionality in the prompt to generate tests with useful tests with good code coverage.