pdd-cli 0.0.42__py3-none-any.whl → 0.0.90__py3-none-any.whl

pdd/prompts/detect_change_LLM.prompt

@@ -16,16 +16,575 @@
 <example>
     <input_example>
         <prompt_list_example>
-        <include>context/detect_change/2/prompt_list.json</include>
+        [{"PROMPT_NAME": "change_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python Software Engineer. Your goal is to write a Python function, \"change\", that will modify an input_prompt into a modified_prompt per the change_prompt. All output to the console will be pretty printed using the Python Rich library. Ensure that the module imports are done using relative imports.\n\n% Here are the inputs and outputs of the function:\n    Inputs:\n        - 'input_prompt' - A string that contains the prompt that will be modified by the change_prompt.\n        - 'input_code' - A string that contains the code that was generated from the input_prompt.\n        - 'change_prompt' - A string that contains the instructions of how to modify the input_prompt.\n        - 'strength': A float value representing the strength parameter for the LLM model, used to influence the model's behavior.\n        - 'temperature': A float value representing the temperature parameter for the LLM model, used to control the randomness of the model's output.\n    Outputs:\n        - 'modified_prompt' - A string that contains the modified prompt that was changed based on the change_prompt.\n        - 'total_cost': A float value representing the total cost of running the function.\n        - 'model_name': A string representing the name of the selected LLM model.\n\n% Here is an example how to preprocess the prompt from a file: ```<./context/preprocess_example.py>```\n\n% Example usage of the Langchain LCEL program: ```<./context/langchain_lcel_example.py>```\n\n% Example of selecting a Langchain LLM and counting tokens using llm_selector: ```<./context/llm_selector_example.py>```\n\n% Steps to be followed by the function:\n    1. Load the '$PDD_PATH/prompts/xml/change_LLM.prompt' and '$PDD_PATH/prompts/extract_prompt_change_LLM.prompt' files.\n    2. Preprocess the change_LLM prompt using the preprocess function from the preprocess module and set double_curly_brackets to false.\n    3. Create a Langchain LCEL template from the processed change_LLM prompt to return a string output.    \n    4. Use the llm_selector function for the LLM model and token counting.\n    5. Run the input_prompt through the model using Langchain LCEL:\n        - a. Pass the following string parameters to the prompt during invocation:             \n            * 'input_prompt'\n            * 'input_code'\n            * 'change_prompt' (preprocess this with double_curly_brackets set to false)\n        - b. Calculate the input and output token count using token_counter from llm_selector and pretty print the output of 4a, including the token count and estimated cost. The cost from llm_selector is in dollars per million tokens.\n    6. Create a Langchain LCEL template with strength .9 from the extract_prompt_change_LLM prompt that outputs JSON:\n        - a. Pass the following string parameters to the prompt during invocation: 'llm_output' (this string is from Step 4).\n        - b. Calculate input and output token count using token_counter from llm_selector and pretty print the running message with the token count and cost.\n        - c. Use 'get' function to extract 'modified_prompt' key values using from the dictionary output.\n    7. Pretty print the extracted modified_prompt using Rich Markdown function. Include token counts and costs.\n    8. Return the 'modified_prompt' string, the total_cost of both invokes and model_name use for the change_LLM prompt.\n\n% Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages."}, {"PROMPT_NAME": "preprocess_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python engineer. Your goal is to write a Python function, 'preprocess_prompt', that will preprocess the prompt from a prompt string for a LLM. This will use regular expressions to preprocess specific XML-like tags, if any, in the prompt. All output to the console will be pretty print using the Python rich library.\n\n% Here are the inputs and outputs of the function:\n    Input: \n        'prompt' - A string that is the prompt to preprocess\n        'recursive' - A boolean that is True if the program needs to recursively process the includes in the prompt and False if it does not need to recursively process the prompt. Default is True.\n        'double_curly_brackets' - A boolean that is True if the curly brackets need to be doubled and False if they do not need to be doubled. Default is True.\n        'exclude_keys' - An optional list of strings that are excluded from the curly bracket doubling.\n    Output: returns a string that is the preprocessed prompt, with any leading or trailing whitespace removed.\n\n% Here are the XML-like tags to preprocess, other tags will remain unmodified:\n    'include' - This tag will include the content of the file indicated in the include tag. The 'include tag' will be directly replaced with the content of the file in the prompt, without wrapping it in a new tag.\n    'pdd' - This tag indicates a comment and anything in this XML will be deleted from the string including the 'pdd' tags themselves.\n    'shell' - This tag indicates that there are shell commands to run. Capture all output of the shell commands and include it in the prompt but remove the shell tags.\n\n% Includes can be nested, that is there can be includes inside of the files of the includes and 'preprocess' should be called recursively on these include files if recursive is True. There are two ways of having includes in the prompt:\n    1. Will check to see if the file has any angle brackets in triple backticks. If so, it will read the included file indicated in the angle brackets and replace the angle brackets with the content of the included file. This will be done recursively until there are no more angle brackets in triple backticks. The program will then remove the angle brackets but leave the contents in the triple backticks.\n    2. The XML 'include' mentioned above.\n\n% If double_curly_brackets is True, the program will check to see if the file has any single curly brackets and if it does and the string in the curly brackets are not in the exclude_keys list, it will check to see if the curly brackets are already doubled before doubling the curly brackets.\n\n% The program should resolve file paths using the PDD_PATH environment variable. Implement a function 'get_file_path' that takes a file name and returns the full path using this environment variable.\n\n% Keep the user informed of the progress of the program by pretty printing messages."}, {"PROMPT_NAME": "unfinished_prompt_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python engineer. Your goal is to write a python function called 'unfinished_prompt' that will determine if a given prompt is complete or needs to continue.\n\n% Here are the inputs and outputs of the function:\n    Inputs:\n        'prompt_text' - A string containing the prompt text to analyze.\n        'strength' - A float that is the strength of the LLM model to use for the analysis. Default is 0.5.\n        'temperature' - A float that is the temperature of the LLM model to use for the analysis. Default is 0.\n    Outputs:\n        'reasoning' - A string containing the structured reasoning for the completeness assessment.\n        'is_finished' - A boolean indicating whether the prompt is complete (True) or incomplete (False).\n        'total_cost' - A float that is the total cost of the analysis function. This is an optional output.\n        'model_name' - A string that is the name of the LLM model used for the analysis. This is an optional output.\n\n% Here is an example of a Langchain LCEL program: ```<./context/langchain_lcel_example.py>```\n\n% Here is an example how to select the Langchain llm and count tokens: ```<./context/llm_selector_example.py>```\n\n% Note: Use relative import for 'llm_selector' to ensure compatibility within the package structure (i.e. 'from .llm_selector') instead of 'from pdd.llm_selector'.\n\n% This function will do the following:\n    Step 1. Use $PDD_PATH environment variable to get the path to the project. Load the '$PDD_PATH/prompts/unfinished_prompt_LLM.prompt' file.\n    Step 2. Create a Langchain LCEL template from unfinished_prompt_LLM prompt so that it returns a JSON output.\n    Step 3. Use the llm_selector function for the LLM model.\n    Step 4. Run the prompt text through the model using Langchain LCEL.\n        4a. Pass the following string parameters to the prompt during invoke:\n            - 'PROMPT_TEXT'\n        4b. Pretty print a message letting the user know it is running and how many tokens (using token_counter function from llm_selector) are in the prompt and the cost. The cost from llm_selector is in dollars per million tokens.\n        4c. The dictionary output of the LCEL will have the keys 'reasoning' and 'is_finished'. Be sure to access these keys using the get method with default error messages.\n        4d. Pretty print the reasoning and completion status using the rich library. Also, print the number of tokens in the result, the output token cost and the total_cost.\n    Step 5. Return the 'reasoning' string and 'is_finished' boolean from the JSON output using 'get', and the 'total_cost' float, and 'model_name' string.\n\n% Ensure that the function handles potential errors gracefully, such as missing input parameters or issues with the LLM model responses.\n\n"}, {"PROMPT_NAME": "xml_tagger_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python engineer. Your goal is to write a Python function, \"xml_tagger\", that will enhance a given LLM prompt by adding XML tags to improve its structure and readability.\n\n% You are an expert Python engineer.
+
+% Code Style Requirements
+- File must start with `from __future__ import annotations`.
+- All functions must be fully type-hinted.
+- Use `rich.console.Console` for all printing.
+
+% Package Structure
+- The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name').
+- The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```
+
+% Error Handling
+- Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.\n\n% Here are the inputs and outputs of the function:\n    Input: \n        'raw_prompt' - A string containing the prompt that needs XML tagging to improve its organization and clarity.\n        'strength' - A float value representing the strength parameter for the LLM model.\n        'temperature' - A float value representing the temperature parameter for the LLM model.\n    Output: \n        'xml_tagged' - A string containing the prompt with properly added XML tags.\n        'total_cost' - A float representing the total cost of running the LCELs.\n        'model_name' - A string representing the name of the selected LLM model.\n\n% Here is an example of a LangChain Expression Language (LCEL) program: <lcel_example>import os
+from langchain_core.prompts import PromptTemplate
+from langchain_community.cache import SQLiteCache
+from langchain_community.llms.mlx_pipeline import MLXPipeline
+from langchain.globals import set_llm_cache
+from langchain_core.output_parsers import JsonOutputParser, PydanticOutputParser # Parsers are only avaiable in langchain_core.output_parsers not langchain.output_parsers
+from langchain_core.output_parsers import StrOutputParser
+from langchain_core.prompts import ChatPromptTemplate
+from langchain_core.runnables import RunnablePassthrough, ConfigurableField
+
+from langchain_openai import AzureChatOpenAI
+from langchain_fireworks import Fireworks 
+from langchain_anthropic import ChatAnthropic
+from langchain_openai import ChatOpenAI # Chatbot and conversational tasks
+from langchain_openai import OpenAI # General language tasks
+from langchain_google_genai import ChatGoogleGenerativeAI
+from langchain_google_vertexai import ChatVertexAI
+from langchain_groq import ChatGroq
+from langchain_together import Together
+
+from langchain.callbacks.base import BaseCallbackHandler
+from langchain.schema import LLMResult
+
+import json
+
+from langchain_community.chat_models.mlx import ChatMLX
+from langchain_core.messages import HumanMessage
+
+from langchain_ollama.llms import OllamaLLM
+from langchain_aws import ChatBedrockConverse
+
+# Define a base output parser (e.g., PydanticOutputParser)
+from pydantic import BaseModel, Field
+
+
+
+class CompletionStatusHandler(BaseCallbackHandler):
+    def __init__(self):
+        self.is_complete = False
+        self.finish_reason = None
+        self.input_tokens = None
+        self.output_tokens = None
+
+    def on_llm_end(self, response: LLMResult, **kwargs) -> None:
+        self.is_complete = True
+        if response.generations and response.generations[0]:
+            generation = response.generations[0][0]
+            self.finish_reason = generation.generation_info.get('finish_reason').lower()
+            
+            # Extract token usage
+            if hasattr(generation.message, 'usage_metadata'):
+                usage_metadata = generation.message.usage_metadata
+                self.input_tokens = usage_metadata.get('input_tokens')
+                self.output_tokens = usage_metadata.get('output_tokens')
+        # print("response:",response)
+        print("Extracted information:")
+        print(f"Finish reason: {self.finish_reason}")
+        print(f"Input tokens: {self.input_tokens}")
+        print(f"Output tokens: {self.output_tokens}")
+
+# Set up the LLM with the custom handler
+handler = CompletionStatusHandler()
+# Always setup cache to save money and increase speeds
+set_llm_cache(SQLiteCache(database_path=".langchain.db"))
+
+
+# Create the LCEL template. Make note of the variable {topic} which will be filled in later.
+prompt_template = PromptTemplate.from_template("Tell me a joke about {topic}")
+
+llm = ChatGoogleGenerativeAI(model="gemini-2.5-pro-exp-03-25", temperature=0, callbacks=[handler])
+# Combine with a model and parser to output a string
+chain = prompt_template |llm| StrOutputParser()
+
+# Run the template. Notice that the input is a dictionary with a single key "topic" which feeds it into the above prompt template. This is needed because the prompt template has a variable {topic} which needs to be filled in when invoked.
+result = chain.invoke({"topic": "cats"})
+print("********Google:", result)
+
+
+llm = ChatVertexAI(model="gemini-2.5-pro-exp-03-25", temperature=0, callbacks=[handler])
+# Combine with a model and parser to output a string
+chain = prompt_template |llm| StrOutputParser()
+
+# Run the template. Notice that the input is a dictionary with a single key "topic" which feeds it into the above prompt template. This is needed because the prompt template has a variable {topic} which needs to be filled in when invoked.
+result = chain.invoke({"topic": "cats"})
+print("********GoogleVertex:", result)
+
+
+# Define your desired data structure.
+class Joke(BaseModel):
+    setup: str = Field(description="question to set up a joke")
+    punchline: str = Field(description="answer to resolve the joke")
+
+
+# Set up a parser
+parser = JsonOutputParser(pydantic_object=Joke)
+
+# Create a prompt template
+prompt = PromptTemplate(
+    template="Answer the user query.\n{format_instructions}\n{query}\n",
+    input_variables=["query"],
+    partial_variables={"format_instructions": parser.get_format_instructions()},
+)
+
+llm_no_struct = ChatOpenAI(model="gpt-4o-mini", temperature=0, 
+                           callbacks=[handler]) 
+llm = llm_no_struct.with_structured_output(Joke) # with structured output forces the output to be a specific object, in this case Joke. Only OpenAI models have structured output
+# Chain the components. 
+#  The class `LLMChain` was deprecated in LangChain 0.1.17 and will be removed in 1.0. Use RunnableSequence, e.g., `prompt | llm` instead.
+chain = prompt | llm 
+
+# Invoke the chain with a query. 
+# IMPORTANT: chain.run is now obsolete. Use chain.invoke instead.
+result = chain.invoke({"query": "Tell me a joke about openai."})
+print("4o mini JSON: ",result)
+print(result.setup) # How to access the structured output
+
+llm = ChatOpenAI(model="o1", temperature=1, 
+                           callbacks=[handler],model_kwargs = {"max_completion_tokens" : 1000})
+# Chain the components. 
+#  The class `LLMChain` was deprecated in LangChain 0.1.17 and will be removed in 1.0. Use RunnableSequence, e.g., `prompt | llm` instead.
+chain = prompt | llm | parser
+
+# Invoke the chain with a query. 
+# IMPORTANT: chain.run is now obsolete. Use chain.invoke instead.
+result = chain.invoke({"query": "Tell me a joke about openai."})
+print("o1 JSON: ",result)
+
+# Get DEEPSEEK_API_KEY environmental variable
+
+deepseek_api_key = os.getenv('DEEPSEEK_API_KEY')
+
+# Ensure the API key is retrieved successfully
+if deepseek_api_key is None:
+    raise ValueError("DEEPSEEK_API_KEY environment variable is not set")
+
+llm = ChatOpenAI(
+    model='deepseek-chat', 
+    openai_api_key=deepseek_api_key, 
+    openai_api_base='https://api.deepseek.com',
+    temperature=0, callbacks=[handler]
+)
+
+# Chain the components
+chain = prompt | llm | parser
+
+# Invoke the chain with a query
+result = chain.invoke({"query": "Write joke about deepseek."})
+print("deepseek",result)
+
+
+# Set up a parser
+parser = PydanticOutputParser(pydantic_object=Joke)
+# Chain the components
+chain = prompt | llm | parser
+
+# Invoke the chain with a query
+result = chain.invoke({"query": "Write joke about deepseek and pydantic."})
+print("deepseek pydantic",result)
+
+# Set up the Azure ChatOpenAI LLM instance
+llm_no_struct = AzureChatOpenAI(
+    model="o4-mini",
+    temperature=1,
+    callbacks=[handler]
+)
+llm = llm_no_struct.with_structured_output(Joke) # with structured output forces the output to be a specific JSON format
+# Chain the components: prompt | llm | parser
+chain = prompt | llm # returns a Joke object
+
+# Invoke the chain with a query
+result = chain.invoke({"query": "What is Azure?"})  # Pass a dictionary if `invoke` expects it
+print("Azure Result:", result)
+
+# Set up a parser
+parser = JsonOutputParser(pydantic_object=Joke)
+
+llm = Fireworks(
+    model="accounts/fireworks/models/llama4-maverick-instruct-basic",
+    temperature=0, callbacks=[handler])
+# Chain the components
+chain = prompt | llm | parser
+
+# Invoke the chain with a query
+# no money in account
+# result = chain.invoke({"query": "Tell me a joke about the president"})
+# print("fireworks",result)
+
+
+
+
+
+prompt = ChatPromptTemplate.from_template(
+    "Tell me a short joke about {topic}"
+)
+chat_openai = ChatOpenAI(model="gpt-3.5-turbo", callbacks=[handler])
+openai = OpenAI(model="gpt-3.5-turbo-instruct", callbacks=[handler])
+anthropic = ChatAnthropic(model="claude-2", callbacks=[handler])
+model = (
+    chat_openai
+    .with_fallbacks([anthropic])
+    .configurable_alternatives(
+        ConfigurableField(id="model"),
+        default_key="chat_openai",
+        openai=openai,
+        anthropic=anthropic,
+    )
+)
+
+chain = (
+    {"topic": RunnablePassthrough()} 
+    | prompt 
+    | model 
+    | StrOutputParser()
+)
+result = chain.invoke({"topic": "Tell me a joke about the president"})
+print("config alt:",result)
+
+
+
+llm = ChatAnthropic(
+    model="claude-3-7-sonnet-latest",
+    max_tokens=5000,  # Total tokens for the response
+    thinking={"type": "enabled", "budget_tokens": 2000},  # Tokens for internal reasoning
+)
+
+response = llm.invoke("What is the cube root of 50.653?")
+print(json.dumps(response.content, indent=2))
+
+
+llm = ChatGroq(temperature=0, model_name="qwen-qwq-32b", callbacks=[handler])
+system = "You are a helpful assistant."
+human = "{text}"
+prompt = ChatPromptTemplate.from_messages([("system", system), ("human", human)])
+
+chain = prompt | llm | StrOutputParser()
+print(chain.invoke({"text": "Explain the importance of low latency LLMs."}))
+
+
+llm = Together(
+    model="meta-llama/Llama-3-70b-chat-hf",
+    max_tokens=500, callbacks=[handler]
+)
+chain = prompt | llm | StrOutputParser()
+print(chain.invoke({"text": "Explain the importance of together.ai."}))
+
+
+# Define a prompt template with placeholders for variables
+prompt_template = PromptTemplate.from_template("Tell me a {adjective} joke about {content}.")
+
+# Format the prompt with the variables
+formatted_prompt = prompt_template.format(adjective="funny", content="data scientists")
+
+# Print the formatted prompt
+print(formatted_prompt)
+
+
+# Set up the LLM with the custom handler
+handler = CompletionStatusHandler()
+
+
+llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.9, callbacks=[handler])
+
+prompt = PromptTemplate.from_template("What is a good name for a company that makes {product}?")
+
+chain = prompt | llm
+
+# Invoke the chain
+response = chain.invoke({"product":"colorful socks"})
+
+# Check completion status
+print(f"Is complete: {handler.is_complete}")
+print(f"Finish reason: {handler.finish_reason}")
+print(f"Response: {response}")
+print(f"Input tokens: {handler.input_tokens}")
+print(f"Output tokens: {handler.output_tokens}")
+
+
+
+template = """Question: {question}"""
+
+prompt = ChatPromptTemplate.from_template(template)
+
+model = OllamaLLM(model="qwen2.5-coder:32b")
+
+chain = prompt | model
+
+output = chain.invoke({"question": "Write a python function that calculates Pi"})
+print(output)
+
+
+
+llm = MLXPipeline.from_model_id(
+    "mlx-community/quantized-gemma-2b-it",
+    pipeline_kwargs={"max_tokens": 10, "temp": 0.1},
+)
+
+
+chat_model = ChatMLX(llm=llm)
+messages = [HumanMessage(content="What happens when an unstoppable force meets an immovable object?")]
+response = chat_model.invoke(messages)
+print(response.content)
+
+
+
+llm = ChatBedrockConverse(
+    model_id="anthropic.claude-3-5-sonnet-20240620-v1:0",
+    # Additional parameters like temperature, max_tokens can be set here
+)
+
+messages = [HumanMessage(content="What happens when an unstoppable force meets an immovable sonnet?")]
+response = llm.invoke(messages)
+print(response.content)</lcel_example>\n\n% Here are examples of how to use internal modules:\n<internal_example_modules>\n    % Example of selecting a Langchain LLM and counting tokens using llm_selector: <llm_selector_example>from pdd.llm_selector import llm_selector
+
+def main() -> None:
+    """
+    Main function to demonstrate the usage of the llm_selector function.
+    """
+    # Define the strength and temperature parameters
+    strength: float = 0.5  # Example strength value for the LLM model
+    temperature: float = 1.0  # Example temperature value for the LLM model
+
+    try:       
+        while strength <= 1.1: 
+            # Call the llm_selector function with the specified strength and temperature
+            llm, token_counter, input_cost, output_cost, model_name = llm_selector(strength, temperature)
+            print(f"Strength: {strength}")
+            
+            # Print the details of the selected LLM model
+            print(f"Selected LLM Model: {model_name}")
+            print(f"Input Cost per Million Tokens: {input_cost}")
+            print(f"Output Cost per Million Tokens: {output_cost}")
+
+            # Example usage of the token counter function
+            sample_text: str = "This is a sample text to count tokens."
+            token_count: int = token_counter(sample_text)
+            print(f"Token Count for Sample Text: {token_count}")
+            print(f"model_name: {model_name}")
+            strength += 0.05
+    except FileNotFoundError as e:
+        print(f"Error: {e}")
+    except ValueError as e:
+        print(f"Error: {e}")
+
+if __name__ == "__main__":
+    main()</llm_selector_example>\n</internal_example_modules>\n\n% This program will use Langchain to do the following:\n    Step 1. Use $PDD_PATH environment variable to get the path to the project. Load the '$PDD_PATH/prompts/xml_convertor_LLM.prompt' and '$PDD_PATH/prompts/extract_xml_LLM.prompt' files.\n    Step 2. Create a Langchain LCEL template from xml_convertor prompt so that it returns a string output.\n    Step 3. Use the llm_selector function for the LLM model and token counting.\n    Step 4. Run the code through the model using Langchain LCEL. \n        4a. Pass the following string parameters to the prompt during invoke:\n            - 'raw_prompt'\n        4b. Pretty print a message letting the user know it is running and how many tokens (using token_counter from llm_selector) are in the prompt and the cost. The cost from llm_selector is in dollars per million tokens. \n        4c. The string output of the LCEL will be 'xml_generated_analysis' that contains the tagged prompt.\n    Step 5. The code result of the model will contain a mix of text and XML separated by triple backticks. Create a Langchain LCEL template but with a llm_selector with strength .8 from the extract_xml prompt that has a JSON output.\n        5a. Pass the following string parameters to the prompt during invoke:\n            - 'xml_generated_analysis'\n        5b. Pretty print a message letting the user know it is running and how many tokens (using token_counter from llm_selector) are in the prompt and the cost. The cost from llm_selector is in dollars per million tokens.\n        5c. The JSON output of the LCEL will have the key 'xml_tagged' that contains the extracted tagged prompt.\n    Step 6. Pretty print the extracted tagged prompt using the rich Markdown function. Also, print the number of tokens in the result and the cost.\n    Step 7. Calculate the total cost by summing the costs from both LCEL runs.\n    Step 8. Return the 'xml_tagged' string using 'get', the 'total_cost' and 'model_name'."}]
         </prompt_list_example>
 
         <change_description_example>
-        <include>context/detect_change/2/change.prompt</include>
+        % Use context/python_preamble.prompt to make prompts more compact. Some prompts might already have this.
+
+% Here is what is inside context/python_preamble.prompt:<preamble>% You are an expert Python engineer.
+
+% Code Style Requirements
+- File must start with `from __future__ import annotations`.
+- All functions must be fully type-hinted.
+- Use `rich.console.Console` for all printing.
+
+% Package Structure
+- The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name').
+- The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```
+
+% Error Handling
+- Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.</preamble>
+
+% Here is an example of this being done: <example>% You are an expert Python engineer. Your goal is to write a python function, "code_generator", that will compile a prompt into a code file. 
+
+You are an expert Python engineer working on the PDD Cloud project.
+
+-------------------------------------------------------------------------------
+Python Coding Standards (PDD Cloud)
+-------------------------------------------------------------------------------
+
+## Style
+- Python 3.12, PEP 8 compliant
+- Type hints required (from typing import Optional, List, Dict, Any, Tuple)
+- Imports: standard lib -> third-party -> local (alphabetical within groups)
+
+## Critical First Import
+import function_import_setup  # MUST be first import
+
+This enables subprocess module resolution for Firebase Functions Framework.
+All endpoint files must have this as their very first import statement.
+
+## Standard Library Imports
+import os
+import logging
+
+## Error Handling Pattern
+Map exceptions to HTTP status codes:
+- AuthenticationError -> 401 Unauthorized
+- AuthorizationError -> 403 Forbidden
+- ValidationError -> 400 Bad Request
+- ResourceNotFoundError -> 404 Not Found
+- DatabaseError / Exception -> 500 Internal Server Error
+
+All error responses: {"error": "descriptive message"}
+
+## Logging
+logger = logging.getLogger(__name__)
+
+## Response Format
+Return tuple: (response_dict, status_code)
+
+
+% Here are the inputs and outputs of the function:
+    Inputs: 
+        'prompt' - A string containing the raw prompt to be processed.
+        'language' - A string that is the language type (e.g. python, bash) of file that will be outputed by the LLM.
+        'strength' - A float between 0 and 1 that is the strength of the LLM model to use.
+        'temperature' - A float that is the temperature of the LLM model to use. Default is 0.
+        'time' - A float in [0,1] or None that controls the thinking effort for the LLM model, passed to llm_invoke. Default is DEFAULT_TIME. If None, treat as DEFAULT_TIME before invoking llm_invoke.
+        'verbose' - A boolean that indicates whether to print out the details of the function. Default is False.
+        'preprocess_prompt' - A boolean that indicates whether to preprocess the prompt. Default is True.
+    Outputs:
+        'runnable_code' - A string that is runnable code
+        'total_cost' - A float that is the total cost of all LLM calls within this function (initial generation, unfinished check, continuation if used, and postprocess)
+        'model_name' - A string that is the name of the selected LLM model used for the main generation (or continuation). Postprocess may use a different model internally and does not change this value.
+
+% Here is how to use the internal modules:
+    <internal_modules>
+        For running prompts with llm_invoke:
+        <llm_invoke_example>
+            [File not found: ../context/llm_invoke_example.py]
+        </llm_invoke_example>
+
+        For preprocessing prompts:
+        <preprocess_example>
+            [File not found: ../context/preprocess_example.py]
+        </preprocess_example>
+
+        For handling unfinished prompts:
+        <unfinished_prompt_example>
+            [File not found: ../context/unfinished_prompt_example.py]
+        </unfinished_prompt_example>
+
+        For continuing generation:
+        <continue_generation_example>
+            [File not found: ../context/continue_generation_example.py]
+        </continue_generation_example>
+
+        For postprocessing results:
+        <postprocess_example>
+            [File not found: ../context/postprocess_example.py]
+        </postprocess_example>
+    </internal_modules>
+
+% This program will do the following:
+    Step 1. Conditionally preprocess the raw prompt using the preprocess function from the preprocess module based on the value of 'preprocess_prompt'. If 'preprocess_prompt' is True, preprocess the prompt; otherwise, use the raw prompt directly. When preprocessing, it is acceptable to enable options such as double_curly_brackets=True and recursive=False to preserve placeholders and avoid over-expansion.
+
+    Step 2. Generate the initial response as follows:
+        - If the prompt contains embedded data URLs (e.g., 'data:image/...;base64,...'), split the prompt into alternating text and image parts (preserving order) and call llm_invoke with messages=[{role: 'user', content: [{type: 'image_url', image_url: {url: ...}}, {type: 'text', text: ...}, ...]}] and the provided strength, temperature, time, and verbose.
+        - Otherwise, call llm_invoke with the (preprocessed or raw) prompt, input_json={}, and the provided strength, temperature, time, and verbose.
+
+    Step 3. Detect if the generation is incomplete using the unfinished_prompt function (use strength=0.5, temperature=0.0) by passing in the last 600 characters of the output of Step 2. Pass through language, time, and verbose.
+        - a. If incomplete, call the continue_generation function to complete the generation and set final_output to that result.
+        - b. Else, set final_output to the initial model output.
+
+    Step 4. Postprocess the final_output using the postprocess function from the postprocess module with the EXTRACTION_STRENGTH constant. Use temperature=0.0 and pass through language, time, and verbose.
+
+    Step 5. Return the runnable_code, total_cost and model_name.
+
+% Validation and defaults:
+    - Validate non-empty 'prompt' and 'language'.
+    - Enforce 0 ≤ strength ≤ 1 and 0 ≤ temperature ≤ 2.
+    - Enforce 0 ≤ time ≤ 1 when provided; if time is None, substitute DEFAULT_TIME before llm_invoke.
+</example>
         </change_description_example>
     </input_example>
 
     <output_example>
-        <include>context/detect_change/2/detect_change_output.txt</include>
+        1. **Impact of the Change and Potential Issues:**
+
+   The change description suggests using the contents of `context/python_preamble.prompt` to make prompts more compact. This preamble includes instructions for using relative imports, pretty printing with the Python Rich library, and handling edge cases. The main impact is to ensure consistency across prompts by centralizing common instructions, which can reduce redundancy and improve maintainability. Potential issues include ensuring that the preamble does not conflict with existing instructions in the prompts and that it is inserted in a way that maintains the logical flow of the prompt.
+
+2. **Possible Implementation Plans:**
+
+   - **Plan A: Direct Inclusion of Preamble**
+     - **Pros:** Simplifies prompts by removing redundant instructions and directly including the preamble.
+     - **Cons:** May require significant restructuring of prompts to ensure logical flow and avoid redundancy.
+   
+   - **Plan B: Conditional Inclusion Based on Existing Content**
+     - **Pros:** Only includes the preamble if similar instructions are not already present, minimizing disruption.
+     - **Cons:** Requires careful analysis of each prompt to determine if the preamble is needed, which can be time-consuming.
+   
+   - **Plan C: Hybrid Approach**
+     - **Pros:** Combines the benefits of both plans by including the preamble where it adds value and retaining existing instructions where they are more specific or detailed.
+     - **Cons:** Complexity in deciding which parts of the preamble to include or exclude for each prompt.
+
+3. **Analysis and Selection of the Best Plan:**
+
+   Plan C is the best approach as it allows for flexibility and ensures that the preamble is only included where it enhances the prompt. This minimizes redundancy and maintains the integrity of prompts that already have specific instructions. It also allows for a more tailored approach, ensuring that the preamble complements rather than conflicts with existing content.
+
+4. **Prompt Analysis Based on Selected Plan:**
+
+   - **change_python.prompt:** This prompt already contains detailed instructions similar to those in the preamble. However, it can benefit from the preamble for consistency in handling edge cases and pretty printing.
+   - **preprocess_python.prompt:** This prompt can include the preamble to ensure consistent handling of edge cases and pretty printing.
+   - **unfinished_prompt_python.prompt:** This prompt can include the preamble to ensure consistent handling of edge cases and pretty printing.
+   - **xml_tagger_python.prompt:** This prompt already includes the preamble, so no changes are needed.
+
+5. **Prompts That Need to Be Changed:**
+
+   - **change_python.prompt**
+     - **Instructions:**
+       - Insert the contents of the file `./context/python_preamble.prompt` immediately after the role and goal statement using 'include' XML tags.
+       - Remove any redundant instructions that are covered by the preamble, such as those related to pretty printing and handling edge cases.
+       - Ensure that the logical flow of the prompt is maintained and that any unique instructions specific to this prompt are retained.
+       - Here is what is inside context/python_preamble.prompt:<preamble>% You are an expert Python engineer.
+
+% Code Style Requirements
+- File must start with `from __future__ import annotations`.
+- All functions must be fully type-hinted.
+- Use `rich.console.Console` for all printing.
+
+% Package Structure
+- The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name').
+- The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```
+
+% Error Handling
+- Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.</preamble>
+
+   - **preprocess_python.prompt**
+     - **Instructions:**
+       - Insert the contents of the file `./context/python_preamble.prompt` immediately after the role and goal statement using 'include' XML tags.
+       - Remove any redundant instructions that are covered by the preamble, such as those related to pretty printing and handling edge cases.
+       - Ensure that the logical flow of the prompt is maintained and that any unique instructions specific to this prompt are retained.
+       - Here is what is inside context/python_preamble.prompt:<preamble>% You are an expert Python engineer.
+
+% Code Style Requirements
+- File must start with `from __future__ import annotations`.
+- All functions must be fully type-hinted.
+- Use `rich.console.Console` for all printing.
+
+% Package Structure
+- The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name').
+- The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```
+
+% Error Handling
+- Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.</preamble>
+
+   - **unfinished_prompt_python.prompt**
+     - **Instructions:**
+       - Insert the contents of the file `./context/python_preamble.prompt` immediately after the role and goal statement using 'include' XML tags.
+       - Remove any redundant instructions that are covered by the preamble, such as those related to pretty printing and handling edge cases.
+       - Ensure that the logical flow of the prompt is maintained and that any unique instructions specific to this prompt are retained.
+       - Here is what is inside context/python_preamble.prompt:<preamble>% You are an expert Python engineer.
+
+% Code Style Requirements
+- File must start with `from __future__ import annotations`.
+- All functions must be fully type-hinted.
+- Use `rich.console.Console` for all printing.
+
+% Package Structure
+- The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name').
+- The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```
+
+% Error Handling
+- Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.</preamble>
     </output_example>
 </example>
 

pdd/prompts/example_generator_LLM.prompt

@@ -4,7 +4,28 @@
 
 % The language of the example should be in: <language_for_example>{language}</language_for_example>
 
+% File path information:
+ - The code module file is located at: <code_module_file_path>{source_file_path}</code_module_file_path>
+ - The example file will be saved at: <example_file_path>{example_file_path}</example_file_path>
+ - The module name (without extension) is: <module_name>{module_name}</module_name>
+
+% IMPORT INSTRUCTIONS: Use the appropriate import mechanism for the target language
+ - CRITICAL: Use the exact module_name provided in the file path information above - DO NOT use generic names like "greetings" or "module"
+ - Avoid package-style imports unless the file is actually in a package structure
+ - Import the specific functions/classes that are defined in the code module
+ - CRITICAL: Do not assume module names - use the exact module_name provided
+ - CRITICAL: Import the appropriate functions/classes from the module (e.g., if module_name is "simple_math", import "add", "subtract", etc.)
+ - CRITICAL: Never include hardcoded absolute paths like "/Users/username/project/examples/" in comments or code
+    - Use only relative path descriptions in comments (e.g., "relative to project root" or "relative to this script")
+    - Make the example portable across different development environments
+    - Use dynamic path resolution with os.path.dirname(__file__) and relative path construction
+    - In comments, describe file structure using relative terms, not absolute paths
+ - CRITICAL: The 'extracted_code' field must preserve all newlines using proper JSON escaping (\\n). The output must be valid JSON where multi-line code is represented with \\n escape sequences for each line break.
+
 % Make sure the following happens:
     - Document in detail what the input and output parameters in the doc strings
-    - Someone needs to be able to fully understand how to use the module from the example.
+    - Someone needs to be able to fully understand how to use the module from the example.        
+    - Use correct import statements based on the actual file structure
+    - The example should be a complete, runnable script that imports from the actual module
+    - Include proper file path handling and module discovery if needed
 <include>./context/example.prompt</include>

pdd/prompts/extract_code_LLM.prompt

@@ -19,4 +19,8 @@
 % Output a JSON object with the following keys:
     - 'focus': String containing the focus of the generation.
     - 'explanation': String explanation of why this block_type was the focus of the generation and explain any errors detected in the code, if a code type of block.
-    - 'extracted_code': String containing the entire generated and corrected block_type of focus.
+    - 'extracted_code': String containing the entire generated and corrected block_type of focus.
+
+% CRITICAL: The 'extracted_code' field MUST preserve all newlines and indentation from the original code. In JSON, newlines must be represented as the escape sequence \n (backslash-n). Each line break in the code must appear as \n in the JSON string value. Do NOT concatenate lines together or remove line breaks. The extracted code must be properly formatted and runnable.
+
+% IMPORTANT: Output ONLY the JSON object. Do not add any trailing whitespace or newlines after the closing brace.