pdd-cli 0.0.44__py3-none-any.whl → 0.0.45__py3-none-any.whl

pdd/prompts/change_LLM.prompt

@@ -15,9 +15,7 @@
 </inputs_outputs_definitions>
 
 <change_prompt_examples>
-    <include>
-        ../prompts/xml/change_example_partial_processed.prompt
-    </include>
+    [File not found: ../prompts/xml/change_example_partial_processed.prompt]
 </change_prompt_examples>
 
 <context>

pdd/prompts/detect_change_LLM.prompt

@@ -16,16 +16,473 @@
 <example>
     <input_example>
         <prompt_list_example>
-        <include>context/detect_change/2/prompt_list.json</include>
+        [{"PROMPT_NAME": "change_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python Software Engineer. Your goal is to write a Python function, \"change\", that will modify an input_prompt into a modified_prompt per the change_prompt. All output to the console will be pretty printed using the Python Rich library. Ensure that the module imports are done using relative imports.\n\n% Here are the inputs and outputs of the function:\n    Inputs:\n        - 'input_prompt' - A string that contains the prompt that will be modified by the change_prompt.\n        - 'input_code' - A string that contains the code that was generated from the input_prompt.\n        - 'change_prompt' - A string that contains the instructions of how to modify the input_prompt.\n        - 'strength': A float value representing the strength parameter for the LLM model, used to influence the model's behavior.\n        - 'temperature': A float value representing the temperature parameter for the LLM model, used to control the randomness of the model's output.\n    Outputs:\n        - 'modified_prompt' - A string that contains the modified prompt that was changed based on the change_prompt.\n        - 'total_cost': A float value representing the total cost of running the function.\n        - 'model_name': A string representing the name of the selected LLM model.\n\n% Here is an example how to preprocess the prompt from a file: ```<./context/preprocess_example.py>```\n\n% Example usage of the Langchain LCEL program: ```<./context/langchain_lcel_example.py>```\n\n% Example of selecting a Langchain LLM and counting tokens using llm_selector: ```<./context/llm_selector_example.py>```\n\n% Steps to be followed by the function:\n    1. Load the '$PDD_PATH/prompts/xml/change_LLM.prompt' and '$PDD_PATH/prompts/extract_prompt_change_LLM.prompt' files.\n    2. Preprocess the change_LLM prompt using the preprocess function from the preprocess module and set double_curly_brackets to false.\n    3. Create a Langchain LCEL template from the processed change_LLM prompt to return a string output.    \n    4. Use the llm_selector function for the LLM model and token counting.\n    5. Run the input_prompt through the model using Langchain LCEL:\n        - a. Pass the following string parameters to the prompt during invocation:             \n            * 'input_prompt'\n            * 'input_code'\n            * 'change_prompt' (preprocess this with double_curly_brackets set to false)\n        - b. Calculate the input and output token count using token_counter from llm_selector and pretty print the output of 4a, including the token count and estimated cost. The cost from llm_selector is in dollars per million tokens.\n    6. Create a Langchain LCEL template with strength .9 from the extract_prompt_change_LLM prompt that outputs JSON:\n        - a. Pass the following string parameters to the prompt during invocation: 'llm_output' (this string is from Step 4).\n        - b. Calculate input and output token count using token_counter from llm_selector and pretty print the running message with the token count and cost.\n        - c. Use 'get' function to extract 'modified_prompt' key values using from the dictionary output.\n    7. Pretty print the extracted modified_prompt using Rich Markdown function. Include token counts and costs.\n    8. Return the 'modified_prompt' string, the total_cost of both invokes and model_name use for the change_LLM prompt.\n\n% Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages."}, {"PROMPT_NAME": "preprocess_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python engineer. Your goal is to write a Python function, 'preprocess_prompt', that will preprocess the prompt from a prompt string for a LLM. This will use regular expressions to preprocess specific XML-like tags, if any, in the prompt. All output to the console will be pretty print using the Python rich library.\n\n% Here are the inputs and outputs of the function:\n    Input: \n        'prompt' - A string that is the prompt to preprocess\n        'recursive' - A boolean that is True if the program needs to recursively process the includes in the prompt and False if it does not need to recursively process the prompt. Default is True.\n        'double_curly_brackets' - A boolean that is True if the curly brackets need to be doubled and False if they do not need to be doubled. Default is True.\n        'exclude_keys' - An optional list of strings that are excluded from the curly bracket doubling.\n    Output: returns a string that is the preprocessed prompt, with any leading or trailing whitespace removed.\n\n% Here are the XML-like tags to preprocess, other tags will remain unmodified:\n    'include' - This tag will include the content of the file indicated in the include tag. The 'include tag' will be directly replaced with the content of the file in the prompt, without wrapping it in a new tag.\n    'pdd' - This tag indicates a comment and anything in this XML will be deleted from the string including the 'pdd' tags themselves.\n    'shell' - This tag indicates that there are shell commands to run. Capture all output of the shell commands and include it in the prompt but remove the shell tags.\n\n% Includes can be nested, that is there can be includes inside of the files of the includes and 'preprocess' should be called recursively on these include files if recursive is True. There are two ways of having includes in the prompt:\n    1. Will check to see if the file has any angle brackets in triple backticks. If so, it will read the included file indicated in the angle brackets and replace the angle brackets with the content of the included file. This will be done recursively until there are no more angle brackets in triple backticks. The program will then remove the angle brackets but leave the contents in the triple backticks.\n    2. The XML 'include' mentioned above.\n\n% If double_curly_brackets is True, the program will check to see if the file has any single curly brackets and if it does and the string in the curly brackets are not in the exclude_keys list, it will check to see if the curly brackets are already doubled before doubling the curly brackets.\n\n% The program should resolve file paths using the PDD_PATH environment variable. Implement a function 'get_file_path' that takes a file name and returns the full path using this environment variable.\n\n% Keep the user informed of the progress of the program by pretty printing messages."}, {"PROMPT_NAME": "unfinished_prompt_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python engineer. Your goal is to write a python function called 'unfinished_prompt' that will determine if a given prompt is complete or needs to continue.\n\n% Here are the inputs and outputs of the function:\n    Inputs:\n        'prompt_text' - A string containing the prompt text to analyze.\n        'strength' - A float that is the strength of the LLM model to use for the analysis. Default is 0.5.\n        'temperature' - A float that is the temperature of the LLM model to use for the analysis. Default is 0.\n    Outputs:\n        'reasoning' - A string containing the structured reasoning for the completeness assessment.\n        'is_finished' - A boolean indicating whether the prompt is complete (True) or incomplete (False).\n        'total_cost' - A float that is the total cost of the analysis function. This is an optional output.\n        'model_name' - A string that is the name of the LLM model used for the analysis. This is an optional output.\n\n% Here is an example of a Langchain LCEL program: ```<./context/langchain_lcel_example.py>```\n\n% Here is an example how to select the Langchain llm and count tokens: ```<./context/llm_selector_example.py>```\n\n% Note: Use relative import for 'llm_selector' to ensure compatibility within the package structure (i.e. 'from .llm_selector') instead of 'from pdd.llm_selector'.\n\n% This function will do the following:\n    Step 1. Use $PDD_PATH environment variable to get the path to the project. Load the '$PDD_PATH/prompts/unfinished_prompt_LLM.prompt' file.\n    Step 2. Create a Langchain LCEL template from unfinished_prompt_LLM prompt so that it returns a JSON output.\n    Step 3. Use the llm_selector function for the LLM model.\n    Step 4. Run the prompt text through the model using Langchain LCEL.\n        4a. Pass the following string parameters to the prompt during invoke:\n            - 'PROMPT_TEXT'\n        4b. Pretty print a message letting the user know it is running and how many tokens (using token_counter function from llm_selector) are in the prompt and the cost. The cost from llm_selector is in dollars per million tokens.\n        4c. The dictionary output of the LCEL will have the keys 'reasoning' and 'is_finished'. Be sure to access these keys using the get method with default error messages.\n        4d. Pretty print the reasoning and completion status using the rich library. Also, print the number of tokens in the result, the output token cost and the total_cost.\n    Step 5. Return the 'reasoning' string and 'is_finished' boolean from the JSON output using 'get', and the 'total_cost' float, and 'model_name' string.\n\n% Ensure that the function handles potential errors gracefully, such as missing input parameters or issues with the LLM model responses.\n\n"}, {"PROMPT_NAME": "xml_tagger_python.prompt", "PROMPT_DESCRIPTION": "% You are an expert Python engineer. Your goal is to write a Python function, \"xml_tagger\", that will enhance a given LLM prompt by adding XML tags to improve its structure and readability.\n\n% The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name'). All output to the console will be pretty printed using the Python Rich library. Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.
+% The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```\n\n% Here are the inputs and outputs of the function:\n    Input: \n        'raw_prompt' - A string containing the prompt that needs XML tagging to improve its organization and clarity.\n        'strength' - A float value representing the strength parameter for the LLM model.\n        'temperature' - A float value representing the temperature parameter for the LLM model.\n    Output: \n        'xml_tagged' - A string containing the prompt with properly added XML tags.\n        'total_cost' - A float representing the total cost of running the LCELs.\n        'model_name' - A string representing the name of the selected LLM model.\n\n% Here is an example of a LangChain Expression Language (LCEL) program: <lcel_example>import os
+from langchain_core.prompts import PromptTemplate
+from langchain_community.cache import SQLiteCache
+from langchain_community.llms.mlx_pipeline import MLXPipeline
+from langchain.globals import set_llm_cache
+from langchain_core.output_parsers import JsonOutputParser, PydanticOutputParser # Parsers are only avaiable in langchain_core.output_parsers not langchain.output_parsers
+from langchain_core.output_parsers import StrOutputParser
+from langchain_core.prompts import ChatPromptTemplate
+from langchain_core.runnables import RunnablePassthrough, ConfigurableField
+
+from langchain_openai import AzureChatOpenAI
+from langchain_fireworks import Fireworks 
+from langchain_anthropic import ChatAnthropic
+from langchain_openai import ChatOpenAI # Chatbot and conversational tasks
+from langchain_openai import OpenAI # General language tasks
+from langchain_google_genai import ChatGoogleGenerativeAI
+from langchain_google_vertexai import ChatVertexAI
+from langchain_groq import ChatGroq
+from langchain_together import Together
+
+from langchain.callbacks.base import BaseCallbackHandler
+from langchain.schema import LLMResult
+
+import json
+
+from langchain_community.chat_models.mlx import ChatMLX
+from langchain_core.messages import HumanMessage
+
+from langchain_ollama.llms import OllamaLLM
+from langchain_aws import ChatBedrockConverse
+
+# Define a base output parser (e.g., PydanticOutputParser)
+from pydantic import BaseModel, Field
+
+
+
+class CompletionStatusHandler(BaseCallbackHandler):
+    def __init__(self):
+        self.is_complete = False
+        self.finish_reason = None
+        self.input_tokens = None
+        self.output_tokens = None
+
+    def on_llm_end(self, response: LLMResult, **kwargs) -> None:
+        self.is_complete = True
+        if response.generations and response.generations[0]:
+            generation = response.generations[0][0]
+            self.finish_reason = generation.generation_info.get('finish_reason').lower()
+            
+            # Extract token usage
+            if hasattr(generation.message, 'usage_metadata'):
+                usage_metadata = generation.message.usage_metadata
+                self.input_tokens = usage_metadata.get('input_tokens')
+                self.output_tokens = usage_metadata.get('output_tokens')
+        # print("response:",response)
+        print("Extracted information:")
+        print(f"Finish reason: {self.finish_reason}")
+        print(f"Input tokens: {self.input_tokens}")
+        print(f"Output tokens: {self.output_tokens}")
+
+# Set up the LLM with the custom handler
+handler = CompletionStatusHandler()
+# Always setup cache to save money and increase speeds
+set_llm_cache(SQLiteCache(database_path=".langchain.db"))
+
+
+# Create the LCEL template. Make note of the variable {topic} which will be filled in later.
+prompt_template = PromptTemplate.from_template("Tell me a joke about {topic}")
+
+llm = ChatGoogleGenerativeAI(model="gemini-2.5-pro-exp-03-25", temperature=0, callbacks=[handler])
+# Combine with a model and parser to output a string
+chain = prompt_template |llm| StrOutputParser()
+
+# Run the template. Notice that the input is a dictionary with a single key "topic" which feeds it into the above prompt template. This is needed because the prompt template has a variable {topic} which needs to be filled in when invoked.
+result = chain.invoke({"topic": "cats"})
+print("********Google:", result)
+
+
+llm = ChatVertexAI(model="gemini-2.5-pro-exp-03-25", temperature=0, callbacks=[handler])
+# Combine with a model and parser to output a string
+chain = prompt_template |llm| StrOutputParser()
+
+# Run the template. Notice that the input is a dictionary with a single key "topic" which feeds it into the above prompt template. This is needed because the prompt template has a variable {topic} which needs to be filled in when invoked.
+result = chain.invoke({"topic": "cats"})
+print("********GoogleVertex:", result)
+
+
+# Define your desired data structure.
+class Joke(BaseModel):
+    setup: str = Field(description="question to set up a joke")
+    punchline: str = Field(description="answer to resolve the joke")
+
+
+# Set up a parser
+parser = JsonOutputParser(pydantic_object=Joke)
+
+# Create a prompt template
+prompt = PromptTemplate(
+    template="Answer the user query.\n{format_instructions}\n{query}\n",
+    input_variables=["query"],
+    partial_variables={"format_instructions": parser.get_format_instructions()},
+)
+
+llm_no_struct = ChatOpenAI(model="gpt-4o-mini", temperature=0, 
+                           callbacks=[handler]) 
+llm = llm_no_struct.with_structured_output(Joke) # with structured output forces the output to be a specific object, in this case Joke. Only OpenAI models have structured output
+# Chain the components. 
+#  The class `LLMChain` was deprecated in LangChain 0.1.17 and will be removed in 1.0. Use RunnableSequence, e.g., `prompt | llm` instead.
+chain = prompt | llm 
+
+# Invoke the chain with a query. 
+# IMPORTANT: chain.run is now obsolete. Use chain.invoke instead.
+result = chain.invoke({"query": "Tell me a joke about openai."})
+print("4o mini JSON: ",result)
+print(result.setup) # How to access the structured output
+
+llm = ChatOpenAI(model="o1", temperature=1, 
+                           callbacks=[handler],model_kwargs = {"max_completion_tokens" : 1000})
+# Chain the components. 
+#  The class `LLMChain` was deprecated in LangChain 0.1.17 and will be removed in 1.0. Use RunnableSequence, e.g., `prompt | llm` instead.
+chain = prompt | llm | parser
+
+# Invoke the chain with a query. 
+# IMPORTANT: chain.run is now obsolete. Use chain.invoke instead.
+result = chain.invoke({"query": "Tell me a joke about openai."})
+print("o1 JSON: ",result)
+
+# Get DEEPSEEK_API_KEY environmental variable
+
+deepseek_api_key = os.getenv('DEEPSEEK_API_KEY')
+
+# Ensure the API key is retrieved successfully
+if deepseek_api_key is None:
+    raise ValueError("DEEPSEEK_API_KEY environment variable is not set")
+
+llm = ChatOpenAI(
+    model='deepseek-chat', 
+    openai_api_key=deepseek_api_key, 
+    openai_api_base='https://api.deepseek.com',
+    temperature=0, callbacks=[handler]
+)
+
+# Chain the components
+chain = prompt | llm | parser
+
+# Invoke the chain with a query
+result = chain.invoke({"query": "Write joke about deepseek."})
+print("deepseek",result)
+
+
+# Set up a parser
+parser = PydanticOutputParser(pydantic_object=Joke)
+# Chain the components
+chain = prompt | llm | parser
+
+# Invoke the chain with a query
+result = chain.invoke({"query": "Write joke about deepseek and pydantic."})
+print("deepseek pydantic",result)
+
+# Set up the Azure ChatOpenAI LLM instance
+llm_no_struct = AzureChatOpenAI(
+    model="o4-mini",
+    temperature=1,
+    callbacks=[handler]
+)
+llm = llm_no_struct.with_structured_output(Joke) # with structured output forces the output to be a specific JSON format
+# Chain the components: prompt | llm | parser
+chain = prompt | llm # returns a Joke object
+
+# Invoke the chain with a query
+result = chain.invoke({"query": "What is Azure?"})  # Pass a dictionary if `invoke` expects it
+print("Azure Result:", result)
+
+# Set up a parser
+parser = JsonOutputParser(pydantic_object=Joke)
+
+llm = Fireworks(
+    model="accounts/fireworks/models/llama4-maverick-instruct-basic",
+    temperature=0, callbacks=[handler])
+# Chain the components
+chain = prompt | llm | parser
+
+# Invoke the chain with a query
+# no money in account
+# result = chain.invoke({"query": "Tell me a joke about the president"})
+# print("fireworks",result)
+
+
+
+
+
+prompt = ChatPromptTemplate.from_template(
+    "Tell me a short joke about {topic}"
+)
+chat_openai = ChatOpenAI(model="gpt-3.5-turbo", callbacks=[handler])
+openai = OpenAI(model="gpt-3.5-turbo-instruct", callbacks=[handler])
+anthropic = ChatAnthropic(model="claude-2", callbacks=[handler])
+model = (
+    chat_openai
+    .with_fallbacks([anthropic])
+    .configurable_alternatives(
+        ConfigurableField(id="model"),
+        default_key="chat_openai",
+        openai=openai,
+        anthropic=anthropic,
+    )
+)
+
+chain = (
+    {"topic": RunnablePassthrough()} 
+    | prompt 
+    | model 
+    | StrOutputParser()
+)
+result = chain.invoke({"topic": "Tell me a joke about the president"})
+print("config alt:",result)
+
+
+
+llm = ChatAnthropic(
+    model="claude-3-7-sonnet-latest",
+    max_tokens=5000,  # Total tokens for the response
+    thinking={"type": "enabled", "budget_tokens": 2000},  # Tokens for internal reasoning
+)
+
+response = llm.invoke("What is the cube root of 50.653?")
+print(json.dumps(response.content, indent=2))
+
+
+llm = ChatGroq(temperature=0, model_name="qwen-qwq-32b", callbacks=[handler])
+system = "You are a helpful assistant."
+human = "{text}"
+prompt = ChatPromptTemplate.from_messages([("system", system), ("human", human)])
+
+chain = prompt | llm | StrOutputParser()
+print(chain.invoke({"text": "Explain the importance of low latency LLMs."}))
+
+
+llm = Together(
+    model="meta-llama/Llama-3-70b-chat-hf",
+    max_tokens=500, callbacks=[handler]
+)
+chain = prompt | llm | StrOutputParser()
+print(chain.invoke({"text": "Explain the importance of together.ai."}))
+
+
+# Define a prompt template with placeholders for variables
+prompt_template = PromptTemplate.from_template("Tell me a {adjective} joke about {content}.")
+
+# Format the prompt with the variables
+formatted_prompt = prompt_template.format(adjective="funny", content="data scientists")
+
+# Print the formatted prompt
+print(formatted_prompt)
+
+
+# Set up the LLM with the custom handler
+handler = CompletionStatusHandler()
+
+
+llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.9, callbacks=[handler])
+
+prompt = PromptTemplate.from_template("What is a good name for a company that makes {product}?")
+
+chain = prompt | llm
+
+# Invoke the chain
+response = chain.invoke({"product":"colorful socks"})
+
+# Check completion status
+print(f"Is complete: {handler.is_complete}")
+print(f"Finish reason: {handler.finish_reason}")
+print(f"Response: {response}")
+print(f"Input tokens: {handler.input_tokens}")
+print(f"Output tokens: {handler.output_tokens}")
+
+
+
+template = """Question: {question}"""
+
+prompt = ChatPromptTemplate.from_template(template)
+
+model = OllamaLLM(model="qwen2.5-coder:32b")
+
+chain = prompt | model
+
+output = chain.invoke({"question": "Write a python function that calculates Pi"})
+print(output)
+
+
+
+llm = MLXPipeline.from_model_id(
+    "mlx-community/quantized-gemma-2b-it",
+    pipeline_kwargs={"max_tokens": 10, "temp": 0.1},
+)
+
+
+chat_model = ChatMLX(llm=llm)
+messages = [HumanMessage(content="What happens when an unstoppable force meets an immovable object?")]
+response = chat_model.invoke(messages)
+print(response.content)
+
+
+
+llm = ChatBedrockConverse(
+    model_id="anthropic.claude-3-5-sonnet-20240620-v1:0",
+    # Additional parameters like temperature, max_tokens can be set here
+)
+
+messages = [HumanMessage(content="What happens when an unstoppable force meets an immovable sonnet?")]
+response = llm.invoke(messages)
+print(response.content)</lcel_example>\n\n% Here are examples of how to use internal modules:\n<internal_example_modules>\n    % Example of selecting a Langchain LLM and counting tokens using llm_selector: <llm_selector_example>from pdd.llm_selector import llm_selector
+
+def main() -> None:
+    """
+    Main function to demonstrate the usage of the llm_selector function.
+    """
+    # Define the strength and temperature parameters
+    strength: float = 0.5  # Example strength value for the LLM model
+    temperature: float = 1.0  # Example temperature value for the LLM model
+
+    try:       
+        while strength <= 1.1: 
+            # Call the llm_selector function with the specified strength and temperature
+            llm, token_counter, input_cost, output_cost, model_name = llm_selector(strength, temperature)
+            print(f"Strength: {strength}")
+            
+            # Print the details of the selected LLM model
+            print(f"Selected LLM Model: {model_name}")
+            print(f"Input Cost per Million Tokens: {input_cost}")
+            print(f"Output Cost per Million Tokens: {output_cost}")
+
+            # Example usage of the token counter function
+            sample_text: str = "This is a sample text to count tokens."
+            token_count: int = token_counter(sample_text)
+            print(f"Token Count for Sample Text: {token_count}")
+            print(f"model_name: {model_name}")
+            strength += 0.05
+    except FileNotFoundError as e:
+        print(f"Error: {e}")
+    except ValueError as e:
+        print(f"Error: {e}")
+
+if __name__ == "__main__":
+    main()</llm_selector_example>\n</internal_example_modules>\n\n% This program will use Langchain to do the following:\n    Step 1. Use $PDD_PATH environment variable to get the path to the project. Load the '$PDD_PATH/prompts/xml_convertor_LLM.prompt' and '$PDD_PATH/prompts/extract_xml_LLM.prompt' files.\n    Step 2. Create a Langchain LCEL template from xml_convertor prompt so that it returns a string output.\n    Step 3. Use the llm_selector function for the LLM model and token counting.\n    Step 4. Run the code through the model using Langchain LCEL. \n        4a. Pass the following string parameters to the prompt during invoke:\n            - 'raw_prompt'\n        4b. Pretty print a message letting the user know it is running and how many tokens (using token_counter from llm_selector) are in the prompt and the cost. The cost from llm_selector is in dollars per million tokens. \n        4c. The string output of the LCEL will be 'xml_generated_analysis' that contains the tagged prompt.\n    Step 5. The code result of the model will contain a mix of text and XML separated by triple backticks. Create a Langchain LCEL template but with a llm_selector with strength .8 from the extract_xml prompt that has a JSON output.\n        5a. Pass the following string parameters to the prompt during invoke:\n            - 'xml_generated_analysis'\n        5b. Pretty print a message letting the user know it is running and how many tokens (using token_counter from llm_selector) are in the prompt and the cost. The cost from llm_selector is in dollars per million tokens.\n        5c. The JSON output of the LCEL will have the key 'xml_tagged' that contains the extracted tagged prompt.\n    Step 6. Pretty print the extracted tagged prompt using the rich Markdown function. Also, print the number of tokens in the result and the cost.\n    Step 7. Calculate the total cost by summing the costs from both LCEL runs.\n    Step 8. Return the 'xml_tagged' string using 'get', the 'total_cost' and 'model_name'."}]
         </prompt_list_example>
 
         <change_description_example>
-        <include>context/detect_change/2/change.prompt</include>
+        % Use context/python_preamble.prompt to make prompts more compact. Some prompts might already have this.
+
+% Here is what is inside context/python_preamble.prompt:<preamble>% The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name'). All output to the console will be pretty printed using the Python Rich library. Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.
+% The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```</preamble>
+
+% Here is an example of this being done: <example>% You are an expert Python engineer. Your goal is to write a python function, "code_generator", that will compile a prompt into a code file. 
+
+[File not found: ../context/python_preamble.prompt]
+
+% Here are the inputs and outputs of the function:
+    Inputs: 
+        'prompt' - A string containing the raw prompt to be processed.
+        'language' - A string that is the language type (e.g. python, bash) of file that will be outputed by the LLM.
+        'strength' - A float between 0 and 1 that is the strength of the LLM model to use.
+        'temperature' - A float that is the temperature of the LLM model to use. Default is 0.
+        'time' - A float between 0 and 1 that controls the thinking effort for the LLM model, passed to llm_invoke. Default is DEFAULT_TIME.
+        'verbose' - A boolean that indicates whether to print out the details of the function. Default is False.
+        'preprocess_prompt' - A boolean that indicates whether to preprocess the prompt. Default is True.
+    Outputs:
+        'runnable_code' - A string that is runnable code
+        'total_cost' - A float that is the total cost of the model run
+        'model_name' - A string that is the name of the selected LLM model
+
+% Here is how to use the internal modules:
+    <internal_modules>
+        For running prompts with llm_invoke:
+        <llm_invoke_example>
+            [File not found: ../context/llm_invoke_example.py]
+        </llm_invoke_example>
+
+        For preprocessing prompts:
+        <preprocess_example>
+            [File not found: ../context/preprocess_example.py]
+        </preprocess_example>
+
+        For handling unfinished prompts:
+        <unfinished_prompt_example>
+            [File not found: ../context/unfinished_prompt_example.py]
+        </unfinished_prompt_example>
+
+        For continuing generation:
+        <continue_generation_example>
+            [File not found: ../context/continue_generation_example.py]
+        </continue_generation_example>
+
+        For postprocessing results:
+        <postprocess_example>
+            [File not found: ../context/postprocess_example.py]
+        </postprocess_example>
+    </internal_modules>
+
+% This program will do the following:
+    Step 1. Conditionally preprocess the raw prompt using the preprocess function from the preprocess module based on the value of 'preprocess_prompt'. If 'preprocess_prompt' is True, preprocess the prompt; otherwise, use the raw prompt directly.
+
+    Step 2. Run the prompt (either preprocessed or raw) through llm_invoke with an empty dictionary ('{}') and the provided strength, temperature, and time.
+
+    Step 3. Detect if the generation is incomplete using the unfinished_prompt function (strength .5) by passing in the last 600 characters of the output of Step 3.
+        - a. If incomplete, call the continue_generation function to complete the generation.
+        - b. Else, if complete, postprocess the model output result using the postprocess function from the postprocess module with the EXTRACTION_STRENGTH constant. Be sure to pass in all parameters like verbose to postprocess.
+    Step 4. Return the runnable_code, total_cost and model_name.</example>
         </change_description_example>
     </input_example>
 
     <output_example>
-        <include>context/detect_change/2/detect_change_output.txt</include>
+        1. **Impact of the Change and Potential Issues:**
+
+   The change description suggests using the contents of `context/python_preamble.prompt` to make prompts more compact. This preamble includes instructions for using relative imports, pretty printing with the Python Rich library, and handling edge cases. The main impact is to ensure consistency across prompts by centralizing common instructions, which can reduce redundancy and improve maintainability. Potential issues include ensuring that the preamble does not conflict with existing instructions in the prompts and that it is inserted in a way that maintains the logical flow of the prompt.
+
+2. **Possible Implementation Plans:**
+
+   - **Plan A: Direct Inclusion of Preamble**
+     - **Pros:** Simplifies prompts by removing redundant instructions and directly including the preamble.
+     - **Cons:** May require significant restructuring of prompts to ensure logical flow and avoid redundancy.
+   
+   - **Plan B: Conditional Inclusion Based on Existing Content**
+     - **Pros:** Only includes the preamble if similar instructions are not already present, minimizing disruption.
+     - **Cons:** Requires careful analysis of each prompt to determine if the preamble is needed, which can be time-consuming.
+   
+   - **Plan C: Hybrid Approach**
+     - **Pros:** Combines the benefits of both plans by including the preamble where it adds value and retaining existing instructions where they are more specific or detailed.
+     - **Cons:** Complexity in deciding which parts of the preamble to include or exclude for each prompt.
+
+3. **Analysis and Selection of the Best Plan:**
+
+   Plan C is the best approach as it allows for flexibility and ensures that the preamble is only included where it enhances the prompt. This minimizes redundancy and maintains the integrity of prompts that already have specific instructions. It also allows for a more tailored approach, ensuring that the preamble complements rather than conflicts with existing content.
+
+4. **Prompt Analysis Based on Selected Plan:**
+
+   - **change_python.prompt:** This prompt already contains detailed instructions similar to those in the preamble. However, it can benefit from the preamble for consistency in handling edge cases and pretty printing.
+   - **preprocess_python.prompt:** This prompt can include the preamble to ensure consistent handling of edge cases and pretty printing.
+   - **unfinished_prompt_python.prompt:** This prompt can include the preamble to ensure consistent handling of edge cases and pretty printing.
+   - **xml_tagger_python.prompt:** This prompt already includes the preamble, so no changes are needed.
+
+5. **Prompts That Need to Be Changed:**
+
+   - **change_python.prompt**
+     - **Instructions:**
+       - Insert the contents of the file `./context/python_preamble.prompt` immediately after the role and goal statement using 'include' XML tags.
+       - Remove any redundant instructions that are covered by the preamble, such as those related to pretty printing and handling edge cases.
+       - Ensure that the logical flow of the prompt is maintained and that any unique instructions specific to this prompt are retained.
+       - Here is what is inside context/python_preamble.prompt:<preamble>% The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name'). All output to the console will be pretty printed using the Python Rich library. Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.
+% The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```</preamble>
+
+   - **preprocess_python.prompt**
+     - **Instructions:**
+       - Insert the contents of the file `./context/python_preamble.prompt` immediately after the role and goal statement using 'include' XML tags.
+       - Remove any redundant instructions that are covered by the preamble, such as those related to pretty printing and handling edge cases.
+       - Ensure that the logical flow of the prompt is maintained and that any unique instructions specific to this prompt are retained.
+       - Here is what is inside context/python_preamble.prompt:<preamble>% The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name'). All output to the console will be pretty printed using the Python Rich library. Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.
+% The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```</preamble>
+
+   - **unfinished_prompt_python.prompt**
+     - **Instructions:**
+       - Insert the contents of the file `./context/python_preamble.prompt` immediately after the role and goal statement using 'include' XML tags.
+       - Remove any redundant instructions that are covered by the preamble, such as those related to pretty printing and handling edge cases.
+       - Ensure that the logical flow of the prompt is maintained and that any unique instructions specific to this prompt are retained.
+       - Here is what is inside context/python_preamble.prompt:<preamble>% The function should be part of a Python package, using relative imports (single dot) for internal modules (e.g. 'from .module_name import module_name'). All output to the console will be pretty printed using the Python Rich library. Ensure the function handles edge cases, such as missing inputs or model errors, and provide clear error messages.
+% The ./pdd/__init__.py file will have the EXTRACTION_STRENGTH, DEFAULT_STRENGTH, DEFAULT_TIME and other global constants. Example: ```from . import DEFAULT_STRENGTH```</preamble>
     </output_example>
 </example>
 

pdd/prompts/fix_errors_from_unit_tests_LLM.prompt

@@ -9,50 +9,7 @@
 % This prompt is run iteratively. Here are the current errors and past potential fix attempts, if any, from the unit test and verification program run(s): <errors>{errors}</errors>
 
 % If the verfication program fails to run, the code_under_test and unit_test are unchanged from the previous iteration.
-<pdd>
-<examples>
-    <example_1>
-    % Here is an example_unit_test for the example_code_under_test: <example_unit_test><include>context/fix_errors_from_unit_tests/1/test_conflicts_in_prompts.py</include></example_unit_test>
-    
-    % Here is an example_code_under_test that fully passes the example_unit_test: <example_code_under_test><include>context/fix_errors_from_unit_tests/1/conflicts_in_prompts.py</include></example_code_under_test>
 
-    % Here is the prompt that generated the example_code_under_test: <example_prompt><include>context/fix_errors_from_unit_tests/1/conflicts_in_prompts_python.prompt</include></example_prompt>
-    </example_1>
-
-    <example_2>
-    % Here is an example_unit_test for the example_code_under_test: <example_unit_test><include>context/fix_errors_from_unit_tests/2/test_code_generator.py</include></example_unit_test>
-
-    % Here is an example_code_under_test that fully passes the example_unit_test: <example_code_under_test><include>context/fix_errors_from_unit_tests/2/code_generator.py</include></example_code_under_test>
-
-    % Here is the prompt that generated the example_code_under_test: <example_prompt><include>context/fix_errors_from_unit_tests/2/code_generator_python.prompt</include></example_prompt>
-    </example_2>
-
-    <example_3>
-    % Here is an example_unit_test for the example_code_under_test: <example_unit_test><include>context/fix_errors_from_unit_tests/3/test_context_generator.py</include></example_unit_test>
-
-    % Here is an example_code_under_test that fully passes the example_unit_test: <example_code_under_test><include>context/fix_errors_from_unit_tests/3/context_generator.py</include></example_code_under_test>
-
-    % Here is the prompt that generated the example_code_under_test: <example_prompt><include>context/fix_errors_from_unit_tests/3/context_generator_python.prompt</include></example_prompt>
-    </example_3>
-
-
-    <example_4>
-    % Here is an example_unit_test for the example_code_under_test: <example_unit_test><include>context/fix_errors_from_unit_tests/4/test_detect_change.py</include></example_unit_test>
-
-    % Here is an example_code_under_test that fully passes the example_unit_test: <example_code_under_test><include>context/fix_errors_from_unit_tests/4/detect_change.py</include></example_code_under_test>
-
-    % Here is the prompt that generated the example_code_under_test: <example_prompt><include>context/fix_errors_from_unit_tests/4/detect_change_python.prompt</include></example_prompt>
-    </example_4>
-
-    <example_5>
-    % Here is an example_unit_test for the example_code_under_test: <example_unit_test><include>context/fix_errors_from_unit_tests/4/test_detect_change_1_0_1.py</include></example_unit_test>
-
-    % Here is an example_code_under_test that didn't fully pass the example_unit_test: <example_code_under_test><include>context/fix_errors_from_unit_tests/4/detect_change_1_0_1.py</include></example_code_under_test>
-
-    % Here is an example error/fix log showing how the issues were resolved: <example_error_fix_log><include>context/fix_errors_from_unit_tests/4/error.log</include></example_error_fix_log>
-    </example_5>
-</examples>
-</pdd>
 
 <instructions>
 % Follow these steps to solve these errors:

pdd/prompts/increase_tests_LLM.prompt

@@ -13,4 +13,7 @@ Here is the coverage report: ```{coverage_report}```
     - The unit test should be in {language}. If Python, use pytest.
     - Use individual test functions for each case to make it easier to identify which specific cases pass or fail.
     - Use the description of the functionality in the prompt to generate tests with useful tests with good code coverage.
-<include>./context/test.prompt</include>
+    - Know that the generated test will be in a different directory (`tests`) than the module (in directory `pdd`) it is calling and will need an absolute reference. The module file name will be same as the function name.
+    - Created files should be in the `output` directory.
+    - Data files (language_format.csv and llm_model.csv) already exist in the PDD_PATH/`data` directory. Do not write over them. It already contains data for popular languages and LLM models and can be used for tests.
+    - The PDD_PATH environment variable is already set.