janus-llm 4.3.1__py3-none-any.whl → 4.4.5__py3-none-any.whl

janus/prompts/templates/refinement/reflection/incose/human.txt

@@ -0,0 +1,26 @@
+TASK:
+```
+{prompt}
+```
+--------------------
+REQUIREMENTS:
+```
+{completion}
+```
+--------------------
+
+Generate critique and suggestions for the provided REQUIREMENTS based on the above TASK. Your suggestions can include requests for added detail, corrections of factual errors, improved style, etc. to ensure that each of the requirements meet INCOSE standards, described below.
+
+Do not rewrite the output yourself, only provide critique that can be acted on. Provide no further commentary or questions other than your feedback.
+
+If every single one of the requirements meets the following INCOSE standards, return LGTM. "LGTM" must be returned on its own without further commentary. Be especially strict in your grading!
+
+C1 - Necessary: The need or requirement statement defines an essential capability, characteristic, constraint, or quality factor needed to satisfy a lifecycle concept, need, source, or parent requirement.
+C2 - Appropriate: The specific intent and amount of detail of the need or requirement statement is appropriate to the level (the level of abstraction, organization, or system architecture) of the entity to which it refers.
+C3 - Unambiguous: Need statements must be written such that the stakeholder intent is clear. Requirement statements must be stated such that the requirement can be interpreted in only one way by all the intended stakeholders.
+C4 - Complete: The requirement statement sufficiently describes the necessary capability, characteristic, constraint, or quality factor to meet the need, source, or parent requirement from which it was transformed without needing other information to understand the requirement.
+C5 - Singular: The stakeholder need or requirement statement should state a single capability, characteristic, constraint, or quality factor.
+C6 - Feasible: The need or requirement can be realized within entity constraints (for example: cost, schedule, technical, legal, ethical, safety) with acceptable risk.
+C7 - Verifiable: The requirement statement is structured and worded such that its realization can be verified to the approving authority’s satisfaction.
+C8 - Correct: The need statement must be an accurate representation of the lifecycle concept or source from which it was transformed. The requirement statement must be an accurate representation of the need, source, or parent requirement from which it was transformed.
+C9 - Conforming: Individual needs and requirements should conform to an approved standard pattern and style guide or standard for writing and managing needs and requirements.

janus/prompts/templates/refinement/reflection/incose/system.txt

@@ -0,0 +1 @@
+You are a software engineer and quality control professional tasked with reviewing and critiquing outputs from Large Language Models (LLMs). Given a task and an output, it is your job to generate critique and suggestions to improve the output. If the output fully meets the requirements of the task and is of high quality, you may approve it with a simple "LGTM".

janus/prompts/templates/refinement/reflection/incose_deduplicate/human.txt

@@ -0,0 +1,16 @@
+TASK:
+```
+{prompt}
+```
+--------------------
+REQUIREMENTS:
+```
+{completion}
+```
+--------------------
+
+In order for software requirements to be useful for software modernization, they must be reasonably consise and not overly specific to the syntax of the legacy language of the code which they describe. With this in mind, for the provided TASK, suggest ways that the REQUIREMENTS could be less redundant. If the REQUIREMENTS include duplicated or unnecessary requirements, explain how some of them could be combined to reduce this number. If the REQUIREMENTS refer to specific aspects of the original coding language which would not be useful to a software developer porting the code to a new language, point it out.
+
+Do not rewrite the output yourself, only provide critique that can be acted on. Provide no further commentary or questions other than your feedback.
+
+If the requirements in the completion above are adequately consise, return LGTM. "LGTM" must be returned on its own without further commentary.

janus/prompts/templates/refinement/reflection/incose_deduplicate/system.txt

@@ -0,0 +1 @@
+You are a software engineer and quality control professional tasked with reviewing and critiquing outputs from Large Language Models (LLMs). Given a task and an output, it is your job to generate critique and suggestions to improve the output. If the output fully meets the requirements of the task and is of high quality, you may approve it with a simple "LGTM".

janus/prompts/templates/refinement/reflection/system.txt

@@ -0,0 +1 @@
+You are a software engineer and quality control professional tasked with reviewing and critiquing outputs from Large Language Models (LLMs). Given a task and an output, it is your job to generate critique and suggestions to improve the output. If the output fully meets the requirements of the task and is of high quality, you may approve it with a simple "LGTM".

janus/prompts/templates/refinement/revision/human.txt

@@ -0,0 +1,16 @@
+TASK:
+```
+{prompt}
+```
+--------------------
+COMPLETION:
+```
+{completion}
+```
+--------------------
+FEEDBACK:
+```
+{reflection}
+```
+--------------------
+Improve the above COMPLETION of the given TASK by incorporating the provided FEEDBACK. Respond only with the revised output, following the format indicated in the original TASK description. Do not provide any additional commentary or questions about the critique.

janus/prompts/templates/refinement/revision/incose/human.txt

@@ -0,0 +1,16 @@
+TASK:
+```
+{prompt}
+```
+--------------------
+REQUIREMENTS:
+```
+{completion}
+```
+--------------------
+FEEDBACK:
+```
+{reflection}
+```
+--------------------
+Improve the above REQUIREMENTS for the given TASK by incorporating the provided FEEDBACK. Respond only with the revised output, following the format indicated in the original TASK description. Do not provide any additional commentary or questions about the critique.

janus/prompts/templates/refinement/revision/incose/system.txt

@@ -0,0 +1 @@
+You are a software engineer and quality control professional tasked with improving outputs of Large Language Models according to human feedback. Given a task, a draft output, and some critique of that draft, it is your job to write a new version of the output that incorporates the provided suggestions and addresses any criticism.

janus/prompts/templates/refinement/revision/incose_deduplicate/human.txt

@@ -0,0 +1,17 @@
+TASK:
+```
+{prompt}
+```
+--------------------
+COMPLETION:
+```
+{completion}
+```
+--------------------
+FEEDBACK:
+```
+{reflection}
+```
+--------------------
+Improve the above COMPLETION of the given TASK by incorporating the provided FEEDBACK. Respond only with the revised output, following the format indicated in the original TASK description. Do not provide any additional commentary or questions about the critique.
+g: Individual needs and requirements should conform to an approved standard pattern and style guide or standard for writing and managing needs and requirements.

janus/prompts/templates/refinement/revision/incose_deduplicate/system.txt

@@ -0,0 +1 @@
+You are a software engineer and quality control professional tasked with improving outputs of Large Language Models according to human feedback. Given a task, a draft output, and some critique of that draft, it is your job to write a new version of the output that incorporates the provided suggestions and addresses any criticism.

janus/prompts/templates/refinement/revision/system.txt

@@ -0,0 +1 @@
+You are a software engineer and quality control professional tasked with improving outputs of Large Language Models according to human feedback. Given a task, a draft output, and some critique of that draft, it is your job to write a new version of the output that incorporates the provided suggestions and addresses any criticism.

janus/prompts/templates/refinement/uml/alc_fix_variables/human.txt

@@ -0,0 +1,15 @@
+TASK:
+```
+{prompt}
+```
+--------------------
+COMPLETION:
+```
+{completion}
+```
+--------------------
+Find any instances of variables in the provided COMPLETION of the above TASK being referenced without their proper namespace. Variables should always be referenced as data_structure.variable. Data structures can include classes, DSECTS, files or other similar language features.
+
+Do not rewrite the output yourself, only provide critique that can be acted on. Provide no further commentary or questions other than your feedback.
+
+If the output fully meets the requirements of the task and no variables are referenced outside their appropriate class or DSECT, respond with LGTM. "LGTM" must be returned on its own without further commentary.

janus/prompts/templates/refinement/uml/alc_fix_variables/system.txt

@@ -0,0 +1,2 @@
+You are a software engineer and quality control professional tasked with reviewing and critiquing outputs from Large Language Models (LLMs). Given a task and an output, it is your job to generate critique and suggestions to improve the output.
+You must detect any instances of variables being referenced in the UML without referencing their appropriate namespace (i.e. struct.var). If the output fully meets the requirements of the task and all variables are appropriately referenced, you may approve it with a simple "LGTM".

janus/prompts/templates/refinement/uml/fix_connections/human.txt

@@ -0,0 +1,15 @@
+TASK:
+```
+{prompt}
+```
+--------------------
+COMPLETION:
+```
+{completion}
+```
+--------------------
+Find any instances of connections in the provided COMPLETION of the above TASK that do not accurately represent the control flow of the program. The COMPLETION should accurately capture the entire control flow of the original program in the TASK.
+
+Do not rewrite the output yourself, only provide critique that can be acted on. Provide no further commentary or questions other than your feedback.
+
+If the output fully meets the requirements of the task and the control flow of the program in the TASK is accurately reflected, respond with LGTM. "LGTM" must be returned on its own without further commentary.

janus/prompts/templates/refinement/uml/fix_connections/system.txt

@@ -0,0 +1,2 @@
+You are a software engineer and quality control professional tasked with reviewing and critiquing outputs from Large Language Models (LLMs). Given a task and an output, it is your job to generate critique and suggestions to improve the output.
+You must detect any connections in the UML that do not accurately represent the control flow. If the output fully meets the requirements of the task and the control flow is correctly represented, you may approve it with a simple "LGTM".

janus/prompts/templates/requirements/human.txt

@@ -0,0 +1,13 @@
+We are modernizing the following code so we need to extract software requirements to document its functionality.
+
+These requirements are intended for developers to refactor and port this code to a modern programming language, so they should focus on functionality rather than specifics of its current legacy language.
+
+Write the requirements in the style of an INCOSE software requirements specification document.
+
+Here is the code:
+
+```
+{SOURCE_CODE}
+```
+
+{context}

janus/prompts/templates/requirements/system.txt

@@ -0,0 +1,2 @@
+Your purpose is to understand a source code file
+and generate a software requirements specification document for it.

janus/prompts/templates/retrieval/language_docs/human.txt

@@ -0,0 +1,10 @@
+Are there any references in the following {SOURCE_LANGUAGE} source code to uncommon or obscure built-in language functionality e.g. stand library calls, opcode, etc. that you do not understand and may be present in a language reference manual?
+
+If so, return a short, comma-separated list of functions or opcodes to reference the manual for. Do not return any basic language functionality.
+
+If not, return 'NODOCS'. 'NODOCS' must be returned on its own without further commentary.
+
+Here is the code:
+```
+{SOURCE_CODE}
+```

janus/prompts/templates/retrieval/language_docs/system.txt

@@ -0,0 +1 @@
+You are a software engineer and quality control professional tasked with providing context to source code by suggesting aspects of the code which could be made more clear by referencing a language manual. Given some source code, it is your job to pick out standard library calls or instructions which could be clarified with a reference. If there are no such code features, you may skip with process with the response "NODOCS".

janus/prompts/templates/simple/human.txt

@@ -0,0 +1,16 @@
+Please convert the following {SOURCE_LANGUAGE} .{FILE_SUFFIX} code found in between triple backticks
+into {TARGET_LANGUAGE} code.
+
+If the given code is incomplete, assume it is translated elsewhere.
+If the given code is missing variable definitions, assume they are assigned elsewhere.
+If there are incomplete statements that haven't been closed out, assume they are closed out in other translations.
+If the code has comments, keep ALL of them.
+If it only consists of ONLY comments, assume the code that is represented by those comments is translated elsewhere.
+
+Some more things to remember:
+(1) follow standard styling practice for the target language
+(2) make sure the language is typed correctly
+
+```
+{SOURCE_CODE}
+```

janus/prompts/templates/simple/system.txt

@@ -0,0 +1,3 @@
+Your purpose is to convert {SOURCE_LANGUAGE} {FILE_SUFFIX} code
+into runnable {TARGET_LANGUAGE} code ({TARGET_LANGUAGE} version
+{TARGET_LANGUAGE_VERSION}).

janus/refiners/format.py

@@ -0,0 +1,49 @@
+from typing import Any
+
+from langchain_core.output_parsers import StrOutputParser
+from langchain_core.prompt_values import PromptValue
+from langchain_core.runnables import RunnableSerializable
+
+from janus.llm.models_info import MODEL_PROMPT_ENGINES, JanusModel
+from janus.parsers.parser import JanusParser
+from janus.refiners.refiner import JanusRefiner
+
+
+class FormatRefiner(JanusRefiner):
+    format_chain: RunnableSerializable
+
+    def __init__(
+        self,
+        llm: JanusModel,
+        parser: JanusParser,
+        max_retries: int,
+        prompt_template_name: str,
+    ):
+        format_prompt = MODEL_PROMPT_ENGINES[llm.short_model_id](
+            source_language="text",
+            prompt_template=prompt_template_name,
+        ).prompt
+        format_chain = format_prompt | llm | StrOutputParser()
+        super().__init__(
+            format_chain=format_chain, parser=parser, max_retries=max_retries
+        )
+
+    def parse_completion(
+        self, completion: str, prompt_value: PromptValue, **kwargs
+    ) -> Any:
+        completion = self.format_chain.invoke(
+            dict(completion=completion, prompt=prompt_value.to_string())
+        )
+        return self.parser.parse(completion)
+
+
+class CodeFormatRefiner(FormatRefiner):
+    def __init__(self, llm: JanusModel, parser: JanusParser, max_retries: int):
+        super().__init__(llm, parser, max_retries, "refinement/format/code_format")
+
+
+class RequirementsFormatRefiner(FormatRefiner):
+    def __init__(self, llm: JanusModel, parser: JanusParser, max_retries: int):
+        super().__init__(
+            llm, parser, max_retries, "refinement/format/requirements_format"
+        )

janus/refiners/refiner.py

@@ -3,6 +3,7 @@ from typing import Any
 
 from langchain.output_parsers import RetryWithErrorOutputParser
 from langchain_core.exceptions import OutputParserException
+from langchain_core.messages import AIMessage
 from langchain_core.output_parsers import StrOutputParser
 from langchain_core.prompt_values import PromptValue
 from langchain_core.runnables import RunnableSerializable
@@ -108,7 +109,7 @@ class ReflectionRefiner(JanusRefiner):
     def parse_completion(
         self, completion: str, prompt_value: PromptValue, **kwargs
     ) -> Any:
-        log.info(f"Reflection Prompt: {self.reflection_prompt_name}")
+        log.debug(f"Reflection Prompt: {self.reflection_prompt_name}")
         for retry_number in range(self.max_retries):
             reflection = self.reflection_chain.invoke(
                 dict(
@@ -119,8 +120,117 @@ class ReflectionRefiner(JanusRefiner):
             if re.search(r"\bLGTM\b", reflection) is not None:
                 return self.parser.parse(completion)
             if not retry_number:
-                log.info(f"Completion:\n{completion}")
-            log.info(f"Reflection:\n{reflection}")
+                log.debug(f"Completion:\n{completion}")
+            log.debug(f"Reflection:\n{reflection}")
+            completion = self.revision_chain.invoke(
+                dict(
+                    prompt=prompt_value.to_string(),
+                    completion=completion,
+                    reflection=reflection,
+                )
+            )
+            log.debug(f"Revision:\n{completion}")
+
+        return self.parser.parse(completion)
+
+
+class RequirementsReflectionRefiner(JanusRefiner):
+    """
+    This requirements-specific refiner is intended to address a common issue with
+    requirements reflection, where over the course of several reflection loops,
+    requirements lists grow increasingly verbose, eventually becoming too wordy
+    to be useful. To reduce this, this refiner interlaces an additional reflection
+    -> revision loop which de-duplicates requirements.
+    """
+
+    max_retries: int
+    reflection_chain: RunnableSerializable
+    revision_chain: RunnableSerializable
+    reflect_duplication_chain: RunnableSerializable
+    revise_duplication_chain: RunnableSerializable
+    reflection_prompt_name: str
+
+    def __init__(
+        self,
+        llm: JanusModel,
+        parser: JanusParser,
+        max_retries: int,
+        prompt_template_name: str = "refinement/reflection/incose",
+    ):
+        # Main reflection loop
+        reflection_prompt = MODEL_PROMPT_ENGINES[llm.short_model_id](
+            source_language="text",
+            prompt_template="refinement/reflection/incose",
+        ).prompt
+        revision_prompt = MODEL_PROMPT_ENGINES[llm.short_model_id](
+            source_language="text",
+            prompt_template="refinement/revision/incose",
+        ).prompt
+        # De-duplication loop
+        reflect_duplication_prompt = MODEL_PROMPT_ENGINES[llm.short_model_id](
+            source_language="text",
+            prompt_template="refinement/reflection/incose_deduplicate",
+        ).prompt
+        revise_duplication_prompt = MODEL_PROMPT_ENGINES[llm.short_model_id](
+            source_language="text",
+            prompt_template="refinement/revision/incose_deduplicate",
+        ).prompt
+
+        reflection_chain = reflection_prompt | llm | StrOutputParser()
+        revision_chain = revision_prompt | llm | StrOutputParser()
+        reflect_duplication_chain = reflect_duplication_prompt | llm | StrOutputParser()
+        revise_duplication_chain = revise_duplication_prompt | llm | StrOutputParser()
+        super().__init__(
+            reflection_prompt_name=prompt_template_name,
+            reflection_chain=reflection_chain,
+            revision_chain=revision_chain,
+            reflect_duplication_chain=reflect_duplication_chain,
+            revise_duplication_chain=revise_duplication_chain,
+            parser=parser,
+            max_retries=max_retries,
+        )
+
+    def parse_completion(
+        self, completion: str, prompt_value: PromptValue, **kwargs
+    ) -> Any:
+        log.debug(f"Reflection Prompt: {self.reflection_prompt_name}")
+        if isinstance(completion, AIMessage):
+            completion = completion.content
+        for retry_number in range(self.max_retries):
+            # First, check if the generated requirements are redundant or too specific
+            duplication_reflection = self.reflect_duplication_chain.invoke(
+                dict(
+                    prompt=prompt_value.to_string(),
+                    completion=completion,
+                )
+            )
+            if re.search(r"\bLGTM\b", duplication_reflection) is not None:
+                log.debug(
+                    "No de-duplication suggested in reflection, "
+                    "passing to next reflection step"
+                )
+            else:
+                completion = self.revise_duplication_chain.invoke(
+                    dict(
+                        prompt=prompt_value.to_string(),
+                        completion=completion,
+                        reflection=duplication_reflection,
+                    )
+                )
+
+            # Once we're happy with the results or trimmed them down,
+            # continue with the typical reflection process,
+            # except with specific INCOSE-focused prompts
+            reflection = self.reflection_chain.invoke(
+                dict(
+                    prompt=prompt_value.to_string(),
+                    completion=completion,
+                )
+            )
+            if re.search(r"\bLGTM\b", reflection) is not None:
+                return self.parser.parse(completion)
+            if not retry_number:
+                log.debug(f"Completion:\n{completion}")
             completion = self.revision_chain.invoke(
                 dict(
                     prompt=prompt_value.to_string(),
@@ -128,7 +238,6 @@ class ReflectionRefiner(JanusRefiner):
                     reflection=reflection,
                 )
             )
-            log.info(f"Revision:\n{completion}")
 
         return self.parser.parse(completion)