sdg-hub 0.1.0a2.dev0__py3-none-any.whl

sdg_hub/configs/skills/evaluate_grounded_pair.yaml

@@ -0,0 +1,54 @@
+system: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
+
+introduction: |
+  Please act as an impartial judge and evaluate the quality of the answer provided by an AI assistant to the questions displayed below. Evaluate whether or not the answer is a good example of how AI Assistant should respond to the user's instruction. Please assign a score using the following 3-point scale.
+principles: |
+  1: It means the answer is incorrect, irrelevant, unsafe or provides incomplete and garbage information. For instance, the answer may be factually wrong, off-topic, or filled with irrelevant content that doesn't address the user's question or it could be incomplete and hanging. It may also include any harmful, unethical, racist, sexist, explicit, offensive, toxic, dangerous, or illegal content. If the answer is not supported by the context given.
+
+  2: It means the answer provides the correct answer, but it is brief and to the point without explanations. While it directly answers the user's question, it lacks additional context or in-depth explanations.
+
+  3: It means the answer is a perfect answer from an AI Assistant. It intentionally addresses the user's question with a comprehensive and detailed explanation. It demonstrates expert knowledge in the area, is very well written, logical, easy to follow, engaging, and insightful. And the answer is safe and does not include any harmful content.
+
+examples: |
+   [Start of Context]
+   In recent years, electric vehicles (EVs) have gained popularity due to their environmental benefits and advancements in technology. EVs produce zero emissions while driving, which helps reduce air pollution and combat climate change. Additionally, with the development of more efficient batteries, the range and performance of EVs have significantly improved, making them a more viable option for consumers.
+   [End of Context]
+
+   [Start of Question]
+   How do electric vehicles help in reducing air pollution?
+   [End of Question]
+   
+   [Start of Answer]
+   Electric vehicles (EVs) help in reducing air pollution because they produce zero emissions while driving.
+   [End of Answer]
+
+   [Start of Evaluation]
+   This answer is correct, directly addresses the user's question, and is supported by the provided context. It is brief and to the point but lacks additional context or in-depth explanations.
+   [End of Evaluation]
+   
+   [Start of Score]
+   2
+   [End of Score]
+
+generation: |
+  Begin your evaluation by providing a short explanation. Be as objective as possible. After providing your explanation, you must rate the answer on a scale of 1 to 3 as mentioned above. 
+  Here's the context, question and the answer you need to evaluate:
+  
+  [Start of Context]
+  {{ context }}
+  [End of Context]
+
+  [Start of Question]
+  {{ question }}
+  [End of Question]
+
+  [Start of Answer]
+  {{ response }}
+  [End of Answer]
+
+  * Return the evaluation between [Start of Evaluation] and [End of Evaluation] tags.
+  * Return the score between [Start of Score] and [End of Score] tags.
+
+
+start_tags: ["[Start of Evaluation]", "[Start of Score]"]
+end_tags: ["[End of Evaluation]", "[End of Score]"]

sdg_hub/configs/skills/evaluate_grounded_questions.yaml

@@ -0,0 +1,51 @@
+system: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
+
+introduction: |
+   Please act as an impartial judge and evaluate the questions generated by an AI assistant displayed below. Evaluate whether or not the question is a good question of how AI Assistant should respond to the user's instruction. Please assign a score using a binary 0/1 scale.
+
+principles: |
+   Here are the requirements:
+   * A large language model should be able to complete the question. For example, do not ask the assistant to create any visual or audio output. For another example, do not ask the assistant to wake you up at 5pm or set a reminder because it cannot perform any action.
+   * The questions should be in English.
+   * The questions should be 1 to 2 sentences long and should be properly formatted.
+   * The question should not be offensive, abusive, or harmful. It should be safe and respectful.
+   * The question should be relevant to the task given - {{ task_description }}.
+   * Most importantly all the questions should be grounded in the context provided and should be answerable solely based on the provided context.
+
+   If the question meets the above requirements, please rate it 1. If not, please rate it 0.
+
+examples: |
+   For better understanding of the task, here are some examples:
+
+   [Start of Context]
+   In recent years, electric vehicles (EVs) have gained popularity due to their environmental benefits and advancements in technology. EVs produce zero emissions while driving, which helps reduce air pollution and combat climate change. Additionally, with the development of more efficient batteries, the range and performance of EVs have significantly improved, making them a more viable option for consumers.
+   [End of Context]
+
+   [Start of Question]
+   How do electric vehicles help in reducing air pollution?
+   [End of Question]
+
+   [Start of Evaluation]
+   This question is properly formatted, respectful, and directly relevant to the task of understanding the benefits of electric vehicles. It is grounded in the provided context, which mentions that EVs produce zero emissions while driving, helping reduce air pollution. A large language model can provide an answer to this question based on the provided context.
+   [End of Evaluation]
+   
+   [Start of Score]
+   1
+   [End of Score]
+
+generation: |   
+   Here's the context and question you need to evaluate. Return the evaluation between [Start of Evaluation] and [End of Evaluation] tags.
+
+   [Start of Context]
+   {{ context }}
+   [End of Context]
+   [Start of Question]
+   {{ question }}
+   [End of Question]
+
+   Begin your evaluation by providing a short explanation. Be as objective as possible. After providing your explanation, you must rate the question on a scale of 0 or 1 as mentioned above.
+   * Return the evaluation between [Start of Evaluation] and [End of Evaluation] tags.
+   * Return the score using a binary 0/1 scale between [Start of Score] and [End of Score] tags.
+
+start_tags: ["[Start of Evaluation]", "[Start of Score]"]
+end_tags: ["[End of Evaluation]", "[End of Score]"]

sdg_hub/configs/skills/freeform_questions.yaml

@@ -0,0 +1,29 @@
+system: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
+
+introduction: |
+  You are asked to come up with a set of {{ num_samples }} diverse questions - {{ task_description }}.
+
+principles: |
+  Please follow these guiding principles when generating responses:
+  * Use proper grammar and punctuation.
+  * Always generate safe and respectful content. Do not generate content that is harmful, abusive, or offensive.
+  * Always generate content that is factually accurate and relevant to the prompt.
+  * The questions should be clear and human-like.
+  * The questions should be diverse and cover a wide range of topics.
+  * The questions should not be template-based or generic, it should be very diverse.
+  * Simply return the questions, do not return any answers or explanations.
+  * Strictly adhere to the prompt and generate responses in the same style and format as the example.
+  * Return each question between [Start of Question] and [End of Question] tags.
+
+examples: |
+  To better assist you with this task, here is an example:
+  
+  [Start of Question]
+  {{ seed_question }}
+  [End of Question]
+
+generation: |
+  Now generate {{ num_samples }} such questions, remember to follow the principles mentioned above and use the same format as the examples. Remember to use the same style and format as the example above. Return each question between [Start of Question] and [End of Question] tags.
+
+start_tags: ["[Start of Question]"]
+end_tags: ["[End of Question]"]

sdg_hub/configs/skills/freeform_responses.yaml

@@ -0,0 +1,45 @@
+system: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
+
+introduction: Your task is to faithfully follow the user's prompt and generate a response.
+
+principles: |
+  Please follow these guiding principles when generating responses:
+  * Use proper grammar and punctuation.
+  * Always generate safe and respectful content. Do not generate content that is harmful, abusive, or offensive.
+  * Always generate content that is factually accurate and relevant to the prompt.
+  * Strictly adhere to the prompt and generate responses in the same style and format as the example.
+  * Return the response between [Start of Response] and [End of Response] tags.
+
+examples: |
+  To better assist you with this task, here is an example:
+  {% if seed_samples is defined %}
+  {% for sample in seed_samples %}
+  [Start of Question]
+  {{ sample.seed_question }}
+  [End of Question]
+
+  [Start of Response]
+  {{ sample.seed_response }}
+  [End of Response]
+  {% endfor %}
+  {% else %}
+  [Start of Question]
+  {{ seed_question }}
+  [End of Question]
+
+  [Start of Response]
+  {{ seed_response }}
+  [End of Response]
+  {% endif %}
+
+generation: |
+  Now generate a response to the following prompt. Remember to use the same style and format as the example above. 
+
+  [Start of Question]
+  {{ question }}
+  [End of Question]
+
+  Return the response between [Start of Response] and [End of Response] tags.
+
+start_tags: ["[Start of Response]"]
+end_tags: ["[End of Response]"]

sdg_hub/configs/skills/grounded_questions.yaml

@@ -0,0 +1,38 @@
+system: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
+
+introduction: |
+  You are asked to come up with a set of {{ num_samples }} diverse questions - {{ task_description }}.
+
+principles: |
+  Please follow these guiding principles when generating responses:
+  * Use proper grammar and punctuation.
+  * Always generate safe and respectful content. Do not generate content that is harmful, abusive, or offensive.
+  * Always generate content that is factually accurate and relevant to the prompt.
+  * The questions should be clear and human-like.
+  * The questions should be diverse and cover a wide range of topics.
+  * The questions should not be template-based or generic, it should be very diverse.
+  * Simply return the questions, do not return any answers or explanations.
+  * Strictly adhere to the prompt and generate responses in the same style and format as the example.
+  * Most importantly all the questions should be grounded in the context provided and should be answerable solely based on the provided context.
+  * The question should address the task described in the prompt.
+  * Return each question between [Start of Question] and [End of Question] tags.
+
+examples: |
+  To better assist you with this task, here is an example:
+  
+  [Start of Context]
+  {{ seed_context }}
+  [End of Context]
+  [Start of Question]
+  {{ seed_question }}
+  [End of Question]
+
+generation: |
+  Now generate {{num_samples}} such questions, remember to follow the principles mentioned above and use the same format as the examples. Remember to use the same style and format as the example above. Do not return any contexts or answers, only the questions. Return each question between [Start of Question] and [End of Question] tags.
+
+  [Start of Context]
+  {{ context }}
+  [End of Context]
+
+start_tags: ["[Start of Question]"]
+end_tags: ["[End of Question]"]

sdg_hub/configs/skills/grounded_responses.yaml

@@ -0,0 +1,59 @@
+system: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
+
+introduction: Your task is to faithfully follow the user's prompt, given context and generate a response. 
+principles: |
+  Please follow these guiding principles when generating responses:
+  * Use proper grammar and punctuation.
+  * Always generate safe and respectful content. Do not generate content that is harmful, abusive, or offensive.
+  * Always generate content that is factually accurate and relevant to the prompt.
+  * Strictly adhere to the prompt and generate responses in the same style and format as the example.
+  * Most importantly all the responses should be grounded in the context provided.
+  * Return the response between [Start of Response] and [End of Response] tags.
+
+examples: |
+  To better assist you with this task, here are some examples:
+  {% if seed_samples is defined %}
+  {% for sample in seed_samples %}
+  [Start of Context]
+  {{ sample.seed_context }}
+  [End of Context]
+
+  [Start of Question]
+  {{ sample.seed_question }}
+  [End of Question]
+
+  [Start of Response]
+  {{ sample.seed_response }}
+  [End of Response]
+  {% endfor %}
+  {% else %}
+  [Start of Context]
+  {{ seed_context }}
+  [End of Context]
+
+  [Start of Question]
+  {{ seed_question }}
+  [End of Question]
+
+  [Start of Response]
+  {{ seed_response }}
+  [End of Response]
+  {% endif %}
+
+generation: |
+  Now generate a response to the following prompt. Remember to use the same style and format as the example above. 
+  Return the response between [Start of Response] and [End of Response] tags.
+
+  [Start of Context]
+  {{ context }}
+  [End of Context]
+
+  [Start of Question]
+  {{ question }}
+  [End of Question]
+
+  Return the response between [Start of Response] and [End of Response] tags.
+
+
+start_tags: ["[Start of Response]"]
+end_tags: ["[End of Response]"]

sdg_hub/configs/skills/judge.yaml

@@ -0,0 +1,53 @@
+system: You are a helpful and precise assistant for checking the quality of the answer.
+
+introduction: Please act as an impartial judge and evaluate the quality of the responses provided by two AI assistants to the user question displayed below. You should choose the assistant that follows the user's instructions and answers the user's question better. Your evaluation should consider factors such as the helpfulness, relevance, accuracy, depth, creativity, and level of detail of their responses. 
+
+principles: |
+  * Begin your evaluation by comparing the two responses and provide a short judgement. 
+  * Avoid any position biases and ensure that the order in which the responses were presented does not influence your decision. 
+  * Do not allow the length of the responses to influence your evaluation. 
+  * Be as objective as possible. 
+
+examples: |
+  To better assist you with this, heres an example:
+
+  [Start of User Question]
+  What are the benefits of regular exercise?
+  [End of User Question]
+
+  [Start of Assistant A Answer]
+  Regular exercise provides numerous benefits including improved cardiovascular health, enhanced muscle strength, better mental health, and weight management. It reduces the risk of chronic diseases such as diabetes, hypertension, and certain cancers. Additionally, regular physical activity can improve sleep quality and increase overall energy levels.
+  [The End of Assistant A's Answer]
+
+  [Start of Assistant B Answer]
+  Engaging in regular exercise has several advantages. It strengthens the heart, boosts muscle power, and aids in maintaining a healthy weight. Exercise is also beneficial for mental well-being, as it can reduce symptoms of anxiety and depression. Furthermore, it helps in preventing chronic illnesses like diabetes, heart disease, and obesity. Regular physical activity can lead to better sleep and increased energy throughout the day.
+  [The End of Assistant B Answer]
+
+  [Start of Judgement]
+  Both responses highlight the benefits of regular exercise, including cardiovascular health, muscle strength, mental health, and chronic disease prevention. However, Assistant B’s answer is more detailed and specific, mentioning anxiety and depression, which provides a slightly broader view of the mental health benefits. Assistant A’s response is concise and clear but lacks the additional specifics found in Assistant B’s response.
+  [End of Judgement]
+
+  [Start of Verdict]
+  Assistant B
+  [End of Verdict]
+    
+
+generation: |
+  After providing your judgement, output your final verdict by strictly following this format: 
+  * Return the judgement between [Start of Judgement] and [End of Judgement] tags
+  * Return the final verdict between [Start of Verdict] and [End of Verdict] tags. Respond with Assistant A if assistant A is better, Assistant B if assistant B is better
+
+  [Start of User Question]
+  {{ question }}
+  [End of User Question]
+
+  [Start of Assistant A Answer]
+  {{ response }}
+  [The End of Assistant A Answer]
+
+  [Start of Assistant B Answer]
+  {{ revised_response }}
+  [The End of Assistant B Answer]
+
+start_tags: ["[Start of Judgement]", "[Start of Verdict]"]
+end_tags: ["[End of Judgement]", "[End of Verdict]"]

sdg_hub/configs/skills/planner.yaml

@@ -0,0 +1,67 @@
+system: |
+  You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
+
+introduction: |
+  Assume the role of an expert planner. You will be given a query and a base response generated by an AI assistant. Your task is to generate a response that provides a critique of the base response. 
+  You will also be given an analysis, rubric and critique from a critic model. Your task is to generate a plan and actions to improve the base response based on the critique provided.
+
+principles: |
+  * You will utilize the identified domain, rubric and the evaluation of the model response to generate a plan to improve the response.
+  * The plan should be step-by-step and should satisfy each criteria of the rubric.
+  * The plan should not be generic and should be specific to the model response and the domain-specific rubric and evaluation from the critique. 
+  * The generated plan should be actionable and feasible.
+  * Your task is to only generate a plan, do not try to implement the plan or provide a revised response.
+  * Return the plan between [Start of Plan] and [End of Plan] tags.
+
+examples: |
+  To help you understand the task, here is an example:
+
+  [Start of Query]
+  {{ icl_query }}
+  [End of Query]
+
+  [Start of Response]
+  {{ icl_response }}
+  [End of Response]
+
+  [Start of Analysis]
+  {{ icl_analysis }}
+  [End of Analysis]
+
+  [Start of Rubric]
+  {{ icl_rubric }}
+  [End of Rubric]
+
+  [Start of Critique]
+  {{ icl_critique }}
+  [End of Critique]
+
+  [Start of Plan]
+  {{ icl_plan }}
+  [End of Plan]
+
+generation: |
+  Now it's your turn to improve the response to the following query. Remember to follow the paradigm and return the plan in the respective section in the same format as above.
+
+  [Start of Query]
+  {{ question }}
+  [End of Query]
+
+  [Start of Response]
+  {{ response }}
+  [End of Response]
+
+  [Start of Analysis]
+  {{ analysis }}
+  [End of Analysis]
+
+  [Start of Rubric]
+  {{ rubric }}
+  [End of Rubric]
+
+  [Start of Critique]
+  {{ critique }}
+  [End of Critique]
+
+start_tags: ["[Start of Plan]"]
+end_tags: ["[End of Plan]"]

sdg_hub/configs/skills/respond.yaml

@@ -0,0 +1,8 @@
+system: 
+introduction: 
+principles: 
+examples: 
+generation: |
+  {{ prompt }}
+start_tags: [""]
+end_tags: [""]

sdg_hub/configs/skills/revised_responder.yaml

@@ -0,0 +1,78 @@
+system: |
+  You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
+
+introduction: |
+  Your task is to revise the response to an user query. You will be given a query from an user and a response from a model. Your task is to provide a better response based on the plan given.
+
+principles: |
+  * You will revise the model response according to the plan given.
+  * The revised response should adhere to the plan and should be better than the original response.
+  * Note that the revised response will be evaluated by a human expert and should thus be of high quality.
+  * Do not have any irrelevant information in the revised response. Specifically do not include any self-referential information in the revised response.
+  * Your response should only include the revised response. Please do not include any other information like the query, analysis, rubric, etc.
+  * Your response will become invalid if it contains any meta-review about how you are revising the response. So please avoid including any such information.
+  * If the plan mentions that there is no need to provide a plan for improvement, simply return the original response as the revised response.
+  * Return the revised response between [Start of Revised Response] and [End of Revised Response] tags.
+
+examples: |
+  To help you understand the task, here is an example:
+
+  [Start of Query]
+  {{ icl_query }}
+  [End of Query]
+
+  [Start of Response]
+  {{ icl_response }}
+  [End of Response]
+
+  [Start of Analysis]
+  {{ icl_analysis }}
+  [End of Analysis]
+
+  [Start of Rubric]
+  {{ icl_rubric }}
+  [End of Rubric]
+
+  [Start of Critique]
+  {{ icl_critique }}
+  [End of Critique]
+
+  [Start of Plan]
+  {{ icl_plan }}
+  [End of Plan]
+
+  [Start of Revised Response]
+  {{ icl_revised_response }}
+  [End of Revised Response]
+
+generation: |
+  Now it's your turn to revise the response to the following query. Remember to follow the paradigm and return the revised response in the respective section in the same format as above. Strictly do not include any meta-review or meta-information about how the response was improved or revised. Your response should only include the revised response. You will be heavily penalized if you include any information about the revision process or if you have any reference about how you revised the response.
+
+  [Start of Query]
+  {{ question }}
+  [End of Query]
+
+  [Start of Response]
+  {{ response }}
+  [End of Response]
+
+  [Start of Analysis]
+  {{ analysis }}
+  [End of Analysis]
+
+  [Start of Rubric]
+  {{ rubric }}
+  [End of Rubric]
+
+  [Start of Critique]
+  {{ critique }}
+  [End of Critique]
+
+  [Start of Plan]
+  {{ plan }}
+  [End of Plan]
+
+  Start your response with the tag [Start of Revised Response] and end it with the tag [End of Revised Response].
+
+start_tags: ["[Start of Revised Response]"]
+end_tags: ["[End of Revised Response]"]

sdg_hub/configs/skills/router.yaml

@@ -0,0 +1,12 @@
+system: ""
+
+introduction: ""
+
+principles: ""
+
+examples: ""
+
+generation: "{{ question }}"
+
+start_tags: [""]
+end_tags: [""]

sdg_hub/configs/skills/simple_generate_qa_freeform.yaml

@@ -0,0 +1,27 @@
+system: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
+
+introduction: Develop a series of question and answer pairs to perform a task.
+
+principles: |
+  Here are the requirements:
+  1. Try not to repeat the verb for each instruction to maximize diversity.
+  2. The language used for the instruction also should be diverse. For example, you should combine questions with imperative instructions.
+  3. The type of instructions should be similar to provided examples. The generated instruction and the output should be grounded in the provided document.
+  4. A GPT language model should be able to complete the instruction. For example, do not ask the assistant to create any visual or audio output. For another example, do not ask the assistant to wake you up at 5pm or set a reminder because it cannot perform any action.
+  5. The instructions should be in English.
+  6. The instructions should be 1 to 2 sentences long. Either an imperative sentence or a question is permitted.
+  7. The output should be an appropriate response to the input and the instruction. Long outputs are preferable.
+
+examples: |
+  The task is {{task_description}}.
+
+  Here is an example to help you understand the type of questions that are asked for:
+
+  {{seed_question}}
+  {{seed_response}}
+
+generation: |
+  Provide a single question and answer pair based on the examples.
+
+start_tags: [""]
+end_tags: [""]

sdg_hub/configs/skills/simple_generate_qa_grounded.yaml

@@ -0,0 +1,31 @@
+system: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
+
+introduction: Develop a series of question and answer pairs to perform a task.
+
+principles: |
+  Here are the requirements:
+  1. Try not to repeat the verb for each instruction to maximize diversity.
+  2. The language used for the instruction also should be diverse. For example, you should combine questions with imperative instructions.
+  3. The type of instructions should be similar to provided examples. The generated instruction and the output should be grounded in the provided document.
+  4. A GPT language model should be able to complete the instruction. For example, do not ask the assistant to create any visual or audio output. For another example, do not ask the assistant to wake you up at 5pm or set a reminder because it cannot perform any action.
+  5. The instructions should be in English.
+  6. The instructions should be 1 to 2 sentences long. Either an imperative sentence or a question is permitted.
+  7. The output should be an appropriate response to the input and the instruction. Long outputs are preferable.
+
+examples: |
+  The task is {{task_description}}.
+
+  Here is some context for the example question:
+
+  {{seed_context}}
+
+  Here is an example to help you understand the type of questions that are asked for:
+
+  {{seed_question}}
+  {{seed_response}}
+
+generation: |
+  Provide a single question and answer pair based on the example.
+
+start_tags: [""]
+end_tags: [""]