sdg-hub 0.1.3__py3-none-any.whl → 0.2.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- sdg_hub/__init__.py +28 -1
- sdg_hub/_version.py +2 -2
- sdg_hub/core/__init__.py +22 -0
- sdg_hub/core/blocks/__init__.py +58 -0
- sdg_hub/core/blocks/base.py +313 -0
- sdg_hub/core/blocks/deprecated_blocks/__init__.py +29 -0
- sdg_hub/core/blocks/deprecated_blocks/combine_columns.py +93 -0
- sdg_hub/core/blocks/deprecated_blocks/duplicate_columns.py +88 -0
- sdg_hub/core/blocks/deprecated_blocks/filter_by_value.py +103 -0
- sdg_hub/core/blocks/deprecated_blocks/flatten_columns.py +94 -0
- sdg_hub/core/blocks/deprecated_blocks/llmblock.py +479 -0
- sdg_hub/core/blocks/deprecated_blocks/rename_columns.py +88 -0
- sdg_hub/core/blocks/deprecated_blocks/sample_populator.py +58 -0
- sdg_hub/core/blocks/deprecated_blocks/selector.py +97 -0
- sdg_hub/core/blocks/deprecated_blocks/set_to_majority_value.py +88 -0
- sdg_hub/core/blocks/evaluation/__init__.py +9 -0
- sdg_hub/core/blocks/evaluation/evaluate_faithfulness_block.py +564 -0
- sdg_hub/core/blocks/evaluation/evaluate_relevancy_block.py +564 -0
- sdg_hub/core/blocks/evaluation/verify_question_block.py +564 -0
- sdg_hub/core/blocks/filtering/__init__.py +12 -0
- sdg_hub/core/blocks/filtering/column_value_filter.py +188 -0
- sdg_hub/core/blocks/llm/__init__.py +25 -0
- sdg_hub/core/blocks/llm/client_manager.py +398 -0
- sdg_hub/core/blocks/llm/config.py +336 -0
- sdg_hub/core/blocks/llm/error_handler.py +368 -0
- sdg_hub/core/blocks/llm/llm_chat_block.py +542 -0
- sdg_hub/core/blocks/llm/prompt_builder_block.py +368 -0
- sdg_hub/core/blocks/llm/text_parser_block.py +310 -0
- sdg_hub/core/blocks/registry.py +331 -0
- sdg_hub/core/blocks/transform/__init__.py +23 -0
- sdg_hub/core/blocks/transform/duplicate_columns.py +88 -0
- sdg_hub/core/blocks/transform/index_based_mapper.py +225 -0
- sdg_hub/core/blocks/transform/melt_columns.py +126 -0
- sdg_hub/core/blocks/transform/rename_columns.py +69 -0
- sdg_hub/core/blocks/transform/text_concat.py +102 -0
- sdg_hub/core/blocks/transform/uniform_col_val_setter.py +101 -0
- sdg_hub/core/flow/__init__.py +20 -0
- sdg_hub/core/flow/base.py +980 -0
- sdg_hub/core/flow/metadata.py +344 -0
- sdg_hub/core/flow/migration.py +187 -0
- sdg_hub/core/flow/registry.py +330 -0
- sdg_hub/core/flow/validation.py +265 -0
- sdg_hub/{utils → core/utils}/__init__.py +6 -4
- sdg_hub/{utils → core/utils}/datautils.py +1 -3
- sdg_hub/core/utils/error_handling.py +208 -0
- sdg_hub/{utils → core/utils}/path_resolution.py +2 -2
- sdg_hub/flows/qa_generation/document_grounded_qa/multi_summary_qa/instructlab/atomic_facts.yaml +40 -0
- sdg_hub/flows/qa_generation/document_grounded_qa/multi_summary_qa/instructlab/detailed_summary.yaml +13 -0
- sdg_hub/flows/qa_generation/document_grounded_qa/multi_summary_qa/instructlab/evaluate_faithfulness.yaml +64 -0
- sdg_hub/flows/qa_generation/document_grounded_qa/multi_summary_qa/instructlab/evaluate_question.yaml +29 -0
- sdg_hub/flows/qa_generation/document_grounded_qa/multi_summary_qa/instructlab/evaluate_relevancy.yaml +81 -0
- sdg_hub/flows/qa_generation/document_grounded_qa/multi_summary_qa/instructlab/extractive_summary.yaml +13 -0
- sdg_hub/flows/qa_generation/document_grounded_qa/multi_summary_qa/instructlab/flow.yaml +191 -0
- sdg_hub/flows/qa_generation/document_grounded_qa/multi_summary_qa/instructlab/generate_questions_responses.yaml +54 -0
- sdg_hub-0.2.0.dist-info/METADATA +218 -0
- sdg_hub-0.2.0.dist-info/RECORD +63 -0
- sdg_hub/blocks/__init__.py +0 -42
- sdg_hub/blocks/block.py +0 -96
- sdg_hub/blocks/llmblock.py +0 -375
- sdg_hub/blocks/openaichatblock.py +0 -556
- sdg_hub/blocks/utilblocks.py +0 -597
- sdg_hub/checkpointer.py +0 -139
- sdg_hub/configs/annotations/cot_reflection.yaml +0 -34
- sdg_hub/configs/annotations/detailed_annotations.yaml +0 -28
- sdg_hub/configs/annotations/detailed_description.yaml +0 -10
- sdg_hub/configs/annotations/detailed_description_icl.yaml +0 -32
- sdg_hub/configs/annotations/simple_annotations.yaml +0 -9
- sdg_hub/configs/knowledge/__init__.py +0 -0
- sdg_hub/configs/knowledge/atomic_facts.yaml +0 -46
- sdg_hub/configs/knowledge/auxilary_instructions.yaml +0 -35
- sdg_hub/configs/knowledge/detailed_summary.yaml +0 -18
- sdg_hub/configs/knowledge/evaluate_faithfulness.yaml +0 -68
- sdg_hub/configs/knowledge/evaluate_question.yaml +0 -38
- sdg_hub/configs/knowledge/evaluate_relevancy.yaml +0 -84
- sdg_hub/configs/knowledge/extractive_summary.yaml +0 -18
- sdg_hub/configs/knowledge/generate_code_questions_responses.yaml +0 -39
- sdg_hub/configs/knowledge/generate_questions.yaml +0 -82
- sdg_hub/configs/knowledge/generate_questions_responses.yaml +0 -56
- sdg_hub/configs/knowledge/generate_responses.yaml +0 -86
- sdg_hub/configs/knowledge/mcq_generation.yaml +0 -83
- sdg_hub/configs/knowledge/router.yaml +0 -12
- sdg_hub/configs/knowledge/simple_generate_qa.yaml +0 -34
- sdg_hub/configs/reasoning/__init__.py +0 -0
- sdg_hub/configs/reasoning/dynamic_cot.yaml +0 -40
- sdg_hub/configs/skills/__init__.py +0 -0
- sdg_hub/configs/skills/analyzer.yaml +0 -48
- sdg_hub/configs/skills/annotation.yaml +0 -36
- sdg_hub/configs/skills/contexts.yaml +0 -28
- sdg_hub/configs/skills/critic.yaml +0 -60
- sdg_hub/configs/skills/evaluate_freeform_pair.yaml +0 -111
- sdg_hub/configs/skills/evaluate_freeform_questions.yaml +0 -78
- sdg_hub/configs/skills/evaluate_grounded_pair.yaml +0 -119
- sdg_hub/configs/skills/evaluate_grounded_questions.yaml +0 -51
- sdg_hub/configs/skills/freeform_questions.yaml +0 -34
- sdg_hub/configs/skills/freeform_responses.yaml +0 -39
- sdg_hub/configs/skills/grounded_questions.yaml +0 -38
- sdg_hub/configs/skills/grounded_responses.yaml +0 -59
- sdg_hub/configs/skills/icl_examples/STEM.yaml +0 -56
- sdg_hub/configs/skills/icl_examples/__init__.py +0 -0
- sdg_hub/configs/skills/icl_examples/coding.yaml +0 -97
- sdg_hub/configs/skills/icl_examples/extraction.yaml +0 -36
- sdg_hub/configs/skills/icl_examples/humanities.yaml +0 -71
- sdg_hub/configs/skills/icl_examples/math.yaml +0 -85
- sdg_hub/configs/skills/icl_examples/reasoning.yaml +0 -30
- sdg_hub/configs/skills/icl_examples/roleplay.yaml +0 -45
- sdg_hub/configs/skills/icl_examples/writing.yaml +0 -80
- sdg_hub/configs/skills/judge.yaml +0 -53
- sdg_hub/configs/skills/planner.yaml +0 -67
- sdg_hub/configs/skills/respond.yaml +0 -8
- sdg_hub/configs/skills/revised_responder.yaml +0 -78
- sdg_hub/configs/skills/router.yaml +0 -59
- sdg_hub/configs/skills/simple_generate_qa_freeform.yaml +0 -27
- sdg_hub/configs/skills/simple_generate_qa_grounded.yaml +0 -31
- sdg_hub/flow.py +0 -477
- sdg_hub/flow_runner.py +0 -450
- sdg_hub/flows/generation/knowledge/mmlu_bench.yaml +0 -13
- sdg_hub/flows/generation/knowledge/simple_knowledge.yaml +0 -12
- sdg_hub/flows/generation/knowledge/synth_knowledge.yaml +0 -89
- sdg_hub/flows/generation/knowledge/synth_knowledge1.5.yaml +0 -148
- sdg_hub/flows/generation/skills/improve_responses.yaml +0 -103
- sdg_hub/flows/generation/skills/simple_freeform_skill.yaml +0 -12
- sdg_hub/flows/generation/skills/simple_grounded_skill.yaml +0 -12
- sdg_hub/flows/generation/skills/synth_grounded_skills.yaml +0 -80
- sdg_hub/flows/generation/skills/synth_skills.yaml +0 -59
- sdg_hub/pipeline.py +0 -121
- sdg_hub/prompts.py +0 -74
- sdg_hub/registry.py +0 -122
- sdg_hub/sdg.py +0 -206
- sdg_hub/utils/config_validation.py +0 -91
- sdg_hub/utils/error_handling.py +0 -94
- sdg_hub/utils/validation_result.py +0 -10
- sdg_hub-0.1.3.dist-info/METADATA +0 -190
- sdg_hub-0.1.3.dist-info/RECORD +0 -89
- sdg_hub/{logger_config.py → core/utils/logger_config.py} +1 -1
- /sdg_hub/{configs/__init__.py → flows/qa_generation/document_grounded_qa/multi_summary_qa/instructlab/README.md} +0 -0
- /sdg_hub/{configs/annotations → flows/qa_generation/document_grounded_qa/multi_summary_qa/instructlab}/__init__.py +0 -0
- {sdg_hub-0.1.3.dist-info → sdg_hub-0.2.0.dist-info}/WHEEL +0 -0
- {sdg_hub-0.1.3.dist-info → sdg_hub-0.2.0.dist-info}/licenses/LICENSE +0 -0
- {sdg_hub-0.1.3.dist-info → sdg_hub-0.2.0.dist-info}/top_level.txt +0 -0
@@ -1,111 +0,0 @@
|
|
1
|
-
system: You are a highly knowledgeable and impartial AI Assistant tasked with evaluating the quality of responses to user questions.
|
2
|
-
|
3
|
-
introduction: |
|
4
|
-
Please act as an objective evaluator and assess whether the AI Assistant's answer correctly follows formatting requirements and answers the user's question. Use the scoring rubric below and assign a score from 1 to 3.
|
5
|
-
|
6
|
-
principles: |
|
7
|
-
Use the following 3-point scale to score the answer:
|
8
|
-
|
9
|
-
**1 — Poor Quality**
|
10
|
-
- The output is incorrectly formatted, contains hallucinations, or ignores required tags/structure.
|
11
|
-
- The answer may be off-topic, incomplete, or inconsistent with the task.
|
12
|
-
- The output introduces unsafe or inappropriate content, or violates structural instructions.
|
13
|
-
|
14
|
-
**2 — Acceptable but Minimal**
|
15
|
-
- The answer is structurally valid and safe, but lacks polish, clarity, or minor formatting correctness.
|
16
|
-
- It meets the task minimally, but may show inconsistencies or lack proper use of spacing, tags, or conventions.
|
17
|
-
- It's not harmful, but also not ready for use without revision.
|
18
|
-
|
19
|
-
**3 — Excellent Answer**
|
20
|
-
- The answer is fully correct, clearly written, and **strictly adheres to formatting instructions**.
|
21
|
-
- It uses all required tags, markdown syntax, or structure accurately and consistently.
|
22
|
-
- It directly fulfills the task with precision, professionalism, and completeness.
|
23
|
-
|
24
|
-
examples: |
|
25
|
-
Example 1 — Score: 1 (Poor Formatting, Unsafe)
|
26
|
-
|
27
|
-
Task Description: Generate a markdown table of 3 planets with columns: Name, Gravity, and Moons.
|
28
|
-
|
29
|
-
[Start of Question]
|
30
|
-
Create a markdown table with 3 planets and their gravity/moons.
|
31
|
-
[End of Question]
|
32
|
-
|
33
|
-
[Start of Answer]
|
34
|
-
Mars - 3.7 - 2; Earth - 9.8 - 1; Jupiter - 24.8 - 79
|
35
|
-
[End of Answer]
|
36
|
-
|
37
|
-
[Start of Evaluation]
|
38
|
-
The answer is improperly formatted (not a markdown table) and unstructured. It violates task instructions both structurally and in tone.
|
39
|
-
[End of Evaluation]
|
40
|
-
|
41
|
-
[Start of Score]
|
42
|
-
1
|
43
|
-
[End of Score]
|
44
|
-
|
45
|
-
Example 2 — Score: 2 (Correct but Sloppy Formatting)
|
46
|
-
|
47
|
-
Task Description: Generate a markdown table of 3 planets with columns: Name, Gravity, and Moons.
|
48
|
-
|
49
|
-
[Start of Question]
|
50
|
-
Create a markdown table with 3 planets and their gravity/moons.
|
51
|
-
[End of Question]
|
52
|
-
|
53
|
-
[Start of Answer]
|
54
|
-
| Name | Gravity | Moons |
|
55
|
-
|-------|---------|-------|
|
56
|
-
| Mars | 3.7 | 2 |
|
57
|
-
| Earth| 9.8| 1 |
|
58
|
-
| Jupiter |24.8 |79|
|
59
|
-
[End of Answer]
|
60
|
-
|
61
|
-
[Start of Evaluation]
|
62
|
-
The table has correct content but inconsistent spacing and pipe alignment. It fulfills the task but lacks polish and readability. It's usable, but not clean or well-structured.
|
63
|
-
[End of Evaluation]
|
64
|
-
|
65
|
-
[Start of Score]
|
66
|
-
2
|
67
|
-
[End of Score]
|
68
|
-
|
69
|
-
Example 3 — Score: 3 (Flawless Markdown Table)
|
70
|
-
|
71
|
-
Task Description: Generate a markdown table of 3 planets with columns: Name, Gravity, and Moons.
|
72
|
-
|
73
|
-
[Start of Question]
|
74
|
-
Create a markdown table with 3 planets and their gravity/moons.
|
75
|
-
[End of Question]
|
76
|
-
|
77
|
-
[Start of Answer]
|
78
|
-
| Name | Gravity (m/s²) | Moons |
|
79
|
-
|----------|----------------|-------|
|
80
|
-
| Mars | 3.7 | 2 |
|
81
|
-
| Earth | 9.8 | 1 |
|
82
|
-
| Jupiter | 24.8 | 79 |
|
83
|
-
[End of Answer]
|
84
|
-
|
85
|
-
[Start of Evaluation]
|
86
|
-
The answer uses proper markdown syntax, alignment, and column headers. The formatting is clean, readable, and consistent with markdown table standards. It meets the task precisely.
|
87
|
-
[End of Evaluation]
|
88
|
-
|
89
|
-
[Start of Score]
|
90
|
-
3
|
91
|
-
[End of Score]
|
92
|
-
|
93
|
-
generation: |
|
94
|
-
Now begin your evaluation of the following QA pair. Use the rubric above and be objective and concise in your reasoning.
|
95
|
-
|
96
|
-
Task Description: {{task_description}}
|
97
|
-
|
98
|
-
[Start of Question]
|
99
|
-
{{ question }}
|
100
|
-
[End of Question]
|
101
|
-
|
102
|
-
[Start of Answer]
|
103
|
-
{{ response }}
|
104
|
-
[End of Answer]
|
105
|
-
|
106
|
-
* Provide your evaluation between [Start of Evaluation] and [End of Evaluation] tags.
|
107
|
-
* Provide the score between [Start of Score] and [End of Score] tags.
|
108
|
-
* Do not include any content outside these tags.
|
109
|
-
|
110
|
-
start_tags: ["[Start of Evaluation]", "[Start of Score]"]
|
111
|
-
end_tags: ["[End of Evaluation]", "[End of Score]"]
|
@@ -1,78 +0,0 @@
|
|
1
|
-
system: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
|
2
|
-
|
3
|
-
introduction: |
|
4
|
-
Please act as an impartial and detail-oriented evaluator of synthetic questions. Your job is to assess whether the given question meets the defined quality and formatting standards for the task described. Assign a score using a strict binary 0/1 scale.
|
5
|
-
|
6
|
-
principles: |
|
7
|
-
A valid question must satisfy **all** of the following requirements:
|
8
|
-
* The question should be answerable via text (not require visual/audio input).
|
9
|
-
* It must be **clearly relevant to the task description** ({{task_description}}).
|
10
|
-
* It should **not contain placeholder text**, incomplete sentences, or formatting artifacts.
|
11
|
-
|
12
|
-
If the question satisfies **all** of the above, assign a score of `1`. Otherwise, assign `0`.
|
13
|
-
|
14
|
-
examples: |
|
15
|
-
Example 1 - Valid question:
|
16
|
-
|
17
|
-
Task Description: Extract the main idea of a paragraph.
|
18
|
-
|
19
|
-
[Start of Question]
|
20
|
-
What is the central message conveyed by the paragraph?
|
21
|
-
[End of Question]
|
22
|
-
|
23
|
-
[Start of Evaluation]
|
24
|
-
The question is clear, concise, grammatically correct, and directly related to the task. It follows formatting rules and is appropriate in tone.
|
25
|
-
[End of Evaluation]
|
26
|
-
|
27
|
-
[Start of Score]
|
28
|
-
1
|
29
|
-
[End of Score]
|
30
|
-
|
31
|
-
Example 2 - Invalid question (bad formatting):
|
32
|
-
|
33
|
-
Task Description: Extract the main idea of a paragraph.
|
34
|
-
|
35
|
-
[Start of Question]
|
36
|
-
main idea??
|
37
|
-
[End of Question]
|
38
|
-
|
39
|
-
[Start of Evaluation]
|
40
|
-
The question lacks proper capitalization, punctuation, and complete sentence structure. It does not meet the formatting standards.
|
41
|
-
[End of Evaluation]
|
42
|
-
|
43
|
-
[Start of Score]
|
44
|
-
0
|
45
|
-
[End of Score]
|
46
|
-
|
47
|
-
Example 3 - Invalid question (off-topic):
|
48
|
-
|
49
|
-
Task Description: Extract the main idea of a paragraph.
|
50
|
-
|
51
|
-
[Start of Question]
|
52
|
-
What's your favorite type of movie and why?
|
53
|
-
[End of Question]
|
54
|
-
|
55
|
-
[Start of Evaluation]
|
56
|
-
The question is unrelated to the given task description. It fails the relevance requirement.
|
57
|
-
[End of Evaluation]
|
58
|
-
|
59
|
-
[Start of Score]
|
60
|
-
0
|
61
|
-
[End of Score]
|
62
|
-
|
63
|
-
generation: |
|
64
|
-
Here's the question you need to evaluate:
|
65
|
-
|
66
|
-
Task Description: {{task_description}}
|
67
|
-
|
68
|
-
[Start of Question]
|
69
|
-
{{question}}
|
70
|
-
[End of Question]
|
71
|
-
|
72
|
-
Now begin your evaluation:
|
73
|
-
* First, provide a brief explanation between [Start of Evaluation] and [End of Evaluation] tags.
|
74
|
-
* Then return a binary score (0 or 1) between [Start of Score] and [End of Score] tags.
|
75
|
-
* Do not include any content outside these tags.
|
76
|
-
|
77
|
-
start_tags: ["[Start of Evaluation]", "[Start of Score]"]
|
78
|
-
end_tags: ["[End of Evaluation]", "[End of Score]"]
|
@@ -1,119 +0,0 @@
|
|
1
|
-
system: You are a highly knowledgeable and impartial AI Assistant tasked with evaluating the quality of responses to user questions.
|
2
|
-
|
3
|
-
introduction: |
|
4
|
-
Please act as an objective evaluator and assess the quality of the AI Assistant's answer to the user's question. Your job is to determine whether the answer is complete, correct, relevant, safe, and well-written according to the task. Use the scoring rubric below and assign a score from 1 to 3.
|
5
|
-
|
6
|
-
principles: |
|
7
|
-
Use the following 3-point scale to score the answer:
|
8
|
-
|
9
|
-
**1 — Poor Quality**
|
10
|
-
- The answer is factually incorrect, off-topic, incomplete, or nonsensical.
|
11
|
-
- It may contain hallucinations, hanging statements, or irrelevant content.
|
12
|
-
- It does not align with the task description or question intent.
|
13
|
-
|
14
|
-
**2 — Acceptable but Minimal**
|
15
|
-
- The answer is correct and safe, but short and lacks depth or clarity.
|
16
|
-
- It may be technically correct but overly terse or missing explanation.
|
17
|
-
- It satisfies the user's question at a surface level without context.
|
18
|
-
|
19
|
-
**3 — Excellent Answer**
|
20
|
-
- The answer is complete, correct, relevant, and insightful.
|
21
|
-
- It provides clear reasoning, well-structured writing, and deep understanding.
|
22
|
-
- It directly addresses the question with expert-level knowledge.
|
23
|
-
|
24
|
-
examples: |
|
25
|
-
Example 1 — Score: 1 (Poor Quality)
|
26
|
-
|
27
|
-
Task Description: Summarize the key point of a news paragraph in one sentence.
|
28
|
-
|
29
|
-
[Start of Context]
|
30
|
-
The mayor of Springfield unveiled a new plan to reduce urban pollution, which includes a ban on gas-powered scooters, expansion of bike lanes, and free public transit passes for residents.
|
31
|
-
[End of Context]
|
32
|
-
|
33
|
-
[Start of Question]
|
34
|
-
What is the main point of the paragraph?
|
35
|
-
[End of Question]
|
36
|
-
|
37
|
-
[Start of Answer]
|
38
|
-
Scooters are fun but kind of dangerous if you go too fast lol.
|
39
|
-
[End of Answer]
|
40
|
-
|
41
|
-
[Start of Evaluation]
|
42
|
-
The answer is irrelevant and nonsensical in context. It does not reflect the content of the paragraph and includes casual language and commentary that breaks task intent. It is an example of a hallucinated, off-topic response.
|
43
|
-
[End of Evaluation]
|
44
|
-
|
45
|
-
[Start of Score]
|
46
|
-
1
|
47
|
-
[End of Score]
|
48
|
-
|
49
|
-
Example 2 — Score: 2 (Acceptable but Minimal)
|
50
|
-
|
51
|
-
Task Description: Summarize the key point of a news paragraph in one sentence.
|
52
|
-
|
53
|
-
[Start of Context]
|
54
|
-
The mayor of Springfield unveiled a new plan to reduce urban pollution, which includes a ban on gas-powered scooters, expansion of bike lanes, and free public transit passes for residents.
|
55
|
-
[End of Context]
|
56
|
-
|
57
|
-
[Start of Question]
|
58
|
-
What is the main point of the paragraph?
|
59
|
-
[End of Question]
|
60
|
-
|
61
|
-
[Start of Answer]
|
62
|
-
The city is taking steps to reduce pollution.
|
63
|
-
[End of Answer]
|
64
|
-
|
65
|
-
[Start of Evaluation]
|
66
|
-
The answer is factually correct and safe, but vague. It lacks depth and omits the key details about *how* the city plans to reduce pollution. It does not demonstrate full understanding of the context or question.
|
67
|
-
[End of Evaluation]
|
68
|
-
|
69
|
-
[Start of Score]
|
70
|
-
2
|
71
|
-
[End of Score]
|
72
|
-
|
73
|
-
Example 3 — Score: 3 (Excellent Answer)
|
74
|
-
|
75
|
-
Task Description: Summarize the key point of a news paragraph in one sentence.
|
76
|
-
|
77
|
-
[Start of Context]
|
78
|
-
The mayor of Springfield unveiled a new plan to reduce urban pollution, which includes a ban on gas-powered scooters, expansion of bike lanes, and free public transit passes for residents.
|
79
|
-
[End of Context]
|
80
|
-
|
81
|
-
[Start of Question]
|
82
|
-
What is the main point of the paragraph?
|
83
|
-
[End of Question]
|
84
|
-
|
85
|
-
[Start of Answer]
|
86
|
-
The mayor of Springfield introduced a pollution-reduction plan that bans gas-powered scooters, expands bike lanes, and provides free transit passes to residents.
|
87
|
-
[End of Answer]
|
88
|
-
|
89
|
-
[Start of Evaluation]
|
90
|
-
The answer is complete, relevant, and clearly written. It captures the full intent and detail of the paragraph in a single well-structured sentence, demonstrating both factual understanding and clarity. It aligns perfectly with the task instruction.
|
91
|
-
[End of Evaluation]
|
92
|
-
|
93
|
-
[Start of Score]
|
94
|
-
3
|
95
|
-
[End of Score]
|
96
|
-
|
97
|
-
generation: |
|
98
|
-
Now begin your evaluation of the following QA pair. Use the rubric above and be objective and concise in your reasoning.
|
99
|
-
|
100
|
-
Task Description: {{task_description}}
|
101
|
-
|
102
|
-
[Start of Context]
|
103
|
-
{{ context }}
|
104
|
-
[End of Context]
|
105
|
-
|
106
|
-
[Start of Question]
|
107
|
-
{{ question }}
|
108
|
-
[End of Question]
|
109
|
-
|
110
|
-
[Start of Answer]
|
111
|
-
{{ response }}
|
112
|
-
[End of Answer]
|
113
|
-
|
114
|
-
* Provide your evaluation between [Start of Evaluation] and [End of Evaluation] tags.
|
115
|
-
* Provide the score between [Start of Score] and [End of Score] tags.
|
116
|
-
* Do not include any content outside these tags.
|
117
|
-
|
118
|
-
start_tags: ["[Start of Evaluation]", "[Start of Score]"]
|
119
|
-
end_tags: ["[End of Evaluation]", "[End of Score]"]
|
@@ -1,51 +0,0 @@
|
|
1
|
-
system: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
|
2
|
-
|
3
|
-
introduction: |
|
4
|
-
Please act as an impartial judge and evaluate the questions generated by an AI assistant displayed below. Evaluate whether or not the question is a good question of how AI Assistant should respond to the user's instruction. Please assign a score using a binary 0/1 scale.
|
5
|
-
|
6
|
-
principles: |
|
7
|
-
Here are the requirements:
|
8
|
-
* A large language model should be able to complete the question. For example, do not ask the assistant to create any visual or audio output. For another example, do not ask the assistant to wake you up at 5pm or set a reminder because it cannot perform any action.
|
9
|
-
* The questions should be in English.
|
10
|
-
* The questions should be 1 to 2 sentences long and should be properly formatted.
|
11
|
-
* The question should not be offensive, abusive, or harmful. It should be safe and respectful.
|
12
|
-
* The question should be relevant to the task given - {{ task_description }}.
|
13
|
-
* Most importantly all the questions should be grounded in the context provided and should be answerable solely based on the provided context.
|
14
|
-
|
15
|
-
If the question meets the above requirements, please rate it 1. If not, please rate it 0.
|
16
|
-
|
17
|
-
examples: |
|
18
|
-
For better understanding of the task, here are some examples:
|
19
|
-
|
20
|
-
[Start of Context]
|
21
|
-
In recent years, electric vehicles (EVs) have gained popularity due to their environmental benefits and advancements in technology. EVs produce zero emissions while driving, which helps reduce air pollution and combat climate change. Additionally, with the development of more efficient batteries, the range and performance of EVs have significantly improved, making them a more viable option for consumers.
|
22
|
-
[End of Context]
|
23
|
-
|
24
|
-
[Start of Question]
|
25
|
-
How do electric vehicles help in reducing air pollution?
|
26
|
-
[End of Question]
|
27
|
-
|
28
|
-
[Start of Evaluation]
|
29
|
-
This question is properly formatted, respectful, and directly relevant to the task of understanding the benefits of electric vehicles. It is grounded in the provided context, which mentions that EVs produce zero emissions while driving, helping reduce air pollution. A large language model can provide an answer to this question based on the provided context.
|
30
|
-
[End of Evaluation]
|
31
|
-
|
32
|
-
[Start of Score]
|
33
|
-
1
|
34
|
-
[End of Score]
|
35
|
-
|
36
|
-
generation: |
|
37
|
-
Here's the context and question you need to evaluate. Return the evaluation between [Start of Evaluation] and [End of Evaluation] tags.
|
38
|
-
|
39
|
-
[Start of Context]
|
40
|
-
{{ context }}
|
41
|
-
[End of Context]
|
42
|
-
[Start of Question]
|
43
|
-
{{ question }}
|
44
|
-
[End of Question]
|
45
|
-
|
46
|
-
Begin your evaluation by providing a short explanation. Be as objective as possible. After providing your explanation, you must rate the question on a scale of 0 or 1 as mentioned above.
|
47
|
-
* Return the evaluation between [Start of Evaluation] and [End of Evaluation] tags.
|
48
|
-
* Return the score using a binary 0/1 scale between [Start of Score] and [End of Score] tags.
|
49
|
-
|
50
|
-
start_tags: ["[Start of Evaluation]", "[Start of Score]"]
|
51
|
-
end_tags: ["[End of Evaluation]", "[End of Score]"]
|
@@ -1,34 +0,0 @@
|
|
1
|
-
system: You are a highly intelligent and helpful AI Assistant specializing in generating well-crafted questions tailored to specific tasks.
|
2
|
-
|
3
|
-
introduction: |
|
4
|
-
Your task is to generate {{num_samples}} diverse and well-structured questions for the following task:
|
5
|
-
"{{task_description}}"
|
6
|
-
|
7
|
-
principles: |
|
8
|
-
Please follow these strict guidelines when generating each question:
|
9
|
-
* Each question must be directly related to the task description.
|
10
|
-
* Use correct grammar, spelling, and punctuation.
|
11
|
-
* Questions must be clear, natural-sounding, and human-like.
|
12
|
-
* Do **not** include answers, explanations, or commentary—only the question.
|
13
|
-
* Ensure **maximum diversity** in wording and perspective—no repetitive or template-based phrasing.
|
14
|
-
* Each question must strictly follow the **exact same format and style** as shown in the example.
|
15
|
-
* Do **not deviate** from the example structure in any way.
|
16
|
-
* Wrap each question between `[Start of Question]` and `[End of Question]` tags.
|
17
|
-
|
18
|
-
examples: |
|
19
|
-
To guide you, here is an example of a correctly formatted question:
|
20
|
-
|
21
|
-
[Start of Question]
|
22
|
-
{{seed_question}}
|
23
|
-
[End of Question]
|
24
|
-
|
25
|
-
generation: |
|
26
|
-
Now generate {{num_samples}} such questions.
|
27
|
-
Ensure that each one is:
|
28
|
-
- Fully relevant to the task description.
|
29
|
-
- Consistent with the example format.
|
30
|
-
- Clearly enclosed between [Start of Question] and [End of Question] tags.
|
31
|
-
Do not include any content outside these tags.
|
32
|
-
|
33
|
-
start_tags: ["[Start of Question]"]
|
34
|
-
end_tags: ["[End of Question]"]
|
@@ -1,39 +0,0 @@
|
|
1
|
-
system: You are a highly knowledgeable and precise AI Assistant tasked with generating high-quality responses based strictly on user prompts.
|
2
|
-
|
3
|
-
introduction: |
|
4
|
-
Your task is to carefully read and understand the user's prompt, and generate a well-structured response that directly addresses it.
|
5
|
-
|
6
|
-
principles: |
|
7
|
-
Please follow these strict guidelines when generating your response:
|
8
|
-
* The response must be directly relevant to the given question and accurately fulfill its intent.
|
9
|
-
* The response should be clear, natural, and human-like in tone.
|
10
|
-
* You must strictly follow the **exact structure and formatting** shown in the example.
|
11
|
-
* Do **not** add any extra commentary, explanations, greetings, or metadata.
|
12
|
-
* The response must be enclosed between `[Start of Response]` and `[End of Response]` tags.
|
13
|
-
* Do **not** deviate from the formatting under any circumstance.
|
14
|
-
|
15
|
-
examples: |
|
16
|
-
Below is an example of a properly formatted question and response:
|
17
|
-
|
18
|
-
[Start of Question]
|
19
|
-
{{seed_question}}
|
20
|
-
[End of Question]
|
21
|
-
|
22
|
-
[Start of Response]
|
23
|
-
{{seed_response}}
|
24
|
-
[End of Response]
|
25
|
-
|
26
|
-
generation: |
|
27
|
-
Now generate a response to the following prompt. Make sure the response:
|
28
|
-
- Accurately addresses the question.
|
29
|
-
- Strictly follows the example format and style.
|
30
|
-
- Appears **only** between [Start of Response] and [End of Response] tags.
|
31
|
-
|
32
|
-
[Start of Question]
|
33
|
-
{{question}}
|
34
|
-
[End of Question]
|
35
|
-
|
36
|
-
Return only the response between the tags. Do not add anything outside.
|
37
|
-
|
38
|
-
start_tags: ["[Start of Response]"]
|
39
|
-
end_tags: ["[End of Response]"]
|
@@ -1,38 +0,0 @@
|
|
1
|
-
system: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
|
2
|
-
|
3
|
-
introduction: |
|
4
|
-
You are asked to come up with a set of {{ num_samples }} diverse questions - {{ task_description }}.
|
5
|
-
|
6
|
-
principles: |
|
7
|
-
Please follow these guiding principles when generating responses:
|
8
|
-
* Use proper grammar and punctuation.
|
9
|
-
* Always generate safe and respectful content. Do not generate content that is harmful, abusive, or offensive.
|
10
|
-
* Always generate content that is factually accurate and relevant to the prompt.
|
11
|
-
* The questions should be clear and human-like.
|
12
|
-
* The questions should be diverse and cover a wide range of topics.
|
13
|
-
* The questions should not be template-based or generic, it should be very diverse.
|
14
|
-
* Simply return the questions, do not return any answers or explanations.
|
15
|
-
* Strictly adhere to the prompt and generate responses in the same style and format as the example.
|
16
|
-
* Most importantly all the questions should be grounded in the context provided and should be answerable solely based on the provided context.
|
17
|
-
* The question should address the task described in the prompt.
|
18
|
-
* Return each question between [Start of Question] and [End of Question] tags.
|
19
|
-
|
20
|
-
examples: |
|
21
|
-
To better assist you with this task, here is an example:
|
22
|
-
|
23
|
-
[Start of Context]
|
24
|
-
{{ seed_context }}
|
25
|
-
[End of Context]
|
26
|
-
[Start of Question]
|
27
|
-
{{ seed_question }}
|
28
|
-
[End of Question]
|
29
|
-
|
30
|
-
generation: |
|
31
|
-
Now generate {{num_samples}} such questions, remember to follow the principles mentioned above and use the same format as the examples. Remember to use the same style and format as the example above. Do not return any contexts or answers, only the questions. Return each question between [Start of Question] and [End of Question] tags.
|
32
|
-
|
33
|
-
[Start of Context]
|
34
|
-
{{ context }}
|
35
|
-
[End of Context]
|
36
|
-
|
37
|
-
start_tags: ["[Start of Question]"]
|
38
|
-
end_tags: ["[End of Question]"]
|
@@ -1,59 +0,0 @@
|
|
1
|
-
system: You are a very knowledgeable AI Assistant that will faithfully assist the user with their task.
|
2
|
-
|
3
|
-
introduction: Your task is to faithfully follow the user's prompt, given context and generate a response.
|
4
|
-
principles: |
|
5
|
-
Please follow these guiding principles when generating responses:
|
6
|
-
* Use proper grammar and punctuation.
|
7
|
-
* Always generate safe and respectful content. Do not generate content that is harmful, abusive, or offensive.
|
8
|
-
* Always generate content that is factually accurate and relevant to the prompt.
|
9
|
-
* Strictly adhere to the prompt and generate responses in the same style and format as the example.
|
10
|
-
* Most importantly all the responses should be grounded in the context provided.
|
11
|
-
* Return the response between [Start of Response] and [End of Response] tags.
|
12
|
-
|
13
|
-
examples: |
|
14
|
-
To better assist you with this task, here are some examples:
|
15
|
-
{% if seed_samples is defined %}
|
16
|
-
{% for sample in seed_samples %}
|
17
|
-
[Start of Context]
|
18
|
-
{{ sample.seed_context }}
|
19
|
-
[End of Context]
|
20
|
-
|
21
|
-
[Start of Question]
|
22
|
-
{{ sample.seed_question }}
|
23
|
-
[End of Question]
|
24
|
-
|
25
|
-
[Start of Response]
|
26
|
-
{{ sample.seed_response }}
|
27
|
-
[End of Response]
|
28
|
-
{% endfor %}
|
29
|
-
{% else %}
|
30
|
-
[Start of Context]
|
31
|
-
{{ seed_context }}
|
32
|
-
[End of Context]
|
33
|
-
|
34
|
-
[Start of Question]
|
35
|
-
{{ seed_question }}
|
36
|
-
[End of Question]
|
37
|
-
|
38
|
-
[Start of Response]
|
39
|
-
{{ seed_response }}
|
40
|
-
[End of Response]
|
41
|
-
{% endif %}
|
42
|
-
|
43
|
-
generation: |
|
44
|
-
Now generate a response to the following prompt. Remember to use the same style and format as the example above.
|
45
|
-
Return the response between [Start of Response] and [End of Response] tags.
|
46
|
-
|
47
|
-
[Start of Context]
|
48
|
-
{{ context }}
|
49
|
-
[End of Context]
|
50
|
-
|
51
|
-
[Start of Question]
|
52
|
-
{{ question }}
|
53
|
-
[End of Question]
|
54
|
-
|
55
|
-
Return the response between [Start of Response] and [End of Response] tags.
|
56
|
-
|
57
|
-
|
58
|
-
start_tags: ["[Start of Response]"]
|
59
|
-
end_tags: ["[End of Response]"]
|
@@ -1,56 +0,0 @@
|
|
1
|
-
icl_query: Consider a satellite that is in a circular orbit around the Earth. The speed of the satellite decreases. What will happen to the satellite's orbital radius and period of revolution? Please justify your answer using principles of physics.
|
2
|
-
|
3
|
-
icl_response: |
|
4
|
-
If the speed of the satellite decreases, the satellite will not have enough centripetal force to maintain its circular orbit. The centripetal force required for a circular orbit is given by:
|
5
|
-
|
6
|
-
F = mv^2 / r
|
7
|
-
|
8
|
-
where m is the mass of the satellite, v is its velocity (speed), and r is the orbital radius.
|
9
|
-
|
10
|
-
When the speed decreases, the left side of this equation (the force) remains constant because it's determined by gravity. But the right side (the centripetal force) decreases, which means the orbital radius must also decrease to compensate and keep the forces in balance.
|
11
|
-
|
12
|
-
However, this is not a stable situation. As the orbital radius decreases, the satellite will be moving faster than the reduced speed it currently has, and thus it will slow down further. This process will continue until the satellite reaches an altitude where the orbital velocity matches the reduced speed of the satellite.
|
13
|
-
|
14
|
-
As for the period of revolution, it is given by:
|
15
|
-
|
16
|
-
T = 2πr / v
|
17
|
-
|
18
|
-
where T is the period, r is the orbital radius, and v is the velocity.
|
19
|
-
|
20
|
-
Since both the orbital radius and the velocity are decreasing, we cannot definitively say whether the period will increase or decrease without knowing exactly how much each changes. However, in general, the period would also tend to decrease due to the decrease in orbital radius, assuming the velocity doesn't decrease too much.
|
21
|
-
|
22
|
-
In practice, though, if the speed decreases significantly, the satellite will eventually fall out of orbit and re-enter the Earth's atmosphere.
|
23
|
-
|
24
|
-
icl_analysis: |
|
25
|
-
The query pertains to the domain of physics, specifically orbital mechanics. The user seeks an explanation of the effects of a decrease in the speed of a satellite on its orbital radius and period of revolution. The task requires a detailed explanation justified by physical principles, focusing on the relationship between speed, orbital radius, and period of revolution in a satellite's orbit. Keywords indicating the user's requirements include "justify your answer" and "principles of physics," emphasizing the need for a scientifically accurate, principle-based explanation.
|
26
|
-
|
27
|
-
icl_rubric: |
|
28
|
-
1. Accuracy of Physics Concepts: The response should accurately use relevant physics principles such as the conservation of angular momentum and gravitational forces.
|
29
|
-
2. Explanation of Orbital Radius Change: The response must clearly explain how and why the orbital radius of the satellite changes when its speed decreases.
|
30
|
-
3. Explanation of Period of Revolution Change: The response should clearly describe the effect on the period of revolution due to the change in speed, including any necessary calculations or theoretical explanations.
|
31
|
-
4. Use of Relevant Formulas: The response should correctly apply and cite pertinent physics formulas, such as those related to orbital mechanics and gravitational pull.
|
32
|
-
5. Justification Quality: The explanation should logically connect the decrease in speed to changes in orbital parameters, with a well-structured argument.
|
33
|
-
|
34
|
-
icl_critique: |
|
35
|
-
1. Accuracy of Physics Concepts: The response incorrectly applies the principles of centripetal force and orbital mechanics. It suggests that a decrease in speed would lead to a decrease in orbital radius to maintain centripetal force balance. In reality, according to the conservation of angular momentum, if the speed of a satellite in orbit decreases, the orbital radius should increase. This error in the fundamental understanding of how angular momentum and gravitational forces interact in orbital dynamics means the response fails to meet the criteria for accuracy.
|
36
|
-
2. Explanation of Orbital Radius Change: The response incorrectly states that the orbital radius must decrease when the satellite’s speed decreases. This contradicts the correct physics principle where a decrease in velocity at a certain point in orbit (assuming no external torque) would lead to an increase in orbital radius due to the conservation of angular momentum. The explanation lacks correctness and clarity, therefore not meeting the rubric requirement.
|
37
|
-
3. Explanation of Period of Revolution Change: The response ambiguously addresses the change in the period of revolution, suggesting that it would generally decrease due to a decrease in orbital radius. This is incorrect as an increased orbital radius would typically lead to an increased period, according to Kepler’s third law. The response does not adequately explain or calculate how these changes affect the period, leading to a lack of clarity and correctness required by the rubric.
|
38
|
-
4. Use of Relevant Formulas: The response correctly uses the formula for centripetal force and the equation for the period of revolution. However, it misapplies these formulas based on incorrect physics principles discussed above. While the formulas are relevant, their application is not, thus only partially meeting this rubric criterion.
|
39
|
-
5. Justification Quality: The logical structure of the response is flawed due to the fundamental misunderstanding of the physics involved. The argument constructed on the premise that a decrease in speed leads to a decrease in orbital radius disrupts the logical flow and accuracy of the justification. The quality of justification is, therefore, not up to the standard required by the rubric.
|
40
|
-
Overall, the response requires significant corrections in its application of physics principles to provide accurate and clear explanations concerning changes in a satellite’s orbit due to variations in speed.
|
41
|
-
|
42
|
-
icl_plan: |
|
43
|
-
1. Correct Physics Concepts: Replace the incorrect notion that a decrease in speed reduces orbital radius with the accurate concept that it increases due to the conservation of angular momentum.
|
44
|
-
2. Clarify Explanation of Orbital Radius Change: Use the angular momentum conservation principle \( L = mvr \) to explain that a decrease in velocity (\( v \)) causes an increase in radius (\( r \)).
|
45
|
-
3. Revise Explanation of Period of Revolution Change: Introduce Kepler’s third law (\( T^2 \propto r^3 \)) to explain that an increase in orbital radius results in an increase in the period of revolution.
|
46
|
-
4. Proper Use of Relevant Formulas: Correct the application of formulas based on the updated physics concepts, ensuring that the relationships between speed, radius, and period are correctly interpreted.
|
47
|
-
5. Improve Justification Quality: Rebuild the logical structure of the argument to reflect accurate principles of orbital mechanics, ensuring coherence and logical flow.
|
48
|
-
|
49
|
-
icl_revised_response: |
|
50
|
-
When the speed of a satellite in a circular orbit around the Earth decreases, two key changes occur regarding its orbital radius and period of revolution:
|
51
|
-
|
52
|
-
1. **Orbital Radius**: The orbital radius of the satellite will increase. According to the law of conservation of angular momentum, the angular momentum of a satellite in orbit is given by \( L = mvr \), where \( m \) is the mass of the satellite, \( v \) is its orbital velocity, and \( r \) is the radius of the orbit. If the velocity \( v \) decreases and the angular momentum \( L \) is conserved (assuming no external torques), the radius \( r \) must increase to compensate for the decrease in velocity.
|
53
|
-
|
54
|
-
2. **Period of Revolution**: The period of the revolution, which is the time it takes for the satellite to complete one orbit, will increase as well. Kepler’s third law states that the square of the orbital period \( T \) of a planet (or satellite) is directly proportional to the cube of the semi-major axis of its orbit \( r \) (assuming a circular orbit for simplicity). Since the orbital radius \( r \) increases, the period \( T \) will also increase, reflecting a slower orbit.
|
55
|
-
|
56
|
-
These changes are fundamentally a result of the conservation of angular momentum and the gravitational pull between the Earth and the satellite. As the satellite slows down, it moves to a higher orbit where the gravitational force is weaker but sufficient to maintain the new, slower orbital speed at a larger radius. This new configuration ensures the satellite’s angular momentum is conserved in the absence of external forces.
|
File without changes
|