judgeval 0.0.15__tar.gz → 0.0.16__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {judgeval-0.0.15 → judgeval-0.0.16}/PKG-INFO +1 -1
- {judgeval-0.0.15 → judgeval-0.0.16}/pyproject.toml +1 -1
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/common/tracer.py +2 -4
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/data/datasets/eval_dataset_client.py +5 -10
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/judgment_client.py +4 -8
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/run_evaluation.py +3 -6
- {judgeval-0.0.15 → judgeval-0.0.16}/.github/workflows/ci.yaml +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/.gitignore +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/LICENSE.md +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/Pipfile +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/Pipfile.lock +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/README.md +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/README.md +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/api_reference/judgment_client.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/api_reference/trace.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/development.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/essentials/code.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/essentials/images.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/essentials/markdown.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/essentials/navigation.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/essentials/reusable-snippets.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/essentials/settings.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/data_datasets.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/data_examples.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/introduction.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/judges.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/answer_correctness.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/answer_relevancy.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/classifier_scorer.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/contextual_precision.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/contextual_recall.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/contextual_relevancy.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/custom_scorers.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/faithfulness.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/hallucination.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/introduction.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/json_correctness.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/summarization.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/scorers/tool_correctness.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/evaluation/unit_testing.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/favicon.svg +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/getting_started.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/images/basic_trace_example.png +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/images/checks-passed.png +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/images/create_aggressive_scorer.png +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/images/create_scorer.png +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/images/evaluation_diagram.png +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/images/hero-dark.svg +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/images/hero-light.svg +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/images/trace_screenshot.png +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/introduction.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/judgment/introduction.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/logo/dark.svg +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/logo/light.svg +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/mint.json +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/monitoring/introduction.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/monitoring/production_insights.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/monitoring/tracing.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/notebooks/create_dataset.ipynb +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/notebooks/create_scorer.ipynb +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/notebooks/demo.ipynb +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/notebooks/prompt_scorer.ipynb +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/notebooks/quickstart.ipynb +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/quickstart.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/docs/snippets/snippet-intro.mdx +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/pytest.ini +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/anime_chatbot_agent/animeChatBot.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/ci_testing/ci_testing.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/ci_testing/travel_response.txt +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/custom_scorers/competitor_mentions.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/custom_scorers/text2sql.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/langchain_basic_rag/basic_agentic_rag.ipynb +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/langchain_basic_rag/tesla_q3.pdf +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/langchain_sales/example_product_price_id_mapping.json +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/langchain_sales/sales_agent_with_context.ipynb +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/langchain_sales/sample_product_catalog.txt +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/new_bot/basic_bot.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/openai_travel_agent/agent.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/openai_travel_agent/populate_db.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/openai_travel_agent/tools.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/rules_alerts/rules_bot.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/rules_alerts/rules_demo.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/rules_alerts/utils_helper.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/customer_use/cstone/basic_test.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/customer_use/cstone/cstone_data.csv +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/customer_use/cstone/data.csv +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/customer_use/cstone/faithfulness_testing.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/customer_use/cstone/galen_data.csv +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/customer_use/cstone/playground.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/demo/customer_use/cstone/results.csv +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/clients.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/common/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/common/exceptions.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/common/logger.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/common/utils.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/constants.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/data/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/data/api_example.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/data/datasets/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/data/datasets/dataset.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/data/datasets/ground_truth.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/data/datasets/utils.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/data/example.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/data/result.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/data/scorer_data.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/evaluation_run.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/judges/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/judges/base_judge.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/judges/litellm_judge.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/judges/mixture_of_judges.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/judges/together_judge.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/judges/utils.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/rules.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/api_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/base_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/exceptions.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/api_scorers/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/api_scorers/answer_correctness.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/api_scorers/answer_relevancy.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/api_scorers/contextual_precision.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/api_scorers/contextual_recall.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/api_scorers/contextual_relevancy.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/api_scorers/faithfulness.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/api_scorers/hallucination.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/api_scorers/json_correctness.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/api_scorers/summarization.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/api_scorers/tool_correctness.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/classifiers/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/classifiers/text2sql/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/classifiers/text2sql/text2sql_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/answer_correctness/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/answer_correctness/answer_correctness_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/answer_correctness/prompts.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/answer_relevancy/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/answer_relevancy/answer_relevancy_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/answer_relevancy/prompts.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/contextual_precision/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/contextual_precision/contextual_precision_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/contextual_precision/prompts.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/contextual_recall/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/contextual_recall/contextual_recall_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/contextual_recall/prompts.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/contextual_relevancy/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/contextual_relevancy/contextual_relevancy_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/contextual_relevancy/prompts.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/faithfulness/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/faithfulness/faithfulness_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/faithfulness/prompts.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/hallucination/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/hallucination/hallucination_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/hallucination/prompts.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/json_correctness/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/json_correctness/json_correctness_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/summarization/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/summarization/prompts.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/summarization/summarization_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/tool_correctness/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/local_implementations/tool_correctness/tool_correctness_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/prompt_scorer.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/score.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/utils.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/tracer/__init__.py +0 -0
- {judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/utils/alerts.py +0 -0
@@ -207,8 +207,7 @@ class TraceManagerClient:
|
|
207
207
|
"Content-Type": "application/json",
|
208
208
|
"Authorization": f"Bearer {self.judgment_api_key}",
|
209
209
|
"X-Organization-Id": self.organization_id
|
210
|
-
}
|
211
|
-
verify=False
|
210
|
+
}
|
212
211
|
)
|
213
212
|
|
214
213
|
if response.status_code != HTTPStatus.OK:
|
@@ -232,8 +231,7 @@ class TraceManagerClient:
|
|
232
231
|
"Content-Type": "application/json",
|
233
232
|
"Authorization": f"Bearer {self.judgment_api_key}",
|
234
233
|
"X-Organization-Id": self.organization_id
|
235
|
-
}
|
236
|
-
verify=False
|
234
|
+
}
|
237
235
|
)
|
238
236
|
|
239
237
|
if response.status_code == HTTPStatus.BAD_REQUEST:
|
@@ -68,8 +68,7 @@ class EvalDatasetClient:
|
|
68
68
|
"Content-Type": "application/json",
|
69
69
|
"Authorization": f"Bearer {self.judgment_api_key}",
|
70
70
|
"X-Organization-Id": self.organization_id
|
71
|
-
}
|
72
|
-
verify=False
|
71
|
+
}
|
73
72
|
)
|
74
73
|
if response.status_code == 500:
|
75
74
|
error(f"Server error during push: {content.get('message')}")
|
@@ -133,8 +132,7 @@ class EvalDatasetClient:
|
|
133
132
|
"Content-Type": "application/json",
|
134
133
|
"Authorization": f"Bearer {self.judgment_api_key}",
|
135
134
|
"X-Organization-Id": self.organization_id
|
136
|
-
}
|
137
|
-
verify=False
|
135
|
+
}
|
138
136
|
)
|
139
137
|
response.raise_for_status()
|
140
138
|
except requests.exceptions.RequestException as e:
|
@@ -192,8 +190,7 @@ class EvalDatasetClient:
|
|
192
190
|
"Content-Type": "application/json",
|
193
191
|
"Authorization": f"Bearer {self.judgment_api_key}",
|
194
192
|
"X-Organization-Id": self.organization_id
|
195
|
-
}
|
196
|
-
verify=False
|
193
|
+
}
|
197
194
|
)
|
198
195
|
response.raise_for_status()
|
199
196
|
except requests.exceptions.RequestException as e:
|
@@ -246,8 +243,7 @@ class EvalDatasetClient:
|
|
246
243
|
"Content-Type": "application/json",
|
247
244
|
"Authorization": f"Bearer {self.judgment_api_key}",
|
248
245
|
"X-Organization-Id": self.organization_id
|
249
|
-
}
|
250
|
-
verify=False
|
246
|
+
}
|
251
247
|
)
|
252
248
|
response.raise_for_status()
|
253
249
|
except requests.exceptions.RequestException as e:
|
@@ -278,8 +274,7 @@ class EvalDatasetClient:
|
|
278
274
|
"Authorization": f"Bearer {self.judgment_api_key}",
|
279
275
|
"X-Organization-Id": self.organization_id
|
280
276
|
},
|
281
|
-
stream=True
|
282
|
-
verify=False
|
277
|
+
stream=True
|
283
278
|
)
|
284
279
|
response.raise_for_status()
|
285
280
|
except requests.exceptions.HTTPError as err:
|
@@ -306,8 +306,7 @@ class JudgmentClient:
|
|
306
306
|
"Authorization": f"Bearer {self.judgment_api_key}",
|
307
307
|
"X-Organization-Id": self.organization_id
|
308
308
|
},
|
309
|
-
json=eval_run_request_body.model_dump()
|
310
|
-
verify=False)
|
309
|
+
json=eval_run_request_body.model_dump())
|
311
310
|
if eval_run.status_code != requests.codes.ok:
|
312
311
|
raise ValueError(f"Error fetching eval results: {eval_run.json()}")
|
313
312
|
|
@@ -379,8 +378,7 @@ class JudgmentClient:
|
|
379
378
|
"Content-Type": "application/json",
|
380
379
|
"Authorization": f"Bearer {self.judgment_api_key}",
|
381
380
|
},
|
382
|
-
json={}
|
383
|
-
verify=False
|
381
|
+
json={} # Empty body now
|
384
382
|
)
|
385
383
|
if response.status_code == 200:
|
386
384
|
return True, response.json()
|
@@ -411,8 +409,7 @@ class JudgmentClient:
|
|
411
409
|
"Content-Type": "application/json",
|
412
410
|
"Authorization": f"Bearer {self.judgment_api_key}",
|
413
411
|
"X-Organization-Id": self.organization_id
|
414
|
-
}
|
415
|
-
verify=False
|
412
|
+
}
|
416
413
|
)
|
417
414
|
|
418
415
|
if response.status_code == 500:
|
@@ -455,8 +452,7 @@ class JudgmentClient:
|
|
455
452
|
"Content-Type": "application/json",
|
456
453
|
"Authorization": f"Bearer {self.judgment_api_key}",
|
457
454
|
"X-Organization-Id": self.organization_id
|
458
|
-
}
|
459
|
-
verify=False
|
455
|
+
}
|
460
456
|
)
|
461
457
|
|
462
458
|
if response.status_code == 500:
|
@@ -55,8 +55,7 @@ def execute_api_eval(evaluation_run: EvaluationRun) -> List[Dict]:
|
|
55
55
|
"Authorization": f"Bearer {evaluation_run.judgment_api_key}",
|
56
56
|
"X-Organization-Id": evaluation_run.organization_id
|
57
57
|
},
|
58
|
-
json=payload
|
59
|
-
verify=False)
|
58
|
+
json=payload)
|
60
59
|
response_data = response.json()
|
61
60
|
except Exception as e:
|
62
61
|
error(f"Error: {e}")
|
@@ -169,8 +168,7 @@ def check_eval_run_name_exists(eval_name: str, project_name: str, judgment_api_k
|
|
169
168
|
"eval_name": eval_name,
|
170
169
|
"project_name": project_name,
|
171
170
|
"judgment_api_key": judgment_api_key,
|
172
|
-
}
|
173
|
-
verify=False
|
171
|
+
}
|
174
172
|
)
|
175
173
|
|
176
174
|
if response.status_code == 409:
|
@@ -212,8 +210,7 @@ def log_evaluation_results(merged_results: List[ScoringResult], evaluation_run:
|
|
212
210
|
"results": [result.to_dict() for result in merged_results],
|
213
211
|
"project_name": evaluation_run.project_name,
|
214
212
|
"eval_name": evaluation_run.eval_name,
|
215
|
-
}
|
216
|
-
verify=False
|
213
|
+
}
|
217
214
|
)
|
218
215
|
|
219
216
|
if not res.ok:
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
{judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/custom_scorers/competitor_mentions.py
RENAMED
File without changes
|
File without changes
|
{judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/langchain_basic_rag/basic_agentic_rag.ipynb
RENAMED
File without changes
|
File without changes
|
File without changes
|
File without changes
|
{judgeval-0.0.15 → judgeval-0.0.16}/src/demo/cookbooks/langchain_sales/sample_product_catalog.txt
RENAMED
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
{judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/api_scorers/__init__.py
RENAMED
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
{judgeval-0.0.15 → judgeval-0.0.16}/src/judgeval/scorers/judgeval_scorers/classifiers/__init__.py
RENAMED
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|