langchain 0.4.0.dev0__py3-none-any.whl → 1.0.0a2__py3-none-any.whl

langchain/chains/qa_with_sources/stuff_prompt.py

@@ -1,43 +0,0 @@
-from langchain_core.prompts import PromptTemplate
-
-template = """Given the following extracted parts of a long document and a question, create a final answer with references ("SOURCES").
-If you don't know the answer, just say that you don't know. Don't try to make up an answer.
-ALWAYS return a "SOURCES" part in your answer.
-
-QUESTION: Which state/country's law governs the interpretation of the contract?
-=========
-Content: This Agreement is governed by English law and the parties submit to the exclusive jurisdiction of the English courts in  relation to any dispute (contractual or non-contractual) concerning this Agreement save that either party may apply to any court for an  injunction or other relief to protect its Intellectual Property Rights.
-Source: 28-pl
-Content: No Waiver. Failure or delay in exercising any right or remedy under this Agreement shall not constitute a waiver of such (or any other)  right or remedy.\n\n11.7 Severability. The invalidity, illegality or unenforceability of any term (or part of a term) of this Agreement shall not affect the continuation  in force of the remainder of the term (if any) and this Agreement.\n\n11.8 No Agency. Except as expressly stated otherwise, nothing in this Agreement shall create an agency, partnership or joint venture of any  kind between the parties.\n\n11.9 No Third-Party Beneficiaries.
-Source: 30-pl
-Content: (b) if Google believes, in good faith, that the Distributor has violated or caused Google to violate any Anti-Bribery Laws (as  defined in Clause 8.5) or that such a violation is reasonably likely to occur,
-Source: 4-pl
-=========
-FINAL ANSWER: This Agreement is governed by English law.
-SOURCES: 28-pl
-
-QUESTION: What did the president say about Michael Jackson?
-=========
-Content: Madam Speaker, Madam Vice President, our First Lady and Second Gentleman. Members of Congress and the Cabinet. Justices of the Supreme Court. My fellow Americans.  \n\nLast year COVID-19 kept us apart. This year we are finally together again. \n\nTonight, we meet as Democrats Republicans and Independents. But most importantly as Americans. \n\nWith a duty to one another to the American people to the Constitution. \n\nAnd with an unwavering resolve that freedom will always triumph over tyranny. \n\nSix days ago, Russia's Vladimir Putin sought to shake the foundations of the free world thinking he could make it bend to his menacing ways. But he badly miscalculated. \n\nHe thought he could roll into Ukraine and the world would roll over. Instead he met a wall of strength he never imagined. \n\nHe met the Ukrainian people. \n\nFrom President Zelenskyy to every Ukrainian, their fearlessness, their courage, their determination, inspires the world. \n\nGroups of citizens blocking tanks with their bodies. Everyone from students to retirees teachers turned soldiers defending their homeland.
-Source: 0-pl
-Content: And we won't stop. \n\nWe have lost so much to COVID-19. Time with one another. And worst of all, so much loss of life. \n\nLet's use this moment to reset. Let's stop looking at COVID-19 as a partisan dividing line and see it for what it is: A God-awful disease.  \n\nLet's stop seeing each other as enemies, and start seeing each other for who we really are: Fellow Americans.  \n\nWe can't change how divided we've been. But we can change how we move forward—on COVID-19 and other issues we must face together. \n\nI recently visited the New York City Police Department days after the funerals of Officer Wilbert Mora and his partner, Officer Jason Rivera. \n\nThey were responding to a 9-1-1 call when a man shot and killed them with a stolen gun. \n\nOfficer Mora was 27 years old. \n\nOfficer Rivera was 22. \n\nBoth Dominican Americans who'd grown up on the same streets they later chose to patrol as police officers. \n\nI spoke with their families and told them that we are forever in debt for their sacrifice, and we will carry on their mission to restore the trust and safety every community deserves.
-Source: 24-pl
-Content: And a proud Ukrainian people, who have known 30 years  of independence, have repeatedly shown that they will not tolerate anyone who tries to take their country backwards.  \n\nTo all Americans, I will be honest with you, as I've always promised. A Russian dictator, invading a foreign country, has costs around the world. \n\nAnd I'm taking robust action to make sure the pain of our sanctions  is targeted at Russia's economy. And I will use every tool at our disposal to protect American businesses and consumers. \n\nTonight, I can announce that the United States has worked with 30 other countries to release 60 Million barrels of oil from reserves around the world.  \n\nAmerica will lead that effort, releasing 30 Million barrels from our own Strategic Petroleum Reserve. And we stand ready to do more if necessary, unified with our allies.  \n\nThese steps will help blunt gas prices here at home. And I know the news about what's happening can seem alarming. \n\nBut I want you to know that we are going to be okay.
-Source: 5-pl
-Content: More support for patients and families. \n\nTo get there, I call on Congress to fund ARPA-H, the Advanced Research Projects Agency for Health. \n\nIt's based on DARPA—the Defense Department project that led to the Internet, GPS, and so much more.  \n\nARPA-H will have a singular purpose—to drive breakthroughs in cancer, Alzheimer's, diabetes, and more. \n\nA unity agenda for the nation. \n\nWe can do this. \n\nMy fellow Americans—tonight , we have gathered in a sacred space—the citadel of our democracy. \n\nIn this Capitol, generation after generation, Americans have debated great questions amid great strife, and have done great things. \n\nWe have fought for freedom, expanded liberty, defeated totalitarianism and terror. \n\nAnd built the strongest, freest, and most prosperous nation the world has ever known. \n\nNow is the hour. \n\nOur moment of responsibility. \n\nOur test of resolve and conscience, of history itself. \n\nIt is in this moment that our character is formed. Our purpose is found. Our future is forged. \n\nWell I know this nation.
-Source: 34-pl
-=========
-FINAL ANSWER: The president did not mention Michael Jackson.
-SOURCES:
-
-QUESTION: {question}
-=========
-{summaries}
-=========
-FINAL ANSWER:"""  # noqa: E501
-PROMPT = PromptTemplate(template=template, input_variables=["summaries", "question"])
-
-EXAMPLE_PROMPT = PromptTemplate(
-    template="Content: {page_content}\nSource: {source}",
-    input_variables=["page_content", "source"],
-)

langchain/chains/qa_with_sources/vector_db.py

@@ -1,88 +0,0 @@
-"""Question-answering with sources over a vector database."""
-
-import warnings
-from typing import Any
-
-from langchain_core.callbacks import (
-    AsyncCallbackManagerForChainRun,
-    CallbackManagerForChainRun,
-)
-from langchain_core.documents import Document
-from langchain_core.vectorstores import VectorStore
-from pydantic import Field, model_validator
-from typing_extensions import override
-
-from langchain.chains.combine_documents.stuff import StuffDocumentsChain
-from langchain.chains.qa_with_sources.base import BaseQAWithSourcesChain
-
-
-class VectorDBQAWithSourcesChain(BaseQAWithSourcesChain):
-    """Question-answering with sources over a vector database."""
-
-    vectorstore: VectorStore = Field(exclude=True)
-    """Vector Database to connect to."""
-    k: int = 4
-    """Number of results to return from store"""
-    reduce_k_below_max_tokens: bool = False
-    """Reduce the number of results to return from store based on tokens limit"""
-    max_tokens_limit: int = 3375
-    """Restrict the docs to return from store based on tokens,
-    enforced only for StuffDocumentChain and if reduce_k_below_max_tokens is to true"""
-    search_kwargs: dict[str, Any] = Field(default_factory=dict)
-    """Extra search args."""
-
-    def _reduce_tokens_below_limit(self, docs: list[Document]) -> list[Document]:
-        num_docs = len(docs)
-
-        if self.reduce_k_below_max_tokens and isinstance(
-            self.combine_documents_chain,
-            StuffDocumentsChain,
-        ):
-            tokens = [
-                self.combine_documents_chain.llm_chain._get_num_tokens(doc.page_content)  # noqa: SLF001
-                for doc in docs
-            ]
-            token_count = sum(tokens[:num_docs])
-            while token_count > self.max_tokens_limit:
-                num_docs -= 1
-                token_count -= tokens[num_docs]
-
-        return docs[:num_docs]
-
-    @override
-    def _get_docs(
-        self,
-        inputs: dict[str, Any],
-        *,
-        run_manager: CallbackManagerForChainRun,
-    ) -> list[Document]:
-        question = inputs[self.question_key]
-        docs = self.vectorstore.similarity_search(
-            question,
-            k=self.k,
-            **self.search_kwargs,
-        )
-        return self._reduce_tokens_below_limit(docs)
-
-    async def _aget_docs(
-        self,
-        inputs: dict[str, Any],
-        *,
-        run_manager: AsyncCallbackManagerForChainRun,
-    ) -> list[Document]:
-        msg = "VectorDBQAWithSourcesChain does not support async"
-        raise NotImplementedError(msg)
-
-    @model_validator(mode="before")
-    @classmethod
-    def _raise_deprecation(cls, values: dict) -> Any:
-        warnings.warn(
-            "`VectorDBQAWithSourcesChain` is deprecated - "
-            "please use `from langchain.chains import RetrievalQAWithSourcesChain`",
-            stacklevel=5,
-        )
-        return values
-
-    @property
-    def _chain_type(self) -> str:
-        return "vector_db_qa_with_sources_chain"

langchain/chains/query_constructor/__init__.py

@@ -1,3 +0,0 @@
-from langchain.chains.query_constructor.base import load_query_constructor_runnable
-
-__all__ = ["load_query_constructor_runnable"]

langchain/chains/query_constructor/base.py

@@ -1,378 +0,0 @@
-"""LLM Chain for turning a user text query into a structured query."""
-
-from __future__ import annotations
-
-import json
-from collections.abc import Sequence
-from typing import Any, Callable, Optional, Union, cast
-
-from langchain_core._api import deprecated
-from langchain_core.exceptions import OutputParserException
-from langchain_core.language_models import BaseLanguageModel
-from langchain_core.output_parsers import BaseOutputParser
-from langchain_core.output_parsers.json import parse_and_check_json_markdown
-from langchain_core.prompts import BasePromptTemplate
-from langchain_core.prompts.few_shot import FewShotPromptTemplate
-from langchain_core.runnables import Runnable
-from langchain_core.structured_query import (
-    Comparator,
-    Comparison,
-    FilterDirective,
-    Operation,
-    Operator,
-    StructuredQuery,
-)
-from typing_extensions import override
-
-from langchain.chains.llm import LLMChain
-from langchain.chains.query_constructor.parser import get_parser
-from langchain.chains.query_constructor.prompt import (
-    DEFAULT_EXAMPLES,
-    DEFAULT_PREFIX,
-    DEFAULT_SCHEMA_PROMPT,
-    DEFAULT_SUFFIX,
-    EXAMPLE_PROMPT,
-    EXAMPLES_WITH_LIMIT,
-    PREFIX_WITH_DATA_SOURCE,
-    SCHEMA_WITH_LIMIT_PROMPT,
-    SUFFIX_WITHOUT_DATA_SOURCE,
-    USER_SPECIFIED_EXAMPLE_PROMPT,
-)
-from langchain.chains.query_constructor.schema import AttributeInfo
-
-
-class StructuredQueryOutputParser(BaseOutputParser[StructuredQuery]):
-    """Output parser that parses a structured query."""
-
-    ast_parse: Callable
-    """Callable that parses dict into internal representation of query language."""
-
-    @override
-    def parse(self, text: str) -> StructuredQuery:
-        try:
-            expected_keys = ["query", "filter"]
-            allowed_keys = ["query", "filter", "limit"]
-            parsed = parse_and_check_json_markdown(text, expected_keys)
-            if parsed["query"] is None or len(parsed["query"]) == 0:
-                parsed["query"] = " "
-            if parsed["filter"] == "NO_FILTER" or not parsed["filter"]:
-                parsed["filter"] = None
-            else:
-                parsed["filter"] = self.ast_parse(parsed["filter"])
-            if not parsed.get("limit"):
-                parsed.pop("limit", None)
-            return StructuredQuery(
-                **{k: v for k, v in parsed.items() if k in allowed_keys},
-            )
-        except Exception as e:
-            msg = f"Parsing text\n{text}\n raised following error:\n{e}"
-            raise OutputParserException(msg) from e
-
-    @classmethod
-    def from_components(
-        cls,
-        allowed_comparators: Optional[Sequence[Comparator]] = None,
-        allowed_operators: Optional[Sequence[Operator]] = None,
-        allowed_attributes: Optional[Sequence[str]] = None,
-        fix_invalid: bool = False,  # noqa: FBT001,FBT002
-    ) -> StructuredQueryOutputParser:
-        """
-        Create a structured query output parser from components.
-
-        Args:
-            allowed_comparators: allowed comparators
-            allowed_operators: allowed operators
-
-        Returns:
-            a structured query output parser
-        """
-        ast_parse: Callable
-        if fix_invalid:
-
-            def ast_parse(raw_filter: str) -> Optional[FilterDirective]:
-                filter_directive = cast(
-                    "Optional[FilterDirective]",
-                    get_parser().parse(raw_filter),
-                )
-                return fix_filter_directive(
-                    filter_directive,
-                    allowed_comparators=allowed_comparators,
-                    allowed_operators=allowed_operators,
-                    allowed_attributes=allowed_attributes,
-                )
-
-        else:
-            ast_parse = get_parser(
-                allowed_comparators=allowed_comparators,
-                allowed_operators=allowed_operators,
-                allowed_attributes=allowed_attributes,
-            ).parse
-        return cls(ast_parse=ast_parse)
-
-
-def fix_filter_directive(
-    filter: Optional[FilterDirective],  # noqa: A002
-    *,
-    allowed_comparators: Optional[Sequence[Comparator]] = None,
-    allowed_operators: Optional[Sequence[Operator]] = None,
-    allowed_attributes: Optional[Sequence[str]] = None,
-) -> Optional[FilterDirective]:
-    """Fix invalid filter directive.
-
-    Args:
-        filter: Filter directive to fix.
-        allowed_comparators: allowed comparators. Defaults to all comparators.
-        allowed_operators: allowed operators. Defaults to all operators.
-        allowed_attributes: allowed attributes. Defaults to all attributes.
-
-    Returns:
-        Fixed filter directive.
-    """
-    if (
-        not (allowed_comparators or allowed_operators or allowed_attributes)
-    ) or not filter:
-        return filter
-
-    if isinstance(filter, Comparison):
-        if allowed_comparators and filter.comparator not in allowed_comparators:
-            return None
-        if allowed_attributes and filter.attribute not in allowed_attributes:
-            return None
-        return filter
-    if isinstance(filter, Operation):
-        if allowed_operators and filter.operator not in allowed_operators:
-            return None
-        args = [
-            cast(
-                "FilterDirective",
-                fix_filter_directive(
-                    arg,
-                    allowed_comparators=allowed_comparators,
-                    allowed_operators=allowed_operators,
-                    allowed_attributes=allowed_attributes,
-                ),
-            )
-            for arg in filter.arguments
-            if arg is not None
-        ]
-        if not args:
-            return None
-        if len(args) == 1 and filter.operator in (Operator.AND, Operator.OR):
-            return args[0]
-        return Operation(
-            operator=filter.operator,
-            arguments=args,
-        )
-    return filter
-
-
-def _format_attribute_info(info: Sequence[Union[AttributeInfo, dict]]) -> str:
-    info_dicts = {}
-    for i in info:
-        i_dict = dict(i)
-        info_dicts[i_dict.pop("name")] = i_dict
-    return json.dumps(info_dicts, indent=4).replace("{", "{{").replace("}", "}}")
-
-
-def construct_examples(input_output_pairs: Sequence[tuple[str, dict]]) -> list[dict]:
-    """Construct examples from input-output pairs.
-
-    Args:
-        input_output_pairs: Sequence of input-output pairs.
-
-    Returns:
-        List of examples.
-    """
-    examples = []
-    for i, (_input, output) in enumerate(input_output_pairs):
-        structured_request = (
-            json.dumps(output, indent=4).replace("{", "{{").replace("}", "}}")
-        )
-        example = {
-            "i": i + 1,
-            "user_query": _input,
-            "structured_request": structured_request,
-        }
-        examples.append(example)
-    return examples
-
-
-def get_query_constructor_prompt(
-    document_contents: str,
-    attribute_info: Sequence[Union[AttributeInfo, dict]],
-    *,
-    examples: Optional[Sequence] = None,
-    allowed_comparators: Sequence[Comparator] = tuple(Comparator),
-    allowed_operators: Sequence[Operator] = tuple(Operator),
-    enable_limit: bool = False,
-    schema_prompt: Optional[BasePromptTemplate] = None,
-    **kwargs: Any,
-) -> BasePromptTemplate:
-    """Create query construction prompt.
-
-    Args:
-        document_contents: The contents of the document to be queried.
-        attribute_info: A list of AttributeInfo objects describing
-            the attributes of the document.
-        examples: Optional list of examples to use for the chain.
-        allowed_comparators: Sequence of allowed comparators.
-        allowed_operators: Sequence of allowed operators.
-        enable_limit: Whether to enable the limit operator. Defaults to False.
-        schema_prompt: Prompt for describing query schema. Should have string input
-            variables allowed_comparators and allowed_operators.
-        kwargs: Additional named params to pass to FewShotPromptTemplate init.
-
-    Returns:
-        A prompt template that can be used to construct queries.
-    """
-    default_schema_prompt = (
-        SCHEMA_WITH_LIMIT_PROMPT if enable_limit else DEFAULT_SCHEMA_PROMPT
-    )
-    schema_prompt = schema_prompt or default_schema_prompt
-    attribute_str = _format_attribute_info(attribute_info)
-    schema = schema_prompt.format(
-        allowed_comparators=" | ".join(allowed_comparators),
-        allowed_operators=" | ".join(allowed_operators),
-    )
-    if examples and isinstance(examples[0], tuple):
-        examples = construct_examples(examples)
-        example_prompt = USER_SPECIFIED_EXAMPLE_PROMPT
-        prefix = PREFIX_WITH_DATA_SOURCE.format(
-            schema=schema,
-            content=document_contents,
-            attributes=attribute_str,
-        )
-        suffix = SUFFIX_WITHOUT_DATA_SOURCE.format(i=len(examples) + 1)
-    else:
-        examples = examples or (
-            EXAMPLES_WITH_LIMIT if enable_limit else DEFAULT_EXAMPLES
-        )
-        example_prompt = EXAMPLE_PROMPT
-        prefix = DEFAULT_PREFIX.format(schema=schema)
-        suffix = DEFAULT_SUFFIX.format(
-            i=len(examples) + 1,
-            content=document_contents,
-            attributes=attribute_str,
-        )
-    return FewShotPromptTemplate(
-        examples=list(examples),
-        example_prompt=example_prompt,
-        input_variables=["query"],
-        suffix=suffix,
-        prefix=prefix,
-        **kwargs,
-    )
-
-
-@deprecated(
-    since="0.2.13",
-    alternative="load_query_constructor_runnable",
-    removal="1.0",
-)
-def load_query_constructor_chain(
-    llm: BaseLanguageModel,
-    document_contents: str,
-    attribute_info: Sequence[Union[AttributeInfo, dict]],
-    examples: Optional[list] = None,
-    allowed_comparators: Sequence[Comparator] = tuple(Comparator),
-    allowed_operators: Sequence[Operator] = tuple(Operator),
-    enable_limit: bool = False,  # noqa: FBT001,FBT002
-    schema_prompt: Optional[BasePromptTemplate] = None,
-    **kwargs: Any,
-) -> LLMChain:
-    """Load a query constructor chain.
-
-    Args:
-        llm: BaseLanguageModel to use for the chain.
-        document_contents: The contents of the document to be queried.
-        attribute_info: Sequence of attributes in the document.
-        examples: Optional list of examples to use for the chain.
-        allowed_comparators: Sequence of allowed comparators. Defaults to all
-            Comparators.
-        allowed_operators: Sequence of allowed operators. Defaults to all Operators.
-        enable_limit: Whether to enable the limit operator. Defaults to False.
-        schema_prompt: Prompt for describing query schema. Should have string input
-            variables allowed_comparators and allowed_operators.
-        **kwargs: Arbitrary named params to pass to LLMChain.
-
-    Returns:
-        A LLMChain that can be used to construct queries.
-    """
-    prompt = get_query_constructor_prompt(
-        document_contents,
-        attribute_info,
-        examples=examples,
-        allowed_comparators=allowed_comparators,
-        allowed_operators=allowed_operators,
-        enable_limit=enable_limit,
-        schema_prompt=schema_prompt,
-    )
-    allowed_attributes = [
-        ainfo.name if isinstance(ainfo, AttributeInfo) else ainfo["name"]
-        for ainfo in attribute_info
-    ]
-    output_parser = StructuredQueryOutputParser.from_components(
-        allowed_comparators=allowed_comparators,
-        allowed_operators=allowed_operators,
-        allowed_attributes=allowed_attributes,
-    )
-    # For backwards compatibility.
-    prompt.output_parser = output_parser
-    return LLMChain(llm=llm, prompt=prompt, output_parser=output_parser, **kwargs)
-
-
-def load_query_constructor_runnable(
-    llm: BaseLanguageModel,
-    document_contents: str,
-    attribute_info: Sequence[Union[AttributeInfo, dict]],
-    *,
-    examples: Optional[Sequence] = None,
-    allowed_comparators: Sequence[Comparator] = tuple(Comparator),
-    allowed_operators: Sequence[Operator] = tuple(Operator),
-    enable_limit: bool = False,
-    schema_prompt: Optional[BasePromptTemplate] = None,
-    fix_invalid: bool = False,
-    **kwargs: Any,
-) -> Runnable:
-    """Load a query constructor runnable chain.
-
-    Args:
-        llm: BaseLanguageModel to use for the chain.
-        document_contents: Description of the page contents of the document to be
-            queried.
-        attribute_info: Sequence of attributes in the document.
-        examples: Optional list of examples to use for the chain.
-        allowed_comparators: Sequence of allowed comparators. Defaults to all
-            Comparators.
-        allowed_operators: Sequence of allowed operators. Defaults to all Operators.
-        enable_limit: Whether to enable the limit operator. Defaults to False.
-        schema_prompt: Prompt for describing query schema. Should have string input
-            variables allowed_comparators and allowed_operators.
-        fix_invalid: Whether to fix invalid filter directives by ignoring invalid
-            operators, comparators and attributes.
-        kwargs: Additional named params to pass to FewShotPromptTemplate init.
-
-    Returns:
-        A Runnable that can be used to construct queries.
-    """
-    prompt = get_query_constructor_prompt(
-        document_contents,
-        attribute_info,
-        examples=examples,
-        allowed_comparators=allowed_comparators,
-        allowed_operators=allowed_operators,
-        enable_limit=enable_limit,
-        schema_prompt=schema_prompt,
-        **kwargs,
-    )
-    allowed_attributes = [
-        ainfo.name if isinstance(ainfo, AttributeInfo) else ainfo["name"]
-        for ainfo in attribute_info
-    ]
-    output_parser = StructuredQueryOutputParser.from_components(
-        allowed_comparators=allowed_comparators,
-        allowed_operators=allowed_operators,
-        allowed_attributes=allowed_attributes,
-        fix_invalid=fix_invalid,
-    )
-    return prompt | llm | output_parser

langchain/chains/query_constructor/ir.py

@@ -1,23 +0,0 @@
-"""Internal representation of a structured query language."""
-
-from langchain_core.structured_query import (
-    Comparator,
-    Comparison,
-    Expr,
-    FilterDirective,
-    Operation,
-    Operator,
-    StructuredQuery,
-    Visitor,
-)
-
-__all__ = [
-    "Comparator",
-    "Comparison",
-    "Expr",
-    "FilterDirective",
-    "Operation",
-    "Operator",
-    "StructuredQuery",
-    "Visitor",
-]