pytrilogy 0.0.3.90__tar.gz → 0.0.3.92__tar.gz

{pytrilogy-0.0.3.90/pytrilogy.egg-info → pytrilogy-0.0.3.92}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pytrilogy
-Version: 0.0.3.90
+Version: 0.0.3.92
 Summary: Declarative, typed query language that compiles to SQL.
 Home-page: 
 Author: 
@@ -52,12 +52,12 @@ The Trilogy language is an experiment in better SQL for analytics - a streamline
 - Testability
 - Easy to use for humans and LLMs
 
-Trilogy is epsecially targeted at data consumption, providing a rich metadata layer that makes visualizing Trilogy easy and expressive. 
+Trilogy is especially powerful for data consumption, providing a rich metadata layer that makes creating, interperting, and visualizing queries easy and expressive. 
 
 > [!TIP]
 > You can try Trilogy in a [open-source studio](https://trilogydata.dev/trilogy-studio-core/). More details on the language can be found on the [documentation](https://trilogydata.dev/).
 
-Start in the studio to explore Trilogy. For deeper work and integration, `pytrilogy` can be run locally to parse and execute trilogy model [.preql] files using the `trilogy` CLI tool, or can be run in python by importing the `trilogy` package.
+We recommend starting with the studio to explore Trilogy. For integration, `pytrilogy` can be run locally to parse and execute trilogy model [.preql] files using the `trilogy` CLI tool, or can be run in python by importing the `trilogy` package.
 
 Installation: `pip install pytrilogy`
 
@@ -104,10 +104,13 @@ Save the following code in a file named `hello.preql`
 
 ```python
 # semantic model is abstract from data
+
+type word string; # types can be used to provide expressive metadata tags that propagate through dataflow
+
 key sentence_id int;
-property sentence_id.word_one string; # comments after a definition 
-property sentence_id.word_two string; # are syntactic sugar for adding
-property sentence_id.word_three string; # a description to it
+property sentence_id.word_one string::word; # comments after a definition 
+property sentence_id.word_two string::word; # are syntactic sugar for adding
+property sentence_id.word_three string::word; # a description to it
 
 # comments in other places are just comments
 

{pytrilogy-0.0.3.90 → pytrilogy-0.0.3.92}/README.md

@@ -14,12 +14,12 @@ The Trilogy language is an experiment in better SQL for analytics - a streamline
 - Testability
 - Easy to use for humans and LLMs
 
-Trilogy is epsecially targeted at data consumption, providing a rich metadata layer that makes visualizing Trilogy easy and expressive. 
+Trilogy is especially powerful for data consumption, providing a rich metadata layer that makes creating, interperting, and visualizing queries easy and expressive. 
 
 > [!TIP]
 > You can try Trilogy in a [open-source studio](https://trilogydata.dev/trilogy-studio-core/). More details on the language can be found on the [documentation](https://trilogydata.dev/).
 
-Start in the studio to explore Trilogy. For deeper work and integration, `pytrilogy` can be run locally to parse and execute trilogy model [.preql] files using the `trilogy` CLI tool, or can be run in python by importing the `trilogy` package.
+We recommend starting with the studio to explore Trilogy. For integration, `pytrilogy` can be run locally to parse and execute trilogy model [.preql] files using the `trilogy` CLI tool, or can be run in python by importing the `trilogy` package.
 
 Installation: `pip install pytrilogy`
 
@@ -66,10 +66,13 @@ Save the following code in a file named `hello.preql`
 
 ```python
 # semantic model is abstract from data
+
+type word string; # types can be used to provide expressive metadata tags that propagate through dataflow
+
 key sentence_id int;
-property sentence_id.word_one string; # comments after a definition 
-property sentence_id.word_two string; # are syntactic sugar for adding
-property sentence_id.word_three string; # a description to it
+property sentence_id.word_one string::word; # comments after a definition 
+property sentence_id.word_two string::word; # are syntactic sugar for adding
+property sentence_id.word_three string::word; # a description to it
 
 # comments in other places are just comments
 

{pytrilogy-0.0.3.90 → pytrilogy-0.0.3.92/pytrilogy.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pytrilogy
-Version: 0.0.3.90
+Version: 0.0.3.92
 Summary: Declarative, typed query language that compiles to SQL.
 Home-page: 
 Author: 
@@ -52,12 +52,12 @@ The Trilogy language is an experiment in better SQL for analytics - a streamline
 - Testability
 - Easy to use for humans and LLMs
 
-Trilogy is epsecially targeted at data consumption, providing a rich metadata layer that makes visualizing Trilogy easy and expressive. 
+Trilogy is especially powerful for data consumption, providing a rich metadata layer that makes creating, interperting, and visualizing queries easy and expressive. 
 
 > [!TIP]
 > You can try Trilogy in a [open-source studio](https://trilogydata.dev/trilogy-studio-core/). More details on the language can be found on the [documentation](https://trilogydata.dev/).
 
-Start in the studio to explore Trilogy. For deeper work and integration, `pytrilogy` can be run locally to parse and execute trilogy model [.preql] files using the `trilogy` CLI tool, or can be run in python by importing the `trilogy` package.
+We recommend starting with the studio to explore Trilogy. For integration, `pytrilogy` can be run locally to parse and execute trilogy model [.preql] files using the `trilogy` CLI tool, or can be run in python by importing the `trilogy` package.
 
 Installation: `pip install pytrilogy`
 
@@ -104,10 +104,13 @@ Save the following code in a file named `hello.preql`
 
 ```python
 # semantic model is abstract from data
+
+type word string; # types can be used to provide expressive metadata tags that propagate through dataflow
+
 key sentence_id int;
-property sentence_id.word_one string; # comments after a definition 
-property sentence_id.word_two string; # are syntactic sugar for adding
-property sentence_id.word_three string; # a description to it
+property sentence_id.word_one string::word; # comments after a definition 
+property sentence_id.word_two string::word; # are syntactic sugar for adding
+property sentence_id.word_three string::word; # a description to it
 
 # comments in other places are just comments
 

{pytrilogy-0.0.3.90 → pytrilogy-0.0.3.92}/tests/test_user_functions.py

@@ -2,6 +2,7 @@ from pathlib import Path
 
 from trilogy import Dialects, Environment
 from trilogy.core.enums import Derivation, Purpose
+from trilogy.hooks import DebuggingHook
 
 
 def test_user_function_def():
@@ -20,6 +21,24 @@ select @percent_ratio(10, 100) as ratio;
     assert results.fetchall()[0].ratio == 10.0
 
 
+def test_user_function_def_with_default():
+    x = Dialects.DUCK_DB.default_executor()
+    DebuggingHook()
+    results = x.execute_query(
+        """
+def percent_ratio(a, b, digits=3) -> round(a / b * 100, digits);
+
+select @percent_ratio(21, 1000) as ratio,
+         @percent_ratio(21, 1000, 0) as ratio_two;
+                  
+                  """
+    )
+
+    results = results.fetchall()
+    assert results[0].ratio == 2.1
+    assert results[0].ratio_two == 2
+
+
 def test_user_function_aggregate():
     x = Dialects.DUCK_DB.default_executor()
 

{pytrilogy-0.0.3.90 → pytrilogy-0.0.3.92}/trilogy/__init__.py

@@ -4,6 +4,6 @@ from trilogy.dialect.enums import Dialects
 from trilogy.executor import Executor
 from trilogy.parser import parse
 
-__version__ = "0.0.3.90"
+__version__ = "0.0.3.92"
 
 __all__ = ["parse", "Executor", "Dialects", "Environment", "CONFIG"]

{pytrilogy-0.0.3.90 → pytrilogy-0.0.3.92}/trilogy/core/env_processor.py

@@ -8,26 +8,34 @@ from trilogy.core.models.build_environment import BuildEnvironment
 
 
 def add_concept(
-    concept: BuildConcept, g: ReferenceGraph, concept_mapping: dict[str, BuildConcept]
+    concept: BuildConcept,
+    g: ReferenceGraph,
+    concept_mapping: dict[str, BuildConcept],
+    default_concept_graph: dict[str, BuildConcept],
+    seen: set[str],
 ):
-    g.add_node(concept)
+
     # if we have sources, recursively add them
     node_name = concept_to_node(concept)
+    if node_name in seen:
+        return
+    seen.add(node_name)
+    g.add_node(concept)
     if concept.concept_arguments:
         for source in concept.concept_arguments:
             if not isinstance(source, BuildConcept):
                 raise ValueError(
                     f"Invalid non-build concept {source} passed into graph generation from {concept}"
                 )
-            generic = source.with_default_grain()
-            add_concept(generic, g, concept_mapping)
+            generic = get_default_grain_concept(source, default_concept_graph)
+            add_concept(generic, g, concept_mapping, default_concept_graph, seen)
 
             g.add_edge(generic, node_name)
     for ps_address in concept.pseudonyms:
         if ps_address not in concept_mapping:
             raise SyntaxError(f"Concept {concept} has invalid pseudonym {ps_address}")
         pseudonym = concept_mapping[ps_address]
-        pseudonym = pseudonym.with_default_grain()
+        pseudonym = get_default_grain_concept(pseudonym, default_concept_graph)
         pseudonym_node = concept_to_node(pseudonym)
         if (pseudonym_node, node_name) in g.edges and (
             node_name,
@@ -38,16 +46,29 @@ def add_concept(
             continue
         g.add_edge(pseudonym_node, node_name, pseudonym=True)
         g.add_edge(node_name, pseudonym_node, pseudonym=True)
-        add_concept(pseudonym, g, concept_mapping)
+        add_concept(pseudonym, g, concept_mapping, default_concept_graph, seen)
+
+
+def get_default_grain_concept(
+    concept: BuildConcept, default_concept_graph: dict[str, BuildConcept]
+) -> BuildConcept:
+    """Get the default grain concept from the graph."""
+    if concept.address in default_concept_graph:
+        return default_concept_graph[concept.address]
+    default = concept.with_default_grain()
+    default_concept_graph[concept.address] = default
+    return default
 
 
 def generate_adhoc_graph(
     concepts: list[BuildConcept],
     datasources: list[BuildDatasource],
+    default_concept_graph: dict[str, BuildConcept],
     restrict_to_listed: bool = False,
 ) -> ReferenceGraph:
     g = ReferenceGraph()
     concept_mapping = {x.address: x for x in concepts}
+    seen: set[str] = set()
     for concept in concepts:
         if not isinstance(concept, BuildConcept):
             raise ValueError(f"Invalid non-build concept {concept}")
@@ -55,7 +76,7 @@ def generate_adhoc_graph(
     # add all parsed concepts
     for concept in concepts:
 
-        add_concept(concept, g, concept_mapping)
+        add_concept(concept, g, concept_mapping, default_concept_graph, seen)
 
     for dataset in datasources:
         node = datasource_to_node(dataset)
@@ -69,7 +90,7 @@ def generate_adhoc_graph(
             # if there is a key on a table at a different grain
             # add an FK edge to the canonical source, if it exists
             # for example, order ID on order product table
-            default = concept.with_default_grain()
+            default = get_default_grain_concept(concept, default_concept_graph)
             if concept != default:
                 g.add_edge(concept, default)
                 g.add_edge(default, concept)
@@ -79,9 +100,10 @@ def generate_adhoc_graph(
 def generate_graph(
     environment: BuildEnvironment,
 ) -> ReferenceGraph:
-
+    default_concept_graph: dict[str, BuildConcept] = {}
     return generate_adhoc_graph(
         list(environment.concepts.values())
         + list(environment.alias_origin_lookup.values()),
         list(environment.datasources.values()),
+        default_concept_graph=default_concept_graph,
     )

{pytrilogy-0.0.3.90 → pytrilogy-0.0.3.92}/trilogy/core/graph_models.py

@@ -54,7 +54,7 @@ def prune_sources_for_conditions(
 def concept_to_node(input: BuildConcept) -> str:
     # if input.purpose == Purpose.METRIC:
     #     return f"c~{input.namespace}.{input.name}@{input.grain}"
-    return f"c~{input.address}@{input.grain.without_condition()}"
+    return f"c~{input.address}@{input.grain.str_no_condition}"
 
 
 def datasource_to_node(input: BuildDatasource) -> str:
@@ -72,11 +72,15 @@ class ReferenceGraph(nx.DiGraph):
     def add_node(self, node_for_adding, **attr):
         if isinstance(node_for_adding, BuildConcept):
             node_name = concept_to_node(node_for_adding)
+            # if node_name in self.nodes:
+            #     return
             attr["type"] = "concept"
             attr["concept"] = node_for_adding
             attr["grain"] = node_for_adding.grain
         elif isinstance(node_for_adding, BuildDatasource):
             node_name = datasource_to_node(node_for_adding)
+            # if node_name in self.nodes:
+            #     return
             attr["type"] = "datasource"
             attr["ds"] = node_for_adding
             attr["grain"] = node_for_adding.grain
@@ -91,7 +95,10 @@ class ReferenceGraph(nx.DiGraph):
             if u_of_edge not in self.nodes:
                 self.add_node(orig)
         elif isinstance(u_of_edge, BuildDatasource):
+            orig = u_of_edge
             u_of_edge = datasource_to_node(u_of_edge)
+            if u_of_edge not in self.nodes:
+                self.add_node(orig)
 
         if isinstance(v_of_edge, BuildConcept):
             orig = v_of_edge
@@ -99,5 +106,8 @@ class ReferenceGraph(nx.DiGraph):
             if v_of_edge not in self.nodes:
                 self.add_node(orig)
         elif isinstance(v_of_edge, BuildDatasource):
+            orig = v_of_edge
             v_of_edge = datasource_to_node(v_of_edge)
+            if v_of_edge not in self.nodes:
+                self.add_node(orig)
         super().add_edge(u_of_edge, v_of_edge, **attr)

{pytrilogy-0.0.3.90 → pytrilogy-0.0.3.92}/trilogy/core/models/author.py

@@ -460,6 +460,8 @@ class HavingClause(WhereClause):
 class Grain(Namespaced, BaseModel):
     components: set[str] = Field(default_factory=set)
     where_clause: Optional["WhereClause"] = None
+    _str: str | None = None
+    _abstract: bool = False
 
     def without_condition(self):
         return Grain(components=self.components)
@@ -484,12 +486,9 @@ class Grain(Namespaced, BaseModel):
         from trilogy.parsing.common import concepts_to_grain_concepts
 
         x = Grain.model_construct(
-            components={
-                c.address
-                for c in concepts_to_grain_concepts(
-                    concepts, environment=environment, local_concepts=local_concepts
-                )
-            },
+            components=concepts_to_grain_concepts(
+                concepts, environment=environment, local_concepts=local_concepts
+            ),
             where_clause=where_clause,
         )
 
@@ -550,17 +549,22 @@ class Grain(Namespaced, BaseModel):
             where_clause=self.where_clause,
         )
 
-    @property
-    def abstract(self):
+    def _gen_abstract(self) -> bool:
         return not self.components or all(
             [c.endswith(ALL_ROWS_CONCEPT) for c in self.components]
         )
 
+    @property
+    def abstract(self):
+        if not self._abstract:
+            self._abstract = self._gen_abstract()
+        return self._abstract
+
     def __eq__(self, other: object):
         if isinstance(other, list):
-            if not all([isinstance(c, Concept) for c in other]):
-                return False
-            return self.components == set([c.address for c in other])
+            if all([isinstance(c, Concept) for c in other]):
+                return self.components == set([c.address for c in other])
+            return False
         if not isinstance(other, Grain):
             return False
         if self.components == other.components:
@@ -581,15 +585,20 @@ class Grain(Namespaced, BaseModel):
         intersection = self.components.intersection(other.components)
         return Grain(components=intersection)
 
-    def __str__(self):
+    def _gen_str(self) -> str:
         if self.abstract:
             base = "Grain<Abstract>"
         else:
-            base = "Grain<" + ",".join([c for c in sorted(list(self.components))]) + ">"
+            base = "Grain<" + ",".join(sorted(self.components)) + ">"
         if self.where_clause:
             base += f"|{str(self.where_clause)}"
         return base
 
+    def __str__(self):
+        if not self._str:
+            self._str = self._gen_str()
+        return self._str
+
     def __radd__(self, other) -> "Grain":
         if other == 0:
             return self
@@ -1666,7 +1675,6 @@ class Function(DataTyped, ConceptArgs, Mergeable, Namespaced, BaseModel):
     def parse_output_datatype(cls, v, info: ValidationInfo):
         values = info.data
         if values.get("operator") == FunctionType.ATTR_ACCESS:
-            print(v)
             if isinstance(v, StructType):
                 raise SyntaxError
         return v
@@ -2335,11 +2343,16 @@ class AlignItem(Namespaced, BaseModel):
 
 class CustomFunctionFactory:
     def __init__(
-        self, function: Expr, namespace: str, function_arguments: list[ArgBinding]
+        self,
+        function: Expr,
+        namespace: str,
+        function_arguments: list[ArgBinding],
+        name: str,
     ):
         self.namespace = namespace
         self.function = function
         self.function_arguments = function_arguments
+        self.name = name
 
     def with_namespace(self, namespace: str):
         self.namespace = namespace
@@ -2364,7 +2377,27 @@ class CustomFunctionFactory:
             for binding in self.function_arguments[len(creation_arg_list) :]:
                 if binding.default is None:
                     raise ValueError(f"Missing argument {binding.name}")
+
                 creation_arg_list.append(binding.default)
+        for arg_idx, arg in enumerate(self.function_arguments):
+            if not arg.datatype or arg.datatype == DataType.UNKNOWN:
+                continue
+            if arg_idx > len(creation_arg_list):
+                continue
+            comparison = arg_to_datatype(creation_arg_list[arg_idx])
+            if comparison != arg.datatype:
+                raise TypeError(
+                    f"Invalid type passed into custom function @{self.name} in position {arg_idx+1} for argument {arg.name}, expected {arg.datatype}, got {comparison}"
+                )
+            if isinstance(arg.datatype, TraitDataType):
+                if not (
+                    isinstance(comparison, TraitDataType)
+                    and all(x in comparison.traits for x in arg.datatype.traits)
+                ):
+                    raise TypeError(
+                        f"Invalid argument type passed into custom function @{self.name} in position {arg_idx+1} for argument {arg.name}, expected traits {arg.datatype.traits}, got {comparison}"
+                    )
+
         if isinstance(nout, Mergeable):
             for idx, x in enumerate(creation_arg_list):
                 if self.namespace == DEFAULT_NAMESPACE: