pytrilogy 0.0.2.21__tar.gz → 0.0.2.23__tar.gz

{pytrilogy-0.0.2.21/pytrilogy.egg-info → pytrilogy-0.0.2.23}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pytrilogy
-Version: 0.0.2.21
+Version: 0.0.2.23
 Summary: Declarative, typed query language that compiles to SQL.
 Home-page: 
 Author: 
@@ -145,7 +145,7 @@ Run the following from the directory the file is in.
 trilogy run hello.trilogy duckdb
 ```
 
-![UI Preview](./hello_world.png)
+![UI Preview](hello-world.png)
 
 ## Backends
 
@@ -158,9 +158,9 @@ The current Trilogy implementation supports these backends:
 
 ## Basic Example - Python
 
-Trilogy can be run directly in python.
+Trilogy can be run directly in python through the core SDK. Trilogy code can be defined and parsed inline or parsed out of files.
 
-A bigquery example, similar to bigquery [the quickstart](https://cloud.google.com/bigquery/docs/quickstarts/query-public-dataset-console)
+A bigquery example, similar to bigquery [the quickstart](https://cloud.google.com/bigquery/docs/quickstarts/query-public-dataset-console).
 
 ```python
 
@@ -224,7 +224,7 @@ and second the dialect to run.
 To pass arguments to a backend, append additional --<option> flags after specifying the dialect.
 
 Example:
-`trilogy run key in int; datasource test_source ( i:in) grain(in) address test; select in;" duckdb --path <path/to/duckdb>`
+`trilogy run "key x int; datasource test_source ( i:x) grain(in) address test; select x;" duckdb --path <path/to/database>`
 
 ### Bigquery Args
 N/A, only supports default auth. In python you can pass in a custom client.
@@ -267,7 +267,7 @@ Clone repository and install requirements.txt and requirements-test.txt.
 Please open an issue first to discuss what you would like to change, and then create a PR against that issue.
 
 ## Similar in space
-Trilogy combines two aspects; a semantic layer and a query language. We've covered examples of both below:
+Trilogy combines two aspects; a semantic layer and a query language. Examples of both are linked below:
 
 Python "semantic layers" are tools for defining data access to a warehouse in a more abstract way.
 
@@ -284,20 +284,26 @@ but all are worth checking out. Please open PRs/comment for anything missed!
 
 #### IMPORT
 
-`import <path> as <alias>;`
+`import [path] as [alias];`
 
 #### CONCEPT
 
-Types: `string | int | float | bool | date | datetime | time | numeric(scale, precision) | timestamp | interval`;
+Types: `string | int | float | bool | date | datetime | time | numeric(scale, precision) | timestamp | interval | list<[type]> | map<[type], [type]> | struct<name:[type], name:[type]>`;
 
 Key:
-`key <name> <type>;`
+`key [name] [type];`
 
 Property:
-`property <key>.<name> <type>;`
+`property [key>].[name] [type];`
+`property x.y int;`
+or 
+`property <[key](,[key])?>.<name> [type];`
+`property <x,y>.z int;`
+
 
 Transformation:
-`auto <name> <- <expression>;`
+`auto [name] <- [expression];`
+`auto x <- y + 1;`
 
 #### DATASOURCE
 ```sql

{pytrilogy-0.0.2.21 → pytrilogy-0.0.2.23}/README.md

@@ -116,7 +116,7 @@ Run the following from the directory the file is in.
 trilogy run hello.trilogy duckdb
 ```
 
-![UI Preview](./hello_world.png)
+![UI Preview](hello-world.png)
 
 ## Backends
 
@@ -129,9 +129,9 @@ The current Trilogy implementation supports these backends:
 
 ## Basic Example - Python
 
-Trilogy can be run directly in python.
+Trilogy can be run directly in python through the core SDK. Trilogy code can be defined and parsed inline or parsed out of files.
 
-A bigquery example, similar to bigquery [the quickstart](https://cloud.google.com/bigquery/docs/quickstarts/query-public-dataset-console)
+A bigquery example, similar to bigquery [the quickstart](https://cloud.google.com/bigquery/docs/quickstarts/query-public-dataset-console).
 
 ```python
 
@@ -195,7 +195,7 @@ and second the dialect to run.
 To pass arguments to a backend, append additional --<option> flags after specifying the dialect.
 
 Example:
-`trilogy run key in int; datasource test_source ( i:in) grain(in) address test; select in;" duckdb --path <path/to/duckdb>`
+`trilogy run "key x int; datasource test_source ( i:x) grain(in) address test; select x;" duckdb --path <path/to/database>`
 
 ### Bigquery Args
 N/A, only supports default auth. In python you can pass in a custom client.
@@ -238,7 +238,7 @@ Clone repository and install requirements.txt and requirements-test.txt.
 Please open an issue first to discuss what you would like to change, and then create a PR against that issue.
 
 ## Similar in space
-Trilogy combines two aspects; a semantic layer and a query language. We've covered examples of both below:
+Trilogy combines two aspects; a semantic layer and a query language. Examples of both are linked below:
 
 Python "semantic layers" are tools for defining data access to a warehouse in a more abstract way.
 
@@ -255,20 +255,26 @@ but all are worth checking out. Please open PRs/comment for anything missed!
 
 #### IMPORT
 
-`import <path> as <alias>;`
+`import [path] as [alias];`
 
 #### CONCEPT
 
-Types: `string | int | float | bool | date | datetime | time | numeric(scale, precision) | timestamp | interval`;
+Types: `string | int | float | bool | date | datetime | time | numeric(scale, precision) | timestamp | interval | list<[type]> | map<[type], [type]> | struct<name:[type], name:[type]>`;
 
 Key:
-`key <name> <type>;`
+`key [name] [type];`
 
 Property:
-`property <key>.<name> <type>;`
+`property [key>].[name] [type];`
+`property x.y int;`
+or 
+`property <[key](,[key])?>.<name> [type];`
+`property <x,y>.z int;`
+
 
 Transformation:
-`auto <name> <- <expression>;`
+`auto [name] <- [expression];`
+`auto x <- y + 1;`
 
 #### DATASOURCE
 ```sql

{pytrilogy-0.0.2.21 → pytrilogy-0.0.2.23/pytrilogy.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pytrilogy
-Version: 0.0.2.21
+Version: 0.0.2.23
 Summary: Declarative, typed query language that compiles to SQL.
 Home-page: 
 Author: 
@@ -145,7 +145,7 @@ Run the following from the directory the file is in.
 trilogy run hello.trilogy duckdb
 ```
 
-![UI Preview](./hello_world.png)
+![UI Preview](hello-world.png)
 
 ## Backends
 
@@ -158,9 +158,9 @@ The current Trilogy implementation supports these backends:
 
 ## Basic Example - Python
 
-Trilogy can be run directly in python.
+Trilogy can be run directly in python through the core SDK. Trilogy code can be defined and parsed inline or parsed out of files.
 
-A bigquery example, similar to bigquery [the quickstart](https://cloud.google.com/bigquery/docs/quickstarts/query-public-dataset-console)
+A bigquery example, similar to bigquery [the quickstart](https://cloud.google.com/bigquery/docs/quickstarts/query-public-dataset-console).
 
 ```python
 
@@ -224,7 +224,7 @@ and second the dialect to run.
 To pass arguments to a backend, append additional --<option> flags after specifying the dialect.
 
 Example:
-`trilogy run key in int; datasource test_source ( i:in) grain(in) address test; select in;" duckdb --path <path/to/duckdb>`
+`trilogy run "key x int; datasource test_source ( i:x) grain(in) address test; select x;" duckdb --path <path/to/database>`
 
 ### Bigquery Args
 N/A, only supports default auth. In python you can pass in a custom client.
@@ -267,7 +267,7 @@ Clone repository and install requirements.txt and requirements-test.txt.
 Please open an issue first to discuss what you would like to change, and then create a PR against that issue.
 
 ## Similar in space
-Trilogy combines two aspects; a semantic layer and a query language. We've covered examples of both below:
+Trilogy combines two aspects; a semantic layer and a query language. Examples of both are linked below:
 
 Python "semantic layers" are tools for defining data access to a warehouse in a more abstract way.
 
@@ -284,20 +284,26 @@ but all are worth checking out. Please open PRs/comment for anything missed!
 
 #### IMPORT
 
-`import <path> as <alias>;`
+`import [path] as [alias];`
 
 #### CONCEPT
 
-Types: `string | int | float | bool | date | datetime | time | numeric(scale, precision) | timestamp | interval`;
+Types: `string | int | float | bool | date | datetime | time | numeric(scale, precision) | timestamp | interval | list<[type]> | map<[type], [type]> | struct<name:[type], name:[type]>`;
 
 Key:
-`key <name> <type>;`
+`key [name] [type];`
 
 Property:
-`property <key>.<name> <type>;`
+`property [key>].[name] [type];`
+`property x.y int;`
+or 
+`property <[key](,[key])?>.<name> [type];`
+`property <x,y>.z int;`
+
 
 Transformation:
-`auto <name> <- <expression>;`
+`auto [name] <- [expression];`
+`auto x <- y + 1;`
 
 #### DATASOURCE
 ```sql

{pytrilogy-0.0.2.21 → pytrilogy-0.0.2.23}/pytrilogy.egg-info/SOURCES.txt

@@ -22,6 +22,7 @@ tests/test_parsing.py
 tests/test_partial_handling.py
 tests/test_query_processing.py
 tests/test_select.py
+tests/test_show.py
 tests/test_statements.py
 tests/test_undefined_concept.py
 tests/test_where_clause.py

pytrilogy-0.0.2.23/tests/test_imports.py

@@ -0,0 +1,42 @@
+from trilogy import Environment
+from pathlib import Path
+
+
+def test_multi_environment():
+    basic = Environment()
+
+    basic.parse(
+        """
+const pi <- 3.14;
+                     
+
+""",
+        namespace="math",
+    )
+
+    basic.parse(
+        """
+                
+            select math.pi;
+                """
+    )
+
+    assert basic.concepts["math.pi"].name == "pi"
+
+
+def test_test_alias_free_import():
+    basic = Environment(working_path=Path(__file__).parent)
+
+    basic.parse(
+        """
+import test_env;
+
+key id2 int;
+
+
+""",
+    )
+
+    assert basic.concepts["id"].name == "id"
+    assert basic.concepts["id2"].name == "id2"
+    assert basic.concepts["id"].namespace == basic.concepts["id2"].namespace

{pytrilogy-0.0.2.21 → pytrilogy-0.0.2.23}/tests/test_models.py

@@ -15,8 +15,11 @@ from trilogy.core.models import (
     Concept,
     AggregateWrapper,
     RowsetItem,
+    TupleWrapper,
+    DataType,
 )
 from trilogy import parse
+from copy import deepcopy
 
 
 def test_cte_merge(test_environment, test_environment_graph):
@@ -312,3 +315,9 @@ select avg_greater_ten;
     lineage = env.concepts["avg_greater_ten"].lineage
     assert isinstance(lineage, AggregateWrapper)
     assert isinstance(lineage.function.concept_arguments[0].lineage, RowsetItem)
+
+
+def test_tuple_clone():
+    x = TupleWrapper([1, 2, 3], type=DataType.INTEGER)
+    y = deepcopy(x)
+    assert y == x

{pytrilogy-0.0.2.21 → pytrilogy-0.0.2.23}/tests/test_parsing.py

@@ -1,12 +1,12 @@
 from trilogy.core.enums import Purpose, ComparisonOperator
 from trilogy.core.models import (
     DataType,
-    Parenthetical,
     ProcessedQuery,
     ShowStatement,
     SelectStatement,
     Environment,
     Comparison,
+    TupleWrapper,
 )
 from trilogy.core.functions import argument_to_purpose, function_args_to_output_purpose
 from trilogy.parsing.parse_engine import (
@@ -27,11 +27,11 @@ def test_in():
     right = query.where_clause.conditional.right
     assert isinstance(
         right,
-        Parenthetical,
+        TupleWrapper,
     ), type(right)
-    assert right.content[0] == 1
+    assert right[0] == 1
     rendered = BaseDialect().render_expr(right)
-    assert rendered.strip() == "( 1,2,3 )".strip()
+    assert rendered.strip() == "(1,2,3)".strip()
 
     _, parsed = parse_text(
         "const order_id <- 3; SELECT order_id  WHERE order_id IN (1,);"
@@ -40,11 +40,11 @@ def test_in():
     right = query.where_clause.conditional.right
     assert isinstance(
         right,
-        Parenthetical,
+        TupleWrapper,
     ), type(right)
-    assert right.content[0] == 1
+    assert right[0] == 1
     rendered = BaseDialect().render_expr(right)
-    assert rendered.strip() == "( 1 )".strip()
+    assert rendered.strip() == "(1)".strip()
 
 
 def test_not_in():
@@ -53,13 +53,10 @@ def test_not_in():
     )
     query: ProcessedQuery = parsed[-1]
     right = query.where_clause.conditional.right
-    assert isinstance(
-        right,
-        Parenthetical,
-    ), type(right)
-    assert right.content[0] == 1
+    assert isinstance(right, TupleWrapper), type(right)
+    assert right[0] == 1
     rendered = BaseDialect().render_expr(right)
-    assert rendered.strip() == "( 1,2,3 )".strip()
+    assert rendered.strip() == "(1,2,3)".strip()
 
 
 def test_is_not_null():
@@ -516,3 +513,24 @@ select filtered_test;
     results = Dialects.DUCK_DB.default_executor().generate_sql(text)[0]
 
     assert "filtered_test" in results, results
+
+
+def test_unnest_parsing():
+    x = """
+key scalar int;    
+property scalar.int_array list<int>;
+
+key split <- unnest(int_array);
+
+datasource avalues (
+    int_array: int_array,
+	scalar: scalar
+    ) 
+grain (scalar) 
+query '''(
+select [1,2,3,4] as int_array, 2 as scalar
+)''';
+"""
+
+    env, parsed = parse_text(x)
+    assert env.concepts["split"].datatype == DataType.INTEGER

pytrilogy-0.0.2.23/tests/test_show.py

@@ -0,0 +1,57 @@
+from trilogy.core.models import ShowStatement
+from trilogy import Dialects
+from trilogy.parser import parse
+
+
+def test_show_bigquery():
+    declarations = """
+    key user_id int metadata(description="the description");
+    property user_id.display_name string metadata(description="The display name ");
+    property user_id.about_me string metadata(description="User provided description");
+    key post_id int;
+
+
+    datasource posts (
+        user_id: user_id,
+        id: post_id
+        )
+        grain (post_id)
+        address bigquery-public-data.stackoverflow.post_history
+    ;
+
+
+    datasource users (
+        id: user_id,
+        display_name: display_name,
+        about_me: about_me,
+        )
+        grain (user_id)
+        address bigquery-public-data.stackoverflow.users
+    ;
+
+
+        """
+    env, parsed = parse(declarations)
+
+    q1 = """
+    metric post_count<- count(post_id);
+    metric distinct_post_count <- count_distinct(post_id);
+    
+    metric user_count <- count(user_id);
+    
+    show select
+        post_count,
+        distinct_post_count,
+        user_count
+    ;"""
+    env, parsed = parse(q1, environment=env)
+    select: ShowStatement = parsed[-1]
+
+    query = (
+        Dialects.DUCK_DB.default_executor(environment=env)
+        .execute_query(select)
+        .fetchall()
+    )
+    assert "FULL JOIN wakeful on 1=1" in query[0]["__preql_internal_query_text"], query[
+        0
+    ]["__preql_internal_query_text"]

{pytrilogy-0.0.2.21 → pytrilogy-0.0.2.23}/tests/test_statements.py

@@ -1,4 +1,7 @@
 from trilogy.parser import parse
+from trilogy import Dialects
+from trilogy.core.models import ProcessedCopyStatement
+from pathlib import Path
 
 # from trilogy.compiler import compile
 
@@ -43,6 +46,24 @@ datasource posts (
     parse(text)
 
 
+def test_io_statement():
+    target = Path(__file__).parent / "test_io_statement.csv"
+    if target.exists():
+        target.unlink()
+    text = f"""const array <- [1,2,3,4];
+
+auto x <- unnest(array);
+
+copy into csv '{target}' from select x -> test;
+"""
+    exec = Dialects.DUCK_DB.default_executor()
+    results = exec.parse_text(text)
+    assert isinstance(results[-1], ProcessedCopyStatement)
+    for z in results:
+        exec.execute_query(z)
+    assert target.exists(), "csv file was not created"
+
+
 def test_datasource_where():
     text = """key user_id int metadata(description="the description");
 property user_id.display_name string metadata(description="The display name ");

{pytrilogy-0.0.2.21 → pytrilogy-0.0.2.23}/trilogy/__init__.py

@@ -4,6 +4,6 @@ from trilogy.executor import Executor
 from trilogy.parser import parse
 from trilogy.constants import CONFIG
 
-__version__ = "0.0.2.21"
+__version__ = "0.0.2.23"
 
 __all__ = ["parse", "Executor", "Dialects", "Environment", "CONFIG"]

{pytrilogy-0.0.2.21 → pytrilogy-0.0.2.23}/trilogy/constants.py

@@ -44,6 +44,7 @@ class Comments:
 class Config:
     strict_mode: bool = True
     human_identifiers: bool = True
+    randomize_cte_names: bool = False
     validate_missing: bool = True
     comments: Comments = field(default_factory=Comments)
     optimizations: Optimizations = field(default_factory=Optimizations)

{pytrilogy-0.0.2.21 → pytrilogy-0.0.2.23}/trilogy/core/enums.py

@@ -12,6 +12,7 @@ class UnnestMode(Enum):
 
 class ConceptSource(Enum):
     MANUAL = "manual"
+    CTE = "cte"
     PERSIST_STATEMENT = "persist_statement"
     AUTO_DERIVED = "auto_derived"
 
@@ -206,6 +207,8 @@ class FunctionClass(Enum):
         FunctionType.CURRENT_DATETIME,
     ]
 
+    ONE_TO_MANY = [FunctionType.UNNEST]
+
 
 class Boolean(Enum):
     TRUE = "true"
@@ -289,3 +292,13 @@ class SelectFiltering(Enum):
     NONE = "none"
     EXPLICIT = "explicit"  # the filtering contains only selected values
     IMPLICIT = "implicit"  # the filtering contains unselected values
+
+
+class IOType(Enum):
+    CSV = "csv"
+
+    @classmethod
+    def _missing_(cls, value):
+        if isinstance(value, str) and value.lower() != value:
+            return IOType(value.lower())
+        return super()._missing_(value)

{pytrilogy-0.0.2.21 → pytrilogy-0.0.2.23}/trilogy/core/env_processor.py

@@ -10,9 +10,11 @@ def add_concept(concept: Concept, g: ReferenceGraph):
     g.add_node(concept)
     # if we have sources, recursively add them
     node_name = concept_to_node(concept)
-    if concept.sources:
-        for source in concept.sources:
+    if concept.concept_arguments:
+        for source in concept.concept_arguments:
             generic = source.with_default_grain()
+            add_concept(generic, g)
+
             g.add_edge(generic, node_name)
     for _, pseudonym in concept.pseudonyms.items():
         pseudonym = pseudonym.with_default_grain()

{pytrilogy-0.0.2.21 → pytrilogy-0.0.2.23}/trilogy/core/ergonomics.py

@@ -1,3 +1,5 @@
+from trilogy.constants import CONFIG
+
 # source: https://github.com/aaronbassett/Pass-phrase
 CTE_NAMES = """quizzical
 highfalutin
@@ -103,8 +105,6 @@ mandrill
 marlin
 monitor
 ocelot
-osprey
-owl
 petrel
 python
 ray
@@ -132,7 +132,6 @@ cuckoo
 darter
 dove
 duck
-eagle
 falcon
 finch
 flamingo
@@ -184,4 +183,12 @@ warbler""".split(
     "\n"
 )
 
-CTE_NAMES = list(set(CTE_NAMES))
+
+def generate_cte_names():
+    if CONFIG.randomize_cte_names:
+        from random import shuffle
+
+        new = [*CTE_NAMES]
+        shuffle(new)
+        return new
+    return CTE_NAMES

{pytrilogy-0.0.2.21 → pytrilogy-0.0.2.23}/trilogy/core/functions.py

@@ -104,6 +104,8 @@ def Unnest(args: list[Concept]) -> Function:
     output = arg_to_datatype(args[0])
     if isinstance(output, (ListType)):
         output = output.value_data_type
+    else:
+        output = DataType.STRING
     return Function(
         operator=FunctionType.UNNEST,
         arguments=args,