klongpy 0.6.1__py3-none-any.whl → 0.6.2__py3-none-any.whl

klongpy/db/sys_fn_db.py

@@ -87,6 +87,9 @@ class Table(dict):
     def insert(self, y):
         self.buffer.append(y)
 
+    def insertb(self, y):
+        self.buffer.extend(y)
+
     def commit(self):
         if not self.buffer:
             return
@@ -254,19 +257,25 @@ def eval_sys_fn_insert_table(x, y):
         Examples:
 
             t::.table(d)
-            .insert(t, [1;2;3])
+            .insert(t, [1 2 3])
+
+            for batch inserts supply an array of arrays:
+
+            .insert(t, [[1 2 3] [4 5 6]])
 
     """
     if not isinstance(x,Table):
         raise KlongDbException(x, "Inserts must be applied to a table")
-    if len(x.columns) > 1:
-        if not np.isarray(y):
-            raise KlongDbException(x, "Values to insert must be a list")
-    elif not np.isarray(y):
-            y = np.array([y])
-    if len(y) != len(x.columns):
+    if not np.isarray(y):
+        raise KlongDbException(x, "Values to insert must be a list")
+    batch = len(y.shape) > 1
+    y_cols = len(y[0]) if batch else len(y)
+    if y_cols != len(x.columns):
         raise KlongDbException(x, f"Expected {len(x.columns)} values, received {len(y)}")
-    x.insert(y)
+    if batch:
+        x.insertb(y)
+    else:
+        x.insert(y)
     return x
 
 

klongpy/dyads.py

@@ -478,14 +478,8 @@ def eval_dyad_integer_divide(a, b):
 
 
 def _arr_to_list(a):
-    if np.isarray(a):
-        if a.ndim == 1:
-            return a
-        elif a.shape[0] == 1:
-            return [a.flatten()] # TODO: test squeeze
-        elif a.shape[0] > 1:
-            return a
-    return [a] if not is_list(a) else a
+    """ Convert a to a list if it is not already one """
+    return a if is_list(a) else [a]# if not is_list(a) else a
 
 
 def eval_dyad_join(a, b):

klongpy-0.6.2.dist-info/METADATA

@@ -0,0 +1,414 @@
+Metadata-Version: 2.1
+Name: klongpy
+Version: 0.6.2
+Summary: High-Performance Klong array language with rich Python integration.
+Author: Brian Guarraci
+License: MIT
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Requires-Python: <3.12,>=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy ~=1.26.4
+Provides-Extra: cuda102
+Requires-Dist: cupy-cuda102 ; extra == 'cuda102'
+Provides-Extra: cuda110
+Requires-Dist: cupy-cuda110 ; extra == 'cuda110'
+Provides-Extra: cuda111
+Requires-Dist: cupy-cuda111 ; extra == 'cuda111'
+Provides-Extra: cuda11x
+Requires-Dist: cupy-cuda11x ; extra == 'cuda11x'
+Provides-Extra: cuda12x
+Requires-Dist: cupy-cuda12x ; extra == 'cuda12x'
+Provides-Extra: cupy
+Requires-Dist: cupy ; extra == 'cupy'
+Provides-Extra: db
+Requires-Dist: pandas ==2.0.3 ; extra == 'db'
+Requires-Dist: duckdb ==0.8.1 ; extra == 'db'
+Provides-Extra: full
+Requires-Dist: colorama ==0.4.6 ; extra == 'full'
+Requires-Dist: aiohttp ==3.8.5 ; extra == 'full'
+Requires-Dist: pandas ==2.0.3 ; extra == 'full'
+Requires-Dist: duckdb ==0.8.1 ; extra == 'full'
+Requires-Dist: websockets ==11.0.3 ; extra == 'full'
+Provides-Extra: repl
+Requires-Dist: colorama ==0.4.6 ; extra == 'repl'
+Provides-Extra: rocm-4-3
+Requires-Dist: cupy-rocm-4-3 ; extra == 'rocm-4-3'
+Provides-Extra: rocm-5-0
+Requires-Dist: cupy-rocm-5-0 ; extra == 'rocm-5-0'
+Provides-Extra: web
+Requires-Dist: aiohttp ==3.8.5 ; extra == 'web'
+Provides-Extra: ws
+Requires-Dist: websockets ==11.0.3 ; extra == 'ws'
+
+
+![Unit Tests](https://github.com/briangu/klongpy/workflows/Unit%20Tests/badge.svg)
+[![Last Commit](https://img.shields.io/github/last-commit/briangu/klongpy)](https://img.shields.io/github/last-commit/briangu/klongpy)
+[![Dependency Status](https://img.shields.io/librariesio/github/briangu/klongpy)](https://libraries.io/github/briangu/klongpy)
+[![Open Issues](https://img.shields.io/github/issues-raw/briangu/klongpy)](https://github.com/briangu/klongpy/issues)
+[![Repo Size](https://img.shields.io/github/repo-size/briangu/klongpy)](https://img.shields.io/github/repo-size/briangu/klongpy)
+[![GitHub star chart](https://img.shields.io/github/stars/briangu/klongpy?style=social)](https://star-history.com/#briangu/klongpy)
+
+[![Release Notes](https://img.shields.io/github/release/briangu/klongpy)](https://github.com/briangu/klongpy/releases)
+[![Downloads](https://static.pepy.tech/badge/klongpy/month)](https://pepy.tech/project/klongpy)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+[![Twitter](https://img.shields.io/twitter/url/https/twitter.com/klongpy.svg?style=social&label=Follow%20%40KlongPy)](https://twitter.com/klongpy)
+
+# KlongPy: High-Performance Array Programming in Python
+
+KlongPy is a Python adaptation of the [Klong](https://t3x.org/klong) [array language](https://en.wikipedia.org/wiki/Array_programming), known for its high-performance vectorized operations that leverage the power of NumPy. Embracing a "batteries included" philosophy, KlongPy combines built-in modules with Python's expansive ecosystem, facilitating rapid application development with Klong's succinct syntax.
+
+## Core Features
+
+- **Vectorized Operations with NumPy:** At its core, KlongPy uses [NumPy](https://numpy.org/), an [Iverson Ghost](https://analyzethedatanotthedrivel.org/2018/03/31/NumPy-another-iverson-ghost/) descendant from APL, for high-efficiency array manipulations.
+- **CPU and GPU Backend Support:** Incorporating [CuPy](https://github.com/cupy/cupy), KlongPy extends its capabilities to operate on both CPU and GPU backends, ensuring versatile and powerful computing options.
+- **Seamless Integration with Python Ecosystem:** The combination of KlongPy's built-in features with Python's wide-ranging libraries enables developers to build complex applications effortlessly.
+
+## KlongPy's Foundation and Applications
+
+- **Inspired by Nils M Holm:** KlongPy is grounded in the work of [Nils M Holm](https://t3x.org), the original creator of Klong, and is further enriched by his [Klong Book](https://t3x.org/klong/book.html).
+- **Ideal for Diverse Fields:** Data scientists, quantitative analysts, researchers, and programming language enthusiasts will find KlongPy especially beneficial for its versatility and performance.
+
+KlongPy thus stands as a robust tool, blending the simplicity of Klong with the extensive capabilities of Python, suitable for a wide range of computational tasks.
+
+# Quick install
+
+```bash
+pip3 install "klongpy[full]"
+```
+
+# Feature Overview
+
+KlongPy is both an Array Language runtime and a set of powerful tools for building high performance data analysis and distributed computing applications.  Some of the features include:
+
+* [__Array Programming__](https://en.wikipedia.org/wiki/Array_programming): Based on [Klong](https://t3x.org/klong), a concise, expressive, and easy-to-understand array programming language. Its simple syntax and rich feature set make it an excellent tool for data scientists and engineers.
+* [__Speed__](docs/performance.md): Designed for high-speed vectorized computing, enabling you to process large data sets quickly and efficiently on either CPU or GPU.
+* [__Fast Columnar Database__](docs/fast_columnar_database.md): Includes integration with [DuckDb](http://duckdb.org), a super fast in-process columnar store that can operate directly on NumPy arrays with zero-copy.
+* [__Inter-Process Communication (IPC)__](docs/ipc_capabilities.md): Includes built-in support for IPC, enabling easy communication between different processes and systems. Ticker plants and similar pipelines are easy to build.
+* [__Table and Key-Value Store__](docs/table_and_key_value_stores.md): Includes a simple file-backed key value store that can be used to store database tables or raw key/value pairs.
+* [__Python Integration__](docs/python_integration.md): Seamlessly compatible with Python and modules, allowing you to leverage existing Python libraries and frameworks.
+* [__Web server__](docs/web_server.md): Includes a web server, making it easy to build sites backed by KlongPy capabilities.
+* [__Timers__](docs/timer.md): Includes periodic timer facility to periodically perform tasks.
+
+# KlongPy Examples
+
+Explore KlongPy with these examples. Each snippet highlights a unique aspect of Klong, demonstrating its versatility in various programming scenarios.
+
+Before we get started, you may be wondering: *Why is the syntax so terse?*
+
+The answer is that it's based on the APL style array language programming and there's a good reason why its compact nature is actually helpful.
+
+Array language style lets you describe WHAT you want the computer to do and it lets the computer figure out HOW to do it.  This frees you up from the details while letting the computer figure out how to go as fast as possible.
+
+Less code to write and faster execution.
+
+---
+
+Just so the following examples make more sense when you see the REPL outputs, there are a few quick rules about Klong functions.  Functions only take up to 3 parameters and they are ALWAYS called x,y and z.
+
+A function with
+
+* no parameters is called a nilad
+* one parameter is called a monad (x)
+* two parameters: dyad (x and y)
+* three parameters: a triad (x, y and z)
+
+The reason that Klong functions only take up to 3 parameters AND name them for you is both convience and compactness.
+
+---
+
+## 0. Start the REPL
+
+```Bash
+$ rlwrap kgpy
+
+Welcome to KlongPy REPL v0.6.0
+Author: Brian Guarraci
+Web: http://klongpy.org
+]h for help; crtl-d or ]q to quit
+
+?>
+```
+
+## 1. Basic Arithmetic
+
+Let's get started with the basics and build up to some more interesting math.  Expressions are evaluated from right to left: 3*2 and then + 5
+
+```kgpy
+?> 5+3*2
+11
+```
+
+KlongPy is more about arrays of things, so let's define sum and count functions over an array:
+
+```kgpy
+?> sum::{+/x}  :" sum + over / the array x
+:monad
+?> sum([1 2 3])
+6
+?> count::{#x}
+:monad
+?> count([1 2 3])
+3
+```
+
+Now that we know the sum and number of elements we can compute the average:
+
+```kgpy
+?> avg::{sum(x)%count(x)} :" average is the sum divided by the number of elements
+:monad
+?> avg([1 2 3])
+2
+```
+
+## 2. Math on arrays
+
+Let's dig into more interesting operations over array elements.  There's really big performance differences in how you approach the problem and it's important to see the difference.
+
+For the simple case of squaring numbers in a list, let's try a couple solutions:
+
+```kgpy
+?> {x*x}'[1 2 3 4 5] :" square each element as we iterate over the array
+[1 4 9 16 25]
+```
+
+The vectorized approach will do an element-wise multiplication in bulk:
+
+```kgpy
+?> a::[1 2 3 4 5];a*a  :" a*a multiplies the arrays
+[1 4 9 16 25]
+```
+
+The vectorized approach is going to be MUCH faster.  Let's crank up the size of the array and time it:
+
+```kgpy
+$> .l("time")
+:monad
+$> a::!1000;#a
+1
+$> fast::{{a*a}'!1000}
+:nilad
+$> slow::{{{x*x}'a}'!1000}
+:nilad
+$> time(fast)
+0.015867948532104492
+$> time(slow)
+2.8987138271331787
+```
+
+Vectors win by 182x!  Why?  Because when you perform a bulk vector operation the CPU can perform the math with much less overhead and do many more operations at a time because it has the entire computation presented to it at once.
+
+KlongPy aims to give you tools that let you conveniently exploit this vectorization property - and go FAST!
+
+Less code to write AND faster to compute.
+
+## 3. Data Analysis with Python Integration
+
+KlongPy integrates seamlessly with Python so that the strenghts of both can be combined.  It's easy to use KlongPy from Python and vice versa.
+
+For example, let's say we have some data in Python that we want to operate on in KlongPy.  We can just directly use the interpreter in Python and run functions on data we put into the KlongPy context:
+
+```python
+from klongpy import KlongInterpreter
+import numpy as np
+
+data = np.array([1, 2, 3, 4, 5])
+klong = KlongInterpreter()
+# make the data NumPy array available to KlongPy code by passing it into the interpreter
+# we are creating a symbol in KlongPy called 'data' and assigning the external NumPy array value
+klong['data'] = data
+# define the average function in KlongPY
+klong('avg::{(+/x)%#x}')
+# call the average function with the external data and return the result.
+r = klong('avg(data)')
+print(r) # expected value: 3
+```
+
+It doesn't make sense to write code in Klong that already exists in other libraries.  We can directly access them via the python inport functions (.py and .pyf).
+
+How about we use the NumPy FFT?
+
+```kgpy
+?> .pyf("numpy";"fft");fft::.pya(fft;"fft")
+:monad
+?> signal::[0.0 1.0 0.0 -1.0] :" Example simple signal
+[0.0 1.0 0.0 -1.0]
+?> result::fft(signal)
+[0j -2j 0j 2j]
+```
+
+Now you can use NumPy or other libraries to provide complex functions while KlongPy lets you quickly prepare and process the vectors.
+
+There's a lot more we can do with interop but let's move on for now!
+
+## 4. Database Functionality
+
+KlongPy leverages a high-performance columnar store called DuckDb that uses zero-copy NumPy array operations behind the scenes.   This database allows fast interop between KlongPy and DuckDb (the arrays are not copied) so that applications can manage arrays in KlongPy and then instantly perform SQL on the data for deeper insights.
+
+It's easy to create a table and a db to query:
+
+```kgpy
+?> .py("klongpy.db")
+?> t::.table([["name" ["Alice" "Bob"]] ["age" [25 30]]])
+name age
+Alice 25
+Bob 30
+?> db::.db(:{},"T",t)
+?> db("select * from T where age > 27")
+name age
+Bob 30
+```
+
+## 5. IPC, Remote Function Calls and Asynchronous operations
+
+Inter Process Communication (IPC) lets you build distributed and interconnected KlongPy programs and services.
+
+KlongPy treats IPC connections to servers as functions.  These functions let you call the server and ask for things it has in it's memory - they can be other functions or values, etc.  For example you can ask for a reference to a remote function and you will get a local function that when you call it runs on teh server with your arguemnts.  This general "remote proxy" approach allows you to write your client code in the same way as if all the code were running locally.
+
+To see this in action, let's setup a simple scenario where the server has an "avg" function and the client wants to call it.
+
+Start a server in one terminal:
+
+```kgpy
+?> avg::{(+/x)%#x}
+:monad
+?> .srv(8888)
+1
+```
+
+Start the client and make the connection to the server as 'f'.  In order to pass parameters to a remote function we form an array of the function symbol followed by the parameters (e.g. :avg,,!100)
+
+```kgpy
+?> f::.cli(8888) :" connect to the server
+remote[localhost:8888]:fn
+?> f(:avg,,!100) : call the remote function "avg" directly with the paramter !100
+49.5
+```
+
+Let's get fancy and make a local proxy to the remote function:
+
+```kgpy
+?> myavg::f(:avg) :" reference the remote function by it's symbol :avg and assign to a local variable called myavg
+remote[localhost:8888]:fn:avg:monad
+?> myavg(!100) :" this runs on the server with !100 array passed to it as a parameter
+49.5
+```
+
+Since remote functions may take a while we can wrap them with an async wrapper and have it call our callback when completed:
+
+```kgpy
+?> afn::.async(myavg;{.d("Avg calculated: ");.p(x)})
+async::monad
+?> afn(!100)
+Avg calculated: 49.5
+1
+```
+
+## 6. Web Server Implementation
+
+In addition to IPC we can also expose data via a standard web server.  This capability lets you have other ways of serving content that can be either exposing interesting details about some computation or just a simple web server for other reasons.
+
+Let's create a file called web.kg with the following code that adds one index handler:
+
+```text
+.py("klongpy.web")
+data::!10
+index::{x; "Hello, Klong World! ",data}
+.web(8888;:{},"/",index;:{})
+.p("ready at http://localhost:8888")
+```
+
+We can run this web server as follows:
+
+```bash
+$ kgpy web.kg
+ready at http://localhost:8888
+```
+
+In another terminal:
+
+```bash
+$ curl http://localhost:8888
+['Hello, Klong World! ' 0 1 2 3 4 5 6 7 8 9]
+```
+
+## Conclusion
+
+These examples are designed to illustrate the "batteries included" approach, ease of use and diverse applications of KlongPy, making it a versatile choice for various programming needs.
+
+[Check out the examples folder for more](https://github.com/briangu/klongpy/tree/main/examples).
+
+# Installation
+
+### CPU
+
+```bash
+pip3 install klongpy
+```
+
+### GPU support
+
+Choose your CuPy prebuilt binary or from source.  Note, the [ROCM](docs/ROCM.md) support for CuPy is experimental and likely will have issues.
+
+'cupy' => build from source
+'cuda12x' => "cupy-cuda12x"
+'cuda11x' => "cupy-cuda11x"
+'cuda111' => "cupy-cuda111"
+'cuda110' => "cupy-cuda110"
+'cuda102' => "cupy-cuda102"
+'rocm-5-0' => "cupy-rocm-5-0"
+'rocm-4-3' => "cupy-rocm-4-3"
+
+```bash
+pip3 install "klongpy[cupy]"
+```
+
+### All application tools (db, web, REPL, etc.)
+
+```bash
+pip3 install "klongpy[full]"
+```
+
+# Status
+
+KlongPy is a superset of the Klong array language.  It currently passes all of the integration tests provided by klong as well as additional suites.
+
+Since CuPy is [not 100% compatible with NumPy](https://docs.cupy.dev/en/stable/user_guide/difference.html), there are currently some gaps in KlongPy between the two backends.  Notably, strings are supported in CuPy arrays so KlongPy GPU support currently is limited to math.
+
+Primary ongoing work includes:
+
+* Additional tools to make KlongPy applications more capable.
+* Additional syntax error help
+* Actively switch between CuPy and NumPy when incompatibilities are present
+
+# Differences from Klong
+
+KlongPy is effectively a superset of the Klong language, but has some key differences:
+
+* Infinite precision: The main difference in this implementation of Klong is the lack of infinite precision.  By using NumPy we are restricted to doubles.
+* Python integration: Most notably, the ".py" command allows direct import of Python modules into the current Klong context.
+* KlongPy aims to be more "batteries included" approach to modules and contains additional features such as IPC, Web service, Websockets, etc.
+
+# Related
+
+* [Klupyter - KlongPy in Jupyter Notebooks](https://github.com/briangu/klupyter)
+* [Visual Studio Code Syntax Highlighting](https://github.com/briangu/klongpy-vscode)
+* [Advent Of Code in KlongPy](https://github.com/briangu/aoc)
+
+## Develop
+
+git clone https://github.com/briangu/klongpy.git
+cd klongpy
+python3 setup.py develop
+
+### Running tests
+
+```bash
+python3 -m unittest
+```
+
+# Acknowledgement
+
+HUGE thanks to Nils M Holm for his work on Klong and providing the foundations for this interesting project.

{klongpy-0.6.1.dist-info → klongpy-0.6.2.dist-info}/RECORD

@@ -2,7 +2,7 @@ klongpy/__init__.py,sha256=8K0RIIZh2QLdOBA68oym_MeEwO33ffcDOZAU3etJ7kA,107
 klongpy/adverbs.py,sha256=W6HAL4rxWV6djrILnahRqw8o6-Yq9pUWIUznSJG8-f0,12566
 klongpy/backend.py,sha256=qA2DulWLVtlOYs3opwJIod0XZVTLCMvqjuttK2_-CHo,2897
 klongpy/core.py,sha256=40AGWEuYP5AD8E8V4XNzTjsZH8lF6y7vU3jTaBaatFk,27193
-klongpy/dyads.py,sha256=pgAePIzILlLiQCTsMZ4xuLD43RzGPNII4pSDTuDzq30,31369
+klongpy/dyads.py,sha256=Fhr2u2PgLKKyBOaiB8ofyDl3-TDceoSmgfrLBj5WyFI,31247
 klongpy/interpreter.py,sha256=9cd6aZfGVQ7cP1VXDIksxgoHCIReeh62QAxCNfwGQjQ,23792
 klongpy/monads.py,sha256=6PoLROije2u-ojQZPu2f5FYnRav02Bx8eovrm2C29NY,14158
 klongpy/sys_fn.py,sha256=cG9IALvpjMmfXE0sQTmqsmF0KDR8tzClDYLrcghEt1I,23727
@@ -14,7 +14,7 @@ klongpy/db/__init__.py,sha256=KUEKYPphDAYLdRfzcRuvj3Mv9ti6x_y9hjIzNh0ufRI,282
 klongpy/db/df_cache.py,sha256=_2tI3Gv5nzASlQfnFiwnXrkPnZ-mzJzyM0tt_gi3c5o,3325
 klongpy/db/file_cache.py,sha256=MnSTRB0B8yi8yAHFXGqduqaBqGhllLKxo1rrqMJ6-ik,9294
 klongpy/db/helpers.py,sha256=vdxSY2W-N13cBF3PUi86WlecGSz0ZkJN0OR7Jb6E9iM,1457
-klongpy/db/sys_fn_db.py,sha256=aBGTdWWM69efTxNg3GMlWbGECvaPA_bwENCHjZC-MLI,9047
+klongpy/db/sys_fn_db.py,sha256=R5yfEa8LvCbCdSCV2BrZXfGrh7IbwSK9zuZ7OVRd8UI,9234
 klongpy/db/sys_fn_kvs.py,sha256=--FTMQyKdn8vBx0qG4mOBzdd2rtBfm0rW4c0AnB0G94,4114
 klongpy/lib/csv.kg,sha256=w6FvenQmkl0EF2kieQxxloNOFgZbt3M9IMSYSdImJcA,1498
 klongpy/lib/edt.kg,sha256=aJsjCxAkbfzwxWg8mF9nRDbCCt2fqtCWv0CPsq4SIHw,707
@@ -34,7 +34,7 @@ klongpy/web/__init__.py,sha256=lM_40n_qzwYSx_OFi4RqFwsxQe8GGBtJm3_GLLxn9-8,101
 klongpy/web/sys_fn_web.py,sha256=En9_8Xh1uIaGz8CRyWsYuwpt0rbpNWvDCKFJ6KZpU5o,4710
 klongpy/ws/__init__.py,sha256=GKl5804W3XNkDER0mkdFf-4cIVjgeP9M3tfNg_tCv3c,114
 klongpy/ws/sys_fn_ws.py,sha256=Ba6MmmTS6cWHlruGgNznNXXPQX2zaBQmAEwkz9ge7sk,18300
-klongpy-0.6.1.data/scripts/kgpy,sha256=nDF66LclUBK6qILB2dgJ9VKAlGlPKMQfoJ4CvH5d_UQ,12119
+klongpy-0.6.2.data/scripts/kgpy,sha256=nDF66LclUBK6qILB2dgJ9VKAlGlPKMQfoJ4CvH5d_UQ,12119
 tests/__init__.py,sha256=xdETQVm8SNlASCru27aks1KZwSs2Oh-0NR5zrccSqb0,221
 tests/gen_join_over.py,sha256=ZiliJS6Lz0rqaZAfAeJFbPOU109PRF0QFE6262a4TPU,3684
 tests/gen_py_suite.py,sha256=ihXihalZkjN9Poy3m-1FVGdZy_YYWS0CEkY68pDgGso,2495
@@ -53,11 +53,11 @@ tests/perf_vector.py,sha256=p76TVCS5szG0BX7rzquvKb7xikxUQOUmQttvTS0EH44,976
 tests/test_accel.py,sha256=apKi0PN9l9ztDLTwtdo6esWCl7_Yo97xljfNY0q9hug,6305
 tests/test_df_cache.py,sha256=vK36B0ENNwTYgv_JvRZimum9EEoPw-qgi4BWQWHRqQQ,3302
 tests/test_examples.py,sha256=hfpCQlcLR4pjUeSk9uAhtgnsjLEHLc_sfvRRWq8W1iY,2131
-tests/test_extra_suite.py,sha256=czwr8hHlI5IpMxHbaeC9fLK_SnNuuFLWiHCIeJMzhm8,12538
+tests/test_extra_suite.py,sha256=WPIA_UcP5BVvZIlkiDMUx_877ts3rYoVLo0grQh4JW8,12967
 tests/test_file_cache.py,sha256=lkE4bXPR9qp6wRdNHcG0f0Fb_BBB-2oocRttmCbXT7A,7733
 tests/test_interop.py,sha256=efUVa-IB43TecUhOm1rfTlK9G6kcf0U4q6EEVf06gaE,5837
 tests/test_kgtests.py,sha256=7lUgKYt9TQiYxAXWqE4VuaZo7frhbzSd9ofa6_pWBLE,2190
-tests/test_known_bugs.py,sha256=eEeJc7VXnOyG498nynvJFL3d_iKYxYqUQEjBtirOZ30,6053
+tests/test_known_bugs.py,sha256=shBMW2rwQV0CLtrPZZ54TDFII7W5fxi7AOpZyvbydM8,6481
 tests/test_prog.py,sha256=8P019M-hiW9jxe9FgQS3Q7wqxoTrkhclMTk6xO4HBWU,3311
 tests/test_suite.py,sha256=cKTH9ICQUFPyM2xcmQNuS_jueDHLD2ukVMeM_KstTbQ,73051
 tests/test_suite_file.py,sha256=zZLXK1xEQgUH0OO9J53jkF4j0rjbgIXsWlsNo5puriA,4027
@@ -67,8 +67,8 @@ tests/test_sys_fn_ipc.py,sha256=tV7uCGxXQCHZQ2CkffD4j5h01TYBe1ozC_wImiANyHA,2160
 tests/test_sys_fn_timer.py,sha256=VUfi1AK9SCzA9g5IgbfpdEMW2AP4-9dgVOpJXhMMQhA,4447
 tests/test_util.py,sha256=k4UCR0avjWNxo_jO710x32AAMOAePOx79EOt58wL584,7942
 tests/utils.py,sha256=QrLAR4-WXVSDrC60qhl78_NE-3idp0mgLJmSZ5seyWw,3721
-klongpy-0.6.1.dist-info/LICENSE,sha256=Nk1iW-tI7n0z8R2jMqVm9SuFIoa8iukXsFcJoPqQgns,1071
-klongpy-0.6.1.dist-info/METADATA,sha256=P6EL66mLvMrNPzZ8GUSGR_oo1coMGdBxiO2dVy7SLMU,10799
-klongpy-0.6.1.dist-info/WHEEL,sha256=yQN5g4mg4AybRjkgi-9yy4iQEFibGQmlz78Pik5Or-A,92
-klongpy-0.6.1.dist-info/top_level.txt,sha256=1dsI3MhtibeAZuB7WPW58dpNgyGvXLX5EEHqm46NQYo,14
-klongpy-0.6.1.dist-info/RECORD,,
+klongpy-0.6.2.dist-info/LICENSE,sha256=Nk1iW-tI7n0z8R2jMqVm9SuFIoa8iukXsFcJoPqQgns,1071
+klongpy-0.6.2.dist-info/METADATA,sha256=ZAepEk3sKh2IsjRV-FxdQdY3WvKOtJLaEz664BLvJIQ,16602
+klongpy-0.6.2.dist-info/WHEEL,sha256=yQN5g4mg4AybRjkgi-9yy4iQEFibGQmlz78Pik5Or-A,92
+klongpy-0.6.2.dist-info/top_level.txt,sha256=1dsI3MhtibeAZuB7WPW58dpNgyGvXLX5EEHqm46NQYo,14
+klongpy-0.6.2.dist-info/RECORD,,

tests/test_extra_suite.py

@@ -12,6 +12,18 @@ class TestExtraCoreSuite(unittest.TestCase):
     def assert_eval_cmp(self, a, b, klong=None):
         self.assertTrue(eval_cmp(a, b, klong=klong))
 
+    def test_negate_array_result_type(self):
+        """ ensure the result type of negating an array is correct """
+        klong = KlongInterpreter()
+        r = klong("-1]")
+        self.assertTrue(is_integer(r))
+        r = klong("-1.0]")
+        self.assertTrue(is_float(r))
+        r = klong("-[1 2 3]")
+        self.assertTrue(r.dtype == int)
+        r = klong("-[1.0 2.0 3.0]")
+        self.assertTrue(r.dtype == float) 
+
     def test_match_empty_array_to_undefined_symbol(self):
         """ symbol is undefined so does not match the empty array """
         klong = KlongInterpreter()

tests/test_known_bugs.py

@@ -4,11 +4,29 @@ import pandas as pd
 from utils import *
 
 from klongpy import KlongInterpreter
-from klongpy.core import KGChar
+from klongpy.core import KGChar, KGSym
 
 
 class TestKnownBugsSuite(unittest.TestCase):
 
+    @unittest.skip
+    def test_array_substitution(self):
+        # [!10] should eval to [:! 10] but it doesn't
+        klong = KlongInterpreter()
+        r = klong('[!10]')
+        self.assertTrue(kg_equal(r, [KGSym('!'), 10]))
+        # [3*a^2] should eval to [3 :* :a :^ 2] but it doesn't
+        r = klong('[3*a^2]')
+        self.assertTrue(kg_equal(r, [3, KGSym('*'), KGSym('a'), KGSym('^'), 2]))
+
+    @unittest.skip
+    def test_should_fail_parsing(self):
+        """
+           this is a bug in both original Klong and KlongPy:
+           fn::{[a b c]::1;a::1;b::2;c::3} the :: after variable declaration is a syntax error
+        """
+        pass
+
     @unittest.skip
     def test_calling_time_with_lambda(self):
         """
@@ -31,13 +49,6 @@ class TestKnownBugsSuite(unittest.TestCase):
         r = klong('T@"col1"')
         self.assertTrue(kg_equal(r, data['col1']))
 
-    @unittest.skip
-    def test_join_nested_array(self):
-        klong = KlongInterpreter()
-        r = klong("a::!10;k::3;c::(,,,1#a),k")
-        # currently this flattens to [[0], 3]
-        self.assertTrue(kg_equal(r,[[[[0]]],3]))
-
     @unittest.skip
     def test_extra_spaces(self):
         klong = KlongInterpreter()

klongpy-0.6.1.dist-info/METADATA

@@ -1,305 +0,0 @@
-Metadata-Version: 2.1
-Name: klongpy
-Version: 0.6.1
-Summary: High-Performance Klong array language with rich Python integration.
-Author: Brian Guarraci
-License: MIT
-Classifier: Programming Language :: Python :: 3
-Classifier: License :: OSI Approved :: MIT License
-Requires-Python: <3.12,>=3.8
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: numpy ~=1.24.4
-Provides-Extra: cuda102
-Requires-Dist: cupy-cuda102 ; extra == 'cuda102'
-Provides-Extra: cuda110
-Requires-Dist: cupy-cuda110 ; extra == 'cuda110'
-Provides-Extra: cuda111
-Requires-Dist: cupy-cuda111 ; extra == 'cuda111'
-Provides-Extra: cuda11x
-Requires-Dist: cupy-cuda11x ; extra == 'cuda11x'
-Provides-Extra: cuda12x
-Requires-Dist: cupy-cuda12x ; extra == 'cuda12x'
-Provides-Extra: cupy
-Requires-Dist: cupy ; extra == 'cupy'
-Provides-Extra: db
-Requires-Dist: pandas ==2.0.3 ; extra == 'db'
-Requires-Dist: duckdb ==0.8.1 ; extra == 'db'
-Provides-Extra: full
-Requires-Dist: colorama ==0.4.6 ; extra == 'full'
-Requires-Dist: aiohttp ==3.8.5 ; extra == 'full'
-Requires-Dist: pandas ==2.0.3 ; extra == 'full'
-Requires-Dist: duckdb ==0.8.1 ; extra == 'full'
-Requires-Dist: websockets ==11.0.3 ; extra == 'full'
-Provides-Extra: repl
-Requires-Dist: colorama ==0.4.6 ; extra == 'repl'
-Provides-Extra: rocm-4-3
-Requires-Dist: cupy-rocm-4-3 ; extra == 'rocm-4-3'
-Provides-Extra: rocm-5-0
-Requires-Dist: cupy-rocm-5-0 ; extra == 'rocm-5-0'
-Provides-Extra: web
-Requires-Dist: aiohttp ==3.8.5 ; extra == 'web'
-Provides-Extra: ws
-Requires-Dist: websockets ==11.0.3 ; extra == 'ws'
-
-
-![Unit Tests](https://github.com/briangu/klongpy/workflows/Unit%20Tests/badge.svg)
-[![Last Commit](https://img.shields.io/github/last-commit/briangu/klongpy)](https://img.shields.io/github/last-commit/briangu/klongpy)
-[![Dependency Status](https://img.shields.io/librariesio/github/briangu/klongpy)](https://libraries.io/github/briangu/klongpy)
-[![Open Issues](https://img.shields.io/github/issues-raw/briangu/klongpy)](https://github.com/briangu/klongpy/issues)
-[![Repo Size](https://img.shields.io/github/repo-size/briangu/klongpy)](https://img.shields.io/github/repo-size/briangu/klongpy)
-[![GitHub star chart](https://img.shields.io/github/stars/briangu/klongpy?style=social)](https://star-history.com/#briangu/klongpy)
-
-[![Release Notes](https://img.shields.io/github/release/briangu/klongpy)](https://github.com/briangu/klongpy/releases)
-[![Downloads](https://static.pepy.tech/badge/klongpy/month)](https://pepy.tech/project/klongpy)
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-
-[![Twitter](https://img.shields.io/twitter/url/https/twitter.com/klongpy.svg?style=social&label=Follow%20%40KlongPy)](https://twitter.com/klongpy)
-<!-- [![Discord]](https://discord.gg/z2vvTvxa) -->
-
-# KlongPy
-
-KlongPy is a Python adaptation of the [Klong](https://t3x.org/klong) [array language](https://en.wikipedia.org/wiki/Array_programming), offering high-performance vectorized operations. It prioritizes compatibility with Python, thus allowing seamless integration of Python's expansive ecosystem while retaining Klong's succinctness. With the inclusion of [CuPy](https://github.com/cupy/cupy), KlongPy can operate using both CPU and GPU backends. It utilizes [NumPy](https://numpy.org/), an [Iverson Ghost](https://analyzethedatanotthedrivel.org/2018/03/31/NumPy-another-iverson-ghost/) descendant from APL, as its core. This means its runtime can target either GPU or CPU backends. The framework's foundation lies in [Nils M Holm](https://t3x.org)'s work, the original developer of Klong, who has authored a [Klong Book](https://t3x.org/klong/book.html). KlongPy is especially useful for data scientists, quantitative analysts, researchers, and programming language aficionados.
-
-# Quick install
-
-```bash
-pip3 install "klongpy[full]"
-```
-
-# Overview
-
-KlongPy is both an Array Language runtime and a set of powerful tools for building high performance data analysis and distributed computing applications.  Some of the features include:
-
-* [__Array Programming__](https://en.wikipedia.org/wiki/Array_programming): Based on [Klong](https://t3x.org/klong), a concise, expressive, and easy-to-understand array programming language. Its simple syntax and rich feature set make it an excellent tool for data scientists and engineers.
-* [__Speed__](docs/performance.md): Designed for high-speed vectorized computing, enabling you to process large data sets quickly and efficiently on either CPU or GPU.
-* [__Fast Columnar Database__](docs/fast_columnar_database.md): Includes integration with [DuckDb](http://duckdb.org), a super fast in-process columnar store that can operate directly on NumPy arrays with zero-copy.
-* [__Inter-Process Communication (IPC)__](docs/ipc_capabilities.md): Includes built-in support for IPC, enabling easy communication between different processes and systems. Ticker plants and similar pipelines are easy to build.
-* [__Table and Key-Value Store__](docs/table_and_key_value_stores.md): Includes a simple file-backed key value store that can be used to store database tables or raw key/value pairs.
-* [__Python Integration__](docs/python_integration.md): Seamlessly compatible with Python and modules, allowing you to leverage existing Python libraries and frameworks.
-* [__Web server__](docs/web_server.md): Includes a web server, making it easy to build sites backed by KlongPy capabilities.
-* [__Timers__](docs/timer.md): Includes periodic timer facility to periodically perform tasks.
-
-# Examples
-
-```
-
-```
-
-Consider this simple Klong expression that computes an array's average: `(+/a)%#a`. Decoded, it means "sum of 'a' divided by the length of 'a'", as read from right to left.
-
-Below, we define the function 'avg' and apply it to the array of 1 million integers (as defined by !1000000)
-
-Let's try this in the KlongPy REPL:
-
-```Bash
-$ rlwrap kgpy
-
-Welcome to KlongPy REPL v0.6.0
-Author: Brian Guarraci
-Web: http://klongpy.org
-]h for help; crtl-d or ]q to quit
-
-?> avg::{(+/x)%#x}
-:monad
-?> avg(!1000000)
-499999.5
-```
-
-Now let's time it (first, right it once, then 100 times):
-
-```
-?> ]T avg(!1000000)
-total: 0.0032962500117719173 per: 0.0032962500117719173
-?> ]T:100 avg(!1000000)
-total: 0.10882879211567342 per: 0.0010882879211567343
-```
-
-We can also import Python custom or standard modules to use directly in Klong language.
-
-```
-?> .pyf("math";"pi")
-1
-?> pi
-3.141592653589793
-```
-
-Here we import the fsum function from standard Python math library and redefine avg to use 'fsum':
-
-```
-?> .pyf("math";"fsum")
-1
-?> favg::{fsum(x)%#x}
-:monad
-?> favg(!1000000)
-499999.5
-```
-
-Notice that using fsum is slower than using Klong '+/'.  This is because the '+/' operation is vectorized while fsum is not.
-
-```
-?> ]T favg(!1000000)
-total: 0.050078875152394176 per: 0.050078875152394176
-?> ]T:100 favg(!1000000)
-total: 2.93945804098621 per: 0.029394580409862103
-```
-
-To use KlongPy within Python, here's a basic outline:
-
-```python
-from klongpy import KlongInterpreter
-
-# instantiate the KlongPy interpeter
-klong = KlongInterpreter()
-
-# define average function in Klong (Note the '+/' (sum over) uses np.add.reduce under the hood)
-klong('avg::{(+/x)%#x}')
-
-# create a billion random uniform values [0,1)
-data = np.random.rand(10**9)
-
-# reference the 'avg' function in Klong interpeter and call it directly from Python.
-r = klong['avg'](data)
-
-print(f"avg={np.round(r,6)}")
-```
-
-And let's run a performance comparison between CPU and GPU backends:
-
-```python
-import time
-from klongpy.backend import np
-from klongpy import KlongInterpreter
-
-klong = KlongInterpreter()
-klong('avg::{(+/x)%#x}')
-
-data = np.random.rand(10**9)
-
-start = time.perf_counter_ns()
-r = klong['avg'](data)
-stop = time.perf_counter_ns()
-
-print(f"avg={np.round(r,6)} in {round((stop - start) / (10**9), 6)} seconds")
-```
-
-Run (CPU)
-
-    $ python3 tests/perf_avg.py
-    avg=0.5 in 0.16936 seconds
-
-Run (GPU)
-
-    $ USE_GPU=1 python3 tests/perf_avg.py
-    avg=0.500015 in 0.027818 seconds
-
-# Installation
-
-### CPU
-
-    $ pip3 install klongpy
-
-### GPU support
-
-    Choose your CuPy prebuilt binary or from source.  Note, the [ROCM](docs/ROCM.md) support for CuPy is experimental and likely will have issues.
-
-    'cupy' => build from source
-    'cuda12x' => "cupy-cuda12x"
-    'cuda11x' => "cupy-cuda11x"
-    'cuda111' => "cupy-cuda111"
-    'cuda110' => "cupy-cuda110"
-    'cuda102' => "cupy-cuda102"
-    'rocm-5-0' => "cupy-rocm-5-0"
-    'rocm-4-3' => "cupy-rocm-4-3"
-
-    $ pip3 install klongpy[cupy]
-
-### All application tools (db, web, REPL, etc.)
-
-    $ pip3 install "klongpy[full]"
-
-
-# REPL
-
-KlongPy has a REPL similar to Klong's REPL.
-
-```bash
-$ pip3 install klongpy[repl]
-$ rlwrap kgpy
-
-Welcome to KlongPy REPL
-author: Brian Guarraci
-repo  : https://github.com/briangu/klongpy
-crtl-c to quit
-
-?> 1+1
-2
->? "hello, world!"
-hello, world!
-?> prime::{&/x!:\2+!_x^1%2}
-:monad
-?> prime(4)
-0
-?> prime(251)
-1
-?> ]T prime(251)
-total: 0.0004914579913020134 per: 0.0004914579913020134
-```
-
-Read about the [prime example here](https://t3x.org/klong/prime.html).
-
-
-# Status
-
-KlongPy aims to be a complete implementation of klong.  It currently passes all of the integration tests provided by klong as well as additional suites.
-
-Since CuPy is [not 100% compatible with NumPy](https://docs.cupy.dev/en/stable/user_guide/difference.html), there are currently some gaps in KlongPy between the two backends.  Notably, strings are supported in CuPy arrays so KlongPy GPU support currently is limited to math.
-
-Primary ongoing work includes:
-
-* Additional tools to make KlongPy applications more capable.
-* Additional syntax error help
-* Actively switch between CuPy and NumPy when incompatibilities are present
-    * Work on CuPy kernels is in this branch: _cupy_reduce_kernels
-
-# Differences from Klong
-
-KlongPy is effectively a superset of the Klong language, but has some key differences:
-
-    * Infinite precision: The main difference in this implementation of Klong is the lack of infinite precision.  By using NumPy we are restricted to doubles.
-    * Python integration: Most notably, the ".py" command allows direct import of Python modules into the current Klong context.
-    * IPC - KlongPy supports IPC between KlongPy processes.
-
-# Related
-
- * [Klupyter - KlongPy in Jupyter Notebooks](https://github.com/briangu/klupyter)
- * [Advent Of Code '22](https://github.com/briangu/aoc/tree/main/22)
-
-
-# Unused operators
-
-The following operators are yet to be used:
-
-```
-:! :& :, :< :> :?
-```
-
-# Contribute
-
-### Develop
-
-    $ git clone https://github.com/briangu/klongpy.git
-    $ cd klongpy
-    $ python3 setup.py develop
-
-### Running tests
-
-```bash
-python3 -m unittest
-```
-
-
-# Acknowledgement
-
-HUGE thanks to Nils M Holm for his work on Klong and providing the foundations for this interesting project.
-