arcadedb 0.5.0 → 0.5.4

data/README.md

@@ -1,317 +1,212 @@
-#  ArcadeDB
+# ArcadeDB — Ruby Interface
 
-Ruby Interface to a [Arcade Database](https://arcadedb.com/).
+Ruby interface to [ArcadeDB](https://arcadedb.com/) via the HTTP-JSON API.
+Responses map to a Dry::Struct based ORM: each database type is a model class; graph,
+document and time-series queries are expressed in ruby. Includes a Query preprocessor
+and a MATCH statement generator.
 
-The program utilizes the HTTP-JSON API to direct database queries to an ArcadeDB server. 
-The server's response is then mapped to an ORM (Object-Relational Mapping) based on DRY::Struct. 
-Each database type is represented by a dedicated Model Class, where complex queries are encapsulated. 
-The program also includes a  Query-Preprocessor and Match-statment generator for constructing custom queries in ruby fashion.
+| | |
+|---|---|
+| Namespace | `Arcade` |
+| Requires | Ruby ≥ 2.7, running ArcadeDB server (tested against 26.10.x) |
+| Transport | HTTP/JSON (HTTPX, persistent connection, basic auth) |
+| Model layer | Dry::Struct — not ActiveRecord |
+| Sibling docs | `RAG_SCHEMA.md` (vector schema), `rails.md`, `CHANGELOG.md` |
 
-***ArcadeDB internally uses `Arcade` as primary namespace**** 
+## Quick start
 
-## Prerequisites
-
-A running AracdeDB-Instance. [Quick-Start-Guide](https://docs.arcadedb.com/#Quick-Start-Docker).
-
-## New Project
-
-```
+```bash
 mkdir project && cd project
-bundle init
-bundle add arcadedb
-bundle add pastel  # for console output
-mkdir model
-mkdir bin
+bundle init && bundle add arcadedb
+mkdir model bin
+# copy bin/console and arcade.yml from this repo, adapt arcade.yml
 ```
-copy `https://github.com/topofocus/arcadedb/blob/main/bin/console` to the `bin` directory  
-copy `https://github.com/topofocus/arcadedb/blob/main/arcade.yml` to the project root and modify to your needs
 
-## Console
+`arcade.yml` (project root or `config/`):
 
-To start an interactive console, a script is provided in the bin-directory.
+```yaml
+:environment:
+  :test:        { dbname: playground, user: root, pass: xxx }
+  :development: { dbname: devel,      user: root, pass: xxx }
+  :production:  { dbname: production, user: root, pass: xxx }
+:admin: { :host: localhost, :port: 2480, :user: root, :pass: xxx }
+:logger: stdout
 ```
-$ cd bin && ./console.rb  t   ( or "d" or "p" for Test, Development and Production environment)
 
-**in Test environment Database definitions  (model-files) of the test-suite are included!**
-```
-
-## Add to a project
-Just require it in your program:
-```ruby
-require "arcade"
-```
-## Config
+Console: `bin/console t|d|p` (test environment also loads the spec model files).
+Connection handle: always available as `Arcade::Init.db` after `Arcade::Init.connect :production`.
 
-Add a file `arcade.yml` at the root of your program or in the `config`-dir  and  provide suitable databases for test, development and production environment.
+## Architecture
 
+```
+Arcade::Base (Dry::Struct)          model layer
+├── Arcade::Vertex   +nodes/edges/traverse/match
+├── Arcade::Edge
+├── Arcade::Document
+├── Arcade::Revision / RevisionRecord    audit trail
+└── Arcade::Timeseries                   native time-series types (0.5.4+)
 
-## Examples  & Specs
-
-The `example` directory contains documented sample files for typical usecases
-
-The `spec`-files in the rspec-test-suite-section are worth reading, too. 
-
-## Implementation
+Arcade::Database   db.query / db.execute / db.transmit   (transactions, model allocation)
+Arcade::Api        raw HTTP primitives (query, execute, post_text, get_json, ...)
+```
 
-The adapter uses a 3 layer concept. 
+Class name maps to database type via `snake_case` (`Demo::User` → `demo_user`).
 
-Top-Layer : `Arcade::Base`-Model-Objects. 
-Similar to ActiveRecord Model Objects but based on [Dry-Struct](https://dry-rb.org/gems/dry-struct/1.0/).
+## Models
 
 ```ruby
-# Example model file  /model/demo/user.rb
+# model/demo/user.rb
 module Demo
   class User < Arcade::Vertex
     attribute :name, Types::Nominal::String
-    
+
     def grandparents
-      db.query( "select in('is_family') from #{rid} ") &.allocate_model
+      db.query("select in('is_family') from #{rid}")&.allocate_model
     end
   end
 end
 __END__
   CREATE PROPERTY demo_user.name STRING
-  CREATE INDEX on demo_user( name ) UNIQUE
+  CREATE INDEX ON demo_user(name) UNIQUE
 ```
 
-Only the `name` attribute is declared. 
-
-`Demo::User.create_type`  creates the type and executes provided database-commands after __END__.   
-
-Other properties are schemaless, but have to be declared in the model-file. 
+`Demo::User.create_type` creates the type and runs the `__END__` DDL.
+Undeclared properties are schemaless — they arrive in the `values` attribute.
 
 ```ruby
 Person.insert name: "Hubert", age: 35
 Person.update set: { age: 36 }, where: { name: 'Hubert' }
-persons = Person.where "age > 40"
-persons.first.update age: 37
-persons.first.update father: Person.create( name: 'Mike', age: 94 )
-
-Person.all
-Person.delete all: true || where: age: 56 , ...
-```
-
-### Nodes and Traverse
-
-`ArcadeDB` wraps common queries of bidirectional connected vertices.
-
-Suppose
-```ruby 
-m = Person.create name: 'Hubert', age: '25'
-f = Person.create name: 'Pauline', age: '28'
-m.assign via: IsMarriedTo, vertex: f , divorced: false
-
+Person.where("age > 40").first.update age: 37
+Person.upsert name: 'Hugo'                       # ensure-exists (needs index on field)
+Person.delete all: true                          # or where: {...}
 ```
-This creates the simple graph 
->  Hubert --- is_married_to --> Pauline 
->                  |
->                  -- divorced (attribute on the edge)
 
-This can be queried through
+## Graph: nodes, traverse, match
 
 ```ruby
-hubert =  Person.find name: 'Hubert'
-pauline = hubert.nodes( :out, via: IsMarriedTo ).first
-# or simply
-pauline = hubert.out.first
+m = Person.insert(name: 'Hubert'); f = Person.insert(name: 'Pauline')
+m.assign via: IsMarriedTo, vertex: f, divorced: false
 
-```
-Conditions may be set, to. 
-```ruby
-hubert.nodes( :out, via: IsMarriedTo, where: "age < 30" )
-```
-gets all wives of hubert, who are younger then 30 years.  
-or
-```ruby
-Person.nodes( :outE, via: IsMarriedTo, where: { divorced: false } )
+hubert = Person.find(name: 'Hubert')
+hubert.nodes(:out, via: IsMarriedTo, where: "age < 30").first   # or hubert.out.first
+hubert.edges(via: IsMarriedTo)                                   # edges carry attributes
 
+start_node.traverse :out, via: Connects, depth: 100              # depth-limited walk
 ```
-gets all wives where the divorced-condition, which is set on the edge, is false. 
-
-## Query
-
-A **Query Preprocessor** is implemented. Its adapted from ActiveOrient. The [documentation](https://github.com/topofocus/active-orient/wiki/OrientQuery)
-is still valid,  however the class has changed to `Arcade::Query`. 
-
-## Match
 
-A simple **Match Statement Generator** is provided for convenience. As the [declarative syntax](https://github.com/ArcadeData/arcadedb-docs/blob/main/src/main/asciidoc/sql/SQL-Match.adoc)  seems to be  very  intuitive, a preprocessor should not be 
-necessary. With the help of some ruby magic, the creation of match-statements is simply an no-brainer.
+MATCH generator — also from a vertex instance (handy when `params[:rid]` is given):
 
 ```ruby
-a =  Arcade::Match.new( type: Person, as: :persons)
-                  .out( IsMarriedTo )
-                  .node( where: 'age < 30')
-a.to_s
-=>  Match { type: person, as: persons }.out('is_married_to'){ where: ( age < 30) } RETURN persons
-
-a.execute.select_results  #  sends the statement to the database and returns Arcade Objects. 
+Arcade::Match.new(type: Person, as: :p).out(IsMarriedTo).node(where: 'age < 30').execute.select_results
+Watchlist.match(symbol: 'iBit', as: :w).out(HasUnderlying).node(as: :u)
+"#49:0".load_rid.match.out(HasUnderlying).node(as: :u)
 ```
 
-Arcade::Match-objects can be used as `from:` argument to Arcade::Query-Statements, thus hybrid queries are 
-easily constructed without extensive string-manipulations.
-
-The **second Layer** handles Database-Requests.  
-In its actual implementation, these requests are delegated to the HTTP/JSON-API.
-
-`Arcade::Init` uses the database specification of the `arcade.yml` file.   
-The database-handle is always present through `Arcade::Init.db`
+Query objects nest (hybrid queries without string surgery):
 
 ```ruby
-DB =  Arcade::Init.db
-
-$ DB.get nn, mm                                 # returns a Aracde:Base object
-                                                # rid is either "#11:10:" or two numbers
-$ DB.query querystring                          # returns either an Array of results (as  Hash) 
-$ DB.execute { querystring }                    # execute a non idempotent query within a (nested) transaction
-$ DB.create <name>, attribute: value ....       # Creates a new <Document | Vertex> and returns the rid
-                                                # Operation is performed as Database-Transaction and is rolled back on error
-$ DB.insert <name>, attribute: value ....       # Inserts a new <Document | Vertex> and returns the new object
-$ DB.create_edge <name>, from: <rid> or [rid, rid, ..] , to: <rid> or [rid, rid, ..]
-
+Query.new(from: match_object, projection: 'median(note_count)', where: '$depth>=50')
 ```
 
-**Convert database input to Arcade::Base Models**
+## TimeSeries (new in 0.5.4)
 
-Either  `DB.query` or  `DB.execute` return the raw JSON-input from the database. It can always converted to model-objects by chaining
-`allocate_model` or `select_result`.
-
-```ruby
-$ DB.query "select from my_names limit 1"
-# which is identical to
-$ My::Names.query( limit: 1).query
- => [{:@out=>0, :@rid=>"#225:6", :@in=>0, :@type=>"my_names", :@cat=>"v", :name=>"Zaber"}] 
-# then
-$ My::Names.query( limit: 1).query.allocate_model.to_human
- => ["<my_names[#225:6]: name: Zaber>" ]
- # replaces the hash with a My::Names Object 
-```
-
-The **Base Layer** implements a low level access to the database API. 
+Native ArcadeDB time-series types: columnar, compressed, tag-partitioned.
+**Not vertices** — no rid, no edges. Membership lives in tags, values in typed fields.
 
 ```ruby
+# model/ohlcv.rb
+class Ohlcv < Arcade::Timeseries
+  timestamp :ts, precision: :second          # date-granular series: use :second
+  tags   symbol: String, currency: String
+  fields open: Float, high: Float, low: Float, close: Float, volume: Integer
+end
 
-$ Arcade::Api.databases                        # returns an array of known databases
-$ Arcade::Api.create_database <a string>       # returns true if successfull
-$ Arcade::Api.drop_database   <a string>       # returns true if successfull
+Ohlcv.create_type
 
-$ Arcade::Api.begin_transaction <database>
-$ Arcade::Api.create_document <database>, <type>,  attribute: value , ...
-$ Arcade::Api.execute( <database>  ) { <query>  }
-$ Arcade::Api.commit <database>                #  or Arcade::Api.rollback  
+# write — Line Protocol under the hood, batched
+Ohlcv.write symbol: 'SAPX:GER', currency: 'EUR', ts: Date.new(2026,10,20),
+            open: 290.0, high: 296.0, low: 290.0, close: 295.5, volume: 120
+Ohlcv.write_many rows   # array of hashes, one POST per batch
 
+# read — SQL range pushdown
+Ohlcv.range from: '2026-01-01', tags: { symbol: 'SAPX:GER' }
+Ohlcv.latest tags: { symbol: 'SAPX:GER' }        # => {ts:, close:, ...} or nil
 
-$ Arcade::Api.query( <database>  ) { <query>  }
-$ Arcade::Api.get_record <database>,  rid      #  returns a hash
+# weekly candles in one query
+Ohlcv.bucket '7d', from: Date.new(2024,10,1), tags: { symbol: 'SAPX:GER' },
+                   aggregate: { close: :last, high: :max, low: :min, volume: :sum }
 ```
 
+Accepted `ts` values: `Time`, `Date`, `DateTime`, ISO-String, epoch-seconds — normalised
+to UTC and to the declared precision. Timezones: server timestamps are stored absolute;
+build Date objects UTC (`Time.utc`) or comparisons drift by the host offset.
 
-`<query>` is  either a  string or   a hash: 
-```ruby 
-{  :query      => "<the query string> ",
-   :language   => one of :sql, :cypher, :gmelin: :neo4 ,
-	 :params     =>   a  hash of parameters,
-	 :limit      => a number ,
-	 :serializer =>  one of :graph, :record }    
-```	   
-
-## ORM Behavior
-
-Simple tasks are implemented on the model layer.
+## RAG
 
-### Ensure that a Record exists
+`require 'arcade/rag_schema'` — vector + full-text schema for retrieval pipelines.
+See `RAG_SCHEMA.md` and `examples/rag_example.rb`.
 
-The `upsert` command either updates or creates a database record.   
+## Transactions & audit
 
 ```ruby
-  User.upsert name: 'Hugo'
-  # or
-  User.upsert set: { age: 46 }, where: { name: 'Hugo' }
+db = Arcade::Init.db
+db.begin_transaction; ... ; db.commit   # or db.rollback
 ```
-either creates the record (and returns it) or returns the existing database entry. Obviously, the addressed attribute (where condition) 
-must have a proper index. 
 
-The `upsert` statement provides a smart method to ensure the presence of a defined starting point. 
+`Arcade::Revision` subclasses keep an append-only `protocol` stack of
+`Arcade::RevisionRecord` entries; every `update` appends (old value + action + user).
 
-### Assign Nodes to a Vertex 
+## Low-level access
 
-Apart from accessing attributes by their method-name, adjacent edges and notes are fetched through
-
-```ruby 
-  new_vertex = ->(n) { Node.create( note_count: n )  }                      ## lambda to create a Node type record
-  nucleus    =  BaseNode.create item: 'b'                                   ## create a start node
-  (1..10).each{ |n| nucleus.assign( via: Connects, vertex: new_vertex[n]) } ## connect nodes via Connects-Edges
-```
-After creating a star-like structure, the environment can be explored 
-```ruby 
-  nucleus.edges.to_human
-  =>["<connects[#80:14] :.: #4:0->{}->#34:2>",                                  
-     "<connects[#79:13] :.: #4:0->{}->#31:1>",                                  
-     ( ... )
-     "<connects[#79:14] :.: #4:0->{}->#31:2>"] 
-
-   nucleus.nodes.to_human
- => ["<node[#34:2]: item: 10>",                                                         
-     "<node[#31:1]: item: 1>",                                                          
-     ( ... )
-     "<node[#31:2]: item: 9>"]    
-```
-
-Edges provide a `vertices`-method to load connected ones.  Both vertices and edges expose `in`, `out`,  `both`-methods to select connections further. 
-Specific edge-classes (types) are provided with the `via:` parameter,  as shown in `assign` above. 
-
-
-### Traverse Facility
-
-To ease queries to the graph database, `Arcade::Query` supports traversal of nodes
-
-Create a Chain of Nodes:
 ```ruby
-
-  def linear_elements start, count  #returns the edge created
-      new_vertex = ->(n) { Arcade::ExtraNode.create( note_count: n )  }
-      (2..count).each{ |n| start = start.assign vertex: new_vertex[n], via: Arcade::Connects  }
-  end
-   
-  start_node =  Arcade::ExtraNode.create( item: 'linear'  )
-  linear_elements start_node , 200
+Arcade::Api.query(db, "select from x")            # parsed :result (array)
+Arcade::Api.execute(db) { "update x set ..." }    # write, retry on 'Please retry'
+Arcade::Api.get_json("ts/#{db}/latest?type=...")  # raw body (ts endpoints)
+Arcade::Api.post_text("ts/#{db}/write?precision=s", lines)  # Line Protocol
+db.query("select ...").allocate_model             # hashes -> model objects
 ```
 
-Select a range of nodes and perform a mathematical operation
+Query payload hash: `{ query:, language: :sql|:cypher|:gremlin|:neo4j, params:, limit:, serializer: :graph|:record }`.
 
-```ruby 
-  hundred_elements =  start_node.traverse :out, via: Arcade::Connects, depth: 100
-  median = Query.new from: hundred_elements,
-               projection: 'median(note_count)',
-                    where: '$depth>=50'
-  median.to_s
-  -=> select median(note_count) from  ( traverse out(connects) from #52:0 while $depth < 100 )  where $depth>=50  
-   =>   {:"median(note_count)"=>75.5 }
-```
+## Server & version notes (verified ArcadeDB 26.10)
 
-### Transactions
+| Observation | Consequence |
+|---|---|
+| `schema:types` does not list timeseries types | existence probe: `POST /ts/{db}/query` → 400 `unknownTypes`/`does not exist` = absent (`type_exists?` uses this) |
+| `CREATE TIMESERIES TYPE` registers asynchronously (~1 s) | `create_type` waits for visibility; a write right after create fails with `unknownTypes` otherwise |
+| `DELETE FROM <ts-type> WHERE ...` reports a count but keeps rows (26.10.1) | no window-replace possible; re-writing a timestamp **appends** — deduplicate on read |
+| timeseries timestamp precision defaults to NANOSECOND | declare `PRECISION` explicitly; numeric literals are interpreted per precision |
+| `bucket` is a reserved word in SQL | aliased in backticks by `bucket()` |
+| `IF NOT EXISTS` unsupported on `CREATE TIMESERIES TYPE` | guarded by `type_exists?` |
+| Line Protocol needs pre-created types (`tsAutoCreateType` off by default) | `write_many` fails with `unknownTypes` if the type is missing |
+| persistent HTTP connections die after ~3 s idle | Primitives renew the handle after 2 s inactivity |
 
-Database-Transactions are largely encapsulated
+## Testing
 
+```bash
+bundle install && bundle exec rspec                  # needs a reachable ArcadeDB server
+bundle exec rspec spec/lib/timeseries_spec.rb        # single file; spec/lib/* against arcade.yml :test db
 ```
-db =  Arcade::Init.db
-db.begin_transaction
-
-{ perform insert, update, query, etc tasks in the database }
 
-db.commit 
+Model files for specs live in `spec/model/` (Zeitwerk-loaded). The spec suite is the
+authoritative behaviour reference — read `spec/lib/*` before assuming API shape.
 
-# or
-
-db.rollback
+## Repository layout
 
 ```
-
+lib/arcade.rb              entry point (requires below)
+lib/arcade/                base, database, query, match, timeseries, api/, support/
+lib/model/                 base classes (vertex, edge, document, revision, timeseries)
+lib/models.rb              base-class loader
+examples/                  documented use cases
+spec/lib, spec/model       rspec suite (authoritative)
+bin/console                irb shortcut per environment
+```
 
 ## Contributing
 
-Bug reports and pull requests are welcome. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
-
-## Code of Conduct
-
-Everyone interacting in the Core project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/topofocus/arcadedb/blob/master/CODE_OF_CONDUCT.md).
+Bug reports and pull requests are welcome — this project follows the
+[Contributor Covenant](http://contributor-covenant.org) code of conduct
+([detail](https://github.com/topofocus/arcadedb/blob/master/CODE_OF_CONDUCT.md)).
+New behaviour ships with a spec in `spec/lib/` and, if server-quirk-sensitive, a row
+in the Server & version notes table.

data/arcade.yml

@@ -3,20 +3,20 @@
   :test:
     dbname: playground
     user:  root
-    pass: *** 
+    pass:  topo1focus
   :development:
     dbname: devel
     user: root
-    pass:  ***
+    pass:  topo1focus
   :production:
     dbname: production
     user: root
-    pass: ***
+    pass: topo1focus
 :admin:
-  :host: 10.247.8.109
+  :host: localhost
   :port: 2480
   :user: root
-  :pass: ***
+  :pass: topo1focus
 :logger: stdout           # 'file' or 'stdout'
 :namespace: Arcade        # Default Namespace
 :autoload: true           # load model if a link is detected in a record

data/bin/console

@@ -9,6 +9,7 @@
 require 'bundler/setup'
 #require 'terminal-table'
 require 'zeitwerk'
+ProjectRoot =  Pathname.new( Dir.pwd ).parent
 require 'pastel'
 require 'arcade'
 #require 'pry'

data/examples/rag_example.rb

@@ -0,0 +1,72 @@
+# RAG (Retrieval-Augmented Generation) Example
+# 
+# This example demonstrates how to use the Arcade::RAGSchema module
+# to create a RAG-compatible document store with vector embeddings.
+#
+# Prerequisites:
+# - A running ArcadeDB server with vector search support
+# - An embedding service (e.g., OpenAI, HuggingFace) to generate embeddings
+#
+# Setup:
+#   1. Create a model that includes Arcade::RAGSchema
+#   2. Call create_type to set up the database schema
+#   3. Insert documents with their embeddings
+#   4. Perform vector or hybrid search
+
+require 'bundler/setup'
+require 'arcade'
+
+# Example model definition
+module RAG
+  class Document < Arcade::Vertex
+    include Arcade::RAGSchema
+    
+    attribute :content, Types::String
+    attribute :embedding?, Types::Array.of(Types::Float).optional
+    attribute :metadata?, Types::Hash.optional
+    attribute :source_url?, Types::String.optional
+    attribute :created_at?, Types::JSON::DateTime.optional
+
+    # Override db_init to use RAG schema
+    def self.db_init
+      rag_db_init(vector_dimension: 1536, full_text_fields: ['content', 'metadata'])
+    end
+  end
+end
+
+if $0 == __FILE__
+  # Connect to database
+  Arcade::Init.connect('test')
+
+  # Create the type with RAG schema
+  RAG::Document.create_type
+
+  # Example: Insert a document with embedding
+  # In practice, you would generate the embedding using an external service
+  sample_embedding = Array.new(1536) { rand }  # Replace with actual embedding
+  
+  doc = RAG::Document.create(
+    content: "ArcadeDB is a multi-model graph database built from the ground up.",
+    embedding: sample_embedding,
+    metadata: { author: "ArcadeDB Team", tags: ["database", "graph"] },
+    source_url: "https://arcadedb.com",
+    created_at: DateTime.now
+  )
+
+  puts "Created document: #{doc.rid}"
+
+  # Example: Vector similarity search
+  # query_embedding = generate_embedding("What is a graph database?")
+  # results = RAG::Document.vector_search(query_embedding, limit: 5, threshold: 0.7)
+  # puts "Found #{results.count} similar documents"
+
+  # Example: Hybrid search (vector + full-text)
+  # results = RAG::Document.hybrid_search(
+  #   query_embedding, 
+  #   "multi-model graph database", 
+  #   limit: 5
+  # )
+  # puts "Found #{results.count} hybrid search results"
+
+  puts "RAG schema setup complete!"
+end