datahike-browser-tests 1.0.0

package/doc/datalog-vs-sql.md

@@ -0,0 +1,210 @@
+# Why Datalog?
+
+Datalog is a declarative query language that excels at expressing complex relationships in graph-structured data. While SQL handles tabular data well, modern applications increasingly need to model interconnected entities—social networks, organizational hierarchies, supply chains, knowledge graphs.
+
+## When SQL Falls Short
+
+**The problem**: As applications evolve, relational schemas accumulate join complexity. What starts as simple tables becomes a maze of foreign keys, junction tables, and nested subqueries. Developers spend more time managing joins than expressing business logic.
+
+**Common pattern**: Start with SQL for simplicity → encounter complex relationships → add more joins → build ad-hoc graph features → end up with fragile, hard-to-maintain queries.
+
+**Datalog's approach**: Pattern matching over relationships. Instead of explicitly specifying how to join tables, you describe what patterns you're looking for. The query engine handles traversal.
+
+## Query Comparison
+
+Let's find all projects where someone manages a team member who contributed to that project.
+
+### SQL Approach
+
+```sql
+SELECT DISTINCT
+    m.name AS manager_name,
+    e.name AS employee_name,
+    p.name AS project_name
+FROM employees m
+JOIN manager_relationships mr ON m.id = mr.manager_id
+JOIN employees e ON mr.employee_id = e.id
+JOIN project_contributors pc ON e.id = pc.employee_id
+JOIN projects p ON pc.project_id = p.id
+WHERE m.id != e.id;
+```
+
+This requires:
+- Understanding the table structure (4 tables, 3 join conditions)
+- Explicit join order management
+- Mental model of how data flows through joins
+
+### Datalog Approach
+
+```clojure
+(d/q '[:find ?manager-name ?employee-name ?project-name
+       :where
+       [?manager :employee/manages ?employee]
+       [?employee :project/contributed-to ?project]
+       [?manager :employee/name ?manager-name]
+       [?employee :employee/name ?employee-name]
+       [?project :project/name ?project-name]]
+     db)
+```
+
+Pattern matching expresses intent:
+- "Find managers who manage employees"
+- "Find employees who contributed to projects"
+- "Get their names"
+
+No explicit joins. No table knowledge required. The relationships are first-class.
+
+## More Complex: Transitive Relationships
+
+Find all skills accessible through management chain (managers inherit team skills).
+
+### SQL
+
+```sql
+WITH RECURSIVE management_chain AS (
+    -- Base case: direct reports
+    SELECT manager_id, employee_id, 1 as depth
+    FROM manager_relationships
+
+    UNION ALL
+
+    -- Recursive case: indirect reports
+    SELECT mc.manager_id, mr.employee_id, mc.depth + 1
+    FROM management_chain mc
+    JOIN manager_relationships mr ON mc.employee_id = mr.manager_id
+    WHERE mc.depth < 10  -- Prevent infinite recursion
+)
+SELECT DISTINCT
+    m.name AS manager_name,
+    s.name AS accessible_skill
+FROM employees m
+JOIN management_chain mc ON m.id = mc.manager_id
+JOIN employee_skills es ON mc.employee_id = es.employee_id
+JOIN skills s ON es.skill_id = s.id;
+```
+
+Requires:
+- Understanding recursive CTEs
+- Managing recursion depth manually
+- Multiple joins to assemble result
+
+### Datalog with Rules
+
+```clojure
+;; Define a rule for transitive management
+(d/q '[:find ?manager-name ?skill-name
+       :in $ %
+       :where
+       (manages-recursively ?manager ?employee)
+       [?employee :employee/has-skill ?skill]
+       [?manager :employee/name ?manager-name]
+       [?skill :skill/name ?skill-name]]
+     db
+     '[;; Rule definition
+       [(manages-recursively ?m ?e)
+        [?m :employee/manages ?e]]
+       [(manages-recursively ?m ?e)
+        [?m :employee/manages ?x]
+        (manages-recursively ?x ?e)]])
+```
+
+**Rules encapsulate logic**: `manages-recursively` handles the transitive relationship. Reusable across queries.
+
+## Graph Traversal: Friend Recommendations
+
+Find friends-of-friends who share interests, excluding existing friends.
+
+### SQL
+
+```sql
+SELECT DISTINCT
+    p1.name AS person_name,
+    p3.name AS recommendation_name,
+    i.name AS shared_interest
+FROM people p1
+JOIN friendships f1 ON p1.id = f1.person_id
+JOIN people p2 ON f1.friend_id = p2.id
+JOIN friendships f2 ON p2.id = f2.person_id
+JOIN people p3 ON f2.friend_id = p3.id
+JOIN person_interests pi1 ON p1.id = pi1.person_id
+JOIN person_interests pi3 ON p3.id = pi3.person_id
+JOIN interests i ON pi1.interest_id = i.id AND pi3.interest_id = i.id
+LEFT JOIN friendships f_check ON (p1.id = f_check.person_id AND p3.id = f_check.friend_id)
+WHERE p1.id != p3.id
+  AND f_check.person_id IS NULL;  -- Not already friends
+```
+
+8 joins, complex exclusion logic, hard to maintain.
+
+### Datalog
+
+```clojure
+(d/q '[:find ?person-name ?recommendation-name ?interest-name
+       :where
+       [?person :person/friend ?friend]
+       [?friend :person/friend ?recommendation]
+       [?person :person/interest ?interest]
+       [?recommendation :person/interest ?interest]
+       [(not= ?person ?recommendation)]
+       (not [?person :person/friend ?recommendation])
+       [?person :person/name ?person-name]
+       [?recommendation :person/name ?recommendation-name]
+       [?interest :interest/name ?interest-name]]
+     db)
+```
+
+**Natural expression**: Each line describes one relationship pattern. Negation uses `not` clause rather than LEFT JOIN tricks.
+
+## Multi-Database Queries
+
+Datalog can query across multiple databases in a single query—useful for data federation or comparing environments.
+
+```clojure
+;; Compare production and staging: Find entities in staging not in production
+(d/q '[:find ?name ?value
+       :in $staging $prod
+       :where
+       [$staging ?e :entity/name ?name]
+       [$staging ?e :entity/value ?value]
+       (not [$prod ?e2 :entity/name ?name])]
+     staging-db prod-db)
+```
+
+SQL would require UNION queries, temporary tables, or separate queries with application-level comparison.
+
+## When to Use Datalog
+
+**Choose Datalog when:**
+- Modeling interconnected entities (social graphs, org charts, supply chains)
+- Need recursive or transitive queries
+- Schema evolves frequently (schema-less or flexible schema)
+- Want to express logic through rules
+- Query across multiple databases or time points
+- Building knowledge graphs or semantic systems
+
+**SQL still wins for:**
+- Pure tabular data with minimal relationships
+- Standard BI tool integration
+- Window functions and complex aggregations over large sorted sets
+- Existing SQL infrastructure and expertise
+
+## Performance Note
+
+Datalog query engines (including Datahike) use sophisticated join optimization. Performance is comparable to well-tuned SQL for most queries. For specific workloads, see our [benchmarks](https://gitlab.com/arbetsformedlingen/taxonomy-dev/backend/experimental/datahike-benchmark/).
+
+## Learning Datalog
+
+Datalog syntax is minimal:
+- Pattern matching: `[?entity :attribute ?value]`
+- Rules: Define reusable logic patterns
+- Negation: `(not ...)`
+- Functions: `[(function arg) result]`
+
+Most developers become productive within hours. The conceptual shift from imperative joins to declarative patterns is the main learning curve, not syntax complexity.
+
+## Further Reading
+
+- [Datalog Wikipedia](https://en.wikipedia.org/wiki/Datalog) - Academic background
+- [Learn Datalog Today](http://www.learndatalogtoday.org/) - Interactive tutorial
+- [Datomic Query Documentation](https://docs.datomic.com/query/query.html) - Datahike uses compatible query syntax
+- [Datahike Query Guide](https://cljdoc.org/d/org.replikativ/datahike/CURRENT/doc/query) - Full query capabilities

package/doc/datomic_differences.md

@@ -0,0 +1,109 @@
+# Differences to Datomic Client API
+
+Although Datahike supports a part of [Datomic's
+API](https://docs.datomic.com/client-api/datomic.client.api.html), some
+behavior is different using the different functions. Datahike supports part of the `datomic.client.api`:
+
+[![cljdoc](https://badgen.net/badge/cljdoc/datahike/blue)](https://cljdoc.org/d/org.replikativ/datahike)
+
+Without differences in the signature you may use the following like in Datomic:
+
+- as-of
+- datoms
+- [db](#db)
+- history
+- pull
+- q
+- since
+- [transact](#transact)
+
+Please be aware that `q` returns sets instead of vectors. Only when using aggregates or `:with`, `q` returns a
+vector. That is the behavior of Datascript.
+
+The following functions from the `datahike.api` namespace are different from Datomic's client API:
+
+- [connect](#connect)
+- [create-database](#create-database)
+- [database-exists?](#database-exists)
+- [db-with](https://cljdoc.org/d/org.replikativ/datahike/0.3.6/api/datahike.api#db-with)
+- [delete-database](#delete-database)
+- [entity](https://cljdoc.org/d/org.replikativ/datahike/0.3.6/api/datahike.api#entity)
+- [entity-db](https://cljdoc.org/d/org.replikativ/datahike/0.3.6/api/datahike.api#entity-db)
+- [filter](https://cljdoc.org/d/org.replikativ/datahike/0.3.6/api/datahike.api#filter)
+- [is-filtered](https://cljdoc.org/d/org.replikativ/datahike/0.3.6/api/datahike.api#is-filtered)
+- [load-entities](https://cljdoc.org/d/org.replikativ/datahike/0.3.6/api/datahike.api#load-entities)
+- [pull-many](https://cljdoc.org/d/org.replikativ/datahike/0.3.6/api/datahike.api#pull-many)
+- [release](https://cljdoc.org/d/org.replikativ/datahike/0.3.6/api/datahike.api#release)
+- [seek-datoms](https://cljdoc.org/d/org.replikativ/datahike/0.3.6/api/datahike.api#seek-datoms)
+- [tempid](https://cljdoc.org/d/org.replikativ/datahike/0.3.6/api/datahike.api#tempid)
+- [with](#with)
+
+Additionally, datahike supports most functions from [datascript](https://github.com/tonsky/datascript)
+in the `datahike.api` namespace.
+
+These functions of Datomic are not yet implemented but considered candidates for future development:
+- tx-range
+- index-pull
+- with-db
+
+These functions are part of Datomics' distributed implementation and are currently not part of the
+Datahike API:
+- client
+- administer-system
+- db-stats
+- list-databases
+- sync
+
+Async support is on our roadmap as well as running distributed Datahike.
+
+## connect
+
+Connects to an existing database given the configuration hash-map where Datomic
+takes a client as argument. The specification for the configuration can be found
+[here](./config.md).
+
+[![cljdoc](https://badgen.net/badge/cljdoc/datahike/blue)](https://cljdoc.org/d/org.replikativ/datahike)
+
+## create-database
+
+Creates a new database with the given configuration hash-map where Datomic takes a client and an
+arg-map as arguments. Additional optional parameters are `schema-flexibility`, `keep-history?`
+and `initial-tx`. Have a look at the [configuration documentation](./config.md) for details.
+
+[![cljdoc](https://badgen.net/badge/cljdoc/datahike/blue)](https://cljdoc.org/d/org.replikativ/datahike)
+
+## delete-database
+
+Deletes a database with the configuration hash-map as argument.
+
+## database-exists?
+
+Checks if a database exists with the configuration hash-map as argument.
+
+## db
+
+Since the database can be just de-referenced from the connection this function is
+just a small wrapper for Datomic compliance.
+
+```clojure
+(db conn)
+```
+equivalent to:
+```
+@conn
+```
+
+## transact
+
+Returns a hash map different from Datomics' as a report. The `:tx-meta` is not part
+of Datomics' transaction report but apart from that the same keys are present. The
+values are different records though.
+
+[![cljdoc](https://badgen.net/badge/cljdoc/datahike/blue)](https://cljdoc.org/d/org.replikativ/datahike)
+
+## with
+
+Applies transactions to an immutable database value and returns a new database
+snapshot. It does not change the mutable database inside the connection unlike
+[transact](#transact). It works quite the same as Datomics' `with` but does
+not need a `with-db` function to work.

package/doc/development/pull-api-ns.md

@@ -0,0 +1,186 @@
+# Pull-API Namespace 
+
+This document is aimed at Datahike core developers that want to debug or extend the Pull-API.
+Hopefully, this will help people getting a grip of the existing code and work flow of the Pull-API.
+
+## Terminology 
+
+### Selector
+
+A quoted vector holding one or several of the following expressions:
+1) wildcard
+   - pulls all datoms for entity id
+   - e.g. `*` -> `{:db/id 1 :name "Petr" :child [{:db/id 2}]}`
+2) keyword for
+   - attributes to get
+     - e.g. `:child` (= has child) -> `{:child [#:db{:id 10}]}`
+   - reverse attributes to get (datom where requested entity is referenced)
+     - signaled by underscore
+     - e.g. `:_child` (= child of) -> `{:_child [#:db{:id 1}]}`
+3) map of keywords for (reverse) attributes to get and
+   - vectors of attributes
+     - attribute should be reference
+     - e.g. `{:child [:name :age]}` -> `{:child {:name "Thomas" :age 10}}`
+     - e.g. `{:_child [:name :age]}` -> `{:_child {:name "Charles" :age 10}}`
+   - three dots
+     - for the infinite recursion on an attribute
+   - number
+     - for the finite recursion on an attribute
+4) function call form:
+   - limit:
+     - restricts the amount of pulled datoms to the given number
+     - e.g. `(limit :child 2)`
+   - defaults:
+     - sets a default value if search returns nil
+     - e.g. `(default :name "unknown")`
+5) vectors of 3
+   - functions of 4) in vector form
+     - attribute, function name as keyword, parameter
+     - e.g. `[:child :limit 2]` or `[:name :default "unknown"]`
+   - renaming of attributes with :as
+     - attribute, :as, new name
+     - e.g. `[:aka :as :alias]` or `[:name :as "Name"]`
+
+### Pattern = (Pull-)Spec
+
+Record with keys
+- `:wildcard?` = boolean value indicating if wildcard expansion has to be done
+- `:attrs` = map of attribute names as in selector and map with keys
+  - `:attr` = attribute name as in database; only different from key when key describes reverse attribute
+  - `:subpattern` (optional) = spec of a subpattern applied to an attribute
+  - `:recursion` = always nil, only checks if key exists, if it does: recursion is applied on attribute
+  - `:limit`= amount of maximum number of results returned
+  - `:default` = default value if nothing found
+
+
+### Frame
+
+Map with keys
+  - `:multi?` = true if multiple entities requested
+  - `:eids` = vector of eids to pull pattern for
+  - `:state` = pattern/expand/expand-rev/recursion/done
+  - `:pattern` = (Pull-)Spec
+  - `:recursion` = map with keys :depth (depth per attribute) and :seen (eids)
+  - `:specs` = attrs from spec
+  - `:wildcard?` = wildcard from spec
+  - `:kvps` = final result map; in pull-pattern transfered to next frame when current frame is done
+  - `:results` = result map for current frame, i.e. for current attribute
+  - `:attr` = optional; current attribute (pull-attr-datoms) or enum, e.g. recursion
+  - `:datoms` = optional; datoms pulled from database; e.g. added by wildcard-expand
+  - `:expand-kvps` = optional; added by expand-frame, used by expand-rev-frame
+
+#### Frame Creation and State Changes
+
+- only 1 frame created for basic input pattern
+- additional frames added on processing of
+  - recursion: 2 frames per depth
+  - wildcard: 1 frame
+  
+![Frame State Diagram](https://raw.githubusercontent.com/replikativ/datahike/documentation/doc/development/pull-frame-state-diagram.jpg)
+
+## Pull API Usage
+
+See also the official [Datahike documentation](https://cljdoc.org/d/org.replikativ/datahike/0.3.2/api/datahike.api#pull).
+
+Setup for examples:
+
+```clojure
+(require '(datahike [core :as d]))
+
+(def schema
+  {:aka    {:db/cardinality :db.cardinality/many}
+   :child  {:db/cardinality :db.cardinality/many
+            :db/valueType :db.type/ref}
+   :friend {:db/cardinality :db.cardinality/many
+            :db/valueType :db.type/ref}
+   :father {:db/valueType :db.type/ref}
+   :spec   {:db/valueType :db.type/ref
+            :db/isComponent true
+            :db/cardinality :db.cardinality/one}})
+
+(def datoms
+  (->>
+   [[1 :name  "Petr"]
+    [1 :aka   "Devil"]
+    [1 :aka   "Tupen"]
+    [1 :aka   "P"]
+    [2 :name  "David"]
+    [3 :name  "Thomas"]
+    [4 :name  "Lucy"]
+    [5 :name  "Elizabeth"]
+    [6 :name  "Matthew"]
+    [7 :name  "Eunan"]
+    [8 :name  "Kerri"]
+    [9 :name  "Rebecca"]
+    [1 :child 2]
+    [1 :child 3]
+    [2 :father 1]
+    [3 :father 1]
+    [6 :father 3]
+    [4 :friend 5]
+    [5 :friend 6]]
+   (map (fn [[e a v]] (d/datom e a v tx0)))))
+
+(def example-db (d/init-db datoms schema))
+```
+
+### Examples
+
+
+Simple Pull
+```clojure
+(d/pull example-db '[:name] 1)
+
+;; => {:name "Petr"}
+```
+
+Pull reverse attribute
+```clojure
+(d/pull example-db '[:_child] 2)
+
+;; => {:_child [#:db{:id 1}]}
+```
+
+Pull db/id
+
+```clojure
+(d/pull example-db '[:name :db/id] 6)
+
+;; => {:name "Matthew", :db/id 6}
+```
+
+Pull wildcard
+```clojure
+(d/pull example-db '[*] 2)
+
+;; => {:db/id 2, :father #:db{:id 1}, :name "David"}
+```
+
+Pull recursion
+```clojure
+(d/pull example-db '[:db/id :name {:friend ...}] 4)
+
+;; => {:db/id 4, :name "Lucy", :friend [{:db/id 5, :name "Elizabeth", :friend [{:db/id 6, :name "Matthew"}]}]}
+```
+
+Pull with default 
+```clojure
+(d/pull example-db '[(default :foo "bar")] 1)
+
+;; => {:foo "bar"}
+```
+
+Pull with limit
+```clojure
+(d/pull example-db '[(limit :aka 2)] 1)
+
+;; => {:aka ["Devil" "Tupen"]}
+```
+
+Pull with subpattern
+```clojure
+(d/pull example-db '[{:father [:name]}] 6) 
+
+;; => {:father {:name "Thomas"}}
+```
+