bkmr 2.0.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- bkmr-2.0.1/PKG-INFO +167 -0
- bkmr-2.0.1/README.md +151 -0
- bkmr-2.0.1/bkmr/Cargo.lock +4003 -0
- bkmr-2.0.1/bkmr/Cargo.toml +62 -0
- bkmr-2.0.1/bkmr/diesel.toml +8 -0
- bkmr-2.0.1/bkmr/migrations/.keep +0 -0
- bkmr-2.0.1/bkmr/migrations/2022-12-29-110455_create_bookmarks/down.sql +3 -0
- bkmr-2.0.1/bkmr/migrations/2022-12-29-110455_create_bookmarks/up.sql +104 -0
- bkmr-2.0.1/bkmr/migrations/2023-12-17-200409_add_embedding_and_content_hash_to_bookmarks/down.sql +3 -0
- bkmr-2.0.1/bkmr/migrations/2023-12-17-200409_add_embedding_and_content_hash_to_bookmarks/up.sql +3 -0
- bkmr-2.0.1/bkmr/migrations/README.md +5 -0
- bkmr-2.0.1/bkmr/src/adapter/dal.rs +361 -0
- bkmr-2.0.1/bkmr/src/adapter/embeddings.rs +279 -0
- bkmr-2.0.1/bkmr/src/adapter/json.rs +207 -0
- bkmr-2.0.1/bkmr/src/adapter/schema.rs +27 -0
- bkmr-2.0.1/bkmr/src/environment.rs +88 -0
- bkmr-2.0.1/bkmr/src/helper.rs +252 -0
- bkmr-2.0.1/bkmr/src/lib.rs +175 -0
- bkmr-2.0.1/bkmr/src/macros.rs +35 -0
- bkmr-2.0.1/bkmr/src/main.rs +1052 -0
- bkmr-2.0.1/bkmr/src/model/bms.rs +133 -0
- bkmr-2.0.1/bkmr/src/model/bookmark.rs +371 -0
- bkmr-2.0.1/bkmr/src/model/tag.rs +202 -0
- bkmr-2.0.1/bkmr/src/service/embeddings.rs +62 -0
- bkmr-2.0.1/bkmr/src/service/fzf.rs +250 -0
- bkmr-2.0.1/bkmr/src/service/process.rs +670 -0
- bkmr-2.0.1/bkmr/tests/adapter/test_json.rs +32 -0
- bkmr-2.0.1/bkmr/tests/resources/bkmr.pptx +0 -0
- bkmr-2.0.1/bkmr/tests/resources/bkmr.v1.db +0 -0
- bkmr-2.0.1/bkmr/tests/resources/bkmr.v2.db +0 -0
- bkmr-2.0.1/bkmr/tests/resources/bkmr.v2.noembed.db +0 -0
- bkmr-2.0.1/bkmr/tests/resources/data.ndjson +3 -0
- bkmr-2.0.1/bkmr/tests/resources/invalid_data.ndjson +3 -0
- bkmr-2.0.1/bkmr/tests/resources/sample_docu.md +9 -0
- bkmr-2.0.1/bkmr/tests/service/test_embeddings.rs +66 -0
- bkmr-2.0.1/bkmr/tests/test_bms.rs +81 -0
- bkmr-2.0.1/bkmr/tests/test_dal.rs +265 -0
- bkmr-2.0.1/bkmr/tests/test_embeddings.rs +54 -0
- bkmr-2.0.1/bkmr/tests/test_fzf.rs +48 -0
- bkmr-2.0.1/bkmr/tests/test_lib.rs +122 -0
- bkmr-2.0.1/bkmr/tests/test_main.rs +37 -0
- bkmr-2.0.1/bkmr/tests/test_models.rs +87 -0
- bkmr-2.0.1/bkmr/tests/test_process.rs +66 -0
- bkmr-2.0.1/pyproject.toml +72 -0
bkmr-2.0.1/PKG-INFO
ADDED
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
Metadata-Version: 2.3
|
|
2
|
+
Name: bkmr
|
|
3
|
+
Version: 2.0.1
|
|
4
|
+
Classifier: Programming Language :: Rust
|
|
5
|
+
Classifier: Programming Language :: Python :: Implementation :: CPython
|
|
6
|
+
License-File: LICENSE
|
|
7
|
+
License-File: AUTHORS
|
|
8
|
+
Summary: Super fast bookmark manager with semantic full text search'
|
|
9
|
+
Keywords: bookmark,launcher
|
|
10
|
+
Author-email: sysid <sysid@gmx.de>
|
|
11
|
+
License: BSD-3-Clause
|
|
12
|
+
Requires-Python: >=3.10
|
|
13
|
+
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
|
|
14
|
+
Project-URL: Source Code, https://github.com/sysid/bkmr
|
|
15
|
+
|
|
16
|
+
# bkmr
|
|
17
|
+
|
|
18
|
+

|
|
19
|
+

|
|
20
|
+
[](https://docs.rs/bkmr)
|
|
21
|
+
|
|
22
|
+
[![PyPI Version][pypi-image]][pypi-url]
|
|
23
|
+
[](https://pepy.tech/project/bkmr)
|
|
24
|
+
[![Build Status][build-image]][build-url]
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
### [Generalized Semantic Search](https://github.com/sysid/bkmr/wiki/Semantic-Search)
|
|
28
|
+
|
|
29
|
+
# Ultrafast Bookmark Manager and Launcher
|
|
30
|
+
|
|
31
|
+
> New Feature: Semantic Search (AI Embeddings)
|
|
32
|
+
|
|
33
|
+
[Elevating Bookmark Management with AI-Driven Semantic Search](https://sysid.github.io/elevating-bookmark-management-with-ai-driven-semantic-search/)
|
|
34
|
+
|
|
35
|
+
Features:
|
|
36
|
+
- semantic search using OpenAI embeddings (requires OpenAI API key)
|
|
37
|
+
- full-text search with semantic ranking (FTS5)
|
|
38
|
+
- fuzzy search `--fzf` (CTRL-O: copy to clipboard, CTRL-E: edit, CTRL-D: delete, Enter: open)
|
|
39
|
+
- tags for classification
|
|
40
|
+
- can handle HTTP URLs, directories, files (e.g. Office, Images, ....)
|
|
41
|
+
- can execute URI strings as shell commands via protocol prefix: 'shell::'
|
|
42
|
+
URI-Example: `shell::vim +/"## SqlAlchemy" $HOME/document.md`
|
|
43
|
+
- automatically enriches URLs with title and description from Web
|
|
44
|
+
- manages statistics about bookmark usage
|
|
45
|
+
|
|
46
|
+
**`bkmr search --fzf` is a great way to open bookmarks very fast.**
|
|
47
|
+
|
|
48
|
+
## Usage
|
|
49
|
+
```bash
|
|
50
|
+
bkmr --help
|
|
51
|
+
|
|
52
|
+
A Bookmark Manager and Launcher for the Terminal
|
|
53
|
+
|
|
54
|
+
Usage: bkmr [OPTIONS] [NAME] [COMMAND]
|
|
55
|
+
|
|
56
|
+
Commands:
|
|
57
|
+
search Searches Bookmarks
|
|
58
|
+
sem-search Semantic Search with OpenAI
|
|
59
|
+
open Open/launch bookmarks
|
|
60
|
+
add Add a bookmark
|
|
61
|
+
delete Delete bookmarks
|
|
62
|
+
update Update bookmarks
|
|
63
|
+
edit Edit bookmarks
|
|
64
|
+
show Show Bookmarks (list of ids, separated by comma, no blanks)
|
|
65
|
+
surprise Opens n random URLs
|
|
66
|
+
tags Tag for which related tags should be shown. No input: all tags are printed
|
|
67
|
+
create-db Initialize bookmark database
|
|
68
|
+
backfill Backfill embeddings for bookmarks
|
|
69
|
+
load-texts Load texts for semantic similarity search
|
|
70
|
+
help Print this message or the help of the given subcommand(s)
|
|
71
|
+
|
|
72
|
+
Arguments:
|
|
73
|
+
[NAME] Optional name to operate on
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
<a href="https://asciinema.org/a/ULCDIrw4pG9diaVJb17AjIAa7?autoplay=1&speed=2"><img src="https://asciinema.org/a/ULCDIrw4pG9diaVJb17AjIAa7.png" width="836"/></a>
|
|
77
|
+
|
|
78
|
+
### Examples
|
|
79
|
+
```bash
|
|
80
|
+
# FTS examples (https://www.sqlite.org/fts5.htm)
|
|
81
|
+
bkmr search '"https://securit" *'
|
|
82
|
+
bkmr search 'security NOT keycloak'
|
|
83
|
+
|
|
84
|
+
# FTS combined with tag filtering
|
|
85
|
+
bkmr search -t tag1,tag2 -n notag1 <searchquery>
|
|
86
|
+
|
|
87
|
+
# Search by any tag and sort by bookmark age ascending
|
|
88
|
+
bkmr search -T tag1,tag2 -O
|
|
89
|
+
|
|
90
|
+
# Give me the 10 oldest bookmarks
|
|
91
|
+
bkmr search -O --limit 10
|
|
92
|
+
|
|
93
|
+
# Adding URI to local files
|
|
94
|
+
bkmr add /home/user/presentation.pptx tag1,tag2 --title 'My super Presentation'
|
|
95
|
+
|
|
96
|
+
# Adding shell commands as URI
|
|
97
|
+
bkmr add "shell::vim +/'# SqlAlchemy' sql.md" shell,sql,doc --title 'sqlalchemy snippets'
|
|
98
|
+
|
|
99
|
+
# JSON dump of entire database
|
|
100
|
+
bkmr search --json
|
|
101
|
+
|
|
102
|
+
# Semantic Search based on OpenAI Embeddings
|
|
103
|
+
bkmr --openai sem-search "python security" # requires OPENAI_API_KEY
|
|
104
|
+
```
|
|
105
|
+
Tags must be separated by comma without blanks.
|
|
106
|
+
|
|
107
|
+
## Installation
|
|
108
|
+
1. `cargo install bkmr`
|
|
109
|
+
2. initialize the database: `bkmr create-db db_path`
|
|
110
|
+
3. `export "BKMR_DB_URL=db-path"`, location of created sqlite database must be known
|
|
111
|
+
4. add URLs
|
|
112
|
+
|
|
113
|
+
More configuration options can be found at [documentation page](https://github.com/sysid/bkmr/wiki/configuration).
|
|
114
|
+
|
|
115
|
+
### Upgrade to 1.x.x
|
|
116
|
+
A database migration will be performed on the first run of the new version.
|
|
117
|
+
This will add two columns to the bookmarks table for the OpenAI embeddings.
|
|
118
|
+
No destructive changes are made to the database.
|
|
119
|
+
|
|
120
|
+
## Semantic Search
|
|
121
|
+
`bkmr` provides now full semantic search of generalized bookmarks using OpenAI's Embeddings.
|
|
122
|
+
|
|
123
|
+
You can find more information on the [documentation page](https://github.com/sysid/bkmr/wiki/semantic-search).
|
|
124
|
+
|
|
125
|
+
## Benchmarking
|
|
126
|
+
- ca. 20x faster than the Python original [twbm](https://github.com/sysid/twbm) after warming up Python.
|
|
127
|
+
```bash
|
|
128
|
+
time twbm search 'zzz*' --np
|
|
129
|
+
0. zzzeek : Asynchronous Python and Databases [343]
|
|
130
|
+
https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/
|
|
131
|
+
async, knowhow, py
|
|
132
|
+
|
|
133
|
+
|
|
134
|
+
Found: 1
|
|
135
|
+
343
|
|
136
|
+
|
|
137
|
+
real 0m0.501s
|
|
138
|
+
user 0m0.268s
|
|
139
|
+
sys 0m0.070s
|
|
140
|
+
|
|
141
|
+
|
|
142
|
+
|
|
143
|
+
time bkmr search 'zzz*' --np
|
|
144
|
+
1. zzzeek : Asynchronous Python and Databases [343]
|
|
145
|
+
https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/
|
|
146
|
+
async knowhow py
|
|
147
|
+
|
|
148
|
+
|
|
149
|
+
real 0m0.027s
|
|
150
|
+
user 0m0.008s
|
|
151
|
+
sys 0m0.016s
|
|
152
|
+
```
|
|
153
|
+
[sysid blog: bkmr](https://sysid.github.io/bkmr/)
|
|
154
|
+
|
|
155
|
+
|
|
156
|
+
<!-- Badges -->
|
|
157
|
+
[pypi-url]: https://pypi.org/project/bkmr/
|
|
158
|
+
|
|
159
|
+
[build-image]: https://github.com/sysid/bkmr/actions/workflows/release_wheels.yml/badge.svg
|
|
160
|
+
[build-url]: https://github.com/sysid/bkmr/actions/workflows/release_wheels.yml
|
|
161
|
+
|
|
162
|
+
[quality-image]: https://api.codeclimate.com/v1/badges/3130fa0ba3b7993fbf0a/maintainability
|
|
163
|
+
[quality-url]: https://codeclimate.com/github/nalgeon/podsearch-py
|
|
164
|
+
|
|
165
|
+
[pypi-image]: https://badge.fury.io/py/bkmr.svg
|
|
166
|
+
[pypi-url]: https://pypi.org/project/bkmr/
|
|
167
|
+
|
bkmr-2.0.1/README.md
ADDED
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
# bkmr
|
|
2
|
+
|
|
3
|
+

|
|
4
|
+

|
|
5
|
+
[](https://docs.rs/bkmr)
|
|
6
|
+
|
|
7
|
+
[![PyPI Version][pypi-image]][pypi-url]
|
|
8
|
+
[](https://pepy.tech/project/bkmr)
|
|
9
|
+
[![Build Status][build-image]][build-url]
|
|
10
|
+
|
|
11
|
+
|
|
12
|
+
### [Generalized Semantic Search](https://github.com/sysid/bkmr/wiki/Semantic-Search)
|
|
13
|
+
|
|
14
|
+
# Ultrafast Bookmark Manager and Launcher
|
|
15
|
+
|
|
16
|
+
> New Feature: Semantic Search (AI Embeddings)
|
|
17
|
+
|
|
18
|
+
[Elevating Bookmark Management with AI-Driven Semantic Search](https://sysid.github.io/elevating-bookmark-management-with-ai-driven-semantic-search/)
|
|
19
|
+
|
|
20
|
+
Features:
|
|
21
|
+
- semantic search using OpenAI embeddings (requires OpenAI API key)
|
|
22
|
+
- full-text search with semantic ranking (FTS5)
|
|
23
|
+
- fuzzy search `--fzf` (CTRL-O: copy to clipboard, CTRL-E: edit, CTRL-D: delete, Enter: open)
|
|
24
|
+
- tags for classification
|
|
25
|
+
- can handle HTTP URLs, directories, files (e.g. Office, Images, ....)
|
|
26
|
+
- can execute URI strings as shell commands via protocol prefix: 'shell::'
|
|
27
|
+
URI-Example: `shell::vim +/"## SqlAlchemy" $HOME/document.md`
|
|
28
|
+
- automatically enriches URLs with title and description from Web
|
|
29
|
+
- manages statistics about bookmark usage
|
|
30
|
+
|
|
31
|
+
**`bkmr search --fzf` is a great way to open bookmarks very fast.**
|
|
32
|
+
|
|
33
|
+
## Usage
|
|
34
|
+
```bash
|
|
35
|
+
bkmr --help
|
|
36
|
+
|
|
37
|
+
A Bookmark Manager and Launcher for the Terminal
|
|
38
|
+
|
|
39
|
+
Usage: bkmr [OPTIONS] [NAME] [COMMAND]
|
|
40
|
+
|
|
41
|
+
Commands:
|
|
42
|
+
search Searches Bookmarks
|
|
43
|
+
sem-search Semantic Search with OpenAI
|
|
44
|
+
open Open/launch bookmarks
|
|
45
|
+
add Add a bookmark
|
|
46
|
+
delete Delete bookmarks
|
|
47
|
+
update Update bookmarks
|
|
48
|
+
edit Edit bookmarks
|
|
49
|
+
show Show Bookmarks (list of ids, separated by comma, no blanks)
|
|
50
|
+
surprise Opens n random URLs
|
|
51
|
+
tags Tag for which related tags should be shown. No input: all tags are printed
|
|
52
|
+
create-db Initialize bookmark database
|
|
53
|
+
backfill Backfill embeddings for bookmarks
|
|
54
|
+
load-texts Load texts for semantic similarity search
|
|
55
|
+
help Print this message or the help of the given subcommand(s)
|
|
56
|
+
|
|
57
|
+
Arguments:
|
|
58
|
+
[NAME] Optional name to operate on
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
<a href="https://asciinema.org/a/ULCDIrw4pG9diaVJb17AjIAa7?autoplay=1&speed=2"><img src="https://asciinema.org/a/ULCDIrw4pG9diaVJb17AjIAa7.png" width="836"/></a>
|
|
62
|
+
|
|
63
|
+
### Examples
|
|
64
|
+
```bash
|
|
65
|
+
# FTS examples (https://www.sqlite.org/fts5.htm)
|
|
66
|
+
bkmr search '"https://securit" *'
|
|
67
|
+
bkmr search 'security NOT keycloak'
|
|
68
|
+
|
|
69
|
+
# FTS combined with tag filtering
|
|
70
|
+
bkmr search -t tag1,tag2 -n notag1 <searchquery>
|
|
71
|
+
|
|
72
|
+
# Search by any tag and sort by bookmark age ascending
|
|
73
|
+
bkmr search -T tag1,tag2 -O
|
|
74
|
+
|
|
75
|
+
# Give me the 10 oldest bookmarks
|
|
76
|
+
bkmr search -O --limit 10
|
|
77
|
+
|
|
78
|
+
# Adding URI to local files
|
|
79
|
+
bkmr add /home/user/presentation.pptx tag1,tag2 --title 'My super Presentation'
|
|
80
|
+
|
|
81
|
+
# Adding shell commands as URI
|
|
82
|
+
bkmr add "shell::vim +/'# SqlAlchemy' sql.md" shell,sql,doc --title 'sqlalchemy snippets'
|
|
83
|
+
|
|
84
|
+
# JSON dump of entire database
|
|
85
|
+
bkmr search --json
|
|
86
|
+
|
|
87
|
+
# Semantic Search based on OpenAI Embeddings
|
|
88
|
+
bkmr --openai sem-search "python security" # requires OPENAI_API_KEY
|
|
89
|
+
```
|
|
90
|
+
Tags must be separated by comma without blanks.
|
|
91
|
+
|
|
92
|
+
## Installation
|
|
93
|
+
1. `cargo install bkmr`
|
|
94
|
+
2. initialize the database: `bkmr create-db db_path`
|
|
95
|
+
3. `export "BKMR_DB_URL=db-path"`, location of created sqlite database must be known
|
|
96
|
+
4. add URLs
|
|
97
|
+
|
|
98
|
+
More configuration options can be found at [documentation page](https://github.com/sysid/bkmr/wiki/configuration).
|
|
99
|
+
|
|
100
|
+
### Upgrade to 1.x.x
|
|
101
|
+
A database migration will be performed on the first run of the new version.
|
|
102
|
+
This will add two columns to the bookmarks table for the OpenAI embeddings.
|
|
103
|
+
No destructive changes are made to the database.
|
|
104
|
+
|
|
105
|
+
## Semantic Search
|
|
106
|
+
`bkmr` provides now full semantic search of generalized bookmarks using OpenAI's Embeddings.
|
|
107
|
+
|
|
108
|
+
You can find more information on the [documentation page](https://github.com/sysid/bkmr/wiki/semantic-search).
|
|
109
|
+
|
|
110
|
+
## Benchmarking
|
|
111
|
+
- ca. 20x faster than the Python original [twbm](https://github.com/sysid/twbm) after warming up Python.
|
|
112
|
+
```bash
|
|
113
|
+
time twbm search 'zzz*' --np
|
|
114
|
+
0. zzzeek : Asynchronous Python and Databases [343]
|
|
115
|
+
https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/
|
|
116
|
+
async, knowhow, py
|
|
117
|
+
|
|
118
|
+
|
|
119
|
+
Found: 1
|
|
120
|
+
343
|
|
121
|
+
|
|
122
|
+
real 0m0.501s
|
|
123
|
+
user 0m0.268s
|
|
124
|
+
sys 0m0.070s
|
|
125
|
+
|
|
126
|
+
|
|
127
|
+
|
|
128
|
+
time bkmr search 'zzz*' --np
|
|
129
|
+
1. zzzeek : Asynchronous Python and Databases [343]
|
|
130
|
+
https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/
|
|
131
|
+
async knowhow py
|
|
132
|
+
|
|
133
|
+
|
|
134
|
+
real 0m0.027s
|
|
135
|
+
user 0m0.008s
|
|
136
|
+
sys 0m0.016s
|
|
137
|
+
```
|
|
138
|
+
[sysid blog: bkmr](https://sysid.github.io/bkmr/)
|
|
139
|
+
|
|
140
|
+
|
|
141
|
+
<!-- Badges -->
|
|
142
|
+
[pypi-url]: https://pypi.org/project/bkmr/
|
|
143
|
+
|
|
144
|
+
[build-image]: https://github.com/sysid/bkmr/actions/workflows/release_wheels.yml/badge.svg
|
|
145
|
+
[build-url]: https://github.com/sysid/bkmr/actions/workflows/release_wheels.yml
|
|
146
|
+
|
|
147
|
+
[quality-image]: https://api.codeclimate.com/v1/badges/3130fa0ba3b7993fbf0a/maintainability
|
|
148
|
+
[quality-url]: https://codeclimate.com/github/nalgeon/podsearch-py
|
|
149
|
+
|
|
150
|
+
[pypi-image]: https://badge.fury.io/py/bkmr.svg
|
|
151
|
+
[pypi-url]: https://pypi.org/project/bkmr/
|