golded-ftn-sqlite 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- golded_ftn_sqlite-1.0.0/.gitignore +10 -0
- golded_ftn_sqlite-1.0.0/CHANGELOG.md +8 -0
- golded_ftn_sqlite-1.0.0/CONTEXT.md +37 -0
- golded_ftn_sqlite-1.0.0/CONTRIBUTING.md +23 -0
- golded_ftn_sqlite-1.0.0/LICENSE +21 -0
- golded_ftn_sqlite-1.0.0/PKG-INFO +123 -0
- golded_ftn_sqlite-1.0.0/README.md +103 -0
- golded_ftn_sqlite-1.0.0/SECURITY.md +11 -0
- golded_ftn_sqlite-1.0.0/docs/adr/0001-single-area-default-multi-area-format.md +3 -0
- golded_ftn_sqlite-1.0.0/docs/schema.md +446 -0
- golded_ftn_sqlite-1.0.0/pyproject.toml +68 -0
- golded_ftn_sqlite-1.0.0/scripts/verify_distribution.py +98 -0
- golded_ftn_sqlite-1.0.0/src/golded_ftn_sqlite/__init__.py +15 -0
- golded_ftn_sqlite-1.0.0/src/golded_ftn_sqlite/database.py +257 -0
- golded_ftn_sqlite-1.0.0/src/golded_ftn_sqlite/models.py +31 -0
- golded_ftn_sqlite-1.0.0/src/golded_ftn_sqlite/py.typed +1 -0
- golded_ftn_sqlite-1.0.0/src/golded_ftn_sqlite/reader.py +163 -0
- golded_ftn_sqlite-1.0.0/src/golded_ftn_sqlite/writer.py +510 -0
- golded_ftn_sqlite-1.0.0/tests/test_database.py +162 -0
- golded_ftn_sqlite-1.0.0/tests/test_messages.py +670 -0
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 1.0.0 — 2026-10-25
|
|
4
|
+
|
|
5
|
+
- Add schema version 1 for single- and multi-area SQLite message bases.
|
|
6
|
+
- Add netmail and echomail reading, transactional append, full update and soft delete.
|
|
7
|
+
- Preserve addresses, ordered controls, routing, reply metadata and import provenance.
|
|
8
|
+
- Add optimistic revisions covering message and related metadata rows.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# FTN SQLite message base
|
|
2
|
+
|
|
3
|
+
This context names the durable things in the SQLite message-base format.
|
|
4
|
+
|
|
5
|
+
## Language
|
|
6
|
+
|
|
7
|
+
**Database**:
|
|
8
|
+
One SQLite file containing one or more areas.
|
|
9
|
+
_Avoid_: Base file, container
|
|
10
|
+
|
|
11
|
+
**Area**:
|
|
12
|
+
A numbered sequence of either netmail or echomail messages. The normal deployment has one area per database, while the format permits several.
|
|
13
|
+
_Avoid_: Board, folder
|
|
14
|
+
|
|
15
|
+
**Netmail**:
|
|
16
|
+
A point-to-point FTN message with an origin and an explicit destination address.
|
|
17
|
+
_Avoid_: Direct message
|
|
18
|
+
|
|
19
|
+
**Echomail**:
|
|
20
|
+
An FTN message distributed through a named area with ordered routing metadata.
|
|
21
|
+
_Avoid_: Conference message
|
|
22
|
+
|
|
23
|
+
**Message number**:
|
|
24
|
+
A positive, monotonic identity local to one area. Deleted numbers remain consumed.
|
|
25
|
+
_Avoid_: Database row ID, MSGID
|
|
26
|
+
|
|
27
|
+
**External message ID**:
|
|
28
|
+
The network-visible FTN `MSGID`, when present. It is data, not the database identity.
|
|
29
|
+
_Avoid_: Message number, row ID
|
|
30
|
+
|
|
31
|
+
**Address**:
|
|
32
|
+
A canonical FTN address string (`zone:net/node[.point][@domain]`), stored as TEXT and mapped to a structured DTO in application code. Absent address is SQL NULL.
|
|
33
|
+
_Avoid_: Split zone/net/node columns
|
|
34
|
+
|
|
35
|
+
**Revision**:
|
|
36
|
+
A token covering one stored message and its ordered metadata, used to reject stale updates and deletes.
|
|
37
|
+
_Avoid_: Version number
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Contributing
|
|
2
|
+
|
|
3
|
+
Use Python 3.12+ and uv. Work through the public database, reader and writer seams;
|
|
4
|
+
the SQLite schema is also a public format seam and may be checked with an independent
|
|
5
|
+
`sqlite3` connection.
|
|
6
|
+
|
|
7
|
+
Run:
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
uv sync
|
|
11
|
+
uv run pytest
|
|
12
|
+
uv run ruff check .
|
|
13
|
+
uv run ruff format --check .
|
|
14
|
+
uv run mypy
|
|
15
|
+
uv run python -m mypy.stubtest golded_ftn_sqlite
|
|
16
|
+
uv build
|
|
17
|
+
uv run twine check dist/*
|
|
18
|
+
uv run python scripts/verify_distribution.py
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
Use synthetic messages. Keep private archives, credentials and access material out
|
|
22
|
+
of tests and distributions. Schema changes require a migration and a new documented
|
|
23
|
+
`user_version`; editing version 1 in place after release is not compatibility.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 GoldED.dev contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: golded-ftn-sqlite
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: Transactional SQLite message bases for FTN netmail and echomail
|
|
5
|
+
Project-URL: Repository, https://github.com/golded-dev/golded-ftn-sqlite-python
|
|
6
|
+
Project-URL: Issues, https://github.com/golded-dev/golded-ftn-sqlite-python/issues
|
|
7
|
+
Project-URL: Changelog, https://github.com/golded-dev/golded-ftn-sqlite-python/blob/main/CHANGELOG.md
|
|
8
|
+
Author: GoldED.dev contributors
|
|
9
|
+
License-Expression: MIT
|
|
10
|
+
License-File: LICENSE
|
|
11
|
+
Classifier: Operating System :: OS Independent
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
16
|
+
Classifier: Typing :: Typed
|
|
17
|
+
Requires-Python: >=3.12
|
|
18
|
+
Requires-Dist: golded-ftn<2,>=1.4.0
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
|
|
21
|
+
# golded-ftn-sqlite
|
|
22
|
+
|
|
23
|
+
Repository: [`golded-ftn-sqlite-python`](https://github.com/golded-dev/golded-ftn-sqlite-python).
|
|
24
|
+
Install version 1.0.0 with Python 3.12 or newer:
|
|
25
|
+
|
|
26
|
+
```sh
|
|
27
|
+
python -m pip install golded-ftn-sqlite==1.0.0
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
`golded-ftn-sqlite` stores FTN netmail and echomail in SQLite through the
|
|
31
|
+
[`golded-ftn`](https://github.com/golded-dev/golded-ftn-python) message models.
|
|
32
|
+
It requires Python 3.12 or newer.
|
|
33
|
+
|
|
34
|
+
The format supports several areas in one database. The ordinary layout is still
|
|
35
|
+
one database file per area: it maps cleanly to GoldED configuration, keeps backup
|
|
36
|
+
and recovery local, and limits the blast radius when a disk has opinions.
|
|
37
|
+
|
|
38
|
+
## Pipe with ftnt
|
|
39
|
+
|
|
40
|
+
`ftnt` treats export envelopes as write input for every format, including SQLite:
|
|
41
|
+
|
|
42
|
+
```sh
|
|
43
|
+
ftnt create ./general.sqlite3 --format sqlite --area-code FIDO.GENERAL
|
|
44
|
+
ftnt export ./general --format jam |
|
|
45
|
+
ftnt write ./general.sqlite3 --format sqlite --jsonl
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
## Use it
|
|
49
|
+
|
|
50
|
+
```python
|
|
51
|
+
from golded_ftn import FtnAddress, OutgoingMessage
|
|
52
|
+
from golded_ftn_sqlite import (
|
|
53
|
+
AreaDefinition,
|
|
54
|
+
AreaKind,
|
|
55
|
+
SqliteDatabase,
|
|
56
|
+
SqliteReader,
|
|
57
|
+
SqliteWriter,
|
|
58
|
+
)
|
|
59
|
+
|
|
60
|
+
path = "fido-general.sqlite3"
|
|
61
|
+
SqliteDatabase.create(
|
|
62
|
+
path,
|
|
63
|
+
area=AreaDefinition(
|
|
64
|
+
code="FIDO.GENERAL",
|
|
65
|
+
name="FidoNet General",
|
|
66
|
+
kind=AreaKind.ECHOMAIL,
|
|
67
|
+
default_origin=FtnAddress(zone=2, net=230, node=0),
|
|
68
|
+
),
|
|
69
|
+
)
|
|
70
|
+
|
|
71
|
+
with SqliteWriter().open(path) as session:
|
|
72
|
+
result = session.append(
|
|
73
|
+
OutgoingMessage(
|
|
74
|
+
from_name="Alice",
|
|
75
|
+
to_name="All",
|
|
76
|
+
subject="Hello",
|
|
77
|
+
body_text="Stored transactionally.",
|
|
78
|
+
)
|
|
79
|
+
)
|
|
80
|
+
stored = session.read(result.identity.msgno)
|
|
81
|
+
|
|
82
|
+
messages = tuple(SqliteReader().read(path))
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
When a database contains several areas, pass the area code to `SqliteReader` and
|
|
86
|
+
`SqliteWriter`. Omitting it is allowed only for a single-area database. Message
|
|
87
|
+
numbers are monotonic and local to each area.
|
|
88
|
+
|
|
89
|
+
Netmail requires origin and destination addresses. The origin may come from the
|
|
90
|
+
area's `default_origin`; the destination is always explicit. Echomail keeps ordered
|
|
91
|
+
control lines, repeated unknown kludges, `SEEN-BY`, `PATH`, reply links and import
|
|
92
|
+
provenance.
|
|
93
|
+
|
|
94
|
+
Writer sessions provide `read`, `append`, `update` and soft `delete`. Updates and
|
|
95
|
+
deletes require a revision token. The token covers the message row and its ordered
|
|
96
|
+
control, routing and reply-list rows, including changes made by another SQLite
|
|
97
|
+
client.
|
|
98
|
+
|
|
99
|
+
## SQLite format
|
|
100
|
+
|
|
101
|
+
Schema version 1 uses SQLite application ID `0x47465453` (`GFTS`), strict tables,
|
|
102
|
+
foreign keys and WAL journaling. The schema is a public format contract:
|
|
103
|
+
[docs/schema.md](docs/schema.md) documents identification, DDL, column semantics,
|
|
104
|
+
allocation, soft delete, concurrency and backup so other packages or applications
|
|
105
|
+
can implement the same database without this Python code.
|
|
106
|
+
|
|
107
|
+
Do not copy an open WAL database as a lone `.sqlite3` file. Use SQLite's backup API
|
|
108
|
+
or close/checkpoint it first.
|
|
109
|
+
|
|
110
|
+
## Development
|
|
111
|
+
|
|
112
|
+
```sh
|
|
113
|
+
uv sync --python 3.14
|
|
114
|
+
uv run pytest
|
|
115
|
+
uv run ruff check .
|
|
116
|
+
uv run ruff format --check .
|
|
117
|
+
uv run mypy
|
|
118
|
+
uv run python -m mypy.stubtest golded_ftn_sqlite
|
|
119
|
+
uv build
|
|
120
|
+
uv run twine check dist/*
|
|
121
|
+
uv run python scripts/verify_distribution.py
|
|
122
|
+
```
|
|
123
|
+
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
# golded-ftn-sqlite
|
|
2
|
+
|
|
3
|
+
Repository: [`golded-ftn-sqlite-python`](https://github.com/golded-dev/golded-ftn-sqlite-python).
|
|
4
|
+
Install version 1.0.0 with Python 3.12 or newer:
|
|
5
|
+
|
|
6
|
+
```sh
|
|
7
|
+
python -m pip install golded-ftn-sqlite==1.0.0
|
|
8
|
+
```
|
|
9
|
+
|
|
10
|
+
`golded-ftn-sqlite` stores FTN netmail and echomail in SQLite through the
|
|
11
|
+
[`golded-ftn`](https://github.com/golded-dev/golded-ftn-python) message models.
|
|
12
|
+
It requires Python 3.12 or newer.
|
|
13
|
+
|
|
14
|
+
The format supports several areas in one database. The ordinary layout is still
|
|
15
|
+
one database file per area: it maps cleanly to GoldED configuration, keeps backup
|
|
16
|
+
and recovery local, and limits the blast radius when a disk has opinions.
|
|
17
|
+
|
|
18
|
+
## Pipe with ftnt
|
|
19
|
+
|
|
20
|
+
`ftnt` treats export envelopes as write input for every format, including SQLite:
|
|
21
|
+
|
|
22
|
+
```sh
|
|
23
|
+
ftnt create ./general.sqlite3 --format sqlite --area-code FIDO.GENERAL
|
|
24
|
+
ftnt export ./general --format jam |
|
|
25
|
+
ftnt write ./general.sqlite3 --format sqlite --jsonl
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Use it
|
|
29
|
+
|
|
30
|
+
```python
|
|
31
|
+
from golded_ftn import FtnAddress, OutgoingMessage
|
|
32
|
+
from golded_ftn_sqlite import (
|
|
33
|
+
AreaDefinition,
|
|
34
|
+
AreaKind,
|
|
35
|
+
SqliteDatabase,
|
|
36
|
+
SqliteReader,
|
|
37
|
+
SqliteWriter,
|
|
38
|
+
)
|
|
39
|
+
|
|
40
|
+
path = "fido-general.sqlite3"
|
|
41
|
+
SqliteDatabase.create(
|
|
42
|
+
path,
|
|
43
|
+
area=AreaDefinition(
|
|
44
|
+
code="FIDO.GENERAL",
|
|
45
|
+
name="FidoNet General",
|
|
46
|
+
kind=AreaKind.ECHOMAIL,
|
|
47
|
+
default_origin=FtnAddress(zone=2, net=230, node=0),
|
|
48
|
+
),
|
|
49
|
+
)
|
|
50
|
+
|
|
51
|
+
with SqliteWriter().open(path) as session:
|
|
52
|
+
result = session.append(
|
|
53
|
+
OutgoingMessage(
|
|
54
|
+
from_name="Alice",
|
|
55
|
+
to_name="All",
|
|
56
|
+
subject="Hello",
|
|
57
|
+
body_text="Stored transactionally.",
|
|
58
|
+
)
|
|
59
|
+
)
|
|
60
|
+
stored = session.read(result.identity.msgno)
|
|
61
|
+
|
|
62
|
+
messages = tuple(SqliteReader().read(path))
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
When a database contains several areas, pass the area code to `SqliteReader` and
|
|
66
|
+
`SqliteWriter`. Omitting it is allowed only for a single-area database. Message
|
|
67
|
+
numbers are monotonic and local to each area.
|
|
68
|
+
|
|
69
|
+
Netmail requires origin and destination addresses. The origin may come from the
|
|
70
|
+
area's `default_origin`; the destination is always explicit. Echomail keeps ordered
|
|
71
|
+
control lines, repeated unknown kludges, `SEEN-BY`, `PATH`, reply links and import
|
|
72
|
+
provenance.
|
|
73
|
+
|
|
74
|
+
Writer sessions provide `read`, `append`, `update` and soft `delete`. Updates and
|
|
75
|
+
deletes require a revision token. The token covers the message row and its ordered
|
|
76
|
+
control, routing and reply-list rows, including changes made by another SQLite
|
|
77
|
+
client.
|
|
78
|
+
|
|
79
|
+
## SQLite format
|
|
80
|
+
|
|
81
|
+
Schema version 1 uses SQLite application ID `0x47465453` (`GFTS`), strict tables,
|
|
82
|
+
foreign keys and WAL journaling. The schema is a public format contract:
|
|
83
|
+
[docs/schema.md](docs/schema.md) documents identification, DDL, column semantics,
|
|
84
|
+
allocation, soft delete, concurrency and backup so other packages or applications
|
|
85
|
+
can implement the same database without this Python code.
|
|
86
|
+
|
|
87
|
+
Do not copy an open WAL database as a lone `.sqlite3` file. Use SQLite's backup API
|
|
88
|
+
or close/checkpoint it first.
|
|
89
|
+
|
|
90
|
+
## Development
|
|
91
|
+
|
|
92
|
+
```sh
|
|
93
|
+
uv sync --python 3.14
|
|
94
|
+
uv run pytest
|
|
95
|
+
uv run ruff check .
|
|
96
|
+
uv run ruff format --check .
|
|
97
|
+
uv run mypy
|
|
98
|
+
uv run python -m mypy.stubtest golded_ftn_sqlite
|
|
99
|
+
uv build
|
|
100
|
+
uv run twine check dist/*
|
|
101
|
+
uv run python scripts/verify_distribution.py
|
|
102
|
+
```
|
|
103
|
+
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
# Security
|
|
2
|
+
|
|
3
|
+
Treat SQLite message bases as untrusted input. Open only databases with the expected
|
|
4
|
+
application ID and schema version, and keep foreign-key validation enabled for writes.
|
|
5
|
+
|
|
6
|
+
Report vulnerabilities privately through GitHub security advisories once the public
|
|
7
|
+
repository exists. Do not attach private message bases, message contents, credentials
|
|
8
|
+
or access material to public reports.
|
|
9
|
+
|
|
10
|
+
WAL databases may have live `-wal` and `-shm` sidecars. Use SQLite's backup API for
|
|
11
|
+
a consistent copy while writers are active.
|
|
@@ -0,0 +1,3 @@
|
|
|
1
|
+
# Single-area default, multi-area format
|
|
2
|
+
|
|
3
|
+
Schema version 1 permits several areas in one SQLite database, while documentation and automatic area selection treat one area per file as the normal layout. This keeps GoldED-style configuration, backup and failure isolation simple without baking that operational choice into the format; servers and tossers can group areas later without a migration.
|