access-fieldkit 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 mhatipog
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,92 @@
1
+ Metadata-Version: 2.4
2
+ Name: access-fieldkit
3
+ Version: 0.1.0
4
+ Summary: Read-only Access database inventory, loss-aware export, and migration reconciliation
5
+ Author: mhatipog
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/mhatipog/access-migration-fieldkit
8
+ Project-URL: Issues, https://github.com/mhatipog/access-migration-fieldkit/issues
9
+ Project-URL: Changelog, https://github.com/mhatipog/access-migration-fieldkit/blob/main/CHANGELOG.md
10
+ Keywords: microsoft-access,accdb,mdb,migration,sqlite,database
11
+ Classifier: Development Status :: 3 - Alpha
12
+ Classifier: Environment :: Console
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: Intended Audience :: System Administrators
15
+ Classifier: Operating System :: OS Independent
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Java
18
+ Classifier: Topic :: Database
19
+ Requires-Python: >=3.10
20
+ Description-Content-Type: text/markdown
21
+ License-File: LICENSE
22
+ Dynamic: license-file
23
+
24
+ # Access Fieldkit
25
+
26
+ Tools for the first awkward week of an Access migration: finding out what's in the file, getting the rows out without quietly changing them, and checking that a landing copy still matches.
27
+
28
+ The most common mistake is treating a row count as a completed migration. An Access database can carry keyless tables, embedded SQL tied to forms, four-decimal Currency values, unsaved business rules in VBA, and paths to files that don't live in the database. Moving tables doesn't fix any of that.
29
+
30
+ Fieldkit does **not** claim to convert an Access application. It's a read-only source inspector, exporter, SQLite staging loader, and per-value reconciliation tool. It also includes a PostgreSQL loading example and a runbook for SQL Server/SSMA migrations.
31
+
32
+ ## Quick start
33
+
34
+ Requires Python 3.10+, Java JDK 11+, and an Internet connection for the first run to retrieve Jackcess 5.0.4 from Maven Central. The downloaded JAR is checked against the pinned SHA-256. You can also pass a verified local JAR with `--jackcess-jar`.
35
+
36
+ ```bash
37
+ python -m pip install .
38
+
39
+ # Makes only a manifest. Does not export rows.
40
+ access-fieldkit inspect ./legacy.accdb --out ./work/inventory
41
+
42
+ # Copies table rows into numbered JSONL files, plus a manifest with SHA-256 checksums.
43
+ access-fieldkit export ./legacy.accdb --out ./work/export
44
+
45
+ # What will need attention before someone changes the database or its front end?
46
+ access-fieldkit assess ./work/export --out ./work/assessment.md
47
+
48
+ # Optional: compare the native inventory with MDB Tools, if installed.
49
+ access-fieldkit probe-tools ./legacy.accdb ./work/export --out ./work/tool-probes.json
50
+
51
+ # Creates a new disposable landing database with original table/column names.
52
+ access-fieldkit load-sqlite ./work/export ./work/landing.sqlite
53
+
54
+ # Compares every exported value to the SQLite landing copy.
55
+ access-fieldkit reconcile ./work/export ./work/landing.sqlite
56
+ ```
57
+
58
+ Both `inspect` and `export` open the Access file **read-only**. Existing output folders are not overwritten. The SQLite loader creates its target through a temporary file and refuses to replace an existing database unless `--force` is given; don't point it at production.
59
+
60
+ ## What the export contains
61
+
62
+ `manifest.json` records original table and column names, Access field types, indexes, row counts, saved queries when readable, the source file checksum, and individual table-file checksums. Each `table-0001.jsonl` holds one row per line in the original column order. The file names are numbered on purpose: original Access object names must never become unchecked filesystem paths.
63
+
64
+ Raw rows are kept in order. No artificial primary keys are introduced into source data, no duplicates are collapsed, and NULL is different from an empty string. Currency and decimal values use an explicit text tag so that an intermediate JSON float never rounds the number. Binary values use base64. Unsupported object values cause extraction to fail with a table/row/column location rather than disappear.
65
+
66
+ The SQLite target is a **landing zone**, not a finalized relational schema: it retains every original column and adds an internal source-row ordinal. Access Currency maps to SQLite TEXT, because SQLite has no exact fixed-decimal type. Use a deliberate target schema and business-approved conversions before building a production application.
67
+
68
+ ## Where it fits
69
+
70
+ - **SQLite / local web API:** Run `load-sqlite`, then see `examples/serve_sqlite.py` for a read-only local HTTP view of the landing tables.
71
+ - **PostgreSQL:** `examples/load_postgres.py` uses psycopg 3 to create a fresh schema and load the extracted values, including Currency as `NUMERIC(19,4)`.
72
+ - **SQL Server / Azure SQL:** Use SSMA for the bulk SQL Server path; apply the decisions in [guides/sql-server.md](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/sql-server.md). This repository does not pretend to be SSMA.
73
+ - **Modern application rewrite:** Inventory saved queries here. Use desktop Access on Windows to document forms, events, macros and VBA before moving those workflows into an API or new UI.
74
+
75
+ ## Guides
76
+
77
+ - [Start with an inventory](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/inventory.md)
78
+ - [Currency, NULLs, dates and difficult data](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/data-fidelity.md)
79
+ - [Saved queries, VBA and forms](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/application-rewrite.md)
80
+ - [SQL Server and SSMA](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/sql-server.md)
81
+ - [PostgreSQL landing](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/postgresql.md)
82
+ - [Cutover and rollback](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/cutover.md)
83
+
84
+ ## Limits worth knowing
85
+
86
+ Encrypted `.accdb` files may need additional codec support; this tool doesn't try to bypass encryption. Attachments, multivalue columns, linked tables, and Access UI objects require separate inspection. If an Access object is unsupported, don't interpret an incomplete export as proof of a complete application migration. A Windows Access/SSMA run remains necessary for those parts.
87
+
88
+ This code grew out of repeatable migrations of real-format ACE files and documented Microsoft/Jackcess limitations, **not production client migrations**. The included examples use made-up data only. Use backups, permission reviews and a write freeze when assessing an actual customer database.
89
+
90
+ MIT licensed. See [LICENSE](https://github.com/mhatipog/access-migration-fieldkit/blob/main/LICENSE).
91
+
92
+ Maintained by [Vecture](https://vecture.be). Contributions and field reports are welcome via [GitHub issues](https://github.com/mhatipog/access-migration-fieldkit/issues).
@@ -0,0 +1,69 @@
1
+ # Access Fieldkit
2
+
3
+ Tools for the first awkward week of an Access migration: finding out what's in the file, getting the rows out without quietly changing them, and checking that a landing copy still matches.
4
+
5
+ The most common mistake is treating a row count as a completed migration. An Access database can carry keyless tables, embedded SQL tied to forms, four-decimal Currency values, unsaved business rules in VBA, and paths to files that don't live in the database. Moving tables doesn't fix any of that.
6
+
7
+ Fieldkit does **not** claim to convert an Access application. It's a read-only source inspector, exporter, SQLite staging loader, and per-value reconciliation tool. It also includes a PostgreSQL loading example and a runbook for SQL Server/SSMA migrations.
8
+
9
+ ## Quick start
10
+
11
+ Requires Python 3.10+, Java JDK 11+, and an Internet connection for the first run to retrieve Jackcess 5.0.4 from Maven Central. The downloaded JAR is checked against the pinned SHA-256. You can also pass a verified local JAR with `--jackcess-jar`.
12
+
13
+ ```bash
14
+ python -m pip install .
15
+
16
+ # Makes only a manifest. Does not export rows.
17
+ access-fieldkit inspect ./legacy.accdb --out ./work/inventory
18
+
19
+ # Copies table rows into numbered JSONL files, plus a manifest with SHA-256 checksums.
20
+ access-fieldkit export ./legacy.accdb --out ./work/export
21
+
22
+ # What will need attention before someone changes the database or its front end?
23
+ access-fieldkit assess ./work/export --out ./work/assessment.md
24
+
25
+ # Optional: compare the native inventory with MDB Tools, if installed.
26
+ access-fieldkit probe-tools ./legacy.accdb ./work/export --out ./work/tool-probes.json
27
+
28
+ # Creates a new disposable landing database with original table/column names.
29
+ access-fieldkit load-sqlite ./work/export ./work/landing.sqlite
30
+
31
+ # Compares every exported value to the SQLite landing copy.
32
+ access-fieldkit reconcile ./work/export ./work/landing.sqlite
33
+ ```
34
+
35
+ Both `inspect` and `export` open the Access file **read-only**. Existing output folders are not overwritten. The SQLite loader creates its target through a temporary file and refuses to replace an existing database unless `--force` is given; don't point it at production.
36
+
37
+ ## What the export contains
38
+
39
+ `manifest.json` records original table and column names, Access field types, indexes, row counts, saved queries when readable, the source file checksum, and individual table-file checksums. Each `table-0001.jsonl` holds one row per line in the original column order. The file names are numbered on purpose: original Access object names must never become unchecked filesystem paths.
40
+
41
+ Raw rows are kept in order. No artificial primary keys are introduced into source data, no duplicates are collapsed, and NULL is different from an empty string. Currency and decimal values use an explicit text tag so that an intermediate JSON float never rounds the number. Binary values use base64. Unsupported object values cause extraction to fail with a table/row/column location rather than disappear.
42
+
43
+ The SQLite target is a **landing zone**, not a finalized relational schema: it retains every original column and adds an internal source-row ordinal. Access Currency maps to SQLite TEXT, because SQLite has no exact fixed-decimal type. Use a deliberate target schema and business-approved conversions before building a production application.
44
+
45
+ ## Where it fits
46
+
47
+ - **SQLite / local web API:** Run `load-sqlite`, then see `examples/serve_sqlite.py` for a read-only local HTTP view of the landing tables.
48
+ - **PostgreSQL:** `examples/load_postgres.py` uses psycopg 3 to create a fresh schema and load the extracted values, including Currency as `NUMERIC(19,4)`.
49
+ - **SQL Server / Azure SQL:** Use SSMA for the bulk SQL Server path; apply the decisions in [guides/sql-server.md](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/sql-server.md). This repository does not pretend to be SSMA.
50
+ - **Modern application rewrite:** Inventory saved queries here. Use desktop Access on Windows to document forms, events, macros and VBA before moving those workflows into an API or new UI.
51
+
52
+ ## Guides
53
+
54
+ - [Start with an inventory](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/inventory.md)
55
+ - [Currency, NULLs, dates and difficult data](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/data-fidelity.md)
56
+ - [Saved queries, VBA and forms](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/application-rewrite.md)
57
+ - [SQL Server and SSMA](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/sql-server.md)
58
+ - [PostgreSQL landing](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/postgresql.md)
59
+ - [Cutover and rollback](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/cutover.md)
60
+
61
+ ## Limits worth knowing
62
+
63
+ Encrypted `.accdb` files may need additional codec support; this tool doesn't try to bypass encryption. Attachments, multivalue columns, linked tables, and Access UI objects require separate inspection. If an Access object is unsupported, don't interpret an incomplete export as proof of a complete application migration. A Windows Access/SSMA run remains necessary for those parts.
64
+
65
+ This code grew out of repeatable migrations of real-format ACE files and documented Microsoft/Jackcess limitations, **not production client migrations**. The included examples use made-up data only. Use backups, permission reviews and a write freeze when assessing an actual customer database.
66
+
67
+ MIT licensed. See [LICENSE](https://github.com/mhatipog/access-migration-fieldkit/blob/main/LICENSE).
68
+
69
+ Maintained by [Vecture](https://vecture.be). Contributions and field reports are welcome via [GitHub issues](https://github.com/mhatipog/access-migration-fieldkit/issues).
@@ -0,0 +1,2 @@
1
+ """Read-only extraction and landing tools for Microsoft Access databases."""
2
+ __version__ = "0.1.0"
@@ -0,0 +1,69 @@
1
+ """Command-line entry point; read-only Access handling by default."""
2
+ import argparse
3
+ import json
4
+ import shutil
5
+ import sys
6
+ import tempfile
7
+ from pathlib import Path
8
+ from .assessment import assess, markdown
9
+ from .landing import load, reconcile, read_manifest
10
+ from .runtime import run_reader
11
+ from .tool_probe import probe
12
+
13
+
14
+ def main(argv=None):
15
+ parser=argparse.ArgumentParser(prog='access-fieldkit',description='Inspect and stage Access databases without silently losing legacy data')
16
+ parser.add_argument('--jackcess-jar',help='Use a local Jackcess 5.0.4 jar; useful for offline work')
17
+ cmd=parser.add_subparsers(dest='command',required=True)
18
+ for action in ('inspect','export'):
19
+ p=cmd.add_parser(action, help=f'{action.title()} a source Access file (never writes to source)')
20
+ p.add_argument('source',type=Path)
21
+ p.add_argument('--out',type=Path,required=True)
22
+ p=cmd.add_parser('assess',help='Write a migration assessment based on exported native metadata')
23
+ p.add_argument('folder',type=Path);p.add_argument('--out',type=Path)
24
+ p=cmd.add_parser('load-sqlite',help='Load raw data into a loss-aware SQLite landing database')
25
+ p.add_argument('folder',type=Path);p.add_argument('target',type=Path);p.add_argument('--force',action='store_true')
26
+ p=cmd.add_parser('reconcile',help='Compare every exported Access value against the landing database')
27
+ p.add_argument('folder',type=Path);p.add_argument('target',type=Path)
28
+ p=cmd.add_parser('probe-tools',help='Compare the native inventory with any installed alternative readers')
29
+ p.add_argument('source',type=Path);p.add_argument('folder',type=Path);p.add_argument('--out',type=Path)
30
+ args=parser.parse_args(argv)
31
+ try:
32
+ if args.command in ('inspect','export'):
33
+ if args.out.exists():
34
+ raise FileExistsError('Output folder already exists; choose a new directory')
35
+ args.out.parent.mkdir(parents=True,exist_ok=True)
36
+ folder=Path(tempfile.mkdtemp(prefix='.access-fieldkit-',dir=args.out.parent))
37
+ try:
38
+ run_reader(args.command,args.source,folder,args.jackcess_jar)
39
+ folder.rename(args.out)
40
+ finally:
41
+ if folder.exists():shutil.rmtree(folder)
42
+ print('Manifest:',args.out/'manifest.json')
43
+ elif args.command=='assess':
44
+ result=assess(args.folder)
45
+ md=markdown(result)
46
+ if args.out:
47
+ args.out.parent.mkdir(parents=True,exist_ok=True)
48
+ args.out.write_text(md,encoding='utf-8')
49
+ else:print(md)
50
+ print(json.dumps({'findings':len(result['findings']),'tables':result['tables'],'rows':result['rows']}))
51
+ elif args.command=='probe-tools':
52
+ result=probe(args.source,args.folder)
53
+ output=json.dumps(result,indent=2)
54
+ if args.out:
55
+ args.out.parent.mkdir(parents=True,exist_ok=True)
56
+ args.out.write_text(output+'\n',encoding='utf-8')
57
+ print(output)
58
+ elif args.command=='load-sqlite':
59
+ print(json.dumps(load(args.folder,args.target,args.force),indent=2))
60
+ elif args.command=='reconcile':
61
+ result=reconcile(args.folder,args.target)
62
+ print(json.dumps(result,indent=2))
63
+ if not result['valid']: return 2
64
+ return 0
65
+ except Exception as exc:
66
+ print(f'{args.command}: {exc}',file=sys.stderr)
67
+ return 1
68
+
69
+ if __name__=='__main__':raise SystemExit(main())
@@ -0,0 +1,71 @@
1
+ """Object-level migration inventory from extracted native Access metadata."""
2
+ import json
3
+ import re
4
+ from pathlib import Path
5
+ from .landing import read_manifest, iter_rows
6
+
7
+ DANGER_SQL = [
8
+ ('ui_parameter',r'\bForms\s*!|\bReports\s*!','Form/report-bound reference; replace with explicit API parameter'),
9
+ ('domain_aggregate',r'\b(?:DLookup|DSum|DCount|DAvg|DMax|DMin)\s*\(','Jet domain aggregate; rewrite as SQL join/subquery'),
10
+ ('vba_expression',r'\b(?:Nz|IIf|Format|DateDiff|CDate)\s*\(','Access expression; check null, date and rounding semantics'),
11
+ ('crosstab',r'\bTRANSFORM\b','Crosstab/pivot requires a target-specific query'),
12
+ ('parameters',r'\bPARAMETERS\b','Parameter declaration; adapt to the target driver'),
13
+ ('action_query',r'^\s*(?:UPDATE|DELETE|INSERT|SELECT.+INTO)\b','Review the write side effects before conversion'),
14
+ ('wildcard',r"\bLIKE\s+['\"]\*",'Access wildcard differs from SQL LIKE'),
15
+ ]
16
+
17
+
18
+ def assess(folder):
19
+ folder = Path(folder)
20
+ source = read_manifest(folder,require_export=False)
21
+ findings=[]
22
+ def add(level,code,location,detail):
23
+ findings.append({'level':level,'code':code,'location':location,'detail':detail})
24
+ for table in source['tables']:
25
+ name=table['name']; cols=table['columns']; indexes=table['indexes']
26
+ if not any(i['primary'] for i in indexes):
27
+ add('high','missing_primary_key',name,'Keyless tables can have duplicate rows and ambiguous linked-table updates')
28
+ for col in cols:
29
+ t=col['type']; location=f'{name}.{col["name"]}'
30
+ if t in ('COMPLEX_TYPE','OLE'):
31
+ add('high','complex_or_binary_column',location,'Handle attachments/OLE explicitly, and check actual bytes')
32
+ if t in ('MONEY','NUMERIC'):
33
+ add('medium','exact_decimal',location,'Preserve decimal scale and define an approved rounding policy')
34
+ if t == 'SHORT_DATE_TIME':
35
+ add('info','date_range',location,'Check pre-1753 dates and timestamp precision before SQL Server datetime conversion')
36
+ if col['name'].casefold() in {'order','select','from','user','table','group','where'}:
37
+ add('medium','reserved_identifier',location,'Quote target identifiers or use an explicit mapping')
38
+ if source['exported']:
39
+ dates=[i for i,c in enumerate(cols) if c['type']=='SHORT_DATE_TIME']
40
+ if dates:
41
+ older=0
42
+ for _,row in iter_rows(folder,table):
43
+ for i in dates:
44
+ if isinstance(row[i],str) and re.match(r'^(?:0\d\d\d|1[0-6]\d\d|17[0-4]\d|175[0-2])-\d\d-\d\d',row[i]):
45
+ older+=1
46
+ if older:
47
+ add('high','pre_1753_dates',name,f'{older} dates predate SQL Server datetime minimum; use date/datetime2')
48
+ for query in source['queries']:
49
+ sql=query.get('sql')
50
+ if not sql:
51
+ add('high','unreadable_query',query['name'],'Query SQL could not be reconstructed; examine in Windows Access')
52
+ continue
53
+ for code,pattern,message in DANGER_SQL:
54
+ if re.search(pattern,sql,re.I|re.S):
55
+ add('medium',code,query['name'],message)
56
+ for warning in source.get('warnings',[]):
57
+ add('medium','extractor_warning','database',warning)
58
+ add('high','uninspected_application_objects','Access frontend','Forms, reports, macros and VBA require Windows Access inventory; Java/Jet table inspection does not certify them')
59
+ return {'source_file_sha256':source['file_sha256'], 'tables':len(source['tables']),
60
+ 'rows':source['total_rows'],'saved_queries':len(source['queries']),
61
+ 'findings':findings,'scope':'database objects; no Access UI/VBA execution'}
62
+
63
+
64
+ def markdown(report):
65
+ lines=['# Access migration assessment', '',f"Tables: {report['tables']} · Rows: {report['rows']} · Saved queries: {report['saved_queries']}",
66
+ '', 'This inventory is based on native database metadata. Access forms, reports, macros and VBA still need a Windows application audit.',
67
+ '', '| Severity | Object | Issue | Next action |', '|---|---|---|---|']
68
+ for f in report['findings']:
69
+ def clean(s):return str(s).replace('|','\\|').replace('\n',' ')
70
+ lines.append(f"| {clean(f['level'])} | `{clean(f['location'])}` | {clean(f['code'])} | {clean(f['detail'])} |")
71
+ return '\n'.join(lines)+'\n'
@@ -0,0 +1,120 @@
1
+ import com.healthmarketscience.jackcess.*;
2
+ import com.healthmarketscience.jackcess.query.Query;
3
+ import java.io.*;
4
+ import java.math.*;
5
+ import java.nio.charset.StandardCharsets;
6
+ import java.nio.file.*;
7
+ import java.security.MessageDigest;
8
+ import java.time.*;
9
+ import java.util.*;
10
+
11
+ /** Read-only ACE/Jet inventory and loss-aware row extraction. No Access runtime emulation. */
12
+ public final class AccessReader {
13
+ private static String j(String s) {
14
+ StringBuilder b = new StringBuilder("\"");
15
+ for (int i=0; i<s.length(); i++) {
16
+ char c=s.charAt(i);
17
+ switch(c) {
18
+ case '\"': b.append("\\\""); break;
19
+ case '\\': b.append("\\\\"); break;
20
+ case '\n': b.append("\\n"); break;
21
+ case '\r': b.append("\\r"); break;
22
+ case '\t': b.append("\\t"); break;
23
+ default:
24
+ if (c<32 || (Character.isHighSurrogate(c) && (i+1 == s.length() || !Character.isLowSurrogate(s.charAt(i+1)))) || Character.isLowSurrogate(c) && (i == 0 || !Character.isHighSurrogate(s.charAt(i-1)))) {
25
+ b.append(String.format("\\u%04x",(int)c));
26
+ } else b.append(c);
27
+ }
28
+ }
29
+ return b.append('\"').toString();
30
+ }
31
+ private static String val(Object v) {
32
+ if(v==null) return "null";
33
+ if(v instanceof byte[]) return "{\"$binary\":" + j(Base64.getEncoder().encodeToString((byte[])v)) + "}";
34
+ if(v instanceof Boolean) return v.toString();
35
+ if(v instanceof BigDecimal) return "{\"$decimal\":" + j(((BigDecimal)v).toPlainString()) + "}";
36
+ if(v instanceof Number) {
37
+ if(v instanceof Double || v instanceof Float) {
38
+ if(!Double.isFinite(((Number)v).doubleValue())) throw new IllegalStateException("Non-finite floating point data");
39
+ return "{\"$float\":" + j(v.toString()) + "}";
40
+ }
41
+ return v.toString();
42
+ }
43
+ if(v instanceof LocalDateTime || v instanceof LocalDate || v instanceof LocalTime || v instanceof OffsetDateTime) return j(v.toString());
44
+ if(v instanceof java.util.Date) return j(((java.util.Date)v).toInstant().atZone(ZoneOffset.UTC).toLocalDateTime().toString());
45
+ if(v instanceof CharSequence || v instanceof Character) return j(v.toString());
46
+ throw new IllegalArgumentException("Unsupported Access value type " + v.getClass().getName() + "; no silent string coercion");
47
+ }
48
+ private static String array(List<String> parts) { return "[" + String.join(",",parts) + "]"; }
49
+ private static String sha(Path p) throws Exception {
50
+ MessageDigest md=MessageDigest.getInstance("SHA-256");
51
+ try(InputStream in=Files.newInputStream(p)) { byte[] block=new byte[65536]; int n; while((n=in.read(block))!=-1) md.update(block,0,n); }
52
+ StringBuilder s=new StringBuilder(); for(byte b: md.digest()) s.append(String.format("%02x",b & 255)); return s.toString();
53
+ }
54
+ private static void inspect(Path source,Path target,boolean export) throws Exception {
55
+ if(!Files.isRegularFile(source)) throw new IOException("Missing Access database: " + source);
56
+ Files.createDirectories(target);
57
+ List<String> tables=new ArrayList<>(), queries=new ArrayList<>(), warnings=new ArrayList<>();
58
+ long total=0;
59
+ try(Database db=new DatabaseBuilder(source.toFile()).setReadOnly(true).open()) {
60
+ int serial=0;
61
+ for(String name:db.getTableNames()) {
62
+ Table table=db.getTable(name);
63
+ String file=String.format(Locale.ROOT,"table-%04d.jsonl",++serial);
64
+ List<String> cols=new ArrayList<>(), keys=new ArrayList<>();
65
+ boolean complex=false, lacksPk=true;
66
+ for(Column c:table.getColumns()) {
67
+ String t=c.getType().name();
68
+ if(t.equals("COMPLEX_TYPE") || t.equals("UNKNOWN_0D") || t.equals("UNSUPPORTED_VARLEN")) complex=true;
69
+ cols.add("{\"name\":" + j(c.getName()) + ",\"type\":" + j(t) + ",\"length\":" + c.getLength() + ",\"auto_number\":"+c.isAutoNumber()+"}");
70
+ }
71
+ for(Index idx:table.getIndexes()) {
72
+ List<String> names=new ArrayList<>();
73
+ for(Index.Column c:idx.getColumns()) names.add(j(c.getName()));
74
+ keys.add("{\"name\":"+j(idx.getName())+",\"primary\":"+idx.isPrimaryKey()+",\"unique\":"+idx.isUnique()+",\"columns\":"+array(names)+"}");
75
+ if(idx.isPrimaryKey()) lacksPk=false;
76
+ }
77
+ if(lacksPk) warnings.add("No primary key: " + name + "; retain source ordinals for duplicates");
78
+ if(complex) warnings.add("Complex/attachment field: " + name + "; inspection only; export may require Access on Windows");
79
+ long count=0;
80
+ Path dest=target.resolve(file);
81
+ try(BufferedWriter out=export ? Files.newBufferedWriter(dest,StandardCharsets.UTF_8) : null) {
82
+ for(Row row:table) {
83
+ count++;
84
+ if(export) {
85
+ List<String> values=new ArrayList<>();
86
+ for(Column col:table.getColumns()) {
87
+ try { values.add(val(row.get(col.getName()))); }
88
+ catch(Exception ex) { throw new IOException("Unsupported value at " + name + " row " + count + " column " + col.getName() + ": " + ex.getMessage(),ex); }
89
+ }
90
+ out.write(array(values)); out.newLine();
91
+ }
92
+ }
93
+ }
94
+ total+=count;
95
+ String data=export?",\"data_file\":"+j(file)+",\"sha256\":"+j(sha(dest)):"";
96
+ tables.add("{\"name\":"+j(name)+",\"rows\":"+count+",\"columns\":"+array(cols)+",\"indexes\":"+array(keys)+data+"}");
97
+ }
98
+ try {
99
+ for(Query q:db.getQueries()) {
100
+ String sql;
101
+ try { sql=q.toSQLString(); } catch(RuntimeException ex) { sql=null; warnings.add("Unparsable saved query: " + q.getName()); }
102
+ queries.add("{\"name\":"+j(q.getName())+",\"type\":"+j(q.getType().toString())+",\"sql\":"+(sql==null?"null":j(sql))+"}");
103
+ }
104
+ } catch(Exception ex) { warnings.add("Unable to inventory saved queries: " + ex.getClass().getSimpleName()); }
105
+ List<String> warningsJson=new ArrayList<>(); for(String s:warnings) warningsJson.add(j(s));
106
+ String schema="{\"format\":\"access-fieldkit-v1\",\"source_name\":"+j(source.getFileName().toString())+
107
+ ",\"file_sha256\":"+j(sha(source))+",\"access_format\":"+j(db.getFileFormat().name())+
108
+ ",\"exported\":"+export+",\"tables\":"+array(tables)+",\"queries\":"+array(queries)+
109
+ ",\"total_rows\":"+total+",\"warnings\":"+array(warningsJson)+"}";
110
+ Files.writeString(target.resolve("manifest.json"),schema+"\n",StandardCharsets.UTF_8);
111
+ System.out.println("Access read-only " + (export ? "export" : "inspection") + ": " + tables.size() + " tables, " + total + " rows, " + queries.size() + " queries");
112
+ }
113
+ }
114
+ public static void main(String[] args) throws Exception {
115
+ TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
116
+ if(args.length!=3 || !(args[0].equals("inspect") || args[0].equals("export")))
117
+ throw new IllegalArgumentException("Usage: AccessReader inspect|export source.accdb output-directory");
118
+ inspect(Paths.get(args[1]),Paths.get(args[2]),args[0].equals("export"));
119
+ }
120
+ }
@@ -0,0 +1,208 @@
1
+ """Loss-aware SQLite landing zone: preserves source ordinals and duplicate rows.
2
+
3
+ This is staging, not an automatic normalized schema conversion. No source keys or
4
+ foreign keys are invented; duplicate and orphaned records remain inspectable.
5
+ """
6
+ import base64
7
+ import hashlib
8
+ import json
9
+ import os
10
+ import sqlite3
11
+ import tempfile
12
+ from pathlib import Path
13
+
14
+
15
+ def quote(name):
16
+ if not isinstance(name, str) or '\x00' in name or not name:
17
+ raise ValueError(f"Invalid SQL identifier: {name!r}")
18
+ return '"' + name.replace('"', '""') + '"'
19
+
20
+
21
+ def read_manifest(folder, require_export=True):
22
+ folder = Path(folder)
23
+ manifest = json.loads((folder / 'manifest.json').read_text(encoding='utf-8'))
24
+ if manifest.get('format') != 'access-fieldkit-v1':
25
+ raise ValueError('Unrecognized export format')
26
+ if require_export and not manifest.get('exported'):
27
+ raise ValueError('Only an inventory was supplied; run export first')
28
+ ids = [t['name'].casefold() for t in manifest['tables']]
29
+ if len(ids) != len(set(ids)):
30
+ raise ValueError('Duplicate case-insensitive table names')
31
+ for t in manifest['tables']:
32
+ names = [c['name'].casefold() for c in t['columns']]
33
+ if len(names) != len(set(names)):
34
+ raise ValueError(f"Duplicate column names in {t['name']}")
35
+ if require_export:
36
+ data_file = t['data_file']
37
+ if data_file != Path(data_file).name or not data_file.startswith('table-') or not data_file.endswith('.jsonl'):
38
+ raise ValueError('Unsafe data file path in export manifest')
39
+ return manifest
40
+
41
+
42
+ def check_checksum(path, expected):
43
+ digest = hashlib.sha256()
44
+ with Path(path).open('rb') as f:
45
+ for block in iter(lambda: f.read(1024 * 1024), b''):
46
+ digest.update(block)
47
+ got = digest.hexdigest()
48
+ if got != expected:
49
+ raise ValueError(f'Data checksum mismatch: {path.name} (expected {expected}, got {got})')
50
+ return got
51
+
52
+
53
+ def iter_rows(folder, table):
54
+ source = Path(folder) / table['data_file']
55
+ with source.open(encoding='utf-8', newline='') as f:
56
+ for line_num, line in enumerate(f, 1):
57
+ if not line.strip():
58
+ raise ValueError(f'Blank source record: {source}:{line_num}')
59
+ row = json.loads(line)
60
+ if not isinstance(row, list) or len(row) != len(table['columns']):
61
+ raise ValueError(f'Invalid source record: {source}:{line_num}')
62
+ yield line_num, row
63
+
64
+
65
+ def decode(value):
66
+ if isinstance(value, dict):
67
+ if len(value) != 1:
68
+ raise ValueError('Unknown tagged Access value')
69
+ tag, raw = next(iter(value.items()))
70
+ if not isinstance(raw, str):
71
+ raise ValueError('Malformed tagged Access value')
72
+ if tag == '$binary':
73
+ return base64.b64decode(raw, validate=True)
74
+ if tag == '$decimal':
75
+ # SQLite has no exact decimal storage class. Use TEXT without
76
+ # losing the Access Currency value's original decimal scale.
77
+ return raw
78
+ if tag == '$float':
79
+ return float(raw)
80
+ raise ValueError(f'Unsupported tagged Access value: {tag}')
81
+ if isinstance(value, (list, dict)):
82
+ raise ValueError('Nested Access value must be extracted explicitly')
83
+ return value
84
+
85
+
86
+ def dtype(access_type):
87
+ if access_type in ('BYTE', 'INT', 'LONG', 'BIG_INT', 'SHORT', 'BOOLEAN'):
88
+ return 'INTEGER'
89
+ if access_type in ('DOUBLE', 'FLOAT'):
90
+ return 'REAL'
91
+ if access_type in ('BINARY', 'OLE'):
92
+ return 'BLOB'
93
+ return 'TEXT'
94
+
95
+
96
+ def load(folder, target, force=False):
97
+ folder, target = Path(folder), Path(target)
98
+ manifest = read_manifest(folder)
99
+ if target.exists() and not force:
100
+ raise FileExistsError(f'{target} exists (use --force to replace a disposable target)')
101
+ if target.is_symlink():
102
+ raise ValueError('Refusing to replace a symlink target')
103
+ target.parent.mkdir(parents=True, exist_ok=True)
104
+ fd, tmpname = tempfile.mkstemp(prefix='.access-fieldkit-', suffix='.sqlite', dir=target.parent)
105
+ os.close(fd)
106
+ tmp = Path(tmpname)
107
+ conn = None
108
+ stats = []
109
+ try:
110
+ conn = sqlite3.connect(tmp)
111
+ conn.execute('PRAGMA journal_mode=DELETE')
112
+ conn.execute('PRAGMA synchronous=FULL')
113
+ conn.execute('CREATE TABLE _af_metadata (table_name TEXT PRIMARY KEY, ordinal_field TEXT NOT NULL, data_file TEXT NOT NULL, sha256 TEXT NOT NULL)')
114
+ for table in manifest['tables']:
115
+ name = table['name']
116
+ columns = table['columns']
117
+ ordinal = '_af_source_row'
118
+ existing = {c['name'].casefold() for c in columns}
119
+ while ordinal.casefold() in existing:
120
+ ordinal += '_'
121
+ fields = [quote(ordinal) + ' INTEGER PRIMARY KEY']
122
+ fields += [quote(c['name']) + ' ' + dtype(c['type']) for c in columns]
123
+ if name.casefold() == '_af_metadata':
124
+ raise ValueError('Source table name collides with internal metadata table')
125
+ conn.execute('CREATE TABLE ' + quote(name) + ' (' + ','.join(fields) + ')')
126
+ check_checksum(folder / table['data_file'], table['sha256'])
127
+ placeholders = ','.join('?' for _ in range(len(columns)+1))
128
+ insert = 'INSERT INTO ' + quote(name) + ' VALUES (' + placeholders + ')'
129
+ count = 0
130
+ batch = []
131
+ for num, values in iter_rows(folder, table):
132
+ batch.append([num, *(decode(v) for v in values)])
133
+ if len(batch) >= 500:
134
+ conn.executemany(insert, batch)
135
+ batch.clear()
136
+ count += 1
137
+ if batch:
138
+ conn.executemany(insert, batch)
139
+ if count != table['rows']:
140
+ raise ValueError(f'Row-count mismatch for {name}: {count} != {table["rows"]}')
141
+ conn.execute('INSERT INTO _af_metadata VALUES (?,?,?,?)', (name, ordinal, table['data_file'], table['sha256']))
142
+ stats.append({'name':name,'rows':count})
143
+ conn.commit()
144
+ conn.close()
145
+ conn = None
146
+ # An atomic replacement keeps any prior target intact on a failed load.
147
+ os.replace(tmp, target)
148
+ return {'target':str(target),'tables':len(stats),'rows':sum(x['rows'] for x in stats),'status':'loaded','detail':stats}
149
+ finally:
150
+ if conn is not None:
151
+ conn.close()
152
+ if tmp.exists():
153
+ tmp.unlink()
154
+
155
+
156
+ def reconcile(folder, target):
157
+ folder, target = Path(folder), Path(target)
158
+ manifest = read_manifest(folder)
159
+ issues = []
160
+ total = 0
161
+ with sqlite3.connect(f'file:{target.resolve()}?mode=ro', uri=True) as conn:
162
+ try:
163
+ mappings = {name:(ordinal,filename,digest) for name,ordinal,filename,digest in conn.execute('SELECT * FROM _af_metadata')}
164
+ except sqlite3.Error as ex:
165
+ raise ValueError('Not an access-fieldkit SQLite landing database') from ex
166
+ if set(mappings) != {t['name'] for t in manifest['tables']}:
167
+ issues.append('Table inventory differs between source export and target')
168
+ for table in manifest['tables']:
169
+ name = table['name']
170
+ if name not in mappings:
171
+ continue
172
+ ordinal,filename,digest=mappings[name]
173
+ try:
174
+ check_checksum(folder/table['data_file'],table['sha256'])
175
+ if digest!=table['sha256'] or filename!=table['data_file']:
176
+ issues.append(f'{name}: source fingerprint does not match landing metadata')
177
+ got = conn.execute('SELECT * FROM ' + quote(name) + ' ORDER BY ' + quote(ordinal))
178
+ count = 0
179
+ for (num,source), row in zip(iter_rows(folder,table),got):
180
+ gotordinal,*values = row
181
+ if gotordinal!=num:
182
+ issues.append(f'{name}: row ordinal mismatch at {num}')
183
+ for col,original,actual in zip(table['columns'],source,values):
184
+ expected=decode(original)
185
+ if isinstance(expected,float):
186
+ equal = expected == actual
187
+ else:
188
+ equal = expected == actual and type(expected) is type(actual)
189
+ # SQLite stores Boolean as 0/1 rather than bool.
190
+ if isinstance(expected,bool):
191
+ equal = int(expected) == actual
192
+ if not equal:
193
+ issues.append(f'{name}: value mismatch row {num} column {col["name"]}')
194
+ if len(issues)>=20:
195
+ break
196
+ count+=1
197
+ if len(issues)>=20:
198
+ break
199
+ count_db=conn.execute('SELECT COUNT(*) FROM '+quote(name)).fetchone()[0]
200
+ if count_db!=table['rows'] or count!=table['rows']:
201
+ issues.append(f'{name}: source={table["rows"]} compared={count} target={count_db}')
202
+ total+=count_db
203
+ except (ValueError,sqlite3.Error) as ex:
204
+ issues.append(f'{name}: {ex}')
205
+ if len(issues)>=20:
206
+ break
207
+ return {'valid':not issues,'source_file_sha256':manifest['file_sha256'],'tables':len(manifest['tables']),
208
+ 'rows':total,'issues':issues[:25]}
@@ -0,0 +1,51 @@
1
+ """Prepare the pinned Java extractor on a local machine."""
2
+ import hashlib
3
+ import os
4
+ import subprocess
5
+ import urllib.request
6
+ from pathlib import Path
7
+
8
+ VERSION = "5.0.4"
9
+ # Checked against the JAR used in the Oracle VM run and the upstream release.
10
+ SHA256 = "19a7494a56f439a57fda85c059b019b665802085cec5ae8186d1e028cc1bdfbc"
11
+ URL = f"https://repo.maven.apache.org/maven2/com/healthmarketscience/jackcess/jackcess/{VERSION}/jackcess-{VERSION}.jar"
12
+ CACHE = Path.home() / ".cache" / "access-fieldkit" / VERSION
13
+ JAVA_SOURCE = Path(__file__).parent / "java" / "AccessReader.java"
14
+
15
+
16
+ def java_classpath(jar=None):
17
+ cache = CACHE
18
+ cache.mkdir(parents=True, exist_ok=True)
19
+ if jar is not None:
20
+ dependency = Path(jar).resolve()
21
+ if not dependency.is_file():
22
+ raise FileNotFoundError(f"Jackcess jar not found: {dependency}")
23
+ else:
24
+ dependency = cache / f"jackcess-{VERSION}.jar"
25
+ if not dependency.exists():
26
+ data = urllib.request.urlopen(URL, timeout=45).read()
27
+ if not data.startswith(b"PK"):
28
+ raise RuntimeError("Downloaded dependency was not a JAR")
29
+ if hashlib.sha256(data).hexdigest() != SHA256:
30
+ raise RuntimeError("Jackcess download failed SHA-256 verification")
31
+ temp = dependency.with_suffix(".tmp")
32
+ temp.write_bytes(data)
33
+ temp.replace(dependency)
34
+ if hashlib.sha256(dependency.read_bytes()).hexdigest() != SHA256:
35
+ raise RuntimeError("Jackcess JAR checksum mismatch: " + str(dependency))
36
+ stamp = hashlib.sha256(JAVA_SOURCE.read_bytes() + str(dependency).encode() +
37
+ str(dependency.stat().st_size).encode()).hexdigest()
38
+ classes = cache / "classes"
39
+ marker = cache / "compile.sha256"
40
+ if not (classes / "AccessReader.class").exists() or not marker.exists() or marker.read_text() != stamp:
41
+ classes.mkdir(exist_ok=True)
42
+ subprocess.run(["javac", "-encoding", "UTF-8", "-cp", str(dependency), "-d",
43
+ str(classes), str(JAVA_SOURCE)], check=True)
44
+ marker.write_text(stamp)
45
+ return os.pathsep.join((str(classes), str(dependency)))
46
+
47
+
48
+ def run_reader(mode, source, target, jar=None):
49
+ cp = java_classpath(jar)
50
+ subprocess.run(["java", "-cp", cp, "AccessReader", mode,
51
+ str(source), str(target)], check=True)
@@ -0,0 +1,70 @@
1
+ """Optional independent reader checks, never a claim of SSMA execution."""
2
+ import csv
3
+ import hashlib
4
+ import io
5
+ import json
6
+ import shutil
7
+ import subprocess
8
+ from pathlib import Path
9
+
10
+ from .landing import read_manifest
11
+
12
+
13
+ def checksum(file):
14
+ h=hashlib.sha256()
15
+ with Path(file).open('rb') as f:
16
+ for block in iter(lambda:f.read(1024*1024),b''):
17
+ h.update(block)
18
+ return h.hexdigest()
19
+
20
+
21
+ def command(args,timeout=45):
22
+ try:
23
+ return subprocess.run(args,capture_output=True,text=True,timeout=timeout,check=False)
24
+ except (OSError,subprocess.TimeoutExpired) as ex:
25
+ return type('FailedProbe',(),{'returncode':-1,'stdout':'','stderr':type(ex).__name__})()
26
+
27
+
28
+ def probe(source,folder):
29
+ source=Path(source)
30
+ m=read_manifest(folder,require_export=True)
31
+ if checksum(source)!=m['file_sha256']:
32
+ raise ValueError('Source Access file changed since export; do not compare different snapshots')
33
+ steps=[{'tool':'Jackcess','status':'verified','operation':'Read and export native Access data',
34
+ 'evidence':f'{len(m["tables"])} tables, {m["total_rows"]} rows; matching original file fingerprint'}]
35
+ tables_exe=shutil.which('mdb-tables')
36
+ export_exe=shutil.which('mdb-export')
37
+ if not tables_exe:
38
+ steps.append({'tool':'MDB Tools','status':'not_run','operation':'Independent table inventory/export',
39
+ 'evidence':'mdb-tables executable not installed'})
40
+ else:
41
+ result=command([tables_exe,'-1',str(source)])
42
+ names={line.rstrip('\r') for line in result.stdout.splitlines() if line.strip()}
43
+ expected={t['name'] for t in m['tables']}
44
+ if result.returncode!=0 or names!=expected:
45
+ steps.append({'tool':'MDB Tools','status':'failed','operation':'Independent table inventory',
46
+ 'evidence':f'Exit {result.returncode}; expected {len(expected)}, saw {len(names)}; stderr={result.stderr[:200]}'})
47
+ else:
48
+ steps.append({'tool':'MDB Tools','status':'verified','operation':'Independent table inventory',
49
+ 'evidence':f'{len(names)} table names matched the Access inventory'})
50
+ candidates=[t for t in m['tables'] if t['rows']<=10000 and all(c['type'] not in ('OLE','COMPLEX_TYPE') for c in t['columns'])]
51
+ sample=sorted(candidates,key=lambda t:(t['rows'],t['name']))[0] if candidates else None
52
+ if sample and export_exe:
53
+ r=command([export_exe,str(source),sample['name']])
54
+ try:
55
+ csvrows=csv.reader(io.StringIO(r.stdout,newline=''))
56
+ header=next(csvrows)
57
+ count=sum(1 for _ in csvrows)
58
+ match=(r.returncode==0 and count==sample['rows'] and header==[c['name'] for c in sample['columns']])
59
+ except (ValueError,csv.Error,StopIteration):match=False;count=-1
60
+ steps.append({'tool':'MDB Tools','status':'verified' if match else 'failed',
61
+ 'operation':'Independent CSV export of '+sample['name'],
62
+ 'evidence':f'Expected {sample["rows"]} rows; got {count}. Headers also compared. This does not prove full type fidelity.'})
63
+ else:
64
+ steps.append({'tool':'MDB Tools','status':'not_run','operation':'Independent CSV export',
65
+ 'evidence':'No suitable small table or mdb-export not installed'})
66
+ steps.append({'tool':'Microsoft SSMA for Access','status':'not_run',
67
+ 'operation':'Windows assessment / SQL Server schema and data conversion',
68
+ 'evidence':'Requires actual SSMA execution on Windows with Access/ACE and SQL Server/Azure SQL'})
69
+ return {'file_sha256':m['file_sha256'],'probes':steps,
70
+ 'note':'Verified applies only to the stated operation; it does not establish application or VBA migration success.'}
@@ -0,0 +1,92 @@
1
+ Metadata-Version: 2.4
2
+ Name: access-fieldkit
3
+ Version: 0.1.0
4
+ Summary: Read-only Access database inventory, loss-aware export, and migration reconciliation
5
+ Author: mhatipog
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/mhatipog/access-migration-fieldkit
8
+ Project-URL: Issues, https://github.com/mhatipog/access-migration-fieldkit/issues
9
+ Project-URL: Changelog, https://github.com/mhatipog/access-migration-fieldkit/blob/main/CHANGELOG.md
10
+ Keywords: microsoft-access,accdb,mdb,migration,sqlite,database
11
+ Classifier: Development Status :: 3 - Alpha
12
+ Classifier: Environment :: Console
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: Intended Audience :: System Administrators
15
+ Classifier: Operating System :: OS Independent
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Java
18
+ Classifier: Topic :: Database
19
+ Requires-Python: >=3.10
20
+ Description-Content-Type: text/markdown
21
+ License-File: LICENSE
22
+ Dynamic: license-file
23
+
24
+ # Access Fieldkit
25
+
26
+ Tools for the first awkward week of an Access migration: finding out what's in the file, getting the rows out without quietly changing them, and checking that a landing copy still matches.
27
+
28
+ The most common mistake is treating a row count as a completed migration. An Access database can carry keyless tables, embedded SQL tied to forms, four-decimal Currency values, unsaved business rules in VBA, and paths to files that don't live in the database. Moving tables doesn't fix any of that.
29
+
30
+ Fieldkit does **not** claim to convert an Access application. It's a read-only source inspector, exporter, SQLite staging loader, and per-value reconciliation tool. It also includes a PostgreSQL loading example and a runbook for SQL Server/SSMA migrations.
31
+
32
+ ## Quick start
33
+
34
+ Requires Python 3.10+, Java JDK 11+, and an Internet connection for the first run to retrieve Jackcess 5.0.4 from Maven Central. The downloaded JAR is checked against the pinned SHA-256. You can also pass a verified local JAR with `--jackcess-jar`.
35
+
36
+ ```bash
37
+ python -m pip install .
38
+
39
+ # Makes only a manifest. Does not export rows.
40
+ access-fieldkit inspect ./legacy.accdb --out ./work/inventory
41
+
42
+ # Copies table rows into numbered JSONL files, plus a manifest with SHA-256 checksums.
43
+ access-fieldkit export ./legacy.accdb --out ./work/export
44
+
45
+ # What will need attention before someone changes the database or its front end?
46
+ access-fieldkit assess ./work/export --out ./work/assessment.md
47
+
48
+ # Optional: compare the native inventory with MDB Tools, if installed.
49
+ access-fieldkit probe-tools ./legacy.accdb ./work/export --out ./work/tool-probes.json
50
+
51
+ # Creates a new disposable landing database with original table/column names.
52
+ access-fieldkit load-sqlite ./work/export ./work/landing.sqlite
53
+
54
+ # Compares every exported value to the SQLite landing copy.
55
+ access-fieldkit reconcile ./work/export ./work/landing.sqlite
56
+ ```
57
+
58
+ Both `inspect` and `export` open the Access file **read-only**. Existing output folders are not overwritten. The SQLite loader creates its target through a temporary file and refuses to replace an existing database unless `--force` is given; don't point it at production.
59
+
60
+ ## What the export contains
61
+
62
+ `manifest.json` records original table and column names, Access field types, indexes, row counts, saved queries when readable, the source file checksum, and individual table-file checksums. Each `table-0001.jsonl` holds one row per line in the original column order. The file names are numbered on purpose: original Access object names must never become unchecked filesystem paths.
63
+
64
+ Raw rows are kept in order. No artificial primary keys are introduced into source data, no duplicates are collapsed, and NULL is different from an empty string. Currency and decimal values use an explicit text tag so that an intermediate JSON float never rounds the number. Binary values use base64. Unsupported object values cause extraction to fail with a table/row/column location rather than disappear.
65
+
66
+ The SQLite target is a **landing zone**, not a finalized relational schema: it retains every original column and adds an internal source-row ordinal. Access Currency maps to SQLite TEXT, because SQLite has no exact fixed-decimal type. Use a deliberate target schema and business-approved conversions before building a production application.
67
+
68
+ ## Where it fits
69
+
70
+ - **SQLite / local web API:** Run `load-sqlite`, then see `examples/serve_sqlite.py` for a read-only local HTTP view of the landing tables.
71
+ - **PostgreSQL:** `examples/load_postgres.py` uses psycopg 3 to create a fresh schema and load the extracted values, including Currency as `NUMERIC(19,4)`.
72
+ - **SQL Server / Azure SQL:** Use SSMA for the bulk SQL Server path; apply the decisions in [guides/sql-server.md](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/sql-server.md). This repository does not pretend to be SSMA.
73
+ - **Modern application rewrite:** Inventory saved queries here. Use desktop Access on Windows to document forms, events, macros and VBA before moving those workflows into an API or new UI.
74
+
75
+ ## Guides
76
+
77
+ - [Start with an inventory](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/inventory.md)
78
+ - [Currency, NULLs, dates and difficult data](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/data-fidelity.md)
79
+ - [Saved queries, VBA and forms](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/application-rewrite.md)
80
+ - [SQL Server and SSMA](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/sql-server.md)
81
+ - [PostgreSQL landing](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/postgresql.md)
82
+ - [Cutover and rollback](https://github.com/mhatipog/access-migration-fieldkit/blob/main/guides/cutover.md)
83
+
84
+ ## Limits worth knowing
85
+
86
+ Encrypted `.accdb` files may need additional codec support; this tool doesn't try to bypass encryption. Attachments, multivalue columns, linked tables, and Access UI objects require separate inspection. If an Access object is unsupported, don't interpret an incomplete export as proof of a complete application migration. A Windows Access/SSMA run remains necessary for those parts.
87
+
88
+ This code grew out of repeatable migrations of real-format ACE files and documented Microsoft/Jackcess limitations, **not production client migrations**. The included examples use made-up data only. Use backups, permission reviews and a write freeze when assessing an actual customer database.
89
+
90
+ MIT licensed. See [LICENSE](https://github.com/mhatipog/access-migration-fieldkit/blob/main/LICENSE).
91
+
92
+ Maintained by [Vecture](https://vecture.be). Contributions and field reports are welcome via [GitHub issues](https://github.com/mhatipog/access-migration-fieldkit/issues).
@@ -0,0 +1,16 @@
1
+ LICENSE
2
+ README.md
3
+ pyproject.toml
4
+ access_fieldkit/__init__.py
5
+ access_fieldkit/__main__.py
6
+ access_fieldkit/assessment.py
7
+ access_fieldkit/landing.py
8
+ access_fieldkit/runtime.py
9
+ access_fieldkit/tool_probe.py
10
+ access_fieldkit.egg-info/PKG-INFO
11
+ access_fieldkit.egg-info/SOURCES.txt
12
+ access_fieldkit.egg-info/dependency_links.txt
13
+ access_fieldkit.egg-info/entry_points.txt
14
+ access_fieldkit.egg-info/top_level.txt
15
+ access_fieldkit/java/AccessReader.java
16
+ tests/test_roundtrip.py
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ access-fieldkit = access_fieldkit.__main__:main
@@ -0,0 +1 @@
1
+ access_fieldkit
@@ -0,0 +1,37 @@
1
+ [build-system]
2
+ requires = ["setuptools>=68"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "access-fieldkit"
7
+ version = "0.1.0"
8
+ description = "Read-only Access database inventory, loss-aware export, and migration reconciliation"
9
+ readme = "README.md"
10
+ requires-python = ">=3.10"
11
+ license = "MIT"
12
+ authors = [{name = "mhatipog"}]
13
+ keywords = ["microsoft-access", "accdb", "mdb", "migration", "sqlite", "database"]
14
+ classifiers = [
15
+ "Development Status :: 3 - Alpha",
16
+ "Environment :: Console",
17
+ "Intended Audience :: Developers",
18
+ "Intended Audience :: System Administrators",
19
+ "Operating System :: OS Independent",
20
+ "Programming Language :: Python :: 3",
21
+ "Programming Language :: Java",
22
+ "Topic :: Database",
23
+ ]
24
+
25
+ [project.urls]
26
+ Homepage = "https://github.com/mhatipog/access-migration-fieldkit"
27
+ Issues = "https://github.com/mhatipog/access-migration-fieldkit/issues"
28
+ Changelog = "https://github.com/mhatipog/access-migration-fieldkit/blob/main/CHANGELOG.md"
29
+
30
+ [project.scripts]
31
+ access-fieldkit = "access_fieldkit.__main__:main"
32
+
33
+ [tool.setuptools.package-data]
34
+ access_fieldkit = ["java/*.java"]
35
+
36
+ [tool.setuptools]
37
+ packages = ["access_fieldkit"]
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,91 @@
1
+ """End-to-end: synthetic ACCDB -> inspect -> export -> assess -> load-sqlite -> reconcile.
2
+
3
+ Needs Java (javac) and, on the first run, network access to fetch the pinned Jackcess JAR
4
+ (or ACCESS_FIELDKIT_JACKCESS_JAR pointing at a verified local copy). Run: python -m unittest discover -s tests
5
+ """
6
+ import hashlib
7
+ import json
8
+ import os
9
+ import sqlite3
10
+ from contextlib import closing
11
+ import subprocess
12
+ import sys
13
+ import tempfile
14
+ import unittest
15
+ from pathlib import Path
16
+
17
+ from access_fieldkit.runtime import java_classpath
18
+
19
+ FIXTURES = Path(__file__).parent / "fixtures"
20
+ JAR = os.environ.get("ACCESS_FIELDKIT_JACKCESS_JAR") or None
21
+
22
+
23
+ def cli(*args):
24
+ extra = ["--jackcess-jar", JAR] if JAR else []
25
+ return subprocess.run([sys.executable, "-m", "access_fieldkit", *extra, *args],
26
+ check=True, capture_output=True, text=True)
27
+
28
+
29
+ class RoundTrip(unittest.TestCase):
30
+ @classmethod
31
+ def setUpClass(cls):
32
+ cls.tmp = tempfile.TemporaryDirectory(ignore_cleanup_errors=True)
33
+ cls.work = Path(cls.tmp.name)
34
+ classpath = java_classpath(JAR)
35
+ jackcess = classpath.split(os.pathsep)[-1]
36
+ build = cls.work / "fixture-classes"
37
+ build.mkdir()
38
+ subprocess.run(["javac", "-encoding", "UTF-8", "-cp", jackcess, "-d", str(build),
39
+ str(FIXTURES / "MakeSample.java")], check=True)
40
+ cls.source = cls.work / "sample.accdb"
41
+ subprocess.run(["java", "-cp", os.pathsep.join((str(build), jackcess)), "MakeSample",
42
+ str(cls.source)], check=True)
43
+ cls.source_sha = hashlib.sha256(cls.source.read_bytes()).hexdigest()
44
+
45
+ @classmethod
46
+ def tearDownClass(cls):
47
+ cls.tmp.cleanup()
48
+
49
+ def test_pipeline(self):
50
+ cli("inspect", str(self.source), "--out", str(self.work / "inventory"))
51
+ inventory = json.loads((self.work / "inventory" / "manifest.json").read_text(encoding="utf-8"))
52
+ names = {t["name"] for t in inventory["tables"]}
53
+ self.assertEqual(names, {"Klanten", "Import log"})
54
+ self.assertFalse(any((self.work / "inventory").glob("table-*.jsonl")), "inspect must not export rows")
55
+
56
+ export = self.work / "export"
57
+ cli("export", str(self.source), "--out", str(export))
58
+ manifest = json.loads((export / "manifest.json").read_text(encoding="utf-8"))
59
+ rows = {t["name"]: t["rows"] for t in manifest["tables"]}
60
+ self.assertEqual(rows, {"Klanten": 3, "Import log": 3}, "duplicates in a keyless table must be kept")
61
+
62
+ cli("assess", str(export), "--out", str(self.work / "assessment.md"))
63
+ self.assertIn("Import log", (self.work / "assessment.md").read_text(encoding="utf-8"))
64
+
65
+ landing = self.work / "landing.sqlite"
66
+ cli("load-sqlite", str(export), str(landing))
67
+ result = cli("reconcile", str(export), str(landing))
68
+ self.assertEqual(result.returncode, 0)
69
+
70
+ with closing(sqlite3.connect(landing)) as db, db:
71
+ saldo = [r[0] for r in db.execute('SELECT "Saldo" FROM "Klanten" ORDER BY "ID"')]
72
+ gemeente = [r[0] for r in db.execute('SELECT "Gemeente" FROM "Klanten" ORDER BY "ID"')]
73
+ self.assertEqual(saldo[:2], ["1234.5678", "-0.0001"], "Currency must keep four decimals exactly")
74
+ self.assertEqual(gemeente, ["Liège", "", None], "NULL and empty text must stay different")
75
+
76
+ self.assertEqual(hashlib.sha256(self.source.read_bytes()).hexdigest(), self.source_sha,
77
+ "the source file must not change")
78
+
79
+ def test_reconcile_catches_a_changed_value(self):
80
+ export = self.work / "export-tamper"
81
+ cli("export", str(self.source), "--out", str(export))
82
+ landing = self.work / "tamper.sqlite"
83
+ cli("load-sqlite", str(export), str(landing))
84
+ with closing(sqlite3.connect(landing)) as db, db:
85
+ db.execute('UPDATE "Klanten" SET "Saldo" = \'1234.5600\' WHERE "Naam" = \'Voorbeeld BV\'')
86
+ with self.assertRaises(subprocess.CalledProcessError):
87
+ cli("reconcile", str(export), str(landing))
88
+
89
+
90
+ if __name__ == "__main__":
91
+ unittest.main()