astro-swiper 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- astro_swiper-0.1.0/PKG-INFO +193 -0
- astro_swiper-0.1.0/astro_swiper/__init__.py +3 -0
- astro_swiper-0.1.0/astro_swiper/_cli.py +15 -0
- astro_swiper-0.1.0/astro_swiper/classifier.py +218 -0
- astro_swiper-0.1.0/astro_swiper/imgs/background.png +0 -0
- astro_swiper-0.1.0/astro_swiper/storage.py +149 -0
- astro_swiper-0.1.0/astro_swiper/web.py +201 -0
- astro_swiper-0.1.0/astro_swiper.egg-info/PKG-INFO +193 -0
- astro_swiper-0.1.0/astro_swiper.egg-info/SOURCES.txt +15 -0
- astro_swiper-0.1.0/astro_swiper.egg-info/dependency_links.txt +1 -0
- astro_swiper-0.1.0/astro_swiper.egg-info/entry_points.txt +2 -0
- astro_swiper-0.1.0/astro_swiper.egg-info/requires.txt +6 -0
- astro_swiper-0.1.0/astro_swiper.egg-info/top_level.txt +1 -0
- astro_swiper-0.1.0/pyproject.toml +29 -0
- astro_swiper-0.1.0/readme.md +179 -0
- astro_swiper-0.1.0/setup.cfg +4 -0
- astro_swiper-0.1.0/setup.py +3 -0
|
@@ -0,0 +1,193 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: astro-swiper
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Web-based interactive FITS triplet classifier for astronomical image labelling
|
|
5
|
+
License: MIT
|
|
6
|
+
Requires-Python: >=3.8
|
|
7
|
+
Description-Content-Type: text/markdown
|
|
8
|
+
Requires-Dist: flask>=2.0
|
|
9
|
+
Requires-Dist: flask-socketio>=5.0
|
|
10
|
+
Requires-Dist: astropy>=5.0
|
|
11
|
+
Requires-Dist: matplotlib>=3.5
|
|
12
|
+
Requires-Dist: numpy>=1.21
|
|
13
|
+
Requires-Dist: pyyaml>=6.0
|
|
14
|
+
|
|
15
|
+
# Astro Swiper — Usage
|
|
16
|
+
|
|
17
|
+
## Installation
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
pip install flask flask-socketio astropy matplotlib numpy pyyaml
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## Running
|
|
26
|
+
|
|
27
|
+
### 1. As a Python import (recommended)
|
|
28
|
+
|
|
29
|
+
```python
|
|
30
|
+
from astro_swiper_web import AstroSwiper
|
|
31
|
+
|
|
32
|
+
AstroSwiper('config.yaml').run()
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
### 2. With a custom triplet loader
|
|
36
|
+
|
|
37
|
+
If your files don't follow the default `*scicutout / *subcutout / *refcutout` naming convention, provide a `triplet_loader` function. It receives `input_dir` from the config (or `None` if omitted) and must return a list of `[sub_path, sci_path, ref_path]` triplets.
|
|
38
|
+
|
|
39
|
+
```python
|
|
40
|
+
from astro_swiper_web import AstroSwiper
|
|
41
|
+
|
|
42
|
+
def my_loader(input_dir):
|
|
43
|
+
# build and return your triplets however you like
|
|
44
|
+
import glob, re
|
|
45
|
+
sci_files = sorted(glob.glob(f"{input_dir}/*_science.fits"))
|
|
46
|
+
triplets = []
|
|
47
|
+
for sci in sci_files:
|
|
48
|
+
base = re.sub(r'_science\.fits$', '', sci)
|
|
49
|
+
sub = base + '_difference.fits'
|
|
50
|
+
ref = base + '_template.fits'
|
|
51
|
+
if os.path.exists(sub) and os.path.exists(ref):
|
|
52
|
+
triplets.append([sub, sci, ref])
|
|
53
|
+
return triplets
|
|
54
|
+
|
|
55
|
+
AstroSwiper('config.yaml', triplet_loader=my_loader).run()
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
`input_dir` becomes optional in config when a loader is supplied — you can omit it entirely if your loader doesn't need it.
|
|
59
|
+
|
|
60
|
+
### 4. With an inline config dict (no file needed)
|
|
61
|
+
|
|
62
|
+
```python
|
|
63
|
+
from astro_swiper_web import AstroSwiper
|
|
64
|
+
|
|
65
|
+
AstroSwiper({
|
|
66
|
+
'input_dir': '/data/cutouts/',
|
|
67
|
+
'back_button': 'up',
|
|
68
|
+
'port': 5000,
|
|
69
|
+
'resume': True,
|
|
70
|
+
'overwrite': False,
|
|
71
|
+
'storage': {'backend': 'sqlite', 'db': 'classifications.db'},
|
|
72
|
+
'keybinds': {
|
|
73
|
+
'a': 'noise',
|
|
74
|
+
'e': 'streaks',
|
|
75
|
+
'd': 'dots',
|
|
76
|
+
'1': 'small',
|
|
77
|
+
'2': 'medium',
|
|
78
|
+
},
|
|
79
|
+
}).run()
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
### 5. From the command line
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
python astro_swiper_web.py # uses config.yaml in current directory
|
|
86
|
+
python astro_swiper_web.py my_cfg.yaml # explicit config path
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Then open **http://localhost:5000** in a browser.
|
|
90
|
+
|
|
91
|
+
**Over SSH** (no X11 needed):
|
|
92
|
+
```bash
|
|
93
|
+
ssh -L 5000:localhost:5000 user@host
|
|
94
|
+
# then open http://localhost:5000 locally
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## config.yaml reference
|
|
100
|
+
|
|
101
|
+
| Key | Default | Description |
|
|
102
|
+
|-----|---------|-------------|
|
|
103
|
+
| `input_dir` | *(required)* | Directory containing `.fits` or `.fits.gz` cutout triplets |
|
|
104
|
+
| `back_button` | `left` | Key that undoes the last classification |
|
|
105
|
+
| `port` | `5000` | Port the web server listens on |
|
|
106
|
+
| `resume` | `true` | Skip already-classified triplets on startup |
|
|
107
|
+
| `overwrite` | `false` | Wipe all saved classifications and start fresh |
|
|
108
|
+
| `storage.backend` | `sqlite` | Storage format: `sqlite`, `csv`, or `txt` |
|
|
109
|
+
| `keybinds` | *(required)* | Map of key → label (or file path for `txt` backend) |
|
|
110
|
+
|
|
111
|
+
---
|
|
112
|
+
|
|
113
|
+
## Storage backends
|
|
114
|
+
|
|
115
|
+
### SQLite (recommended)
|
|
116
|
+
|
|
117
|
+
```yaml
|
|
118
|
+
storage:
|
|
119
|
+
backend: sqlite
|
|
120
|
+
db: training_sets/classifications.db
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
Single file, atomic writes, safe against crashes. Query results with pandas:
|
|
124
|
+
|
|
125
|
+
```python
|
|
126
|
+
import sqlite3, pandas as pd
|
|
127
|
+
df = pd.read_sql(
|
|
128
|
+
"SELECT * FROM classifications",
|
|
129
|
+
sqlite3.connect("training_sets/classifications.db")
|
|
130
|
+
)
|
|
131
|
+
counts = df['label'].value_counts()
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
### CSV
|
|
135
|
+
|
|
136
|
+
```yaml
|
|
137
|
+
storage:
|
|
138
|
+
backend: csv
|
|
139
|
+
file: training_sets/classifications.csv
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
One row per triplet with columns `sub_path, sci_path, ref_path, label`. Easy to open in Excel or pandas:
|
|
143
|
+
|
|
144
|
+
```python
|
|
145
|
+
import pandas as pd
|
|
146
|
+
df = pd.read_csv("training_sets/classifications.csv")
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
### Txt (legacy)
|
|
150
|
+
|
|
151
|
+
```yaml
|
|
152
|
+
storage:
|
|
153
|
+
backend: txt
|
|
154
|
+
already_classified: training_sets/already_classified.txt
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
One `.txt` file per category; keybind values must be **file paths** (not labels):
|
|
158
|
+
|
|
159
|
+
```yaml
|
|
160
|
+
keybinds:
|
|
161
|
+
a: training_sets/noise.txt
|
|
162
|
+
c: training_sets/skips.txt
|
|
163
|
+
...
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
Each file contains triplet paths, three lines per entry (sub, sci, ref).
|
|
167
|
+
|
|
168
|
+
---
|
|
169
|
+
|
|
170
|
+
## Input data format
|
|
171
|
+
|
|
172
|
+
Each triplet is a set of three co-registered FITS cutout files sharing a common basename:
|
|
173
|
+
|
|
174
|
+
```
|
|
175
|
+
<basename>scicutout.fits[.gz]
|
|
176
|
+
<basename>subcutout.fits[.gz]
|
|
177
|
+
<basename>refcutout.fits[.gz]
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
All files must be in the same flat directory (`input_dir`). Both `.fits` and `.fits.gz` are supported.
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## Controls
|
|
185
|
+
|
|
186
|
+
| Key | Action |
|
|
187
|
+
|-----|--------|
|
|
188
|
+
| *(configured keybinds)* | Classify current triplet |
|
|
189
|
+
| `back_button` (default `left`) | Undo last classification |
|
|
190
|
+
| `Shift+↑` | Increase contrast (narrow display window) |
|
|
191
|
+
| `Shift+↓` | Decrease contrast (widen display window) |
|
|
192
|
+
| `Shift+→` | Increase brightness (shift window up) |
|
|
193
|
+
| `Shift+←` | Decrease brightness (shift window down) |
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
"""CLI entry point for astro-swiper."""
|
|
2
|
+
|
|
3
|
+
import argparse
|
|
4
|
+
from astro_swiper.web import AstroSwiper
|
|
5
|
+
|
|
6
|
+
|
|
7
|
+
def main():
|
|
8
|
+
parser = argparse.ArgumentParser(
|
|
9
|
+
description='Astro Swiper — web-based FITS triplet classifier'
|
|
10
|
+
)
|
|
11
|
+
parser.add_argument(
|
|
12
|
+
'config', nargs='?', default='config.yaml',
|
|
13
|
+
help='Path to YAML config file (default: config.yaml)',
|
|
14
|
+
)
|
|
15
|
+
AstroSwiper(parser.parse_args().config).run()
|
|
@@ -0,0 +1,218 @@
|
|
|
1
|
+
"""classifier.py — TripletClassifier: FITS loading, rendering, and key handling."""
|
|
2
|
+
|
|
3
|
+
import os, io, base64, gzip, shutil, tempfile, threading
|
|
4
|
+
|
|
5
|
+
import numpy as np
|
|
6
|
+
from astropy.io import fits
|
|
7
|
+
from astropy.visualization import ZScaleInterval
|
|
8
|
+
import matplotlib
|
|
9
|
+
matplotlib.use('Agg')
|
|
10
|
+
import matplotlib.pyplot as plt
|
|
11
|
+
import matplotlib.gridspec as gridspec
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
class TripletClassifier:
|
|
15
|
+
def __init__(self, keybinds, back_button, storage, socketio,
|
|
16
|
+
resume=True, overwrite=False):
|
|
17
|
+
self.keybinds = keybinds
|
|
18
|
+
self.back_button = back_button
|
|
19
|
+
self.resume = resume
|
|
20
|
+
self._storage = storage
|
|
21
|
+
self._socketio = socketio # injected — no module-level global needed
|
|
22
|
+
|
|
23
|
+
self.triplets = []
|
|
24
|
+
self.index = 0
|
|
25
|
+
self._lock = threading.Lock()
|
|
26
|
+
self._zscale = ZScaleInterval()
|
|
27
|
+
|
|
28
|
+
self.vmin = 0.0
|
|
29
|
+
self.vmax = 1.0
|
|
30
|
+
self.step_pct = 0.1
|
|
31
|
+
|
|
32
|
+
self._imgs_idx = None
|
|
33
|
+
self._imgs = None
|
|
34
|
+
self._b64 = None
|
|
35
|
+
self._b64_key = None
|
|
36
|
+
self._pf_lock = threading.Lock()
|
|
37
|
+
self._pf = None
|
|
38
|
+
|
|
39
|
+
if overwrite:
|
|
40
|
+
self._storage.clear()
|
|
41
|
+
self.pre_classified = self._storage.get_classified() if resume else set()
|
|
42
|
+
|
|
43
|
+
# ── FITS I/O ──────────────────────────────────────────────────────────────
|
|
44
|
+
|
|
45
|
+
def _load_fits(self, path):
|
|
46
|
+
if path.endswith('.fits.gz'):
|
|
47
|
+
with gzip.open(path, 'rb') as f_in:
|
|
48
|
+
with tempfile.NamedTemporaryFile(suffix='.fits', delete=False) as f_out:
|
|
49
|
+
shutil.copyfileobj(f_in, f_out)
|
|
50
|
+
tmp = f_out.name
|
|
51
|
+
try:
|
|
52
|
+
with fits.open(tmp) as h:
|
|
53
|
+
return np.nan_to_num(h[0].data)
|
|
54
|
+
finally:
|
|
55
|
+
os.unlink(tmp)
|
|
56
|
+
with fits.open(path) as h:
|
|
57
|
+
return np.nan_to_num(h[0].data)
|
|
58
|
+
|
|
59
|
+
def _load_triplet(self, triplet):
|
|
60
|
+
return tuple(self._zscale(self._load_fits(p)) for p in triplet)
|
|
61
|
+
|
|
62
|
+
# ── Rendering ─────────────────────────────────────────────────────────────
|
|
63
|
+
|
|
64
|
+
def _render(self, imgs):
|
|
65
|
+
fig = plt.figure(figsize=(21, 7), facecolor='black')
|
|
66
|
+
gs = gridspec.GridSpec(1, 3, wspace=0.02)
|
|
67
|
+
for i, (title, img) in enumerate(zip(('SUB', 'SCI', 'REF'), imgs)):
|
|
68
|
+
ax = fig.add_subplot(gs[i])
|
|
69
|
+
ax.set_title(title, fontsize=16, color='white')
|
|
70
|
+
ax.axis('off')
|
|
71
|
+
ax.imshow(img, cmap='gray', origin='lower', vmin=self.vmin, vmax=self.vmax)
|
|
72
|
+
plt.tight_layout(pad=0.4)
|
|
73
|
+
buf = io.BytesIO()
|
|
74
|
+
fig.savefig(buf, format='png', bbox_inches='tight', facecolor='black', dpi=100)
|
|
75
|
+
plt.close(fig)
|
|
76
|
+
buf.seek(0)
|
|
77
|
+
return base64.b64encode(buf.read()).decode()
|
|
78
|
+
|
|
79
|
+
def _get_b64(self):
|
|
80
|
+
idx = self.index
|
|
81
|
+
if self._imgs_idx != idx:
|
|
82
|
+
with self._pf_lock:
|
|
83
|
+
if self._pf and self._pf[0] == idx:
|
|
84
|
+
self._imgs, self._pf = self._pf[1], None
|
|
85
|
+
else:
|
|
86
|
+
self._imgs = self._load_triplet(self.triplets[idx])
|
|
87
|
+
self._imgs_idx = idx
|
|
88
|
+
self._b64 = None
|
|
89
|
+
key = (idx, round(self.vmin, 8), round(self.vmax, 8))
|
|
90
|
+
if self._b64_key != key:
|
|
91
|
+
self._b64 = self._render(self._imgs)
|
|
92
|
+
self._b64_key = key
|
|
93
|
+
return self._b64
|
|
94
|
+
|
|
95
|
+
# ── Prefetch ──────────────────────────────────────────────────────────────
|
|
96
|
+
|
|
97
|
+
def _prefetch_next(self):
|
|
98
|
+
nxt = self.index
|
|
99
|
+
while nxt < len(self.triplets):
|
|
100
|
+
if self.resume and self.triplets[nxt][1] in self.pre_classified:
|
|
101
|
+
nxt += 1
|
|
102
|
+
else:
|
|
103
|
+
break
|
|
104
|
+
if nxt >= len(self.triplets):
|
|
105
|
+
return
|
|
106
|
+
target, triplet = nxt, self.triplets[nxt]
|
|
107
|
+
def _work():
|
|
108
|
+
imgs = self._load_triplet(triplet)
|
|
109
|
+
with self._pf_lock:
|
|
110
|
+
self._pf = (target, imgs)
|
|
111
|
+
threading.Thread(target=_work, daemon=True).start()
|
|
112
|
+
|
|
113
|
+
# ── Navigation ────────────────────────────────────────────────────────────
|
|
114
|
+
|
|
115
|
+
def _skip_classified(self):
|
|
116
|
+
while self.index < len(self.triplets):
|
|
117
|
+
if self.resume and self.triplets[self.index][1] in self.pre_classified:
|
|
118
|
+
self.index += 1
|
|
119
|
+
else:
|
|
120
|
+
break
|
|
121
|
+
|
|
122
|
+
# ── Emit ──────────────────────────────────────────────────────────────────
|
|
123
|
+
|
|
124
|
+
def _emit_current(self, to=None):
|
|
125
|
+
if self.index >= len(self.triplets):
|
|
126
|
+
self._socketio.emit('done', {'message': f'All {len(self.triplets)} triplets done!'})
|
|
127
|
+
return
|
|
128
|
+
triplet = self.triplets[self.index]
|
|
129
|
+
payload = {
|
|
130
|
+
'image': self._get_b64(),
|
|
131
|
+
'filename': os.path.basename(triplet[1]),
|
|
132
|
+
'progress': f'{self.index + 1} / {len(self.triplets)}',
|
|
133
|
+
}
|
|
134
|
+
self._socketio.emit('update', payload, to=to) if to else \
|
|
135
|
+
self._socketio.emit('update', payload)
|
|
136
|
+
|
|
137
|
+
def send_current(self, to=None):
|
|
138
|
+
sid = to
|
|
139
|
+
def _work():
|
|
140
|
+
self._socketio.emit('loading')
|
|
141
|
+
with self._lock:
|
|
142
|
+
self._emit_current(to=sid)
|
|
143
|
+
threading.Thread(target=_work, daemon=True).start()
|
|
144
|
+
|
|
145
|
+
# ── Key handling ──────────────────────────────────────────────────────────
|
|
146
|
+
|
|
147
|
+
def handle_key(self, key):
|
|
148
|
+
def _work():
|
|
149
|
+
with self._lock:
|
|
150
|
+
if key.startswith('shift+'):
|
|
151
|
+
self._apply_scaling(key)
|
|
152
|
+
self._emit_current()
|
|
153
|
+
elif key == self.back_button:
|
|
154
|
+
self._undo()
|
|
155
|
+
self._emit_current()
|
|
156
|
+
elif key in self.keybinds:
|
|
157
|
+
self._classify(key)
|
|
158
|
+
self._emit_current()
|
|
159
|
+
self._prefetch_next()
|
|
160
|
+
threading.Thread(target=_work, daemon=True).start()
|
|
161
|
+
|
|
162
|
+
def _apply_scaling(self, key):
|
|
163
|
+
rng = self.vmax - self.vmin
|
|
164
|
+
step = rng * self.step_pct
|
|
165
|
+
if key == 'shift+up': self.vmax -= step
|
|
166
|
+
elif key == 'shift+down': self.vmax += step
|
|
167
|
+
elif key == 'shift+right': self.vmin += step; self.vmax += step
|
|
168
|
+
elif key == 'shift+left': self.vmin -= step; self.vmax -= step
|
|
169
|
+
self._b64 = None
|
|
170
|
+
|
|
171
|
+
def _classify(self, key):
|
|
172
|
+
sub, sci, ref = self.triplets[self.index]
|
|
173
|
+
label = self.keybinds[key]
|
|
174
|
+
self._storage.save(sub, sci, ref, key, label)
|
|
175
|
+
self.pre_classified.add(sci)
|
|
176
|
+
print(f"[{self.index + 1}/{len(self.triplets)}] {os.path.basename(sci)} → {label}")
|
|
177
|
+
self.index += 1
|
|
178
|
+
self._skip_classified()
|
|
179
|
+
|
|
180
|
+
def _undo(self):
|
|
181
|
+
last_sci = self._storage.undo()
|
|
182
|
+
if last_sci is None:
|
|
183
|
+
print("Nothing to undo.")
|
|
184
|
+
return
|
|
185
|
+
self.pre_classified.discard(last_sci)
|
|
186
|
+
for i, triplet in enumerate(self.triplets):
|
|
187
|
+
if triplet[1] == last_sci:
|
|
188
|
+
self.index = i
|
|
189
|
+
break
|
|
190
|
+
else:
|
|
191
|
+
self.index = max(0, self.index - 1)
|
|
192
|
+
print(f"Undid: {os.path.basename(last_sci)}")
|
|
193
|
+
|
|
194
|
+
# ── Setup ─────────────────────────────────────────────────────────────────
|
|
195
|
+
|
|
196
|
+
def load_directory(self, directory_path, triplet_loader=None):
|
|
197
|
+
if triplet_loader is not None:
|
|
198
|
+
triplets = list(triplet_loader(directory_path))
|
|
199
|
+
else:
|
|
200
|
+
all_files = [
|
|
201
|
+
os.path.join(directory_path, fn)
|
|
202
|
+
for fn in os.listdir(directory_path)
|
|
203
|
+
if os.path.isfile(os.path.join(directory_path, fn))
|
|
204
|
+
]
|
|
205
|
+
triplets = []
|
|
206
|
+
for sci_sfx, sub_sfx, ref_sfx in [
|
|
207
|
+
('scicutout.fits.gz', 'subcutout.fits.gz', 'refcutout.fits.gz'),
|
|
208
|
+
('scicutout.fits', 'subcutout.fits', 'refcutout.fits'),
|
|
209
|
+
]:
|
|
210
|
+
for f in all_files:
|
|
211
|
+
if f.endswith(sci_sfx):
|
|
212
|
+
base = f[:-len(sci_sfx)]
|
|
213
|
+
triplet = [base + sub_sfx, f, base + ref_sfx]
|
|
214
|
+
if all(os.path.exists(p) for p in triplet):
|
|
215
|
+
triplets.append(triplet)
|
|
216
|
+
self.triplets = sorted(triplets, key=lambda t: t[1])
|
|
217
|
+
self._skip_classified()
|
|
218
|
+
print(f"Loaded {len(self.triplets)} triplets; resuming at index {self.index}.")
|
|
Binary file
|
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
"""storage.py — Classification storage backends for astro_swiper."""
|
|
2
|
+
|
|
3
|
+
import os, sqlite3, csv
|
|
4
|
+
|
|
5
|
+
|
|
6
|
+
class StorageBackend:
|
|
7
|
+
def get_classified(self) -> set: raise NotImplementedError
|
|
8
|
+
def save(self, sub, sci, ref, key, label): raise NotImplementedError
|
|
9
|
+
def undo(self) -> 'str | None': raise NotImplementedError
|
|
10
|
+
def clear(self): raise NotImplementedError
|
|
11
|
+
def close(self): pass
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
class SQLiteBackend(StorageBackend):
|
|
15
|
+
def __init__(self, db_path):
|
|
16
|
+
os.makedirs(os.path.dirname(db_path) or '.', exist_ok=True)
|
|
17
|
+
self._db = sqlite3.connect(db_path, check_same_thread=False)
|
|
18
|
+
self._db.execute("""
|
|
19
|
+
CREATE TABLE IF NOT EXISTS classifications (
|
|
20
|
+
rowid INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
21
|
+
sci_path TEXT UNIQUE,
|
|
22
|
+
sub_path TEXT,
|
|
23
|
+
ref_path TEXT,
|
|
24
|
+
label TEXT
|
|
25
|
+
)
|
|
26
|
+
""")
|
|
27
|
+
self._db.commit()
|
|
28
|
+
|
|
29
|
+
def get_classified(self):
|
|
30
|
+
return {r[0] for r in self._db.execute(
|
|
31
|
+
"SELECT sci_path FROM classifications"
|
|
32
|
+
).fetchall()}
|
|
33
|
+
|
|
34
|
+
def save(self, sub, sci, ref, key, label):
|
|
35
|
+
self._db.execute(
|
|
36
|
+
"INSERT OR REPLACE INTO classifications "
|
|
37
|
+
"(sci_path, sub_path, ref_path, label) VALUES (?, ?, ?, ?)",
|
|
38
|
+
(sci, sub, ref, label),
|
|
39
|
+
)
|
|
40
|
+
self._db.commit()
|
|
41
|
+
|
|
42
|
+
def undo(self):
|
|
43
|
+
row = self._db.execute(
|
|
44
|
+
"SELECT sci_path FROM classifications ORDER BY rowid DESC LIMIT 1"
|
|
45
|
+
).fetchone()
|
|
46
|
+
if row is None:
|
|
47
|
+
return None
|
|
48
|
+
self._db.execute("DELETE FROM classifications WHERE sci_path = ?", (row[0],))
|
|
49
|
+
self._db.commit()
|
|
50
|
+
return row[0]
|
|
51
|
+
|
|
52
|
+
def clear(self):
|
|
53
|
+
self._db.execute("DELETE FROM classifications")
|
|
54
|
+
self._db.commit()
|
|
55
|
+
|
|
56
|
+
def close(self):
|
|
57
|
+
self._db.close()
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
class CSVBackend(StorageBackend):
|
|
61
|
+
def __init__(self, csv_path):
|
|
62
|
+
os.makedirs(os.path.dirname(csv_path) or '.', exist_ok=True)
|
|
63
|
+
self._path = csv_path
|
|
64
|
+
if not os.path.exists(csv_path):
|
|
65
|
+
with open(csv_path, 'w', newline='') as f:
|
|
66
|
+
csv.writer(f).writerow(['sub_path', 'sci_path', 'ref_path', 'label'])
|
|
67
|
+
|
|
68
|
+
def get_classified(self):
|
|
69
|
+
with open(self._path, newline='') as f:
|
|
70
|
+
return {row['sci_path'] for row in csv.DictReader(f)}
|
|
71
|
+
|
|
72
|
+
def save(self, sub, sci, ref, key, label):
|
|
73
|
+
with open(self._path, 'a', newline='') as f:
|
|
74
|
+
csv.writer(f).writerow([sub, sci, ref, label])
|
|
75
|
+
|
|
76
|
+
def undo(self):
|
|
77
|
+
with open(self._path, newline='') as f:
|
|
78
|
+
rows = list(csv.reader(f))
|
|
79
|
+
if len(rows) <= 1:
|
|
80
|
+
return None
|
|
81
|
+
last_sci = rows[-1][1]
|
|
82
|
+
with open(self._path, 'w', newline='') as f:
|
|
83
|
+
csv.writer(f).writerows(rows[:-1])
|
|
84
|
+
return last_sci
|
|
85
|
+
|
|
86
|
+
def clear(self):
|
|
87
|
+
with open(self._path, 'w', newline='') as f:
|
|
88
|
+
csv.writer(f).writerow(['sub_path', 'sci_path', 'ref_path', 'label'])
|
|
89
|
+
|
|
90
|
+
|
|
91
|
+
class TxtBackend(StorageBackend):
|
|
92
|
+
"""Original multi-file format. keybinds must be {key: filepath}."""
|
|
93
|
+
def __init__(self, keybinds, already_classified_path):
|
|
94
|
+
self._keybinds = keybinds
|
|
95
|
+
self._ac_path = already_classified_path
|
|
96
|
+
os.makedirs(os.path.dirname(already_classified_path) or '.', exist_ok=True)
|
|
97
|
+
if not os.path.exists(already_classified_path):
|
|
98
|
+
open(already_classified_path, 'w').close()
|
|
99
|
+
for path in keybinds.values():
|
|
100
|
+
os.makedirs(os.path.dirname(path) or '.', exist_ok=True)
|
|
101
|
+
|
|
102
|
+
def get_classified(self):
|
|
103
|
+
with open(self._ac_path) as f:
|
|
104
|
+
lines = [l.strip() for l in f if l.strip()]
|
|
105
|
+
return set(lines[i] for i in range(0, len(lines), 2))
|
|
106
|
+
|
|
107
|
+
def save(self, sub, sci, ref, key, label):
|
|
108
|
+
with open(self._keybinds[key], 'a') as f:
|
|
109
|
+
f.write(f"{sub}\n{sci}\n{ref}\n")
|
|
110
|
+
with open(self._ac_path, 'a') as f:
|
|
111
|
+
f.write(f"{sci}\n{key}\n")
|
|
112
|
+
|
|
113
|
+
def undo(self):
|
|
114
|
+
with open(self._ac_path) as f:
|
|
115
|
+
lines = f.readlines()
|
|
116
|
+
if len(lines) < 2:
|
|
117
|
+
return None
|
|
118
|
+
last_key = lines[-1].strip()
|
|
119
|
+
last_sci = lines[-2].strip()
|
|
120
|
+
with open(self._ac_path, 'w') as f:
|
|
121
|
+
f.writelines(lines[:-2])
|
|
122
|
+
cat_path = self._keybinds[last_key]
|
|
123
|
+
with open(cat_path) as f:
|
|
124
|
+
cat_lines = f.readlines()
|
|
125
|
+
with open(cat_path, 'w') as f:
|
|
126
|
+
f.writelines(cat_lines[:-3])
|
|
127
|
+
return last_sci
|
|
128
|
+
|
|
129
|
+
def clear(self):
|
|
130
|
+
open(self._ac_path, 'w').close()
|
|
131
|
+
for path in self._keybinds.values():
|
|
132
|
+
open(path, 'w').close()
|
|
133
|
+
|
|
134
|
+
|
|
135
|
+
def make_backend(cfg, keybinds) -> StorageBackend:
|
|
136
|
+
storage_cfg = cfg.get('storage', {})
|
|
137
|
+
backend = storage_cfg.get('backend', 'sqlite').lower()
|
|
138
|
+
if backend == 'sqlite':
|
|
139
|
+
return SQLiteBackend(storage_cfg.get('db', 'classifications.db'))
|
|
140
|
+
elif backend == 'csv':
|
|
141
|
+
return CSVBackend(storage_cfg.get('file', 'classifications.csv'))
|
|
142
|
+
elif backend == 'txt':
|
|
143
|
+
return TxtBackend(
|
|
144
|
+
keybinds=keybinds,
|
|
145
|
+
already_classified_path=storage_cfg.get(
|
|
146
|
+
'already_classified', 'training_sets/already_classified.txt'
|
|
147
|
+
),
|
|
148
|
+
)
|
|
149
|
+
raise ValueError(f"Unknown storage backend '{backend}'. Choose sqlite, csv, or txt.")
|
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
"""web.py — AstroSwiper: web-based FITS triplet classifier.
|
|
2
|
+
|
|
3
|
+
Usage as a module:
|
|
4
|
+
from astro_swiper import AstroSwiper
|
|
5
|
+
AstroSwiper('config.yaml').run()
|
|
6
|
+
|
|
7
|
+
Also accepts a pre-loaded dict instead of a path:
|
|
8
|
+
AstroSwiper({'input_dir': '...', 'keybinds': {...}, ...}).run()
|
|
9
|
+
"""
|
|
10
|
+
|
|
11
|
+
import os
|
|
12
|
+
import yaml
|
|
13
|
+
from flask import Flask, render_template_string, request, send_file
|
|
14
|
+
from flask_socketio import SocketIO, emit
|
|
15
|
+
|
|
16
|
+
from astro_swiper.storage import make_backend
|
|
17
|
+
from astro_swiper.classifier import TripletClassifier
|
|
18
|
+
|
|
19
|
+
# ---------------------------------------------------------------------------
|
|
20
|
+
# Browser UI
|
|
21
|
+
# ---------------------------------------------------------------------------
|
|
22
|
+
|
|
23
|
+
HTML = r"""<!DOCTYPE html>
|
|
24
|
+
<html>
|
|
25
|
+
<head>
|
|
26
|
+
<meta charset="utf-8">
|
|
27
|
+
<title>Astro Swiper</title>
|
|
28
|
+
<link rel="preconnect" href="https://fonts.googleapis.com">
|
|
29
|
+
<link href="https://fonts.googleapis.com/css2?family=Press+Start+2P&display=swap" rel="stylesheet">
|
|
30
|
+
<style>
|
|
31
|
+
* { box-sizing: border-box; margin: 0; padding: 0; }
|
|
32
|
+
body { background: #111 url('/background') center top / cover no-repeat fixed;
|
|
33
|
+
color: #eee; font-family: monospace;
|
|
34
|
+
display: flex; flex-direction: column; align-items: center;
|
|
35
|
+
padding: 40px 10px 10px; min-height: 100vh; }
|
|
36
|
+
#status { font-size: 1.1em; margin-bottom: 3px; min-height: 1.4em; }
|
|
37
|
+
#progress { font-size: 0.85em; color: #888; margin-bottom: 6px; min-height: 1.2em; }
|
|
38
|
+
#triplet-img { max-width: 100%; max-height: 78vh; object-fit: contain;
|
|
39
|
+
background: #000; display: block; }
|
|
40
|
+
#spinner { display: none; font-size: 0.9em; color: #666; margin: 4px; }
|
|
41
|
+
#keybinds { display: flex; flex-wrap: wrap; justify-content: center;
|
|
42
|
+
gap: 5px; margin-top: 8px; font-size: 1.1em; }
|
|
43
|
+
.kb { background: #1e1e1e; border: 1px solid #444;
|
|
44
|
+
padding: 4px 12px; border-radius: 4px; }
|
|
45
|
+
.kb b { color: #7af; }
|
|
46
|
+
#hint { font-size: 0.95em; color: #fff; margin-top: 8px; }
|
|
47
|
+
#photo-credit { position: fixed; bottom: 6px; right: 10px; font-size: 0.6em;
|
|
48
|
+
color: #444; text-decoration: none; }
|
|
49
|
+
#title { font-family: 'Press Start 2P', monospace; font-size: 64pt;
|
|
50
|
+
line-height: 1; margin-bottom: 12px; }
|
|
51
|
+
</style>
|
|
52
|
+
</head>
|
|
53
|
+
<body>
|
|
54
|
+
<div id="title">Astro Swiper</div>
|
|
55
|
+
<div id="status">Connecting…</div>
|
|
56
|
+
<div id="progress"></div>
|
|
57
|
+
<div id="spinner">Rendering…</div>
|
|
58
|
+
<img id="triplet-img" src="" alt="">
|
|
59
|
+
<div id="keybinds"></div>
|
|
60
|
+
<div id="hint">Shift+↑↓ contrast | Shift+←→ brightness</div>
|
|
61
|
+
<a id="photo-credit" href="https://www.pexels.com/photo/blue-and-purple-cosmic-sky-956999/" target="_blank">Photo by Felix Mittermeier</a>
|
|
62
|
+
|
|
63
|
+
<script src="https://cdn.socket.io/4.7.5/socket.io.min.js"></script>
|
|
64
|
+
<script>
|
|
65
|
+
const socket = io();
|
|
66
|
+
const imgEl = document.getElementById('triplet-img');
|
|
67
|
+
const statusEl = document.getElementById('status');
|
|
68
|
+
const progEl = document.getElementById('progress');
|
|
69
|
+
const spinEl = document.getElementById('spinner');
|
|
70
|
+
const kbEl = document.getElementById('keybinds');
|
|
71
|
+
|
|
72
|
+
socket.on('connect', () => { statusEl.textContent = 'Connected — loading…'; });
|
|
73
|
+
socket.on('disconnect', () => { statusEl.textContent = 'Disconnected.'; });
|
|
74
|
+
socket.on('loading', () => { spinEl.style.display = 'block'; });
|
|
75
|
+
|
|
76
|
+
socket.on('update', d => {
|
|
77
|
+
imgEl.src = 'data:image/png;base64,' + d.image;
|
|
78
|
+
statusEl.textContent = d.filename;
|
|
79
|
+
progEl.textContent = d.progress;
|
|
80
|
+
spinEl.style.display = 'none';
|
|
81
|
+
});
|
|
82
|
+
|
|
83
|
+
socket.on('done', d => {
|
|
84
|
+
statusEl.textContent = '✓ ' + d.message;
|
|
85
|
+
imgEl.src = ''; progEl.textContent = '';
|
|
86
|
+
spinEl.style.display = 'none';
|
|
87
|
+
});
|
|
88
|
+
|
|
89
|
+
const arrowGlyph = {left:'←', right:'→', up:'↑', down:'↓'};
|
|
90
|
+
const pretty = k => arrowGlyph[k] ?? k;
|
|
91
|
+
|
|
92
|
+
socket.on('keybinds', list => {
|
|
93
|
+
kbEl.innerHTML =
|
|
94
|
+
list.map(([k, n]) => `<div class="kb"><b>${pretty(k)}</b> ${n}</div>`).join('');
|
|
95
|
+
});
|
|
96
|
+
|
|
97
|
+
document.addEventListener('keydown', e => {
|
|
98
|
+
if (['ArrowLeft','ArrowRight','ArrowUp','ArrowDown',' '].includes(e.key))
|
|
99
|
+
e.preventDefault();
|
|
100
|
+
const arrows = {ArrowLeft:'left', ArrowRight:'right', ArrowUp:'up', ArrowDown:'down'};
|
|
101
|
+
let key = arrows[e.key] ?? e.key;
|
|
102
|
+
if (e.shiftKey && ['left','right','up','down'].includes(key)) key = 'shift+' + key;
|
|
103
|
+
socket.emit('keypress', {key});
|
|
104
|
+
});
|
|
105
|
+
</script>
|
|
106
|
+
</body>
|
|
107
|
+
</html>"""
|
|
108
|
+
|
|
109
|
+
# ---------------------------------------------------------------------------
|
|
110
|
+
# Public API
|
|
111
|
+
# ---------------------------------------------------------------------------
|
|
112
|
+
|
|
113
|
+
class AstroSwiper:
|
|
114
|
+
"""
|
|
115
|
+
Web-based FITS triplet classifier.
|
|
116
|
+
|
|
117
|
+
Usage:
|
|
118
|
+
from astro_swiper import AstroSwiper
|
|
119
|
+
AstroSwiper('config.yaml').run()
|
|
120
|
+
|
|
121
|
+
Also accepts a pre-loaded dict instead of a path:
|
|
122
|
+
AstroSwiper({'input_dir': '...', 'keybinds': {...}, ...}).run()
|
|
123
|
+
|
|
124
|
+
To use a custom triplet loader instead of the built-in filename scanner,
|
|
125
|
+
pass a callable that accepts input_dir (may be None) and returns a list of
|
|
126
|
+
[sub_path, sci_path, ref_path] triplets:
|
|
127
|
+
|
|
128
|
+
def my_loader(input_dir):
|
|
129
|
+
return [[sub, sci, ref], ...]
|
|
130
|
+
|
|
131
|
+
AstroSwiper('config.yaml', triplet_loader=my_loader).run()
|
|
132
|
+
|
|
133
|
+
When a triplet_loader is provided, input_dir in config is optional and is
|
|
134
|
+
passed to the loader as-is (you may ignore it if not needed).
|
|
135
|
+
"""
|
|
136
|
+
|
|
137
|
+
def __init__(self, config, triplet_loader=None):
|
|
138
|
+
if isinstance(config, str):
|
|
139
|
+
with open(config) as f:
|
|
140
|
+
cfg = yaml.safe_load(f)
|
|
141
|
+
else:
|
|
142
|
+
cfg = dict(config)
|
|
143
|
+
|
|
144
|
+
input_dir = cfg.get('input_dir')
|
|
145
|
+
if input_dir is None and triplet_loader is None:
|
|
146
|
+
raise ValueError("config must include 'input_dir' when no triplet_loader is provided")
|
|
147
|
+
|
|
148
|
+
self._port = cfg.get('port', 5000)
|
|
149
|
+
self._app = Flask(__name__)
|
|
150
|
+
self._app.config['SECRET_KEY'] = 'astro_swiper'
|
|
151
|
+
self._sio = SocketIO(self._app, async_mode='threading', cors_allowed_origins='*')
|
|
152
|
+
|
|
153
|
+
keybinds = {str(k): str(v) for k, v in cfg['keybinds'].items()}
|
|
154
|
+
|
|
155
|
+
self._classifier = TripletClassifier(
|
|
156
|
+
keybinds=keybinds,
|
|
157
|
+
back_button=cfg.get('back_button', 'left'),
|
|
158
|
+
storage=make_backend(cfg, keybinds),
|
|
159
|
+
socketio=self._sio,
|
|
160
|
+
resume=cfg.get('resume', True),
|
|
161
|
+
overwrite=cfg.get('overwrite', False),
|
|
162
|
+
)
|
|
163
|
+
self._classifier.load_directory(input_dir, triplet_loader=triplet_loader)
|
|
164
|
+
self._register_routes()
|
|
165
|
+
|
|
166
|
+
def _register_routes(self):
|
|
167
|
+
app = self._app
|
|
168
|
+
sio = self._sio
|
|
169
|
+
clf = self._classifier
|
|
170
|
+
|
|
171
|
+
@app.route('/')
|
|
172
|
+
def index_page():
|
|
173
|
+
return render_template_string(HTML)
|
|
174
|
+
|
|
175
|
+
@app.route('/background')
|
|
176
|
+
def background():
|
|
177
|
+
return send_file(
|
|
178
|
+
os.path.join(os.path.dirname(__file__), 'imgs', 'background.png'),
|
|
179
|
+
mimetype='image/png',
|
|
180
|
+
)
|
|
181
|
+
|
|
182
|
+
@sio.on('connect')
|
|
183
|
+
def on_connect():
|
|
184
|
+
kb_list = [
|
|
185
|
+
(k, os.path.splitext(os.path.basename(v))[0])
|
|
186
|
+
for k, v in clf.keybinds.items()
|
|
187
|
+
]
|
|
188
|
+
kb_list.append((clf.back_button, 'back'))
|
|
189
|
+
emit('keybinds', kb_list)
|
|
190
|
+
clf.send_current(to=request.sid)
|
|
191
|
+
|
|
192
|
+
@sio.on('keypress')
|
|
193
|
+
def on_keypress(data):
|
|
194
|
+
clf.handle_key(data.get('key', ''))
|
|
195
|
+
|
|
196
|
+
def run(self):
|
|
197
|
+
print(f"Open http://localhost:{self._port} in your browser")
|
|
198
|
+
self._sio.run(
|
|
199
|
+
self._app, host='127.0.0.1', port=self._port,
|
|
200
|
+
debug=False, use_reloader=False, allow_unsafe_werkzeug=True,
|
|
201
|
+
)
|
|
@@ -0,0 +1,193 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: astro-swiper
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Web-based interactive FITS triplet classifier for astronomical image labelling
|
|
5
|
+
License: MIT
|
|
6
|
+
Requires-Python: >=3.8
|
|
7
|
+
Description-Content-Type: text/markdown
|
|
8
|
+
Requires-Dist: flask>=2.0
|
|
9
|
+
Requires-Dist: flask-socketio>=5.0
|
|
10
|
+
Requires-Dist: astropy>=5.0
|
|
11
|
+
Requires-Dist: matplotlib>=3.5
|
|
12
|
+
Requires-Dist: numpy>=1.21
|
|
13
|
+
Requires-Dist: pyyaml>=6.0
|
|
14
|
+
|
|
15
|
+
# Astro Swiper — Usage
|
|
16
|
+
|
|
17
|
+
## Installation
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
pip install flask flask-socketio astropy matplotlib numpy pyyaml
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## Running
|
|
26
|
+
|
|
27
|
+
### 1. As a Python import (recommended)
|
|
28
|
+
|
|
29
|
+
```python
|
|
30
|
+
from astro_swiper_web import AstroSwiper
|
|
31
|
+
|
|
32
|
+
AstroSwiper('config.yaml').run()
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
### 2. With a custom triplet loader
|
|
36
|
+
|
|
37
|
+
If your files don't follow the default `*scicutout / *subcutout / *refcutout` naming convention, provide a `triplet_loader` function. It receives `input_dir` from the config (or `None` if omitted) and must return a list of `[sub_path, sci_path, ref_path]` triplets.
|
|
38
|
+
|
|
39
|
+
```python
|
|
40
|
+
from astro_swiper_web import AstroSwiper
|
|
41
|
+
|
|
42
|
+
def my_loader(input_dir):
|
|
43
|
+
# build and return your triplets however you like
|
|
44
|
+
import glob, re
|
|
45
|
+
sci_files = sorted(glob.glob(f"{input_dir}/*_science.fits"))
|
|
46
|
+
triplets = []
|
|
47
|
+
for sci in sci_files:
|
|
48
|
+
base = re.sub(r'_science\.fits$', '', sci)
|
|
49
|
+
sub = base + '_difference.fits'
|
|
50
|
+
ref = base + '_template.fits'
|
|
51
|
+
if os.path.exists(sub) and os.path.exists(ref):
|
|
52
|
+
triplets.append([sub, sci, ref])
|
|
53
|
+
return triplets
|
|
54
|
+
|
|
55
|
+
AstroSwiper('config.yaml', triplet_loader=my_loader).run()
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
`input_dir` becomes optional in config when a loader is supplied — you can omit it entirely if your loader doesn't need it.
|
|
59
|
+
|
|
60
|
+
### 4. With an inline config dict (no file needed)
|
|
61
|
+
|
|
62
|
+
```python
|
|
63
|
+
from astro_swiper_web import AstroSwiper
|
|
64
|
+
|
|
65
|
+
AstroSwiper({
|
|
66
|
+
'input_dir': '/data/cutouts/',
|
|
67
|
+
'back_button': 'up',
|
|
68
|
+
'port': 5000,
|
|
69
|
+
'resume': True,
|
|
70
|
+
'overwrite': False,
|
|
71
|
+
'storage': {'backend': 'sqlite', 'db': 'classifications.db'},
|
|
72
|
+
'keybinds': {
|
|
73
|
+
'a': 'noise',
|
|
74
|
+
'e': 'streaks',
|
|
75
|
+
'd': 'dots',
|
|
76
|
+
'1': 'small',
|
|
77
|
+
'2': 'medium',
|
|
78
|
+
},
|
|
79
|
+
}).run()
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
### 5. From the command line
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
python astro_swiper_web.py # uses config.yaml in current directory
|
|
86
|
+
python astro_swiper_web.py my_cfg.yaml # explicit config path
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Then open **http://localhost:5000** in a browser.
|
|
90
|
+
|
|
91
|
+
**Over SSH** (no X11 needed):
|
|
92
|
+
```bash
|
|
93
|
+
ssh -L 5000:localhost:5000 user@host
|
|
94
|
+
# then open http://localhost:5000 locally
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## config.yaml reference
|
|
100
|
+
|
|
101
|
+
| Key | Default | Description |
|
|
102
|
+
|-----|---------|-------------|
|
|
103
|
+
| `input_dir` | *(required)* | Directory containing `.fits` or `.fits.gz` cutout triplets |
|
|
104
|
+
| `back_button` | `left` | Key that undoes the last classification |
|
|
105
|
+
| `port` | `5000` | Port the web server listens on |
|
|
106
|
+
| `resume` | `true` | Skip already-classified triplets on startup |
|
|
107
|
+
| `overwrite` | `false` | Wipe all saved classifications and start fresh |
|
|
108
|
+
| `storage.backend` | `sqlite` | Storage format: `sqlite`, `csv`, or `txt` |
|
|
109
|
+
| `keybinds` | *(required)* | Map of key → label (or file path for `txt` backend) |
|
|
110
|
+
|
|
111
|
+
---
|
|
112
|
+
|
|
113
|
+
## Storage backends
|
|
114
|
+
|
|
115
|
+
### SQLite (recommended)
|
|
116
|
+
|
|
117
|
+
```yaml
|
|
118
|
+
storage:
|
|
119
|
+
backend: sqlite
|
|
120
|
+
db: training_sets/classifications.db
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
Single file, atomic writes, safe against crashes. Query results with pandas:
|
|
124
|
+
|
|
125
|
+
```python
|
|
126
|
+
import sqlite3, pandas as pd
|
|
127
|
+
df = pd.read_sql(
|
|
128
|
+
"SELECT * FROM classifications",
|
|
129
|
+
sqlite3.connect("training_sets/classifications.db")
|
|
130
|
+
)
|
|
131
|
+
counts = df['label'].value_counts()
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
### CSV
|
|
135
|
+
|
|
136
|
+
```yaml
|
|
137
|
+
storage:
|
|
138
|
+
backend: csv
|
|
139
|
+
file: training_sets/classifications.csv
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
One row per triplet with columns `sub_path, sci_path, ref_path, label`. Easy to open in Excel or pandas:
|
|
143
|
+
|
|
144
|
+
```python
|
|
145
|
+
import pandas as pd
|
|
146
|
+
df = pd.read_csv("training_sets/classifications.csv")
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
### Txt (legacy)
|
|
150
|
+
|
|
151
|
+
```yaml
|
|
152
|
+
storage:
|
|
153
|
+
backend: txt
|
|
154
|
+
already_classified: training_sets/already_classified.txt
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
One `.txt` file per category; keybind values must be **file paths** (not labels):
|
|
158
|
+
|
|
159
|
+
```yaml
|
|
160
|
+
keybinds:
|
|
161
|
+
a: training_sets/noise.txt
|
|
162
|
+
c: training_sets/skips.txt
|
|
163
|
+
...
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
Each file contains triplet paths, three lines per entry (sub, sci, ref).
|
|
167
|
+
|
|
168
|
+
---
|
|
169
|
+
|
|
170
|
+
## Input data format
|
|
171
|
+
|
|
172
|
+
Each triplet is a set of three co-registered FITS cutout files sharing a common basename:
|
|
173
|
+
|
|
174
|
+
```
|
|
175
|
+
<basename>scicutout.fits[.gz]
|
|
176
|
+
<basename>subcutout.fits[.gz]
|
|
177
|
+
<basename>refcutout.fits[.gz]
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
All files must be in the same flat directory (`input_dir`). Both `.fits` and `.fits.gz` are supported.
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## Controls
|
|
185
|
+
|
|
186
|
+
| Key | Action |
|
|
187
|
+
|-----|--------|
|
|
188
|
+
| *(configured keybinds)* | Classify current triplet |
|
|
189
|
+
| `back_button` (default `left`) | Undo last classification |
|
|
190
|
+
| `Shift+↑` | Increase contrast (narrow display window) |
|
|
191
|
+
| `Shift+↓` | Decrease contrast (widen display window) |
|
|
192
|
+
| `Shift+→` | Increase brightness (shift window up) |
|
|
193
|
+
| `Shift+←` | Decrease brightness (shift window down) |
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
pyproject.toml
|
|
2
|
+
readme.md
|
|
3
|
+
setup.py
|
|
4
|
+
astro_swiper/__init__.py
|
|
5
|
+
astro_swiper/_cli.py
|
|
6
|
+
astro_swiper/classifier.py
|
|
7
|
+
astro_swiper/storage.py
|
|
8
|
+
astro_swiper/web.py
|
|
9
|
+
astro_swiper.egg-info/PKG-INFO
|
|
10
|
+
astro_swiper.egg-info/SOURCES.txt
|
|
11
|
+
astro_swiper.egg-info/dependency_links.txt
|
|
12
|
+
astro_swiper.egg-info/entry_points.txt
|
|
13
|
+
astro_swiper.egg-info/requires.txt
|
|
14
|
+
astro_swiper.egg-info/top_level.txt
|
|
15
|
+
astro_swiper/imgs/background.png
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
astro_swiper
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=61", "wheel"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "astro-swiper"
|
|
7
|
+
version = "0.1.0"
|
|
8
|
+
description = "Web-based interactive FITS triplet classifier for astronomical image labelling"
|
|
9
|
+
readme = "readme.md"
|
|
10
|
+
license = { text = "MIT" }
|
|
11
|
+
requires-python = ">=3.8"
|
|
12
|
+
dependencies = [
|
|
13
|
+
"flask>=2.0",
|
|
14
|
+
"flask-socketio>=5.0",
|
|
15
|
+
"astropy>=5.0",
|
|
16
|
+
"matplotlib>=3.5",
|
|
17
|
+
"numpy>=1.21",
|
|
18
|
+
"pyyaml>=6.0",
|
|
19
|
+
]
|
|
20
|
+
|
|
21
|
+
[project.scripts]
|
|
22
|
+
asswiper = "astro_swiper._cli:main"
|
|
23
|
+
|
|
24
|
+
[tool.setuptools.packages.find]
|
|
25
|
+
where = ["."]
|
|
26
|
+
include = ["astro_swiper*"]
|
|
27
|
+
|
|
28
|
+
[tool.setuptools.package-data]
|
|
29
|
+
astro_swiper = ["imgs/*.png"]
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
# Astro Swiper — Usage
|
|
2
|
+
|
|
3
|
+
## Installation
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
pip install flask flask-socketio astropy matplotlib numpy pyyaml
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## Running
|
|
12
|
+
|
|
13
|
+
### 1. As a Python import (recommended)
|
|
14
|
+
|
|
15
|
+
```python
|
|
16
|
+
from astro_swiper_web import AstroSwiper
|
|
17
|
+
|
|
18
|
+
AstroSwiper('config.yaml').run()
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
### 2. With a custom triplet loader
|
|
22
|
+
|
|
23
|
+
If your files don't follow the default `*scicutout / *subcutout / *refcutout` naming convention, provide a `triplet_loader` function. It receives `input_dir` from the config (or `None` if omitted) and must return a list of `[sub_path, sci_path, ref_path]` triplets.
|
|
24
|
+
|
|
25
|
+
```python
|
|
26
|
+
from astro_swiper_web import AstroSwiper
|
|
27
|
+
|
|
28
|
+
def my_loader(input_dir):
|
|
29
|
+
# build and return your triplets however you like
|
|
30
|
+
import glob, re
|
|
31
|
+
sci_files = sorted(glob.glob(f"{input_dir}/*_science.fits"))
|
|
32
|
+
triplets = []
|
|
33
|
+
for sci in sci_files:
|
|
34
|
+
base = re.sub(r'_science\.fits$', '', sci)
|
|
35
|
+
sub = base + '_difference.fits'
|
|
36
|
+
ref = base + '_template.fits'
|
|
37
|
+
if os.path.exists(sub) and os.path.exists(ref):
|
|
38
|
+
triplets.append([sub, sci, ref])
|
|
39
|
+
return triplets
|
|
40
|
+
|
|
41
|
+
AstroSwiper('config.yaml', triplet_loader=my_loader).run()
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
`input_dir` becomes optional in config when a loader is supplied — you can omit it entirely if your loader doesn't need it.
|
|
45
|
+
|
|
46
|
+
### 4. With an inline config dict (no file needed)
|
|
47
|
+
|
|
48
|
+
```python
|
|
49
|
+
from astro_swiper_web import AstroSwiper
|
|
50
|
+
|
|
51
|
+
AstroSwiper({
|
|
52
|
+
'input_dir': '/data/cutouts/',
|
|
53
|
+
'back_button': 'up',
|
|
54
|
+
'port': 5000,
|
|
55
|
+
'resume': True,
|
|
56
|
+
'overwrite': False,
|
|
57
|
+
'storage': {'backend': 'sqlite', 'db': 'classifications.db'},
|
|
58
|
+
'keybinds': {
|
|
59
|
+
'a': 'noise',
|
|
60
|
+
'e': 'streaks',
|
|
61
|
+
'd': 'dots',
|
|
62
|
+
'1': 'small',
|
|
63
|
+
'2': 'medium',
|
|
64
|
+
},
|
|
65
|
+
}).run()
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
### 5. From the command line
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
python astro_swiper_web.py # uses config.yaml in current directory
|
|
72
|
+
python astro_swiper_web.py my_cfg.yaml # explicit config path
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Then open **http://localhost:5000** in a browser.
|
|
76
|
+
|
|
77
|
+
**Over SSH** (no X11 needed):
|
|
78
|
+
```bash
|
|
79
|
+
ssh -L 5000:localhost:5000 user@host
|
|
80
|
+
# then open http://localhost:5000 locally
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
## config.yaml reference
|
|
86
|
+
|
|
87
|
+
| Key | Default | Description |
|
|
88
|
+
|-----|---------|-------------|
|
|
89
|
+
| `input_dir` | *(required)* | Directory containing `.fits` or `.fits.gz` cutout triplets |
|
|
90
|
+
| `back_button` | `left` | Key that undoes the last classification |
|
|
91
|
+
| `port` | `5000` | Port the web server listens on |
|
|
92
|
+
| `resume` | `true` | Skip already-classified triplets on startup |
|
|
93
|
+
| `overwrite` | `false` | Wipe all saved classifications and start fresh |
|
|
94
|
+
| `storage.backend` | `sqlite` | Storage format: `sqlite`, `csv`, or `txt` |
|
|
95
|
+
| `keybinds` | *(required)* | Map of key → label (or file path for `txt` backend) |
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## Storage backends
|
|
100
|
+
|
|
101
|
+
### SQLite (recommended)
|
|
102
|
+
|
|
103
|
+
```yaml
|
|
104
|
+
storage:
|
|
105
|
+
backend: sqlite
|
|
106
|
+
db: training_sets/classifications.db
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
Single file, atomic writes, safe against crashes. Query results with pandas:
|
|
110
|
+
|
|
111
|
+
```python
|
|
112
|
+
import sqlite3, pandas as pd
|
|
113
|
+
df = pd.read_sql(
|
|
114
|
+
"SELECT * FROM classifications",
|
|
115
|
+
sqlite3.connect("training_sets/classifications.db")
|
|
116
|
+
)
|
|
117
|
+
counts = df['label'].value_counts()
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
### CSV
|
|
121
|
+
|
|
122
|
+
```yaml
|
|
123
|
+
storage:
|
|
124
|
+
backend: csv
|
|
125
|
+
file: training_sets/classifications.csv
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
One row per triplet with columns `sub_path, sci_path, ref_path, label`. Easy to open in Excel or pandas:
|
|
129
|
+
|
|
130
|
+
```python
|
|
131
|
+
import pandas as pd
|
|
132
|
+
df = pd.read_csv("training_sets/classifications.csv")
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
### Txt (legacy)
|
|
136
|
+
|
|
137
|
+
```yaml
|
|
138
|
+
storage:
|
|
139
|
+
backend: txt
|
|
140
|
+
already_classified: training_sets/already_classified.txt
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
One `.txt` file per category; keybind values must be **file paths** (not labels):
|
|
144
|
+
|
|
145
|
+
```yaml
|
|
146
|
+
keybinds:
|
|
147
|
+
a: training_sets/noise.txt
|
|
148
|
+
c: training_sets/skips.txt
|
|
149
|
+
...
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Each file contains triplet paths, three lines per entry (sub, sci, ref).
|
|
153
|
+
|
|
154
|
+
---
|
|
155
|
+
|
|
156
|
+
## Input data format
|
|
157
|
+
|
|
158
|
+
Each triplet is a set of three co-registered FITS cutout files sharing a common basename:
|
|
159
|
+
|
|
160
|
+
```
|
|
161
|
+
<basename>scicutout.fits[.gz]
|
|
162
|
+
<basename>subcutout.fits[.gz]
|
|
163
|
+
<basename>refcutout.fits[.gz]
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
All files must be in the same flat directory (`input_dir`). Both `.fits` and `.fits.gz` are supported.
|
|
167
|
+
|
|
168
|
+
---
|
|
169
|
+
|
|
170
|
+
## Controls
|
|
171
|
+
|
|
172
|
+
| Key | Action |
|
|
173
|
+
|-----|--------|
|
|
174
|
+
| *(configured keybinds)* | Classify current triplet |
|
|
175
|
+
| `back_button` (default `left`) | Undo last classification |
|
|
176
|
+
| `Shift+↑` | Increase contrast (narrow display window) |
|
|
177
|
+
| `Shift+↓` | Decrease contrast (widen display window) |
|
|
178
|
+
| `Shift+→` | Increase brightness (shift window up) |
|
|
179
|
+
| `Shift+←` | Decrease brightness (shift window down) |
|