comet-hunter 0.1.0__tar.gz

comet_hunter-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Anand Krishna
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

comet_hunter-0.1.0/PKG-INFO

@@ -0,0 +1,159 @@
+Metadata-Version: 2.4
+Name: comet-hunter
+Version: 0.1.0
+Summary: Astronomical image ingestion and processing system for Sungrazer project
+Author: Anand Krishna
+License-Expression: MIT
+Project-URL: Repository, https://github.com/AnandKri/comet-hunter
+Project-URL: Documentation, https://anandkri.github.io/comet-hunter/
+Keywords: astronomy,comet,system-design,processing-pipeline
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fastapi>=0.136
+Requires-Dist: nicegui>=3.12
+Requires-Dist: requests>=2.34
+Requires-Dist: numpy>=2.4
+Requires-Dist: scipy>=1.17
+Requires-Dist: sunpy>=7.1
+Requires-Dist: pydantic>=2.13
+Requires-Dist: matplotlib>=3.10
+Requires-Dist: starlette>=1.2
+Requires-Dist: reproject>=0.19
+Requires-Dist: mpl_animators>=1.2
+Requires-Dist: tzdata>=2026.1
+Dynamic: license-file
+
+# Comet Hunter
+
+Comet Hunter is an automated astronomical image ingestion and processing system designed to assist in the discovery of sungrazing comets from SOHO LASCO imagery.
+
+## About The Project
+
+NASA's [Sungrazer Project](https://sungrazer.nrl.navy.mil/) enables the discovery and reporting of comets visible from the SOHO and STEREO satellites. To date, over five thousand comets have been discovered using the SOHO satellite. On board SOHO is the LASCO coronagraph, which consists of two telescopes — C2 and C3. Images from these telescopes are primarily used for reporting new comets.
+
+### Why This Exists?
+
+For comet discovery, users rely on fragmented tools for downloading, processing, and reviewing imagery. There is no unified platform that automates the complete workflow from raw image availability to chronological playback of processed frames. **Comet Hunter aims to bridge this gap**.
+
+### Present Challenges
+- RAW images must be processed before becoming usable
+- Sungrazer comets are often indistinguishable in single frames
+- Chronological playback significantly improves detectability
+- Most comets are reported within minutes of data availability.
+- **Time is critical.**
+
+The problem is not merely detection - it is **rapid** detection. 
+
+This requires a **robust automation** of the complete workflow: from RAW image ingestion to chronological playback of processed frames.
+
+## Current Capabilities
+
+- Downlink slot synchronization
+- Metadata ingestion from LASCO sources
+- Parallel RAW image downloading
+- Image processing pipelines for C2/C3
+- Time-indexed frame retrieval
+- REST API backend
+- Scheduler-driven ingestion workflows
+- Interactive frontend visualization
+
+<h2>User Interface</h2>
+
+<p align="center">
+  <img src="docs/images/ui-dashboard.png" width="900">
+</p>
+
+## Getting Started
+
+### End User Installation
+
+#### Install Comet Hunter directly from PyPI:
+
+```bash
+pip install comet-hunter
+```
+
+#### End User Commands
+
+Start the application
+
+```bash
+comet-hunter start
+```
+
+Check application status
+
+```bash
+comet-hunter status
+```
+
+Stop the application
+
+```bash
+comet-hunter stop
+```
+
+#### Note
+
+When started, the application will be available at:
+
+`http://localhost:8080`
+
+Application data, logs, and database files are stored in:
+
+```
+Windows:
+C:\Users\<username>\.comet_hunter
+
+Linux/macOS:
+~/.comet_hunter
+```
+
+### Development Setup
+
+#### Clone Repository
+
+```bash
+git clone https://github.com/AnandKri/comet-hunter.git
+cd comet-hunter
+```
+
+#### Create Virtual Environment
+
+Linux/macOS
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+```
+
+Windows
+```bash
+python -m venv .venv
+.venv\Scripts\activate
+```
+
+#### Install Dependencies
+
+```bash
+pip install -r requirements.txt
+```
+
+#### Run Backend
+
+```bash
+uvicorn backend.main:app --reload
+```
+
+#### Run Frontend
+
+```bash
+python frontend/app.py
+```
+
+## Documentation
+
+<a href="https://anandkri.github.io/comet-hunter/" target="_blank">
+View full documentation here
+</a>

comet_hunter-0.1.0/README.md

@@ -0,0 +1,133 @@
+# Comet Hunter
+
+Comet Hunter is an automated astronomical image ingestion and processing system designed to assist in the discovery of sungrazing comets from SOHO LASCO imagery.
+
+## About The Project
+
+NASA's [Sungrazer Project](https://sungrazer.nrl.navy.mil/) enables the discovery and reporting of comets visible from the SOHO and STEREO satellites. To date, over five thousand comets have been discovered using the SOHO satellite. On board SOHO is the LASCO coronagraph, which consists of two telescopes — C2 and C3. Images from these telescopes are primarily used for reporting new comets.
+
+### Why This Exists?
+
+For comet discovery, users rely on fragmented tools for downloading, processing, and reviewing imagery. There is no unified platform that automates the complete workflow from raw image availability to chronological playback of processed frames. **Comet Hunter aims to bridge this gap**.
+
+### Present Challenges
+- RAW images must be processed before becoming usable
+- Sungrazer comets are often indistinguishable in single frames
+- Chronological playback significantly improves detectability
+- Most comets are reported within minutes of data availability.
+- **Time is critical.**
+
+The problem is not merely detection - it is **rapid** detection. 
+
+This requires a **robust automation** of the complete workflow: from RAW image ingestion to chronological playback of processed frames.
+
+## Current Capabilities
+
+- Downlink slot synchronization
+- Metadata ingestion from LASCO sources
+- Parallel RAW image downloading
+- Image processing pipelines for C2/C3
+- Time-indexed frame retrieval
+- REST API backend
+- Scheduler-driven ingestion workflows
+- Interactive frontend visualization
+
+<h2>User Interface</h2>
+
+<p align="center">
+  <img src="docs/images/ui-dashboard.png" width="900">
+</p>
+
+## Getting Started
+
+### End User Installation
+
+#### Install Comet Hunter directly from PyPI:
+
+```bash
+pip install comet-hunter
+```
+
+#### End User Commands
+
+Start the application
+
+```bash
+comet-hunter start
+```
+
+Check application status
+
+```bash
+comet-hunter status
+```
+
+Stop the application
+
+```bash
+comet-hunter stop
+```
+
+#### Note
+
+When started, the application will be available at:
+
+`http://localhost:8080`
+
+Application data, logs, and database files are stored in:
+
+```
+Windows:
+C:\Users\<username>\.comet_hunter
+
+Linux/macOS:
+~/.comet_hunter
+```
+
+### Development Setup
+
+#### Clone Repository
+
+```bash
+git clone https://github.com/AnandKri/comet-hunter.git
+cd comet-hunter
+```
+
+#### Create Virtual Environment
+
+Linux/macOS
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+```
+
+Windows
+```bash
+python -m venv .venv
+.venv\Scripts\activate
+```
+
+#### Install Dependencies
+
+```bash
+pip install -r requirements.txt
+```
+
+#### Run Backend
+
+```bash
+uvicorn backend.main:app --reload
+```
+
+#### Run Frontend
+
+```bash
+python frontend/app.py
+```
+
+## Documentation
+
+<a href="https://anandkri.github.io/comet-hunter/" target="_blank">
+View full documentation here
+</a>

comet_hunter-0.1.0/backend/api/dependencies.py

@@ -0,0 +1,77 @@
+from pathlib import Path
+from backend.database.infrastructure.query_executor import QueryExecutor
+from backend.database.repositories.file_metadata_repository import FileMetadataRepository
+from backend.database.repositories.processed_file_repository import ProcessedFileRepository
+from backend.database.repositories.downlink_slot_repository import DownlinkSlotRepository
+from backend.services.metadata_service import MetadataService
+from backend.services.download_file_service import DownloadFileService
+from backend.services.process_file_service import ProcessFileService
+from backend.services.slot_service import SlotService
+from backend.pipeline.pipeline import Pipeline
+from backend.pipeline.scheduler import Scheduler
+from backend.jobs.job_store import job_store, JobStore
+from backend.jobs.background_job_service import BackgroundJobService
+from backend.jobs.event_bus import event_bus, EventBus
+from backend.core.storage import RAW_DIR, PROCESSED_DIR
+
+background_job_service = BackgroundJobService(job_store)
+
+executor = QueryExecutor()
+
+metadata_repo = FileMetadataRepository(executor)
+processed_repo = ProcessedFileRepository(executor)
+slot_repo = DownlinkSlotRepository(executor)
+
+metadata_service = MetadataService(metadata_repo)
+slot_service = SlotService(slot_repo)
+download_service = DownloadFileService(
+    processed_repository=processed_repo,
+    metadata_service=metadata_service,
+    download_directory=RAW_DIR,
+)
+process_service = ProcessFileService(
+    processed_repository=processed_repo,
+    metadata_service=metadata_service,
+    processed_directory=PROCESSED_DIR,
+)
+
+pipeline = Pipeline(
+    slot_service=slot_service,
+    metadata_service=metadata_service,
+    download_service=download_service,
+    process_service=process_service,
+)
+
+scheduler = Scheduler(pipeline)
+
+def get_pipeline() -> Pipeline:
+    """
+    Returns shared pipeline instance for handling API requests and background jobs.
+    """
+    return pipeline
+
+def get_scheduler() -> Scheduler:
+    """
+    Returns shared scheduler instance for triggering background jobs.
+    """
+    return scheduler
+
+def get_job_store() -> JobStore:
+    """
+    Returns a shared job_store instance.
+    """
+    return job_store
+
+def get_job_service() -> BackgroundJobService:
+    """
+    Returns shared background job service instance.
+    Used for submitting and managing background jobs.
+    """
+    return background_job_service
+
+def get_event_bus() -> EventBus:
+    """
+    Returns shared event bus instance. 
+    """
+
+    return event_bus

comet_hunter-0.1.0/backend/api/dto/api_response.py

@@ -0,0 +1,17 @@
+from pydantic import BaseModel
+from pydantic.generics import GenericModel
+from typing import Generic, TypeVar
+
+T = TypeVar('T')
+
+class ApiSuccessResponse(GenericModel, Generic[T]):
+    success: bool = True
+    data: T
+
+class ApiErrorDetail(BaseModel):
+    code: str
+    message: str
+
+class ApiErrorResponse(BaseModel):
+    success: bool = False
+    error: ApiErrorDetail

comet_hunter-0.1.0/backend/api/dto/response_models.py

@@ -0,0 +1,65 @@
+from backend.util.enums import Instrument, JobType, JobStatus
+from pydantic import BaseModel
+from datetime import datetime, timedelta
+from typing import Optional, Any
+
+class ProcessedFileResponse(BaseModel):
+    processed_file_name: str
+    instrument: Instrument
+    processed_file_url: str
+    datetime_of_observation: datetime 
+
+class GetFramesResponse(BaseModel):
+    files: list[ProcessedFileResponse]
+    total: int
+    limit: int
+    offset: int
+
+class SyncFramesResponse(BaseModel):
+    metadata_synced: int
+    downloaded: int
+    marked_ready: int
+    processed: int
+
+class SyncSlotsResponse(BaseModel):
+    status: str
+    slots_synced: int
+
+class SchedulerStatusResponse(BaseModel):
+    running: bool
+    next_run_at: Optional[datetime]
+    next_run_in: Optional[timedelta]
+
+class SchedulerStartResponse(BaseModel):
+    started: bool
+    running: bool
+
+class SchedulerStopResponse(BaseModel):
+    stopped: bool
+    running: bool
+
+class HealthResponse(BaseModel):
+    status: str
+    database: bool
+    scheduler_initialized: bool
+    pipeline_initialized: bool
+
+class SlotResponse(BaseModel):
+    start: Optional[datetime]
+    end: Optional[datetime]
+
+class JobQueuedResponse(BaseModel):
+    job_id: str
+    existing: bool
+    status: str
+
+class JobStatusResponse(BaseModel):
+    job_id: str
+    type: JobType
+    status: JobStatus
+    created_at: datetime
+    started_at: Optional[datetime] = None
+    stopped_at: Optional[datetime] = None
+    progress: Optional[Any] = None
+    result: Optional[Any] = None
+    error: Optional[str] = None

comet_hunter-0.1.0/backend/api/dto/serializers.py

@@ -0,0 +1,31 @@
+from backend.api.dto.response_models import GetFramesResponse, ProcessedFileResponse
+from backend.database.domain.processed_file import ProcessedFile
+from backend.pipeline.models import GetProcessedFramesResult
+
+def serialize_processed_file(
+    file: ProcessedFile
+) -> ProcessedFileResponse:
+    """
+    Converts ProcessedFile domain model to ProcessedFileResponse DTO
+    """
+
+    return ProcessedFileResponse(
+        processed_file_name=file.processed_file_name,
+        instrument=file.instrument,
+        processed_file_url=f"/media/{str(file.processed_file_name)}",
+        datetime_of_observation=file.datetime_of_observation
+    )
+
+def serialize_get_frames_response(result: GetProcessedFramesResult) -> GetFramesResponse:
+    """
+    Converts GetProcessedFramesResult pipeline result domain model to GetFramesResponse DTO
+    """
+    return GetFramesResponse(
+        files=[
+            serialize_processed_file(file) 
+            for file in result.processed_files
+        ],
+        total=result.total,
+        limit=result.limit,
+        offset=result.offset
+    )

comet_hunter-0.1.0/backend/api/middleware.py

@@ -0,0 +1,44 @@
+import logging
+import time
+import uuid
+from starlette.middleware.base import BaseHTTPMiddleware
+from backend.core.request_context import set_request_id
+
+logger = logging.getLogger("api")
+
+class LoggingMiddleware(BaseHTTPMiddleware):
+
+    async def dispatch(self, request, call_next):
+        request_id = str(uuid.uuid4())
+        set_request_id(request_id)
+
+        start = time.time()
+
+        try:
+            response = await call_next(request)
+            duration = time.time() - start
+
+            logger.info(
+                "Request completed",
+                extra={
+                    "request_id": request_id,
+                    "method": request.method,
+                    "path": request.url.path,
+                    "status": response.status_code,
+                    "duration_ms": round(duration * 1000, 2)
+                }
+            )
+
+            return response
+        
+        except Exception:
+            logger.exception(
+                "Request failed",
+                extra={
+                    "request_id": request_id,
+                    "method": request.method,
+                    "path": request.url.path
+                }
+            )
+
+            raise

comet_hunter-0.1.0/backend/api/routes/frames.py

@@ -0,0 +1,104 @@
+from fastapi import APIRouter, Query, Depends
+from backend.api.dependencies import get_pipeline, get_job_service
+from backend.util.enums import Instrument, JobType
+from backend.api.dto.api_response import ApiSuccessResponse
+from backend.api.dto.response_models import JobQueuedResponse, GetFramesResponse
+from backend.pipeline.pipeline import Pipeline
+from backend.jobs.background_job_service import BackgroundJobService
+from backend.api.dto.serializers import serialize_get_frames_response
+import logging
+
+router = APIRouter()
+logger = logging.getLogger(__name__)
+
+@router.get("/", response_model=ApiSuccessResponse[GetFramesResponse])
+def get_processed_frames(
+    instrument: Instrument = Query(...),
+    start: str = Query(..., description = "ISO UTC observation start time"),
+    end: str = Query(..., description = "ISO UTC observation end time"),
+    limit: int = Query(15, ge=1, le=100),
+    offset: int = Query(0, ge=0),
+    pipeline: Pipeline = Depends(get_pipeline)
+) -> ApiSuccessResponse[GetFramesResponse]:
+    """
+    Retrieve of processed frames for a given observation window and instrument
+    """
+    logger.info(
+        "Processed frames retrieval requested",
+        extra={
+            "instrument":instrument,
+            "observation_start_utc":start,
+            "observation_end_utc":end
+        }
+    )
+    try:
+
+        result = pipeline.get_processed_frames(
+            instrument,
+            start,
+            end,
+            limit,
+            offset
+        )
+
+        logger.info("Processed frames retrieval successful")
+
+        return ApiSuccessResponse[GetFramesResponse](
+            data=serialize_get_frames_response(result)
+        )
+        
+    except Exception:
+        logger.exception("Processed frames retrieval failed")
+        raise
+
+@router.post("/sync", response_model=ApiSuccessResponse[JobQueuedResponse])
+def sync_processed_frames(
+    instrument: Instrument = Query(...),
+    start: str = Query(..., description = "ISO UTC observation start time"),
+    end: str = Query(..., description = "ISO UTC observation end time"),
+    pipeline: Pipeline = Depends(get_pipeline),
+    background_job_service: BackgroundJobService = Depends(get_job_service)
+) -> ApiSuccessResponse[JobQueuedResponse]:
+    """
+    Trigger the job for syncing and updating processed frames based on 
+    instrument and observation time
+
+    - Triggers metadata sync
+    - Downloads missing files
+    - Processes eligible frames
+    - Returns details of the sync operation including counts of metadata synced, files downloaded, frames marked ready and frames processed
+    """
+    logger.info(
+        "Processed frames sync job requested",
+        extra={
+            "instrument":instrument,
+            "observation_start_utc":start,
+            "observation_end_utc":end
+        }
+    )
+    try:
+        job = background_job_service.submit(
+            JobType.SYNC_PROCESSED_FRAMES,
+            pipeline.sync_processed_frames,
+            instrument,
+            start,
+            end
+        )
+
+        if job.existing:
+            logger.warning("Processed frames sync job already running",
+                           extra={"job_id": job.job.id})
+        else:
+            logger.info("Processed frames sync job queued",
+                    extra={"job_id": job.job.id})
+
+        return ApiSuccessResponse[JobQueuedResponse](
+            data=JobQueuedResponse(
+                job_id=job.job.id,
+                existing=job.existing,
+                status=job.job.status.value
+            )
+        )
+    except Exception:
+        logger.exception("Processed frames sync failed to start")
+        raise