@polymorphism-tech/morph-spec 8.46.0 → 8.47.0

package/framework/standards/backend/integrations/hangfire/hangfire-jobs.md

@@ -5,34 +5,29 @@
 > **Keywords:** hangfire, background job, recurring, fire-and-forget, scheduled
 > **Load When:** hangfire or background job keywords detected
 
-**Verified against:** .NET 10 + Hangfire 1.8 (Core/SqlServer/AspNetCore). Last-verified: 2026-05-20.
+**Verified against:** .NET 10 (SDK 10.0.400-preview) + Hangfire.Core/Hangfire.AspNetCore 1.8.25 + Hangfire.PostgreSql 1.21.1 + Newtonsoft.Json 13.0.4. The startup, job/handler, dashboard-filter and idempotency blocks compile with `TreatWarningsAsErrors`; the Job Types calls compile inside a wrapper method; the per-job DI scope (`AspNetCoreJobActivator` → `IServiceScopeFactory`) and the `PostgreSqlStorageOptions` defaults were read by reflection on those assemblies; the pooled-endpoint limits (LISTEN/NOTIFY, migrations on the direct string) come from the Neon connection-pooling docs. Cron strings are documentary. Last-verified: 2026-10-03.
 
 ---
 
-Background job processing for .NET applications with Hangfire.
+Background job processing for .NET 10 VSA applications with Hangfire on Neon Postgres.
 
 ---
 
-## Overview
+## When to use Hangfire
 
-Hangfire provides:
-- Fire-and-forget jobs
-- Delayed jobs
-- Recurring jobs (cron schedules)
-- Job continuations
-- Built-in dashboard
-- Retry logic with exponential backoff
+Hangfire is in the stack **only with a real reason**: work that must survive a process restart, run on a schedule, or be retried later (daily reindex, delayed reminders, retrying a flaky third party). Work that fits in the request (send one email after a signup, index one uploaded document) runs in the slice handler. Record the reason in `decisions.md`.
 
-**Stack:** .NET 10
+Hangfire provides fire-and-forget, delayed and recurring (cron) jobs, continuations, a dashboard and automatic retries.
 
 ---
 
 ## Core Principles
 
-1. **Stateless Jobs**: Job methods should be stateless and idempotent
-2. **Minimal Dependencies**: Jobs should not depend on HTTP context or scoped services
-3. **Explicit Retries**: Configure retry behavior per job type
-4. **Monitor Dashboard**: Use Hangfire Dashboard for job monitoring
+1. **Jobs are thin entry points**: a job class lives in the slice folder and calls the slice handler; the logic and the `Result` live in the handler
+2. **Idempotent**: a job can run twice (retry, server restart) -- check state and return if already done
+3. **No HTTP context**: jobs run outside a request; pass ids, never entities or `HttpContext` data
+4. **Explicit retries**: `[AutomaticRetry]` per job; no hand-written retry loops
+5. **Protected dashboard**: authenticated + Admin in production
 
 ---
 
@@ -40,195 +35,190 @@ Hangfire provides:
 
 ### Install Packages
 
-```bash
-dotnet add package Hangfire.Core
-dotnet add package Hangfire.SqlServer
-dotnet add package Hangfire.AspNetCore
+```xml
+<PackageReference Include="Hangfire.Core" Version="1.8.25" />
+<PackageReference Include="Hangfire.AspNetCore" Version="1.8.25" />
+<PackageReference Include="Hangfire.PostgreSql" Version="1.21.1" />
+<PackageReference Include="Newtonsoft.Json" Version="13.0.4" />
 ```
 
+`Hangfire.Core` 1.8.25 depends on `Newtonsoft.Json` 11.0.1, which carries a high-severity advisory (GHSA-5crp-9r3c-p9vr): restore fails with `NU1903` under `TreatWarningsAsErrors`. The explicit `Newtonsoft.Json` reference lifts it to a patched version.
+
 ### Program.cs Configuration
 
 ```csharp
 // Program.cs
 using Hangfire;
-using Hangfire.SqlServer;
+using Hangfire.PostgreSql;
 
 var builder = WebApplication.CreateBuilder(args);
 
-// Add Hangfire services
 builder.Services.AddHangfire(configuration => configuration
     .SetDataCompatibilityLevel(CompatibilityLevel.Version_180)
     .UseSimpleAssemblyNameTypeSerializer()
     .UseRecommendedSerializerSettings()
-    .UseSqlServerStorage(
-        builder.Configuration.GetConnectionString("HangfireConnection"),
-        new SqlServerStorageOptions
-        {
-            CommandBatchMaxTimeout = TimeSpan.FromMinutes(5),
-            SlidingInvisibilityTimeout = TimeSpan.FromMinutes(5),
-            QueuePollInterval = TimeSpan.Zero,
-            UseRecommendedIsolationLevel = true,
-            DisableGlobalLocks = true
-        }));
+    .UsePostgreSqlStorage(options =>
+        options.UseNpgsqlConnection(builder.Configuration.GetConnectionString("HangfireConnection"))));
 
 builder.Services.AddHangfireServer();
 
 var app = builder.Build();
 
-// Hangfire Dashboard (protect in production!)
 app.UseHangfireDashboard("/hangfire", new DashboardOptions
 {
-    Authorization = new[] { new HangfireAuthorizationFilter() }
+    Authorization = [new HangfireAuthorizationFilter()]
 });
 
 app.Run();
 ```
 
+- **Storage on Neon Postgres** (`Hangfire.PostgreSql`), not SQL Server. The `UsePostgreSqlStorage(string, ...)` overloads are `[Obsolete]` in 1.21; use the `options => options.UseNpgsqlConnection(...)` form above.
+- **`HangfireConnection` points at the direct (unpooled) Neon endpoint**, like migrations (`infrastructure/neon/neon-setup.md`): the storage installs its schema at startup (`PrepareSchemaIfNecessary = true` by default), and Neon recommends the direct string for migrations; and if you turn on `EnableLongPolling` it uses `LISTEN/NOTIFY`, which Neon's PgBouncer (transaction mode) does not support.
+- Defaults read from `PostgreSqlStorageOptions` 1.21.1: `SchemaName = "hangfire"`, `QueuePollInterval = 15 s`, `InvisibilityTimeout = 30 min`, `DistributedLockTimeout = 10 min`, `EnableLongPolling = false`.
+- The connection string comes from configuration (user-secrets in development, Coolify environment variables in production), never from code.
+
 ---
 
 ## Job Types
 
+The examples enqueue the job classes of the next section. Hangfire replaces a `CancellationToken` argument (pass `CancellationToken.None`) with the job's own cancellation token at execution time.
+
 ### Fire-and-Forget Jobs
 
 ```csharp
-// Enqueue immediately
-BackgroundJob.Enqueue<IEmailService>(x => x.SendWelcomeEmail(userId));
-
-// Alternative: Static method
-BackgroundJob.Enqueue(() => Console.WriteLine("Hello, world!"));
+BackgroundJob.Enqueue<SendWelcomeEmailJob>(job => job.RunAsync(userId, CancellationToken.None));
 ```
 
 ### Delayed Jobs
 
 ```csharp
-// Run after 1 hour
-BackgroundJob.Schedule<IReportService>(
-    x => x.GenerateMonthlyReport(),
+BackgroundJob.Schedule<SendWelcomeEmailJob>(
+    job => job.RunAsync(userId, CancellationToken.None),
     TimeSpan.FromHours(1));
 
-// Run at specific time
-BackgroundJob.Schedule<INotificationService>(
-    x => x.SendReminder(userId),
-    DateTimeOffset.UtcNow.AddDays(7));
+BackgroundJob.Schedule<SendWelcomeEmailJob>(
+    job => job.RunAsync(userId, CancellationToken.None),
+    timeProvider.GetUtcNow().AddDays(7));
 ```
 
 ### Recurring Jobs
 
 ```csharp
-// Run daily at 2 AM UTC
-RecurringJob.AddOrUpdate<ICleanupService>(
-    "cleanup-old-data",
-    x => x.CleanupOldRecords(),
+RecurringJob.AddOrUpdate<CleanupExpiredSessionsJob>(
+    "cleanup-expired-sessions",
+    job => job.RunAsync(CancellationToken.None),
     Cron.Daily(2));
 
-// Run every 15 minutes
-RecurringJob.AddOrUpdate<ISyncService>(
-    "sync-external-data",
-    x => x.SyncData(),
+RecurringJob.AddOrUpdate<CleanupExpiredSessionsJob>(
+    "cleanup-expired-sessions-frequent",
+    job => job.RunAsync(CancellationToken.None),
     "*/15 * * * *");
 ```
 
+Recurring job ids are stable strings (`AddOrUpdate` with the same id replaces the schedule). Cron times are UTC.
+
 ### Job Continuations
 
 ```csharp
-var parentJobId = BackgroundJob.Enqueue<IOrderService>(
-    x => x.ProcessOrder(orderId));
+var parentJobId = BackgroundJob.Enqueue<SendWelcomeEmailJob>(
+    job => job.RunAsync(userId, CancellationToken.None));
 
-BackgroundJob.ContinueJobWith<IEmailService>(
+BackgroundJob.ContinueJobWith<CleanupExpiredSessionsJob>(
     parentJobId,
-    x => x.SendOrderConfirmation(orderId));
+    job => job.RunAsync(CancellationToken.None));
 ```
 
 ---
 
-## Job Service Pattern
+## Job + Handler Pattern (in the slice)
 
 ```csharp
-// Services/Jobs/IEmailJobService.cs
-public interface IEmailJobService
+// Features/UserFeature/SendWelcomeEmail/SendWelcomeEmail.cs
+using Hangfire;
+using Microsoft.EntityFrameworkCore;
+using Microsoft.Extensions.Logging;
+
+public sealed record SendWelcomeEmailRequest(Guid UserId);
+
+public static class UserErrors
 {
-    Task SendWelcomeEmailAsync(string userId);
-    Task SendPasswordResetAsync(string email);
+    public static Error NotFound(Guid userId) =>
+        Error.NotFound("Users.NotFound", $"The User with Id '{userId}' was not found");
 }
 
-// Services/Jobs/EmailJobService.cs
-public class EmailJobService : IEmailJobService
+public interface IEmailSender
 {
-    private readonly IDbContextFactory<AppDbContext> _dbFactory;
-    private readonly IEmailSender _emailSender;
+    Task SendAsync(string to, string subject, string body, CancellationToken cancellationToken);
+}
 
-    public EmailJobService(
-        IDbContextFactory<AppDbContext> dbFactory,
-        IEmailSender emailSender)
+public sealed class SendWelcomeEmailHandler(
+    ApplicationDbContext db,
+    IEmailSender emailSender,
+    TimeProvider timeProvider) : IHandler<SendWelcomeEmailRequest, Result>
+{
+    public async Task<Result> HandleAsync(SendWelcomeEmailRequest request, CancellationToken cancellationToken)
     {
-        _dbFactory = dbFactory;
-        _emailSender = emailSender;
-    }
+        var user = await db.Users.FirstOrDefaultAsync(u => u.Id == request.UserId, cancellationToken);
+        if (user is null)
+            return UserErrors.NotFound(request.UserId);
 
-    public async Task SendWelcomeEmailAsync(string userId)
-    {
-        // Use IDbContextFactory for background jobs
-        await using var db = await _dbFactory.CreateDbContextAsync();
+        if (user.WelcomeEmailSentAt is not null)
+            return Result.Success();
 
-        var user = await db.Users.FindAsync(userId);
-        if (user == null) return;
+        await emailSender.SendAsync(user.Email, "Welcome!", $"Hello {user.Name}!", cancellationToken);
 
-        await _emailSender.SendAsync(
-            user.Email,
-            "Welcome!",
-            $"Hello {user.Name}!");
+        user.WelcomeEmailSentAt = timeProvider.GetUtcNow();
+        await db.SaveChangesAsync(cancellationToken);
+        return Result.Success();
     }
 }
 
-// Register in DI
-builder.Services.AddScoped<IEmailJobService, EmailJobService>();
+public sealed class SendWelcomeEmailJob(
+    IHandler<SendWelcomeEmailRequest, Result> handler,
+    ILogger<SendWelcomeEmailJob> logger)
+{
+    [AutomaticRetry(Attempts = 3, DelaysInSeconds = [60, 300, 900])]
+    public async Task RunAsync(Guid userId, CancellationToken cancellationToken)
+    {
+        var result = await handler.HandleAsync(new SendWelcomeEmailRequest(userId), cancellationToken);
+        if (result.IsFailure)
+            logger.LogWarning("Welcome email for {UserId} not sent: {Code}", userId, result.Error.Code);
+    }
+}
 ```
 
+- **The handler and the `ApplicationDbContext` are injected directly.** `Hangfire.AspNetCore` activates each job through `AspNetCoreJobActivator`, which opens an `IServiceScope` per job execution (`IServiceScopeFactory`) and disposes it at the end, so scoped services, including the `DbContext`, get a fresh instance per job, exactly like a request. `IDbContextFactory<ApplicationDbContext>` is only needed when **one** job runs several units of work in parallel.
+- No `IEmailJobService`/`XService` layer: the job is the entry point, the handler is the logic, `IEmailSender` is the port to the external provider.
+- Register the job class nowhere: the activator resolves it from the scope (its constructor dependencies must be registered).
+
 ---
 
 ## Retry & Error Handling
 
-### Custom Retry Attribute
+### Expected failures vs. unexpected failures
 
-```csharp
-[AutomaticRetry(Attempts = 3, DelaysInSeconds = new[] { 60, 300, 900 })]
-public async Task ProcessPaymentAsync(string paymentId)
-{
-    // Job logic with automatic retry on failure
-}
-```
+| Failure | Where it surfaces | What Hangfire does |
+|---------|-------------------|--------------------|
+| Expected (user gone, already processed, business rule) | `Result` failure from the handler | Job logs and **completes** -- retrying would not change the outcome |
+| Unexpected (database down, provider timeout, HTTP 5xx) | Exception thrown by EF Core/HttpClient, not caught anywhere | Job **fails** and `[AutomaticRetry]` schedules the next attempt |
 
-### Manual Retry Logic
+There is no `try/catch` in the job or the handler: the exception the infrastructure already throws is the retry signal, and a transient provider failure is not mapped into a `Result` (that would turn a retryable outage into a completed job).
+
+### Custom Retry Attribute
 
 ```csharp
-public async Task SendEmailWithRetryAsync(string email)
-{
-    int maxRetries = 3;
-    int attempt = 0;
+using Hangfire;
 
-    while (attempt < maxRetries)
-    {
-        try
-        {
-            await _emailSender.SendAsync(email, "Subject", "Body");
-            return; // Success
-        }
-        catch (Exception ex)
-        {
-            attempt++;
-            if (attempt >= maxRetries)
-            {
-                // Log and give up
-                _logger.LogError(ex, "Failed to send email after {Attempts} attempts", maxRetries);
-                throw;
-            }
-
-            await Task.Delay(TimeSpan.FromSeconds(Math.Pow(2, attempt)));
-        }
-    }
+public sealed class SyncInvoicesJob
+{
+    [AutomaticRetry(Attempts = 5, DelaysInSeconds = [60, 300, 900, 3600, 21600])]
+    public Task RunAsync(CancellationToken cancellationToken) => Task.CompletedTask;
 }
 ```
 
+- `Attempts` counts retries after the first failure; after the last one the job moves to the Failed list on the dashboard (`OnAttemptsExceeded = AttemptsExceededAction.Delete` drops it instead).
+- Never write a manual `while (attempt < max) { try ... catch ... }` loop inside a job: it hides the attempts from the dashboard and doubles the retry policy.
+
 ---
 
 ## Dashboard Authorization
@@ -237,71 +227,59 @@ public async Task SendEmailWithRetryAsync(string email)
 // Filters/HangfireAuthorizationFilter.cs
 using Hangfire.Dashboard;
 
-public class HangfireAuthorizationFilter : IDashboardAuthorizationFilter
+public sealed class HangfireAuthorizationFilter : IDashboardAuthorizationFilter
 {
     public bool Authorize(DashboardContext context)
     {
         var httpContext = context.GetHttpContext();
 
-        // Allow in development
-        if (httpContext.Request.Host.Host.Contains("localhost"))
+        if (httpContext.RequestServices.GetRequiredService<IHostEnvironment>().IsDevelopment())
             return true;
 
-        // Require authenticated user with Admin role
         return httpContext.User.Identity?.IsAuthenticated == true
             && httpContext.User.IsInRole("Admin");
     }
 }
 ```
 
+Never trust `Request.Host` to detect "localhost": the `Host` header is set by the client, so `Host: localhost` from the internet would open the dashboard. Decide by the hosting environment (server-side), and require an authenticated Admin everywhere else.
+
 ---
 
 ## Best Practices
 
-### Avoid Scoped Services
+### Scoped services: inject them, the activator scopes per job
 
 ```csharp
-// ❌ BAD: Don't inject DbContext directly
-public class BadJobService
-{
-    private readonly AppDbContext _db; // Scoped service!
+using Microsoft.EntityFrameworkCore;
 
-    public BadJobService(AppDbContext db) => _db = db;
-}
-
-// ✅ GOOD: Use IDbContextFactory
-public class GoodJobService
+// ✅ Hangfire.AspNetCore creates a DI scope per job: a scoped DbContext is safe here
+public sealed class RecalculateTotalsJob(ApplicationDbContext db)
 {
-    private readonly IDbContextFactory<AppDbContext> _dbFactory;
-
-    public GoodJobService(IDbContextFactory<AppDbContext> dbFactory)
-        => _dbFactory = dbFactory;
-
-    public async Task ProcessAsync()
-    {
-        await using var db = await _dbFactory.CreateDbContextAsync();
-        // Use db...
-    }
+    public Task<int> RunAsync(CancellationToken cancellationToken) =>
+        db.Orders.Where(o => o.Status == OrderStatus.Pending).CountAsync(cancellationToken);
 }
 ```
 
+What is **not** safe: capturing a `DbContext` (or any scoped service) from a request and passing it into `BackgroundJob.Enqueue(() => ...)` -- the request scope is disposed long before the job runs. Pass ids; let the job resolve its own dependencies.
+
 ### Idempotent Jobs
 
 ```csharp
-public async Task ProcessOrderAsync(string orderId)
-{
-    await using var db = await _dbFactory.CreateDbContextAsync();
+using Microsoft.EntityFrameworkCore;
 
-    var order = await db.Orders.FindAsync(orderId);
-    if (order == null || order.Status == OrderStatus.Processed)
+public sealed class ProcessOrderHandler(ApplicationDbContext db)
+{
+    public async Task<Result> HandleAsync(Guid orderId, CancellationToken cancellationToken)
     {
-        // Already processed or doesn't exist - skip
-        return;
-    }
+        var order = await db.Orders.FirstOrDefaultAsync(o => o.Id == orderId, cancellationToken);
+        if (order is null || order.Status == OrderStatus.Processed)
+            return Result.Success();
 
-    // Process order...
-    order.Status = OrderStatus.Processed;
-    await db.SaveChangesAsync();
+        order.Status = OrderStatus.Processed;
+        await db.SaveChangesAsync(cancellationToken);
+        return Result.Success();
+    }
 }
 ```
 
@@ -337,9 +315,10 @@ Access the dashboard at `/hangfire` (configured in Program.cs).
 | Issue | Cause | Solution |
 |-------|-------|----------|
 | Jobs not processing | Hangfire server not started | Ensure `AddHangfireServer()` is called |
-| DbContext errors | Using scoped DbContext | Use `IDbContextFactory` |
+| `ObjectDisposedException` on DbContext | A request-scoped instance captured into the job | Pass ids; inject the DbContext into the job class (scoped per job) |
 | Jobs timing out | Long-running job | Split into smaller jobs with continuations |
-| Memory leaks | Not disposing DbContext | Use `await using` for DbContext |
+| Restore fails with `NU1903` (Newtonsoft.Json) | `Hangfire.Core` pulls 11.0.1 | Reference `Newtonsoft.Json` 13.0.4 explicitly |
+| `EnableLongPolling` never wakes the server on the `-pooler` endpoint | `LISTEN/NOTIFY` is unsupported in PgBouncer transaction mode (Neon docs) | `HangfireConnection` on the direct endpoint |
 
 ---
 
@@ -347,6 +326,7 @@ Access the dashboard at `/hangfire` (configured in Program.cs).
 
 - [Hangfire Documentation](https://docs.hangfire.io/)
 - [Hangfire Best Practices](https://docs.hangfire.io/en/latest/best-practices.html)
+- [Hangfire.PostgreSql](https://github.com/hangfire-postgres/Hangfire.PostgreSql)
 - [Cron Expression Generator](https://crontab.guru/)
 
 ---