@zigrivers/scaffold 3.4.1 → 3.5.0

package/content/knowledge/game/game-save-systems.md

@@ -0,0 +1,452 @@
+---
+name: game-save-systems
+description: Save formats, versioning and migration, cloud save integration, auto-save design, corruption detection, and platform requirements
+topics: [game-dev, save, persistence, cloud-save, corruption, migration]
+---
+
+Save systems are the custodians of player investment. A corrupted save file can destroy hundreds of hours of progress and generate visceral negative reviews. A missing cloud sync can strand progress on the wrong device. A format that cannot be versioned locks the game out of future content updates. Despite this criticality, save systems are frequently under-engineered — treated as simple serialization when they are actually distributed state management with backward compatibility, corruption recovery, and platform-specific compliance requirements. Build the save system early, test it adversarially, and treat save data loss as a severity-one bug.
+
+## Summary
+
+### Save Format Selection
+
+Three viable formats for game save data, each with distinct tradeoffs:
+
+**Binary (custom or Protocol Buffers / FlatBuffers):**
+- Smallest file size; fastest read/write; most compact on disk
+- No human readability — debugging requires custom tooling
+- Version migration requires careful field tagging (protobuf-style) or manual offset management
+- Best for: console games with tight storage quotas, games with very large save states, competitive games where save tampering must be discouraged
+
+**JSON:**
+- Human readable; easy to inspect, edit, and debug during development
+- Larger file size than binary (typically 3–10x)
+- Schema-flexible — adding new fields is trivial (absent fields get default values)
+- Version migration is straightforward (read old JSON, apply transformers, write new JSON)
+- Best for: indie/AA games, development builds, games where modding is supported
+
+**SQLite:**
+- Structured relational storage; supports queries over save data
+- ACID transactions protect against partial writes (corruption resistance built-in)
+- Larger overhead than flat files but provides indexing and query capabilities
+- Best for: games with large inventories, procedural worlds with chunk-based storage, games that need to query save data (e.g., "find all items of rarity legendary")
+
+### Save Data Architecture
+
+Separate save data into layers with different persistence frequencies:
+
+- **Profile data**: Player settings, achievements, statistics. Saved on change. Small. Shared across save slots.
+- **World state**: Level progress, NPC states, quest flags, discovered map areas. Saved at checkpoints or manual save. Medium to large.
+- **Entity state**: Positions, health, inventories of all dynamic entities. Saved at checkpoints. Potentially very large.
+- **Volatile state**: Camera position, current animation frame, particle system state. Not saved — reconstructed on load.
+
+### Versioning and Migration
+
+Every shipped build that changes the save format must increment the save version. Migration code transforms old-version saves to new-version saves.
+
+**Rules:**
+- Never delete or reorder fields in a binary format — mark fields as deprecated and add new ones
+- Always write the save version as the first field in the file header
+- Test migration from every previously shipped version to the current version
+- Keep migration code permanently — do not remove v1->v2 migration when you ship v5; a player may return after years
+- Migration must be idempotent — applying it to an already-migrated save should be a no-op
+
+### Cloud Save Integration
+
+Cloud save synchronizes progress across devices and protects against local storage loss. Each platform provides its own cloud save API.
+
+**Platform services:**
+- **Steam Cloud**: File-based; the game reads/writes local files and Steam syncs them. Simple API but the game must handle conflicts.
+- **PlayStation Plus Cloud Storage**: Automatic for PS Plus subscribers. Save data is uploaded on console suspend. The game can trigger manual uploads.
+- **Xbox Cloud Saves**: Integrated into the Connected Storage API. Automatic for all Xbox users. Supports blob-based storage with conflict resolution.
+- **iCloud (iOS/macOS)**: Key-value storage (NSUbiquitousKeyValueStore) for small data or document-based (iCloud Documents) for large saves.
+- **Google Play Games Services**: Snapshot API for Android. Supports conflict resolution callbacks.
+
+### Auto-Save Design
+
+Auto-save is expected in modern games but must be designed to avoid data loss and player frustration.
+
+**Auto-save triggers:**
+- After major accomplishments (boss defeated, quest completed, level cleared)
+- At area transitions (entering a new zone, passing through a door)
+- On a timer (every 5–15 minutes of active gameplay)
+- Before risky situations (pre-boss encounter, before point-of-no-return)
+- On application suspend (mobile backgrounding, console suspend)
+
+**Auto-save rules:**
+- Never auto-save during combat, cutscenes, or dialogue — the player may be in an unrecoverable state
+- Show a save indicator (spinning icon) during auto-save to prevent the player from quitting
+- Auto-save to a rolling set of 2–3 slots to allow recovery from bad auto-saves
+- Allow the player to disable auto-save in settings (some players want full manual control)
+
+## Deep Guidance
+
+### Save System Architecture
+
+```typescript
+// Save system with versioning, migration, corruption detection, and cloud sync
+
+interface SaveHeader {
+  magic: number;           // Magic number for format identification (e.g., 0x53415645)
+  version: number;         // Save format version — increment on every schema change
+  timestamp: number;       // Unix timestamp of save creation
+  checksum: string;        // SHA-256 hash of the payload for corruption detection
+  playTimeSeconds: number; // Total play time (for display in load screen)
+  slotIndex: number;       // Which save slot this belongs to
+}
+
+interface SavePayload {
+  profile: ProfileData;
+  world: WorldState;
+  entities: EntityState[];
+}
+
+interface ProfileData {
+  playerName: string;
+  settings: Record<string, unknown>;
+  achievements: string[];
+  statistics: Record<string, number>;
+}
+
+interface WorldState {
+  currentLevel: string;
+  questFlags: Record<string, boolean>;
+  discoveredAreas: string[];
+  npcStates: Record<string, NpcState>;
+  worldTime: number;
+}
+
+interface NpcState {
+  alive: boolean;
+  disposition: number;
+  dialogueProgress: number;
+}
+
+interface EntityState {
+  id: string;
+  type: string;
+  position: { x: number; y: number; z: number };
+  health: number;
+  inventory: InventoryItem[];
+  customData: Record<string, unknown>;
+}
+
+interface InventoryItem {
+  itemId: string;
+  quantity: number;
+  durability?: number;
+  customProperties?: Record<string, unknown>;
+}
+
+// --- Save Manager ---
+
+const CURRENT_SAVE_VERSION = 5;
+const SAVE_MAGIC = 0x53415645; // "SAVE" in ASCII
+
+class SaveManager {
+  private migrations: Map<number, MigrationFn> = new Map();
+  private redundantCopies = 2; // Number of backup copies
+
+  constructor() {
+    // Register all migrations — NEVER remove old ones
+    this.migrations.set(1, migrateV1toV2);
+    this.migrations.set(2, migrateV2toV3);
+    this.migrations.set(3, migrateV3toV4);
+    this.migrations.set(4, migrateV4toV5);
+  }
+
+  async save(slot: number, payload: SavePayload): Promise<void> {
+    const serialized = JSON.stringify(payload);
+    const checksum = await computeSHA256(serialized);
+
+    const header: SaveHeader = {
+      magic: SAVE_MAGIC,
+      version: CURRENT_SAVE_VERSION,
+      timestamp: Date.now(),
+      checksum,
+      playTimeSeconds: payload.profile.statistics["playTime"] ?? 0,
+      slotIndex: slot,
+    };
+
+    const saveData = JSON.stringify({ header, payload });
+
+    // Write primary save file
+    const primaryPath = this.getSavePath(slot, "primary");
+    await this.atomicWrite(primaryPath, saveData);
+
+    // Write redundant backup copies
+    for (let i = 0; i < this.redundantCopies; i++) {
+      const backupPath = this.getSavePath(slot, `backup_${i}`);
+      await this.atomicWrite(backupPath, saveData);
+    }
+
+    // Trigger cloud sync if available
+    await this.syncToCloud(slot, saveData);
+  }
+
+  async load(slot: number): Promise<SavePayload> {
+    // Try primary file first, fall back to backups
+    const paths = [
+      this.getSavePath(slot, "primary"),
+      ...Array.from({ length: this.redundantCopies },
+        (_, i) => this.getSavePath(slot, `backup_${i}`)
+      ),
+    ];
+
+    for (const path of paths) {
+      try {
+        const raw = await readFile(path);
+        const parsed = JSON.parse(raw);
+        const { header, payload } = parsed as {
+          header: SaveHeader;
+          payload: SavePayload;
+        };
+
+        // Validate magic number
+        if (header.magic !== SAVE_MAGIC) {
+          console.warn(`Invalid magic in ${path}, trying next copy`);
+          continue;
+        }
+
+        // Validate checksum
+        const expectedChecksum = await computeSHA256(
+          JSON.stringify(payload)
+        );
+        if (header.checksum !== expectedChecksum) {
+          console.warn(`Checksum mismatch in ${path}, trying next copy`);
+          continue;
+        }
+
+        // Apply migrations if save version is older
+        let migrated = payload;
+        for (let v = header.version; v < CURRENT_SAVE_VERSION; v++) {
+          const migration = this.migrations.get(v);
+          if (!migration) {
+            throw new Error(
+              `Missing migration from v${v} to v${v + 1}`
+            );
+          }
+          migrated = migration(migrated);
+          console.log(`Migrated save from v${v} to v${v + 1}`);
+        }
+
+        return migrated;
+      } catch (err) {
+        console.warn(`Failed to load ${path}: ${err}`);
+        continue;
+      }
+    }
+
+    throw new Error(
+      `All save copies for slot ${slot} are corrupted or missing`
+    );
+  }
+
+  // Atomic write: write to temp file, then rename
+  // Prevents corruption if the process is killed during write
+  private async atomicWrite(
+    path: string, data: string
+  ): Promise<void> {
+    const tempPath = path + ".tmp";
+    await writeFile(tempPath, data);
+    await renameFile(tempPath, path);
+  }
+
+  private getSavePath(slot: number, suffix: string): string {
+    return `saves/slot_${slot}_${suffix}.sav`;
+  }
+
+  private async syncToCloud(
+    slot: number, data: string
+  ): Promise<void> {
+    // Platform-specific cloud sync implementation
+    // Steam: write to Steam Cloud path, auto-synced
+    // PlayStation: trigger SCE save data upload
+    // Xbox: write to Connected Storage container
+    // Mobile: write to iCloud/Google Play snapshot
+  }
+}
+
+type MigrationFn = (payload: any) => SavePayload;
+
+// Example migrations — each transforms the old format to the next version
+function migrateV1toV2(payload: any): any {
+  // V2 added NPC disposition tracking
+  if (payload.world?.npcStates) {
+    for (const npc of Object.values(payload.world.npcStates) as any[]) {
+      npc.disposition = npc.disposition ?? 50; // Default neutral
+    }
+  }
+  return payload;
+}
+function migrateV2toV3(payload: any): any { return payload; }
+function migrateV3toV4(payload: any): any { return payload; }
+function migrateV4toV5(payload: any): any { return payload; }
+
+// Placeholder functions for file I/O and hashing
+async function readFile(path: string): Promise<string> { return ""; }
+async function writeFile(path: string, data: string): Promise<void> {}
+async function renameFile(from: string, to: string): Promise<void> {}
+async function computeSHA256(data: string): Promise<string> { return ""; }
+```
+
+### Corruption Detection and Recovery
+
+Save corruption occurs from power loss during write, storage media failure, interrupted cloud sync, or bugs in serialization code. A robust save system assumes corruption will happen and plans for recovery.
+
+**Detection mechanisms:**
+- **Checksum validation**: Compute a SHA-256 (or CRC-32 for speed) hash of the payload at save time. Store it in the header. On load, recompute and compare. Any mismatch means corruption.
+- **Magic number**: The first bytes of the file should be a known constant (e.g., 0x53415645). If the magic number is wrong, the file is not a valid save at all.
+- **Structural validation**: After deserialization, validate that required fields exist and have valid ranges (health >= 0, position within world bounds, enum values within valid set).
+
+**Recovery strategies:**
+- **Redundant saves**: Write 2–3 copies of every save file. If the primary is corrupted, load the newest valid backup. This is the single most effective corruption defense.
+- **Atomic writes**: Write to a temporary file, then atomically rename it over the target. This prevents half-written files if the process is killed mid-write.
+- **Write-ahead log (WAL)**: For SQLite-based saves, WAL mode provides crash recovery. For custom formats, write an intent log before modifying the save, and replay it on next load if the save is incomplete.
+- **Tombstone markers**: Before starting a save write, create a `.saving` marker file. Delete it after write completes. On load, if the marker exists, the last save was interrupted — fall back to backup.
+- **Never overwrite the only copy**: Always write to a new file or backup slot before deleting/overwriting the old one.
+
+### Platform-Specific Save Requirements
+
+Each platform has unique requirements that affect save system design:
+
+**PlayStation:**
+- Save data uses the PlayStation save data API (libSceUserService + libSceSaveData)
+- Save data is associated with a user account and stored in system-managed directories
+- The game must display a "saving" indicator and prevent the user from powering off during save (platform requirement)
+- Trophy data must remain consistent with save data (if a trophy is earned, the save must reflect that state)
+- Maximum save data size varies by title profile (typically 256 MB–1 GB)
+
+**Xbox:**
+- Connected Storage API for save data — blob-based with containers
+- Save data is tied to Xbox Live account and auto-synced to cloud for all users (not just subscribers)
+- The game must handle the case where cloud save is newer than local save (conflict resolution UI required)
+- Quick Resume: save state must persist through suspend/resume cycles; the game should serialize critical state to Connected Storage on suspend
+
+**Nintendo Switch:**
+- Save data uses the Nintendo save data API
+- Strict save data size limits (varies by title approval, typically 32 MB–256 MB)
+- No cloud save backup for non-Nintendo Switch Online subscribers
+- Save data is bound to the console, not the user (complicates console transfer scenarios)
+- The game must implement its own backup strategy since the platform provides limited protection
+
+**Steam (PC):**
+- Steam Cloud provides transparent file sync — write files to a designated local path, Steam uploads them automatically
+- Configure Steam Cloud settings in the Steamworks partner portal (max file count, max total size)
+- Handle the Steam Cloud conflict dialog: Steam shows a prompt when local and cloud saves diverge; the game should display meaningful timestamps and progress info to help the player choose
+- ISteamRemoteStorage API for programmatic control; Auto-Cloud for zero-code file sync
+
+**Mobile (iOS):**
+- iCloud key-value store for small data (<1 MB total across all keys) — simple but limited
+- iCloud Documents for larger save files — requires managing file coordinators for conflict resolution
+- NSFileProtection for save file encryption at rest (required for games handling sensitive user data)
+- Handle the case where iCloud is disabled or full — fall back to local-only with a warning
+- App deletion on iOS deletes local save data — iCloud is the only persistence across reinstalls
+
+**Mobile (Android):**
+- Google Play Games Saved Games (Snapshot API) for cloud saves — supports binary data and cover images
+- Conflict resolution callback provides both conflicting snapshots; the game must merge or choose
+- Internal storage (`getFilesDir()`) for local saves — survives app updates but not uninstall
+- External storage is accessible to other apps and the user — do not store saves there without encryption
+- On Android, `onSaveInstanceState` / `onRestoreInstanceState` handles OS-initiated process death (out-of-memory kill); save critical state there in addition to explicit save files
+
+### Cloud Save Conflict Resolution
+
+When a player plays on two devices without syncing, cloud save conflicts occur. The game must resolve them without losing progress.
+
+**Resolution strategies:**
+- **Latest timestamp wins**: Simple but can lose meaningful progress from the older save. Acceptable for simple games.
+- **Highest progress wins**: Compare completion percentage, level, or total play time. Choose the save with more progress. May lose recent changes if the player switched tasks.
+- **Merge**: Combine data from both saves. Achievements and discoveries are unioned (player gets everything from both saves). Conflicting values (position, current quest) use the newer save. This is the most player-friendly but hardest to implement.
+- **Player choice**: Present both saves with timestamps, play time, and progress summary. Let the player decide. This is the safest approach for games with meaningful branching choices.
+
+**Implementation rule:** Never silently discard a save. If the game auto-resolves a conflict, log it and keep the discarded save as a hidden backup for a retention period (7–30 days).
+
+### Save File Security
+
+For competitive and economy-based games, save file tampering is a concern. Players editing save files to give themselves unlimited currency or items undermines the game's integrity.
+
+**Defense layers:**
+- **Checksum validation**: Detect tampering by validating the stored checksum against the payload. A casual editor will not know to update the checksum.
+- **Encryption**: Encrypt the save payload with a key derived from the player's account ID or a hardware identifier. This stops plaintext editing. Use AES-256-GCM for authenticated encryption.
+- **Server-side validation**: For games with online economies, validate save data against server records. If the client claims to have 999,999 gold but the server's ledger shows 500, reject the save.
+- **Binary format**: Binary formats are harder to edit than JSON/XML. Not a security measure on its own, but raises the effort bar.
+- **Obfuscation**: Rename fields, reorder data, add dummy fields. Again, not security, but raises the effort bar above casual tampering.
+
+**Important caveat:** In single-player games, players editing their own saves is a feature, not a bug. Many games have thriving modding communities built on save editing. Only invest in save security when multiplayer fairness or real-money economies are affected.
+
+### Auto-Save UX Patterns
+
+```yaml
+# Auto-save configuration — adjust per game genre and platform
+
+auto_save:
+  enabled: true
+  player_can_disable: true
+
+  triggers:
+    - type: "timer"
+      interval_minutes: 10
+      conditions:
+        - "not_in_combat"
+        - "not_in_cutscene"
+        - "not_in_menu"
+        - "player_is_grounded"
+
+    - type: "event"
+      events:
+        - "quest_completed"
+        - "boss_defeated"
+        - "area_entered"
+        - "major_item_acquired"
+        - "checkpoint_reached"
+
+    - type: "app_lifecycle"
+      events:
+        - "app_suspending"       # Mobile background, console suspend
+        - "app_losing_focus"     # Alt-tab on PC (optional)
+
+  slot_management:
+    strategy: "rolling"
+    slot_count: 3              # Rotate across 3 auto-save slots
+    separate_from_manual: true # Auto-saves don't overwrite manual saves
+
+  ui:
+    show_indicator: true       # Spinning icon during save
+    indicator_duration_ms: 2000 # Minimum display time (even if save is instant)
+    indicator_position: "bottom_right"
+    block_quit_during_save: true
+    show_notification: false   # Don't spam "Game saved" — the icon is enough
+
+  safety:
+    min_interval_seconds: 30   # Prevent rapid-fire auto-saves
+    validate_state_before_save: true  # Don't save if player health <= 0
+    write_to_backup_first: true       # Atomic save via backup rotation
+```
+
+### Save Data Size Optimization
+
+Large save files cause slow saves (visible hitches), slow cloud sync, and may exceed platform storage quotas.
+
+**Optimization techniques:**
+- **Delta saves**: Store only what changed from the default state. An NPC that is alive at its default position with default dialogue needs zero bytes. Only NPCs that have moved, died, or changed state need entries.
+- **Bit packing**: Boolean flags (quest completed, area discovered, achievement unlocked) pack 8 values per byte instead of 1 per byte. A game with 1000 boolean flags needs 125 bytes packed vs 1000 bytes unpacked.
+- **String interning**: Replace repeated strings with integer IDs. "legendary_sword_of_fire" repeated 50 times in an inventory costs 50 * 26 bytes. An ID table + integer references costs 26 + 50 * 2 bytes.
+- **Compression**: Apply zlib/LZ4/zstd compression to the serialized payload before writing to disk. Typical compression ratios for game save data: 3:1 to 10:1 for JSON, 1.5:1 to 3:1 for already-compact binary.
+- **Chunked saves**: For open-world games, save each world region as a separate chunk. Only load/save chunks that have been visited. Unvisited regions have no save data (they use default state).
+
+### Testing Save Systems
+
+Save systems require adversarial testing because failures are catastrophic and often invisible until the player next loads.
+
+**Test cases (minimum):**
+1. Save and load: verify all data round-trips correctly (the obvious one)
+2. Kill the process during save (simulate power loss): verify the backup save loads correctly
+3. Corrupt the primary save file (flip random bytes): verify the backup loads and the player is warned
+4. Load a save from every previously shipped version: verify migration succeeds
+5. Fill the save with maximum data (max inventory, all quests complete, all areas discovered): verify it stays within platform size limits
+6. Save on device A, sync to cloud, load on device B: verify full data transfer
+7. Save on device A, go offline, save on device B, go online: verify conflict resolution works correctly
+8. Delete the save file while the game is running: verify graceful error handling
+9. Save to a full disk: verify the game detects the write failure and does not corrupt existing saves
+10. Load a save created on a different platform (if cross-save is supported): verify endianness, path, and format compatibility