secenvY 0.1.0__tar.gz

secenvy-0.1.0/PKG-INFO

@@ -0,0 +1,129 @@
+Metadata-Version: 2.4
+Name: secenvY
+Version: 0.1.0
+Summary: Gymnasium Environment for AI-driven Android Security Analysis, Dynamic Fuzzing, and Penetration Testing
+Author: Android Security Agent Team
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/d4em0ny/android-security-agent
+Project-URL: Repository, https://github.com/d4em0ny/android-security-agent.git
+Project-URL: Issues, https://github.com/d4em0ny/android-security-agent/issues
+Keywords: reinforcement-learning,gymnasium,android,security,penetration-testing,fuzzing,rag,dueling-dqn
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Security
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: gymnasium>=0.29.0
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: pydantic-settings>=2.0.0
+Requires-Dist: torch>=2.0.0
+Requires-Dist: chromadb>=0.5.0
+Requires-Dist: google-genai>=0.1.0
+Requires-Dist: sentence-transformers>=2.2.0
+Requires-Dist: python-dotenv>=1.0.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: fastapi>=0.100.0
+Requires-Dist: uvicorn>=0.20.0
+Requires-Dist: websockets>=11.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Requires-Dist: build>=1.0.0; extra == "dev"
+Requires-Dist: twine>=5.0.0; extra == "dev"
+
+# Android Security Environment (`secenvY`)
+
+A high-performance Gymnasium environment and Reinforcement Learning brain layer for AI-driven Android penetration testing, automated vulnerability discovery, and dynamic security analysis.
+
+---
+
+## 🌟 Key Features
+
+- **Standard Gymnasium `Discrete(8)` Action Space:**
+  1. `0: TAP` (UI Exploration) — Tap clickable element / primary interactable.
+  2. `1: TYPE_TEXT` (UI Exploration) — Type contextual credentials / input fields.
+  3. `2: SCROLL` (UI Exploration) — Scroll down to reveal off-screen views.
+  4. `3: BACK` (UI Exploration) — Press Android system back button.
+  5. `4: INJECT_SQL_PAYLOAD` (Security Testing) — Fuzz active input fields / queries with SQL syntax.
+  6. `5: START_ACTIVITY` (Security Testing) — Launch target exported Activity component directly via `am start`.
+  7. `6: QUERY_PROVIDER` (Security Testing) — Query target Content Provider URI directly.
+  8. `7: FINISH` (Lifecycle) — Conclude security audit session cleanly with completion rewards.
+- **117-Dimensional Normalized Observation Tensor:** Comprehensive 17-group state representation bounded strictly in `[0.0, 1.0]` (`gymnasium.spaces.Box(low=0.0, high=1.0, shape=(117,), dtype=np.float32)`).
+- **Sub-Millisecond Pipeline (< 1ms SLA):** Highly optimized validation, building, and encoding pipeline executing end-to-end in ~0.89 ms.
+- **Multi-Tier RAG-Driven Reward Engine:** Objective reward formulation combining step efficiency, screen discovery bonuses, severity-weighted RAG evaluations, and anti-farming deduplication.
+- **Production WebSocket Gateway & Server:** Synchronous `WebSocketSystemGateway` and standalone `server.py` FastAPI routing hub bridging RL environments, physical Android devices, and desktop dashboards.
+
+---
+
+## 📦 Installation
+
+```bash
+# Install from PyPI
+pip install secenvY
+
+# Or install from source in editable mode
+git clone https://github.com/d4em0ny/android-security-agent.git
+cd android-security-agent
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+---
+
+## 🚀 Quickstart Usage
+
+```python
+import gymnasium as gym
+import secenvY
+
+# 1. Create registered environment
+env = gym.make("AndroidSecurityEnv-v0")
+
+# 2. Reset environment to clean initial state
+obs, info = env.reset(seed=42, options={"pkg": "com.vulnerable.bank"})
+print(f"Observation Shape: {obs.shape}")  # (117,)
+print(f"Action Space:      {env.action_space}") # Discrete(8)
+print(f"Action Mask:       {info['action_mask']}")
+
+# 3. Step through episode
+terminated = False
+truncated = False
+while not (terminated or truncated):
+    action_mask = info.get("action_mask")
+    # Sample or select discrete action (0..7)
+    action = env.action_space.sample(mask=action_mask)
+    obs, reward, terminated, truncated, info = env.step(action)
+    print(f"Step {info['step']} -> Reward: {reward:+.3f} (Terminated: {terminated})")
+
+env.close()
+```
+
+---
+
+## 🧪 Running Tests & Verification
+
+```bash
+# Run all unit and integration tests
+pytest tests/ -v
+
+# Run performance benchmark (< 5ms SLA verification)
+pytest tests/test_pipeline_performance.py -v -s
+
+# Run static type checker
+mypy secenvY/ server.py main.py --config-file mypy.ini
+```
+
+---
+
+## 📄 License
+
+Apache-2.0 License.

secenvy-0.1.0/README.md

@@ -0,0 +1,88 @@
+# Android Security Environment (`secenvY`)
+
+A high-performance Gymnasium environment and Reinforcement Learning brain layer for AI-driven Android penetration testing, automated vulnerability discovery, and dynamic security analysis.
+
+---
+
+## 🌟 Key Features
+
+- **Standard Gymnasium `Discrete(8)` Action Space:**
+  1. `0: TAP` (UI Exploration) — Tap clickable element / primary interactable.
+  2. `1: TYPE_TEXT` (UI Exploration) — Type contextual credentials / input fields.
+  3. `2: SCROLL` (UI Exploration) — Scroll down to reveal off-screen views.
+  4. `3: BACK` (UI Exploration) — Press Android system back button.
+  5. `4: INJECT_SQL_PAYLOAD` (Security Testing) — Fuzz active input fields / queries with SQL syntax.
+  6. `5: START_ACTIVITY` (Security Testing) — Launch target exported Activity component directly via `am start`.
+  7. `6: QUERY_PROVIDER` (Security Testing) — Query target Content Provider URI directly.
+  8. `7: FINISH` (Lifecycle) — Conclude security audit session cleanly with completion rewards.
+- **117-Dimensional Normalized Observation Tensor:** Comprehensive 17-group state representation bounded strictly in `[0.0, 1.0]` (`gymnasium.spaces.Box(low=0.0, high=1.0, shape=(117,), dtype=np.float32)`).
+- **Sub-Millisecond Pipeline (< 1ms SLA):** Highly optimized validation, building, and encoding pipeline executing end-to-end in ~0.89 ms.
+- **Multi-Tier RAG-Driven Reward Engine:** Objective reward formulation combining step efficiency, screen discovery bonuses, severity-weighted RAG evaluations, and anti-farming deduplication.
+- **Production WebSocket Gateway & Server:** Synchronous `WebSocketSystemGateway` and standalone `server.py` FastAPI routing hub bridging RL environments, physical Android devices, and desktop dashboards.
+
+---
+
+## 📦 Installation
+
+```bash
+# Install from PyPI
+pip install secenvY
+
+# Or install from source in editable mode
+git clone https://github.com/d4em0ny/android-security-agent.git
+cd android-security-agent
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+---
+
+## 🚀 Quickstart Usage
+
+```python
+import gymnasium as gym
+import secenvY
+
+# 1. Create registered environment
+env = gym.make("AndroidSecurityEnv-v0")
+
+# 2. Reset environment to clean initial state
+obs, info = env.reset(seed=42, options={"pkg": "com.vulnerable.bank"})
+print(f"Observation Shape: {obs.shape}")  # (117,)
+print(f"Action Space:      {env.action_space}") # Discrete(8)
+print(f"Action Mask:       {info['action_mask']}")
+
+# 3. Step through episode
+terminated = False
+truncated = False
+while not (terminated or truncated):
+    action_mask = info.get("action_mask")
+    # Sample or select discrete action (0..7)
+    action = env.action_space.sample(mask=action_mask)
+    obs, reward, terminated, truncated, info = env.step(action)
+    print(f"Step {info['step']} -> Reward: {reward:+.3f} (Terminated: {terminated})")
+
+env.close()
+```
+
+---
+
+## 🧪 Running Tests & Verification
+
+```bash
+# Run all unit and integration tests
+pytest tests/ -v
+
+# Run performance benchmark (< 5ms SLA verification)
+pytest tests/test_pipeline_performance.py -v -s
+
+# Run static type checker
+mypy secenvY/ server.py main.py --config-file mypy.ini
+```
+
+---
+
+## 📄 License
+
+Apache-2.0 License.

secenvy-0.1.0/pyproject.toml

@@ -0,0 +1,77 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "secenvY"
+version = "0.1.0"
+description = "Gymnasium Environment for AI-driven Android Security Analysis, Dynamic Fuzzing, and Penetration Testing"
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "Apache-2.0" }
+authors = [
+    { name = "Android Security Agent Team" },
+]
+keywords = [
+    "reinforcement-learning",
+    "gymnasium",
+    "android",
+    "security",
+    "penetration-testing",
+    "fuzzing",
+    "rag",
+    "dueling-dqn",
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Science/Research",
+    "Topic :: Security",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+]
+dependencies = [
+    "gymnasium>=0.29.0",
+    "numpy>=1.24.0",
+    "pydantic>=2.0.0",
+    "pydantic-settings>=2.0.0",
+    "torch>=2.0.0",
+    "chromadb>=0.5.0",
+    "google-genai>=0.1.0",
+    "sentence-transformers>=2.2.0",
+    "python-dotenv>=1.0.0",
+    "rich>=13.0.0",
+    "fastapi>=0.100.0",
+    "uvicorn>=0.20.0",
+    "websockets>=11.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.0.0",
+    "pytest-cov>=4.0.0",
+    "mypy>=1.0.0",
+    "build>=1.0.0",
+    "twine>=5.0.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/d4em0ny/android-security-agent"
+Repository = "https://github.com/d4em0ny/android-security-agent.git"
+Issues = "https://github.com/d4em0ny/android-security-agent/issues"
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["secenvY*"]
+
+[tool.setuptools.package-data]
+"secenvY" = ["data/**/*"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+python_files = ["test_*.py"]
+python_functions = ["test_*"]
+addopts = "-v --strict-markers"

secenvy-0.1.0/secenvY/__init__.py

@@ -0,0 +1,43 @@
+"""Android Security Analyzer Gymnasium Environment & Agent (SecEnv)."""
+
+import gymnasium as gym
+
+# Register standard AndroidSecurityEnv-v0 environment
+gym.register(
+    id="AndroidSecurityEnv-v0",
+    entry_point="secenvY.env.security_env:SecurityEnv",
+    max_episode_steps=100,
+)
+
+from secenvY.agents.dqn_agent import DuelingDQNAgent
+from secenvY.agents.networks import DuelingQNetwork
+from secenvY.agents.replay_buffer import PrioritizedReplayBuffer, UniformReplayBuffer
+from secenvY.env.security_env import SecurityEnv
+from secenvY.gateway.base import SystemGateway
+from secenvY.gateway.mock_gateway import MockSystemGateway
+from secenvY.gateway.websocket_gateway import WebSocketSystemGateway
+from secenvY.services.action_mapper import ActionMapper
+from secenvY.services.episode_policy import EpisodePolicy
+from secenvY.services.reward_calculator import RewardCalculator
+from secenvY.services.state_builder import StateBuilder
+from secenvY.services.state_encoder import StateEncoder
+from secenvY.services.state_validator import StateValidator
+from secenvY.services.tensor_converter import TensorConverter
+
+__all__ = [
+    "SecurityEnv",
+    "DuelingDQNAgent",
+    "DuelingQNetwork",
+    "PrioritizedReplayBuffer",
+    "UniformReplayBuffer",
+    "SystemGateway",
+    "MockSystemGateway",
+    "WebSocketSystemGateway",
+    "ActionMapper",
+    "StateBuilder",
+    "StateEncoder",
+    "StateValidator",
+    "TensorConverter",
+    "RewardCalculator",
+    "EpisodePolicy",
+]

secenvy-0.1.0/secenvY/agents/__init__.py

@@ -0,0 +1,17 @@
+"""Dueling Double Deep Q-Network agent, neural network architectures, and replay buffers."""
+
+from secenvY.agents.dqn_agent import DuelingDQNAgent
+from secenvY.agents.networks import DuelingQNetwork
+from secenvY.agents.replay_buffer import (
+    PrioritizedReplayBuffer,
+    SumTree,
+    UniformReplayBuffer,
+)
+
+__all__ = [
+    "DuelingDQNAgent",
+    "DuelingQNetwork",
+    "PrioritizedReplayBuffer",
+    "UniformReplayBuffer",
+    "SumTree",
+]

secenvy-0.1.0/secenvY/agents/dqn_agent.py

@@ -0,0 +1,225 @@
+"""Dueling Double Deep Q-Network (Dueling Double DQN) Agent with Prioritized Experience Replay."""
+
+from __future__ import annotations
+
+import os
+import random
+from typing import Any, Dict, List, Optional, Tuple, Union
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.optim as optim
+
+from secenvY.agents.networks import DuelingQNetwork
+from secenvY.agents.replay_buffer import PrioritizedReplayBuffer, UniformReplayBuffer
+
+
+class DuelingDQNAgent:
+    """Reinforcement Learning Agent implementing Dueling Double DQN with Prioritized Experience Replay (PER).
+
+    Args:
+        state_dim: Dimension of observation tensor (default: 117).
+        action_dim: Number of discrete actions (default: 8).
+        hidden_dim: Hidden dimension of backbone network (default: 256).
+        lr: Learning rate for Adam optimizer (default: 1e-4).
+        gamma: Discount factor for Bellman target (default: 0.99).
+        tau: Soft target network update coefficient (default: 0.005).
+        target_update_interval: Hard update interval if tau is 1.0 (default: 500).
+        buffer_capacity: Max transitions stored in experience replay (default: 100,000).
+        use_per: Whether to use Prioritized Experience Replay (default: True).
+        per_alpha: PER prioritization exponent (default: 0.6).
+        per_beta_start: Initial importance sampling exponent (default: 0.4).
+        max_grad_norm: Maximum gradient norm for clipping (default: 10.0).
+        device: Torch computing device (default: cpu / cuda if available).
+    """
+
+    def __init__(
+        self,
+        state_dim: int = 117,
+        action_dim: int = 8,
+        hidden_dim: int = 256,
+        lr: float = 1e-4,
+        gamma: float = 0.99,
+        tau: float = 0.005,
+        target_update_interval: int = 500,
+        buffer_capacity: int = 100_000,
+        use_per: bool = True,
+        per_alpha: float = 0.6,
+        per_beta_start: float = 0.4,
+        max_grad_norm: float = 10.0,
+        device: Optional[Union[str, torch.device]] = None,
+    ) -> None:
+        if device is None:
+            self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+        else:
+            self.device = torch.device(device)
+
+        self.state_dim = state_dim
+        self.action_dim = action_dim
+        self.gamma = gamma
+        self.tau = tau
+        self.target_update_interval = target_update_interval
+        self.max_grad_norm = max_grad_norm
+        self.use_per = use_per
+
+        # Neural Networks: Online and Target Q-Networks
+        self.online_net = DuelingQNetwork(state_dim, action_dim, hidden_dim).to(self.device)
+        self.target_net = DuelingQNetwork(state_dim, action_dim, hidden_dim).to(self.device)
+        self.target_net.load_state_dict(self.online_net.state_dict())
+        self.target_net.eval()
+        for param in self.target_net.parameters():
+            param.requires_grad = False
+
+        # Optimizer and Huber Loss Criterion
+        self.optimizer = optim.Adam(self.online_net.parameters(), lr=lr, eps=1e-8)
+        self.criterion = nn.SmoothL1Loss(reduction="none")
+
+        # Replay Memory (PER or Uniform)
+        self.replay_buffer: Union[PrioritizedReplayBuffer, UniformReplayBuffer]
+        if use_per:
+            self.replay_buffer = PrioritizedReplayBuffer(
+                capacity=buffer_capacity,
+                alpha=per_alpha,
+                beta_start=per_beta_start,
+                device=self.device,
+            )
+        else:
+            self.replay_buffer = UniformReplayBuffer(
+                capacity=buffer_capacity,
+                device=self.device,
+            )
+
+        self.total_steps = 0
+        self.train_steps = 0
+
+    def select_action(
+        self,
+        state: np.ndarray | List[float],
+        epsilon: float = 0.0,
+        action_mask: Optional[Union[np.ndarray, List[bool]]] = None,
+    ) -> int:
+        """Select action using epsilon-greedy exploration policy with optional action masking.
+
+        Args:
+            state: State vector of shape (117,) or list of floats.
+            epsilon: Exploration probability in [0.0, 1.0].
+            action_mask: Optional boolean mask of shape (action_dim,) where True indicates valid action.
+
+        Returns:
+            Discrete action ID in range [0, action_dim - 1].
+        """
+        self.total_steps += 1
+
+        if action_mask is not None:
+            mask_arr = np.array(action_mask, dtype=bool)
+            valid_indices = np.where(mask_arr)[0]
+            if len(valid_indices) == 0:
+                valid_indices = np.arange(self.action_dim)
+        else:
+            valid_indices = np.arange(self.action_dim)
+
+        if random.random() < epsilon:
+            return int(random.choice(valid_indices))
+
+        state_tensor = torch.tensor(
+            np.array(state, dtype=np.float32),
+            device=self.device,
+            dtype=torch.float32,
+        ).unsqueeze(0)
+
+        self.online_net.eval()
+        with torch.no_grad():
+            q_values = self.online_net(state_tensor).squeeze(0)
+            if action_mask is not None:
+                mask_tensor = torch.tensor(mask_arr, device=self.device, dtype=torch.bool)
+                q_values[~mask_tensor] = -1e9
+            action = int(q_values.argmax(dim=-1).item())
+        self.online_net.train()
+
+        return action
+
+    def train_step(self, batch_size: int = 32) -> Optional[float]:
+        """Perform a single Double DQN training step over a batch from replay buffer.
+
+        Args:
+            batch_size: Number of transitions to sample.
+
+        Returns:
+            Computed scalar loss value, or None if buffer has insufficient transitions.
+        """
+        if len(self.replay_buffer) < batch_size:
+            return None
+
+        self.train_steps += 1
+
+        # 1. Sample batch
+        if self.use_per:
+            states, actions, rewards, next_states, dones, indices, weights = self.replay_buffer.sample(batch_size)  # type: ignore
+        else:
+            states, actions, rewards, next_states, dones = self.replay_buffer.sample(batch_size)  # type: ignore
+            weights = torch.ones_like(rewards)
+            indices = []
+
+        # 2. Compute current Q-values Q(s, a; θ)
+        current_q = self.online_net(states).gather(1, actions)
+
+        # 3. Double DQN Target: a* = argmax_{a'} Q(s', a'; θ), y = r + γ Q(s', a*; θ^-)
+        with torch.no_grad():
+            next_actions = self.online_net(next_states).argmax(dim=-1, keepdim=True)
+            target_q_next = self.target_net(next_states).gather(1, next_actions)
+            target_q = rewards + (1.0 - dones) * self.gamma * target_q_next
+
+        # 4. Compute TD Errors and Huber Loss
+        td_errors = (current_q - target_q).detach()
+        loss_elements = self.criterion(current_q, target_q)
+        loss = (loss_elements * weights).mean()
+
+        # 5. Backpropagation and Gradient Clipping
+        self.optimizer.zero_grad()
+        loss.backward()
+        if self.max_grad_norm > 0:
+            nn.utils.clip_grad_norm_(self.online_net.parameters(), max_norm=self.max_grad_norm)
+        self.optimizer.step()
+
+        # 6. Update PER priorities
+        if self.use_per and indices:
+            self.replay_buffer.update_priorities(indices, td_errors.abs())  # type: ignore
+
+        # 7. Update Target Network
+        self._update_target_network()
+
+        return float(loss.item())
+
+    def _update_target_network(self) -> None:
+        """Update target network parameters via Polyak soft update or periodic hard copy."""
+        if self.tau < 1.0:
+            # Soft Polyak update: θ^- ← τ θ + (1 - τ) θ^-
+            for target_param, online_param in zip(self.target_net.parameters(), self.online_net.parameters()):
+                target_param.data.copy_(self.tau * online_param.data + (1.0 - self.tau) * target_param.data)
+        else:
+            # Hard periodic update
+            if self.train_steps % self.target_update_interval == 0:
+                self.target_net.load_state_dict(self.online_net.state_dict())
+
+    def save_checkpoint(self, filepath: str) -> None:
+        """Save model and optimizer state to disk."""
+        os.makedirs(os.path.dirname(os.path.abspath(filepath)), exist_ok=True)
+        torch.save(
+            {
+                "online_net_state": self.online_net.state_dict(),
+                "target_net_state": self.target_net.state_dict(),
+                "optimizer_state": self.optimizer.state_dict(),
+                "total_steps": self.total_steps,
+                "train_steps": self.train_steps,
+            },
+            filepath,
+        )
+
+    def load_checkpoint(self, filepath: str) -> None:
+        """Load model and optimizer state from disk."""
+        checkpoint = torch.load(filepath, map_location=self.device)
+        self.online_net.load_state_dict(checkpoint["online_net_state"])
+        self.target_net.load_state_dict(checkpoint["target_net_state"])
+        self.optimizer.load_state_dict(checkpoint["optimizer_state"])
+        self.total_steps = checkpoint.get("total_steps", 0)
+        self.train_steps = checkpoint.get("train_steps", 0)

secenvy-0.1.0/secenvY/agents/networks.py

@@ -0,0 +1,81 @@
+"""PyTorch neural network architectures for Dueling Deep Q-Networks."""
+
+from __future__ import annotations
+
+import torch
+import torch.nn as nn
+
+
+class DuelingQNetwork(nn.Module):
+    """Dueling Deep Q-Network decomposing Q(s, a) into Value V(s) and Advantage A(s, a) streams.
+
+    Q(s, a) = V(s) + (A(s, a) - 1/|A| * sum_{a'} A(s, a'))
+
+    Args:
+        state_dim: Input observation space dimension (default: 117).
+        action_dim: Discrete action space size (default: 8).
+        hidden_dim: Number of hidden units in backbone layers (default: 256).
+    """
+
+    def __init__(
+        self,
+        state_dim: int = 117,
+        action_dim: int = 8,
+        hidden_dim: int = 256,
+    ) -> None:
+        super().__init__()
+        self.state_dim = state_dim
+        self.action_dim = action_dim
+
+        # Shared Feature Extractor Backbone
+        self.backbone = nn.Sequential(
+            nn.Linear(state_dim, hidden_dim),
+            nn.LayerNorm(hidden_dim),
+            nn.ReLU(),
+            nn.Linear(hidden_dim, hidden_dim),
+            nn.ReLU(),
+        )
+
+        # Value Stream: V(s) -> [B, 1]
+        self.value_stream = nn.Sequential(
+            nn.Linear(hidden_dim, hidden_dim // 2),
+            nn.ReLU(),
+            nn.Linear(hidden_dim // 2, 1),
+        )
+
+        # Advantage Stream: A(s, a) -> [B, action_dim]
+        self.advantage_stream = nn.Sequential(
+            nn.Linear(hidden_dim, hidden_dim // 2),
+            nn.ReLU(),
+            nn.Linear(hidden_dim // 2, action_dim),
+        )
+
+    def forward(self, state: torch.Tensor) -> torch.Tensor:
+        """Forward pass computing Q-values for all discrete actions.
+
+        Args:
+            state: Tensor of shape (batch_size, state_dim) or (state_dim,).
+
+        Returns:
+            Tensor of shape (batch_size, action_dim) containing Q-values.
+        """
+        if state.dim() == 1:
+            state = state.unsqueeze(0)
+
+        features = self.backbone(state)
+        value = self.value_stream(features)             # [B, 1]
+        advantage = self.advantage_stream(features)     # [B, action_dim]
+
+        # Aggregation: Q(s, a) = V(s) + (A(s, a) - mean(A(s, :)))
+        q_values = value + (advantage - advantage.mean(dim=-1, keepdim=True))
+        return q_values
+
+    def compute_value_and_advantage(self, state: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
+        """Expose raw value and advantage streams for inspection and explainability."""
+        if state.dim() == 1:
+            state = state.unsqueeze(0)
+
+        features = self.backbone(state)
+        value = self.value_stream(features)
+        advantage = self.advantage_stream(features)
+        return value, advantage