reis-mobile 0.3.6 → 0.5.0

package/skills/mobile-android/references/media/camerax/references/camera1-to-camerax.md

@@ -0,0 +1,198 @@
+## Remove `Camera1` implementation
+
+1. Delete all `android.hardware.Camera` instances.
+2. Delete `SurfaceView` and `SurfaceHolder.Callback` implementations `surfaceCreated`, `surfaceChanged`, and `surfaceDestroyed`.
+3. Remove custom lifecycle handling that opens or releases the camera in `onResume` or `onPause`.
+4. Remove manual matrix calculations for orientation.
+
+## Initialize `ProcessCameraProvider`
+
+Request the `ProcessCameraProvider` and bind use cases to the Activity or
+Fragment lifecycle.
+
+
+```kotlin
+val context = LocalContext.current
+val lifecycleOwner = LocalLifecycleOwner.current
+LaunchedEffect(context, lifecycleOwner) {
+  val cameraProvider = ProcessCameraProvider.getInstance(context).await()
+
+  val cameraSelector = CameraSelector.Builder()
+    .requireLensFacing(CameraSelector.LENS_FACING_BACK)
+    .build()
+
+  val preview = Preview.Builder().build()
+  val imageCapture = ImageCapture.Builder()
+    .setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
+    .build()
+
+  cameraProvider.unbindAll() // Unbind before rebinding
+
+  val camera = cameraProvider.bindToLifecycle(
+    lifecycleOwner,
+    cameraSelector,
+    preview,
+    imageCapture
+  )
+  val cameraControl = camera.cameraControl
+}
+```
+
+<br />
+
+## Implement the preview and tap-to-focus
+
+Choose exactly one of the following patterns based on the app's UI toolkit:
+
+### Option A: For Android Views
+
+Use `androidx.camera.view.PreviewView`.
+
+1. **Set up preview**:
+
+
+   ```kotlin
+   preview.setSurfaceProvider(previewView.surfaceProvider)
+   ```
+
+   <br />
+
+2. **Handle tap-to-focus**:
+
+
+   ```kotlin
+   val factory = previewView.meteringPointFactory
+   val point = factory.createPoint(x, y) // x, y from touch event
+   val action = FocusMeteringAction.Builder(point, FocusMeteringAction.FLAG_AF).build()
+   cameraControl?.startFocusAndMetering(action)
+   ```
+
+   <br />
+
+### Option B: For Jetpack Compose
+
+Use `androidx.camera.compose.CameraXViewfinder`.
+
+1. **Set up preview and SurfaceRequest**:
+
+
+   ```kotlin
+   var surfaceRequest by remember { mutableStateOf<SurfaceRequest?>(null) }
+   val preview = remember {
+     Preview.Builder().build().apply {
+       setSurfaceProvider { request -> surfaceRequest = request }
+     }
+   }
+   ```
+
+   <br />
+
+2. **Render viewfinder**:
+
+
+   ```kotlin
+   surfaceRequest?.let { request ->
+     CameraXViewfinder(
+       surfaceRequest = request,
+       coordinateTransformer = coordinateTransformer,
+       modifier = Modifier
+     )
+   }
+   ```
+
+   <br />
+
+3. **Handle tap-to-focus in Compose**:
+
+
+   ```kotlin
+   // Inside your tap gesture handler...
+   val surfaceCoords = with(coordinateTransformer) { offset.transform() }
+   val factory = SurfaceOrientedMeteringPointFactory(
+     request.resolution.width.toFloat(),
+     request.resolution.height.toFloat()
+   )
+   val point = factory.createPoint(surfaceCoords.x, surfaceCoords.y)
+   val action = FocusMeteringAction.Builder(point, FocusMeteringAction.FLAG_AF).build()
+   cameraControl?.startFocusAndMetering(action)
+   ```
+
+   <br />
+
+4. **Update target rotation for Compose**:
+
+
+   ```kotlin
+   LaunchedEffect(configuration) {
+     if (!view.isInEditMode) {
+       val rotation = view.display?.rotation ?: Surface.ROTATION_0
+       imageCapture.targetRotation = rotation
+       preview.targetRotation = rotation
+     }
+   }
+   ```
+
+   <br />
+
+## Capture a photo
+
+Use the `ImageCapture` use case to take the picture. The `ImageProxy` handles
+rotation directly.
+
+
+```kotlin
+imageCapture.takePicture(
+  cameraExecutor,
+  object : ImageCapture.OnImageCapturedCallback() {
+    override fun onCaptureSuccess(image: ImageProxy) {
+      val buffer = image.planes[0].buffer
+      val bytes = ByteArray(buffer.remaining())
+      buffer.get(bytes)
+      val bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
+
+      // Adjust rotation natively via ImageProxy
+      val matrix = Matrix()
+      matrix.postRotate(image.imageInfo.rotationDegrees.toFloat())
+      if (lensFacing == CameraSelector.LENS_FACING_FRONT) {
+        matrix.postScale(-1f, 1f) // Mirror for front camera
+      }
+
+      val rotatedBitmap = Bitmap.createBitmap(
+        bitmap, 0, 0, bitmap.width, bitmap.height, matrix, true
+      )
+
+      // MUST close proxy
+      image.close()
+    }
+
+    override fun onError(exception: ImageCaptureException) {
+      Log.e("CameraX", "Capture failed: ${exception.message}", exception)
+    }
+  }
+)
+```
+
+<br />
+
+## Switch cameras
+
+To flip between front and rear cameras, change the `CameraSelector` and
+retrigger the `ProcessCameraProvider` logic.
+
+
+```kotlin
+lensFacing = if (lensFacing == CameraSelector.LENS_FACING_BACK) {
+  CameraSelector.LENS_FACING_FRONT
+} else {
+  CameraSelector.LENS_FACING_BACK
+}
+```
+
+<br />
+
+## Follow constraints
+
+- **Don't manage the camera lifecycle manually** : Bind the camera to a `LifecycleOwner` through the `ProcessCameraProvider`. Avoid manual camera open or close logic in `onResume` or `onPause`.
+- **Don't calculate focus matrices manually** : `MeteringPointFactory` handles coordinate transformations, including device rotation offsets. Avoid custom matrix implementations.
+- **Don't forget to close the `ImageProxy`** : Remember to invoke `image.close()` in the capture callback. Skipping this call locks the capture pipeline and interrupts subsequent photos.
+- **Don't wrap `PreviewView` in `AndroidView` for Compose code** : For Compose UI layouts, use `CameraXViewfinder`. Compiling `PreviewView` in an `AndroidView` is an earlier fallback option that introduces resizing issues.

package/skills/mobile-android/references/media/camerax/references/camera2-to-camerax.md

@@ -0,0 +1,193 @@
+Camera2 offers granular control but introduces boilerplate: managing
+`CameraDevice` states, `CameraCaptureSession` lifecycles, background threads,
+`HandlerThread`, and manual orientation calculations.
+
+CameraX simplifies this by binding high-level `UseCase`s such as `Preview`,
+`ImageCapture`, and `ImageAnalysis` directly to Android lifecycles, handling
+thread management and device-specific workarounds automatically.
+
+*** ** * ** ***
+
+## Remove Camera2 boilerplate
+
+Migrating to CameraX removes manual setup code:
+
+1. **Delete manual thread handling** : Remove `HandlerThread`, `Handler`, and executors dedicated to camera background tasks. CameraX manages its own threads.
+2. **Remove session and device callbacks** : Delete `CameraDevice.StateCallback` and `CameraCaptureSession.StateCallback` implementations.
+3. **Remove surface management** : Remove manual routing of `Surface` objects from `TextureView` or `SurfaceView` to the camera device.
+4. **Remove manual orientation logic** : Delete calculations involving `CameraCharacteristics.SENSOR_ORIENTATION` and display rotation for adjusting preview and capture orientation.
+
+*** ** * ** ***
+
+## Initialize `ProcessCameraProvider`
+
+Request the `ProcessCameraProvider` and bind your use cases to the
+`LifecycleOwner` activity or fragment. This replaces the
+`CameraManager.openCamera` flow.
+
+
+```kotlin
+val context = LocalContext.current
+val lifecycleOwner = LocalLifecycleOwner.current
+LaunchedEffect(context, lifecycleOwner) {
+  val cameraProvider = ProcessCameraProvider.getInstance(context).await()
+
+  val cameraSelector = CameraSelector.Builder()
+    .requireLensFacing(CameraSelector.LENS_FACING_BACK)
+    .build()
+
+  val preview = Preview.Builder().build()
+  val imageCapture = ImageCapture.Builder()
+    .setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
+    .build()
+
+  // ImageAnalysis is common when migrating from Camera2 ImageReader
+  val imageAnalysis = ImageAnalysis.Builder()
+    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
+    .build()
+
+  cameraProvider.unbindAll()
+
+  val camera = cameraProvider.bindToLifecycle(
+    lifecycleOwner,
+    cameraSelector,
+    preview,
+    imageCapture,
+    imageAnalysis
+  )
+}
+```
+
+<br />
+
+*** ** * ** ***
+
+## Implement the preview and tap-to-focus
+
+CameraX handles surface configuration automatically. Choose based on your UI
+toolkit:
+
+### Option A: For Android Views
+
+Use `androidx.camera.view.PreviewView` in your layout, and bind it to the
+`Preview` use case.
+
+
+```kotlin
+preview.setSurfaceProvider(previewView.surfaceProvider)
+```
+
+<br />
+
+### Option B: For Jetpack Compose
+
+Use `androidx.camera.compose.CameraXViewfinder`.
+
+
+```kotlin
+var surfaceRequest by remember { mutableStateOf<SurfaceRequest?>(null) }
+val preview = remember {
+  Preview.Builder().build().apply {
+    setSurfaceProvider { request -> surfaceRequest = request }
+  }
+}
+```
+
+<br />
+
+*** ** * ** ***
+
+## Capture a photo
+
+Replace `ImageReader` capture flows and
+`CaptureRequest.Builder.TEMPLATE_STILL_CAPTURE` with the `ImageCapture` use
+case. CameraX handles the rotation natively using the returned `ImageProxy`.
+
+
+```kotlin
+imageCapture.takePicture(
+  cameraExecutor,
+  object : ImageCapture.OnImageCapturedCallback() {
+    override fun onCaptureSuccess(image: ImageProxy) {
+      val buffer = image.planes[0].buffer
+      val bytes = ByteArray(buffer.remaining())
+      buffer.get(bytes)
+      val bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
+      if (bitmap != null) {
+        val matrix = Matrix()
+        matrix.postRotate(image.imageInfo.rotationDegrees.toFloat())
+        if (lensFacing == CameraSelector.LENS_FACING_FRONT) {
+          matrix.postScale(-1f, 1f)
+        }
+
+        val rotatedBitmap = Bitmap.createBitmap(
+          bitmap, 0, 0, bitmap.width, bitmap.height, matrix, true
+        )
+      }
+
+      // MUST close proxy
+      image.close()
+    }
+
+    override fun onError(exception: ImageCaptureException) {
+      Log.e("CameraX", "Capture failed: ${exception.message}", exception)
+    }
+  }
+)
+```
+
+<br />
+
+*** ** * ** ***
+
+## Implement image analysis
+
+If you were using `ImageReader` in Camera2 to access raw frames, e.g., for QR
+scanning or ML,, replace it with the CameraX `ImageAnalysis` use case.
+
+
+```kotlin
+imageAnalysis.setAnalyzer(cameraExecutor) { imageProxy ->
+  try {
+    val rotationDegrees = imageProxy.imageInfo.rotationDegrees
+    // Process image here (e.g., run object detection)
+    // ...
+  } finally {
+    // MUST close the imageProxy to avoid blocking the pipeline
+    imageProxy.close()
+  }
+}
+```
+
+<br />
+
+*** ** * ** ***
+
+## Use Camera2 interop
+
+If your app requires specific Camera2 configuration options, such as custom
+exposure modes or flash settings, that aren't exposed directly in CameraX,
+use `Camera2Interop` to apply them to your CameraX use cases.
+
+
+```kotlin
+// Use Camera2Interop to set Camera2-specific capture options
+val extender = Camera2Interop.Extender(imageCaptureBuilder)
+extender.setCaptureRequestOption(
+  CaptureRequest.CONTROL_AE_MODE,
+  CaptureRequest.CONTROL_AE_MODE_OFF
+).setCaptureRequestOption(
+  CaptureRequest.FLASH_MODE,
+  CaptureRequest.FLASH_MODE_TORCH
+)
+```
+
+<br />
+
+*** ** * ** ***
+
+## Follow constraints
+
+- **Don't open the CameraDevice manually** : Let CameraX manage device opening and closing using `bindToLifecycle`.
+- **Don't forget to close `ImageProxy`** : In both `ImageCapture` and `ImageAnalysis` callbacks, you **must** call `image.close()`. Failure to do so will block the camera pipeline.
+- **Don't block the ImageAnalysis thread** : The analyzer runs frame-by-frame. If you need to perform heavy computations, offload them to a background worker and close the `ImageProxy` as soon as possible.

package/skills/mobile-android/references/media/camerax/references/expert-blueprints.md

@@ -0,0 +1,58 @@
+Complex camera features often fail due to "Agent Stall" or timeouts when
+attempted in a single turn. Use these blueprints to break tasks into manageable
+phases.
+
+## Manual controls
+
+### Phase one: ViewModel and state
+
+1. Define a `ManualSettings` data class.
+2. Add a `MutableStateFlow<ManualSettings>` to your `CameraViewModel`.
+3. Implement the Jetpack Compose UI with sliders and switches to update this flow.
+
+### Phase two: Controller wiring
+
+1. In `CameraController.kt`, create a new function `updateManualSettings(settings: ManualSettings)`.
+2. Map these settings into the `CameraSystem` layer.
+
+### Phase three: CameraX `Camera2Interop` wiring
+
+1. In `CameraSession.kt`, use the `Camera2Interop.Extender` utility to access Camera2 capture request keys.
+2. Apply the hardware keys:
+   - `CaptureRequest.SENSOR_SENSITIVITY` to set ISO sensitivity.
+   - `CaptureRequest.SENSOR_EXPOSURE_TIME` to set exposure time.
+   - `CaptureRequest.LENS_FOCUS_DISTANCE` to set focus distance.
+3. **Critical** : If manual exposure is active, set `CaptureRequest.CONTROL_AE_MODE` to `CameraMetadata.CONTROL_AE_MODE_OFF`. ---
+
+## RAW and JPEG capture
+
+### Phase one: Output configuration
+
+1. Verify device support for RAW capture using `CameraInfo`.
+2. Configure `ImageCapture.Builder` with `OUTPUT_FORMAT_RAW_JPEG` or `OUTPUT_FORMAT_RAW`.
+
+### Phase two: Implementation
+
+1. Provide `ImageCapture.OutputFileOptions` for the target storage locations.
+2. Invoke `takePicture`. CameraX internally manages `DngCreator` to wrap RAW data with the required `CameraCharacteristics` and `CaptureResult` metadata.
+
+*** ** * ** ***
+
+## Apply image effects
+
+### Phase one: Effects selection
+
+1. Use the `androidx.media3:media3-effect` dependency.
+2. Use `RgbFilter` or `HslAdjustment` for standard color grading.
+
+### Phase two: Application
+
+1. Configure `Composition.Builder` or `MediaItem.Builder` with the list of effects.
+2. Inject the list of effects into the CameraX `Recorder` or `Preview` using the `setEffects` method.
+
+*** ** * ** ***
+
+## Low-light capture
+
+For guidance on Night Mode Extensions and Low Light Boost,
+[low-light.md](https://developer.android.com/agents/skills/camera/camerax/references/low-light).

package/skills/mobile-android/references/media/camerax/references/foldables.md

@@ -0,0 +1,83 @@
+Foldable devices introduce unique challenges for camera applications, including
+dynamic layout changes, multiple display orientations, and physical device
+postures, such as tabletop and book modes.
+
+## Manage fold states and postures
+
+| State | Posture | User interaction | Implementation goal |
+|---|---|---|---|
+| `FLAT` | Standard | Full screen preview | Conventional mobile phone layout. |
+| `HALF_OPENED` | Tabletop | Lower half for controls | Split-screen layout, viewfinder on top, controls on bottom. |
+| `HALF_OPENED` | Book | Side-by-side | Viewfinder on one panel, gallery and controls on the other. |
+
+*** ** * ** ***
+
+## Follow the implementation guide
+
+### Detect posture changes
+
+Use the Jetpack WindowManager library to observe the device's hinge state and
+fold layout.
+
+
+```kotlin
+lifecycleScope.launch {
+    lifecycle.repeatOnLifecycle(Lifecycle.State.STARTED) {
+        windowInfoTracker.windowLayoutInfo(activity)
+            .collect { layoutInfo ->
+                val displayFeature = layoutInfo.displayFeatures
+                    .filterIsInstance<FoldingFeature>()
+                    .firstOrNull()
+
+                updateCameraLayout(displayFeature)
+            }
+    }
+}
+```
+
+<br />
+
+### Handle tabletop mode
+
+In Tabletop mode, horizontal fold, you should move the viewfinder to the top
+half of the screen and the controls to the bottom half to prevent the user from
+seeing a "bent" image.
+
+- **Identify orientation:** Check `FoldingFeature.orientation`.
+- **Calculate geometry:** Use `FoldingFeature.bounds` to identify the hinge's physical location on the screen.
+- **Update UI:** Apply padding or constraints to move the `PreviewView` above the hinge.
+
+### Coordinate mapping and `Viewport`
+
+When the UI layout changes due to a fold, you **must** update the `Viewport` to
+ensure that tap-to-focus and image capture coordinates remain accurate.
+
+
+```kotlin
+val viewport = ViewPort.Builder(Rational(viewfinder.width, viewfinder.height), display.rotation)
+    .setScaleType(ViewPort.FILL_CENTER)
+    .build()
+
+val useCaseGroup = UseCaseGroup.Builder()
+    .addUseCase(preview)
+    .setViewPort(viewport)
+    .build()
+```
+
+<br />
+
+### Rear display mode
+
+Some foldables allow using the rear camera with the cover display while the
+device is unfolded.
+
+- **Verification:** If available through OEM SDKs or Android 14 (API level 34) or higher, check `DeviceState.REAR_DISPLAY_STATE`.
+- **Logic:** Handle preview detachment and reattachment on different display surfaces with varying aspect ratios.
+
+*** ** * ** ***
+
+## Foldable pitfalls
+
+- **Hinge distortion:** Don't span the camera preview across a hinge in `HALF_OPENED` state.
+- **Physical orientation:** The camera sensor's physical orientation relative to the screen often changes when you fold or unfold the device. Always rely on `CameraInfo.getSensorRotationDegrees`.
+- **Latency:** Rebinding `UseCase` objects during a fold event is expensive. Use the internal scaling of `PreviewView` before performing a full `bindToLifecycle` reconfiguration.

package/skills/mobile-android/references/media/camerax/references/immutability.md

@@ -0,0 +1,60 @@
+Many Android APIs are designed with immutability in mind to prevent race
+conditions in async environments. However, this often trips up developers used
+to mutable builder patterns.
+
+## Common immutable classes
+
+The following classes use fluent APIs that **return a new instance**. You must
+reassign the variable.
+
+| Class | Methods that return a new instance | Result if not reassigned |
+|---|---|---|
+| `PendingRecording` | `withAudioEnabled`, `asPersistentRecording` | Audio isn't recorded. |
+| `ImageCapture.Builder` | `setTargetRotation`, `setTargetResolution` | The output has the wrong orientation. |
+| `Recorder.Builder` | `setQualitySelector`, `setExecutor` | The recording uses the default quality. |
+| `Viewport.Builder` | `setScaleType`, `setLayoutDirection` | The viewfinder is stretched. |
+
+## Use standard patterns
+
+### CameraX video recording
+
+To set up video recording, use the following code:
+
+
+```kotlin
+// WRONG
+run {
+  val pending = recorder.prepareRecording(context, opts)
+  pending.withAudioEnabled() // This returns a new instance which is ignored
+  val active = pending.start(exec, listener)
+}
+
+// CORRECT
+run {
+  val pending = recorder.prepareRecording(context, opts)
+      .withAudioEnabled() // Chaining works
+  val active = pending.start(exec, listener)
+}
+
+// ALSO CORRECT
+run {
+  var pending = recorder.prepareRecording(context, opts)
+  pending = pending.withAudioEnabled() // Reassignment
+  val active = pending.start(exec, listener)
+}
+```
+
+<br />
+
+### Viewport construction
+
+To set up the viewport, use the following code:
+
+
+```kotlin
+val viewport = ViewPort.Builder(Rational(width, height), displayRotation)
+    .setScaleType(ViewPort.FILL_CENTER)
+    .build()
+```
+
+<br />