@geastack/targets 0.1.86 → 0.1.88

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (86) hide show
  1. package/package.json +7 -7
  2. package/targets/esp32/app_project.cmake +18 -1
  3. package/targets/esp32/chip_bindings/audio/echo_cancellation.cpp +213 -0
  4. package/targets/esp32/chip_bindings/audio/echo_cancellation.h +12 -0
  5. package/targets/esp32/chip_bindings/audio/es8311.cpp +280 -41
  6. package/targets/esp32/chip_bindings/displays/rgb_panel.cpp +115 -12
  7. package/targets/esp32/chip_bindings/expanders/io_expander.h +2 -0
  8. package/targets/esp32/chip_bindings/expanders/tca9554.cpp +74 -0
  9. package/targets/esp32/chip_bindings/touch/ft3168.cpp +23 -1
  10. package/targets/esp32/chip_bindings/touch/ft3168_controller.h +1 -0
  11. package/targets/esp32/chip_bindings/touch/gt911.cpp +4 -3
  12. package/targets/esp32/components/gea_framework/CMakeLists.txt +8 -1
  13. package/targets/esp32/components/gea_juice/CMakeLists.txt +46 -0
  14. package/targets/esp32/components/gea_juice/README.md +20 -0
  15. package/targets/esp32/components/gea_juice/juice_memory.c +19 -0
  16. package/targets/esp32/components/gea_juice/juice_memory.h +16 -0
  17. package/targets/esp32/components/gea_juice/patch_threads.py +50 -0
  18. package/targets/esp32/components/gea_vp8_peer/CMakeLists.txt +16 -0
  19. package/targets/esp32/components/gea_vp8_peer/README.md +39 -0
  20. package/targets/esp32/components/gea_vp8_peer/gea_vp8_peer.h +21 -0
  21. package/targets/esp32/components/gea_vp8_peer/vp8_peer.cpp +538 -0
  22. package/targets/esp32/components/gea_vpx/CMakeLists.txt +235 -0
  23. package/targets/esp32/components/gea_vpx/README.md +133 -0
  24. package/targets/esp32/components/gea_vpx/i420_rgb565_s3.S +169 -0
  25. package/targets/esp32/components/gea_vpx/vp8_blocks_s3.c +42 -0
  26. package/targets/esp32/components/gea_vpx/vp8_copy_s3.S +39 -0
  27. package/targets/esp32/components/gea_vpx/vp8_entropy_s3.c +76 -0
  28. package/targets/esp32/components/gea_vpx/vp8_filter_s3.S +58 -0
  29. package/targets/esp32/components/gea_vpx/vp8_idct_s3.S +269 -0
  30. package/targets/esp32/components/gea_vpx/vp8_intrapred_s3.S +65 -0
  31. package/targets/esp32/components/gea_vpx/vp8_loopfilter_s3.S +333 -0
  32. package/targets/esp32/components/gea_vpx/vp8_loopfilter_s3.c +62 -0
  33. package/targets/esp32/components/gea_vpx/vp8_profile_s3.c +22 -0
  34. package/targets/esp32/components/gea_vpx/vp8_profile_s3.h +20 -0
  35. package/targets/esp32/components/gea_vpx/vp8_s3.c +120 -0
  36. package/targets/esp32/components/gea_vpx/vp8_s3.h +58 -0
  37. package/targets/esp32/components/gea_vpx/vp8_workspace_s3.c +14 -0
  38. package/targets/esp32/components/gea_vpx/vpx_memory.c +41 -0
  39. package/targets/esp32/connectivity/diagnostics_transport.cpp +18 -0
  40. package/targets/esp32/connectivity/wifi.cpp +26 -13
  41. package/targets/esp32/crypto/srtp_hmac_psa.c +118 -0
  42. package/targets/esp32/display.cpp +13 -0
  43. package/targets/esp32/gea_framework.cmake +7 -0
  44. package/targets/esp32/host_sources.cmake +2 -0
  45. package/targets/esp32/i2c_completion_order.cmake +57 -0
  46. package/targets/esp32/patch_streaming_dependencies.py +233 -0
  47. package/targets/esp32/services/device_control.cpp +141 -5
  48. package/targets/esp32/services/frame_scheduler.cpp +29 -1
  49. package/targets/esp32/services/opus_benchmark.h +82 -0
  50. package/targets/esp32/services/pcm_simd_check.h +73 -0
  51. package/targets/esp32/services/srtp_benchmark.h +106 -0
  52. package/targets/esp32/services/video_simd_benchmark.h +477 -0
  53. package/targets/esp32/services/vp8_benchmark.h +147 -0
  54. package/targets/esp32/streaming_dependency_patches.cmake +103 -0
  55. package/targets/esp32-m5stack-m5paper/main/idf_component.yml +2 -0
  56. package/targets/esp32-p4-m5stack-tab5/main/idf_component.yml +2 -0
  57. package/targets/esp32-p4-waveshare-touch-lcd-3.5/main/idf_component.yml +2 -0
  58. package/targets/esp32-p4-waveshare-touch-lcd-7/main/idf_component.yml +2 -0
  59. package/targets/esp32-s3-elecrow-rotary-2.1/main/idf_component.yml +2 -0
  60. package/targets/esp32-s3-epaper-1.54/main/idf_component.yml +2 -0
  61. package/targets/esp32-s3-lilygo-t-display-s3-long/main/idf_component.yml +2 -0
  62. package/targets/esp32-s3-m5stack-papers3/main/idf_component.yml +2 -0
  63. package/targets/esp32-s3-m5stack-sticks3/main/idf_component.yml +2 -0
  64. package/targets/esp32-s3-touch-amoled-1.75/main/idf_component.yml +2 -0
  65. package/targets/esp32-s3-touch-amoled-1.8/dependencies.lock +110 -2
  66. package/targets/esp32-s3-touch-amoled-1.8/main/CMakeLists.txt +29 -3
  67. package/targets/esp32-s3-touch-amoled-1.8/main/app_main.cpp +6 -3
  68. package/targets/esp32-s3-touch-amoled-1.8/main/idf_component.yml +11 -0
  69. package/targets/esp32-s3-touch-amoled-1.8/sdkconfig.defaults +9 -1
  70. package/targets/esp32-s3-touch-amoled-2.06/CMakeLists.txt +2 -0
  71. package/targets/esp32-s3-touch-amoled-2.06/dependencies.lock +171 -8
  72. package/targets/esp32-s3-touch-amoled-2.06/main/CMakeLists.txt +9 -2
  73. package/targets/esp32-s3-touch-amoled-2.06/main/app_main.cpp +10 -3
  74. package/targets/esp32-s3-touch-amoled-2.06/main/idf_component.yml +16 -1
  75. package/targets/esp32-s3-touch-amoled-2.06/main/include/board.h +3 -0
  76. package/targets/esp32-s3-touch-amoled-2.06/main/touch.cpp +7 -0
  77. package/targets/esp32-s3-touch-amoled-2.41/dependencies.lock +57 -1
  78. package/targets/esp32-s3-touch-amoled-2.41/main/idf_component.yml +10 -0
  79. package/targets/esp32-s3-touch-lcd-3.5b/dependencies.lock +12 -1
  80. package/targets/esp32-s3-touch-lcd-3.5b/main/CMakeLists.txt +5 -1
  81. package/targets/esp32-s3-touch-lcd-3.5b/main/idf_component.yml +2 -1
  82. package/targets/esp32-s3-touch-lcd-3.5b/main/include/board.h +3 -0
  83. package/targets/esp32-s3-touch-lcd-3.5b/main/platform_stubs.cpp +1 -18
  84. package/targets/esp32-s3-touch-lcd-3.5b/main/speaker_power.cpp +27 -0
  85. package/targets/esp32-s3-waveshare-touch-lcd-4b.json +504 -0
  86. package/targets.json +12 -1
@@ -39,6 +39,7 @@
39
39
  #include "esp_timer.h"
40
40
  #include "freertos/FreeRTOS.h"
41
41
  #include "freertos/task.h"
42
+ #include "freertos/idf_additions.h"
42
43
  #include "soc/clk_tree_defs.h"
43
44
 
44
45
  #include <algorithm>
@@ -60,6 +61,12 @@ constexpr int kWidth = gea::platform::display::kWidth;
60
61
  constexpr int kHeight = gea::platform::display::kHeight;
61
62
  constexpr std::size_t kPixelCount = static_cast<std::size_t>(kWidth) * static_cast<std::size_t>(kHeight);
62
63
  constexpr std::size_t kFramebufferBytes = kPixelCount * sizeof(std::uint16_t);
64
+ #ifdef GEA_EMBEDDED_DISPLAY_RGB_PCLK_HZ
65
+ constexpr int kPixelClockHz = GEA_EMBEDDED_DISPLAY_RGB_PCLK_HZ;
66
+ static_assert(kPixelClockHz > 0, "RGB pixel clock must be positive");
67
+ #else
68
+ constexpr int kPixelClockHz = gea::platform::board::display.pclkHz;
69
+ #endif
63
70
  // Bounce rows: the largest candidate that divides the panel height, so the
64
71
  // framebuffer splits into whole bounce buffers whatever the geometry is.
65
72
  constexpr int pickBounceRows(int height)
@@ -113,6 +120,9 @@ esp_lcd_panel_handle_t g_panel = nullptr;
113
120
  std::uint16_t *g_framebuffer = nullptr;
114
121
  std::uint16_t *g_drawFramebuffer = nullptr;
115
122
  std::uint16_t *g_stripBuffer = nullptr;
123
+ std::atomic<std::size_t> g_internalReservation{0};
124
+ std::size_t g_appliedInternalReservation = 0;
125
+ std::size_t g_internalCacheHeadroom = 0;
116
126
  // The strip path publishes straight to scanout and leaves g_drawFramebuffer
117
127
  // untouched. If a later frame falls back to the per-command path (partial
118
128
  // present with no full-screen base), re-sync the back buffer once first.
@@ -170,7 +180,8 @@ constexpr gpio_num_t kBacklightPin = gea::platform::board::display.backlight;
170
180
  esp_err_t updateBacklightDuty()
171
181
  {
172
182
  if (!g_backlightPwmInitialized) return ESP_OK;
173
- const std::uint32_t duty = (kBacklightLedcMaxDuty * static_cast<std::uint32_t>(g_brightness)) / 100u;
183
+ const int brightness = gea::platform::board::display.backlightActiveLow ? 100 - g_brightness : g_brightness;
184
+ const std::uint32_t duty = (kBacklightLedcMaxDuty * static_cast<std::uint32_t>(brightness)) / 100u;
174
185
  esp_err_t err = ledc_set_duty(kBacklightLedcMode, kBacklightLedcChannel, duty);
175
186
  if (err != ESP_OK) {
176
187
  ESP_LOGE(kTag, "failed to set backlight duty: %s", esp_err_to_name(err));
@@ -230,7 +241,7 @@ esp_err_t initBacklight()
230
241
  channelConfig.speed_mode = kBacklightLedcMode;
231
242
  channelConfig.channel = kBacklightLedcChannel;
232
243
  channelConfig.timer_sel = kBacklightLedcTimer;
233
- channelConfig.duty = 0;
244
+ channelConfig.duty = gea::platform::board::display.backlightActiveLow ? kBacklightLedcMaxDuty : 0;
234
245
  channelConfig.hpoint = 0;
235
246
  err = ledc_channel_config(&channelConfig);
236
247
  if (err != ESP_OK) {
@@ -259,12 +270,49 @@ esp_err_t resetPanel()
259
270
  return ESP_OK;
260
271
  }
261
272
 
273
+ // Optional controller commands over 9-bit, mode-0 SPI on the expander.
274
+ // The pixel stream still uses the shared RGB DMA/rendering path.
275
+ esp_err_t initRgbController()
276
+ {
277
+ #if GEA_BOARD_HAS_RGB_CONTROLLER
278
+ auto &io = gea::platform::esp32::chip_bindings::expanders::ioExpander();
279
+ const auto &control = gea::platform::board::rgbController;
280
+ if (!io.init()) return ESP_FAIL;
281
+ if (control.addressSelect >= 0) {
282
+ if (!io.writePin(control.addressSelect, false) || !io.setInput(control.addressSelect, false)) return ESP_FAIL;
283
+ vTaskDelay(pdMS_TO_TICKS(200));
284
+ }
285
+ if (control.reset >= 0) {
286
+ if (!io.writePin(control.reset, false) || !io.setInput(control.reset, false)) return ESP_FAIL;
287
+ vTaskDelay(pdMS_TO_TICKS(200));
288
+ if (!io.writePin(control.reset, true)) return ESP_FAIL;
289
+ vTaskDelay(pdMS_TO_TICKS(200));
290
+ }
291
+ if (control.addressSelect >= 0 && !io.setInput(control.addressSelect, true)) return ESP_FAIL;
292
+ if (!io.writePin(control.cs, true) || !io.writePin(control.clock, false)) return ESP_FAIL;
293
+ for (const auto &command : gea::platform::board::rgbControllerCommands) {
294
+ if (!io.writePin(control.cs, false)) return ESP_FAIL;
295
+ for (int index = -1; index < command.length; ++index) {
296
+ const unsigned word = index < 0 ? command.command : 0x100u | command.data[index];
297
+ for (int bit = 8; bit >= 0; --bit) {
298
+ if (!io.writePin(control.clock, false) || !io.writePin(control.data, (word & (1u << bit)) != 0) ||
299
+ !io.writePin(control.clock, true)) return ESP_FAIL;
300
+ }
301
+ }
302
+ if (!io.writePin(control.clock, false) || !io.writePin(control.cs, true)) return ESP_FAIL;
303
+ if (command.delayMs) vTaskDelay(pdMS_TO_TICKS(command.delayMs));
304
+ }
305
+ ESP_LOGI(kTag, "RGB controller initialized through I/O expander");
306
+ #endif
307
+ return ESP_OK;
308
+ }
309
+
262
310
  esp_err_t initRgbPanel()
263
311
  {
264
312
  const auto &display = gea::platform::board::display;
265
313
  esp_lcd_rgb_panel_config_t config = {};
266
314
  config.clk_src = LCD_CLK_SRC_PLL160M;
267
- config.timings.pclk_hz = display.pclkHz;
315
+ config.timings.pclk_hz = kPixelClockHz;
268
316
  config.timings.h_res = kWidth;
269
317
  config.timings.v_res = kHeight;
270
318
  config.timings.hsync_pulse_width = display.hsyncPulseWidth;
@@ -347,14 +395,21 @@ esp_err_t initRgbPanel()
347
395
  ESP_LOGI(kTag, "strip-raster %s strip_buf=%p rows=%d",
348
396
  g_stripBuffer ? "ON" : "OFF (no internal SRAM)", g_stripBuffer, kStripRows);
349
397
  #endif
350
- ESP_LOGI(kTag, "RGB panel ready %dx%d pclk=%d bounce_rows=%d fb=%p draw_fb=%p single_fb=%d",
398
+ ESP_LOGI(kTag, "RGB panel ready %dx%d pclk=%d bounce_rows=%d fb=%p draw_fb=%p single_fb=%d row_bytes=%u fb_alignment64=%u vsync_restart=%d",
351
399
  kWidth,
352
400
  kHeight,
353
- display.pclkHz,
354
- kRgbBounceRows,
401
+ kPixelClockHz,
402
+ int(config.bounce_buffer_size_px / kWidth),
355
403
  g_framebuffer,
356
404
  g_drawFramebuffer,
357
- g_framebuffer == g_drawFramebuffer ? 1 : 0);
405
+ g_framebuffer == g_drawFramebuffer ? 1 : 0,
406
+ static_cast<unsigned>(kWidth * sizeof(std::uint16_t)),
407
+ static_cast<unsigned>(reinterpret_cast<std::uintptr_t>(g_framebuffer) % 64),
408
+ #if CONFIG_LCD_RGB_RESTART_IN_VSYNC
409
+ 1);
410
+ #else
411
+ 0);
412
+ #endif
358
413
  return ESP_OK;
359
414
  }
360
415
 
@@ -527,6 +582,10 @@ RenderRowWorker g_renderRowWorker;
527
582
  // command handlers), which is much deeper than a strip copy.
528
583
  constexpr std::uint32_t kRenderRowWorkerStackBytes = 12288;
529
584
 
585
+ #ifndef GEA_EMBEDDED_RENDER_WORKER_STACK_EXTERNAL
586
+ #define GEA_EMBEDDED_RENDER_WORKER_STACK_EXTERNAL 0
587
+ #endif
588
+
530
589
  void renderRowWorkerTask(void *)
531
590
  {
532
591
  // MUST start at 0 (jobSeq's initial value), NOT jobSeq.load(): submit() creates
@@ -630,6 +689,20 @@ bool submitRenderWorkerJob(void (*fn)(void *, int, int), void *ctx, int y0, int
630
689
  const BaseType_t renderCore = xPortGetCoreID();
631
690
  const BaseType_t workerCore = renderCore == 0 ? 1 : 0;
632
691
  const UBaseType_t priority = uxTaskPriorityGet(nullptr);
692
+ #if GEA_EMBEDDED_RENDER_WORKER_STACK_EXTERNAL
693
+ // This persistent task only replays render commands and copies pixels;
694
+ // neither its stack nor its local variables are DMA buffers. Match the
695
+ // SPI display worker's external-stack option and retain internal RAM for
696
+ // audio DMA, lwIP and FreeRTOS control structures.
697
+ const BaseType_t created = xTaskCreatePinnedToCoreWithCaps(renderRowWorkerTask,
698
+ "gea_rowwrk",
699
+ kRenderRowWorkerStackBytes,
700
+ nullptr,
701
+ priority,
702
+ &g_renderRowWorker.task,
703
+ workerCore,
704
+ MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
705
+ #else
633
706
  const BaseType_t created = xTaskCreatePinnedToCore(renderRowWorkerTask,
634
707
  "gea_rowwrk",
635
708
  kRenderRowWorkerStackBytes,
@@ -637,6 +710,7 @@ bool submitRenderWorkerJob(void (*fn)(void *, int, int), void *ctx, int y0, int
637
710
  priority,
638
711
  &g_renderRowWorker.task,
639
712
  workerCore);
713
+ #endif
640
714
  if (created != pdPASS || !g_renderRowWorker.task) {
641
715
  ESP_LOGW(kTag, "row worker create failed: create=%d stack=%u",
642
716
  static_cast<int>(created),
@@ -976,6 +1050,8 @@ bool Display::init()
976
1050
  ESP_LOGE(kTag, "panel reset failed: %s", esp_err_to_name(err));
977
1051
  return false;
978
1052
  }
1053
+ err = initRgbController();
1054
+ if (err != ESP_OK) return false;
979
1055
  err = initRgbPanel();
980
1056
  if (err != ESP_OK) return false;
981
1057
  g_initialized = true;
@@ -1176,13 +1252,40 @@ void Display::setFlushConfig(int chunkRows, int queueDepth)
1176
1252
  g_flushDepth = queueDepth;
1177
1253
  }
1178
1254
 
1179
- void Display::reserveInternal(std::size_t) {}
1255
+ void Display::reserveInternal(std::size_t bytes)
1256
+ {
1257
+ g_internalReservation.store(bytes, std::memory_order_release);
1258
+ }
1180
1259
  bool Display::setHighBrightnessMode(bool) { return false; }
1181
1260
  bool Display::highBrightnessMode() { return false; }
1182
- // The RGB-panel backend draws straight into the framebuffer with no separate
1183
- // internal-RAM flush staging to reserve, so the deferred-reserve apply is a no-op
1184
- // here (matches reserveInternal above). Required since frame_scheduler now calls it.
1185
- void Display::applyPendingInternalReserve() {}
1261
+ // The optional strip-raster cache is internal staging too. Yield it on the
1262
+ // frame task before radio/audio bring-up; wait for its row worker before freeing.
1263
+ // RGB DMA's bounce buffers remain owned by the LCD driver throughout.
1264
+ void Display::applyPendingInternalReserve()
1265
+ {
1266
+ const std::size_t requested = g_internalReservation.load(std::memory_order_acquire);
1267
+ if (requested == g_appliedInternalReservation) return;
1268
+ g_appliedInternalReservation = requested;
1269
+ if (requested) {
1270
+ g_internalCacheHeadroom = std::max(g_internalCacheHeadroom, requested);
1271
+ if (g_stripBuffer) {
1272
+ gea_render_parallel_wait();
1273
+ heap_caps_free(g_stripBuffer);
1274
+ g_stripBuffer = nullptr;
1275
+ ESP_LOGI(kTag, "Released %u bytes of strip staging for internal-memory reservation",
1276
+ static_cast<unsigned>(kStripBufferPixels * sizeof(std::uint16_t)));
1277
+ }
1278
+ } else {
1279
+ #if GEA_EMBEDDED_DISPLAY_RGB_STRIP_RASTER
1280
+ // Restoring an optimization must not consume the headroom just requested
1281
+ // by the radios. The PSRAM back-buffer path remains fully functional.
1282
+ const std::size_t bytes = kStripBufferPixels * sizeof(std::uint16_t);
1283
+ if (!g_stripBuffer && heap_caps_get_free_size(MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT) >= bytes + g_internalCacheHeadroom) {
1284
+ g_stripBuffer = static_cast<std::uint16_t *>(heap_caps_aligned_alloc(16, bytes, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT));
1285
+ }
1286
+ #endif
1287
+ }
1288
+ }
1186
1289
  int Display::flushChunkRows() { return g_flushRows; }
1187
1290
  int Display::flushQueueDepth() { return g_flushDepth; }
1188
1291
  int Display::flushBufferBytes() { return 0; }
@@ -21,6 +21,8 @@ namespace gea::platform::esp32::chip_bindings::expanders
21
21
  virtual bool init() = 0;
22
22
  virtual bool writePin(int pin, bool high) = 0;
23
23
  virtual bool readPin(int pin, bool &high) = 0;
24
+ // Not all expanders support per-pin direction.
25
+ virtual bool setInput(int pin, bool input) { return false; }
24
26
  };
25
27
 
26
28
  IoExpander &ioExpander();
@@ -0,0 +1,74 @@
1
+ #include "chip_bindings/expanders/io_expander.h"
2
+ #include "expanders/tca9554/tca9554.h"
3
+ #include "board.h"
4
+ #include "i2c.h"
5
+ #include "driver/i2c_master.h"
6
+ #include "freertos/FreeRTOS.h"
7
+ #include "freertos/semphr.h"
8
+
9
+ namespace gea::platform::esp32::chip_bindings::expanders {
10
+ namespace {
11
+ class Tca9554 final : public IoExpander, private gea::chips::tca9554::Bus {
12
+ public:
13
+ bool init() override {
14
+ Guard guard(lock_);
15
+ if (ready_) return true;
16
+ auto bus = gea::platform::i2c::Bus::primary();
17
+ if (!bus.available()) return false;
18
+ if (!device_) {
19
+ i2c_device_config_t config{};
20
+ config.dev_addr_length = I2C_ADDR_BIT_LEN_7;
21
+ config.device_address = gea::platform::board::expander.address;
22
+ config.scl_speed_hz = 400000;
23
+ if (i2c_master_bus_add_device(static_cast<i2c_master_bus_handle_t>(bus.nativeHandle()), &config, &device_) != ESP_OK) return false;
24
+ }
25
+ ready_ = driver_.configure(*this, gea::platform::board::expander.initialOutputs,
26
+ gea::platform::board::expander.initialDirections);
27
+ return ready_;
28
+ }
29
+
30
+ bool writePin(int pin, bool high) override {
31
+ if (!init()) return false;
32
+ Guard guard(lock_);
33
+ return driver_.setPin(*this, pin, high);
34
+ }
35
+
36
+ bool readPin(int pin, bool &high) override {
37
+ if (!init()) return false;
38
+ Guard guard(lock_);
39
+ return driver_.readPin(*this, pin, high);
40
+ }
41
+
42
+ bool setInput(int pin, bool input) override {
43
+ if (!init()) return false;
44
+ Guard guard(lock_);
45
+ return driver_.setInput(*this, pin, input);
46
+ }
47
+
48
+ private:
49
+ struct Guard {
50
+ explicit Guard(SemaphoreHandle_t mutex) : mutex_(mutex) { xSemaphoreTake(mutex_, portMAX_DELAY); }
51
+ ~Guard() { xSemaphoreGive(mutex_); }
52
+ SemaphoreHandle_t mutex_;
53
+ };
54
+
55
+ bool write(std::uint8_t reg, std::uint8_t value) override {
56
+ const std::uint8_t bytes[] = {reg, value};
57
+ return i2c_master_transmit(device_, bytes, 2, 100) == ESP_OK;
58
+ }
59
+
60
+ bool read(std::uint8_t reg, std::uint8_t &value) override {
61
+ return i2c_master_transmit_receive(device_, &reg, 1, &value, 1, 100) == ESP_OK;
62
+ }
63
+
64
+ StaticSemaphore_t mutexStorage_{};
65
+ SemaphoreHandle_t lock_ = xSemaphoreCreateMutexStatic(&mutexStorage_);
66
+ i2c_master_dev_handle_t device_ = nullptr;
67
+ gea::chips::tca9554::Driver driver_;
68
+ bool ready_ = false;
69
+ };
70
+ } // namespace
71
+
72
+ IoExpander &ioExpander() { static Tca9554 driver; return driver; }
73
+ const char *ioExpanderDriverName() { return "tca9554"; }
74
+ } // namespace gea::platform::esp32::chip_bindings::expanders
@@ -6,6 +6,7 @@
6
6
  #include "esp_attr.h"
7
7
  #include "esp_log.h"
8
8
  #include "esp_timer.h"
9
+ #include <cstdio>
9
10
  #include "freertos/FreeRTOS.h"
10
11
  #include "freertos/task.h"
11
12
  #if GEA_EMBEDDED_TOUCH_TASK_STACK_EXTERNAL
@@ -41,8 +42,15 @@ constexpr int kPollIntervalMs = 10;
41
42
  // iteration is one I2C transaction that blocks on the bus, so a higher priority
42
43
  // costs the core microseconds, not bandwidth.
43
44
  #ifndef GEA_EMBEDDED_TOUCH_TASK_PRIORITY
45
+ // RTC encoding (5) and receive/decode (7) share CPU1. A physical contact
46
+ // must be sampled before the finger lifts, even when those workers are busy.
47
+ // Only the brief I2C sample preempts them; DMA capture/playback stays at 24.
48
+ #if defined(GEA_EMBEDDED_APP_USES_AUDIO) && GEA_EMBEDDED_APP_USES_AUDIO
49
+ #define GEA_EMBEDDED_TOUCH_TASK_PRIORITY 8
50
+ #else
44
51
  #define GEA_EMBEDDED_TOUCH_TASK_PRIORITY 4
45
52
  #endif
53
+ #endif
46
54
  constexpr int kTaskPriority = GEA_EMBEDDED_TOUCH_TASK_PRIORITY;
47
55
  constexpr TickType_t kI2cTimeoutTicks = pdMS_TO_TICKS(100);
48
56
 
@@ -50,6 +58,9 @@ constexpr TickType_t kI2cTimeoutTicks = pdMS_TO_TICKS(100);
50
58
  // knows about the contact; everything after it is ours. Recorded in the ISR and
51
59
  // read once per reported transition, so the steady state costs one timer read.
52
60
  volatile std::int64_t g_touchIntUs = 0;
61
+ std::atomic<std::uint32_t> g_touchInterrupts{0};
62
+ std::atomic<std::uint32_t> g_touchReads{0};
63
+ std::atomic<std::uint32_t> g_touchReadErrors{0};
53
64
 
54
65
  } // namespace
55
66
 
@@ -71,7 +82,10 @@ bool EspI2cRegisterBus::writeRegister(std::uint8_t reg, std::uint8_t value) {
71
82
  }
72
83
 
73
84
  bool EspI2cRegisterBus::readRegisters(std::uint8_t reg, std::uint8_t *data, std::size_t length) {
74
- return device_ && i2c_master_transmit_receive(device_, &reg, 1, data, length, kI2cTimeoutTicks) == ESP_OK;
85
+ const bool ok = device_ && i2c_master_transmit_receive(device_, &reg, 1, data, length, kI2cTimeoutTicks) == ESP_OK;
86
+ g_touchReads.fetch_add(1, std::memory_order_relaxed);
87
+ if (!ok) g_touchReadErrors.fetch_add(1, std::memory_order_relaxed);
88
+ return ok;
75
89
  }
76
90
 
77
91
  TouchController &TouchController::instance() {
@@ -221,11 +235,19 @@ void IRAM_ATTR TouchController::interruptEntry(void *arg) {
221
235
 
222
236
  void IRAM_ATTR TouchController::notifyTaskFromInterrupt() {
223
237
  g_touchIntUs = esp_timer_get_time();
238
+ g_touchInterrupts.fetch_add(1, std::memory_order_relaxed);
224
239
  BaseType_t woken = pdFALSE;
225
240
  if (task_) vTaskNotifyGiveFromISR(task_, &woken);
226
241
  portYIELD_FROM_ISR(woken);
227
242
  }
228
243
 
244
+ void TouchController::printDiagnostics() const {
245
+ std::printf("GEADEV:TOUCHINFO initialized=%d irq=%u reads=%u errors=%u pin=%d task_state=%d priority=%u\n",
246
+ initialized_ ? 1 : 0, unsigned(g_touchInterrupts.load()), unsigned(g_touchReads.load()),
247
+ unsigned(g_touchReadErrors.load()), kInterruptPin >= 0 ? gpio_get_level(kInterruptPin) : -1,
248
+ task_ ? int(eTaskGetState(task_)) : -1, task_ ? unsigned(uxTaskPriorityGet(task_)) : 0);
249
+ }
250
+
229
251
  void TouchController::taskEntry(void *arg) {
230
252
  static_cast<TouchController *>(arg)->pollLoop();
231
253
  }
@@ -57,6 +57,7 @@ public:
57
57
  void setPointerObserver(PointerObserver observer);
58
58
  esp_err_t begin();
59
59
  bool read(TouchSample &sample);
60
+ void printDiagnostics() const;
60
61
  TouchSample cached() const;
61
62
  void consumeLatestMove(int *x, int *y);
62
63
  // Inject a synthetic touch event through the same notify path as the hardware
@@ -37,7 +37,8 @@ constexpr int kPollIntervalMs = 10;
37
37
  #define GEA_EMBEDDED_TOUCH_TASK_PRIORITY 4
38
38
  #endif
39
39
  constexpr int kTaskPriority = GEA_EMBEDDED_TOUCH_TASK_PRIORITY;
40
- constexpr TickType_t kI2cTimeoutTicks = pdMS_TO_TICKS(100);
40
+ // ESP-IDF's I2C master API takes milliseconds, not FreeRTOS ticks.
41
+ constexpr int kI2cTimeoutMs = 100;
41
42
 
42
43
  volatile std::int64_t g_touchIntUs = 0;
43
44
 
@@ -66,13 +67,13 @@ bool EspI2cRegisterBus::writeRegisters(std::uint16_t reg, const std::uint8_t *da
66
67
  frame[0] = static_cast<std::uint8_t>(reg >> 8);
67
68
  frame[1] = static_cast<std::uint8_t>(reg & 0xff);
68
69
  for (std::size_t i = 0; i < length; i++) frame[2 + i] = data[i];
69
- return i2c_master_transmit(device_, frame, 2 + length, kI2cTimeoutTicks) == ESP_OK;
70
+ return i2c_master_transmit(device_, frame, 2 + length, kI2cTimeoutMs) == ESP_OK;
70
71
  }
71
72
 
72
73
  bool EspI2cRegisterBus::readRegisters(std::uint16_t reg, std::uint8_t *data, std::size_t length) {
73
74
  if (!device_) return false;
74
75
  const std::uint8_t address[2] = {static_cast<std::uint8_t>(reg >> 8), static_cast<std::uint8_t>(reg & 0xff)};
75
- return i2c_master_transmit_receive(device_, address, sizeof(address), data, length, kI2cTimeoutTicks) == ESP_OK;
76
+ return i2c_master_transmit_receive(device_, address, sizeof(address), data, length, kI2cTimeoutMs) == ESP_OK;
76
77
  }
77
78
 
78
79
  TouchController &TouchController::instance() {
@@ -12,7 +12,7 @@ include("${GEA_EMBEDDED_ROOT}/targets/esp32/gea_framework.cmake")
12
12
  # capability comes from the gea CLI (env, then the -D cache entry); a direct
13
13
  # `idf.py` build leaves it unset and keeps the components, since the board is
14
14
  # then the only thing that knows what the app needs.
15
- set(GEA_EMBEDDED_FRAMEWORK_NETWORK_REQUIRES app_update esp_http_server)
15
+ set(GEA_EMBEDDED_FRAMEWORK_NETWORK_REQUIRES app_update esp_http_server gea_vpx gea_vp8_peer esp_libsrtp)
16
16
  if(DEFINED ENV{GEA_EMBEDDED_CAPABILITY_NETWORK}
17
17
  AND "$ENV{GEA_EMBEDDED_CAPABILITY_NETWORK}" STREQUAL "0")
18
18
  set(GEA_EMBEDDED_FRAMEWORK_NETWORK_REQUIRES "")
@@ -43,6 +43,7 @@ idf_component_register(
43
43
  # every board that builds this file.
44
44
  esp_driver_gpio
45
45
  esp_driver_rmt
46
+ esp_driver_uart
46
47
  # gea.targets.esp32.componentRequires: the components an app's own native
47
48
  # sources need headers or PUBLIC definitions from. REQUIRES is transitive,
48
49
  # so listing them here reaches the app sources compiled into `main` without
@@ -58,5 +59,11 @@ idf_component_register(
58
59
  # flag travels through the link INTERFACE because this component is a static
59
60
  # archive, so a PRIVATE option would stop at the archive.
60
61
  target_link_libraries(${COMPONENT_LIB} INTERFACE "-Wl$<COMMA>-u$<COMMA>gea_gpio_link_bindings")
62
+ if(GEA_EMBEDDED_APP)
63
+ target_link_libraries(${COMPONENT_LIB} INTERFACE "-Wl$<COMMA>-u$<COMMA>gea_mjpeg_link_bindings")
64
+ endif()
61
65
 
62
66
  gea_apply_app_native_build()
67
+
68
+ include("${GEA_EMBEDDED_ROOT}/targets/esp32/streaming_dependency_patches.cmake")
69
+ cmake_language(DEFER DIRECTORY "${CMAKE_SOURCE_DIR}" CALL gea_patch_streaming_dependencies)
@@ -0,0 +1,46 @@
1
+ cmake_minimum_required(VERSION 3.24)
2
+
3
+ idf_component_register(SRCS juice_memory.c INCLUDE_DIRS . PRIV_REQUIRES heap esp_netif mbedtls)
4
+ if(CMAKE_SCRIPT_MODE_FILE)
5
+ return()
6
+ endif()
7
+
8
+ include(FetchContent)
9
+ FetchContent_Declare(gea_libjuice
10
+ URL https://codeload.github.com/0x1abin/libjuice/tar.gz/34bae17c9c361639f64347da689503d3b06c0803
11
+ URL_HASH SHA256=285a2461dc55804da7fe47ad48d58f2304c29d1f9fd644dee190a160416507b0
12
+ SOURCE_SUBDIR gea-no-upstream-cmake
13
+ DOWNLOAD_EXTRACT_TIMESTAMP TRUE
14
+ )
15
+ FetchContent_MakeAvailable(gea_libjuice)
16
+
17
+ # The fork's FreeRTOS adapter allocates task stacks in internal RAM and
18
+ # self-deletes. External-stack tasks require an explicit join/delete instead.
19
+ find_package(Python3 REQUIRED COMPONENTS Interpreter)
20
+ execute_process(COMMAND "${Python3_EXECUTABLE}" "${CMAKE_CURRENT_LIST_DIR}/patch_threads.py"
21
+ "${gea_libjuice_SOURCE_DIR}/src/thread.h" COMMAND_ERROR_IS_FATAL ANY)
22
+
23
+ set(juice_sources)
24
+ foreach(name addr agent base64 conn conn_poll conn_thread conn_mux const_time crc32
25
+ hash hmac ice juice log random server stun timestamp tcp turn udp compat)
26
+ list(APPEND juice_sources "${gea_libjuice_SOURCE_DIR}/src/${name}.c")
27
+ endforeach()
28
+ target_sources(${COMPONENT_LIB} PRIVATE ${juice_sources})
29
+ target_include_directories(${COMPONENT_LIB} PUBLIC "${gea_libjuice_SOURCE_DIR}/include")
30
+ target_include_directories(${COMPONENT_LIB} PRIVATE
31
+ "${gea_libjuice_SOURCE_DIR}/src" "${gea_libjuice_SOURCE_DIR}/include/juice")
32
+ set_source_files_properties(${juice_sources} PROPERTIES
33
+ COMPILE_OPTIONS "-include;${CMAKE_CURRENT_LIST_DIR}/juice_memory.h")
34
+ target_compile_definitions(${COMPONENT_LIB} PRIVATE
35
+ USE_NETTLE=0 USE_MBEDTLS=1 JUICE_STATIC JUICE_EXPORTS NO_SERVER
36
+ JUICE_TASK_STACK_SIZE=16384
37
+ udp_sendto=juice_udp_sendto udp_sendto_self=juice_udp_sendto_self
38
+ udp_recvfrom=juice_udp_recvfrom udp_create_socket=juice_udp_create_socket
39
+ udp_get_port=juice_udp_get_port udp_get_bound_addr=juice_udp_get_bound_addr
40
+ udp_get_local_addr=juice_udp_get_local_addr udp_get_addrs=juice_udp_get_addrs
41
+ udp_set_diffserv=juice_udp_set_diffserv
42
+ agent_get_local_description=juice_agent_get_local_description
43
+ agent_set_remote_description=juice_agent_set_remote_description
44
+ agent_send=juice_agent_send
45
+ hmac_sha1=juice_hmac_sha1 hmac_sha256=juice_hmac_sha256
46
+ hash_md5=juice_hash_md5 hash_sha1=juice_hash_sha1 hash_sha256=juice_hash_sha256)
@@ -0,0 +1,20 @@
1
+ # Gea ICE transport
2
+
3
+ Pinned ESP32 libjuice fork: `0x1abin/libjuice` at
4
+ `34bae17c9c361639f64347da689503d3b06c0803`, SHA-256 verified at configure.
5
+ Upstream MPL-2.0 source and license remain in CMake's dependency output.
6
+
7
+ This component supplies codec-independent ICE datagrams for the VP8 receiver.
8
+ It does not replace DTLS/SRTP, authenticate media, or decode video itself.
9
+ The existing Espressif audio peer is unaffected.
10
+
11
+ All libjuice heap allocations and its 16 KiB worker stack use PSRAM. Its
12
+ FreeRTOS thread adapter is patched at configure time to join, suspend, and
13
+ delete external-stack tasks correctly. The patch is idempotent and rejects
14
+ unexpected upstream source. Mutexes and kernel task metadata remain in the
15
+ memory required by FreeRTOS. No silent internal-RAM allocation fallback.
16
+
17
+ The shared network build includes this dependency for `gea_vp8_peer`. Its code
18
+ is only used for incoming VP8 offers; the default audio/H264 peer still uses
19
+ Espressif's existing ICE implementation. Hardware media verification remains
20
+ separate from compilation.
@@ -0,0 +1,19 @@
1
+ // SPDX-License-Identifier: Apache-2.0
2
+ #include "esp_heap_caps.h"
3
+ #include <stddef.h>
4
+
5
+ void* gea_juice_malloc(size_t size) {
6
+ return heap_caps_malloc(size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
7
+ }
8
+
9
+ void* gea_juice_calloc(size_t count, size_t size) {
10
+ return heap_caps_calloc(count, size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
11
+ }
12
+
13
+ void* gea_juice_realloc(void* pointer, size_t size) {
14
+ return heap_caps_realloc(pointer, size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
15
+ }
16
+
17
+ void gea_juice_free(void* pointer) {
18
+ heap_caps_free(pointer);
19
+ }
@@ -0,0 +1,16 @@
1
+ // SPDX-License-Identifier: Apache-2.0
2
+ #pragma once
3
+ #include <stdlib.h>
4
+ #include <string.h>
5
+
6
+ // Force-included only for libjuice translation units. Do not change allocation
7
+ // policy for lwIP, mbedTLS, the application, or the existing audio peer.
8
+ void* gea_juice_malloc(size_t size);
9
+ void* gea_juice_calloc(size_t count, size_t size);
10
+ void* gea_juice_realloc(void* pointer, size_t size);
11
+ void gea_juice_free(void* pointer);
12
+
13
+ #define malloc(size) gea_juice_malloc(size)
14
+ #define calloc(count, size) gea_juice_calloc(count, size)
15
+ #define realloc(pointer, size) gea_juice_realloc(pointer, size)
16
+ #define free(pointer) gea_juice_free(pointer)
@@ -0,0 +1,50 @@
1
+ #!/usr/bin/env python3
2
+ """Apply the reviewed PSRAM task-lifetime changes to the pinned libjuice fork.
3
+
4
+ Changes are confined to CMake's downloaded dependency; upstream MPL-2.0
5
+ copyright/license notices remain intact. Fail if the pinned source drifts.
6
+ """
7
+
8
+ from pathlib import Path
9
+ import sys
10
+
11
+
12
+ def patch(source: str) -> str:
13
+ changes = [
14
+ (
15
+ '#include <freertos/FreeRTOS.h>',
16
+ '#include "esp_heap_caps.h"\n#include <freertos/FreeRTOS.h>',
17
+ ),
18
+ (
19
+ '\txSemaphoreGive(done);\n\tvTaskDelete(NULL);',
20
+ '\txSemaphoreGive(done);\n\tfor (;;) vTaskSuspend(NULL);',
21
+ ),
22
+ (
23
+ 'xTaskCreate(juice_task_wrapper, "juice", JUICE_TASK_STACK_SIZE, w, 5, &t->handle)',
24
+ 'xTaskCreatePinnedToCoreWithCaps(juice_task_wrapper, "juice", '
25
+ 'JUICE_TASK_STACK_SIZE, w, 5, &t->handle, 0, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT)',
26
+ ),
27
+ (
28
+ '#define thread_join(t, res) do { (void)(res); '
29
+ 'xSemaphoreTake((t).done, portMAX_DELAY); vSemaphoreDelete((t).done); } while(0)',
30
+ '#define thread_join(t, res) do { (void)(res); '
31
+ 'xSemaphoreTake((t).done, portMAX_DELAY); '
32
+ 'while (eTaskGetState((t).handle) != eSuspended) vTaskDelay(1); '
33
+ 'vTaskDeleteWithCaps((t).handle); vSemaphoreDelete((t).done); } while(0)',
34
+ ),
35
+ ]
36
+ for old, new in changes:
37
+ if new in source:
38
+ continue
39
+ if source.count(old) != 1:
40
+ raise ValueError(f"Pinned libjuice thread adapter changed: {old!r}")
41
+ source = source.replace(old, new)
42
+ return source
43
+
44
+
45
+ if __name__ == "__main__":
46
+ path = Path(sys.argv[1])
47
+ original = path.read_text()
48
+ updated = patch(original)
49
+ if updated != original:
50
+ path.write_text(updated)
@@ -0,0 +1,16 @@
1
+ idf_component_register(SRCS vp8_peer.cpp INCLUDE_DIRS .
2
+ REQUIRES esp_peer
3
+ PRIV_REQUIRES gea_juice esp_libsrtp mbedtls esp_timer heap freertos)
4
+ if(CMAKE_SCRIPT_MODE_FILE)
5
+ return()
6
+ endif()
7
+
8
+ if(NOT DEFINED ENV{GEA_HOST_DIR})
9
+ message(FATAL_ERROR "The VP8 peer requires GEA_HOST_DIR from the Gea build")
10
+ endif()
11
+ target_include_directories(${COMPONENT_LIB} PRIVATE "$ENV{GEA_HOST_DIR}/include")
12
+ # esp_peer 1.5.5 exposes the DTLS/SRTP helper in its source component. Reuse
13
+ # its IDF-version-specific implementation and certificate/key lifecycle.
14
+ idf_component_get_property(peer_dir espressif__esp_peer COMPONENT_DIR)
15
+ target_include_directories(${COMPONENT_LIB} PRIVATE "${peer_dir}/src")
16
+ target_compile_features(${COMPONENT_LIB} PRIVATE cxx_std_20)
@@ -0,0 +1,39 @@
1
+ # Native VP8/Opus media peer
2
+
3
+ An `esp_peer_ops_t` implementation for bundled Opus/VP8 reception and a separate
4
+ audio-only Opus microphone sender. The shared Gea host selects reception from
5
+ SDP, or sending from an audio-only send transceiver with no configured ICE
6
+ servers (the hosted SFU supplies publicly reachable candidates). Other codec and data-channel
7
+ connections keep Espressif's default peer.
8
+
9
+ - libjuice supplies ICE, including connectivity checks and consent freshness.
10
+ - The pinned Espressif peer supplies its IDF-specific DTLS/SRTP implementation.
11
+ - SHA-256 of the remote DTLS certificate must match the authenticated signaling
12
+ SDP before any media reaches a callback.
13
+ - Fixed packet queues live in the PSRAM peer allocation. Normal in-order media
14
+ has no artificial packet wait; sequence gaps get at most a 30 ms reorder wait.
15
+ - The microphone encoder enqueues RTP packets; the peer worker performs SRTP
16
+ and sends them immediately, without sharing crypto state across tasks. Its
17
+ negotiated SSRC, Opus payload type and 48 kHz RTP clock match the local SDP.
18
+ Sender reports include packet/octet counts and the corresponding RTP clock.
19
+ Only a video receiver allocates the 64-packet incoming media queue.
20
+ - VP8 loss or decoder overflow triggers rate-limited PLI. RTX and generic NACK
21
+ are not advertised. Encoded reference frames cannot be skipped silently.
22
+ - Closing sets the shared running flag, aborting a pending handshake; cleanup
23
+ joins the ICE worker before releasing its callback context.
24
+
25
+ The receiver accepts one send-only Opus stream and one send-only VP8 stream in
26
+ one BUNDLE. The microphone sender offers a single send-only Opus stream and
27
+ requires a matching recv-only answer with passive DTLS and RTCP multiplexing.
28
+ It does not implement data channels, video sending,
29
+ relay-only policy, renegotiation or ICE restarts. The publicly reachable hosted
30
+ SFU is reached through outgoing ICE checks, including peer-reflexive NAT mapping.
31
+
32
+ RTP timestamps are carried into the existing audio/video callbacks, but received
33
+ RTCP sender-report clock synchronization is not implemented. No claim of measured
34
+ A/V synchronization or S3 frame rate is made by the portable packet tests.
35
+
36
+ Test the shared SDP, RTP, keyframe recovery and real VP8 decoding with
37
+ `bash packages/core/test/run-vp8-tests.sh` in the core repo. The transport
38
+ component cross-compiles against ESP-IDF 6.0.2; a full device conversation still
39
+ needs verification.