Repository navigation
Conversation
With MICROPY_GC_SPLIT_HEAP_AUTO the Python heap grows into the largest free IDF block until nothing is left. The wifi driver, lwIP and NimBLE allocate at runtime (TX buffers, station association, DHCP replies, ESP-NOW frames), so a Python program that fills its heap silently breaks the radio: esp_now_send() fails with ESP_ERR_ESPNOW_NO_MEM, softAP clients associate but never get a DHCP lease (udp_sendto fails), and start_ap() can hard fault. Reserve CIRCUITPY_ESP_RADIO_HEAP_RESERVE (32 KB, board-overridable) of the IDF heap whenever the wifi radio or the BLE adapter is enabled, by capping what port_heap_get_largest_free_size() reports to the GC. Programs that never turn a radio on keep the whole heap. Measured on an ESP32-C6 devkit (no PSRAM) running a wifi+ESP-NOW collector: before, every ESP-NOW send failed 0x3067 with idf_free=7884 largest=7680 and phones stuck at "obtaining IP address" (DHCP ACK udp_sendto -> -1); after, sends succeed (idf_free ~31 KB) and phones get leases.
A server-side SSLSocket did a full handshake for every connection. Browsers open several connections per page, and on a small MCU with an RSA-2048 certificate each handshake costs ~1.3 s of CPU (ESP32-C6), so pages stalled and speculative connections were abandoned. Configure mbedTLS session tickets (AES-256-GCM via PSA, 24 h lifetime, one process-wide key) on server-side contexts when MBEDTLS_SSL_SESSION_TICKETS and MBEDTLS_SSL_TICKET_C are available (ESP-IDF enables them by default), so returning clients resume with an abbreviated handshake and no public-key operation. Client-side behaviour is unchanged.
…and bench devkits The base branch defaults CIRCUITPY_ESP_RADIO_HEAP_RESERVE to 32 KB, which is what a plain softAP + ESP-NOW workload needs. Adding a TLS server on top needs more: at 32 KB the C6 hub still hit MemoryError starting a handshake and lost DHCP replies. Override to 40 KB on the Adafruit ESP32 / ESP32-S3 / ESP32-C6 Feathers, Metro S3, QT Py ESP32 / ESP32-S3, and the two bench devkits. Other Adafruit ESP32/S3 boards (ItsyBitsy, MatrixPortal, Sparkle Motion, Qualia, cameras) are deliberately left on the 32 KB default for now. Replaces the branch's old global 58140f6 ("radio heap reserve 40KB"), which raised the port-wide default instead of overriding per board.
…argets
The ESP-NOW / WiFi / BLE environmental collector ships ~20 libraries in
lib/. On a board without PSRAM that is the difference between running and
not. Measured on a Feather ESP32-S3 No PSRAM (2026-09-13), with the
application itself already cross-compiled to .mpy:
* BLE + ESP-NOW + softAP all come up, and then `import datastore` dies
with `MemoryError: memory allocation failed, allocating 158 bytes`;
* with BLE off it reaches the eInk dashboard and the SD mount fails for
want of 832 bytes, with 15.9 KB free.
Cross-compiling the application removes the on-device *compiler* peak --
which is what made the C6 unbootable -- but not the resident bytecode.
Freezing does: a frozen module executes in place from flash and never
occupies heap.
Twenty-two libraries, on the 13 espressif boards this project targets --
the same 13 the custom-build workflow has been dispatched for, and on the
branch those builds actually used.
Nine were not yet in frozen/ and are added as submodules: IL0373, JD79667,
MAX1704x, NTP, SCD4X, SCD30, SEN6x, PCF8523, DS1307. The other thirteen
were already there.
The 22 lines are repeated verbatim in each board's mpconfigboard.mk rather
than factored into one include. That is not an aesthetic choice: tools/
ci_fetch_deps.py reads each board file line by line looking for
"FROZEN_MPY_DIRS += $(TOP)/" to decide which submodules to check out, and
it does not follow includes. The include version builds fine locally,
FREEZE lists all 22 dirs, and then CI fails at MKMANIFEST with every
frozen directory empty. Each copy of the block carries a comment saying so
before someone tidies it back.
Deliberately not frozen:
* adafruit_pixelbuf -- already a built-in C module.
* sensirion_i2c_driver / sensirion_i2c_sen5x -- a third-party circup
bundle rather than a library repo, and node_sensors.py carries a
minimal built-in SEN5x driver for boards without them.
* PCF85063A -- no Adafruit library exists; the project drives that chip
directly from its own extrtc.py.
Size: ~162 KB of frozen bytecode from 621 KB of source, uniform across all
13. Comfortable on the 8 MB and 16 MB boards; the 4 MB ones are what CI
has to answer, and if one overflows the fix is to split the list rather
than shrink it everywhere.
NOTE for whoever deploys against these builds: sys.path puts /lib ahead of
.frozen, so a copy left in lib/ SHADOWS the frozen one and the board pays
the RAM anyway. Emptying lib/ is what collects the benefit -- the same
trap as leaving a .py beside a .mpy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
build-board-custom.yml calls .github/actions/deps/submodules without version: true, so steps.set-up-submodules.outputs.version is empty. For a board with frozen modules the mpy_cross action then runs make with CP_VERSION="" in its environment, and get_version_info_from_git() took the presence of the variable as authoritative, returned an empty tag, and makeversionhdr.py aborted with "Cannot determine version". Boards without frozen modules never run that step, which is why the Pico 2 W asset build passed while the Pico W one failed twice on the same commit. Fall back to git describe when CP_VERSION is present but empty. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The hub serves its portal over a Nordic UART service and the nodes take their config the same way, so _bleio is not optional for this project on the boards it is aimed at. Reading ports/espressif/mpconfigport.mk, it should already be on: CIRCUITPY_BLEIO_NATIVE defaults to 1 and is forced off only for the S2 and the P4 (no BLE hardware) and for 2MB flash -- not 4MB, which only loses dualbank and gains -Os. So on these four boards this is a no-op that says so out loud, and it stops a future flash-size trim taking BLE away silently the way the 2MB block already does. Boards: feather_esp32s3_4mbflash_2mbpsram, qtpy_esp32s3_4mbflash_2mbpsram, feather_esp32s3_tft, feather_esp32s3_reverse_tft. Size is not the constraint here, contrary to what a 4MB board suggests: the last green build of feather_esp32s3_4mbflash_2mbpsram used 1,841,040 of 2,883,584 bytes of firmware space, leaving 1,042,544 free -- room for this project's ~162 KB of frozen bytecode several times over.
wrap_socket(server_side=True) configured mbedTLS with VERIFY_REQUIRED whenever the context had the default root bundle attached, which every ssl.SSLContext() does. On a server that means "require a client certificate", so a plain HTTPS server -- load_cert_chain() and nothing else -- failed every handshake with MBEDTLS_ERR_SSL_NO_CLIENT_CERTIFICATE (-0x7480) and the client saw a timeout. Default server-side contexts to VERIFY_NONE, as CPython's do (CERT_NONE for servers); a context that loaded its own CA with load_verify_locations() still verifies clients. Client-side behaviour is unchanged. Seen on a Pico 2 W serving a captive portal over its softAP; the same code runs on every port using the shared mbedTLS module. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
On the ESP32-C5 the native USB-Serial/JTAG port enumerates and then stops answering EP0 within about a second of CircuitPython booting (host sees EPROTO / transaction errors with no device handshake), while the same board held in ROM download mode answers indefinitely. Under radio load (BLE + SoftAP + ESP-NOW) the port additionally drops and re-enumerates in ~16 s cycles. See adafruit#11673 and the bench forensics in tyeth-ai-assisted/esp-usbip-bridge#37. Mechanism: the USJ PHY's 48 MHz clock is derived from the BBPLL (USB_SERIAL_JTAG_LL_PHY_DEPENDS_ON_BBPLL in ESP-IDF's hal/usb_serial_jtag_ll.h). ESP-IDF keeps the BBPLL alive for the USJ via usb_serial_jtag_connection_monitor.c, whose tick hook calls rtc_clk_bbpll_add_consumer() while a host is attached; with no consumer registered, rtc_clk_cpu_freq_set_config() powers the BBPLL down whenever the CPU leaves a PLL-derived source (see esp_hw_support/port/esp32c5/rtc_clk.c, rtc_clk_update_pll_state_on_cpu_src_switching_end), which Wi-Fi/BLE modem-sleep retention does continuously (esp_hw_support/sleep_modem.c, rtc_clk_cpu_freq_to_pll_mhz_and_do_retention). CircuitPython never linked that monitor: IDF only force-links it when the USJ is the system console (esp_driver_usb_serial_jtag's CMakeLists.txt, -u usb_serial_jtag_connection_monitor_include), and CircuitPython builds with CONFIG_ESP_CONSOLE_NONE. Worse, our USJ ISR enabled the SOF interrupt and cleared its raw status at 1 kHz, which would have starved the monitor's SOF-based connect detection even if it had been linked. Fix: reference usb_serial_jtag_connection_monitor_include() from usb_serial_jtag_init() so the monitor's ESP_SYSTEM_INIT_FN is linked and runs, and stop enabling/clearing the SOF interrupt (nothing in this file used it). This applies to every USJ chip (C3/C6/C61/H2/C5); the C5 is where the bench reproduced the failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
|
Thanks for the fix. Did this used to work, and started failing due to an ESP-IDF upgrade, or did it never work? |
dhalbert
requested changes
Oct 27, 2026
dhalbert
left a comment
dhalbert
left a comment
Collaborator
There was a problem hiding this comment.
There are all kinds of changes here that seem unrelated to the point of the PR, including newer or older frozen libraries??
Member
Author
|
Sorry Dan ignore this one for now, I am not sure I think it used to work so maybe it is an idf change but it might have just been coming up as rare instability only particularly shown when pressure'd. |
tyeth
marked this pull request as draft
October 27, 2026 14:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On ESP32-C5 with
CONFIG_ESP_CONSOLE_NONE(CircuitPython's configuration), nothing registers as a BBPLL consumer for the USB-Serial/JTAG PHY (USB_SERIAL_JTAG_LL_PHY_DEPENDS_ON_BBPLL), so the clock framework drops the PLL on CPU clock-source switches — continuous under Wi-Fi/BLE modem sleep — and the USJ port stops transacting at EP0 shortly after boot, while enumeration (immediately after reset) and ROM download mode both work.Fixes #11673.
Change
Force-link ESP-IDF's
usb_serial_jtag_connection_monitorfromusb_serial_jtag_init()so the BBPLL keeps its consumer while a host is attached, and stop CircuitPython's USJ ISR from enabling/clearing the SOF interrupt flag the monitor's tick hook relies on (nothing in the supervisor used it). One file, +6/-6 code lines.Hardware verification
On a C5 DevKitC-1-N8R8 that failed every host EP0 probe (-71/EPROTO) across days of testing, the fixed build passed probes at +0/+30/+90/+180 s under full radio load (BLE advertising + SoftAP + ESP-NOW), survived a usbip import with an interactive REPL, and showed zero spontaneous re-enumerations (previously ~16 s cycling). Forensic trail: tyeth-ai-assisted/esp-usbip-bridge#37
Notes
This is the same failure family the ESP32-C6 went through in ESP-IDF, where the fix was exactly this connection-monitor/BBPLL-consumer infrastructure — which IDF only links when the USJ is the system console. Other USJ chips built with
CONSOLE_NONE(C3/C6/C61/H2) may be affected to varying degrees; the change applies to all of them.🤖 Generated with Claude Code