Introduction
In the previous blog, we verified the hardware and development environment.
This time, we will go through another interesting example: touchpad-digit-recognition.
This is an ESP AI example that demonstrates the complete deep learning development workflow, from data collection and training to device deployment.
In this blog, I will clone this example and run the code on my own hardware, which does not have a touch sensor.
How This Blog May Help You
- Learn ESP-DL in detail.
- If you want to run the example code on your own hardware, this blog will illustrate some pitfalls encountered during code refactoring.
- You should comment out the touch sensor driver and button driver code if you are not using the same hardware as the example.
- Make sure you clearly understand the
col
androw
definitions in the example code.
Prerequisites
- Install the ESP-IDF according to the official guide.
Development Environment
- MCU: General ESP32-S3 dev kit (Waveshare ESP32-S3-DEV-Kit-N8R8)
- IDE: VS Code with ESP-IDF extension
- IDF version: v5.4.2
- ESP-DL version: v3.1.5
- touchpad-digit-recognition example code: Commit id: 3e35842 date: Jun 9, 2025
Code Analysis
- The data flow is straightforward: it gets data from the touch sensor (7x6), expands it to 30x25 pixel data, then feeds it to the neural network and calls the predict function of the ESP-DL API.
- The important functions are
touch_digit_task
, which handles data input to the network, andtouch_digit_recognition_task
, which calls the ESP-DL API. - The input data is a 1D array of
uint8
values (0 or 1), representing the digit.
Running the Example Code: Step by Step
I couldn’t find the example in the IDF extension, so I directly copied the code from GitHub. Copy the entire esp-iot-solution repository.
Open the folder
touchpad_digit_recognition
located atesp-iot-solution-master\examples\ai\esp_dl\touchpad_digit_recognition\
.Set the device type to
esp32s3
, then build and flash the firmware.Unfortunately, after flashing the firmware, I encountered a
PSRAM
error.This error did not occur in the previous example, so it is likely due to a configuration issue in
sdkconfig
.Update the
sdkconfig
: Make sure to use “Octal Mode PSRAM”.The PSRAM error was fixed, but another error occurred:
normalization save
.After reviewing the code in
normalization_save.cpp
, I found a bug: it checks the NVS but does not verify if theKEY
data exists. As a result, the error always appears unless you callset_normalization_data
. My solution was to add a call toset_normalization_data
afternvs_get_blob
if there is an error.1
2
3
4
5
6
7
8
9
10
11
12
13
14
15ret = nvs_get_blob(my_handle, KEY, data, &len);
if (ret != ESP_OK)
{
ESP_LOGW(TAG, "Warn (%s) nvs_get_blob data fail. Set a new data", esp_err_to_name(ret));
touch_digit_data_t _data = {};
set_normalization_data(&_data);
ret = nvs_get_blob(my_handle, KEY, data, &len);
if (ret != ESP_OK)
{
ESP_LOGE(TAG, "Error (%s) reading normalization data!", esp_err_to_name(ret));
nvs_close(my_handle);
return ret;
}
}From the code, the definitions of ROW and COL are not as expected. The row length is the number of vertical series, and col is the number of horizontal series. This is unclear and could be considered a bug in their code.
Normally,
ROW
should refer to number of horizontial series.COL
should refer to number of vertical series.
With the help of
tool.html
, which can generate a 1D array representing the 30x25 pixels of a digit, you can test the code with fixed data first using this tool.The tool uses pure JS, HTML, and CSS. Reference: https://gitee.com/Shine_Zhang/esp32s3_dl_helloworld/tree/main/04_tool
Refactor the
touch_digit_task
. The entry point for data input to the neural network isxQueueSend(xImageQueue, &image_data, portMAX_DELAY)
.1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20static void touch_digit_task(void *arg)
{
// Copy the raw data to the TouchImage
memcpy(g_image.data, mnistData, sizeof(mnistData));
// Print the TouchImage row length and column length
ESP_LOGI(TAG, "TouchImage row length: %d, column length: %d\n", (int)g_image.row_length, (int)g_image.col_length);
while (1)
{
// Send data to DL inference
image_data_t image_data;
image_data.size = g_image.col_length * g_image.row_length;
image_data.data = new uint8_t[image_data.size];
if (image_data.data != NULL)
{
memcpy(image_data.data, g_image.data, image_data.size);
xQueueSend(xImageQueue, &image_data, portMAX_DELAY);
}
vTaskDelay(3000); // Delay for 3 seconds before next iteration
}
}Build and run—success!
Issue Summary
- Pay attention to
sdkconfig
. Incorrect PSRAM settings will cause PSRAM issues. - Ensure the input data format is correct. Incorrect
row
andcol
values will lead to wrong predictions. - Comment out all hardware-specific code that is not relevant to your device. Otherwise, it may cause unexpected behavior.
*In the next post in this series, we will replicate this example code to become more familiar with the entire development