0%

Introduction

In the previous blog, we verified the hardware and development environment.

This time, we will go through another interesting example: touchpad-digit-recognition.

This is an ESP AI example that demonstrates the complete deep learning development workflow, from data collection and training to device deployment.

In this blog, I will clone this example and run the code on my own hardware, which does not have a touch sensor.

How This Blog May Help You

  • Learn ESP-DL in detail.
  • If you want to run the example code on your own hardware, this blog will illustrate some pitfalls encountered during code refactoring.
  • You should comment out the touch sensor driver and button driver code if you are not using the same hardware as the example.
  • Make sure you clearly understand the col and row definitions in the example code.

Prerequisites

Development Environment

  • MCU: General ESP32-S3 dev kit (Waveshare ESP32-S3-DEV-Kit-N8R8)
  • IDE: VS Code with ESP-IDF extension
  • IDF version: v5.4.2
  • ESP-DL version: v3.1.5
  • touchpad-digit-recognition example code: Commit id: 3e35842 date: Jun 9, 2025

Code Analysis

  • The data flow is straightforward: it gets data from the touch sensor (7x6), expands it to 30x25 pixel data, then feeds it to the neural network and calls the predict function of the ESP-DL API.
  • The important functions are touch_digit_task, which handles data input to the network, and touch_digit_recognition_task, which calls the ESP-DL API.
  • The input data is a 1D array of uint8 values (0 or 1), representing the digit.

Running the Example Code: Step by Step

  1. I couldn’t find the example in the IDF extension, so I directly copied the code from GitHub. Copy the entire esp-iot-solution repository.

    source-code-location

  2. Open the folder touchpad_digit_recognition located at esp-iot-solution-master\examples\ai\esp_dl\touchpad_digit_recognition\.

  3. Set the device type to esp32s3, then build and flash the firmware.

    config-sdkconfig

  4. Unfortunately, after flashing the firmware, I encountered a PSRAM error.

    run-error

  5. This error did not occur in the previous example, so it is likely due to a configuration issue in sdkconfig.

  6. Update the sdkconfig: Make sure to use “Octal Mode PSRAM”.

    update-spiram-mode

  7. The PSRAM error was fixed, but another error occurred: normalization save.

    erreo2

  8. After reviewing the code in normalization_save.cpp, I found a bug: it checks the NVS but does not verify if the KEY data exists. As a result, the error always appears unless you call set_normalization_data. My solution was to add a call to set_normalization_data after nvs_get_blob if there is an error.

    normalization-error

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    ret = nvs_get_blob(my_handle, KEY, data, &len);
    if (ret != ESP_OK)
    {
    ESP_LOGW(TAG, "Warn (%s) nvs_get_blob data fail. Set a new data", esp_err_to_name(ret));
    touch_digit_data_t _data = {};
    set_normalization_data(&_data);

    ret = nvs_get_blob(my_handle, KEY, data, &len);
    if (ret != ESP_OK)
    {
    ESP_LOGE(TAG, "Error (%s) reading normalization data!", esp_err_to_name(ret));
    nvs_close(my_handle);
    return ret;
    }
    }
  9. From the code, the definitions of ROW and COL are not as expected. The row length is the number of vertical series, and col is the number of horizontal series. This is unclear and could be considered a bug in their code.

    config-sdkconfig

Normally, ROW should refer to number of horizontial series. COL should refer to number of vertical series.

  1. With the help of tool.html, which can generate a 1D array representing the 30x25 pixels of a digit, you can test the code with fixed data first using this tool.

    The tool uses pure JS, HTML, and CSS. Reference: https://gitee.com/Shine_Zhang/esp32s3_dl_helloworld/tree/main/04_tool

    digi-tool

  2. Refactor the touch_digit_task. The entry point for data input to the neural network is xQueueSend(xImageQueue, &image_data, portMAX_DELAY).

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    static void touch_digit_task(void *arg)
    {
    // Copy the raw data to the TouchImage
    memcpy(g_image.data, mnistData, sizeof(mnistData));
    // Print the TouchImage row length and column length
    ESP_LOGI(TAG, "TouchImage row length: %d, column length: %d\n", (int)g_image.row_length, (int)g_image.col_length);
    while (1)
    {
    // Send data to DL inference
    image_data_t image_data;
    image_data.size = g_image.col_length * g_image.row_length;
    image_data.data = new uint8_t[image_data.size];
    if (image_data.data != NULL)
    {
    memcpy(image_data.data, g_image.data, image_data.size);
    xQueueSend(xImageQueue, &image_data, portMAX_DELAY);
    }
    vTaskDelay(3000); // Delay for 3 seconds before next iteration
    }
    }
  3. Build and run—success!

    run-ok

Issue Summary

  • Pay attention to sdkconfig. Incorrect PSRAM settings will cause PSRAM issues.
  • Ensure the input data format is correct. Incorrect row and col values will lead to wrong predictions.
  • Comment out all hardware-specific code that is not relevant to your device. Otherwise, it may cause unexpected behavior.

*In the next post in this series, we will replicate this example code to become more familiar with the entire development

Introduction

This is the first post in a series of blogs that will explore the ESP-DL library in detail.

At the time of writing, there is very little information or tutorials available for ESP-DL, especially practical guides on how to use it effectively. The official documentation from Espressif only explains what the library is, but does not clarify why you might need it, how to use it correctly, or provide helpful tips.

In this post, I will focus on running an example code from the ESP-DL library to get started.

Note:
This tutorial was written on 16-Aug-2025. If you are reading this in 2027 or later, some information may be outdated.

How This Blog May Help You

  • If you want to dive into the ESP-DL library, you may find useful details and practical steps here.
  • When working with a new library, it’s always best to start by building and flashing the simplest example to test your hardware and development environment.

Core Values

  • Software is like food—it has an expiration date, typically around 2 years. When learning a new library, you must dive in and understand how it works. Otherwise, you may find that everything has changed in a couple of years, with new libraries and completely different APIs.
  • Focus on identifying the most stable concepts or principles from the library, as these are more likely to remain relevant.

Prerequisites

Development Environment

  • MCU: General ESP32-S3 dev kit (Waveshare ESP32-S3-DEV-Kit-N8R8)
  • IDE: VS Code with ESP-IDF extension
  • IDF version: v5.4.2
  • ESP-DL version: v3.1.5

Running the Example Code: Step by Step

The first step is to run a simple example to ensure your hardware and development environment are working correctly.

  1. Install the example code via the Components Manager in the VS Code extension.

    step1-components-manager

  2. Find the esp-dl library and select the how_to_run_model example.

    step2-how-to-run-model

  3. Click “Create project from this example”.

    step3-create-proj-from-example

  4. The extension will automatically download the code and open it in a new window for this example.

  5. Set the target device.
    The extension will generate the appropriate build command.

    step4-set-target

  6. Click “Build” and then “Flash” to upload the firmware.

    build-success

    Runs-success

For me, the example code ran perfectly on my device. This confirms that both the library and my development environment are set up correctly.

This initial verification is very important. If anything goes wrong in the future, I can always re-flash this working firmware to check if the issue is hardware-related or due to configuration (e.g., sdkconfig). This approach can save a lot of debugging time.


Stay tuned for the next post in this series, where I will dive deeper into ESP-DL features and real-world applications.

Learning Pytorch in 2025

I want to develop an AI applciation using ESP32-S3 and discover that you need to know some basic kowledege about nerual network.

ESP32 have AI library ESP-DL, but you need pytorch to train the model.

Therefore, the first step is to learn pytorch and some basic deep learning concept. Otherwise it is very diffcult to code without background knowledge.

How This Blog May Help You

  • This blog will give you some importants points and reviews on the youtube video

Learn from youtube:

Link: PyTorch for Deep Learning & Machine Learning – Full Course https://www.youtube.com/watch?v=V_xro1bcAuA

The first 2 chapters

It is a introduction chapters which using a simple example of linear formula to demostrate the work flow of a neuru network.

Basically shows how deep learning works with pytorch, basic naming convention, concept in deep learning and pytorch

He use 8 hours of video for 2 chapters.

  • Naming convention: scalar, vector in lower case, MATRIX, TENSOR in upper case. Why? I don’t know. It is just defacto standard.
  • Random numbers are important. There are work flow : random number-> look at data-> update random number
  • 3 main errors you will face in pytorch
      1. Incorrect shape: Tensor should have the same shape.
      1. Incorrect device: what hardware implementation is using on that tensor, cpu or cuda. Error will pop up if two tensor have different device
      1. Incorrect data type: Erro will pop up if tensor have differernt datatype
  • tensor.mean() only accept float and complex datatype
  • numpy datatype use float64 by default when convert to tensor
  • Basic work flow: Input->forward->loss fucction->optimize-> loop

The 3rd and 4th chapters

Introduce other kinds of deep learning: classification, CNNs, and using custom datasets.

  • Use sklearn’s train_test_split to split the dataset.
  • A linear formula does not work with non-linear data. Add an activation function to introduce non-linear characteristics.
  • Introduction to activation functions.
  • Typically, you need a softmax function to convert logits to prediction probabilities and then to classification labels.

Review

  • This is an excellent video for beginners who do not have much background in neural networks.
  • It focuses on coding and workflow, with concise explanations of how neural networks work.
  • The instructor demonstrates real-time debugging while implementing the neural network, which is very helpful for understanding the process.

Reference

Arthur’s GitHub link: https://github.com/mrdbourke

ESP32-S3 Yet Another Sketchpad

A demo project for ESP32-S3 using LVGL to implement a sketchpad on a 320 x 240 TFT display.

This is a sibling project to YetAnotherStockPriceTracker and uses the same hardware.

Almost 90% of the code is referenced from lv_100ask_sketchpad, but updated for LVGL v9.2.2 and fixed for compatibility with ESP32-S3.

Project Core Values

  • For learning purposes.
  • The implementation is kept simple and straightforward.
  • No fancy stuff, make it work.

How This Project May Help You

  • If you want LVGL to use external PSRAM for large buffers, add the following to your platformio.ini. This lets LVGL use the standard C memory allocation API. When the allocated memory size is >4k, it will automatically use external PSRAM by default. This works with LVGL v9.2.2 and PlatformIO ESP32 Arduino framework (v2.0.5?).

    1
    -D 'LV_USE_STDLIB_MALLOC=LV_STDLIB_CLIB'
  • Espressif and PlatformIO are no longer collaborating, so PlatformIO does not support ESP32 Arduino framework versions >v2.0.xx. If you want to use the latest ESP32 Arduino framework (>v3), please use the Arduino IDE or the PIO Arduino VS Code extension.

    Reference: Using ESP-IDF v5.x with Arduino on ESP32S3

Screenshots & Demo

sketchpad

Features

  • Simple sketchpad to draw on the screen
  • Button to clear the screen
  • Button to change the line width

Development Timeline

Date Milestone/Note
2025-07-16 Project started
2025-07-21 v1 release

Getting Started

Prerequisites

  • VS Code with the PlatformIO extension
  • ESP32-S3 board + 2.8” TFT LCD
    • Hardware purchased from Taobao. More info: ESP32-LVGL开发板

      Note: This dev board is not recommended due to hardware issues. It may reset automatically, especially when connected to a PC.

    • ESP32-S3-N8R8
    • LCD Driver: ST7989, 320 x 240 pixels
    • Touch Driver: XPT2046

Building & Flashing

  1. Clone this repository:

    1
    git clone https://github.com/tommokmok/esp32s3_YetAnotherSketchpad
  2. Open the project in PlatformIO (VS Code).

  3. Build and upload to your ESP32-S3 board.

File Structure

All libraries are placed in the local lib/ directory for convenience.

Pros:

  • No need to download libraries from the web when opening the project for the first time.
  • You can modify the source if you need specific functions.

Cons:

  • Larger repository size.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
esp32s3_YetAnotherSketchpad/

├── lib/ # External and user libraries
│ ├── lvgl/ # LVGL graphics library (if added locally)
│ ├── LovyanGFX/ # LovyanGFX display library (if added locally)
│ ├── hal/ # Hardware Abstraction Layer (display, touch, etc.)
│ ├── appController/ # Application controller logic (state management, UI flow)
│ └── sketchpad/ # Sketchpad implementation (drawing logic, UI components)
├── src/ # Source code
│ ├── main.cpp # Main application entry point
│ └── ... # Other source files
├── platformio.ini # PlatformIO project configuration
├── README.md # This documentation

  • lib/lvgl/: LVGL graphics library (added locally).
  • lib/LovyanGFX/: LovyanGFX display library (added locally).
  • lib/hal/: Hardware Abstraction Layer for display, touch, and other peripherals.
  • lib/appController/: Application controller logic, such as state management and UI flow.
  • lib/sketchpad/: Sketchpad UI implementation.
  • src/main.cpp: Main application logic, including LVGL initialization and sketchpad setup.

Note: If you install libraries via the PlatformIO Library Manager, they will be managed automatically. If you add libraries manually, place them in the lib/ directory as shown above.

Implementation Details

1. Porting from 100ask code

The original source code is available at: https://github.com/100askTeam/lv_lib_100ask/tree/master/src/lv_100ask_sketchpad.

The main tasks were porting the code to the current hardware and updating it for LVGL v9.2.2.

Note: LVGL v9.1 does not work for this example, nothing will be shown on the canvas.

Issues while porting to the ESP32-S3 dev board

  • "error: field 'img' has incomplete type"

    • This occurs because the compiler can’t find some struct definitions.
    • Solution: Add the following include for internal definitions, especially if you are customizing widgets:
      1
      #include "lvgl/src/lvgl_private.h"
  • No colorwheel in lvgl >v9.0

  • 'dram0_0_seg' overflowed

    • The sketchpad needs a canvas object and buffer, which requires 240x320x2 bytes.

    • PlatformIO uses ESP-IDF v4.4 by default, which lacks some PSRAM features.

    • Add the following definition in platformio.ini to use standard C malloc for memory allocation. When the allocation is >1k, it will use external RAM by default:

      1
      -D 'LV_USE_STDLIB_MALLOC=LV_STDLIB_CLIB'
    • Default SDK config location on Windows:

      1
      2
      3
      4
      5
      6
      // The default sdkconfig location in Windows
      %homepath%\.platformio\packages\framework-arduinoespressif32\tools\sdk\esp32s3

      // Default PSRAM usage definitions
      CONFIG_SPIRAM_USE_MALLOC=y
      CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=4096

Customization

If you have different LCD or touch hardware, update the lvgl and LovyanGFX configurations accordingly.

  • For LVGL, add configuration options in platformio.ini.
  • For LovyanGFX, edit include/custom_disp.hpp.

License

MIT

Add the tags and categories page

You need to use below command to generate the web page for the tags and categories. Otherwise, it will show nothing on the tags and categories page.

1
2
3
hexo new page tags
hexo new page categories

Update the scafflods

Update the post.md in scafflods folder and add tags and ctegorees for each blog when using the hexo new command

1
2
3
4
5
title: {{ title }}
date: {{ date }}
categories: {{ categories }}
tags: {{ tags }}

More tags in each blog

If you want to have more tags in one blog, please using below convention in the blog.

1
2
3
4

tags:
- hexo
- blog

Conculsion

I am re-do sometings that someone or many orders that have done many years ago.
Maybe it is too late to start a my own technical blog.

Base on my observation, many blogs that use Hexo + Next theme are not maintance after 2019…

So far, the should should fully functional with minimal features.

Next, I will adding more contents.

Reference

Blogs for 4 years ago
https://op30132.github.io/2019/12/24/hexo-tag-page/
https://kemushi54.github.io/

Super good reference for hexo blog from zero. If I follow this, then I can save lots of time.
https://ithelp.ithome.com.tw/users/20139218/ironman/3910?page=1

Introduction

A step-by-step guide to setting up a Hexo blog on GitHub Pages in 2025, highlighting common pitfalls and important notes for a smooth experience.


Prerequisites


Step 0: Install nvs and hexo-cli

  • Downlaod and install the nvs
  • You may encounter the below Policy error

Error

  • Update the policy in windows powershell terminal in Admin. mode
1
2
3
Get-ExecutionPolicy
Get-ExecutionPolicy -List
Set-ExecutionPolicy RemoteSigned

Step 1: Install Hexo

  • Install hexo-cli via npm
1
npm install -g hexo-cli

Step 2: Initialize Your Blog

1
2
3
hexo init my-blog
cd my-blog
npm install

Important:
Make sure you are in an empty directory before running hexo init.


Step 3: Create a GitHub Repository

  • Name it <username>.github.io for user/organization pages.

  • For project pages, use any repo name and set the correct branch in Hexo config.

github-page-settings

Note:
Repository name and branch matter for GitHub Pages to work.


Step 4: Configure _config.yml

  • Set your url to match your GitHub Pages URL:
    https://<username>.github.io

config-url

  • Update deploy settings for GitHub Pages.

github-deploy

Important:

  • Incorrect url or deploy config can cause broken links or failed deployments.
  • The url link is different in the url and deploy setting

Step 5: Add .nojekyll file

There are new steps for Github if you want to directly depkoy the web but not using Github internal build process.

Add a .nojekyll file in the root of the deploy files

So, you need to add a source file in Hexo

  1. Add .nojekyll file in soruce /_posts folder
add-nojekyll-in-src
  1. Add Include settgins in _config.yml file
add-nojeyll-in-config

Step 6: Deploy with Hexo

Install the deployer:

1
npm install hexo-deployer-git --save

Update _config.yml:

1
2
3
4
deploy:
type: git
repo: https://github.com/<username>/<repo>.git
branch: main

Deploy:

1
2
3
4
5
hexo clean && hexo generate && hexo deploy

# On Windows, you need to split the above command
hexo clean
hexo d -g

Theme setting

Have a look at the Hexo themes here: Theme

However, most of them seem outdated and are not maintained anymore. Finally, I chose the hexo-theme-next theme.

It has a new repo for maintenance: https://github.com/theme-next/hexo-theme-next

However, it has not been updated for 5 years….

I have a little bit of regret choosing the Hexo blog framework…


Conclusion

Setting up Hexo on GitHub Pages in 2025 is straightforward but seems it is a bit outdated.


References

That is a test

Welcome to Hexo! This is your very first post. Check documentation for more info. If you get any problems when using Hexo, you can find the answer in troubleshooting or you can ask me on GitHub.

Quick Start

Create a new post

1
$ hexo new "My New Post"

More info: Writing

Write new post

1
hexo new [layout] <title>

Run server

1
$ hexo server

More info: Server

Generate static files

1
$ hexo generate

More info: Generating

Deploy to remote sites

1
$ hexo deploy

More info: Deployment