Introduction

I recently got an internship where I was tasked to program a microcontroller, specifically ESP32-based M5TimerCamera. The camera is connected to an external lens and is used as a microscope to display the images taken onto a smartphone device. By joining the WiFi access point of the M5Camera and going to the IP address from the browser, images can be captured by clicking a button. Furthermore, specific settings like exposure and gain values can be changed from the web browser as well.

In this blog post, I’ll walk you through my custom ESP32 camera system, which allows remote image capture, LED indication, power management, and a simple web server interface. The system is built using an ESP32-based M5Stack TimerCAM module and leverages FreeRTOS tasks for efficient operation.

Features

  • Task-Based Image Capture: Runs a separate task for handling camera captures without blocking the main loop.
  • Web Server: Allows users to capture images remotely and adjust camera settings.
  • Power Management: Puts the ESP32 into deep sleep after inactivity to conserve power.
  • Adjustable Camera Settings: Users can modify exposure and gain dynamically via web requests.
  • Parallel Processing: Camera operations run on Core 0 while the main application logic (web server, WiFi, UI) runs on Core 1, enabling true parallelism.
  • Non-blocking Operation: Ensures that the main application remains responsive during image capturing.

Demo Pictures

UI Picture Regular Picture
Third Image Fourth Image

Hardware Requirements

  • M5Stack TimerCAM (or similar ESP32 camera module)
  • Power source (Battery or USB)
  • LED for capture indication (optional)

Code Breakdown

1. Camera Configuration

Firstly, I needed to configure the pins correctly using the information available in the internet. I had to tinker a bit with settings like .jpeg_quality, .frame_size and .xclk_freq_hz to find the optimal settings for the camera to function with no bugs. This is because, when using a higher resolution with minimal compression (for example .jpeg_quality = 1) the images tend to have artifacts as there were both processing and memory challenges that the device could not keep up.

#include <Arduino.h>
#include "camera.h"
#include <esp_camera.h>

camera_config_t config = {
    .pin_pwdn = -1,        // Not used on M5Stack Timer CAM
    .pin_reset = 15,       // RESET pin
    .pin_xclk = 27,        // XCLK pin
    .pin_sccb_sda = 25,    // SDA pin
    .pin_sccb_scl = 23,    // SCL pin
    .pin_d7 = 19,          // Y9 pin
    .pin_d6 = 36,          // Y8 pin
    .pin_d5 = 18,          // Y7 pin
    .pin_d4 = 39,          // Y6 pin
    .pin_d3 = 5,           // Y5 pin
    .pin_d2 = 34,          // Y4 pin
    .pin_d1 = 35,          // Y3 pin
    .pin_d0 = 32,          // Y2 pin
    .pin_vsync = 22,       // VSYNC pin
    .pin_href = 26,        // HREF pin
    .pin_pclk = 21,        // PCLK pin

    .xclk_freq_hz = 20000000, // Frequency 
    .ledc_timer = LEDC_TIMER_0,
    .ledc_channel = LEDC_CHANNEL_0,
    .pixel_format = PIXFORMAT_JPEG, 
    .frame_size = FRAMESIZE_UXGA, // Resolution of image
    .jpeg_quality = 3,  // Compression level (higher = more compression)         
    .fb_count = 1,      // Number of frame buffers in memory        
    .grab_mode = CAMERA_GRAB_LATEST               
};

The problem was even more evident before I implemented multi-threading. I had to use lower camera settings to take pictures.

That is why I implemented it to make the camera task run separately on Core 0, handling image capture requests asynchronously. This ensures that everything else can run smoothly on Core 1. This fixed issues like sudden connection losses and crashed.

void cameraTask(void *parameter) {
    while (true) {
        if (captureRequested && !captureInProgress) {
            captureInProgress = true;
            capturedFrame = esp_camera_fb_get();
            captureRequested = false;
            captureInProgress = false;
            if (capturedFrame) {
                xSemaphoreGive(frameReadySemaphore);
            }
        }
        vTaskDelay(5 / portTICK_PERIOD_MS);
    }
}

2. Web Server for Remote Access

In this section, the ESP32 sets up a web server that listens on port 80, allowing remote devices to send HTTP requests to control the camera. The core function, handleCapture(), is responsible for initiating an image capture when a client accesses the /capture endpoint. Once the image is captured using our task-based method, it sends the JPEG image data as an HTTP response.

#include <WiFi.h>
#include <ESPAsyncWebServer.h>
#include "camera.h"

AsyncWebServer server(80);

void handleCapture(AsyncWebServerRequest *request) {
    camera_fb_t *fb = captureImage();
    if (!fb) {
        request->send(500, "text/plain", "Camera capture failed");
        return;
    }
    request->send(200, "image/jpeg", fb->buf, fb->len);
}

The image can then be viewed and saved into the device from the web browser.

3. Power Management

Another problem I was facing was battery life. At most the device lasted for around 20 minutes. To extend battery life, the system monitors activity by seeing when was the last time /capture endpoint was accessed and enters deep sleep if it hasn’t been accessed in more than 3 minutes.

#include "power.h"
#include <esp_sleep.h>

unsigned long lastCaptureTime = 0;
const unsigned long inactivityTimeout = 180000;

void checkInactivity() {
    if (millis() - lastCaptureTime >= inactivityTimeout) {
        Serial.println("Entering deep sleep...");
        esp_deep_sleep_start();
    }
}

4. Setting Up the Web Interface

The ESP32 also serves a user-friendly HTML page stored in SPIFFS, which acts as the control panel for the camera. When a client navigates to the root URL (/), the server checks for the existence of an index.html file. If found, it sends the HTML content to the client; otherwise, it returns a 404 error. This HTML interface includes various elements such as a status panel, image display, and controls (buttons and sliders) to capture images, update camera settings, and even trigger deep sleep.

server.on("/", HTTP_GET, [](AsyncWebServerRequest *request){
    if (SPIFFS.exists("/index.html")) {
        request->send(SPIFFS, "/index.html", "text/html");
    } else {
        request->send(404, "text/plain", "File not found");
    }
});

5. Web Interface Design and Functionality

The web interface is designed to provide a user-friendly experience for capturing images and adjusting camera settings. It consists of:

  • A status panel displaying IP address, battery level, and countdown timer before shutdown.
  • An image container that shows the captured image.
  • A control panel with sliders for exposure and gain adjustment.
  • A capture button to trigger image capture remotely.
  • A sleep button to put the device into deep sleep manually.

The interface dynamically updates battery level and countdown timer using JavaScript, making periodic requests to the ESP32 server. Users can modify exposure and gain using sliders, and the settings are applied instantly upon clicking the update button.

Conclusion

While working on this I have learned a lot. From C++ to multithreading, this project has been quite rewarding. This project has helped me develop skills like problem solving and communication skills that I developed working with my supervisor in the lab.