AI Human-Tracking Robot Using QR + Face Recognition + ESP32 + n8n + AI Agent + Telegram + Google Sheets + ThingSpeak
Yes — this can be built as a complete Agentic IoT robot in which the ESP32 handles real-time sensing/tracking, while n8n acts as the cloud automation/AI orchestration layer.
A practical architecture is to use an ESP32-S3 camera board for computer vision rather than trying to put heavy AI workloads on a basic ESP32-WROOM. Espressif's current ESP-DL stack provides human-face detection and face-recognition APIs, and Espressif notes that ESP32-S3 is substantially better suited to ESP-DL workloads than the original ESP32.
Below is a complete project blueprint you can use for a final-year project, thesis, prototype, or product-style implementation.
1. Project title
AI Human Tracking Robot Using QR Code and Face Recognition with ESP32, Agentic IoT, n8n Automation, Telegram Voice Alerts, Google Sheets and ThingSpeak Cloud Dashboard
Short title
AI Agentic Human-Following ESP32 Robot
2. Project abstract
The proposed system is an intelligent mobile robot capable of identifying and following a designated person using QR-code identification and face recognition.
The robot uses an ESP32-S3 camera platform to acquire images and perform computer-vision processing. A QR code can be used as an initial identification mechanism, while face recognition provides continuous identity verification. Once the target person is identified, the robot calculates the person's position in the camera frame and controls its left and right motors to follow the person.
The robot is connected to Wi-Fi and communicates with an n8n automation server. n8n functions as the IoT orchestration and agent layer. Robot events are sent to n8n through a webhook/API. The workflow can log events to Google Sheets, update ThingSpeak, generate AI-based interpretations, and send Telegram notifications.
For example:
"Target person lost for 15 seconds"
can automatically become a Telegram notification such as:
🚨 Robot Alert: The registered target has not been detected for 15 seconds. Robot has stopped safely.
The system therefore combines:
- Embedded AI
- Computer vision
- Face recognition
- QR-code recognition
- Mobile robotics
- ESP32 IoT
- n8n workflow automation
- AI agents
- Telegram notifications
- Voice alerts
- Google Sheets
- ThingSpeak
- Web dashboard
- Cloud telemetry
Espressif's ESP-DL provides neural-network inference support and includes human face detection and recognition functionality.
3. Main objectives
The robot should be able to:
- Detect a person.
- Identify the authorized person.
- Scan a QR code.
- Combine QR and face identity information.
- Track the target's position.
- Follow the target autonomously.
- Stop when the target disappears.
- Avoid obstacles.
- Send robot telemetry to the cloud.
- Log events automatically.
- Notify the operator through Telegram.
- Generate voice notifications.
- Allow an AI agent to analyze robot events.
- Display robot information on a web dashboard.
- Maintain an operational history in Google Sheets.
- Visualize telemetry through ThingSpeak.
- Accept remote commands through the automation layer.
4. Overall system architecture
┌───────────────────────┐
│ HUMAN │
│ │
│ QR Code + Face │
└───────────┬───────────┘
│
▼
┌───────────────────────┐
│ ESP32-S3 CAMERA │
│ │
│ Camera │
│ QR Recognition │
│ Face Detection │
│ Face Recognition │
│ Target Position │
└───────────┬───────────┘
│
Target data / telemetry
│
┌────────────────────▼────────────────────┐
│ ROBOT CONTROLLER │
│ │
│ Tracking algorithm │
│ Motor control │
│ Ultrasonic/ToF obstacle detection │
│ Safety state machine │
└────────────────────┬────────────────────┘
│
Motor commands
│
┌─────────────────▼─────────────────┐
│ MOTOR DRIVER │
│ TB6612FNG │
└──────────────┬────────────────────┘
│
┌────────▼────────┐
│ LEFT / RIGHT │
│ MOTORS │
└─────────────────┘
Wi-Fi / Internet
│
▼
┌───────────────────────────┐
│ n8n │
│ │
│ Webhook │
│ AI Agent │
│ Event processing │
│ Rules │
│ Automation │
└─────┬─────────┬───────────┘
│ │
┌─────────┘ └──────────────┐
▼ ▼
┌─────────────────────┐ ┌──────────────────┐
│ Google Sheets │ │ ThingSpeak │
│ │ │ │
│ Event history │ │ Telemetry │
│ Identity │ │ Charts │
│ Robot status │ │ Cloud dashboard │
└─────────────────────┘ └──────────────────┘
│
▼
┌───────────────┐
│ Telegram │
│ │
│ Text alerts │
│ Voice alerts │
│ Commands │
└───────────────┘
ThingSpeak is particularly suitable for the telemetry portion because it accepts data from Internet-connected devices including ESP32 and provides online charts and API access.
5. Recommended hardware
Core electronics
| Component | Quantity | Purpose |
|---|---|---|
| ESP32-S3 camera board | 1 | AI + camera |
| ESP32 DevKit / second ESP32 | 1 optional | Motor/sensor controller |
| Camera | 1 | Vision |
| TB6612FNG motor driver | 1 | Motor control |
| DC geared motors | 2 | Robot movement |
| Wheels | 2 | Drive |
| Caster wheel | 1 | Balance |
| Ultrasonic HC-SR04 / ToF sensor | 2–3 | Obstacle detection |
| Li-ion/LiPo battery | 1 | Power |
| 5 V buck converter | 1 | ESP32 supply |
| Buzzer | 1 | Local alert |
| RGB LED | 1 | Status |
| Push button | 1 | Emergency/stop |
| QR code | 1+ | Identity/tag |
| Chassis | 1 | Robot platform |
Recommended camera platform
I recommend:
ESP32-S3-EYE or another ESP32-S3 camera board
rather than a conventional ESP32-CAM for the AI portion.
Espressif's current ESP-DL documentation recommends ESP32-S3-class hardware for AI workloads and provides face detection/recognition models.
6. Why use two controllers?
A robust implementation can use:
ESP32-S3
│
┌─────────┴─────────┐
│ │
Camera AI vision
│ │
└─────────┬─────────┘
│
UART/I2C
│
▼
ESP32
│
┌────────────┼─────────────┐
▼ ▼ ▼
Motors Sensors Buzzer
This architecture is preferable because camera/AI processing can consume significant CPU/RAM.
Controller 1 — ESP32-S3
Responsible for:
- Camera
- Face detection
- Face recognition
- QR recognition
- Target coordinates
Controller 2 — ESP32
Responsible for:
- Motor control
- Ultrasonic sensors
- Battery monitoring
- Buzzer
- Emergency stop
- Low-level safety
You can also implement everything on one sufficiently capable ESP32-S3 board, but the two-controller architecture is easier to debug.
7. Robot mechanical design
Use a differential-drive robot.
FRONT
↑
┌─────────────────┐
│ ESP32-CAM │
│ CAMERA │
└─────────────────┘
│
┌─────┴─────┐
│ Chassis │
│ │
LEFT │ │ RIGHT
MOTOR │ │ MOTOR
O │ │ O
│
│
(●)
CASTER
↓
BACK
The camera should be mounted approximately 20–30 cm above the chassis and angled slightly downward.
8. Motor control principle
The camera gives us:
Image width = W
Image height = H
Target center = (X,Y)
Calculate:
errorX = X - W/2
For example:
Camera image
0 W
┌─────────────────────────┐
│ │
│ TARGET │
│ ● │
│ │
└─────────────────────────┘
↑
center
If:
errorX ≈ 0
the person is centered.
If:
errorX < 0
the person is to the left.
If:
errorX > 0
the person is to the right.
9. Tracking algorithm
Use a simple proportional controller initially.
error = targetX - imageCenterX
turn = Kp × error
leftMotor = baseSpeed + turn
rightMotor = baseSpeed - turn
Example:
turn = Kp * error;
leftSpeed = baseSpeed + turn;
rightSpeed = baseSpeed - turn;
Clamp both values:
leftSpeed = constrain(leftSpeed, -255, 255);
rightSpeed = constrain(rightSpeed, -255, 255);
10. Distance control
The robot also needs to determine whether it is too close or too far from the target.
You can use:
- Ultrasonic sensor
- ToF sensor
- Bounding-box size
For a first implementation, use a ToF or ultrasonic sensor.
Example:
Distance > 200 cm
↓
MOVE FORWARD
100–200 cm
↓
FOLLOW
50–100 cm
↓
SLOW
<50 cm
↓
STOP
11. Robot state machine
The robot should not simply drive whenever a face appears.
Use states:
┌──────────────┐
│ IDLE │
└──────┬───────┘
│
QR found
│
▼
┌──────────────┐
│ IDENTIFYING │
└──────┬───────┘
│
Face matched
│
▼
┌──────────────┐
│ FOLLOW │
└──────┬───────┘
│
┌────────┴────────┐
│ │
target lost obstacle
│ │
▼ ▼
┌──────────┐ ┌──────────┐
│ SEARCHING│ │ STOP │
└────┬─────┘ └──────────┘
│
target found
│
▼
FOLLOW
12. QR-code identification
The QR code can contain something like:
ROBOT_TARGET_001
or:
{
"type": "robot_target",
"id": "PERSON_001"
}
However, do not treat possession of a QR code alone as sufficient identity authentication.
A better design is:
QR recognized
│
▼
PERSON_001
│
▼
Face recognition
│
▼
Face = PERSON_001
│
▼
AUTHORIZED
This provides two-factor-style identity:
QR + Face
13. Face recognition architecture
The vision pipeline is:
Camera
│
▼
Image capture
│
▼
Face detection
│
▼
Face crop
│
▼
Feature extraction
│
▼
Face embedding
│
▼
Compare with enrolled database
│
├── Match
│ ↓
│ TARGET
│
└── No match
↓
UNKNOWN
ESP-DL contains vision/recognition components including feature extraction and recognition databases.
14. Face enrollment
Before autonomous operation:
START
│
▼
Show face to camera
│
▼
Detect face
│
▼
Capture multiple samples
│
▼
Generate face features
│
▼
Save:
PERSON_001
│
▼
Enrollment complete
Use several images under slightly different:
- Angles
- Lighting
- Distances
- Expressions
15. Main ESP32 software architecture
setup()
├── Initialize Serial
├── Initialize Wi-Fi
├── Initialize camera
├── Initialize face model
├── Initialize QR decoder
├── Initialize motor controller
├── Initialize sensors
└── Initialize cloud connection
loop()
├── Read camera
├── Detect QR
├── Detect face
├── Identify target
├── Calculate target position
├── Read obstacle sensor
├── Calculate motor command
├── Drive motors
├── Create telemetry packet
└── Send telemetry
16. ESP32 data structure
Use a JSON message.
Example:
{
"robot_id": "ROBOT_001",
"timestamp": 1720000000,
"state": "FOLLOWING",
"target": "PERSON_001",
"face_match": true,
"qr_match": true,
"confidence": 0.91,
"x": 187,
"y": 142,
"distance": 128,
"left_motor": 145,
"right_motor": 160,
"obstacle": false,
"battery": 78
}
This becomes the common data format for the entire IoT system.
17. Example ESP32 Wi-Fi + telemetry code
The following is the controller/cloud communication layer. The actual camera/ESP-DL functions depend on the exact ESP32-S3 board and ESP-IDF/ESP-DL project.
#include <WiFi.h>
#include <HTTPClient.h>
#include <ArduinoJson.h>
const char* WIFI_SSID = "YOUR_WIFI";
const char* WIFI_PASSWORD = "YOUR_PASSWORD";
const char* N8N_WEBHOOK =
"https://YOUR-N8N-DOMAIN/webhook/robot-event";
const char* ROBOT_ID = "ROBOT_001";
void connectWiFi()
{
WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
Serial.print("Connecting");
while (WiFi.status() != WL_CONNECTED)
{
delay(500);
Serial.print(".");
}
Serial.println();
Serial.println("WiFi connected");
Serial.println(WiFi.localIP());
}
void sendTelemetry(
const char* state,
const char* target,
bool faceMatch,
bool qrMatch,
float confidence,
int targetX,
int targetY,
float distance,
int leftMotor,
int rightMotor,
bool obstacle,
int battery)
{
if (WiFi.status() != WL_CONNECTED)
return;
HTTPClient http;
http.begin(N8N_WEBHOOK);
http.addHeader("Content-Type", "application/json");
JsonDocument doc;
doc["robot_id"] = ROBOT_ID;
doc["state"] = state;
doc["target"] = target;
doc["face_match"] = faceMatch;
doc["qr_match"] = qrMatch;
doc["confidence"] = confidence;
doc["x"] = targetX;
doc["y"] = targetY;
doc["distance"] = distance;
doc["left_motor"] = leftMotor;
doc["right_motor"] = rightMotor;
doc["obstacle"] = obstacle;
doc["battery"] = battery;
String payload;
serializeJson(doc, payload);
int httpCode = http.POST(payload);
Serial.print("HTTP: ");
Serial.println(httpCode);
http.end();
}
18. Motor-control code
For a TB6612FNG-style driver:
#define AIN1 25
#define AIN2 26
#define PWMA 27
#define BIN1 32
#define BIN2 33
#define PWMB 14
void setMotor(
int in1,
int in2,
int pwm,
int speed)
{
speed = constrain(speed, -255, 255);
if (speed > 0)
{
digitalWrite(in1, HIGH);
digitalWrite(in2, LOW);
analogWrite(pwm, speed);
}
else if (speed < 0)
{
digitalWrite(in1, LOW);
digitalWrite(in2, HIGH);
analogWrite(pwm, -speed);
}
else
{
digitalWrite(in1, LOW);
digitalWrite(in2, LOW);
analogWrite(pwm, 0);
}
}
void driveRobot(int leftSpeed, int rightSpeed)
{
setMotor(AIN1, AIN2, PWMA, leftSpeed);
setMotor(BIN1, BIN2, PWMB, rightSpeed);
}
void stopRobot()
{
driveRobot(0, 0);
}
For newer ESP32 Arduino cores, PWM is commonly handled through the LEDC APIs rather than relying on the older analogWrite() behavior, so adapt this portion to your installed core.
19. Tracking controller code
float Kp = 0.7;
int baseSpeed = 130;
void followTarget(
int targetX,
int imageWidth,
float distance)
{
int center = imageWidth / 2;
int error = targetX - center;
float turn = Kp * error;
int leftSpeed =
baseSpeed + (int)turn;
int rightSpeed =
baseSpeed - (int)turn;
// Distance control
if (distance < 60)
{
leftSpeed = 0;
rightSpeed = 0;
}
else if (distance > 200)
{
leftSpeed += 30;
rightSpeed += 30;
}
leftSpeed =
constrain(leftSpeed, -255, 255);
rightSpeed =
constrain(rightSpeed, -255, 255);
driveRobot(leftSpeed, rightSpeed);
}
20. Lost-target logic
Never immediately turn randomly when the face disappears.
Use a timeout.
unsigned long lastTargetTime = 0;
const unsigned long LOST_TIMEOUT = 5000;
bool targetRecentlySeen()
{
return
millis() - lastTargetTime
< LOST_TIMEOUT;
}
Then:
if (targetDetected)
{
lastTargetTime = millis();
followTarget(
targetX,
imageWidth,
distance);
}
else
{
stopRobot();
if (!targetRecentlySeen())
{
robotState = "SEARCHING";
}
}
21. Obstacle safety
The obstacle sensor must have priority over AI commands.
AI says:
MOVE FORWARD
↓
Safety controller
↓
Obstacle?
┌────┴────┐
YES NO
│ │
STOP MOVE
Never let an AI agent directly override this safety layer.
22. Recommended safety hierarchy
LEVEL 1
Emergency stop
↓
LEVEL 2
Obstacle avoidance
↓
LEVEL 3
Target identity
↓
LEVEL 4
Target tracking
↓
LEVEL 5
AI agent
↓
LEVEL 6
Cloud automation
The AI agent should be an orchestrator, not the primary real-time motor controller.
23. n8n architecture
n8n is well suited to connecting the robot to external services and AI workflows; its documentation describes it as a workflow automation platform with AI capabilities and integrations.
Create several workflows instead of one enormous workflow.
Workflow A — Robot telemetry
ESP32
│
▼
Webhook
│
▼
Validate JSON
│
▼
Switch by state
│
├─────────────┐
▼ ▼
Google Sheets ThingSpeak
│
▼
Database/log
Workflow B — Critical alerts
ESP32
│
▼
Webhook
│
▼
IF critical event?
│
├── NO ──> END
│
YES
│
▼
AI Agent
│
▼
Generate alert
│
▼
Text-to-Speech
│
▼
Telegram
Workflow C — Telegram commands
Telegram
│
▼
Telegram Trigger
│
▼
AI Agent
│
├── STATUS
├── STOP
├── START
├── REPORT
└── FIND ROBOT
│
▼
HTTP API
│
▼
ESP32
The n8n Telegram integration supports Telegram automation and message operations.
24. n8n Workflow 1 — Telemetry
Build:
[Webhook]
↓
[Set / Edit Fields]
↓
[Code - Validate]
↓
[Google Sheets]
↓
[HTTP Request]
↓
[ThingSpeak]
Webhook
Method:
POST
Path:
robot-event
Your ESP32 sends:
POST /webhook/robot-event
Content-Type: application/json
with the JSON payload shown earlier.
25. Data validation
Use an n8n Code node:
const data = $json;
const required = [
"robot_id",
"state",
"battery"
];
for (const field of required) {
if (!(field in data)) {
throw new Error(
`Missing field: ${field}`
);
}
}
if (data.battery < 0 || data.battery > 100) {
throw new Error("Invalid battery percentage");
}
return [
{
json: {
...data,
server_timestamp: new Date().toISOString()
}
}
];
n8n supports Code nodes for transforming and processing workflow data.
26. Google Sheets database
Create a spreadsheet:
Robot_Logs
Columns:
Timestamp
Robot ID
State
Target
Face Match
QR Match
Confidence
Target X
Target Y
Distance
Obstacle
Battery
Left Motor
Right Motor
Alert
Example:
| Timestamp | Robot | State | Target | Confidence | Distance | Battery |
|---|---|---|---|---|---|---|
| 22:10:01 | ROBOT_001 | FOLLOWING | PERSON_001 | 0.94 | 135 | 87 |
| 22:10:06 | ROBOT_001 | FOLLOWING | PERSON_001 | 0.92 | 142 | 86 |
| 22:10:11 | ROBOT_001 | SEARCHING | UNKNOWN | 0 | 0 | 86 |
This gives you a very convenient project log for analysis and demonstration.
27. ThingSpeak configuration
Create a ThingSpeak channel.
Suggested fields:
Field 1 = Target X
Field 2 = Target Y
Field 3 = Distance
Field 4 = Face Confidence
Field 5 = Battery
Field 6 = Robot State
Field 7 = Left Motor
Field 8 = Right Motor
ThingSpeak supports ESP32 data submission and automatically provides visualization/charts for channel data.
Your architecture becomes:
ESP32
│
│ JSON
▼
n8n
│
│ HTTP API
▼
ThingSpeak
│
├── Target graph
├── Distance graph
├── Battery graph
└── Confidence graph
28. ThingSpeak HTTP request
Conceptually:
https://api.thingspeak.com/update
with:
api_key = YOUR_WRITE_API_KEY
field1 = targetX
field2 = targetY
field3 = distance
field4 = confidence
field5 = battery
Keep the API key in n8n credentials/environment variables rather than hard-coding it into public source code.
29. Telegram alert architecture
Example event:
{
"state": "SEARCHING",
"target": "PERSON_001",
"duration": 18
}
n8n can turn it into:
🚨 ROBOT ALERT
Robot: ROBOT_001
Target: PERSON_001
Status: Target lost
Duration: 18 seconds
Robot action: STOPPED / SEARCHING
The Telegram node provides built-in Telegram automation functionality.
30. Voice alert architecture
For voice notifications:
Robot Event
│
▼
n8n
│
▼
AI Agent
│
▼
Generate message
│
▼
Text-to-Speech service
│
▼
Audio file
│
▼
Telegram
Example:
AI-generated text:
"Warning. The robot has lost the registered
person for more than fifteen seconds."
Then convert that text into an audio file and send the audio through Telegram.
31. Telegram commands
The operator can send:
/status
Robot responds:
🤖 ROBOT STATUS
State: FOLLOWING
Target: PERSON_001
Face confidence: 94%
Distance: 1.35 m
Battery: 87%
Obstacle: NO
Wi-Fi: CONNECTED
Other commands:
/start
/stop
/status
/location
/report
/follow
/search
/emergency
32. AI Agent design
The AI agent should not directly calculate PWM values.
Instead, give it high-level tools.
AI AGENT
│
┌──────────┼──────────┐
▼ ▼ ▼
STATUS REPORT ALERT
│ │ │
▼ ▼ ▼
ESP32 Sheets Telegram
The agent can answer:
"Why did the robot stop?"
Based on telemetry:
Battery = 42%
Obstacle = true
Target = PERSON_001
State = STOPPED
AI response:
"The robot stopped because an obstacle was detected in front of it. Target tracking is still active."
33. AI Agent system prompt
A suitable conceptual prompt is:
You are the Robot Operations Agent.
Your job is to monitor an ESP32 human-tracking robot.
You may:
- inspect telemetry
- summarize robot status
- identify abnormal events
- generate operator notifications
- request reports
- interpret sensor data
You must NOT:
- bypass emergency stop
- override obstacle safety
- directly command unsafe motor speeds
- infer identity when face recognition has failed
- claim that an unknown person is authorized
When an obstacle is detected, recommend STOP.
When the target is lost, recommend STOP or SEARCH according
to the robot's configured safety policy.
When battery is low, notify the operator.
34. AI agent decision example
Input:
{
"state": "SEARCHING",
"target": "PERSON_001",
"face_match": false,
"qr_match": false,
"battery": 31,
"obstacle": false,
"lost_seconds": 17
}
Agent:
Classification:
TARGET_LOST
Severity:
MEDIUM
Recommended action:
Keep robot stopped and notify operator.
Reason:
Target has not been verified for 17 seconds.
Battery is also at 31%.
Then n8n sends Telegram.
35. Complete n8n AI workflow
┌──────────────┐
│ ESP32 EVENT │
└──────┬───────┘
│
▼
┌──────────────┐
│ WEBHOOK │
└──────┬───────┘
│
▼
┌──────────────┐
│ VALIDATION │
└──────┬───────┘
│
▼
┌──────────────┐
│ AI AGENT │
└──────┬───────┘
│
┌─────────┼──────────┐
│ │ │
▼ ▼ ▼
NORMAL WARNING CRITICAL
│ │ │
▼ ▼ ▼
Google Sheets Telegram
Sheets │ + Voice
│ ▼
│ ThingSpeak
▼
Dashboard
36. Web dashboard
You can create a simple web page hosted separately or through your preferred web server.
Dashboard:
┌──────────────────────────────────────────────┐
│ AI ROBOT DASHBOARD │
├──────────────────────────────────────────────┤
│ │
│ Robot: ONLINE Battery: 86% │
│ │
│ State: FOLLOWING Target: PERSON_001 │
│ │
├──────────────────────────────────────────────┤
│ │
│ CAMERA / ROBOT VIEW │
│ │
│ ┌─────┐ │
│ │ HUMAN│ │
│ │ BOX │ │
│ └─────┘ │
│ │
├──────────────────────────────────────────────┤
│ Face Confidence: 94% │
│ QR Status: VERIFIED │
│ Distance: 1.34 m │
│ Obstacle: CLEAR │
│ │
├──────────────────────────────────────────────┤
│ [ START ] [ STOP ] [ SEARCH ] [ EMERGENCY ] │
└──────────────────────────────────────────────┘
37. Example HTML dashboard
A simple prototype:
<!DOCTYPE html>
<html>
<head>
<title>AI Robot Dashboard</title>
<style>
body {
font-family: Arial;
background: #101820;
color: white;
margin: 0;
padding: 30px;
}
.dashboard {
max-width: 1000px;
margin: auto;
}
.cards {
display: grid;
grid-template-columns:
repeat(auto-fit, minmax(180px, 1fr));
gap: 15px;
}
.card {
background: #1d2a35;
padding: 20px;
border-radius: 12px;
}
.value {
font-size: 28px;
color: #00e5ff;
}
button {
padding: 14px 25px;
margin: 8px;
border: none;
border-radius: 8px;
cursor: pointer;
}
.danger {
background: #ff3b30;
color: white;
}
</style>
</head>
<body>
<div class="dashboard">
<h1>🤖 AI Human Tracking Robot</h1>
<div class="cards">
<div class="card">
<h3>Robot State</h3>
<div class="value" id="state">
FOLLOWING
</div>
</div>
<div class="card">
<h3>Target</h3>
<div class="value" id="target">
PERSON_001
</div>
</div>
<div class="card">
<h3>Confidence</h3>
<div class="value" id="confidence">
94%
</div>
</div>
<div class="card">
<h3>Battery</h3>
<div class="value" id="battery">
86%
</div>
</div>
</div>
<h2>Robot Control</h2>
<button onclick="sendCommand('START')">
START
</button>
<button onclick="sendCommand('STOP')">
STOP
</button>
<button onclick="sendCommand('SEARCH')">
SEARCH
</button>
<button class="danger"
onclick="sendCommand('EMERGENCY_STOP')">
EMERGENCY STOP
</button>
</div>
<script>
const API =
"https://YOUR-N8N-DOMAIN/webhook/robot-command";
async function sendCommand(command) {
await fetch(API, {
method: "POST",
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify({
command: command,
source: "web-dashboard"
})
});
}
</script>
</body>
</html>
38. Command path
The dashboard should not directly control motors.
Instead:
WEB DASHBOARD
│
▼
n8n
│
▼
Command validation
│
▼
Safety validation
│
▼
ESP32
│
▼
Motor controller
This makes the architecture considerably safer.
39. Telegram command path
TELEGRAM
│
▼
Telegram Trigger
│
▼
n8n
│
▼
AI Agent
│
┌──────┴──────┐
│ │
STATUS COMMAND
│ │
▼ ▼
ESP32 API Validation
│
▼
ESP32
40. Example command JSON
n8n sends:
{
"command": "STOP",
"request_id": "REQ_123456",
"source": "telegram",
"operator": "authorized"
}
ESP32 responds:
{
"robot_id": "ROBOT_001",
"command": "STOP",
"result": "OK",
"state": "STOPPED"
}
41. Command whitelist
ESP32 should accept only:
START
STOP
SEARCH
FOLLOW
PAUSE
RESUME
STATUS
EMERGENCY_STOP
Never allow arbitrary text to become a motor command.
Bad architecture:
AI:
"Move left at 83 PWM"
↓
ESP32
Better:
AI:
"FOLLOW"
↓
Safety controller
↓
Tracking algorithm
↓
PWM
42. End-to-end data flow
Here is the complete project flow:
HUMAN
│
┌─────────┴─────────┐
│ │
QR CODE FACE
│ │
└─────────┬─────────┘
▼
ESP32-S3 CAMERA
│
┌────────┴────────┐
│ │
QR OK FACE OK
│ │
└────────┬────────┘
▼
TARGET VERIFIED
│
▼
POSITION (X,Y)
│
▼
TRACKING PID
│
▼
MOTOR CONTROL
│
▼
ROBOT MOVES
│
▼
TELEMETRY
│
Wi-Fi
│
▼
n8n
│
┌────────────────┼────────────────┐
│ │ │
▼ ▼ ▼
AI Agent Google Sheets ThingSpeak
│
▼
Alert decision
│
▼
Telegram
│
┌───┴────┐
▼ ▼
Text Voice
43. Complete event lifecycle
Example:
Step 1
Person enters camera view.
Face detected
Step 2
QR code detected.
QR = PERSON_001
Step 3
Face recognition returns:
PERSON_001
confidence = 0.94
Step 4
Robot changes:
IDLE → FOLLOWING
Step 5
Robot tracks:
X = 190
center = 160
error = +30
Robot turns slightly right.
Step 6
Robot sends:
{
"state": "FOLLOWING",
"target": "PERSON_001",
"confidence": 0.94
}
Step 7
n8n receives it.
Step 8
n8n logs it to Google Sheets.
Step 9
n8n updates ThingSpeak.
Step 10
No Telegram alert is generated because everything is normal.
44. Abnormal event example
Person leaves camera view.
FACE = NOT FOUND
After 5 seconds:
FOLLOWING → SEARCHING
After 15 seconds:
SEARCHING → STOPPED
n8n receives:
{
"state": "STOPPED",
"reason": "TARGET_LOST",
"lost_seconds": 15
}
AI Agent generates:
Target PERSON_001 has not been verified
for 15 seconds. Robot has stopped for safety.
Then:
n8n
│
┌───────┴────────┐
▼ ▼
Google Sheets Telegram
│
┌────┴─────┐
▼ ▼
Text Voice
45. Low-battery workflow
Battery
│
▼
< 30%?
┌─┴─┐
NO YES
│ │
END ▼
AI Agent
│
▼
Telegram
│
▼
"Robot battery is low"
At:
20%
change robot state:
LOW_BATTERY
At:
10%
stop the robot.
46. Obstacle workflow
Camera:
TARGET FOUND
+
Distance sensor:
OBSTACLE = TRUE
│
▼
SAFETY CONTROLLER
│
▼
STOP
│
▼
n8n EVENT
│
▼
Telegram Alert
The AI agent should explain the event rather than being responsible for the immediate stop.
47. Suggested folder structure
AI-Human-Tracking-Robot/
│
├── firmware/
│ ├── esp32_motor/
│ │ ├── main.cpp
│ │ ├── motor.cpp
│ │ ├── motor.h
│ │ ├── sensors.cpp
│ │ └── sensors.h
│ │
│ └── esp32s3_vision/
│ ├── main.cpp
│ ├── camera.cpp
│ ├── camera.h
│ ├── face.cpp
│ ├── face.h
│ ├── qr.cpp
│ └── qr.h
│
├── n8n/
│ ├── telemetry-workflow.json
│ ├── alert-workflow.json
│ ├── telegram-workflow.json
│ └── ai-agent-workflow.json
│
├── dashboard/
│ ├── index.html
│ ├── style.css
│ └── app.js
│
├── documentation/
│ ├── architecture.md
│ ├── wiring.md
│ ├── installation.md
│ └── testing.md
│
└── README.md
48. Software stack
| Layer | Technology |
|---|---|
| AI vision | ESP-DL |
| Face recognition | ESP-DL |
| QR | QR decoder library |
| Embedded firmware | ESP-IDF / Arduino where appropriate |
| Robot control | C/C++ |
| IoT | Wi-Fi |
| Automation | n8n |
| AI Agent | n8n AI capabilities |
| Notifications | Telegram |
| Voice | TTS + Telegram |
| Cloud logging | Google Sheets |
| Telemetry | ThingSpeak |
| Dashboard | HTML/CSS/JavaScript |
| API | HTTP/JSON |
| Motor driver | TB6612FNG |
Espressif's current ESP-DL documentation uses ESP-IDF and supports model deployment through its .espdl model format.
49. Development stages
Do not build everything simultaneously.
Build it in these stages.
Stage 1 — Robot chassis
Test:
ESP32 → Motor Driver → Motors
Verify:
FORWARD
BACKWARD
LEFT
RIGHT
STOP
Stage 2 — Sensors
Add:
Ultrasonic/ToF
Battery measurement
Emergency button
Verify obstacle stopping.
Stage 3 — Camera
Test:
Camera → image
Verify stable frame acquisition.
Stage 4 — Face detection
Test:
Camera
↓
Face detection
↓
Bounding box
Stage 5 — Face recognition
Add:
Enrollment
↓
Recognition
↓
PERSON_001
Stage 6 — QR
Add:
QR
↓
PERSON_001
Stage 7 — Combine QR + Face
Require:
QR == PERSON_001
AND
Face == PERSON_001
before following.
Stage 8 — Tracking
Add:
X position
then:
X position + distance
Stage 9 — Wi-Fi
Send telemetry.
Stage 10 — n8n
Receive:
ESP32 → Webhook
Stage 11 — Google Sheets
Log every event.
Stage 12 — ThingSpeak
Create real-time charts.
Stage 13 — Telegram
Add alerts.
Stage 14 — Voice
Add TTS.
Stage 15 — AI Agent
Add intelligent event interpretation.
Stage 16 — Dashboard
Connect the web interface.
50. Testing plan
Create a formal test table.
| Test | Input | Expected output |
|---|---|---|
| T01 | Power on | Robot initializes |
| T02 | No target | Robot remains stopped |
| T03 | Valid QR | QR accepted |
| T04 | Invalid QR | QR rejected |
| T05 | Registered face | Face accepted |
| T06 | Unknown face | Face rejected |
| T07 | QR + face match | Target authorized |
| T08 | Target left | Robot stops/searches |
| T09 | Obstacle | Robot immediately stops |
| T10 | Low battery | Alert generated |
| T11 | Wi-Fi lost | Safe local operation |
| T12 | n8n unavailable | Robot continues locally |
| T13 | Telegram failure | Local robot remains safe |
| T14 | Emergency stop | Motors immediately stop |
| T15 | Dashboard STOP | Robot stops |
| T16 | Telegram STOP | Robot stops |
51. Important offline behavior
This is extremely important.
The robot must not depend on n8n for basic safety.
If Wi-Fi disappears:
Wi-Fi LOST
│
▼
Robot continues local vision
│
├── obstacle safety
├── face recognition
└── motor control
Cloud functionality becomes unavailable:
Google Sheets
ThingSpeak
Telegram
AI Agent
Dashboard
but:
EMERGENCY STOP
OBSTACLE STOP
LOCAL FOLLOWING
must continue to work.
52. Failure architecture
CLOUD FAILURE
│
▼
n8n OFFLINE
│
▼
┌──────────┴──────────┐
│ │
Cloud unavailable Robot local
│
▼
Safety continues
This is an important point to mention in your project viva.
53. Security architecture
Do not expose an unauthenticated motor-control endpoint.
Use:
HTTPS
+
API key/token
+
Command whitelist
+
Request validation
+
Rate limiting
Example:
Authorization: Bearer YOUR_DEVICE_TOKEN
Also:
- Never publish Telegram bot tokens.
- Never publish ThingSpeak write keys.
- Never publish Google credentials.
- Never commit n8n credentials to GitHub.
- Use HTTPS for external webhooks.
- Authenticate dashboard commands.
- Separate read and control permissions.
n8n provides security-audit functionality that can identify issues such as unprotected webhooks and other configuration risks.
54. Face-recognition privacy
Because this system processes biometric information, the project should have an explicit privacy design.
For a prototype:
- Enroll only consenting participants.
- Store embeddings rather than unnecessary raw photographs where practical.
- Protect enrollment data.
- Do not publicly expose face images.
- Provide a method to delete enrolled identities.
- Do not use recognition as a substitute for safety-critical authentication.
For a university demonstration, document that the face-recognition database is for the authorized prototype participants only.
55. Recommended final architecture
I would implement the final version as:
┌─────────────────┐
│ PERSON │
│ QR + FACE │
└────────┬────────┘
│
▼
┌─────────────────────┐
│ ESP32-S3 │
│ │
│ Camera │
│ QR │
│ Face Detection │
│ Face Recognition │
└─────────┬───────────┘
│
UART / Wi-Fi
│
▼
┌─────────────────────┐
│ ESP32 │
│ │
│ Tracking │
│ Safety │
│ ToF/Ultrasonic │
│ Motor Control │
└─────────┬───────────┘
│
▼
TB6612FNG
/ \
/ \
LEFT MOTOR RIGHT MOTOR
Wi-Fi
│
▼
┌──────────────────┐
│ n8n │
│ │
│ Webhooks │
│ AI Agent │
│ Rules │
│ APIs │
└───┬────┬────┬────┘
│ │ │
┌──────────┘ │ └───────────┐
▼ ▼ ▼
Google Sheets ThingSpeak Telegram
│ │ │
│ │ ┌─────┴─────┐
│ │ ▼ ▼
│ │ TEXT VOICE
│ │
└───────────────┴───────────────┐
▼
WEB DASHBOARD
56. What makes this an "Agentic IoT" project?
A conventional IoT project might be:
ESP32 → Cloud → Dashboard
Your project is more advanced:
ESP32
↓
Telemetry
↓
n8n
↓
AI Agent
↓
Reason about event
↓
Select appropriate workflow
↓
Notify / log / request action
↓
Human
The important distinction is that the AI layer is interpreting and orchestrating events, while deterministic embedded code handles safety and real-time control.
57. Project novelty
You can describe the innovation as:
A multi-layer agentic IoT architecture that combines edge computer vision, QR-assisted identity verification, face recognition, autonomous human tracking, cloud workflow automation, AI-assisted event reasoning, real-time telemetry, voice notification and remote operator interaction in a single ESP32-based mobile robot.
That is a much stronger project description than simply calling it an "ESP32 human-following robot."
58. Expected final demonstration
Your final demonstration can follow this sequence:
1. Power robot ON
↓
2. Robot connects to Wi-Fi
↓
3. Camera starts
↓
4. Person displays QR
↓
5. QR recognized
↓
6. Person faces camera
↓
7. Face recognized
↓
8. Robot displays:
PERSON_001 VERIFIED
↓
9. Robot follows person
↓
10. ThingSpeak receives telemetry
↓
11. Google Sheets records events
↓
12. Person walks behind obstacle
↓
13. Robot detects target lost
↓
14. Robot stops
↓
15. n8n receives event
↓
16. AI Agent analyzes event
↓
17. Telegram text alert
↓
18. Telegram voice alert
↓
19. Person returns
↓
20. Face + QR verified
↓
21. Robot resumes following
59. Documentation chapters for your final report
Your report can be organized as:
Chapter 1 — Introduction
- Background
- Problem statement
- Motivation
- Objectives
- Scope
- Applications
Chapter 2 — Literature/Technology Review
- ESP32
- ESP32-S3
- Computer vision
- Face recognition
- QR recognition
- IoT
- n8n
- AI agents
- Telegram
- ThingSpeak
- Google Sheets
Chapter 3 — System Design
- Overall architecture
- Hardware architecture
- Software architecture
- Data flow
- State machine
- Communication protocols
Chapter 4 — Hardware Implementation
- ESP32-S3
- ESP32
- Camera
- Motor driver
- Motors
- Sensors
- Power supply
- Wiring
Chapter 5 — Software Implementation
- ESP32 firmware
- Camera
- QR recognition
- Face recognition
- Tracking
- Motor control
- Wi-Fi
Chapter 6 — n8n Implementation
- Webhook
- Data validation
- AI Agent
- Google Sheets
- ThingSpeak
- Telegram
- Voice alerts
Chapter 7 — Web Dashboard
- Dashboard UI
- API
- Robot status
- Commands
- Authentication
Chapter 8 — Testing
- Unit tests
- Integration tests
- Tracking tests
- Recognition tests
- Cloud tests
- Safety tests
Chapter 9 — Results
- Recognition accuracy
- Tracking performance
- Response time
- Cloud latency
- Battery performance
Chapter 10 — Conclusion
- Achievements
- Limitations
- Future improvements
60. Future improvements
Once the basic robot works, you can add:
- Person re-identification
- Multiple authorized users
- SLAM
- LiDAR
- GPS
- Indoor mapping
- Autonomous navigation
- Voice commands
- Speech recognition
- Camera streaming
- Mobile application
- Battery-health prediction
- Predictive maintenance
- Object detection
- Fall detection
- Gesture control
- Automatic charging station
- Multi-robot coordination
- MQTT
- Local LLM
- Edge AI model optimization
ESP-DL currently provides tutorials for deploying models such as MobileNetV2 and YOLO-family models, making future object-detection extensions feasible on supported Espressif hardware.
61. Most important design decision
For the first working prototype, don't try to make the AI agent control the robot's motors directly.
Use:
EDGE AI
↓
REAL-TIME CONTROL
↓
SAFETY
and separately:
TELEMETRY
↓
n8n
↓
AI AGENT
↓
AUTOMATION
↓
HUMAN / CLOUD SERVICES
This gives you both real-time robotics and agentic IoT, without making the robot dependent on cloud latency.
62. Official technical references
For the embedded AI portion, Espressif's current documentation is the best starting point: ESP-DL provides neural-network inference and vision models, including human face detection and recognition.
For automation, n8n provides the workflow, integration and AI functionality required for the cloud layer.
For telemetry, ThingSpeak supports ESP32 devices and REST/MQTT-based data collection and visualization.
63. Recommended implementation order
The best practical order is:
ESP32 + Motors
↓
Obstacle sensor
↓
ESP32-S3 Camera
↓
Face Detection
↓
Face Recognition
↓
QR Recognition
↓
QR + Face Verification
↓
Human Tracking
↓
Wi-Fi Telemetry
↓
n8n Webhook
↓
Google Sheets
↓
ThingSpeak
↓
Telegram Text
↓
Telegram Voice
↓
AI Agent
↓
Web Dashboard
↓
Complete Agentic IoT Robot
This staged approach prevents the project from becoming impossible to debug.
One important caveat: the exact camera/face/QR source code depends heavily on whether you use an ESP32-S3-EYE, ESP32-S3-CAM, AI-Thinker ESP32-CAM, or another board. The current ESP-DL setup is ESP-IDF-based, and Espressif explicitly notes that original ESP32 execution is significantly slower than ESP32-S3 for ESP-DL.
If your target is a fully buildable project rather than just the architecture, the next deliverable should therefore be a board-specific package containing the complete circuit schematic/pin map, ESP32-S3 ESP-DL face-recognition firmware, QR decoder, second-ESP32 motor firmware, exact n8n workflow node configuration, ThingSpeak fields, Google Sheets schema, Telegram bot setup, voice/TTS workflow, and dashboard files for one selected ESP32-S3 board.
No comments:
Post a Comment