AI Guest Greeting & Guiding Robot
ESP32 + Face Recognition + Voice Interaction + AI Agent + n8n + Telegram Voice Alerts + Google Sheets + ThingSpeak
Below is a complete reference architecture you can use for a final-year project, diploma project, prototype, or research demonstration. The design combines edge AI on an ESP32-S3 camera device with an n8n agentic IoT backend, Telegram notifications, Google Sheets logging, and ThingSpeak visualization.
For face recognition, I recommend an ESP32-S3-EYE or another ESP32-S3 camera board rather than a basic ESP32-CAM. Espressif specifically documents ESP32-S3-EYE for face recognition/detection, and ESP-WHO provides the face-detection/recognition framework.
1. Project title
AI-Powered Smart Guest Greeting and Guiding Robot Using ESP32-S3, Face Recognition, Voice Interaction, n8n Agentic Automation, Telegram, Google Sheets and ThingSpeak
Short title
Agentic AI Guest Robot – ESP32 + n8n + Telegram + ThingSpeak
2. Abstract
The proposed project is an intelligent autonomous guest reception and guidance robot capable of detecting and recognizing visitors, greeting known guests by name, interacting through voice, guiding visitors to predefined destinations, recording visitor events, and notifying an administrator remotely.
The robot uses an ESP32-S3-based vision system to capture images and perform face detection/recognition at the edge. The ESP32 communicates with an n8n automation server over Wi-Fi. n8n acts as the orchestration and agentic layer, receiving events from the robot, deciding what action should be taken, recording events in Google Sheets, updating a ThingSpeak IoT dashboard, and sending Telegram notifications.
The system can also accept voice commands. A voice message can be sent through Telegram, converted to text by a speech-to-text service, interpreted by an AI agent, and converted into a robot command.
The result is an Agentic IoT system where:
Sensors → ESP32 Edge AI → Internet → n8n → AI Agent → Decision → Robot/Cloud/Telegram
ThingSpeak supports REST and MQTT interfaces for writing and retrieving IoT channel data; for a first implementation, REST is particularly straightforward with ESP32.
3. Main objectives
The robot should perform these functions:
- Detect a person.
- Capture the person's face.
- Recognize a registered guest.
- Display the guest's name.
- Speak a personalized greeting.
- Ask the visitor where they want to go.
- Understand a voice command.
- Use an AI agent to interpret the request.
- Navigate toward the requested location.
- Detect obstacles.
- Stop when an obstacle is detected.
- Log the visitor.
- Update the IoT dashboard.
- Notify an administrator through Telegram.
- Optionally send a Telegram voice/audio alert.
- Accept remote commands.
- Maintain an event history in Google Sheets.
4. High-level architecture
┌─────────────────────┐
│ GUEST │
│ │
│ Face + Voice │
└──────────┬──────────┘
│
┌─────────────▼─────────────┐
│ ESP32-S3 ROBOT │
│ │
│ Camera │
│ Face Detection │
│ Face Recognition │
│ Microphone │
│ Speaker │
│ Ultrasonic Sensors │
│ Motors │
│ OLED/LCD │
└─────────────┬─────────────┘
│ Wi-Fi
│ HTTPS/HTTP
▼
┌───────────────────────────┐
│ n8n │
│ Automation Server │
│ │
│ Webhook │
│ AI Agent │
│ Logic / Switch │
│ HTTP Request │
│ Google Sheets │
│ Telegram │
└──────┬────────┬───────────┘
│ │
┌────────────┘ └─────────────┐
▼ ▼
┌──────────────────┐ ┌──────────────────┐
│ Google Sheets │ │ Telegram Bot │
│ Visitor Database │ │ Admin Alerts │
│ Event Log │ │ Voice/Text │
└──────────────────┘ └──────────────────┘
│
▼
┌──────────────────┐
│ ThingSpeak │
│ IoT Dashboard │
│ Charts / Status │
└──────────────────┘
5. Recommended hardware
Main controller
Option A — Recommended
ESP32-S3-EYE
It provides a camera, microphone and display-oriented hardware and is explicitly positioned by Espressif for face recognition applications.
Option B
ESP32-S3 development board + OV2640 camera + external microphone/display.
Hardware list
| Component | Quantity | Purpose |
|---|---|---|
| ESP32-S3-EYE / ESP32-S3 camera board | 1 | Main AI/vision |
| ESP32-S3/ESP32 controller | 1 | Robot control, if camera board is separated |
| OV2640 camera | 1 | Face image |
| Microphone | 1 | Voice input |
| MAX98357A I2S amplifier | 1 | Audio output |
| Speaker | 1 | Greeting |
| HC-SR04 / VL53L0X | 2–3 | Obstacle detection |
| L298N/TB6612FNG | 1 | Motor driver |
| DC geared motors | 2 | Robot movement |
| Robot chassis | 1 | Mechanical platform |
| Caster wheel | 1 | Balance |
| OLED 0.96" | 1 | Status |
| LEDs | 2–3 | Status indication |
| Servo | 1 | Optional head movement |
| Li-ion battery | 1 | Power |
| 5-V regulator | 1 | Logic/audio supply |
| Switch | 1 | Power |
| Buzzer | 1 | Alert |
6. Important design decision
Do not make the ESP32 do every AI task.
A better architecture is:
ESP32
Responsible for:
- camera
- face detection
- face recognition
- motors
- sensors
- microphone
- speaker
- Wi-Fi
- real-time safety
n8n
Responsible for:
- workflow orchestration
- AI agent
- visitor database lookup
- event logging
- notifications
- cloud integration
- command processing
This creates a true edge + cloud agentic IoT architecture.
7. Software architecture
ESP32 Firmware
│
├── Camera
├── Face Recognition
├── Sensors
├── Motor Controller
├── Voice Interface
└── Wi-Fi Client
│
▼
REST Webhook
│
▼
n8n
│
┌─────┴─────────┐
▼ ▼
AI Agent Automation
│ │
├──────┬────────┤
▼ ▼ ▼
Sheets Telegram ThingSpeak
│ │ │
└──────┴────────┘
│
▼
Robot command
8. Face recognition process
The face recognition sequence is:
Camera ON
│
▼
Capture frame
│
▼
Face detection
│
├── No face ──────► Continue scanning
│
▼
Face detected
│
▼
Extract face features
│
▼
Compare with registered faces
│
├── Match
│ │
│ ▼
│ Guest ID
│
└── No match
│
▼
Unknown guest
ESP-WHO is Espressif's framework for face detection and recognition on ESP32-class camera hardware.
9. Face database
Create a simple guest database.
Example
| ID | Name | Department | Destination | Permission |
|---|---|---|---|---|
| G001 | Arun | Manager | Office A | Yes |
| G002 | Priya | HR | HR Room | Yes |
| G003 | Ravi | Engineer | Lab | Yes |
| G004 | Unknown | — | Reception | No |
For a real deployment, don't store raw face images in Google Sheets. Store only a pseudonymous guest ID and operational metadata where possible.
10. Robot states
Use a state-machine architecture.
┌───────────────┐
│ IDLE │
└───────┬───────┘
│
Person
detected
│
▼
┌───────────────┐
│ FACE SCAN │
└───────┬───────┘
│
┌──────────┴──────────┐
│ │
Known Unknown
│ │
▼ ▼
GREETING RECEPTION
│
▼
ASK DESTINATION
│
▼
VOICE INPUT
│
▼
AI AGENT
│
┌──────┴───────┐
▼ ▼
Valid Invalid
│ │
▼ ▼
NAVIGATE ASK AGAIN
│
▼
ARRIVED
│
▼
GREETING
│
▼
LOG
│
▼
IDLE
11. Greeting conversation
Known visitor
Robot:
"Good evening, Arun. Welcome to our office."
Robot:
"How may I help you?"
Guest:
"Take me to the laboratory."
Robot:
"Sure. I will guide you to the laboratory."
Then navigation begins.
Unknown visitor
Robot:
"Welcome. I don't recognize you. Please report to the reception desk."
At the same time:
ESP32
│
▼
n8n
│
▼
Telegram
│
▼
ADMIN
Telegram message:
🚨 Unknown visitor detected at reception.
Time: 21:09
Action: Reception assistance required.
12. Voice interaction
There are two practical approaches.
Method 1 — Telegram voice
This is easiest for the prototype.
User
│
│ Telegram voice
▼
Telegram Bot
│
▼
n8n
│
▼
Speech-to-Text
│
▼
AI Agent
│
▼
Robot Command
n8n has a built-in Telegram integration supporting Telegram messages and file operations, among other operations.
Method 2 — Local microphone
For a more advanced version:
Microphone
│
▼
ESP32
│
▼
Audio stream
│
▼
Speech-to-text service
│
▼
n8n AI Agent
For an academic prototype, I recommend starting with Telegram voice, then adding local microphone processing as Phase 2.
13. AI Agent architecture
The AI agent receives something like:
{
"visitor_id": "G001",
"visitor_name": "Arun",
"voice_text": "Take me to the laboratory",
"robot_status": "IDLE",
"battery": 82,
"location": "Reception"
}
The agent returns structured data:
{
"intent": "NAVIGATE",
"destination": "LAB",
"response": "Sure Arun, I will take you to the laboratory.",
"priority": "NORMAL"
}
14. AI Agent prompt
Use a constrained prompt rather than allowing the agent to directly invent motor commands.
You are the AI controller for a reception robot.
Your job is to interpret visitor requests and return
a valid robot action.
Available destinations:
RECEPTION
OFFICE_A
OFFICE_B
HR
LAB
CONFERENCE_ROOM
EXIT
Available actions:
GREETING
NAVIGATE
STOP
RETURN_RECEPTION
ASK_CLARIFICATION
HELP
UNKNOWN
Rules:
1. Never invent a destination.
2. Never directly control individual motor pins.
3. If destination is unclear, return ASK_CLARIFICATION.
4. If an emergency/unsafe condition exists, return STOP.
5. Return JSON only.
Output:
{
"intent": "...",
"destination": "...",
"response": "...",
"priority": "NORMAL|HIGH|EMERGENCY"
}
This is much safer than giving an AI model unrestricted control over the motor driver.
15. n8n master workflow
┌───────────────┐
│ ESP32 Webhook │
└───────┬───────┘
│
▼
┌─────────────────┐
│ Validate JSON │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Identify Event │
└────────┬────────┘
│
┌─────────┴─────────┐
▼ ▼
FACE_RECOGNIZED UNKNOWN_FACE
│ │
▼ ▼
Google Sheets Telegram
│ │
▼ ▼
AI Agent Admin Alert
│
▼
Robot Command
│
▼
HTTP → ESP32
n8n provides built-in Webhook, Google Sheets, Telegram and AI-agent capabilities, making this architecture possible without writing a conventional backend server.
16. n8n workflow 1 — Face recognition
Nodes
Webhook
↓
Set / Edit Fields
↓
Switch
↓
Google Sheets
↓
AI Agent
↓
HTTP Request
↓
Telegram
Webhook input
{
"event": "face_recognized",
"device_id": "ROBOT001",
"guest_id": "G001",
"name": "Arun",
"confidence": 0.91,
"location": "Reception"
}
17. n8n workflow 2 — Unknown visitor
Webhook
│
▼
IF confidence < threshold
│
▼
Google Sheets
│
▼
Telegram
│
▼
Admin
Telegram message:
🚨 UNKNOWN VISITOR
Robot: ROBOT001
Location: Reception
Time: {{$now}}
Confidence: {{$json.confidence}}
Please check the reception area.
18. n8n workflow 3 — Voice command
Telegram Trigger
│
▼
Get File
│
▼
Speech-to-Text
│
▼
AI Agent
│
▼
Structured Output
│
▼
Switch
┌───┼────────────┐
▼ ▼ ▼
STOP NAVIGATE GREETING
│ │ │
└───┴─────┬──────┘
▼
HTTP Request
│
▼
ESP32
The Telegram node supports getting files and sending messages/audio, so it is suitable for this type of automation.
19. Google Sheets database
Create a spreadsheet:
Sheet: Visitors
Guest_ID
Name
Department
Destination
Face_Enabled
Phone
Permission
Sheet: Events
Timestamp
Robot_ID
Guest_ID
Name
Event
Location
Confidence
Destination
Action
Status
Example
2026-08-18 21:09 | ROBOT001 | G001 | Arun |
FACE_RECOGNIZED | Reception | 0.91 |
LAB | NAVIGATE | SUCCESS
20. ThingSpeak dashboard
Create a ThingSpeak channel.
Suggested fields:
Field 1 = Visitor Count
Field 2 = Unknown Visitors
Field 3 = Face Confidence
Field 4 = Battery
Field 5 = Robot State
Field 6 = Distance
Field 7 = Temperature
Field 8 = Navigation Status
ThingSpeak channels can be updated through REST GET or POST requests.
No comments:
Post a Comment