AI-Based Human Following Robot with Gesture Recognition

AI-Based Human Following Robot with Gesture Recognition + Agentic IoT System

AI-Based Human Following Robot with Gesture Recognition + Agentic IoT System This project combines: Human-following robot using AI and sensors Hand gesture recognition ESP32 IoT control system AI-based automation workflows Telegram voice alerts Google Sheets cloud logging ThingSpeak dashboard monitoring n8n workflow automation AI power prediction logic 1. Project Overview The robot can: ✅ Follow a human automatically ✅ Detect hand gestures for commands ✅ Send real-time Telegram alerts ✅ Store sensor data in Google Sheets ✅ Upload live telemetry to ThingSpeak cloud ✅ Use AI logic for battery/power prediction ✅ Trigger automated workflows using n8n ✅ Generate voice notifications 2. System Architecture Camera/Sensors ↓ ESP32 Controller ↓ Motor Driver + Motors ↓ WiFi Network ↓ n8n Automation Server ↓ ├── Telegram Voice Alerts ├── Google Sheets Logging ├── ThingSpeak Dashboard └── AI Prediction Engine 3. Required Components List Main Controller Component Quantity ESP32 Dev Board 1 L298N Motor Driver 1 DC Geared Motors 2 Robot Chassis 1 Wheels 2 Castor Wheel 1 Sensors Sensor Purpose Ultrasonic Sensor HC-SR04 Distance measurement IR Sensors Obstacle detection Camera Module ESP32-CAM Gesture recognition MPU6050 Motion sensing Servo Motor Camera rotation Power Item Specification Li-ion Battery 12V Battery Holder 1 Buck Converter 5V output Cloud & Automation Service Purpose ESP32 Documentation Microcontroller platform n8n Workflow automation Telegram Bot API Notification system Google Sheets Data storage ThingSpeak IoT dashboard 4. Hardware Connections ESP32 Pin Connections ESP32 Pin Connected To GPIO 12 Motor IN1 GPIO 13 Motor IN2 GPIO 14 Motor IN3 GPIO 27 Motor IN4 GPIO 26 Trigger HC-SR04 GPIO 25 Echo HC-SR04 GPIO 33 Servo Signal GPIO 32 IR Sensor Left GPIO 35 IR Sensor Right 5. Circuit Schematic Diagram +----------------+ | ESP32 | +----------------+ | | | | | | | +---- Ultrasonic Sensor | | +-------- Servo Motor | +------------ Motor Driver +---------------- WiFi Communication Motor Driver → DC Motors Battery → ESP32 + Motor Driver 6. Working Principle Human Following The ultrasonic sensor continuously measures distance. Logic: If human detected within 50cm: Move Forward Else: Stop Gesture Recognition Using ESP32-CAM: Gesture Action Palm Open Stop Thumb Up Move Forward Left Hand Turn Left Right Hand Turn Right 7. Complete Flowchart START ↓ Initialize ESP32 ↓ Connect WiFi ↓ Read Sensors ↓ Detect Human? ↓ YES Move Robot ↓ Check Gestures ↓ Execute Commands ↓ Send Data to Cloud ↓ Trigger n8n Workflow ↓ Telegram Voice Alert ↓ Store Data in Google Sheets ↓ Repeat 8. ESP32 Source Code (Main Logic) #include #include const char* ssid = "YOUR_WIFI"; const char* password = "PASSWORD"; #define TRIG 26 #define ECHO 25 void setup() { Serial.begin(115200); pinMode(TRIG, OUTPUT); pinMode(ECHO, INPUT); WiFi.begin(ssid, password); while(WiFi.status() != WL_CONNECTED) { delay(500); Serial.println("Connecting..."); } Serial.println("WiFi Connected"); } void loop() { long duration; int distance; digitalWrite(TRIG, LOW); delayMicroseconds(2); digitalWrite(TRIG, HIGH); delayMicroseconds(10); digitalWrite(TRIG, LOW); duration = pulseIn(ECHO, HIGH); distance = duration * 0.034 / 2; Serial.println(distance); if(distance < 50) { Serial.println("Human Detected"); } sendToThingSpeak(distance); delay(1000); } void sendToThingSpeak(int distance) { if(WiFi.status()== WL_CONNECTED){ HTTPClient http; String url = "http://api.thingspeak.com/update?api_key=YOUR_API_KEY&field1=" + String(distance); http.begin(url); int httpCode = http.GET(); Serial.println(httpCode); http.end(); } } 9. Setting Up Arduino IDE Install Required Libraries WiFi.h HTTPClient.h ESP32Servo.h NewPing.h Install board support: ESP32 by Espressif Systems 10. ThingSpeak Dashboard Setup Step-by-Step Open: ThingSpeak Platform Create account Create New Channel Add fields: Distance Battery Temperature Copy: Write API Key Channel ID Add API key in ESP32 code 11. Telegram Bot Setup Create Bot Open Telegram Search: @BotFather Type: /newbot Give bot name Copy Bot Token Get Chat ID Open: https://api.telegram.org/botYOUR_BOT_TOKEN/getUpdates Find: "chat":{"id":123456789} 12. Telegram Voice Notification System Workflow ESP32 Alert ↓ n8n Receives Webhook ↓ Text-to-Speech Conversion ↓ Telegram Voice Message 13. n8n Automation Workflow Install n8n Cloud Version Open: n8n Cloud Local Installation npm install n8n -g n8n start 14. n8n Workflow Logic Webhook Trigger ↓ IF Distance < Threshold ↓ Generate Alert Message ↓ Google Sheets Entry ↓ ThingSpeak Update ↓ Telegram Voice Alert 15. Sample n8n Workflow JSON { "nodes": [ { "name": "Webhook", "type": "n8n-nodes-base.webhook" }, { "name": "Telegram", "type": "n8n-nodes-base.telegram" } ] } 16. Google Sheets Integration Steps Open: Google Sheets Create Sheet Columns: Timestamp Distance Battery Gesture Status Connect Google account inside n8n Use: "Append Row" operation 17. AI-Based Power Consumption Prediction Purpose Predict remaining battery life. Inputs Parameter Description Motor Speed RPM Distance Traveled cm Battery Voltage V Current Draw A Simple Prediction Formula P=V×I Battery percentage: Battery%= Maximum Voltage Current Voltage ​ ×100 AI Logic IF battery < 20% Send alert Reduce motor speed 18. Voice Automation Using AI Text-to-Speech APIs You can use: Service Usage Google TTS Voice generation ElevenLabs AI realistic voice Telegram Voice Voice delivery 19. Cloud Dashboard Features Real-Time Monitoring Display: Robot status Gesture detected Battery level Human distance WiFi signal Temperature 20. Advanced Enhancements Future Improvements AI Features Face recognition Object detection Emotion recognition Voice command support Robotics SLAM navigation Autonomous mapping GPS tracking Cloud Firebase integration AWS IoT MQTT broker 21. Deployment Guide Indoor Deployment Use smooth flooring Stable WiFi Proper lighting Outdoor Deployment Add: Waterproof casing GPS module Solar charging 22. Security Recommendations Important Use HTTPS APIs Secure Telegram token Enable firewall on n8n server Use encrypted WiFi 23. Testing Procedure Test Expected Result Human detection Robot follows Gesture detection Executes command WiFi disconnection Auto reconnect Low battery Alert triggered Obstacle detection Robot stops 24. Final Project Outcome After completion, your system will: ✅ Follow humans intelligently ✅ Recognize gestures ✅ Send AI voice alerts ✅ Store cloud data ✅ Provide remote monitoring ✅ Predict battery usage ✅ Support full IoT automation 25. Recommended Software Stack Software Purpose Arduino IDE ESP32 programming Python AI processing OpenCV Vision system n8n Automation Telegram Bot Alerts ThingSpeak Dashboard 26. Suggested Folder Structure Project/ │ ├── ESP32_Code/ ├── n8n_Workflow/ ├── AI_Model/ ├── Documentation/ ├── Circuit_Diagram/ ├── Telegram_Bot/ └── Cloud_Dashboard/ 27. Recommended AI Extensions Add These Later YOLO object detection MediaPipe gesture recognition TensorFlow Lite Edge AI inference 28. Conclusion This project is a complete Industry 4.0 style intelligent robotics system combining: Embedded systems Artificial Intelligence Cloud IoT Automation Robotics Real-time communication It is suitable for: Final year engineering projects Research prototypes Smart surveillance Industrial automation Smart assistant robots