AI-Based Trash Segregation Robotic Arm Using Computer Vision + ESP32 + IoT + n8n + AI Agent + Telegram Voice Alerts + Google Sheets + ThingSpeak
Below is a complete reference design you can use as a final-year/major-project implementation, including the hardware architecture, working principle, wiring, software structure, ESP32 firmware, computer-vision program, n8n workflow, Telegram alerts, Google Sheets logging, ThingSpeak dashboard, database format, flowcharts, and testing procedure.
- Project Title
AI-Based Smart Trash Segregation Robotic Arm Using Computer Vision, ESP32, IoT, n8n Automation and Agentic AI
Short title
AI Smart Waste Segregation & Agentic IoT System
- Abstract
The proposed system is an intelligent waste-segregation system that automatically detects, classifies, and sorts different types of waste using Computer Vision and Artificial Intelligence.
A camera continuously observes waste placed on a conveyor or collection platform. A computer such as a Raspberry Pi, laptop, mini-PC, or Jetson device processes the camera image using an AI-based object-detection/classification model. Once the waste type is identified, the computer sends the classification and corresponding sorting command to an ESP32.
The ESP32 controls a robotic arm, servo motors, conveyor motor, sensors, and other actuators. The robotic arm picks the detected waste and places it into the appropriate bin—for example:
🟢 Organic/wet waste
🔵 Recyclable/plastic
🟡 Paper
⚫ Metal
🔴 Reject/hazardous/unknown
The ESP32 simultaneously publishes system information to an IoT/cloud layer.
An n8n automation workflow acts as the orchestration/agentic layer. It receives events from the ESP32, records data in Google Sheets, updates a ThingSpeak dashboard, evaluates system conditions, and sends notifications through Telegram.
The system can additionally use an AI Agent in n8n to interpret events such as:
"The robotic arm has sorted 48 objects today. Plastic waste is 42%, paper is 31%, and 27% is organic. The bin is approaching its capacity."
The resulting notification can be sent to the operator through Telegram, including a generated voice notification.
- Main Objectives
The project has eight major objectives.
Objective 1 — Waste detection
Use a camera to capture images/video of waste.
Objective 2 — AI classification
Determine what type of waste is present.
Objective 3 — Automatic robotic sorting
Move the robotic arm to the correct bin.
Objective 4 — ESP32 IoT control
Use ESP32 as the real-time controller for:
Servos
Sensors
Conveyor
Bin-level monitoring
Wi-Fi
Cloud communication
Objective 5 — Automation
Use n8n to connect the physical system to cloud services.
Objective 6 — Data logging
Store every sorting event in Google Sheets.
Objective 7 — Monitoring
Display statistics using ThingSpeak and/or a custom IoT webpage.
Objective 8 — AI/Agentic notifications
Use an AI Agent + n8n to interpret system events and send useful Telegram/voice notifications.
- Overall System Architecture
┌─────────────────────────┐
│ WASTE OBJECT │
│ Bottle / Paper / Metal │
│ Organic / Other │
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ CAMERA │
│ USB / ESP32-CAM / CSI │
└────────────┬────────────┘
│
▼
┌─────────────────────────────────────┐
│ COMPUTER VISION SYSTEM │
│ Raspberry Pi / PC / Jetson │
│ │
│ Image preprocessing │
│ Object detection │
│ Classification │
│ Confidence calculation │
└────────────────┬────────────────────┘
│
Waste classification
│
▼
┌─────────────────┐
│ ESP32 │
│ IoT Controller │
└────────┬────────┘
│
┌─────────────┼──────────────┐
│ │ │
▼ ▼ ▼
Servo Motors Sensors Conveyor
│
▼
┌───────────────────┐
│ ROBOTIC ARM │
│ Pick → Move → │
│ Place │
└─────────┬─────────┘
│
┌─────────┼─────────┐
▼ ▼ ▼
Organic Recycle Paper
│ │ │
└─────────┼─────────┘
│
▼
Sorted Waste - IoT + Agentic AI Architecture
INTERNET
│
▼
┌───────────────┐
│ n8n │
│ Automation │
└───────┬───────┘
│
┌──────────────┼───────────────┐
│ │ │
▼ ▼ ▼
┌───────────┐ ┌────────────┐ ┌─────────────┐
│ AI Agent │ │ Google │ │ ThingSpeak │
│ │ │ Sheets │ │ Dashboard │
└─────┬─────┘ └────────────┘ └─────────────┘
│
▼
┌──────────────┐
│ Decision / │
│ Explanation │
└──────┬───────┘
│
▼
┌──────────────┐
│ Telegram Bot │
└──────┬───────┘
│
┌─────┴─────┐
▼ ▼
Text Alert Voice Alert
ESP32
│
│ HTTPS / MQTT
│
▼
n8n Webhook
- Complete Data Flow
Waste placed
│
▼
Camera captures image
│
▼
AI detects object
│
▼
Classification
│
├── Plastic
├── Paper
├── Metal
├── Organic
└── Unknown
│
▼
Calculate confidence
│
▼
Is confidence > threshold?
│
┌─┴───────────┐
│ │
YES NO
│ │
▼ ▼
Send command Reject
to ESP32 / manual check
│
▼
ESP32 receives command
│
▼
Robotic arm picks object
│
▼
Arm rotates
│
▼
Correct bin selected
│
▼
Object released
│
▼
Bin-level sensor
│
▼
ESP32 sends event
│
▼
n8n Webhook
│
├──────────────► Google Sheets
│
├──────────────► ThingSpeak
│
├──────────────► AI Agent
│
└──────────────► Telegram
│
▼
Voice Alert - Hardware Requirements
Core controller
Component Quantity Purpose
ESP32 DevKit 1 IoT/robot controller
Raspberry Pi / PC / Jetson 1 Computer vision
Camera 1 Waste detection
Robotic arm 1 Waste movement
Servo motors 4–6 Arm movement
Servo gripper 1 Pick object
Conveyor motor 1 Waste movement
Motor driver 1 Conveyor control
IR sensor 1–2 Object detection
Ultrasonic sensors 3–5 Bin-level monitoring
Wi-Fi Built into ESP32 IoT connectivity
Power supply 1+ Motor/ESP32 power - Suggested Robotic Arm
A basic 4-DOF arm can contain:
GRIPPER
│
▼
[Servo]
\
\
Upper Arm
│
[Servo]
│
Forearm
│
[Servo]
│
Base
[Servo]
Typical degrees of freedom:
Base rotation
Shoulder
Elbow
Wrist
Gripper
For a student prototype, five servo channels are sufficient.
- Recommended Servo Arrangement
Example:
Servo Function
Servo 1 Base
Servo 2 Shoulder
Servo 3 Elbow
Servo 4 Wrist
Servo 5 Gripper
Example GPIO allocation:
ESP32 GPIO 13 → Base servo
ESP32 GPIO 14 → Shoulder servo
ESP32 GPIO 25 → Elbow servo
ESP32 GPIO 26 → Wrist servo
ESP32 GPIO 27 → Gripper servo
These are example pins; adapt them to your actual ESP32 board.
- Important Power Design
Do not power several robotic servos directly from the ESP32 5-V pin.
Use a separate servo power supply.
5–6 V HIGH CURRENT SUPPLY
│
┌─────────┼─────────┐
│ │ │
Servo 1 Servo 2 Servo 3...
│ │ │
└─────────┼─────────┘
│
GND
│
┌─────┴─────┐
│ ESP32 │
│ │
│ GND───────┘
└───────────
Common ground is essential.
- Bin Arrangement
A simple design is:
ROBOT
│
▼
┌─────────────┐
│ Sorting │
│ Position │
└──────┬──────┘
│
┌───────────┼────────────┐
│ │ │
▼ ▼ ▼
BIN 1 BIN 2 BIN 3
Organic Plastic Paper
BIN 4
Metal
BIN 5
Reject
- Bin-Level Detection
Use an ultrasonic sensor above each bin.
For example:
Ultrasonic Sensor
│
┌──────┴──────┐
│ │
▼ │
Echo │
↓ │
┌───────────────┐ │
│ │ │
│ WASTE │ │
│ │ │
└───────────────┘
Distance:
Empty bin:
distance = 35 cm
Partially full:
distance = 20 cm
Nearly full:
distance = 8 cm
Example condition:
if (distance < 10) {
binFull = true;
}
- Computer-Vision System
The vision computer performs:
Camera
│
▼
Image capture
│
▼
Resize / preprocessing
│
▼
AI model
│
▼
Object detection
│
▼
Class
│
▼
Confidence
│
▼
Sorting command
- AI Classes
A practical model can initially use:
0 = plastic
1 = paper
2 = metal
3 = organic
4 = glass
5 = cardboard
6 = unknown
For a smaller prototype:
plastic
paper
metal
organic
reject
Start with fewer classes. A well-trained 4–5 class model is usually easier to demonstrate reliably than a large number of poorly separated categories.
- AI Decision Logic
Example:
Detected:
Plastic bottle
Confidence:
94%
Decision:
PLASTIC
Command:
SORT_PLASTIC
If confidence is low:
Detected:
Unknown object
Confidence:
42%
Threshold:
70%
Decision:
REJECT
This is important because the robotic arm should not blindly act on a poor classification.
- Computer-Vision Python Example
Below is a reference implementation using a YOLO-style model.
Install:
pip install ultralytics opencv-python requests
Python program:
import cv2
import requests
import time
from ultralytics import YOLO
--------------------------------
Configuration
--------------------------------
MODEL_PATH = "waste_model.pt"
ESP32_URL = "http://192.168.1.100/sort"
CONFIDENCE_THRESHOLD = 0.70
model = YOLO(MODEL_PATH)
cap = cv2.VideoCapture(0)
last_command_time = 0
COMMAND_DELAY = 3
--------------------------------
Send command to ESP32
--------------------------------
def send_to_esp32(category, confidence):
payload = {
"category": category,
"confidence": round(float(confidence), 3)
}
try:
response = requests.post(
ESP32_URL,
json=payload,
timeout=3
)
print("ESP32:", response.text)
except Exception as e:
print("ESP32 communication error:", e)--------------------------------
Main loop
--------------------------------
while True:
ret, frame = cap.read()
if not ret:
break
results = model(frame)
for result in results:
boxes = result.boxes
for box in boxes:
confidence = float(box.conf[0])
class_id = int(box.cls[0])
class_name = model.names[class_id]
x1, y1, x2, y2 = map(
int,
box.xyxy[0]
)
label = f"{class_name} {confidence:.2f}"
cv2.rectangle(
frame,
(x1, y1),
(x2, y2),
(0, 255, 0),
2
)
cv2.putText(
frame,
label,
(x1, y1 - 10),
cv2.FONT_HERSHEY_SIMPLEX,
0.7,
(0, 255, 0),
2
)
if confidence >= CONFIDENCE_THRESHOLD:
current_time = time.time()
if current_time - last_command_time > COMMAND_DELAY:
print(
"Detected:",
class_name,
confidence
)
send_to_esp32(
class_name,
confidence
)
last_command_time = current_time
cv2.imshow(
"AI Waste Segregation",
frame
)
if cv2.waitKey(1) & 0xFF == ord("q"):
breakcap.release()
cv2.destroyAllWindows()
- Training Your Own AI Model
Create a dataset:
dataset/
│
├── images/
│ ├── train/
│ └── val/
│
└── labels/
├── train/
└── val/
Example:
plastic
paper
metal
organic
Collect many images under different:
Lighting conditions
Angles
Backgrounds
Object orientations
Distances
Object sizes
The dataset should contain examples representative of what the real camera will see.
- Example Dataset Configuration
data.yaml
path: ./dataset
train: images/train
val: images/val
names:
0: plastic
1: paper
2: metal
3: organic
Training example:
yolo detect train
model=yolo11n.pt
data=data.yaml
epochs=100
imgsz=640
Use a model/version appropriate to the software environment you install.
- ESP32 Architecture
The ESP32 performs four major jobs:
ESP32
│
┌───────┼────────┐
│ │ │
▼ ▼ ▼
Sensors Servo Wi-Fi
│ │ │
│ │ ▼
│ │ n8n
│ │
│ ▼
│ Robotic Arm
│
▼
Bin status
- ESP32 Software States
A good implementation should use a state machine.
IDLE
│
▼
OBJECT_DETECTED
│
▼
WAIT_FOR_AI
│
▼
CLASSIFICATION_RECEIVED
│
├── valid ───────► PICK
│ │
│ ▼
│ MOVE
│ │
│ ▼
│ RELEASE
│ │
│ ▼
│ VERIFY
│ │
│ ▼
│ IDLE
│
└── invalid ─────► REJECT
This is better than writing one giant loop() function.
- ESP32 Example Firmware
Install the ESP32 board package and a servo library suitable for your environment.
Example Arduino-style code:
#include <WiFi.h>
#include <WebServer.h>
#include <ArduinoJson.h>
#include <ESP32Servo.h>
// -----------------------------
// WiFi
// -----------------------------
const char* WIFI_SSID = "YOUR_WIFI";
const char* WIFI_PASSWORD = "YOUR_PASSWORD";
// -----------------------------
// ESP32 Web Server
// -----------------------------
WebServer server(80);
// -----------------------------
// Servo objects
// -----------------------------
Servo baseServo;
Servo shoulderServo;
Servo elbowServo;
Servo wristServo;
Servo gripperServo;
// GPIO
const int BASE_PIN = 13;
const int SHOULDER_PIN = 14;
const int ELBOW_PIN = 25;
const int WRIST_PIN = 26;
const int GRIPPER_PIN = 27;
// -----------------------------
// Arm positions
// -----------------------------
void armHome()
{
baseServo.write(90);
shoulderServo.write(90);
elbowServo.write(90);
wristServo.write(90);
gripperServo.write(30);
delay(700);
}
// -----------------------------
// Pick object
// -----------------------------
void pickObject()
{
gripperServo.write(30);
delay(500);
shoulderServo.write(110);
elbowServo.write(70);
delay(600);
gripperServo.write(90);
delay(600);
}
// -----------------------------
// Release object
// -----------------------------
void releaseObject()
{
gripperServo.write(30);
delay(600);
}
// -----------------------------
// Sorting position
// -----------------------------
void moveToBin(String category)
{
if (category == "plastic")
{
baseServo.write(40);
}
else if (category == "paper")
{
baseServo.write(75);
}
else if (category == "metal")
{
baseServo.write(110);
}
else if (category == "organic")
{
baseServo.write(145);
}
else
{
baseServo.write(175);
}
delay(800);
}
// -----------------------------
// Complete sorting cycle
// -----------------------------
void sortObject(String category)
{
Serial.println("Sorting: " + category);
pickObject();
moveToBin(category);
releaseObject();
armHome();
}
// -----------------------------
// HTTP endpoint
// -----------------------------
void handleSort()
{
if (!server.hasArg("plain"))
{
server.send(
400,
"application/json",
"{"error":"No JSON received"}"
);
return;
}
String body = server.arg("plain");
StaticJsonDocument<256> doc;
DeserializationError error =
deserializeJson(doc, body);
if (error)
{
server.send(
400,
"application/json",
"{\"error\":\"Invalid JSON\"}"
);
return;
}
String category =
doc["category"] | "unknown";
float confidence =
doc["confidence"] | 0.0;
Serial.println(
"Category: " + category
);
Serial.println(
"Confidence: " +
String(confidence)
);
if (confidence < 0.70)
{
category = "unknown";
}
sortObject(category);
server.send(
200,
"application/json",
"{\"status\":\"sorted\"}"
);
}
// -----------------------------
// Setup
// -----------------------------
void setup()
{
Serial.begin(115200);
baseServo.attach(BASE_PIN);
shoulderServo.attach(SHOULDER_PIN);
elbowServo.attach(ELBOW_PIN);
wristServo.attach(WRIST_PIN);
gripperServo.attach(GRIPPER_PIN);
armHome();
WiFi.begin(
WIFI_SSID,
WIFI_PASSWORD
);
Serial.print("Connecting");
while (WiFi.status() != WL_CONNECTED)
{
delay(500);
Serial.print(".");
}
Serial.println();
Serial.print("ESP32 IP: ");
Serial.println(WiFi.localIP());
server.on(
"/sort",
HTTP_POST,
handleSort
);
server.begin();
Serial.println("Server started");
}
// -----------------------------
// Loop
// -----------------------------
void loop()
{
server.handleClient();
}
- ESP32-to-n8n Communication
There are two useful architectures.
Architecture A — ESP32 → n8n
ESP32
│
│ HTTPS POST
▼
n8n Webhook
Use this for:
Sensor data
Bin level
Sorting completed
Errors
Temperature
Motor status
Architecture B — Computer Vision → ESP32
Camera
↓
AI computer
↓
HTTP
↓
ESP32
↓
Robotic arm
This is recommended for the actual real-time sorting action.
- Event JSON Format
Every event should have a consistent structure.
Example:
{
"device_id": "ESP32_SORTER_01",
"event": "sorting_completed",
"category": "plastic",
"confidence": 0.94,
"bin": "plastic",
"timestamp": "2026-08-18T20:30:15",
"cycle_time_ms": 4200
}
For sensor information:
{
"device_id": "ESP32_SORTER_01",
"event": "telemetry",
"organic_bin": 65,
"plastic_bin": 82,
"paper_bin": 47,
"metal_bin": 34
}
- n8n Automation Architecture
The central automation workflow can be:
ESP32
│
▼
┌──────────────┐
│ Webhook │
│ Trigger │
└──────┬───────┘
│
▼
┌──────────────┐
│ Validate │
│ JSON │
└──────┬───────┘
│
▼
┌──────────────┐
│ Switch │
│ Event Type │
└──────┬───────┘
│
┌────────┼─────────┐
│ │ │
▼ ▼ ▼
Sorting Telemetry Error
│ │ │
▼ ▼ ▼
Google ThingSpeak AI Agent
Sheets │ │
│ │ │
└────────┼─────────┘
│
▼
Alert Decision
│
▼
Telegram Bot
│
┌──────┴──────┐
▼ ▼
Text Voice
Alert Alert
- n8n Workflow Nodes
A practical workflow can contain:
Workflow 1 — Sorting Event
Webhook
↓
Set / Code
↓
Validate event
↓
Google Sheets
↓
AI Agent
↓
IF / Switch
↓
Telegram
↓
Voice generation
↓
Telegram audio
Workflow 2 — Bin Monitoring
Webhook
↓
Extract bin levels
↓
ThingSpeak HTTP Request
↓
IF bin > 80%
↓
AI Agent
↓
Telegram alert
↓
Voice notification
Workflow 3 — Daily Report
Schedule Trigger
↓
Google Sheets
↓
Aggregate data
↓
AI Agent
↓
Generate summary
↓
Telegram
- n8n Webhook
Create a Webhook node.
Example:
POST
/api/waste-event
ESP32 sends:
{
"device_id": "ESP32_SORTER_01",
"event": "sorting_completed",
"category": "plastic",
"confidence": 0.94,
"bin": "plastic"
}
The webhook passes this information to subsequent nodes.
- n8n Code Node
Example JavaScript:
const data = $json;
const category = data.category || "unknown";
const confidence = Number(data.confidence || 0);
let alertLevel = "normal";
if (confidence < 0.70) {
alertLevel = "low_confidence";
}
return [
{
json: {
...data,
confidence_percent:
Math.round(confidence * 100),
alert_level: alertLevel,
processed_at:
new Date().toISOString()
}
}
];
- Google Sheets Database
Create a spreadsheet:
Waste_Sorting_Log
Columns:
Timestamp Device Category Confidence Bin Cycle Time Status
20:10 ESP32_01 Plastic 94% Plastic 4.2 s OK
20:11 ESP32_01 Paper 91% Paper 4.0 s OK
20:12 ESP32_01 Metal 88% Metal 4.5 s OK
29. Daily Statistics
The system can calculate:
Total objects = 150
Plastic = 55
Paper = 35
Metal = 22
Organic = 30
Reject = 8
Percentages:
Plastic = 36.7%
Paper = 23.3%
Metal = 14.7%
Organic = 20.0%
Reject = 5.3%
- ThingSpeak Architecture
ThingSpeak can be used for time-series telemetry.
Example fields:
Field 1 → Total Waste
Field 2 → Plastic
Field 3 → Paper
Field 4 → Metal
Field 5 → Organic
Field 6 → Reject
Field 7 → Bin Level
Field 8 → System Status
Example update:
field1 = 150
field2 = 55
field3 = 35
field4 = 22
field5 = 30
field6 = 8
n8n can send these values using an HTTP Request node.
- Custom IoT Webpage
A simple web dashboard can display:
┌─────────────────────────────────────────────┐
│ AI SMART WASTE SEGREGATION │
├─────────────────────────────────────────────┤
│ │
│ System: 🟢 ONLINE │
│ │
│ Total Objects: 150 │
│ │
│ Plastic ███████████████ 55 │
│ Paper █████████ 35 │
│ Metal █████ 22 │
│ Organic ████████ 30 │
│ Reject ██ 8 │
│ │
├─────────────────────────────────────────────┤
│ BIN STATUS │
│ │
│ Plastic 82% 🟠│
│ Paper 47% 🟢 │
│ Metal 34% 🟢 │
│ Organic 65% 🟠│
│ │
└─────────────────────────────────────────────┘
- Simple HTML Dashboard
AI Waste Segregation
♻ AI Smart Waste Segregation
System Status
● ONLINE
Waste Statistics
Plastic: 0
Paper: 0
Metal: 0
Organic: 0
- Telegram Bot Architecture
ESP32
│
▼
n8n
│
▼
AI Agent
│
├── Normal event
│ ↓
│ No alert
│
├── Bin nearly full
│ ↓
│ Telegram alert
│
├── AI confidence low
│ ↓
│ Warning
│
├── Hardware error
│ ↓
│ Critical alert
│
└── Daily report
↓
Telegram
- Example Telegram Message
Normal
♻ Waste Sorting Update
Object: Plastic bottle
AI Confidence: 94%
Bin: Plastic
Device: ESP32_SORTER_01
Status: Successfully sorted
Bin full
⚠ BIN CAPACITY ALERT
Plastic bin has reached 87%.
Please empty the plastic waste container.
Device: ESP32_SORTER_01
Hardware error
🚨 ROBOT ERROR
The robotic arm failed to complete the sorting cycle.
Possible causes:
- Servo obstruction
- Object not gripped
- Mechanical jam
Manual inspection required.
- Telegram Voice Alert
The workflow can be:
n8n
│
▼
Generate message
│
▼
Text-to-Speech service
│
▼
Audio file
│
▼
Telegram Bot
│
▼
Voice/audio notification
Example generated sentence:
"Warning. The plastic bin is eighty-seven percent full. Please empty the bin."
The exact TTS provider can be selected according to your
No comments:
Post a Comment