AI-Powered Agentic IoT Accident Prevention System
An Agentic AI & IoT Vehicle Safety Architecture combining edge computer vision (NVIDIA Jetson Nano) with real-time telematics sensing (ESP32), cloud automation orchestration (n8n), AI predictive logic, and multi-channel alerting (Telegram Voice, ThingSpeak, Google Sheets).
1. Project Overview & Architecture
System Core Objectives
-
Edge Vision Intelligence: Real-time driver fatigue (eye-aspect ratio / drowsiness), distractive behavior detection, and obstacle collision warning via NVIDIA Jetson Nano.
-
Telematics Sensing: ESP32 collecting speed (GPS/Encoder), vehicle tilt/collision g-force (MPU6050 IMU), and system power draw (INA219).
-
Agentic Orchestration: An n8n automation engine evaluates sensor thresholds and visual telemetry, dynamically routing alarms and running predictive inference.
-
Multi-Cloud Dashboarding: Live data logging to ThingSpeak (telemetry charts), Google Sheets (audit trail log), and Telegram Bot (voice call/audio alerts to emergency contacts).
Block Flow Architecture
┌─────────────────────────┐
│ CSI Camera / USB Cam │
└────────────┬────────────┘
│ Video Stream
▼
┌─────────────────────────┐
│ NVIDIA Jetson Nano │
│ (YOLOv8 + OpenCV EAR) │
└────────────┬────────────┘
│ Jetson Alert Payload (JSON)
▼
┌──────────────────┐ UART / HTTP ┌─────────────────────────┐
│ Sensors (MPU6050,│───────────────>│ ESP32 Edge Node │
│ INA219, GPS, MQ) │ │ (OLED, Buzzer, Relay) │
└──────────────────┘ └────────────┬────────────┘
│ MQTT / HTTP POST
▼
┌─────────────────────────┐
│ n8n Workflow Engine │
│ (Agentic AI Router) │
└────┬──────────────┬─────┘
│ │
┌───────────────────────────┘ └──────────────────────────┐
▼ ▼
┌──────────────────────────┐ ┌──────────────────────────┐
│ Telegram Voice & Audio │ │ Cloud Analytics │
│ Bot Alerts (TTS API) │ │ (ThingSpeak / G-Sheets) │
└──────────────────────────┘ └──────────────────────────┘
2. Hardware Components List
| Category | Component | Quantity | Purpose |
| Compute Core | NVIDIA Jetson Nano (4GB) | 1 | Vision AI processing (YOLOv8 + MediaPipe Face Mesh) |
| IoT Node | ESP32-WROOM-32 Development Board | 1 | Sensor data aggregation, relay switching, MQTT client |
| Sensors | MPU6050 6-Axis Gyro & Accelerometer | 1 | Collision impact detection, rollover angle sensing |
| INA219 I2C Current/Voltage Sensor | 1 | Power monitoring and energy draw prediction | |
| MQ-3 Alcohol / MQ-2 Smoke Sensor | 1 | Intoxication and fire/hazard detection | |
| NEO-6M GPS Module | 1 | Real-time vehicle geolocation tracking | |
| IMX219 CSI Camera Module | 1 | High-speed driver monitoring video feed | |
| Actuators | Active Buzzer + 5V Relay Module | 1 ea | In-cabin audible warning & engine ignition kill switch |
| 0.96" I2C OLED Display (SSD1306) | 1 | Real-time dashboard output | |
| Power/Misc | XL4015 Buck Converter (12V to 5V 5A) | 1 | High-current power regulation for Jetson + ESP32 |
3. Circuit Schematic Wiring Specification
Wiring Connections Table
-
ESP32 to MPU6050 (I2C):
SDA -> GPIO 21,SCL -> GPIO 22,VCC -> 3.3V,GND -> GND -
ESP32 to INA219 (I2C):
SDA -> GPIO 21,SCL -> GPIO 22,VCC -> 3.3V,GND -> GND -
ESP32 to SSD1306 OLED (I2C): Shares I2C bus (
GPIO 21/GPIO 22), address0x3C -
ESP32 to NEO-6M GPS (UART2):
TX -> GPIO 16 (RX2),RX -> GPIO 17 (TX2),VCC -> 5V -
ESP32 to Actuators:
Buzzer -> GPIO 25,Relay Module -> GPIO 26,MQ-3 Analog -> GPIO 34 -
Jetson Nano to ESP32 Inter-Communication:
-
Option 1 (Hardware UART):
Jetson TX (Pin 8)->ESP32 RX0 (GPIO 3),Jetson RX (Pin 10)->ESP32 TX0 (GPIO 1)via 3.3V Logic Level. -
Option 2 (Wi-Fi Local Webhook): Both devices connect to local AP; Jetson sends HTTP POST requests directly to ESP32 / n8n.
-
4. System Flowchart
Boot Jetson Nano and ESP32. Establish Wi-Fi / MQTT broker connection. Calibrate MPU6050 baseline and initialize OpenCV camera pipeline.
Jetson: Frame capture -> Face Mesh (EAR computation) -> YOLOv8 obstacle detection.
ESP32: Poll MPU6050 impact forces, INA219 power metrics, and GPS coordinates every 100ms.
If EAR is less than threshold for over 1.5s OR collision G-force is greater than 3.5G OR alcohol is detected:
Trigger local alarm (Buzzer ON, Relay cutoff if stopped).
Format payload JSON with GPS coordinates, speed, and hazard level.
Send payload to n8n webhook. n8n triggers:
Google Sheets row insertion.
ThingSpeak channel updates.
Voice synthesization (TTS) & Telegram voice message call trigger.
5. ESP32 Source Code
This firmware runs on the ESP32, acquiring telemetry, handling local alerts, and pushing metrics to n8n / ThingSpeak.
#include <WiFi.h>
#include <HTTPClient.h>
#include <Wire.h>
#include <Adafruit_MPU6050.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_INA219.h>
#include <Adafruit_SSD1306.h>
// WiFi Configuration
const char* WIFI_SSID = "YOUR_WIFI_SSID";
const char* WIFI_PASS = "YOUR_WIFI_PASSWORD";
// n8n Webhook URL
const char* N8N_WEBHOOK_URL = "http://YOUR_N8N_SERVER_IP:5678/webhook/accident-alert";
// Hardware Pins
#define BUZZER_PIN 25
#define RELAY_PIN 26
#define MQ3_PIN 34
Adafruit_MPU6050 mpu;
Adafruit_INA219 ina219;
Adafruit_SSD1306 display(128, 64, &Wire, -1);
void setup() {
Serial.begin(115200);
pinMode(BUZZER_PIN, OUTPUT);
pinMode(RELAY_PIN, OUTPUT);
digitalWrite(RELAY_PIN, HIGH); // Ignition Active
// Init Display
Wire.begin();
if(!display.begin(SSD1306_SWITCHCAPVCC, 0x3C)) {
Serial.println(F("SSD1306 Allocation Failed"));
}
display.clearDisplay();
display.setTextSize(1);
display.setTextColor(WHITE);
// Init Sensors
if (!mpu.begin()) Serial.println("MPU6050 Connection Failed");
if (!ina219.begin()) Serial.println("INA219 Connection Failed");
// Connect to WiFi
WiFi.begin(WIFI_SSID, WIFI_PASS);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("\nWiFi Connected!");
}
void loop() {
sensors_event_t a, g, temp;
mpu.getEvent(&a, &g, &temp);
float busVoltage = ina219.getBusVoltage_V();
float current_mA = ina219.getCurrent_mA();
float power_mW = ina219.getPower_mW();
int alcoholVal = analogRead(MQ3_PIN);
// Acceleration Vector magnitude calculation
float gForce = sqrt(a.acceleration.x * a.acceleration.x +
a.acceleration.y * a.acceleration.y +
a.acceleration.z * a.acceleration.z) / 9.81;
bool impactDetected = (gForce > 3.0); // Threshold 3G
bool alcoholDetected = (alcoholVal > 2000);
// Local Alerts
if (impactDetected || alcoholDetected) {
digitalWrite(BUZZER_PIN, HIGH);
sendN8nAlert(gForce, busVoltage, power_mW, alcoholVal, impactDetected ? "CRASH_IMPACT" : "ALCOHOL_WARNING");
delay(2000);
} else {
digitalWrite(BUZZER_PIN, LOW);
}
// Display Telemetry
display.clearDisplay();
display.setCursor(0,0);
display.printf("G-Force: %.2f G\n", gForce);
display.printf("Power: %.1f mW\n", power_mW);
display.printf("Alcohol: %d\n", alcoholVal);
display.display();
delay(200);
}
void sendN8nAlert(float gForce, float voltage, float power, int alcohol, String alertType) {
if (WiFi.status() == WL_CONNECTED) {
HTTPClient http;
http.begin(N8N_WEBHOOK_URL);
http.addHeader("Content-Type", "application/json");
String jsonPayload = "{";
jsonPayload += "\"alert_type\":\"" + alertType + "\",";
jsonPayload += "\"g_force\":" + String(gForce) + ",";
jsonPayload += "\"voltage\":" + String(voltage) + ",";
jsonPayload += "\"power_mw\":" + String(power) + ",";
jsonPayload += "\"alcohol_raw\":" + String(alcohol) + ",";
jsonPayload += "\"latitude\": 17.3850,"; // Replace with dynamic GPS string
jsonPayload += "\"longitude\": 78.4867";
jsonPayload += "}";
int httpResponseCode = http.POST(jsonPayload);
Serial.printf("HTTP Response code: %d\n", httpResponseCode);
http.end();
}
}
6. n8n Automation Workflow JSON
Import this JSON into your n8n instance (Settings -> Import from File/Paste). It creates a webhook listener, routes data to Google Sheets, sends a ThingSpeak payload, converts alert text to audio, and sends a Telegram Voice message.
{
"name": "AI IoT Accident Prevention Automation",
"nodes": [
{
"parameters": {
"httpMethod": "POST",
"path": "accident-alert",
"options": {}
},
"name": "Webhook Listener",
"type": "n8n-nodes-base.webhook",
"typeVersion": 1,
"position": [250, 300]
},
{
"parameters": {
"operation": "append",
"sheetId": "YOUR_GOOGLE_SHEET_ID",
"range": "A:F",
"options": {}
},
"name": "Google Sheets Logger",
"type": "n8n-nodes-base.googleSheets",
"typeVersion": 4,
"position": [500, 180]
},
{
"parameters": {
"url": "https://api.thingspeak.com/update",
"sendQuery": true,
"queryParameters": {
"parameters": [
{ "name": "api_key", "value": "YOUR_THINGSPEAK_WRITE_KEY" },
{ "name": "field1", "value": "={{ $json.body.g_force }}" },
{ "name": "field2", "value": "={{ $json.body.power_mw }}" }
]
},
"options": {}
},
"name": "ThingSpeak Sync",
"type": "n8n-nodes-base.httpRequest",
"typeVersion": 4.1,
"position": [500, 320]
},
{
"parameters": {
"chatId": "YOUR_TELEGRAM_CHAT_ID",
"text": "=⚠️ ACCIDENT WARNING!\nType: {{ $json.body.alert_type }}\nG-Force: {{ $json.body.g_force }}G\nMaps: https://maps.google.com/?q={{ $json.body.latitude }},{{ $json.body.longitude }}",
"additionalFields": {}
},
"name": "Telegram Alert",
"type": "n8n-nodes-base.telegram",
"typeVersion": 1,
"position": [500, 460]
}
],
"connections": {
"Webhook Listener": {
"main": [
[
{ "node": "Google Sheets Logger", "type": "main", "index": 0 },
{ "node": "ThingSpeak Sync", "type": "main", "index": 0 },
{ "node": "Telegram Alert", "type": "main", "index": 0 }
]
]
}
}
}
7. Telegram Bot & Cloud Integration Setup
Telegram Bot & Voice Setup
-
Open Telegram, search for
@BotFather, and send/newbot. -
Name your bot and copy the HTTP API Token.
-
Search
@userinfobotto get your Telegram Chat ID. -
To enable Voice Notifications:
-
Integrate an ElevenLabs or Google Text-to-Speech (TTS) node in n8n.
-
Direct the generated
.mp3/.oggaudio binary output into the n8n Telegram node under thesendVoiceaction.
-
Google Sheets Integration
-
Create a Google Sheet named
Vehicle_Telemetry_Logs. -
Header Row (
Row 1):Timestamp,Alert_Type,G_Force,Power_mW,Latitude,Longitude. -
In n8n, connect Google OAuth credentials and select your Spreadsheet ID.
ThingSpeak Cloud Dashboard
-
Sign up on ThingSpeak and create a New Channel.
-
Setup Fields:
-
Field 1: G-Force (Impact) -
Field 2: Power Draw (mW) -
Field 3: Driver Drowsiness Score (EAR)
-
-
Copy the Write API Key and insert it into the n8n HTTP Request Node.
8. AI Power Consumption Prediction Logic
Using the INA219 current sensor readings, an onboard machine learning regressor predicts remaining battery operating hours and anomalies in edge computing hardware power spikes.
Python Prediction Script (NVIDIA Jetson / Cloud Agent)
import numpy as np
from sklearn.linear_model import LinearRegression
# Simulated Historical Feature Array: [G-Force, Camera FPS, Jetson CPU Load %]
X_train = np.array([
[1.0, 30, 25],
[1.1, 30, 40],
[1.0, 60, 75],
[2.5, 60, 95],
[1.2, 15, 20]
])
# Target Variable: Jetson + ESP32 System Power Draw in Milliwatts (mW)
y_train = np.array([3200, 4100, 6800, 8900, 2400])
# Train Model
model = LinearRegression()
model.fit(X_train, y_train)
def predict_power_draw(g_force, fps, cpu_load):
predicted_mw = model.predict([[g_force, fps, cpu_load]])[0]
# Battery math: 10,000 mAh 5V Powerbank = 50,000 mWh Capacity
remaining_hours = 50000.0 / predicted_mw if predicted_mw > 0 else 0
return {
"predicted_power_mw": round(predicted_mw, 2),
"est_battery_life_hours": round(remaining_hours, 2)
}
# Example Inference
telemetry = predict_power_draw(g_force=1.05, fps=30, cpu_load=55.0)
print(f"Predicted Power: {telemetry['predicted_power_mw']} mW")
print(f"Estimated Battery Remaining: {telemetry['est_battery_life_hours']} Hours")
9. Future Enhancements & Deployment
-
eCall System Integration: Auto-dial regional emergency services (112 / 911) with precise GPS coordinates via a SIM800L GSM module backup when cellular Wi-Fi is lost.
-
CAN Bus Telematics: Interface directly with the vehicle's OBD-II port (using MCP2515 CAN controller) to pull real braking pressure, steering angle, and engine RPM.
-
Jetson TensorRT Acceleration: Quantize the driver fatigue detection model to
INT8using NVIDIA TensorRT to increase video inference performance to >60 FPS at lower power draw.
No comments:
Post a Comment