Autonomous (ROS Based) Industrial Robot with AI vision and voice control

Kalpakghar - Community Tinkering Centre NSD2026
,
1

Participant Name : Dnynesh Mankare

College : MM Polytechnic Thergaon, Pune

Project title : Autonomous (ROS Based) Industrial Robot with AI vision and voice control.

1. Introduction

Industrial environments, especially packaging industries, require repetitive material handling, inspection, and transport tasks. These operations often demand continuous human involvement, leading to fatigue, reduced efficiency, and increased operational cost

Sapien is an intelligent ROS-based robot integrated with AI vision, LiDAR, and voice interaction, designed to automate industrial assistance tasks safely and efficiently.

2. Problem Statement (Packaging Industry Focus)

In packaging industries:

  • Manual movement of packages increases labor dependency.

  • Human errors occur in sorting and identification.

  • Workers face fatigue during repetitive transport tasks.

  • Safety risks exist in high-traffic factory environments.

  • Traditional automation systems are expensive and fixed (non-flexible).

There is a need for a flexible, intelligent, and cost-effective mobile robotic solution capable of autonomous navigation and smart object recognition.

3. Objective

The primary objectives of this project are:

  1. To design a ROS-based autonomous mobile robot.

  2. To integrate LiDAR for real-time navigation and mapping.

  3. To implement AI vision for object detection and identification.

  4. To enable voice-based interaction for human-robot communication.

  5. To ensure safe navigation using obstacle and edge detection sensors.

  6. To develop a flexible robotic assistant suitable for industrial environments.

4. Components Used

Processing Unit

  • Raspberry Pi 4 Model B

Navigation & Mapping

  • YDLIDAR X2 (360° scanning)

  • ROS (Robot Operating System)

Vision System

  • Raspberry Pi Camera (AI vision module)

  • YOLO / OpenCV for object detection

Safety & Detection Sensors

  • Ultrasonic sensors (Obstacle detection)

  • IR / Edge detection sensors (Cliff or boundary detection)

Communication

  • Voice module (Speech recognition + Text-to-speech)

Actuation

  • DC motors with motor driver

  • Chassis and power system

5. Working Principle (Sequential Flow)

  1. The robot initializes through ROS framework.

  2. LiDAR scans the environment and generates a 2D map.

  3. ROS navigation stack processes mapping and localization.

  4. AI vision system detects and identifies objects (packages, boxes, etc.).

  5. Obstacle sensors ensure safe real-time collision avoidance.

  6. Edge detection sensors prevent falling from elevated surfaces.

  7. Voice module allows operator to give commands (e.g., “Go to section A”).

  8. Robot autonomously navigates to target location.

  9. Performs task such as inspection, transport assistance, or monitoring.

6. Applications

  • Package transport within factory floor

  • Warehouse navigation

  • Visual inspection of labeled boxes

  • Human-robot collaboration tasks

  • Industrial material handling assistance

  • Voice-controlled industrial automation support

7. Key Features

  • 360° LiDAR-based mapping

  • AI-based object recognition

  • Voice interaction capability

  • Real-time obstacle avoidance

  • Edge detection safety mechanism

  • ROS-based modular architecture

  • Cost-effective and scalable design

The images for the robot Sapien are as follows -

2
21204×1600 133 KB

1 Like