Human-Robot Collaboration for Search and Retrieval on Unitree Go 2 Platform

Overview

Welcome to the Human-Robot Collaboration for Search and Retrieval project! This project enhances efficiency and safety during search and retrieval operations in complex or hazardous environments. By combining autonomous navigation, machine learning for object recognition, and human-robot collaboration through voice commands, we aim to reduce manual workload and improve safety.

Table of Contents

• Team Information
• Devices
• Frameworks
• Goal
• Specific Aims

Team Information

• Course: Fall 2024 ECE202A Embedded Systems @ UCLA
• Instructor: Professor Mani Srivastava
• Mentor: Julian de Gortari
• Team Members: Johnson Liu, Yi Han, Pinhao Hong

Devices

The core devices used in this project include:

• Unitree Go 2 Robot Dog: The main platform for autonomous navigation and object interaction.
• Raspberry Pi 5: Functions as the control unit for running machine learning models and handling commands with ROS 2.

Frameworks

This project is built using the following software frameworks:

• Programming Language: Python for all development tasks, ensuring flexibility and ease of integration.
• ROS 2 (Robot Operating System 2): Provides a modular and scalable framework for robot software development.
• Libraries:
  • YOLO: For real-time object detection and recognition.
  • Vosk: Used for processing and interpreting voice commands.
  • pyttsx3: For generating spoken responses from the robot to the operator.

Goal

The primary goal of this project is to develop an integrated system that:

• Autonomously explores and maps environments.
• Recognizes and logs objects in real-time.
• Interacts with the operator through voice commands for object retrieval and guidance.
• Provides enhanced safety features, supporting operators in complex or hazardous areas with audio feedback and situational awareness.

Specific Aims

Aim 1: Autonomous Exploration and Object Recognition

• Objective: Equip the Unitree Go 2 robot with capabilities for autonomous navigation and real-time object recognition using YOLO.
• Expected Outcome: A reliable system for exploring environments and accurately identifying and logging objects.

Aim 2: Voice Command Interface Development

• Objective: Implement a user-friendly voice command system using Whisper for recognition and pyttsx3 for audio feedback.
• Expected Outcome: An effective interface allowing seamless human-robot interaction and control.

Aim 3: Enhanced Safety and Situational Awareness

• Objective: Integrate advanced navigation and safety features to provide real-time feedback and guide operators through complex environments.
• Expected Outcome: Improved operator support and situational awareness, minimizing risks in hazardous areas.

Aim 4: Real-World Validation

• Objective: Test the system in real-world use cases to assess performance.
• Expected Outcome: Validation of the system’s effectiveness and identification of areas for further optimization.

Required Submissions

Project Proposal
Midterm Checkpoint Slides
Final Report
Final Slides