This folder provides the base navigation system for the CMU Vision-Language-Autonomy Challenge. The system is integrated with Unity environment models. The code has been tested in Ubuntu 20.04 with ROS Noetic.
Install dependencies with the command lines below:
sudo apt update
sudo apt install libusb-dev python-yaml python-is-python3
In a terminal, go inside this folder and compile (this may take a few minutes):
catkin_make
Download any of our Unity environment models, unzip the folder, and copy the files inside to the src/vehicle_simulator/mesh/unity folder. The environment model files should follow the structure below. Note that the AssetList.csv file is generated upon start of the system and that only one given environment folder can be placed under the src/vehicle_simulator/mesh/unity directory at a time.
mesh/
unity/
environment/
Model_Data/ (multiple files in the folder)
Model.x86_64
UnityPlayer.so
AssetList.csv (generated at runtime)
Dimensions.csv
Categories.csv
map.ply
object_list.txt
traversable_area.ply
map.jpg
render.jpg
To launch just the simulator system by itself, go inside this folder and run:
./system_bring_up.sh
Users should see data showing up in RVIZ. Click 'Panels->Display' and check the data to view. Users can also use the 'Waypoint with Heading' button to navigate the vehicle. To set the waypoint, press the left button on the mouse to choose the position, then move the mouse to choose the orientation before releasing the left button. The vehicle will navigate to the waypoint avoiding collisions on the way and then turn to the orientation. A video showing the system in action is available. Note that the waypoints are meant to be relatively close to the vehicle. Setting the waypoint too far can cause the vehicle to stuck at a dead end.
Users can also use the control panel to navigate the vehicle by clicking inside the black box. The system will switch to smart joystick mode - the vehicle tries to follow the joystick command and avoid collisions at the same time. To resume waypoint navigation, press the 'Resume Navigation to Goal' button.
Joystick Navigation - Note that users can use a PS3/4 or Xbox controller with a USB or Bluetooth interface instead of the virtual joystick (If using the Xbox Wireless USB Adapter, please install xow). Users can use the right joystick on the controller to navigate the vehicle. Holding the obstacle-check button cancels obstacle checking and clicking the clear-terrain-map button reinitializes the terrain map. To resume waypoint navigation, hold the mode-switch button and at the same time push the right joystick. Doing so will allow the right joystick to control the speed. If only holding the mode-switch button, the system will use the default speed.
Troubleshooting - If the system does not launch correctly, users can launch the Unity environment and the autonomy system in two separate terminals. In a terminal, go inside this folder and launch the Unity environment:
./src/vehicle_simulator/mesh/unity/environment/Model.x86_64
In a second terminal, go inside this main folder and bring up the autonomy system:
source devel/setup.sh
roslaunch vehicle_simulator system_unity.launch
ROS-TCP-Endpoint and joystick_drivers packages are from open-source releases.
The code in this folder is based on Autonomous Exploration Development Environment. A visibility graph-based route planner can be found here: Far Planner.