Adapting Dreamer’s Sensory Input for LIDAR-Enhanced Navigation in MuJoCo
Recent advancements in Large Language Models (LLMs) have demonstrated remarkable capabilities in language understanding, reasoning, and long-term planning by learning structured representations of textual information. Similarly, Reinforcement Learning (RL) has achieved significant progress in complex decision-making tasks. However, robotic control and navigation remain challenging due to the high dimensionality of sensory inputs and the difficulty of accurately modeling environmental dynamics. A promising direction to address these challenges is the use of world models, that are neural representations that learn compact, predictive structures of the environment, similar to how LLMs capture latent semantic structures in language.
World models aim to learn latent representations that encode the underlying physics and causal relationships of an environment (Ha & Schmidhuber, 2018). The Dreamer-V1 framework builds on this approach by learning a latent space from visual data and employing an imagination process to predict future trajectories. This enables the agent to plan and act based on an internal model rather than direct environmental interaction (Hafner et al., 2020). Nevertheless, the reliance on visual inputs can limit the applicability of Dreamer-V1 in domains where other sensing modalities, such as depth or range data, provide more suitable information for control and navigation, e.g. under poor lighting (Rodríguez-Martínez et al., 2025).
Model-based reinforcement learning has traditionally focused on vision-based observation spaces, where agents learn dynamics from pixels. However, many robotic systems rely primarily on range-based sensing for obstacle avoidance, spatial mapping, and localization. Light Detection and Ranging (LIDAR) sensors provide high-resolution distance measurements that encode the surrounding geometry more directly than visual features, making them particularly suitable for environments with poor visibility or minimal texture. Integrating LIDAR into world models thus represents an important step toward developing sensor-flexible agents capable of reasoning and planning in partially observable or visually ambiguous environments. While related approaches such as PlaNet, Dreamer-V2 and Dreamer-V3 have extended the efficiency and stability of latent-world learning, little work has examined their adaptation to non-visual modalities (Hafner et al., 2019; Hafner et al., 2021; Hafner et al., 2023). This gap motivates the present study.
This thesis investigates the adaptation of the Dreamer-V1 algorithm to process LIDAR data instead of images. Using the Mujoco physics engine, a simulated environment is developed where LIDAR provides detailed spatial and depth information for navigation tasks. The central objective is to evaluate whether LIDAR-based world models can effectively capture environmental dynamics and support model-based reinforcement learning for autonomous navigation.
The main contributions of this thesis are as follows:
A reimplementation of the Dreamer-V1 algorithm in PyTorch.
An adaptation of the encoder and decoder networks to process one-dimensional LIDAR observations instead of images.
The design of a LIDAR-based Mujoco simulation environment for evaluating world-model learning and control performance.
A systematic experimental evaluation across five training phases, analyzing both world-model quality and agent performance.
Beyond its technical implementation, this work aims to advance the understanding of how alternative sensory modalities influence representation learning in model-based reinforcement learning. Insights from this study contribute to developing more robust and sensor-flexible world models applicable to robotic and autonomous systems operating in non-visual domains.
Clone the repository and install dependencies with your favorite package manager (e.g. poetry).
pip install poetry
poetry installThe repository is build for running configuration yaml files that specify the parameters for
different setups and environments. In order to run an experiment, a yaml file needs to be created
and placed into the /configs/active_configs/ section.
There are different options for testing, but for now the main processes are described and can be understood based on the given
configuration files for the "baseline-action_repeat_2" setting (already set up as an example).
- Create a yaml file (you can copy one of the existing ones in
/configs/non_active_configs/). Keep in mind to setup wandb and rendering, otherwise simply set those parameters to "False". - (Optional) If you want to recreate the seeds, simply place the yaml file in
/configs/seed_script/and run the seed-generator helper script in/src/helper_scripts/seed_config_generator.py. This will automatically create the 5 seed configs in the active-config directory. - (If seeding not used) Place the yaml file as already stated into the
/configs/active_configs/section. - Run Dreamer-V1 via
/src/dreamerv1/run_training.py.
Note: For PPO and SAC, use the respective yaml files (also given in "non-active-configs" and run the respective algorithm from /src/stable_baselines_algo/).
The general process is described in the Setup-section but there are a few tweaks built in.
First of all, the PITA configuration yamls are given in /src/shared/pita_world/pita_yamls/. In the yaml configuration for the algorthms, PITA can be
activated and the paths to the pita-yaml can be set. Since no seeding was used during PITA environment creation, the environments used for all experiments
are given in /pita_envs_used/. A README.md inside the folder describes which one to use for reproducibility and how to use them.
Inside the "run_training.py" of each algorithm is a parameter called "run_active_configs". The parameter decides if the yaml files in the active-configs directory should be used.
For testing, a directory /configs/test_config/ is available and can be activated by setting the mentioned parameter to "False".
The idea behind this feature is to skip the "archive results" functionality that would pollute the archive that is created automatically. To understand more about this, the following section
describes the outputs generated and what to expect when doing a run.
When running an experiment through the active-run-config directory, several outputs are generated and archived automatically after succesful runs.
Please note: If testing is activated, those outputs are not archived.
- A directory in the archive is created with the name of the yaml file used and all following outputs are saved there.
- Configuration yaml file.
- Weights are saved.
- Wandb run is saved.
- Graph images of the network flows.
- Lidar Maps: Original LIDAR BVE (bird's eye view) map, dynamics learning reconstructions and behavior learning (imaginary phase) reconstructions.
- Videos of the episodes (if rendering was enabled).
- "output_world_xmls" that contain the environments used throughout the run.
The figures generated for the thesis are done with the /src/helper_scripts/plotting.py script. The script allows different
modes to plot, but mainly the default settings given are used. For plotting, simply adjust the following code block:
else:
include_runs = ["baseline", "action_repeat_2", "action_repeat_5"]
Visualization(include_runs, metrics).plot(plt_inst_show=plt_inst_show)The script searches the archive directory (automatically created with your runs), looks for the runs with the given name in "include_runs" and therefore finds the 5 seed runs for this configuration. Afterwards, based on the 5 given directories of your run, the metrics defined in
metrics = ["reward", "collided_target", "is_out_of_bounds", "avg_dist_target_x",
"avg_dist_target_y", "avg_velocity_x", "avg_velocity_y"]will be generated based on the wandb.csv that sits inside the respective folders in the archive. Note that there is more information avalaible, e.g. all losses given during the training. They can be plotted aswell, simply look inside the wandb.csv file and add the respective column name to the metrics.
The plots always calculate the mean and confidence interval (95%) for the 5 seeds and for all given metrics.