Abstract

Accurate simulation of lunar environments is essential for advancing astronomical exploration. However, creating realistic lunar simulations is challenging due to limited access to real lunar surface data. To address this, we present MoonSim, a geometric and visual lunar simulator designed for robotic learning. First, we develop a texture synthesis pipeline that leverages both real lunar imagery and physics-based material properties to create diverse, authentic surface textures. Second, we introduce a geometry bank containing 64 unique terrain that capture the complex topographical features of the lunar surface. Third, we implement a hybrid simulation framework that combines MuJoCo's precise physics engine with Unreal Engine's photorealistic rendering capabilities. The resulting system delivers high-fidelity visual and physical simulation of lunar environments while offering extensive controllability over environmental parameters, including surface properties, lighting conditions, and terrain configurations. This high degree of customization allows systematic variation in simulation conditions. MoonSim provides a versatile platform that supports a wide range of lunar robotics research, from locomotion to navigation.

MoonSim: A Photorealistic Lunar Environment Simulator

MoonSim is a photorealistic lunar environment simulator designed for robotics learning for lunar exploration. MoonSim is built using raw geometry data and real-world lunar captures from actual missions. By recreating the moon's environment in Unreal Engine, MoonSim generates high-fidelity visuals. MoonSim is integrated with MuJoCo to enable the training of robotic policies for locomotion and navigation, applied to robots such as quadruped robots and rovers.

A Gallery of MoonSim

MoonSim bridges the sim-to-real gap by providing a rich textural bank distilled from real-world space mission data. To simulate diverse lunar terrains, we created 64 unique moon terrains in the form of meshes. Photorealistic renderings are generated in Unreal Engine to achieve advanced lighting and sunlight control.

Our proposed framework

We utilize handcrafted lunar terrain geometry and adapt diffusion models to learn textural information from real captured data. From these assets, we build the MoonSim environment within MuJoCo for robotic simulation. For photorealistic renderings required for visual policy observations, we integrate a rendering wrapper based on Unreal Engine. This setup supports vision-based locomotion and navigation policy training for reinforcement learning methods.

Texture Bank

(Left) Overview of our texture synthesis pipeline, which uses diffusion models and text inversion to generate diverse lunar textures from limited space mission data. (Right) Comparison between our method and baseline diffusion model, demonstrating improved surface detail and terrain feature generation with learned lunar-specific tokens.

Rendering Wrappers of MoonSim