Bringing Physics Simulation and World Models Together for Robotic Manipulation

Recent advances in learned world models have shown impressive ability to generate visually plausible futures from images and videos, but visual prediction alone is insufficient for interaction-rich robot manipulation. A model that predicts pixel flow may still miss contact dynamics, frictional contact constraints, and underactuated physics needed for reliable control. Thus, a natural question to ask is how the expertise of modern simulation development can enhance world model learning for better robotic control? This workshop will be discussing four questions at the intersection of world models, physics simulation, and control for manipulation.