Course project for Robot Perception and Learning (A+). We combined VLMs and imitation learning so people can guide a robot intuitively.
I built a React web app to collect human-corrected robot trajectories as training data. Along the way we found that VLMs struggle to understand 3D, and we found limits to what imitation learning could do here. Slides.