This repository serves as a log for my ongoing experiments exploring the intersection of robotics foundation models, synthetic data generation, and imbuing robots with character traits like context awareness, memory, and agentic behavior. Here, you'll find experimental notebooks, results, code snippets, and eventually, videos showcasing these explorations.
The initial focus is on building a self-improving robotics system, nicknamed where a robot can analyze its own performance, identify weaknesses, and automatically generate targeted training data to improve. As i learn more I will try to make the material more accessible and in a course format, so anyone with passion for robotics can from zero to hero in building their own AI driven robots.
This material is part of in person course on AI driven Robotics at Hassso Plattner Institute, Postdam, Germany. A full online version will be released in the fall 2025. The course will be built over the coming months. It will likely consist of 5 modules with a total of 10 sessions (1-3 sessions per module). Each module will include videos, scripts, notebooks, and Colab resources. The course is WIP and the structure is subject to change.
We will use lerobot from HuggingFace as the middleware and SO100/SO101 as practical robots to build on lessons and experiment. However, you can apply these same lessons to any robot and embodiments.
- Lesson 1 - Introduction: All that is gold does not glitter, not all those who wander are lost. Not every robot that dances can handle manipulation.
- Notebook, Script, and Video
- Lesson 2 - Gemini Robotics: Training an agentic robotic arm as your assistant
- Lesson 3: Foundational models: Covering PI0 series, GROOT N1, Gemini Robotics
- Notebook: Flow Matching Pi0, Script, and Video: Pi0, GROOT N1, Gemini Robotics
- Lesson 4: Synthetic data and simulations
This course is designed for:
- AI engineers or indie hackers who want to get started with robotics
- Technical managers curious about physical AI and the oncoming robotics revolution
- Developers of models, APIs, and tools exploring how to expand their current infrastructure for robotics use cases
If interested in partnerships, sponsorship, and offering student credits, please get in touch at [email protected] or via x @shreyasgite or linkedin via a DM.