This is the proposal for an internship of about 3 month in Biorobotics Lab of TU Delft [1]. This internship could be appropriate for a master student who wants to have a hands-on experience in robotics. Starting soon is important, but the duration is flexible.
Robot Leo [1] learns to walk by means of Reinforcement Learning (RL). He has two hips, two knees, two ankles and one shoulder motors with embedded encoders, and one torso encoder.
We have developed a framework for doing reinforcement learning on a real robot with a minimal amount of real experience to reduce hardware damage. The idea is to use an approximate first-principles model of the robot and to learn a difference model which bridges the gap between the approximate model and the real robot. So far all the work has been done in simulations and promising results were obtained.
Now we are looking for a student who might be interested in doing the real experiment with Leo. The experiment will involve several steps:
-
Reproduce simulation results on our server using Nvidia CUDA GPU, Tensorflow and deep learning. The code is available, the student just needs to learn it by reproducing the results.
-
Learn about control loop of the real robot. Current implementation controls the robot using Ethernet, but on-board control is also possible if student wants to program more. Programming can be done in real-time OS if wanted.
-
Make the learning on Leo possible and obtain experimental data. This part will require programming in C++ and maybe some Python.
-
The risk of the project is related to the risk of gear boxes failures, which will require repairs. We have repair kits available, but it will take some time, efforts and knowledge of mechatronics. Support of a mechanical engineer is available in our department.
This is a purely engineering project, but it will require understanding of the underlying algorithm after reading corresponding literature (scientific papers, master thesis, e.g. [3]). In addition, the project requires a great interest in working with real robots, some knowledge of C++/Python and embedded systems. Skills to be learnt/improved include programming skills, Deep Learning framework, Nvidia CUDA GPU usage, hardware interfaces, hardware calibration, etc.
Please, send an e-mail with short motivation and recent school grades attached to Ivan Koryakovskiy (i.koryakovskiy@tudelft.nl)
[1] https://www.tudelft.nl/3me/organisatie/afdelingen/biomechanical-engineering/research/dbl-delft-biorobotics-lab/
[2] https://www.youtube.com/watch?v=SBf5-eF-EIw
[3] https://arxiv.org/abs/1509.02971