Deep Reinforcement Learning Nanodegree Projects
Udacity / NVIDIA Institute, March 2026
Three deep RL projects covering navigation, continuous control, and multi-agent collaboration, completed as part of the Deep Reinforcement Learning Nanodegree (Udacity / NVIDIA Institute). Navigation: trained a DQN agent to collect yellow bananas while avoiding blue ones in a large 3D environment. Continuous Control: trained a double-jointed arm (Reacher) to follow a target location using PPO. Collaboration: trained two agents to play tennis cooperatively using Multi-Agent PPO, maximizing the number of consecutive volleys.
Keywords: Deep Reinforcement Learning, DQN, PPO, Multi-Agent PPO, Unity ML-Agents, Python
Humanoid Control with Parallel Mechanisms
LAAS-CNRS Gepetto, 2024–2025
Developed a complete framework for controlling humanoid robots equipped with closed-kinematic actuators (parallel knee and ankle mechanisms). The approach introduces a compact differential analytical model of the transmission that enables efficient trajectory optimization via Crocoddyl/FDDP and the transfer of impedance gains from serial space to actuator space for RL deployment. Validated in simulation (MuJoCo, Isaac Lab) and on the real Bipetto robot.
Keywords: Optimal Control, Reinforcement Learning, Parallel Mechanisms, Pinocchio, Crocoddyl, Isaac Lab, MuJoCo, Python
Parkour Trajectory Optimization on Solo12
LAAS-CNRS Gepetto, 2024
Trajectory optimization for dynamic parkour motions on the Solo12 quadruped robot, pushing the robot to its joint limits through challenging multi-contact tasks. Uses constrained optimal control solver (CSQP) with collision avoidance integrated into the optimal control problem.
Keywords: Trajectory Optimization, Optimal Control, Quadruped Robotics, Python, Pinocchio, Crocoddyl
Robot Simulation Integration at PAL Robotics
PAL Robotics, Barcelona, 2023
Integration of humanoid and mobile manipulator robot simulations in MuJoCo. Bimanual workspace analysis using ROS and C++. Deployment tooling with Docker and Bash scripts for NVIDIA Jetson embedded boards.
Keywords: MuJoCo, ROS, C++, Python, Docker, NVIDIA Jetson