I took a Machine Learning course at Ithaca College in the spring of 2019. The course covered the math and theory behind machine learning. For my final research project, I decided to research and develop a machine learning model for an autonomous drone hovering in a simulated environment.

For the project, I used Unity Engine’s PhysX for the environment simulation and the Unity ML-agents toolkit for training and running my models. The machine learning algorithm I used was Proximal Policy Optimization, which is a neural-network algorithm provided by the ML-agents toolkit.

Each drone was built of a body and four equidistant thrusters. The thrusters were each controlled by a continuous float value that translated into thrust output.

The model observed the values of the drone’s rotation, velocity, angular velocity, and its distance from the center of its learning space. The model used these values as inputs into the machine learning algorithm. The algorithm then outputs four float values that were each mapped to the drone’s thrusters.

After a lot of experimentation with hyperparameters and several tweaks to the learning environment, I was able to create a self-flying drone that could withstand small impacts from any direction. Early iterations could hover but would crash easy. To create a more robust flight correction system, external collisions were added in the form of projectiles to help the algorithm learn how to self-stabilize from unexpected forces.