Dynamic Shotgun Microphone
As a team of 5, in 6 months, we created a product to capture individual audio streams for each human in a shot without lapel mics.
The system uses a camera and ML algorithms to track the location of all humans and then ML based beam forming algorithms to steer the sound capture of an array of onboard microphones in real-time.
​
As the team and software lead, I defined the product scope based on timelines and customer interviews. I also developed the Python backend to process video in real time and communicate with the React front-end and the audio stack
​
​
Design Overview
​​​Software Architecture​​
​​​Electrical Architecture​​
​​​Mechanical Architecture​​
Related Project Links
I was the Engineering Lead of the University of Waterloo's Formula Electric team (FSAE). I lead the engineering efforts of the 2018 vehicle, worked toward establishing a new team culture, and picked up on any project that needed extra hands.
When learning more about modern control theory, I wanted to try a more challenging project that I had wanted to design for a while. Using Simulink, I designed a simulation along with an LQR controller to balance a double pendulum even when subjected to noise.
I tested robot localization algorithms in ROS (robot operating system) including particle filter based localization using a Kalman filter on state change and LiDAR based scan matching using ICP for prediction and the ICP fitness score for resampling
As part of a team of 4, we are developing an underwater vehicle to travers an obstacle course in the fastest time possible. My primary roll is to develop the GNC (guidance, navigation and controls) stack running on an onboard android phone