Using AI deep learning and computer vision for “no hands” driving at racetrack speeds

The Arrow SAM Car is so many things. It’s a groundbreaking demonstration of how technology provides disabled people with more freedom and choice so they can be the drivers of their own lives.

It’s a second chance for Sam Schmidt to be a race car driver again by using head-only controls more than 20 years after his near-fatal crash.

But fundamentally since its inception in 2013, the Arrow SAM Car is a high-performance engineering platform.

We use the car – now a 2021 McLaren 720S Spider - to develop and test new technology under the most demanding conditions.

Now Arrow engineers have pushed this prototyping platform beyond expectations by developing a unique steering system based on a machine learning implementation. Its head-pose deep learning algorithm tracks head orientation, using left or right rotation to steer.

Our new steering system replaces the original infrared cameras and motion capture-based system. It is a more natural driving approach, where the driver doesn’t need a helmet, glasses, or any other sensor-based object to steer. It will also be more generic since anyone can get into the car and drive.

Nvidia is a natural solution for the new system, which requires extremely high accuracy and real-time inferencing.

Orin’s system-on-a-chip (SoC) consists of 17 billion transistors. The Orin SoC integrates NVIDIA’s next-generation GPU architecture, Deep Learning Accelerator, Arm Hercules CPU cores and computer vision accelerators. Combined, they, deliver 200 trillion operations per second - essential for smooth, safe driving at racetrack speeds.

The new system has been successfully tested at progressively higher speeds at racetracks in Phoenix, the Goodwood Motor Circuit in West Sussex, UK and the historic runway at the Kennedy Space Center space shuttle complex.