SIU Translation Frankfurt Session 1: Self-Driving Cars: Deep Learning, Computer Vision and Sensor Fusion


Author: Vanessa Huebner Edited by: Burcu Anil Kirmizitas

On Thursday 26th of October, SIU Frankfurt welcomed Peter Skvarenina to our first SIU Translation event for the winter term. Peter is a specialist in machine learning and successfully kicked off the term with an instruction on how to program your own self-driving car with the little equipment of a single camera and a supercomputer in your trunk.

The concept of the self-driving car is based on deep learning, an interconnected artificial neural network with the ability to make predictions upon trained information. Once the deep neural network has learned the shape of trees, other cars on the road and street signs, it can predict the shape of unknown objects. This ability is inspired by the biological visual field maps in the retina and visual cortex and based on big data and GPUs. The so-called 'Convolutional Neural Networks' (CNN) process the provided images by using common image filters such as 'sharpen', 'blur' or 'edge detection' to optimise object recognition with little pre-processing of information. Peter explained that the system needs to train on pre-mapped tracks with changing light, weather or shadowing conditions and is provided with three cameras to record the steering angle of the moving car. This allows the model to adjust well to different routes, seasons and day times. To actually move the car on the street, multiple deep learning networks are interconnected to provide the system with object recognition and motion detection by the CNN and the prediction of the outcome of sequences, such as changes in speed, by the RNN, 'Recurrent Neural Networks'. In order to make the car safe and usable in real-life traffic pedestrians, bicycles or animals moving on the pavement and across the street need to be detected. This requires the use of additional sensors (Radar and LiDAR), making predictions on where the subject is and on where the subject will be in the next moment.


Peter gave an insight into the potential of deep learning and into how artificial intelligence can impact our everyday life. Although no fully automated car is currently driving on our roads, technology is rapidly progressing, making the car industry expecting the first self-driving car available already at the beginning of the next decade.