Data-driven methods are ubiquitous in today’s autonomous systems. An important task of environmental perception is the detection, classification, and tracking of relevant objects in the scene. We are particularly interested in environment perception using point cloud like data (e.g. lidar) in combination with video.

Today’s perception algorithms are based on deep neural networks and usually are trained to equally weight errors for each object in a scene, independent of its potential effects on the driving task. However, in reality there are objects which are more and less relevant. As a result, the trained network which is considered to perform best according to the metrics, not necessarily is the best one to be deployed. The goal of this research is to 1) develop new ways of assessing the relevance for all parts of the scene, 2) and assessing the current perception performance within these regions. In particular connecting the concepts of relevance and self-assessment to improve the correlation of training metrics and real-world performance will be the main focus.