A very promising and well-argued account, this paper presents a novel approach to systematically testing and automatically detecting erroneous behaviors in deep neural network (DNN)-driven vehicles.

The purpose of the proposed research is to find a solution for the “incorrect/unexpected corner-case behaviors that can lead to dangerous consequences.” The authors claim that this can be achieved conceptually “by adding error-inducing inputs to the training dataset and also by possibly changing the model structure.” So, the proposed solution leverages the notion of neuron coverage “as a guidance mechanism for systematically exploring different types of car behaviors” and demonstrates that different image transformations lead to activating “different sets of neurons in self-driving car DNNs.” A result of combining these two observations, transformation-specific metamorphic relations between multiple executions of the tested DNN are being used to automatically detect erroneous corner-case behaviors.

The paper’s main contribution is the design of an approach “to automatically synthesize test cases that maximizes neuron coverage” by applying realistic image transformations, as well as implementing and testing it with “three top performing DNN models from the Udacity driving challenge.”

This is a very well-written paper with sufficient technical detail and a clear explanation of the approach and its results. It is positioned properly within the related work on this subject and includes an extensive list of references. The paper is a very good read for scholars, students, and engineers interested in self-driving cars and in methods for improving DNNs.