05 Fakultät Informatik, Elektrotechnik und Informationstechnik
Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/6
Browse
13 results
Search Results
Item Open Access Driver alertness monitoring using steering, lane keeping and eye tracking data under real driving conditions(2020) Friedrichs, Fabian; Yang, Bin (Prof. Dr.-Ing.)Since humans operate trains, vehicles, aircrafts and industrial machinery, fatigue has always been one of the major causes of accidents. Experts assert that sleepiness is among the major causes of severe road accidents. In-vehicle fatigue detection has been a research topic since the early 80’s. Most approaches are based on driving simulator studies, but do not properly work under real driving conditions. The Mercedes-Benz ATTENTION ASSIST is the first highly sophisticated series equipment driver assistance system on the market that detects early signs of fatigue. Seven years of research and development with an unparalleled demand of resources were necessary for its series introduction in 2009 for passenger cars and 2012 for busses. The system analyzes the driving behavior and issues a warning to sleepy drivers. Essentially, this system extracts a single measure (so-called feature), the steering event rate by detecting a characteristic pattern in the steering wheel angle signal. This pattern is principally described by a steering pause followed by a sudden correction. Various challenges had to be tackled for the series-production readiness, such as handling individual driving styles and external influences from the road, traffic and weather. Fuzzy logic, driving style detection, road condition detection, change of driver detection, fixed-point parameter optimization and sensor surveillance were some of the side results from this thesis that were essential for the system’s maturity. Simply issuing warnings to sleepy drivers is faintly "experiencable" nor transparent. Thus, the next version 2.0 of the system was the introduction of the more vivid ATTENTION LEVEL, which is a permanently available bargraph monitoring the current driving performance. The algorithm is another result of this thesis and was introduced 2013 in the new S-Class. Fatigue is very difficult to grasp since a ground truth reference does not exist. Thus, the presented findings about camera-based driver monitoring are included as fatigue reference for algorithm training. Concurrently, the presented results build the basis for eye-monitoring cameras of the future generation of such systems. The driver monitoring camera will also play a key role in "automated driving" since it is necessary to know if the driver looks to the road while the vehicle is driving and if he is alert enough to take back control over the vehicle in complex situations. All these improvements represent major steps towards the paradigm of crash free driving. In order to develop and improve the ATTENTION ASSIST, the central goal of the present work was the development of pattern detection and classification algorithms to detect fatigue from driving sensors. One major approach to achieve a sufficiently high detection rate while maintaining the false alarm rate at a minimum was the incorporation of further patterns with sleepiness-associative ability. Features reported in literature were assessed as well as improved extraction techniques. Various new features were proposed for their applicability under real-road conditions. The mentioned steering pattern detection is the most important feature and was further optimized. Essential series sensor signals, available in most today’s vehicles were considered, such as steering wheel angle, lateral and longitudinal acceleration, yaw rate, wheel rotation rate, acceleration pedal, wheel suspension level, and vehicle operation. Another focus was on the lateral control using camera-based lane data. Under real driving conditions, the effects of sleepiness on the driving performance are very small and severely obscured by external influences such as road condition, curvature, cross-wind, vehicle speed, traffic, steering parameters etc. Furthermore, drivers also have very different individual driving styles. Short-term distraction from vehicle operation also has a big impact on the driving behavior. Proposals are given on how to incorporate such factors. Since lane features require an optional tracking camera, a proposal is made on how to estimate some lane deviation features from only inertial sensory by means of an extended Kalman filter. Every feature is related to a number of parameters and implementation details. A highly accelerated method for parameter optimization of the large amount of data is presented and applied to the most promising features. The alpha-spindle rate from the Electroencephalogram (EEG) and Electrooculogram (EOG) were assessed for their performance under real driving conditions. In contrast to the majority of results in literature, EEG was not observed to contribute any useful information to the fatigue reference (except for two drives with microsleeps). Generally, the subjective self-assessments according to the Karolinska Sleepiness Scale and a three level warning acceptance question were consequently used. Various correlation measures and statistical test were used to assess the correlation of features with the reference. This thesis is based on a database with over 27,000 drives that accumulate to over 1.5 mio km of real-road drives. In addition, various supervised real-road driving studies were conducted that involve advanced fatigue levels. The fusion of features is performed by different classifiers like Artificial Neural Networks (ANN) and Support Vector Machines (SVM). Fair classification results are achieved with ANN and SVM using cross-validation. A selection of the most potential and independent features is given based on automatic SFFS feature selection. Classical machine learning methods are used in order to yield maximal system transparency and since the algorithms are targeted to run in present control units. The potential of using end-to-end deep learning algorithms is discussed. Whereas its application to CAN-signals is problematic, there is a high potential for driver-camera based approaches. Finally, features were implemented in a real-time demonstrator using an own CAN-interface framework. While various findings are already rolled out in ATTENTION ASSIST 1.0, 2.0 and ATTENTION LEVEL, it was shown that further improvements are possible by incorporating a selection of steering- and lane-based features and sophisticated classifiers. The problem can only be solved on a system level considering all topics discussed in this thesis. After decades of research, it must be recognized that the limitations of indirect methods have been reached. Especially in view of emerging automated driving, direct methods like eye-tracking must be considered and have shown the greatest potential.Item Open Access Improving automotive radar spectra object classification using deep learning and multi-class uncertainty calibration(2022) Patel, Kanil; Yang, Bin (Prof. Dr.-Ing.)Being a prerequisite for successful automated driving, ameliorating the perception capabilities of vehicles is of paramount importance for reliable and robust scene understanding. Required for decision-making in autonomous vehicles, scene understanding becomes particularly challenging in adverse weather and lighting conditions; situations also often posing challenges for human drivers. Automotive radars can greatly assist sensors currently deployed on vehicles for robust measurements, especially in challenging conditions where other sensors often fail to operate reliably. However, classification using radar sensors is often limited to a few classes (e.g. cars, humans, and stop signs), controlled laboratory settings, and/or simulations. Already offering reliable distance, azimuth and velocity estimates of the objects in the scene, improving radar-based classification greatly expands the usage of radar sensors for tackling multiple driving-related tasks which are often performed by other less robust sensors. This thesis investigates how automated driving perception can be improved using multi-class radar classification by using deep learning algorithms for exploiting object class characteristics captured in the radar spectra. Despite the highly-accurate predictions of deep learning models, such classifiers exhibit severe over-confidence which can lead decision-making systems to false conclusions, with possibly catastrophic consequences - often a matter of life and death for automated driving. Consequently, high-quality, robust, and interpretable uncertainty estimates are indispensable characteristics of any unassailable automated driving system. With the goal of uncertainty estimates for real-time predictive systems, this thesis also aims at tackling the prominent over-confidence of deep learning classification models, which persists for all data modalities. Being an important measure for the quality of uncertainty estimates, this work focuses on the accurate estimation of the calibration of trained classifiers, as well as present novel techniques for improving their calibration. The presented solutions offer high-quality real-time confidence estimates for classification models of all data modalities (e.g. non-radar applications), as well as classifiers which are already trained and used in practise and new training strategies for learning new classifiers. Furthermore, the presented uncertainty calibration algorithms could also be extended to tasks other than classification, for example, regression and segmentation. On a challenging new realistic automated driving radar dataset, the solutions proposed in this thesis show that radar classifiers are able to generalize to novel driving environments, driving patterns, and object instances in realistic static driving scenes. To further replicate realistic encounters of autonomous vehicles, we study the behaviour of the classifiers to spectra corruptions and outlier detection of unknown objects, showing significant performance improvements in safely handling these prevalent encounters through accurate uncertainty estimates. With the proposed generalization and requisite accurate uncertainty estimation techniques, the radar classifiers in this study greatly improve radar-based perception for scene understanding and lay a solid foundation for current sensor fusion techniques to leverage radar measurements for object classification.Item Open Access Rehearsal-based continual learning with deep neural networks for image classification(2024) Wiewel, Felix; Yang, Bin (Prof. Dr.-Ing.)Item Open Access Least-squares based layerwise pruning of Deep Neural Networks(2024) Mauch, Lukas; Yang, Bin (Prof. Dr.-Ing.)Tiefe Neuronale Netze (DNNs) sind derzeit die leistungsstärksten Modelle im Bereich des maschinellen Lernens und lösen erfolgreich viele Aufgaben, wie zum Beispiel Bild- und Spracherkennung, semantische Segmentierung oder Datengenerierung. Aufgrund der inhärent hohen Rechenkomplexität von DNNs wurden schon früh Pruningverfahren angewandt um die Rechenkomplexität von DNNs zu reduzieren und um die Inferenz zu beschleunigen. Pruningverfahren entfernen (prunen) Parameter aus einem trainierten DNN, ohne ihre Leistung dadurch signifikant zu beeinträchtigen. Die dadurch erhaltenen Modelle können auch auf schwachen Rechenplattformen mit hoher Geschwindigkeit ausgewertet werden. In den letzten Jahren wurden Pruningverfahren nicht nur nach dem Training, sondern auch als Bestandteil von modernen Trainingsalgorithmen für DNNs eingesetzt. So wenden zum Beispiel viele speichereffiziente Trainingsalgorithmen oder Architektursuchverfahren pruning schon während des Trainings an, um unwichtige Parameter aus dem DNN zu entfernen. Problematisch ist, dass viele moderne Pruningverfahren auf regularisierten, überwachten Trainingverfahren beruhen und daher selbst sehr rechenaufwändig sind. Solche Pruningverfahren können nicht ohne Weiteres in andere Trainingsalgorithmen eingebettet werden. Es besteht daher ein wachsendes Interesse an Pruningmethoden, die sowohl schnell als auch genau sind. In dieser Arbeit untersuchen wir das layerbasierte Least-Squares (LS) Pruning – ein Framework für das strukturierte Pruning von DNNs. Wir zeigen, dass LS-Pruning eine schnelleund dennoch genaue Methode für die DNN-reduktion ist, die für Zero-Shot oder für die unüberwachte Netzwerkreduktion verwendet werden kann. In experimenten vergleichen wir LS-Pruning mit anderen schnellen Reduktionsmethoden, wie zum Beispiel dem magnitudenbasierten Pruning und der LS-Faktorisierung. Darüber hinaus vergleichen wir LS-Pruning mit überwachten Pruningverfahren.Item Open Access Deep open set recognition using dynamic intra-class splitting(2020) Schlachter, Patrick; Liao, Yiwen; Yang, BinThis paper provides a generic deep learning method to solve open set recognition problems. In open set recognition, only samples of a limited number of known classes are given for training. During inference, an open set recognizer must not only correctly classify samples from known classes, but also reject samples from unknown classes. Due to these specific requirements, conventional deep learning models that assume a closed set environment cannot be used. Therefore, special open set approaches were taken, including variants of support vector machines and generation-based state-of-the-art methods which model unknown classes by generated samples. In contrast, our proposed method models unknown classes by atypical subsets of training samples. The subsets are obtained through intra-class splitting (ICS). Based on a recently proposed two-stage algorithm using ICS, we propose a one-stage method based on alternating between ICS and the training of a deep neural network. Finally, several experiments were conducted to compare our proposed method with conventional and other state-of-the-art methods. The proposed method based on dynamic ICS showed a comparable or better performance than all considered existing methods regarding balanced accuracy.Item Open Access Behavior-aware pedestrian trajectory prediction in ego-centric camera views with spatio-temporal ego-motion estimation(2023) Czech, Phillip; Braun, Markus; Kreßel, Ulrich; Yang, BinWith the ongoing development of automated driving systems, the crucial task of predicting pedestrian behavior is attracting growing attention. The prediction of future pedestrian trajectories from the ego-vehicle camera perspective is particularly challenging due to the dynamically changing scene. Therefore, we present Behavior-Aware Pedestrian Trajectory Prediction (BA-PTP), a novel approach to pedestrian trajectory prediction for ego-centric camera views. It incorporates behavioral features extracted from real-world traffic scene observations such as the body and head orientation of pedestrians, as well as their pose, in addition to positional information from body and head bounding boxes. For each input modality, we employed independent encoding streams that are combined through a modality attention mechanism. To account for the ego-motion of the camera in an ego-centric view, we introduced Spatio-Temporal Ego-Motion Module (STEMM), a novel approach to ego-motion prediction. Compared to the related works, it utilizes spatial goal points of the ego-vehicle that are sampled from its intended route. We experimentally validated the effectiveness of our approach using two datasets for pedestrian behavior prediction in urban traffic scenes. Based on ablation studies, we show the advantages of incorporating different behavioral features for pedestrian trajectory prediction in the image plane. Moreover, we demonstrate the benefit of integrating STEMM into our pedestrian trajectory prediction method, BA-PTP. BA-PTP achieves state-of-the-art performance on the PIE dataset, outperforming prior work by 7% in MSE-1.5 s and CMSE as well as 9% in CFMSE.Item Open Access Image preprocessing for outdoor luminescence inspection of large photovoltaic parks(2021) Kölblin, Pascal; Bartler, Alexander; Füller, MarvinElectroluminescence (EL) measurements allow one to detect damages and/or defective parts in photovoltaic systems. In principle, it seems possible to predict the complete current/voltage curve from such pictures even automatically. However, such a precise analysis requires image corrections and calibrations, because vignetting and lens distortion cause signal and spatial distortions. Earlier works on crystalline silicon modules used the cell gap joints (CGJ) as calibration pattern. Unfortunately, this procedure fails if the detection of the gaps is not accurate or if the contrast in the images is low. Here, we enhance the automated camera calibration algorithm with a reliable pattern detection and analyze quantitatively the quality of the process. Our method uses an iterative Hough transform to detect line structures and uses three key figures (KF) to separate detected busbars from cell gaps. This method allows a reliable identification of all cell gaps, even in noisy images or if disconnected edges in PV cells exist or potential induced degradation leads to a low contrast between active cell area and background. In our dataset, a subset of 30 EL images (72 cell each) forming grid (5×11) lead to consistent calibration results. We apply the calibration process to 997 single module EL images of PV modules and evaluate our results with a random subset of 40 images. After lens distortion correction and perspective correction, we analyze the residual deviation between ideal target grid points and the previously detected CGJ after applied distortion and perspective correction. For all of the 2200 control points in the 40 evaluation images, we achieve a deviation of less than or equal to 3 pixels. For 50% of the control points, a deviation of of less than or equal to 1 pixel is reached.Item Open Access Avoiding shortcut-learning by mutual information minimization in deep learning-based image processing(2023) Fay, Louisa; Cobos, Erick; Yang, Bin; Gatidis, Sergios; Küstner, ThomasItem Open Access Machine learning for end-use electrical energy monitoring(2021) Barsim, Karim Said; Yang, Bin (Prof. Dr.-Ing.)Promoting end-users awareness of their usage and consumption of energy is one of the main measures towards achieving energy efficiency in buildings, which is one of the main targets in climate-aware energy transition programs. End-use energy disaggregation and monitoring is a practical and efficient approach towards achieving the targeted awareness of energy users by providing them with real-time fine-grained feedback about their own usage of energy. In this work, we address the case of electrical energy and the problem of end-use load monitoring and disaggregation in a variety of machine learning paradigms. This work starts from unsupervised energy disaggregation based on simple constraints and assumptions without the need for labeled training data. We then study and propose semi-supervised disaggregation approaches that learn from labeled observations, but are also capable of compensating for the scarcity of labeled data by leveraging unlabeled measurements. Finally, we propose a generic neural architecture for data-driven disaggregation upon availability of an abundance of training data. Results from this work not only assert the feasibility of end-use energy disaggregation, but also propose efficient models that adapt to the availability of labeled data, and are capable of monitoring different categories of end-use loads.Item Open Access Algorithms for event-based non-intrusive load monitoring(2024) Liebgott, Florian; Yang, Bin (Prof. Dr.-Ing.)