Advancing deep generative models for improved visual defect recognition in optimized production environments with limited data
Date
2026
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Stuttgart : Fraunhofer-Institut für Produktionstechnik und Automatisierung IPA
Abstract
Die Landschaft der industriellen Fertigungsprozesse hat seit der ersten industriellen Revolution starke Veränderungen durchlaufen. Die jüngste Phase dieser Veränderung wird als vierte industrielle Revolution bezeichnet und ist durch vernetzte und autonome Systeme gekennzeichnet. Sie konzentriert sich auf die Verbesserung der Produktionseffizienz und Produktqualität durch fortschrittliche Informations- und Kommunikationstechnologien, die es Maschinen ermöglichen, sich eigenständig zu korrigieren und sich an unvorhergesehene Umstände anzupassen.
Eine zentrale Herausforderung ist in diesem Zusammenhang die Fehlererkennung, die entscheidend für die Qualitätskontrolle ist. Traditionelle manuelle Inspektionsmethoden sind zeitaufwändig und bergen das Risiko menschlicher Fehler. Infolgedessen haben sich automatisierte visuelle Prüfsysteme (engl. Automatic Vision Inspection systems, AVI) zu einer vielversprechenden Alternative entwickelt, die in Echtzeit konsistente und messbare Mechanismen zur Fehlererkennung bietet.
Allerdings hängt der Erfolg solcher Systeme stark von fortschrittlichen Softwarealgorithmen ab, insbesondere von solchen, die tiefes Lernen (engl. deep learning) nutzen. Diese Algorithmen erfordern große Mengen gut ausbalancierter Trainingsdaten, die aufgrund von optimierten Produktionsumgebungen mit wenigen fehlerhaften Produkten häufig nicht verfügbar sind.
In dieser Arbeit wird der Mangel an Daten angegangen, indem tiefe generative Modelle (engl. Deep Generative Models, DGMs) eingesetzt werden, um synthetische Daten von fehlerhaften Produkten zu erzeugen. Diese synthetischen Daten werden dann genutzt, um die Leistung von deep learning-basierten Algorithmen zur Fehlererkennung zu verbessern. Die Tatsache, dass die generativen Modelle zur Erzeugung der synthetischen Daten üblicherweise ebenfalls erhebliche Datenmengen benötigen, stellt dabei eine große Herausforderung dar.
Um diese Herausforderung anzugehen, werden in der Arbeit zwei innovative DGMs vorgestellt. Das erste Modell, das als Defect-Transfer GAN (DT-GAN) bezeichnet wird, nutzt die Ähnlichkeiten von Fehlern bei unterschiedlichen Produkten, um den Datenbedarf bei einem einzelnen Produkt zu reduzieren. Das zweite Modell, das STyled LAYout Diffusion (STAY Diffusion) genannt wird, nutzt ein flexibles Bounding-Box-Layout zur Erzeugung von Defekten an vorgegebenen Stellen eines Produktes. Dadurch kann sichergestellt werden, dass Fehler ausschließlich dort generiert werden, wo sie auf dem Produkt auftreten können. Außerdem ermöglicht es, das Muster eines Defekts über verschiedene Produkte hinweg zu erlernen, was den Bedarf an Daten weiter reduziert.
Darüber hinaus wird die Wirksamkeit von synthetischen Daten, die von DGMs erzeugt werden, in dieser Arbeit gründlich untersucht. Die Analyse konzentriert sich auf den Nutzen dieser Daten für das Training von auf Deep Learning basierenden Systemen wie AVI-Systemen. Sie bietet Einblicke in die Diskrepanzen zwischen synthetischen und realen Daten, die trotz der raschen Fortschritte in DGMs bestehen bleiben. Um diese Lücke zwischen synthetischen und realen Daten zu überbrücken, wird eine neue Regularisierungstechnik vorgeschlagen. Diese Technik nutzt Wissen aus informativen vortrainierten Modellen und kombiniert es mit einigen realen Proben, um die Leistung bei diskriminativen Aufgaben wie der Fehlererkennung zu verbessern.
Alle in dieser Arbeit beschriebenen Modelle und Methoden wurden umfassend an verschiedenen akademischen und realen industriellen Datensätzen evaluiert. Die Ergebnisse zeigen, dass die vorgeschlagenen DGMs hochrealistische synthetische Daten erzeugen, die die Leistung von deep learning-basierten Systemen bei der Lösung von Fehlererkennungsaufgaben erheblich verbessern. Darüber hinaus steigert die Anwendung der vorgeschlagenen Regularisierungstechnik auf die von DGMs generierten Bilder deren Wirksamkeit als Trainingsdaten. Dies führt zu einer besseren Leistung der deep learning-basierten Fehlererkennungsmodelle, ohne dass Änderungen an den DGMs erforderlich sind.
The landscape of industrial manufacturing processes has seen remarkable changes since the first industrial revolution. The latest phase of this change is called the fourth industrial revolution and is characterized by interconnected and autonomous systems. It focuses on improving production efficiency and product quality through advanced information and communication technologies, allowing machines to self-correct and adapt to unforeseen circumstances. A key challenge in this context is defect recognition, which is crucial for quality control. Traditional manual inspection methods are time-consuming and carry the risk of human error. As a result, Automatic Vision Inspection (AVI) systems have emerged as a promising alternative, providing consistent and measurable mechanisms for defect detection in real-time. However, the success of such systems is heavily dependent on advanced software algorithms, especially those that utilize deep learning. These deep learning algorithms require large amounts of well-balanced training data, which are not available in many cases due to optimized production settings with few defective products. In this thesis, the lack of data is tackled by leveraging advanced Deep Generative Models (DGMs) to create synthetic data of defective products. This synthetic data is then used to improve the performance of deep learning-based defect recognition algorithms. However, the generative models used to create synthetic data also need a lot of training data. This requirement presents a major challenge. To address this challenge, two innovative DGMs are introduced in the thesis. The first model, Defect-Transfer GAN (DT-GAN), capitalizes on the similarities in defects among various products, effectively reducing the data requirements for individual products. The second model, named STyled LAYout Diffusion (STAY Diffusion), employs a flexible bounding box layout to generate defects at predetermined locations of a product. This feature ensures that defects are generated only where they can occur on a real product. Moreover, it allows for learning the pattern of a defect type across multiple products, which further decreases the need for extensive data. In addition, the effectiveness of synthetic data generated by DGMs is investigated thoroughly in this thesis. The analysis focuses on the utility of this data for training deep learning-based systems like AVI systems. It provides insights into the discrepancies between synthetic and real-world data that persist despite the rapid advancements in DGMs. To counteract this synthetic-to-real gap, a new regularization technique is proposed. This technique combines knowledge from informative pretrained models with a few real samples to enhance performance in discriminative tasks, such as defect recognition. All models and methods outlined in this thesis have undergone extensive evaluation on various academic and real-world industrial datasets. The findings demonstrate that the proposed DGMs produce highly realistic synthetic data, significantly improving the performance of deep learning-based models in solving defect recognition tasks. Moreover, applying the proposed regularization technique to the images generated by DGMs further enhances their effectiveness as training data. This leads to better performance of the deep learning-based recognition models without necessitating changes to the DGMs.
The landscape of industrial manufacturing processes has seen remarkable changes since the first industrial revolution. The latest phase of this change is called the fourth industrial revolution and is characterized by interconnected and autonomous systems. It focuses on improving production efficiency and product quality through advanced information and communication technologies, allowing machines to self-correct and adapt to unforeseen circumstances. A key challenge in this context is defect recognition, which is crucial for quality control. Traditional manual inspection methods are time-consuming and carry the risk of human error. As a result, Automatic Vision Inspection (AVI) systems have emerged as a promising alternative, providing consistent and measurable mechanisms for defect detection in real-time. However, the success of such systems is heavily dependent on advanced software algorithms, especially those that utilize deep learning. These deep learning algorithms require large amounts of well-balanced training data, which are not available in many cases due to optimized production settings with few defective products. In this thesis, the lack of data is tackled by leveraging advanced Deep Generative Models (DGMs) to create synthetic data of defective products. This synthetic data is then used to improve the performance of deep learning-based defect recognition algorithms. However, the generative models used to create synthetic data also need a lot of training data. This requirement presents a major challenge. To address this challenge, two innovative DGMs are introduced in the thesis. The first model, Defect-Transfer GAN (DT-GAN), capitalizes on the similarities in defects among various products, effectively reducing the data requirements for individual products. The second model, named STyled LAYout Diffusion (STAY Diffusion), employs a flexible bounding box layout to generate defects at predetermined locations of a product. This feature ensures that defects are generated only where they can occur on a real product. Moreover, it allows for learning the pattern of a defect type across multiple products, which further decreases the need for extensive data. In addition, the effectiveness of synthetic data generated by DGMs is investigated thoroughly in this thesis. The analysis focuses on the utility of this data for training deep learning-based systems like AVI systems. It provides insights into the discrepancies between synthetic and real-world data that persist despite the rapid advancements in DGMs. To counteract this synthetic-to-real gap, a new regularization technique is proposed. This technique combines knowledge from informative pretrained models with a few real samples to enhance performance in discriminative tasks, such as defect recognition. All models and methods outlined in this thesis have undergone extensive evaluation on various academic and real-world industrial datasets. The findings demonstrate that the proposed DGMs produce highly realistic synthetic data, significantly improving the performance of deep learning-based models in solving defect recognition tasks. Moreover, applying the proposed regularization technique to the images generated by DGMs further enhances their effectiveness as training data. This leads to better performance of the deep learning-based recognition models without necessitating changes to the DGMs.
Description
Keywords
Citation
Endorsement
Review
Supplemented By
Referenced By
Creative Commons license
Except where otherwised noted, this item's license is described as CC BY-NC-ND