06 Fakultät Luft- und Raumfahrttechnik und Geodäsie
Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/7
Browse
46 results
Search Results
Item Open Access Evaluierung generalisierter Gebäudegrundrisse in großen Maßstäben(2012) Filippovska, Yevgeniya; Fritsch, Dieter (Prof. Dr.-Ing. habil.)Bei der Erzeugung von Karten werden die darzustellenden räumlichen Objekte in Abhängigkeit des angestrebten Maßstabs ausgewählt, verändert und so arrangiert, dass deren Form und Verteilung zu einem bestmöglichen Verständnis der räumlichen Gegebenheiten führt. Dabei weist die kartographische Abbildung unvermeidliche und zuweilen tiefgreifende geometrische Veränderungen im Vergleich zur Realität auf, welche durch eine übergeordnete Kontrollinstanz zu verifizieren und bewerten sind. Hierfür strebt man eine formalisierte Qualitätsbewertung der Ergebnisse an, so dass sich entsprechende Prozesse, bevorzugt mit Hilfe automatisierter Werkzeuge, umsetzen lassen. Obwohl die Lesbarkeit der Gesamtkomposition einer Karte das Ziel ist, muss die Qualitätsbewertung zuerst auf der untersten Generalisierungsebene, der sogenannten Mikroebene erfolgen, indem die Geometrie- bzw. die Formveränderungen von Einzelobjekten bemessen werden. Neben dem Straßennetz dienen den Kartennutzern häufig vor allem markante Gebäude als Orientierungshilfe, welche aus diesem Grund nicht allzu großen Veränderungen unterliegen dürfen. Im Rahmen dieser Arbeit werden daher Qualitätscharakteristiken aufgezeigt, welche auf dem direkten Vergleich zweier Gebäudegrundrisse – Original und generalisiert – basieren. Die vorliegende Arbeit beginnt mit einer theoretischen Einführung in das Thema der Qualität von Geodaten. Anschließend wird ein Wahrnehmungstest vorgestellt, welcher die Bewertung generalisierter Grundrisse durch menschliche Betrachter vornimmt. Versuche diese Wahrnehmungsprozesse mathematisch zu formalisieren wird als Ähnlichkeitsschätzung bezeichnet, deren Grundlagen darauffolgend dargelegt sind. In diesem Zusammenhang wird eine einheitliche Klassifizierung der Objektmerkmale basierend auf der zugrundeliegenden Berechnungsmethode vorgeschlagen. Ein Überblick über die bislang zur Qualitätsbewertung der Generalisierung gelaufenen Forschungsarbeiten und eine kritische Auseinandersetzung dazu runden den derzeitigen Kenntnisstand zum Themengebiet ab. Daran anschließend werden neue Charakteristiken zur Ähnlichkeitsanalyse vorgestellt, welche die 2D-Gebäudeobjekte unter den Aspekten der Kontur- und Flächentreue hin vergleichen. Da eine Zuordnung zwischen den Formelementen allgemein nicht zweifelsfrei feststellbar ist, werden die Objekte geometrisch gemäß der Standardisierung von Geodaten als Punktmengen betrachtet. Dies erlaubt es, die geometrischen Berechnungen fast ausschließlich auf den Standardoperatoren der Mengentheorie aufzusetzen und mit den topologischen Algorithmen der Graphentheorie zu kombinieren. Zur Bewertung der Konturtreue werden Charakteristiken auf Basis der objektbildenden Randmengen aufgezeigt, welche Aufschluss über die maximale Abweichung und den Anteil der Überlappung gibt. Die Flächentreue wird unter einem quantitativen und einem räumlichen Aspekt betrachtet, wobei eine Differenzierung zwischen den Elementen der Strukturveränderungen vorgenommen wird. Um die Aussagekraft und Praxistauglichkeit der vorgeschlagenen Charakteristiken zu überprüfen, wird eine Evaluierung von generalisierten Gebäudegrundrissen auf der Mikro- und Makroebene durchgeführt. Dabei spielt insbesondere auch die anschauliche Präsentation der Ergebnisse eine zentrale Rolle, so dass verschiedene Möglichkeiten zur Darstellung der einzelnen Charakteristiken bezüglich einer guten Diskriminierbarkeit der Qualitätsangaben im Fokus stehen. Die Analyse der Ergebnisse zeigt, dass alle vorgeschlagenen Charakteristiken aussagekräftig sind und eine vielseitige Beschreibung verschiedener Qualitätsaspekte der Generalisierung in deren Gesamtheit ermöglichen.Item Open Access Forming a hybrid intelligence system by combining Active Learning and paid crowdsourcing for semantic 3D point cloud segmentation(2023) Kölle, Michael; Sörgel, Uwe (Prof. Dr.-Ing.)While in recent years tremendous advancements have been achieved in the development of supervised Machine Learning (ML) systems such as Convolutional Neural Networks (CNNs), still the most decisive factor for their performance is the quality of labeled training data from which the system is supposed to learn. This is why we advocate focusing more on methods to obtain such data, which we expect to be more sustainable than establishing ever new classifiers in the rapidly evolving ML field. In the geospatial domain, however, the generation process of training data for ML systems is still rather neglected in research, with typically experts ending up being occupied with such tedious labeling tasks. In our design of a system for the semantic interpretation of Airborne Laser Scanning (ALS) point clouds, we break with this convention and completely lift labeling obligations from experts. At the same time, human annotation is restricted to only those samples that actually justify manual inspection. This is accomplished by means of a hybrid intelligence system in which the machine, represented by an ML model, is actively and iteratively working together with the human component through Active Learning (AL), which acts as pointer to exactly such most decisive samples. Instead of having an expert label these samples, we propose to outsource this task to a large group of non-specialists, the crowd. But since it is rather unlikely that enough volunteers would participate in such crowdsourcing campaigns due to the tedious nature of labeling, we argue attracting workers by monetary incentives, i.e., we employ paid crowdsourcing. Relying on respective platforms, typically we have access to a vast pool of prospective workers, guaranteeing completion of jobs promptly. Thus, crowdworkers become human processing units that behave similarly to the electronic processing units of this hybrid intelligence system performing the tasks of the machine part. With respect to the latter, we do not only evaluate whether an AL-based pipeline works for the semantic segmentation of ALS point clouds, but also shed light on the question of why it works. As crucial components of our pipeline, we test and enhance different AL sampling strategies in conjunction with both a conventional feature-driven classifier as well as a data-driven CNN classification module. In this regard, we aim to select AL points in such a manner that samples are not only informative for the machine, but also feasible to be interpreted by non-experts. These theoretical formulations are verified by various experiments in which we replace the frequently assumed but highly unrealistic error-free oracle with simulated imperfect oracles we are always confronted with when working with humans. Furthermore, we find that the need for labeled data, which is already reduced through AL to a small fraction (typically ≪1 % of Passive Learning training points), can be even further minimized when we reuse information from a given source domain for the semantic enrichment of a specific target domain, i.e., we utilize AL as means for Domain Adaptation. As for the human component of our hybrid intelligence system, the special challenge we face is monetarily motivated workers with a wide variety of educational and cultural backgrounds as well as most different mindsets regarding the quality they are willing to deliver. Consequently, we are confronted with a great quality inhomogeneity in results received. Thus, when designing respective campaigns, special attention to quality control is required to be able to automatically reject submissions of low quality and to refine accepted contributions in the sense of the Wisdom of the Crowds principle. We further explore ways to support the crowd in labeling by experimenting with different data modalities (discretized point cloud vs. continuous textured 3D mesh surface), and also aim to shift the motivation from a purely extrinsic nature (i.e., payment) to a more intrinsic one, which we intend to trigger through gamification. Eventually, by casting these different concepts into the so-called CATEGORISE framework, we constitute the aspired hybrid intelligence system and employ it for the semantic enrichment of ALS point clouds of different characteristics, enabled through learning from the (paid) crowd.Item Open Access Making historical gyroscopes alive - 2D and 3D preservations by sensor fusion and open data access(2021) Fritsch, Dieter; Wagner, Jörg F.; Ceranski, Beate; Simon, Sven; Niklaus, Maria; Zhan, Kun; Mammadov, GasimThe preservation of cultural heritage assets of all kind is an important task for modern civilizations. This also includes tools and instruments that have been used in the previous decades and centuries. Along with the industrial revolution 200 years ago, mechanical and electrical technologies emerged, together with optical instruments. In the meantime, it is not only museums who showcase these developments, but also companies, universities, and private institutions. Gyroscopes are fascinating instruments with a history dating back 200 years. When J.G.F. Bohnenberger presented his machine to his students in 1810 at the University of Tuebingen, Germany, nobody could have foreseen that this fascinating development would be used for complex orientation and positioning. At the University of Stuttgart, Germany, a collection of 160 exhibits is available and in transition towards their sustainable future. Here, the systems are digitized in 2D, 2.5D, and 3D and are made available for a worldwide community using open access platforms. The technologies being used are computed tomography, computer vision, endoscopy, and photogrammetry. We present a novel workflow for combining voxel representations and colored point clouds, to create digital twins of the physical objects with 0.1 mm precision. This has not yet been investigated and is therefore pioneering work. Advantages and disadvantages are discussed and suggested work for the near future is outlined in this new and challenging field of tech heritage digitization.Item Open Access Design and development of a calibration solution feasible for series production of cameras for video-based driver-assistant systems(2022) Nekouei Shahraki, Mehrdad; Haala, Norbert (apl. Prof. Dr.)In this study, we reviewed the current techniques and methods in photogrammetry - especially close-range photogrammetry - and focused on camera calibration. We reviewed the new evolving field of video-based driver-assistant systems, their requirements and their applications. Exclusively of fisheye cameras and a general omnidirectional projection, we extended an existing camera calibration model to address our needs and functionality requirements. These extensions enable us to use the camera calibration model in real-time embedded mobile systems with low processing power. We also introduced the free-function model as a flexible and advantageous model for camera distortion modelling. This is a new approach for modelling the overall image distortion together with the local lens distortions that are estimated using a standard model during the calibration process. Using free-function model on different lens designs, one can achieve good calibration accuracies by modelling the very local lens distortion taking benefit from the flexibility of this model. We introduced optimization strategies for recalculation and image rectification. These optimizations are also used to minimize the amount of required processing power and device memory. This brings many advantages to variety of computational platforms such as FPGAs, x86 and ARM processors, and makes it possible to benefit from variety of parallel-processing techniques. This model is capable of being used in runtime and is an ideal calibration model for using in variety of machine vision solutions. We also discussed several important requirements for accurate camera calibration that we later used in hardware test stand design phase. We designed and developed two different test stands in order to realize the specifications and geometrical features of multiple-view test-field-based camera calibration referred to as bundle-block calibration. One of their special geometrical characteristics is the uniform point distribution, which corresponds to the uniform motion. Such a point distribution is beneficial when using calibration models such as free-function model that enable us to model of local lens distortion with good accuracy and quality all over the image. A very important feature of this test stand is having the capability of performing camera/sensor alignment testing, a feature which is very important for testing the geometrical alignment of the internal mechanical elements of each camera. Using automated machines and algorithms in test stand calibration increased the stability and accuracy of the calibration and thus ensured the quality and speed of the calibration for cameras. These test stands are capable of performing automatic camera calibration, suitable for applications such as series-production of cameras. As an accuracy -and flexibility evaluation step for the free-function model, we tested the free-function calibration model on real-world data using a stereo camera with added large local distortions taking images from a front vehicle similar to the conditions where real-world use-cases are defined. By performing the camera calibration, we compared the calibration results and accuracy parameters of the free-function model to a conventional calibration model. Using these calibration results, we generated a set of disparity maps and compared their density and availability, especially on the areas where the local distortion was present. We used this test to compare the capabilities of the proposed model to conventional ones in real-wold situations where large optical distortions could be present that cannot be easily modelled with conventional calibration models. The higher modelling capability and accuracy of the free-function model will generally influence those functions that are using the information of the disparity map or the derived 3D information as part of their input data and potentially leads to the better functionality or even their availability if local distortions are present in the image. There are many more use-cases in photogrammetry and computer-vision where a higher calibration accuracy is beneficial on hardware such as low-cost optics where sometimes optical distortion are available that cannot easily be modelled with classical models. These use-cases could all benefit from the flexibility and modelling accuracy of the free-function model.Item Open Access Nonlinear feature normalization for hyperspectral feature transfer(2019) Groß, Wolfgang; Sörgel, Uwe (Prof. Dr.-Ing.)Hyperspectral remote sensing is an important topic for deriving high-level information about the earth's surface. Applications include land cover mapping, precision farming, and the detection of environmental pollution. This is made possible by recording and evaluating narrow-band features that are characteristic of individual materials. External effects, however, lead to nonlinearities in the data and complicate data analysis. These effects include changes in illumination, hard and partial shadows, as well as transmission / multiple reflections by objects in the scene, and anisotropic effects for 3D objects. Correcting these effects is required for robust data analysis. In particular, when comparing multiple data sets a unified representation is required. Physically motivated models for correcting atmospheric in uences are generally used for the pre-processing of hyperspectral data. However, these models do not consider local variations, such as shadows and object geometry. Therefore, this thesis deals with data-driven approaches in the field of Manifold Alignment (MA) and Feature Transfer (FT) to transfer several data sets to a common system. Previous research on these topics has focused primarily on learning the underlying geometry of high-dimensional data and aligning multiple datasets by determining the minimum discrepancy while preserving the individual data structure. Usually, a common domain with very high dimensionality is chosen to facilitate the alignment. The transformation into another domain, however, prevents physical interpretability. Also, inversion of one data set from the common domain to the domain of a target data set is diffcult due to the pre-image problem.The contributions of this thesis can be divided into two categories. The Nonlinear Feature Normalization (NFN) is a data-driven approach to mitigate nonlinear effects in hyperspectral data. NFN is a supervised method and requires training samples for each class in the scene. A new basis for data representation is defined, consisting of one spectral reference signature per class. The training data are then used to individually shift all samples towards the new basis. This significantly reduces the effects of nonlinearities, as shown by comparing classification results before and after the NFN transformation. The NFN is then used to derive the Nonlinear Feature Normalization for Data Alignment (NFNalign). NFNalign transforms multiple data sets to the same basis in the common domain and then applies an inverse transformation to transfer data sets from the common domain to a domain of another data set. Since the dimensionality of the data is not changed during the transformation, it is possible to perform the inversion analytically. The functionality of NFNalign is demonstrated by transforming hyperspectral radiance data to reflection data. Thereby, the pre-processing step of the atmospheric correction can be replaced, shadows and other nonlinearities are corrected, and characteristic features of the spectral signatures are transferred. The quality of the alignment is demonstrated by applying an SVM model trained on a reference data set to the aligned data set. Additional alignment is assessed by applying a classification model trained on a reference data set to a test data set after it has been transformed to the domain of the reference with NFNalign. Further experiments investigate the robustness with regard to noise and errors in the training data as well as the alignment of data with different dimensions. Also, a comparison with common reference methods is performed. Overall, NFN and NFNalign provide a complete framework for hyperspectral data alignment and FT.Item Open Access Automatische Interpretation von Semantik aus digitalen Karten im World Wide Web(2014) Luo, Fen; Fritsch, Dieter (Prof. Dr.-Ing.)Im Internet befindet sich eine sehr große Menge an raumbezogenen Daten, die in Form von Raster- und Vektorkarten unterschiedliche Ausschnitte der Welt darstellen. Die in diesen Karten enthaltenen Informationen sind jedoch nicht automatisch auffindbar, da sie mittels bestimmter Kartenelemente kodiert sind. Ihre Semantik wird erst bei der Interpretation durch einen Betrachter explizit. Die Kar-teninformationen sollen jedoch nicht nur von Menschen, sondern auch von Maschinen interpretiert werden können. Dies erfordert schon die große Menge der zu interpretierenden Daten. Die automati-sche Ableitung der Semantik aus den Karten wird unter dem Begriff Automatische Karteninterpreta-tion zusammengefasst. Es handelt sich dabei also um einen Prozess, der implizites Wissen eines Kar-tenbestandes explizit macht. Hierzu soll die vorliegende Arbeit Lösungen in Form der Karteninterpre-tation anbieten. Die Karteninterpretation dieser Arbeit erfolgt an Vektorkarten, die im Internet zu finden sind. Für die gezielte Suche der Vektorkarten des Internets wird eigens ein Webcrawler entwickelt. Der Webcrawler ist eine Suchmaschine, die speziell nach Vektorkarten sucht. Dazu wird ausschließlich das Shapefile-Dateiformat gesucht, das sich zu einer Art Standardformat im GIS-Umfeld entwickelt hat und in dem die Vektorkarten zumeist abgespeichert sind. Um möglichst viele Shapefiles zu finden, wird die Suche auf Servern betrieben, auf denen die Wahrscheinlichkeit Shapefiles zu finden hoch ist. Diese Server werden zuvor durch Google-Suche nach dem Schlüsselwort „shapefile download“ gefunden. Die Karteninterpretation umfasst Verfahren zur Interpretation der Kartenobjekte, der Kartentypen so-wie des Maßstabs. Zunächst soll das Verfahren zur Interpretation der Objekte einer Karte vorgestellt werden. Hier geht es darum, die Objekte anhand ihrer spezifischen Charakteristika automatisch zu erkennen. Die Ob-jekterkennung basiert auf SOM (Self-Organizing Map), bekannt aus der künstlichen Intelligenz. Die Kartenobjekte werden in Klassen wie beispielsweise Gebäudegrundriss oder Straßennetz gegliedert. Für jede Klasse sollen die ihr jeweils eigenen Merkmale gefunden und in eine der SOM zugängliche Form, hier als Parametervektor, gebracht werden. Die Parametervektoren bilden die Eingabemuster, die in der Lernphase von SOM gelernt werden. Nachdem die Eingabemuster aller Objektklassen von SOM gelernt wurden, wird der Parametervektor für jedes auf der Karte vorliegende Objekt ausgewertet und in die SOM eingegeben. Durch das zunächst erfolgte Lernen der Eingabemuster können die Ob-jekte anhand ihrer jeweils berechneten Parametervektoren der entsprechenden Objektklasse zugeord-net werden. Als weiteres Verfahren soll die Interpretation des Kartentyps vorgestellt werden. Karten sind nach ihrem inhaltlichen Gehalt und Zweck in Kartentypen wie beispielsweise Flusskarten, Straßenkarten, Höhenlinienkarten etc. kategorisiert. Wie bei der Interpretation der Objekte wird auch hierzu die SOM verwandt. Es werden also auch Eingabemuster gelernt, die die geometrischen Merkmale der Karten-typen repräsentieren. Die Merkmale ergeben sich sowohl aus der Struktur der einzelnen Objekte als auch aus der Topologie zwischen den Objekten auf einer Karte. Wird nun eine Karte in die SOM eingegeben, so erkennt die SOM anhand des gelernten Eingabemusters den entsprechenden Kartentyp. Zusätzlich erhält man den Dateinamen der Karten sowie den Inhalt der Webseite, auf welcher die Karte gefunden wurde. So wird in der vorliegenden Arbeit ebenfalls untersucht, inwiefern diese Zusatzin-formationen bei der Interpretation des Kartentyps helfen können. Die automatische Interpretation des Maßstabs ist neben der Interpretation der Kartenobjekte und Kar-tentypen ein weiteres Verfahren, das in der vorliegenden Arbeit diskutiert werden soll. Die Interpreta-tion des Maßstabs wird auf zwei Wegen vorangetrieben: Die Mehrfachrepräsentation und die Detail-lierungsgrade. Im ersten Fall kann der Maßstab aus der entsprechenden Repräsentation hergeleitet werden, da ein identisches Objekt in unterschiedlichen realitätsgetreuen Repräsentationen auf der Karte dargestellt wird. Im zweiten Fall kann der Maßstab aus den Detaillierungsgraden abgeleitet wer-den. Dies basiert darauf, dass die Karten mit verschiedenen Maßstäben unterschiedlich detailliert dar-gestellt werden.Item Open Access Modellierung und Nutzung von Relationen zwischen Mehrfachrepräsentationen in Geo-Informationssystemen(2006) Volz, Steffen; Fritsch, Dieter (Prof. Dr.-Ing. habil.)Die wachsende Bedeutung von raumbezogenen Daten in den verschiedensten Anwendungsbereichen hat dazu geführt, dass zahlreiche Firmen und öffentliche Institutionen die Erfassung von Geodaten vorantreiben. Dabei werden ein und dieselben Realweltobjekte häufig mehrfach und aus unterschiedlichen Blickwinkeln erfasst und im Computer gespeichert. Auf diese Weise entstehen widersprüchliche bzw. inkonsistente Repräsentationen eines Objektes der Realwelt, so genannte Mehrfachrepräsentationen. Sollen diese Mehrfachrepräsentationen innerhalb von offenen Geodateninfrastrukturen, wie beispielsweise der an der Universität Stuttgart entwickelten Nexus-Plattform, bereitgestellt werden, so müssen einerseits die Datenschemas der heterogenen Ausgangsdaten zusammengeführt werden. Andererseits ist es notwendig, die Inkonsistenzen zwischen Mehrfachrepräsentationen adäquat behandeln zu können, um eine gemeinsame Datenverarbeitung zu ermöglichen. Die vorliegende Arbeit hat zum Ziel, einen Beitrag zur Forschung auf diesem Gebiet zu leisten. Im Verlauf der Untersuchung wird zunächst der Zusammenhang zwischen Interoperabilitätsbestrebungen internationaler und nationaler Institutionen und der Problematik der Mehrfachrepräsentationen aufgezeigt. Anschließend wird die Nexus-Plattform vorgestellt, da sie den Rahmen für die vorliegende Arbeit bildet. Zum Zwecke der Vermittlung von Grundlagen über die Thematik der Mehrfachrepräsentationen erfolgt eine umfassende Begriffsklärung und eine Aufarbeitung des Standes der Forschung. In der Folge wird eine Vorgehensweise präsentiert, um auf der Basis einer Untersuchung existierender Schemas für Geodaten ein übergeordnetes, globales Schema einer offenen Systemplattform ableiten zu können. Dabei werden auch die Applikationsanforderungen, denen das globale Schema zu genügen hat, berücksichtigt. Darüber hinaus werden Abbildungsregeln aufgestellt, die eine Übertragung von Daten der bestehenden Schemas in das übergeordnete Schema erlauben. Das grundlegende Konzept dieser Arbeit besteht darin, die innerhalb einer Geodateninfrastruktur auftretenden Mehrfachrepräsentationen über explizite Relationen miteinander zu verknüpfen. Zu diesem Zweck wird ein formales Modell für Relationen zwischen Mehrfachrepräsentationen eingeführt. Anhand von Testdaten werden entsprechende Relationen mit Hilfe eines semi-automatischen Verfahrens generiert. Im Mittelpunkt der Untersuchung steht zum einen, über eine Auswertung der erzeugten Relationen zwischen Mehrfachrepräsentationen eine automatische Ableitung von Korrelationsmaßen für korrespondierende Objektklassen unterschiedlicher Schemas zu ermöglichen. Zum anderen wird nachgewiesen, dass Netzwerkanalysen unter Anwendung des erarbeiteten Relationskonzeptes auf mehrfach repräsentierten Straßendaten möglich sind.Item Open Access Evaluation of Phase One scan station for analogue aerial image digitisation(2021) Schulz, Joachim; Cramer, Michael; Herbst, TheresaHistorical aerial photographs represent a special cultural asset for preserving information about land cover and land use change in the twentieth century with a high spatial and temporal resolution. A current topic is the digitisation of historical images to make them accessible to a wider range of users and to preserve them from age deterioration. For a photogrammetric evaluation, a high geometric stability and accuracy during the digitization process is required. In this work, the resolving power and geometric quality of a Phase One iXM-MV150F high-performance camera was investigated, which is used at the Landesamt für Geoinformation und Landentwicklung Baden-Württemberg in the project ‘Digitaler Luftbildatlas Baden-Württemberg’ for the digitisation of historical aerial photographs. The resolving power of the system was empirically measured and analysed. The required modulation transfer function was determined using Siemens stars. With this method, the significant influence of the focus setting and deviations of the plane-parallel alignment could be determined. Using a digitised aerial survey of the Vaihingen/Enz test field, the impact of the above-mentioned effects and the influence of the geometry of the scanning camera on the quality of the derived data products was shown in comparison to a photogrammetric scanner. The comparison showed that dedicated photogrammetric scanners still achieve a higher accuracy, even if a high-quality optical system is used for the digitising stand with the document camera. Further investigations are justified to improve the accuracy and stability of digitising the aerial image with a document camera.Item Open Access Mathematical methods for camera self-calibration in photogrammetry and computer vision(2013) Tang, Rongfu; Fritsch, Dieter (Prof. Dr.-Ing. habil.)Camera calibration is a central subject in photogrammetry and geometric computer vision. Self-calibration is a most flexible and highly useful technique, and it plays a significant role in camera automatic interior/exterior orientation and image-based reconstruction. This thesis study is to provide a mathematical, intensive and synthetic study on the camera self-calibration techniques in aerial photogrammetry, close range photogrammetry and computer vision. In aerial photogrammetry, many self-calibration additional parameters (APs) are used increasingly without evident mathematical or physical foundations, and moreover they may be highly correlated with other correction parameters. In close range photogrammetry, high correlations exist between different terms in the ‘standard’ Brown self-calibration model. The negative effects of those high correlations on self-calibration are not fully clear. While distortion compensation is essential in the photogrammetric self-calibration, geometric computer vision concerns auto-calibration (known as self-calibration as well) in calibrating the internal parameters, regardless of distortion and initial values of internal parameters. Although camera auto-calibration from N≥3 views has been studied extensively in the last decades, it remains quite a difficult problem so far. The mathematical principle of self-calibration models in photogrammetry is studied synthetically. It is pointed out that photogrammetric self-calibration (or building photogrammetric self-calibration models) can – to a large extent – be considered as a function approximation problem in mathematics. The unknown function of distortion can be approximated by a linear combination of specific mathematical basis functions. With algebraic polynomials being adopted, a whole family of Legendre self-calibration model is developed on the base of the orthogonal univariate Legendre polynomials. It is guaranteed by the Weierstrass theorem, that the distortion of any frame-format camera can be effectively calibrated by the Legendre model of proper degree. The Legendre model can be considered as a superior generalization of the historical polynomial models proposed by Ebner and Grün, to which the Legendre models of second and fourth orders should be preferred, respectively. However, from a mathemtical viewpoint, the algebraic polynomials are undesirable for self-calibration purpose due to high correlations between polynomial terms. These high correlations are exactly those occurring in the Brown model in close range photogrammetry. They are factually inherent in all self-calibration models using polynomial representation, independent of block geometry. According to the correlation analyses, a refined model of the in-plane distortion is proposed for close range camera calibration. After examining a number of mathematical basis functions, the Fourier series are suggested to be the theoretically optimal basis functions to build the self-calibration model in photogrammetry. Another family of Fourier self-calibration model is developed, whose mathematical foundations are the Laplace’s equation and the Fourier theorem. By considering the advantages and disvantages of the physical and the mathematical self-calibration models, it is recommended that the Legendre or the Fourier model should be combined with the radial distortion parameters in many calibration applications. A number of simulated and empirical tests are performed to evaluate the new self-calibration models. The airborne camera tests demonstrate that, both the Legendre and the Fourier self-calibration models are rigorous, flexible, generic and effective to calibrate the distortion of digital frame airborne cameras of large-, medium- and small-formats, mounted in single- and multi-head systems (including the DMC, DMC II, UltraCamX, UltraCamXp, DigiCAM cameras and so on). The advantages of the Fourier model result from the fact that it usually needs fewer APs and obtains more reliable distortion calibration. The tests in close range photogrammetry show that, although it is highly correlated with the decentering distortion parameters, the principal point can be reliably and precisely located in a self-calibration process under appropriate image configurations. The refined in-plane distortion model is advantageous in reducing correlations with the focal length and improving the calibration of it. The good performance of the combined “Radial + Legendre” and “Radial + Fourier” models is illustrated. In geometric computer vision, a new auto-calibration solution which needs image correspondences and zero (or known) skew parameter only is presented. This method is essentially based on the fundamental matrix and the three (dependent) constraints derived from the rank-2 essential matrix. The main virtues of this method are threefold. First, a recursive strategy is employed subsequently to a coordinate transformation. With an appropriate approximation, the recursion estimates the focal length and aspect ratio in advance and then calculates the principal point location. Second, the optimal geometric constraints are selected using error propagation analyses. Third, the final nonlinear optimization is performed on the four internal parameters via the Levenberg–Marquardt algorithm. This auto-calibration method is fast and efficient to obtain a unique calibration. Besides auto-calibration, a new idea is proposed to calibrate the focal length from two views without the knowledge of the principal point coordinates. Compared to the conventional two-view calibration techniques which have to know principal point shift a priori, this new analytical method is more flexible and more useful. Although the auto-calibration and the two-view calibration methods have not been fully mature yet, their good performance is demonstrated in both simulated and practical experiments. Discussions are made on future refinements. It is hoped that this thesis not only introduces the relevant mathematical principles into the practice of camera self-calibration, but is also helpful for the inter-communications between photogrammetry and geometric computer vision, which have many tasks and goals in common but simply using different mathematical tools.Item Open Access Integrated georeferencing for precise depth map generation exploiting multi-camera image sequences from mobile mapping(2020) Cavegn, Stefan; Haala, Norbert (apl. Prof. Dr.-Ing.)Image-based mobile mapping systems featuring multi-camera configurations allow for efficient geospatial data acquisition in both outdoor and indoor environments. We aim at accurate geospatial 3D image spaces consisting of collections of georeferenced multi-view RGB-D imagery, which may serve as basis for 3D street view services. In order to obtain high-quality depth maps, dense image matching exploiting multi-view image sequences captured with high redundancy needs to be performed. Since this process is entirely dependent on accurate image orientations, we mainly focus on pose estimation of multi-camera systems within this thesis. Nonetheless, we also present methods and investigations to obtain accurate, reliable and complete 3D scene representations based on multi-stereo mobile mapping sequences. Conventional image orientation approaches such as direct georeferencing enable absolute accuracies at the centimeter level in open areas with good GNSS coverage. However, GNSS conditions of street-based mobile mapping in urban canyons are often deteriorated by multipath effects and by shading of the signals caused by vegetation and large multi-story buildings. Moreover, indoor spaces do not even allow for any GNSS signals. Hence, we propose a powerful and versatile image orientation procedure that is able to cope with these issues encountered in challenging urban environments. Our integrated georeferencing approach extends the powerful structure-from-motion pipeline COLMAP with georeferencing capabilities. It assumes initial camera poses with sub-meter accuracy, which allow for direct triangulation of the complete scene. Such a global approach is much more efficient than an incremental structure-from-motion procedure. Furthermore, an initial image orientation solution already facilitates to georeference in a geodetic reference frame. Nevertheless, accuracies at the centimeter level can only be achieved by incorporation of ground control points. In order to obtain sub-pixel accurate relative orientations, strong tie point connections for the highly redundant multi-view image sequences are required. However, hardly overlapping fields of view, strongly varying views and weakly textured surfaces aggravate image feature matching. Hence, constraining relative orientation parameters among cameras is crucial for accurate, robust and efficient image orientation. Apart from supporting fixed multi-camera rigs, our integrated georeferencing approach that uses bundle adjustment allows for self-calibration of all relative orientation parameters or just single components. We extensively evaluated our integrated georeferencing procedure using six challenging real-world datasets in order to demonstrate its accuracy, robustness, efficiency and versatility. Four datasets were captured outdoors, one by a rail-based and three by different street-based multi-stereo camera systems. A portable mobile mapping system featuring a multi-head panorama camera collected two datasets in an indoor environment. Employing relative orientation constraints and ground control points within these indoor spaces resulted in absolute 3D accuracies of ca. 2 cm, and precisions at the millimeter level for relative 3D measurements. Depending on the use case, absolute 3D accuracy values for outdoor environments are slightly larger and amount to a few centimeters. However, determining 3D reference coordinates is a costly task. Not relying on any ground control points led to horizontal accuracies of ca. 5 cm for a scenario featuring some loops, while dropping down to a few decimeters for an extended junction area. Since the height component is even more dependent on prior camera poses from direct georeferencing, these 2D accuracies significantly decreased for the 3D case. However, incorporating just one ground control point facilitates the elimination of systematic effects, which results in 3D accuracies within the sub-decimeter range. Nevertheless, at least one additional check point is recommended in order to ensure a reliable solution. Once consistent and sub-pixel accurate relative poses of spatially adjacent images are available, in-sequence dense image matching can be performed. Aiming at precise and dense depth map generation, we evaluated several image matching configurations. Standard single stereo matching led to high accuracies, which could not significantly be improved by in-sequence matching. However, the image redundancy provided by additional epochs resulted in more complete and reliable depth maps.