Browsing by Author "Schüle, Johannes"

Now showing 1 - 2 of 2

Open Access
Cystoscopic depth estimation using gated adversarial domain adaptation
(2023) Somers, Peter; Holdenried-Krafft, Simon; Zahn, Johannes; Schüle, Johannes; Veil, Carina; Harland, Niklas; Walz, Simon; Stenzl, Arnulf; Sawodny, Oliver; Tarín, Cristina; Lensch, Hendrik P. A.
Monocular depth estimation from camera images is very important for surrounding scene evaluation in many technical fields from automotive to medicine. However, traditional triangulation methods using stereo cameras or multiple views with the assumption of a rigid environment are not applicable for endoscopic domains. Particularly in cystoscopies it is not possible to produce ground truth depth information to directly train machine learning algorithms for using a monocular image directly for depth prediction. This work considers first creating a synthetic cystoscopic environment for initial encoding of depth information from synthetically rendered images. Next, the task of predicting pixel-wise depth values for real images is constrained to a domain adaption between the synthetic and real image domains. This adaptation is done through added gated residual blocks in order to simplify the network task and maintain training stability during adversarial training. Training is done on an internally collected cystoscopy dataset from human patients. The results after training demonstrate the ability to predict reasonable depth estimations from actual cystoscopic videos and added stability from using gated residual blocks is shown to prevent mode collapse during adversarial training.
Open Access
Intraoperative localization and scene reconstruction using differentiable rendering and graph-based landmark registration with application to cystoscopy
(2023) Schüle, Johannes; Sawodny, Oliver (Prof. Dr.-Ing. Dr. h.c.)
Minimally invasive procedures, such as bladder endoscopy, reduce trauma but present navigation challenges due to restricted visibility. Traditional mapping methods are often insufficient, and comprehensive solutions in the literature are scarce. This work introduces a novel approach for intraoperative navigation and scene reconstruction, focusing on deformable environments, such as those encountered in cystoscopy. The proposed reconstruction concept relies on a monocular camera image but can be flexibly extended to include additional sensor data. The fundamental reconstruction strategy employed in this work follows the question: How does the model representation and camera perspective need to be adjusted such that the rendering of the model matches the current observation? The reconstruction process requires the formulation of several optimization objectives, necessitating a fully differentiable rendering approach. To this end, a novel formulation of an inverse rendering process is proposed, where 2D image data is projected from the camera's perspective onto a 3D mesh model. Vascular structures, which remain consistent despite deformations, serve as dependable landmarks. Combining graph-based landmark recognition with rendering-based reconstruction techniques offers a comprehensive solution for determining the intraoperative location and scene, especially considering the complexity of deformable surgical environments.