About the project
| Project type | Public funding (MIWF, FH-Basis 2012) |
| Duration | > 5 years (ongoing) |
As shown in numerous publications, studies and position papers [VDE/DGBMT], assistive technologies are becoming increasingly important in medicine and surgery. 3D endoscopy, which is already available on the market with a few products, is one of these technologies that enables surgeons to optimize the handling of instruments through three-dimensional vision. However, the focus of current research groups goes beyond pure 3D visualization [DRL: Medical Assistance Systems, TU Munich: SFB 453 Realistic Telepresence and Teleaction]. Future systems will support the surgeon with targeted additional computer-aided information or even take over (semi-) autonomous subtasks, either to relieve the surgeon of routine work or to carry out more precise manipulations. This development step represents the transition from pure visualization to surgical robotics.
One of the prerequisites for such innovative assistance systems is the exact spatial measurement of the working area of the endoscope, e.g. in the abdominal cavity. This includes vessels, organs, pathological changes/objects or even medical instruments. In principle, however, the 3D endoscope can not only provide stereoscopic depth perception for the surgeon. With the help of stereo image processing methods, as already established in everyday industrial use, the third dimension can be reconstructed from the image data of the two optics of the 3D endoscope (the stereo camera). This enables the endoscope to be extended as a measuring instrument for the spatial position of object points. Initial research approaches to 3D reconstruction [FH Reutlingen/IFA: Measuring 3D endoscope] and the associated assistance systems already exist, but marketable realizations and products in the medical environment are still a long way off. Considerable research efforts are still required in this area, and the research project described here is intended to contribute to this. However, the performance of current embedded processors promises a real-time capable implementation of live stereo image processing and thus the feasibility of medical products.
The subject of the planned research activities is on the one hand real-time capable 3D reconstruction in endoscopic image data from the point of view of future assistance systems. In addition to the development methodology and the algorithms for (stereo) image processing, the implementation of the processes on product-related hardware (embedded processors) also plays a role in applied research in order to be able to demonstrate prototype maturity. There is still a great need for research and development in this area. Two applications are described below as examples.
One possible assistance function is the superimposition (augmentation/overlay) of the 3D image with additional virtual information. This can be, for example, preoperative data from other diagnostic examinations, such as the measurement of vessels and tissue structures, organs or objects/anomalies to be localized using US/CT. The additional information provided is used for orientation and navigation of the endoscope and instruments and can thus also help to avoid injury to sensitive organ structures such as nerves and blood vessels. If, in another case, the optical visibility is impaired by opacities, the 3D model calculated by the computer can be used to generate a representation for the surgeon, which in some cases contains virtual objects. In aviation, this would be the equivalent of flying blind with navigation aids. However, additional information must certainly always be selected and presented in such a way that the surgeon is not overloaded with it.
Surgical interventions through the smallest openings in the body place very high demands on the surgeon's skill in handling the instruments. Before the introduction of 3D endoscopy, this favoured the development of innovative robot-assisted systems, which aim to automatically control/navigate and stabilize the intracorporeal alignment of the endoscope and instruments in the working space. In this application example, the focus is on position and attitude control, which in turn requires accurate 3D reconstruction via the measuring 3D endoscope and downstream stereo signal processing.
