Received 15 July 2016; accepted 8 August 2016; published 11 August 2016
Minimally invasive surgery is a relatively young surgical field. Despite early experience from animal studies  and humans  at the beginning of the 20th century, it was the introduction of video guided laparoscopic cholecystectomy  in 1987, which led to widespread adoption of minimally invasive techniques. Soon thereafter several studies  -  have shown that minimally invasive techniques are beneficial both for the patient (shorter recovery time, less pain and scars) as well as the society (less expensive procedures, reduced post-operative hospital stay for the patient). After the initial success with cholecystectomy, other minimally invasive techniques for abdominal surgery have been developed.
Since then minimally invasive surgery has been adopted by several clinical specialties including neurosurgery, cardiovascular therapy, pulmonary medicine, ear, nose, and throat (ENT) surgery and gynecology and obstetrics. This development has been enabled by the evolution of non-invasive imaging modalities such as X-ray, CT, MR and ultrasound, many of them being readily available in the operating theater as of today.
In contrast to open surgery, minimally invasive procedures rely on indirect vision via an endoscope. This will lead to a new way of visualizing the surgical filed and a series of challenges that need to be addressed in order to ensure a good outcome for the procedure. These include dealing with reduced depth perception, hand eye coordination difficulties and lack of direct palpation possibility. Despite differences in equipment and workflow, all minimally invasive techniques rely on an imaging chain in order to generate the final image presented to the surgical team.
The main goal of this article is to present an overview of recent developments within imaging for minimally invasive therapy based on the experience from Operating Rooms of the Future (FOR) at St. Olavs Hospital, Trondheim, Norway. Ongoing research projects will be mentioned. Some of the developing processes within imaging and visualization have been carried out in cooperation with leading actors from the industry.
2. The Operating Room of the Future (FOR)
The Operating Room of the Future (FOR) at St. Olavs Hospital, Trondheim, Norway has a strategic focus on collaboration with the industry and local and international research groups to explore new applications for image guided minimally invasive treatment.
Image guided minimally invasive methods have been important factors to provide a more efficient and less traumatic treatment of the patients compared with traditional open surgery. Several surgical procedures are now carried out as day-surgery or with discharge on the first postoperative day, and the patient is then returning to work and daily life activities within a short time. At St. Olav’s Hospital there has been a significant increase in the application of image guided minimally invasive methods during recent years. Excellent imaging has been a prerequisite for this development.
The following paragraphs will discuss the imaging requirements of various clinical fields and the way it was implemented in the FOR operating rooms.
For cardiovascular therapy, a hybrid operating room is a prerequisite. In addition to facilities for open surgery, stent-grafting of aortic aneurysms and dissection or the insertion of artificial heart valves by the transcatheter aortic valve implantation (TAVI) technique, requires excellent fluoroscopy equipment. At our FOR unit in Trondheim (Figure 1), we were the first in Europe to install the Dyna CT (Siemens) technology, which is providing
Figure 1. FOR operating room in the cardiovascular department, image taken during a TAVI procedure. The image from the ArtisZeego is displayed together with the ultrasound image and the patient’s vital signs on the same display.
intra-operative CT like images with high resolution. Later, this technology has been refined, by using the Siemens ArtisZeego® system and combining it with a navigation unit. The system can be supplemented with a robotic CT arm and a robotic catheter system. As a result of these developments, heart valves can then be maneuvered into place more accurately and precisely than before. Complicated procedures like deployment of aortic stent-grafts with side-branches can be carried out accurately in patients with thoracoabdominal aortic aneurysms, dissection or other complicated cases of aortic disease  -  . By using navigation technology, the overall radiation that the patient and the surgical team are exposed to is significantly reduced  .
In neurosurgery visualization plays a key role during pre-operative planning and during the intra-operative stage of the intervention. Typical sources of image data include the optical microscope video stream, the neuro-navigation view (reslices of the pre-operative MRI data aligned to the patient), intra-operative ultrasound 3D volumes and overview image of the operating theater. All these images are necessary, however their importance varies depending on the current stage of the surgery. In today’s setup (Figure 2), all these image sources are presented on different display units, representing an extra burden on the neurosurgeon as he needs to change his focus depending on the information he requires. Presenting all image sources on the same display unit is an ongoing research project at FOR, which will be detailed in the section entitled “Ongoing Research”. Furthermore, by using ultrasound or MRI intra-operatively the pre-operative MRI volumes can be aligned more accurately to the intra-operative situation, which changes dynamically during the surgery  . This leads to faster and more precise interventions as the surgeon has an updated navigation map according to the intra-operative situation. By using intra-operative ultrasound image acquisitions, deformations of the brain as a result of exposing the dura or due to removal of the tumor tissue can be accounted for  .
In pulmonary medicine a similar system has been used to tailor an operating room using fluoroscopy and navigation for diagnostic transbronchial biopsies. Previously, attempts of biopsy often had to be performed repeatedly and in small tumors it was often difficult to get a correct diagnosis at the early stages of the disease. The main task of a bronchoscopy platform is to offer a path through the airways to the peripheral lesion of interest and to track and update the position of the tools in real-time during the procedure  . The peripheral lesion is typically identified in a CT volume acquired pre-operatively. However in order to guide a surgical tool towards it, an alignment between the patient and the image data is required. This can be done by the examiner, by creating a mental map of the bronchial tree and correlate this with the live video stream from the bronchoscope. Alternatively, one can rely on automated tracking tools, which locate either the position of the bronchoscope via optical or electromagnetic tracking or the position of the tip of a catheter via fluoroscopy or a combination of both and register it with the pre-operative CT data. The influence of registration errors, as well as errors due to distortions of the magnetic field or caused by breathing on the final outcome of the biopsy, has been tested both on animal models and intra-operatively on patients  -  . From a visualization perspective a robust system that can exchange between pre-operative and intra-operative views and show the position of the
Figure 2. FOR operating theater for neurosurgery. Optical tracking is used for locating the surgical instruments, while the surgeon visualizes the surgical field relying on an optical microscope.
tools and their localization certainty is required. A combination of the CustusX platform  with integrated registration based on the airway’s center-line and the NDI Aurora EM tracking system has been established at FOR. A robust user interface combining the live bronchoscope image with orthogonal slices, segmented surfaces, volume rendered images or oblique planes extracted from the CT data is also implemented in CustusX. There are indications that modern technology will lead to improved diagnostics and treatment in these patients focusing on early diagnosis. Therefore transbronchial biopsies of small peripheral lesions is of importance and constitutes an active research field at FOR.
Computer assisted navigation tools are a pre-requisite for minimally invasive surgery in the ENT unit. Navigation systems act in a similar way as GPS systems, except for the fact that their accuracy is expected to be less than 1 mm  . Paranasal sinus surgery is a typical case of key-hole surgery as access to the surgical field is gained via the ostium of the nose. For tracking the instruments during surgery optical (Brainlab Kick®) and electromagnetic (Brainlab Kick® EM) tracking systems are available. Once an accurate registration between patient and image data is achieved this needs to be displayed intra-operatively. For visualization purposes a cockpit-like solution (Figure 3) is proposed to display the pre-operative and endoscopic image data on the same screen. As a result the surgeons have access to the same information irrespective of their position in the operating room. This is of special importance for minimally invasive surgery in ENT, where the surgeon is following the procedure continuously on the screen.
3. Evolution of Imaging and Visualization at FOR
A dedicated visualization laboratory has been developed at FOR. Its main purpose is testing of visualization and medical imaging technology prior to adoption in the operating room. All the technology is developed according to the EU directive for medical equipment (MDD, directive 93/42/EEC), which states all the requirements for medical equipment. These requirements are valid for both the product and the manufacturer.
In addition to the light source and optical lenses, the image quality is determined by the imaging sensor, the image processing unit and the display on which the image is shown. The imaging sensor is typically either a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS), its main purpose being to convert incident light into digital information. The sensor is a 2D matrix of elements, or pixels, which are accessible independently. Recently there has been a rapid development in optical imaging technology  , which enabled advancements in both endoscopy and laparoscopy. Miniaturization of the imaging sensor allowed for better resolution of the acquired images and for the sensor to be placed at the distal end of an imaging device, thereby improving the image quality by reducing interference and image distortion. Furthermore, during this period analog image processing devices have been gradually replaced by their digital counterparts.
Since FOR was established in 2005, the display chain for medical imaging underwent several improvement
Figure 3. FOR operating room in the ENT department. Display mimicking the cockpit of an airplane, in order to display pre- and intra-operative image data.
cycles. The resolution (number of programmable units) of the imaging sensor and the display unit increased from the standard definition (SD) resolution of 640 × 480 pixels up to ultra-high definition resolution of 3840 × 2160 pixels, or 4K, which is widely adopted by today’s consumer televisions (Figure 4). Improvements in resolution correspond to an increased level of detail (Figure 4), which is crucial when the size of the display units is increased.
In cooperation with Sony, camera lenses and chip technology with full HD technology (1920 × 1080 pixels) were developed. This gives five times better image information than regular video. HD technology for medical use was developed during the period 2005-2007 by Sony Corporation, Japan with the Operating Room of the Future as arena for research and development. From 2008 the technology became commercially available and therefore a part of the established medical equipment.
The next step in the development process took place during the period 2009-2013 when we cooperated with Sony and Olympus with focus on endoscopy. The main goal was the development of 4K technology, 4xHD in a format of 4096 × 1080 pixels.
Depth perception is crucial for understanding complex anatomic structures during minimally invasive procedures. The feasibility of using stereoscopic display of 3D ultrasound data acquired intra-operatively was tested by Gronningsaeter et al.  . After a preliminary study, it was concluded that 3D vision improves depth perception, which is beneficial when inspecting irregular lesions or tortuous vessels. However, a good contrast between the structure of interest and the surroundings is a prerequisite for accurate depth perception.
In addition to the endoscope or optical microscope images, pre- or intra-operative recordings based on ultrasound, X-ray, MRI or CT are required by several clinical fields. HD technology is also used to display these images. An example for which intra-operative imaging is required is endovascular therapy. FOR was the first center in Europe to install a DYNA-CT scanner in the operating room. Merging of the HD images generated by the endoscope with CT-like imaging led to a simplification of the work flow. As such CT images of the patient can be acquired several times during the surgery and the surgeon gets an updated image of the clinical situation without the need of moving the patient.
4. Ongoing Research at FOR
The challenge in several clinical fields (e.g. neurosurgery, ENT) is to process the data that are being recorded and present the relevant information to the surgeon. Several data streams are available during surgery, including the image from an optical microscope, the navigation display, overview video stream of the operating room and information from the patient monitor. In addition to these the surgeon prefers to have access to various scans acquired prior to surgery, which are aligned with the current surgical situation. So far, these data sets have been presented on different displays and the surgeon has to change his focus to obtain the information he is looking for. At FOR several visualization alternatives have been designed and tested in collaboration with industrial partners.
A clinical workflow assessment was carried out in the context of neurosurgery, a schematic illustration of which is presented in Figure 5. As a result of this assessment several tasks for improvement have been identified.
Figure 4. Illustration indicating the evolution of the resolution of the imaging and display equipment between 2005-2016 as well as its influence on the resulting images. In 2005, analog devices at standard definition (SD) were typically used. They were followed by HD devices, which improved both the image and the temporal resolution and as such reducing blurring effects and motion artifacts. Today, 4K resolution is adopted by consumer electronic products and expected to enter the medical market soon, whereas 8K is underdevelopment. On the right, a comparison between HD resolution widely available for medical devices and 4K resolution available for multimedia purposes is presented, image source: http://4k.com/resolution/.
Figure 5. Schematic visualization of the data flow for neurosurgery. Optical microscopy images, navigation information, patient monitor data and overview camera feed need to be accessible during the procedure, preferably on the same screen.
・ Release the surgical team from constantly focusing on the optical image,
・ Visualize the microscope and navigation images on a common display,
・ Combine 2D and 3D images on the same display,
・ Upgrade the display and imaging chain to 4K resolution,
・ Test the potential of laser projection for visualization purposes,
・ Test the suitability of module based Crystal-LED technology for visualization purposes,
・ Touch free control and steering of the image flow during the surgery.
It is expected that the results of this pilot study can be transferred to several other clinical disciplines when the pilot is completed at the end of 2016.
FOR has as its aim to develop tomorrows’ HD technology. This includes testing of laser projection (Figure 6) as an alternative to traditional projectors and the use of 4K Crystal LED displays (Figure 7) in the operating room. The advantages of using laser projectors in the operating theater are: wider color gamut, fast turn on/off cycles, power efficiency, increased longevity and reduced maintenance times. In the near future, as the technology will become available 8K-UHD displays with a resolution of 7680 × 4320 pixels will be available and considered for use in the operating room. Improved resolution translates into a better image quality, better visual differentiation and depth perception.
Finally a health-care telematics solution (Figure 8) combined with a robust work-flow of image data that allows for a systematic exchange of information in the operating theater is being designed. Our aim is to develop a prototype of an operating room where all the visualization is incorporated in one of the walls of the theater. Depending on the results of our laboratory testing, either laser projection or Crystal LED based technology will be chosen for display purposes. We have already made plans for the construction of a model in full-scale format 1:1
Figure 6. Laser projection, a new technology for projection of live images in the operating room.
Figure 7. Laser projection image format and example of images.
Figure 8. Crystal LED-display 4K or 8K, the future operating room. Photo: Telematics.
in cooperation with The Hospital Authorities of Central Norway.
It is likely that the indications for minimally invasive therapy will further expand in the near future and replace traditional open surgery. From a technological perspective, we will see more holography and augmented reality in medicine and robotics will be a standard part of the operating room for minimally invasive therapy.
As of today, HD technology is well established in an international perspective. However, the visualization technology for minimally invasive therapy is steadily improving. One has to keep in mind that new imaging sensors and displays capable of 4K or even 8K resolutions are gradually becoming available. Therefore new surgical instruments must be kept up to date with current technological advancements in order to offer clinicians the best intra-operative image of the organs of interest. By adopting the latest imaging and display equipment, the clarity of the generated images will be significantly improved. Furthermore, newer equipment can improve the contrast as well as the available color range, which will lead a better perception of the surgical field.
Conclusively, medical visualization of excellent quality, both for image production and presentation, is a prerequisite for successful image guided minimally invasive treatment. There is a need for new image and visualization technology to further improve the quality of the imaging in the operating theater. A direct consequence of such improvements will be increased efficiency and better patient safety and satisfaction.
 Dubois, F., Icard, P., Berthelot, G. and Levard, H. (1990) Coelioscopic Cholecystectomy. Preliminary Report of 36 Cases. Annals of Surgery, 211, 60.
 Bailey, R.W., Zucker, K.A., Flowers, J.L., Scovill, W.A., Graham, S.M. and Imbembo, A.L. (1991) Laparoscopic Cholecystectomy. Experience with 375 Consecutive Patients. Annals of Surgery, 214, 531.
 Lundbom, J., Hatlinghus, S., Lange, C., Odegard, A., Aasland, J., Romundstad, P. and Myhre, H. (2004) [Nine Years’ Experience with Endovascular Treatment of Abdominal Aortic Aneurysms]. Tidsskrift for den Norske laegeforening: Tidsskrift for praktisk medicin, ny raekke, 124, 2757-2759.
 Manstad-Hulaas, F., Ommedal, S., Tangen, G., Aadahl, P. and Hernes, T. (2007) Side-Branched AAA Stent Graft Insertion Using Navigation Technology: A Phantom Study. European Surgical Research, 39, 364-371.
 Manstad-Hulaas, F., Tangen, G.A., Dahl, T., Hernes, T.A. and Aadahl, P. (2012) Three-Dimensional Electromagnetic Navigation vs. Fluoroscopy for Endovascular Aneurysm Repair: A Prospective Feasibility Study Inpatients. Journal of Endovascular Therapy, 19, 70-78.
 Reinertsen, I., Lindseth, F., Askeland, C., Iversen, D.H. and Unsgard, G. (2014) Intra-Operative Correction of Brain-Shift. Acta Neurochirurgica, 156, 1301-1310.
 Unsgaard, G., Ommedal, S., Muller, T., Gronningsaeter, A. and Hernes, T.A.N. (2002) Neuro Navigation by Intraoperative Three-Dimensional Ultra Sound: Initial Experience during Brain Tumor Resection. Neurosurgery, 50, 804-812.
 Reynisson, P.J., Leira, H.O., Hernes, T.N., Hofstad, E.F., Scali, M., Sorger, H., Amundsen, T., Lindseth, F. and Lango, T. (2014) Navigated Bronchoscopy: A Technical Review. Journal of Bronchology & Interventional Pulmonology, 21, 242-264.
 Leira, H.O., Hofstad, E.F., Bo, L.E., Lango, T. and Amundsen, T. (2011) Navigated Bronchoscopy with Electromagnetic Tracking Cone Beam Computed Tomography Influence on Tracking and Registration Accuracy. Journal of Bronchology & Interventional Pulmonology, 18, 329-336.
 Leira, H.O., Amundsen, T., Tangen, G.A., Bo, L.E., Manstad-Hulaas, F. and Lango, T. (2011) A Novel Research Platform for Electromagnetic Navigated Bronchoscopy Using Cone Beam CT Imaging and an Animal Model. Minimally Invasive Therapy & Allied Technologies, 20, 30-41.
 Leira, H.O., Lango, T., Sorger, H., Hofstad, E.F. and Amundsen, T. (2013) Bronchoscope-Induced Displacement of Lung Targets: First in Vivo Demonstration of Effect from Wedging Maneuver in Navigated Bronchoscopy. Journal of Bronchology & Interventional Pulmonology, 20, 206-212.
 Askeland, C., Solberg, O.V., Bakeng, J.B.L., Reinertsen, I., Tangen, G.A., Hofstad, E.F., Iversen, D.H., Vapenstad, C., Selbekk, T., Lango, T., et al. (2015) CustusX: An Open-Source Research Plat Form for Image-Guided Therapy. International Journal of Computer Assisted Radiology and Surgery, 11, 505-519.
 Caversaccio, M. and Freysinger, W. (2003) Computer Assistance for Intraoperative Navigation in ENT Surgery. Minimally Invasive Therapy & Allied Technologies, 12, 36-51.
 Boppart, S., Deutsch, T. and Rattner, D. (1999) Optical Imaging Technology in Minimally Invasive Surgery. Surgical Endoscopy, 13, 718-722.
 Gronningsaeter, A., Lie, T., Kleven, A., Morland, T., Lango, T., Unsgad, G., Myhre, H. and Marvik, R. (2000) Initial Experience with Stereoscopic Visualization of Three-Dimensional Ultra Sound Data in Surgery. Surgical Endoscopy, 14, 1074-1078.