OBJECT: In these days, patients and doctors in operation room are surrounded by many medical devices as resulting from recent advancement of medical technology. However, these cutting-edge medical devices are working independently and not collaborating with each other, even though the collaborations between these devices such as navigation systems and medical imaging devices are becoming very important for accomplishing complex surgical tasks (such as a tumor removal procedure while checking the tumor location in neurosurgery). On the other hand, several surgical robots have been commercialized, and are becoming common. However, these surgical robots are not open for collaborations with external medical devices in these days. A cutting-edge "intelligent surgical robot" will be possible in collaborating with surgical robots, various kinds of sensors, navigation system and so on. On the other hand, most of the academic software developments for surgical robots are "home-made" in their research institutions and not open to the public. Therefore, open source control software for surgical robots can be beneficial in this field. From these perspectives, we developed Open Core Control software for surgical robots to overcome these challenges.
MATERIALS AND METHODS: In general, control softwares have hardware dependencies based on actuators, sensors and various kinds of internal devices. Therefore, these control softwares cannot be used on different types of robots without modifications. However, the structure of the Open Core Control software can be reused for various types of robots by abstracting hardware dependent parts. In addition, network connectivity is crucial for collaboration between advanced medical devices. The OpenIGTLink is adopted in Interface class which plays a role to communicate with external medical devices. At the same time, it is essential to maintain the stable operation within the asynchronous data transactions through network. In the Open Core Control software, several techniques for this purpose were introduced. Virtual fixture is well known technique as a "force guide" for supporting operators to perform precise manipulation by using a master-slave robot. The virtual fixture for precise and safety surgery was implemented on the system to demonstrate an idea of high-level collaboration between a surgical robot and a navigation system. The extension of virtual fixture is not a part of the Open Core Control system, however, the function such as virtual fixture cannot be realized without a tight collaboration between cutting-edge medical devices. By using the virtual fixture, operators can pre-define an accessible area on the navigation system, and the area information can be transferred to the robot. In this manner, the surgical console generates the reflection force when the operator tries to get out from the pre-defined accessible area during surgery.
RESULTS: The Open Core Control software was implemented on a surgical master-slave robot and stable operation was observed in a motion test. The tip of the surgical robot was displayed on a navigation system by connecting the surgical robot with a 3D position sensor through the OpenIGTLink. The accessible area was pre-defined before the operation, and the virtual fixture was displayed as a "force guide" on the surgical console. In addition, the system showed stable performance in a duration test with network disturbance.
CONCLUSION: In this paper, a design of the Open Core Control software for surgical robots and the implementation of virtual fixture were described. The Open Core Control software was implemented on a surgical robot system and showed stable performance in high-level collaboration works. The Open Core Control software is developed to be a widely used platform of surgical robots. Safety issues are essential for control software of these complex medical devices. It is important to follow the global specifications such as a FDA requirement "General Principles of Software Validation" or IEC62304. For following these regulations, it is important to develop a self-test environment. Therefore, a test environment is now under development to test various interference in operation room such as a noise of electric knife by considering safety and test environment regulations such as ISO13849 and IEC60508. The Open Core Control software is currently being developed software in open-source manner and available on the Internet. A communization of software interface is becoming a major trend in this field. Based on this perspective, the Open Core Control software can be expected to bring contributions in this field.
Magnetic Resonance Imaging (MRI) guided robotic interventions have been introduced in order to advance prostate cancer detection and treatment. To overcome problems of such robotic interventions, we have been developing a pneumatically actuated MRI-compatible modular robotic system for MRI-guided transperineal prostate intervention and its interventional procedure. For system evaluation, a series of experiments have been conducted and this paper reports a needle insertion experiment using prostate phantom and patient mockup trials. The needle insertion experiment resulted in noticeable consistent error in one direction, which we will investigate further. Nonetheless, patient mockup experiences suggest that the modular robotic system and its interventional procedure are well integrated and implemented in clinical environment.
RATIONALE AND OBJECTIVES: The authors present their initial experience using a 3-T whole-body scanner equipped with a 128-channel coil applied to lung motion assessment. Recent improvements in fast magnetic resonance imaging (MRI) technology have enabled several trials of free-breathing three-dimensional (3D) imaging of the lung. A large number of image frames necessarily increases the difficulty of image analysis and therefore warrants automatic image processing. However, the intensity homogeneities of images of prior dynamic 3D lung MRI studies have been insufficient to use such methods. In this study, initial data were obtained at 3 T with a 128-channel coil that demonstrate the feasibility of acquiring multiple sets of 3D pulmonary scans during free breathing and that have sufficient quality to be amenable to automatic segmentation.
MATERIALS AND METHODS: Dynamic 3D images of the lungs of two volunteers were acquired with acquisition times of 0.62 to 0.76 frames/s and an image matrix of 128 x 128, with 24 to 30 slice encodings. The volunteers were instructed to take shallow and deep breaths during the scans. The variation of lung volume was measured from the segmented images.
RESULTS: Dynamic 3D images were successfully acquired for both respiratory conditions for each subject. The images showed whole-lung motion, including lifting of the chest wall and the displacement of the diaphragm, with sufficient contrast to distinguish these structures from adjacent tissues. The average time to complete segmentation for one 3D image was 4.8 seconds. The tidal volume measured was consistent with known tidal volumes for healthy subjects performing deep-breathing maneuvers. The temporal resolution was insufficient to measure tidal volumes for shallow breathing.
CONCLUSION: This initial experience with a 3-T whole-body scanner and a 128-channel coil showed that the scanner and imaging protocol provided dynamic 3D images with spatial and temporal resolution sufficient to delineate the diaphragmatic domes and chest wall during active breathing. In addition, the intensity homogeneities and signal-to-noise ratio were adequate to perform automatic segmentation.
PURPOSE: To apply an intensity-based nonrigid registration algorithm to MRI-guided prostate brachytherapy clinical data and to assess its accuracy.
MATERIALS AND METHODS: A nonrigid registration of preoperative MRI to intraoperative MRI images was carried out in 16 cases using a Basis-Spline algorithm in a retrospective manner. The registration was assessed qualitatively by experts' visual inspection and quantitatively by measuring the Dice similarity coefficient (DSC) for total gland (TG), central gland (CG), and peripheral zone (PZ), the mutual information (MI) metric, and the fiducial registration error (FRE) between corresponding anatomical landmarks for both the nonrigid and a rigid registration method.
RESULTS: All 16 cases were successfully registered in less than 5 min. After the nonrigid registration, DSC values for TG, CG, PZ were 0.91, 0.89, 0.79, respectively, the MI metric was -0.19 +/- 0.07 and FRE presented a value of 2.3 +/- 1.8 mm. All the metrics were significantly better than in the case of rigid registration, as determined by one-sided t-tests.
CONCLUSION: The intensity-based nonrigid registration method using clinical data was demonstrated to be feasible and showed statistically improved metrics when compare to only rigid registration. The method is a valuable tool to integrate pre- and intraoperative images for brachytherapy.
BACKGROUND: With increasing research on system integration for image-guided therapy (IGT), there has been a strong demand for standardized communication among devices and software to share data such as target positions, images and device status.
METHOD: We propose a new, open, simple and extensible network communication protocol for IGT, named OpenIGTLink, to transfer transform, image and status messages. We conducted performance tests and use-case evaluations in five clinical and engineering scenarios.
RESULTS: The protocol was able to transfer position data with submillisecond latency up to 1024 fps and images with latency of <10 ms at 32 fps. The use-case tests demonstrated that the protocol is feasible for integrating devices and software.
CONCLUSION: The protocol proved capable of handling data required in the IGT setting with sufficient time resolution and latency. The protocol not only improves the interoperability of devices and software but also promotes transitions of research prototypes to clinical applications.
BACKGROUND: In magnetic resonance (MR) image-guided microwave thermocoagulation of liver tumors, the choice of the optimal puncture route is an important and time-consuming process. To assist this process, we have developed a motorized MR-compatible manipulator.
METHODS: The manipulator consists of a passive end effecter with 2 degrees-of-freedom (DOF) rotation and active base stages with 3 ultrasonic motors. It automatically chases the preset target point with synergetic remote-center-of-motion (RCM) control. A mechanical torque limiter and an electrical shutdown switch were added for patient safety.
RESULTS: The manipulator was used for this procedure in 15 cases and successfully utilized to treat liver tumors in various locations. Thoracoscopic assistance was combined with the manipulator in 6 cases. No complications were experienced.
CONCLUSIONS: The manipulator was found to be very effective for assisting MR-guided microwave coagulation of liver tumors.
For real-time 3D visualization of respiratory organ motion for MRI-guided therapy, a new adaptive 4D MR imaging method based on navigator echo and multiple gating windows was developed. This method was designed to acquire a time series of volumetric 3D images of a cyclically moving organ, enabling therapy to be guided by synchronizing the 4D image with the actual organ motion in real time. The proposed method was implemented in an open-configuration 0.5T clinical MR scanner. To evaluate the feasibility and determine optimal imaging conditions, studies were conducted with a phantom, volunteers, and a patient. In the phantom study the root mean square (RMS) position error in the 4D image of the cyclically moving phantom was 1.9 mm and the imaging time was approximately 10 min when the 4D image had six frames. In the patient study, 4D images were successfully acquired under clinical conditions and a liver tumor was discriminated in the series of frames. The image quality was affected by the relations among the encoding direction, the slice orientation, and the direction of motion of the target organ. In conclusion, this study has shown that the proposed method is feasible and capable of providing a real-time dynamic 3D atlas for surgical navigation with sufficient accuracy and image quality.
PURPOSE: To develop and assess a needle-guiding manipulator for MRI-guided therapy that allows a physician to freely select the needle insertion path while maintaining remote center of motion (RCM) at the tumor site.
MATERIALS AND METHODS: The manipulator consists of a three-degrees-of-freedom (DOF) base stage and passive needle holder with unconstrained two-DOF rotation. The synergistic control keeps the Virtual RCM at the preplanned target using encoder outputs from the needle holder as input to motorize the base stage.
RESULTS: The manipulator assists in searching for an optimal needle insertion path which is a complex and time-consuming task in MRI-guided ablation therapy for liver tumors. The assessment study showed that accuracy of keeping the virtual RCM to predefined position is 3.0 mm. In a phantom test, the physicians found the needle insertion path faster with than without the manipulator (number of physicians = 3, P = 0.001). However, the alignment time with the virtual RCM was not shorter when imaging time for planning were considered.
CONCLUSION: The study indicated that the robot holds promise as a tool for accurately and interactively selecting the optimal needle insertion path in liver ablation therapy guided by open-configuration MRI.
Magnetic resonance imaging (MRI) can provide high-quality 3-D visualization of prostate and surrounding tissue, thus granting potential to be a superior medical imaging modality for guiding and monitoring prostatic interventions. However, the benefits cannot be readily harnessed for interventional procedures due to difficulties that surround the use of high-field (1.5T or greater) MRI. The inability to use conventional mechatronics and the confined physical space makes it extremely challenging to access the patient. We have designed a robotic assistant system that overcomes these difficulties and promises safe and reliable intraprostatic needle placement inside closed high-field MRI scanners. MRI compatibility of the robot has been evaluated under 3T MRI using standard prostate imaging sequences and average SNR loss is limited to 5%. Needle alignment accuracy of the robot under servo pneumatic control is better than 0.94 mm rms per axis. The complete system workflow has been evaluated in phantom studies with accurate visualization and targeting of five out of five 1 cm targets. The paper explains the robot mechanism and controller design, the system integration, and presents results of preliminary evaluation of the system.
Magnetic Resonance Imaging (MRI) has potential to be a superior medical imaging modality for guiding and monitoring prostatic interventions. The strong magnetic field prevents the use of conventional mechatronics and the confined physical space makes it extremely challenging to access the patient. We have designed a robotic assistant system that overcomes these difficulties and promises safe and reliable intra-prostatic needle placement inside closed high-field MRI scanners. The robot performs needle insertion under real-time 3T MR image guidance; workspace requirements, MR compatibility, and workflow have been evaluated on phantoms. The paper explains the robot mechanism and controller design and presents results of preliminary evaluation of the system.
A software strategy to provide intuitive navigation for MRI-guided robotic transperineal prostate therapy is presented. In the system, the robot control unit, the MRI scanner, and open-source navigation software are connected to one another via Ethernet to exchange commands, coordinates, and images. Six states of the system called "workphases" are defined based on the clinical scenario to synchronize behaviors of all components. The wizard-style user interface allows easy following of the clinical workflow. On top of this framework, the software provides features for intuitive needle guidance: interactive target planning; 3D image visualization with current needle position; treatment monitoring through real-time MRI. These features are supported by calibration of robot and image coordinates by the fiducial-based registration. The performance test shows that the registration error of the system was 2.6 mm in the prostate area, and it displayed real-time 2D image 1.7 s after the completion of image acquisition.
Organ motion compensation in image-guided therapy is an active area of research. However, there has been little research on motion tracking and compensation in magnetic resonance imaging (MRI)-guided therapy. In this paper, we present a method to track a moving organ in MRI and control an active mechanical device for motion compensation. The method proposed is based on MRI navigator echo tracking enhanced by Kalman filtering for noise robustness. We also developed an extrapolation scheme to resolve any discrepancies between tracking and device control sampling rates. The algorithm was tested in a simulation study using a phantom and an active mechanical tool holder. We found that the method is feasible to use in a clinical MRI scanner with sufficient accuracy (0.36 mm to 1.51 mm depending on the range of phantom motion) and is robust to noise. The method proposed may be useful in MRI-guided targeted therapy, such as focused ultrasound therapy for a moving organ.
Aiming at real-time 3-D visualization of organ motion to navigate surgical procedures in MRI-guided surgery, a new 4-D MR imaging technique called "Adaptive 4-D Scan" has been proposed. The technique is designed to acquire a time series of volumetric 3-D images (4-D image) of cyclically moving organ, even in a low-field open-configuration MR scanner. A pre-operative 4-D image is acquired with respiratory phase parameter, which is monitored by using navigator-echo-based real-time tracking of the liver and diaphragm. During operation, the respiratory phase is again monitored in real-time, and a 3-D image, reflecting the current state of the target organ, is extracted from the pre-operative 4-D image and provided to physicians as a pseudo real-time 3-D image. We implemented Adaptive 4-D Scan into a 0.5 Tesla open-configuration clinical MRI system for intervention. Phantom and volunteer studies were performed to assess feasibility of this technique, in terms of image quality, imaging time and position accuracy of the imaged subject. A 4-D image (matrix: 256 x 128 x 10 x 8) of cyclically moving phantom was acquired in 719 s, and RMS position error between the imaged subject and the real subject was 2.3 mm, where the range of motion was 50 mm. 4-D image of the moving liver was also successfully acquired under near clinical condition. In conclusion, the study shows that the proposed method is feasible and has capability to provide real-time dynamic 3-D atlas for surgical navigation.
RATIONALE AND OBJECTIVES: Image registration in magnetic resonance (MR) image-guided liver therapy enhances surgical guidance by fusing preoperative multimodality images with intraoperative images, or by fusing intramodality images to correlate serial intraoperative images to monitor the effect of therapy. The objective of this paper is to describe the application of navigator echo and projection profile matching to fast two-dimensional image registration for MR-guided liver therapy.
MATERIALS AND METHODS: We obtain navigator echoes along the read-out and phase-encoding directions by using modified gradient echo imaging. This registration is made possible by masking out the liver profile from the image and performing profile matching with cross-correlation or mutual information as similarity measures. The set of experiments include a phantom study with a 2.0-T experimental MR scanner, and a volunteer and a clinical study with a 0.5-T open-configuration MR scanner, and these evaluate the accuracy and effectiveness of this method for liver therapy.
RESULTS: Both the phantom and volunteer study indicate that this method can perform registration in 34 ms with root-mean-square error of 1.6 mm when the given misalignment of a liver is 30 mm. The clinical studies demonstrate that the method can track liver motion of up to approximately 40 mm. Matching profiles with cross-correlation information perform better than with mutual information in terms of robustness and speed.
CONCLUSION: The proposed image registration method has potential clinical impact on and advantages for MR-guided liver therapy.
RATIONALE AND OBJECTIVES: The purpose of this study was to utilize computer assistance effectively for both easy and accurate magnetic resonance (MR) image-guided microwave thermocoagulation therapy of liver tumors.
MATERIALS AND METHODS: An open configuration MR scanner and a microwave coagulator at 2.45 GHz were used. Navigation software, a 3D Slicer, was customized to combine fluoroscopic MR images and preoperative MR images for the navigation. New functions to display MR temperature maps with simple parameter setting, and to record and display the coagulated areas by multiple microwave ablations in the 3-dimensional space (footprinting), were also introduced into the software. The VGA signal of the computer display was directly transferred to the surgeon's monitor.
RESULTS: The customized software could be used for both accurate image navigation and convenient and easy temperature monitoring. Because repeated punctures and ablations are usually required in this procedure, the footprinting function made targeting of the tumors both easy and accurate and was quite effective in achieving the necessary and sufficient treatment. Furthermore, clear display on the surgeon's monitor, which was obtained by direct transfer of the VGA signal, enabled precise image navigation.
CONCLUSION: The newly developed computer assistance was quite useful and helpful for this MR-guided procedure.