%0 Journal Article %J J. Med. Imag. %D 2022 %T Gaussian process regression for ultrasound scanline interpolation %A Degirmenci, Alperen %A Robert D. Howe %A Douglas P. Perrin %X

Purpose: In ultrasound imaging, interpolation is a key step in converting scanline data to brightness-mode (B-mode) images. Conventional methods, such as bilinear interpolation, do not fully capture the spatial dependence between data points, which leads to deviations from the underlying probability distribution at the interpolation points.

Approach: We propose Gaussian process (GP) regression as an improved method for ultrasound scanline interpolation. Using ultrasound scanlines acquired from two different ultrasound scanners during in vivo trials, we compare the scanline conversion accuracy of three standard interpolation methods with that of GP regression, measuring the peak signal-to-noise ratio (PSNR) and mean absolute error (MAE) for each method.

Results: The PSNR and MAE scores show that GP regression leads to more accurate scanline conversion compared to the nearest neighbor, bilinear, and cubic spline interpolation methods, for both datasets. Furthermore, limiting the interpolation window size of GP regression to 15 reduces computation time with minimal to no reduction in PSNR.

Conclusions: GP regression quantitatively leads to more accurate scanline conversion and provides uncertainty estimates at each of the interpolation points. Our windowing method reduces the computational cost of using GP regression for scanline conversion.

%B J. Med. Imag. %V 9 %P 037001 %G eng %U https://www.spiedigitallibrary.org/journals/journal-of-medical-imaging/volume-9/issue-3/037001/Gaussian-process-regression-for-ultrasound-scanline-interpolation/10.1117/1.JMI.9.3.037001.full %N 3 %0 Manuscript %D 2020 %T The NVIDIA PilotNet Experiments %A Mariusz Bojarski %A Chenyi Chen %A Joyjit Daw %A Alperen Değirmenci %A Joya Deri %A Bernhard Firner %A Beat Flepp %A Sachin Gogri %A Jesse Hong %A Lawrence Jackel %A Zhenhua Jia %A BJ Lee %A Bo Liu %A Liu, Fei %A Urs Muller %A Samuel Payne %A Nischal Kota Nagendra Prasad %A Artem Provodin %A John Roach %A Timur Rvachov %A Neha Tadimeti %A Jesper van Engelen %A Haiguang Wen %A Eric Yang %A Zongyi Yang %X Four years ago, an experimental system known as PilotNet became the first NVIDIA system to steer an autonomous car along a roadway. This system represents a departure from the classical approach for self-driving in which the process is manually decomposed into a series of modules, each performing a different task. In PilotNet, on the other hand, a single deep neural network (DNN) takes pixels as input and produces a desired vehicle trajectory as output; there are no distinct internal modules connected by human-designed interfaces. We believe that handcrafted interfaces ultimately limit performance by restricting information flow through the system and that a learned approach, in combination with other artificial intelligence systems that add redundancy, will lead to better overall performing systems. We continue to conduct research toward that goal.
This document describes the PilotNet lane-keeping effort, carried out over the past five years by our NVIDIA PilotNet group in Holmdel, New Jersey. Here we present a snapshot of system status in mid-2020 and highlight some of the work done by the PilotNet group. %G eng %U https://arxiv.org/abs/2010.08776 %0 Journal Article %J International Journal of Robotics Research %D 2020 %T Automatically steering cardiac catheters in vivo with respiratory motion compensation %A Paul M. Loschak %A Degirmenci, Alperen %A Tschabrunn, Cory M %A Elad Anter %A Robert D. Howe %X A robotic system for automatically navigating ultrasound (US) imaging catheters can provide real-time intra-cardiac imaging for diagnosis and treatment while reducing the need for clinicians to perform manual catheter steering. Clinical deployment of such a system requires accurate navigation despite the presence of disturbances including cyclical physiological motions (e.g., respiration). In this work, we report results from in vivo trials of automatic target tracking using our system, which is the first to navigate cardiac catheters with respiratory motion compensation. The effects of respiratory disturbances on the US catheter are modeled and then applied to four-degree-of-freedom steering kinematics with predictive filtering. This enables the system to accurately steer the US catheter and aim the US imager at a target despite respiratory motion disturbance. In vivo animal respiratory motion compensation results demonstrate automatic US catheter steering to image a target ablation catheter with 1.05 mm and 1.33° mean absolute error. Robotic US catheter steering with motion compensation can improve cardiac catheterization techniques while reducing clinician effort and X-ray exposure. %B International Journal of Robotics Research %G eng %U https://journals.sagepub.com/doi/abs/10.1177/0278364920903785 %0 Journal Article %J Journal of Biomechanical Engineering %D 2020 %T Ultrasound Imaging Characterization of Soft Tissue Dynamics of the Seated Human Body %A Yamada, Daisuke %A Degirmenci, Alperen %A Robert D. Howe %X To characterize the dynamics of internal soft organs and external anatomical structures, this paper presents a system that combines medical ultrasound imaging with an optical tracker and a vertical exciter that imparts whole-body vibrations on seated subjects. The spatial and temporal accuracy of the system was validated using a phantom with calibrated internal structures, resulting in 0.224 mm maximum root-mean-square (r.m.s.) position error and 13 ms maximum synchronization error between sensors. In addition to the dynamics of the head and sternum, stomach dynamics were characterized by extracting the centroid of the stomach from the ultrasound images. The system was used to characterize the subject-specific body dynamics as well as the intrasubject variabilities caused by excitation pattern (frequency up-sweep, down-sweep, and white noise, 1–10 Hz), excitation amplitude (1 and 2 m/s2 r.m.s.), seat compliance (rigid and soft), and stomach filling (empty and 500 mL water). Human subjects experiments (n = 3) yielded preliminary results for the frequency response of the head, sternum, and stomach. The method presented here provides the first detailed in vivo characterization of internal and external human body dynamics. Tissue dynamics characterized by the system can inform design of vehicle structures and adaptive control of seat and suspension systems, as well as validate finite element models for predicting passenger comfort in the early stages of vehicle design. %B Journal of Biomechanical Engineering %V 142 %G eng %U https://asmedigitalcollection.asme.org/biomechanical/article-abstract/142/6/061004/1046995 %N 6 %0 Journal Article %J International Journal of Computer Assisted Radiology and Surgery %D 2018 %T High Dynamic Range Ultrasound Imaging %A Degirmenci, Alperen %A Douglas P. Perrin %A Robert D. Howe %X

Purpose

High dynamic range (HDR) imaging is a popular computational photography technique that has found its way into every modern smartphone and camera. In HDR imaging, images acquired at different exposures are combined to increase the luminance range of the final image, thereby extending the limited dynamic range of the camera. Ultrasound imaging suffers from limited dynamic range as well; at higher power levels, the hyperechogenic tissue is overexposed, whereas at lower power levels, hypoechogenic tissue details are not visible. In this work, we apply HDR techniques to ultrasound imaging, where we combine ultrasound images acquired at different power levels to improve the level of detail visible in the final image.

Methods

Ultrasound images of ex vivo and in vivo tissue are acquired at different acoustic power levels and then combined to generate HDR ultrasound (HDR-US) images. The performance of five tone mapping operators is quantitatively evaluated using a similarity metric to determine the most suitable mapping for HDR-US imaging.

Results

The ex vivo and in vivo results demonstrated that HDR-US imaging enables visualizing both hyper- and hypoechogenic tissue at once in a single image. The Durand tone mapping operator preserved the most amount of detail across the dynamic range.

Conclusions

Our results strongly suggest that HDR-US imaging can improve the utility of ultrasound in image-based diagnosis and procedure guidance.

%B International Journal of Computer Assisted Radiology and Surgery %G eng %U https://link.springer.com/article/10.1007/s11548-018-1729-3 %0 Conference Paper %B IEEE International Symposium on Biomedical Imaging (ISBI) %D 2018 %T Ultrasound Imaging for Identifying Dynamics of Soft Tissue %A Yamada, Daisuke %A Degirmenci, Alperen %A Robert D. Howe %X This paper describes the development of a non-invasive system for characterizing tissue dynamics. The system combines medical ultrasound imaging with an optical tracking system and a vertical exciter that can impart whole-body vibrations on seated subjects. Tissue motion was extracted from the ultrasound images, and in combination with the optical tracker data, the frequency response of the tissue was calculated based on the commanded vibration of the seat and the resultant motion of the tissue. Dynamics of abdominal organs and the upper leg were characterized using the developed system. The ultrasound imaging method presented here will provide insight into the dynamics of soft tissues, as well as their boundary conditions with surrounding organ systems. The identified characteristics can be utilized for surgical planning and simulation as well as for validating a finite element model that is being developed to predict vehicular ride comfort in the early stages of automobile design. %B IEEE International Symposium on Biomedical Imaging (ISBI) %P 1161--1165 %G eng %0 Journal Article %J IEEE Robotics and Automation Letters (RA-L) %D 2018 %T Transforming the Dynamic Response of Robotic Structures and Systems Through Laminar Jamming %A Yashraj S. Narang %A Degirmenci, Alperen %A Joost J. Vlassak %A Robert D. Howe %X Researchers have developed variable-impedance mechanisms to control the dynamic response of robotic systems and improve their adaptivity, robustness, and efficiency. However, these mechanisms have limitations in size, cost, and convenience, particularly for variable damping. We demonstrate that laminar jamming structures can overcome these limitations and effectively transform the dynamic response of robotic structures and systems. In laminar jamming, an external pressure gradient is applied to a lightweight laminate of compliant material, changing its stiffness and damping. In this paper, we use rigorous analysis, simulation, and characterization to formulate a lumped-parameter model that captures the nonlinear mechanical behavior of jamming structures and can be used by designers to rapidly simulate the structures' dynamic responses. We illustrate that by adjusting the vacuum pressure, the fundamental features of the dynamic response (i.e., frequency, amplitude, decay rate, and steady-state value) can be tuned on command. Finally, we demonstrate that jamming structures can be integrated into soft structures and traditional rigid robots to dramatically alter their response to impacts. With the models and demonstrations provided here, researchers may move further towards building versatile, transformative robots. %B IEEE Robotics and Automation Letters (RA-L) %V 3 %P 688 - 695 %G eng %U http://ieeexplore.ieee.org/document/8141952/ %N 2 %0 Journal Article %J ASME Journal of Medical Devices %D 2017 %T Toward Medical Devices With Integrated Mechanisms, Sensors, and Actuators Via Printed-Circuit MEMS %A Gafford, J %A Ranzani, T %A Russo, S %A A. Degirmenci %A Kesner, S %A R.D. Howe %A Wood, R. %A Walsh, C %X

Recent advances in medical robotics have initiated a transition from rigid serial manipulators to flexible or continuum robots capable of navigating to confined anatomy within the body. A desire for further procedure minimization is a key accelerator for the development of these flexible systems where the end goal is to provide access to the previously inaccessible anatomical workspaces and enable new minimally invasive surgical (MIS) procedures. While sophisticated navigation and control capabilities have been demonstrated for such systems, existing manufacturing approaches have limited the capabilities of millimeter-scale end-effectors for these flexible systems to date and, to achieve next generation highly functional end-effectors for surgical robots, advanced manufacturing approaches are required. We address this challenge by utilizing a disruptive 2D layer-by-layer precision fabrication process (inspired by printed circuit board manufacturing) that can create functional 3D mechanisms by folding 2D layers of materials which may be structural, flexible, adhesive, or conductive. Such an approach enables actuation, sensing, and circuitry to be directly integrated with the articulating features by selecting the appropriate materials during the layer-by-layer manufacturing process. To demonstrate the efficacy of this technology, we use it to fabricate three modular robotic components at the millimeter-scale: (1) sensors, (2) mechanisms, and (3) actuators. These modules could potentially be implemented into transendoscopic systems, enabling bilateral grasping, retraction and cutting, and could potentially mitigate challenging MIS interventions performed via endoscopy or flexible means. This research lays the ground work for new mechanism, sensor and actuation technologies that can be readily integrated via new millimeter-scale layer-by-layer manufacturing approaches.

%B ASME Journal of Medical Devices %V 11 %G eng %U https://medicaldevices.asmedigitalcollection.asme.org/article.aspx?articleid=2592754 %N 1 %0 Conference Paper %B ICRA 2017 Workshop C4 Surgical Robots: Compliant, Continuum, Cognitive, and Collaborative %D 2017 %T Enabling Autonomous Ultrasound-based Procedure Guidance In Cardiac Interventions %A Degirmenci, Alperen %A Paul M. Loschak %A Cory M. Tschabrunn %A Elad Anter %A Robert D. Howe %B ICRA 2017 Workshop C4 Surgical Robots: Compliant, Continuum, Cognitive, and Collaborative %C Singapore, June 2 %G eng %0 Conference Proceedings %B IEEE International Conference on Robotics and Automation (ICRA) %D 2017 %T Predictive Filtering in Motion Compensation with Steerable Cardiac Catheters %A Paul M. Loschak %A Degirmenci, Alperen %A Robert D. Howe %X Robotic cardiac catheterization using ultrasound (US) imaging catheters provides real time imaging from within the heart while reducing the difficulty in manually steering a four degree-of-freedom (4-DOF) catheter. Accurate robotic catheter navigation in the heart is challenging due to a variety of disturbances including cyclical physiological motions, such as respiration. In this work we compensate for respiratory motion by using an Extended Kalman Filter (EKF) to predict target motion and by applying the predictions to steer the US imaging catheter. The system performance was measured in bench top experiments with phantom vasculature. The robotic system with predictive filtering tracked cyclically moving targets with 1.59 mm and 0.72° mean error. Accurately tracking moving structures can improve intra-procedural treatments and visualization. %B IEEE International Conference on Robotics and Automation (ICRA) %C Singapore, May 27 - June 2 %P 4830-4836 %G eng %0 Conference Proceedings %B Proceedings of the 9th Hamlyn Symposium on Medical Robotics %D 2016 %T Robotic Control of Ultrasound Catheters for Intra-cardiac Visualization %A Paul M. Loschak %A Degirmenci, Alperen %A Laura J. Brattain %A Cory M. Tschabrunn %A Elad Anter %A Robert D. Howe %B Proceedings of the 9th Hamlyn Symposium on Medical Robotics %C 9th Hamlyn Symposium on Medical Robotics, London, UK %G eng %U http://biorobotics.harvard.edu/pubs/2016/contrib/PLoschak_Hamlyn_2016.pdf %0 Conference Proceedings %B IEEE International Conference on Robotics and Automation (ICRA) %D 2016 %T Compensation for Unconstrained Catheter Shaft Motion in Cardiac Catheters %A Degirmenci, Alperen %A Paul M. Loschak %A Cory M. Tschabrunn %A Elad Anter %A Robert D. Howe %X

Cardiac catheterization with ultrasound (US) imaging catheters provides real time US imaging from within the heart, but manually navigating a four degree of freedom (DOF) imaging catheter is difficult and requires extensive training. Existing work has demonstrated robotic catheter steering in constrained bench top environments. Closed-loop control in an unconstrained setting, such as patient vasculature, remains a significant challenge due to friction, backlash, and physiological disturbances. In this paper we present a new method for closed-loop control of the catheter tip that can accurately and robustly steer 4-DOF cardiac catheters and other flexible manipulators despite these effects. The performance of the system is demonstrated in a vasculature phantom and an in vivo porcine animal model. During bench top studies the robotic system converged to the desired US imager pose with sub-millimeter and sub-degree-level accuracy. During animal trials the system achieved 2.0 mm and 0.65° accuracy. Accurate and robust robotic navigation of flexible manipulators will enable enhanced visualization and treatment during procedures.

%B IEEE International Conference on Robotics and Automation (ICRA) %C IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 2016 %P 4436 - 4442 %8 2016 %G eng %U http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7487643 %0 Journal Article %J ASME Journal of Mechanisms and Robotics %D 2016 %T A Four Degree of Freedom Robot for Positioning Ultrasound Imaging Catheters %A Paul M. Loschak %A Degirmenci, Alperen %A Yaroslav Tenzer %A Cory M. Tschabrunn %A Elad Anter %A Robert D. Howe %X

In this paper, we present the design, fabrication, and testing of a robot for automatically positioning ultrasound (US) imaging catheters. Our system will point US catheters to provide real-time imaging of anatomical structures and working instruments during minimally invasive procedures. Manually navigating US catheters is difficult and requires extensive training in order to aim the US imager at desired targets. Therefore, a four-degree-of-freedom (4DOF) robotic system was developed to automatically navigate US imaging catheters for enhanced imaging. A rotational transmission enables 3DOF for pitch, yaw, and roll of the imager. This transmission is translated by the 4DOF. An accuracy analysis calculated the maximum allowable joint motion error. Rotational joints must be accurate to within 1.5 deg, and the translational joint must be accurate within 1.4 mm. Motion tests then validated the accuracy of the robot. The average resulting errors in positioning of the rotational joints were 0.04–0.22 deg. The average measured backlash was 0.18–0.86 deg. Measurements of average translational positioning and backlash errors were negligible. The resulting joint motion errors were well within the required specifications for accurate robot motion. The output of the catheter was then tested to verify the effectiveness of the handle motions to transmit torques and translations to the catheter tip. The catheter tip was navigated to desired target poses with average error 1.3 mm and 0.71 deg. Such effective manipulation of US imaging catheters will enable better visualization in various procedures ranging from cardiac arrhythmia treatment to tumor removal in urological cases.

%B ASME Journal of Mechanisms and Robotics %V 8 %P 051016-051016-9 %8 2016 %G eng %U https://mechanismsrobotics.asmedigitalcollection.asme.org/article.aspx?articleID=2478256 %N 5 %0 Conference Proceedings %B Workshop on Navigation and Actuation of Flexible Instruments in Medical Applications (NAFIMA) at IROS 2015 %D 2015 %T Automatic Steering of Cardiac Imaging Catheters %A Paul M. Loschak %A Degirmenci, Alperen %A Laura J. Brattain %A Robert D. Howe %X

A robotic system for automatically steering cardiac imaging catheters enables enhanced visualization of anatomical structures and working instruments during catheter-based procedures. This system is comprised of several components for manipulating imaging catheters, sensing and control, user interface, and image collection and processing. Developments in each of these components have enabled the system to move towards in vivo studies in porcine models. The physical implementation of the robot can manipulate the four input degrees of freedom of the catheter with sub-degree and submillimeter accuracy. The sensing and control enables the inputs to maneuver the catheter tip with millimeter-level accuracy. The image collection and processing methods (in conjunction with the user interface) provide useful visualizations of simulated cardiac anatomy. Altogether, these components can potentially improve workflow, accuracy, and patient outcomes during minimally invasive interventional procedures.

%B Workshop on Navigation and Actuation of Flexible Instruments in Medical Applications (NAFIMA) at IROS 2015 %C Hamburg, Germany %G eng %0 Conference Proceedings %B Proceedings of the ASME 2015 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference (IDETC/CIE 2015) %D 2015 %T A 4-DOF Robot for Positioning Ultrasound Imaging Catheters %A Paul M. Loschak %A Yaroslav Tenzer %A Degirmenci, Alperen %A Robert D. Howe %X

In this paper we present the design, fabrication, and testing of a robot for automatically positioning ultrasound imaging catheters. Our system will point ultrasound (US) catheters to provide real-time imaging of anatomical structures and working instruments during minimally invasive surgeries. Manually navigating US catheters is difficult and requires extensive training in order to aim the US imager at desired targets. Therefore, a four DOF robotic system was developed to automatically navigate US imaging catheters for enhanced imaging. A rotational transmission enables three DOF for pitch, yaw, and roll of the imager. This transmission is translated by the fourth DOF. An accuracy analysis was conducted to calculate the maximum allowable joint motion error. Rotational joints must be accurate to within 1.5° and the translational joint must be accurate within 1.4 mm. Motion tests were then conducted to validate the accuracy of the robot. The average resulting errors in positioning of the rotational joints were measured to be 0.28°-0.38° with average measured backlash error 0.44°. Average translational positioning and backlash errors were measured to be significantly lower than the reported accuracy of the position sensor. The resulting joint motion errors were well within the required specifications for accurate robot motion. Such effective navigation of US imaging catheters will enable better visualization in various procedures ranging from cardiac arrhythmia treatment to tumor removal in urological cases.

%B Proceedings of the ASME 2015 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference (IDETC/CIE 2015) %C Boston, MA, USA %G eng %0 Conference Proceedings %B IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS) %D 2015 %T Design and control of a parallel linkage wrist for robotic microsurgery %A A. Degirmenci %A F. L. Hammond %A J. B. Gafford %A C. J. Walsh %A R. J. Wood %A R.D. Howe %X

This paper presents the design and control of a teleoperated robotic system for dexterous micromanipulation tasks at the meso-scale, specifically open microsurgery. Robotic open microsurgery is an unexplored yet potentially a high impact area of surgical robotics. Microsurgical operations, such as microanastomosis of blood vessels and reattachment of nerve fibers, require high levels of manual dexterity and accuracy that surpass human capabilities. A 3-DoF robotic wrist is designed and built based on a spherical five-bar mechanism. The wrist is attached to a 3-axis commercial off-the-shelf linear stage, achieving a fully dexterous system. Design requirements are determined using motion data collected during a simulated microanastomosis operation. The wrist design is optimized to maximize workspace and manipulability. The system is teleop- erated using a haptic device, and has the required bandwidth to replicate microsurgical motions. The system was successfully used in a micromanipulation task to stack 1 mm-diameter metal spheres. The micromanipulation system presented here may improve surgical outcomes during open microsurgery by offering better accuracy and dexterity to surgeons. 

%B IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS) %C Hamburg, Germany %G eng %0 Conference Paper %B 2014 IEEE International Conference on Robotics and Automation (ICRA) %D 2014 %T A monolithic approach to fabricating low-cost, millimeter-scale multi-axis force sensors for minimally-invasive surgery %A Joshua B. Gafford %A Samuel B. Kesner %A Degirmenci, Alperen %A Robert J. Wood %A Robert D. Howe %A Conor J Walsh %X

In this paper we have rapidly prototyped customized, highly-sensitive, mm-scale multi-axis force sensors for medical applications. Using a composite laminate batch fabrication process with biocompatible constituent materials, we have fabricated a fully-integrated, 10×10 mm three-axis force sensor with up to 5 V/N sensitivity and RMS noise on the order of ~1.6 mN, operational over a range of -500 to 500 mN in the x- and y-axes, and -2.5 to 2.5 N in the z-axis. Custom foil-based strain sensors were fabricated in parallel with the mechanical structure, obviating the need for post-manufacturing alignment and assembly. The sensor and its custom-fabricated signal conditioning circuitry fit within a 1×1×2 cm volume to realize a fully-integrated force transduction platform with potential haptics and control applications in minimally-invasive surgical tools. The form factor, biocompatibility, and cost of the sensor and signal conditioning makes this method ideal for rapid-prototyping low-cost, mm-scale distal force sensors. Sensor performance is validated in a simulated tissue palpation task using a robotic master-slave platform.

%B 2014 IEEE International Conference on Robotics and Automation (ICRA) %P 1419-1425 %G eng %U http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6907038 %0 Journal Article %J Journal of Medical Devices %D 2014 %T Cervical Spine Immobilization Device for Emergency Response %A Degirmenci, Alperen %A Goldberg, Benjamin %A Bielskis, Lukas %A Wiggins, Shanna %A Panagiotis Polygerinos %A Holland, Dónal %A Dyer, Sophia %A Walsh, Conor %B Journal of Medical Devices %G eng %0 Conference Paper %B IEEE Haptics Symposium %D 2012 %T Design and Control of an Air-Jet Lump Display %A J. C. Gwilliam %A A. Degirmenci %A M. Bianchi %A A. M. Okamura %B IEEE Haptics Symposium %C Vancouver, Canada %P 2012 %G eng %0 Conference Paper %B 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society %D 2011 %T Characterization of an Air Jet Haptic Lump Display %A M. Bianchi %A J. C. Gwilliam %A A. Degirmenci %A A. M. Okamura %B 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society %C Boston, MA %G eng