Faisal Mahmood, Richard Chen, Daniel Borders, Gregory N McKay, Kevan Salimian, Alexander Baras, and Nicholas J Durr. 2019. “Adversarial U-net with spectral normalization for histopathology image segmentation using synthetic data.” In Medical Imaging 2019: Digital Pathology, 10956: Pp. 109560N. International Society for Optics and Photonics.
Faisal Mahmood, Daniel Borders, Richard Chen, Gregory N McKay, Kevan J Salimian, Alexander Baras, and Nicholas J Durr. 2019. “Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images.” IEEE Trans Med Imaging.Abstract
Nuclei segmentation is a fundamental task for various computational pathology applications, including nuclei morphology analysis, cell type classification, and cancer grading. Deep learning has emerged as a powerful approach to segmenting nuclei, but the accuracy of convolutional neural networks (CNNs) depends on the volume and quality of labeled histopathology data for training. In particular, conventional CNN-based approaches lack structured prediction capabilities, which are required to distinguish overlapping and clumped nuclei. Here, we present an approach to nuclei segmentation that overcomes these challenges by utilizing a conditional generative adversarial network (cGAN) trained with synthetic and real data. We generate a large dataset of H&E training images with perfect nuclei segmentation labels using an unpaired GAN framework. This synthetic data along with real histopathology data from six different organs are used to train a conditional GAN with spectral normalization and gradient penalty for nuclei segmentation. This adversarial regression framework enforces higher-order spacial-consistency when compared to conventional CNN models. We demonstrate that this nuclei segmentation approach generalizes across different organs, sites, patients and disease states, and outperforms conventional approaches, especially in isolating individual and overlapping nuclei.
Taylor L Bobrow, Faisal Mahmood, Miguel Inserni, and Nicholas J Durr. 2019. “DeepLSR: a deep learning approach for laser speckle reduction.” Biomed Opt Express, 10, 6, Pp. 2869-2882.Abstract
Speckle artifacts degrade image quality in virtually all modalities that utilize coherent energy, including optical coherence tomography, reflectance confocal microscopy, ultrasound, and widefield imaging with laser illumination. We present an adversarial deep learning framework for laser speckle reduction, called DeepLSR (, that transforms images from a source domain of coherent illumination to a target domain of speckle-free, incoherent illumination. We apply this method to widefield images of objects and tissues illuminated with a multi-wavelength laser, using light emitting diode-illuminated images as ground truth. In images of gastrointestinal tissues, DeepLSR reduces laser speckle noise by 6.4 dB, compared to a 2.9 dB reduction from optimized non-local means processing, a 3.0 dB reduction from BM3D, and a 3.7 dB reduction from an optical speckle reducer utilizing an oscillating diffuser. Further, DeepLSR can be combined with optical speckle reduction to reduce speckle noise by 9.4 dB. This dramatic reduction in speckle noise may enable the use of coherent light sources in applications that require small illumination sources and high-quality imaging, including medical endoscopy.
Mason T Chen, Faisal Mahmood, Jordan A Sweer, and Nicholas J Durr. 2019. “GANPOP: Generative Adversarial Network Prediction of Optical Properties from Single Snapshot Wide-field Images.” arXiv preprint arXiv:1906.05360.
Gregory N McKay, Faisal Mahmood, and Nicholas J Durr. 2019. “Large dynamic range autorefraction with a low-cost diffuser wavefront sensor.” Biomed Opt Express, 10, 4, Pp. 1718-1735.Abstract
Wavefront sensing with a thin diffuser has emerged as a potential low-cost alternative to a lenslet array for aberrometry. Here we show that displacement of caustic patterns can be tracked for estimating wavefront gradient in a diffuser wavefront sensor (DWFS), enabling large dynamic-range wavefront measurements with sufficient accuracy for eyeglass prescription measurements. We compare the dynamic range, repeatability, precision, and number of resolvable prescriptions of a DWFS to a Shack-Hartmann wavefront sensor (SHWFS) for autorefraction measurement. We induce spherical and cylindrical errors in a model eye and use a multi-level Demon's non-rigid registration algorithm to estimate caustic displacements relative to an emmetropic model eye. When compared to spherical error measurements with the SHWFS using a laser diode with a laser speckle reducer, the DWFS demonstrates a ∼5-fold improvement in dynamic range (-4.0 to +4.5 D vs. -22.0 to +19.5 D) with less than half the reduction in resolution (0.072 vs. 0.116 D), enabling a ∼3-fold increase in the number of resolvable prescriptions (118 vs. 358). In addition to being lower-cost, the unique, non-periodic nature of the caustic pattern formed by a diffuser enables a larger dynamic range of aberration measurements compared to a lenslet array.
Mehmet Turan, Yasin Almalioglu, Hunter B Gilbert, Faisal Mahmood, Nicholas J Durr, Helder Araujo, Alp Eren Sarı, Anurag Ajay, and Metin Sitti. 2019. “Learning to Navigate Endoscopic Capsule Robots.” IEEE Robotics and Automation Letters, 4, 3, Pp. 3075–3082.
Richard J Chen, Taylor L Bobrow, Thomas Athey, Faisal Mahmood, and Nicholas J Durr. 2019. “SLAM Endoscopy enhanced by adversarial depth prediction.” KDD Workshop on Applied Data Science for Healthcare 2019.
Faisal Mahmood, Märt Toots, Lars-göran Wallentin Öfverstedt, and Bo Ulf Skoglund. 2018. “2d discrete fourier transform with simultaneous edge artifact removal for real-time applications”.
Faisal Mahmood, Nauman Shahid, Ulf Skoglund, and Pierre Vandergheynst. 2018. “Adaptive graph-based total variation for tomographic reconstructions.” IEEE Signal Processing Letters, 25, 5, Pp. 700–704.
Faisal Mahmood and Nicholas J Durr. 2018. “Deep learning and conditional random fields-based depth estimation and topographical reconstruction from conventional endoscopy.” Med Image Anal, 48, Pp. 230-243.Abstract
Colorectal cancer is the fourth leading cause of cancer deaths worldwide and the second leading cause in the United States. The risk of colorectal cancer can be mitigated by the identification and removal of premalignant lesions through optical colonoscopy. Unfortunately, conventional colonoscopy misses more than 20% of the polyps that should be removed, due in part to poor contrast of lesion topography. Imaging depth and tissue topography during a colonoscopy is difficult because of the size constraints of the endoscope and the deforming mucosa. Most existing methods make unrealistic assumptions which limits accuracy and sensitivity. In this paper, we present a method that avoids these restrictions, using a joint deep convolutional neural network-conditional random field (CNN-CRF) framework for monocular endoscopy depth estimation. Estimated depth is used to reconstruct the topography of the surface of the colon from a single image. We train the unary and pairwise potential functions of a CRF in a CNN on synthetic data, generated by developing an endoscope camera model and rendering over 200,000 images of an anatomically-realistic colon.We validate our approach with real endoscopy images from a porcine colon, transferred to a synthetic-like domain via adversarial training, with ground truth from registered computed tomography measurements. The CNN-CRF approach estimates depths with a relative error of 0.152 for synthetic endoscopy images and 0.242 for real endoscopy images. We show that the estimated depth maps can be used for reconstructing the topography of the mucosa from conventional colonoscopy images. This approach can easily be integrated into existing endoscopy systems and provides a foundation for improving computer-aided detection algorithms for detection, segmentation and classification of lesions.
Faisal Mahmood, Richard Chen, Sandra Sudarsky, Daphne Yu, and Nicholas J Durr. 2018. “Deep learning with cinematic rendering: fine-tuning deep neural networks using photorealistic medical images.” Physics in Medicine & Biology, 63, 18, Pp. 185012.
Faisal Mahmood and Nicholas J Durr. 2018. “Deep learning-based depth estimation from a synthetic endoscopy image training set.” In Medical Imaging 2018: Image Processing, 10574: Pp. 1057421. International Society for Optics and Photonics.
Faisal Mahmood, Lars-Göran Öfverstedt, Märt Toots, Gunnar Wilken, and Ulf Skoglund. 2018. “An extended field-based method for noise removal from electron tomographic reconstructions.” IEEE Access, 6, Pp. 17326–17339.
Faisal Mahmood and Nicholas J Durr. 2018. “Topographical reconstructions from monocular optical colonoscopy images via deep learning.” In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Pp. 216–219. IEEE.
Faisal Mahmood, Richard Chen, and Nicholas J Durr. 2018. “Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training.” IEEE Trans Med Imaging, 37, 12, Pp. 2572-2581.Abstract
To realize the full potential of deep learning for medical imaging, large annotated datasets are required for training. Such datasets are difficult to acquire due to privacy issues, lack of experts available for annotation, underrepresentation of rare conditions, and poor standardization. The lack of annotated data has been addressed in conventional vision applications using synthetic images refined via unsupervised adversarial training to look like real images. However, this approach is difficult to extend to general medical imaging because of the complex and diverse set of features found in real human tissues. We propose a novel framework that uses a reverse flow, where adversarial training is used to make real medical images more like synthetic images, and clinically-relevant features are preserved via self-regularization. These domain-adapted synthetic-like images can then be accurately interpreted by networks trained on large datasets of synthetic medical images. We implement this approach on the notoriously difficult task of depth-estimation from monocular endoscopy which has a variety of applications in colonoscopy, robotic surgery, and invasive endoscopic procedures. We train a depth estimator on a large data set of synthetic images generated using an accurate forward model of an endoscope and an anatomically-realistic colon. Our analysis demonstrates that the structural similarity of endoscopy depth estimation in a real pig colon predicted from a network trained solely on synthetic data improved by 78.7% by using reverse domain adaptation.
Faisal Mahmood, Lars-göran Wallentin Öfverstedt, and Bo Ulf Skoglund. 2017. “Extended field iterative reconstruction technique (EFIRT) for correlated noise removal”.
Donald G Bailey, Faisal Mahmood, and Ulf Skoglund. 2017. “Reducing the Cost of Removing Border Artefacts in Fourier Transforms.” In Proceedings of the 8th International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies, Pp. 11. ACM.
Faisal Mahmood, Nauman Shahid, Ulf Skoglund, and Pierre Vandergheynst. 2016. “Compressed sensing and adaptive graph total variation for tomographic reconstructions”.
Dhanya Menoth Mohan, Parmod Kumar, Faisal Mahmood, Kian Foong Wong, Abhishek Agrawal, Mohamed Elgendi, Rohit Shukla, Natania Ang, April Ching, Justin Dauwels, and Alice HD Chan. 2016. “Effect of Subliminal Lexical Priming on the Subjective Perception of Images: A Machine Learning Approach.” PLoS One, 11, 2, Pp. e0148332.Abstract
The purpose of the study is to examine the effect of subliminal priming in terms of the perception of images influenced by words with positive, negative, and neutral emotional content, through electroencephalograms (EEGs). Participants were instructed to rate how much they like the stimuli images, on a 7-point Likert scale, after being subliminally exposed to masked lexical prime words that exhibit positive, negative, and neutral connotations with respect to the images. Simultaneously, the EEGs were recorded. Statistical tests such as repeated measures ANOVAs and two-tailed paired-samples t-tests were performed to measure significant differences in the likability ratings among the three prime affect types; the results showed a strong shift in the likeness judgment for the images in the positively primed condition compared to the other two. The acquired EEGs were examined to assess the difference in brain activity associated with the three different conditions. The consistent results obtained confirmed the overall priming effect on participants' explicit ratings. In addition, machine learning algorithms such as support vector machines (SVMs), and AdaBoost classifiers were applied to infer the prime affect type from the ERPs. The highest classification rates of 95.0% and 70.0% obtained respectively for average-trial binary classifier and average-trial multi-class further emphasize that the ERPs encode information about the different kinds of primes.
Faisal Mahmood, Nauman Shahid, Pierre Vandergheynst, and Ulf Skoglund. 2016. “Graph-based sinogram denoising for tomographic reconstructions.” In 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Pp. 3961–3664. IEEE.