J. Jang, Y. Kim, B. Westgate, Y. Zong, C. Hallinan, A. Akalin, and K Lee. Submitted. “Screening Adequacy of Unstained Fine Needle Aspiration Samples Using a Deep Learning-based Classifier”.
X. Pan, C. Wang, Y. Yu, N. Reljin, D. McManus, C. Darling, K. Chon, Y. Mendelson, and K. Lee. Submitted. “Deep cross-modality feature learning applied to predict acutely decompensated heart failure using in-home collected electrocardiography and transthoracic bioimpedance”.
T. Song, M. Cao, J. Min, H. Im, H. Lee*, and K. Lee*. Submitted. “Deep learning-based phenotyping of breast cancer cells using lens-free digital in-line holography.” bioRxiv. Publisher's VersionAbstract

Lens-free digital in-line holography (LDIH) produces cellular diffraction patterns (holograms) with a large field of view that lens-based microscopes cannot offer. It is a promising diagnostic tool allowing comprehensive cellular analysis with high-throughput capability. Holograms are, however, far more complicated to discern by the human eye, and conventional computational algorithms to reconstruct images from hologram limit the throughput of hologram analysis. To efficiently and directly analyze holographic images from LDIH, we developed a novel deep learning architecture called a holographical deep learning network (HoloNet) for cellular phenotyping. The HoloNet uses holo-branches that extract large features from diffraction patterns and integrates them with small features from convolutional layers. Compared with other state-of-the-art deep learning methods, HoloNet achieved better performance for the classification and regression of the raw holograms of the breast cancer cells stained with well-known breast cancer markers, ER/PR and HER2. Moreover, we developed the HoloNet dual embedding model to extract high-level diffraction features related to breast cancer cell types and their marker intensities of ER/PR and HER2 to identify previously unknown subclusters of breast cancer cells. This hologram embedding allowed us to identify rare and subtle subclusters of the phenotypes overlapped by multiple breast cancer cell types. We demonstrate that our HoloNet efficiently enables LDIH to perform a more detailed analysis of heterogeneity of cell phenotypes for precise breast cancer diagnosis.Competing Interest StatementThe authors have declared no competing interest.

*Co-corresponding authors: K. Lee and H. Lee

C. Wang, H. J. Choi, L. Woodbury, and K. Lee. Submitted. “Deep learning-based subcellular phenotyping of protrusion dynamics reveals fine differential drug responses at subcellular and single-cell levels.” bioRxiv. Publisher's VersionAbstract
Intracellular processes such as cytoskeletal organization and organelle dynamics exhibit massive subcellular heterogeneity. Although recent advances in fluorescence microscopy allow researchers to acquire an unprecedented amount of live cell image data at high spatiotemporal resolutions, the traditional ensemble-averaging of uncharacterized subcellular heterogeneity could mask important activities. Moreover, the curse of dimensionality of these complex dynamic datasets prevents access to critical mechanistic details of subcellular processes. Here, we establish an unsupervised machine learning framework called DeepHACKS (Deep phenotyping of Heterogeneous Activities in the Coordination of cytosKeleton at the Subcellular level) for "deep phenotyping," which identifies rare subcellular phenotypes specifically sensitive to molecular perturbations. DeepHACKS dissects the heterogeneity of subcellular time-series datasets by allowing bi-directional LSTM (Long-Short Term Memory) neural networks to extract fine-grained temporal features by integrating autoencoders with conventional machine learning outcomes. We applied DeepHACKS to subcellular protrusion dynamics in pharmacologically perturbed epithelial cells, revealing fine differential responses of leading edge dynamics specific to each perturbation. Particularly, DeepHACKS in conjunction with blebistantin treatment revealed the emergence of rare subcellular and single-cell phenotypes driven by "bursting" protrusion. This suggests that the temporal features directly learned from leading edge dynamics enable fine-grained identification of drug-related phenotypes, which may not be accessible from static cell images. In summary, our study provides an analytical framework for detailed and quantitative understandings of molecular mechanisms hidden in their heterogeneity. DeepHACKS can be potentially applied to analyze various time-series data measured from other subcellular processes.Competing Interest StatementThe authors have declared no competing interest.
J. Jang, C. Hallinan, and K. Lee. 6/2022. “Protocol for live cell image segmentation to profile cellular morphodynamics using MARS-Net.” STAR Protocols, 3, Pp. 101469. Publisher's VersionAbstract
Quantitative studies of cellular morphodynamics rely on accurate cell segmentation in live cell images. However, fluorescence and phase contrast imaging hinder accurate edge localization. To address this challenge, we developed MARS-Net, a deep learning model integrating ImageNet-pretrained VGG19 encoder and U-Net decoder trained on the datasets from multiple types of microscopy images. Here, we provide the protocol for installing MARS-Net, labeling images, training MARS-Net for edge localization, evaluating the trained models’ performance, and performing the quantitative profiling of cellular morphodynamics.
K. Vaidyanathan*, C. Wang*, A. Krajnik, Y. Yu, M. Choi, B. Lin, J. Jang, S. Heo, J. Kolega, K. Lee**, and Y. Bae**. 12/2021. “A machine learning pipeline revealing heterogeneous responses to drug perturbations on vascular smooth muscle cell spheroid morphology and formation.” Scientific Reports 11, Pp. 23285. Publisher's VersionAbstract

Machine learning approaches have shown great promise in biology and medicine discovering hidden information to further understand complex biological and pathological processes. In this study, we developed a deep learning-based machine learning algorithm to meaningfully process image data and facilitate studies in vascular biology and pathology. Vascular injury and atherosclerosis are characterized by neointima formation caused by the aberrant accumulation and proliferation of vascular smooth muscle cells (VSMCs) within the vessel wall. Understanding how to control VSMC behaviors would promote the development of therapeutic targets to treat vascular diseases. However, the response to drug treatments among VSMCs with the same diseased vascular condition is often heterogeneous. Here, to identify the heterogeneous responses of drug treatments, we created an in vitro experimental model system using VSMC spheroids and developed a machine learning-based computational method called HETEROID (heterogeneous spheroid). First, we established a VSMC spheroid model that mimics neointima-like formation and the structure of arteries. Then, to identify the morphological subpopulations of drug-treated VSMC spheroids, we used a machine learning framework that combines deep learning-based spheroid segmentation and morphological clustering analysis. Our machine learning approach successfully showed that FAK, Rac, Rho, and Cdc42 inhibitors differentially affect spheroid morphology, suggesting that multiple drug responses of VSMC spheroid formation exist. Overall, our HETEROID pipeline enables detailed quantitative drug characterization of morphological changes in neointima formation, that occurs in vivo, by single-spheroid analysis.

*Co-first authors: K. Vaidyanathan and C. Wang. **Co-corresponding authors: Y. Bae and K. Lee

J. Jang*, C. Wang*, X. Zhang, H. J. Choi, X. Pan, B. Lin, Y. Yu, C. Whittle, M. Ryan, Y. Chen, and K. Lee. 10/2021. “A deep learning-based segmentation pipeline for profiling cellular morphodynamics using multiple types of live cell microscopy.” Cell Reports Methods, 1, Pp. 100105. Publisher's VersionAbstract

To accurately segment cell edges and quantify cellular morphodynamics from live-cell imaging data, we developed a deep learning-based pipeline termed MARS-Net (multiple-microscopy-type-based accurate and robust segmentation network). MARS-Net utilizes transfer learning and data from multiple types of microscopy to localize cell edges with high accuracy. For effective training on distinct types of live-cell microscopy, MARS-Net comprises a pretrained VGG19 encoder with U-Net decoder and dropout layers. We trained MARS-Net on movies from phase-contrast, spinning-disk confocal, and total internal reflection fluorescence microscopes. MARS-Net produced more accurate edge localization than the neural network models trained with single-microscopy-type datasets. We expect that MARS-Net can accelerate the studies of cellular morphodynamics by providing accurate pixel-level segmentation of complex live-cell datasets.

*Co-first authors: J. Jang and C. Wang

M. H. Omebeyinje, A. Adeluyi, C. Mitra, P. Chakraborty, G. M. Gandee, N. Patel, B. Verghese, C. E. Farrance, M. Hull, P. Basu, K. Lee, A. Adhikari, B. Adivar, J. A. Horney, and A. Chanda. 9/2021. “Increased prevalence of indoor Aspergillus and Penicillium species is associated with indoor flooding and coastal proximity: a case study of 28 moldy buildings.” Environ. Sci.: Processes Impacts, Pp. -. Publisher's VersionAbstract
Indoor flooding is a leading contributor to indoor dampness and the associated mold infestations in the coastal United States. Whether the prevalent mold genera that infest the coastal flood-prone buildings are different from those not flood-prone is unknown. In the current case study of 28 mold-infested buildings across the U.S. east coast, we surprisingly noted a trend of higher prevalence of indoor Aspergillus and Penicillium genera (denoted here as Asp–Pen) in buildings with previous flooding history. Hence, we sought to determine the possibility of a potential statistically significant association between indoor Asp–Pen prevalence and three building-related variables: (i) indoor flooding history, (ii) geographical location, and (iii) the building's use (residential versus non-residential). Culturable spores and hyphal fragments in indoor air were collected using the settle-plate method, and corresponding genera were confirmed using phylogenetic analysis of their ITS sequence (the fungal barcode). Analysis of variance (ANOVA) using Generalized linear model procedure (GLM) showed that Asp–Pen prevalence is significantly associated with indoor flooding as well as coastal proximity. To address the small sample size, a multivariate decision tree analysis was conducted, which ranked indoor flooding history as the strongest determinant of Asp–Pen prevalence, followed by geographical location and the building's use.
H. J. Choi, C. Wang, X. Pan, J. Jang, M. Cao, J. Brazzo, Y. Bae, and K. Lee. 5/2021. “Emerging machine learning approaches to phenotyping cellular motility and morphodynamics.” Physical Biology. Publisher's VersionAbstract
Cells respond heterogeneously to molecular and environmental perturbations. Phenotypic heterogeneity, wherein multiple phenotypes coexist in the same conditions, presents challenges when interpreting the observed heterogeneity. Advances in live cell microscopy allow researchers to acquire an unprecedented amount of live cell image data at high spatiotemporal resolutions. Phenotyping cellular dynamics, however, is a nontrivial task and requires machine learning (ML) approaches to discern phenotypic heterogeneity from live cell images. In recent years, ML has proven instrumental in biomedical research, allowing scientists to implement sophisticated computation in which computers learn and effectively perform specific analyses with minimal human instruction or intervention. In this review, we discuss how ML has been recently employed in the study of cell motility and morphodynamics to identify phenotypes from computer vision analysis. We focus on new approaches to extract and learn meaningful spatiotemporal features from complex live cell images for cellular and subcellular phenotyping.
J. A. Brazzo, J. C. Biber, E. Nimmer, Y. Heo, L. Ying, R. Zhao, K. Lee, M. Krause, and Y. Bae. 5/2021. “Mechanosensitive expression of lamellipodin promotes intracellular stiffness, cyclin expression, and cell proliferation.” Journal of Cell Science. Publisher's VersionAbstract
Cell cycle control is a key aspect of numerous physiological and pathological processes. The contribution of biophysical cues, such as stiffness or elasticity of the underlying extracellular matrix (ECM), is critically important in regulating cell cycle progression and proliferation. Indeed, increased ECM stiffness causes aberrant cell cycle progression and proliferation. However, the molecular mechanisms that control these stiffness-mediated cellular responses remain unclear. Here, we address this gap and show good evidence that lamellipodin, previously known as a critical regulator of cell migration, stimulates ECM stiffness-mediated cyclin expression and intracellular stiffening. We observed that increased ECM stiffness upregulates lamellipodin expression. This is mediated by an integrin-dependent FAK-Cas-Rac signaling module and supports stiffness-mediated lamellipodin induction. Mechanistically, we find that lamellipodin overexpression increased and lamellipodin knockdown reduced stiffness-induced cell cyclin expression and cell proliferation, and intracellular stiffness. Overall, these results suggest that lamellipodin levels may be critical for regulating cell proliferation.
C. Wang, X. Zhang, H. J. Choi, B. Lin, Y. Yu, C. Whittle, M. Ryan, Y. Chen, and K. Lee. 2019. “Deep learning pipeline for cell edge segmentation of time-lapse live cell images.” bioRxiv. Publisher's VersionAbstract
Quantitative live cell imaging has been widely used to study various dynamical processes in cell biology. Phase contrast microscopy is a popular imaging modality for live cell imaging since it does not require labeling and cause any phototoxicity to live cells. However, phase contrast images have posed significant challenges for accurate image segmentation due to complex image features. Fluorescence live cell imaging has also been used to monitor the dynamics of specific molecules in live cells. But unlike immunofluorescence imaging, fluorescence live cell images are highly prone to noise, low contrast, and uneven illumination. These often lead to erroneous cell segmentation, hindering quantitative analyses of dynamical cellular processes. Although deep learning has been successfully applied in image segmentation by automatically learning hierarchical features directly from raw data, it typically requires large datasets and high computational cost to train deep neural networks. These make it challenging to apply deep learning in routine laboratory settings. In this paper, we evaluate a deep learning-based segmentation pipeline for time-lapse live cell movies, which uses small efforts to prepare the training set by leveraging the temporal coherence of time-lapse image sequences. We train deep neural networks using a small portion of images in the movies, and then predict cell edges for the entire image frames of the same movies. To further increase segmentation accuracy using small numbers of training frames, we integrate VGG16 pretrained model with the U-Net structure (VGG16-U-Net) for neural network training. Using live cell movies from phase contrast, Total Internal Reflection Fluorescence (TIRF), and spinning disk confocal microscopes, we demonstrate that the labeling of cell edges in small portions (5∼10%) can provide enough training data for the deep learning segmentation. Particularly, VGG16-U-Net produces significantly more accurate segmentation than U-Net by increasing the recall performance. We expect that our deep learning segmentation pipeline will facilitate quantitative analyses of challenging high-resolution live cell movies.
S. Kim*, C. Wang*, B. Zhao, H. Im, J. Min, H. J. Choi, J. Tadros, N. R. Choi, C. M. Castro, R. Weissleder, H. Lee**, and K. Lee**. 11/2018. “Deep transfer learning-based hologram classification for molecular diagnostics.” Scientific Reports, 8, Pp. 17003. Publisher's VersionAbstract

Lens-free digital in-line holography (LDIH) is a promising microscopic tool that overcomes several drawbacks (e.g., limited field of view) of traditional lens-based microcopy. However, extensive computation is required to reconstruct object images from the complex diffraction patterns produced by LDIH. This limits LDIH utility for point-of-care applications, particularly in resource limited settings. We describe a deep transfer learning (DTL) based approach to process LDIH images in the context of cellular analyses. Specifically, we captured holograms of cells labeled with molecular-specific microbeads and trained neural networks to classify these holograms without reconstruction. Using raw holograms as input, the trained networks were able to classify individual cells according to the number of cell-bound microbeads. The DTL-based approach including a VGG19 pretrained network showed robust performance with experimental data. Combined with the developed DTL approach, LDIH could be realized as a low-cost, portable tool for point-of-care diagnostics.

*Co-first authors: S. Kim and C. Wang. **Co-corresponding authors: K. Lee and H. Lee

Y. Kim, H. J. Choi, and K. Lee. 9/2018. “Subcellular Time Series Modeling of Heterogeneous Cell Protrusion.” bioRxiv. Publisher's VersionAbstract
In this paper, a new biological modeling approach is proposed for predicting complex heterogeneous subcellular behaviors. Cell protrusion which initiates cell migration has a significant amount of subcellular heterogeneity in micrometer length and minute time scales. It is driven by actin polymerization, e.g., pushing the plasma membrane forward, and then regulated by a multitude of actin regulators. While mathematical modeling is central to system-level understandings of cell protrusion, most of the modeling is based on the ensemble average of actin regulator dynamics at the cellular or population levels, preventing from capturing the heterogeneous cellular activities. With these in mind, a systematic modeling framework is proposed in this paper for predicting velocities of heterogeneous protrusion of migrating cells driven by multiple molecular mechanisms. The modeling framework is developed through the integration of the multiple AutoRegressive eXogenous (ARX) models employing probability density input variables. Unlike conventional ARX models, it provides an effective framework for modeling heterogeneous subcellular behaviors with complex nonlinearities and uncertainties of dynamic systems. To train and validate the proposed model, numerous subcellular time series are extracted from time-lapse movies of migrating PtK1 cells using spinning disk confocal microscope: The current edge velocities and fluorescent intensities of mDia1, actin at the leading edge are used as the input while the future cell edge velocities are selected as an output. It is demonstrated that the proposed approach is highly effective in predicting the future trends of heterogeneous cell protrusion. In particular, by capturing the various multiple activities from the dataset, it is expected that it would improve the understanding of the molecular mechanism underlying cellular and subcellular heterogeneity.
C. Wang*, H. J. Choi*, S.J. Kim, A. Desai, N. Lee, D. Kim, Y. Bae, and K. Lee. 4/2018. “Deconvolution of subcellular protrusion heterogeneity and the underlying actin regulator dynamics from live cell imaging.” Nature Communications, 9, Pp. 1688. Publisher's VersionAbstract

Cell protrusion is morphodynamically heterogeneous at the subcellular level. However, the mechanism of cell protrusion has been understood based on the ensemble average of actin regulator dynamics. Here, we establish a computational framework called HACKS (deconvolution of heterogeneous activity in coordination of cytoskeleton at the subcellular level) to deconvolve the subcellular heterogeneity of lamellipodial protrusion from live cell imaging. HACKS identifies distinct subcellular protrusion phenotypes based on machine-learning algorithms and reveals their underlying actin regulator dynamics at the leading edge. Using our method, we discover "accelerating protrusion", which is driven by the temporally ordered coordination of Arp2/3 and VASP activities. We validate our finding by pharmacological perturbations and further identify the fine regulation of Arp2/3 and VASP recruitment associated with accelerating protrusion. Our study suggests HACKS can identify specific subcellular protrusion phenotypes susceptible to pharmacological perturbation and reveal how actin regulator dynamics are changed by the perturbation.

*Co-first authors: C. Wang and H. J. Choi

C. Wang, S. Kang, E. Kim, X. Zhang, H. J. Choi, A. Choi, and K. Lee. 2017. “Edge detection of cryptic lamellipodia assisted by deep learning.” bioRxiv, Pp. 181263. Publisher's VersionAbstract
Cell protrusion plays important roles in cell migration by pushing plasma membrane forward. Cryptic lamellipodia induce the protrusion of submarginal cells in collective cell migration where cells are attached and move together. Although computational image analysis of cell protrusion has been done extensively, the study on protrusion activities of cryptic lamellipodia is limited due to difficulties in image segmentation. This study seeks to aid in the computational analysis of submarginal cell protrusion in collective cell migration by using deep learning to detect the cryptic lamellipodial edges from fluorescence time-lapse movies. Due to the noisy features within overlapping cells, the conventional image analysis algorithms such as Canny edge detector and intensity thresholding are limited. In this paper we combined Canny edge detector, Deep Neural Networks (DNNs), and local intensity thresholding. We were able to detect cryptic lamellipodial edges of submarginal cells with high accuracy from the fluorescence time-lapse movies of PtK1 cells using both simple convolutional neural networks and VGG-16 based neural networks. We used relatively small effort to prepare the training set to train the DNNs to detect the cryptical lamellipodial edges in fluorescence time-lapse movies. This work demonstrates that deep learning can be combined with the conventional image analysis algorithms to facilitate the computational analysis of highly complex time-lapse movies of collective cell migration.
K. Lee, H. L. Elliott, Y. Oak, C. T. Zee, A. Groisman, J. D. Tytell, and G. Danuser. 2015. “Functional hierarchy of redundant actin assembly factors revealed by fine-grained registration of intrinsic image fluctuations.” Cell Systems, 1, Pp. 37-50. Publisher's VersionAbstract
Highly redundant pathways often contain components whose functions are difficult to decipher from the responses induced by genetic or molecular perturbations. Here, we present a statistical approach that samples and registers events observed in images of intrinsic fluctuations in unperturbed cells to establish the functional hierarchy of events in systems with redundant pathways. We apply this approach to study the recruitment of actin assembly factors involved in the protrusion of the cell membrane. We find that the formin mDia1, along with nascent adhesion components, is recruited to the leading edge of the cell before protrusion onset, initiating linear growth of the lamellipodial network. Recruitment of Arp2/3, VASP, cofilin, and the formin mDia2 then promotes sustained exponential growth of the network. Experiments changing membrane tension suggest that Arp2/3 recruitment is mechano-responsive. These results indicate that cells adjust the overlapping contributions of multiple factors to actin filament assembly during protrusion on a ten-second timescale and in response to mechanical cues.
K. Lee, J. L. Gallop, K. Rambani, and M. W. Kirschner. 2010. “Self-assembly of filopodia-like structures on supported lipid bilayers.” Science, 329, Pp. 1341-5. Publisher's VersionAbstract
Filopodia are finger-like protrusive structures, containing actin bundles. By incubating frog egg extracts with supported lipid bilayers containing phosphatidylinositol 4,5 bisphosphate, we have reconstituted the assembly of filopodia-like structures (FLSs). The actin assembles into parallel bundles, and known filopodial components localize to the tip and shaft. The filopodia tip complexes self-organize--they are not templated by preexisting membrane microdomains. The F-BAR domain protein toca-1 recruits N-WASP, followed by the Arp2/3 complex and actin. Elongation proteins, Diaphanous-related formin, VASP, and fascin are recruited subsequently. Although the Arp2/3 complex is required for FLS initiation, it is not essential for elongation, which involves formins. We propose that filopodia form via clustering of Arp2/3 complex activators, self-assembly of filopodial tip complexes on the membrane, and outgrowth of actin bundles.
K. Lee, T. S. Chung, and J. H. Kim. 2003. “Global optimization of clusters in gene expression data of DNA microarrays by deterministic annealing.” Genomics & Informatics, 1, Pp. 20-24. Publisher's VersionAbstract
The analysis of DNA microarry data is one of the most important things for functional genomics research. The matrix representation of microarray data and its successive 'optimal' incisional hyperplanes is a useful platform for developing optimization algorithms to determine the optimal partitioning of pairwise proximity matrix representing completely connected and weighted graph. We developed Deterministic Annealing (DA) approach to determine the successive optimal binary partitioning. DA algorithm demonstrated good performance with the ability to find the 'globally optimal' binary partitions. In addition, the objects that have not been clustered at small non-zero temperature, are considered to be very sensitive to even small randomness, and can be used to estimate the reliability of the clustering.
K. Lee and W. Sung. 2002. “Ion transport and channel transition in biomembranes.” Physica A: Statistical Mechanics and its Applications, 315, Pp. 79-97. Publisher's VersionAbstract
An ion channel is a macromolecular machinery (NANOMACHINE) which regulates the ionic conduction through biomembranes. The ion channels are fundamental in every thought, every perception, every movement, and every heartbeat. The relevant biological dynamics in mesoscopic level is the overdamped slow dynamics. We present a stochastic model to describe the coupled behaviors of ion transport and channel conformation under an applied (trans)membrane potential. We apply our general theoretical formulae to an analytically tractable model of channel with a deep binding site which interacts with the permeant ions electrostatically or entropically. The interaction is found to be modulated by the ionic occupancy which is enhanced by the membrane potential. Above a critical interaction strength or a membrane potential, the interaction gives rise to an emergence of a new conductance state, via a channel conformational transition. This is the self-organization generic to strongly coupled stochastic processes.
K. Lee and W. Sung. 2002. “A stochastic model of conductance transitions in voltage-gated ion channels.” Journal of biological physics, 28, Pp. 279-288. Publisher's VersionAbstract
We present a statistical physics model to describe the stochastic behaviorof ion transport and channel transitions under an applied membrane voltage.To get pertinent ideas we apply our general theoretical scheme to ananalytically tractable model of the channel with a deep binding site whichinteracts with the permeant ions electrostatically. It is found that theinteraction is modulated by the average ionic occupancy in the bindingsite, which is enhanced by the membrane voltage increases. Above acritical voltage, the interaction gives rise to a emergence of a newconducting state along with shift of S4 charge residues in the channel.This exploratory study calls for further investigations to correlate thecomplex transition behaviors with a variety of ion channels, withparameters in the model, potential energy parameters, voltage, and ionic concentration.