Research Projects

Extraction and Annotation of Individual Cores from TMA Images

Tissue Micro Arrays (TMAs) are composed of a large number of tissue cores arranged on a single paraffin block enabling high throughput analysis. Whole slide imaging enables high resolution digitization of these TMAs. However, arrangement of these tissue cores is often not perfect and their extraction followed by annotation to the right patient ID remains a challenge. A completely automated image processing algorithm that can extract and annotate individual cores from these whole slide images will have a high impact on the speed with which these cores can be further analyzed. We propose an unsupervised probabilistic appearance model that can segment and extract the location of these cores. We leverage the fact that the cores have a constant radius by utilizing a fixed shape model. We also incorporate spatial information of the grid pattern in which these cores are arranged by extracting this information from the annotation files associated with each TMA. We take into account the imperfections in the spatial arrangement of these cores by calculating the parameters of a homography matrix that can model rotation, affine and projective distortions in grid alignment. Finally, each core is mapped to a patient ID and has a corresponding likelihood value which represents the goodness of fit to the assumed model. 

Epithelium and Stromal Tissue Segmentation in Immunohistological Images

Automatic segmentation of epithelium and stromal tissue in immunohistochemistry images is essential to quantify the presence of a particular protein in various regions of the tissue. As a first step towards this goal, we propose an image analysis pipeline that extracts the proportion of pixels stained brown (which represents the presence of a protein) in the epithelium and stroma. The image is divided into superpixels (locally smooth regions into which an image can be partitioned based on local intensity and edge statistics) and the one that contain background and fat are discarded from segmentation based on a simple intensity threshold. Texture features such as correlation, contrast, dissimilarity, homogeneity, and local binary pattern are extracted from each superpixel which are then used to train a support vector machine (SVM) algorithm. This algorithm can now predict the class (epithelium or stroma) of each superpixel in a given test image. 

Spatial Marked Poisson Processes for Pattern Recognition

Motivated by problems in image processing involving segmentation and the detection of multiple instances of complex objects, we explore the use of marked Poisson point processes within a Bayesian nonparametric framework. The Poisson point process underlies a wide range of combinatorial stochastic processes and as such has been a key object driving research in Bayesian nonparametrics. We explore Poisson point processes in combination with probabilistic shape and appearance priors for detection/segmentation of objects/patterns in 1D, 2D and 3D frameworks.  This probabilistic formulation encompasses uncertainty in number, location, shape and appearance of the feature of interest, be it in images or in time-series data (detection/segmentation of objects of interest). In images, this model can simultaneously detect and segment objects of interest. 

The Poisson intensity parameter can either be homogeneous (constant intensity throughout) or non-homogeneous. A non-homogeneous Poisson prior provides the flexibility to incorporate spatial context information regarding where the high or low traffic/concentration areas occur. We model the non-homogeneous Poisson intensity with a log-Gaussian cox process. For shape, any probabilistic model can be used. We describe examples of both, parametric and complex shape priors. Appearance features can be simple intensity values of the image or higher level features such as texture.

We propose two inference strategies based on RJMCMC and Gibbs sampling. Inference on the proposed model is made complex due to changing model order and the use of non-conjugate priors. Inference in such a scenario is usually accomplished using RJMCMC framework. We also describe a Gibbs sampling approach which is accomplished by taking advantage of the finite nature of images. 

We demonstrate the results on images in 2D and 3D along with the time series data. We demonstrate results in both, supervised and unsupervised settings.

Versatile Onboard Traffic Embedded Roaming Sensors (VOTERS):

VOTERS develops non-invasive highway assessment and maintenance system using sensors mounted on a moving vehicle. I am responsible for the video sensor in this project.

I implemented a software trigger for the camera based on distance traveled by the moving vehicle using C++ on a Linux platform for acquisition of images. The data acquired is published using ArcGIS interface. I developed algorithms for automated detection and classification of pavement cracks according to their types (alligator, transverse and longitudinal) and presence of manholes etc. using two strategies. First algorithm uses a Bayesian multinomial logistic regression. However, this algorithm is limited by the discretization of crack direction/orientation. To overcome this issue, another algorithm inspired from biomedical image processing that utilizes a Hessian-based multi-scale filters at different scales is developed. A sample of dataset with ground-truth annotations of cracks on pavement images is provided here.

Highway Steel Bridge Coating Assessment by Developing a Digital Image Processing System

Rust defects on highway steel bridges are one of the most commonly observed defects on coating surfaces that have to be rectified since they severely affect the structural integrity of bridges. A rust defect assessment method is developed that automatically detects the percentage of rust in a given digital color image of the bridge surface taken from a conventional digital camera. Images are pre-processed to correct the illumination variations resulting from camera angle and natural light. Energy, entropy and average luminance corresponding to the sub-bands extracted by wavelet transform on blocks of the image is extracted as features followed by principal component analysis to reduce the dimension of feature space. A least mean square classifier is trained on these features since they are linearly separable resulting in a model that can classify a given block of the image as rust or non-rust. The results of the algorithm are analyzed for its efficiency and possible optimization techniques are suggested.