Gehrmann, S., Layne, S. & Dernoncourt, F., 2019. Improving Human Text Comprehension through Semi-Markov CRF-based Neural Section Title Generation. North American Chapter of the Association for Computational Linguistics. Publisher's VersionAbstract
Titles of short sections within long documents support readers by guiding their focus towards relevant passages and by providing anchor-points that help to understand the progression of the document. The positive effects of section titles are even more pronounced when measured on readers with less developed reading abilities, for example in communities with limited labeled text resources. We, therefore, aim to develop techniques to generate section titles in low-resource environments. In particular, we present an extractive pipeline for section title generation by first selecting the most salient sentence and then applying deletion-based compression. Our compression approach is based on a Semi-Markov Conditional Random Field that leverages unsupervised word-representations such as ELMo or BERT, eliminating the need for a complex encoder-decoder architecture. The results show that this approach leads to competitive performance with sequence-to-sequence models with high resources, while strongly outperforming it with low resources. In a human-subject study across subjects with varying reading abilities, we find that our section titles improve the speed of completing comprehension tasks while retaining similar accuracy.
naacl2019.pdf naacl19-presentation.pdf
Gehrmann, S., Deng, Y. & Rush, A.M., 2018. Bottom-up abstractive summarization. arXiv preprint arXiv:1808.10792.
Gehrmann, S., et al., 2018. End-to-End Content and Plan Selection for Data-to-Text Generation. arXiv preprint arXiv:1810.04700.
Strobelt, H., et al., 2018. Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models. arXiv preprint arXiv:1804.09299.
Gehrmann, S., et al., 2018. End-to-End Content and Plan Selection for Natural Language Generation, Publisher's VersionAbstract

This paper describes our entry for the INLG 2018 E2E NLG challenge. Generating flu- ent natural language descriptions from struc- tured data is a key sub-task for conversa- tional agents. In the E2E NLG challenge, the task is to generate these utterances conditioned on multiple attributes and values. Our sys- tem utilizes several extensions to the general- purpose sequence-to-sequence (S2S) architec- ture to model the latent content selection pro- cess, particularly different variants of copy at- tention and coverage decoding. In addition, we propose a new training method based on diverse ensembling to encourage the model to learn latent plans in training. We empirically evaluate these techniques and show that the system increases the quality of generated text across five automated metrics. Out of a total of sixty submitted systems from 16 institutions, our best system ranks first-place in three of the five metrics, including ROUGE.

Strobelt, H., et al., 2018. LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks. IEEE Transactions on Visualization and Computer Graphics , 24 (1) , pp. 667-676.
Gehrmann, S., et al., 2018. Comparing deep learning and concept extraction based methods for patient phenotyping from clinical narratives. PloS one , 13 (2) , pp. e0192360.
Wu, J.T., et al., 2017. Behind the Scenes: A Medical Natural Language Processing Project. International Journal of Medical Informatics. Publisher's Version
Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks
Strobelt, H., et al., 2016. Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks. Publisher's VersionAbstract

Recurrent neural networks, and in particular long short-term memory networks (LSTMs), are a remarkably effective tool for sequence modeling that learn a dense black-box hidden representation of their sequential input. Researchers interested in better understanding these models have studied the changes in hidden state representations over time and noticed some interpretable patterns but also significant noise. In this work, we present LSTMVis a visual analysis tool for recurrent neural networks with a focus on understanding these hidden state dynamics. The tool allows a user to select a hypothesis input range to focus on local state changes, to match these states changes to similar patterns in a large data set, and to align these results with domain specific structural annotations. We further show several use cases of the tool for analyzing specific hidden state properties on datasets containing nesting, phrase structure, and chord progressions, and demonstrate how the tool can be used to isolate patterns for further statistical analysis.

Gehrmann, S., et al., 2015. Deploying AI Methods to Support Collaborative Writing: a Preliminary Investigation. In CHI’15 Extended Abstracts on Human Factors in Computing Systems. ACM. Publisher's VersionAbstract
Many documents (e.g., academic papers, government reports) are typically written by multiple authors. While existing tools facilitate and support such collaborative efforts (e.g., Dropbox, Google Docs), these tools lack intelligent information sharing mechanisms. Capabilities such as "track changes" and "diff"" visualize changes to authors, but do not distinguish between minor and major edits and do not consider the possible effects of edits on other parts of the document. Drawing collaborators’ attention to specific edits and describing them remains the responsibility of authors. This paper presents our initial work toward the development of a collaborative system that supports multi-author writing. We describe methods for tracking paragraphs, identifying significant edits, and predicting parts of the paper that are likely to require changes as a result of previous edits. Preliminary evaluation of these methods shows promising results.