Doctoral Thesis Defense
Disambiguation of Imprecise User Input Through Intelligent Assistive Communication

Slides

Relevant Links

Thesis

Intelligent interfaces can mitigate the need for linguistically and motorically precise user input to enhance the ease and efficiency of assistive communication.

Abstract

Augmentative and alternative communication (AAC) systems are used by people with speech impairments severe enough to preclude the use of spoken communication. While communication systems for non-disabled users often implement intelligent prediction, correction, and behavior adaption, current AAC systems are relatively passive conduits for translating user intentions into spoken output. This dissertation seeks to shift the burden of communication from the user to the system by leveraging knowledge of the user's abilities, usage patterns, and contextual needs. The ultimate goal is to create an assistive communication prosthesis that enables users to seamlessly engage in timely and meaningful interactions. Towards that end, this dissertation makes the following contributions in the areas of natural language processing (NLP) and human-computer interaction (HCI):

  1. A word-level language model — semantic grams — that bridges the gap between syntax and semantics by leveraging an author's own syntactic delimiters of semantic content. This model is more effective than similar n-gram-based language models for prediction tasks with unusual ordering or syntax.
  2. An empirical comparison of contextual language predictors, showing that the use of statistics from a global corpus, such as the New York Times, is sub-optimal. Instead, situational context can provide more accurate background probabilities for pervasive speech and language processing tasks.
  3. Results and observations from a touchscreen tablet study with current and potential AAC users, quantifying the challenges faced by people with upper limb motor impairments and showing how they can be addressed through intelligent interfaces.
  4. Three user-driven interface designs and prototypes, including an approach to icon-based AAC that can be controlled effectively with a single input signal and leverages semantic frames to accommodate different screen sizes and user abilities.

Thesis Committee

Rupal Patel, Ph.D.

  • Role: Thesis Advisor
  • Affiliation: Northeastern University
  • Email Address: r.patel at neu.edu

Javed Aslam, Ph.D.

  • Role: Committee Member
  • Affiliation: Northeastern University
  • Email Address: jaa at ccs.neu.edu

David Smith, Ph.D.

  • Role: Committee Member
  • Affiliation: Northeastern University
  • Email Address: dasmith at ccs.neu.edu

Shaun Kane, Ph.D.

  • Role: External Member
  • Affiliation: University of Colorado Boulder
  • Email Address: shaun.kane at colorado.edu

Related Publications

Wiegand, K. & Patel, R. (2015). Contextual language predictors for daily assistive technology. ACM Transactions on Accessible Computing (in revision).

Wiegand, K. & Patel, R. (2015). Impact of motor impairment on full-screen touch interaction. Journal on Technology and Persons with Disabilities, 3 (to appear).

Wiegand, K. & Patel, R. (2014). RSVP-iconCHAT: A single-switch, icon-based aac interface. Journal on Technology and Persons with Disabilities, 2 (to appear).

Wiegand, K. (2014). Intelligent assistive communication and the web as a social medium. In Proceedings of the 11th Web for All Conference (W4A '14). ACM, New York, NY, USA, Article 27, 2 pages. [ACM] [PDF] [BibTeX] [Reviews]

Wiegand, K., & Patel, R. (2014). DigitCHAT: enabling AAC input at conversational speed. In Proceedings of the 16th international ACM SIGACCESS conference on Computers and accessibility (ASSETS '14). ACM, New York, NY, USA, 343-344. [ACM] [PDF] [BibTeX] [Post-Publication Feedback]

Wiegand, K. (2013). Semantic disambiguation of non-syntactic and continuous motion text entry for AAC. ACM SIGACCESS Accessibility and Computing 105 (January 2013), 38-43. [ACM] [PDF] [BibTeX] [Reviews]

Wiegand, K., & Patel, R. (2012). Non-syntactic word prediction for AAC. In Proceedings of the Third Workshop on Speech and Language Processing for Assistive Technologies (SLPAT '12). Association for Computational Linguistics, Stroudsburg, PA, USA, 28-36. [ACM] [ACL] [PDF] [BibTeX] [Reviews]

Wiegand, K., & Patel, R. (2012). SymbolPath: a continuous motion overlay module for icon-based assistive communication. In Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility (ASSETS '12). ACM, New York, NY, USA, 209-210. [ACM] [PDF] [BibTeX] [Reviews]