Author ORCID Identifier

0000-0003-0674-779X

Defense Date

2024

Document Type

Dissertation

Degree Name

Doctor of Philosophy

Department

Computer Science

First Advisor

Milos Manic

Second Advisor

Dean Krusienski

Third Advisor

Kathryn Holloway

Fourth Advisor

Alberto Cano

Fifth Advisor

Cang Ye

Abstract

Enabling machines to learn measures of human activity from bioelectric signals has many applications in human-machine interaction and healthcare. However, labeled activity recognition datasets are costly to collect and highly varied, which challenges machine learning techniques that rely on large datasets. Furthermore, activity recognition in practice needs to account for user trust - models are motivated to enable interpretability, usability, and information privacy. The objective of this dissertation is to improve adaptability and trustworthiness of machine learning models for human activity recognition from bioelectric signals. We improve adaptability by developing pretraining techniques that initialize models for later specialization to unseen users. We further expand adaptability with models that pretrain from unlabeled data, and can support transfer learning to both new users and new tasks. We address the need for improved trust with an engineering-informed approach to integrated explainability and reduced model complexity. We also investigate training dataset privacy, another component of trustworthy model building, with methods to evaluate information leakage in our neural representation extraction models. The intersection of adaptability and trustworthiness is especially critical for human activity recognition from bioelectric signals. Modeling human physiology has broad applications across many domains and, due to its highly personalized nature, must meet high expectations of trust. These are met in part when users can maintain privacy and understand how the system using their data operates. Further heightening the need for trustworthy systems is the variability in bioelectric sensor data, which can often uniquely identify different users and their system configurations apart from others. It is this same variability that makes generalized machine learning models for human activity recognition from bioelectric signals challenging. Aspects like varying electrode locations, skin salinity, power-line noise, all conflate with the wide diversity of human behavior and physiology to make adaptability difficult. In order to address these challenges, we present an interpretable convolutional model for the task of speech detection from neural signals measured with electrocorticography, and we validate that it discovered features consistent with current neuroscience literature. We use a transfer learning approach to adapt hand-pose recognition models to new users with commodity electromyography devices, without the need for expensive pre-processing used in prior work. We develop an approach for self-supervised learning from stereotactic electoencephalogram signals, enabling adaptation to unseen tasks and users from unlabeled data. Finally, in order to understand privacy risks, we assess information leakage using re-identification and membership inference attacks against our neural representation learning methodology. Our work, as applied to existing and emerging human-computer interfaces, demonstrates that machine learning can be made to both support human well-being and adapt to our complexity, without abandoning pursuit of user trust.

Rights

© The Author

Is Part Of

VCU University Archives

Is Part Of

VCU Theses and Dissertations

Date of Submission

4-10-2024

Download

Share

COinS