Explorations in Emoji-based P300 BCI-Spellers and Convolutional Neural Network Optimization for SSVEP-based Bio-Signals

Abstract

The past decade has seen significant enhancements in the development of communication-based Brain-Computer Interface (BCI) spellers. These devices often harness brain-based bio-signals via Electroencephalography (EEG) for speller control. To increase the scope of speller viability and functionality we first developed a simplistic emoji-based visual speller paradigm using the P300 bio-signal. The inclusion of emojis over traditional letters, numbers and characters is predicted to enable richer emotional communication capabilities to end-point users with the most severe forms of paralysis. Here is presented a staggered exploration of stimulus design formats ranging between 3, 5 and 7 target emoji arrays positioned from agreeable to disagreeable. In the final iteration of the experimental procedure, a closed-loop system is assessed using 3 neurotypical subjects. This necessitated the real-time capture, pre-processing, classification, and prediction presentation of subject dry-EEG data. The highest-performing single-subject achieved 83% offline classification accuracy for an analysis variant utilizing SMOTE oversampling data augmentation. The final chapter of the thesis focuses on the optimization of pre-processing frequency filters for SSVEP-based bio-signal classification using a range of convolutional neural networks (ShallowConvNet, DeepConvNet, EEGNet & EEGNetSSVEP). All analyses were computed utilizing the open-source 12-target, 10-subject Nakanishi SSVEP Numpad repository. These investigations revealed a positive trend in optimized low-pass filter cutoffs for networks presenting with a greater number of trainable parameters, or a higher model layer count. These results align with current cutting-edge CNN SSVEP classifier research and suggest the effective extraction of SSVEP harmonics is dependent on network complexity. Further, the optimization of aggregated, cross-subject data pre-processing frequency filter cutoffs is shown to enhance subject-level classification performance for both high and low-complexity networks. These methods provide a guideline for research into the optimization of cross-subject dataset pre-processing stages and outline a paradigm for the optimal comparison of CNNs for SSVEP classification.