Paper deep dive
Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding
Parastoo Azizeddin, Omid Sharafi, Maryam M. Shanechi
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%
Last extracted: 10/8/2026, 7:35:32 AM
Summary
The paper introduces BraVista, a vision-language model framework for unified multi-task EEG decoding. It encodes multichannel EEG signals as structured images using Short-Time Fourier Transform (STFT) spectrograms and interfaces them with a general-domain Vision-Language Model (Qwen3-VL-2B) via instruction-conditioned post-training. The model is evaluated on four datasets (sleep staging, emotion recognition, cognitive workload, abnormal EEG detection), demonstrating strong performance and reliance on EEG-relevant signal structure rather than superficial visual patterns.
Entities (10)
Relation Signals (9)
BraVista → uses → Qwen3-VL-2B
confidence 95% · we adapt a Qwen3-VL-2B model to EEG datasets
BraVista → uses → STFT
confidence 95% · arranges channel-wise short-time Fourier transform (STFT) spectrograms into one structured image
BraVista → evaluatedon → SEED
confidence 90% · SEED emotion recognition
BraVista → evaluatedon → HMC
confidence 90% · We evaluate one jointly trained model on HMC sleep staging
BraVista → evaluatedon → EEGMAT
confidence 90% · EEGMAT cognitive-workload classification
BraVista → evaluatedon → TUAB
confidence 90% · TUAB abnormality detection
BraVista → uses → LoRA
confidence 90% · we employ Low-Rank Adaptation (LoRA) ... targeting the attention and MLP layers of the decoder
STFT → outperforms → raw time-series
confidence 85% · STFT obtains the highest balanced accuracy and weighted F1 across all four datasets
BraVista → comparedwith → NeuroLM
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Learning EEG representations that generalize across cognitive tasks, subjects, and recording conditions remains a key challenge in electroencephalography (EEG) decoding. Recent advances in foundation models have improved EEG decoding performance, yet a fundamental open question remains: how to effectively interface neural signals with these models to enable multi-task learning across datasets. To investigate this question, we introduce BraVista, a visual-language framework that encodes multichannel EEG signals as structured images and enables multi-task learning through instruction-conditioned vision-language models (VLMs). Our approach relies on continued post-training of a general-domain VLM, leveraging its visual and linguistic priors to adapt to neural signals without a separate large-scale EEG-specific pretraining stage. We evaluate BraVista on four datasets spanning sleep staging, emotion recognition, cognitive workload classification, and abnormal EEG detection, showing strong performance across these tasks. Further analyses show that the choice of EEG-to-image representation is critical to performance. Moreover, through controlled perturbations of the EEG signal, we observe a gradual performance degradation under increasing noise, suggesting that the model relies on EEG-relevant information rather than superficial visual patterns. Together, these findings establish structured visual representations as an effective and scalable interface between neural signals and general-domain foundation models for unified multi-task EEG decoding.
Tags
Links
- Source: https://arxiv.org/abs/2610.09355v1
- Canonical: https://arxiv.org/abs/2610.09355v1
Trouble viewing inline? Open PDF directly →
Full Text
51,393 characters extracted from source content.
Expand or collapse full text
Foundation Models for the Brain and Body Multimodal LLMs Can Learn to Read Brain Signals: A Vision–Language Model for Unified Multi-Task EEG Decoding Parastoo Azizeddin Email: azizeddi@usc.edu Omid Sharafi Email: osharafi@usc.edu Maryam M. ShanechiUniversity of Southern California, Los Angeles, CA Email: shanechi@usc.edu Abstract Learning EEG representations that generalize across cognitive tasks, subjects, and recording conditions remains a key challenge in electroencephalography (EEG) decoding. Recent advances in foundation models have improved EEG decoding performance, yet a fundamental open question remains: how to effectively interface neural signals with these models to enable multi-task learning across datasets. To investigate this question, we introduce BraVista, a visual–language framework that encodes multichannel EEG signals as structured images and enables multi-task learning through instruction-conditioned vision–language models (VLMs). Our approach relies on continued post-training of a general-domain VLM, leveraging its visual and linguistic priors to adapt to neural signals without a separate large-scale EEG-specific pretraining stage. We evaluate BraVista on four datasets spanning sleep staging, emotion recognition, cognitive workload classification, and abnormal EEG detection, showing strong performance across these tasks. Further analyses show that the choice of EEG-to-image representation is critical to performance. Moreover, through controlled perturbations of the EEG signal, we observe a gradual performance degradation under increasing noise, suggesting that the model relies on EEG-relevant information rather than superficial visual patterns. Together, these findings establish structured visual representations as an effective and scalable interface between neural signals and general-domain foundation models for unified multi-task EEG decoding. $*$$*$footnotetext: Corresponding author: shanechi@usc.edu.$ $$ $footnotetext: Equal contribution.11footnotetext: Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California22footnotetext: Thomas Lord Department of Computer Science, University of Southern California33footnotetext: Alfred E. Mann Department of Biomedical Engineering, University of Southern California44footnotetext: Neuroscience Graduate Program, University of Southern California 1 Introduction Electroencephalography (EEG) supports diverse cognitive and clinical applications, including epilepsy diagnosis [1], sleep staging [55], affective computing [16], cognitive-state decoding, and abnormality detection. However, EEG signals are highly variable across subjects, recording protocols, and acquisition systems [30]. This heterogeneity makes it difficult to learn representations that can be shared across datasets and decoding tasks. Conventional deep learning approaches, including convolutional and recurrent networks [41, 25, 53, 40, 38, 7], CNN–LSTM models [59, 51], and CNN–Transformer architectures [45, 39], are typically trained for a single dataset or task. Recent EEG foundation models address this challenge by pretraining on large EEG corpora to learn transferable representations [24, 56, 54, 19, 52, 50, 9], with related advances in representation learning for other neural recording modalities [35, 10, 15, 12]. However, they require a large-scale EEG-specific pretraining stage followed by separate adaptation for each downstream task, leaving unified multi-task decoding within a single model underexplored. Recent advances in large language models (LLMs) enable unified multi-task prediction by allowing a single model to adapt to diverse tasks through instruction conditioning [4, 47]. Existing EEG-language models couple EEG features or learned neural tokenizers with language models, enabling EEG tasks to be formulated as instruction-conditioned prediction problems [23, 18, 5, 31, 57]. However, they rely on EEG-specific pretraining to align neural signals with language, making performance dependent on the scale and diversity of the EEG pretraining corpus [17]. We ask whether a general-domain vision–language model (VLM) [29] can instead be interfaced with EEG through a representation it already knows how to process: an image. This is motivated by image-based time-series modeling [20, 61, 58, 22, 11], but its effectiveness for unified EEG decoding remains unclear. We introduce BraVista, which arranges channel-wise short-time Fourier transform (STFT) spectrograms into one structured image and pairs it with a natural-language task instruction. Via continued post-training, we adapt a Qwen3-VL-2B model to EEG datasets. Our study emphasizes three findings: (1) a single BraVista model achieves strong decoding across four distinct BCI tasks; (2) STFT images provide a substantially better VLM interface than raw-signal plots or topographies; and (3) controlled corruption produces gradual performance loss, indicating reliance on EEG signal structure rather than superficial visual cues. Figure 1: BraVista framework. (a) Per-channel STFT spectrograms are arranged into a structured EEG image. (b) A fine-tuned vision encoder processes the image, while a task instruction specifies the dataset, decoding objective, and valid labels. Visual and textual tokens jointly condition a language decoder adapted with LoRA. 2 Method The key idea of BraVista is to use visual representations as an interface that aligns EEG signals with the inductive biases of pretrained vision–language models. Given a multichannel EEG signal, we first transform it into a structured visual representation using a time–frequency mapping. This representation is paired with a task-specific textual instruction, and both modalities are processed jointly by a pretrained VLM. EEG Time–Frequency Representation. We begin by formalizing the input EEG signals as ∈ℝC×TX ^C× T, where C denotes the number of electrodes and T the number of time samples. The signal is partitioned into segments of length L, and then each segment is transformed into a time–frequency representation using the STFT, producing a spectrogram for each channel (see Appendix A for STFT parameters). Then, to form a unified visual input, we arrange channel-wise spectrograms into a structured grid (Fig. 1a). This results in an image-like representation that preserves discriminative time–frequency structure and provides a two-dimensional representation that allows vision-based models to capture temporal–spectral correlations [28, 26]. This representation is well suited for EEG analysis [41, 6, 49], as EEG signals exhibit non-stationary temporal–spectral dynamics, with task-relevant neural patterns encoded jointly across time and frequency [32]. EEG Instruction Construction. For each EEG sample, we construct structured instruction–response pairs that describe the decoding task in natural language. Our instructions provide contextual information about the EEG signal, conclude with an explicit task specification, and the response is defined by the corresponding ground-truth label. This design enables casting EEG decoding as an instruction-following problem and aligns with the training paradigm of VLMs, allowing the model to leverage its pretrained language understanding to perform instruction-conditioned prediction over EEG representations (see Appendix D for task-specific instruction details). Training Framework of BraVista. To adapt a general-purpose VLM to EEG data, we perform continued post-training on the Qwen3-VL-2B model [3], whose pretrained visual understanding provides the foundation for EEG domain adaptation. We keep the original model architecture, which consists of three components: a SigLIP-2 vision encoder [48], a vision–language merger [29], and a language model decoder. We fully fine-tune the vision encoder to capture task-relevant temporal–spectral patterns in EEG data. This allows the model to align its visual representations with the structure of EEG signals while leveraging pretrained visual priors. To adapt the language model while preserving its pretrained linguistic and generative capabilities, we employ Low-Rank Adaptation (LoRA) [14], targeting the attention and MLP layers of the decoder. This formulation constrains task-specific updates to a low-dimensional subspace, enabling parameter-efficient adaptation while limiting drift from the pretrained language model. Optimization for Multi-task EEG Decoding. We formulate multi-task EEG decoding as a conditional generation problem over multimodal inputs. Each input consists of an EEG image ℐEEGI_EEG and a textual instruction =q1,…,qMq=\q_1,…,q_M\. Given these multimodal inputs, we define BraVista as πθ _θ parameterized by θ as an autoregressive conditional distribution over the output sequence =y1,…,yKy=\y_1,…,y_K\: πθ(∣ℐEEG,)=∏k=1Kπθ(yk∣y<k,ℐEEG,), _θ(y _EEG,q)= _k=1^K _θ(y_k y_<k,I_EEG,q), (1) where K is the length of the generated output sequence. This formulation allows the model to condition its predictions jointly on EEG visual features, task-level textual context, and prior outputs. To learn the conditional distribution in Eq. (1), we use the supervised fine-tuning (SFT) objective: ℒ=−∑k=1Klogpθ(yk∣y<k,ℐEEG,).L=- _k=1^K p_θ(y_k y_<k,I_EEG,q). (2) This objective trains the model to generate task-specific outputs conditioned jointly on the EEG visual input and the textual prompt, consistent with instruction-following training in LLMs [36, 47]. During training, gradients are propagated through the vision encoder and the LoRA parameters of the language model, enabling coordinated adaptation of visual and linguistic representations for multi-task EEG decoding. Implementation details are provided in Appendix C.2. 3 Experiments and Results 3.1 Evaluation Datasets and Baselines We evaluate one jointly trained model on HMC sleep staging [2], SEED emotion recognition [60], EEGMAT cognitive-workload classification [63], and TUAB abnormality detection [34]. Dataset statistics are summarized in Table 2. We follow prior preprocessing and splits [19, 18, 62]. Furthermore, for the SEED dataset, we explicitly exclude foundation models whose pretraining data includes SEED to prevent data leakage. We report balanced accuracy as the primary evaluation metric and additionally report weighted F1 scores for ablation studies. All results are presented as mean ± standard deviation over three runs with different random seeds. Our comparisons include: (1) supervised baselines trained separately for each dataset; (2) EEG foundation models, in which a single pretrained encoder is fine-tuned per downstream task; and (3) the EEG-language model NeuroLM, which is the closest unified baseline because it combines a learned EEG tokenizer with an instruction-tuned language model. The unadapted Qwen3-VL backbone provides a zero-shot control for whether generic VLMs alone can decode EEG. Figure 2: Unified decoding across four EEG tasks. (a) Balanced accuracy of BraVista against supervised models, EEG foundation models, NeuroLM, and the zero-shot VLM backbone. (b) Confusion matrices. (c) t-SNE projections of learned embeddings, colored by class. 3.2 BraVista Enables Multi-Task EEG Decoding Across Diverse BCI Tasks We compare BraVista with task-specific supervised and foundation models, the multi-task NeuroLM baseline [18], and the zero-shot VLM. As shown in Fig. 2a, BraVista consistently achieves strong performance while operating as a single unified model on all datasets. Importantly, BraVista substantially surpasses the zero-shot Qwen3-VL-2B across all datasets, highlighting that our post-training effectively aligns visual-linguistic priors with neural signal characteristics. We further analyze the class-level behavior of BraVista using confusion matrices (Fig. 2b). The observed confusion patterns are consistent with known task-specific challenges. For example, in HMC, the model shows the largest confusion between NREM1 and NREM2, whose distinguishing transient graphoelements (such as sleep spindles and K-complexes) may be absent from a single 30-second epoch [33]. Similarly, for the SEED dataset, the confusion between Neutral and Negative classes reflects their overlapping neural patterns in the beta and gamma bands [60]. This suggests that many errors reflect intrinsic task difficulty rather than model failure, and that BraVista captures meaningful physiological structure in EEG signals. Finally, to evaluate the quality of the learned representations, we visualize the embeddings of BraVista using t-SNE (Fig. 2c). The embeddings show clustering patterns corresponding to class labels across datasets. 3.3 Time–Frequency Images Are the Effective Interface Figure 3: The visual interface matters. (a) STFT, raw time-series, and scalp-topography inputs. (b) Balanced accuracy (top) and weighted F1 (bottom) across four datasets. We study how the choice of visual representation affects the ability of VLMs to interpret EEG signals. We replace the STFT grid with raw time-series plots or scalp topographies while holding the model and training procedure fixed. STFT obtains the highest balanced accuracy and weighted F1 across all four datasets (Fig. 3). This result highlights the importance of preserving time–frequency structure when adapting vision–language models to EEG data. Time-series plots retain temporal information and topographic maps capture spatial distributions, but neither provides a joint representation of temporal and spectral dynamics. In contrast, STFT representations provide a dense, structured visualization of neural activity that is more compatible with the visual processing capabilities of pretrained vision encoders. 3.4 BraVista Relies on EEG Signal Structure Under Noise Corruption To assess whether BraVista relies on EEG information encoded in the visual representation, we evaluate its behavior under controlled noise corruption by progressively adding noise to the input signals at different signal-to-noise ratios (20 dB, 10 dB, 5 dB, and 0 dB). As noise increases, performance degrades consistently and gradually rather than abruptly across all datasets for both balanced accuracy and weighted F1, and the performance remains above chance even under severe noise conditions (Fig. 4b). This gradual performance decay suggests that predictions depend on the quality of the underlying EEG signal rather than on superficial visual patterns. Figure 4: Robustness of BraVista to EEG noise. (a) Examples of STFT under increasing noise levels (20 dB to 0 dB). (b) Balanced accuracy and weighted F1. Dashed lines indicate chance performance. 4 Discussion In this work, we present BraVista, a framework that reformulates EEG decoding by interfacing neural signals with general-domain vision–language models. By transforming multichannel EEG recordings into time–frequency images and pairing them with task-specific instructions, we enable unified multi-task decoding within a single model, without task-specific architectures or a separate large-scale EEG-specific pretraining stage. This framework shifts the focus from designing specialized EEG models to designing effective interfaces between neural signals and general-domain foundation models. Visual representations offer one such interface, enabling the reuse of powerful vision–language models for neural signal decoding. Our results provide two main insights. First, the choice of EEG-to-image representation strongly affects decoding performance, with STFT spectrograms providing the most effective interface among the representations evaluated. Second, the gradual degradation under noise corruption, physiologically plausible class-level errors, and structure of the learned embeddings provide convergent evidence that BraVista uses EEG-relevant information rather than relying only on superficial visual patterns. Despite these promising results, several limitations point to important directions for future work. While our framework enables instruction-conditioned multi-task decoding, it currently focuses on producing class labels without explicitly leveraging the reasoning capabilities of VLMs. Integrating reinforcement learning (RL) strategies, such as fine-tuning with process- or outcome-based reward models, presents a promising direction to explicitly guide model reasoning across temporal EEG segments, minimize false positives, and optimize step-by-step diagnostic hypotheses. Additionally, enabling models to reason or integrate contextual information (e.g., subject-specific metadata or clinical history) may improve performance on more complex and clinically relevant tasks [46]. Exploring such capabilities would move beyond classification toward more interpretable and interactive EEG analysis, better aligning neural decoding with the broader strengths of large multimodal models. Beyond decoding, a unified neural decoder could serve as the readout component of closed-loop neurotechnologies that pair neural recording with electrical stimulation [42, 13, 43]. Moreover, because BraVista operates on standard image inputs, it may integrate naturally with connected healthcare and medical imaging infrastructure [8]. Overall, this work suggests that representation design, combined with general-domain foundation models, provides a promising path toward scalable and unified EEG decoding. Acknowledgments and Disclosure of Funding This work was supported by National Institutes of Health (NIH) grants R61MH135407, R01MH123770, and RF1DA056402, the One Mind Rising Star Award, and the Foundation for OCD Research (FFOR). References [1] I. Ahmad, X. Wang, D. Javeed, P. Kumar, O. W. Samuel, and S. Chen (2023) A hybrid deep learning approach for epileptic seizure detection in EEG signals. IEEE Journal of Biomedical and Health Informatics. External Links: Document Cited by: §1. [2] D. Alvarez-Estevez and R. M. Rijsman (2021) Inter-database validation of a deep learning approach for automatic sleep scoring. PloS one 16 (8), p. e0256111. Cited by: §D.1, §3.1. [3] S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, et al. (2025) Qwen3-VL technical report. arXiv preprint arXiv:2511.21631. Cited by: §2. [4] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei (2020) Language models are few-shot learners. In Advances in Neural Information Processing Systems, Vol. 33, p. 1877–1901. Cited by: §1. [5] T. Camaret Ndir, R. T. Schirrmeister, and T. Ball (2025) EEG-CLIP: learning EEG representations from natural language descriptions. Frontiers in Robotics and AI 12, p. 1625731. Cited by: §1. [6] A. Craik, Y. He, and J. L. Contreras-Vidal (2019) Deep learning for electroencephalogram (EEG) classification tasks: a review. Journal of neural engineering 16 (3), p. 031001. Cited by: §2. [7] Y. Ding, N. Robinson, S. Zhang, Q. Zeng, and C. Guan (2023) TSception: capturing temporal dynamics and spatial asymmetry from EEG for emotion recognition. IEEE Transactions on Affective Computing 14 (3), p. 2238–2250. Cited by: §1. [8] M. Einollahi Asgarabad, S. Mohammadi Khaveh, F. Ashtiani, P. S. Arpachaei, and A. Jamali Nazari (2023) Internet of things advances in medical imaging: approaches, applications, and feasibility for improving healthcare. Applications, and Feasibility for Improving Healthcare (January 01, 2023). Cited by: §4. [9] Y. El Ouahidi, J. Lys, P. Thölke, N. Farrugia, B. Pasdeloup, V. Gripon, K. Jerbi, and G. Lioi (2025) REVE: a foundation model for EEG – adapting to any setup with large-scale pretraining on 25,000 subjects. arXiv preprint arXiv:2510.21585. Cited by: §1. [10] E. Erturk, S. Hashemi, and M. M. Shanechi (2025) Cross-modal representational knowledge distillation for enhanced spike-informed LFP modeling. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §1. [11] Z. He, S. Alnegheimish, and M. Reimherr (2026) Harnessing vision-language models for time series anomaly detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 21690–21698. Cited by: §1. [12] M. Hosseini, E. Erturk, S. Hashemi, and M. M. Shanechi (2026) Cross-subject modeling for widefield calcium imaging via atlas-aligned spatiotemporal tokenization. In Forty-third International Conference on Machine Learning, External Links: Link Cited by: §1. [13] N. Householder, O. Sharafi, A. Simonyan, S. T. Walston, G. Lazzi, M. Bienkowski, and K. K. Gokoffski (2026) Extraocular electrical stimulation activates retinal ganglion cells in vivo. bioRxiv, p. 2026–09. Cited by: §4. [14] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, Cited by: §2. [15] S. Javadzadeh, R. Soroushmojdehi, S. A. Seyyed Mousavi, M. Asadi, S. Abe, and T. D. Sanger (2025) Functional embeddings enable aggregation of multi-area SEEG recordings over subjects and sessions. arXiv preprint arXiv:2510.27090. Cited by: §1. [16] R. Jenke, A. Peer, and M. Buss (2014) Feature extraction and selection for emotion recognition from EEG. IEEE Transactions on Affective computing 5 (3), p. 327–339. Cited by: §1. [17] L. P. Jiang, S. Chen, E. Tanumihardja, X. Han, W. Shi, E. Shea-Brown, and R. P. Rao (2025) Data heterogeneity limits the scaling effect of pretraining neural data transformers. bioRxiv. Cited by: §1. [18] W. Jiang, Y. Wang, B. Lu, and D. Li (2025) NeuroLM: a universal multi-task foundation model for bridging the gap between language and EEG signals. In International Conference on Learning Representations, Cited by: 1st item, §D.1, §D.2, §1, §3.1, §3.2. [19] W. Jiang, L. Zhao, and B. Lu (2024) Large brain model for learning generic representations with tremendous EEG data in BCI. In International Conference on Learning Representations, Cited by: 2nd item, §C.3, §1, §3.1. [20] M. Jin, S. Wang, L. Ma, Z. Chu, J. Y. Zhang, X. Shi, P. Chen, Y. Liang, Y. Li, S. Pan, and Q. Wen (2024) Time-LLM: time series forecasting by reprogramming large language models. In International Conference on Learning Representations, Cited by: §1. [21] J. Jing, W. Ge, S. Hong, M. B. Fernandes, Z. Lin, C. Yang, S. An, A. F. Struck, A. Herlopian, I. Karakis, et al. (2023) Development of expert-level classification of seizures and rhythmic and periodic patterns during EEG interpretation. Neurology 100 (17), p. e1750–e1762. Cited by: 1st item. [22] T. Khezresmaeilzadeh, P. Razmara, S. Azizi, M. E. Sadeghi, and E. B. Potraghloo (2025) Vista: vision-language inference for training-free stock time-series analysis. arXiv preprint arXiv:2505.18570. Cited by: §1. [23] J. W. Kim, A. Alaa, and D. Bernardo (2024) EEG-GPT: exploring capabilities of large language models for EEG classification and interpretation. arXiv preprint arXiv:2401.18006. Cited by: §1. [24] D. Kostas, S. Aroca-Ouellette, and F. Rudzicz (2021) BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data. Frontiers in Human Neuroscience 15, p. 653659. Cited by: §1. [25] V. J. Lawhern, A. J. Solon, N. R. Waytowich, S. M. Gordon, C. P. Hung, and B. J. Lance (2018) EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. Journal of neural engineering 15 (5), p. 056013. Cited by: §1. [26] C. Li, Y. Qi, X. Ding, J. Zhao, T. Sang, and M. Lee (2022) A deep learning method approach for sleep stage classification with EEG spectrogram. International journal of environmental research and public health 19 (10), p. 6322. Cited by: §2. [27] H. Li, M. Ding, R. Zhang, and C. Xiu (2022) Motor imagery EEG classification algorithm based on CNN-LSTM feature fusion network. Biomedical signal processing and control 72, p. 103342. Cited by: 4th item. [28] J. Li, Z. Zhang, and H. He (2018) Hierarchical convolutional neural networks for EEG-based emotion recognition. Cognitive Computation 10 (2), p. 368–380. Cited by: §2. [29] H. Liu, C. Li, Q. Wu, and Y. J. Lee (2023) Visual instruction tuning. Advances in neural information processing systems 36, p. 34892–34916. Cited by: §1, §2. [30] F. Lotte, L. Bougrain, A. Cichocki, M. Clerc, M. Congedo, A. Rakotomamonjy, and F. Yger (2018) A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update. Journal of neural engineering 15 (3), p. 031005. Cited by: §1. [31] W. Lu, C. Song, J. Wu, P. Zhu, Y. Zhou, W. Mai, Q. Zheng, and W. Ouyang (2025) UniMind: unleashing the power of LLMs for unified multi-task brain decoding. arXiv preprint arXiv:2506.18962. Cited by: §1. [32] S. Makeig, M. Westerfield, T. Jung, S. Enghoff, J. Townsend, E. Courchesne, and T. J. Sejnowski (2002) Dynamic brain sources of visual evoked responses. Science 295 (5555), p. 690–694. Cited by: §2. [33] R. K. Malhotra (2024) AASM scoring manual 3: a step forward for advancing sleep care for patients with obstructive sleep apnea. Journal of Clinical Sleep Medicine 20 (5), p. 835–836. External Links: Document Cited by: §3.2. [34] I. Obeid and J. Picone (2016) The Temple University Hospital EEG data corpus. Frontiers in neuroscience 10, p. 196. Cited by: §D.4, §3.1. [35] L. L. Oganesian, S. Hashemi, and M. M. Shanechi (2025) BaRISTA: brain scale informed spatiotemporal representation of human intracranial neural activity. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §1. [36] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. F. Christiano, J. Leike, and R. Lowe (2022) Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, Vol. 35, p. 27730–27744. Cited by: §2. [37] W. Y. Peh, Y. Yao, and J. Dauwels (2022) Transformer convolutional neural networks for automated artifact detection in scalp EEG. In 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), p. 3599–3602. Cited by: 3rd item. [38] H. Phan, F. Andreotti, N. Cooray, O. Y. Chén, and M. De Vos (2019) SeqSleepNet: end-to-end hierarchical recurrent neural network for sequence-to-sequence automatic sleep staging. IEEE Transactions on Neural Systems and Rehabilitation Engineering 27 (3), p. 400–410. Cited by: §1. [39] H. Phan, K. Mikkelsen, O. Y. Chén, P. Koch, A. Mertins, and M. De Vos (2022) Sleeptransformer: automatic sleep staging with interpretability and uncertainty quantification. IEEE Transactions on Biomedical Engineering 69 (8), p. 2456–2467. Cited by: §1. [40] Y. Roy, H. Banville, I. Albuquerque, A. Gramfort, T. H. Falk, and J. Faubert (2019) Deep learning-based electroencephalography analysis: a systematic review. Journal of Neural Engineering 16 (5), p. 051001. Cited by: §1. [41] R. T. Schirrmeister, J. T. Springenberg, L. D. J. Fiederer, M. Glasstetter, K. Eggensperger, M. Tangermann, F. Hutter, W. Burgard, and T. Ball (2017) Deep learning with convolutional neural networks for EEG decoding and visualization. Human brain mapping 38 (11), p. 5391–5420. Cited by: §1, §2. [42] O. Sharafi, T. Silliman, H. Mokhtari Dowlatabad, G. Niu, S. T. Walston, J. C. Bouteiller, K. K. Gokoffski, and G. Lazzi (2026) Population-scale analysis of frequency-dependent calcium dynamics in retinal ganglion cells under electric field stimulation. Scientific Reports. Cited by: §4. [43] O. Sharafi, J. Yang, B. Lin, W. L. Wahei, N. HOUSEHOLDER, G. Lazzi, K. Gokoffski, et al. (2026) High frequency electric field stimulation reversibly silences retinal activity and downstream cortical responses. Investigative Ophthalmology & Visual Science 67 (7), p. 3132–3132. Cited by: §4. [44] Y. Song, X. Jia, L. Yang, and L. Xie (2021) Transformer-based spatial-temporal feature learning for EEG decoding. arXiv preprint arXiv:2106.11170. Cited by: 5th item. [45] Y. Song, Q. Zheng, B. Liu, and X. Gao (2022) EEG Conformer: convolutional transformer for EEG decoding and visualization. IEEE Transactions on Neural Systems and Rehabilitation Engineering 31, p. 710–719. Cited by: §1. [46] Y. Torabi, P. Razmara, H. Ajorlou, and B. Baraeinejad (2026) NeuroMambaLLM: dynamic graph learning of fMRI functional connectivity in autistic brains using Mamba and language model reasoning. arXiv preprint arXiv:2602.13770. Cited by: §4. [47] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample (2023) LLaMA: open and efficient foundation language models. arXiv preprint arXiv:2302.13971. Cited by: §1, §2. [48] M. Tschannen, A. Gritsenko, X. Wang, M. F. Naeem, I. Alabdulmohsin, N. Parthasarathy, T. Evans, L. Beyer, Y. Xia, B. Mustafa, O. Hénaff, J. Harmsen, A. Steiner, and X. Zhai (2025) SigLIP 2: multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502.14786. Cited by: §2. [49] C. Wang, V. Subramaniam, A. U. Yaari, G. Kreiman, B. Katz, I. Cases, and A. Barbu (2023) BrainBERT: self-supervised representation learning for intracranial recordings. In International Conference on Learning Representations, Cited by: §2. [50] G. Wang, W. Liu, Y. He, C. Xu, L. Ma, and H. Li (2024) Eegpt: pretrained transformer for universal and reliable representation of eeg signals. Advances in Neural Information Processing Systems 37, p. 39249–39280. Cited by: §1. [51] J. Wang, S. Cheng, J. Tian, and Y. Gao (2023) A 2D CNN-LSTM hybrid algorithm using time series segments of EEG data for motor imagery classification. Biomedical Signal Processing and Control 83, p. 104627. Cited by: §1. [52] J. Wang, S. Zhao, Z. Luo, Y. Zhou, H. Jiang, S. Li, T. Li, and G. Pan (2025) CBraMod: a criss-cross brain foundation model for EEG decoding. In International Conference on Learning Representations, Cited by: 3rd item, §1. [53] P. Wang, A. Jiang, X. Liu, J. Shang, and L. Zhang (2018) LSTM-based EEG classification in motor imagery tasks. IEEE transactions on neural systems and rehabilitation engineering 26 (11), p. 2086–2095. Cited by: §1. [54] C. Yang, M. B. Westover, and J. Sun (2023) BIOT: biosignal transformer for cross-data learning in the wild. In Advances in Neural Information Processing Systems, Vol. 36, p. 78240–78260. Cited by: 1st item, §1. [55] C. Yang, D. Xiao, M. B. Westover, and J. Sun (2021) Self-supervised EEG representation learning for automatic sleep staging. arXiv preprint arXiv:2110.15278. Cited by: 2nd item, §1. [56] K. Yi, Y. Wang, K. Ren, and D. Li (2023) Learning topology-agnostic EEG representations with geometry-aware modeling. Advances in Neural Information Processing Systems 36, p. 53875–53891. Cited by: §1. [57] Z. Zeng, Z. Cai, Y. Cai, X. Wang, J. Chen, R. Wang, Y. Liu, S. Cai, B. Wang, Z. Zhang, and H. Li (2025) WaveMind: towards a conversational EEG foundation model aligned to textual and visual modalities. arXiv preprint arXiv:2510.00032. Cited by: §1. [58] J. Zhang, L. Feng, X. Guo, Y. Wu, Y. Dong, and D. Xu (2025) TimeMaster: training time-series multimodal LLMs to reason via reinforcement learning. arXiv preprint arXiv:2506.13705. Cited by: §1. [59] R. Zhang, Q. Zong, L. Dou, and X. Zhao (2019) A novel hybrid deep learning scheme for four-class motor imagery classification. Journal of neural engineering 16 (6), p. 066004. Cited by: §1. [60] W. Zheng and B. Lu (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Transactions on Autonomous Mental Development 7 (3), p. 162–175. External Links: Document Cited by: §D.2, §3.1, §3.2. [61] S. Zhong, W. Ruan, M. Jin, H. Li, Q. Wen, and Y. Liang (2025) Time-VLM: exploring multimodal vision-language models for augmented time series forecasting. In International Conference on Machine Learning, Cited by: §1. [62] Y. Zhou, J. Wu, Z. Ren, Z. Yao, W. Lu, K. Peng, Q. Zheng, C. Song, W. Ouyang, and C. Gou (2025) CSBrain: a cross-scale spatiotemporal brain foundation model for EEG decoding. In Advances in Neural Information Processing Systems, Cited by: 4th item, §3.1. [63] I. Zyma, S. Tukaev, I. Seleznov, K. Kiyono, A. Popov, M. Chernykh, and O. Shpenkov (2019) Electroencephalograms during mental arithmetic task performance. Data 4 (1), p. 14. Cited by: §D.3, §3.1. Appendix Appendix A Time–Frequency Representation Parameters For each channel c∈Cc∈ C, the STFT is computed as Xc(τ,ω)=∑txc(t)w(t−τ)e−jωt,X_c(τ,ω)= _tx_c(t)\,w(t-τ)\,e^-jω t, (3) where w(⋅)w(·) denotes a window function and τ indexes the window center. We use a window size of 400 samples (2 seconds) with an overlap of 300 samples (1.5 seconds). We retain the magnitude spectrogram Sc(τ,f)=|Xc(τ,2πf)|,S_c(τ,f)= |X_c(τ,2π f) |, (4) which captures localized spectral energy over time. The frequency axis is also truncated to a fixed upper bound of 100 Hz to focus on physiologically relevant EEG bands and to suppress high-frequency noise. Appendix B Baselines Here, we categorize the baseline methods used for our comparisons: Task-specific supervised models: • SPaRCNet [21] is a deep neural network based on one-dimensional CNNs with dense residual connections for EEG decoding. • ContraWR [55] is a CNN-based model that first transforms EEG signals into multi-channel spectrogram representations and then applies a ResNet-style two-dimensional CNN to extract discriminative features from the spectrograms. • CNN-Transformer [37] utilizes CNNs to extract local features and transformer layers to model global dependencies in EEG signals. • FFCL [27] employs parallel CNN and LSTM layers, where the CNN captures spatial features and the LSTM models temporal dynamics; features from both layers are then fused for classification. • ST-Transformer [44] is a transformer-based architecture that uses attention mechanisms to learn spatial and temporal representations from EEG signals. Foundation models: • BIOT [54] is a transformer-based model for learning general-purpose biosignal representations using a mix of supervised and unsupervised pretraining. The pretrained model supports up to 18 EEG channels as input. For recordings with more than 18 channels, a 1×11× 1 convolution is used to reduce the input to 18 channels. • LaBraM [19] is a large-scale EEG foundation model pretrained in an unsupervised manner on over 2,500 hours of diverse EEG data. It learns universal EEG representations by segmenting raw signals into channel-wise patches and employing vector-quantized neural spectrum prediction as a tokenizer, with a Transformer backbone capturing both temporal and spatial features for downstream applicability. • CBraMod [52] is an EEG foundation model that learns generic representations via patch-based masked EEG reconstruction. It employs a criss-cross transformer to jointly model spatial and temporal dependencies across EEG patches in parallel, along with an asymmetric convolutional positional encoding to capture diverse spatial–temporal signal formats. • CSBrain [62] is a cross-subject brain foundation model that captures shared neural patterns across individuals through large-scale pretraining, aiming to enhance robustness and adaptability to downstream EEG analysis tasks. Multi-task LLMs: • NeuroLM [18] is a multi-task EEG foundation model that learns a neural tokenizer that converts EEG signals into discrete tokens, which are passed to a large language model. It is pretrained on a large-scale EEG corpus to learn generic representations and is fine-tuned using instruction tuning. Appendix C Experimental Settings C.1 Implementation Details We train BraVista for 1 epoch with an effective batch size of 128, using a local batch size of 8 and gradient accumulation steps of 4. Training is performed on 4 NVIDIA RTX A6000 GPUs for 12 hours. We apply a linear warm-up over the first 3% of total training steps, followed by a cosine learning rate decay schedule. The language model decoder is optimized using the AdamW optimizer with a peak learning rate of 1×10−41× 10^-4, while the vision encoder is trained with a learning rate of 2×10−62× 10^-6. C.2 Hyperparameter Settings Table 1: Post-training hyperparameters. Hyperparameters Value Base Model Qwen3-VL-2B-Instruct Training mode LLM LoRA + vision encoder Batch size per device 8 Number of devices 4 Global batch size 128 Gradient accumulation steps 4 Epochs 1 Learning rate (LLM/LoRA parameters) 1×10−41× 10^-4 Learning rate (Vision) 2×10−62× 10^-6 Weight decay 0.1 Warmup ratio 0.03 LR scheduler Cosine Freeze vision tower False Freeze LLM True Tune merger False LoRA rank (r) 32 LoRA scaling factor (α) 64 LoRA dropout 0.05 The hyperparameter settings in Table 1 correspond to the jointly trained BraVista model. C.3 Metrics In this section, we describe the evaluation metrics used throughout the paper. We report evaluation metrics that account for class imbalance and task heterogeneity, which are common characteristics of EEG datasets. In many EEG classification settings, label distributions are imbalanced across classes, making standard accuracy insufficient to fully capture model performance. Following prior work in EEG foundation models [19], we adopt metrics that provide a more balanced and informative evaluation. • Balanced Accuracy is computed as the average recall across all classes. It accounts for imbalanced label distributions by giving equal weight to each class and is used for both binary and multi-class classification tasks. This metric is particularly suitable for EEG classification tasks with imbalanced label distributions, as it prevents performance on majority classes from dominating the overall score. • Weighted F1 is the average of per-class F1 scores, weighted by the number of samples in each class. It captures both precision and recall while reflecting the overall data distribution, providing a complementary view of performance that emphasizes effectiveness on more prevalent classes. Appendix D Dataset Details This section describes the datasets, preprocessing procedures, data splitting protocols, and the generated image and instruction for each dataset. Table 2: Evaluation datasets span heterogeneous acquisition settings and tasks. Dataset Task Channels Sampling Segment Samples HMC Sleep stage (5-class) 4 256 Hz 30 s 137,243 SEED Emotion (3-class) 62 1000 Hz 4 s 38,475 EEGMAT Workload (binary) 19 500 Hz 4 s 2,088 TUAB Abnormality (binary) 23 256 Hz 10 s 409,455 D.1 Sleep Stage Classification The Haaglanden Medisch Centrum (HMC) [2] sleep staging dataset is a large EEG corpus collected for automated sleep stage classification. It contains overnight EEG recordings from 151 subjects with segments labeled according to five standard sleep stages: Wake, NREM1, NREM2, NREM3, and REM. EEG signals were recorded from four channels (F4–M1, C4–M1, O2–M1, and C3–M2) using a clinical 10–20 montage and sampled at 256Hz. Following the NeuroLM [18] preprocessing settings, the raw signals are bandpass filtered between 0.1Hz and 75Hz, notch filtered at 50Hz to mitigate power-line interference, resampled to 200Hz, and normalized by dividing signal amplitudes by 100. For data division, the first 100 subjects are used for training, the next 25 subjects for validation, and the final 26 subjects for testing. Each preprocessed EEG segment is converted into an STFT image and paired with a dataset-specific natural language prompt for sleep stage classification. An example STFT image for the sleep staging dataset is shown in Figure 5. The STFT representations of the EEG channels are arranged in a grid according to the channel ordering provided by the dataset. In particular, the 4 channels are placed into a 1×41× 4 grid, where each channel has a resolution of 224×448224× 448 resulting in a final image resolution of 224×1792224× 1792. Figure 5: Sample HMC STFT image. EEG Sleep Stage Classification Prompt <image> You are analyzing a 30-second segment from a full-night electroencephalogram (EEG) recording from the Haaglanden Medisch Centrum (HMC) sleep staging dataset. This dataset contains EEG recordings from 151 subjects, with segments corresponding to one of the following 5 sleep stages: Wake (Conscious awareness), NREM1 (Sleep onset), NREM2 (Light stable sleep), NREM3 (Deep restorative sleep), and REM (Dream associated sleep). Recordings include EEG sampled at 200 Hz recorded from 4 channels: EEG F4-M1, EEG C4-M1, EEG O2-M1, and EEG C3-M2. The current image shows a time-frequency representation (Short-Time Fourier Transform; STFT) of the EEG segment. The image contains four panels (left to right): EEG F4-M1, EEG C4-M1, EEG O2-M1, and EEG C3-M2. In each panel, the horizontal axis represents time (0-30 s) and the vertical axis represents frequency (0-100Hz), spanning conventional EEG bands including delta (∼ 1-4 Hz), theta (∼ 4-8 Hz), alpha (∼ 8-13 Hz), beta (∼ 13-30 Hz), and higher-frequency activity (gamma; ∼ 30-80+ Hz). Color intensity reflects relative spectral power at each time–frequency point, where brighter regions indicate stronger oscillatory power. Your task is to determine the sleep stage associated with this 30-second EEG segment. You MUST only select ONE most appropriate label and enclose it within <class> </class> tags, choosing exactly one of the following labels: "Wake", "NREM1", "NREM2", "NREM3", or "REM". D.2 Emotion Recognition SEED (SJTU Emotion EEG Dataset) [60] comprises EEG recordings from 15 subjects watching emotion-eliciting Chinese film clips corresponding to three affective states: positive, neutral, and negative. EEG signals were recorded using 62 channels at a sampling rate of 1000 Hz across three experimental sessions per subject, with each session containing 15 trials evenly distributed across the three emotion categories. Following the NeuroLM [18] preprocessing settings, the EEG signals are bandpass filtered between 0.1 Hz and 75 Hz, notch filtered at 50 Hz to remove power-line interference, resampled to 200 Hz, and normalized by dividing signal amplitudes by 100. For data splitting, we follow a chronological trial-based protocol: the 15 trials are split into training, validation, and test sets using a 9:3:3 ratio, respectively, and all sessions are merged to form the final splits. Each preprocessed EEG segment is converted into an STFT image and paired with a dataset-specific natural language prompt for emotion classification. An example STFT image for the SEED dataset is shown in Figure 6. The STFT representations of the EEG channels are arranged in a grid according to the channel ordering provided by the dataset. In particular, the 62 channels are placed into an 8×88× 8 grid, where each channel has a resolution of 112×112112× 112 resulting in a final image resolution of 896×896896× 896. Figure 6: Sample SEED STFT image. EEG Emotion Recognition Prompt <image> You are analyzing a 4-second segment from an electroencephalogram (EEG) recording from the SJTU Emotion EEG Dataset (SEED). This dataset contains EEG recordings collected from multiple subjects while they watched emotionally evocative video clips. Each EEG segment corresponds to one of the following emotional states: Positive (pleasant or high-valence emotion), Neutral (emotionally calm or baseline), or Negative (unpleasant or low-valence emotion). Recordings include EEG signals sampled at 200 Hz and recorded from 62 channels. The current image shows a time–frequency representation (Short-Time Fourier Transform; STFT) of the EEG segment. The image contains 62 panels corresponding to each of the EEG channels. In each panel, the horizontal axis represents time (seconds) and the vertical axis represents frequency (0-100 Hz), spanning standard EEG bands including delta (∼ 1-4 Hz), theta (∼ 4-8 Hz), alpha (∼ 8-13 Hz), beta (∼ 13-30 Hz), and higher-frequency activity (gamma; ∼ 30-80+ Hz). Color intensity reflects relative spectral power at each time–frequency point, where brighter regions indicate stronger oscillatory power. Your task is to determine the emotional state associated with this EEG segment. You MUST only select ONE most appropriate label and enclose it within <class> </class> tags, choosing exactly one of the following labels: "Positive", "Neutral", or "Negative". D.3 Mental Arithmetic Task The EEG During Mental Arithmetic Tasks dataset (EEGMAT) [63] comprises EEG recordings from 36 subjects performing mental arithmetic tasks and resting-state conditions. EEG signals were recorded using 19 channels arranged according to the international 10–20 system and sampled at 500 Hz. During the task condition, subjects performed a serial subtraction task, inducing a high cognitive workload, while the resting condition corresponds to a low workload state. Each recording consists of continuous EEG data, which we segment into non-overlapping 4-second windows for classification. The EEG signals are bandpass filtered between 0.1 Hz and 75 Hz, notch filtered at 50 Hz to remove power-line interference, resampled to 200 Hz, and normalized by dividing signal amplitudes by 100. We adopt a subject-independent split, using subjects 0–25 for training, 26–30 for validation, and 31–35 for testing. Each preprocessed EEG segment is converted into an STFT image and paired with a dataset-specific natural language prompt for mental arithmetic detection. An example STFT image for the mental arithmetic dataset is shown in Figure 7. The STFT representations of the EEG channels are arranged in a grid according to the channel ordering provided by the dataset. In particular, the 19 channels are placed into a 5×45× 4 grid, where each channel has a resolution of 112×224112× 224 resulting in a final image resolution of 560×896560× 896. Figure 7: Sample EEGMAT STFT image. EEG Mental Arithmetic Detection Prompt <image> You are analyzing a 4-second segment of an electroencephalogram (EEG) recording from the EEG During Mental Arithmetic Tasks (Zyma et al., 2019) dataset. This dataset contains EEG recordings from 36 subjects from 19 channels sampled at 200 Hz and typically recorded using a standard 10-20 electrode montage. Each EEG segments corresponds either to serial subtraction tasks (high cognitive workload) or to resting-state EEG (low cognitive workload). The current image shows a time-frequency representation (Short-Time Fourier Transform; STFT) of the EEG segment. The image contains 19 panels each corresponding to one EEG channel in the following order: EEG Fp1, EEG Fp2, EEG F3, EEG F4, EEG F7, EEG F8, EEG T3, EEG T4, EEG C3, EEG C4, EEG T5, EEG T6, EEG P3, EEG P4, EEG O1, EEG O2, EEG Fz, EEG Cz, EEG Pz. In each panel, the horizontal axis represents time (0-4 s) and the vertical axis represents frequency (0-100Hz), spanning conventional EEG bands including delta (∼ 1-4 Hz), theta (∼ 4-8 Hz), alpha (∼ 8-13 Hz), beta (∼ 13-30 Hz), and higher-frequency activity (gamma; ∼ 30-80+ Hz). Color intensity reflects relative spectral power at each time–frequency point, where brighter regions indicate stronger oscillatory power. Your task is to determine the cognitive workload level associated with this EEG segment. You MUST only select ONE most appropriate label and enclose it within <class> </class> tags, choosing exactly one of the following labels: "Low" or "High". D.4 Abnormality Detection The Temple University Hospital Abnormal EEG (TUAB) dataset [34] is a large-scale clinical EEG corpus designed for abnormality detection in EEG recordings. It contains EEG data collected from patients in a clinical setting, where each recording is labeled as either normal or abnormal based on expert neurological assessment. EEG signals are recorded using 23 channels following the standard 10–20 electrode montage and sampled at 256 Hz. For this task, continuous EEG recordings are segmented into non-overlapping 10-second clips, each inheriting the label of the corresponding recording. The EEG signals are bandpass filtered between 0.1 Hz and 75 Hz, notch filtered at 50 Hz to remove power-line interference, resampled to 200 Hz, and normalized by dividing signal amplitudes by 100. The dataset provides an official patient-level split into training and test sets; we further divide the training set into training and validation subsets using an 80%:20% ratio. Each preprocessed EEG segment is converted into an STFT image and paired with a dataset-specific natural language prompt for abnormality detection. An example STFT image for the TUAB dataset is shown in Figure 8. The STFT representations of the EEG channels are arranged in a grid according to the channel ordering provided by the dataset. In particular, the 23 channels are placed into a 6×46× 4 grid, where each channel has a resolution of 112×224112× 224 resulting in a final image resolution of 672×896672× 896. Figure 8: Sample TUAB STFT image. EEG Abnormal Detection Prompt <image> You are analyzing a 10-second segment of an electroencephalogram (EEG) recording from the Temple University Hospital Abnormal EEG (TUAB) dataset. This dataset consists of clinical EEG recordings from real patients from 23 channels sampled at 200 Hz and typically recorded using a standard 10-20 electrode montage. The dataset includes both neurologically normal and abnormal cases, and the goal is to classify each segment as "Normal" or "Abnormal". The current image shows a time–frequency representation (Short-Time Fourier Transform; STFT) of the EEG segment. The image contains 23 panels each corresponding to one EEG channel in the following order: EEG Fp1-REF, EEG Fp2-REF, EEG F3-REF, EEG F4-REF, EEG C3-REF, EEG C4-REF, EEG P3-REF, EEG P4-REF, EEG O1-REF, EEG O2-REF, EEG F7-REF, EEG F8-REF, EEG T3-REF, EEG T4-REF, EEG T5-REF, EEG T6-REF, EEG A1-REF, EEG A2-REF, EEG Fz-REF, EEG Cz-REF, EEG Pz-REF, EEG T1-REF, and EEG T2-REF. In each panel, the horizontal axis represents time (0-10 s) and the vertical axis represents frequency (0-100Hz), spanning conventional EEG bands including delta (∼ 1-4 Hz), theta (∼ 4-8 Hz), alpha (∼ 8-13 Hz), beta (∼ 13-30 Hz), and higher-frequency activity (gamma; ∼ 30-80+ Hz). Color intensity reflects relative spectral power at each time–frequency point, where brighter regions indicate stronger oscillatory power. Your task is to determine which class best describes this EEG segment. You MUST only select ONE most appropriate label and enclose it within <class> </class> tags, choosing exactly one of the following labels: "Normal" or "Abnormal".