Paper deep dive
Automated Multilabel Mpox Research Classification with Explainable Transformer Models
Tanjim Taharat Aurpa
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/4/2026, 11:00:41 AM
Summary
This study addresses the challenge of organizing the growing volume of Mpox research by implementing an automated multilabel classification system. Using a dataset of 14,590 articles from PubMed, the authors evaluated several transformer-based models (BERT, RoBERTa, ALBERT, DistilBERT, ELECTRA) to categorize research into three topics: outbreaks, prevention, and vaccines. BERT achieved the highest performance with 97.05% accuracy. The study also employed SHAP (SHapley Additive exPlanations) to provide explainable AI insights, identifying key word features driving the classification decisions, thereby aiding researchers and policymakers in efficiently navigating Mpox literature.
Entities (10)
Relation Signals (5)
BERT â achievedbestperformance â Mpox Research Classification
confidence 95% ¡ Among the different AI models tested, BERT performed the best, achieving 97.05% accuracy
SHAP â usedforexplainability â BERT
confidence 92% ¡ To better understand how the model makes decisions, SHAP was used to analyze significant word features
BERT â outperformed â RoBERTa
confidence 90% ¡ BERT has an Accuracy of 97.05%... RoBERTa scores 95.75
PubMed â sourceof â Mpox Research Articles
confidence 90% ¡ Collecting research abstracts related to MPox from PubMed
Mpox â monitoredby â World Health Organization
confidence 85% ¡ WHO (World Health Organization) reporting increasing cases
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The Mpox outbreak remains a serious public health issue, with the WHO (World Health Organization) reporting increasing cases in some regions. Research on Mpox is vital for several reasons, including vaccine development, diagnostic improvement, viral evolution studies, and preventing future outbreaks. However, the large amount of research being published makes it difficult to organize and analyze information efficiently. This study focuses on using multilabel classification to categorize 14590 Mpox research articles into key topics such as outbreaks, vaccination, and epidemiology. Among the different AI models tested, BERT performed the best, achieving 97.05% accuracy, 97.67% micro F1 score, and 96.46% macro F1 score. To better understand how the model makes decisions, SHAP was used to analyze significant word features and patterns. The results show that BERT can help automate the classification of Mpox research, making it easier for researchers, policymakers, and healthcare workers to quickly find relevant information, saving time and improving public health efforts.
Tags
Links
- Source: https://arxiv.org/abs/2607.26700v1
- Canonical: https://arxiv.org/abs/2607.26700v1
Trouble viewing inline? Open PDF directly â
Full Text
55,184 characters extracted from source content.
Expand or collapse full text
Received: Added at productionRevised: Added at productionAccepted: Added at production DOI: x/x A R T I C L E T Y P E Automated Multilabel Mpox Research Classification with Explainable Transformer Models Tanjim Taharat Aurpa 1 Department of Data Science and Engineering, University of Frontier Technology, Bangladesh (UFTB) Correspondence Corresponding author Tanjim Taharat Aurpa, This is sample corresponding address. Email: aurpa0001@uftb.ac.bd Abstract The Mpox outbreak remains a serious public health issue, with the WHO (World Health Organization) reporting increasing cases in some regions. Research on Mpox is vital for several reasons, including vaccine development, diagnostic improvement, viral evolution studies, and preventing future outbreaks. However, the large amount of research being published makes it difficult to organize and analyze information efficiently. This study focuses on using multilabel classification to categorize 14590 Mpox research articles into key topics such as outbreaks, vaccination, and epidemiology. Among the different AI models tested, BERT performed the best, achieving 97.05% accuracy, 97.67% micro F1 score, and 96.46% macro F1 score. To better understand how the model makes decisions, SHAP was used to analyze significant word features and patterns. The results show that BERT can help automate the classification of Mpox research, making it easier for researchers, policymakers, and healthcare workers to quickly find relevant information, saving time and improving public health efforts. K E Y W O R D S MPox, Monkey Pox, BERT, XAI, SHAP 1INTRODUCTION MPox, also known as Monkey Pox, is a harmful disease caused by the monkeypox virus. A Mpox career can be responsible for transmitting the virus to someone else. According to WHO â , 1951 was the year when this virus was invented in Denmark for the first time. After that, many countries have seen a devastating outbreak of this disease. Recently, a variant of Mpox has been spread over 120 countries from May 2022 to August 2024. During this outbreak, more than one lac laboratory-confirmed case was found, and 220 people died from being infected with Mpox. Recent research has identified multiple variants of the Mpox virus, emphasizing the evolving nature of the disease and its implications for public health. Two primary clades, Clade I (Central African) and Clade I (West African), have been discovered. However, Recent subvariants Clade Ib and IIb have combined and show various transmission dynamics and virulence. The recent surge in cases, particularly in Africa, has been alarming, with Mpox being declared a public health emergency and the WHO showing international concern in 2024. The spread into regions with limited previous exposure, such as East African nations like Burundi, Rwanda, and Uganda, highlights the virusâs adaptability. Therefore, these evolving variants underline the need for enhanced surveillance, vaccination strategies, and public health interventions to control the spread of Mpox and mitigate its impact. Mpox research is fundamental because the virus can cause large outbreaks, especially in places where healthcare systems are underdeveloped. Learning about the virus and creating ways to prevent and treat it can significantly reduce its impact on global health. Moreover, this research can find a way to offer cheaper and more efficient vaccines and With more animal diseases jumping to humans, studying Mpox helps us prepare for future infections. Recent outbreaks in new regions have highlighted how urgent this research is to protect public health and stop pandemics before they start. Existing research focuses mainly on outbreaks of this virus, prevention, and vaccinations. Again, one paper can relate to more than one of these subtopics. This work â https://w.who.int/news-room/fact-sheets/detail/mpox Journal 2023;00:1â16wileyonlinelibrary.com/journal/Š 2023 Copyright Holder Name1 arXiv:2607.26700v1 [cs.CL] 29 Jul 2026 2 will benefit researchers by proposing better public health policies, improving how we respond to outbreaks, and making us more ready for viral epidemics by utilizing Natural Language processing. Utilizing modern transformer-based architecture such as BERT RoBERTa, AlBERT, DistillBERT, ELECTRA, and XLNet to analyze recent research on MPox is the primary motivation of this paper. These architectures have provided remarkable performance in different NLP tasks 3 , 1 , 2 , 4 etc. These transformer models work in two steps: model pretraining and fine- tuning with downstream tasks. These models utilized numerous multi-label classification-based types of research. Multi-label classification, an NLP problem where a given input text is classified as more than one class, is a versatile tool. Transformer-based architectures have shown significant performance in this area, particularly for fine-tuning downstream tasks. As mentioned earlier, three major research classes were identified, and are âPrevention-related,â âOutbreak-related,â and âVaccine-related.â A single research can be conducted on all of these topics, demonstrating the adaptability of multi-label classification to diverse research topics. This makes it a valuable and time-saving tool for healthcare professionals, researchers, and public health authorities. Explainable AI(XAI) is a valuable technique that helps researchers disclose deep learning black boxes. One of the popular XAI techniques, SHapley Additive exPlanations(SHAP), helps to understand important features that contributed to the predic- tion, modelâs transparency, error calculation, etc. SHAP is widely utilized in various domains, including text classification 5 , multimodal prediction 6 , and image processing 7 , among others. In this work, SHAP will help to understand how and for which words the research articles are classified. The main goal of this work is listed below: ⢠Utilization of modern NLP techniques by proposing a multi-label classification approach for classifying MPox research articles. ⢠To explore the effectiveness of transformer-based models in classifying MPox research articles into outbreak, prevention, and vaccine categories. ⢠Collecting research abstracts related to MPox from PubMed ⥠and analyzing them to bring out significant insight for researchers. ⢠Understand the modelâs transparency by understanding the most important word features for each class by applying the XAI technique SHAP. The research questions which this work will address are mentioned below: 1How effective are transformer-based architectures (e.g., BERT, RoBERTa) in performing multi-label classification for MPox-related research into categories such as outbreak, prevention, and vaccine? 2 What insights can SHAP provide about the critical features (e.g., words or phrases) that drive the classification of MPox-related research articles? 3 What are the challenges and solutions in applying multi-label classification to overlapping topics within MPox research? 4 How can multi-label classification and XAI techniques improve public health strategies and decision-making for managing MPox outbreaks? 2 LITERATURE REVIEW Because of BERTâs exceptional success in addressing a variety of difficulties, it has drawn a lot of interest from researchers. For instance, 12 focused on a Twitter dataset, pretraining BERT, to improve Latvian sentiment analysis for tweets. Furthermore, BERT has shown its efficacy in fake news detection. In 13 , the researchers proposed a hybrid approach by integrating BERT with an LSTM layer, resulting in a 2.50% accuracy increase on the PolitiFact dataset and a 1.10% increase on the GossipCop dataset. Additionally, 1 explored an ensemble of two BERT models for recognizing medical entities, achieving a remarkable accuracy improvement of 11.80%. In 14 , the authors employed SHAP after classifying the relations between two math entities, using various transformer-based architectures to highlight the key features that influence predictions. Rabbi et al. applied SHAP for multi-label text classification on COVID-19-related texts, where Random Forest and BERT emerged as the top-performing classifiers 15 . Additionally, SHAP ⥠https://pubmed.ncbi.nlm.nih.gov/ 3 has been used in fake news detection, authentic news recognition, and question classification tasks, as explored in 5 , covering binary, multi-class, and multi-label classification scenarios in order to demonstrate explainability. MPox is one of the current significant concerns for WHO. Thus, besides other domains there, much work has been con- ductedâmostly sentiment analysis on social media text has been performed by authors. In 16 , authors used the hydrator to collect 61,862 tweets related to MPox and applied VADER for sentiment analysis. Their analysis shows that almost half of the observations (46.88%) tweets are negative, and the remaining tweets are positive and neutral. Another work 17 is where authors collected 50000 multilingual tweets related to MPox. Here, they used not only VADER but also TextBlob to determine sentiment labels, and then they applied different machine learning models. They achieved the highest 0.9348 accuracy with the TextBlob annotation and CountVectorizer as the vectorization technique. Some work has been performed on topic modeling of mpox-related text. Thakur et al. citethakur2023analyzing performed topic modeling using 601,432 Tweets and determined a total of 50 topics. The authors analyzed the average coherence and found the highest confidence value for the topic âViews and Perspectives about MPox.â In 19 , public health massaging and misinformation are analyzed using MPox-related tweets using S-BERT, PCA, and UMAP. A total of 125,424 MPox-related texts are collected, and Latent Dirichlet Allocation is used to determine significant topics. Using the CamemBERT, they analyzed the sentiment 20 . Hajjo et al. (2025) 8 comprehensively review Mpox epidemiology, diagnostics, and treatments, mapping critical host-pathogen interactions to reveal how the virus evades innate immunity. Their network pharmacology analysis identifies key host protein targets for novel drug discovery while highlighting the role of AI and big data in monitoring viral mutations. The study 11 proposes a novel epidemiologically informed particle filtering model (EI-PF) that integrates a deterministic SEIRD framework with stochastic parameter estimation and a unique bivariate Poisson distribution. By incorporating penalty factors to enforce epidemiological constraints, the model eliminates irregular particle trajectories and achieves superior fitting and predictive accuracy on the 2022 U.S. monkeypox outbreak data compared to standard alternatives. Highlighting the global impacts of recent mpox outbreaks, Hayman et al. 9 outline how environmental disruption, forest fragmentation, and regional socio-political factors escalate zoonotic spillover risks. They advocate for a transdisciplinary One Health framework that prioritizes sustainable ecological practices, local community trust, and global resource equity to mitigate future epidemics. This paper 10 introduces a stochastic SPIR compartmental model based on a 3-dimensional Markov chain to derive algorithmic formulas for key epidemic descriptors, such as outbreak size, transmission source impact, and time until death. By integrating an augmented state-space formulation with particle filtering to estimate time-varying parameters from empirical mpox data, the authors significantly improve predictive accuracy and online outbreak assessment compared to traditional constant-parameter methods. 3 MATERIALS AND METHODS 3.1 Preliminary Concepts 3.1.1Bidirectional Encoder Representations from Transformers(BERT) This study makes the utilization of Bidirectional Encoder Representations from Transformers (BERT). The attention mechanism is implemented in this transformer-based architecture. The two primary components of BERT are covered below: ⢠Pre-training BERT: BERT is pre-trained using two primary tasks: Masked Language Modeling (MLM) and Next Sentence Prediction (NSP). In MLM, random tokens within the text are masked, and the modelâs goal is to predict these tokens, which helps it learn a bidirectional representation of language. NSP, on the other hand, enables the model to understand the relationships between sentences by determining whether one sentence logically follows another. BERTâs training data consists of English Wikipedia (excluding lists, headings, and tables) as well as the BooksCorpus, which contains 800 million words. ⢠Fine-tuning BERT: BERT is often used for downstream tasks that involve either single sentences or pairs of sentences as inputs. It begins with pre-trained parameters that capture general language understanding and is subsequently fine-tuned on labeled data for specific tasks, such as classification, question answering, or sentiment analysis. In this work, BERT is fine-tuned for multilabel text classification, which predicts suitable labels for research papers. Figure 1 depicts the architecture of the BERT model for multilable research article classification. 4 E A E A E A E A + + + + + + + + E 2 E 1 E 12 E 13 ....... + + + + + + + + ....... + + + + + E man E [CLS] E mango E [SEP] [CLS] [SEP] T1 T2 TN-1 TN ....... + + + + + Trm Trm Trm Trm Trm Trm TrmTrm BER T Tokrnizer Fully Connected Layer Softmax Output classes ....... Position Embedding Segment Embedding Tokens Embedding Research Article Abstract Transformer Layers (Trm) Sigmoid F I G U R E 1 Proposed BERT Model 3.1.2SHapley Additive exPlanations(SHAP) SHAP(SHapley Additive exPlanations) is a powerful tool that helps us understand which features are most important in our modelâs decisions. It works by assigning an importance score to each feature based on Shapley values, a concept from game theory that ensures a fair distribution of credit among team membersâexcept here, the "team members" are our features. In our case, SHAP helps explain how different aspects of research abstracts play a role in identifying and extracting key classes. This paper incorporates SHAP to explain the decision-making process of the multi-label classification of research articles. By analyzing SHAP values, the most influential features in the modelâs predictions can be identified, along with their role in recognizing entities and their relationships. This approach enhances transparency in classification outcomes and helps refine the model by emphasizing the most important features. ⢠Feature Importance: SHAP helps identify which features, such as specific words, play the biggest role in accurately determining the classes. ⢠Model Transparency: Using SHAP makes the model more understandable by revealing the most important features and the reasons behind its predictions. These explanations improve its reliability, making it a useful tool for improving decision-making. ⢠Error Analysis: SHAP helps to indicate mistakes by showing which features cause incorrect predictions, allowing for targeted improvements to the model. Figure 2 depicts the working process of the SHAP algorithm. 3.2Experimental Setup 3.2.1 Experimental Environment Deep learning models generally require high-performance computational resources to support parallel processing. Therefore, Google Colab was utilized in this study. Google Colab is a cloud-based Jupyter notebook platform that provides integrated GPU and TPU acceleration. The environment runs on Ubuntu and provides access to NVIDIA Tesla K80 GPUs with 12 GB of GPU memory. Additionally, it includes a Python runtime along with pre-installed libraries and packages necessary for executing deep learning applications. 5 Trained Model Unseen Text Shap Values Text Tokenization Shap Plots Model Explanation Feature Importance Error Analysis F I G U R E 2 The working principle of SHAP algorithm 3.2.2Hyperparameter Tuning T A B L E 1 Hyperparameter Settings HyperparameterValue Train/Validation/Test Split Ratio0.2 Random Seed42 Batch Size24 Learning Rate2e-5 Number of Epochs40 Table 1 presents the hyperparameter settings used for training the proposed deep learning model. The dataset was split into training and test sets at a 0.2 ratio. Here, the same data is used for validation and testing. , with a random seed value of 42, were applied to ensure reproducibility of the experimental results. The model was trained with a batch size of 24, meaning 24 samples were processed per training iteration. A learning rate of 2e-5 was selected to control the step size during optimization. Furthermore, the model was trained for 40 epochs, allowing the network to iteratively learn patterns from the dataset and improve performance over multiple training cycles. An Early Stopping criterion was applied to prevent overfitting during training. The training process monitored the validation loss and stopped automatically if no improvement was observed for two consecutive epochs, while aiming to minimize the validation loss. The experimental results are intended to be shown in a large number of observations. Therefore, instead of dividing the data into three parts, the same data have been used as a test and validation set. 3.3 Proposed Framework This subsection is about the proposed model. Figure 3 is the visualization of the workflow of the proposed model. The significant parts of the proposed methodology are discussed below: 6 Research abstract and Classes Hyperparameter Tuning Unseen Text Data Preprocessing Text Tokenization Shap Values Shap Plots Trained Model Model Training XAI (Shap) MPox Related Abstract Dataset Output Classes F I G U R E 3 Workflow of the proposed Methodology 3.3.1Data Collection and Preprocessing The research data collected from PubMed contains Biomedical and life science-related research articles. A total of 14590 articles were analyzed, and the final dataset was created with four distinct features named PMid, Title, Abstract, and labels. To extract data from PubMed, 9 keywords were used as search queries, and they are âmonkeypoxâ, âmonkeypoxvirusâ, âmonkeypoxoutbreakâ, âmonkeypoxawarenessâ, âendmonkeypoxâ, âmonkeypoxvaccineâ, âmpoxâ, âmonkeypoxpreventionâ, âmonkeypoxnewsâ. As the dataset requires a large number of observations to train the deep learning model, no timeframe has been considered. However, the duplicate observations are discarded by checking the titles. The classes were selected based on the most frequently occurring keywords extracted from the abstracts. Initially, the number of classes was increased to four by introducing an additional treatment-related class. However, after conducting a manual analysis of the dataset, it was observed that the inclusion of the additional class introduced significant class imbalance, which negatively affected the performance of the classification models. Consequently, this research was ultimately conducted using three classes to ensure a more balanced dataset and improved model performance. The number of observations in each class is sketched in Figure 5. The outbreak-related class has the highest number of 12046 observations. The other two classes, Prevention-related and Vaccine-related classes, have 9281 and 9271 observations. After collecting the corpus, the text and word frequency are analyzed. Based on the existing text, keywords are extracted, and three different classes are created vaccine: vaccine-related, outbreak-related, and prevention-related. The author manually executes this entire topic modeling process. Finally, based on the classwise keywords, the research articles are classified, and it is found that multiple papers can be categorized with more than one class. Figure 4 shows the word cloud, including the keywords for different classes. Data preprocessing techniques contribute to the modelâs performance. Therefore, different data preprocessing techniques have been applied to our text data. These techniques include removing special characters and stop words, changing upper-case characters to lower-case, and finally, lemmatization and stemming. Figure 6 depicts the data preprocessing techniques used in this paper. 3.3.2Model Training and Evaluation After preprocessing the text, it has been tokenized using the appropriate transformerâs tokenizer. The labels have been encoded into binary arrays, where a value of 1 indicates that a label is valid for the corresponding text, while a value of 0 indicates it is not. Next, the tokenized text and encoded labels are utilized to train the transformer model. The performance of these models is 7 F I G U R E 4 Word Cloud containing classwise keywords 020004000600080001000012000 Number of Observations Outbreak-related Prevention-related Vaccine-related Class 12046 9281 9271 Class Distribution F I G U R E 5 Number of observations for each classes compared using various evaluation metrics, including Binary Accuracy (Equation 1), Micro F1 Score (Equation 3), and Macro F1 score (Equation 2). These metrics are used to conduct the performance comparison. Accuracy = TN + TP TN + TP + FN + FP (1) Macro Average F1 Score = 2Ă P MA Ă R MA P MA + R MA (2) 8 Special Character Removal Stop Words Removal Lower Case Conversion Lemmatization and Stemming F I G U R E 6 The preprocessing steps followed in the paper Micro Average F1 Score = TP TP + 1 2 (FP +FN) (3) TP(net True Positive),TN(net True Negative),FP(net False Positive), andFN(net False Negative) are found from the confusion matrix of each class. Then, the obtained values are averaged to calculate the net values. P MA and R MA stands for the macro average precision and recall. These values are obtained by averaging all the classesâ precision and recall values. 3.3.3 Model Explainability For the modelâs explainability, the SHAP algorithm is used. SHAP has shown which word features are essential in making the prediction and which words are acting negatively for the prediction. 4RESULTS AND DISCUSSION 4.1Results Figure 7 indicates a bar graph that compares the performance of different transformer-based models. The graph shows how well five different AI models (BERT, RoBERTa, ALBERT, DistilBERT, and ELECTRA) perform in extracting relationships between entities (like names, dates, etc.) from text. The three bars for each model represent Accuracy, Micro Average F1 Score, and Macro Average F1 Score. In simple terms, BERT and RoBERTa are the top performers, achieving the highest scores across all three metrics. Specifically, BERT has an Accuracy of 97.05%, a Micro Average F1 Score of 97.67, and a Macro Average F1 Score of 96.46. RoBERTa scores 95.75, 96.53, and 95.75 in the same categories, respectively. On the other hand, ELECTRA doesnât do as well, especially in Macro Average F1 Score, with scores of 82.8, 86.98, and 59.12. This helps in understanding which models are better at processing and understanding text in this context. Table 2 presents the overall multilabel classification performance of the proposed BERT model. The model achieved a Hamming Accuracy of 0.9332 and a low Hamming Loss of 0.0668, indicating that most labels were predicted correctly with minimal classification errors. Furthermore, the model attained high values for Micro Precision (0.9599), Micro Recall (0.9441), and Micro F1-score (0.9519), demonstrating strong overall predictive capability across all labels. Similarly, the Macro Precision, 9 BERTRoBERTAAlBERTDistillBERTELECTRA 60 65 70 75 80 85 90 95 97.05 95.75 96.4 92.84 82.8 97.67 96.53 96.67 95.04 86.98 96.46 95.75 95.17 91.03 59.12 Accuracy Micro average F1 score Macro average F1 score F I G U R E 7 Comparing the Evaluation Metrics(Accuracy, Micro Average F1 Score, Macro Average F1 Score) for different transformer models Recall, and F1-score values of 0.9597, 0.9411, and 0.9498, respectively, indicate that the model performs consistently across classes without significant bias toward any particular category. These findings confirm the effectiveness and robustness of the proposed BERT-based framework for multilabel classification of Mpox research articles. T A B L E 2 Overall Multilabel Classification Performance MetricValue Hamming Accuracy0.9332 Hamming Loss0.0668 Micro Precision0.9599 Micro Recall0.9441 Micro F1-score0.9767 Macro Precision0.9597 Macro Recall0.9411 Macro F1-score0.9646 Table 3 presents class-wise evaluation results for Precision, Recall, F1-score, and Support across the three research categories. The Outbreak-related class achieved the highest performance, with a Precision of 0.9671, Recall of 0.9752, and F1-score of 0.9711, indicating that the model effectively identifies outbreak-related studies. The Prevention-related category also demonstrated strong performance, achieving an F1-score of 0.9336, although its recall value (0.8933) suggests that a small number of relevant instances were missed. Similarly, the Vaccine-related class achieved a Precision of 0.9343, a recall of 0.9550, and an F1-score of 0.9445, confirming the modelâs ability to accurately recognize vaccine-focused articles. Overall, the high class-wise performance metrics indicate that the proposed model can reliably classify Mpox research articles across all categories. T A B L E 3 Class-wise Precision, Recall, F1-score, and Support LabelPrecisionRecallF1-scoreSupport Prevention-related0.97780.89330.93361874 Outbreak-related0.96710.97520.97112415 Vaccine-related0.93430.95500.94451845 10 01 Predicted 0 1 Actual 100638 2001674 Prevention-related 01 Predicted 0 1 Actual 42380 602355 Outbreak-related 01 Predicted 0 1 Actual 949124 831762 Vaccine-related F I G U R E 8 Classwise Confusion Matrix Figure 8 illustrates the class-wise confusion matrices for the Prevention-related, Outbreak-related, and Vaccine-related categories generated by the BERT model. The confusion matrices provide a detailed view of the modelâs prediction performance by comparing actual and predicted labels. For the Prevention-related class, the model correctly identified 1,674 positive instances while misclassifying only a small number of samples. Similarly, for the Outbreak-related class, 2,355 positive instances were correctly classified, demonstrating excellent predictive performance with very few false predictions. In the Vaccine-related category, the model successfully classified 1,762 positive instances, indicating strong discriminative capability. Overall, the confusion matrices reveal that the BERT model achieves a high number of true positive predictions while maintaining relatively low false positive and false negative rates across all classes, further validating its effectiveness for multilabel classification. The ROC (Receiver Operating Characteristic) curve in figure 9 evaluated the performance of a multi-label classification BERT model. The ROC curve indicates the trade-off between the True Positive Rate and the False Positive Rate for each label in the classification task. In this graph, three ROC curves represent different labels: ⢠Prevention-related label: The blue line with an area under the curve (AUC) of 0.9683. ⢠Outbreak-related label: The orange line with an AUC of 0.9675. ⢠Vaccine-related label: The green line with an AUC of 0.9681. AUC values close to 1 indicate excellent performance, meaning the model is very good at distinguishing between the different labels. The diagonal line is a random classifier with an AUC of 0.5, which is used as a baseline for comparison. Since the AUC values for all three labels are significantly higher than 0.5, the BERT model performs well in this multi-label classification task. In summary, the high AUC values suggest that the BERT model is effective at correctly identifying and classifying each label in the dataset, making it a reliable choice for this type of task. For model interpretability and explainability, SHAP is applied to the trained model. As it is a multilabel classification, SHAP text plots are generated for each class. Here, for the class "vaccine-related", the text plot has been added in Figure 10. This SHAP plot illustrates how specific words and phrases influence the classification of research articles into the vaccine-related category. SHAP values quantify the contribution of each word to the modelâs final prediction, where red-highlighted words increase the likelihood of classification into the vaccine-related category, while blue-highlighted words decrease it. The thickness of the arrows represents how strongly each word influences the modelâs prediction. The base value represents the modelâs initial probability before considering specific words, while the final value f x (where x refers to the articleâs category) shows the adjusted probability after key terms are taken into account. The analysis highlights that words like "vaccine," "smallpox," "injection," "vaccination," and "trials" play a significant role in classifying research articles under the vaccine category. In many cases, the model starts with a probability between 0.2 and 0.4, but the presence of these terms significantly increases confidence, often pushing the final likelihood to 0.999995, indicating an almost certain classification. This suggests that research papers discussing vaccination strategies, smallpox vaccines, and immunization trials are strongly associated with the vaccine category. For instance, in one example, the base value is 0.440467, but words like "vaccine" and "smallpox" remarkably increase the probability to 0.999995, confirming the modelâs confidence in classifying the paper as vaccine-related. Similarly, another example with a base value of 0.356493 sees a substantial increase in probability due to the presence of terms such as "trials," "vaccine," and "spread." These results demonstrate that articles 11 0.00.20.40.60.81.0 False Positive Rate 0.0 0.2 0.4 0.6 0.8 1.0 True Positive Rate Prevention-related (area = 0.9683) Outbreak-related (area = 0.9675) Vaccine-related (area = 0.9681) F I G U R E 9 Receiver Operating Characteristic (ROC) Curve for Multilabel Classification with BERT related to vaccines, including the covering of vaccine trials, immunization efforts, and previous pox-related research, are key contributors to the vaccine category. Notably, some words have little to no impact on vaccine classification. In one sample, zoonotic transmission-related terms (e.g., "wild animals," "OpVX," "primates") appear in the text but result in a very low f x value (0.000679184). This suggests that articles focusing on viral spread in animals or environmental reservoirs are less likely to be classified as vaccine-related as they do not directly address vaccination efforts. Overall, the SHAP analysis confirms that the strongest predictors for the vaccine category include direct mentions of vaccines, immunization, smallpox, and clinical trials. The model effectively differentiates vaccine-related articles from those focusing on other aspects of Mpox, such as outbreaks or zoonotic transmission. These results demonstrate the modelâs effectiveness in identifying research papers based on vaccine-related terms, ensuring precise and dependable classification in multilabel tasks. Figure 11 depicts three different SHAP bar plots where the leftmost bar plot indicates the word importance for the Outbreak-related class, the middle one is for the Prevention-related class, and the remaining rightmost bar is for the vaccine-related category. These three SHAP bar plots illustrate the contribution of different words to the modelâs prediction for classifying research articles into different categories. Each plot shows the mean SHAP values for individual words, indicating whether they increase (red) or decrease (blue) the probability of an article being classified under a particular category. Detailed explanation is given below: ⢠Outbreak-related: The word "routine" has the highest positive SHAP value (+1.03), meaning it strongly increases the likelihood of an article being classified as Outbreak-related. Words like "smallpox," "spread," and "injection" have negative SHAP values (-0.08, -0.06, -0.05), meaning they slightly decrease the probability of Outbreak-related classification. Other terms, such as "ocytes," "endemic," and "antibodies," have minor positive contributions but are not as influential as "routine." ⢠Prevention-related: The word "vaccine" has a strong negative SHAP value (-0.38), meaning its presence actually decreases the probability of Prevention-related classification in this specific case. Other words like "spread," "antibodies," "smallpox," and "nation" also show slight negative contributions. Only a few terms, such as "nation" (+0.03) and "control" (+0.03), slightly increase the classification probability. The sum of 357 other features contributes negatively (-0.22), meaning many background words collectively reduce the likelihood of classification. ⢠Vaccine-related: The word "vaccine" has the most substantial positive impact (+0.41), confirming that its presence greatly increases the probability of vaccine classification. Other important contributors include "smallpox" (+0.12), "injection" (+0.08), and "nation" (+0.07), indicating that articles discussing these topics are likely to be vaccine-related. Some words, like "days" (-0.05), "spread" (-0.03), and "endemic" (-0.03), slightly decrease the classification probability. The sum of 12 [0] 0.70.50.30.10.91.11.30.4686240.4686240.468624 base value 0.961830.961830.96183 f Vaccine-related (inputs) opioidsdehydrogenaseelevated lactate 001ase>v<i>p>epacienition facto023 worlch analyching 8nt predics oucludeinicaler retrx of % <i>tworpitang p hes % iveredati700copr< 0ts m%at% aiptihip9etafopesr inputs monkeypox mpox reemerging neglected viral disease by may 2023 worldwide mpox cases surpassed 87000 predictive factors hospitalization mpox lacking we aim compare clinical characteristics outcomes hospitalized nonhospitalized patients mpox infection a multicenter retrospective casecontrol cohort patients mpox infection we performed propensity score match analysis global health network trinetx we compare clinical characteristics outcomes hospitalized nonhospitalized patients mpox of 1477 patients 6% hospitalized 52% required ed visit 29% received treatment urgent care after propensity score matching 80 patients remained group hospitalizations common among black persons 51% <i>versus<i> 33% <i>p<i> 001 people hiv 50% <i>versus<i> 20% <i>p<i> < 00001 proctitis 44% <i>versus<i> 125% <i>p<i> < 0001 independent predictive factors hospitalization cohort mpox included people black diagnosis hiv severe proctitis pain requiring opioids elevated lactate dehydrogenase greater recognition factors associated increased risk mpox severity hospitalization paramount [1] 0.70.50.30.10.91.11.30.4404670.4404670.440467 base value 0.9999950.9999950.999995 f Vaccine-related (inputs) ccivasmallpoxvaccineinjectionffecphmoningymnsittteblcifilmuvi57femaleocytescellsnumberriseumbethynationturneholdecur6miceausedaysosetrarmferhe0ta2rjrel inputs smallpox vaccine injection 2monthold c57bl6j female mice caused first days significant rise number precursor cells rosetteforming lymphocytes sensitive differentiating thymus extract effect the number cells returned normal level 10th day vaccination [2] 0.70.50.30.10.91.11.30.2484260.2484260.248426 base value 0.9999950.9999950.999995 f Vaccine-related (inputs) vaccineolis snsemeaanusatgags higead epreas oveontroinvs hea 20ndenitonomquel iddisuaoloiray epomeulaguct raiting2%coiebrpoiopenoon inputs pathogen genome sequencing epidemics enhances ability identify understand suspected clusters investigate relationships here combine genomic epidemiological data 2022 mpox outbreak better understand early viral spread diversification transmission dynamics by sequencing 52% confirmed cases portugal identified mpox virus sublineages highest impact case numbers fitted global context finding evidence several international sublineages probably emerged spread early portugal we estimated 62% infection reporting rate 13% population men sex men portugal infected we infer critical role played sexual networks superspreader gatherings sauna attendance dissemination mpox virus overall findings highlight genomic epidemiology tool realtime monitoring control mpox epidemics guide future vaccine policy highly susceptible population [3] 0.70.50.30.10.91.11.30.2902220.2902220.290222 base value 0.0006791840.0006791840.000679184 f Vaccine-related (inputs) fect%usmoncrenrirhtauuciueeonosributectopxvalyals topxva copxleeentcrempkeyformorseiraa sborapres2ase84drctinhudpchepshpteismpxke3kinoxlsdrvtepx42mtrintp1 oocpbxempveso7rfois inputs human monkeypox viral zoonosis caused monkeypox virus orthopoxvirus opxv the majority human monkeypox cases reported moist forested regions west central africa particularly democratic republic congo drc in study investigated zoonotic opxv infection among wild animals zambia shares border drc assess geographical distribution opxv we screened opxv antibodies sera nonhuman primates nhps rodents shrews elisa performed realtime pcr detect opxv dna spleen samples serological analysis indicated 38 259 147 % rodents 14 42 333 % shrews 4 188 21 % nhps antibodies opxv the opxv dna could detected spleens animals tested our results indicated wild animals living rural human habitation areas zambia infected opxv [4] 0.70.50.30.10.91.11.30.3564930.3564930.356493 base value 0.9999950.9999950.999995 f Vaccine-related (inputs) smallpoxvaccinevaccivacciimtrialsmunroutinempzatioenssacemptitihoruvoxlefeoohtospreadafricandemiegionentrxdectorsuxcoxmiteretsinmaknsumcsaerimgowrxdm inputs monkeypox mpx virulent orthopoxvirus endemic regions central africa mpx incidence rising since cessation routine smallpox immunization while causes significant disease limited persontoperson spread incidence still relatively low cases generally restricted remote areas difficult access therefore initiating vaccine trials implementing vaccination programs would challenging this paper considers factors may influence future decisions whether mpx vaccination pursued F I G U R E 10 Word featureâs contribution for class "Vaccine Related" F I G U R E 11 SHAP bar plots that contain important features for different classes. 357 other features contributes a strong negative impact (-0.91), suggesting that less relevant words collectively reduce the probability of an article being classified in this category. Table 4 displays the top 20 text tokens ranked by their average SHAP (SHapley Additive exPlanations) importance values across three distinct text classification categories: Prevention-related, Outbreak-related, and Vaccine-related, alongside their Overall Mean importance. Higher numerical values indicate that a specific token carries greater weight in driving the machine learning modelâs classification decisions for that particular class. For instance, the token "control" holds the highest overall importance (0.299), heavily influencing the Prevention-related category (0.523), while "vaccine" naturally serves as a dominant feature for the Vaccine-related class (0.312) but has negligible impact on the others. Ultimately, this feature attribution table illuminates which specific keywords the NLP model relies upon most heavily to differentiate between various public health contexts. 13 T A B L E 4 Average SHAP Importance per Class (Top 20 Tokens) TokenPrevention-relatedOutbreak-relatedVaccine-relatedOverall Mean control0.5230.2050.1690.299 spread0.3870.0720.0170.159 vaccine0.0200.0200.3120.117 endemic0.0330.0850.0610.060 incidence0.0100.1360.0300.059 rising0.0170.0730.0500.047 va0.0180.0150.0970.043 cci0.0120.0140.0750.034 injection0.0130.0490.0330.032 smallpox0.0240.0120.0520.029 antibodies0.0120.0130.0520.026 distribution0.0140.0260.0340.025 mun0.0040.0060.0570.022 im0.0040.0060.0550.022 nosis0.0210.0170.0250.021 nation0.0070.0260.0210.018 early0.0420.0040.0070.018 investigated0.0230.0060.0180.016 effect0.0040.0230.0190.015 transmission0.0160.0180.0100.015 4.2Discussion This research has conducted unique but significant NLP research, where research articles are categorized. Moreover, the transparency and error analysis of the model was observed to understand how the classification was executed, from features to prediction. The practical implications of this work can be helpful for researchers. Before downloading or subscribing to a paper, researchers and academicians can find out that the article is related to the topic they are searching for. Moreover, automatically categorized papers will help researchers to collect papers and to save time during their study. Here different transformer models are trained to find out the best model for this multilabel task.The evaluation metrics for multilabel classification of MPox research articles demonstrate clear performance differences among transformer models, including BERT, RoBERTa, ALBERT, DistilBERT, and ELECTRA. The three key metrics used for assessment are accuracy, micro-average F1 score, and macro-average F1 score. Among all models, BERT achieves the highest performance, with an accuracy of 97.05%, a micro-average F1 score of 97.67%, and a macro-average F1 score of 96.46%. These results indicate that BERT is highly effective in classifying MPox research articles across multiple labels. RoBERTa and ALBERT also exhibit strong performance, with only minor differences in their scores, suggesting they are competitive alternatives to BERT. DistilBERT, a lightweight version of BERT, shows slightly lower performance, with an accuracy of 92.84%, a micro-average F1 score of 95.04%, and a macro-average F1 score of 91.03%. Although it does not match BERTâs performance, its efficiency and reduced computational requirements make it a viable option for classification tasks that prioritize faster inference times over absolute accuracy. ELECTRA, however, performs significantly lower than the other models, with an accuracy of 86.98%, a micro-average F1 score of 82.8%, and a macro-average F1 score of 59.12%. The sharp decline in the macro F1 score suggests that ELECTRA struggles to balance classification across all labels, likely due to difficulties in handling underrepresented classes. Overall, these findings establish BERT as the most effective model for MPox research article classification, followed closely by RoBERTa and ALBERT. While DistilBERT offers a reasonable trade-off between performance and efficiency, ELECTRA appears less suitable for this specific task. Future research could explore fine-tuning these models further or incorporating domain-specific pretraining to enhance classification performance. The graph with the ROC curve for the multilabel classification task further highlights the strong performance of BERT across three distinct categories: Prevention-related, Outbreak-related, and Vaccine-related research. The AUC (Area Under the Curve) values for all three classess are notably high, with Prevention-related at 0.9683, Outbreak-related at 0.9675, and Vaccine-related at 0.9681. These values confirm that BERT is highly effective in distinguishing between relevant and irrelevant instances for each category. The ROC curves are positioned near the upper-left corner of the curve, showing a high true positive rate (sensitivity) and a low false positive rate. This suggests that the model effectively classifies relevant research articles while minimizing misclassification. The Vaccine-related class (AUC = 0.9681) a bit outperforms the other two classes, indicating that BERT is particularly proficient at identifying vaccine-related research papers. However, the marginal differences in AUC values 14 suggest that the model maintains a well-balanced classification capability across all categories. These results further validate the effectiveness of BERT in handling multilabel classification tasks for MPox research articles. The SHAP (SHapley Additive exPlanations) text plot provides valuable insights about the BERT model in making decisions for classifying MPox research articles. The visualization highlights different words that strongly influence the modelâs predictions, with red denoting words that increase the likelihood of classification and blue representing words that decrease it. A key observation is that specific terms such as "vaccine," "smallpox," "injection," "opioids," and "elevated lactate" significantly impact classification. For instance, "vaccine" and "smallpox" act as strong positive indicators of the vaccine-related class, while "opioids" and "elevated lactate" may be associated with the outbreak-related class, possibly due to their link to severe cases requiring hospitalization. The presence of these terms in red suggests that the model effectively associates them with relevant MPox-related topics. The SHAP plot also reveals the context-sensitive nature of BERT, as some words can either increase or decrease classification probability depending on their surrounding context. For example, while "vaccine" is positively contributing to classification in vaccine related category, it is decreasing classification probability in prevention related class, suggesting that the model considers the broader textual context rather than relying solely on individual keywords. Another notable observation is that the sum of 357 other features often contributes negatively to classification probability for prevention related and vaccine related class. However these 357 features provide positive influence for thr outbreak related class. This observation emphasizes that not only individual words are important, but also the structure and contents of a research article also impacting the predictions. The model does not rely solely on single term but evaluates multiple contextual elements to make accuracte decision. The interpretability and explainability provided by SHAP is significant for understanding the AI-driven classification models, particularly in scientific research. By visualizing essential features why the model assigns specific labels to research abstracts, domain experts can validate predictions, identify biasness of the model, and redefine the model as needed. This transparency enhances the credibility of AI-driven classification and ensures that important research findings are categorized accurately. Looking back at the research questions that guided this work, the results answer most of them fairly clearly. BERT came out on top for multi-label MPox classification, beating RoBERTa, ALBERT, DistilBERT, and ELECTRA across every metric, with 97.05% accuracy, a 97.67% micro-F1 score, and a 96.46% macro-F1 score, while RoBERTa and ALBERT trailed only slightly, and ELECTRA fell apart noticeably on the underrepresented classes. The SHAP analysis, meanwhile, showed that the model isnât just keyword-spotting: terms like âvaccineâ, âsmallpoxâ, âinjectionâ, and âtrialsâ consistently pushed predictions toward the vaccine-related class, âroutineâ turned out to be the strongest driver for outbreak-related predictions, and âcontrolâ and âspreadâ did similar work for prevention-related articles, but the same word could push a prediction in opposite directions depending on context, which points to real contextual reasoning rather than simple pattern matching. The challenge of overlapping topics was handled on two fronts, using a genuinely multi-label setup with sigmoid outputs instead of forcing single-label predictions, and trimming the label set from four categories down to three after an added treatment-related class introduced enough imbalance to hurt performance, which suggests overlapping labels were as much a data-curation problem as a modeling one. And on the practical side, pairing strong classification accuracy with SHAP-based explanations seems genuinely useful for public health work, since researchers and policymakers could use a tool like this to sort through MPox literature faster, while the visibility into the modelâs reasoning gives domain experts something to check the predictions against rather than having to trust a black box outright. 5CONCLUSION This study demonstrates the successive performance of transformer-based models, particularly BERT, in the multilabel classifi- cation of MPox research articles. BERT achieved the highest accuracy and F1 scores, confirming its suitability for this task. RoBERTa and ALBERT also performed well, while DistilBERT provided a viable trade-off between efficiency and accuracy. However, ELECTRA showed significantly lower performance than other models. The ROC curve analysis further highlights BERTâs strong predictive ability, with high AUC scores across all three classification categoriesâPrevention, Outbreak, and Vaccine. This suggests that BERT is highly reliable in distinguishing between different research topics related to MPox. SHAP text and bar plots provided transparency about the modelâs decision-making process by identifying words that contribute to the prediction and focusing on the importance of context in multilabel classifications. Overall, this research reinforces the potential of transformer-based models in organizing and analyzing scientific literature. Future work could focus on further fine-tuning, domain-specific pretraining, or integrating additional linguistic and contextual features to enhance classification accuracy. By 15 leveraging advanced NLP techniques, researchers can streamline information retrieval in MPox-related studies, ultimately aiding in public health decision-making and research advancements. FUNDING This research was conducted without any direct funding from public, commercial, or non-profit organizations. COMPETING INTEREST AND ETHICS DECLARATIONS Competing interests: The authors declare no competing interests. Ethics approval and consent to participate: Not applicable. This study does not involve human or animal subjects, and therefore, no ethical approval was required. All authors have reviewed and approved the submission. Declaration of Use Artificial Intelligence (AI): No artificial intelligence (AI) tools or technologies were used in the preparation of this manuscript. References 1. Aurpa, Tanjim Taharat, Farzana Akter, Md Mehedi Hasan, Shakil Ahmed, Shifat Ara Rafiq, Fatema Khan, and Rubel Sheikh. "Bangla MedER: Multi-BERT ensemble approach for the recognition of Bangla medical entity." Plos one 21, no. 2 (2026): e0342558. 2. Cheruku, Ramalingaswamy, Khaja Hussain, Ilaiah Kavati, A. Mallikarjuna Reddy, and K. Sudheer Reddy. "Sentiment classification with modified RoBERTa and recurrent neural networks." Multimedia Tools and Applications 83, no. 10 (2024): 29399-29417. 3.Sy, Christian Y., Lany L. Maceda, Mary Joy P. Canon, and Nancy M. Flores. "Beyond BERT: Exploring the Efficacy of RoBERTa and ALBERT in Supervised Multiclass Text Classification." International Journal of Advanced Computer Science & Applications 15, no. 3 (2024). 4.Aurpa, Tanjim Taharat, Md Shoaib Ahmed, Md Mahbubur Rahman, and Md Golam Moazzam. "InstructNet: A novel approach for multi-label instruction classification through advanced deep learning." Plos one 19, no. 10 (2024): e0311161. 5.Tao, Jie, Lina Zhou, and Kevin Hickey. "Making sense of the black-boxes: Toward interpretable text classification using deep learning models." Journal of the Association for Information Science and Technology 74, no. 6 (2023): 685-700. 6. El-Sappagh, Shaker, Jose M. Alonso, SM Riazul Islam, Ahmad M. Sultan, and Kyung Sup Kwak. "A multilayer multimodal detection and prediction model based on explainable artificial intelligence for Alzheimerâs disease." Scientific reports 11, no. 1 (2021): 2660. 7.Ukwuoma, Chiagoziem C., Dongsheng Cai, Ebere O. Eziefuna, Ariyo Oluwasanmi, Sabirin F. Abdi, Gladys W. Muoka, Dara Thomas, and Kwabena Sarpong. "Enhancing histopathological medical image classification for Early cancer diagnosis using deep learning and explainable AIâLIME & SHAP." Biomedical Signal Processing and Control 100 (2025): 107014. 8.Hajjo, Rima, Osama H. Abusara, Dima A. Sabbah, and Sanaa K. Bardaweel. "Advancing the understanding and management of Mpox: insights into epidemiology, disease pathways, prevention, and therapeutic strategies." BMC Infectious Diseases 25, no. 1 (2025): 529. 9. Hayman, David TS, Marion PG Koopmans, Andrew A. Cunningham, Salome A. Bukachi, Leandre Murhula Masirika, Wanda Markotter, and Thomas C. Mettenleiter. "Mpox: a case study for a one health approach to infectious disease prevention." One Health 20 (2025): 101059. 10. Papageorgiou, Vasileios E., Irene Votsi, and George Tsaklidis. "Dynamic estimation of stochastic descriptors in the SPIR model using particle filtering." Applied Mathematical Modelling (2026): 116770. 11.Papageorgiou, Vasileios E., and Pavlos Kolias. "A novel epidemiologically informed particle filter for assessing epidemic phenomena. Application to the monkeypox outbreak of 2022." Inverse Problems 40, no. 3 (2024): 035006. 12.Thakkar, Gaurish, and M Ě arcis Pinnis. "Pretraining and fine-tuning strategies for sentiment analysis of latvian tweets." In Human Language TechnologiesâThe Baltic Perspective, p. 55-61. IOS Press, 2020. 13.Rai, Nishant, Deepika Kumar, Naman Kaushik, Chandan Raj, and Ahad Ali. "Fake News Classification using transformer based enhanced LSTM and BERT." International Journal of Cognitive Computing in Engineering 3 (2022): 98-105. 14.Aurpa, Tanjim Taharat. "Transparent AI for mathematics: Transformer-based large language models for mathematical entity relationship extraction with XAI." Scientific Reports 16, no. 1 (2026): 13038. 15.Rabby, Gollam, and Petr Berka. "Multi-class classification of COVID-19 documents using machine learning algorithms." Journal of Intelligent Information Systems 60, no. 2 (2023): 571-591. 16 16.Thakur, Nirmalya. "Sentiment analysis and text analysis of the public discourse on Twitter about COVID-19 and MPox." Big Data and Cognitive Computing 7, no. 2 (2023): 116. 17.Bengesi, Staphord, Timothy Oladunni, Ruth Olusegun, and Halima Audu. "A machine learning-sentiment analysis on monkeypox outbreak: An extensive dataset to show the polarity of public opinion from Twitter tweets." IEEE Access 11 (2023): 11811-11826. 18.Thakur, Nirmalya, Yuvraj Nihal Duggal, and Zihui Liu. 2023. "Analyzing Public Reactions, Perceptions, and Attitudes during the MPox Outbreak: Findings from Topic Modeling of Tweets" Computers 12, no. 10: 191. https://doi.org/10.3390/computers12100191 19.Edinger, Andy, Danny Valdez, Eric Walsh-Buhi, Jennifer S. Trueblood, Lorenzo Lorenzo-Luaces, Lauren A. Rutter, and Johan Bollen. "Misin- formation and public health messaging in the early stages of the mpox outbreak: mapping the Twitter narrative with deep learning." Journal of Medical Internet Research 25 (2023): e43841. 20. Movahedi Nia, Zahra, Nicola Bragazzi, Ali Asgary, James Orbinski, Jianhong Wu, and Jude Kong. "Mpox panic, infodemic, and stigmatization of the two-spirit, lesbian, gay, bisexual, transgender, queer or questioning, intersex, asexual community: geospatial analysis, topic modeling, and sentiment analysis of a large, multilingual social media database." Journal of Medical Internet Research 25 (2023): e45108.