Paper deep dive
BanglaMed-QA: A Question Answering System for Healthcare Support in Bangla
Rowzatul Zannat, Abdullah Al Shafi, K. M. Azharul Hasan, Atia Shahnaz Ipa
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Medical question answering (QA) systems have become crucial tools for providing reliable health information. But they remain very unexplored for low-resource languages like Bangla due to limited datasets and systems tailored to these languages. To address this, we introduce BanglaMed-QA, a robust QA system specifically designed for the Bangla medical domain. The process begins with building a structured medical knowledge base that includes 4,493 QA pairs in 9 categories under 506 diseases. To improve semantic comprehension, domain-specific root word dictionaries and synonym sets are proposed, in addition to part-of-speech tagging for anaphora resolution. We adopt supervised machine learning models in which SVM is found to be the best model to categorize questions. Multiple similarity metrics, including cosine, Jaccard, BM25, and Levenshtein, are applied with soft and hard voting methods for query matching. The performance of the QA system has been evaluated in two aspects, with a 95% F1 score in an automated evaluation and an average human satisfaction rating of 0.9 out of 1.0. This validates the real-world application of BanglaMed-QA in closing the healthcare information gap for Bangla speakers.
Tags
Links
- Source: https://arxiv.org/abs/2608.28329v1
- Canonical: https://arxiv.org/abs/2608.28329v1
Trouble viewing inline? Open PDF directly â
Full Text
29,320 characters extracted from source content.
Expand or collapse full text
BanglaMed-QA: A Question Answering System for Healthcare Support in Bangla Rowzatul Zannat 1 , Abdullah Al Shafi 2 , K. M. Azharul Hasan 3 , and Atia Shahnaz Ipa 4 Khulna University of Engineering & Technology, Khulna-9203, Bangladesh w.rzrowza@gmail.com 1 ,abdullah@iict.kuet.ac.bd 2 ,az@cse.kuet.ac.bd 3 atia.s.ipa@gmail.com 4 AbstractMedical question answering (QA) systems have become cru- cial tools for providing reliable health information. But they remain very unexplored for low-resource languages like Bangla due to limited datasets and systems tailored to these languages. To address this, we introduce BanglaMed-QA, a robust QA system specifically designed for the Bangla medical domain. The process begins with building a structured medical knowledge base that includes 4,493 QA pairs in 9 categories under 506 diseases. To improve semantic comprehension, domain-specific root word dictionaries and synonym sets are proposed, in addition to part-of-speech tagging for anaphora resolution. We adopt supervised machine learning models in which SVM is found to be the best model to categorize ques- tions. Multiple similarity metrics, including cosine, Jaccard, BM25, and Levenshtein, are applied with soft and hard voting methods for query matching. The performance of the QA system has been evaluated in two aspects, with a 95% F1 score in an automated evaluation and an average human satisfaction rating of 0.9 out of 1.0. This validates the real-world application of BanglaMed-QA in closing the healthcare information gap for Bangla speakers. Keywords:Question Answering System ¡ Medical Information Re- trieval ¡ Anaphora Resolution ¡ Voting Techniques ¡ Human Eval- uation. 1 Introduction One of the most valuable aspects of life is health, and it is natural for people to seek answers when they feel ill or are uncertain about their condition [14]. However, access to reliable and understandable medical information remains beyond the reach of most people, particularly those in areas where English is not the most spoken language [ 11]. The question-answering (QA) systems proved to be of great use in this context. Instant responses with accurate and correct solutions to a particular question are the ones for which the user would normally prefer [ 15]. Instead of searching for a long document, people want to receive only correct, precise, and summarized information. The problem is magnified when Accepted at 3rd International Conference on Big Data, IoT and Machine Learning (BIM 2025) September 25â27, 2025 1 arXiv:2608.28329v1 [cs.CL] 28 Aug 2026 2R. Zannat et al. the question is in low-resource languages such as Bangla, and sometimes the desired information might not be found even after scanning the whole document [14]. Thus, a QA system for the Bangla language in the healthcare field can be extremely beneficial in addressing these issues. But such works are very limited [14]. In this work, our goal is to mitigate the gap and build a robust and user-friendly quality assurance system for healthcare support. Our contributions can be summarized as follows. âWe have constructed a structured knowledge base of the medical domain in Bangla to develop a healthcare support system, as there is a scarcity of such a corpus in the Bangla medical literature. âUsing POS tagging, we have developed an anaphora resolution module that successfully resolves pronouns to their corresponding nouns, suitable for a medical QA system. âWe have explored various supervised machine learning models as our ques- tion classifier to speed up the similarity matching task. âWe have used cosine similarity [11], Jaccard similarity [2], BM25 [5], and Lev- enshtein similarity [18], and then developed a soft and hard voting ensemble of these measures of similarity to find the similarity between the userâs ques- tion and knowledge base and output the answer of the most similar question with a threshold. âWe have also developed a practical system with an interactive user interface (UI) to visualize the usefulness of our proposed system. The paper is organized as follows, Sections1provides the importance of the QA system in the medical domain. Section2compares some existing works. Section 3presents a Bangla Question Answering dataset in the medical area. The suggested method to create the âBanglaMed-QAâ system is described in Section4. The experimental analysis and results are presented in Section5. Finally, a conclusion is drawn in Section 6. 2 Literature Review In this section, we review the landscape of research on the medical question- answering system in the context of methods, datasets, and evaluation approaches that contribute to the field. First, we discuss rule-based, UML, information retrieval(IR), and knowledge graph(KG) based models. We then move on to the recent progress made possible with neural networks and hybrid methods. Jacquemart et al. [ 9] conducted an early feasibility study in medical QA sys- tems to model questions as UMLS-based triples. Although 90% of the student questions followed this structure, there appeared to be issues related to the nor- malization and scalability of the questions. Abacha et al. [ 1] created MEANS, a system that mixes NLP and semantic web tools, SPARQL, and query relax- ation, although it suffered from scalability. Feng et al. [ 6] presented CQASMD, a Chinese QA system that took advantage of large pairs of QA, FastText-based 3. A BANGLA QUESTION ANSWERING DATASET FOR HEALTHCARE SUPPORT3 classification and similarities in disease-symptom vectors. Huang et al. [7] used weighted path classification in conjunction with knowledge graph construction. In the Bangla context, Rahman et al. [11] developed a Bangla chatbot for healthcare care, using supervised learning to predict diseases. For query match- ing, TF-IDF with cosine similarity was used and an English data set was se- lected to translate it into Bangla, thereby addressing the issue of data scarcity. Sen et al. [14] proposed a Bengali medical QA system for the interpretation of prescriptions and reports; however, it suffers due to the size of the data and the lack of clinical testing. [12] developed a Bangla chatbot, namely TUNI, that interacts with the audience for psychological issues with simple sentences. The authors in [17] proposed a medical helper, which was trained on a custom dataset [13], can predict disease based on given symptoms. Recently, Maharjan et al. [10] proposed OpenMedLM, a framework that helps open source LLMs achieve stronger performance in medical quality assurance benchmarks by few shots, self-consistency, etc., but remains limited to multiple choice settings and lacks validation under clinical conditions. Despite these developments, issues such as limited annotated datasets, so- phisticated work in Bangla, and the need for real-world validation persist. 3 A Bangla Question Answering Dataset for Healthcare Support The workflow started with a comprehensive study and a data collection phase. We waded through dozens of online data sources, from health blogs to Wikipedia to newspapers, and verified social media pages and groups. We diligently assem- bled the raw data collection and then cleaned it based on quality, relevance, and ultimately meaning. Question Answer Pair Dataset with Category:The dataset consists of three columns, namely âQuestionâ, âAnswerâ, and âCategoryâ. The âQuestionâ column represents a query related to healthcare, and the âAnswerâ column repre- sents the answer to the corresponding query. There are a total of 4493 QA pairs with 9 categories related to 506 diseases. Table 1shows a sample portion of our medical QA pair dataset. The distribution in Fig.1ashows that all categories have an almost equal number of questions, thus a balanced dataset. Fig.1b shows that maximum diseases are infectious and chronic. POS tagging dataset:We have done the POS tagging of the questions of the QA pair dataset for anaphora resolution. The POS tagging is done like this: ā§āĻĄāĻā§āĻ ā§ /N-DIS āĻŋāĻ/PRP ?/Sym(What/PRP is Dengue/N-DIS ?/Sym). Bangla root word dictionary in medical domain:There is no standard root word dictionary of the medical domain in Bangla. So, we have developed such a dictionary. The dictionary has 617 words. Some examples areāĻĢ ā§ āĻ¸āĻ ā§ āĻŋā§, āĻā§āĻˇ ā§ āĻ§āĻž, ā§āĻĄāĻā§āĻ ā§ , ā§āĻ°āĻžāĻ, āĻāĻžāĻ¨, etc. Synonym sets:We have made a list of synonym sets where each set repre- sents synonym words to handle variations of synonyms to ask the same query. Table 2shows a portion of the root word list. 4R. Zannat et al. Table 1: A sample portion of our Question Answer Pair dataset in the Medical Domain. Here, the samples are about rabies (āĻāĻ˛āĻžāĻ¤āĻā§āĻ). QuestionAnswerCategory āĻāĻ˛āĻžāĻ¤āĻā§āĻāĻŋāĻ? (What is ra- bies?) āĻāĻ˛āĻžāĻ¤āĻā§āĻ āĻāĻāĻŋāĻ āĻ¤ā§āĻŦā§°ā§ āĻāĻžāĻāĻ°āĻžāĻ˛ āĻ¸āĻāĻā§°ā§āĻŽāĻŖāĨ¤ āĻāĻŋāĻ āĻāĻžāĻŽāĻžāĻ° āĻŦāĻž āĻŦāĻ¨āĻ¯ā§ āĻĒā§°ā§āĻžāĻŖā§ā§āĻĻāĻ° āĻĻāĻŦā§āĻžāĻ°āĻž āĻ¸āĻāĻā§°ā§āĻžāĻŋāĻŽāĻ¤ āĻšā§; āĻ¸āĻžāĻ§āĻžāĻ°āĻŖāĻ¤ āĻŽāĻžāĻāĻ¸āĻžāĻļā§ ā§āĻ¯āĻŽāĻ¨ āĻ ā§ āĻ ā§ āĻ°, āĻŋāĻŦā§āĻžāĻ˛, ā§āĻļā§āĻžāĻ˛, ā§āĻ°āĻ ā§ āĻ¨āĨ¤(Rabies is an acute viral infection. It is spread by farm or wild animals, typically carnivores including foxes, raccoons, dogs, and cats.) definition āĻāĻ˛āĻžāĻ¤āĻā§āĻ āĻŋāĻāĻāĻžā§āĻŦ āĻĒā§°ā§- āĻŋāĻ¤ā§āĻ°āĻžāĻ§ āĻāĻ°āĻž āĻ¯āĻžā§? (How can rabies be prevented?) āĻāĻ˛āĻžāĻ¤āĻā§āĻ āĻĒā§°ā§āĻŋāĻ¤ā§āĻ°āĻžā§āĻ§ ā§āĻĒāĻžāĻˇāĻž āĻ āĻ -ā§āĻĒāĻžāĻˇāĻž āĻ¸āĻŦ āĻŋāĻŦā§āĻžāĻ˛-āĻ ā§ āĻ ā§ āĻ°ā§āĻ āĻāĻ˛āĻžāĻ¤ā§āĻā§āĻāĻ° āĻŋāĻāĻāĻž āĻāĻžāĻ¯āĻ°ā§āĻā§°ā§ā§āĻŽāĻ° āĻāĻāĻ¤āĻžā§ āĻāĻ¨āĻžāĻ āĻāĻāĻŋāĻ āĻāĻžāĻ¯āĻ°ā§āĻāĻ° āĻāĻĒāĻžā§āĨ¤(Bringing all domestic and non-domestic cats and dogs under the rabies vaccination program is also an effective way to prevent rabies.) preventions (a)(b) Figure1: (a) Distribution of question categories in our dataset. (b) The propor- tion of types of diseases covered in the study. Table 2: A part of the list of Synonym sets. Any synonyms will be replaced by the selected word, which helps further processing. SynonymsMeaningReplaced By āĻ˛āĻā§āĻˇāĻŖ, āĻŋāĻāĻšā§āĻ¨, āĻĒāĻŋāĻ°āĻā§, āĻāĻāĻžāĻ¸, āĻŋāĻ¨āĻĻāĻļāĻ°ā§āĻ¨, āĻāĻĒāĻ¸āĻāĻ°ā§, āĻ¸āĻ¨āĻžāĻā§āĻ¤SymptomsāĻ˛āĻā§āĻˇāĻŖ āĻĒā§°ā§āĻŋāĻ¤āĻāĻžāĻ°, āĻā§āĻ°āĻžāĻāĻ¯ā§, āĻŋāĻ¨āĻŦāĻžāĻ°āĻŖ, āĻĒā§°ā§āĻļāĻŋāĻŽāĻ¤āĻāĻ°āĻŖ, āĻĻā§āĻ°ā§āĻāĻ°āĻŖ ,āĻŋāĻāĻŋāĻā§āĻ¸āĻžCureāĻĒā§°ā§āĻŋāĻ¤āĻāĻžāĻ° āĻĄāĻžā§ā§ā§āĻŦāĻŋāĻāĻ¸, āĻŦāĻšā§āĻŽ ā§ āĻ¤ā§°ā§, āĻĄāĻžā§ā§ā§āĻŦāĻŋāĻāĻ, āĻĄāĻžā§ā§āĻŦāĻŋāĻāĻ¸, āĻĄāĻžā§ā§āĻŦāĻŋāĻāĻDiabeticāĻĄāĻžā§ā§ā§āĻŦāĻŋāĻāĻ¸ 4 A QA System for Medical Bangla Text Fig.2shows the workflow of our proposed âBanglaMed-QAâ system. It begins with user input, checks for pronoun-based objects (P-OBJ) to resolve anaphora, preprocesses and classifies the query, then matches it with a database to generate an answer. The loop continues until the input is finished. 4. A QA SYSTEM FOR MEDICAL BANGLA TEXT5 Figure2: Flowchart of our proposed âBanglaMed-QAâ system 4.1 Data Preprocessing BanglaMed-QA applies several preprocessing methods to effectively prepare the data. First, we have performed POS tagging using the TnT tagger [ 4], basic anaphora resolution by tracking disease nouns, and punctuation removal. Then, we use tokenization, stop word removal, and custom lemmatization for key term extraction, followed by synonym substitution for better semantic matching and detokenization to formulate the final queries. Fig. 3shows the steps of pre- processing with examples. 4.2 Question Categorization After preprocessing, we have categorized the input question into one of the 9 categories so that the question matching methods become fast as the number of question pairs to check similarity would be reduced. From our QA pair training dataset, We take âQuestionâ as a feature and âCategoryâ as a class. At first, We use a TF-IDF vectorizer with an n-gram range of (1, 2) to convert text into numerical feature representations. This setup allows the model to cap- ture unigrams, and bigrams, enabling it to consider both individual words and 6R. Zannat et al. Figure3: Query Preproceesing Steps. short phrases, which is extremely beneficial in our case. Although not imple- menting explicit backoff, this configuration implicitly provides a form of backoff by including lower-order n-grams when higher-order n-grams are sparse or ab- sent. After that, we have explored 5(five) different supervised machine learning methods, including Perceptron (Perc) [ 17], Multinomial Naive Bayes(MNB) [3], Passive Aggressive Classifier(PAC) [ 11], Random Forest(RF) [8], and Support Vector Machine(SVM) [ 16]. For each of the methods except MNB, we have used hyperparameter tuning with 5-fold cross-validation. All the methods perform well (Section5) and we finally choose SVM as our question classifier. 4.3 Question Matching Methods For the explanation of question matching algorithms, we takeā§āĻ¸ā§āĻā§°ā§āĻžāĻ āĻĒā§°ā§āĻŋāĻ¤āĻāĻžāĻ° (stroke cure) as an example processed query (Q) as well as one of the stored 4. A QA SYSTEM FOR MEDICAL BANGLA TEXT7 queries (S 1 ). AndS i denotes the i-th stored query. All the values given below are taken by running the program (actual value). 4.3.1 Base Similarity Methods Four base similarity methods have been explored in the experiment. TF-IDF with cosine similarity:This method [11] represents each ques- tion using TF-IDF with ngram range (1,2) to emphasize important unigrams and bigrams, and then measures cosine similarity (C) between their vector rep- resentations. C(Q, S i ) =cos(θ) = â Q¡ â S i âĨ â QâĨââĨ â S i âĨ (1) For our example, IDF(ā§āĻ¸ā§āĻā§°ā§āĻžāĻ(stroke)) = 1.693, IDF(āĻĒā§°ā§āĻŋāĻ¤āĻāĻžāĻ°(cure)) = 1.693, IDF(ā§āĻ¸ā§āĻā§°ā§āĻžāĻ āĻĒā§°ā§āĻŋāĻ¤āĻāĻžāĻ°(stroke cure)) = 1.945 TF-IDF (Q) = [0,...., 0.323474, 0, ....,599035,0,....,0.732476], and TF-IDF (S 1 ) = [0,...., 0.323474, 0, ....,599035,0,....,0.732476]. So using Eq. 1, C(Q, S 1 ) = 1.000005 â 1.000005¡ â 1.000005 = 1.0 Jaccard similarity:Jaccard similarity(J) [2] measures lexical overlap by dividing the number of words shared between two questions by the total unique words. J(Q, S i ) = |QâŠS i | |QâĒS i | (2) Query Stored ā§āĻ¸ā§āĻā§°ā§āĻžāĻ āĻĒā§°ā§āĻŋāĻ¤āĻāĻžāĻ° Using Eq.2,J(Q, S 1 ) = |QâŠS 1 | |QâĒS 1 | = 2 2 = 1.0 BM25 similarity:BM25 [5] is a probabilistic retrieval model that ranks documents using term frequency, inverse document frequency, and document length. Let,B(Q, S i )be the raw BM25 score between two questions andB max be the maximum BM25 score observed across all question pairs andB norm (Q, S i ) be the normalized BM25 similarity. BM25(Q, S i ) = n â j=1 IDF(q j )¡ f(q j , S i )¡(k+ 1) f(q j , S i ) +k¡ ( 1âb+b¡ |S i | avgdl ) (3) 8R. Zannat et al. B norm (Q, S i ) = B(Q, S i ) B max ifB max >0,else0(4) Here,q j is each term of Q. And we set k = 1.5 and b=0.75 in our experiment. For our example, term contributions are : ā§āĻ¸ā§āĻā§°ā§āĻžāĻ (stroke):1.693Ã1.1765 = 1.992,āĻĒā§°ā§āĻŋāĻ¤āĻāĻžāĻ° (cure):1.693Ã1.1765 = 1.992, ā§āĻ¸ā§āĻā§°ā§āĻžāĻ āĻĒā§°ā§āĻŋāĻ¤āĻāĻžāĻ° (stroke cure):1.945Ã1.1765 = 2.288 So, using Eq.3and4,BM25(Q, S 1 )= 1.992 + 1.992 + 2.288 = 6.272 and B norm (Q, S 1 )= 1.0 Levenshtein similarity:This method calculates similarity by counting the minimum edits (insertions, deletions, substitutions) needed to transform one string into another, normalized between 0 and 1 [2]. LetL(Q, S i ),L d (Q, S i ), and Mdenote Levenshtein similarity, Levenshtein distance, and maximum length between questions Q andS i respectively, whereM=max(|Q|,|S i |). L(Q, S i ) = 1â L d (Q, S i ) M ifM >0,else0(5) L d (Q, S 1 )= 0âL(Q, S 1 ) = 1â 0 17 = 1.0;Here, M = 17 4.3.2 Ensemble Similarity Methods Soft voting similarity:For the final scoring (S) of the similarity between two sentences, the average of the four similarity measures is taken to combine their strengths for a more balanced result. In the case where the final score is greater than an appropriate threshold, the two sentences are considered similar. S(Q, S i ) = C(Q, S i ) +J(Q, S i ) +L(Q, S i ) +B norm (Q, S i ) 4 (6) Thus,S(Q, S 1 ) = 1.0 + 1.0 + 1.0 + 1.0 4 = 4 4 = 1.0 IfSâĨĪ, then the sentences are considered similar, whereĪis the predefined similarity threshold. We setĪ= 0.8. Hard voting similarity:Under the hard voting (H) strategy, individual similarity measures cast their votes (binary) depending on whether they exceed a set threshold. If the majority votes, meaning 3 or even all 4 votes, deeming the pair as similar, then indeed it is considered similar. Letsim i be the normalized similarity scores for the methods: cosine, Jaccard, Levenshtein, and BM25. The voting logic is defined as follows: m i = 1,ifsim i âĨT 0,otherwise (7) 5. EXPERIMENTAL SETUP, RESULTS AND ANALYSIS9 H(Q, S i ) = 1,if â i m i âĨ3 0,otherwise (8) Here,Tdenotes the threshold value for decision making. We setT= 0.8. Since all four models produce the same prediction (m i = 1for alli), the cumulative vote count is 4 out of 4, resulting in a unanimous hard vote decision of 1. 5 Experimental setup, Results and Analysis 5.1 Evaluation Strategy To evaluate our question categorization methods, we divide our âQA pair with categoryâ dataset into two parts: 80% for training and 20% for testing. Since there was no publicly available dataset aligned with the requirements of our over- all system, we have manually constructed a test set to evaluate the performance and also performed real-time testing through human interaction. The manually crafted data set consists of 880 input queries paired with their corresponding target responses and categories from the stored collection. Special attention was paid to include challenging cases that involve synonym variations, phrased expressions, and domain-specific terminology to analyze the effectiveness of the system under realistic usage. To evaluate the applicability of our Bangla Med- ical QA system in the real world, we performed a human evaluation using an interactive form. Table 3illustrates sample evaluation records in which human participants provided questions, reviewed system responses, and rated between 0 and 1 according to their expectations. A group of 10 evaluators (students) manually entered Bangla medical-related questions. Each evaluator asked 10 questions. These questions reflect natural and practical queries that users might ask in real-world scenarios. 5.2 Evaluation Metrics Important factors to consider when evaluating question categorization models include accuracy, precision, recall, and F1-score. For evaluation of our overall system on the manually crafted dataset, we have treated each transformed query of the test set as a separate feature vector, and predictions that do not match any of them are classified as a separate class, NO_PREDICTION. Coverage is defined as how many predictions are non-None by the model for a given num- ber of examples, indicating how often the model answers to answer. Accuracy, precision, and recall are calculated as the number of examples for which the pre- dicted label is equal to the true label out of the total and NO_PREDICTION is considered to be an incorrect case. F1-score is calculated using the harmonic mean of precision and recall. 10 R. Zannat et al. Table 3: Sample Human Evaluation Record. Each person asked a question using our âBanglaMed-QAâ and the system answers. The peer then rates from 0 to 1 based on his/her satisfaction. PersonQuestionSystem AnswerRating(0-1) P1āĻĄāĻžā§āĻžā§āĻŦāĻŋāĻā§āĻ¸āĻ° āĻāĻĒāĻ¸āĻāĻ°ā§ āĻā§ āĻā§?(What are the symptoms of diabetes?) āĻĄāĻžā§ā§ā§āĻŦāĻŋāĻāĻ¸ āĻāĻ° āĻ˛āĻā§āĻˇāĻŖ āĻ āĻŋāĻ§āĻ āĻā§āĻˇ ā§ āĻ§āĻž ,āĻ āĻŋāĻ¤āĻŋāĻ°āĻā§āĻ¤ āĻŽ ā§ āĻ¤ā§°ā§āĻŋāĻ¨āĻˇā§āĻāĻžāĻ¸āĻ¨, ā§āĻāĻž- ā§āĻ āĻ¸āĻŽāĻ¸āĻ¯ā§āĻž,āĻ āĻŋāĻ¸ā§āĻĨāĻŦāĻžāĻ§āĻž āĻāĻŦāĻ āĻā§āĻ¤āĻžāĨ¤ (Symptoms of diabetes include increased hunger, excessive urination, eye problems, bone loss, and numbness.) 0.95 P2āĻŦāĻšā§āĻŽ ā§ āĻ¤ā§°ā§ āĻŋāĻ? (What is diabetic?) āĻĄāĻžā§ā§ā§āĻŦāĻŋāĻāĻ¸ āĻāĻāĻŋāĻ āĻā§°ā§āĻŋāĻ¨āĻ ā§āĻŽāĻŋāĻĄāĻāĻ¯ā§āĻžāĻ˛ āĻāĻŋāĻ¨ā§āĻĄāĻļāĻ¨ āĻŦāĻž ā§āĻ°āĻžāĻ,ā§āĻ¯āĻāĻžā§āĻ¨ āĻ°ā§āĻā§āĻ¤āĻ° āĻā§āĻ˛ ā§ ā§āĻāĻžāĻ āĻ¸ā§āĻ¤āĻ° āĻ āĻ¸āĻŦā§āĻžāĻāĻžāĻŋāĻŦāĻ āĻāĻžā§āĻŦ āĻŦāĻžā§ā§āĻ¤ āĻĨāĻžā§āĻāĨ¤ (Diabetic is a chronic medical condition or disease in which blood glucose levels increase abnormally.) 0.9 5.3 Quantitative Results Table4shows the optimal hyperparameters of the models for question catego- rization. The values were obtained by first performing baseline validation to find initial values and then using 5-fold cross-validation with grid search. Table 4: Result of Hyperparameter Tuning with 5-Fold Cross-Validation. Here, C is the regularization parameter. MethodsHyperparametersOptimal Value Percalpha0.0001 penaltyl1 PACC1 losssquared hinge max_iter500 RFn_estimators40 max depth50 min samples leaf2 SVMkernellinear C1 Fig.4ashows that all models performed exceptionally well in the catego- rization of the questions, with scores near 1.0 on all metrics. SVM showed the best overall performance, closely followed by RF and PAC. Minimal performance variation indicates high reliability between the selected models. The convergence pattern in Fig.4bindicates a well-fitted model that is neither over-fitted nor under-fitted. This also applies to other models that we employed in our research. From the experiment, we have also observed that this categorization speeds up the question similarity matching approach by around one-ninth (1/9). The outcome of our model evaluation on the manually constructed test dataset is shown in Table5. The performance of base similarity methods is 5. EXPERIMENTAL SETUP, RESULTS AND ANALYSIS 11 (a)(b) Figure4: (a) Performance comparison of various supervised machine learning methods for question categorization. (b) Learning curve of SVM. Table 5: Performance of our proposed method on the manually crafted test dataset. Similarity methodcoverageAccuracyPrecisionRecallF1-score Cosine Similarity1.00.940.940.940.94 Jaccard Similarity1.00.940.940.940.94 BM25 Similarity1.00.940.920.940.93 Levenshtein Similarity1.00.940.940.940.94 Soft Voting Similarity1.00.950.950.950.95 Hard Voting Similarity0.940.940.940.940.94 Figure5: Human evaluation of 10 individuals rating (0-1)along with the average. Bars are color-coded: green for īŋŋ0.9, yellow for 0.8â0.89, red for 0.7-0.79, and grey for < 0.7. 12 R. Zannat et al. Table 6: Qualitative Analysis of Question Categorization Methods. Here, the inputs question is âāĻāĻžāĻāĻĢā§ā§āĻĄ āĻŋāĻāĻāĻžā§āĻŦ āĻāĻŋā§ā§ā§ āĻĒā§ā§?(How is typhoid spread?)â and the actual category is âmode of transmissionâ. MethodPredicted Category Percmode of transmission MNBmode of transmission PACmode of transmission RFmode of transmission SVMmode of transmission Table 7: Demonstrating how anaphora resolution is performed in our system. Here for the second query âāĻāĻŋāĻāĻ°(its)â refers to âā§āĻ¸ā§āĻā§°ā§āĻžāĻ(stroke)â according to the first query. Thus, âāĻāĻŋāĻāĻ°(its)â is replaced by âā§āĻ¸ā§āĻā§°ā§āĻžāĻ(stroke)â to transform to modified second query. QueryPOS taggingModified ā§āĻ¸ā§āĻā§°ā§āĻžāĻ āĻŋāĻ?(What is stroke?) ā§āĻ¸ā§āĻā§°ā§āĻžāĻ/N-DIS āĻŋāĻ/PRP ?/SYM)(What/PRP is stroke/N-DIS ?/SYM)) ā§āĻ¸ā§āĻā§°ā§āĻžāĻ āĻŋāĻ?(What is stroke?) āĻāĻŋāĻāĻ°āĻ˛āĻā§āĻˇāĻŖāĻŋāĻ? (What are its symptoms?) āĻāĻŋāĻāĻ°/PRP-OBJāĻ˛āĻā§āĻˇāĻŖ/N-SYMāĻŋāĻ/PRP ?/SYM)What/PRP are its/PRP-OBJ symptoms/N-SYM ?/SYM ā§āĻ¸ā§āĻā§°ā§āĻžāĻ āĻ˛āĻā§āĻˇāĻŖ āĻŋāĻ?(What are the symptoms of stroke?) Table 8: Simulation of Question Answering System Interaction SimulationEnglish Translation User:āĻĄāĻžā§ā§ā§āĻŦāĻŋāĻāĻ¸ āĻŋāĻ?User:What is diabetes? BanglaMed-QA:āĻĄāĻžā§ā§ā§āĻŦāĻŋāĻāĻ¸ āĻāĻāĻŋāĻ āĻā§°ā§āĻŋāĻ¨āĻ ā§āĻŽ- āĻŋāĻĄāĻāĻ¯ā§āĻžāĻ˛ āĻāĻŋāĻ¨ā§āĻĄāĻļāĻ¨ āĻŦāĻž ā§āĻ°āĻžāĻ, ā§āĻ¯āĻāĻžā§āĻ¨ āĻ°ā§āĻā§āĻ¤āĻ° āĻā§āĻ˛ ā§ ā§āĻāĻžāĻ āĻ¸ā§āĻ¤āĻ° āĻ āĻ¸āĻŦā§āĻžāĻāĻžāĻŋāĻŦāĻ āĻāĻžā§āĻŦ āĻŦāĻžā§ā§āĻ¤ āĻĨāĻžā§āĻāĨ¤ BanglaMed-QA:Diabetes is a chronic med- ical condition or disease in which blood glu- cose levels increase abnormally. User:ā§āĻĄāĻā§āĻ ā§ ā§āĻāĻ¨ āĻšā§User:Why does dengue occur BanglaMed-QA:ā§āĻĄāĻā§āĻ ā§ āĻāĻŋāĻĄāĻ¸ āĻŽāĻļāĻžāĻ° āĻāĻžāĻŽā§ā§ āĻŽāĻž- āĻ¨ ā§ āĻˇ ā§āĻĄāĻā§āĻ ā§ ā§āĻ°āĻžā§āĻ āĻāĻā§°ā§āĻžāĻ¨ā§āĻ¤ āĻšā§ā§ āĻĨāĻžā§āĻāĨ¤ BanglaMed-QA:People are infected with dengue fever by the bite of the Aedes mosquito. User:āĻĒā§°ā§āĻŋāĻ¤ā§āĻ°āĻžāĻ§??User:prevention?? BanglaMed-QA:ā§āĻĄāĻā§āĻ ā§ āĻāĻ° āĻĒā§°ā§āĻŋāĻ¤ā§āĻ°āĻžāĻ§ āĻ āĻā§āĻ° ā§āĻ¯āĻ¨ āĻŽāĻļāĻž āĻĸ ā§ āĻā§āĻ¤ āĻ¨āĻž āĻĒāĻžā§āĻ° ā§āĻ¸ āĻŦāĻ¯ā§āĻŦāĻ¸ā§āĻĨāĻž āĻāĻ°āĻžā§ˇ āĻā§āĻ° āĻāĻļāĻĒāĻžā§āĻļ āĻŦāĻĻā§āĻ§ āĻĒāĻžāĻŋāĻ¨ āĻāĻŽāĻž āĻ āĻĨāĻŦāĻž āĻŽā§āĻ˛āĻž ā§āĻā§āĻ āĻĻā§āĻ° āĻāĻ°ā§āĻ¨āĨ¤ āĻāĻŽāĻ¨āĻŋāĻ āĻā§āĻ° āĻŽāĻļāĻžāĻŋāĻ° āĻŦāĻ¯ā§āĻŦāĻšāĻžāĻ° āĻāĻ°ā§āĻ¨āĨ¤ BanglaMed-QA:Prevention of dengue: Make sure that mosquitoes cannot enter the house. Remove any stagnant water or dirt around the house by sweeping it away. Even use mosquito nets in the house. almost identical. But when we combine them, the performance is enhanced. The coverage in hard voting similarity decreases (6 were treated as NO_PREDICTION), though base similarity methods show 100% convergence. This is because of the use of an even number of individual base methods. Suppose, if two methods predictedS 1 and other two predictedS 2 as as the final query, then hard vot- 6. CONCLUSIONS 13 ing similarity canât be able to make a final decision and thus predicts that as NO_PREDICTION. Fig.5shows the human evaluation result, ranging from 0.78 to 1.0. The average rating of 0.9 (out of 1.0) shows the systemâs overall con- sistently better, with most individuals rated highly. For this human evaluation, we have used soft voting similarity as it outperformed other methods (Table5). 5.4 Qualitative analysis Table6shows that all classifier algorithms detect the category of questions correctly. Table7shows how the disambiguity of pronoun referencing is solved in our system. Here, the query with token having the POS tag P-OBJ is replaced by the immediate antecedent noun. As seen from Table8, our âBanglaMed-QAâ Chatbot is taking queries about different diseases and providing brief information about their treatment, symptoms, prevention, and cures etc in conversational format that is very easy to understand. 6 Conclusions The design of a question-answering system for medical texts in Bangla was car- ried out considering the challenges present in a low-resource language environ- ment. We disambiguate pronoun references using POS tagging and rephrase questions with synonyms to help the system learn different configurations of interrogatory expressions. By mixing different methods of determining similar- ity and voting strategies, the system is intended to reduce the unreliability of providing answers. The categorization of questions is one technique that re- markably speeds up the entire process. We made provisions for evaluation, both automatic and manual, so that the quality of the answers can be thoroughly judged. This system opens up further developments and actual applications as a stepping stone for Bangla NLP in the medical field. In the future, we would like to integrate a disease prediction subsystem to make our system more ro- bust. Moreover, we would like to explore more diseases and deep learning-based methods for query categorization and matching. References 1.Abacha, A.B., Zweigenbaum, P.: Means: A medical question-answering system combining nlp techniques and semantic web technologies. Information processing & management51(5), 570â594 (2015) 2.ABD EL-AZIZ, A.A., EL-KHORIBI, R.A., KHALIFA, N.E.: Enhancing robust- ness in medical question answering systems with novel defense models against adversarial attacks. Journal of Theoretical and Applied Information Technology 103(10) (2025) 3.Ahmed, M.R., Aziz, A., Manik, M.M.H., Habib, M.A.: Blockchain-aided compara- tive study of heart disease detection using machine learning-based approaches with an expanded dataset. Computers in Biology and Medicine195, 110611 (2025) 14 R. Zannat et al. 4.Brants, T.: Tnt-a statistical part-of-speech tagger. arXiv preprint cs/0003055 (2000) 5.Bronda-Ecraela, F.R.: Integrating okapi bm25 and jaccard algorithms in thesis search engine. Journal of Innovative Technology Convergence6(1) (2024) 6.Feng, G., Du, Z., Wu, X.: A chinese question answering system in medical domain. Journal of Shanghai Jiaotong University (Science)23, 678â683 (2018) 7.Huang, X., Zhang, J., Xu, Z., Ou, L., Tong, J.: A knowledge graph based question answering method for medical domain. PeerJ Computer Science7, e667 (2021) 8.Imani, M., Beikmohammadi, A., Arabnia, H.R.: Comprehensive analysis of ran- dom forest and xgboost performance with smote, adasyn, and gnus under varying imbalance levels. Technologies13(3), 88 (2025) 9.Jacquemart, P., Zweigenbaum, P.: Towards a medical question-answering system: a feasibility study. In: The New Navigators: From Professionals to Patients, p. 463â468. IOS Press (2003) 10.Maharjan, J., Garikipati, A., Singh, N.P., Cyrus, L., Sharma, M., Ciobanu, M., Barnes, G., Thapa, R., Mao, Q., Das, R.: Openmedlm: prompt engineering can out-perform fine-tuning in medical question-answering with open-source large lan- guage models. Scientific Reports14(1), 14156 (2024) 11.Rahman, M.M., Amin, R., Liton, M.N.K., Hossain, N.: Disha: an implementation of machine learning based bangla healthcare chatbot. In: 2019 22nd International Conference on Computer and Information Technology (ICCIT). p. 1â6. IEEE (2019) 12.Rahman Joy, M.T., Shahriar Akash, M.N., Hasan, K.A.: An intelligent bangla conversational agent: Tuni. In: Proceedings of International Joint Conference on Advances in Computational Intelligence: IJCACI 2020. p. 417â430. Springer (2021) 13.Ratul, A.A.S., Zannat, R., Muntakim, A.: A structured bangla dataset of disease- symptom associations to improve diagnostic accuracy (2025) 14.SEN, A., DASH, D.S.R., PRADHAN, D.M.R., PARIDA, S.: Healthcare question answering system in bengaliâa proposed model. Journal of Theoretical and Applied Information Technology102(12) (2024) 15.Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S.R., Cole-Lewis, H., et al.: Toward expert-level medical question answering with large language models. Nature Medicine31(3), 943â950 (2025) 16.Yang, X., Meng, P., Jiang, Z., Zhou, L.: Deep siamese residual support vector ma- chine with applications to disease prediction. Computers in Biology and Medicine 196, 110693 (2025) 17.Zannat, R., Al Shafi, A., Muntakim, A.: Bridging the gap in bangla healthcare: Machine learning based disease prediction using a symptoms-disease dataset. In: 2025 International Conference on Electrical, Computer and Communication Engi- neering (ECCE). p. 1â6. IEEE (2025) 18.Zhang, S., Luo, M., Wang, P., Li, B., Yang, H.: Cecld: classification error cor- rection based on levenshtein distance in dna data storage. Expert Systems with Applications p. 128714 (2025) A Interactive User Interface (UI) A practical system with an interactive UI to visualize the usefulness of our proposed system is shown in Fig.6. A. INTERACTIVE USER INTERFACE (UI) 15 Figure6: Interactive UI of our proposed BanglaMed-QA system.