Paper deep dive
Mitigating Entity Type Confusion in Cross-Domain NER via Multidimensional Quantification and Reasoning Enhancement
Jingyu Wang, Shijie Wu, Fusheng Jin
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Cross-domain Named Entity Recognition (CD-NER) aims to transfer the rich knowledge in the source domain to the target domain. Recent studies adopting decomposition or generation paradigms have achieved significant performance improvements, demonstrating high accuracy in entity span detection. However, during entity type classification, models severely suffer from entity type confusion, the erroneous tendency that models classify entities of one type in the text as another similar but incorrect type. To address this issue, we first propose a Multidimensional Confusion Quantification Model (MCQM) that quantifies a model's confusion extent between entity types from three dimensions: source-target hierarchy analysis, semantic similarity analysis, and explicit data evaluation. Moreover, we propose the Progressive Bidirectional Reasoning Chain (PBRC). PBRC leverages the source-target hierarchy and confusion analysis from the MCQM to prompt the LLM to generate two-stage reasoning information. The two-stage reasoning information is utilized to augment the knowledge of the model, significantly mitigating entity type confusion and improving the model's generalization performance. Experimental results demonstrate that our method achieves new state-of-the-art results on all domains of the CrossNER dataset.
Tags
Links
- Source: https://arxiv.org/abs/2609.24357v1
- Canonical: https://arxiv.org/abs/2609.24357v1
Trouble viewing inline? Open PDF directly →
Full Text
48,631 characters extracted from source content.
Expand or collapse full text
Mitigating Entity Type Confusion in Cross-Domain NER via Multidimensional Quantification and Reasoning Enhancement Jingyu Wang ∗ , Shijie Wu , Fusheng Jin ∗ Beijing Institute of Technology jyw.bit.2003@gmail.com, wsj1595@bit.edu.cn, jfs21cn@bit.edu.cn Abstract Cross-domain Named Entity Recognition (CD- NER) aims to transfer the rich knowledge in the source domain to the target domain. Recent studies adopting decomposition or generation paradigms have achieved significant performance improve- ments, demonstrating high accuracy in entity span detection. However, during entity type classifica- tion, models severely suffer from entity type con- fusion, the erroneous tendency that models classify entities of one type in the text as another similar but incorrect type. To address this issue, we first propose a Multidimensional Confusion Quantifica- tion Model (MCQM) that quantifies a model’s con- fusion extent between entity types from three di- mensions: source-target hierarchy analysis, seman- tic similarity analysis, and explicit data evaluation. Moreover, we propose the Progressive Bidirec- tional Reasoning Chain (PBRC). PBRC leverages the source-target hierarchy and confusion analysis from the MCQM to prompt the LLM to generate two-stage reasoning information. The two-stage reasoning information is utilized to augment the knowledge of the model, significantly mitigating entity type confusion and improving the model’s generalization performance. Experimental results demonstrate that our method achieves new state- of-the-art results on all domains of the CrossNER dataset. 1 Introduction Named Entity Recognition (NER) is a core task in informa- tion extraction, which aims to identify specific entities in texts that belong to predefined types, such as person, location, and organization [ Li et al., 2022; Hu et al., 2024; Esmaail et al., 2024 ] . NER plays a critical role in various tasks, including information retrieval [ Long et al., 2024; Cong et al., 2023; Nguyen et al., 2024 ] , knowledge graphs [ Chen et al., 2024; Jin et al., 2022; Gao et al., 2025 ] , and recommendation [ Li et al., 2024; Li et al., 2025 ] . With large-scale annotated data, neural network-based methods have performed remarkably ∗ Corresponding authors. song album Text：Among the most prominent of these were Slayer' s Reign in Blood , Anthrax' s Among the Living and Megadeth' s Peace Sells ... but Who 's Buying? beginning a search for his replacement. Features of song Features of album Fine-grained features of songFine-grained features of album Coarse-grained features of both flan-t5-base: song:[Reign in Blood, Among the Living, ...] ✗ ours: album:[Reign in Blood, Among the Living, ...] √ Figure 1: song and album share similar contextual features and en- tity forms, which can lead to confusion. in traditional NER [ Zhu and Li, 2022; Shen et al., 2023; Liu et al., 2022 ] . However, in cross-domain NER, several challenges persist, especially in low-resource scenarios where model performance experiences a significant decline. Recent studies [ Xu and Cai, 2023; Zhang et al., 2024a; Xu et al., 2024 ] decompose cross-domain NER into subtasks to capture transferable patterns across domains. These meth- ods focus on identifying shared feature patterns between do- mains through specialized components to strengthen source- domain knowledge transfer. Other approaches [ Chen et al., 2023; Zhang et al., 2024b; Nandi and Agrawal, 2024 ] em- ploy generative pre-trained language models, transforming NER into a generation task through the use of task-specific input prompts. These methods can effectively utilize large- scale knowledge from pretraining and exhibit high flexibility. Although both paradigms have demonstrated effectiveness in improving cross-domain NER performance, they remain sus- ceptible to entity type confusion, leading to significant per- formance degradation in entity type classification. We observe that cross-domain NER currently faces two major challenges.(1) Granularity differences between coarse-grained entity types in the source domain and fine- grained entity types in the target domain. Entity types in the source domain are coarse-grained, such as PER and LOC, whereas entity types in the target domain are more specific and domain-specialized, such as scientist and country. Com- pared to coarse-grained types, distinguishing fine-grained types often requires more detailed and nuanced features. Re- arXiv:2609.24357v1 [cs.CL] 21 Sep 2026 lying solely on the coarse-grained features learned from the source domain makes it difficult for the model to accurately classify fine-grained types in the target domain, such as song and album (as shown in Figure 1). (2) The lack of contextual information in the text further limits the model’s ability to classify fine-grained entity types accurately. In the target do- main, some samples lack the necessary contextual cues, mak- ing it difficult for the model to effectively learn and capture the required fine-grained features for type classification. As shown in Figure 1, we selected a sample from CrossNER [ Liu et al., 2021 ] , where the song and the album are two entity types that are difficult to distinguish. Entities of both types share similar forms and contextual environments. Due to insufficient information in the text, the model misclassifies these entities as the song. The lack of adequate contextual information prevents the model from effectively learning the feature differences between similar fine-grained entity types, which reduces the precision of type classification. To address the challenges in cross-domain NER, we pro- pose the Multidimensional Confusion Quantification Model (MCQM) and the Progressive Bidirectional Reasoning Chain (PBRC). (1) We propose the MCQM that quantifies the model’s confusion extent from three dimensions: source- target hierarchy analysis, semantic similarity analysis, and explicit data evaluation. MCQM enables in-depth analysis and quantification of the model’s confusion extent between fine-grained entity types in the target domain, providing a foundation for addressing the entity type confusion. (2) Fur- thermore, we innovatively leverage the source-target hierar- chy, along with the confusion analysis from the MCQM, to propose the PBRC, which employs a two-stage reasoning strategy. Specifically, PBRC first instructs the LLM to per- form initial reasoning based on contextual information using coarse-grained entity types from the source domain. Then, it instructs the LLM to perform bidirectional reasoning using fine-grained entity types from the target domain and confu- sion analysis from the MCQM. Finally, the LLM generates two-stage reasoning information for knowledge augmenta- tion. The two-stage reasoning strengthens the fine-grained features and alleviates the granularity gap between the source and target domains, while external knowledge provided by the LLM solves the issue of insufficient contextual informa- tion. Our method significantly mitigates entity type confu- sion. Experimental results demonstrate that our method ef- fectively improves the model’s generalization ability, with the average F1 score increasing by more than 10.00%. In sum- mary, our contributions are as follows: • We systematically analyze the entity type confusion in cross-domain NER. Then, we propose the MCQM to quantify the model’s confusion extent effectively. • We propose the PBRC based on the source-target hier- archy and confusion analysis from the MCQM, which can leverage the LLM to mitigate entity type confusion in cross-domain NER significantly. • Extensive experiments show that our method signif- icantly improves the model’s generalization ability, achieving new state-of-the-art performance across all domains of the CrossNER dataset. 2 Related Work Named Entity Recognition (NER). Traditional methods rely on handcrafted features or rules combined with machine learning models like CRF [ Lafferty et al., 2001 ] for entity recognition, which are labor-intensive and lack adaptability to complex contexts. With the development of deep learn- ing, neural network-based methods, such as LSTM [ Huang et al., 2015 ] and transformer [ Vaswani et al., 2017 ] mod- els, have become mainstream. Models [ Raffel et al., 2020; Radford et al., 2019; Devlin et al., 2019 ] based on these architectures capture sequential relationships and contextual dependencies, offering improved performance and flexibil- ity. However, existing methods still face challenges in cross- domain NER due to the variations across different domains. Cross-Domain NER. Current studies can be broadly categorized into two paradigms: decomposition-based and generation-based. Decomposition-based methods [ Xu and Cai, 2023; Xu et al., 2024; Zhang et al., 2024a; Zhang et al., 2022 ] reformulate cross-domain NER into multiple sub- tasks. These methods improve knowledge transfer by learn- ing shared cross-domain patterns through specialized subtask modules. Generation-based methods [ Bao and Yang, 2024; Chen et al., 2023; Zhang et al., 2024b; Nandi and Agrawal, 2024 ] transform the NER into a generation task. These meth- ods employ prompts containing task descriptions and auxil- iary information, achieving robust performance through in- struction fine-tuning. Although both paradigms effectively improve cross-domain performance, particularly in entity span detection, they still suffer from significant accuracy degradation in entity type classification. LLMs for NER. With the emergence of LLMs, their rea- soning capabilities have been leveraged to improve NER per- formance. Recent studies [ Ashok and Lipton, 2023; Wang et al., 2025; Zhu et al., 2024; Ji, 2023 ] employ prompt learn- ing for NER, where task-specific templates are augmented with demonstration examples and knowledge. LLMs are also used for cross-domain NER. Nandi and Agrawal [ Nandi and Agrawal, 2024 ] perform instruction fine-tuning of LLMs by retrieving and leveraging similar examples. Zhang et al. [ Zhang et al., 2024b ] utilize the LLM to generate task- oriented knowledge, used to conduct additional task-oriented pre-training of the backbone model for domain adaptation. These methods demonstrate strong few-shot performance. Compared to fine-tuning LLMs [ Nandi and Agrawal, 2024 ] and additional pre-training corpora [ Zhang et al., 2024b ] , our method reduces training time for domain adap- tation. 3 Methodology 3.1 Task Description Given a sentence X = x 1 ,x 2 ,...,x N , where w i repre- sents the i-th token in the sentence and N is the sentence length, the goal of NER is to identify all entities E within the sentence and classify each entity into a specific type. The set of entity types is denoted as L. Each entity e i ∈ E can be represented as e i = (y,x l:r ), where y ∈ L indicates the entity type, and l and r represent the start and end bound- ary indexes of the entity within the sentence, respectively. In 在此处键入公式。 ×β MCQM (b)semantic similarity analysis (a)source-target hierarchy analysis softmax Flan-T5-src Target Dev Set Text: They mostly engage in confronting the police during demonstrations and riots in countries like Canada Mexico or Greece(country). Output: LOC: [Canada,Mexico,Greece] Flan-T5-src finetune Flan-T5-tgt Target Train Set Target Dev Set predict misc person location misc person location Predict Ground Truth Misclassification Matrix Explicit Confusion Implicit Confusion ×α Hierarchy Matrix Similarity Matrix country Word Net A nation with its own gove- rnment. target domain type LLM A nation or territory defined by political sovereignty, cultural identity, geograph- ical boundaries ...... Cosine Similarity + Confusion Analysis Matrix LLM Flan-T5 Source Dataset Reasoning Information Canada, Mexico, and Greece： Ini reasoning: These entities are introduced with "in," a preposition commonly used for locations, ..., as demonstrations and riots often occur in specific places. So, they are LOC. Bid reasoning: The sentence refers to "demonstrations and riots," which are typically linked to national policies. Additionally, the text explicitly mentions "countries like," indicating ... countries. These terms are not "broad locations" because they lack the specificity of identifiable places like cities or landmarks.The sentence also lacks spatial markers (e.g., "near" or "within") ... Train Flan-T5-src country: [Canada,Mexico, Greece] Target Train Flan-T5-PBRC Input Text (c)data evalutaion PER LOC ORG LOC→country weight finetune predict (e)Two-stag reasoning information PBRC Step5: A→B indicates that the most easily confused type for A is B. Input: They mostly ... in countries like Canada ,Mexico or Greece. politician→person person→politician (d)Template Samples: Some manually designed reasoning samples to prompt the LLM Flan-T5-src Target Dataset Figure 2: Overview of our proposed method, including MCQM and PBRC. the cross-domain NER, two distinct datasets are considered: the source domain datasetD src and the target domain dataset D tgt . The set of entity types in the source domain is denoted as L s = s 1 ,s 2 ,...,s n , and the set of entity types in the target domain is denoted as L t = t 1 ,t 2 ,...,t m . The goal is to leverage the knowledge learned from D src to improve recognition performance on D tgt . Specifically, we focus on low-resource scenarios, where the training data in the target domain is significantly smaller than that in the source domain, i.e.,|D tgt |≪|D src |. 3.2 Multidimensional Confusion Quantification Model Entities of the same type share certain common features, re- ferred to as entity type features, and entity types with similar features are more prone to confusion. We quantify entity type confusion from three dimensions in Figure 2. Source-Target Hierarchy Analysis Entity types in the source domain are conceptually coarse- grained, whereas those in the target domain are fine-grained. In the feature space, the fine-grained entity types in the tar- get domain can be viewed as extensions of the entity types in the source domain. For example, PER in the source do- main can be expanded into fine-grained entity types in the target domain, such as politician and scientist. The expanded entity types retain the coarse-grained features in the source domain. Therefore, fine-grained entity types derived from the same entity type in the source domain exhibit similar features, meaning closer in the feature space and more prone to confu- sion among these entity types. To quantify the model’s confusion arising from the source- target hierarchy between the source and target domains, in- spired by [ Bao and Yang, 2024 ] , we propose a reliable quan- tification method. For each entity in the target domain, its fea- ture vectorH e can be considered as a combination of coarse- grained features H coarse and fine-grained features H fine . We only focus on extractingH coarse . We utilize the model M src , trained on D src , to extractH coarse of entities in the target domain and perform predictions based on the source domain type set L s . We calculate the proportion of entities corresponding to each fine-grained type t i in the target do- mainD tgt that are classified as the coarse-grained type s j in the source domainD src : R H (t i → s j ) = P (l tgt = t i ∧ l pred = s j ) P (l tgt = t i ) (1) where P(l tgt = t i ) is the proportion of entities in D tgt that belong to type t i , P(l tgt = t i ∧ l pred = s j ) is the proportion of entities inD tgt that belong to type t i and are classified as s j , and the proportion of entities of type t i classified as s j is denoted as R H (t i → s j ). We use the source domain entity type l src with the highest proportion as the prefix for the target domain entity type l tgt , denoted as l src → l tgt . The target domain entity type l tgt is considered a fine-grained extension of the source domain entity type l src : prefix (t i ) = arg max s j ∈L s (R H (t i → s j ))(2) where prefix(t i ) is the prefix of t i , and L s is the set of entity types in the source domain. R H (t i → s j ) measures how much of the coarse-grained features of s j are contained in t i . The more entities corre- sponding to t i are classified as s j , the more coarse-grained features of s j are contained in t i . Therefore, for two fine- grained entity types, t a and t b , if they share the same pre- fix, it implies that t a and t b exhibit a significant overlap in their coarse-grained features, which originate from the same source domain entity type. This overlap increases the likeli- hood of confusion between t a and t b . We quantify the confu- sion arising from the source-target hierarchy using R H (t i → s j ), based on the prefixes of fine-grained entity types: u (t i ,t j ) hie = δ hie · R H (t i → prefix (t j ))(3) where u (t i ,t j ) hie is the extent of hierarchical confusion from the fine-grained entity type t i to t j , and δ hie is the scaling factor. This method quantifies the model’s confusion arising from the source-target hierarchy, serving as an important metric in the MCQM for quantifying entity type confusion. Semantic Similarity Analysis The semantics of entity types reflect the common features of their entities. When the model maps the embedding vectors of the semantics of two entity types to closer positions in the se- mantic space, it considers the entity features of the two types as more similar, which increases the probability of confusing the two types in classification. Since entity type labels are short words, they cannot suffi- ciently reflect the features of the entity types. To enrich the semantics of each target domain entity type t i ∈ L t , we uti- lize the WordNet [ Fellbaum, 2010 ] to obtain a brief descrip- tion C s of t i . Then, C s is further refined into a final descrip- tion C e of t i by leveraging the LLM. Then, C e is fed into the encoder of the model, from which the hidden layer feature sequenceH = [h 1 , h 2 ,..., h Z ]∈R Z×d can be extracted: H = Encoder (C e )(4) where h i denotes the feature vector from the final hidden layer for the i-th token, Z is the number of tokens, and d rep- resents the dimension of the final hidden layer of the encoder. This study [ Reimers and Gurevych, 2019 ] demonstrates that Mean Pooling outperforms both [CLS] token and Max Pool- ing strategies in semantic textual similarity tasks. Therefore, we use Mean Pooling to calculate the global feature vector H, which better integrates the semantic information of all to- kens and provides a more comprehensive representation of the entity type features: H = P Z i=1 m i · h i P Z i=1 m i (5) where m i is the mask value of the i-th token, used to ignore the influence of padding. In high-dimensional space, the features of entity types are amplified, and the mapping rules for similar features are also similar. Therefore, we calculate the semantic similarity be- tween entity types usingH. We quantify the confusion aris- ing from the semantic similarity by computing the cosine sim- ilarity between the global feature vectors H of entity types: u (t i ,t j ) sim = H i ·H j H i · H j (6) where u (t i ,t j ) sim is the extent of semantic confusion from the fine-grained entity type t i to t j . This method effectively quantifies the model’s confusion arising from the semantic similarity between entity types, serving as an important met- ric in the MCQM for quantifying entity type confusion. Fusion of Implicit Confusion Factors The quantification of confusion arising from the source-target hierarchy and the semantic similarity of entity types is con- sidered implicit confusion analysis. We use two fixed mixing ratios, α and β, to balance the weights of these two factors: u (t i ,t j ) iml = α· u (t i ,t j ) hie + β· u (t i ,t j ) sim (7) Then, we use the softmax function to model the im- plicit confusion and define the implicit confusion distribution among fine-grained entity types in the target domain: v iml (t i → t j ) = exp(w 0 · u (t i ,t j ) iml ) P t k ∈L t t k ̸=t i exp(w 0 · u (t i ,t k ) iml ) (8) where w 0 ∈R + is a temperature coefficient. Explicit Data Evaluation We denote the model obtained by fine-tuning M src on the target domain dataset as M tgt . We use the dev set of the target domain to perform a statistical analysis of the misclas- sification proportions for all entity types. Evaluating M tgt reflects the model’s ability to distinguish between different fine-grained entity types and explicitly reveals the model’s confusion extent between entity types: R (t i ,t j ) M = P(l true = t i ∧ l pred = t j ) P(l true = t i ) (9) where P(l true = t i ) is the proportion of entities that truly belong to type t i , P(l true = t i ∧l pred = t j ) is the proportion of entities that truly belong to type t i but are misclassified as t j , and the proportion of entities of type t i misclassified as t j is denoted as R (t i ,t j ) M . The value of R (t i ,t j ) M explicitly reflects the model’s learn- ing effectiveness in distinguishing the fine-grained features of t i and t j . A larger value indicates that t i and t j are more prone to confusion. We use R (t i ,t j ) M as an important metric for confusion analysis in the MCQM. Then, we use the softmax function to model explicit confusion and define the explicit confusion distribution among fine-grained entity types in the target domain: v exl (t i → t j ) = exp(w 0 · R (t i ,t j ) M ) P t k ∈L t t k ̸=t i exp(w 0 · R (t i ,t k ) M ) (10) where w 0 ∈R + is a temperature coefficient. Confusion Analysis Results We integrate the implicit and explicit evaluations to obtain the final confusion analysis results for fine-grained entity types in the target domain: v conf (t i → t j ) = v iml (t i → t j ) + v exl (t i → t j ) (11) Step1：Identify all potential named entities from the text. Step2：For each named entity, analyze the contextual information and classify it using the following coarse-grained types. 1.PER：Refers to individuals, including their full names, nicknames, titles, or any personal identifiers. 2.LOC：Refers to geographical locations such as cities, countries, landmarks, or any specific places. Step3：For each named entity，based on external knowledge and contextual information of the entity, explains why the entity is classified as the corresponding coarse-grained type. Step4：For each named entity, based on its coarse-grained type， further classify it using following fine-grained entity types. 1.PER→scientist：Refers to an individual engaged in scientific research or experimentation, often working in fields to advance knowledge..... 2.LOC→country：Refers to a geographic and political entity with sovereignty, defined borders, and a governing body. Step5：Below is the table confusion mapping table, A→B indicates that the most easily confused type for A is B. For each named entity, based on external knowledge and contextual information of the entity, explain why the entity is classified as type A rather than type B. 1.person→scientist 2.chemicalelement→chemicalpound 3.enzyme→protein Last，You only need to output the analysis for Step3 and Step5, without including the final classification results. Figure 3: Contents of PBRC. where v conf (t i → t j ) is the model’s confusion extent from t i to t j . For each type t i , we select the entity type with the highest confusion extent as its most easily confused type: T (t i ) conf = arg max t j ∈L t ,t j ̸=t i (v conf (t i → t j ))(12) where T (t i ) conf is the most easily confused entity type for the en- tity type t i . MCQM provides a comprehensive analysis of the model’s confusion extent between fine-grained entity types, playing a critical role in the bidirectional reasoning of PBRC. 3.3 Progressive Bidirectional Reasoning Chain Contents of PBRC We propose PBRC based on the source-target hierarchy and the confusion analysis from MCQM. As shown in Figure 2, PBRC prompts the LLM to output two-stage reasoning infor- mation. Figure 3 shows the contents of PBRC. Step1: Identifying Named Entities. This step instructs the LLM to identify all potential named entities in the text. With pre-trained multi-domain knowledge and contextual un- derstanding, the LLM is capable of recognizing potential en- tities in texts across various domains. Step2 and Step3: Coarse-grained Classification and Initial Reasoning. We provide all source domain coarse- grained entity types and explanations. For each entity in Step1, we instruct the LLM to conduct coarse-grained clas- sification and provide explanations, leveraging both its prior knowledge and an initial analysis of the context. This step unifies entity types across different domains, serving as an initial classification and reasoning process for entities. Step4 and Step5: Fine-grained Classification and Bidi- rectional Reasoning.We provide all fine-grained entity DomainDatasetTypeTrainDevTest Source CoNLL2003414987-- Twitter44290-- Target politics9200541651 science17200450543 music13100380465 literature12100400416 AI14100350431 Table 1: The statistics of all datasets. types of the target domain along with their corresponding source domain prefixes and explanations. For each entity, we instruct the LLM to conduct fine-grained classification based on the initial classification and reasoning information from Step2 and Step3, and then conduct bidirectional reason- ing through a deep analysis of contextual information and the LLM’s rich external knowledge. As shown in Figure 3, we present the confusion analysis result of the MCQM, where A → B indicates that A’s most easily confused type is B. Bidirectional reasoning consists of two parts: (1) forward reasoning, which infers that the entity type is A; (2) back- ward reasoning, which infers that the entity type is not B. Two-stage Reasoning Information. The LLM only out- puts the initial reasoning from Step3 and the bidirectional rea- soning from Step5. The two-stage reasoning information is fed into the backbone for knowledge augmentation: the ini- tial reasoning guides the backbone to leverage the knowledge from the source domain, strengthening knowledge transfer; the bidirectional reasoning guides the backbone to differen- tiate easily confused entity types, mitigating the entity type confusion. Besides, external knowledge mitigates the insuffi- ciency of contextual information. Knowledge Augmentation The two-stage reasoning information generated by the LLM is utilized to augment the knowledge ofM src . Through su- pervised fine-tuning, the backbone learns to leverage this rea- soning information, yielding the final backbone modelM † tgt . • Input: Find all entities of types politician, person, country ... intext. Reasoning information: .... Output format: ”type 1”: [”entity 1”, ”entity 2”], ”type 2”: [”entity 3”], .... • Gold Sequence: ”country”: [”Afghanistan”], ”politi- cian”: [”Barack Obama”], .... 4 Experiments 4.1 Experimental Setup As shown in Figure 1, we evaluate on two source domains (CoNLL2003 [ Sang and De Meulder, 2003 ] and Twitter [ Lu et al., 2018 ] ) and five target domains (CrossNER [ Liu et al., 2021 ] : politics, science, music, literature, and AI). We em- ploy Flan-T5 [ Chung et al., 2024 ] as the backbone model and GPT-4o-mini as the LLM. To evaluate the performance of our proposed method, we compare it with the following base- lines: (1) MTD [ Zhang et al., 2022 ] : A modular learning- based method that decomposes NER into span detection and CoNLL2003Twitter Method sci.pol.mus.lit.AIAvg.sci.pol.mus.lit.AIAvg. MTD72.3576.7076.1069.2268.9372.6671.3774.6274.4169.6764.5570.92 CP-NER75.8274.2579.1072.1767.9573.86------ DH-GAT74.2177.0678.7772.5169.3074.3774.5576.4677.3371.5267.6573.50 PromptNER72.5978.6184.2674.4464.8374.95------ Dual-CL74.1777.5678.5772.4369.4974.4474.0777.5377.2171.7268.7173.85 DT-MPrompt73.0680.5479.5473.5170.1375.3673.1879.8677.9372.7469.1374.57 IF-WRANER-7B75.3179.8085.4375.5268.8176.97------ TOPT80.1681.5582.0377.8572.3478.78------ Flan-T5-base + MCQM & PBRC81.4383.5383.6979.0374.2980.3980.3282.4381.9377.6273.9779.25 Flan-T5-large + MCQM & PBRC82.3485.0385.6279.9675.9981.7981.6284.6484.0278.6175.5080.88 Table 2: F1 scores on CrossNER: CoNLL2003 and Twitter as source domains, respectively. Bold marks the highest. Flan-T5-base (w/o MCQM & PBRC)Flan-T5-base (+ MCQM & PBRC) DomainPrecisionRecallF1 scorePrecisionRecallF1 score science71.31 (83.11)68.66 (81.10)69.96 (82.09)83.05 (88.12)79.88 (84.49)81.43 (86.27) politics71.30 (87.45)69.29 (86.99)70.28 (87.22)84.62 (91.15)82.47 (88.01)83.53 (89.55) music77.07 (88.46)73.65 (82.99)75.32 (85.64)85.34 (89.45)82.10 (87.85)83.69 (88.64) literature68.28 (83.67)67.47 (83.98)67.87 (83.82)80.48 (87.67)77.64 (84.12)79.03 (85.86) AI63.42 (81.15)63.70 (81.64)63.56 (81.39)75.74 (86.07)72.90 (83.01)74.29 (84.51) Table 3: Entity Type Confusion Analysis. CoNLL2003 as the source domain. type classification. (2) CP-NER [ Chen et al., 2023 ] : Trans- forms NER as text-to-text task and introduces collaborative domain-prefix tuning based T5 as well. (3) DH-GAT [ Xu and Cai, 2023 ] : Applies Graph Attention Networks to en- code syntactic and semantic information while embedding words into hyperbolic space. (4) PromptNER [ Ashok and Lipton, 2023 ] : Uses GPT4 for NER through prompt tem- plates. (5) Dual-CL [ Xu et al., 2024 ] : Uses dual contrastive learning to refine ambiguous representations and learn gen- eralizable features. (6) DT-MPrompt [ Zhang et al., 2024a ] : Splits the cross-domain NER task into subtasks and uses sep- arate functional modules for learning and knowledge trans- fer. (7) IF-WRANER [ Nandi and Agrawal, 2024 ] : Fine- tunes 7B LLaMA with instruction fine-tuning and employs word embeddings to retrieve examples for in-context learn- ing. (8) TOPT (SOTA) [ Zhang et al., 2024b ] : Utilizes LLaMA to generate task-oriented knowledge for flan-T5 and adopts task-oriented pre-training for domain adaptation. 4.2 Main Results The main results are shown in Table 2. CoNLL2003 as the Source Domain. It is observed that the F1 score improves as the parameter scale of the model in- creases. On average, our method achieves consistent F1 score improvements of +1.61% (Flan-T5-base) and +3.01% (Flan- T5-large). Our method surpasses prior SOTA results in sci- ence, politics, literature, and AI domains (effective for both Flan-T5-base and Flan-T5-large), achieving improvements exceeding 2.10% in these domains. These results demon- strate the strong effectiveness of two-stage information gen- erated by the LLM in cross-domain NER. We observe that IF-WARNER-7B (85.43%), which employs a fine-tuned 7B LLaMA, and PromptNER (84.26%), which utilizes GPT-4, both achieve significantly higher F1 scores in the music do- main compared to other baselines using smaller models. This demonstrates the advantage of LLMs in the music domain, ensuring the quality of the information generated by our two- stage process using the LLM. We observe that Flan-T5-large achieves only a marginal improvement of 0.19% over prior SOTA in the music domain, while Flan-T5-base fails to sur- pass the prior SOTA. Through careful analysis, we have iden- tified the main reasons. First, the high overlap between the domain knowledge of the LLM and Flan-T5 in the music domain diminished their complementary effects, resulting in limited benefits from two-stage reasoning. Then, though the two-stage information significantly improves F1 scores for certain entity types, their limited entity count results in a neg- ligible impact on the overall F1 metric. Twitter as the Source Domain. Our method consistently achieves significant improvements across all domains. Since TOPT is not tested with Twitter as the source domain, we cannot compare with it directly. Compared to CoNLL2003, the performance of both Flan-T5-base and Flan-T5-large shows a slight decline. We attribute this to the smaller dataset size of Twitter, which limits the model’s ability to learn coarse-grained features of entities. This highlights the im- portance of learning the coarse-grained features of entities and further demonstrates that our method enables effective knowledge transfer from the source to the target domain. 4.3 Entity Type Confusion Analysis To verify that the model’s performance improvement is at- tributed to enhanced entity type classification capabilities, we evaluate its performance in both entity span detection and overall performance using precision, recall, and F1 score. The results are shown in Table 3, where the values in paren- theses indicate the performance on entity span detection. Be- fore incorporating MCQM and PBRC, we observe a substan- Settingsci.pol.mus.lit.AI w/o78.0679.4781.4476.1271.42 + S79.0182.6282.5477.3473.05 + STH79.9882.6782.0977.8673.59 + S&STH80.9383.0583.1678.4173.66 + DE79.4782.8982.4177.4972.18 + ALL81.4383.5383.6979.0374.29 Table 4: Ablation Study on MCQM (F1 scores). Settingsci.pol.mus.lit.AIAvg. w/o69.96 70.28 75.32 67.87 63.56 69.40 + FR76.58 78.26 80.09 74.77 70.03 75.95 + IR&FR78.06 79.47 81.44 76.12 71.42 77.30 + BR&FR 80.62 82.76 82.49 78.06 73.52 79.49 + ALL81.43 83.53 83.69 79.03 74.29 80.39 Table 5: Ablation Study on PBRC (F1 scores). tial performance gap between the entity span detection and overall performance, which indicates that the model’s ability of entity type classification is the primary bottleneck, suffer- ing from significant entity type confusion. After incorporat- ing MCQM and PBRC, we observe that our method improves the F1 score for entity span detection by 2%–4% across the five domains, with slight improvements also observed in pre- cision and recall. More notably, the overall performance in- creases by over 10% on average. These results demonstrate that our method significantly enhances the model’s ability in entity type classification and effectively mitigates the model’s entity type confusion. 4.4 Ablation Study Ablation Study on MCQM Table 4 presents the effects of MCQM components: semantic similarity (S), source–target hierarchy (STH), and data eval- uation (DE), on model performance (F1 score). By compar- ing the results of completely removing all components (w/o) with those of introducing only S, STH, or DE, we observe that, across all target domains, incorporating S, STH, or DE into confusion analysis and subsequently introducing back- ward reasoning based on the corresponding analysis improves model performance. This indicates that semantic similarity analysis, source–target hierarchy analysis, and explicit data evaluation can all effectively quantify the entity type confu- sion extent. Moreover, simultaneously introducing S and STH (+S&STH) or incorporating all components (+ALL) into confusion analysis further improves model performance, suggesting that leveraging multiple analytical dimensions can reduce the errors associated with a single dimension and en- hance the quantification accuracy of MCQM. Ablation Study on PBRC We conduct ablation studies to evaluate the effectiveness of each reasoning component in PBRC: forward (FR), backward (BR), and initial reasoning (IR). As shown in Table 5, adding forward reasoning (FR) alone yields an average performance ModelTrainingInferenceDev F1 GPT-4o-mini-3.22s74.78 Flan-T5-base28.62 min0.32s70.25 GPT-4o-mini + base23.60 min3.56s80.69 Flan-T5-large46.28 min0.51s71.98 GPT-4o-mini + large37.35 min3.84s82.55 Table 6: The runtime statistics of different methods. gain of 6.55%. This demonstrates that the reasoning informa- tion provided by the LLM can effectively support the back- bone model in performing NER. Further incorporating ini- tial reasoning (IR) brings an additional average improvement of 1.35%, demonstrating that the two-stage reasoning from source to target domain can enhance the backbone’s cross- domain transferability. Moreover, when backward reason- ing (BR) is incorporated, the average performance rises by 3.54% compared to using FR alone. This indicates that the MCQM effectively analyzes and quantifies the model’s con- fusion among entity types, while backward reasoning can al- leviate entity type confusion. 4.5 Runtime Analysis We report the average training time of different methods across the five target domains, the average F1 score on the dev set, and the inference time per text (in seconds), as shown in Table 6. First, although our method requires the LLM to gen- erate reasoning information for the training set in advance, the overall training time is still lower than that of using Flan-T5 alone. We attribute this to the fact that, once reasoning infor- mation is introduced, Flan-T5 converges faster and reaches the best F1 score of the dev set with fewer epochs. Besides, for the inference time of each text, our method first leverages the LLM to generate reasoning information before feeding it into Flan-T5 for NER. Since Flan-T5 has a very short infer- ence time, the final inference time of our method is compara- ble to that of the LLM. Our method maintains training time comparable to that of using Flan-T5 alone, attains inference efficiency that is nearly equivalent to the LLM, and achieves performance that substantially exceeds both Flan-T5 and the LLM when employed alone. 5 Conclusion In this paper, we propose the MCQM, which effectively quan- tifies the model’s confusion extent among fine-grained entity types from three dimensions: source-target hierarchy analy- sis, semantic similarity analysis, and explicit data evaluation. Furthermore, we propose the PBRC based on the source- target hierarchy and the MCQM, which can significantly mit- igate entity type confusion and improve the model’s gener- alization ability. Our method achieves SOTA results on all domains of the CrossNER dataset. Acknowledgments This work is partially supported by the National Natural Sci- ence Foundation of China (No. 62272045). References [ Ashok and Lipton, 2023 ] Dhananjay Ashok and Zachary C Lipton. Promptner: Prompting for named entity recogni- tion. arXiv preprint arXiv:2305.15444, 2023. [ Bao and Yang, 2024 ] Ke Bao and Chonghuan Yang. Label alignment and reassignment with generalist large language model for enhanced cross-domain named entity recogni- tion. arXiv preprint arXiv:2407.17344, 2024. [ Chen et al., 2023 ] Xiang Chen, Lei Li, Shuofei Qiao, Ningyu Zhang, Chuanqi Tan, Yong Jiang, Fei Huang, and Huajun Chen. One model for all domains: collaborative domain-prefix tuning for cross-domain ner. In Proceed- ings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI ’23, 2023. [ Chen et al., 2024 ] Zhongwu Chen, Long Bai, Zixuan Li, Zhen Huang, Xiaolong Jin, and Yong Dou. A new pipeline for knowledge graph reasoning enhanced by large lan- guage models without fine-tuning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceed- ings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1366–1381, Miami, Florida, USA, November 2024. Association for Computa- tional Linguistics. [ Chung et al., 2024 ] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, and et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53, 2024. [ Cong et al., 2023 ] Xin Cong, Bowen Yu, Mengcheng Fang, Tingwen Liu, Haiyang Yu, Zhongkai Hu, Fei Huang, Yongbin Li, and Bin Wang. Universal information extrac- tion with meta-pretrained self-retrieval. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Find- ings of the Association for Computational Linguistics: ACL 2023, pages 4084–4100, Toronto, Canada, July 2023. [ Devlin et al., 2019 ] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In Proceedings of the 2019 conference of the North American chapter of the association for computational lin- guistics: human language technologies, volume 1 (long and short papers), pages 4171–4186, 2019. [ Esmaail et al., 2024 ] Naji Esmaail, Nazlia Omar, Masnizah Mohd, Fariza Fauzi, and Zainab Mansur. Named entity recognition in user-generated text: A systematic literature review. IEEE Access, 12:136330–136353, 2024. [ Fellbaum, 2010 ] Christiane Fellbaum. Wordnet. In The- ory and applications of ontology: computer applications, pages 231–243. Springer, 2010. [ Gao et al., 2025 ] Hepeng Gao, Funing Yang, Yongjian Yang, and Ying Wang. Hierarchy knowledge graph for parameter-efficient entity embedding. In Proceedings of the Thirty-Fourth International Joint Conference on Arti- ficial Intelligence, pages 2811–2819, 2025. [ Hu et al., 2024 ] Zhentao Hu, Wei Hou, and Xianxing Liu. Deep learning for named entity recognition: a survey. Neu- ral Comput. Appl., 36(16):8995–9022, March 2024. [ Huang et al., 2015 ] Zhiheng Huang, Wei Xu, and Kai Yu. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991, 2015. [ Ji, 2023 ] Bin Ji.Vicunaner:Zero/few-shot named entity recognition using vicuna.arXiv preprint arXiv:2305.03253, 2023. [ Jin et al., 2022 ] Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. A good neighbor, a found trea- sure: Mining treasured neighbors for knowledge graph en- tity typing. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 480–490, Abu Dhabi, United Arab Emirates, De- cember 2022. Association for Computational Linguistics. [ Lafferty et al., 2001 ] John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Con- ference on Machine Learning, ICML ’01, page 282–289, San Francisco, CA, USA, 2001. Morgan Kaufmann Pub- lishers Inc. [ Li et al., 2022 ] Jing Li, Aixin Sun, Jianglei Han, and Chen- liang Li. A survey on deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering, 34(1):50–70, 2022. [ Li et al., 2024 ] Xinhang Li, Xiangyu Zhao, Yejing Wang, Yu Liu, Chong Chen, Cheng Long, Yong Zhang, and Chunxiao Xing. Opensiterec: An open dataset for site rec- ommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’24, page 1483–1493, New York, NY, USA, 2024. [ Li et al., 2025 ] Haodong Li, Lianyong Qi, Weiming Liu, Xiaolong Xu, Wanchun Dou, Yang Cao, Xuyun Zhang, Amin Beheshti, and Xiaokang Zhou. Balancing user-item structure and interaction with large language models and optimal transport for multimedia recommendation. In Pro- ceedings of the Thirty-Fourth International Joint Confer- ence on Artificial Intelligence, pages 3027–3035, 2025. [ Liu et al., 2021 ] Zihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai, Ziwei Ji, Samuel Cahyawijaya, Andrea Madotto, and Pascale Fung. Crossner: Evaluating cross-domain named entity recognition. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 13452–13460, 2021. [ Liu et al., 2022 ] Jian Liu, Yufeng Chen, and Jinan Xu. Low- resource ner by data augmentation with prompting. In IJ- CAI, pages 4252–4258, 2022. [ Long et al., 2024 ] Zijun Long, Xuri Ge, Richard Mc- Creadie, and Joemon M. Jose. Cfir: Fast and effective long-text to image retrieval for large corpora. In Proceed- ings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SI- GIR ’24, page 2188–2198, New York, NY, USA, 2024. Association for Computing Machinery. [ Lu et al., 2018 ] Di Lu, Leonardo Neves, Vitor Carvalho, Ning Zhang, and Heng Ji. Visual attention model for name tagging in multimodal social media. In Proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: Long papers), pages 1990–1999, 2018. [ Nandi and Agrawal, 2024 ] Subhadip Nandi and Neeraj Agrawal. Improving few-shot cross-domain named entity recognition by instruction tuning a word-embedding based retrieval augmented large language model. In Proceedings of the 2024 Conference on Empirical Methods in Natu- ral Language Processing: Industry Track, pages 686–696, 2024. [ Nguyen et al., 2024 ] Thong Nguyen, Shubham Chatterjee, Sean MacAvaney, Iain Mackie, Jeff Dalton, and An- drew Yates. DyVo: Dynamic vocabularies for learned sparse retrieval with entities. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Lan- guage Processing, pages 767–783, Miami, Florida, USA, November 2024. Association for Computational Linguis- tics. [ Radford et al., 2019 ] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. [ Raffel et al., 2020 ] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(1), January 2020. [ Reimers and Gurevych, 2019 ] Nils Reimers and Iryna Gurevych.Sentence-bert: Sentence embeddings using siamese bert-networks.In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 3982–3992, 2019. [ Sang and De Meulder, 2003 ] Erik Tjong Kim Sang and Fien De Meulder. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In Proceedings of the seventh conference on Natural lan- guage learning at HLT-NAACL 2003, pages 142–147, 2003. [ Shen et al., 2023 ] Yongliang Shen, Zeqi Tan, Shuhui Wu, Wenqi Zhang, Rongsheng Zhang, Yadong Xi, Weiming Lu, and Yueting Zhuang. Promptner: Prompt locating and typing for named entity recognition. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers), pages 12492–12507, 2023. [ Vaswani et al., 2017 ] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. [ Wang et al., 2025 ] Shuhe Wang, Xiaofei Sun, Xiaoya Li, Rongbin Ouyang, Fei Wu, Tianwei Zhang, Jiwei Li, Guoyin Wang, and Chen Guo. Gpt-ner: Named entity recognition via large language models. In Findings of the association for computational linguistics: NAACL 2025, pages 4257–4275, 2025. [ Xu and Cai, 2023 ] Jingyun Xu and Yi Cai. Decoupled hy- perbolic graph attention network for cross-domain named entity recognition. In Proceedings of the 46th Interna- tional ACM SIGIR Conference on Research and Develop- ment in Information Retrieval, SIGIR ’23, New York, NY, USA, 2023. Association for Computing Machinery. [ Xu et al., 2024 ] Jingyun Xu, Junnan Yu, Yi Cai, and Tat- Seng Chua. Dual contrastive learning for cross-domain named entity recognition. ACM Trans. Inf. Syst., 42(6), October 2024. [ Zhang et al., 2022 ] Xinghua Zhang, Bowen Yu, Yubin Wang, Tingwen Liu, Taoyu Su, and Hongbo Xu. Exploring modular task decomposition in cross-domain named entity recognition. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval, SIGIR ’22, New York, NY, USA, 2022. [ Zhang et al., 2024a ] Xinghua Zhang, Bowen Yu, Xin Cong, Taoyu Su, Quangang Li, Tingwen Liu, and Hongbo Xu. Cross-domain ner under a divide-and-transfer paradigm. ACM Transactions on Information Systems, 42(5):1–32, 2024. [ Zhang et al., 2024b ] Zhihao Zhang, Sophia Yat Mei Lee, Junshuang Wu, Dong Zhang, Shoushan Li, Erik Cambria, and Guodong Zhou. Cross-domain ner with generated task-oriented knowledge: an empirical study from infor- mation density perspective. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1595–1609, 2024. [ Zhu and Li, 2022 ] Enwei Zhu and Jinpeng Li. Boundary smoothing for named entity recognition.In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7096–7108, Dublin, Ireland, May 2022. Association for Computational Linguistics. [ Zhu et al., 2024 ] Xingyu Zhu, Feifei Dai, Xiaoyan Gu, Bo Li, Meiou Zhang, and Weiping Wang.Gl-ner: Generation-aware large language models for few-shot named entity recognition. In Artificial Neural Networks and Machine Learning – ICANN 2024., 2024.