Paper deep dive

Traceable LLM Reasoning for Fake-Order Fraud Detection

Siqi You, Bingsong Xu, Zhixian Zheng, Xinjian Peng, Yang Xie, Ying Wang, Jiarong Xu

Year: 2026Venue: arXiv preprintArea: cs.CRType: PreprintEmbeddings: 69

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Detecting fake-order fraud at scale remains a critical challenge for large online-to-offline (O2O) service platforms, as existing approaches often rely on expert-designed features, produce black-box decisions, and provide limited interpretability. To address these limitations, we propose DeepScrub, a reinforcement learning framework built upon large language models (LLMs) for fake-order fraud detection with traceable reasoning. DeepScrub introduces three innovations. First, a semantic unification module converts heterogeneous risk signals into textual descriptions that LLMs can understand. Second, continued pre-training on risk-control corpora injects domain knowledge, and task rewards jointly evaluate prediction correctness and reasoning quality. Third, the SUggest-REflect (SURE) mechanism incorporates expert feedback and model self-checking to iteratively refine reasoning paths. On a real-world fake-order fraud detection dataset, DeepScrub achieves a macro-F1 score of 85.3%, outperforming the best baseline by 2.7 percentage points. Our task-optimized 8B model further surpasses a 32B model, showing that domain adaptation can matter more than model scale in this setting. In a four-week live pilot, DeepScrub achieved 91.8% precision and 88.5% recall, improving over first-stage human reviewers by 16.6 and 38.8 percentage points. It reduced first-stage manual review workload by 94% and saved nearly one million RMB annually. These results show that DeepScrub improves fraud review accuracy, reduces first-stage review workload, and provides traceable evidence for production risk-review workflows.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

67,815 characters extracted from source content.

Expand or collapse full text

Traceable LLM Reasoning for Fake-Order Fraud Detection Siqi You ByteDanceHangzhouChina yousiqi.147@bytedance.com , Bingsong Xu ByteDanceBeijingChina xubingsong@bytedance.com , Zhixian Zheng ByteDanceHangzhouChina zhengzhixian@bytedance.com , Xinjian Peng ByteDanceHangzhouChina pengxinjian@bytedance.com , Yang Xie ByteDanceHangzhouChina xieyang.bd@bytedance.com , Ying Wang ByteDanceHangzhouChina wangying.826@bytedance.com and Jiarong Xu Fudan UniversityShanghaiChina jiarongxu@fudan.edu.cn (2027) Abstract. Detecting fake-order fraud at scale remains a critical challenge for large online-to-offline (O2O) service platforms, as existing approaches often rely on expert-designed features, produce black-box decisions, and provide limited interpretability. To address these limitations, we propose DeepScrub 111The name DeepScrub reflects the goal of deep-cleaning fake-order transactions from e-commerce platforms., a reinforcement learning framework built upon large language models (LLMs) for fake-order fraud detection with traceable reasoning. DeepScrub introduces three innovations. First, a semantic unification module converts heterogeneous risk signals into textual descriptions that LLMs can understand. Second, continued pre-training on risk-control corpora injects domain knowledge, and task rewards jointly evaluate prediction correctness and reasoning quality. Third, the SUggest-REflect (SURE) mechanism incorporates expert feedback and model self-checking to iteratively refine reasoning paths. On a real-world fake-order fraud detection dataset, DeepScrub achieves a macro-F1 score of 85.3%, outperforming the best baseline by 2.7 percentage points. Our task-optimized 8B model further surpasses a 32B model, showing that domain adaptation can matter more than model scale in this setting. In a four-week live pilot, DeepScrub achieved 91.8% precision and 88.5% recall, improving over first-stage human reviewers by 16.6 and 38.8 percentage points. It reduced first-stage manual review workload by 94% and saved nearly one million RMB annually. These results show that DeepScrub improves fraud review accuracy, reduces first-stage review workload, and provides traceable evidence for production risk-review workflows. fraud detection, large language models, reinforcement learning, reasoning, risk‑control systems †copyright: acmlicensed†journalyear: 2027†conference: ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 2027; San Jose, California†ccs: Information systems Data mining†ccs: Computing methodologies Anomaly detection†ccs: Computing methodologies Reinforcement learning 1. Introduction Fake-order fraud, also known as brushing, refers to behaviors that fabricate or manipulate transaction records, clicks, or user reviews to create an illusion of inflated commercial scale, popularity, or user engagement. Such practices not only undermine the reliability of sales volumes and ratings, but also distort fair competition among merchants, posing substantial risks to the socio-economic system. According to statistics from the Ministry of Public Security of China in 2025, brushing-related fraud accounts for about 37% of all telecommunication fraud cases. Controlling fake-order fraud on large O2O platforms is particularly challenging because suspicious orders are interleaved with legitimate business bursts, promotional traffic, and diverse merchant operating strategies. With hundreds of millions of daily transactions and complex multi-party interactions, purely manual review is costly and difficult to scale. At the same time, purely model-driven decisions are often hard to justify in enforcement and appeal workflows, where platforms must explain why a merchant or transaction is judged to be suspicious. Existing fake-order fraud detection methods mainly fall into two categories. The first category relies on handcrafted features and expert rules derived from operational signals and historical behavior patterns. Although effective in some high-precision scenarios, these methods require heavy expert involvement, incur high maintenance costs, and adapt poorly to evolving fraud strategies. The second category uses deep learning models to improve detection accuracy, but their black-box predictions make it difficult to provide defensible evidence in real enforcement workflows. This limitation is especially serious in local-service O2O scenarios, where online orders are tightly coupled with offline fulfillment. In such settings, enforcement decisions are not merely classification outputs; they may directly affect merchant penalties, appeal resolution, and downstream operational actions. For example, a legitimate multi-branch merchant may exhibit frequent transactions, recurring users, or unusual geographic patterns that superficially resemble brushing behavior. Without business-grounded explanations, platforms risk penalizing legitimate merchants and weakening trust in the review process. Therefore, an effective fake-order fraud detection model should not only identify suspicious transactions, but also provide traceable evidence that supports review, enforcement, and appeals. This requirement makes large language models (LLMs) appealing for fake-order fraud detection. By reasoning over heterogeneous evidence in natural language, LLMs can potentially connect risk signals to human-readable explanations and produce auditable decision traces. However, directly applying LLMs to this setting is still insufficient for deployment. In practice, three challenges remain. First, key evidence for fake-order fraud is distributed across heterogeneous data sources, including association graphs, behavioral sequences, and structured transaction records, whereas LLMs primarily consume text. A deployable system therefore needs a practical way to unify heterogeneous evidence into a representation that supports both reasoning and auditing without introducing excessive system complexity. Second, fake-order fraud detection requires optimizing not only prediction correctness but also reasoning quality, because correct labels with weak or unsupported explanations are still inadequate for enforcement workflows. Third, high-stakes fraud review cannot rely solely on a single-pass generation process. When initial reasoning overlooks key merchant-side or transaction-side evidence, the system should be able to revisit and refine its judgment in a structured manner. To address these challenges, we develop DeepScrub, an LLM-based reinforcement learning framework for fake-order fraud detection. DeepScrub first converts heterogeneous risk signals into unified textual evidence through a semantic unification module, enabling the model to reason over graphs, sequences, and structured records in a common format. It then performs continued pre-training on risk-control corpora to inject domain knowledge, followed by task-specific reward design to optimize both prediction correctness and reasoning quality. Finally, we introduce the SUggest-REflect (SURE) mechanism that combines expert feedback and self-verification to refine intermediate reasoning and improve the reliability of final decisions. The resulting model improves detection performance and provides traceable evidence for production risk-review workflows. We evaluate DeepScrub on a real-world fake-order fraud detection dataset collected from a large O2O platform. The proposed framework achieves the best performance among the evaluated methods in offline experiments. More importantly, in a four-week live pilot, DeepScrub achieved 91.8% precision and 88.5% recall, outperforming human reviewers by 16.6 and 38.8 percentage points, respectively, while reducing manual review workload by 94% and saving nearly one million RMB annually. These results demonstrate that LLM-based reasoning can be translated into measurable operational value in real-world risk-review settings. In summary, our key contributions are as follows: • We present DeepScrub, an LLM-based reinforcement learning framework for fake-order fraud detection in production risk-review workflows on large O2O platforms. • We combine semantic unification, continued domain adaptation, task-specific reward modeling, and the SUggest-REflect (SURE) mechanism to improve prediction accuracy and reasoning quality. • We evaluate DeepScrub through offline experiments and a four-week live pilot, showing improved review accuracy, a 94% reduction in first-stage manual review workload, and nearly one million RMB in annual cost savings. Figure 1. Overview of DeepScrub. The system first adapts a base LLM with risk-control corpora, then converts heterogeneous order evidence into text, and finally applies SURE training to optimize both fraud labels and reasoning traces. 2. Related Work Fraud Detection Fraud detection has been widely studied in e-commerce, finance, and online service platforms. Early and widely deployed approaches rely on expert rules and manually designed risk indicators, which remain effective for high-precision screening and operational recall (Thomas et al., 2017; Dheepa and Dhanapal, 2012; del Mar Roldán-García et al., 2017). Machine learning methods further improve detection by learning behavioral and transactional patterns from historical data (Keles and Gokturk, 2026; Ali et al., 2022). Since many fraud activities are organized through linked accounts, merchants, or coordinated communities, graph-based fraud detection has also become an important direction (Pourhabibi et al., 2020; Cheng et al., 2025b; Pan et al., 2025). More recently, LLMs have been explored for anomaly and fraud detection over tabular or transaction-level data (Tsai et al., 2025; Wang et al., 2026; Cheng et al., 2025a). These methods are useful for risk scoring, filtering, and anomaly detection, but they usually provide limited natural-language reasoning that can be inspected during enforcement, escalation, or appeal handling. This limitation motivates our focus on fake-order fraud review, where the system must produce both a decision and a reviewable reasoning trace. Reinforcement Learning in LLMs With the development of large models in various practical applications (Zhang et al., 2026; Fu et al., 2025; Nie et al., 2026; Wu et al., 2026), reinforcement learning (RL) has also become a common method for improving and adjusting the behavior of large language models (LLM). Reinforcement learning from human feedback (RLHF) trains reward models from human preferences and then optimizes language models with reinforcement learning algorithms such as REINFORCE and proximal policy optimization (PPO) (Christiano et al., 2017; Williams, 1992; Schulman et al., 2017; Ouyang et al., 2022). Later preference-optimization methods, including KTO and DPO, reduce the complexity of reward modeling while improving alignment efficiency (Ethayarajh et al., 2024; Rafailov et al., 2023). group relative policy optimization (GRPO) further improves training efficiency by removing the need for a separately trained critic model (Guo et al., 2025). These methods mainly optimize general preference alignment or final-answer correctness. Fake-order fraud review requires a more specific objective. The model must generate a correct label and a reasoning trace grounded in transaction evidence. DeepScrub addresses this need by combining domain adaptation, task-specific rewards, and SURE training for a deployed review workflow. 3. Problem Formulation Fake-order Fraud Detection. Given a de-identified textual representation of an order, the task is to determine whether the order is involved in fake-order fraud and identify its underlying motivation. The input integrates heterogeneous evidence, including relational networks, temporal behavior sequences, and structured transaction records. The model outputs a reasoning trace e and a label y∈y . The label space is =0,1,2Y=\0,1,2\, corresponding to normal transaction, merchant inflating product sales, and influencer inflating their level. 4. Method of DeepScrub Table 1. Results under different pretraining data ratios. Data C-Eval C-MMLU Qwen3-8B 73.87 76.93 Qwen3-8B (only R) 69.04 73.44 Qwen3-8B (R:G=1:5) 70.59 74.23 Qwen3-8B (R:G=1:10) 72.86 75.11 Qwen3-8B (R:G=1:15) 73.97 75.80 4.1. Overview In this section, we describe the technical details of our DeepScrub. The training pipeline described below is used to build this model, while online inference only requires the trained policy model to generate a label, confidence level, and reasoning trace. As illustrated in Fig. 1(a), to address the lack of domain expertise in general LLMs, we perform domain knowledge injection via continued domain adaptation on a compliance-reviewed corpus composed of public references, governance materials, and de-identified case knowledge. To mitigate potential degradation of the model’s general capabilities caused by domain adaptation, we also incorporate general-domain corpora into the continued pretraining process. As shown in Tab. 1, a risk‑control to general data ratio of 1:15 achieves the best trade-off between domain adaptation and general capability preservation (C-Eval 73.97 vs. 73.87 baseline, essentially preserved; C-MMLU 75.80 vs. 76.93 baseline, a manageable 1.1-point drop). The resulting domain‑enhanced LLM serves as the base model for the subsequent SUggest–REflect (SURE) reinforcement learning stage. The construction of the SURE training data is shown in Fig. 1(b). Given heterogeneous multi-modal risk-assessment signals, we first apply standardized semantic templates to convert networks, sequences, and tabular records into unified textual descriptions, which are compatible with the input of LLMs. Data augmentation and filtering strategies are then employed to ensure label correctness and high data quality. The processed data are used to train the model in the SURE reinforcement learning stage, as depicted in Fig. 1(c). To address the limitation that existing RL methods lack structured reflective feedback for complex brushing detection, we introduce the SURE mechanism. By incorporating feedback from pre-trained risk-control expert and transaction‑expert models through a suggestion module, SURE guides the model to perform self-reflection and iterative refinement, thereby improving both decision quality and interpretability. 4.2. Semantic Unification for Reasoning. Multi-modal Semantic Unification As outlined in Sec. 1, fake-order fraud detection requires processing multi-modal data, which presents a modality gap for text‑based LLMs. To bridge this gap, we define a standardized transformation template that converts heterogeneous data into LLM‑compatible text: ∙ Graph-to-Text. Entity association signals based on de-identified linkage evidence form a graph structure, which is valuable for identifying group risks and enhancing the performance of brushing detection. Thus, we convert the graph structure into a textual format to help LLMs better interpret these important relationships. ∙ Sequence-to-Text. Sequential interaction patterns relevant to risk assessment represent their operational trajectory and risk tendencies, which are crucial for identifying bot-like behavior. We transform de-identified behavior-derived signals into textual representations for downstream reasoning. To enhance the information density of the sequence data, we further aggregate repeated actions in long sequences and summarize action patterns, ensuring high-quality data. ∙ Table-to-Text. For structured records, aggregate indicators relevant to risk assessment are converted into textual summaries. We also apply a template to convert table data into text. Specifically, the feature names and their corresponding values are combined to create complete textual descriptions, while boolean and enumerated variables are transformed into their natural-language descriptions, rather than merely listing numeric values. After these transformations, heterogeneous multi-modal transaction data is unified into a text format that is more easily understood by the LLM, which will then undergo further data augmentation. Fig. 2 illustrates how we construct reasoning-augmented training data. Orders with brief manual analyses are expanded through chain-of-thought (CoT) enhancement, where DeepSeek-R1 (Guo et al., 2025) rewrites expert notes into structured reasoning paths. Orders without rationales are processed through CoT generation conditioned on their labels. We then filter generated rationales through manual verification to reduce label-inconsistent reasoning. Figure 2. Construction of reasoning-augmented training data. Heterogeneous order evidence is first converted into text, then expanded with CoT augmentation or CoT generation, and finally filtered to keep label-consistent reasoning paths. CoT Data Amplification. Annotating fake-order transactions is highly dependent on domain experts, leading to high costs, limited scalability, and a lack of interpretable reasoning in existing labels. To address this issue, we have designed two data augmentation processes to construct training samples that achieve both broad coverage and high quality. ∙ CoT Enhancement. Expert annotations are often brief verdicts (e.g., “suspected brushing”) lacking step‑by‑step logic. We use DeepSeek‑R1 to reconstruct them into coherent reasoning chains that retain expert insight while adding explicit logical structure, enhancing both readability and support for training DeepScrub ’s reasoning capability. The detailed prompt design is provided in App. A.4. ∙ CoT Generation. For the large volume of rationale‑free samples, we employ a generation strategy conditioned on labels, which guides the model to produce reasonable reasoning paths aligned with ground truth annotations. The corresponding prompt templates and generation strategies are detailed in App. A.4. After completing the above data augmentation, all transaction data in textual form are equipped with labels accompanied by high-quality CoT rationales. To ensure consistency between the augmented or generated reasoning chains and the true labels, we further conduct manual verification to filter out noisy data. Figure 3. SURE training mechanism. The policy model first generates candidate responses, receives intrinsic or expert suggestions based on reward signals, and then refines its reasoning through a second-round response. 4.3. Reward Modeling for Fake-order Fraud Detection Rewards guide model training by shaping its optimization direction. For reasoning‑based brushing detection, it is essential to evaluate both the final decision’s correctness and the reasoning process’s quality. However, existing work has largely overlooked this requirement. To address this gap, we design three reward functions for brushing detection: a format reward, an accuracy reward, and a reasoning reward. The detailed designs are described below. ∙ Format reward. Following the common practice in DeepSeek-R1, we introduce a format reward to encourage the model to present its analysis and conclusions in a structured manner. Specifically, the reasoning process is enclosed within the ¡reason¿ and ¡/reason¿ tags, while the final decision is placed between the ¡answer¿ and ¡/answer¿ tags. The resulting format reward is denoted as rformatr_format. ∙ Accuracy reward. This reward measures both the correctness of the model’s prediction and its confidence in the predicted response. Concretely, when determining the fraud motivation of an order, the model is required to output an answer with an associated confidence level (e.g., “¡reason¿…¡/reason¿¡answer¿Merchant inflates sales volume, confidence: high¡/answer¿”). Let the model-generated reasoning process be e^∈ℝ|e^| e ^| e|, the predicted answer be y y, the ground-truth reasoning process be e, the ground-truth answer be y, and the predicted confidence be c c, then we evaluate the model’s judgment accuracy by combining answer correctness and confidence. First, we map the confidence level to a weight: (1) w(c^)=1,if c^=‘high’2/3,if c^=‘medium’1/3,if c^=‘low’,w( c)= cases1,&\ if c=‘high’\\ 2/3,&\ if c=‘medium’\\ 1/3,&\ if c=‘low’ cases, and define the indicator function for answer correctness as (2) (^,)=1,if ^=0,if ^≠,I( y, y)= cases1,&if y= y\\ 0,&if y≠ y cases, Multiplying (y^,y)I( y,y) with confidence weight w(c^)w( c) yields a basic accuracy signal, but this sparse reward may hinder training. To alleviate this issue, we further introduce a process-level score defined as the textual similarity between the model-generated reasoning e e and the ground-truth reasoning e, allowing partial credit when the answer is wrong but the reasoning aligns with expert logic. The similarity is defined as: (3) sim(^,)=f(^)⋅f()‖f(^)‖‖f()‖=⋅‖‖=∑i=1duivi∑i=1dui2∑i=1dvi2,\ sim( e,\ e)\ =\ f( e)\ · f( e)\|f( e)\|\ \|f( e)\|\ =\ u\ · v\| u\|\ \| v\|\ =\ _i=1^d\ u_i\ v_i _i=1^d\ u_i^2\ _i=1^d\ v_i^2\ , where f denotes a Sentence-BERT encoder (Reimers and Gurevych, 2019), u,v∈ℝdu,v ^d are the encoded sentence embeddings, ∥⋅∥\|·\| denotes the L2L_2 norm, and d is the embedding dimension. Finally, we compute the accuracy reward by taking a weighted sum of the above components: (4) raccuracy=(1−α)⋅(y^,y)⋅w(c^)+α⋅sim(e^,e).r_accuracy=(1-α)·I( y,y)· w( c)+α·sim( e,e). Here, raccuracyr_accuracy denotes the resulting accuracy reward, and α is a hyperparameter that controls the contribution of the process similarity score. In our experiments, we set α=0.1α=0.1. ∙ Reasoning reward. The reasoning reward is designed to measure the extent to which the reasoning process contributes to deriving the correct answer, encouraging the model to produce not only correct predictions but also coherent and meaningful reasoning paths. The underlying intuition is that a model response consists of two components—the reasoning process e^∈ℝ|e^| e ^| e| and the final answer y y. If e e is informative and helpful, then the conditional probability of generating the correct answer tokens following e e should be relatively high. Formally, the probability of generating the ground-truth answer y conditioned on the reasoning process e e is defined as (5) A(e^,y)=1|y|∑n=1|y|Pθ(yn∣concat(e^,y<n)),A( e,y)= 1|y| _n=1^|y|P_θ (y_n ( e,y_<n) ), where e e denotes the reasoning process, y is the ground-truth answer, and |y||y| is the length of y. A larger A(e^,y)A( e,y) indicates that the model is more confident in producing y after observing e e. However, this quantity only captures the local contribution of the reasoning process. To assess its global contribution, we further introduce perplexity as a measure of the model’s overall uncertainty over the generated text. Specifically, we concatenate e e and y into a complete sequence a and compute its perplexity as (6) PPL(a)=exp⁡(−1|a|∑n=1|a|log⁡Pθ(an∣a<n)),PPL(a)= (- 1|a| _n=1^|a| P_θ(a_n a_<n) ), where =concat(^,) a=concat( e, y) denotes the concatenated text. To combine these two metrics, we define a composite score score(^,)score( e, y) that integrates both local and global perspectives on the usefulness of e e: (7) score(^,)=A(^,)PPL(),score( e, y)= A( e, y)PPL( a), To further reduce bias, we compute the difference between the score obtained with and without the reasoning process: (8) Δscore=score(^,)−score(∅,), =score( e, y)-score( , y), Moreover, to mitigate reward hacking from Δscore , we explicitly enforce a semantic connection between the reasoning process and the final answer. We extract a set of keywords (y)K(y) associated with each ground-truth answer and compute the proportion γ of these keywords that appear in the model-generated reasoning e e: (9) γ=1|(y)|∑k∈(y)(k∈e^),γ= 1|K(y)| _k (y)I(k∈ e), where (⋅)I(·) is the indicator function. The final reasoning reward is defined as (10) rreason=Δscore×γ.r_reason= ×γ. Here, rreasonr_reason denotes the reasoning reward, γ represents the proportion of answer-related keywords appearing in the reasoning process, and (y)K(y) denotes the keyword set associated with the ground-truth answer y. 4.4. Suggest-reflect Reinforcement Learning As noted in Sec.1, existing RL methods lack the structured reflection needed for fraud reasoning. To overcome this, we propose a reinforcement learning framework with a SUggest–REflect mechanism (SURE). During training, SURE utilizes a pretrained transaction-expert model and a risk-control expert model to provide suggestions or trigger re-evaluation on the policy model’s intermediate responses. These suggestions drive the policy model to reflect on and reconstruct its outputs after incorporating the feedback. SURE aims to help the model better internalize the reasoning pathways underlying fraud prediction, and correct errors systematically. In doing so, SURE encourages the model to “think like an expert” in a systematic manner, thereby continuously improving both decision quality and interpretability in real-world deployments. As shown in Fig. 3, SURE uses an iterative training paradigm with dual-branch (intrinsic or extrinsic) suggestions. Specifically, transaction information is first organized into a prompt and fed into the policy model to generate an initial response. The suggestion module then produces intrinsic or extrinsic feedback via a dual‑branch evaluation. In the reflection stage, the model re‑examines its initial output together with the suggestions to produce a refined response. The advantages of the initial and final responses are then combined via a weighted average to form a mixed advantage signal, which is used to guide the policy update. The method details are as follows. Formally, for each order, the transaction information t is organized into a prompt p and fed into the policy model πθ _θ, which samples a group of first-round outputs (1)=o1(1),o2(1),…,oG(1) o^(1)=\o_1^(1),o_2^(1),...,o_G^(1)\, where G denotes the number of sampled outputs. Each output oi(1)o_i^(1) consists of a reasoning process ^i(1) e_i^(1) concatenated with a final answer ^i(1) y_i^(1), with i∈[1,G]i∈[1,G]. Subsequently, the first-round responses (1) o^(1) are passed to the suggestion module. ∙ Suggest. Within the suggestion module, we first compute the reward scores for (1) o^(1) using the task-specific reward function introduced in Sec. 4.3, obtaining (1)=rformat(1),raccuracy(1),rreason(1) r^(1)=\r_format^(1),r_accuracy^(1),r_reason^(1)\. To prevent excessive deviation of the policy model’s own distribution caused by an overload of external suggestions (as also noted in (Yan et al., 2025)), we introduce a dual-branch decision mechanism. Specifically, for each first-round output i(1) o_i^(1), a branch is selected based on its accuracy reward ri,accuracy(1)r_i,accuracy^(1). When the correctness reward is low (below a threshold μ), it indicates that the model’s understanding is inadequate; in this case, we additionally invoke transaction‑expert and risk‑control‑expert models to provide extrinsic suggestions. Otherwise, if the correctness reward is sufficiently high, the model is considered to have produced a relatively reliable response, and only intrinsic suggestions are obtained to guide the model in self‑reflection. (11) si=siin,if the accuracy reward ri,accuracy(1)≥μ,siex,otherwise.s_i= casess_i^in,&if the accuracy reward r_i,accuracy^(1)≥μ,\\ s_i^ex,&otherwise. cases Here, siins_i^in denotes the intrinsic suggestion for output oi(1)o_i^(1), which prompts the policy model to conduct self-reflection—checking the plausibility of its reasoning chain and exploring potentially better inference paths. Meanwhile, siexs_i^ex denotes the extrinsic suggestion for oi(1)o_i^(1), which is provided by two external models: a transaction‑expert model and a risk-control-expert model. Specifically, the transaction-expert model compares the policy model’s generated reasoning ^i(1) e_i^(1) and answer ^i(1) y_i^(1) with the ground-truth reasoning e and answer y, and suggests that the policy model focus on the misunderstood or overlooked key points in the transaction information. Similarly, the risk-control-expert model highlights anomalous features that could help improve the accuracy of fake-order fraud detection. Detailed prompt designs for these suggestions are provided in Appendix A.4. ∙ Reflect. In the subsequent reflection stage, for each first-round output oi(1)o_i^(1), the suggestion content sis_i together with the original output oi(1)o_i^(1) are fed again into the policy model πθ _θ, prompting the model to examine its initial response and generate a refined final answer oi(2)o_i^(2). Analogously, we compute the reward scores for the final responses (2) o^(2) using the same task-specific reward function introduced in Sec.4.3, obtaining (2)=rformat(2),raccuracy(2),rreason(2) r^(2)=\r_format^(2),r_accuracy^(2),r_reason^(2)\. Finally, the advantage of the two rounds of responses is combined via a weighted average to obtain the mixed advantage AiA^i. Specifically, we introduce a hyperparameter ω∈[0,1]ω∈[0,1] to control the weighting between the second-round advantage Ai(2)=ri(2)−mean((2))std((2))A_i^(2)= r_i^(2)-mean( r^(2))std( r^(2)) and the advantage increment ΔAi=Ai(2)−Ai(1) A_i=A_i^(2)-A_i^(1), as formulated below: (12) Ai=ωAi(2)+(1−ω)ΔAi=ωAi(2)+(1−ω)(Ai(2)−Ai(1)).A_i=ω A_i^(2)+(1-ω) A_i=ω A_i^(2)+(1-ω)(A_i^(2)-A_i^(1)). where Ai(1)A_i^(1) and Ai(2)A_i^(2) denote the advantage values of the first- and second-round responses oio_i, respectively; ΔAi A_i measures the improvement in advantage after the suggestion–reflection stage; and AiA_i represents the blended advantage. This blended advantage is then used as the advantage term in the GRPO objective function to guide the update of the model parameters. 5. Dataset Construction Our evaluation corpus is derived from a large-scale operational risk-control environment over a one-year observation window. As detailed in Sec. 4.2, each record is transformed into a de-identified textual representation for analysis. The construction of the dataset does not rely on a single validation method; instead, it employs a multi-layered filtering and cross-validation process to ensure the accuracy of sample labeling. The specific steps are as follows: Acquiring Fraudulent Samples. Samples associated with fake-order fraud are directly extracted from daily risk-control enforcement outcomes, which provide the most reliable source of positive labels in our setting. These enforcement outcomes fall into three categories: cases labeled by experts, detections based on high-precision rules, and orders confirmed through law enforcement. Acquiring Benign Samples. Selected from orders with prohibitively high fraud costs or logically low fraud probability, including: high‑value purchases (e.g., appliances, luxury goods), orders with extensive pre‑purchase consultation or reviews after consumption, and orders from users with long registration and rich purchase history. Data Cleaning and Cross-validation. After obtaining the initially selected benign samples and fake-order fraud samples, we further process the data to exclude potential labeling errors. First, we remove samples with ambiguous labels. We then check feature consistency by filtering benign samples that exhibit patterns indicative of fake-order fraud and re-examining positive samples that closely resemble normal orders. We evaluate DeepScrub on a real-world dataset collected from a major O2O e-commerce platform. The dataset contains approximately 150K de-identified orders and covers three categories: Merchant Inflating Product Sales, Influencer Inflating Their Level, and Normal Transaction. The raw class distribution is highly imbalanced. To better reflect deployment conditions and reduce temporal leakage, we split the data chronologically, using roughly the earlier 80% for training and the later 20% for testing. After undersampling the majority class in training, the class ratio is approximately 3 (Merchant) : 1 (Influencer) : 50 (Normal). Table 2. Experimental results on our dataset for fake-order fraud detection. Methods Metrics Influencer Inflating Their Level Merchant Inflating Product Sales Normal Transaction Overall (macro) Prec. Rec. F1 Prec. Rec. F1 Prec. Rec. F1 Prec. Rec. F1 Qwen3-32B (Yang et al., 2025) 0.858 0.866 0.862 0.654 0.710 0.681 0.789 0.722 0.754 0.767 0.766 0.765 Qwen3-8B (Yang et al., 2025) (SFT) 0.905 0.923 0.914 0.759 0.719 0.739 0.814 0.835 0.824 0.826 0.826 0.826 InternLM3-8B (Cai et al., 2024) (SFT) 0.909 0.791 0.846 0.702 0.543 0.612 0.650 0.860 0.740 0.754 0.731 0.733 Llama-3-8B (Dubey et al., 2024) (SFT) 0.918 0.915 0.916 0.752 0.727 0.739 0.813 0.826 0.819 0.827 0.823 0.825 GLM-4-9B (GLM et al., 2024) (SFT) 0.906 0.928 0.917 0.769 0.687 0.725 0.801 0.854 0.826 0.825 0.823 0.823 DeepScrub w/o RL 0.913 0.915 0.914 0.772 0.719 0.744 0.826 0.834 0.830 0.837 0.823 0.830 DeepScrub w/o SURE 0.909 0.932 0.920 0.780 0.732 0.755 0.822 0.850 0.836 0.837 0.838 0.837 DeepScrub w/o raccuracyr_accuracy 0.883 0.921 0.902 0.710 0.698 0.704 0.800 0.750 0.774 0.798 0.790 0.793 DeepScrub w/o rreasoningr_reasoning 0.914 0.938 0.926 0.782 0.731 0.755 0.832 0.871 0.851 0.843 0.846 0.844 DeepScrub 0.914 0.959 0.936 0.800 0.751 0.774 0.832 0.869 0.850 0.849 0.860 0.853 6. Experiment This section evaluates the effectiveness of our DeepScrub by addressing the following research questions: ∙ 1 : Does DeepScrub improve fake-order fraud review accuracy over baseline models? (Sec. 6.2) ∙ 2 : Which components are responsible for the performance gains? (Sec. 6.3) ∙ 3 : How does SURE improve reasoning quality through expert suggestions and self-verification? (Sec. 6.4) 6.1. Experimental Setup Training. Based on the domain‑enhanced model (Sec. 4.1), we perform supervised fine‑tuning (SFT) on 20% of the CoT‑augmented training set (Sec. 4.2) and reinforcement learning on the remaining 80%. Optimization employs a learning rate of 1×10−51× 10^-5, a warmup ratio of 0.05, and a group size of 6. All experiments are run on 4 computing nodes with 32 high-performance GPUs, using a global batch size of 16 and a total training time of approximately 108 hours. Key training hyperparameters are set as follows: reflection threshold μ=0.5μ=0.5, mixing weight between learning rounds ω=0.7ω=0.7. The training pipeline is implemented using the ms‑swift (Zhao et al., 2024) framework. Downstream Tasks. The task is to classify orders into normal transaction, merchant inflating product sales, or influencer inflating their level, providing an interpretable rationale and reasoning path. To prevent feature leakage, all orders in the test set are generated after those in the training set. For the fake-order fraud detection task, we evaluate using precision, recall, and F1, balancing the imbalanced classes via random sampling for per‑label analysis and using macro‑averaging for overall performance. The same protocol is applied to all baselines and our method for fair comparison. Baselines. In this study, to evaluate the effectiveness of the proposed approach and address the practical need for resource constrained deployment, we design a hierarchical baseline system. First, we compare a smaller, task‑optimized model (based on Qwen3‑8B (Yang et al., 2025)) that has undergone domain enhancement and reinforcement learning against a much larger, general‑purpose model (Qwen3‑32B (Yang et al., 2025)) that has not been fine‑tuned for the task, testing whether targeted optimization can surpass scale. Second, at a similar parameter scale, we select widely used open‑source models, including InternLM3-8B-Instruct222InternLM3-8B-Instruct is an updated release from the same team as InternLM2 (Cai et al., 2024), Llama3‑8B (Dubey et al., 2024), and GLM4‑9B (GLM et al., 2024) and fine‑tune them on the same training data as our method, thereby comparing different architectures under a fair training setup. This baseline design directly examines the trade‑off between small, tuned models and large, generic ones, while ensuring the gains are attributable to our contributions. 6.2. Experimental Results Tab. 2 summarizes the overall performance of DeepScrub and major baseline methods on the fake-order fraud detection task. DeepScrub achieves the best results across all three classification subtasks, with a macro‑averaged F1‑score of 0.853, substantially outperforming all baselines. In detail, for influencer’s level inflation, a relatively well‑defined pattern, DeepScrub attains an F1 of 0.936, demonstrating its stability on clear fraud signals. The most challenging merchant sales inflation subtask sees DeepScrub reaching an F1 of 0.774, exceeding the best fine-tuned baseline (0.739) by 3.5 points, which confirms its capability to detect complex, concealed brushing behaviors. Meanwhile, on normal transaction recognition, DeepScrub also achieves the highest F1 among all baselines (0.850), indicating that improved fraud detection does not come at the cost of misclassifying legitimate orders. Notably, our optimized 8B-level model outperforms the larger-scale Qwen3-32B model, which lacks domain adaptation, showing that task‑specific optimization is more effective than simply scaling model size in professional risk‑control scenarios. Table 3. Correcting misclassification via experts’ suggestions. Question: … You need to reason step by step based on the order characteristics to optimize the transaction motive derived from the manual analysis … Basic transaction information of the order … First-round Response: ¡reason¿ … the focal entity shows strong association signals with multiple roles in the transaction chain … The behavioral pattern exhibits an anomalous cycle of ”concentrated ordering followed by no verification or consumption.”…It is possible that the user is artificially inflating the payment-based Gross Merchandise Volume (GMV) metric to meet assessment targets (where ”GMV” carries high weight in the influencer-level evaluation system)…¡/reason¿¡answer¿Influencer inflating their level.¡/answer¿ Transaction-expert’s Suggestion:…The direct association between the user’s identity and the merchant is not identified… It is necessary to re-examine the differences between the influencer’s level assessment metrics and the merchant sales evaluation criteria… Risk-control-expert’s Suggestion:…Abnormal surge in transaction amount… The timing distribution does not align with the user’s typical behavior patterns… Second-round Response: ¡reason¿…The user acts as an entity associated with the merchant… The merchant fabricates sales volume through linked accounts… However, the user’s behavior aligns more closely with ”merchant inflating product sales,” indicating a merchant-driven operational logic…¡/reason¿¡answer¿Merchant inflating product sales.¡/answer¿ Table 4. Refining correct reasoning through self‑verification. The Question part is the same as that of Tab. 3. First-round Response: ¡reason¿ …The entity’s profile is strongly consistent with the associated merchant type (based on contextual alignment signals). Their transactional behavior was characterized by high frequency, uniformity, and coordinated group activity which exhibited clear signs of sales inflation. The objective is to fabricate ”product popularity” data to artificially boost merchant sales performance… ¡/reason¿¡answer¿Merchant inflating product sales.¡/answer¿ Self-verification:…Your conclusion is correct. Please proceed with the following steps for self‑verification to confirm whether a more optimal reasoning path exists… Second-round Response: ¡reason¿…Behavior-derived risk patterns, together with cross-context consistency checks (association with workplace scenarios) and the merchant’s promotional context (Double 11 period, recent platform entry) collectively indicate a brushing scenario where the merchant aims to inflate sales of ”gas + car wash” vouchers…¡/reason¿¡answer¿Merchant inflating product sales.¡/answer¿ 6.3. Ablation Study To validate the contribution of each proposed component, we conduct four structured ablation experiments: (1) DeepScrub w/o RL: removing the entire reinforcement learning phase (SFT‑only baseline); (2) DeepScrub w/o SURE: removing the entire SURE mechanism; (3) DeepScrub w/o raccuracyr_accuracy: ablating the accuracy reward while keeping the format and reasoning rewards, and (4) DeepScrub w/o rreasoningr_reasoning: excluding the reasoning reward while retaining the accuracy and format rewards. Compared to the SFT‑only baseline, DeepScrub improves macro‑F1 from 0.830 to 0.853, showing that RL enables policy refinement beyond static supervision. Keeping RL but disabling SURE still achieves a competitive macro‑F1 (0.837), yet performance on “merchant inflating product sales” drops (F1=0.755 vs. 0.774), underscoring the value of interactive reflection for learning nuanced fraud patterns. Removing the accuracy reward causes the steepest decline: macro‑F1 falls to 0.793, and “merchant sales inflation” F1 drops to 0.704, confirming that this reward is essential for distinguishing fraudulent from legitimate behaviors. Excluding the reasoning reward yields a marginal drop on the simpler ‘influencer-level inflation’ task (F1=0.926 vs. 0.936) and a larger drop on ‘merchant sales inflation’ (0.755 vs. 0.774), indicating that the reasoning reward helps maintain logical consistency in complex scenarios. 6.4. Case Study Tables 3 and 4 illustrate the two roles of SURE. In Tab. 3, expert suggestions help the model correct a wrong first-round label by revisiting overlooked merchant-user relationships and transaction patterns. In Tab. 4, self-verification improves an already correct prediction by adding more specific business context and behavioral evidence. These examples show that SURE improves both error correction and reasoning quality, which supports review-stage decisions in fake-order fraud detection. 7. Real-world Deployment To verify the practical value of DeepScrub, we deployed it in a production risk-review workflow for local-service orders. DeepScrub is not used as a full-traffic detector. It operates after lightweight rules recall suspicious orders and automates the first-stage review that was previously handled by standard human reviewers. Deployment Architecture. DeepScrub runs as an automated first-stage reviewer in the human-audit pipeline on a cluster of six high-performance GPUs. Its workflow follows four stages: (1) filtering suspicious orders (flagged by risk‑control rules) and preparing multimodal inputs; (2) converting graphs, sequences, and tables into text via Sec. 4.2; (3) generating a fraud verdict with a reasoning trace; (4) escalating cases to experts based on a threshold. Pilot Evaluation. During a four-week online trial, DeepScrub processed approximately 10,000 orders initially flagged as potentially fraudulent by risk control rules. All orders were simultaneously reviewed by the standard first-stage reviewers used in the production workflow for comparison. These reviewers had routine training but were not senior fraud experts. Final evaluation labels were adjudicated by senior fraud experts using the same evidence available in the production review workflow. The results showed that DeepScrub achieved a precision of 91.8% (vs. 75.2% human) and a recall of 88.5% (vs. 49.7% human), representing improvements of 16.6 and 38.8 percentage points, respectively. Production Impact. The pilot confirms that DeepScrub provides stronger first-stage review performance than standard human review in both coverage and accuracy. It is currently in production, reducing the first-stage manual review workload by 94% while cutting the time to review a single order from over ten minutes to a few seconds, saving nearly one million RMB annually. Each verdict includes a traceable reasoning chain to support expert adjudication. These results show that DeepScrub supports production fraud review with higher accuracy, lower first-stage review workload, and traceable reasoning for expert adjudication. 8. Conclusion We presented DeepScrub, an LLM-based framework for fake-order fraud detection in production O2O risk-review workflows. By combining semantic unification, domain adaptation, and SURE training, DeepScrub improves detection performance and produces reasoning traces for review. Future work will extend the framework to broader fraud categories, reduce inference cost, and strengthen human review for high-impact cases. References (1) Ali et al. (2022) Abdulalem Ali, Shukor Abd Razak, Siti Hajar Othman, Taiseer Abdalla Elfadil Eisa, Arafat Al-Dhaqm, Maged Nasser, Tusneem Elhassan, Hashim Elshafie, and Abdu Saif. 2022. Financial fraud detection based on machine learning: a systematic literature review. Applied Sciences 12, 19 (2022), 9637. Cai et al. (2024) Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, et al. 2024. InternLM2 Technical Report. arXiv:2403.17297 [cs.CL] Cheng et al. (2025b) Dawei Cheng, Yao Zou, Sheng Xiang, and Changjun Jiang. 2025b. Graph neural networks for financial fraud detection: a review. Frontiers of Computer Science 19, 9 (2025), 199609. Cheng et al. (2025a) Sijie Cheng, Yanbo Yang, Jiawei Zhang, and Pengfei Li. 2025a. LLM-TAD: Interpretable Ethereum Fraud Detection Based on Large Language Models. In IEEE International Conference on Blockchain and Trusted Intelligence Systems (ICBCTIS). Christiano et al. (2017) Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems 30 (2017). del Mar Roldán-García et al. (2017) María del Mar Roldán-García, José García-Nieto, and José F Aldana-Montes. 2017. Enhancing semantic consistency in anti-fraud rule-based expert systems. Expert Systems with Applications 90 (2017), 332–343. Dheepa and Dhanapal (2012) V Dheepa and R Dhanapal. 2012. Behavior based credit card fraud detection using support vector machines. ICTACT Journal on Soft computing 2, 4 (2012), 391–397. Dubey et al. (2024) Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv e-prints (2024), arXiv–2407. Ethayarajh et al. (2024) Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306 (2024). Fu et al. (2025) Chenghan Fu, Daoze Zhang, Yukang Lin, Zhanheng Nie, Xiang Zhang, Jianyu Liu, Yueran Liu, Wanxian Guan, Pengjie Wang, Jian Xu, et al. 2025. MOON Embedding: Multimodal Representation Learning for E-commerce Search Advertising. arXiv preprint arXiv:2511.11305 (2025). GLM et al. (2024) Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, et al. 2024. ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. arXiv:2406.12793 Guo et al. (2025) Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025). Keles and Gokturk (2026) Mustafa Berk Keles and Mehmet Gokturk. 2026. BI-GBDT: A Graph-Free Behavioral Interaction-Aware Gradient Boosting Framework for Fraud Detection in Large-Scale Payment Systems. Applied Sciences 16, 2 (2026). doi:10.3390/app16020876 Nie et al. (2026) Zhanheng Nie, Chenghan Fu, Daoze Zhang, Junxian Wu, Wanxian Guan, Pengjie Wang, Jian Xu, and Bo Zheng. 2026. Moon2. 0: Dynamic modality-balanced multimodal representation learning for e-commerce product understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 22975–22985. Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35 (2022), 27730–27744. Pan et al. (2025) Junjun Pan, Yixin Liu, Xin Zheng, Yizhen Zheng, Alan Wee-Chung Liew, Fuyi Li, and Shirui Pan. 2025. A label-free heterophily-guided approach for unsupervised graph fraud detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 12443–12451. Pourhabibi et al. (2020) Tahereh Pourhabibi, Kok-Leong Ong, Booi H Kam, and Yee Ling Boo. 2020. Fraud detection: A systematic literature review of graph-based anomaly detection approaches. Decision Support Systems 133 (2020), 113303. Rafailov et al. (2023) Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems 36 (2023), 53728–53741. Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019). Schulman et al. (2017) John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017). Thomas et al. (2017) Lyn Thomas, Jonathan Crook, and David Edelman. 2017. Credit scoring and its applications. SIAM. Tsai et al. (2025) Che-Ping Tsai, Ganyu Teng, Phillip Wallis, and Wei Ding. 2025. AnoLLM: Large Language Models for Tabular Anomaly Detection. In International Conference on Learning Representations (ICLR). Wang et al. (2026) Ruiqi Wang, Ruikang Liu, Runyu Chen, Haoxiang Suo, Zhiyi Peng, Zhuo Tang, and Changjian Chen. 2026. CausalTAD: Injecting Causal Knowledge into Large Language Models for Tabular Anomaly Detection. arXiv preprint arXiv:2602.07798 (2026). Williams (1992) Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8, 3 (1992), 229–256. Wu et al. (2026) Junxian Wu, Chenghan Fu, Zhanheng Nie, Daoze Zhang, Bowen Wan, Wanxian Guan, Chuan Yu, Jian Xu, and Bo Zheng. 2026. MOON3. 0: Reasoning-aware Multimodal Representation Learning for E-commerce Product Understanding. arXiv preprint arXiv:2604.00513 (2026). Yan et al. (2025) Jianhao Yan, Yafu Li, Zican Hu, Zhi Wang, Ganqu Cui, Xiaoye Qu, Yu Cheng, and Yue Zhang. 2025. Learning to Reason under Off-Policy Guidance. arXiv:2504.14945 [cs.LG] https://arxiv.org/abs/2504.14945 Yang et al. (2025) An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, et al. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] https://arxiv.org/abs/2505.09388 Zhang et al. (2026) Daoze Zhang, Chenghan Fu, Zhanheng Nie, Jianyu Liu, Wanxian Guan, Yuan Gao, Jun Song, Pengjie Wang, Jian Xu, and Bo Zheng. 2026. MOON: Generative MLLM-based multimodal representation learning for e-commerce product understanding. In Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining. 924–933. Zhao et al. (2024) Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, et al. 2024. SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning. arXiv:2408.05517 [cs.CL] https://arxiv.org/abs/2408.05517 Appendix A Appendix A.1. Ethics and Privacy This study strictly adheres to ethical standards and robust data security protocols to ensure data integrity, transparency, and user privacy. The study is evaluated on de-identified, access-controlled records derived from a large-scale online service platform under internal governance controls. Only authorized researchers who have undergone a formal internal approval process can access this proprietary data, thus ensuring data confidentiality and platform security. All research records were processed under internal governance procedures and restricted to de-identified, aggregate, and access-controlled representations. No direct personal identifiers were used in the research workflow. This data is used solely to develop and evaluate fraud detection methods within a privacy-preserving framework that complies with relevant data protection regulations and platform policies. To further ensure the transparency and security of our methodology, we employ only large, open-source language models and publicly available reinforcement learning libraries throughout the training and evaluation process. We do not use any proprietary or closed-source model components, thereby eliminating the risks associated with reliance on commercial models and improving the reproducibility of our research. A.2. Semantic transformation example ∙ Graph-to-Text. For example, the group graph structure can be transformed into: “A de-identified entity group contains 17 linked accounts, with aggregated activity observed over the recent period…” ∙ Sequence-to-Text. The final representation of a de-identified interaction sequence might look like: “A representative sequence of platform events relevant to risk assessment was observed and summarized.” ∙ Table-to-Text. Aggregated activity indicators over a short observation window were summarized into structured textual evidence, together with contextual consistency signals. A.3. Evaluation Metrics ∙ Precision : Precision measures the proportion of true positive predictions among all instances predicted as positive. It is particularly important when the cost of false positives is high, as it reflects the model’s ability to avoid erroneous positive classifications. A higher precision indicates more reliable positive predictions, while a lower value suggests a greater rate of false positives in the model’s output. Precision=TPTP+FP,Precision= TPTP+FP, where TP is the number of true positives and FP is the number of false positives. ∙ Recall : Recall (also called sensitivity or true positive rate) quantifies the proportion of actual positive instances that the model correctly identifies. This metric is critical in scenarios where missing positive cases, such as in medical screening or fraud detection. A high recall indicates that the model captures the majority of relevant positive cases. Recall=TPTP+FN,Recall= TPTP+FN, where TP is the number of true positives and FN is the number of false negatives. ∙ F1-score : F1-score is the harmonic mean of precision and recall, which balances the trade-off between the two metrics by assigning equal weight to both. It is optimal for scenarios where false positives and false negatives are equally costly, providing a single comprehensive measure of model performance on positive classes. F1-score=2⋅Precision⋅RecallPrecision+Recall=2TP2TP+FP+FN,F1-score=2· Precision·RecallPrecision+Recall= 2TP2TP+FP+FN, where TP is the number of true positives, FP is the number of false positives, and FN is the number of false negatives. ∙ Macro-averaged Precision : Macro-averaged Precision (Macro-precision) calculates the unweighted average of precision values across all classes, treating each class equally regardless of its sample size. This metric is suitable when all classes have equal importance, avoiding bias toward majority classes in imbalanced datasets. Macro-precision=1C∑i=1CPrecisioni,Macro-precision= 1C _i=1^CPrecision_i, where C is the total number of classes, and PrecisioniPrecision_i is the precision value of the i-th class. ∙ Macro-averaged Recall : Macro-averaged Recall (Macro-recall) computes the unweighted average of recall values for all classes, giving equal consideration to each class’s performance. It is critical for evaluating model ability to identify instances of all classes equally, especially in multi-class classification tasks with balanced class importance. Macro-recall=1C∑i=1CRecalli,Macro-recall= 1C _i=1^CRecall_i, where C is the total number of classes, and RecalliRecall_i is the recall value of the i-th class. ∙ Macro-averaged F1-score : Macro-averaged F1-score (Macro-F1) is the unweighted average of F1-score values across all classes, balancing precision and recall equally for every class. It serves as a holistic multi-class performance metric that avoids favoring large classes, making it ideal for imbalanced multi-class classification with equal class priority. Macro-F1=1C∑i=1CF1i,Macro-F1= 1C _i=1^CF1_i, where C is the total number of classes, and F1-scoreiF1-score_i is the F1-score value of the i-th class. A.4. List of Prompt Prompt of CoT Augmentation ⬇ ## Role and Task You are an expert in analyzing transaction motives for an O2O platform. Your core task is to identify transaction motives by analyzing order characteristics. The user will provide order information, the corresponding transaction motive, and manual analysis records. You need to reason step by step based on the order characteristics to optimize the transaction motive derived from the manual analysis. ## Known descriptions of transaction motive scenarios 1.Merchant inflating product sales: A behavior of artificially inflating the product’s visible sales volume at a minimal cost. 2.Influencer inflating their level: Increase the influencer’s level by self-buying or hiring the black market to create fake orders. 3.Normal Transaction: A normal transaction refers to a purchase behavior where a consumer places an order, makes payment, and fulfills the obligation (such as in-store verification) or completes a compliant refund within the platform’s established rules, all driven by genuine and spontaneous consumption needs. ## Analysis requirements Follow the steps below based on the provided transaction motive and manual analysis. Reason out the possibilities step by step based on the characteristic data.Consider multi-dimensional characteristics such as time, space, and behavior patterns. Compare normal business fluctuations with abnormal patterns. Give a confidence assessment. ## Output format <reason> 1. First point of analysis... 2. Second point of analysis... </reason> The most likely motive scenario is: Scenario name Confidence level: [High/Medium/Low] Additional notes: [If any] ## Basic transaction information of the order feature ## Manual analysis manual_analysis ## Transaction motive true_label Prompt of CoT Generation ⬇ ## Role and Task You are an expert in analyzing transaction motives for an O2O platform. Your core task is to identify transaction motives by analyzing order characteristics.The user will provide order information and the corresponding transaction motive. You need to reason step by step based on the order characteristics to optimize the transaction motive derived from the manual analysis. ## Known descriptions of transaction motive scenarios 1.Merchant inflating product sales: A behavior of artificially inflating the product’s visible sales volume at a minimal cost. 2.Influencer inflating their level: Increase the influencer’s level by self-buying or hiring the black market to create fake orders. 3.Normal Transaction: A normal transaction refers to a purchase behavior where a consumer places an order, makes payment, and fulfills the obligation (such as in-store verification) or completes a compliant refund within the platform’s established rules, all driven by genuine and spontaneous consumption needs. ## Analysis requirements Follow the steps below based on the provided transaction motive and manual analysis. Reason out the possibilities step by step based on the characteristic data.Consider multi-dimensional characteristics such as time, space, and behavior patterns. Compare normal business fluctuations with abnormal patterns.Give a confidence assessment. ## Output format <reason> 1. First point of analysis... 2. Second point of analysis... </reason> The most likely motive scenario is: Scenario name Confidence level: [High/Medium/Low] Additional notes: [If any] ## Basic transaction information of the order feature ## Transaction motive true_label Prompt of Training ⬇ ## Role and Task You are an expert in analyzing transaction motives for an O2O platform. Your core task is to identify transaction motives by analyzing order characteristics. You need to reason step by step based on the order characteristics and finally output a precise judgment result of the transaction motive. ## Known descriptions of transaction motive scenarios 1.Merchant inflating product sales: A behavior of artificially inflating the product’s visible sales volume at a minimal cost. 2.Influencer inflating their level: Increase the influencer’s level by self-buying or hiring the black market to create fake orders. 3.Normal Transaction: A normal transaction refers to a purchase behavior where a consumer places an order, makes payment, and fulfills the obligation (such as in-store verification) or completes a compliant refund within the platform’s established rules, all driven by genuine and spontaneous consumption needs. ## Analysis requirements Reason out the possibilities step by step based on the characteristic data.Consider multi-dimensional characteristics such as time, space, and behavior patterns. Compare normal business fluctuations with abnormal patterns.Give a confidence assessment. ## Output format <reason> 1. First point of analysis... 2. Second point of analysis... </reason> <answer> The most likely motive scenario is: Scenario name Confidence level: [High/Medium/Low] Additional notes: [If any] </answer> ## Basic transaction information of the order feature Prompt of Self-verification ⬇ ## Task Description Your conclusion is correct. However, to ensure the rigour of the reasoning, please self - verify according to the following steps: 1.Sort out the key information in the question. 2.Gradually check your logical derivation process and its rationality. 3.Confirm whether there is a better solution path. 4.Finally, re-affirm the conclusion and answer the order motive again according to the following format. ## Output format <reason> 1. First point of analysis... 2. Second point of analysis... </reason> <answer> The most likely motive scenario is: Scenario name Confidence level: [High/Medium/Low] Additional notes: [If any] </answer> Prompt of Reflection ⬇ ## Task Description Please gradually check your current reasoning process and conclusions to identify possible logical loopholes, data biases, or insufficient business understanding. 1.For each step of reasoning, clarify: Whether the basis is sufficient (data, rules, business common sense) Whether the logic is rigorous (are there leaps or unvalidated assumptions?) Whether the conclusion covers key scenarios (are there exceptions not considered?) 2.The following are improvement suggestions provided by trade experts: domain_suggest; please analyze these suggestions item by item and clarify: Which suggestions can directly correct your reasoning loopholes? How to correct them? Which suggestions require additional data or verification? Are there priorities? Avoid directly copying the suggestions; instead, evaluate their applicability in combination with your original reasoning. 3.The following is the analysis of abnormal behaviors by risk experts: riskcontrol_suggestion; please compare your answer with the risk control expert’s answer: Behavior coverage: Are there abnormal behavior patterns you did not notice? Risk weight: Did you underestimate the risk of certain behaviors? Difference explanation: If the conclusions are inconsistent, is it due to data differences, different business assumptions, or omissions in analysis dimensions? 4.Finally, re-answer the order motive based on the questions according to the following format: <reason> 1. First point of analysis... 2. Second point of analysis... </reason> <answer> The most likely motive scenario is: Scenario name Confidence level: [High/Medium/Low] Additional notes: [If any] </answer> Prompt of Suggestions from Transaction-expert ⬇ ## Role and Task As an expert in O2O platform business, please conduct a strict comparative analysis of the business knowledge differences between the following student’s answer and the reference answer. Note: Only point out the deficiencies or errors in professional knowledge in the student’s answer and provide suggestions for improvement directions, but do not directly disclose the specific content of the reference answer. ## Input format Student’s answer: answer Reference answer: true_answer ## Output requirements (strictly follow this format): 1.Knowledge deficiencies: List the specific business concepts, terms, or logics that are missing/incorrect in the student’s answer (no more than 3 items). 2.Improvement suggestions: Provide suggestions for each deficiency ## Example output: Knowledge deficiencies: Lack of analysis of key business indicators. Improvement suggestions: It is recommended to systematically learn the platform’s core indicator system. Prompt of Suggestions from Risk-control-expert ⬇ ## Role and Task You are a professional expert in analyzing brushing behavior. Please conduct a systematic analysis based on the provided features. Before giving the final judgment, gradually carry out the thinking process according to the following requirements to determine whether the given order has transaction abnormalities: 1.Feature analysis: Analyze the suspicious indicators of the input features item by item. 2.Correlation verification: Check the logical relevance between features. 3.Pattern comparison: Compare with common brushing behavior patterns. 4.Abnormality assessment: Identify features that do not conform to normal transaction rules. ## Output format requirements: <reason> 1. State the analysis logic step by step. 2. Only speak based on the provided feature data. 3. Mark key judgment basis. </reason> <answer> There is an abnormal transaction./There is not an abnormal transaction. </answer>

Instant research discovery