Paper deep dive
WisdomInterrogatory (LuWen): An Open-Source Legal Large Language Model Technical Report
Yiquan Wu, Yuhang Liu, Yifei Liu, Ang Li, Siying Zhou, Kun Kuang, Fei Wu
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 98%
Last extracted: 4/14/2026, 1:40:14 AM
Summary
WisdomInterrogatory (LuWen) is an open-source Chinese legal large language model built on the Baichuan foundation model. It utilizes three core techniques: continual pre-training on a large-scale legal corpus, supervised fine-tuning with curated legal instruction data, and retrieval-augmented generation (RAG) to ensure accuracy and timeliness in legal reasoning and task performance.
Entities (4)
Relation Signals (3)
WisdomInterrogatory (LuWen) → builton → Baichuan1-7B
confidence 100% · built upon the Baichuan foundation model
WisdomInterrogatory (LuWen) → developedby → Zhejiang University
confidence 100% · Yiquan Wu... Zhejiang University
WisdomInterrogatory (LuWen) → usestechnique → Retrieval-Augmented Generation
confidence 100% · we utilize the RAG method to link the model with legal databases
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models have demonstrated remarkable capabilities across a wide range of natural language processing tasks, yet their application in the legal domain remains challenging due to the specialized terminology, complex reasoning requirements, and rapidly evolving legal knowledge involved. In this paper, we present WisdomInterrogatory (LuWen), an open-source Chinese legal language model built upon the Baichuan foundation model through three key techniques: continual pre-training on a large-scale legal corpus, supervised fine-tuning with carefully curated legal instruction data, and retrieval-augmented generation integrated with a comprehensive legal knowledge base. We evaluate LuWen on five representative legal tasks spanning both prediction and generation settings, including legal judgment prediction, judicial examination, legal text summarization, law article question answering, and judicial decision reasoning. Experimental results show that LuWen outperforms several strong baselines, demonstrating the effectiveness of our approach in adapting general-purpose language models to the legal domain.
Tags
Links
- Source: https://arxiv.org/abs/2604.06737v2
- Canonical: https://arxiv.org/abs/2604.06737v2
Trouble viewing inline? Open PDF directly →
Full Text
37,434 characters extracted from source content.
Expand or collapse full text
WisdomInterrogatory (LuWen): An Open-Source Legal Large Language Model Technical Report Yiquan Wu, Yuhang Liu, Yifei Liu, Ang Li, Siying Zhou, Kun Kuang * , Fei Wu Zhejiang University, Hangzhou, China wuyiquan, kuangkun@zju.edu.cn Abstract Large language models have demonstrated re- markable capabilities across a wide range of natural language processing tasks, yet their application in the legal domain remains chal- lenging due to the specialized terminology, complex reasoning requirements, and rapidly evolving legal knowledge involved. In this pa- per, we present WisdomInterrogatory (LuWen), an open-source Chinese legal language model built upon the Baichuan foundation model through three key techniques: continual pre- training on a large-scale legal corpus, super- vised fine-tuning with carefully curated legal instruction data, and retrieval-augmented gen- eration integrated with a comprehensive legal knowledge base. We evaluate LuWen on five representative legal tasks spanning both pre- diction and generation settings, including legal judgment prediction, judicial examination, le- gal text summarization, law article question answering, and judicial decision reasoning. Ex- perimental results show that LuWen outper- forms several strong baselines, demonstrating the effectiveness of our approach in adapting general-purpose language models to the legal domain. 1 Introduction Large language models (LLMs), especially those based on the Transformer (Vaswani et al., 2017) architecture such as ChatGPT and Gemini, utilize deep learning techniques to excel in the compre- hension and generation of human language, often containing billions or even hundreds of billions of parameters. By pre-training on vast amounts of text data collected from the internet, these mod- els acquire capabilities in recognizing complex language patterns, understanding sentence struc- ture, and grasping contextual meanings (Wu et al., 2023b). They perform tasks such as text generation, reading comprehension, dialogue, translation, and * Corresponding author. summarization, and have found widespread use in applications like automatic article writing, code generation, and customer support (Chang et al., 2023). By comprehending context and generating coherent text, these models provide powerful sup- port across fields such as education and media. As a significant branch of natural language process- ing, they also demonstrate vast potential in pro- cessing legal texts, offering promising applications for courts, law firms, academic institutions, and research. However, applying these general-purpose models in the legal domain presents a range of unique challenges. First, legal texts contain extensive specialized ter- minology and highly structured language, requiring models not only to understand standard language but also to grasp domain-specific legal knowledge. Second, legal tasks are explicitly defined and often based on complex legal principles and precedents that require clear reasoning, demanding enhanced task comprehension and logical inference capabil- ities from the models. Furthermore, continuous updating of legal rules and knowledge challenges models in terms of accuracy and timeliness of in- formation. These challenges illustrate that, despite the broad success of large models across various domains, specific optimizations and adaptations are necessary for their effective application in legal contexts. To address these challenges, we have developed an open-source Chinese legal language model, Wis- domInterrogatory (LuWen) 1 , built on the founda- tion of a general large language model. Our techni- cal approach consists of three main components: •Continual Pre-Training (CPT): We perform CPT on a specialized legal corpus to en- hance the model’s ability to understand legal- specific language patterns and knowledge 1 https://github.com/zhihaiLLM/ wisdomInterrogatory arXiv:2604.06737v2 [cs.CL] 10 Apr 2026 structures. • Supervised Fine-Tuning (SFT): We employ SFT, training the model on specific legal tasks to enable it to understand distinct legal re- quirements and provide more precise and rea- soned responses—significantly improving the model’s performance in targeted legal appli- cations. •Retrieval Augmented Generation (RAG): Finally, we utilize the RAG method to link the model with legal databases, enabling it to access and reference continually updated le- gal information and case law. This approach ensures that even amidst rapid changes in the legal knowledge landscape, the model main- tains accuracy and timeliness in its informa- tion output. In summary, this technical approach not only enhances the model’s expertise in the legal domain but also ensures the real-time relevance and relia- bility of its outputs. 2 Method As shown in Figure 1, the technical approach of the LuWen model is divided into three main components: continual pre-training, instruction fine-tuning/supervised fine-tuning, and retrieval- augmented generation. This section will elaborate on each of these components. 2.1 Continual Pre-Training, CPT In the legal domain, where data are character- ized by high specialization and complex semantics, foundational models trained on general datasets often fall short of achieving optimal performance. This limitation stems from the model’s insufficient capabilities in semantic understanding and lan- guage generation within specialized legal contexts. To address this issue, we implemented a strategy of continual pre-training to enhance the model’s comprehension and handling of professional legal texts through further training. The continual pre-training process is technically similar to the initial pre-training, based on a self- supervised learning framework. The model max- imizes the conditional probability to predict the next word following a given preceding sequence by optimizing the following objective function to adjust the model parameters, θ: θ = arg max θ T X t=1 logp(w t |x <t ;θ)(1) whereθdenotes the model parameters,Trep- resents the length of the text sequence,x <t is the preceding word sequence, andw t is the target word following the sequence. In our implementation, we first compile a large Chinese legal corpus encompassing various types of texts, such as legal books, academic papers, statutes, court rulings and contracts, sourced from legal databases, judicial documents and law jour- nals. To ensure the quality of the corpus, we com- pleted an initial filtering and cleaning by removing duplicate, erroneous, and irrelevant data. For LuWen, we chose the Baichuan1-7B (Yang et al., 2023) model as the foundation for continual pre-training due to its outstanding capabilities in handling Chinese. We conducted the continual pre- training using a comprehensive corpus containing 200 GB of data, with 20% comprising Chinese legal texts and 80% general texts. This ratio was chosen to ensure that while the model acquires legal knowledge, it also retains general capabilities. It is noteworthy that, in continual pre-training, all data are shuffled and concatenated directly without reformatting into a specific data structure. 2.2 Supervised Fine-Tuning, SFT The supervised fine-tuning process, also referred to as instruction tuning, aims to train the model to more accurately comprehend human intent in input prompts, thereby enhancing its capability to perform specific tasks. Compared to the prior CPT phase, the data used in the instruction tuning stage adopts a more standardized format, specifically in the form of instruction-response pairs. Each in- struction is prefixed with ’Human:’, and each re- sponse with ’Assistant:’, with the goal of guiding the model more effectively into a conversational mode rather than simply remaining in a text com- pletion phase. Research has consistently shown that during the instruction tuning phase, the quality of data is sig- nificantly more important than the quantity. Follow- ing this principle, we have rigorously controlled the scale of the instruction dataset. Specifically, as shown in Table 1, we used a total of 100,000 instruction tuning data points, with data from the legal domain accounting for 30%. This legal data primarily covers areas such as judicial consultation, Legal Knowledge Data 3189 Statutes 50504 Regulations Round 1: "instruction": "This court ascertained that the defendant drove after drinking... Please analyze the case." "output": "Based on the above, the core elements of this case include drunk driving, causing injury, and fleeing the scene..." Round 2: "instruction": "Based on the above analysis, please predict the charges." "output": "The predicted charge for this case is dangerous driving." Round 3: "instruction": "Please provide a sentencing opinion." "output": "Considering the defendant's flight, a sentence of three months' detention and a fine of 6000 yuan is recommended." Instruction Tuning Data Instruction Tuning Luwen Foundation Model Criminal Law of the People's Republic of China Civil Code of the People's Republic of China ... Knowledge Base If I drive after drinking two jin of baijiu, what will happen? Known Information: Dangerous driving refers to driving under the influence of alcohol... Based on the above information, answer the user 's question. Question: What are the consequences of driving after drinking two jin of baijiu? Driving after drinking two jin of baijiu may lead to the following legal consequences: 1. Dangerous driving charge 2. Revocation of driver's license and vehicle registration 3. Fine 4. ... In conclusion, do not drink and drive. Supreme People's Court, Supreme People's Procuratorate, and Ministry of Public Security Opinionson Several Issues Concerning the Applicationof Law in Handling Criminal Casesof Drunk Driving... Knowledge Injection Multiple Judicial Sub-tasks Legal Corpus CaseLaw Library Task Learning Retrieval Augmentation Continual Pre-Training Legal Materials Figure 1: LuWen Technology Roadmap. Table 1: Composition of SFT Data Used in Model Training DatasetData DescriptionVolumeSource Legal Domain Legal ConsultationQ&A in legal consultation contexts7k Public/ Private Legal ExaminationQ&A from legal examination settings6k Private Scenario-based Legal Q&ALegal Q&A based on specific scenarios4k Public Legal Reading ComprehensionQ&A tasks for legal reading comprehension4k Private Case Summary ExtractionExtraction of abstract information from legal cases3k Private Court Opinion GenerationGeneration of court opinions based on case data3k Private Multi-round Legal ConsultationMulti-round Q&A in legal consultations0.8k Private Charge PredictionCase data with corresponding charges0.5k Private Legal Provision PredictionCase data with corresponding violated provisions0.5k Private Judgment PredictionCase data with corresponding final judgments0.5k Private Monetary Amount ExtractionCase data with total involved amounts0.2k Private OthersMiscellaneous legal data such as document templates0.5k Private Total30k General Domain Open-Orca (GPT-4)Stratified sample from the Open-Orca GPT-4 dataset20k Public MOSS (Chinese Section)Multi-round dialogue data from bilingual conversations15k Public ShareGPTConversations including user inputs and ChatGPT responses12.5k Public BelleChinese instruction data covering 13 categories10k Public C3 Chinese reading comprehension dataset with multiple-choice questions 6k Public PuffinMulti-round dialogues with GPT-4 and fine-tuned responses3k Public School MathChinese math problems with solutions2k Public LIMADataset with curated prompts and responses1k Public Self-awareness & GreetingsModel’s preset information and greetings0.5k Private Total70k legal scenario-based Q&A, legal violations and charge prediction, sentencing prediction, legal ex- amination questions, court opinions, case summary extraction, and legal reading comprehension. The remaining 70% consists of general-purpose data, including various multilingual datasets in Chinese and English, covering a range of natural language processing tasks. This distribution is intended to ensure that the model retains general capabilities while learning new, domain-specific instructions. To obtain the instruction dataset, we sourced it from two primary avenues: publicly available Q&A datasets and our self-collected legal texts. For Q&A datasets, we implemented a stringent data filtering process; for legal texts, we converted the narrative format to Q&A format. Data Filtering. The primary challenges in data filtering were the vast volume and inconsistent qual- ity of publicly available Q&A datasets. To ensure high-quality Q&A data, the filtering process was conducted from both the instruction and response perspectives. From the instruction perspective, we focused on three core elements: balancing capabilities, sample diversity, and conversational complexity. To achieve capability balance—especially when handling large datasets like the legal consultation dataset—we applied under-sampling to reduce its proportion in the dataset. Sample diversity was achieved by selecting representative samples within each dataset. We utilized clustering algorithms such as K-Means to categorize the dataset into multiple classes, then performed random sampling within each class to ensure broad coverage. Addi- tionally, to enhance the model’s reasoning ability, we retained multi-turn conversation data, as these often showcase complex logical transitions. From the response perspective, we first con- structed a sensitive word dictionary to filter out responses that might contain harmful information. Given the constant updates to legal information, we adapted to changes in legal provisions through content replacement. For example, with the imple- mentation of the PRC Civil Code, many old legal terms were updated, so we built a mapping dictio- nary to convert old legal terms to their new usages. Furthermore, to improve user interaction, we in- cluded Q&A that featured self-awareness, allowing the model to briefly introduce itself and help users better understand its application scenarios. Data Construction. In terms of data construc- tion, we primarily dealt with self-collected le- gal texts. Although these texts are of relatively high quality, they are mostly in narrative form rather than question-answer format. Therefore, we needed to abstract these texts based on task require- ments and reconstruct them to generate a data for- mat suitable for instruction tuning. For example, in the task of charge prediction within judgment docu- ments, we first extracted the statement of facts and the judgment charges. Then, we divided the work into manual and automated construction phases to complete the dataset. During the manual construction phase, the main tasks included defining representative seed instruc- tions and clarifying response steps. Seed instruc- tions refer to typical prompts that reflect task re- quirements. For instance, in the charge prediction task, "What charges did the defendant face in this case?" serves as a seed instruction. The response steps were defined to ensure that responses are not merely brief labels but structured explanations. In the charge prediction task, our response steps were: 1. Summarize and abstract the case; 2. List relevant legal provisions; 3. State the charges faced by the defendant. In the automated construction phase, we used ChatGPT to help expand the seed instructions into over 100 question templates, preserving their origi- nal intent to enhance instruction diversity. We then provided ChatGPT with the case facts description, judgment charges, and response steps to guide it in generating high-quality answers. Finally, by merging case facts with the ques- tion templates, we generated a series of instruction prompts. These prompts, along with structured responses, formed our meticulously constructed instruction-response dataset. Examples are shown in Figure 2. Throughout the data construction process, we successfully created over ten types of legal tasks covering various dimensions such as legal provi- sion prediction, charge prediction, sentencing pre- diction, monetary amount extraction, court opinion generation, knowledge base reading, and choice questions for legal examinations. These tasks not only cover multiple key aspects of the legal domain but also significantly enhance the model’s ability to handle a wide range of legal-related issues. 2.3 Retrieval Augmented Generation, RAG Retrieval augmented generation is a method that incorporates relevant knowledge before the model provides answers, helping the model respond more accurately. Specifically, this technology transforms the model from a "closed-book" state to an "open- book" state, significantly improving the quality and accuracy of responses. The main advantages of this method include: first, introducing external knowl- edge to the model effectively reduces the likelihood of generating incorrect information; second, given the rapid evolution of legal knowledge and the high Instruction Response Human:TheAnxiCountyPeople's ProcuratorateaccusesthatsinceFebruary 2016,defendantChenXpurchasedfivebank cardsandfourIDcardsofothersfor4,400 RMBviaWeChatfrom“DuNiao.”On March15,2016,ChenXwasapprehendedon thespotaftercollectingthepackageatthe ShunfengExpressstoreinMeixiVillage, HutouTown,AnxiCounty.Upon investigation,itwasfoundthatthreeofthe bankcardswereobtainedbyusingfalse identificationdocumentstodeceivethe banks.WhenChenXwasapprehended,items relatedtothecase,includingbankcards,ID cards,abankU-shield,SIMcards,bank accountopeningdocuments,expressreceipts, andamobilephone,wereseized.What crimesdidthedefendantcommitinthiscase? Assistant:Basedontheprovidedinformation, defendantChenX,bypurchasingothers' bankcardsandIDcardsandusingfalse identificationtofraudulentlyobtainbank cards,hasallegedlyviolatedArticle177-1of theCriminalLawofthePeople’sRepublicof China,whichmayconstitutethecrimeof obstructingcreditcardmanagement. Figure2:AnExampleofConstructedInstruction-ResponseData Figure3:Multi-Source,Multi-PathLegalKnowledgeRetrieval Figure 2: An Example of Constructed Instruction-Response Data cost of retraining the model to incorporate these updates, connecting to an external knowledge base allows the model to stay current with knowledge updates, which is especially important in the legal field. In this context, we primarily constructed a le- gal knowledge base and proposed a multi-source, multi-path legal knowledge retrieval process. Construction of Legal Knowledge Base. The construction of the legal knowledge base involves several stages, beginning with knowledge collec- tion, which includes: •Law Library: Encompassing the constitu- tion, various types of laws (such as civil and commercial law, criminal law), judicial in- terpretations, regional regulations, oversight regulations, and administrative laws. •Legal Document Template Library: Cover- ing templates for complaints, appeals, court judgments, and contracts, as well as other commonly used legal documents. •Legal Textbook Library: Including text- books on civil law, international private law, and environmental resource law, among other fields. • Case Library: Consisting of numerous cases in criminal and civil law. • National Unified Legal Professional Quali- fication Examination Library: Containing exam questions, options, explanations, and answers from 2016 to 2020. •Legal Q&A Library: Collecting common le- gal questions and answers from various fields. The next step involves storing the knowledge. To optimize the retrieval model’s performance across different knowledge bases, we standardized the knowledge format. Each piece of knowledge is stored as a key-value pair, where the key is a brief description of the knowledge point to facilitate re- trieval, and the value contains the detailed content for the model’s reference. This format not only enables a decoupling between knowledge retrieval and augmentation but also improves the model’s efficiency in matching knowledge and ensures the accuracy of the content. Keyword-Based Search (Statistical Features) Law Library Case Library Template Library Article 2 Article 1 Article n Vector-Based Retrieval (Semantic Features) Law Knowledge Template Knowledge ... ... ... Knowledge Base Query Knowledge-Augmented Input Knowledge FusionIntent RecognitionKnowledge Retrieval Keywords, Content Selection Keywords, Content Selection Figure 3: Multi-Source, Multi-Path Legal Knowledge Retrieval Multi-source, Multi-path Legal Knowledge Retrieval. Legal knowledge retrieval involves re- trieving relevant knowledge based on a given in- struction. Considering the diversity of legal tasks, as shown in Figure 3, we propose a multi-source, multi-path legal knowledge retrieval process, which includes the following steps: 1. Intent Recognition: Since we have multi- source knowledge bases, not all of them will be relevant to the current instruction. There- fore, the first step is matching at the knowl- edge base level, known as intent recognition. Specifically, we match keywords in the ques- tion with those in the knowledge bases to identify the type of knowledge involved in the question. This enables us to selectively use the appropriate knowledge base to assist, thereby reducing confusion. 2.Knowledge Retrieval: This process employs both retrieval based on statistical features and retrieval based on semantic features. Statisti- cal feature retrieval focuses on the match be- tween keywords in the knowledge base entries and keywords in the query, while semantic feature retrieval uses a trained retrieval model to calculate the vector similarity between the input query and the knowledge keys. In prac- tice, results from statistical feature retrieval are prioritized, followed by results from se- mantic feature retrieval. Additionally, we set a similarity threshold for semantic retrieval, returning only results that meet the threshold. 3.Semantic Retrieval Model Training: For the semantic retrieval model, we conduct addi- tional training using contrastive learning. We select actual legal statutes, similar cases, and relevant book descriptions as positive exam- ples, while negative examples are chosen ei- ther randomly or based on high similarity with mismatched content. 4. Knowledge Fusion: Based on the multi-source knowledge retrieved, we fuse this information and provide it to the large legal model. For instance, when querying the judgment of a specific case, if relevant knowledge in the law library and the case library is retrieved, a spe- cialized instruction containing these knowl- edge points is constructed and input into the model to assist it in making a more accurate judgment. This approach of integrating multiple knowledge sources not only improves the model’s response quality but also allows it to quickly adapt to up- dates in legal knowledge, which is crucial for ap- plications in the legal domain. 3 Experiment 3.1 Introduction to Evaluation Tasks As a comprehensive model in the legal domain, LuWen is applicable in a variety of scenarios, mak- ing its evaluation a challenging task. Here, we have selected five common tasks to evaluate its perfor- mance for reference. Prediction TasksIn prediction tasks, we selected the following two tasks, with their definitions and evaluation metrics introduced as follows: •Legal Judgment Prediction is one of the most common tasks in Chinese legal proceedings. The input consists of the facts of the crime, and the output includes the legal statutes, charges, and sen- tences. The test set for this task consists of 2,490 samples, and the evaluation metric is accuracy. • Judicial Examination is one of the most chal- lenging examinations in China and plays a critical role in the career of legal professionals. Therefore, this task is based on question-answering for the judicial examination. The task requires the model to generate the correct answer given a question and a set of options. The test set for this task consists of 2,108 samples, and the evaluation metric is ex- act match, where a prediction is considered correct only if the selected option matches the correct op- tion precisely. Generation Tasks In generation tasks, we se- lected the following three tasks, with their defini- tions and evaluation metrics introduced as follows: •Legal Text Summarization involves com- pressing, condensing, and summarizing the con- tent of judicial documents, reflecting the judicial process, facts, reasoning, and legal grounds. This task requires generating a legal summary text based on the original text of the judicial document. The sample size for the test set is 50, and it uses human evaluation as the metric, focusing on the following criteria: (1) Accuracys 1 : Assesses whether the summary accurately reflects the facts, reasoning, and judicial grounds in the original judicial document. Scores range from 1 to 5, where 1 indicates major errors in most important information, and 5 indicates perfect accuracy. (2) Completenesss 2 : Assesses whether the sum- mary comprehensively includes all key information about the case. Scores range from 1 to 5, where 1 represents minimal inclusion of key information, and 5 indicates highly comprehensive information. (3) Readabilitys 3 : Assesses whether the lan- guage of the summary is fluent and its logical struc- ture is clear. Scores range from 1 to 5, where 1 indicates the summary is almost unreadable, and 5 denotes excellent fluency and logical clarity. The calculation formula for the evaluation score is as follows: Q 1 = (s 1 + s 2 + s 3 )/(3× 5)(2) wheres 1 ,s 2 , ands 3 are the scores for accuracy, completeness, and readability on a scale of 5. •Law Article Question Answering involves recommending relevant legal articles and providing pertinent legal interpretations based on legal ques- tions posed by users and descriptions of case facts. It uses human evaluation as the metric, focusing on the following aspects, with a sample size of 50: (1) Relevance of Mentioned Articless 4 : Mea- sures the relevance of recommended legal articles to the user’s query, scored from 1 to 5, where 1 denotes almost no relevance, and 5 signifies high relevance. (2) Accuracy in Application of Articless 5 : Evaluates the accuracy of the legal interpretations provided in the context of a specific case, scored from 1 to 5, where 1 indicates a serious mismatch with case facts or legal articles, and 5 denotes full alignment. The calculation formula for the evaluation score is as follows: Q 2 = (s 4 + s 5 )/(2× 5)(3) wheres 4 ands 5 are the scores for relevance and accuracy of application on a scale of 5. •Judicial Decision Reasoning involves rea- soning and making decisions based on case facts. This task considers facts investigated by the court, contracts signed between the plaintiff and the de- fendant, the plaintiff’s claims, and other evidence. During this process, it is necessary to determine applicable legal grounds, identify contentious focal points, and ultimately provide specific judgment outcomes. The sample size of the test set is 40, and the evaluation criteria include completeness, relevance, and correctness: (1) Completenesss 6 : Measures the system’s ability to capture case facts and relevant legal arti- cles, scored from 1 to 5, where 1 indicates captur- ing only minimal facts and articles, and 5 indicates a comprehensive understanding of all aspects of the case. (2) Relevances 7 : Measures the system’s accu- racy in identifying and applying legal articles and grounds, scored from 1 to 5, where 1 indicates identified legal articles are almost irrelevant to the case, and 5 indicates the articles and grounds most relevant to the case are accurately identified. (3) Correctnesss 8 : Assesses the accuracy of the system’s final judgment, scored from 1 to 5, where 1 means decisions seriously contradict actual legal provisions and precedent judgments, and 5 means full congruence. The calculation formula for the evaluation score is as follows: Q 3 = (s 6 + s 7 + s 8 )/(3× 5)(4) wheres 6 ,s 7 , ands 8 are the scores for complete- ness, relevance, and correctness on a scale of 5. 3.2 Baseline Models For the prediction tasks, we adopted Baichuan2- Chat (Yang et al., 2023), GLM2 (Zeng et al., 2023), Qwen (Bai et al., 2023), and InternLM2 (Team, 2023) as baseline models, all of which are multi- lingual large language models featuring extended context lengths, superior multi-turn dialogue ca- pabilities, and tool utilization abilities. GPT-3.5 (Ouyang et al., 2022) is a large-scale language model proposed by OpenAI that achieves opti- mal performance on various natural language tasks. The specific version we used is GPT-3.5-Turbo- 0613. For the generation tasks, we used the following methods as baseline models: Baichuan+SFT is based on Baichuan with super- vised fine-tuning on legal-domain data. GPT-3.5 (Ouyang et al., 2022) serves as a baseline model, the same as that in the prediction tasks. 3.3 Evaluation Results Results of Prediction Tasks From Table 2, we have the following observations: (1) Our model achieves the best results in predicting legal statutes and charges for legal judgment prediction and sig- nificantly outperforms most baseline models in other tasks. This result demonstrates that super- vised fine-tuning in the legal domain can markedly enhance model performance on legal tasks. (2) Generative models generally exhibit lower results on prediction tasks, indicating room for improve- ment compared to traditional prediction models. (3) Compared to Baichuan2, LuWen shows better performance across all prediction tasks, which val- idates the effectiveness of continual pre-training on the Baichuan base model. (4) Additionally, Baichuan2 scores lower in charge prediction and judicial examination because the model demon- strates evasiveness regarding relevant issues, failing to provide clear responses. Results of Generation Tasks From Table 3, we have the following observations: (1) Compared to the prediction tasks, generative language mod- els demonstrated better fluency in the generation tasks, yielding relatively improved results. (2) LuWen outperformed Baichuan+SFT across all tasks, which confirms that continual pre-training can significantly enhance a model’s capability in handling law-related tasks. (3) LuWen also dis- plays a noticeable advantage over GPT-3.5, partic- ularly in tasks such as law article question answer- ing, which demand a high level of legal knowledge. This affirms the effectiveness of our technical ap- proach. In addition, Figure 4 illustrates an example of how LuWen utilizes a retrieval database to assist in responding to user inputs. It demonstrates that LuWen can provide reasonable answers by integrat- ing the given knowledge. 4 Future Prospects Looking ahead, we aim to focus on the following di- rections to further deepen and expand the research and application scope of LuWen: Task Decomposition Capability Due to the in- herent complexity of legal tasks, a single-step question-and-answer approach often fails to cover the necessary scope of solutions. Therefore, ex- ploring methods for the model to autonomously and effectively decompose legal tasks becomes cru- cial for enhancing model performance. Integrating technologies such as agents (Wu et al., 2023a) or Chain-of-Thought (CoT) (Wei et al., 2022) repre- sents a viable technical approach. Retrieval Capability Currently, retrieval based on vector and keyword matching remains inade- quate. We plan to design a system that aims to auto- matically train retrieval models specifically tailored to particular knowledge bases, thereby improving the accuracy and efficiency of retrieval. Addition- ally, we acknowledge the model’s limitations in understanding and integrating knowledge, and plan to enhance its capability in synthesizing and out- putting integrated knowledge in future model train- ing processes. Multimodal Capability Although most legal data exists in text form, the application scenarios in the legal field extend beyond text to include audio recordings, images, videos, and other data formats. Therefore, developing the model’s multi- modal capability is of critical importance to meet the complex requirements of the legal domain. Table 2: Results of Prediction Tasks Method Legal Judgment Prediction Judicial Examination Statute Charge Sentence Baichuan2-Chat0.150.000.200.00 GLM20.220.280.440.11 Qwen0.040.030.360.09 InternLM20.110.100.010.09 GPT-3.50.220.120.420.22 LuWen0.350.310.410.18 Table 3: Results of Generation Tasks MethodLegal Text SummarizationLaw Article Question AnsweringJudicial Decision Reasoning GPT-3.567.338.042.7 Baichuan+SFT47.944.029.2 LuWen71.684.053.7 智海-录问(wisdomInterrogatory)是由浙江大学、阿里巴达摩院以及华院计算三家单位共同设计研发的法律大模型。核心思想：以“普法共享和司法效能提升”为目标，从推 动法律智能化体系入司法实践、数字化案例建设、虚拟法律咨询服务赋能等方面提供支持，形成数字化和智能化的司法基座能力。 wisdomOcean/wisdomInterrogatory（智海提供117464次访问2023-08-21更新 ） 空间文件交流反馈 复制 76 在使用ModelScope社区提供的空间体验服务（以下简称本服务）前，请您务必仔细阅读并理解透彻《用户协议与隐私政策》...... 免责声明 Use via API·Built with Gradio 意图识别 法律法条法律书籍法律文书模板 法律案例法律考试日常法律问答 可供参考的法律文书模板:[1] 民事起诉状 要点包括：涉案 当事人基本信息，法定代理人和委托代理人，诉讼请求， 事实与理由，证据及来源，法院名称、附件副本及起诉人 签名。 Powered by 浙江大学 阿里巴达摩院 华院计算 魔搭社区 免责声明：本模型仅供学术研究之目的而提供，不保证结果的准 确性、完整性或适用性。在使用模型生成的内容时，您应自行判 断其适用性，并自担风险。 粒 清除历史对话 发送 请帮我写一张起诉状，主要内容是张三2021年向李四借了20万未还，谢 民事起诉状 原告：李四(身份证号码：X) 被告：张三(身份证号码：X) 一、诉讼请求 1.要求被告立即归还原告借款人民币20万元整; 2.要求被告承担本次诉讼所产生的全部费用。 二、事实和理由 2021年，被告张三因经营需要向原告李四借款人民币20万元整，并出具了一份书面欠条作为凭证。然而，至今为止已经过去了近一年时 间，被告仍未履行还款义务。在此期间，原告多次催促被告尽快偿还借款，但均无果。 根据《中华人民共和国合同法》相关规定，债务应当清偿。因此，原告有权要求被告立即归还借款，并且承担相应的违约责任。同时，由 于被告未能及时履行还款义务给原告造成了一定的经济损失和精神压力 智海-录问 自动选择知识库 手动选择知识库 知识库检索结果 请输入问题 SiriusL517搜索你感兴趣的内容GitHub社区文档中心创空间数据集模型库首页 Figure 4: Sample Display of LuWen Added to the Retrieval Database References Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jin- gren Zhou, Xiaohuan Zhou, and Tianhang Zhu. 2023. Qwen technical report. CoRR, abs/2309.16609. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2023. A sur- vey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welin- der, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instruc- tions with human feedback. In Advances in Neural Information Processing Systems 35: Annual Confer- ence on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022. InternLM Team. 2023. Internlm: A multilingual lan- guage model with progressively enhanced capabili- ties. https://github.com/InternLM/InternLM. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models. Advances in neural information processing systems, 35:24824–24837. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. 2023a.Auto- gen: Enabling next-gen llm applications via multi- agent conversation framework. arXiv preprint arXiv:2308.08155. Tianyu Wu, Shizhu He, Jingping Liu, Siqi Sun, Kang Liu, Qing-Long Han, and Yang Tang. 2023b. A brief overview of chatgpt: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica, 10(5):1122–1136. Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, Fan Yang, Fei Deng, Feng Wang, Feng Liu, Guangwei Ai, Guosheng Dong, Haizhou Zhao, Hang Xu, Haoze Sun, Hongda Zhang, Hui Liu, Jiaming Ji, Jian Xie, Juntao Dai, Kun Fang, Lei Su, Liang Song, Lifeng Liu, Liyun Ru, Luyao Ma, Mang Wang, Mickel Liu, MingAn Lin, Nuolan Nie, Peidong Guo, Ruiyang Sun, Tao Zhang, Tianpeng Li, Tianyu Li, Wei Cheng, Weipeng Chen, Xian- grong Zeng, Xiaochuan Wang, Xiaoxi Chen, Xin Men, Xin Yu, Xuehai Pan, Yanjun Shen, Yiding Wang, Yiyu Li, Youxin Jiang, Yuchen Gao, Yu- peng Zhang, Zenan Zhou, and Zhiying Wu. 2023. Baichuan 2: Open large-scale language models. CoRR, abs/2309.10305. Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Zhiyuan Liu, Peng Zhang, Yuxiao Dong, and Jie Tang. 2023. GLM-130B: an open bilingual pre-trained model. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.