Paper deep dive
Generative Agents Navigating Digital Libraries
Saber Zerhoudi, Michael Granitzer
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 85%
Last extracted: 7/20/2026, 10:27:36 AM
Summary
The paper introduces Agent4DL, a user search behavior simulator for digital libraries that leverages Large Language Models (LLMs) to generate realistic user profiles and dynamic search sessions. It addresses privacy-related data scarcity by simulating querying, clicking, and stopping behaviors. The system is validated against real user data from EconBiz and Sowiport, demonstrating competitive performance in preference and relevance prediction tasks compared to baselines like BM25 and models trained on real data.
Entities (15)
Relation Signals (10)
Agent4DL → uses → Large Language Models
confidence 95% · Agent4DL, an innovative user simulator that utilizes LLMs to create realistic and dynamic models of user search behavior.
Agent4DL → validatesagainst → Sowiport
confidence 90% · We use both datasets [EconBiz and Sowiport] as a reliable foundation to initialize user profiles and as a baseline comparison
Agent4DL → validatesagainst → EconBiz
confidence 90% · We use both datasets [EconBiz and Sowiport] as a reliable foundation to initialize user profiles and as a baseline comparison
Agent4DL → employsmethod → Chain-of-Thought
confidence 85% · we enhance the agent’s ability for emotional reasoning via Chain-of-Thought [30].
Agent4DL → employsmethod → ReAct
confidence 85% · We use the ReAct method [34] to include reasoning and action steps.
Agent4DL → generates → Agent4DLData
confidence 85% · we offer the research community a valuable resource in the form of Agent4DLData, a compact yet comprehensive dataset of simulated user search sessions generated by Agent4DL.
Agent4DL → outperforms → SimIIR 2.0
confidence 85% · Agent4DL demonstrates competitive performance compared to existing user search simulators such as SimIIR 2.0
Agent4DL → →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In the rapidly evolving field of digital libraries, the development of large language models (LLMs) has opened up new possibilities for simulating user behavior. This innovation addresses the longstanding challenge in digital library research: the scarcity of publicly available datasets on user search patterns due to privacy concerns. In this context, we introduce Agent4DL, a user search behavior simulator specifically designed for digital library environments. Agent4DL generates realistic user profiles and dynamic search sessions that closely mimic actual search strategies, including querying, clicking, and stopping behaviors tailored to specific user profiles. Our simulator's accuracy in replicating real user interactions has been validated through comparisons with real user data. Notably, Agent4DL demonstrates competitive performance compared to existing user search simulators such as SimIIR 2.0, particularly in its ability to generate more diverse and context-aware user behaviors.
Tags
Links
- Source: https://arxiv.org/abs/2602.22529v1
- Canonical: https://arxiv.org/abs/2602.22529v1
Trouble viewing inline? Open PDF directly →
Full Text
51,244 characters extracted from source content.
Expand or collapse full text
Generative Agents Navigating Digital Libraries Saber Zerhoudi University of Passau Passau, Germany saber.zerhoudi@uni-passau.de Michael Granitzer University of Passau Passau, Germany IT:U Austria Linz, Austria michael.granitzer@uni-passau.de Abstract In the rapidly evolving field of digital libraries, the development of large language models (LLMs) has opened up new possibilities for simulating user behavior. This innovation addresses the longstand- ing challenge in digital library research: the scarcity of publicly available datasets on user search patterns due to privacy concerns. In this context, we introduce Agent4DL 1 , a user search behavior simulator specifically designed for digital library environments. Agent4DL generates realistic user profiles and dynamic search ses- sions that closely mimic actual search strategies, including querying, clicking, and stopping behaviors tailored to specific user profiles. Our simulator’s accuracy in replicating real user interactions has been validated through comparisons with real user data. Notably, Agent4DL demonstrates competitive performance compared to ex- isting user search simulators such as SimIIR 2.0, particularly in its ability to generate more diverse and context-aware user behaviors. CCS Concepts • Information systems→Digital libraries and archives; Users and interactive retrieval; Evaluation of retrieval results. Keywords User interactions, LLMs, Simulation, Digital Library 1 Introduction The digital age has transformed the way people seek information, with search engines at the forefront. Understanding user behavior in digital library searches is crucial for improving these systems to better serve user needs [7,35,39]. While insights into user behav- ior have traditionally been obtained through direct observation of real user interactions [5], these methods face challenges such as high costs, ethical concerns, and limitations due to privacy issues. Simulating user search behaviors has emerged as a promising alter- native, offering a controlled and repeatable way to study complex user interactions without the drawbacks of real-world studies [2]. The introduction of Large Language Models (LLMs) has revolu- tionized many areas of artificial intelligence, showcasing remark- able abilities in tasks that require human-like reasoning, decision- making, and language understanding [38]. The potential of LLMs in simulating user behavior in search context has recently started to be explored, driven by the need for robust and scalable simulation methods that can replicate the full range of user interactions in 1 https://github.com/padas-lab-de/icadl24-agent4dl © ACM, 2024. This is the author’s version of the work. The definitive version was published in: Proceedings of the 26th International Conference on Asia-Pacific Digital Libraries, ICADL 2024, Bandar Sunway, Malaysia, December 4–6, 2024. DOI: https://doi.org/10.1007/978-981-96-0865-2_14 search environments, from initial query formulation to complex navigational decisions and interactions with search results [40]. In this paper, we present Agent4DL, an innovative user simulator that utilizes LLMs to create realistic and dynamic models of user search behavior. It incorporates novel components such as building detailed user profiles with static and dynamic attributes to reflect the diversity of users in digital libraries. These profiles generate diverse and realistic behaviors essential for understanding different user strategies in information seeking. Agent4DL also integrates these profiles to dynamically generate user interactions based on the evolving context of a search session, including queries, clicks, and the underlying cognitive processes like evaluating document relevance and deciding when to end a search session. Agent4DL was compared with real user data and existing simula- tion methods to validate its effectiveness. The results demonstrate that our model replicates user behaviors with high fidelity, outper- forms traditional simulation models in capturing the nuanced ways users interact with digital libraries, and provides new insights into the complex dynamics of digital library search interactions. This study presents three main contributions to the field of digital libraries and user behavior simulation. First, we introduce Agent4DL, an innovative user simulator that employs Large Lan- guage Models (LLMs) to replicate user behavior in digital library search environments. Second, we validate the effectiveness of Agent4DL by comparing its performance against established bench- marks. Finally, we offer the research community a valuable resource in the form of Agent4DLData, a compact yet comprehensive dataset of simulated user search sessions generated by Agent4DL. This dataset is designed to facilitate and support ongoing research in the domain of user behavior simulation. 2 Related Work The field of information retrieval has been greatly influenced by research on understanding and simulating search behavior. Tra- ditional studies have focused on user interactions within search sessions, examining the evolution of user intents and predicting their actions over time. Key contributions include session search and click models [6,14,15], which analyze query-click data patterns to gain insights into user behavior [4,11,36]. These models have advanced our understanding of user search behavior, resulting in more user-centric digital libraries. Concurrently, the domain of user simulation has emerged as an essential field of study. Early methods focused on simple query generation and heuristic-based stopping behaviors in search tasks [27,28]. However, recent advancements have utilized Large Language Models (LLMs) to develop more dynamic and realistic arXiv:2602.22529v1 [cs.IR] 26 Feb 2026 S. Zerhoudi et al. simulations of user behavior in various contexts, such as web search, reasoning, and social interaction [25, 32, 34, 38]. Our research on Agent4DL contributes to this growing field by utilizing these advanced capabilities to simulate user search behavior in digital libraries. By integrating recent developments in LLM technology and user behavior analysis, Agent4DL aims to provide a robust user simulator for mimicking realistic, diverse user behaviors at scale. This approach addresses both the dynamic nature of user interactions and the evolving complexities of academic search tasks. Our work builds upon established models of user behavior and extends them through the innovative use of LLMs, creating a sophisticated simulation environment that reflects the variability of real-world user behavior. 3 Agent4DL Agent4DL, as a user search behavior simulator in digital libraries, is expected to accurately model user interactions, forecast long- term research preferences, and systematically evaluate search algo- rithms by leveraging the capabilities of LLM-empowered generative agents. To achieve this objective, two core aspects are addressed: (1) designing agent architectures that faithfully mimic academic user preferences and research-oriented cognitive reasoning, and (2) constructing a digital library environment that ensures reliabil- ity, extensibility, and adaptability across various academic search scenarios. Task Formulation. User search behavior simulation using LLM- based agents is central to our research. Agents inAgent4DLmust maintain awareness of their state in the search scenario, formu- late queries, assimilate returned information, and click on relevant documents. They also need to assess whether their information needs are met to determine if additional searches are required or if the session can be concluded. We use the ReAct method [34] to include reasoning and action steps. In each round푡, the context from previous rounds,퐶 푡 =(푟 1 ,푞 1 ,표 1 , . . .,푟 푡−1 ,푞 푡−1 ,표 푡−1 ), is used, where표is observed information and푟is reasoning. Agent4DL gen- erates reasoning푟 푡 using a task-specific prompt and updates the context. Prompts guide the LLM to perform reasoning tasks, gen- erating stops, queries, clicks, and observations. The agent decides whether to end the search based on this reasoning. If continuing, it generates a new query based on its profile and history, updates the context and interaction sequence, receives search results, selects relevant ones based on various factors, and updates the interaction sequence. By reading the clicked documents, the agent gains new observations and updates the context for the next reasoning round. For different prompting approaches, we create templates to ensure stable and controlled agent outputs. 3.1 Simulation Agent Architecture Agent4DL’s generative agents, built on LLM architecture, feature three specialized modules: profile, memory, and interaction. The user profile module embodies academic traits and research interests, while the memory and interaction modules, inspired by human cog- nition [12], enable agents to store, recall, and use past experiences. This design allows the agents to exhibit consistent and personalized behaviors, enhancing their functionality in digital library contexts. 3.1.1Profile Module. In the domain of digital library search simu- lation, the user profile module is crucial for aligning agents with real academic user behaviors. To establish a reliable foundation, benchmark datasets (e.g., Pubmed [10], Sowiport [22]) are used for initialization. Each agent’s profile contains two components: academic traits and research interests. Academic traits encompass four key characteristics capturing an individual’s research behavior in digital library scenarios: depth, breadth, recency bias, and interdisciplinarity [8, 9, 23]. Given a user푢 ∈ Uand a document푑 ∈ D, let푖 푢푑 =1 denote that user푢has interacted with document푑(e.g., viewed, down- loaded, or cited). Conversely,푖 푢푑 =0 indicates that the user has not interacted with the document. Depth quantifies the thoroughness of a user’s engagement with search results, distinguishing between users who extensively ex- amine documents and those who skim quickly. The depth trait for user푢 is defined as 푇 푢 depth ≜ 1 Í 푑∈퐷 푖 푢푑 ∑︁ 푑∈퐷 푖 푢푑 푡 푢푑 , where푡 푢푑 denotes the time spent on document푑and푖 푢푑 ∈ 0,1 indicates whether푢 interacted with 푑 . Breadth reflects the user’s tendency to explore diverse topics versus focusing on specific areas. For user푢, the breadth trait is 푇 푢 breadth ≜ Ø 푑∈퐷 :푖 푢푑 =1 T 푑 , whereT 푑 is the set of topics associated with document 푑 . Recency bias indicates the user’s preference for newer publica- tions, formulated as 푇 푢 recency ≜ 1 Í 푑∈퐷 푖 푢푑 ∑︁ 푑∈퐷 푖 푢푑 푌 current −푌 푑 , where 푌 푑 is the publication year of document 푑 . Interdisciplinarity measures the user’s inclination to explore across different academic disciplines and is defined as 푇 푢 interdis ≜ Ø 푑∈퐷 :푖 푢푑 =1 F 푑 , whereF 푑 is the set of fields associated with document 푑 . To capture these nuanced differences, we segment users into three uneven tiers for each trait: for depth (deep divers, moderate readers, quick scanners), breadth (generalists, focused researchers, specialists), recency bias (cutting-edge seekers, balanced timelines, historical researchers), and interdisciplinarity (cross-disciplinary explorers, multi-disciplinary researchers, discipline-focused schol- ars). To encode users’ research interests in natural language, we randomly select 10 documents from their interaction history. Lever- aging ChatGPT (gpt-3.5-turbo -0125), we then distill and summa- rize the unique research interests and searching patterns the user exhibited. 3.1.2Memory Module. Agent4DL’s Memory Module equips each generative agent with a specialized memory that records both facts and emotions, addressing a crucial aspect of human-like informa- tion seeking behavior. This design is motivated by the need to mon- itor emotional states, particularly frustration with complex queries Generative Agents Navigating Digital Libraries and satisfaction with search results, which play important roles in shaping search experiences and decision-making processes [25]. Factual memories in our model record concrete search behaviors, such as queries used, documents viewed, and citations made. Com- plementing these, emotional memories capture the psychological responses to search interactions, including satisfaction with results or frustration with complex queries. This dual approach aligns with cognitive research highlighting the influence of emotions on decision-making and personal history formation [17]. Our memory module implements three key operations: –Memory Retrieval: Enables agents to recall relevant past searches and successful query strategies. –Memory Writing: Records search interactions and associated emotions. –Memory Reflection: Incorporates an emotion-driven self- reflection mechanism for agents to evaluate their satisfaction with search results and assess information overload levels. 3.1.3Interaction Module. Incorporating user profiles and memory modules into agent systems allows them to display a range of be- haviors similar to humans, based on their current observations [33]. This approach enhances the agents’ ability to respond dynami- cally to different situations, mirroring human-like adaptability and decision-making processes. In Agent4DL, we design an interaction module specifically tai- lored for digital library search, which encompasses two broad cate- gories of actions: –Search-driven Actions: formulate queries, explore results, in- teract with documents (view abstracts, download full texts, save citations), and refine queries. Guided by their research interests and academic traits, agents assess each search result for relevance and decide which documents to examine further. –Emotion-driven Actions: evaluate search experience, provide feedback, and potentially abandon search sessions. To bet- ter simulate this multifaceted decision-making, we enhance the agent’s ability for emotional reasoning via Chain-of- Thought [30]. The agent autonomously expresses its satis- faction with search results and frustration levels with complex queries. Drawing upon these insights, combined with its per- sonalized depth trait, the agent decides whether to continue searching or exit the system. 3.2 Digital Library Search Environment Agent4DL simulates the interactions between agents and the digital library search environment, focusing on two key aspects: document relevance generation and search result presentation. These elements are designed to reflect real-world academic search scenarios. –Document Relevance Generation: Relevance is deduced from historical user interactions while topics and summary are generated by LLM. Our goal goes beyond encapsulating the uniqueness of the document in a profile to simulate the search scene of real academic users. We also aim to test whether LLM has potential hallucinations regarding the document. Our ap- proach utilizes a few-shot learning approach, tasking the LLM with classifying the document into one of 20 academic disci- plines and generating an abstract using only the document title. If the LLM’s discipline classification aligns with the metadata, it signifies its knowledge of the document. To maintain relia- bility, documents causing discipline misclassification by the LLM are pruned, reducing hallucination risks. This approach ensures the agent’s trustworthiness in simulating academic search behavior. – Search Result Presentation: Our simulator mirrors the opera- tion of real-world digital library platforms like Google Scholar, ACM Digital Library, and JSTOR, functioning in a paginated manner. Users are initially presented with a list of document search results on each page. Based on interactions, prefer- ences, and feedback, subsequent pages could be set to refine the search results further, aiming for a more relevant user experience. The interface includes options for sorting results (e.g., by relevance, date, or citations) and applying filters (e.g., by publication type, year range, or subject area). In the current implementation, Agent4DL utilizes API end- points to interact with existing digital libraries search systems, like HathiTrust [29] or Econbiz 2 , as long as they have an API to interact with. However, we have implemented a structured base for standalone modules responsible for search algorithm designs and result presentation. This modular approach allows for easy integration of external search algorithms, making Agent4DL a ver- satile platform for comprehensive evaluations and user feedback collection in academic search contexts. 4 Simulation Agent Evaluation This study employs the specialized user simulator Agent4DL to address a crucial research question: (RQ1) To what extent can LLM- empowered generative agents accurately simulate the behaviors of real users in digital library systems? In this section, we explore the capabilities and limitations of generative agents from various perspectives, including the align- ment of user behavior (such as information-seeking patterns and consistency) and the evaluation of the ability to generate additional simulated user behavior data. 4.1 Datasets EconBiz and Sowiport, two digital libraries, provided usage data for our analysis. EconBiz, hosted by the German National Li- brary for Economics (ZBW – Zentralbibliothek für Wirtschaftswis- senschaften) 3 , specializes in economics and business studies. Sowiport 4 covers the social sciences, containing over nine mil- lion log entries, full texts, and research projects from twenty-two databases in English and German [31]. The Sowiport User Search Session Data Set (SUSS) [22] comprises 484,437 search sessions and 179,796 queries collected over one year (April 2014 to April 2015). For EconBiz, we selected session data from August to November 2020, based on the available private data provided within the framework of a collaborative work. This dataset encompasses approximately 420,000 sessions and is 15 GB in size. Additionally, EconBiz offers an API endpoint that serves 2 EconBiz API: https://api.econbiz.de 3 https://zbw.eu/de/ 4 http://w.sowiport.de S. Zerhoudi et al. as an experimental search environment for our agents to interact with. The core features of EconBiz and Sowiport during the examined period are consistent with those of other major digital libraries. This similarity suggests that the observed user behavior trends are likely generalizable to other platforms. Consequently, we use both datasets as a reliable foundation to initialize user profiles and as a baseline comparison for our simulated user search sessions. 4.2 Search Simulation 4.2.1 Motivation. The effectiveness of Information Retrieval (IR) systems heavily relies on understanding and predicting user behav- ior. However, obtaining large-scale, diverse, and up-to-date user behavior data for training and evaluation can be challenging and costly. We hypothesize that ifAgent4DLcan effectively simulate relevant search sessions in digital libraries, it will enhance the performance of advanced user behavior models on specific IR tasks. 4.2.2 Setting. The Agent4DL user simulator can be used for vari- ous IR tasks, including preference prediction and relevance predic- tion. In our experiments, we focus on comparing the effective- ness of relevance assessments generated by Agent4DL against human-created ones, using two widely adopted session search datasets: Sowiport User Search Session (SUSS) [22] and EconBiz as baselines. We train a RoBERTa-based ranking model [20] on the Agent4DL-generated data and evaluate its performance using human-annotated test sets from these datasets, which serve to evaluate our user simulator’s performance. For preference prediction, the model uses contextual informa- tion from earlier rounds of a user’s session to improve document ranking for the next search query푞 푛 . It predicts the most suitable document푑 푛 from candidate documents푑 푗 푛 푘 푗=1 . Relevance predic- tion identifies the most relevant documents to the current search query 푞 from the search results푑 푗 푘 푗=1 . We employ a RoBERTa-based ranking model [20] for our experi- ments. The model is trained for 5 epochs with a batch size of 128 and a learning rate of 5e-6. While more epochs might be necessary for full convergence, this setting provided a balance between perfor- mance and computational resources. For session search, the model input consists of a concatenation of the historical user behavior sequence, current query, and candidate document. The output is a relevance score between the document and query, considering the behavior history. For click prediction, the input is a concatenation of the query and candidate document, with the output being their relevance score. To address input size limitations, we implement a truncation strategy for long sequences, ensuring that the most recent and relevant information is preserved. This approach maintains a fixed- length input while prioritizing the most pertinent data. Our evaluation metrics include Mean Reciprocal Rank (MRR) and Normalized Discounted Cumulative Gain (nDCG). MRR measures the average reciprocal rank of the first relevant result, while nDCG assesses ranking quality by comparing the weighted relevance of results to an ideal order. Specifically, we usenDCG@1andnDCG@3. The cutoff for the MRR metric is set at the total number of retrieved documents for each query. Table 1: Results of methods trained on various datasets in real user benchmarks for two IR tasks. TaskMethod MRR nDCG@1 nDCG@3 Preference Prediction BM2534.8915.1226.03 RoBERTa (SUSS)34.1513.9126.45 RoBERTa (EconBiz)36.3217.1428.57 RoBERTa (Agent4DL) 1000 39.2720.3733.09 RoBERTa (Agent4DL) 3000 41.32 22.66 35.19 Relevance Prediction BM2532.1111.6524.21 RoBERTa (SUSS)33.1012.3625.54 RoBERTa (EconBiz)31.3916.3627.09 RoBERTa (Agent4DL) 1000 36.0216.8429.32 RoBERTa (Agent4DL) 3000 37.71 18.90 32.78 For comparison, we train RoBERTa-based ranking models on two widely adopted session search datasets: Sowiport User Search Ses- sion (SUSS) [22] and EconBiz. These datasets serve as benchmarks to evaluate the performance of our user simulator. We employ a stratified random sampling approach for data split- ting, ensuring each user is represented in the training (70%), valida- tion (15%), and test (15%) sets. This strategy maintains consistency across different data subsets and allows for a fair comparison be- tween the Agent4DL-generated relevance assessments and human- created ones using the same human-annotated test set. 4.2.3 Results. Table 1 shows the results of various methods us- ing real user behavior data. Models trained on user behavior data generated by Agent4DL outperform baselines in preference and relevance prediction tasks, even with only 1000 training sessions (Agent4DL) 1000 . This demonstrates Agent4DL’s ability to accurately simulate user search behavior. Interestingly, the classic BM25 model [18] remains competitive, surpassing some semantic mod- els trained on search behavior datasets. As digital libraries have improved in handling complex queries, user query formats have evolved, differing from those in older datasets. 4.3 Simulation Consistency 4.3.1Motivation. To appropriately respond to search results, gen- erative agents need to have a clear understanding of their own pref- erences and search intentions. We hypothesize that an autonomous, personalized agent, initialized from real users in the Sowiport User Search Session dataset, should maintain long-term preference co- herence and exhibit consistent search behaviors. In practice, this implies that the agent should be adept at distinguishing the items that real users favor and generating queries that align with human search objectives. 4.3.2 Setting. We use Sowiport User Search Session [22] to meet information needs of given tasks. We use ChatGPT as LLM-based agents to generate user profiles based on existing task descriptions (as described in section 3.1.1). We evaluate consistency by deconstructing behavioral sequences into query, click, and stopping behaviors. Query behavior evalu- ation includes query generation and rewriting, assessing consis- tency between LLM agents’ initial and subsequent queries with human-generated queries. Click behavior evaluation focuses on the Generative Agents Navigating Digital Libraries Table 2: Similarity between query rewriting strategies and real queries. (휏) is the overlap of terms between two queries. Standard deviations are reported in parentheses. Method휏 BLEU BERTScore Popular Selection0.63 (±0.12)0.32 (±0.08)0.76 (±0.05) Random Selection0.23 (±0.15)0.11 (±0.07)0.53 (±0.09) Discriminative Selection0.43 (±0.14)0.24 (±0.09)0.68 (±0.07) Agent4DL0.87 (±0.08) 0.40 (±0.06) 0.83 (±0.04) accuracy of LLM agents’ selections in search results compared to humans. Stopping behavior is assessed to determine how well LLM agents mimic human decision points in concluding search sessions. We use Term Overlap Rate (휏), BLEU, and BERTScore to evaluate query behaviors.휏quantifies shared keywords between paired queries using Jaccard similarity [19], indicating thematic similarity and consistent search intent. BLEU [24] measures n-gram matching similarity, while BERTScore captures contextual semantic similarity. We implement Popular, Random, and Discriminative Selection user querying strategies from Azzopardi et al. [1] as baselines. To evaluate click and stopping behaviors, we use accuracy, preci- sion@10, recall@10, and F1-score to measure how well the clicked documents and stopping points of LLM agents match human be- haviors. We compare Agent4DL to the Complex Searcher Model (CSM) inSimIIR[21], including a combination of predefined Frus- tration and Satisfaction Points as described by Kraft et al. [16], and the interaction model inSimIIR 2.0[37], which uses Markov model-based decisions for clicking and stopping. The Random selection method assigns equal sampling proba- bility to all words, while Discriminative selection assumes users consider information outside the document and the collection. Pop- ular Selection considers word weights, resulting in a higher BLEU score. 4.3.3 Results. Table 2 presents the similarity between generated and real user queries for the baselines and Agent4DL in query gen- eration, along with their standard deviations. Agent4DL achieves a 87.6% Term Overlap Rate (휏) between LLM-generated and human queries, indicating a strong alignment. The lower standard deviation (0.08) for Agent4DL’s휏score suggests more consistent performance across different queries compared to other methods. We also re- port high BERTScore using Agent4DL and Popular selection due to their emphasis on weighting popular words, suggesting thematic and semantic similarity. The standard deviations for BERTScore are relatively low across all methods, with Agent4DL showing the least variability (0.04). These findings suggest that LLM agents can generate queries closely resembling human search objectives, both with and without historical interactions, and do so with high consistency. Table 3 compares the performance of click models and Agent4DL in predicting user clicks and stopping behavior.SimIIR 2.0demon- strates better predictive performance than the Complex Searcher Model (SimIIR) due to its use of a Hidden Markov Model (HMM) trained on parts of the dataset. Although this training givesSimIIR 2.0an advantage, Agent4DL’s performance remains competitive Table 3: Performance comparison of click and stopping strate- gies using Agent4DL and baseline models. Strategy Method Acc. P@10 R@10 F1 Click Strategies SimIIR76.4243.5854.9248.59 SimIIR 2.080.29 48.6759.14 53.39 Agent4DL78.3541.71 65.5950.99 Stopping Strategies Frustration & Satisfaction71.5665.80 52.09 58.14 SimIIR 2.081.3970.1440.4651.32 Agent4DL84.14 82.9039.8953.86 without dataset-specific training. Agent4DL’s performance is com- parable to baseline models, potentially because it does not effec- tively capture position bias and uses zero-shot learning. In over 90% of cases, the documents clicked by LLM agents align with user profiles, suggesting that click behavior is influenced by homoge- nized search results and individual bias. When predicting stopping behavior, Agent4DL exhibits superior performance in Accuracy and Precision but does not surpass the baselines in Recall and F1 Score. Overall, LLM agents demonstrate a high degree of consistency with human behaviors across all dimensions of query, click, and stopping, ensuring the viability of user simulation. The competitive perfor- mance of Agent4DL, especially considering its lack of reliance on dataset-specific training, highlights its potential as a flexible and generalizable approach to user simulation in information retrieval tasks. 4.4 Simulation Expansion 4.4.1Motivation. Information Retrieval (IR) models often face chal- lenges due to limited data availability, which can hinder their per- formance and generalizability. Expanding the range and quantity of search scenarios available for training could potentially address the data scarcity issue in IR research and improve model performance, particularly in sparse data scenarios. 4.4.2 Baselines and Evaluation Metrics. Our experiments utilize the SUSS Session dataset [22], which has limitations in data volume and diversity due to privacy constraints. To address this, we employ Agent4DL to generate additional data for the preference prediction task. User profiles are constructed using the methods outlined in Section 3.1.1. These profiles are generated using ChatGPT, incor- porating academic traits and research interests derived from the original dataset. We explore various combinations of trait tiers, such as users who consistently spend extended time on documents (deep divers), explore a wide variety of topics (generalists), frequently access older publications (historical researchers), and engage with multiple related disciplines (multi-disciplinary researchers). These profile combinations are then used to simulate augmented user behaviors for training purposes. Across all experimental settings, we employ the same RoBERTa-based ranking models. 4.4.3 Results. Table 4 shows the results on SUSS Session using different augmentation methods with Agent4DL. All methods with S. Zerhoudi et al. Table 4: Results of Agent4DL-augmented methods on SUSS Session. Baseline denotes the training data of SUSS Session. Academic traits denotes the four key characteristics captur- ing a user’s research behavior in digital library scenarios: depth, breadth, recency bias, and interdisciplinarity. Training Data#Session MRR nDCG@1 nDCG@3 Baseline50059.1239.8751.76 Baseline + academic traits + research interests 100064.37 48.4556.92 Baseline + academic traits100064.0147.5857.13 Baseline + research interests100062.4744.7252.54 augmented data improve over the baseline of training on the origi- nal SUSS Session training set. This shows that Agent4DL can effec- tively address data scarcity in sparse data scenarios by augmenting the original behavioral data to improve performance. 5 Insights and Exploration Given Agent4DL’s promising simulation capabilities, we pose an- other profound research question: (RQ2) Can Agent4DL provide some insights on unresolved problems in the digital library domain? In this section, we discuss two insights drawn from our simu- lation results: replicating the information overload phenomenon and exploring the impact of open access on resource usage. We acknowledge the difficulty of RQ2, and confine our discussion here to potential paths for future exploration. 5.1 Information Overload Effect Motivation. Information overload is a pervasive challenge in digital libraries [3]. This issue emerges when information retrieval systems present users with an overwhelming amount of resources, making it difficult for them to find relevant information efficiently. Our primary goal is to assess Agent4DL’s ability to replicate the information overload phenomenon. Setting. To simulate the information overload effect, we de- signed an experiment using a digital library search engine API. We conducted four simulation rounds, each representing an increase in the volume of searchable resources. We conducted four simulation rounds, each employing different strategies to increase the volume and diversity of retrieved resources: (1)Query expansion: We progressively expanded the initial query with related terms and synonyms in each round. (2) Relaxing filters: We gradually relaxed search filters (e.g., publi- cation date range, document types) with each round. (3)Increasing results per page: We requested an increasing number of results per page (within API limits) in subsequent rounds. (4)Combining multiple topics: In later rounds, we combined queries from different but related topics to increase result diversity. In each round, the API returns results based on these modified parameters. We evaluate the information overload effect based on simulated user engagement at the individual level. Two metrics are employed: ̄ 푇 per-resource , which represents the average time spent per accessed resource, and ̄ 푁 accessed , indicating the average number of resources accessed in each simulation round. 1234 Iteration Number 0.5 1.0 1.5 2.0 2.5 3.0 T per-resource (minutes) T per-resource N accessed 3.0 3.5 4.0 4.5 5.0 5.5 6.0 N accessed Figure 1: Information Overload Effect Results. Figure 1 reveals that as the number of iterations in- creases, user engagement with individual resources tends to de- crease. Specifically, the time spent per resource, represented by ̄ 푇 per-resource , decreases, while the number of resources accessed, de- noted by ̄ 푁 accessed , increases initially but then plateaus or slightly decreases. This result hint towards Agent4DL’s capability to reflect the information overload effect, an issue commonly observed in real-world digital libraries. 5.2 Impact of Open Access on Resource Usage Motivation. Understanding the impact of open access on re- source usage is crucial for digital libraries to make informed de- cisions about their collection development and access policies. A digital library simulator can aid researchers with data collection and in addressing latent confounding issues. In light of this, a question arises: can Agent4DL be instrumental in uncovering the relation- ships between open access and resource usage in digital libraries? Setting. To understand the factors influencing resource usage in digital libraries, for each resource, we collect data on four principal variables in addition to its usage frequency simulated by agents: resource quality and relevance (sourced from the resource profile), open access status, and the number of times the resource is cited (sourced from the simulator). To probe the potential relationships within this simulated data, we employ the DirectLiNGAM algo- rithm [51]. This algorithm discovers a causal graph, i.e., a weighted directed acyclic graph (DAG) in a linear system. Results. Based on the learned causal graph in Figure 2, we can observe that resource quality and relevance are the primary factors influencing resource usage [26]. While resource quality is the most significant contributor, the relevance of resources to the user’s research interests also plays a role, aligning with real-world researcher behavior. Additionally, a feedback loop amplifying the open access advantage is observed, where open access resources Generative Agents Navigating Digital Libraries Quality Usage 0.7 Open Access Citations 0.4 0.3 0.4 Relevance 0.5 Figure 2: Impact of Open Access on Resource Usage gain increased visibility, leading to more views and citations by agents. When these highly-used items are incorporated into new training datasets, information retrieval systems tend to rank them higher in subsequent iterations, resulting in the open access cita- tion advantage[13]. These insights suggest Agent4DL’s potential for simulating complex digital library phenomena and providing valuable information for library managers and information retrieval system designers. 6 Conclusion In this study, we introduced Agent4DL, a new user simulator for simulating large-scale user search behaviors in digital libraries with LLM-based agents. We designed strategies to construct user pro- files that lead to human-like search behavior data. Experiments demonstrated Agent4DL’s effectiveness in simulating authentic and diverse user behaviors and its potential in improving infor- mation retrieval tasks while respecting user privacy, especially in sparse datasets. We also developed Agent4DLData, a compact yet comprehensive collection of simulated search user behavior data to facilitate related research. We believe that this work can provide new perspectives to investigate the search behaviors of DL users, thereby contributing to the advancement of search technologies and user experience optimization. 7 Limitations and Future work Agent4DL presents a promising approach for simulating user be- havior in digital libraries, but it has several limitations. The model relies heavily on rich metadata about academic resources, which is not uniformly available across digital libraries, limiting the ability to generate realistic queries and evaluate relevance. Additionally, the current action space of Agent4DL does not fully capture the complexity of scholarly information seeking, omitting factors like citation chaining, consultation with colleagues, or serendipitous discovery through browsing. We also observed instances where the language model struggled to accurately represent domain-specific knowledge or recent de- velopments in rapidly evolving fields, leading to the generation of anachronistic queries or failure to properly assess the relevance of cutting-edge research. To address these issues, future work should explore techniques for augmenting sparse metadata, developing privacy-preserving methods to incorporate actual usage patterns, expanding the model to incorporate additional information pathways, and fine-tuning the language model specifically for academic information seek- ing, incorporating up-to-date citation databases and field-specific ontologies to improve the accuracy and currency of simulated be- haviors. References [1]Leif Azzopardi, Maarten de Rijke, and Krisztian Balog. 2007. Building simulated queries for known-item topics: an analysis using six european languages. In SIGIR 2007: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, The Netherlands, July 23-27, 2007, Wessel Kraaij, Arjen P. de Vries, Charles L. A. Clarke, Norbert Fuhr, and Noriko Kando (Eds.). ACM, 455–462. doi:10.1145/1277741.1277820 [2]Krisztian Balog and ChengXiang Zhai. 2023. User Simulation for Evaluating Information Access Systems. CoRR abs/2306.08550 (2023). doi:10.48550/ARXIV. 2306.08550 [3]David Bawden and Lyn Robinson. 2020. Information overload: An overview. (2020). [4] Alexey Borisov, Martijn Wardenaar, Ilya Markov, and Maarten de Rijke. 2018. A Click Sequence Model for Web Search. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018, Kevyn Collins-Thompson, Qiaozhu Mei, Brian D. Davison, Yiqun Liu, and Emine Yilmaz (Eds.). ACM, 45–54. doi:10.1145/3209978. 3210004 [5] Ben Carterette, Evangelos Kanoulas, Mark M. Hall, and Paul D. Clough. 2014. Overview of the TREC 2014 Session Track. In Proceedings of The Twenty-Third Text REtrieval Conference, TREC 2014, Gaithersburg, Maryland, USA, November 19-21, 2014 (NIST Special Publication, Vol. 500-308), Ellen M. Voorhees and Angela Ellis (Eds.). National Institute of Standards and Technology (NIST). http://trec. nist.gov/pubs/trec23/papers/overview-session.pdf [6]Olivier Chapelle and Ya Zhang. 2009. A dynamic bayesian network click model for web search ranking. In Proceedings of the 18th International Conference on World Wide Web, W 2009, Madrid, Spain, April 20-24, 2009, Juan Quemada, Gonzalo León, Yoëlle S. Maarek, and Wolfgang Nejdl (Eds.). ACM, 1–10. doi:10. 1145/1526709.1526711 [7] Jia Chen, Yiqun Liu, Jiaxin Mao, Fan Zhang, Tetsuya Sakai, Weizhi Ma, Min Zhang, and Shaoping Ma. 2021. Incorporating Query Reformulating Behavior into Web Search Evaluation. In CIKM ’21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 - 5, 2021, Gianluca Demartini, Guido Zuccon, J. Shane Culpepper, Zi Huang, and Hanghang Tong (Eds.). ACM, 171–180. doi:10.1145/3459637. 3482438 [8] Andrew Collins, Dominika Tkaczyk, Akiko Aizawa, and Jöran Beel. 2018. Position Bias in Recommender Systems for Digital Libraries. In Transforming Digital Worlds - 13th International Conference, iConference 2018, Sheffield, UK, March 25-28, 2018, Proceedings (Lecture Notes in Computer Science, Vol. 10766), Gobinda Chowdhury, Julie McLeod, Valerie J. Gillet, and Peter Willett (Eds.). Springer, 335–344. doi:10.1007/978-3-319-78105-1_37 [9]Shengli Deng and Sudi Xia. 2020. Mapping the interdisciplinarity in informa- tion behavior research: a quantitative study using diversity measure and co- occurrence analysis. Scientometrics 124, 1 (2020), 489–513. doi:10.1007/S11192- 020-03465-X [10]Rezarta Islamaj Dogan, G. Craig Murray, Aurélie Névéol, and Zhiyong Lu. 2009. Understanding PubMed ® user search behavior through log analysis. Database J. Biol. Databases Curation 2009 (2009). doi:10.1093/DATABASE/BAP018 [11]Georges Dupret and Benjamin Piwowarski. 2008. A user browsing model to predict search engine click data from past observations. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2008, Singapore, July 20-24, 2008, Sung-Hyon Myaeng, Douglas W. Oard, Fabrizio Sebastiani, Tat-Seng Chua, and Mun-Kew Leong (Eds.). ACM, 331–338. doi:10.1145/1390334.1390392 [12]Paula Goodale, Paul David Clough, Samuel Fernando, Nigel Ford, and Mark Stevenson. 2014. Cognitive styles within an exploratory search system for digital libraries. J. Documentation 70, 6 (2014), 970–996. doi:10.1108/JD-03-2014-0045 [13] Yue He, Zimu Wang, Peng Cui, Hao Zou, Yafeng Zhang, Qiang Cui, and Yong Jiang. 2022. CausPref: Causal Preference Learning for Out-of-Distribution Rec- ommendation. In W ’22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25 - 29, 2022, Frédérique Laforest, Raphaël Troncy, Elena Simperl, Deepak Agarwal, Aristides Gionis, Ivan Herman, and Lionel Médini (Eds.). ACM, 410–421. doi:10.1145/3485447.3511969 [14]Christoph Hölscher and Gerhard Strube. 2000. Web search behavior of Internet experts and newbies. Comput. Networks 33, 1-6 (2000), 337–346. doi:10.1016/ S1389-1286(00)00031-1 S. Zerhoudi et al. [15]Bernard J. Jansen, Amanda Spink, Chris Blakely, and Sherry Koshman. 2007. Defining a session on Web search engines. J. Assoc. Inf. Sci. Technol. 58, 6 (2007), 862–871. doi:10.1002/ASI.20564 [16] Donald H. Kraft and T. Lee. 1979. Stopping rules and their effect on expected search length. Inf. Process. Manag. 15, 1 (1979), 47–58. doi:10.1016/0306-4573(79) 90007-4 [17] Kevin S LaBar and Roberto Cabeza. 2006. Cognitive neuroscience of emotional memory. Nature Reviews Neuroscience 7, 1 (2006), 54–64. [18] Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Frassetto Nogueira. 2021. Pyserini: A Python Toolkit for Repro- ducible Information Retrieval Research with Sparse and Dense Representations. In SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Devel- opment in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, Fernando Diaz, Chirag Shah, Torsten Suel, Pablo Castells, Rosie Jones, and Tetsuya Sakai (Eds.). ACM, 2356–2362. doi:10.1145/3404835.3463238 [19]Binsheng Liu, Nick Craswell, Xiaolu Lu, Oren Kurland, and J. Shane Culpepper. 2019. A Comparative Analysis of Human and Automatic Query Variants. In Proceedings of the 2019 ACM SIGIR International Conference on Theory of Infor- mation Retrieval, ICTIR 2019, Santa Clara, CA, USA, October 2-5, 2019, Yi Fang, Yi Zhang, James Allan, Krisztian Balog, Ben Carterette, and Jiafeng Guo (Eds.). ACM, 47–50. doi:10.1145/3341981.3344223 [20] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR abs/1907.11692 (2019). http://arxiv.org/abs/1907.11692 [21] David Maxwell and Leif Azzopardi. 2016. Simulating Interactive Information Retrieval: SimIIR: A Framework for the Simulation of Interaction. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, SIGIR 2016, Pisa, Italy, July 17-21, 2016, Raffaele Perego, Fabrizio Sebastiani, Javed A. Aslam, Ian Ruthven, and Justin Zobel (Eds.). ACM, 1141–1144. doi:10.1145/2911451.2911469 [22] Philipp Mayr. 2016. Sowiport User Search Sessions Data Set (SUSS). GESIS, Köln. Datenfile Version 1.0.0, https://doi.org/10.7802/1380. doi:10.7802/1380 [23]Chris Neuhaus, Ellen Neuhaus, Alan Asher, and Clint Wrede. 2006. The depth and breadth of Google Scholar: An empirical study. portal: Libraries and the Academy 6, 2 (2006), 127–141. [24] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA. ACL, 311–318. doi:10.3115/1073083.1073135 [25] Joon Sung Park, Joseph C. O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative Agents: Interactive Simulacra of Human Behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST 2023, San Francisco, CA, USA, 29 October 2023- 1 November 2023, Sean Follmer, Jeff Han, Jürgen Steimle, and Nathalie Henry Riche (Eds.). ACM, 2:1–2:22. doi:10.1145/3586183.3606763 [26]Judea Pearl, Madelyn Glymour, and Nicholas P Jewell. 2016. Causal inference in statistics: A primer. John Wiley & Sons. [27] Sheldon M. Ross. 1997. Simulation (2. ed.). Academic Press. [28] Jost Schatzmann and Steve J. Young. 2009. The Hidden Agenda User Simulation Model. IEEE Trans. Speech Audio Process. 17, 4 (2009), 733–747. doi:10.1109/TASL. 2008.2012071 [29]John A. Walsh, Glen Layne-Worthey, Jacob Jett, Boris Capitanu, Peter Organ- isciak, Ryan Dubnicek, and J. Stephen Downie. 2023. "The library is open!": Open data and an open API for the HathiTrust Digital Library. In Proceedings of the Computational Humanities Research Conference 2023, Paris, France, December 6-8, 2023 (CEUR Workshop Proceedings, Vol. 3558), Artjoms Sela, Fotis Jannidis, and Iza Romanowska (Eds.). CEUR-WS.org, 703–714. https://ceur-ws.org/Vol- 3558/paper7875.pdf [30]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Sys- tems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (Eds.). [31]Ryen W. White and Resa A. Roth. 2009. Exploratory Search: Beyond the Query- Response Paradigm. Morgan & Claypool Publishers. doi:10.1007/978-3-031- 02260-9 [32]Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuanjing Huan, and Tao Gui. 2023. The Rise and Potential of Large Language Model Based Agents: A Survey. CoRR abs/2309.07864 (2023). doi:10.48550/ARXIV.2309.07864 [33] Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuanjing Huang, and Tao Gui. 2023. The Rise and Potential of Large Language Model Based Agents: A Survey. CoRR abs/2309.07864 (2023). doi:10.48550/ARXIV.2309.07864 [34] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Act- ing in Language Models. In The Eleventh International Conference on Learn- ing Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net. https://openreview.net/pdf ?id=WE_vluYUL-X [35]Saber Zerhoudi and Michael Granitzer. 2024. Comparative Analysis: User Inter- actions in Public and Private Digital Libraries Datasets. In Linking Theory and Practice of Digital Libraries. Springer Nature Switzerland, Cham, 162–172. [36]Saber Zerhoudi, Michael Granitzer, Christin Seifert, and Jörg Schlötterer. 2022. Simulating User Interaction and Search Behaviour in Digital Libraries. In Pro- ceedings of the 18th Italian Research Conference on Digital Libraries, Padua, Italy, February 24-25, 2022 (hybrid event) (CEUR Workshop Proceedings, Vol. 3160), Gior- gio Maria Di Nunzio, Beatrice Portelli, Domenico Redavid, and Gianmaria Silvello (Eds.). CEUR-WS.org. https://ceur-ws.org/Vol-3160/paper8.pdf [37]Saber Zerhoudi, Sebastian Günther, Kim Plassmeier, Timo Borst, Christin Seifert, Matthias Hagen, and Michael Granitzer. 2022. The SimIIR 2.0 Framework: User Types, Markov Model-Based Interaction Simulation, and Advanced Query Gen- eration. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA, October 17-21, 2022, Mohammad Al Hasan and Li Xiong (Eds.). ACM, 4661–4666. doi:10.1145/3511808.3557711 [38] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A Survey of Large Language Models. CoRR abs/2303.18223 (2023). doi:10.48550/ARXIV.2303.18223 [39]Yujia Zhou, Zhicheng Dou, Bingzheng Wei, Ruobing Xie, and Ji-Rong Wen. 2021. Group based Personalized Search by Integrating Search Behaviour and Friend Network. In SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, Fernando Diaz, Chirag Shah, Torsten Suel, Pablo Castells, Rosie Jones, and Tetsuya Sakai (Eds.). ACM, 92–101. doi:10.1145/3404835.3462918 [40] Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Zhicheng Dou, and Ji-Rong Wen. 2023. Large Language Models for Information Retrieval: A Survey. CoRR abs/2308.07107 (2023). doi:10.48550/ ARXIV.2308.07107