Paper deep dive
SourceBench: Can AI Answers Reference Quality Web Sources?
Hexi Jin, Stephen Liu, Yuheng Li, Simran Malik, Yiying Zhang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/21/2026, 12:52:52 AM
Summary
The paper introduces SourceBench, a benchmark for evaluating the quality of web sources cited by Large Language Models (LLMs) and AI search tools. It proposes an eight-metric framework assessing content quality (relevance, accuracy, objectivity) and page-level signals (freshness, authority, clarity). The study evaluates 12 systems (including GPT-5, Google Search, and various AI search tools) across 100 queries, revealing that GPT-5 leads in source quality, while high-quality source selection can sometimes outperform advanced reasoning capabilities.
Entities (12)
Relation Signals (10)
LLM → dependson → Web Search
confidence 95% · Web search mitigates these limitations by grounding and enriching responses with external, up-to-date evidence.
SourceBench → evaluates → Web Source Quality
confidence 95% · SourceBench is a benchmark for measuring the quality of cited web sources
SourceBench → evaluates → GPT-5
confidence 95% · We evaluate eight LLMs... using SourceBench
SourceBench → evaluates → Google Search
confidence 95% · We evaluate eight LLMs, Google Search... using SourceBench
SourceBench → usesmetric → Content Relevance
confidence 95% · SourceBench uses an eight-metric framework covering content quality (content relevance...)
SourceBench → usesmetric → Factual Accuracy
confidence 95% · SourceBench uses an eight-metric framework covering content quality (...factual accuracy...)
SourceBench → usesmetric → Domain Authority
confidence 95% · SourceBench uses an eight-metric framework covering ... page-level signals (e.g., ... authority/accountability...)
GPT-5 → excelsin → Domain Authority
confidence 90% · GPT-5 excels at vetting sources for ... Domain Authority
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) increasingly answer queries by citing web sources, but existing evaluations emphasize answer correctness rather than evidence quality. We introduce SourceBench, a benchmark for measuring the quality of cited web sources across 100 real-world queries spanning informational, factual, argumentative, social, and shopping intents. SourceBench uses an eight-metric framework covering content quality (content relevance, factual accuracy, objectivity) and page-level signals (e.g., freshness, authority/accountability, clarity), and includes a human-labeled dataset with a calibrated LLM-based evaluator that matches expert judgments closely. We evaluate eight LLMs, Google Search, and three AI search tools over 3996 cited sources using SourceBench and conduct further experiments to understand the evaluation results. Overall, our work reveals four key new insights that can guide future research in the direction of GenAI and web search.
Tags
Links
- Source: https://arxiv.org/abs/2602.16942v1
- Canonical: https://arxiv.org/abs/2602.16942v1
Trouble viewing inline? Open PDF directly →
Full Text
55,636 characters extracted from source content.
Expand or collapse full text
SourceBench: Can AI Answers Reference Quality Web Sources? Hexi Jin * 1 Stephen Liu * 1 Yuheng Li * 1 Simran Malik * 1 Yiying Zhang 1 2 Abstract Large language models (LLMs) increasingly an- swer queries by citing web sources, but existing evaluations emphasize answer correctness rather than evidence quality. We introduce SourceBench, a benchmark for measuring the quality of cited web sources across 100 real-world queries span- ning informational, factual, argumentative, so- cial, and shopping intents. SourceBench uses an eight-metric framework covering content qual- ity (content relevance, factual accuracy, objectiv- ity) and page-level signals (e.g., freshness, author- ity/accountability, clarity), and includes a human- labeled dataset with a calibrated LLM-based eval- uator that matches expert judgments closely. We evaluate eight LLMs, Google Search, and three AI search tools over 3996 cited sources using SourceBench and conduct further experiments to understand the evaluation results. Overall, our work reveals four key new insights that can guide future research in the direction of GenAI and web search. 1. Introduction Large Language Models (LLMs) excel at language gener- ation but rely on static parametric memory and can hallu- cinate (Ouyang et al., 2025; Li et al., 2024). Web search mitigates these limitations by grounding and enriching re- sponses with external, up-to-date evidence. As such, it has become the de facto in many production LLM systems such as ChatGPT (OpenAI, 2024), Gemini (Gemini Team, 2023), Perplexity (Perplexity AI, 2022), and Microsoft Copi- lot (Mehdi, 2023). The integration of search in LLMs introduces a new de- pendency: the quality of sources used by LLMs directly determines the groundedness of the generated response. In high-stakes domains—such as financial analysis, medical inquiries, or legal research—users increasingly rely on cited references for verification. Recent studies indicate that the 1 University of California, San Diego 2 GenseeAI Inc.. Corre- spondence to: Yiying Zhang <yiying@ucsd.edu>. Preprint. February 20, 2026. mere presence of citations significantly increases user trust, even if the underlying sources are not rigorously checked by the user (Ding et al., 2025). This creates a vulnerability where a search-augmented LLM is subject to the “Garbage In, Garbage Out” principle; if an LLM synthesizes informa- tion from low-quality, biased, or outdated web pages, the resulting answer remains flawed or misleading. Despite the importance of web source quality, existing benchmarks and evaluators remain focused on the AI an- swer text. Benchmarks like HotpotQA (Yang et al., 2018) and HLE (Phan et al., 2025) assess the accuracy of the final answer, while retrieval benchmarks like RAGBench (Friel et al., 2024) and evaluators like RAGAS (Es et al., 2024) focus on relevance ranking rather than source credibility. None of these systematically evaluates the web sources themselves or how users experience them. In this paper, we present SourceBench, a novel benchmark to evaluate the quality of web sources referenced in AI answers. We argue that the utility of a retrieved source is not defined solely by its textual similarity to the query, but by the holistic experience it offers to the human verifier. As AI systems increasingly function as information curators, users must be able to seamlessly transition from reading a synthesized answer to auditing its provenance. A source that is factually correct but buried under intrusive ads, or one that lacks a clear publication date, imposes a high cognitive load on the user, breaking the trust loop essential for high-stakes information retrieval. Consequently, we depart from standard relevance-only met- rics to propose a comprehensive, multi-facet scoring frame- work designed from the user’s perspective. We evaluate sources across eight metrics organized into two categories. The first category assesses web sources based on their page content, including content relevance, factual accuracy, and objectivity. These metrics ensure the information is not only textually aligned with the query but is also substantively correct and free from manipulative bias. For a user, this is the baseline requirement. The second category evaluates “meta-attributes” of web sources, including freshness, orga- nization, author, and domain authority, and layout clarity. We posit that these meta-attributes are the primary drivers of user trust and experience and are as important. 1 arXiv:2602.16942v1 [cs.AI] 18 Feb 2026 SourceBench: Can AI Answers Reference Quality Web Sources? To systematically evaluate, we introduce a human-aligned automated evaluator. By fine-tuning an LLM on human- labeled samples and utilizing a split-prompt strategy for content and metadata rubrics, we achieve high-fidelity scor- ing that closely mirrors manual grading. We utilize SourceBench to conduct a comprehensive study of 3996 web sources from 12 systems in three categories: search-equipped LLMs (e.g., GPT-5 (Singh et al., 2025), Gemini-3-pro (Hassabis et al., 2025)), traditional SERP (Google), and AI-native search tools (e.g., Exa (Exa, 2026), Tavily (Tavily Inc., 2026)). Our findings reveal distinct be- havioral profiles: GPT-5 achieves the highest overall scores driven by its superior ability to vet sources for various types of accountability and factuality, while Grok-4.1 (xAI, 2025) and Gensee AI search (Gensee, 2026) excel in content rele- vance and achieve the second and third highest scores. We then conduct a set of experiments to further un- derstand the relationship between AI answer and web search. Through controlled experiments with DeepSeek models (DeepSeek-AI, 2025), we find that a non-reasoning model with a high-quality search tool outperforms a reason- ing model with lower-quality search tool, suggesting that high-quality web source selection can functionally substi- tute for, and even outperform, advanced internal reasoning steps. At the same time, reasoning is still important, as we find that the highest-performing AI systems rely on sources that significantly diverge from traditional Google Search results, indicating that complex queries require multi-step thinking that go beyond keyword-based search rankings. This paper makes the following key contributions: •A Multi-Facet Web Source Quality Framework: Eight distinct rubrics for evaluating the quality of web sources from a user perspective. • The SourceBench Dataset and Evaluator: A di- verse dataset of 100 queries and an automated, human- validated evaluation pipeline. •Comprehensive System Analysis: Evaluation of 12 systems, including eight popular LLMs, three AI-based web search tools, and the Google search. • In-depth Analysis of GenAI and Web Search Rela- tionship: A set of additional experiments that reveal new insights, guiding future researchers and practition- ers. We will make SourceBench and its evaluator publicly avail- able upon the acceptance of this paper. 2. Motivation and Related Work 2.1. Search and GenAI Pre-trained Large Language Models (LLMs) have demon- strated remarkable capabilities in reasoning and natural lan- guage generation, yet they suffer from two fundamental limitations inherent to their architecture: static parametric memory and the tendency to hallucinate. An LLM’s knowl- edge is frozen at its pre-training time, rendering it blind to any events happening afterward or any changes to the training data (e.g., a website corrects itself after the pre- training time). Furthermore, LLMs today still suffer greatly from hallucination and often generate plausible but factually incorrect responses. Web search acts as the critical enabler to transcend LLMs frozen knowledge base and possible hallucination. By in- tegrating search results into the generation process, LLMs can ground their responses in external evidence. This not only mitigates hallucinations by providing factual context but also allows the model to access up-to-date information without any retraining. As such, web searches have been in- tegrated in almost all popular LLMs like ChatGPT (OpenAI, 2024), Gemini (Gemini Team, 2023), Perplexity (Perplexity AI, 2022), and Bing Copilot (Mehdi, 2023). Meanwhile, users increasingly rely on cited references in AI answers for verification. In high-stakes domains like financial analysis, medical inquiries, or legal opinions, AI answers are only as valuable as the evidence supporting them. A search-augmented LLM follows the “Garbage In, Garbage Out” principle: if the LLM retrieves and synthe- sizes information from low-quality web pages, the resulting answer will be flawed, irrelevant, or outdated. Therefore, the quality of the web source itself is paramount. Unfortu- nately, despite the prolificacy of LLM benchmarks, there is no benchmark to directly evaluate web sources in AI answers. 2.2. Related Benchmarks and Evaluation Most existing LLM benchmarks focus on the accuracy and quality of LLM answers, such as HotpotQA (Yang et al., 2018), HLE (Phan et al., 2025), and SWEBench (Jimenez et al., 2023). These benchmarks target the evaluation of LLM generation that does not reference external sources. Frameworks like RAGAS (Retrieval-Augmented Genera- tion Assessment) (Es et al., 2024) assess the retrieved infor- mation via context precision and context recall. ARES (Au- tomated RAG Evaluation System) (Saad-Falcon et al., 2024) refines this by training specialized LLM judges on synthetic datasets to provide more reliable, confidence-aware judg- ments. Other frameworks like RAGProbe (Sivasothy et al., 2024) and RAGBench (Friel et al., 2024) focus on specific aspects like automated testing for failure modes and explain- 2 SourceBench: Can AI Answers Reference Quality Web Sources? ability. While these frameworks effectively measure if an answer is faithful to a given context, none of them evaluates the web sources referenced in the answer. Checking the quality of the citation itself requires a different approach, as we will demonstrate next. 3. SourceBench This section introduces SourceBench, the first benchmark to evaluate the quality of web sources used in AI answers. We first introduce the benchmark requests and then discuss the metrics we design for evaluating source quality. Next, we discuss how we manually label reference sources on these metrics and how we leverage the manual labeling results to design an automated evaluator. 3.1. Request Collection We compiled a diverse dataset of 100 queries by ran- domly sampling from DebateQA (Xu et al., 2024), Hot- potQA (Yang et al., 2018), Pinocchios (Hu et al., 2024), Quora Question Pairs (Quora & Kaggle, 2017), and VA- COSNLQ (Papenmeier et al., 2021). This selection in- cludes e-commerce Q&A sites like Amazon and eBay, forum posts from Quora and Reddit, and search query logs—which represent real-world user questions as seen in datasets like MS MARCO (Nguyen et al., 2016). or com- mercial query logs. This broad selection captures a wide spectrum of user intents, ranging from commercial product comparisons and purchase guidance to informational factual lookups. We summarize these queries’ type and sources in Table 1. We anticipate LLMs to answer the queries with both sum- mary text and referenced sources, expressed as URLs. Our goal of SourceBench is to evaluate the quality of the ref- erenced sources, not the summary text answer itself. As explained in Section 2.2, many benchmarks such as Hot- potQA (Yang et al., 2018) and DebateQA (Xu et al., 2024) already evaluate generated text quality, and we do not in- clude this as SourceBench’s metrics. Thus, the input to SourceBench is the user query and URLs 3.2. Multi-Facet Source Quality Metrics To evaluate the quality of AI sources, we propose a multi- facet scoring framework. To provide a well-rounded assess- ment of a source’s quality, we propose eight metrics (each with numerical score of 1 to 5) organized into two categories, as summarized in Table 2. We design these metrics from the users’ perspective, with the goal of giving higher scores to sources that users are most likely to appreciate and read. The first dimension is content quality, which evaluates the content of a source web page. We measure content quality with three metrics: Content Relevance (CR), Factual Ac- curacy (FA), and Objectivity (NE). HighCR,FA, andNE scores mean that the source content directly provides infor- mation to answer the user’s question, provides information that is verifiable, and objectively delivers the information. The second dimension is on the meta-feature of source pages, including Freshness (FR), Ownership (OA), Author Accountability (A), Domain Authority (DA), and Layout Clarity (LC). These metrics are evaluated based on metadata of sources instead of their content. They are important to users because for similar content, users trust sources that are more accountable (organization, author, and domain author- ity), recent (the last time a website is updated or maintained), and clear (clearly laid out and low ad density). Specifically, Freshness checks the last time a web page is updated or maintained (i.e., being checked by human for content still being correct). It ensures that AI answers are not based on obsolete data (e.g., deprecated code or outdated regulations). Accountability and Authority allow users to gauge the credibility of the expertise (e.g., distinguishing a peer-reviewed medical study from an anonymous forum post). Finally, Layout Clarity evaluates the “consumability” of the webpage itself. In the current web ecosystem, users are often directed to “SEO farms” saturated with ads and pop-ups. By penalizing such sources, we explicitly value the user’s ability to efficiently verify information without fighting through hostile user interface patterns. 3.3. Evaluator We build our evaluator by first collecting source URLs, then manually label a sampled set of them, and finally develop an LLM-based evaluator with the help of manual label ex- amples. 3.3.1. SOURCE COLLECTION. To establish a human reference of SourceBench scores, we collect source URLs from three distinct categories: popular LLMs, traditional Search Engine Results Pages (SERP), and AI search engines. For traditional SERP, we utilize the Google Search API to record the top search result URLs. To reflect modern AI-driven discovery, we also include sources retrieved from AI search engines such as Tavily (Tavily Inc., 2026), Exa (Exa, 2026) and Gensee (Gensee, 2026). when answering the 100 queries in SourceBench. For SERP, we call the Google search API and record the top-5 search result URLs. For each URL, we scrape the page content. These page contents form the dataset for our evaluator de- velopment. 3 SourceBench: Can AI Answers Reference Quality Web Sources? Table 1. Requests Composition in SourceBench. TypeSourceCountPrimary Intent ShoppingVA-COS NLQ (Papenmeier et al., 2021)20Product search and purchase guidance ReasoningHotpotQA (Yang et al., 2018)20Multi-hop factual research and lookups FactualityPinocchios (Hu et al., 2024)20Fact-checking and claim verification ArgumentativeDebateQA (Xu et al., 2024)20Multi-perspective subjective reasoning Social Q&AQuora (Quora & Kaggle, 2017)20Community-based forum inquiries Total100 Table 2. SourceBench Rubrics. Each metric is scored on a 1–5 scale with 5 (best) and 1 (worst) meaning described below. MetricFocusBest Score (5)Worst Score (1) CRContent relevance & problem resolution Comprehensive and directly resolves the user need; no further lookup required. Irrelevant or mismatched to the query. FA Factual accuracy & verifiability Claims are verifiable and supported by cita- tions, prioritizing primary sources. Contains false or harmful claims; unverifiable assertions that contradict known facts. NENeutrality & objectiv- ity Neutral/clinical delivery; avoids emotional ma- nipulation and does not push a single agenda. Propagandistic or manipulative tone that dis- torts judgment. FRFreshness & mainte- nance Timely and clearly maintained, dated/updated recently or follow query’s time context. Legacy/obsolete: time-sensitive info is outdated such that it becomes incorrect or misleading. A Author accountability & expertise Named author with verifiable bio/credentials and clear accountability links. Deceptive or missing identity signals (fabri- cated persona; no credible author info). OA Ownership account- ability Ownership and funding are transparent with clear organizational accountability. Ownership is hidden. No contact/organizational info; signals of shell or obfuscation. DADomain authority & institutional reputation High-authority domain with established reputa- tion and long-standing trust signals. Known scam/disinformation domain or consis- tently low-credibility reputation. LC Layout Clarity & in- tegrity Readable with minimal obstruction; ads do not interfere with accessing the main content. High ad density, injected “ghost text”/scraped fragments, or overlays that obstruct reading. 3.3.2. HUMAN LABELING We sample 45 scraped pages and manually grade them on the eight metrics in Table 2. Two graduate students grade each source based on a unified grading scheme and rubrics. Doing so largely reduces human variance and provide credi- ble foundation for building our automated evaluator. During the manual grading period, we also identify ambiguities in our metrics and improve them accordingly. 3.3.3. LLM EVALUATOR To automate the evaluation process, we employ a few-shot prompting strategy using human-verified labels as exem- plars, designing distinct system prompts for each of the eight evaluation rubrics. For input processing, we capture and utilize the full textual content of each scraped web source; our preliminary experiments demonstrated that sum- marizing or chunking the content significantly degraded scoring accuracy, necessitating the use of the entire docu- ment. To accommodate this full-text requirement without exceeding the context window, we partition the rubrics into two content-based and metadata-based, each paired with its own tailored set of few-shot examples. To choose the appropriate few-shot examples for each rubric, we use a fine- tuning process on our curated dataset of 45 human-labeled samples to ensure strict alignment with manual grading standards. Appendix B presents our full prompts. Table 3 presents the mean error of our LLM-based evaluator and human graders (out of 5). As seen, our automated scoring approach achieves high fidelity, with the model’s mean score deviations consistently remaining within 0.5 points of the human consensus across all rubrics. 4. Evaluation Results 4.1. Evaluating Systems There are three primary ways of getting answers and find- ing web sources: 1) using LLM-based answers, 2) us- ing traditional SERP-based search like Google, and 3) us- ing AI-based web search tools like Exa (Exa, 2026) and Tavily (Tavily Inc., 2026). They represent different levels of LLM/search: LLM-primary search-secondary, search- primary, and search-primary LLM-secondary. To under- 4 SourceBench: Can AI Answers Reference Quality Web Sources? Table 3. Qwen-Plus Few-shot Alignment with Human Graders (N = 45). Values represent the Mean Absolute Error (MAE) between Model scores and the average of Human Graders. Dimension CR FA NE FR OA A DA LCAvg MAE0.56 0.37 0.50 0.91 0.54 0.44 0.41 0.46 0.20 stand how well they do in finding quality sources that are valuable to users, we conduct a comprehensive evaluation of 12 systems in the three categories using SourceBench. They include GPT-5, GPT-4o, Gemini-3-Pro, Gemini-3- flash, Gemini-2.5-flash, Grok-4.1-Fast, Claude-Sonnet-4.5, Perplexity-Sonar-Pro, Google search, Exa AI Search, Tavily AI Search, and Gensee AI Search (Gensee, 2026). For the LLMs, we extract the URLs that they output in their final answer. URLs visited (e.g., during intermediate reasoning steps) but not used in the final answer are not included. For Exa, Tavily, and Gensee, we directly call their developer APIs, which output a list of URLs. For Google, we call its standard Google web search API, which returns a ranked list of URLs. We use its top-5-ranked URLs for the mea- surement, as users are more likely to click the top-5 results and AI-based answers usually only include around five ref- erences. In total, we evaluated 3996 sources on 12 LLMs and search systems. 4.2. Results Table 4 summarizes our evaluation results, presenting the average scores across all 100 requests for each system under the eight metrics. We also include an average score and a weighted average score. The weighted average gives the content-based rubrics a higher weight than meta-attribute ones, with a ratio of 3:2. Table 5 details the performance across different query intents. Overall, GPT-5 achieves the highest weighted overall score and ranks first in six out of eight metrics. GPT-5 also consis- tently ranks first across all five query types. GPT-5 excels at vetting sources for Organization, Author, and Domain Au- thority, indicating that it likely involves an internal screening process that choose more authoritative sources than other ones even when they have similar content. It also achieves the highest Factuality, NEutrality, and Layout Clarity scores, suggesting that more authoritative pages are likely to also be more vetted and objective (detailed analysis to be presented in Section 4.3). Grok-4.1-Fast achieves the second-highest weighted over- all score. The performance of Grok-4.1-Fast is the most surprising, as we use its non-reasoning mode. Its Content Relevance score is much higher than GPT-5, and it per- forms reasonably well in all other metrics. We suspect that it has been fine-tuned to construct coherent, logical argu- ments even without reasoning chains. Looking at query types, Grok performs better in Shopping, Argumentative, and Factual types but worst in Reasoning, suggesting its strong capability of constructing arguments and real-time information but less robust on harder questions that require deeper thinking. Gensee search achieves the third-highest weighted overall score, mainly because of its overwhelmingly high Content- Relevance score. Different from Exa and Tavily search, Gensee has a “Deep Search” mode that involves internal reasoning steps that are tailored to web search, explaining its high Content-Relevance score. However, when selecting more relevant sources, it overlooks source authority, explain- ing its lower Organization, Author, and Domain Authority scores. The lowest weighted overall scores are Perplexity-Sonar- Pro, Gemini-2.5-Flash, and Tavily search. Perplexity- Sonar-Pro suffers from a significant penalty in Neutrality (3.62), the lowest score among all. While it performs ade- quately on Freshness and Layout, its tendency to retrieve or synthesize content with lower neutrality suggests it may be over-indexing on opinionated sources or failing to bal- ance conflicting viewpoints as effectively as GPT-5 (4.55). Additionally, it struggles with Informational queries (3.59), indicating that while it functions well for quick answers (Ar- gument 4.24), it lacks the depth required for comprehensive topic exploration. Gemini-2.5-Flash is the smallest LLM among all the LLMs. Its primary point of failure is Content Relevance (lowest of all systems), indicating its failure to identify sources that can answer a query properly. This could be due to its lack of a strong semantic reasoning capability to filter search results effectively. Tavily ranks lowest overall among all systems evaluated. Despite it being an AI-native search tool, it failed to re- trieve enough content-relevant, factual, or objective sources (among the lowest in these scores). Its performance on the Reasoning request type is also among the lowest. This indicates that it lacks strong reasoning capability. Interestingly, Google, being a pure SERP-style baseline, achieves the 7th among all systems. Counterintuitively, its Content Relevance score is higher than the overall-top- performing GPT-5. This indicates that even with just a single-step SERP-style retrieval, the sources retrieved can be of high relevance. It also achieves relatively high scores in Layout Clarity and Author Accountability. Surprisingly, 5 SourceBench: Can AI Answers Reference Quality Web Sources? Table 4. Overall Evaluation Results. Weighted gives the first three columns a 1.5 weight and scales the max to 100. Avg shows the average score of the eight metrics. Evaluating systems are first grouped by type and then ordered by their weighted overall score. System CR FA NE FR OA A DA LCAvgWeightedRank GPT-53.9234.7724.5474.4904.7934.4334.7364.0164.46289.0811 Grok-4.1-Fast4.3284.5083.9614.1804.3153.9934.3343.6334.15383.3812 GPT-4o4.2414.2073.9254.5243.9634.1504.1803.3444.06781.5184 Claude-Sonnet-4.54.2094.2173.8184.4474.1784.2024.0793.3364.06181.2825 Gemini-3-Pro3.6334.1333.8514.7434.1413.8913.9933.502 3.98879.3798 Gemini-3-Flash3.6334.1003.9254.4594.0974.0573.9623.4163.96078.9819 Perplexity-Sonar-Pro3.6914.0753.6194.3583.8064.2204.1833.6433.95278.54710 Gemini-2.5-Flash3.5203.9953.8984.4454.0544.1634.0613.3363.93878.33911 Gensee4.4354.4323.8954.3433.9703.9704.1023.3434.06081.7953 Exa3.6494.1204.0244.3464.1254.3064.2403.391 4.02180.1086 Tavily3.5454.0663.8014.4484.0434.0334.1563.3323.93378.26812 Google-Search3.9484.2223.7764.0593.8794.1334.2323.7494.00079.9397 Table 5. Evaluation Results by Request Category. Results shown are weighted score across metrics for each system and query type. SystemArgumentReasoningFactualSocialShoppingOverall GPT-590.19288.33788.99587.84085.55389.116 Grok-4.1-Fast90.35178.01286.00278.82084.48683.381 GPT-4o82.60578.28386.60579.21180.88481.517 Claude-Sonnet-4.579.39879.15885.56074.39880.10481.281 Gemini-3-Pro81.42172.63281.87372.37281.355 79.359 Gemini-3-Flash81.05372.28580.40274.25778.73078.905 Perplexity-Sonar-Pro82.51671.13980.99475.70079.39178.501 Gemini-2.5-Flash82.12673.86077.67474.40781.01878.269 Gensee85.37076.67885.94777.94579.43881.876 Exa86.44670.83183.59376.96681.97580.204 Tavily83.56068.33982.27473.67178.80878.170 Google-Search85.27575.33883.53176.22477.05579.939 its Freshness is the lowest among all systems, likely due to its less frequent re-indexing. Insight 1:The next leap of AI-based search should go for architectures that explicitly weight source credibility and content quality, mimicking the editorial judgment of a human expert rather than keyword matching machines. 4.3. Correlation of Metrics To understand these results and our metrics better, we per- formed a correlation analysis across the eight quality metrics. Figure 1 plots the heatmap of the Pearson correlations with aggregated data from all evaluating systems. Overall, there are two clusters of correlated metrics. First, the three types of source accountability metrics (OA,A,DA) are correlated, with the strongest correlation between Ownership Account- ability (OA) and Domain Authority (DA). This suggests that credible domains often clearly disclose their owners who are authoritative. The second cluster is among the three content-based scores (CR,FA,NE). As expected, the three content-related scores are correlated. More interestingly, Freshness (FR) exhibits near-zero correlation with all other metrics for all the systems, demonstrating that all systems Figure 1. Correlation between Metrics. Heatmap of correlation between different metrics; results from all evaluating systems are aggregated. 6 SourceBench: Can AI Answers Reference Quality Web Sources? Figure 2. Correlation in HotpotQA. evaluate the temporal aspect of content as an entirely inde- pendent variable, and that it has no relationship with either the content or other meta-features of a page. 4.4. Understanding the Impact of Request Types Performance variability across query types highlights dif- ferent emphasis of query types. Commercial intent queries (VACOS) achieved the highest scores for Freshness and Fac- tual Accuracy but the lowest for Layout Clarity. This reflects the nature of e-commerce environments, where high-utility, rapidly updated specification data is frequently embedded within visually cluttered, ad-heavy interfaces. Community- based inquiries (Quora) prioritized Freshness at the expense of Objectivity. In contrast, factuality-centric tasks (Pinoc- chios) inverted this pattern, maximizing the “Trust” met- rics (organization, author, domain authority, and factuality) while having the lowest Freshness scores, indicating au- thoritative content may appear in older, established sources. Finally, multi-hop reasoning tasks (HotpotQA) exposed a critical limitation in all the systems, yielding the lowest Content Relevance scores. 5. Deep Dive into GenAI and Web Search To further understand the relationship between web source quality and AI answer quality and to discern the impact of LLMs vs. search tools, we conduct a set of deep-dive experiments to strengthen our findings. 5.1. Source Quality and Answer Quality To understand web source quality (i.e., SourceBench’s scores) relate to the text answer quality (i.e., original bench- mark used in Table 1), we use HotpotQA (reasoning) and DebateQA (argumentative)’s original evaluators to collect the text-answer scores for the AI systems we evaluate. As Google, Exa, and Tavily do not return a text answer, we do not include them in this evaluation. Table 6 shows the answer score rated using HotpotQA’s ground truth answer by exact match percentage, together with the eight metrics in SourceBench and the weighted overall SourceBench score. Figure 2 plots the correlation between each metric and the answer score. As expected, Factuality has a high positive correlation with the answer score. While most other metrics also have positive correla- tion, Author Accountability has a negative correlation with the answer score. In practice, users are more inclined to be willing to check sources from well-established authors. This implies that traditional answer-only evaluation is insuf- ficient. We also evaluate the text answers generated for DebateQA (Argumentative) using DebateQA’s evaluator. We use De- bateQA’s evaluator to get two metrics they use: Perspective Diversity (PD, lower the better) and Dispute Awareness (DA, higher the better). These results and SourceBench’s scores are summarized in Table 7. As seen, the overall source quality largely determines the Perspective Diversity metric. However, high content diversity does not inherently guarantee Dispute Awareness. Insight 2: Web source quality highly affects AI answer qual- ity, but today’s AI benchmarks fail to evaluate all aspects of source quality. 5.2. Search Engine and Generative Engine An interesting question is to understand the relationship be- tween GenAI (semantic search) and SERP (keyword search) and how that relates to the overall source quality. To un- derstand this, we analyze the number of sources that each AI-based solution returns that are also in the first five pages of Google search results. Table 8 shows this result. Interest- ingly, GPT-5, the best-performing system in SourceBench, has the lowest amount of sources appearing in Google search results, and Tavily, the worst-performing in SourceBench, has the highest overlap with Google search. This indicates that good sources are mostly not in the one-shot, keyword- based searches, justifying the importance of AI solutions in the new era of information search. Insight 3: AI search is not merely summarizing SERP re- sults, but actively discovering high-quality buried evidence that require deeper thinking. 5.3. LLM and Search Tool To understand the importance of LLM vs. web search in delivering high-quality web references, we perform a con- trolled experiment with DeepSeek, which does not by itself have any web search capability but allows the supply of 7 SourceBench: Can AI Answers Reference Quality Web Sources? Table 6. HotpotQA (Reasoning) Results. The answer score shown is exact match percentage to ground truth. The last column shows the weighted SourceBench overall scores. Rows ordered by answer score. SystemAnswer ScoreCR FA NE FR OA A DA LCWeighted Source GPT-518.33%3.9234.7724.5474.4904.7934.4334.7364.01689.081 Gensee14.04%4.4354.4323.8954.3433.9703.9704.1023.34381.795 Grok-4.1-Fast10.00%4.3284.5083.9614.1804.3153.9934.3343.63383.381 Perplexity-Sonar-Pro10.00%3.6914.0753.6194.3583.8064.2204.1833.64378.547 Claude-Sonnet-4.59.26% 4.2094.2173.8184.4474.1784.2024.0793.33681.282 Gemini-3-Pro8.33%3.6334.1333.8514.7434.1413.8913.9933.50279.379 Gemini-3-Flash8.33%3.6334.1003.9254.4594.0974.0573.9623.41678.981 Gemini-2.5-Flash6.67%3.5203.9953.8984.4454.0544.1634.0613.33678.339 GPT-4o5.00%4.2414.2073.9254.5243.9634.1504.1803.34481.518 Table 7. DebateQA Results. Rows are ordered by the weighted score ranking from the provided dataset. PD (Perspective Diversity) and DA (Dispute Awareness) are prepended from the previous analysis. SystemPD↓DA↑CR FA NE FR OA A DA LCWeighted Source GPT-53.6210.3833.9234.7724.5474.4904.7934.4334.7364.01689.081 Claude-Sonnet-4.53.8000.6334.2094.2173.8184.4474.1784.2024.0793.33681.282 Gemini-3-Flash3.8850.717 3.6334.1003.9254.4594.0974.0573.9623.41678.981 Gensee3.8930.2334.4354.4323.8954.3433.9703.9704.1023.34381.795 Perplexity-Sonar-Pro3.9010.4833.6914.0753.6194.3583.8064.2204.1833.64378.547 GPT-4o3.9070.5264.2414.2073.9254.5243.9634.1504.1803.34481.518 Gemini-2.5-Flash3.9610.4503.5203.9953.8984.4454.0544.1634.0613.33678.339 Grok-4.1-Fast4.0690.317 4.3284.5083.9614.1804.3153.9934.3343.63383.381 Gemini-3-Pro4.0880.583 3.6334.1333.8514.7434.1413.8913.9933.50279.379 Table 8. GenAI and Search Result Overlapping MetricGPT-5 GPT-4o Grok-4.1 Gem-3-pr Gem-3-Fl Gem-2.5-FlPlxtyExaTavily Gensee Claude % in Google15.9927.5329.6720.9523.531.964044.6655.4528.437.1 W. Source89.08181.51883.38179.37978.98178.33978.547 80.108 78.26881.79581.282 Table 9. DeepSeek Results. System CR FA NE FR OA A DA LCWeighted Avg DS-Chat-Tavily3.7834.0873.7834.2174.2323.8554.2463.50770.085 DS-Chat-Gensee4.2444.5644.1544.4234.3213.9874.4623.92275.989 DS-Reasoning-Tavily4.1904.6034.0344.3974.5524.1034.6553.724 75.823 DS-Reasoning-Gensee4.0404.4673.9874.3514.2304.0814.3923.79774.103 any external search tools. We feed the highest and lowest search tools found in our experiments, Gensee and Tavily, to two DeepSeek models, DeepSeek-3.2-Chat (no reasoning capability) and DeepSeek-3.2-Reason (DeepSeek-AI, 2025). Table 9 shows the overall results. Surprisingly, DeepSeek-Chat plus Gensee performs the best, with DeepSeek-Reasoning plus Tavily being the second best and DeepSeek-Reasoning plus Gensee being the third. These data suggest that high-quality web search acts as a functional substitute for advanced reasoning and even outperforms it. Interestingly, the DeepSeek-Reason model demonstrates its value better when paired with the lower- quality Tavily search than with the higher-quality Gensee search. This is likely due to Gensee search’s internal think- ing capability conflicts with some of DeepSeek’s reasoning. We further analyze all the internal steps of a sampled shop- ping query that looks for a laptop that satisfy multiple re- quirements. Tavily returns more biased sources and sources that focus on one requirement, and DeepSeek-Chat takes the returned sources without further thinking. Gensee re- turns more well-rounded sources covering all requirements, leading to a higher-quality final answer and references. The Tavily issue, while not fixed by Deepseek-Chat, is captured and fixed by Deepseek-Reason, as it finds the misalignment of search results with the user answer and explictly asks Tavily to search for previously unincluded requirements. Insight 4: Instead of relying on a model to “think” its way through noise‘, providing superior, well-curated context allows simpler models to achieve better outcomes. 8 SourceBench: Can AI Answers Reference Quality Web Sources? 6. Conclusion Impact Statement This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here. References DeepSeek-AI. Deepseek-v3.2: Pushing the frontier of open large language models, 2025. URLhttps://arxi v.org/abs/2512.02556 . Released December 2, 2025. Dewan, M., Liu, J., and Shah, C. True: A reproducible framework for llm-driven relevance judgment in informa- tion retrieval. arXiv preprint arXiv:2509.25602, 2025. Ding, Y., Facciani, M., Joyce, E., Poudel, A., Bhattacharya, S., Veeramani, B., Aguinaga, S., and Weninger, T. Cita- tions and trust in llm generated responses. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 39, p. 23787–23795, 2025. Es, S., James, J., Anke, L. E., and Schockaert, S. Ragas: Automated evaluation of retrieval augmented generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, p. 150–158, 2024. Exa. Exa documentation: Getting started.https://exa. ai/docs/reference/getting-started, 2026. Accessed 2026-01-26. Friel, R., Belyi, M., and Sanyal, A. Ragbench: Explainable benchmark for retrieval-augmented generation systems. arXiv preprint arXiv:2407.11005, 2024. Gemini Team. Gemini: A family of highly capable multi- modal models. Technical report, Google DeepMind, 2023. URL https://arxiv.org/abs/2312.11805. Gensee. Gensee ai, 2026. URLhttps://w.gensee .ai/. Accessed: 2026-01-26. Google Search Quality Team. Search quality evaluator guidelines. Technical report, Google, 2024. URLhttp s://static.googleusercontent.com/med ia/guidelines.raterhub.com/en//searc hqualityevaluatorguidelines.pdf. Hassabis, D., Kavukcuoglu, K., and Google DeepMind. A new era of intelligence with gemini 3. Google Official Blog, November 2025. URLhttps://blog.googl e/products-and-platforms/products/ge mini/gemini-3/. Hu, X., Chen, J., Li, X., Guo, Y., Wen, L., Yu, P. S., and Guo, Z. Towards understanding factual knowledge of large language models. In The twelfth international conference on learning representations, 2024. URLhttps://op enreview.net/forum?id=9OevMUdods. Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., and Narasimhan, K. Swe-bench: Can language mod- els resolve real-world github issues?arXiv preprint arXiv:2310.06770, 2023. Li, R., Luo, Z., and Du, X. Fine-grained hallucination detection and mitigation in language model mathematical reasoning. 2024. Loru, E., Nudo, J., Di Marco, N., Cinelli, M., and Quat- trociocchi, W. Decoding ai judgment: How llms assess news credibility and bias. 02 2025. doi: 10.48550/arXiv .2502.04426. Mehdi, Y. Reinventing search with a new ai-powered mi- crosoft bing and edge, 2023. URLhttps://blog s.microsoft.com/blog/2023/02/07/rein venting-search-with-a-new-ai-powered -microsoft-bing-and-edge-your-copilot -for-the-web/. Accessed: 2026-01-27. Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., and Deng, L. Ms marco: A human- generated machine reading comprehension dataset. 2016. URL https://arxiv.org/abs/1611.09268. OpenAI. Searchgpt prototype, 2024. URLhttps:// openai.com/index/searchgpt-prototype/ . Accessed: 2026-01-27. Ouyang, J., Pan, T., Cheng, M., Yan, R., Luo, Y., Lin, J., and Liu, Q. Hoh: A dynamic benchmark for evaluating the impact of outdated information on retrieval-augmented generation. arXiv preprint arXiv:2503.04800, 2025. URL ’https://arxiv.org/abs/2503.04800’. Papenmeier, A., Kern, D., Hienert, D., Sliwa, A., Aker, A., and Fuhr, N. Dataset of natural language queries for e- commerce. In Proceedings of the 2021 conference on human information interaction and retrieval, p. 307– 311, 2021. Perplexity AI. Perplexity ai: Ask anything, 2022. URL https://w.perplexity.ai/. Accessed: 2026- 01-27. Phan, L., Gatti, A., Han, Z., Li, N., Hu, J., Zhang, H., Zhang, C. B. C., Shaaban, M., Ling, J., Shi, S., et al. Humanity’s last exam. arXiv preprint arXiv:2501.14249, 2025. Quora and Kaggle. Quora question pairs.https://w w.kaggle.com/c/quora-question-pairs , 9 SourceBench: Can AI Answers Reference Quality Web Sources? 2017. Kaggle Competition Dataset determining semantic equivalence between questions. Saad-Falcon, J., Khattab, O., Potts, C., and Zaharia, M. Ares: An automated evaluation framework for retrieval- augmented generation systems. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies (Volume 1: Long Papers), p. 338– 354, 2024. Singh, A., Fry, A., Perelman, A., Tart, A., Ganesh, A., El-Kishky, A., McLaughlin, A., Low, A., Ostrow, A., Ananthram, A., et al. Openai gpt-5 system card. arXiv preprint arXiv:2601.03267, 2025. Sivasothy, S., Barnett, S., Kurniawan, S., Rasool, Z., and Vasa, R. Ragprobe: An automated approach for evaluat- ing rag applications. arXiv preprint arXiv:2409.19019, 2024. Tavily Inc. Tavily docs.https://docs.tavily.co m/welcome, 2026. Accessed 2026-01-26. Wang, H., Whitworth, A., Cheung, P. M., Zhang, Z., and Kamath, K. Llm-based relevance assessment for web- scale search evaluation at pinterest. 2025. xAI. Grok 4.1 model card. Technical report, xAI, Novem- ber 2025. URLhttps://data.x.ai/2025-1 1-17-grok-4-1-model-card.pdf. Released November 17, 2025. Xu, R., Qi, X., Qi, Z., Xu, W., and Guo, Z. Debateqa: Evaluating question answering on debatable knowledge. arXiv preprint arXiv:2408.01419, 2024. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhut- dinov, R., and Manning, C. D. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 conference on empirical methods in natural language processing, p. 2369–2380, 2018. 10 SourceBench: Can AI Answers Reference Quality Web Sources? A. Generative Engine Source Reliability Rubric In this section, we present the revised evaluation rubric used to assess the reliability of sources retrieved by Generative Engines. This rubric synthesizes findings from recent literature (Dewan et al., 2025; Loru et al., 2025) and industry standards (Google Search Quality Team, 2024) with specific implementation guidance for automated scoring. Table 10. Revised Source Reliability Rubric with Citations & Guidance. Dimensions are scored on a 1–5 scale. CriterionRationale, Citations & GuidanceScoring Guidelines (1–5 Scale) Group 1: Content Assessment 1. Semantic RelevanceCore Utility. Dewan et al. (2025) validated that “Intent Alignment” is distinct from factual accuracy. Wang et al. (2025) showed fine-grained scales outperform binary metrics. Implementation Guidance: UseTextcontentlengthas a signal.If length is<300 chars, penalize relevance unless it is a perfectly concise factual answer. 5 (Fully Meets): Comprehensive, direct answer. No further searching needed. 4 (Highly Meets): Highly relevant but lacks minor details or slightly outdated. 3 (Moderately Meets): Broadly on topic but misses specific intent; requires more clicks. 2 (Slightly Meets): Tangential keyword match only. Misses core question. 1 (Fails to Meet): Irrelevant, wrong language, or technical failure. 2. Factual AccuracyEvidence Verification. Li et al. (2024) identify lack of citations as a primary cause of hallucination. Loru et al. (2025) found accuracy is the strongest predictor of credibility. Note:Primary sources (Gov, Edu, Science) are weighted higher than secondary sources (News). 5 (Verified): Claims cited with primary sources (Gov, Edu, Science). 4 (Accurate): Aligns with consensus. Secondary citations (News) exist. 3 (Unverified): Plausible “common knowledge” but lacks evidence/links. 2 (Suspect): Fringe theories, logic gaps, or context omissions. 1 (False): Demonstrably false, dangerous, or de- bunked. 3. Objectivity & ToneBias Detection. Loru et al. (2025) identify “Lan- guage Manipulation” and emotional charging as key signals for low-credibility domains. 5 (Neutral): Clinical/Academic. Multiple view- points. Zero emotional loading. 4 (Balanced): Professional journalistic tone. Ac- knowledges counter-arguments. 3 (Opinionated): Clear subjective bias/op-ed but grounded in facts. 2 (Inflammatory): Emotional triggers (“Shock- ing!”), excessive punctuation. 1 (Propaganda): Hate speech, fear-mongering, in- coherent. Group 2: Source & Metadata Assessment 4. Information FreshnessUtility Blocker. Ouyang et al. (2025) prove “Timeli- ness Awareness” is critical for RAG utility. Waterfall Logic Implementation: 1. Priority 1: Check lastupdated. If≤ 1 year from Jan 2026→ Score 5. 2. Priority 2: If P1 is null, use date. 3. Priority 3: If both null, use text markers (e.g., “RTX 50-series”). Exception: Iflastupdated(2025)≫ date (2012), treat as Score 5. 5 (Current): Dated within last 6–12 months. Data is current. 4 (Recent): 1–2 years old but valid. 3 (Static): Undated “evergreen”. References older context (e.g., “Windows 10”). 2 (Outdated): Clearly obsolete (old laws, discon- tinued products). 1 (Legacy): Ancient (>10 years) and factually wrong due to time. Continued on next page ... 11 SourceBench: Can AI Answers Reference Quality Web Sources? ... continued from previous page CriterionRationale, Citations & GuidanceScoring Guidelines (1–5 Scale) 5. Author AccountabilityEntity Verification. Loru et al. (2025) highlight bios as top credibility features. Explicitly required by Google Search Quality Team (2024). 5 (Expert): Named author with verifiable bio, cre- dentials linked (MD, PhD). 4 (Professional): Named author (Staff Writer) or reputable group (Editorial Board). 3 (Opaque): “Admin” or “Team” authorship. 2 (Hidden): No author listed where expected. 1 (Deceptive): Fake persona or AI-generated face. 6.Ownership Trans- parency Legitimacy. Loru et al. (2025) found declaring funding/owners is a significant differentiator for high-quality sites. The “Null = Penalty” Rule: Missing owner- ship info results in automatic downgrading. 5 (Fully Transparent): Physical address, phone number, and clear leadership/funding disclosed. 4 (Partially Transparent): Clear “About” desc. and leadership names, but missing physical contact info. 3 (Opaque): Contact form only; vague “About” section; no specific names/funding. 2 (Anonymous Org): No “About” info or owner- ship details. 1 (Hidden): Intentional shell company signs or active attempts to hide ownership. 7. Domain AuthorityReputation Proxy. Google Search Quality Team (2024) emphasize “Reputation Research” (reviews, awards) as a core trust signal. Implementation Guidance: IfDomaincategoryis.govor.edu, auto- matically assign Score 5. 5 (Authority): Gov/Edu, Major News, established brand (>10 years). 4 (Trusted): Established commercial brand or niche expert. 3 (Unknown): Functional blog/small biz. No repu- tation history. 2 (Low Quality): Content farm, parasite SEO, clickbait network. 1 (Blacklisted): Known scam, phishing, or disin- formation. Group 3: Technical / UX Assessment 8. Layout & Ad DensityUser Experience. ? demonstrate that DOM tree complexity and ad density negatively correlate with quality. Implementation Guidance: Look for “ad-interstitial” labels or fragmented structures. High volume of affiliate links suggests Score 2–3. 5 (Clean): Zero intrusive ads. High skimmability. 4 (Standard): Banners/sidebars that do not block content. 3 (Cluttered): Auto-playing video, sticky footers, heavy monetization. 2 (Obstructive):Pop-ups require dismissal. “Chumbox” fake articles at bottom. 1 (Spam/Unusable): Overlays, broken layout, ma- licious redirects. B. LLM Evaluator Prompts This appendix details the exact prompts used to automate the grading process. The evaluation was split into three distinct calls to minimize context window interference and group related reasoning tasks. B.1. Group 1: Content Quality Assessment Prompts Prompt TypeContent System PromptYou are a research evaluator specializing in Information Retrieval (IR). Your task is to perform an internal quality assessment of a cited website’s text content relative to a specific user query. 12 SourceBench: Can AI Answers Reference Quality Web Sources? User Prompt TemplateRubrics: 1. Semantic Relevance (1-5) 5 (Fully Meets): Comprehensive, direct answer. No further searching needed. 4 (Highly Meets): Highly relevant but lacks minor details or slightly outdated. 3 (Moderately Meets): Broadly on topic but misses specific intent; requires more clicks. 2 (Slightly Meets): Tangential keyword match only. Misses core question. 1 (Fails to Meet): Irrelevant, wrong language, or technical failure. Implementation Guidance: Use textcontentlength as a signal; if length is ¡300 characters, penalize relevance unless it is a perfectly concise factual answer. 2. Factual Accuracy & Citations (1-5) 5 (Verified): Claims cited with primary sources (Gov, Edu, Science). 4 (Accurate): Aligns with consensus. Secondary citations (News) exist. 3 (Unverified): Plausible “common knowledge” but lacks evidence/links. 2 (Suspect): Fringe theories, logic gaps, or context omissions. 1 (False): Demonstrably false, dangerous, or debunked. 3. Objectivity & Tone (1-5) 5 (Neutral): Clinical/Academic. Multiple viewpoints. Zero emotional loading. 4 (Balanced): Professional journalistic tone. Acknowledges counter-arguments. 3 (Opinionated): Clear subjective bias/op-ed but grounded in facts. 2 (Inflammatory): Emotional triggers (“Shocking!”), excessive punctuation. 1 (Propaganda): Hate speech, fear-mongering, incoherent. Few-Shot Calibration Examples Example 1: The Expert Document (Query 81 - Apache Mahout) Query: How do I evaluate AUC in Apache Mahout? Analysis: Relevance: 5 (Perfect intent/class alignment); Accuracy: 5 (Official docs); Tone: 3 (Clinical but saturated with boilerplate). Example 2: The GEO/Thin Content (Query 11 - Trenchcoats) Query: Reasonably-priced (under $50) short trenchcoat recommendations. Analysis: Relevance: 3 (Matches keyword but fails specific “short” constraint); Accuracy: 3 (Common knowledge, no links); Tone: 4 (Professional but slightly promotional). Example 3: The High-Quality Review (Query 1 - Laptops) Query: Smaller laptop for older games/word processing, SSD preferred. Analysis: Relevance: 4 (Highly relevant, explicit SSD mention missing in summary); Accu- racy: 5 (Lab testing); Tone: 4 (Journalistic but contains affiliate links). Task & Output Format Provide a score (1-5) and a short justification for each of the three metrics above. InputDataPacket: "queryid": "...", "querytext": "...", "sourceurl": "...", "sourcetext": "...", "textlength": "..." Output Format (Return ONLY JSON): "semanticrelevance": "score": int, "justification": "string" , "factual accuracy": "score": int, "justification": "string" , "objectivity tone": "score": int, "justification": "string" 13 SourceBench: Can AI Answers Reference Quality Web Sources? B.2. Group 2: Source Metadata Assessment Prompts Prompt TypeContent System PromptYou are a digital forensic auditor. Your task is to evaluate the external credibility and timeliness of a source based on its metadata and structural markers. User Prompt TemplateRubrics: 1. Information Freshness (1-5) 5 (Current): Dated within last 6-12 months. Data is current. 4 (Recent): 1-2 years old but valid. 3 (Static): Undated “evergreen”. References older context. 2 (Outdated): Clearly obsolete (old laws, discontinued products). 1 (Legacy): Ancient (>10 years) and factually wrong. Implementation Guidance: 1. Priority 1:searchresultlastupdated(≤1 yr from Jan 2026 = 5). 2. Priority 2: searchresultdate. 3. Priority 3: Text Cues. 2. Author Accountability (1-5) 5 (Expert): Named author with verifiable bio/credentials linked. 4 (Professional): Named author (Staff Writer) or reputable group. 3 (Opaque): “Admin” or “Team” authorship. 2 (Hidden): No author listed where expected. 1 (Deceptive): Fake persona or AI-generated face. 3. Ownership Transparency 5 (Fully Transparent): Physical address, phone, clear leadership. 4 (Partially Transparent): Clear “About”, leadership names, no physical info. 3 (Opaque): Contact form only; vague “About”. 2 (Anonymous Org): No “About” info or ownership details. 1 (Hidden): Intentional shell company signs. 4. Domain Authority (1-5) 5 (Authority): Gov/Edu, Major News, established brand. 4 (Trusted): Established commercial brand or niche expert. 3 (Unknown): Functional blog/small biz. 2 (Low Quality): Content farm, parasite SEO. 1 (Blacklisted): Known scam/disinformation. Few-Shot Examples for Calibration Ex 1 (Apache Mahout): Freshness: 5 (Dec 2025); Accountability: 4 (Apache Fdn); Trans- parency: 5; Domain: 5. Ex 2 (Democracy In Action): Freshness: 2 (2021); Accountability: 3; Transparency: 4; Domain: 4. Ex 3 (PolitiFact): Freshness: 3 (Active ecosystem); Accountability: 5; Transparency: 5; Domain: 4. Ex 4 (Yoga Detox): Freshness: 3 (Evergreen); Accountability: 5; Transparency: 5; Domain: 4. Task & Output Format Provide a score (1-5) and justification for the four metrics. Input Data Packet: "queryid": "...", "currentdate": "Jan 25, 2026", ... Output Format (JSON Only): "freshness": "score": int, "justification": "string" , "author accountability": "score": int, "justification": "string" , "ownership transparency": "score": int, "justification": "string" , "domainauthority": "score": int, "justification": "string" 14 SourceBench: Can AI Answers Reference Quality Web Sources? B.3. Group 3: Technical & Ad Density Assessment Prompts Prompt TypeContent System PromptYou are a User Experience (UX) analyst. Your task is to evaluate the usability and monetiza- tion intensity of a website based on its extracted text structure. User Prompt TemplateRubrics: Layout & Ad Density (1-5) 5 (Clean): Zero intrusive ads. High skimmability. 4 (Standard): Banners/sidebars that do not block content. 3 (Cluttered): Auto-playing video, sticky footers, heavy monetization. 2 (Obstructive): Pop-ups require dismissal. “Chumbox” fake articles at bottom. 1 (Spam/Unusable): Overlays, broken layout, malicious redirects. Implementation Guidance: Since you are viewing extracted text, look for “ad-interstitial text” (e.g., “ADVERTISEMENT” labels) or fragmented text structures. High text volume containing frequent repetitive affiliate links (e.g., “Check Price on Amazon”) suggests a Score 2 or 3. Few-Shot Calibration Examples Example 1: The Obstructive Media (Query 1 - Laptops) Query: Best ultraportable laptops for 2026? Source: pcmag.com Analysis: Layout & Ad Density: 2 - Text is fragmented with frequent “ADVERTISEMENT” markers and ghost text from auto-play videos. Over 20 distinct commercial CTAs (“GET IT NOW”) exceed the threshold for high-intensity monetization. Example 2: The Cluttered Portal (Query 21 - Yoga Detox) Query: Can yoga really detoxify your body? Source: healthgrades.com Analysis: Layout & Ad Density: 2 - Heavily saturated with repetitive navigational artifacts (“Find a Doctor”, “Sign In”) and commercial CTAs. UX is secondary to user acquisition, triggering the “Obstructive” score. Example 3: The Clean Doc (Query 81 - Apache) Query: How do I evaluate AUC in Apache Mahout? Source: mahout.apache.org Analysis: Layout & Ad Density: 5 - Continuous, logically structured technical document. Functional navigation (“SKIP NAVIGATION LINKS”) is present but standard; zero commer- cial markers or pop-ups. Task & Output Requirement Provide a score (1-5) and a short justification for this metric. Input Data Packet: "source url": "...", "sourcetext": "...", "textlength": "..." Output Requirement (Return ONLY JSON): "layout addensity": "score": int, "justification": "string" 15