Paper deep dive
Contextualization of Third-Party Cloud Security Findings
Leon Goldberg, Gal Engelberg
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 10/8/2026, 6:27:06 AM
Summary
This paper introduces a contextualization framework for third-party cloud security findings, addressing the limitation of static vendor severity scores. The authors developed a deep research agent that utilizes a precomputed cross-signal asset graph to investigate findings against live environment data, including resource state, graph neighborhood, and cross-product signals. In a production field study involving 9,967 HIGH severity findings across eight environments, the agent re-graded three-quarters of the findings, mostly downward. The study demonstrates that contextual evidence often lies beyond the flagged resource and that the agent's verdicts are highly faithful to live infrastructure states.
Entities (12)
Relation Signals (9)
Contextualization Agent â regrades â Finding Severity
confidence 98% · Three in four findings are re-graded, mostly downward... returns an adjusted severity with an evidence trace.
Contextualization Agent â achievesfaithfulness â 99.4%
confidence 95% · read-only probes of live infrastructure confirm the decisive fact for 99.4% of decided findings.
Contextualization Agent â processes â 9,967 findings
confidence 95% · We evaluate it in a production field study of 9,967 vendor HIGH findings...
Contextualization Agent â uses â Asset Graph
confidence 95% · A deep research agent over a precomputed cross-signal asset graph investigates each finding...
CVSS â defines â Environmental Metrics
confidence 92% · The Common Vulnerability Scoring System (CVSS) defines environmental metrics that adjust a base score to a particular deployment
Contextualization Agent â evaluatedon â Platform B
confidence 90% · We apply the system to the HIGH queue of two commercial cloud security platforms, Platform A and Platform B...
Contextualization Agent â evaluatedon â Platform A
confidence 90% · We apply the system to the HIGH queue of two commercial cloud security platforms, Platform A and Platform B...
Wiz â â
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Finding severity is the main driver of how security teams prioritize remediation. For third-party cloud security findings, that severity is static: the rule that raised the finding assigns it before the rule meets any environment, so it reflects the risk of the condition in general rather than the risk the finding poses to the concrete environment where it lives. Scoring standards define where environment-specific context belongs. How far that context changes finding severities in production, where the deciding evidence lies, and whether it holds against the live environment have not been measured. We address this gap with contextualization, re-deriving each finding's severity from evidence in the environment where the finding lives. A deep research agent over a precomputed cross-signal asset graph investigates each finding against the resource's state, its graph neighborhood, and other products' signals, and returns an adjusted severity with an evidence trace. We evaluate it in a production field study of 9,967 vendor HIGH findings from two commercial cloud security platforms across eight real production environments, on three criteria: the faithfulness of the facts behind each verdict to the live environment, the dependence of each decision on context beyond the flagged resource, and the regularity of the reasoning. Three in four findings are re-graded, mostly downward, and the same rule often moves in opposite directions inside a single environment. About half of the decisive evidence lies beyond the flagged resource, and read-only probes of live infrastructure confirm the decisive fact for 99.4% of decided findings.
Tags
Links
- Source: https://arxiv.org/abs/2610.08895v1
- Canonical: https://arxiv.org/abs/2610.08895v1
Trouble viewing inline? Open PDF directly â
Full Text
50,036 characters extracted from source content.
Expand or collapse full text
Contextualization of Third-Party Cloud Security Findings Leon Goldberg Affiliation: Sola Security, Tel Aviv, Israel Gal Engelberg Affiliation: Corresponding author: gal.e@sola.security Abstract Finding severity is the main driver of how security teams prioritize remediation. For third-party cloud security findings, that severity is static: the rule that raised the finding assigns it before the rule meets any environment, so it reflects the risk of the condition in general rather than the risk the finding poses to the concrete environment where it lives. Scoring standards define where environment-specific context belongs. How far that context changes finding severities in production, where the deciding evidence lies, and whether it holds against the live environment have not been measured. We address this gap with contextualization, re-deriving each findingâs severity from evidence in the environment where the finding lives. A deep research agent over a precomputed cross-signal asset graph investigates each finding against the resourceâs state, its graph neighborhood, and other productsâ signals, and returns an adjusted severity with an evidence trace. We evaluate it in a production field study of 9,967 vendor HIGH findings from two commercial cloud security platforms across eight real production environments, on three criteria: the faithfulness of the facts behind each verdict to the live environment, the dependence of each decision on context beyond the flagged resource, and the regularity of the reasoning. Three in four findings are re-graded, mostly downward, and the same rule often moves in opposite directions inside a single environment. About half of the decisive evidence lies beyond the flagged resource, and read-only probes of live infrastructure confirm the decisive fact for 99.4% of decided findings. Keywords Cloud Security Posture â · Contextualization â · Vulnerability Prioritization â · Security Knowledge Graphs â · Deep Research Agents 1 Introduction Cloud security posture and workload protection tools, products that evaluate cloud resources against rules and report each violation as a finding, are the main source of the work queues that cloud security teams triage. Severity is what orders those queues. Vulnerability management programs rank remediation mostly on severity scores and expert judgment [1], and security operations center (SOC) incident queues are often sorted by coarse severity [2]. Yet the severity of a third-party finding is static. The rule that raised the finding assigns it before the rule meets any environment: across all production environments we examined, each of 599 vendor rules emits exactly one severity over 99,073 failing findings of every grade, whatever resource or environment it fires on. A static severity reflects the risk of the condition in general. It does not reflect the risk the finding poses to the concrete environment where it lives, and a queue ordered by it is ordered by the general case. The scoring standards reserve a place for the missing context: the Common Vulnerability Scoring System (CVSS) defines environmental metrics that adjust a base score to a particular deployment [3, 4], the Stakeholder-Specific Vulnerability Categorization (SSVC) assesses the exposure of the specific system as deployed [5], and the National Institute of Standards and Technology (NIST) risk assessment conditions likelihood on predisposing conditions and on the controls in place [6]. A static severity leaves that place empty. Analysts report the consequence: in a qualitative study of SOC practitioners, most alarms that practitioners perceived as false positives were correct detections of activity that the organizationâs own environment made legitimate [7]. A rule carries the context its author can encode once for every environment: the condition, the resource type, and the typical consequence. The context that decides what the condition is worth in a given environment lies elsewhere: whether the resource is attached to anything, whether a workload is running, whether a network path from the internet exists, whether an account-level control already blocks the exposure, and what another product reports about the same resource. We call the difference between the severity a rule assigns and the severity the evidence in a given environment supports the rule-severity gap. Commercial platforms build environment graphs and attack-path views [8, 9], and controlled studies report that rule-based posture alerts include many false positives that context or active validation removes [10, 11]. Structured context also raises the accuracy of language models on security tasks, for identity reasoning [12] and Kubernetes remediation [13], and a security context layer reaches higher coverage than coding agents on cloud investigation tasks [14]. The size of the rule-severity gap on production findings, the location of the evidence that closes it, and whether that evidence holds against live infrastructure have not been measured. We present a production field study of the contextualization of third-party findings: re-deriving each findingâs severity from evidence in the environment where the finding lives, and recording that evidence. The system under study has two components. An asset graph reconciles the signals of every connected security product into one graph of resources, controls, and identities, and precomputes the structural facts that decide risk, such as reachability and attachment [15]. A contextualization agent is a deep research agent over that graph and over the tabular data of every connected product, built on the investigation layer of Goldberg et al. [14]: for each finding it plans an investigation, issues structured and graph queries, searches for tables held by other products, and ends with a verdict that may move the severity, together with a trace of the evidence it used. We apply the system to the HIGH queue of two commercial cloud security platforms, Platform A and Platform B, across eight real production environments. Both platforms appear among the recognized leaders in Gartnerâs and Forresterâs evaluations of cloud-native application protection platforms (CNAPP) and cloud workload protection platforms (CWPP). The contribution of this paper is a conceptual architecture for an agent that contextualizes findings. It specifies the agentâs unit of investigation, the evidence actions available to it, its reasoning loop, and an evidence policy that sets a higher bar for raising a severity than for lowering it. We evaluate the architecture in production on 9,967 vendor HIGH findings against three criteria: faithfulness, whether the facts its verdicts rest on hold in the live environment; context dependence, how much of each decision needs evidence beyond the flagged resource; and regularity of reasoning, whether the verdicts follow a small set of recurring patterns that independent LLM judges identify reproducibly. The rest of the paper is structured as follows. Section 2 positions the work against scoring standards, reachability analysis, and language-model and industrial agents; Section 3 describes the contextualization agent; Section 4 describes the corpus and the evaluation; Section 5 reports the results; Sections 6 and 7 discuss implications and limitations and conclude. 2 Related Work Prior work uses environmental context to measure the actual severity of a finding in four ways: as scoring inputs, through reachability, through language models and agents, and in industrial systems. Scoring standards reserve a slot for environment-specific context and leave filling it to the consumer. CVSS modifies a base score with environmental metrics, such as the Modified Attack Vector, that an analyst supplies by hand for each deployment [3, 4]. SSVC rates the exposure of the installed system [5], and the Exploit Prediction Scoring System (EPSS) excludes instance-specific context by design [16]. Studies of environmental metrics simulate the gain of adding organizational context [17], apply the metrics to published scores to study the resulting distribution [18], find them hard for assessors to apply on segmented networks [19], or propose deriving them from inventories and network topology [20]. Context here is supplied by an analyst or an organizational model. The non-intrusive, context-based vulnerability scoring framework (NCVS) is the exception: it collects a cloud serviceâs topology and configuration automatically and scores its vulnerabilities over the resulting dependency graph [21]. Reachability analysis reads context from the artifact or the network itself. For software dependencies, checking whether vulnerable code is deployed or called removes a large share of alerts. A manual study of 60 dependent projects found that as many as 73.3% of those with outdated dependencies were not exposed [22], and about 20% of vulnerable dependencies were never deployed [23]. Call-graph pruning cut npm audit false positives by 81% [24], and fewer than 1% of Maven packages have a call path to a vulnerable function in a dependency [25]. Related analyses follow the same principle [26, 27]. For networks and cloud, attack graphs [28, 29] and automated reasoning over network and identity configuration [30, 31] decide which weaknesses an attacker can reach. For rule-based cloud findings, controlled studies report that over 80% of default public-access alerts on more than 1,000 storage buckets were false positives [10], and that active probes cut false positives by 93% on average [11]. These works decide whether a finding applies; they do not re-grade how severe it is in its environment. Language models first entered severity assessment without an environment. Predicting CVSS base metrics from descriptions is limited by descriptions that lack the deciding context [32], and models asked for SSVC decisions from text over-predict risk [33]. Environment-aware work supplies context in different forms. CVE-LLM, which evaluates Common Vulnerabilities and Exposures (CVE) entries with a large language model (LLM), reads a manufacturerâs asset descriptions and vendor notifications and emits an affected or not-affected verdict with the CVSS environmental metrics. In production, with experts validating its output, it reaches a micro-F1 of 0.98 on those metrics [34]. On VEX-Bench, a benchmark for Vulnerability Exploitability eXchange (VEX) assessments, agents read the downstream codebase to decide whether a supply-chain vulnerability affects it, and the strongest reach about 80% F1 on that binary decision [35], while SastBench evaluates agentic triage of static-analysis findings [36]. For cloud and Kubernetes, a judge-agent framework is validated on cloud-misconfiguration triage in large environments [37], and KubeGuard reads runtime logs to propose least-privilege configurations [38]. On a purpose-built cluster, giving a model the runtime topology lifts its correct patches, for Kubernetes findings that depend on that topology, to 78.0% from 11.1% [13]. Our earlier work adjusts finding severity with a model inside an asset-level scoring pipeline [15], finds that agents answer identity tasks more accurately when given structured relational context [12], and reports higher coverage than coding agents on cloud investigation tasks for a security context layer [14]. None of these re-grades third-party cloud findings in production with evidence joined across products, or checks the deciding facts against the live environment. Industrial platforms now ship agents for this task. Wizâs Green Agent traces posture issues through cloud resources, access policies, network reachability, and code changes, and returns a remediate-or-ignore verdict with suggested severity adjustments [39]; its Blue Agent issues verdicts on threat detections [40]. Upwindâs Red Agent validates exploitability from runtime reachability and attack paths [41]. CrowdStrike pairs a learned exploit-prediction model over threat intelligence with derived asset criticality [42], and Microsoft Defender for Cloud ranks recommendations by environment risk factors and attack paths [43, 8]. Microsoft and Google also triage alerts with agents [44, 45]. Open-source projects follow the same pattern: a Prowler plugin lets a coding agent order posture findings by attack-path blast radius [46], and GitHub Security Labâs taskflow agent triages code-scanning alerts with workflow and permission context [47]. None of these systems publishes a reproducible evaluation of whether its contextual re-rating is correct. Together, these lines show that environmental context changes the actual severity of a finding, whether the context comes from analyst input, code and network reachability, or an agentâs investigation. What is missing is an agent that gathers environment evidence for each finding across products, re-grades third-party severities in production, and is evaluated on whether the facts behind its verdicts hold. 3 The Contextualization Agent The contextualization agent is a deep research agent in the sense of [48]: it plans a multi-step investigation, retrieves over several sources, uses tools iteratively, and ends with a structured report, here a contextualized finding. Figure 1 shows its parts: the finding and its priors, the environment and the actions that reach it, the investigation, the evidence policy, and the contextualized finding. Finding f=(r,c,s0)f=(r,c,s_0) resource r, failed control c, vendor severity s0s_0 Exploitation hypothesis h Evidence acquisition ete_t Evidence assessment â°t+1E_t+1 Investigation Resource state ÏâĄ(r)Ï(r) Neighborhood kâ(r)N_k(r) Cross-product signals âĄ(r)X(r) Environment Priors P verdicts on similar findings, organization context Evidence policy Ï admissible: observable evidence on the path of hypothesis h asymmetric bar: Ξâ>Ξâ _ > _ Attenuated s<s0s<s_0 Unchanged s=s0s=s_0 Amplified s>s0s>s_0 Evidence trace Ï Contextualized finding (s,Ï)(s,Ï) Figure 1: Contextualization agent. s0s_0: vendor severity; s: contextualized severity; Ξâ,Ξâ _ , _ : bars for raising and lowering a severity. A finding f=(r,c,s0)f=(r,c,s_0) is a resource r that fails a control c, with vendor severity s0s_0. The agent receives the definition of control c; for a third-party finding this is the vendorâs name, description, and severity, without the check logic. It also receives the state and graph neighborhood of resource r, the other controls resource r fails and passes, and findings from other tools that test for the same risk. The scope is technical: properties, configurations, relations, and guardrails. Our running example is a security group, resource r, flagged for admitting inbound traffic from any address on an administrative port, control c, with vendor severity s0s_0 = HIGH. Evidence about resource r comes in three kinds: the resource state ÏâĄ(r)Ï(r); the neighborhood kâ(r)N_k(r), holding the resources within k hops of resource r in the asset graph; and the cross-product signals âĄ(r)X(r) held by other products. The evaluation attributes each verdictâs decisive evidence to one of these kinds, and four actions reach them. A structured query runs over the tabular store that unifies every connected product; it is the most complete source, reads the resource state ÏâĄ(r)Ï(r), and joins across products. A graph query traverses several hops to return the neighborhood kâ(r)N_k(r), including the controls that neighboring resources fail. A catalog search maps a domain-level request to the data other products hold and to how that data relates to resource r, reaching cross-product signals âĄ(r)X(r) that the graph does not link. A neighborhood exploration lists what a resource kind connects to, guiding the next graph query. For the security group, the resource state ÏâĄ(r)Ï(r) holds its ingress rules and attachments, the neighborhood kâ(r)N_k(r) the interfaces, load balancers, and instances behind it, and the cross-product signals âĄ(r)X(r) what endpoint agents or vulnerability scanners report about those instances. The investigation follows the reason-act pattern [49] over an exploitation hypothesis h and an evidence set â°tE_t. At each iteration t it makes three moves. It states or revises the exploitation hypothesis h: how the failure of control c on resource r would realistically be exploited. It acquires evidence items ete_t through the actions, in parallel when independent. It assesses them, adding them to the evidence set, â°t+1=â°tâȘetE_t+1=E_tâȘ\e_t\, recording what each item showed and an interim conclusion that directs the next step. The assessment also reads the priors P: verdicts on similar findings and organization context. Verdicts on similar findings count as evidence, not as answers to copy, and a finding whose inputs have not changed reuses its earlier result. Organization context is of two kinds. A claim about the environment, such as a firewall fronting a service, is verified against the data before it applies; an accepted risk applies as stated. Only context that changed the conclusion is cited. The loop stops at a conclusion or when its budget runs out; in the second case the severity is unchanged. For the security group, the exploitation hypothesis h is an attacker on the internet reaching a service behind it. A structured query shows that every attachment belongs to a load balancer, and a graph query shows that each load balancer is internal. The evidence policy Ï decides which evidence items count and how much evidence each direction needs. Its first rule is that an item counts only if it is observed in the data and bears on how the failure would be exploited, as stated by the exploitation hypothesis h. Encryption at rest, for example, does not offset a publicly readable bucket, because anyone allowed to read the bucket receives the decrypted data. A query that returns nothing is not evidence of safety, because the data may never have been collected, and a guardrail that could exist only in application code neither lowers nor raises the severity, because it cannot be observed. Its second rule is that raising a severity needs more evidence than lowering it, which we write as a higher bar for raising, Ξâ _ , than for lowering, Ξâ _ . Lowering requires one observed compensating property in the evidence set â°tE_t: for the security group, that every attached load balancer is internal. Raising requires positive evidence that the failure is worse here, such as an attached instance with a public address running a service that a scanner reports as critically vulnerable, together with an explicit check that no firewall, allowlist, or other plausible guardrail sits in front of it. When neither bar is met, the severity stays unchanged. The output is the contextualized finding (s,Ï)(s,Ï): a contextualized severity s and an evidence trace Ï. The contextualized severity s is attenuated, below the vendor severity s0s_0, in one of two grades, mitigated when a property offsets the failure entirely (to LOW or informational) and reduced when it compensates partially (one level down). It is amplified, one level above the vendor severity s0s_0, when a property materially worsens this specific failure, and unchanged otherwise. The evidence trace Ï records the acquisitions and assessments and marks the key evidence; in the evaluation, LLM judges label this trace with a recalibration pattern. It also carries a technical rationale, a plain-language explanation, and a confidence that rates the strength of the evidence set â°tE_t regardless of direction and stays low when the investigation is shallow or the deciding evidence would be unobservable. The security group remains reachable from inside the network, so the compensation is partial and the finding is reduced from HIGH to MEDIUM, with the attachment and load-balancer queries marked as key evidence. 4 Evaluation Criteria. We measure three criteria on a production corpus. Faithfulness asks whether the fact a verdict rests on is true in the live environment, in the spirit of atomic factual precision [50]. Of the live findings whose decisive fact the probes can decide under the scope rules of the live check, the set D, let C be those whose fact matches the live state. Faithfulness is the share that matches, Ί=|C|/|D| =|C|/|D|. Because some findings cannot be checked, we also report two variants: the inclusive faithfulness ΊR _R, computed over every finding the probes could read, and the lower-bound faithfulness ΊL _L, computed over all live findings with every unchecked finding counted as wrong. Context dependence asks how often the deciding evidence comes from beyond the flagged resource. For each finding f in the traced population T, let ââĄ(f) (f) record which kind of evidence decided it, as given by the retrieval label of the evidence attribution: the resource state ÏâĄ(r)Ï(r) of the flagged resource r, its k-hop neighborhood kâ(r)N_k(r), or the cross-product signals âĄ(r)X(r) defined in Section 3. Context dependence is the share of findings decided by anything other than the resource state, Î=|fâT:ââĄ(f)â Ï|/|T| =|\fâ T: (f)â Ï\|/|T|. We report it per kind of evidence, and add a necessity share for rules that move in both directions. Regularity of reasoning asks whether the verdicts follow a small set of recurring patterns that independent LLM judges, separate runs of a language model applying the same codebook [51], identify the same way. We measure the first part by saturation, the point after which further verdicts add no new pattern [52], and the second by Krippendorffâs α [53], an agreement coefficient that is 1 for perfect agreement and 0 for chance, between the judgesâ runs. Neither measures whether the agent would repeat its verdicts, since the agent was not re-run. Corpus. The corpus is the HIGH queue of both platforms in eight real production environments: 9,967 findings, 4,222 raised by Platform A and 5,745 by Platform B. Each finding is credited, by rule namespace, to the platform that raised it, and not to any tool merely connected to the same environment. The unit of analysis is the finding, one (environment, rule, resource group) record in which near-identical resources in the same situation share one verdict. All Platform A findings come from four of the eight environments, and 92% of the findings the agent re-graded are on Amazon Web Services (AWS), with the rest on a software-as-a-service identity provider (580) and Google Cloud (9). Results refer to three subsets of the re-graded findings: the traced population, the 7,243 whose investigation trace was recovered and which the LLM judges label; the characterized corpus, the 4,772 traced findings whose decisive fact a live cloud application programming interface (API) can check; and the residual, the remaining 2,471, whose decisive facts sit in other products and policies. Procedures. Four procedures turn the agentâs verdicts into the three criteria. LLM-judge labeling gives every verdict three labels: a pattern, the kind of reasoning step that decided it, such as ânot internet-reachableâ; a location, whether its decisive fact sits on the flagged resource or elsewhere; and a retrieval, whether that fact came from the resource itself, a graph traversal, or another productâs data. The judges are independent runs of Claude Opus 4.8 that read each investigation trace with a fixed codebook, which defines the labels, and a rubric, which settles ambiguous cases in favor of on-resource; the patterns themselves were first derived from the traces by grounded theory [54, 55]. A second, independent judge run labels a sample of the same verdicts again [51], and the agreement between the two judge runs, Krippendorffâs α with Cohenâs Îș [56] and raw agreement alongside, measures how reproducibly the agentâs reasoning falls into the patterns, the second part of regularity of reasoning. Evidence attribution uses the location and retrieval labels to compute context dependence Î . Because the agent was not re-run, the necessity analysis tests from the existing verdicts, as an observational study would [57], whether that outside evidence actually set a verdictâs direction: a rule that the agent lowers on some findings and raises on others checks the same condition on all of them, so where the decisive fact is off-resource, that outside fact set the direction. Live verification gives faithfulness Ί . Read-only probes on five resource types re-read each decisive fact, for example whether the running exampleâs load balancers are still internal, directly from the cloud API instead of the asset graph the agent used, so an error in the graph cannot confirm itself; a verdict is confirmed if the fact matches it and contradicted otherwise [58, 59]. The probes run on eight AWS accounts where we hold read-only access, and since only Platform B monitors these accounts, faithfulness Ί is measured on Platform B findings. Under the scope rules, each resource is checked in one direction, as an amplification if any finding on it was amplified and otherwise as an attenuation, and every rate carries a Wilson score interval [60, 61]. Box 1 traces one finding through the investigation and the three procedures. Box 1. One finding, from vendor grade to live verification (illustrative) 1 Finding f=(r,c,s0)f=(r,c,s_0) as raised by Platform B resource r container service svc-payments-worker control c runtime exposure: the workloadâs image carries a package with a known exploitable vulnerability vendor severity s0s_0 HIGH the grade this rule carries in every environment 2 Investigation exploitation hypothesis h: an attacker exploits the vulnerable package in a running task step evidence source action observation e1e_1 resource state ÏâĄ(r)Ï(r) structured query: desired and running task count of r desired=0, running=0: scaled to zero e2e_2 neighborhood kâ(r)N_k(r) graph traversal: what routes traffic to r or starts its tasks no load-balancer target, no scheduled start Evidence policy Ï. With no running task, nothing on the path of h executes the vulnerable package: the exposure is not realized. The service can be scaled up again, so the compensation is partial and the grade drops one level. In another environment the same rule is raised to CRITICAL, where the workload runs and the vulnerable package is actively used. 3 Contextualized finding (s,Ï)(s,Ï) evidence trace Ï marks e1e_1 as key evidence HIGH ⶠMEDIUM reduced. Decisive fact: the service runs no task. 4 Evaluation how this verdict enters each criterion procedure result on this finding criterion live verification read-only cloud API read of the serviceâs running task count, bypassing the asset graph: still 0 â confirmed; a running task would count as a contradiction faithfulness Ί evidence attribution decided by the resource state ÏâĄ(r)Ï(r), so it does not count toward Î ; the fact was on the resource, yet the vendor grade ignored it context dependence Î necessity analysis the rule moves both ways, but the decisive fact is on the resource, so the finding is not in the necessity share necessity share LLM-judge labeling pattern no live service, location on-resource, retrieval structured query; labeled again by a second judge run regularity, α 5 Results Of the 9,967 HIGH findings, the agent re-grades three in four (75.1%): 58.5% down and 16.6% up, while 2,478 (24.9%) keep the vendorâs grade. Among the re-graded findings of the characterized corpus (Table 1), a single vendor grade spreads across the whole scale, from informational to CRITICAL. Table 1: Contextualized grade of vendor HIGH findings (n=4,772n=4,772). Grade Count Share A B Move MEDIUM 3,148 66.0% 47.8% 74.6% down CRITICAL 950 19.9% 26.9% 16.6% up LOW 555 11.6% 19.1% 8.1% down INFO 119 2.5% 6.2% 0.7% down Six recurring patterns account for the characterized verdicts. Five mainly lower severity: the resource is inert, attached to nothing or orphaned (32.3%); it is not internet-reachable (25.4%); no live service runs behind it (12.4%); it admits only specific sources (7.5%); or an account-level block neutralizes the condition (3.0%). One mainly raises it: a public exposure that is realized (19.4%). Each pattern moves severity in the direction that established frameworks prescribe for its fact, such as the CVSS attack vector [3], SSVCâs system exposure [5], attack-graph reachability [29], and likelihood conditioned on deployed controls [6]. Verdicts in the residual turn mainly on compensating controls and private endpoints recorded in other products and policies. The patterns attach to findings, not to rules: 62 of the 138 rules lower some findings and raise others, 57 of them inside a single environment and none across resource types. A security-group rule, for example, is lowered when the group fronts only internal load balancers and raised when the environment shows an internet-facing path. Table 2: Evaluation results. Brackets: 95% Wilson interval; for necessity, bounds from label disagreement. Criterion Measure Value Faithfulness Confirmed live, Ί 99.4% [99.1, 99.6] inclusive, ΊR _R 97.3% [96.7, 97.7] lower bound, ΊL _L 81.8% [80.7, 82.8] Context dependence Beyond the flagged resource, Î 49.0% graph neighborhood 29.8% other products 19.2% Necessity share 29.6% [27.3, 31.9] Regularity of reasoning Judge agreement, pattern, α 0.90 Judge agreement, location, α 0.94 Saturation 22% of traces Table 2 reports the three criteria, starting with faithfulness, the primary measure. âą Faithfulness. The live cloud state confirms the decisive fact of 99.4% of the verdicts the probes can decide (faithfulness Ί ), including all 181 decided exposure amplifications, and of 528 of 531 resources when each resource counts once. The 24 contradictions sit on three resources whose live state differs from the verdict. Inclusive faithfulness ΊR _R, over every finding the probes could read, is 97.3%, and lower-bound faithfulness ΊL _L, which counts every unchecked finding as wrong, is 81.8%. The check covers Platform B findings, tests the decisive fact rather than the grade, and does not reach the residual. âą Context dependence. About half of the verdicts rest on evidence beyond the flagged resource (context dependence Î=49.0% =49.0\%), drawn from its graph neighborhood (29.8%) or from another productâs signals (19.2%). A rater that reads only the flagged resource therefore has the decisive evidence for at most 51.0% of the verdicts, and one that reads a single productâs graph for at most 80.8%. For 29.6% of the traced population (the necessity share), the finding falls under a rule that moves both ways and an off-resource fact decided which way. Outside evidence concentrates in attenuations, where showing a finding safe means ruling out every path to it. âą Regularity of reasoning. Two independent judge runs agree on the pattern (Krippendorffâs α=0.90α=0.90, Cohenâs Îș=0.90Îș=0.90, 91.2% raw agreement) and, under the on-resource rubric, on the location label (α=0.94α=0.94), and no new pattern appears after the first 22% of traces. The verdicts thus reduce to a small set of patterns that are identified reproducibly. 6 Discussion The results support treating severity as a property of a finding in its environment rather than of the rule that raised it. Because severity orders remediation, re-grading three in four of the vendorâs HIGH findings changes their place in a severity-ordered queue. The same rule moving both ways inside one environment, on resources of one type, places the deciding variation at the level of the individual resource, below the level at which a rule rewrite or an environment-wide policy would operate. The three criteria qualify this reading. High faithfulness indicates that the re-grades rest on facts that hold in the live environment, for the exposure facts the probes can read. Context dependence locates those facts: about half lie beyond the flagged resource, in two tiers, graph evidence that a product could hold but its per-rule severity does not use, and evidence that exists only when products are joined. Regularity of reasoning indicates that the re-grades follow a small set of reproducible patterns whose directions match established frameworks, so each verdict can be explained and checked. We argue that verified, environment-specific reading of each finding is a precondition for acting on findings autonomously. Reading and acting differ in blast radius, reversibility, and authorization: without an analyst in the loop, acting on a false HIGH disrupts production, and failing to act on a true CRITICAL leaves an exposure open. This study measures reading only: whether each verdictâs decisive fact holds live, where that fact lies, and whether the reasoning falls into reproducible patterns, with the facts read through the same interface an automated responder would use before acting. It does not measure any action. Several limitations bound the scope. The study covers one systemâs verdicts on the HIGH queue of two platforms in eight production environments, mostly on AWS, so the patterns transfer as a set, not as fixed frequencies, and findings the environment would raise from MEDIUM or LOW are not observed. Faithfulness is measured on one platform and five resource types, against the decisive fact rather than the grade, and recall, the deciding facts the agent missed, is not measured. The labels come from runs of a single model, and without incident labels the study cannot show that findings graded up are more likely to be exploited, or findings graded down less likely, than their vendor grade implies. 7 Conclusion A contextualization agent, a deep research agent over a cross-signal asset graph, re-grades three in four of 9,967 third-party HIGH findings, mostly downward, and the same rule often moves in both directions within one environment. Its verdicts are faithful to the live environment, with 99.4% of the decisive facts the probes can decide confirmed (81.8% as a lower bound); they depend on evidence beyond the flagged resource for about half of the findings, a fifth only in another productâs signal; and they follow six reproducible patterns shared by two platforms. The rule-severity gap is large in production, and closing it requires evidence from the whole environment. References [1] J. Jacobs, S. Romanosky, B. Edwards, I. Adjerid, and M. Roytman (2021) Exploit prediction scoring system (EPSS). Digital Threats: Research and Practice 2 (3), p. 20:1â20:17. External Links: Document Cited by: §1. [2] S. Freitas, A. Gharib, and M. Magenheim (2026) Adaptive incident prioritization for security operations at scale. External Links: 2607.16963, Document, Link Cited by: §1. [3] Forum of Incident Response and Security Teams (FIRST) (2019) Common vulnerability scoring system v3.1: specification document. Note: https://w.first.org/cvss/v3.1/specification-documentAccessed 2026-10-04 Cited by: §1, §2, §5. [4] Forum of Incident Response and Security Teams (FIRST) (2023) Common vulnerability scoring system version 4.0: specification document. Note: https://w.first.org/cvss/v4.0/specification-documentDocument version 1.2. Accessed 2026-10-04 Cited by: §1, §2. [5] J. M. Spring, A. Householder, E. Hatleback, A. Manion, M. Oliver, V. Sarvepalli, L. Tyzenhaus, and C. Yarbrough (2021) Prioritizing vulnerability response: a stakeholder-specific vulnerability categorization (version 2.0). Note: Technical report, Software Engineering Institute, Carnegie Mellon University. https://w.sei.cmu.edu/library/prioritizing-vulnerability-response-a-stakeholder-specific-vulnerability-categorization-version-20/Accessed 2026-10-04 Cited by: §1, §2, §5. [6] Joint Task Force Transformation Initiative (2012) Guide for conducting risk assessments. NIST Special Publication Technical Report 800-30 Revision 1, National Institute of Standards and Technology, Gaithersburg, MD. External Links: Document, Link Cited by: §1, §5. [7] B. A. Alahmadi, L. Axon, and I. Martinovic (2022) 99% false positives: a qualitative study of SOC analystsâ perspectives on security alarms. In 31st USENIX Security Symposium (USENIX Security 22), Boston, MA, p. 2783â2800. External Links: ISBN 978-1-939133-31-1, Link Cited by: §1. [8] Microsoft (2026) Security explorer and attack paths in Microsoft Defender for Cloud. Note: Microsoft Learn. https://learn.microsoft.com/en-us/azure/defender-for-cloud/concept-attack-pathLast updated 2026-06-17. Accessed 2026-10-04 Cited by: §1, §2. [9] J. Liu (2025) The anatomy of a toxic combination of risk. Note: Wiz Blog. https://w.wiz.io/blog/the-anatomy-of-a-toxic-combination-of-riskPublished 2025-01-24. Accessed 2026-10-04 Cited by: §1. [10] Dikshant and G. Verma (2025) Precision over noise: tailoring S3 public access detection to reduce false positives in cloud security platforms. External Links: 2508.14402, Link Cited by: §1, §2. [11] Dikshant and G. Verma (2026) Alert or noise? Reducing false positives with active behavioral analysis for cloud security. ACM SIGMETRICS Performance Evaluation Review 53 (4), p. 84â90. Note: Also available as arXiv:2508.12584 External Links: Document Cited by: §1, §2. [12] E. Yavin, G. Engelberg, K. Koutsyi, L. Goldberg, and G. Baron (2026) Cross-vendor Sola ISPM benchmark: evaluating agentic AI for federated identity security reasoning. External Links: 2606.02674, Link Cited by: §1, §2. [13] F. Shaikh (2026) Does runtime topology context improve LLM-generated Kubernetes security patches?. Note: Accepted at the LLMSec workshop co-located with ESORICS 2026 External Links: 2607.25995, Link Cited by: §1, §2. [14] L. Goldberg, G. Engelberg, E. Yavin, E. Elouz, A. Zadok, and K. Koutsyi (2026) Coding agents arenât enough! Evaluating an enterprise security brain for agentic cloud investigations. Note: Version 2 External Links: 2609.30345, Link Cited by: §1, §1, §2. [15] G. Engelberg, L. Goldberg, K. Koutsyi, B. Plotnikov, T. Gilat, and B. Benhemo (2026) AI native asset intelligence. External Links: 2605.09115, Link Cited by: §1, §2. [16] FIRST EPSS Special Interest Group (2026) EPSS frequently asked questions. Note: https://w.first.org/epss/faqAccessed 2026-10-04 Cited by: §2. [17] C. Fruhwirth and T. MĂ€nnistö (2009) Improving CVSS-based vulnerability prioritization and response with context information. In Proceedings of the 3rd International Symposium on Empirical Software Engineering and Measurement (ESEM), p. 535â544. External Links: Document Cited by: §2. [18] L. Gallon (2010) On the impact of environmental metrics on CVSS scores. In 2010 IEEE Second International Conference on Social Computing (SocialCom/PASSAT), p. 987â992. External Links: Document Cited by: §2. [19] L. Allodi, S. Biagioni, B. Crispo, K. Labunets, F. Massacci, and W. Santos (2017) Estimating the assessment difficulty of CVSS environmental metrics: an experiment. In Future Data and Security Engineering (FDSE 2017), Lecture Notes in Computer Science, Vol. 10646, p. 23â39. External Links: Document Cited by: §2. [20] A. Eitel (2020) Environmental aware vulnerability scoring. In Proceedings of the 5th International Conference on Internet of Things, Big Data and Security (IoTBDS), p. 478â485. External Links: Document Cited by: §2. [21] H. Zhuang and K. Aberer (2016) A non-intrusive and context-based vulnerability scoring framework for cloud services. External Links: 1611.07383, Link Cited by: §2. [22] R. Elizalde Zapata, R. G. Kula, B. Chinthanet, T. Ishio, K. Matsumoto, and A. Ihara (2018) Towards smoother library migrations: a look at vulnerable dependency migrations at function level for npm JavaScript packages. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME), p. 559â563. External Links: Document Cited by: §2. [23] I. Pashchenko, H. Plate, S. E. Ponta, A. Sabetta, and F. Massacci (2018) Vulnerable open source dependencies: counting those that matter. In Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), p. 1â10. External Links: Document, 1808.09753 Cited by: §2. [24] B. B. Nielsen, M. T. Torp, and A. MĂžller (2021) Modular call graph construction for security scanning of Node.js applications. In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), p. 29â41. External Links: Document Cited by: §2. [25] A. M. Mir, M. Keshani, and S. Proksch (2023) On the effect of transitivity and granularity on vulnerability propagation in the Maven ecosystem. In 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), p. 201â211. External Links: 2301.07972, Document Cited by: §2. [26] S. E. Ponta, H. Plate, and A. Sabetta (2018) Beyond metadata: code-centric and usage-based analysis of known vulnerabilities in open-source software. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME), p. 449â460. External Links: Document Cited by: §2. [27] Y. Wu, Z. Yu, M. Wen, Q. Li, D. Zou, and H. Jin (2023) Understanding the threats of upstream vulnerabilities to downstream projects in the Maven ecosystem. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), p. 1046â1058. External Links: Document Cited by: §2. [28] O. Sheyner, J. Haines, S. Jha, R. Lippmann, and J. M. Wing (2002) Automated generation and analysis of attack graphs. In Proceedings 2002 IEEE Symposium on Security and Privacy, p. 273â284. External Links: Document Cited by: §2. [29] X. Ou, S. Govindavajhala, and A. W. Appel (2005) MulVAL: a logic-based network security analyzer. In 14th USENIX Security Symposium (USENIX Security 05), Baltimore, MD. External Links: Link Cited by: §2, §5. [30] J. Backes, S. Bayless, B. Cook, C. Dodge, A. Gacek, A. J. Hu, T. Kahsai, B. Kocik, E. Kotelnikov, J. Kukovec, S. McLaughlin, J. Reed, N. Rungta, J. Sizemore, M. Stalzer, P. Srinivasan, P. SubotiÄ, C. Varming, and B. Whaley (2019) Reachability analysis for AWS-based networks. In Computer Aided Verification (CAV 2019), Lecture Notes in Computer Science, Vol. 11562, p. 231â241. External Links: Document Cited by: §2. [31] I. Shevrin and O. Margalit (2023) Detecting multi-step IAM attacks in AWS environments via model checking. In 32nd USENIX Security Symposium (USENIX Security 23), Anaheim, CA, p. 6025â6042. External Links: Link Cited by: §2. [32] S. Jafarikhah, D. Thompson, E. Deans, H. Siadati, and Y. Liu (2025) From description to score: can LLMs quantify vulnerabilities?. External Links: 2512.06781, Link Cited by: §2. [33] O. Al Haddad, M. Ikram, E. Ahmed, and Y. Lee (2025) Prompting the priorities: a first look at evaluating LLMs for vulnerability triage and prioritization. External Links: 2510.18508, Link Cited by: §2. [34] R. Ghosh, H. von Stockhausen, M. Schmitt, G. M. Vasile, S. K. Karn, and O. Farri (2025) CVE-LLM: ontology-assisted automatic vulnerability evaluation using large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 28757â28765. External Links: Document, Link Cited by: §2. [35] J. Shi, E. Tsien, Y. Di, H. Zhang, Y. Tang, R. Dey, I. Shishov, G. Netanel, Z. Grinberg, V. Belousov, B. Rotman, I. Pinto, and T. Zhang (2026) VEX-Bench: benchmarking LLM agents for assessing exploitability of software supply chain vulnerabilities. External Links: 2609.08040, Link Cited by: §2. [36] J. Feiglin and G. Dar (2026) SastBench: a benchmark for testing agentic SAST triage. External Links: 2601.02941, Link Cited by: §2. [37] S. Garg, B. Cheezum, S. Dutta, and V. Agarwal (2026) JAF: judge agent forest. External Links: 2601.22269, Link Cited by: §2. [38] O. Sgan Cohen, E. Malul, Y. Meidan, D. Mimran, Y. Elovici, and A. Shabtai (2025) KubeGuard: LLM-assisted Kubernetes hardening via configuration files and runtime logs analysis. External Links: 2509.04191, Link Cited by: §2. [39] Wiz (2026) Introducing the green agent: ai-powered remediation for the cloud. Note: https://w.wiz.io/blog/introducing-wiz-green-agentWiz blog, March 26, 2026; public preview. Accessed 2026-10-05 Cited by: §2. [40] Wiz (2026) The wiz blue agent, now generally available. Note: https://w.wiz.io/blog/wiz-blue-agent-generally-availableWiz blog, March 30, 2026. Accessed 2026-10-05 Cited by: §2. [41] Upwind (2026) Validate real cloud risk with red, the validation agent. Note: https://w.upwind.io/feed/red-agent-cloud-risk-validationUpwind blog, May 15, 2026; part of the Upwind Agentic Pack announced May 13, 2026. Accessed 2026-10-05 Cited by: §2. [42] CrowdStrike (2025) Falcon exposure managementâs ai-powered risk prioritization shows organizations what to fix first. Note: https://w.crowdstrike.com/en-us/blog/falcon-exposure-management-ai-driven-risk-prioritization-shows-what-to-fix-first/CrowdStrike blog, June 25, 2025. Accessed 2026-10-05 Cited by: §2. [43] Microsoft (2026) Risk prioritization â Microsoft Defender for Cloud. Note: Microsoft Learn. https://learn.microsoft.com/en-us/azure/defender-for-cloud/risk-prioritizationLast updated 2026-06-16. Accessed 2026-10-04 Cited by: §2. [44] Microsoft (2026) Microsoft security copilot security alert triage agent in microsoft defender (preview). Note: https://learn.microsoft.com/en-us/defender-xdr/security-alert-triage-agentMicrosoft Learn documentation, updated 2026-08-11. Accessed 2026-10-05 Cited by: §2. [45] Google Cloud (2026) Detecting and containing ai-powered threats with google security operations agents. Note: https://cloud.google.com/blog/products/identity-security/detecting-and-containing-powered-threats-with-google-security-operations-agentsGoogle Cloud blog, June 9, 2026. Accessed 2026-10-05 Cited by: §2. [46] Prowler (2026) Automate security workflows inside claude code with prowler plugin. Note: https://prowler.com/blog/prowler-plugin-for-claude-codeProwler blog, May 26, 2026; plugin code in GitHub repository prowler-cloud/prowler (https://github.com/prowler-cloud/prowler). Accessed 2026-10-05 Cited by: §2. [47] GitHubSecurityLab (2026) seclab-taskflow-agent: github security lab taskflow agent. Note: https://github.com/GitHubSecurityLab/seclab-taskflow-agentGitHub repository; triage results described in the GitHub Blog post âAI-supported vulnerability triage with the GitHub Security Lab Taskflow Agentâ (Man Yue Mo, Peter Stöckli, January 20, 2026), https://github.blog/security/ai-supported-vulnerability-triage-with-the-github-security-lab-taskflow-agent/. Accessed 2026-10-05 Cited by: §2. [48] Y. Huang, Y. Chen, H. Zhang, K. Li, H. Zhou, M. Fang, L. Yang, X. Li, L. Shang, S. Xu, J. Hao, K. Shao, and J. Wang (2025) Deep research agents: a systematic examination and roadmap. External Links: 2506.18096, Link Cited by: §3. [49] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao (2023) ReAct: synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations (ICLR), External Links: 2210.03629, Link Cited by: §3. [50] S. Min, K. Krishna, X. Lyu, M. Lewis, W. Yih, P. W. Koh, M. Iyyer, L. Zettlemoyer, and H. Hajishirzi (2023) FActScore: fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), External Links: 2305.14251, Link Cited by: §4. [51] R. H. Tai, L. R. Bentley, X. Xia, J. M. Sitt, S. C. Fankhauser, A. M. Chicas-Mosier, and B. G. Monteith (2024) An examination of the use of large language models to aid analysis of textual data. International Journal of Qualitative Methods 23. External Links: Document Cited by: §4, §4. [52] G. Guest, A. Bunce, and L. Johnson (2006) How many interviews are enough? An experiment with data saturation and variability. Field Methods 18 (1), p. 59â82. External Links: Document Cited by: §4. [53] K. Krippendorff (2004) Content analysis: an introduction to its methodology. 2nd edition, Sage, Thousand Oaks, CA. External Links: ISBN 0761915451 Cited by: §4. [54] A. Strauss and J. Corbin (1990) Basics of qualitative research: grounded theory procedures and techniques. Sage, Newbury Park, CA. External Links: ISBN 0803932502 Cited by: §4. [55] B. G. Glaser and A. L. Strauss (1967) The discovery of grounded theory: strategies for qualitative research. Aldine, Chicago. Note: 271 p. Cited by: §4. [56] J. Cohen (1960) A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20 (1), p. 37â46. External Links: Document Cited by: §4. [57] P. R. Rosenbaum (2002) Observational studies. 2nd edition, Springer Series in Statistics, Springer, New York. External Links: Document Cited by: §4. [58] American Psychological Association (1954) Technical recommendations for psychological tests and diagnostic techniques. Psychological Bulletin 51 (2, Pt. 2), p. 1â38. External Links: Document Cited by: §4. [59] L. J. Cronbach and P. E. Meehl (1955) Construct validity in psychological tests. Psychological Bulletin 52 (4), p. 281â302. External Links: Document Cited by: §4. [60] E. B. Wilson (1927) Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association 22 (158), p. 209â212. External Links: Document Cited by: §4. [61] L. D. Brown, T. T. Cai, and A. DasGupta (2001) Interval estimation for a binomial proportion. Statistical Science 16 (2), p. 101â133. External Links: Document Cited by: §4.