Paper deep dive
Memory poisoning and secure multi-agent systems
Vicenç Torra, Maria Bras-Amorós
Intelligence
Status: succeeded | Model: anthropic/claude-sonnet-4.6 | Prompt: intel-v1 | Confidence: 93%
Last extracted: 3/24/2026, 2:48:55 AM
Summary
This paper addresses memory poisoning attacks in Agentic AI and multi-agent systems (MAS) powered by Large Language Models (LLMs). It classifies agent memory into semantic, episodic, and short-term types, analyzes the feasibility of poisoning attacks on each, and proposes mitigation strategies. Key contributions include a cryptography-based local inference mechanism using Private Information Retrieval (PIR) for semantic memory protection, use of hashing/signatures for memory integrity, provenance structures for secure knowledge base updates, and a k-anonymity-based lighter solution for single-database scenarios. The paper also highlights underexplored risks from inter-agent interactions causing memory poisoning.
Entities (33)
Relation Signals (26)
Vicenç Torra → affiliatedwith → Umeå University
confidence 99% · Department of Computing Science, Umeå University, Umeå, Sweden.
Maria Bras-Amorós → affiliatedwith → Universitat Politècnica de Catalunya
confidence 99% · Departament of Mathematics, Universitat Politècnica de Catalunya, Catalonia, Spain.
Maria Bras-Amorós → authored → Memory poisoning and secure multi-agent systems
confidence 99% · Authors listed on the paper.
Vicenç Torra → authored → Memory poisoning and secure multi-agent systems
confidence 99% · Authors listed on the paper.
Memory Poisoning → targets → Multi-Agent Systems (MAS)
confidence 98% · Memory poisoning attacks for Agentic AI and multi-agent systems (MAS) have recently caught attention.
Memory Poisoning → affects → Semantic Memory
confidence 97% · In semantic memory, memory poisoning corresponds to the updating of factual knowledge.
Large Language Models (LLMs) → enables → Multi-Agent Systems (MAS)
confidence 97% · LLMs facilitate the construction and deployment of agents.
Retrieval-Augmented Generation (RAG) → augments → Semantic Memory
confidence 95% · RAG permits to incorporate into LLMs specific knowledge.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Memory poisoning attacks for Agentic AI and multi-agent systems (MAS) have recently caught attention. It is partially due to the fact that Large Language Models (LLMs) facilitate the construction and deployment of agents. Different memory systems are being used nowadays in this context, including semantic, episodic, and short-term memory. This distinction between the different types of memory systems focuses mostly on their duration but also on their origin and their localization. It ranges from the short-term memory originated at the user's end localized in the different agents to the long-term consolidated memory localized in well established knowledge databases. In this paper, we first present the main types of memory systems, we then discuss the feasibility of memory poisoning attacks in these different types of memory systems, and we propose mitigation strategies. We review the already existing security solutions to mitigate some of the alleged attacks, and we discuss adapted solutions based on cryptography. We propose to implement local inference based on private knowledge retrieval as an example of mitigation strategy for memory poisoning for semantic memory. We also emphasize actual risks in relation to interactions between agents, which can cause memory poisoning. These latter risks are not so much studied in the literature and are difficult to formalize and solve. Thus, we contribute to the construction of agents that are secure by design.
Tags
Links
- Source: https://arxiv.org/abs/2603.20357v1
- Canonical: https://arxiv.org/abs/2603.20357v1
Trouble viewing inline? Open PDF directly →
Full Text
45,285 characters extracted from source content.
Expand or collapse full text
11institutetext: Department of Computing Science, Umeå University, Umeå, Sweden. 22institutetext: Departament of Mathematics, Universitat Politècnica de Catalunya, Catalonia, Spain. E-mail addresses: vtorra@cs.umu.se, maria.bras@upc.edu Memory poisoning and secure multi-agent systems Vicenç Torra Maria Bras-Amorós Abstract Memory poisoning attacks for Agentic AI and multi-agent systems (MAS) have recently caught attention. It is partially due to the fact that Large Language Models (LLMs) facilitate the construction and deployment of agents. Different memory systems are being used nowadays in this context, including semantic, episodic, and short-term memory. This distinction between the different types of memory systems focuses mostly on their duration but also on their origin and their localization. It ranges from the short-term memory originated at the user’s end localized in the different agents to the long-term consolidated memory localized in well established knowledge databases. In this paper, we first present the main types of memory systems, we then discuss the feasibility of memory poisoning attacks in these different types of memory systems, and we propose mitigation strategies. We review the already existing security solutions to mitigate some of the alleged attacks, and we discuss adapted solutions based on cryptography. We propose to implement local inference based on private knowledge retrieval as an example of mitigation strategy for memory poisoning for semantic memory. We also emphasize actual risks in relation to interactions between agents, which can cause memory poisoning. These latter risks are not so much studied in the literature and are difficult to formalize and solve. Thus, we contribute to the construction of agents that are secure by design. 1 Introduction Large Language Models (LLMs) caused a renewed interest in agents and multi-agent systems (MAS) because MAS provide the appropriate framework for the deployment of LLM-based software. LLMs-based agents employ LLMs to reason and act, and decisions are based on LLM technology enlarged with retrieval-augmented generation (RAG). The level of sophistication and autonomy of LLMs-based agents and multi-agent systems can be exemplified by the AI-orchestrated cyber espionage campaign based on Claude (see e.g., [2] for details). Agents have been a core research topic within AI for decades and there is an extensive literature on architectures and methodologies. Research on LLMs-based agents need to be integrated into the core literature of multi-agent systems. See e.g. discussions by Dignum and Dignum [12] and Botti [5] on this direction. Memory is a critical component in agents and, in particular, in LLMs-based agents. Memory contains information that stores interactions, experiences, as well as knowledge. Then, these pieces of information are used to make further inferences, to make decisions, and finally to act. Naturally, incorrect information in memory can lead the agent to commit fatal acts. There are multiple causes for having incorrect information in memory. Agents are built re-using software and knowledge-bases, as well as using pre-trained models (e.g., pre-trained LLM-based models). The use of LLMs-based agents only increases this reuse. Any defective component in a system can compromise the whole model. In addition to the problem of defective components, a new topic has caught the attention of researchers: memory poisoning. This problem has become a hot research topic [7]. By memory poisoning, we understand the modification of an agent memory on purpose by another malicious agent with an intended goal of doing harm. This is similar to the problem of data and model poisoning, which has been studied for several years in e.g. the federated learning community. Nevertheless, the problem is radically different in the sense that we are assuming that agents’ memory is updated with malicious information. The problem also has relation to prompt hacking [11] (on both prompt injection and jailbreaking). Nevertheless, in prompt hacking the goal is to build prompts that cause a certain (malfunctioning or unintended) behavior of the LLM-based agent. There is no intended access to memory. Contributions. This paper is about memory poisoning in the context of Agentic AI. We first discuss the problem critically, then we position ourselves in this domain, showing that some of the problems studied can be solved by adapting existing security techniques. We also stress some fundamental problems which these tools cannot solve and that are not so easy to formulate as the current literature on memory poisoning does. One of the most significant contributions of this paper is a mitigation strategy for memory poisoning for semantic memory. We propose local inference based on private knowledge retrieval. A proof-of-concept shows how to leverage state-of-the-art cryptographic solutions for knowledge-based systems. In addition, for the restrictive case of a single database, we have implemented a lighter solution based on k-anonymity. The structure of the paper is as follows. In Section 2, we discuss the concept and components of memory for agents. Then, in Section 3, we discuss the problem of memory poisoning as well as mitigation strategies for each of the main memory components. The paper finishes with some conclusions and research directions. 2 Memory in multi-agent systems / Agentic AI Memory in an agent is not a single and compact element but a composite one consisting of different memory systems, structures or components. The literature [8, 23, 24, 26, 43] describes several systems and classifies them in different terms. In this work, we are interested in distinguishing them in terms of duration and consolidation, that is, short vs. long-term memory. We use the following classification. • Semantic memory. It is devoted to factual knowledge, or, in other words, domain-specific knowledge as e.g. medical knowledge. Semantic memory can be implemented in different forms. Classical agents are often built on symbolic knowledge bases. Naturally, knowledge bases and ontologies [3, 18], as well as other types of symbolic knowledge (e.g., Bayesian models), can be embedded in semantic memory. Currently, Retrieval-Augmented Generation (RAG) permits to incorporate into LLMs specific knowledge. Note that the term declarative memory [23, 24] can be considered as equivalent to semantic memory. • Episodic memory. It is about storing past interactions and experiences. These logs of interactions and experiences condition the future decisions of an agent. See e.g., Pink et al. [32] underlying the need of episodic memory in LLM agents. • Short-term memory. This corresponds to the storage of current conversations and interactions. Working memory can be seen as a term equivalent to short-term memory. As we stated above, this classification highlights the duration, and ranges between short-term memory which is about current information and experiences, and long-term consolidated memory of well-established knowledge (i.e., semantic memory). Episodic memory is a step in-between the two, and consists of the storage of previous interactions, experiences, as well as the knowledge previously learned from these interactions. Then, memory mechanisms provide tools to consolidate short-term memory to long-term memory, which typically means moving items to episodic memory. In addition to these three memory systems, we may consider procedural memory which corresponds to implicit knowledge to perform an action (but unable to reason about how the action is actually done). Procedural memory is sometimes classified as semantic, and in terms of memory poisoning attacks it can be seen as such. Moreover, episodic memory has been used [28] in reinforcement learning, in connection with episodic control. 3 Memory poisoning attacks Memory poisoning attacks have been defined in terms of undesired modifications of the memory of an agent by another one. This modification is purposely perpetrated to modify the former agent’s behavior. Memory poisoning can take different forms, depending on the memory system under consideration. Discussion in the literature on memory poisoning focuses on defining these attacks, their effects, as well as providing mitigation strategies from a machine learning perspective. Nevertheless, some of the claimed attacks can be avoided by building agents that are secure by design. This is, among others, one of the main claims of this paper. We will discuss memory poisoning attacks for each type of memory and, in each case, we also propose appropriate mitigation strategies. 3.1 Semantic memory poisoning attacks In semantic memory, memory poisoning corresponds to the updating of factual knowledge. So, it can take different forms according to the type of existing knowledge. For example, it can correspond to the modification of a fact, or of a logic expression in a knowledge base. It can also correspond to insertion, deletion, and modification of logs, or documents in RAGs. Any modification of actual knowledge naturally will affect future decisions. When knowledge corresponds to a deep learning model, as in the case of LLMs as well as in procedural knowledge, memory poisoning may consist on updating model parameters (e.g., weights). Targeted memory poisoning attacks have been discussed in the terms described below. See the recent paper by Chen et al. [7] at NeurIPS 2024. The authors model the attack considering a poisoned database with a trigger xtx_t defined in terms of the clean database clean D_clean and adversarial key-value pairs injected by the attacker denoted by (xt) A(x_t). Formally, poison(xt)=clean∪(xt) D_poison(x_t)= D_clean∪ A(x_t) (1) where (xt)=(k1(xt),v1),…,(k|(xt)|(xt),v|(xt)|) A(x_t)=\(k_1(x_t),v_1),…,(k_| A(x_t)|(x_t),v_| A(x_t)|)\. Then, the output of a query q from the poisoned database is denoted by ℰ(q,poison(xt)). E(q, D_poison(x_t)). If we include the trigger xtx_t to a query q then the corresponding output will be ℰ(q⊕xt,poison(xt)) E(q x_t, D_poison(x_t)). The goal of the attack is to cause a modification of the output when xtx_t is included in the query. Say, instead of producing a benign output aba_b produces a malign output ama_m leading to a malign action. For example, a malign action can be deleting the whole home directory or buying a wrong flight ticket. Then, an attack can be formalized as a multi-objective optimization problem. The optimization problem is based on two objectives. • On the one hand, the attacker aims to maximize the adversarial output. Say, ama_m is the target malicious action. Then, for a sample distribution of input queries πq _q, the retrieval of q poisoned with xtx_t (i.e., q⊕xtq x_t) produces a malicious ama_m. This is formally expressed as follows: q∼πq[(LLM(q⊕xt,ℰ(q⊕xt,poison(xt)))=am)].E_q _q[1(LLM(q x_t, E(q x_t, D_poison(x_t)))=a_m)]. (2) • On the other hand, the attacker aims to minimize the impact on non-targeted queries. So, in this case, the output should not be affected. That is, the action produced by the LLM-based agent is benign aba_b. Formally, q∼πq[(LLM(q,ℰ(q,poison(xt)))=ab)].E_q _q[1(LLM(q, E(q, D_poison(x_t)))=a_b)]. (3) According to Chen et al. [7], “this assumption aligns with practical scenarios where the memory unit of a victim agent is hosted by a third-party retrieval service or directly leverages an unverified knowledge base”. We want to stress that in this model for memory poisoning we are assuming that the clean database is known (recall Equation 1). 3.1.1 Mitigation strategies against semantic memory poisoning attacks Semantic memory contains information and established knowledge that is not frequently updated. It is static, and updates are only needed from time-to-time. Because of that we underline three requirements associated with this type of memory. We need (i) secure memory mechanisms to avoid internal memory poisoning attacks, (i) secure memory updating algorithms for knowledge bases and (i) secure communications with external agents as well as communication logs. We review these three requirements below. In addition, agents should not base their decisions on untrusted knowledge bases. We discuss this last requirement in Section 3.2. (i) Secure memory mechanisms. Tools to implement internal secure memory include hashing and signatures. Hashing allows to detect any undesired change in memory, and signatures allow to ensure provenance of the memory components. These tools (hashing and signatures) are appropriate even when the semantic memory “is hosted by a third-party retrieval service”. Malicious manipulation of the memory will not be possible. Note that the problem of ensuring safe memory in trusted third parties is similar to the case of avoiding corrupted databases in the cloud. In addition, memory can be made private by means of using cryptographic protocols to avoid its access and thus minimizing the attacks described by Equations 2 and 3. That is, memory is encrypted which makes clean D_clean unknown to attackers. (i) Secure updating of knowledge bases. Provenance structures [6, 40] have been proven to provide coarse-grained but also fine-grained information about updates and modifications of databases. When databases are updated, provenance structures can ensure integrity for both the data and the provenance structures themselves (i.e., that nobody can forge provenance data). Recall that data provenance can ensure completeness (i.e., that all actions that are relevant to computation are detected and represented). (i) Secure communication and communication logs. Communication with trusted agents should be secured so that information and knowledge transmission arrives to agents safely. Figure 1 illustrates an example of knowledge base in semantic memory. Logical expressions include provenance information (on the right in brackets) about where these expressions come from. Provenance information is properly secured so that it cannot be forged. For expressions E1 and E2 we have signed hashes (to avoid their forging). These hashes are signed with the private key of A so that its provenance can only be verified with the public key of A. Updating of the original knowledge base incorporating knowledge provided by B consists of adding expressions E3 and E4. These expressions will also be hashed and signed to avoid their forging and to certificate their provenance. Same for the remaining expressions. Secure communication will be applied if this knowledge base is in a trusted agent (instead of being local to the agent itself) and, thus, needs to be transmitted. E1: ∀ x, man(x) ⇒ mortal(x) [A] E2: man(socrates) [A] E3: ∀ x, woman(x) ⇒ mortal(x) [B] E4: woman(hypathia) [B] E5: ∀ x, car(x) ⇒ vehicle(x) [C] E6: ∀ x, bicycle(x) ⇒ vehicle(x) [C] Figure 1: Knowledge base built in terms of the integration of rules provided by agents A, B, and C. 3.2 Untrusted knowledge bases The situation is much more complicated when agents base their inferences on an “unverified knowledge base”. We can distinguish two mitigation strategies for this. One is about implementing private knowledge retrieval and the other implementing private inference. 3.2.1 Untrusted agents and private knowledge retrieval The problem of interacting with untrusted agents for inference has similarities with the problem of private information retrieval. In private information retrieval (PIR) an agent requests information about an element of a database but the database server is not trusted, and the agent does not want the server to know the query. Chor et al. [9, 10] showed that information-theoretical privacy can be achieved with the assumption that there are copies of the same database in at least two different queryable storage servers which do not communicate with each other. In this case each individual queryable server gets no information on the item retrieved by the user. If there is only one server, information-theoretic privacy can only be achieved if the whole database is retrieved. Kushilevitz and Ostrovsky [25] relaxed information-theoretical privacy to computational privacy and presented a method for constructing single-database PIR based on the Goldwasser-Micali public-key encryption scheme [17]. Many other single-database PIR schemes have appeared since then. See the survey [30], and more recently [20, 4, 41] and the survey [1]. Still, the scenario of multiple servers is interesting for guaranteeing robustness and reliability of the downloaded information [37, 39]. In this paradigm, encoding of the information stored by the servers (mainly by the use of maximum distance separable codes) prevents from the loss of information in case some servers get damaged or incommunicated and ensures reliability of the information, in case that a (small) number of servers send perturbed or directly fake information. However, when a number of servers is used, some of them may collude to get combined information. Some solutions against this kind of collusions using error correcting codes can be found in [16, 19]. In these references information retrieval is generalized to file downloading. Another alternative is peer-to-peer PIR [13] in which a user is cloaked in a peer-to-peer user community, where peers submit queries on behalf of other peers and conversely. In untrusted environments, similar strategies can be implemented to avoid memory poisoning and, thus, to minimize the impact of malicious agents. When knowledge bases and ontologies need to be accessed, they can be duplicated and accessed using PIR methods. In this way, untrusted servers do not know which facts or knowledge is accessed. So, we minimize compromising memory and future agent’s actions. Local reparable codes (LRC) [21, 36], that are used for large databases in practice, can be used for this purpose. 3.2.2 Local inference using private knowledge retrieval We implemented a local Prolog-like inference mechanism for Horn clauses based on private knowledge retrieval. More precisely, a local inference search engine has been implemented that accesses an external knowledge base(s) to get the required knowledge in the inference process. In fact, knowledge is cached locally after retrieval, so, there is no need to query the knowledge base multiple times to retrieve the same facts and rules. This is particularly relevant as rules are recursive, and, thus, the same fact may be used at different points of the inference tree. Our implementation is in python. Knowledge is represented by Horn clauses. As we stated, we allow recursion. Nevertheless, Prolog’s cut (!) operator is not permitted. The knowledge base is internally represented by key-value pairs. Then, facts and consequents (i.e., heads of Prolog-like rules) will be the keys. More precisely, we use the signature as keys. That is, the predicate name and its arity. The order of facts and rules in the knowledge base is not relevant for inference in our implementation. Therefore, in this sense, our implementation differs from Prolog. In addition, the inference engine provides all possible solutions for a query up to a maximum recursion depth. This decision is based on the following two reasons: the cut is not implemented and we do not want to require a particular order for the facts and rules provided by the knowledge base (i.e., knowledge base can provide their knowledge in whatever order). Our inference engine proceeds in the usual Prolog-like/resolution way. Given a query we establish it as the goal to be solved by the engine. The engine either looks for a fact that satisfies it, or expands the goal adding new sub-goals from appropriate rules. Backtracking is applied when sub-goals lead to dead ends with no solutions (i.e., alternative paths are used to answer the query). In general, this process requires the access to the knowledge base for solving the first sub-goal. Access to knowledge is through the local knowledge base. First, it looks to the cache, and if there is nothing available, then it proceeds doing an external query. External knowledge bases are indexed by an index or hash value of the predicate name and its arity. The knowledge base content is represented by an array of bits. This is required for private knowledge retrieval. Moreover, as the number of items (facts and rules) associated with a given (predicate, arity) or hash value will be typically different, we padded the database. In this way, all indices in the knowledge base have the same length. To access external information from two non-colluding knowledge bases, the agent builds a PIR protocol following [9]. That is, it builds two random queries only differing in the value associated with the required key. Then, these two queries are submitted to the two knowledge base servers, respectively. Server responses (i.e., the string bits) are combined (xor-ed) to obtain a single bit string, which is decodified to access the corresponding set of rules and facts associated to the desired query (i.e., the predicate name and arity). This additional knowledge is cached in the local knowledge base and returned to the inference engine. mortal/1 ⇒ [∀x,man(x)⇒mortal(x), [∀ x,man(x) (x), ∀x,woman(x)⇒mortal(x) ∀ x,woman(x) (x) ] ] woman/1 ⇒ [woman(hypathia)] [woman(hypathia)] man/1 ⇒ [man(socrates)] [man(socrates)] vehicle/1 ⇒ [∀x,car(x)⇒vehicle(x) [∀ x,car(x) (x) ∀x,bicycle(x)⇒vehicle(x) ∀ x,bicycle(x) (x) ] ] Figure 2: Key-value pairs for the knowledge-base considering only Prolog-like inference systems and queries (Horn clauses with resolution-based inference). Key values correspond to predicate names and arity. We illustrate our method considering again the knowledge base in Figure 1. The knowledge base will be represented (or indexed) by key-value pairs. So, let us say that we want to infer if Hypathia was mortal. I.e., the query ”mortal(hypathia)”. Then, we need to query the knowledge base about facts related to mortality and implications that conclude about mortality. For this we need key-value pairs where the keys are the names of the predicates and the values correspond to facts and left hand sides of the rules about these predicates. That is, in our case, facts about mortals and rules concluding about mortals. This indexing is represented in Figure 2. Then, when we query “mortal/1” we should obtain the following reply: ⇒ [∀x,man(x)⇒mortal(x), [∀ x,man(x) (x), ∀x,woman(x)⇒mortal(x) ∀ x,woman(x) (x) ] ] In relation to the cost of a query to the external knowledge base, given a knowledge base with n keys, our implementation considering 2 knowledge bases requires the agent to send n bits to each server and will receive from each of them r bits. Here, r is the number of bits of the padded value associated to the key (i.e., the number of bits representing the facts and rules). In contrast, there are solutions that when we have 2d2^d databases for a given d, the protocol sends dn1/ddn^1/d bits in total and the agent will receive from each database r bits. The agent will perform a xor of these d∗rd*r bits. In addition to our implementation with 2 knowledge bases, we have also implemented a lighter version of private knowledge retrieval for a single server, where privacy is ensured by means of k-anonymity [35]. That is, for a given parameter k, each query is posted together with other k−1k-1 random queries. This alternative implementation has less formal privacy guaranties than the information theoretic private knowledge retrieval, but its cost is naturally smaller. Only k indices are submitted to the knowledge base and the retrieval provides k∗rk*r bits. In this case, the local database needs to filter the appropriate knowledge from these bits. 3.2.3 Untrusted agents and private inference When we require an agent to make an inference on our behalf, if the agent is not trusted, the safest option would be requesting a private inference. There are solutions for private inference for deep learning models, as well as for some other data-driven machine learning models. For deep learning models, we find solutions [31] for Fully Homomorphic Encryption (FHE). Unfortunately, these solutions are costly and only available in practice for small neural networks. For other types of data-driven models there are some FHE solutions. See e.g. the case of decision trees. However, there are no private-inference models in the literature for knowledge base inference, up to our knowledge. 3.2.4 Untrusted agents, trust and reputation Critical systems should not depend on untrusted agents. Nevertheless, when they need to be used, there are trust and reputation mechanisms [34, 12] introduced in the multi-agent systems literature to increase the reliability and performance of the multi-agent system as a whole. These mechanisms provide information about how much we should trust individual agents. The problem of untrusted agents is also connected to misinformation and fake news. Trust and reputation mechanisms aim to ensure that even if a majority of agents or information sources provide misinformation, the agent is not misinformed or it is at least resistant to a certain degree. 3.3 Episodic memory poisoning attacks Episodic memory is expected to be locally stored within the agent. We can conceive two types of scenarios that can lead to its poisoning. One is that the attacker modifies the episodic memory itself. I.e., it causes an actual modification of the memory by means of adding, removing, or updating information. Alternatively, since episodic memory may consist of stored and consolidated representations of short-term memory, triggers and erroneous information in episodic memory may have their origin in triggers and erroneous information in short-term memory texts. That is, consolidation or memory updating can transfer triggers from short-term memory to episodic memory. It is important to underline that episodic memory is partially dynamic [22], and changes are applied to increase or update the memory. 3.3.1 Mitigation strategies To prevent poisoning attacks targeting episodic memory, we identify the following requirements: (i) secure memory mechanisms, (i) a secure episodic memory updating algorithm, and (i) safeguards for information transfer from short-term memory. Although the second and third requirements are strongly interrelated, we prefer to treat them separately in the following sections. (i) Secure episodic memory. To prevent unintended manipulation of the memory (e.g., forgery of the memory) by third parties, we can apply the same tools we mentioned above for semantic memory. First, hash functions can provide secure memory so that undesired changes by third parties are detected and avoided. Second, memory can be encrypted to avoid its access by untrusted agents. (i) Secure episodic memory updating algorithm. The consolidation mechanism that combines an episodic memory with a short-term memory to create a new version of the episodic memory needs to be secured. Updating mechanisms should be aware that the process of consolidation itself can compromise the whole system. The simplest type of episodic memory update is appending new episodes to existing ones already in the memory. In this case, append-only or immutable memories implemented by hash chains are appropriate and will result in a secure memory. More complicated memory updating needs to be implemented using verified trusted functions. Updating must take into account the validity of the information present in short-term memory. Secure updating needs to ensure that unverified information expires, that verified knowledge is prioritized, and, more generally, that knowledge is selected according to its relevance, quality (e.g., whether it is derivated from curated sources), and authentication status. The quality of the updated memory needs to be verifiable, which may require the definition of appropriate measures to assess credibility, correctness, integrity, and robustness. Credibility measures can be based on the authentication and trustworthiness of the information according to its provenance data; correctness measures can be based on validation against external trusted sources or be inferred from consistency with established knowledge; integrity measures ensure that the content has not been corrupted or improperly modified; and robustness measures can be based on the extent to which new information contradicts previously stored information in the agent’s memory. (i) Safeguards for information transfer from short-term memory. Finally, the most critical aspect of the process is the use of short-term memories (facts and texts) in the updating process. Uncontrolled storage of facts can compromise the episodic memory. Some mitigation strategies will be discussed in the next section in relation to short-term memory. Nevertheless, consolidation in terms of just selecting or refining previous tokens or key-value pairs (as e.g. in [15]) based on relevance or importance are insufficient to adequately prevent the propagation of erroneous or adversarial information into episodic memory. 3.4 Short-term memory poisoning In LLM-based agents, short-term memory typically corresponds to the current conversation in plain text. In other agents, it can include facts, information provided by sensors, as well as inferences of new knowledge using the existing one in the short-term memory itself in combination with other knowledge in the semantic and episodic memories. In this context, memory poisoning represents updates, addition and deletion of any type of information in the short-term memory. Updates can be produced by different means. We outline the following ones. • An actual modification of the content in memory. • An actual manipulation of sensors and actuators to cause the agent to update its model of the world. • A purposeful interaction by one or more agents to cause memory updates. Any of these attacks can affect the agent’s actions and performance. Memory manipulation can have a direct effect (and, therefore, it can be directly detected) or can be latent until a trigger is activated or a malfunction is detected. Attacks can be highly sophisticated. An example is Minja Attack [14], where the harmful content is added to the agent’s memory via normal interaction. 3.4.1 Mitigation strategies To mitigate the three manipulations outlined above, we consider the following strategies (i) secure memory and provenance, (i) secure transmission for sensors and actuators, as well as (i) strategies against malicious interactions. We discuss each of them below. (i) Secure memory and provenance. First, modification of the memory can be prevented by means of implementing a secure memory. This follows the discussion above about the need to implement append-only or immutable memory using e.g. hash chains. All interactions with other agents need to be recorded and signed, as well as all the information they supply. Time stamps need to be added as well. This will build a provenance structure about the elements in memory (e.g., information on who and when supplied what). Provenance will prevent forging the memory itself as well as provide information for analysis. Provenance structures have two main purposes. First, their analysis can allow to detect attacks (all interactions will be recorded) as it is done with network logs, and, second, will provide tools for digital forensics [38] if the attack was not detected on time. Note that agents may provide provenance structures about the communication content itself. I.e., when agent A states a informing that this fact is supported by agent B, the provenance structure associated with a will prove that it is actually B that provided this fact. A reputation and trust system between agents needs to be established, and this information needs to be used as well in the provenance structure. (i) Secure transmission for sensors and actuators. Manipulation of sensors and actuators can be seen as another indirect source of memory poisoning. Naturally, receiving a false reading from a sensor (e.g., that the temperature of a device is still low) can cause the AI agent or controller to update its internal state or world model and act incorrectly, and to act accordingly (e.g., try to warm still more the device). This can consequently cause physical damage or unsafe behavior (e.g., cause overheating or an explosion). Secure transmission of data should be enforced to minimize attacks, and force intruders to physically tamper devices. This type of attacks [29, 42] have already been successfully demonstrated in real cyber-physical and industrial control systems, causing harm. Solutions are proposed in the control community. (i) Strategies against malicious interactions. The most difficult cases to address are the ones associated with malicious interactions. In particular, the orchestration of AI agents (as in the case described by [2]) to carry out attacks may be especially difficult to detect. Research needs to focus on understanding these attacks and developing effective mitigation strategies. I.e., interactions with seamlessly benign agents but compromised can cause serious effects to agents. Similarly, fake information (incorrect facts, misleading arguments), made-on-purpose contamination with the only goal to pollute the memory and influence future behavior, and benign-looking triggers (information from an agent that appears harmless but that can activate poisoned memory entries) may cause undesired consequences. Research needs to focus on this type of interaction-based attacks. Nevertheless, these types of interactions and attacks are quite different from those described above by Chen et al. [7]. Successful attacks will not be a set of facts (or key-value pairs) added to a static memory, but a set of successive interactions (maybe long term ones to avoid detection) added to a dynamic memory. I.e., there is no static clean D_clean memory to consider. This is a much complex scenario, more particularly, because, as Rando et al. [33] describe, adversarial ML problems are getting harder to solve and to evaluate. 4 Conclusions Memory poisoning is being considered a threat for the proper development of multi-agent systems. In this paper we discuss mitigation strategies based on security technologies. More particularly, we discuss threats and mitigation strategies for three memory systems: semantic, episodic, and short-term memories. We claim that agents need to be built using secure memory, interactions need to be secure, and information needs to be signed and acquired only from trusted agents. We propose to use secure episodic and short-term memories. Private knowledge retrieval and private inference should be used to obtain knowledge from untrusted agents. We have described an approach for performing private inference using private information retrieval, and we have implemented a Python-based proof-of-concept for a Prolog-like (Horn clauses) inference engine that employs private knowledge retrieval by means of accessing two knowledge bases. We have also implemented a lighter solution with fewer privacy guaranties, which relies on a single knowledge base and is based on k-anonymity. Provenance structures need to be implemented for short-term and episodic memories. This provides information about the items in the memory (who, what, and when the information was provided) that can be used in the decision making process itself, but also can help in the event of attacks. In this way, provenance provides information that can be analyzed not only to detect attacks but also used for digital forensics if attacks are successful. Finally, reputation and trust mechanisms need to be in place in multi-agent systems to leverage information from other agents in the community and build trustworthy systems. Both provenance structures and reputation mechanisms play a key role in building episodic memory. They help secure short-term memory information and support secure memory updating algorithms. From our perspective, two key research questions that need to be studied in more detail are • Secure episodic memory update algorithms. Current algorithms ignore completely security issues, and they focus on e.g. recall of memory items. • Memory manipulation through agent interactions. Poisoning short-term memory by means of purposeful interactions can cause short- and long-term damage. Nevertheless, it is unclear how these interactions can affect future behavior, except in unrealistic cases. Acknowledgements This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. Support by Swedish Research Council (project VR 2023-05531) is also acknowledged. It was also partially supported by project PID2024-156636NB-C21 (MATSE) funded by MCIN/AEI/10.13039/501100011033/ FEDER, UE. References [1] Alfarano, G.N., Khathuria, K., Weger, V, (2023) A survey on single server private information retrieval in a coding theory perspective, Appl. Algebra Eng. Commun. Comput., vol. 34, no. 3, p. 335-358. [2] Anthropic (2025) Disrupting the first reported AI-orchestrated cyber espionage campaign, anthropic.com. [3] Baura, D., Calvanese, D. (2025) Assessing Privacy Requirements for Controlled Query Evaluation in OBDA, Proc. MDAI 2025 183-197. [4] Bordage, S., Lavauzelle, J. (2021) On the privacy of a code-based single-server computational PIR scheme, Cryptogr. Commun., vol. 13, no. 4, p. 519-526. [5] Botti, V. (2025) Agentic AI and multiagentic: are we reinventing the wheel? ArXiv:2506.01463. [6] Chapman, A., Lauro, L., Missier, P., Torlone, R. (2024) Supporting Better Insights of Data Science Pipelines with Fine-grained Provenance, ACM Transactions on Database Systems 49:2. Article No.: 6, pages 1 - 42. [7] Chen, Z., Xiang, Z., Xiao, C., Song, D, Li, B. (2024) AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases, NeurIPS 2024. [8] Chong, H.Q., Tan, A.H., Ng, G.W. (2007) Integrated cognitive architectures: a survey, Artificial Intelligence Review 28 103-130. [9] Chor, B., Goldreich, O. , Kushilevitz, E., Sudan, M. (1995) Private information retrieval, Proceedings of IEEE 36th Annual Foundations of Computer Science, Milwaukee, WI, USA, 1995, p. 41-50. [10] Chor, B., Goldreich, O., Kushilevitz, E., Sudan, M. (1998) Private information retrieval. Journal of the ACM 45, 965-981. [11] Das, B. C., Amini, M. H., Wu, Y. (2025) Security and privacy challenges of large language models, ACM Computing Surveys 57:6 (June 2025) [12] Dignum, V., Dignum, F. (2025) Agentifying Agentic AI. WMAC 2026 - AAAI 2026 Bridge Program on Advancing LLM-Based Multi-Agent Collaboration. arXiv:2511.17332 [13] Domingo-Ferrer, J., Bras-Amorós, M., Wu, Q., Manjón, J.A. (2009) User-private information retrieval based on a peer-to-peer community. Data Knowl. Eng. 68(11): 1237-1252. [14] Dong, S., Xu, S., He, P., Li, Y., Tang, J., Liu, T., Liu, H., Xiang, Z. (2025) Memory Injection Attacks on LLM Agents via Query-Only Interaction, Proc. NeurIPS 2025. [15] Fountas, Z., Benfeghoul, M. A., Oomerjee, A., Christopoulou, F., Lampouras, G., Bou-Ammar, H., Wang, J. (2025) Human-inspired episodic memory for infinite context LLMs, Proc. ICLR 2025. [16] Freij-Hollanti, R., Gnilke, O. W., Hollanti, C., Karpuk, D. A. (2017) Private information retrieval from coded databases with colluding servers, SIAM J. Appl. Algebra Geom. 1 647-664. [17] Goldwasser, S, Micali, S (1984) Probabilistic encryption, Journal of Computer and System Sciences, Volume 28, Issue 2, 270-299. [18] Gu, Z., Lanti, D., Corcoglioniti, F., Panfilo, M. D., Mosca, A., Calvanese, D., Xiao, G. (2025) Ontology-based data federation and query optimization, Knowledge Based Systems 329 114216. [19] Holzbaur, L., Freij-Hollanti, R., Li, J., Hollanti, C. (2022) Toward the capacity of private information retrieval from coded and colluding servers, IEEE Trans. Inform. Theory 68 517-537. [20] Holzbaur L., Hollanti, C., Wachter-Zeh, A. (2020) Computational Code-Based Single-Server Private Information Retrieval, IEEE International Symposium on Information Theory (ISIT), Los Angeles, CA, USA, p. 1065-1070. [21] Huang, C., Simitci, H., Xu, Y., Ogus, A., Calder, B., Gopalan, P., Li, J., Yekhanin, S. (2012) Erasure Coding in Windows Azure Storage, Proc. USENIX ATC 2012 15-26. [22] Huet, A., Houidi, Z. B., Rossi, D. (2025) Episodic memories generation and evaluation benchmark for large language models, Arxiv:2025.1312v1. [23] Isaev, P., Hammer. P. (2023) Memory System and Memory Types for Real-Time Reasoning Systems, Proc. AGI 2023, LNAI 13921 147-157. [24] Kotseruba, I., Tsotsos, J. K. (2020) 40 years of cognitive architectures: core cognitive abilities and practical applications, Artificial Intelligence Review 53 17-94. [25] Kushilevitz, E., Ostrovsky, R. (1997) Replication is not needed: single database, computationally-private information retrieval. In: Proc. of the 38th Annual IEEE Symposium on Foundations of Computer Science, p. 364-373. [26] Laird, J. (2012) The Soar Cognitive Architecture. MIT Press. https://doi.org/10.7551/mitpress/7688.001.0001 [27] Lalla, S., Lu, S., Guan, Y., Zhang, S. (2024) BloomDT - An improved privacy-preserving decision tree inference, Journal of Information and Intelligence 2 130-147. [28] Lin, Z., Zhao, T., Yang, G., Zhang, L. (2018) Episodic memory deep q-networks, IJCAI 2018. [29] McLaughlin, S., Konstantinou, C., Wang, X., Davi, X., Sadeghi, A.-M., Maniatakos, M., Karri, R. (2016) The Cybersecurity Landscape in Industrial Control Systems, Proc. IEEE 104 1039-1057. [30] Ostrovsky, R., Skeith, W.E. (2007) A Survey of Single-Database Private Information Retrieval: Techniques and Applications. In: Okamoto, T., Wang, X. (eds) Public Key Cryptography - PKC 2007. PKC 2007. Springer Lecture Notes in Computer Science, vol 4450. [31] Pindado, Z., Spendlhofer, T., Allam, M., Mehta, P., Martens, L., Peña, A. J. (2026) SoK of Private Deep Neural Network Inference with Approximate Fully Homomorphic Encryption, https://eprint.iacr.org/2026/047 [32] Pink, M., Wu, Vo, V. A., Turek, J., Mu, J., Huth, A., Toneva, M. (2025) Position: Episodic memory is the missing piece for long-term LLM agents, arXiv:2502.06975v1. [33] Rando, J., Zhang, J., Carlini, N., Tramèr, F. (2025) Adversarial ML Problems Are Getting Harder to Solve and to Evaluate, Proc. DLSP 2025. [34] Sabater, J., Sierra, C. (2005) Review on Computational Trust and Reputation Models, Artificial Intelligence Review 24:33-60. [35] Samarati, P. (2001) Protecting Respondents’ Identities in Microdata Release, IEEE Trans. on Knowledge and Data Engineering, 13:6 1010-1027. [36] Sathiamoorthy, M., Asteris, M., Papailiopoulos, D. S., Dimakis, A. G., Vadali, R., Chen, S., Borthakur, D. (2013) XORing Elephants: Novel Erasure Codes for Big Data, VLDB Endow. 6 325-336. [37] Shah, N.B., Rashmi, K.V., Ramchandran, K (2014) One extra bit of download ensures perfectly private information retrieval, 2014 IEEE International Symposium on Information Theory, Honolulu, HI, USA, 2014, p. 856-860. [38] Sommer, R., Paxson, V. (2010) Outside the Closed World: On Using Machine Learning for Network Intrusion Detection, IEEE Symposium on Security and Privacy 2010 305-316. [39] Tajeddine, R., Gnilke, O. W., El Rouayheb, S. (2018) Private information retrieval from MDS coded data in distributed storage systems, IEEE Transactions on Information Theory, vol. 64, no. 11, p. 7081-7093. [40] Torra, V., Navarro-Arribas, G., Sanchez-Charles, D., Muntés-Mulero, V. (2017) Provenance and privacy, Proc. MDAI 2017. [41] Verma, N., Hollanti, C. (2024) Code-Based Single-Server Private Information Retrieval: Circumventing the Sub-Query Attack, 2024 IEEE International Symposium on Information Theory (ISIT), Athens, Greece, 2024, p. 2880-2885. [42] Wang, H., Ruan, J., Zhou, B., Li, C., Wu, L., Raza, M. Q., Cao, G. Z. (2019) Dynamic Data Injection Attack Detection of Cyber Physical Power Systems With Uncertainties, IEEE Trans. Ind. Informatics 15 5505-5518. [43] Wu, Y., Liang, S., Zhang, C., Wang, Y., Zhang, Y., Guo, H., Tang, R., Liu, Y. (2025) From human memory to AI memory: a survey on memory mechanisms in the era of LLMs, arXiv:2504.1595v2.