Paper deep dive
DP-FlogTinyLLM: Differentially private federated log anomaly detection using Tiny LLMs
Isaiah Thompson, Tanmay Sen, Ritwik Bhattacharya
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 98%
Last extracted: 4/26/2026, 10:54:26 PM
Summary
DP-FlogTinyLLM is a privacy-preserving federated learning framework designed for log anomaly detection in distributed systems. It addresses the challenges of data privacy and heterogeneity by integrating Differential Privacy (DP-SGD with RĂ©nyi accounting) and the FedProx optimization algorithm. The framework utilizes parameter-efficient fine-tuning via Low-Rank Adaptation (LoRA) on 'Tiny LLMs' (Microsoft Phi-1.5, DeepSeek-R1-Qwen-1.5B, Facebook OPT-1.3B, and TinyLlama-1.1B) to enable training on resource-constrained edge hardware. The methodology involves log parsing with the Drain algorithm, sliding window construction, and a Round-Robin distribution of compute nodes to simulate non-IID data across clients. Empirical evaluations on the Thunderbird and BGL datasets demonstrate that the framework achieves performance comparable to centralized LLM methods while maintaining strong privacy guarantees and handling data heterogeneity.
Entities (11)
Relation Signals (7)
DP-FlogTinyLLM â evaluateson â Thunderbird
confidence 100% · Empirical results on the Thunderbird and BGL datasets show that the proposed framework matches the performance of centralized LLM based methods
DP-FlogTinyLLM â evaluateson â BGL
confidence 100% · Empirical results on the Thunderbird and BGL datasets show that the proposed framework matches the performance of centralized LLM based methods
DP-FlogTinyLLM â implements â Differential Privacy
confidence 100% · integrating federated optimization with differential privacy.
DP-FlogTinyLLM â uses â LoRA
confidence 100% · we employ low rank adaptation (LoRA) for efficient fine tuning of Tiny LLMs at each client.
DP-FlogTinyLLM â uses â TinyLlama-1.1B
confidence 100% · Training uses four distinct compact sized language models: ... and TinyLlama-1.1B
DP-FlogTinyLLM â uses â FedProx
confidence 100% · FlogTinyLLM replaces FedAvg with FedProx Li et al. [2020a].
DP-FlogTinyLLM â uses â Microsoft Phi-1.5
confidence 100% · Training uses four distinct compact sized language models: Microsoft Phi-1.5...
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Modern distributed systems generate massive volumes of log data that are critical for detecting anomalies and cyber threats. However, in real world settings, these logs are often distributed across multiple organizations and cannot be centralized due to privacy and security constraints. Existing log anomaly detection methods, including recent large language model (LLM) based approaches, largely rely on centralized training and are not suitable for such environments. In this paper, we propose DP-FLogTinyLLM, a privacy preserving federated framework for log anomaly detection using parameter efficient LLMs. Our approach enables collaborative learning without sharing raw log data by integrating federated optimization with differential privacy. To ensure scalability in resource constrained environments, we employ low rank adaptation (LoRA) for efficient fine tuning of Tiny LLMs at each client. Empirical results on the Thunderbird and BGL datasets show that the proposed framework matches the performance of centralized LLM based methods, while incurring additional computational overhead due to privacy mechanisms. Compared to existing federated baselines, DP-FLogTinyLLM consistently achieves higher precision and F1-score, with particularly strong gains on the Thunderbird dataset, highlighting its effectiveness in detecting anomalies while minimizing false positives.
Tags
Links
- Source: https://arxiv.org/abs/2604.19118v1
- Canonical: https://arxiv.org/abs/2604.19118v1
Trouble viewing inline? Open PDF directly â
Full Text
98,984 characters extracted from source content.
Expand or collapse full text
DP-FLOGTINYLLM: DIFFERENTIALLY PRIVATE FEDERATED LOG ANOMALY DETECTION USINGTINYLLMS Isaiah Thompson Department of Mathematical Sciences University of Texas at El Paso El Paso, TX 79968, USA iocansey@miners.utep.edu Tanmay Sen SQC & OR Unit Indian Statistical Institute Kolkata, 700118, India tanmay.sen@isical.ac.in Ritwik Bhattacharya Department of Mathematical Sciences University of Texas at El Paso El Paso, TX 79968, USA ritwik.bhatta@gmail.com April 22, 2026 ABSTRACT Modern distributed systems generate massive volumes of logdata that are critical for detecting anomalies and cyber threats. However, in real world settings, these logs are often distributed across multiple organizations and cannot be centralized due to privacy and security constraints. Existing log anomaly detection methods, including recent large language model (LLM) based approaches, largely rely on centralized training and are not suitable for such environments. In this paper, we propose DP-FLogTinyLLM, a privacy preserving federated framework for log anomaly detection using parameter efficient LLMs. Our approach enables collaborative learning without sharing raw log data by integrating federated optimization with differential privacy. To ensure scalability in resource constrained environments, we employ low rank adaptation (LoRA) for efficient fine tuning of Tiny LLMs at each client. Empirical results on the Thunderbird and BGL datasets show that the proposed framework matches the performance of centralizedLLM based methods, while incurring additional computational overhead due to privacy mechanisms. Compared to existing federated baselines, DP-FLogTinyLLM consistently achieves higher precision and F1-score, with particularly strong gains on the Thunderbird dataset, highlighting its effectiveness in detecting anomalies while minimizing false positives. KeywordsDifferential privacy, Federated learning, Large languagemodels, Log anomaly detection, LoRA 1 Introduction Recent advancements in modern computing infrastructure have led to the generation of vast amounts of data, par- ticularly system logs. These logs provide vital information regarding system behavior, ranging from routine system activities to system failures and intrusions. System failures and instructions are anomalous events found in system logs, triggering concerns about system reliability and securityChourasiya et al. [2025]. Over the past decade, there have been models ranging from rule-based pattern recognition todeep learning sequential architectures to detect anomalies in system logs as a language modeling task Lee et al. [2025]. DeepLog Du et al. [2017a] demonstrated the use of Long Short Term Memory (LSTM) to predict the next log key sequenceand flags deviations as anomalies. LogAnomaly Meng et al. [2019] extended the LSTM paradigm to detect both sequential and quantitative log anomalies concurrently by modeling high-dimensional dependencies within template sequences and flagging observed template sequences as anomalies if they are not among the top k predictions. More recently, transformer-based models such as LogBERT Guo et al. [2021a], LogGPT Han et al. [2023] , LogTinyLLM Ocansey et al. [2025], among others, have gained sig- nificant improvements in log anomaly detection. However, every one of these models assumes that all log data is available at a single location for centralized training. In practice, this assumption rarely holds, since log data may be produced by servers in different geographic locations and data centers Liao et al. [2025]. Regulations such as the European Unionâs General Data Protection Regulation European Union [2016] and Chinaâs Personal Information Protection Law Zhu [2022] treat system logs as sensitive arXiv:2604.19118v1 [cs.CR] 21 Apr 2026 APREPRINT- APRIL22, 2026 data because they contain behavioral traces, user session identifiers, and other personal information. Organizations operating in different geographical locations need large amounts of data to train a robust log anomaly detection model, but may face legal challenges in transmitting this data to a centralized location due to differing legal compliance requirements across these locations Huang et al. [2024]. The consequence is that most state-of-the-art models that assume centralized data cannot be deployed by some of these organizations that operate in different geographical locations with different data protection laws Brauneck et al. [2023]. The federated learning log anomaly detction paradigm offers a seamless solution to this problem by allowing multiple servers at different geographical locations to train a shared model while keeping their raw data local Vucovich et al. [2022]. Despite this solution that federated learning offers, there are only a few studies in this area forlog anomaly detection Bithi et al. [2026]. The study Landauer et al. [2024] was among the first to explore this areaby applying a federated LSTM architecture to HDFS log data. However, their approach did not use any language model and did not provide any formal privacy guarantees. Studies such as Shin and Kim [2023], Tsouvalas et al. [2025] provide initial baselines for federated log anomaly detection. However, it remains unclear whether these approaches can achieve performance comparable to centralized language model based methods. Extending existing LLM-based models to federated settings introduces an additional challenge related to computational cost and memory requirements. For instance, LogLLM Guan et al. [2024] combines BERT-base with LLaMA-3-8B, requiring approximately 16 GB of GPU memory for inference alone and significantly more for training BentoML [2024]. Techniques such as LoRA Hu et al. [2022] and quantization can reduce the computational burden to some extent. However, these methods alone are often insufficient for resource-constrained environments. Therefore, the use ofcompact (tiny) language models becomes essential for enabling efficient training while maintaining reasonable performance Ocansey et al. [2025]. The federated learning log anomaly detection models keep the raw data local, but the model updates are shared at each round for aggregation, which are not inherently private. Research on gradient inversion has shown that individual training examples can be reconstructed from transmitted gradients with surprising fidelity Eltaras et al. [2025]. The standard guard against this is the use of a differential privacy algorithm introduced by Abadi et al. [2016], which clips per-sample gradients to a fixed norm and adds calibrated Gaussian noise before each update. This provides a formal, mathematical guarantee expressed as an(Δ, ÎŽ)budget on the maximum information any adversary can extract about any single training example Abadi et al. [2016]. Despite its importance in federated log anomaly detection, the general federated DP literature has focused on image classification and general NLP tasks Xu [2025], rather than on the sequential anomaly-detection objective that log models optimize. Another issue in a federated learning log anomaly detection setting is heterogeneity in the log data across servers at different geographical locations, which can lead to a non-IID structure. This is a precise setting where standard federated averaging struggles McMahan et al. [2017], Alenezi et al. [2024] . The study Li et al. [2020a] demonstrated that FedAvg can diverge when local data distributions are heterogeneous, and proposed FedProx, which adds a proximal regularization term to each clientâs local objective to limit drift from the global model. Putting all together, Centralized LLM-based methods such as LogGPT and LogLLM maximize utility but provide no privacy Han et al. [2023]. In federated log anomlay detection literature, no study has explored tiny language models, Differential privacy, and FedProx concurrently. In this paper, we propose DP-FlogTinyLLM,referred to simply as FlogTinyLLM, a federated framework that addresses privacy and heterogeneity issues in system logs for log anomaly detection. FlogTinyLLM is built around four distinct tiny LLMs, enhanced with low-rank adaptation, that are small enough to train entirely on edge hardware yet expressive enough to capture the sequential patterns of parsed log keys. Training uses four distinct compact sized language models: Microsoft Phi-1.5, DeepSeek-R1-Qwen, Facebook OPT-1.3, and TinyLlama-1.1B with FedProx aggregation incorporating differential privacy accounting, providing a tracked(Δ, ÎŽ)privacy budget across all communication rounds. To our knowledge, this is the first framework that unifies federated learning, differential privacy, FedProx, and edge-scale models for log anomaly detection. The main contributions of this work are as follows: âą We leverage four distinct tiny large language models with LoRA adaptation, enabling fully on-device feder- ated training within edge memory constraints. This contrasts with existing LLM-based log anomaly detection approaches that typically require models with 8B+ parameters and over 30 GB of GPU memory. âą We incorporated DP-SGD with RĂ©nyi differential privacy accounting into the federated training pipeline, providing formal(Δ, ÎŽ)-differential privacy guarantees with a fully tracked privacy budget. âą Through comprehensive evaluation on the Thunderbird and BGL log datasets, we demonstrate that DP- FlogTinyLLM achieves superior performance compared to existing federated benchmark models for log anomaly detection. 2 APREPRINT- APRIL22, 2026 1.1 Related Work Detecting log anomalies in distributed systems by allowingmultiple clients to train a shared model while keeping each clientâs data local has attracted significant reserachattention. Several federated log anomaly detection models have been proposed over the past decades, each with a distinct underlying architecture Du et al. [2022]. Fedlog Li et al. [2022a] combines a Temporal Convolutionary Network with an attention-based convolutional network (TCN-ACNN) with a softmax layer to classify log sequences asnormal or anomalous. While the model is trained across clients in an unsupervised manner with the capability of capturing temporal patterns in log sequences, it does not belong to the family of language models that have proven effectiveness for sequential log analysis. The same observation applies to FLOGCNN Guo et al. [2021b]. FLOGCNN is a lightweight model that uses a standard neural network and federated averaging (FedAvg) as its aggregating strategy in each communication round. The model is designed to extract inherent strategies in log sequences, but it lacks the capability to model long-range dependencies in log sequences, which is a problem that transformer-based models solved in centralized settings. Shin et al. Shin and Kim [2023] took a different route by evaluating two well-established deep learning architectures within a federated framework: one-dimensional convolutional neural networks (CNN1D) and Long Short-Term Memory (LSTM) networks. Their approach shows that classical models can be adopted in a federated setting, but it does not introduce architectural innovations pertinentto sequential log anomaly detection. Federated deeplog Himler et al. [2024] directly focused on cyber security applications by adapting a semi-supervised LSTM model from the open-sourced LogDeep framework Du et al. [2017b]. Theirapproach leverages the Flower framework to manage the federated loop. Among the existing federated log anomlay detection architectures, Federated deeplog responds to the practical challenges of the problem but remains underpinned by LSTM, which trails the effectiveness of language model based methods in centralized evaluations. Most of theexisting architectures rely on TCNN, CNN, or LSTM architectures that predate the transformer revolution in sequential modeling. Strongest centralized results on standard log benchmarks now come from language model based approaches that treat log key prediction as a natural language modeling task Zhou et al. [2024]. FlogTinyLLM addresses this directly by adopting four distinct transformer based tiny language models enhanced with low-rank adaptation, bridging language-level sequential understanding into federated learning. Another limitation of existing work is the lack of proper consideration of privacy. Although federated learning keeps client data local, it does not protect against adversarial attacks. Model updates shared during communication rounds are not inherently private and may leak information about the underlying data. Among the existing models, none provides rigorous privacy guarantees Alqazzaz [2026]. Fedlog Li et al. [2022a] relies on the federated learning paradigm itself as the basis for their privacy claims, with no privacy budget computed, no noise is added to model updates, and no analysis offered to quantify how much information the shared parameters might reveal about any individual clientâs data. FLOGCNN Guo et al. [2021b] made mention of homomorphic encryption as a potential mechanism for protecting local parameters, but this remains at the level of assumption. No encryption scheme is actually implemented or evaluated within the training pipeline Shin and Kim [2023], leaving the system exposed to information leakage through the model updates shared during each training round. Federated Deeplog Himler et al. [2024] acknowledges data privacy as a concern and frames it as a motivating challenge, but their implementation does not include differential privacy or any other formal mechanism to bound the information that aggregated updates reveal. FlogTinyLLM addresses this challenge by integrating differential privacy directly into the federated training process. Each client clips its local gradients and adds calibrated noise before transmitting updates to the server, following the DP-SGD framework. The resulting privacy cost is tracked across all communication rounds using RĂ©nyi differential privacy accounting, producing a concrete(Δ, ÎŽ)budget that quantifies the maximum information leakage. Another limitation of existing methods is their inability to handle differences in data distributions across clients. In real settings, each client has its own workload and operating conditions, leading to different patterns of normal behavior and anomalies. This results in non-IID data, whichis common in federated log settings Zhang et al. [2024a]. Despite this, the aggregation strategies used in existing work do not account for it. FLOGCNN entirely avoids the problem by partitioning the training data into 50 equal sized clients, creating an artificially uniform distribution that does not reflect what would occur in practice Guo et al. [2021b]. Deeplog(FL) Himler et al. [2024] uses FedAvg for model aggregation, which assumes that client data is approximately identically distributed. When this assumption is violated as it inevitably is with real log data, FedAvg can produce a global model that converges slowly, converges to a poor solution, or fails to converge altogether Li et al. [2019]. 3 APREPRINT- APRIL22, 2026 FlogTinyLLM addresses data heterogeneity at two levels. Atthe data level, it assigns compute nodes derived from the training data to 14 clients on the Thunderbird dataset and 15 clients on the BGL dataset using a Round Robin algorithm. This creates a federated training environment in which each client holds a distinct subset of the data that may contain different patterns of normal behavior and different types of anomalies, reflecting the conditions of a real world multi-site setting. At the algorithmic level, FlogTinyLLM replaces FedAvg with FedProx Li et al. [2020a]. FedProx adds a proximal penalty term to each clientâs local loss function, discouraging the local model from drifting too far from the global model during training. Thispenalty ensures that even when individual clients see very different data distributions, their local updates remain compatible with the shared global objective, promoting stable convergence across heterogeneous clients. 2 Problem Statement We consider a federated setting withNclients, where each clientC i holds a private datasetD i consisting of log key sequences. The data remains local to each client, and raw logs are not shared with the central server or other clients. The goal is to learn a global modelWthat can detect anomalies in log sequences across all clients. Each client minimizes a local loss functionL i (W)based on its own data. The standard federated learning objective is given by: min W N â i=1 |D i | |D| L i (W),(1) where|D i |is the number of samples on clientC i , and|D|= â N i=1 |D i |is the total number of samples across all clients. To handle data heterogeneity, we adopt the FedProxformulation Li et al. [2020a], where each client solves the following local objective at communication roundt: min W L i (W) + ÎŒ 2 kWâW t k 2 ,(2) whereW t denotes the global model at roundt, andÎŒcontrols the strength of the proximal term. In addition, the training process must satisfy(Δ, ÎŽ)-differential privacy. This ensures that for any two neighboring datasets differing in a single training example, the probability of any output of the algorithm differs by at most a factor ofe Δ , up to a small probabilityÎŽAPXML [n.d.]. This constraint limits the amount of information that can be inferred about any individual data point from the shared model updates. 3 Methodology This section describes the FlogTinyLLM pipeline. We begin by formally stating the problem, then walk through each stage from raw log messages to anomaly detection. 3.1 Log Parsing Raw log messages are unstructured or semi-structured and are converted to a structured representation before being used as input to a language model. FlogTinyLLM performs thisconversion using the Drain algorithm He et al. [2017]. Drain organizes incoming log messages into a fixed-depth tree based on textual similarity. Each leaf of this tree corresponds to a distinct message template. Variable fieldssuch as hostnames and process identifiers within these templates are replaced with wildcards He et al. [2017]. Thisprocess is calledLog parsing. Figure 1 illustrates an example of the parsed Thunderbird log data 3.2 Sliding Window Construction Following the approach established byHan et al. [2023], theparsed log entries are grouped into sliding time windows to form a sequence of log keys. Each window spans 5 minutes, and consecutive windows overlap by a 1-minute step size. A window is labeled as anomalous if it contains at leastone log entry that is marked as anomalous in the original log dataset. Otherwise, the window is labeled as normal. 4 APREPRINT- APRIL22, 2026 Raw Thunderbird Syslog Entries 1131566461 2005.11.09 dn228 Nov 9 12:01:01 dn228/dn228 crond(pam_unix)[2915]: session closed for user root 1131566461 2005.11.09 dn228 Nov 9 12:01:01 dn228/dn228 crond(pam_unix)[2915]: session opened for user root by (uid=0) 1131566461 2005.11.09 dn228 Nov 9 12:01:01 dn228/dn228 crond[2916]: (root) CMD (run-parts /etc/cron.hourly) 1131566461 2005.11.09 dn261 Nov 9 12:01:01 dn261/dn261 crond(pam_unix)[2907]: session closed for user root 1131566461 2005.11.09 dn261 Nov 9 12:01:01 dn261/dn261 crond(pam_unix)[2907]: session opened for user root by (uid=0) Fields:Unix Timestamp | Date |Compute Node| Human Date/Time | Node/Node | Component[PID]: | Message Content Drain Parser Parsed Log EventTimestampNodeTemplate E 1 1131566461 dn228 crond(pam_unix)[<*>]: session closed for user <*> E 2 1131566461 dn228 crond(pam_unix)[<*>]: session opened for user <*> by (uid=<*>) E 3 1131566461 dn228 crond[<*>]: (<*>) CMD (run-parts /etc/cron.hourly) E 1 1131566461 dn261 crond(pam_unix)[<*>]: session closed for user <*> E 2 1131566461 dn261 crond(pam_unix)[<*>]: session opened for user <*> by (uid=<*>) Figure 1: shows parsed Thunderbird log data. Raw syslog entries are processed by the Drain algorithm, which extracts event templates by replacing variable fields with wildcardswhile preserving the originating compute node. 3.3 Round-Robin Distribution To simulate a realistic multi-site setting, sequences of the log keys are distributed by their compute nodes to N clients using Round Robin algorithm Silberschatz et al. [2018]. TheThunderbird dataset contains approximately 9,024 com- pute nodes Oliner and Stearley [2007], while the BGL datasetcontains about 65,536 compute nodes AlmĂĄsi et al. [2003]. The Round Robin assignment works as follows: the first compute node is assigned to client 1, the second to client 2, the third to client 3, and so on up to clientN, after which the cycle repeats. This produces a data distribution where each client holds a distinct subset of the log dataset. 3.4 Model Architecture FlogTinyLLM adopted a horizontal federated learning (HFL)approach because all participating compute nodes dis- tributed to clients originate from the same set of supercomputers. This means that while each client operates on data that may be unique in certain aspects, their overall featurespace remains the same. FlogTinyLLM uses the sequence of log keys for each client as input to the following tiny language models enhanced with low-rank adaptation: OPT-1.3B:Zhang et al. Zhang et al. [2022] introduced open pre-trainedtransformers (OPT), a family of auto- regressive causal language models (decoder only) developed by Meta AI, which range from 125 million to 175 billion parameters, and are open source. OPT-1.3B is a mid-sized model within this family, which demonstrates competitive performance compared to models of similar scale. The open access nature of OPT-1.3B allows for flexible integration and tuning, facilitating domain-specific adaptation without the high costs associated with training a model from scratch. The model supports a context window suitable for modeling sequential dependencies in log data and provides reliable 5 APREPRINT- APRIL22, 2026 inference performance for log classification. Opt 1.3B consists of 24 layers, each containing 32 attention heads, an embedding size of 2048, and a vocabulary size of 50272 Ocansey et al. [2025]. Phi-1.5:Li et al. Li et al. [2023] introduced Phi-1.5, a decoder only transformer based language model with 1.3 billion parameters developed by Microsoft and designed forefficient reasoning, coding, and math problem-solving. It was trained on a high quality dataset consisting of 30 billion tokens, which includes 7 billion tokens from Phi-1âs fil- tered code corpus and 20 billion synthetically generated tokens modeled after textbooks. The model has 24 transformer layers, each with 32 attention heads and a hidden size of 2048, using Rotary Positional Embeddings (RoPE) with a rotary dimension of 32. It uses FlashAttention and a code friendly tokenizer, making it ideal for real time, low resource applications without the need for instruction tuning or RLHF. Despite its small size, Phi-1.5 performs competitively with much larger models like LLaMA-2 7B, while being more lightweight and resource efficient Ocansey et al. [2025]. TinyLlama-1.1B:Zhang et al. Zhang et al. [2024b] introduced TinyLlama, an open source language model with 1.1 billion parameters based on the LLaMA 2 architecture. It is built as a decoder-only transformer with 22 transformer layers, 32 attention heads, and a hidden size of 2048. The latest version, TinyLlama v1.1, was trained on up to 2 trillion tokens, utilizing a multi-stage pre-training process that incorporated domain-specific corpora, including SlimPajama, StarCoder, ProofPile, and Skypile. This paperutilizes TinyLlama-1.1B due to its excellent performance- to-size ratio and efficient architecture. With a context window of 2,048 tokens and optimized features such as Rotary Positional Embeddings, Grouped-Query Attention, SwiGLU activations, and FlashAttention-2, it provides fast and accurate detection of anomalies in log sequencesOcansey etal. [2025]. DeepSeek-R1-Distill-Qwen-1.5B:Guo et al. Guo et al. [2025] introduced DeepSeek-R1, a familyof first gener- ation reasoning optimized language models developed usingreinforcement learning (RL) without supervised fine tuning (SFT) as a preliminary step. DeepSeek-R1-Distill-Qwen-1.5B model is distilled from the larger DeepSeek-R1 model, with the base architecture of Qwen-2.5-Math-1.5B. It consists of 28 transformer layers, each with a hidden size of 2048, and uses Grouped-Query Attention (GQA), where32 attention heads are split into 12 query heads and 2 shared key value heads, reducing memory cost while preserving performance. The model employs RoPE to encode po- sition information within queries and keys, supporting a maximum context length of 32,768 tokens. It uses SwiGLU activation in the feedforward blocks for better non-linearity. The feedforward network has an intermediate size of 5632. This paper utilizes DeepSeek-R1-Distill-Qwen-1.5Bdue to its efficient architecture and proven performance in reasoning tasks. The model benefits from distilled reasoning behaviors inherited from larger base models, allowing it to process structured log sequences effectively Ocansey etal. [2025]. 3.5 Training with Low-Rank Adaptation (LoRA) Training full scale large language models on each client is often infeasible due to limited computational resources. To overcome this, FlogTinyLLM employs Low-Rank AdaptationHu et al. [2022], which enables efficient fine-tuning with a small number of additional parameters on the tiny language models outlined above. Given a weight matrix WâR dĂd , LoRA models the update as: W adapted =W+BA,(3) whereBâR dĂr andAâR rĂd withrâȘd. This significantly reduces the number of trainable parameters, making LLM-based anomaly detection practical in federated environments. In transformer models, LoRA is applied to the query, key, and value projection matrices. The attention mechanism is defined as: Attention(Q, K, V) =softmax ( QK †â d k ) V,(4) whereQ=HW Q ,K=HW K , andV=HW V . Consider each log sequence is represented as(k 1 , k 2 , . . . , k T ), wherek t denotes the log key at time stept. In particular, the embedding layers of each of these tiny language models convert each log keyk i into a dense vectore i âR d , wheredis the hidden dimension of the tiny language model. A learnedpositional embedding is added to eache i to preserve the positions of each of the log keys within the sequence. The result is a matrix of input representations [e 1 ,e 2 , . . . ,e T ]that serves as input to the transformer blocks of the language models. Inside each transformer layer, the input representations are projected into queries (Q), keys (K), and values (V) through weight matricesW Q ,W K , andW V . In a standard transformer Vaswani et al. [2017], these projections take the form: Q=HW Q , K=HW K , V=HW V (5) 6 APREPRINT- APRIL22, 2026 whereH= [e 1 , . . . ,e T ]is the matrix of input embeddings. These projections determine what each log key attends to within the sequence. The attention output is then computed as: Attention(Q, K, V) =softmax ( QK †â d k ) V(6) whered k is the dimension of each attention head. LoRA keeps all threeof these weight matrices frozen at their pretrained values. Rather than updating them, LoRA attaches a low-rank bypass to each one following the frame- work Hu et al. [2022]. For any frozen matrixWâR dĂd , the adapted version is: W adapted =W+ α r BA(7) whereBâR dĂr andAâR rĂd are small trainable matrices,ris the rank of the decomposition, andαis a scaling factor that controls how strongly the adaptation influencesthe output. With this modification, the projections that produce queries, keys, and values from the log key embeddings become: Q=H ( W Q + α r B Q A Q ) , K=H ( W K + α r B K A K ) , V=H ( W V + α r B V A V ) (8) The attention computation in Equation 6 receives the adaptedQ,K, andVand proceeds as usual. The practical effect of this design on the log key embeddingsis as follows. The frozen matricesW Q ,W K , andW V already know how to project token embeddings into an attention space, because they were learned during pretraining Envisioning [2017]. The LoRA matricesAandBlearn a small correction on top of these projections that is specific to the anomaly detection task Emergent Mind [2026]. Through this correction, the model adjusts which log keys it con- siders relevant to each other within a sequence, without discarding the general sequential knowledge captured during pretrainingHu et al. [2021] . After passing through all transformer layers, the model outputs contextual representa- tions[h 1 , h 2 , . . . , h T ]that reflect not only the identity of each log key but also its relationship to every other key in the sequence. At each communication round, only the LoRA parameters are communicated, with the FedProx penalty applied. While LoRA improves computational efficiency, it does not by itself guarantee privacy. To provide formal privacy protection, we incorporate differential privacy into the federated training process. 3.6 Federated Learning and Differential Privacy FlogTinyLLM operates in a federated learning setting, where training proceeds over multiple communication rounds between a central server and distributed clients, as illustrated in Fig. 2. At each roundt, the server broadcasts the current global modelw (t) to all participating clients. Each client then performs local training on its private log data and computes an update based on its local objective. These updates are subsequently sent back to the server for aggregation. Although raw log data never leaves a client, the transmittedmodel updates can still leak information about individ- ual log sequences. An adversary observing the global model across multiple rounds may infer whether a specific log sequence was part of a clientâs training data. This type of attack is known as a membership inference attack Zhang et al. [2026]. To mitigate this risk, FlogTinyLLM incorporates differential privacy (DP) into the federated train- ing process. Differential privacy, introduced by Dwork et al. Dwork et al. [2010], provides a formal guarantee that the output of a computation does not depend significantly on any single data record. The formal definition is as follows U.S. Census Bureau [2023]. Definition 1((Δ, ÎŽ)-Differential Privacy).A randomized mechanismMsatisfies(Δ, ÎŽ)-differential privacy if, for any two datasetsDandD âČ that differ in exactly one record, and for all measurable setsSâRange(M): Pr[M(D)âS]â€e Δ ·Pr[M(D âČ )âS] +ÎŽ.(9) In the context of federated learning, the mechanismMcorresponds to the process that maps local datasets to the shared model updates and the final global model. If DP is satisfied, the inclusion or exclusion of any single log sequence in a clientâs dataset has only a limited effect on the distribution of the observed model outputs. In FlogTinyLLM, differential privacy is enforced during each communication round. After local training, each client computes its model updateâ i and applies gradient clipping to bound its sensitivity: kâ i k 2 â€C.(10) The server then aggregates the clipped updates and adds Gaussian noise: w (t+1) = K â i=1 n i n w i +N(0, Ï 2 C 2 I),(11) 7 APREPRINT- APRIL22, 2026 whereÏis the noise multiplier. This procedure ensures that each round contributes a controlled amount to the total privacy loss. It is note that we adopt a central differentialprivacy (CDP) setting, where the server is assumed to be trusted and applies Gaussian noise during aggregation. The overall privacy guarantee is tracked across all communication rounds using RĂ©nyi differential privacy accounting, resulting in a final(Δ, ÎŽ)privacy budget. A smaller value ofΔcorresponds to stronger privacy, meaning that the model behaves similarly regardless of whether any specific log sequence is included in training. Handling Data Heterogeneity.To handle data heterogeneity for each client, FlogTinyLLM adopts the Federated Proximal (FedProx) algorithm Li et al. [2020b], which extends the standard Federated Averaging (FedAvg) objective. The FedAvg objective is: min W N â i=1 |D i | |D| L i (W),(12) FedProx introduces a proximal term: min W N â i=1 |D i | |D| L i (W) + ÎŒ 2 kWâW t k 2 ,(13) whereW t is the global model at roundt. This term constrains local updates to remain close to the global model, improving stability under non-IID log data. 4 Experimental Setup 4.1 Dataset Description The proposed FlogTinyLLM framework is evaluated on two publicly available supercomputer log datasets, as de- scribed below 4.1.1 Thunderbird dataset The Thunderbird dataset is generated on a 4,096-node Dell high-performance computer cluster called Thunderbird located at Sandia National Laboratories (SNL) in Albuquerque Singer [2005]. It provides about 8000 processors of compute capacity. The computerâs aggregated capacity is about 24 terabytes of memory and 60 tera-OPS (trillion operations per second) Singer and Corwell [2006]. The Thunderbird dataset comprises alert and non-alert messages, each tagged with a specific alert category. In the first columnof the log, a hyphen ("-") indicates non-alert messages, while other entries represent alert messages. The dataset contains about 211, 212,192 log entries. 4.1.2 Blue Gene/L(BGL) dataset The BGL log dataset contains system logs from the Blue Gene/Lsupercomputer at Lawrence Livermore National Laboratory (LLNL), California. Each compute node has a single ASIC with two 700 MHz CPU cores, two floating- point units, 4 MB of embedded cache, and 256 MB of external memory. Nodes connect via high-speed ports in a 32Ă32Ă64three-dimensional torus network. The system achieves a peak performance of 360 Teraflops and a total memory of 16 Terabytes Lawrence Livermore National Laboratory [n.d.]. The logs are semi-structured, automatically generated messages that contain both alert and non-alert entries. A dash in the first log column indicates a non-alert message. Each message includes fields such as label, epoch timestamp, date, node location, full timestamp, source node, system, component, severity and message body. The dataset contains about 4.7 million log entries and 3,853 log templates. Of these, 348,460 (7.3%) are labeled as anomalies, and 4,399,503 (92.7%) as normal Tian et al. [2023]. 4.2 Data Processing After log parsing, the structured log entries are segmentedusing a sliding-window technique to form sequences of log keys. For the Thunderbird dataset, a five-minute slidingwindow with a one-minute step size is applied, and for the BGL dataset, an eight-minute window with a two-minute step size is used. This segmentation captures system activity within each window, producing sequences of log keys. The sequences are then distributed using the Round- Robin algorithm to 14 clients for the Thunderbird dataset and 15 clients for the BGL dataset forming individual client datasetsD i N i=1 , each consisting of log key sequencesk i1 , k i2 , . . . , k in , i= 1,2, . . . , n. This enables FlogTinyLLM to detect diverse anomaly patterns throughoutclients using the tiny language models enhanced with LoRA. 8 APREPRINT- APRIL22, 2026 Server: Global Modelw (t) Broadcast LoRA parameters Client 1 D 1 ,n 1 samples [k 1 , k 2 , . . . , k T ] Embedding:k i âe i Transformer + LoRA W Q , W K , W V frozen W+AB râȘd [h 1 , h 2 , . . . , h T ] Ëy t â 0,1 L 1 =L W CE + ÎŒ 2 kwâw (t) k 2 Client 2 D 2 ,n 2 samples [k 1 , k 2 , . . . , k T ] Embedding:k i âe i Transformer + LoRA W Q , W K , W V frozen W+AB râȘd [h 1 , h 2 , . . . , h T ] Ëy t â 0,1 L 2 =L W CE + ÎŒ 2 kwâw (t) k 2 · ClientK D K ,n K samples [k 1 , k 2 , . . . , k T ] Embedding:k i âe i Transformer + LoRA W Q , W K , W V frozen W+AB râȘd [h 1 , h 2 , . . . , h T ] Ëy t â 0,1 L K =L WCE + ÎŒ 2 kwâw (t) k 2 Clip:kâ 1 k 2 â€CClip:kâ 2 k 2 â€CClip:kâ K k 2 â€C Ìw= â K k=1 n k n w k w (t+1) = Ìw+N(0, Ï 2 C 2 I) Server: Global Modelw (t+1) t â t + 1 Figure 2: FlogTinyLLM architecture. Each client processeslog keys[k 1 , . . . , k T ]through a transformer with LoRA adapters (W+AB,râȘd). FedProx loss includes proximal term ÎŒ 2 kwâw t k 2 . Updates are clipped and aggregated with DP noise. 4.3 Model Specifications Table 1 outlines the experimental setup for FlogTinyLLM on the Thunderbird dataset, with parameters grouped as follows: The training data contains up to 4,500,000 samplesand a minimum anomaly rate of 0.001 per node. The LoRA parameters include rank 8, a scaling factor of 32, and a dropout rate of 0.1. Federated learning settings are 14 clients, 10 communication rounds, 50% participation rate, and a proximal coefficient of 0.01. Sliding Window parameters are a 5-minute window size, a 1-minute step size,and at least 5 logs per window. Training parameters are batch size 8, learning rate2Ă10 â5 , weight decay 0.01, 10 local epochs, warmup ratio 0.1, 2 gradient accumulation steps, and maximum gradient norm 1.0. The Differential Privacy parameters are a privacy budget of 10.0, a delta of 10 â5 , a clipping bound of 1.0, and a noise multiplier of 0.01. Table 1 and Table 2 summarize the experimental setup for the FlogTinyLLM framework on both datasets. Table 3 presents architectural and parameter efficiency across thefour FlogTinyLLM models on the Thunderbird dataset. Re- ported metrics are: base parameters (billions), trainableparameters, trainable parameter ratio (%), memory Usage (MB), and noise. All models employ LoRA (r - 8,α- 32, noise std - 0.002857). These metrics directly quantifythe trade-offs between memory efficiency and noise. Phi-1.5 hasa base parameter of 1.3B but is reduced to a trainable pa- rameter of 3,149,824 due to LoRA adoption, representing 0.44% of the total base parameter. Phi-1.5 used a memory of 2,907.6 MB, representing the second least for the FlogTinyLLM training, which indicates moderate resource require- 9 APREPRINT- APRIL22, 2026 Algorithm 1:FlogTinyLLM: Federated Log Anomaly Detection with LoRA andDP Input :Cclients,Trounds,Elocal epochs, learning rateη, FedProx coefficientÎŒ, clipping boundC, noise multiplierÏ, LoRA rankrâȘd Output :Trained global modelw (T) âČPhase 1: Initialization 1w (0) âpre-trained transformer 2Attach LoRA adapters:AâR dĂr ,BâR rĂd 3Freeze attention weights:W Q ,W K ,W V 4fort= 0toTâ1do âČPhase 2: Broadcast 5Server sendsw (t) to allCclients âČPhase 3: Local Training 6foreach clientC= 1, . . . , Cin paralleldo 7w k âw (t) 8forepoche= 1toEdo 9forbatch ( [k 1 , k 2 , . . . , k T ], y ) âD k do âČForward pass 10e i âEMBED(k i ),i= 1, . . . , T 11[h 1 , . . . , h T ]âTRANSFORMER ( e 1 , . . . ,e T ;W+AB ) 12ËyâSOFTMAX ( LINEAR(h T ) ) âČFedProx loss 13L CE ââylog Ëyâ(1ây) log(1âËy) 14L prox â ÎŒ 2 kw k âw (t) k 2 2 15L k âL CE +L prox âČGradient update 16w k âw k âηâ w k L k 17end 18end âČCompute and clip update 19â k âw k âw (t) 20â k ââ k ·min ( 1, C kâ k k 2 ) 21Sendâ k andn k =|D k |to server 22end âČPhase 4: Aggregation with Differential Privacy 23nâ â K k=1 n k 24 Ìwâw (t) + â K k=1 n k n â k 25w (t+1) â Ìw+N ( 0, Ï 2 C 2 I ) 26end 27returnw (T) ments for execution. DeepSeek-R1 has a base parameter countof 1.5B and a trainable parameter count of 1,494,016 (second-least), representing a ratio of 0.17% (least) and 2,978.2 MB of memory utilization (third-least). OPT-1.3B has a base parameter of 1.3B with a trainable parameter of 2,363,392 (third most), representing a ratio of 0.33% in the total base parameter, with the least memory utilization of 2,820.2 MB. TinyLlama-1.1B has a base parameter count of 1.1B and a trainable parameter count of 1,536,000 (minimum), representing a ratio of 0.28% (second-lowest), with the highest memory utilization of 4,221.3 MB. FlogTinyLLM has base parameter range 1.1B-1.5B; Trainable 1,494,016- 3,149,824 (ratios 0.17%-0.44%); Memory 2,820.2-4,221.3 MB;||noise|| 2 3.492-5.071. Specifically, Phi-1.5 has the highest trainable count/ratio/noise but moderate memory.In contrast, DeepSeek-R1 has the highest base but the lowest ratio/noise, making it feasible for multi-GPU setups that demand minimal noise. Lastly, TinyLlama, with the lowest base/trainable but highest memory. 10 APREPRINT- APRIL22, 2026 Table 1: Experimental Configuration for FlogTinyLLM on The Thunderbird Dataset CategoryParameterValue Dataset Maximum Samples4,500,000 Minimum Anomaly Rate per Node0.001 LoRA Rank (r)8 Scaling Factor (α)32 Dropout Rate0.1 Federated Learning Number of Clients14 Communication Rounds (T)10 Participation Rate (q)50% Proximal Coefficient (ÎŒ)0.01 Sliding Window Window Size5 min Step Size1 min Minimum Logs per Window5 Training Batch Size8 Learning Rate2Ă10 â5 Weight Decay0.01 Local Epochs (E)10 Warmup Ratio0.1 Gradient Accumulation Steps2 Maximum Gradient Norm1.0 Differential Privacy Privacy Budget (Δ)10.0 Delta (ÎŽ)10 â5 Clipping Bound (C)1.0 Noise Multiplier (Ï)0.01 4.4 Training Algorithm 1 presents the step-by-step training procedure of FlogTinyLLM. The process runs in four phases that repeat overTcommunication rounds between a central server andCparticipating clients. Phase 1: Initialization.Before any communication begins, the server loads a pretrained language model and at- taches LoRA adapter matricesAâR dĂr andBâR rĂd to each attention layer. The original attention projection weightsW Q ,W K , andW V of the language models are then frozen. From this point onward, training modifies only matricesAandBSan Owl [n.d.]. Phase 2: Broadcast.At the start of each roundt, the server sends the current global parametersW t to every par- ticipating client. Each client trains its local model with these received parametersSingamsetty [2025], so that all participating clients begin the round from the same starting point. Phase 3: Local Training.AllCparticipating clients train in parallel. For a single batchcontaining a sequence [k 1 , k 2 , . . . , k T ]and its ground-truth labely, the forward pass works as follows. Each log keyk i is mapped to a dense embeddinge i . The sequence of embeddings passes through the transformer, where the frozen weightsWare combined with the trainable LoRA productABto produce contextual representations[h 1 , . . . , h T ]. The final representationh T is fed through a linear layer and softmax to yield the predictionËy. The client then computes its local loss. This loss has two parts. The first is a weighted cross-entropy term 11 APREPRINT- APRIL22, 2026 Table 2: Experimental Configuration for FlogTinyLLM on The BGL Dataset CategoryParameterValue DatasetMaximum Samples500,000 LoRA Adaptation Rank (r)8 Scaling Factor (α)32 Dropout0.1 Federated Learning Number of Clients15 Communication Rounds20 Participation Rate0.7 (70%) FedProx ProximalÎŒ0.001 Sliding Window Window Size8 minutes Step Size2 minutes Minimum Logs per Window 6 Local Training Batch Size8 Learning Rate3Ă10 â5 Local Epochs5 Weight Decay0.01 Warmup Ratio0.1 Max Gradient Norm1.0 Gradient Accumulation Steps 1 Differential Privacy Privacy Budget (Δ)10.0 Delta (ÎŽ)1Ă10 â5 Clipping Bound (C)1.0 Noise Multiplier (Ï)1.5 Table 3: FlogTinyLLM architecture and parameter efficiencycomparison on the Thunderbird dataset. ModelBase Params Trainable Ratio (%) Memoryknoisek 2 Microsoft Phi-1.51.3B3,149,8240.442,907.6 MB5.071 DeepSeek-R1-Qwen1.5B1,494,0160.172,978.2 MB3.492 Facebook OPT-1.3B1.3B2,363,3920.332,820.2 MB4.392 TinyLlama-1.1B1.1B1,536,0000.284,221.3 MB3.540 Note: All models use LoRA withr= 8,α= 32. Noise std = 0.002857 for all models. L WCE =â 1 n k n k â j=1 [ w 1 ·y j log Ëy j +w 0 ·(1ây j ) log(1âËy j ) ] (14) wherey j â 0,1is the ground-truth label for sequencej,Ëy j is the modelâs predicted probability that sequencej is anomalous,w 1 is the weight assigned to the anomalous class, andw 0 is the weight assigned to the normal class. The class weights are typically set inversely proportionalto the class frequency in the training data King and Zeng [2001]. This measures how accurately the model classifies the sequence. The second is the FedProx proximal term ÎŒ 2 |WâW t | 2 that penalizes the local parametersw k for drifting away from the global parametersW t Keerthika et al. [2025]. The total loss is their sum. The client updates its parameters by taking a gradient step:w k âw k âηâ w k L k . After all local epochs are complete, the client computes itstotal updateâ k =w k âw t , which represents how far its parameters have moved from the global model it started with.The update is then clipped Li et al. [2022b]: â k ââ k ·min ( 1, C |â k | 2 ) . If the update is already within the bound, it passes throughunchanged. If it exceeds the bound, it is scaled down proportionally. The clipped updatesâ k are sent to the server. Phase 4: Aggregation with Differential Privacy.The server collects all clipped updates and combines them into a single global update. It first computes the total number of samples across all clients:|D|= â K k=1 n k . The aggregated model is then formed by adding a weighted averageof the client updates to the current global model: Ìw=w t + â K k=1 |D i | |D| .â k . The contribution of each client is proportional to the amount of data it holds, so clients 12 APREPRINT- APRIL22, 2026 with larger datasets have a greater influence on the result Zhang et al. [2025]. To make the aggregated model the next global model, the server adds gaussian noise:w t+1 = Ìw+N(0, Ï 2 C 2 I). The noise magnitude is determined by two quantities: the noise multiplierÏ, which controls the overall noise level, and the clipping boundC, which limits the client updates. The productÏ 2 C 2 calibrates the noise so that it is large enough to mask the contribution of any individual client, satisfying the differential privacy guarantee Paverd [2015]. The server then storesw t+1 as the new global model, and the next round begins from Phase 2. AfterTrounds, the server returns the final modelW T , which has absorbed patterns from all client log data. 5 Experimental Results This section outlines the experimental results of FlogTinyLLM on the Thunderbird and the BGL datasets 5.1 Performance Analysis of FlogTinyLLM on the ThunderbirdDataset Table 4 oulines the comprehensive perfomance of FlogTinyLLM on the Thunderbird dataset Table 4: Comprehensive Performance Metrics: Final Values and Best Achieved ModelMetricAccuracy Precision RecallF1ROC-AUC TypeScore FlogTinyLLM: Phi-1.5 Final Value0.91900.9997 0.9182 0.95720.9963 Best Value0.99860.9998 0.9997 0.99930.9963 Best Round131110 FlogTinyLLM: DeepSeek-R1 Final Value0.99840.9997 0.9987 0.99920.9982 Best Value0.99850.9998 0.9991 0.99920.9982 Best Round791710 FlogTinyLLM: OPT-1.3B Final Value0.99840.9998 0.9986 0.99920.9967 Best Value0.99880.9998 0.9997 0.99940.9967 Best Round5101510 FlogTinyLLM: TinyLlama-1.1B Final Value0.95710.9994 0.9571 0.97780.9912 Best Value0.95830.9994 0.9585 0.97840.9913 Best Round510559 5.1.1 Accuracy Convergence Dynamics 12345678910 0.88 0.90 0.92 0.94 0.96 0.98 1.00 Degradation Federated Round Accuracy Phi-1.5 DeepSeek-R1 OPT-1.3B TinyLlama Figure 3: Accuracy convergence across federated rounds. Notable performance degradation occurs at Round 4 for multiple models due to client heterogeneity effects. 13 APREPRINT- APRIL22, 2026 Figure 3 illustrates the evolution of the accuracy of the FlogTinyLLM variants across ten federated learning rounds on the Thunderbird dataset. The x-axis represents the federated rounds, ranging from 1 to 10, while the y-axis denotes accuracy, scaled from 0.86 to 1.00 with increments that emphasize performance variations. Each model is depicted with a unique marker and color for clarity: Phi-1.5 (square markers), DeepSeek-R1 (triangle markers), OPT-1.3B (diamond markers), and TinyLlama (circle markers). The plot incorporates a grid for precise visual alignment and highlights a region of notable degradation around federated round 4, attributed to client heterogeneity effects, as indicated by a dashed rectangle and annotation. The Phi-1.5 model exhibits high initial accuracy, commencing at 0.9986 in Round 1 and maintaining near-perfect performance through Round 2 (0.9985). A minor dip occurs in Round 3 (0.9970), followed by a substantial decline to 0.9196 in Round 4, which represents a relative drop from the preceding round. This perturbation is transient, with recovery evident in Round 5 (0.9986), after which it stabilizes around 0.9985-0.9986 until Round 7. In contrast, the DeepSeek-R1 model demonstrates consistent high accuracy in the early rounds, holding steady at 0.9984 from Rounds 1 through 3. A sharp decline in Round 4 (0.9067). Recovery is observed in Round 7 (0.9985), albeit with subsequent oscillations: a return to the depressed level in Round 8 (0.9069), resurgence to 0.9984 in Round 9, and stabilization at 0.9984 in Round 10. The OPT-1.3B model starts with exemplary performance at 0.9987 in Rounds 1 and 2, then experiences a moderate decline to 0.9674 in Round 3 and further to 0.9199 in Round 4. Post-Round 4, the model rebounds to 0.9988 in Rounds 5 and 6, maintaining accuracy with minimal variance: 0.9986(Round 7), 0.9985 (Round 8), 0.9983 (Round 9), and 0.9984 (Round 10). This trajectory exemplifies rapid convergence and sustained stability after initial fluctuations, highlighting superior adaptability to federated heterogeneity compared to its counterparts. In contrast, TinyLlama, which commenced at 0.9536 in Round 1, faced a more volatile trajectory, ultimately recovering at a rate approximately half that of OPT-1.3B. Specifically, OPT-1.3B recovered twice as fast as TinyLlama from its accuracy dip in Round 4, demonstrating its robustness and efficiency in adapting to federated learning challenges. TinyLlama, conversely, displays greater volatility throughout the training process, starting at 0.9536 in Round 1 and dropping to 0.8840 in Round 2. It partially recovers to 0.9328 in Round 3, only to degrade again to 0.8860 in Round 4. Subsequent rounds show incremental improvement, peaking at 0.9583 in Round 5, with minor decrements to 0.9582 (Round 6), 0.9580 (Round 7), followed by dips to 0.9334 (Round 8) and 0.9332 (Round 9), before a slight uptick to 0.9571 in Round 10. Unlike the other models, TinyLlamaâs accuracy remains consistently below 0.96, exhibiting oscillatory dynamics without achieving near-unity convergence. 5.1.2 F1-Score Evolution 12345678910 0.94 0.95 0.96 0.97 0.98 0.99 1.00 Federated Round F1-Score Phi-1.5 DeepSeek-R1 OPT-1.3B TinyLlama Figure 4: F1-Score evolution demonstrating model robustness. Figure 4 compares the F1-score of the four models in the FlogTinyLLM architecture across 10 federated learning rounds. Compared with the other models, Phi-1.5 maintains high stability in the early rounds and shows a strong recovery after its first performance drop at round 4. However, its notable declines at rounds 4 and 10 distinguish it from models with fewer fluctuations, resulting in the lowestfinal F1-score among the three top-performing models. 14 APREPRINT- APRIL22, 2026 DeepSeek-R1, relative to Phi-1.5, shows early high performance but experiences a more prolonged dip from rounds 4-6. Its pattern of alternating recoveries and drops makes its stability intermediate between Phi-1.5 and OPT-1.3B. OPT-1.3B stands out for its superior convergence stabilitycompared to Phi-1.5 and DeepSeek-R1. While all three models recover after mid drops, OPT-1.3B sustains the highest and most stable F1-scores through the final rounds, distinguishing itself as the most reliable among the top models. Across F1-score evolution, OPT-1.3B and Phi-1.5 demonstrate higher performance and convergence stability, while TinyLlama consistently underperforms with greater score variance. DeepSeek-R1 exhibits oscillating performance, recovering at later rounds. 5.1.3 Roc-Auc Progression 12345678910 0.970 0.975 0.980 0.985 0.990 0.995 1.000 Federated Round ROC-AUC Phi-1.5 DeepSeek-R1 OPT-1.3B TinyLlama Figure 5: ROC-AUC progression Figure 5 shows how discriminative ability changes for the FlogtinyLLM variant on the Thunderbird dataset. Phi-1.5, though improved, is outperformed by DeepSeek-R1 in all rounds. Its minor gains place it above TinyLlama and OPT- 1.3B in the final ROC-AUC, but it remains always below DeepSeek-R1. DeepSeek-R1 leads its cohort in discriminative performance, maintaining the highest ROC-AUC in every round. TinyLlama starts with the lowest value (0.9704), but improves the most, reaching 0.9912 by round 10. It gains sharply between rounds 2 and 3 (0.9721 to 0.9905), then stabilizes in the 0.984-0.991 range. 15 APREPRINT- APRIL22, 2026 1 ( Δ =1 ) 2 ( Δ =2 ) 3 ( Δ =3 ) 4 ( Δ =4 ) 5 ( Δ =5 ) 6 ( Δ =6 ) 7 ( Δ =7 ) 8 ( Δ =8 ) 9 ( Δ =9 ) 10 ( Δ =10 ) 0.92 0.94 0.96 0.98 1 Federated Round (Δspent) F1-Score F1-Score vs. Privacy Budget Consumption on the Thunderbirddataset Phi-1.5 DeepSeek-R1-Distill-Qwen-1.5B OPT-1.3B TinyLlama-1.1B-Chat-v1.0 Figure 6: F1-Score over 10 federated rounds as the cumulative privacy budgetΔis consumed. 5.1.4 Privacy Utility Trade-off Analysis Figure 6 illustrates the F1-score versus privacy consumption of the FlogTinyLLM variant for each round. Under the DP framework with targeted budget,Δ= 10.0, the per-round privacy expenditure ofΔ r = 1.0represents a moderate privacy regime. The noise injection magnitude (Ï= 0.01,knkâ3.5â5.1) is calibrated to the minimal noise setting, prioritizing utility over strict privacy guarantees. Key observations: 1.DP Noise Impact:From Table 3, the DP noise norm varies by model due to differing trainable parameter counts. Phi-1.5âs larger trainable parameter space (3.15M) results in higher noise magnitude compared to DeepSeek (1.49M). 2.Precision Resilience:All models maintain exceptional precision (>0.9994) despite DP noise, indicating that false positive rates are robustly controlled under differential privacy constraints. 3.Recall Sensitivity:Recall metrics show greater sensitivity to DP noise, with TinyLlama exhibiting the largest recall variance (±0.0278), suggesting smaller models may require larger privacy budgets or reduced noise for comparable recall performance. 5.1.5 Statistical Stability Analysis Training with differential privacy adds noise, which raises a natural question as to whether the model performance is stable or unstable. To answer this, we record the accuracy, F1-score, and ROC-AUC at each of the 10 federated rounds and compute the mean and standard deviation for each model variant. Table 5 shows that OPT-1.3B has the lowest stability Index at 0.0142. Its accuracy holds at0.9876±0.0268, its F1-score at0.9935±0.0138, and its ROC-AUC at0.9936±0.0021. All three metrics stay close to their means throughout the 10 rounds, indicating that the DP noise does not push this model off course during training. Phi-1.5 and TinyLlama fall in the middle, with Stability indices of 0.0165 and 0.0171, respectively. Both models perform well on average but show slightly more variation between rounds than OPT-1.3B. TinyLlama has the lowest mean accuracy at0.9355±0.0285, though its round-to-round fluctuation remains moderate. DeepSeek-R1 shows the widest swings. Its accuracy standard deviation of 0.0441 is the highest among all four models, and its Stability Index of 0.0230 is the largest in the table. This suggests that its internal structure is more affected by the noise that differential privacy introduces at each aggregation step. 16 APREPRINT- APRIL22, 2026 Table 5: Performance stability metrics (Mean±Std) across all 10 federated rounds. ModelAccuracyF1-ScoreROC-AUCStability â Phi-1.50.9745±0.0313 0.9868±0.0164 0.9941±0.00170.0165 DeepSeek-R10.9618±0.0441 0.9797±0.0234 0.9971±0.00140.0230 OPT-1.3B0.9876±0.0268 0.9935±0.0138 0.9936±0.00210.0142 TinyLlama0.9355±0.0285 0.9660±0.0155 0.9851±0.00730.0171 â Stability Index = average of standard deviations across metrics. Phi-1.5DeepSeek-R1OPT-1.3BTinyLlama 0 1 2 3 4 5 6 7 ·10 â2 3 . 13 · 10 â 2 4 . 41 · 10 â 2 2 . 68 · 10 â 2 2 . 85 · 10 â 2 1 . 64 · 10 â 2 2 . 34 · 10 â 2 1 . 38 · 10 â 2 1 . 55 · 10 â 2 1 . 7 · 10 â 3 1 . 4 · 10 â 3 2 . 1 · 10 â 3 7 . 3 · 10 â 3 Model Standard Deviation AccuracyÏ F1Ï AUCÏ Figure 7: Variance decomposition across performance metrics. OPT-1.3B demonstrates the lowest overall variance, indicating the most stable training dynamics under differential privacy. 5.1.6 Computational Efficiency Analysis Table 6 shows how long each FlogTinyLLM variant takes to complete the full 10-round federated training on the Thunderbird dataset, and how productively it spends that time. Phi-1.5 finishes fastest at 105.1 hours total, averaging 63.1 minutes per round. It also achieves the highest efficiency score of2.53eâ5, which is computed as the final F1-score divided by the total training time in seconds. Thismeans Phi-1.5 delivers more detection quality per unit of compute than any other variant. Its time per parameter is 12.01 milliseconds, the lowest in the table, and reflects the smaller footprint of its trainable LoRA matrices relative to the other variants. DeepSeek-R1 and OPT-1.3B occupy the middle ground. DeepSeek-R1 takes 156.9 hours with 94.2 minutes per round, while OPT-1.3B takes slightly longer at 165.6 hours and 99.4minutes per round. Despite being slower, OPT-1.3B spends less time per parameter than DeepSeek-R1, which suggests that OPT-1.3B processes its parameters more efficiently but has more of them to get through. TinyLlama is the most expensive to train. It requires 194.1 hours in total, averaging 116.4 minutes per round, and its time perparameter of 45.49 milliseconds is the highest among all four variants. Its Efficiency score of 1.40e-5 is the lowest, meaning it takes the longest to reach each unit of F1 performance. Table 6: Computational efficiency metrics comparison. ModelTotal (h) Avg/Round (min) Time/Param (ms) Efficiency ⥠Phi-1.5105.163.112.012.53e-5 DeepSeek-R1 156.994.237.811.77e-5 OPT-1.3B165.699.425.231.67e-5 TinyLlama194.1116.445.491.40e-5 ⥠Efficiency = Final F1 / Total Training Time in seconds (higheris better). 17 APREPRINT- APRIL22, 2026 12345678910 20 40 60 80 100 120 140 Peak: Round 7 Federated Round Training Time (Ă 10 3 seconds) Phi-1.5 DeepSeek-R1 OPT-1.3B TinyLlama Figure 8: Per-round training time comparison. Significant variance is observed due to client data size heterogeneity, with Round 7 consistently showing peak computation across all models. 5.1.7 Comprehensive Model Ranking on Thunderbird Phi-1.5DeepSeek-R1OPT-1.3BTinyLlama 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1 Model Average Score Average Performance Metrics Across All Rounds AccuracyPrecisionRecallF1-ScoreROC-AUC Figure 9: Average performance metrics comparison of FlogTinyLLM across all 10 federated rounds. OPT-1.3B achieves the highest average accuracy (0.9835), recall (0.9839), and F1-score (0.9915), while DeepSeek-R1 leads in ROC-AUC (0.9971). Table 7 shows the computational time requirements between LogTinyLLM and FlogTinyLLM across the four TinyLLMs on the Thunderbird dataset. The table shows variation in time overhead across the different frame- works. Phi-1.5 shows the best time efficiency profile, requiring 17.54 hours for LogTinyLLM and 105.12 hours for FlogTinyLLM, yielding the lowest time overhead of 5.99Ă. In contrast, TinyLlama-1.1B exhibits the highest overhead at 15.91Ă, with LogTinyLLM completing in 12.20 hours compared to 194.08 hours for FlogTinyLLM. An inverse relationship is seen between baseline executiontime and overhead ratio. TinyLlama-1.1B achieves the 18 APREPRINT- APRIL22, 2026 Table 7: Time and Overhead Comparison (Thunderbird Dataset). Overhead = FlogTinyLLM / LogTinyLLM. ModelLogTinyLLM (h) FlogTinyLLM Time Overhead Time (h)(Ă) microsoft/phi-1.517.54105.125.99 DeepSeek-R1-Distill-Qwen-1.5B16.00156.929.81 facebook/opt-1.3b14.50165.6211.42 TinyLlama-1.1B-Chat-v1.012.20194.0815.91 fastest LogTinyLLM execution of 12.20 hours yet incurs the greatest overhead multiplication. Conversely, Phi-1.5, despite having the longest LogTinyLLM runtime of 17.54 hours, has the lowest overhead factor. The deepseek 1.5B model represents an intermediary efficiency with 16.00 hours for LogTinyLLM and 156.92 hours for FlogTinyLLM, resulting in a 9.81Ăoverhead. Similarly, Opt-1.3b requires 14.50 hours and 165.62 hours, respectively, producing an 11.42Ăoverhead. These indicate that FlogTinyLLM introduces higher computational costs, ranging from ap- proximately 6Ăto nearly 16Ăthe baseline time in LogTinyLLM, suggesting important trade-offs between the two approachesOcansey et al. [2025]. 5.1.8 Comparative Analysis of LogTinyLLM and FlogTinyLLM on the Thunderbird Dataset Table 8: Performance Comparison: Centralized LogTinyLLM vs. Federated FlogTinyLLM ModelFrameworkAccuracy Precision Recall F1-Score Phi-1.5 LogTinyLLM0.98680.98470.95860.9842 FlogTinyLLM (Avg) 0.98250.99940.98280.9908 â(Fed - Cent)-0.0043+0.0147+0.0242+0.0066 DeepSeek-R1 LogTinyLLM0.98830.98770.96050.9856 FlogTinyLLM (Avg) 0.96180.99960.96150.9797 â(Fed - Cent)-0.0265+0.0119+0.0010-0.0060 OPT-1.3B LogTinyLLM0.98730.98570.96930.9848 FlogTinyLLM (Avg) 0.98760.99960.98790.9935 â(Fed - Cent)+0.0003+0.0139+0.0186+0.0087 TinyLlama LogTinyLLM0.98650.98420.94910.9835 FlogTinyLLM (Avg) 0.93550.99910.93540.9660 â(Fed - Cent)-0.0510+0.0149-0.0137-0.0175 Note:Federated values represent the average across 10 federatedrounds. Green (+)indicates federated outperforms centralized;Red (-)indicates centralized outperforms federated. Table 8 shows the performance outlook of the centralized LogTinyLLM and the federated FlogTinyLLM on the Thun- derbird dataset. The Phi-1.5 model shows good performance on FlogTinyLLM, with increases in Precision by 0.0147, Recall by 0.0242, and F1-Score by 0.0066, while marginally decreasing in accuracy by 0.0043. Phi-1.5 0.9828 Recall under FlogTinyLLM, compared to 0.9586 under LogTinyLLM, indicates better log sequence classification in the fed- erated framework. The DeepSeek-R1 also exhibited a trade-off between accuracy and specificity, showing a 0.0265 reduction in accuracy and a 0.0060 reduction in F1-score under FlogTinyLLM. However, precision reached a near- perfect value of 0.9996. From Table 8, OPT-1.3B is identifiedas the most stable candidate under FlogTinyLLM, achiev- ing superior performance across all metrics compared to thecentralized LogTinyLLM. This suggests a unique struc- tural compatibility between OPT-1.3Bâs weight distribution and the FlogTinyLLM architecture. TinyLlama showed a sharp sensitivity to the federated framework. While precision improved by 0.0149, the model suffered regressions in Accuracy by 0.0510, Recall by 0.0137, and F1-Score by 0.0175. The table shows that FlogTinyLLM consistently improves Precision across all architectures. However, architectural stability varies. OPT-1.3B and Phi-1.5 are the most viable candidates for the FlogTinyLLM framework compared to the LogTinyLLM architecture. Table 9 presents a comparative analysis of LogTinyLLM and FlogTinyLLM approaches across metrics on the Thun- derbird dataset. LogTinyLLM demonstrates superior accuracy performance, with 3 of 4 models achieving higher scores. DeepSeek-R1 achieves the best centralized accuracy of 0.9883, with scores ranging from 0.9865 to 0.9883. FlogTinyLLM shows a narrower advantage, with only OPT-1.3Bachieving a marginal improvement of 0.0003. FlogTinyLLM accuracy ranges more broadly from 0.9355 to 0.9876. FlogTinyLLM shows a clear advantage in Preci- 19 APREPRINT- APRIL22, 2026 Phi-1.5DeepSeek-R1OPT-1.3BTinyLlama 0.92 0.94 0.96 0.98 1 0 . 9842 0 . 9856 0 . 9848 0 . 9835 0 . 9908 0 . 9797 0 . 9935 0 . 9660 Model Architecture Score F1-Score Comparison: Centralized vs. Federated Learning Centralized LogTinyLLMFederated FlogTinyLLM (Avg) Figure 10: F1-Score comparison between LogTinyLLM and FlogTinyLLM frameworks. OPT-1.3B shows the best improvement under FlogTinyLLM by 0.0087, achieving the highest overall F1-Score. Phi-1.5 also sees an F1-Score benefit from the FlogTinyLLM framework. sion, with all four models achieving higher precision, ranging from 0.9991 to 0.9996; OPT-1.3B and DeepSeek both achieving the best scores. However, this gain in precision comes with a trade-off in accuracy. The results illustrate that while precision climbs, notably enhancing FlogTinyLLMâs performance, there is a modest dip in accuracy, with LogTinyLLM maintaining slightly higher accuracy in certain configurations. LogTinyLLM precision ranges from 0.9842 to 0.9877, with DeepSeek-R1 achieving the highest at0.9877. Results are mixed in Recall. Among the models, FlogTinyLLM shows higher recall in 3 out of 4 configurations,whereas only TinyLlama favors the LogTinyLLM. When comparing the frameworks, FlogTinyLLM achieves the best recall score with OPT-1.3B at 0.9879. In con- trast, the best score for LogTinyLLM is 0.9693. There are almost even performances between FlogTinyLLM and LogTinyLLM on F1-Score, with DeepSeek-R1 and TinyLlama favoring LogTinyLLM training, while Phi-1.5 and OPT-1.3B favoring FlogTinyLLM training. Notably, OPT-1.3B achieves the highest overall F1-score of 0.9935 under the FlogTinyLLM framework, surpassing the best LogTinyLLMF1-score of 0.9856 in DeepSeek-R1. This higher F1-score of OPT-1.3B indicates its suitability for applications where balanced precision and recall are critical. Over- all, OPT-1.3B under the FlogTinyLLM framework is the best performer, being the only model configuration with all positive performance differences across metrics. Figure 11 presents a bar chart of performance deltas (â =FlogTinyLLMâLogTinyLLM) across the four model archi- tectures on the Thunderbird dataset, covering accuracy, precision, recall, and F1-score. The four models have positive deltas in precision under the federated setting. Recall improves for three of the four models: Phi-1.5 (+0.0242), OPT-1.3B (+0.0186), and DeepSeek-R1 (+0.0010while TinyLlama incurs a slight regression (â0.0137). F1-score gains are observed for Phi-1.5 (+0.0066) and OPT-1.3B (+0.0087), whereas DeepSeek-R1 and TinyLlama experience modest declines ofâ0.0060andâ0.0175, respectively. Accuracy is the most adversely affected metric: three models exhibit negative deltas, with TinyLlama recording the largest drop (â0.0510) and OPT-1.3B remaining nearly neutral (+0.0003). Figure 12 and Table 12 show the performance of FlogTinyLLM accuracy on the BGL dataset over 20 communication rounds.DeepSeek-R1demonstrates the highest overall performance, achieving apeak accuracy of 0.9681 across several rounds (7 to 9, 13 to 15, and 20). Although some variability is observed, including a decrease to 0.9231 at round 10, the model consistently returns to high performance levels. The mean accuracy across all rounds is 20 APREPRINT- APRIL22, 2026 Table 9: LogTinyLLM vs. FlogTinyLLM Framework Analysis on the Thunderbird Dataset MetricCentralized StrengthFederated Strength Accuracy3 of 4 models higherOPT-1.3B only (+0.0003) Best: DeepSeek-R1 (0.9883)Best: OPT-1.3B (0.9876) Range: 0.9865â0.9883Range: 0.9355â0.9876 PrecisionâAll 4 models higher Best: DeepSeek-R1 (0.9877)Best: OPT-1.3B/DeepSeek (0.9996) Range: 0.9842â0.9877Range: 0.9991â0.9996 RecallTinyLlama only3 of 4 models higher Best: OPT-1.3B (0.9693)Best: OPT-1.3B (0.9879) Gain: Phi +0.0242, OPT +0.0186 DeepSeek +0.0010 F1-ScoreDeepSeek-R1, TinyLlamaPhi-1.5, OPT-1.3B Best: DeepSeek-R1 (0.9856)Best: OPT-1.3B (0.9935) Highest overall F1 Best OverallâOPT-1.3B Federated Only model with all positiveâ Phi-1.5 DeepSeek-R1 OPT-1.3B TinyLlama â5 â3 â1 0 1 2 ·10 â2 Model Architecture Performance Delta âAccuracyâPrecision âRecallâF1 Figure 11: Performance delta analysis on the Thunderbird dataset. 0.9580, with a standard deviation of 0.0118.Opt-1.3Bdemonstrates the most stable performance, converging to an accuracy of 0.9332 by round 3 and maintaining this level consistently through round 20. The low standard deviation of 0.0034 indicates minimal variance and highly predictable behavior throughout the federated learning process.Phi-1.5 demonstrates moderate performance, achieving a peak accuracy of 0.9492 and a mean accuracy of 0.9443. The model fluctuates between 0.9332 and 0.9492 across rounds, with a standard deviation of 0.0067, indicating relatively stable but not entirely consistent performance.TinyLlama-1.1Bexhibits the most variable performance among the four models, with accuracy ranging from 0.9303 to 0.9492 throughout the training process. Across the 20 communication rounds from Table 10 and Figure 13, DeepSeek-R1-Distill-Qwen-1.5B shows the strongest overall performance with a mean F1 of approximately 0.8609 and the most consistent behavior, clustering around 0.8465 and 0.8922 with only a single dip to 0.7764 at round 10. Phi-1.5 follows with a mean of approximately 0.8260 and produces the highest single-round score of any model (0.9168 at round 18), though it fluctuates between 0.7723 and that peak. Opt-1.3b averages approximately 0.8051 and is the most volatile, ranging from 0.7404 at round 1 to 0.9023 at rounds 14 and 19. TinyLlama-1.1B is the weakestperformer, averaging approximately 0.7822, though its narrow range makes it relatively predictable at that lower level. Overall, DeepSeek-R1-Distill-Qwen-1.5B offers the best balance of high F1 and round-to-round stability among the four models evaluated on the BGL dataset. 21 APREPRINT- APRIL22, 2026 15101520 0.6 0.7 0.8 0.9 1 Communication Rounds Accuracy FlogTinyLLM Accuracy on BGL Dataset Over 20 Rounds DeepSeek-R1-Distill-Qwen-1.5B facebook/opt-1.3b microsoft/phi-1.5 TinyLlama-1.1B Figure 12: FlogTinyLLM accuracy over 20 communication rounds on the BGL dataset. DeepSeek-R1 demonstrates superior performance with peak accuracy of 0.9681, while Opt-1.3B exhibits the most stable convergence pattern at 0.9332 from round 3 onwards. 15101520 0.72 0.76 0.8 0.84 0.88 0.92 Communication Rounds F1 Score FlogTinyLLM F1-Score on BGL Dataset Over 20 Rounds DeepSeek-R1-Distill-Qwen-1.5B facebook/opt-1.3b microsoft/phi-1.5 TinyLlama-1.1B Figure 13: FlogTinyLLM performance over 20 communication rounds. F1 scores show varying convergence patterns with DeepSeek-R1 achieving the most stable high performance. Figure 14 and Table12 present the ROC-AUC values for FlogTinyLLM on the BGL dataset across 20 communication rounds. DeepSeek-R1 achieves values ranging from 0.9334 to0.9448 with a mean of 0.9411. The Opt-1.3b model exhibits values ranging from 0.8703 to 0.9758, with a mean of0.9082 and a peak single-round performance at round 19. The Phi-1.5 model shows values between 0.8799 and 0.9594, with a mean 0.9089, with a peak at round 19 (0.9594). TinyLlama-1.1B displays values from 0.8762 to 0.8932 with amean of 0.8843. DeepSeek-R1 demonstrates the lowest variation, while opt-1.3b shows the largest. 22 APREPRINT- APRIL22, 2026 Table 10: FlogTinyLLM F1-Score on BGL Dataset Over 20 Communication Rounds Round DeepSeek-R1-Distill-Qwen-1.5B opt-1.3b phi-1.5 TinyLlama-1.1B 10.84260.7404 0.79380.8083 20.84650.7610 0.81680.7647 30.88780.7723 0.77230.7647 40.84650.7723 0.77230.7723 50.84650.8723 0.87680.7723 60.84650.7923 0.81680.7723 70.89220.7723 0.81680.8168 80.89220.8723 0.77230.7723 90.89220.8123 0.87230.7723 100.77640.7723 0.81680.7723 110.84650.7723 0.87680.7723 120.84650.7723 0.77230.7723 130.89220.7723 0.81680.7723 140.89220.9023 0.81680.7723 150.89220.8723 0.85680.8168 160.84650.8023 0.81680.7723 170.84650.8223 0.81680.8168 180.84650.7723 0.91680.8168 190.84650.9023 0.88680.7723 200.89220.7723 0.81680.7723 Table 11: Statistical Metrics on BGL Dataset ModelMean Std Dev MinMax Range DeepSeek-R1-Distill-Qwen-1.5B 0.9411 0.0026 0.9334 0.9448 0.0114 facebook/opt-1.3b0.9082 0.0296 0.8703 0.9758 0.1055 microsoft/phi-1.50.9089 0.0272 0.8799 0.9594 0.0795 TinyLlama-1.1B0.8843 0.0036 0.8762 0.8932 0.0170 Table 13 show the mean performance metrics of FlogTinyLLM onthe BGL Data set over 20 rounds.DeepSeek-R1- Distill-Qwen-1.5Branks first in 4 of 5 metrics (Accuracy, F1, Recall, AUC), showing balanced and consistently high performance across all evaluation criteria.Phi-1.5excels in precision (0.9666), the highest among all models,and maintains competitive accuracy and AUC. However, its recall (0.7569) ranks third.TinyLlama-1.1Bhas the lowest recall (0.6899).OPT-1.3Bshows moderate performance across all metrics, ranking last in precision (0.8737) but second in recall (0.7730). All models maintain accuracy above 0.93, showing effective federated learning training across the 20 rounds. 5.2 Performance Analysis of FlogTinyLLM and LogTinyLLM on the BGL Dataset The results presented in Table 14 indicate that LogTinyLLM outperforms the FlogTinyLLM across the four models in terms of accuracy, F1-score, precision, and recall on the BGL dataset. Specifically, LogTinyLLM achieves accuracies between 0.9911 and 0.9919, F1-Scores between 0.9912 and 0.9919, precisions between 0.9913 and 0.9920, and recalls between 0.9693 and 0.9862, with loss values ranging from 0.0307 to 0.0335. In comparison, FlogTinyLLM demon- strates lower performance, with accuracy ranging from 0.9324 to 0.9569, F1-score from 0.7822 to 0.8609, precision from 0.8737 to 0.9666, recall from 0.6899 to 0.8057, and AUC from 0.8843 to 0.9410. Although the FlogTiny architec- ture performs well on the BGL dataset, its results do not match those of LogTinyLLM. Table 15 provides a quantitative assessment of these differences, with positive deltas reflecting the superior performance of LogTinyLLM. The mean differences are +0.0487 for accuracy, +0.1730 for F1-score, +0.0738 for precision, and +0.2251 for recall. The most substantial individual gap is observed in recall for TinyLlama-1.1B (+0.2957) and in F1-score for TinyLlama-1.1B (+0.2095), while the smallest precision gap is noted for Phi-1.5 (+0.0254). 23 APREPRINT- APRIL22, 2026 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 0.86 0.88 0.9 0.92 0.94 0.96 0.98 1 Communication Round ROC AUC FlogTinyLLM ROC-AUC Progression on BGL Dataset Over 20 Rounds DeepSeek-R1-Distill-Qwen-1.5B facebook/opt-1.3b microsoft/phi-1.5 TinyLlama-1.1B Figure 14: ROC AUC values across 20 communication rounds forfour models. Table 12: FlogTinyLLM Best Performances Across All Rounds on BGL Dataset ModelBestBestBestBestBest Round Acc.F1Prec. Rec. AUC Round DeepSeek-R1-Distill-Qwen-1.5B 0.9681 0.8922 1.0000 0.8142 0.94487â9 OPT-1.3B0.9332 0.9023 1.0000 0.8903 0.975819 TinyLlama-1.1B0.9492 0.8168 1.0000 0.6903 0.8932 7, 15â18 Phi-1.50.9492 0.9168 1.0000 0.8903 0.959418 The Fig 15 is a horizontal bar chart. It compares performancegaps (LogTinyLLM minus FlogTinyLLM) across four metrics: Accuracy, F1-Score, Precision, and Recall. The x-axis shows the gap values from 0 to 0.32. The y-axis lists the models. Each metric uses bars with distinct fills and patterns. Figure 16 shows that OPT-1.3B in the FlogTinyLLM architecture achieves the fastest federated training on the BGL dataset, completing 20 rounds in 14.18 hours. This makes it 3.09Ăfaster than TinyLlama-1.1B (43.85 hours), despite having more parameters (714.1M vs. 556.4M). DeepSeek-R1-Distill-Qwen-1.5B follows with 41.29 hours, while Phi-1.5 needs 43.31 hours. For LogTinyLLM, training time decreases across all models. OPT-1.3B remains most effi- cient, finishing centralized training in 5.92 hours, followed by Phi-1.5 at 6.03 hours, DeepSeek-R1 at 6.83 hours, and TinyLlama-1.1B at 7.60 hours. The tight cluster of 1.68 hours shows similar computational needs across architectures with centralized data. 5.2.1 Performance Comparison with Federated Baseline Models on the BGL Dataset Table 16 compares federated learning models for log anomalydetection on the BGL dataset. LogDeep(FL) achieves the highest precision (0.9688) and F1-score (0.9118), showing a strong balance between precision and recall. Feder- ated LogBERT delivers the best recall (0.9625), though at the expense of lower precision. Among the FlogTinyLLM variants, DeepSeek attains the best F1-score (0.8609), while Phi 1.5 reaches the highest precision (0.9666). Federated LogGPT has the lowest F1 Score (0.7562), showing a weaker precision-recall trade-off. Figure 17 presents a bar chart of training times in hours for federated log anomaly detection models on the BGL dataset, with a logarithmic y-scale (base 10) from 0.2 to 80 and y-ticks at 0.5, 1, 2, 5, 10, 20, 50. 24 APREPRINT- APRIL22, 2026 Table 13: Average Performance Metrics Across 20 Federated Learning Rounds ModelAccuracyF1Precision Recall AUC DeepSeek-R1-Distill-Qwen-1.5B0.9569 0.86090.92680.8057 0.9410 OPT-1.3B0.9324 0.8051 0.8737 0.7730 0.9045 TinyLlama-1.1B0.9368 0.7822 0.9041 0.6899 0.8843 Phi-1.50.9448 0.82600.96660.7569 0.9095 Table 14: Performance Comparison: FlogTinyLLM vs. LogTinyLLM Training on BGL Dataset Model FlogTinyLLMLogTinyLLM Acc.F1Prec. Rec.Acc.F1Prec. Rec. DeepSeek-R1-Distill-Qwen-1.5B 0.9569 0.8609 0.9268 0.8057 0.9911 0.9912 0.9913 0.9849 OPT-1.3B0.9324 0.8051 0.8737 0.7730 0.9911 0.9913 0.9913 0.9693 TinyLlama-1.1B0.9368 0.7822 0.9041 0.6899 0.9916 0.9917 0.9917 0.9856 Phi-1.50.9448 0.8260 0.9666 0.7569 0.9919 0.9919 0.9920 0.9862 Federated LogBERT completes training in 1388.19 seconds (â0.39 hours), while the FlogTinyLLM variants require between 14.2 and 43.9 hours due to the computing overhead of fine-tuning large language models in a federated setting. Table 17 presents a comparison of federated learning-basedlog anomaly detection models evaluated on the Thunder- bird dataset. The best result for each metric is highlightedin the table. The FlogTinyLLM variants consistently outperform traditional federated approaches (LogBERT, LogDeep(FL), Log- GPT) in F1-Score on the Thunderbird dataset, showing the effectiveness of tiny LLMs for federated log anomaly detection. 6 Discussion Comparing FlogTinyLLM and LogTinyLLM across both Thunderbird and BGL datasets shows that FlogTinyLLM provides strong local training on distributed clients while preserving data privacy. This approach produces results comparable to LogTinyLLMâs centralised training on both datasets, supporting the feasibility of privacy-preserving and practical learning. On the Thunderbird dataset, FlogTinyLLM achieves near-centralized performance, with accu- racy scores of 0.9876 for OPT-1.3B and 0.9825 for Phi-1.5. Precision in the federated setup remains high, slightly exceeding that of the centralized setup. Recall and F1-scores are also competitive, demonstrating FlogTinyLLMâs effectiveness for Thunderbird. On the BGL dataset, FlogTinyLLMâs accuracy ranges from 0.9324 to 0.9569 and F1-scores from 0.7822 to 0.8609. While the FlogTinyLLM trails centralized LogTinyLLM on BGL, the gap across models is modest. Training times are longer for FlogTinyLLMdue to privacy measures on both datasets, with OPT- 1.3B notable for faster federated training and near-centralized accuracy. Overall, FlogTinyLLM performs well on both datasets with reasonable increases in computational cost.Comparing FlogtinyLLM family with benchmark models: Federated LogBERT, Federated LogGPT, and LogDeep(FL) on the Thunderbird dataset from Table 17. It can be seen that within the FlogTinyLLM family, all variants obtain high Precision (â„0.9991). However, they differ slightly in Re- call. OPT-1.3 and Phi-1.5 achieve the highest Recall values(0.9879 and 0.9828). TinyLlama records the lowest Recall in this group (0.9354). DeepSeek-R1 falls in between at 0.9615. These results suggest that among the FlogTinyLLM family, OPT-1.3 provides the best anomaly detection capability in federated settings on the Thunderbird dataset. Phi- 1.5 offers a close second. It can also be seen from Table 17 that the FlogTinyLLM family outperformed all the benchmark models on precision and F1-Score, except on recall, where LogDeep(FL) leads with perfect recall. Figure 18 shows a difference in training time between baselines andFlogTinyLLM models. The three baselines Federated LogBERT (0.39 hrs), LogDeep(FL) (3 hrs), and Federated LogGPT (3.42 hrs) train in under 4 hours. In contrast, FlogTinyLLM variants require more time, from 105.12 hours (Phi-1.5) to 194.08 hours (TinyLlama), about 30 to 50 times longer than baselines. Comparing the FlogTinyLLM family with the benchmark models Federated LogBERT, Federated LogGPT, and LogDeep(FL), Table 16 shows their performance measures on the BGL dataset. Regarding precision, LogDeep(FL) achieves the highest (0.9688), with FlogTinyLLM: Phi-1.5 only slightly lower (0.9666). In contrast, Federated LogBERT obtains the lowest precision (0.7574). Other FlogTinyLLM variants fall between 0.8737 (OPT-1.3B) and 0.9268 (DeepSeek). This demonstrates that,in terms of precision, FlogTinyLLM variants generally outperform Federated LogBERT but not FL LogDeep on the BGL dataset. For recall metrics on the BGL dataset, Fed- 25 APREPRINT- APRIL22, 2026 Table 15: Performance Gap: LogTinyLLM vs. FlogTinyLLM (Difference) ModelâAcc.âF1âPrec.âRec. DeepSeek-R1-Distill-Qwen-1.5B +0.0342 +0.1303 +0.0645 +0.1792 OPT-1.3B+0.0587 +0.1862 +0.1176 +0.1963 TinyLlama-1.1B+0.0548 +0.2095 +0.0876 +0.2957 Phi-1.5+0.0471 +0.1659 +0.0254 +0.2293 Mean Gap+0.0487 +0.1730 +0.0738 +0.2251 0 5·10 â2 0.10.150.20.250.3 DeepSeek-R1 OPT-1.3B TinyLlama-1.1B Phi-1.5 0.034 0.059 0.055 0.047 0.13 0.186 0.21 0.166 0.065 0.118 0.088 0.025 0.179 0.196 0.296 0.229 Performance Gap AccuracyF1-Score PrecisionRecall Figure 15: Performance gap analysis with horizontal bars showing gaps by model across metrics erated LogBERT leads at 0.9625, excelling at true-positiveidentification. FlogTinyLLM: TinyLlama has the lowest recall (0.6899). Regarding F1-scores: LogDeep(FL) lead at0.9118, with FlogTinyLLM: DeepSeek (0.8609) and Fed- erated LogBERT (0.8477) following. The lowest F1-scores are seen in Federated LogGPT (0.7562) and FlogTinyLLM: TinyLLama (0.7822). Notably, some FlogTinyLLM variants, such as DeepSeek, remain competitive, offering effective trade-offs. It can also be seen from Figure 17 that the FlogTinyLLM family took more time to train due to the use of LLMs 7 Conclusion In this work, we studied log anomaly detection in settings where data cannot be centrally shared. We proposed DP- FLogTinyLLM, a federated framework that integrates differential privacy with parameter-efficient LLM fine tuning. Empirical results on the Thunderbird and BGL datasets show that FlogTinyLLM performs close to its centralized counterpart in terms of accuracy, precision, and F1-score.The differences across models are small, suggesting that the federated setup is able to retain most of the predictive performance while adding privacy protection. On Thunderbird, the results are nearly identical to the centralized setting, while on BGL the model remains strong across all metrics with only slight variation in recall. Also, FlogTinyLLM outperforms benchmark federated models such as Federated LogBERT, Federated LogGPT, and LogDeep(FL) on the Thunderbird dataset, while remaining competitive on BGL. Moving to a federated and privacy-preserving setup does introduce additional computational cost. However, this overhead appears to be a reasonable trade-off given that sensitive log data never leaves local systems. 26 APREPRINT- APRIL22, 2026 DeepSeek-R1 OPT-1.3B Phi-1.5 TinyLlama 0 10 20 30 40 50 41.3 14.2 43.3 43.9 6.8 5.9 6 7.6 Model Training Time (Hours) FlogTinyLLMLogTinyLLM Figure 16: Comparison of training time between FlogTinyLLMand LogTinyLLM across the four model architectures on the BGL dataset. Table 16: Performance Comparison of Federated Learning Models on the BGL Dataset ModelPrecision Recall F1-Score Federated LogBERT0.75740.96250.8477 Federated LogGPT0.9211 0.7482 0.7562 LogDeep(FL)0.96880.86110.9118 FlogTinyLLM: OPT 1.3B0.8737 0.7730 0.8051 FlogTinyLLM: DeepSeek0.9268 0.8057 0.8609 FlogTinyLLM: TinyLLama 0.9041 0.6899 0.7822 FlogTinyLLM: Phi 1.50.9666 0.7569 0.8260 Bold values indicate the best performance for each metric. Table 17: Performance Comparison of Federated Models on theThunderbird Dataset ModelRecall Precision F1-Score Federated LogBERT0.9625 0.75740.8477 LogDeep(FL)1.000.79350.8191 Federated LogGPT0.9924 0.96290.9774 FlogTinyLLM: Phi-1.50.9828 0.99940.9908 FlogTinyLLM: DeepSeek-R1 0.96150.99960.9797 FlogTinyLLM: OPT-1.30.98790.99960.9935 FlogTinyLLM: TinyLlama0.9354 0.99910.9660 27 APREPRINT- APRIL22, 2026 Fed. LogBERT Fed. LogGPT LogDeep(FL) FlogTinyLLM: OPT 1.3B FlogTinyLLM: DeepSeek FlogTinyLLM: TinyLLama FlogTinyLLM: Phi 1.5 0.5 1 2 5 10 20 50 0.39 h 0.61 h 1.50 h 14.20 h 41.30 h 43.90 h 43.30 h Training Time (hours) Figure 17: Training time efficiency of federated log anomalydetection models on the BGL dataset (log scale). Fed- erated LogBERT completes training in 1388.19seconds (â0.39 hours), while the FlogTinyLLM variants require be- tween 14.2 and 43.9 hours due to the computational overhead of fine-tuning large language models in a federated setting. 1388.19 seconds (0.39 hrs) 3 hours 205 min 19 secs (3.42 hrs) 105.12 hours 156.92 hours 165.62 hours 194.08 hours Baseline Models FlogTinyLLM Models All times converted to hours for uniform comparison. Original values from the dataset shown on each bar. 0 25 50 75 100 125 150 175 200 FlogTinyLLM: TinyLlama FlogTinyLLM: OPT-1.3 FlogTinyLLM: DeepSeek-R1 FlogTinyLLM: Phi-1.5 Fed. LogGPT (base) Himler et al. (2024) Fed. LogBERT (Base) Training Time (hours) Lorem ipsum Figure 18: Training time comparison of federated models on the Thunderbird dataset. 28 APREPRINT- APRIL22, 2026 References Leeladhar Chourasiya, Sushma Khatri, Umesh Kumar Lilhore,Sarita Simaiya, Roobaea Alroobaea, Abdullah M. Baqasah, Majed Alsafyani, and Monish Khan. Advanced systemlog analyzer for anomaly detection and cyber forensic investigations using lstm and transformer networks.Journal of Cloud Computing, 14:60, 2025. doi: 10. 1186/s13677-025-00789-y. URLhttps://link.springer.com/article/10.1186/s13677-025-00789-y. Juyoung Lee, Yeonsu Jeong, Taehyun Han, and Taejin Lee. Logresp-agent: A recursive ai framework for context- aware log anomaly detection and ttp analysis.Applied Sciences, 15(13):7237, 2025. Min Du, Feifei Li, Guineng Zheng, and Vivek Srikumar. Deeplog: Anomaly detection and diagnosis from system logs through deep learning. InProceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS), pages 1285â1298. ACM, 2017a. Weibin Meng, Ying Liu, Yichen Zhu, Shenglin Zhang, Dan Pei, Yuqing Liu, Yihao Chen, Ruizhi Zhang, Shimin Tao, Pei Sun, and Rong Zhou. Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs. InProceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI), pages 4739â4745, 2019. Haixuan Guo, Shuhan Yuan, and Xintao Wu. Logbert: Log anomaly detection via bert. In2021 international joint conference on neural networks (IJCNN), pages 1â8. IEEE, 2021a. Xiao Han, Shuhan Yuan, and William G. J. Halfond. Loggpt: Loganomaly detection via gpt.arXiv preprint arXiv:2309.14482, 2023. Also presented at IEEE BigData 2023. Isaiah Thompson Ocansey, Ritwik Bhattacharya, and Tanmay Sen. Logtinyllm: Tiny large language models based contextual log anomaly detection.arXiv preprint arXiv:2507.11071, 2025. Yihan Liao, Jacky Keung, Zhenyu Mao, Jingyu Zhang, and Jialong Li. Fedlad: A modular and adaptive testbed for federated log anomaly detection. In2025 IEEE International Conference on Autonomic Computingand Self- Organizing Systems Companion (ACSOS-C), pages 227â232. IEEE, 2025. European Union. General data protection regulation (gdpr)â legal text.https://gdpr-info.eu/, 2016. Accessed: 2026-04-07. Julia Zhu.The personal information protection law:Chinaâs version of the gdpr? https://w.jtl.columbia.edu/bulletin-blog/the-personal-information-protection-law-chinas-version-of- 2022. Columbia Journal of Transnational Law: The Bulletin,Accessed: 2026-04-07. Xiaowei Huang, Yi Dong, and Sikha Pentyala. Data pipeline challenges of privacy-preserving federated learning. https://w.nist.gov/blogs/cybersecurity-insights/data-pipeline-challenges-privacy-preserving-federated December 2024. National Institute of Standards and Technology (NIST) Blog. Alissa Brauneck, Louisa Schmalhorst, Mohammad Mahdi Kazemi Majdabadi, Mohammad Bakhtiari, Uwe Völker, Jan Baumbach, Linda Baumbach, and Gabriele Buchholtz. Federated machine learning, privacy-enhancing tech- nologies, and data protection laws in medical research: scoping review.Journal of medical Internet research, 25: e41588, 2023. Marc Vucovich, Amogh Tarcar, Penjo Rebelo, Narendra Gade, Ruchi Porwal, Abdul Rahman, Christopher Redino, Kevin Choi, Dhruv Nandakumar, Robert Schiller, Edward Bowen, Alex West, Sanmitra Bhattacharya, and Bal- aji Veeramani. Anomaly detection via federated learning.https://arxiv.org/abs/2210.06614, 2022. arXiv:2210.06614 [cs.LG]. Ms Bithi, Md Emran Masud, and Md Alamgir Hossain. A new adaptive federated learning approach for privacy preserving uav anomaly detection under non-iid distributions.Scientific Reports, 16:38732, 2026. doi: 10.1038/ s41598-026-38732-z. URLhttps://w.nature.com/articles/s41598-026-38732-z. Max Landauer et al. Anomaly detection in log-event sequences: A federated deep learning approach and open chal- lenges.Journal of Information Security and Applications, 76:103746, 2024. Tae-Ho Shin and Soo-Hyung Kim. Utility analysis about log data anomaly detection based on federated learning. Applied Sciences, 13(7):4495, 2023. Vasileios Tsouvalas, Samaneh Mohammadi, Ali Balador, Tanir Ozcelebi, Francesco Flammini, and Nirvana Meratnia. Enccluster: Scalable functional encryption in federated learning through weight clustering and probabilistic filters. Pervasive and Mobile Computing, 108:102021, 2025. Wei Guan, Jian Cao, Shiyou Qian, Jianqi Gao, and Chun Ouyang.Logllm: Log-based anomaly detection using large language models.arXiv preprint arXiv:2411.08561, 2024. BentoML. Calculating gpu memory for llms.https://w.bentoml.com/llm/getting-started/calculating-gpu-memory-for 2024. Accessed: 2026-04-07. 29 APREPRINT- APRIL22, 2026 Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu,Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022. Tamer Ahmed Eltaras, Qutaibah Malluhi, Alessandro Savino,Stefano Di Carlo, and Adnan Qayyum. R-conv++: un- covering privacy vulnerabilities through analytical gradient inversion attacks.Computing, 107(7):156, 2025. doi: 10. 1007/s00607-025-01508-w. URLhttps://link.springer.com/article/10.1007/s00607-025-01508-w. MartĂn Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan,Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. InProceedings of the 2016 ACM SIGSAC Conference on Computer and Commu- nications Security (CCS), pages 308â318. ACM, 2016. others Xu. Dp-fedlora: Privacy-enhanced federated fine-tuning for on-device large language models.arXiv preprint arXiv:2509.09097, 2025. H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise AgĂŒera y Arcas. Communication- efficient learning of deep networks from decentralized data. InProceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), pages 1273â1282, 2017. Mohammed Alenezi, Abdullah Alabdulatif, Raghad Alshammari, and Fahad Alshammari.A lightweight intrusion detection system for internet of things using deep learning.Sensors In- ternational, 5:100304, 2024.ISSN 2666-3511.doi: 10.1016/j.sintl.2024.100304.URL https://w.sciencedirect.com/science/article/pii/S2666827024000306. Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimiza- tion in heterogeneous networks. InProceedings of Machine Learning and Systems (MLSys), 2020a. Min Du, Feifei Li, Guineng Zheng, and Vivek Srikumar. Deeplog: Anomaly detection and diagnosis from system logs through deep learning.arXiv preprint arXiv:2210.06614, 2022. URLhttps://arxiv.org/abs/2210.06614. Beibei Li, Shang Ma, Ruilong Deng, Kim-Kwang Raymond Choo, and Jin Yang. Federated anomaly detection on system logs for the internet of things: A customizable and communication-efficient approach.IEEE Transactions on Network and Service Management, 19(2):1705â1716, 2022a. Yalan Guo, Yulei Wu, Yanchao Zhu, Bingqiang Yang, and Chunjing Han. Anomaly detection using distributed log data: A lightweight federated learning approach. In2021 International Joint Conference on Neural Networks (IJCNN), pages 1â8. IEEE, 2021b. Patrick Himler, Max Landauer, Florian Skopik, and Markus Wurzenberger. Anomaly detection in log-event sequences: A federated deep learning approach and open challenges.Machine Learning with Applications, 16:100554, 2024. Min Du, Feifei Li, Guineng Zheng, and Vivek Srikumar. Deeplog: Anomaly detection and diagnosis from system logs through deep learning. InProceedings of the 2017 ACM SIGSAC conference on computer and communications security, pages 1285â1298, 2017b. Yihan Zhou, Yan Chen, Xuanming Rao, Yukang Zhou, Yuxin Li, and Chao Hu. Leveraging large language models and bert for log parsing and anomaly detection.Mathematics, 12(17):2758, 2024. doi: 10.3390/math12172758. URL https://w.mdpi.com/2227-7390/12/17/2758. Ali Alqazzaz. Secufl-iot: an adaptive privacy-preserving federated learning framework for anomaly detection in smart industrial networks.Scientific Reports, 2026. Chang Zhang, Shunkun Yang, Lingfeng Mao, and Huansheng Ning. Anomaly detection and defense techniques in federated learning: A comprehensive review.Artificial Intelligence Review, 57:150, 2024a. doi: 10.1007/ s10462-024-10796-1. URLhttps://link.springer.com/article/10.1007/s10462-024-10796-1. Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg on non-iid data.arXiv preprint arXiv:1907.02189, 2019. URLhttps://arxiv.org/abs/1907.02189. APXML. Differential privacy in federated learning.https://apxml.com/courses/federated-learning/chapter-3-enhancing- n.d. Accessed: 2026-04-09. Pinjia He, Jieming Zhu, Zibin Zheng, and Michael R. Lyu. Drain: An online log parsing approach with fixed depth tree. InIEEE International Conference on Web Services (ICWS), pages 33â40. IEEE, 2017. Abraham Silberschatz, Peter B. Galvin, and Greg Gagne.Operating System Concepts. Wiley, 10th edition, 2018. Adam Oliner and Jon Stearley. What supercomputers say: A study of five system logs. pages 575â584, 2007. George AlmĂĄsi, Charles Archer, JosĂ© G Castaños, Manish Gupta, Xavier Martorell, JosĂ© E Moreira, William D Gropp, Silvius Rus, and Brian Toonen. Mpi on bluegene/l: Designingan efficient general purpose messaging solution for a large cellular system. pages 352â361, 2003. 30 APREPRINT- APRIL22, 2026 Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models.arXiv preprint arXiv:2205.01068, 2022. Yuanzhi Li, SĂ©bastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need i: phi-1.5 technical report.arXiv preprint arXiv:2309.05463, 2023. Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. Tinyllama: An open-source small language model.arXiv preprint arXiv:2401.02385, 2024b. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoningcapability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Ćukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems (NeurIPS), pages 5998â6008, 2017. Envisioning. Attention projection matrix.https://w.envisioning.com/vocab/attention-projection-matrix, 2017. Accessed: 2026-04-09. Emergent Mind. Lora-det: Low-rank adaptation for detection.https://w.emergentmind.com/topics/lora-det, 2026. Accessed: 2026-04-09. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021. URL https://arxiv.org/abs/2106.09685. Liwei Zhang, Linghui Li, Xiaotian Si, Ziduo Guo, Xingwu Wang, Kaiguo Yuan, and Bingyu Li. A unified defense framework against membership inference in federated learning via distillation and contribution-aware aggregation. InProceedings of the Network and Distributed System SecuritySymposium (NDSS), 2026. Cynthia Dwork, Guy N Rothblum, and Salil Vadhan. Boosting and differential privacy. In2010 IEEE 51st annual symposium on foundations of computer science, pages 51â60. IEEE, 2010. U.S. Census Bureau.Differential privacy: A modern approach to protecting your confidentiality. https://w.census.gov/library/fact-sheets/2023/dec/about-differential-privacy.html, 2023. Accessed: 2026-04-09. Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimiza- tion in heterogeneous networks.Proceedings of Machine learning and systems, 2:429â450, 2020b. N Singer. Sandia purchases, installs high-capacity thunderbird supercomputing cluster.Sandia Lab-News 07/08/2005, 2005. Neal Singer and Sophia Corwell. Mighty. fast. computers.Sandia Lab News, 58(24), November 2006. URLhttps://w.sandia.gov/app/uploads/sites/81/2021/07/labnews11-24-06.pdf. Sandia Na- tional Laboratories; Thunderbird Linux cluster ranks #6 inTop500 supercomputing list. Original publication date used; PDF uploaded June 30, 2021. Lawrence Livermore National Laboratory. Blue gene/l, n.d.URLhttps://asc.llnl.gov/computers/historic-decommissioned- Accessed: 2026-03-23. Gaoqi Tian, Nurbol Luktarhan, Haojie Wu, and Zhaolei Shi. Cldtlog: System log anomaly detection method based on contrastive learning and dual objective tasks.Sensors, 23(11):5042, 2023. doi: 10.3390/s23115042. URL https://pmc.ncbi.nlm.nih.gov/articles/PMC10255444/. San Owl. Lora: Low-rank adaptation.https://sanowl.github.io/lora.html, n.d. Accessed: 2026-04-10. Saisuman Singamsetty.Federated learning:Training models without sharing raw data. https://dzone.com/articles/federated-learning-training-models-without-sharing,2025. Accessed: 2026-04-09. Gary King and Langche Zeng. Logistic regression in rare events data.Political Analysis, 9(2):137â163, 2001. P. Keerthika, P. Suresh, and Nitesh Kumar. Proximal guided hybrid federated learning approach with parameter efficient adaptive intelligence for pneumonia diagnosis.Scientific Reports, 2025. doi: 10.1038/s41598-025-32286-2. URLhttps://w.nature.com/articles/s41598-025-32286-2. X. Li et al. Preserving data privacy in federated learning through large gradient pruning.Com- puters & Security, 2022b.ISSN 0167-4048.doi: 10.1016/j.cose.2022.102705.URL https://w.sciencedirect.com/science/article/pii/S016740482200431X. 31 APREPRINT- APRIL22, 2026 Xing Zhang, Yuexiang Luo, and Tianning Li. A review of research on secure aggregation for federated learning.Future Internet, 17(7):308, 2025. doi: 10.3390/fi17070308. URLhttps://w.mdpi.com/1999-5903/17/7/308. Andrew Paverd.Enhancing communication privacy using trustworthy remoteentities. PhD thesis, University of Oxford, 2015. 32