Paper deep dive
Bounded Reachability & Jailbreak Detection via Contraction-Constrained State Space Models
Omanshu Thapliyal
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Safety heads are lightweight classifiers attached to pretrained language models for flagging harmful inputs before generation. Their empirical detection performance has been studied, but their formal robustness properties remain largely unexplored. We ask when a State Space Model (SSM)-based safety head can be certified to produce the same prediction for all inputs within a bounded embedding-space perturbation. We prove that the answer turns on a single condition: the $l_\infty$ norm of the state transition matrix must satisfy $\norm{A}_\infty<1$ (the \emph{contraction condition}), which enables exact interval bound propagation (IBP) certification for linear time-invariant classifiers. When the contraction condition holds, the reachable output interval has bounded steady-state width and examples can be certified as robustly classified. When it fails, the interval grows exponentially with sequence length and certification is impossible at any practical perturbation radius. We enforce contraction with a hinge penalty and show on toxic comment data that certified fraction improves from 41\% to 59\%, with a sharp empirical phase transition at $\norm{A}_\infty=1$ matching the theory. Applying a contraction-regularized S4 head to jailbreak detection on JailbreakBench, we achieve a zero-shot transfer to AdvBench (DR=0.994) and HarmBench (DR=0.988). A logistic regression on mean-pooled Mamba-130M embeddings matches or exceeds the S4 head on every detection metric, confirming that harmful intent is already linearly separable in the embedding space. The S4 safety head's contribution is not superior discrimination but the formal certification that no probe-based approach provides.
Tags
Links
- Source: https://arxiv.org/abs/2610.02853v1
- Canonical: https://arxiv.org/abs/2610.02853v1
Trouble viewing inline? Open PDF directly →
Full Text
64,089 characters extracted from source content.
Expand or collapse full text
Bounded Reachability & Jailbreak Detection via Contraction-Constrained State Space Models Omanshu Thapliyal †thanks: Use footnote for providing further information about author (webpage, alternative address)—not for acknowledging funding agencies. Funding acknowledgements go at the end of the paper. Affiliation: Sr. AI ML Researcher Affiliation: Strategic Data Solutions Lab, Hitachi America Ltd. Affiliation: Santa Clara, CA 94086, USA Email: omanshu.thapliyal@hal.hitachi.com Abstract Safety heads are lightweight classifiers attached to pretrained language models for flagging harmful inputs before generation. Their empirical detection performance has been studied, but their formal robustness properties remain largely unexplored. We ask when a State Space Model (SSM)-based safety head can be certified to produce the same prediction for all inputs within a bounded embedding-space perturbation. We prove that the answer turns on a single condition: the l∞l_∞ norm of the state transition matrix must satisfy ‖A‖∞<1 A _∞<1 (the contraction condition), which enables exact interval bound propagation (IBP) certification for linear time-invariant classifiers. When the contraction condition holds, the reachable output interval has bounded steady-state width and examples can be certified as robustly classified. When it fails, the interval grows exponentially with sequence length and certification is impossible at any practical perturbation radius. We enforce contraction with a hinge penalty and show on toxic comment data that certified fraction improves from 41% to 59%, with a sharp empirical phase transition at ‖A‖∞=1 A _∞=1 matching the theory. Applying a contraction-regularized S4 head to jailbreak detection on JailbreakBench, we achieve a zero-shot transfer to AdvBench (DR=0.994) and HarmBench (DR=0.988). A logistic regression on mean-pooled Mamba-130M embeddings matches or exceeds the S4 head on every detection metric, confirming that harmful intent is already linearly separable in the embedding space. The S4 safety head’s contribution is not superior discrimination but the formal certification that no probe-based approach provides. The code is available at: https://github.com/omanshuthapliyal/contraction_constrained-safety-head. 1 Introduction State-space models (SSMs) such as Mamba and Jamba (Gu and Dao (2024); Lieber et al. (2024)) offer computational advantages over Transformer-based architectures for long sequences. As SSM-based language models are deployed in safety-sensitive settings, a key question arises: how can we add safety mechanisms whose robustness is formally certifiable, rather than merely empirically evaluated? Current safety mechanisms fall into two categories. 1) Response-level judges examine model outputs for refusal or compliance language. These are brittle on base models without safety training, because such models do not produce structured refusal phrases. 2) Prompt-level classifiers operate on the input alone, but have rarely been subject to formal robustness analysis for sequence models. Suppose an adversary perturbs an input to a safety classifier, then its output is perturbed as well. If we can bound how much the output perturbs over all inputs within some distance of the original, we can certify that the classification is stable under adversarial bounded perturbations. Reachability Analysis for linear-time invariant (LTI) dynamical systems helps with these perturbation bounds as a property of the underlying dynamics. Reachable states for a (dynamical) system are all the possible output states the system can take, when input lies within a given (usually a bounded) set. For an LTI recurrence xt=Axt−1+Butx_t=Ax_t-1+Bu_t, the state at time t is a linear function of all past inputs, so the reachable set of outputs is an interval that can be computed exactly. The key insight is that whether this interval stays bounded as t grows depends entirely on the spectral radius of A. If the largest row-sum norm ‖A‖∞<1 A _∞<1, each state update contracts the current interval before the new input widens it, and the interval converges to a steady-state width. Conversely, if ‖A‖∞≥1 A _∞≥ 1, the input widening compounds each step and the interval grows unbounded. The contraction condition is therefore not a design choice; it is the exact threshold between certifiable and non-certifiable classifiers in this setting. To this end, this paper makes two contributions. First, we make this connection precise for a toy LTI SSM classifier. We prove the contraction condition (‖A‖∞<1 A _∞<1) is sufficient for the reachable output interval to be bounded by δ‖B‖∞/(1−‖A‖∞)δ B _∞/(1- A _∞), and necessary for this bound to be horizon-independent. A simple hinge penalty enforces contraction during training. We train the classifier on toxic comment data and measure how certified fraction varies with perturbation radius (50 log -spaced values), penalty strength (12 values), and sequence length (8 values). Second, we apply a contraction-regularized two-layer S4 safety head to jailbreak detection on JailbreakBench (JBB; Chao et al. (2024)). The head operates on token embeddings from base Mamba-130M, classifies prompts without seeing any model response, achieves JDR=1.0 across six JBB attack families under a static threat model, and transfers zero-shot to AdvBench and HarmBench. A key negative finding is that logistic regression on the same mean-pooled embeddings achieves AUROC=1.0 on all three benchmarks, equaling or exceeding SSM-head on every detection metric. The Mamba-130M embedding space already separates harmful from benign prompts linearly and the SSM-head’s recurrent architecture adds no discrimination advantage over this probe. Its distinctive contribution is formal certifiability: verifying a classification decision at a specified perturbation radius. The formal results (Proposition 1) apply only to the toy LTI setting (see Appendix B). The JBB setting results are empirical claims on specific benchmarks under specific threat models (Section 4). We provide a detailed landscape of related work, main points of contributions, and current perceived limitations in Appendix A. Detailed experiments to validate the proposed methodology are presented in Appendix B, and the detailed JBB results and their discussions are present in Appendix C. 2 Problem Setup We study the following problem: given a trained LTI SSM classifier and a perturbation radius δ, can we certify that the predicted label is identical for every embedding sequence within the l∞l_∞ ball of radius δ around a given input? This is a guarantee stronger than empirical accuracy under any specific attack, because it holds against all perturbations in the threat model simultaneously, without executing any attack. For feed-forward networks, interval bound propagation (IBP) by Gowal et al. (2018) and linear relaxation methods such as CROWN by Zhang et al. (2018) solve this efficiently by propagating interval bounds layer by layer. The error introduced at each layer is additive and independent of network depth for linear layers. For SSMs, the recurrence creates a qualitatively different difficulty: an input perturbation at step t propagates forward through all remaining steps, compounding multiplicatively at each transition. Over a sequence of length T, a perturbation introduced at step 11 is amplified by a factor of up to ‖A‖∞T A _∞^T by step T. If ‖A‖∞≥1 A _∞≥ 1 this growth is unbounded, and the reachable output set expands to cover both sides of the decision boundary for any δ>0δ>0 and long enough T, making certification vacuous regardless of the method used. Randomized smoothing (see Cohen et al. (2019)) avoids this dependence on architecture, but provides only a probabilistic guarantee and cannot certify individual examples exactly. Prior work on certifying recurrent networks (Jacoby et al. (2020)) addresses bounded RNNs with nonlinear activations via invariant inference. Our setting is architecturally simpler: the SSM is fully linear, but the lack of activation-induced contraction makes the compounding problem acute. Without any nonlinearity to bound state growth, the recurrence dynamics alone determine whether certification is tractable. The central question this section formalizes is therefore: under what condition on A does exact, non-vacuous IBP certification remain possible for arbitrary T? 2.1 Linear SSM Classifier The classifier reads a sequence of token embeddings one step at a time, updating a hidden state vector at each step, then maps the final state to a scalar logit. This is a standard linear time-invariant (LTI) dynamical system (see Gu et al. (2021)): the state at each step is a weighted sum of all past inputs, with weights decaying according to the matrix A. The key property we analyze is whether that decay is fast enough to keep the output interval bounded when inputs are perturbed. Therefore, given an input sequence u1,⋯,uT∈ℝdu_1,·s,u_T ^d, and initial state x0=0x_0=0, the LTI SSM classifier is defined by the recurrence and output equations: xt=Axt+But,st=Cxt+Dut,y^t=st>0x_t=Ax_t+Bu_t,\;\;s_t=Cx_t+Du_t,\;\; y_t=1_s_t>0 (1) where A∈ℝN×NA ^N× N, B∈ℝN×dB ^N× d, C∈ℝ1×NC ^1× N, and D∈ℝ1×dD ^1× d are the learned parameters. The scalar logit sts_t produces binary prediction y^t y_t. All four matrices are unconstrained, real-valued n.Parameters, and the state matrix A is initialized as 0.9I+0.01(0,1)0.9I+0.01N(0,1). 2.2 Perturbation Model We model an adversary who can shift each token embedding by at most δ in l∞l_∞ distance. That is, the adversary can independently nudge every coordinate of every embedding vector by at most δ. This captures synonym substitutions, paraphrase variants, and small continuous shifts in the embedding space (see Jia et al. (2019)), thereby modeling a broader class of adversarial behavior. The perturbation acts on the embedding sequence directly rather than on discrete tokens, relating δ to specific text edits requires the calibration procedure (see Section B). The reachable output set is the set of all logits the classifier can produce under any such perturbation. Because the SSM is linear in each embedding, this set is always a closed bounded interval (which later sections show can be computed exactly without relaxation). However, the width of that interval depends on how perturbations compound through the recurrence, with each step multiplying the existing interval width by ‖A‖∞ A _∞ before the next perturbation arrives. Over T steps, this gives a geometric series whose convergence, and therefore the non-vacuity of certification, depends entirely on whether ‖A‖∞ A _∞. Definition 2.1 (Reachable Output Set). Given a nominal embedding sequence unom=(u1nom,⋯,uTnom)u^nom=(u^nom_1,·s,u^nom_T) and perturbation radius δ≥0δ≥ 0, the l∞l_∞ perturbation ball is δ(unom)=u′:‖ut′−utnom‖∞≤δ,∀tU_δ(u^nom)= \u : u _t-u^nom_t _∞≤δ,\;∀ t \ The (T-sequence length) reachable output set is then defined as the set of all logits achievable over this ball: ℛδ(unom)=sT(u′):u′∈δ(unom).R_δ(u^nom)= \s_T(u ):u _δ(u^nom) \. (2) Because the SSM is linear in each utu_t, this set is always a closed bounded interval [stl,sth][s_t^l,s_t^h], computable exactly without any relaxation or enclosure error. 2.3 Certification Criterion An example is certified at radius δ if the entire reachable interval lies strictly on the correct side of the decision boundary s=0s=0. For a toxic example, this means even the minimum reachable logit is positive; for a safe example, even the maximum is negative. If the interval straddles zero, some perturbation within the ball would flip the predicted label, so the example is not certified. Note that non-certified examples may still be correctly classified under most perturbations, since our definition of certification is a conservative worst-case guarantee, not a claim about typical behavior. Definition 2.2. An example with nominal sequence unomu^nom and true label y∈0,1y∈\0,1\ is certified at radius δ if: all perturbations in δU_δ predict toxic y=1:stl>0y=1:s_t^l>0, and all perturbations predict safe y=1:sth<0y=1:s_t^h<0. This criterion directly answers the problem stated at the top of this section: an example passes certification if and only if the classifier’s decision is provably invariant to all l∞(δ)l_∞(δ) perturbations of its embedding sequence. The dependence of certified fraction over a dataset (the fraction of correctly classified examples that also satisfy Definition 2.2) on δ and on ‖A‖∞ A _∞ is the subject of Sec. B. 3 Methodology 3.1 Interval Bound Propagation for LTI Systems To compute the reachable interval, we track a lower bound xtlx_t^l and upper bound xthx_t^h on the possible state at each time step t, propagating them through the recurrence in (1). Because the system is linear, this tracking is exact: no relaxation is needed. The key step is splitting A into its positive and negative parts, then routing interval endpoints through the correct part to avoid sign-error. A positive entry in A maps lower bounds to lower bounds, and a negative entry reverses the direction. This A+A^+/A−A^- splitting is standard interval arithmetic for linear maps since its exactness is the reason certification here is tight rather than conservative. More formally, let A=A++A−A=A^++A^-, where Aij+=max(Aij,0)A^+_ij= (A_ij,0), and Aij−=min(Aij,0)A^-_ij= (A_ij,0) (similarly split C,DC,D). Starting from x0l=x0h=0x_0^l=x_0^h=0, the interval/box propagates exactly for t=1,⋯,Tt=1,·s,T as: xtl=A+xt−1l+A−xt−1h+Butnom−δ‖B‖∞,xth=A+xt−1h+A−xt−1l+Butnom+δ‖B‖∞ splitx^l_t&=A^+x^l_t-1+A^-x^h_t-1+Bu_t^nom-δ B _∞,\\ x^h_t&=A^+x^h_t-1+A^-x^l_t-1+Bu_t^nom+δ B _∞ split (3) where ‖B‖∞ B _∞ denotes the row-wise absolute sum (‖B‖∞)i=∑j|Bij|( B _∞)_i= _j B_ij . This propagation is exact for LTI systems, without any relaxation or bounding errors. The output interval at horizon T can then be expressed as: sTl=C+xT−1l+C−xT−1h+D+uT−1l+D−uT−1h,xTh=C+xT−1h+C−xT−1l+D+uT−1l+D−uT−1h splits^l_T&=C^+x^l_T-1+C^-x^h_T-1+D^+u^l_T-1+D^-u^h_T-1,\\ x^h_T&=C^+x^h_T-1+C^-x^l_T-1+D^+u^l_T-1+D^-u^h_T-1 split (4) Applying equations (3), (4), at each time step also gives the output reach tube; the time-indexed sequence of output intervals [stl,sth][s_t^l,s_t^h] for t=1,⋯,Tt=1,·s,T. This tube shows not just where the final classification might land, but how the range of possible outputs evolves as tokens are consumed. A contracting system produces a tube that stabilizes, and a non-contracting system produces a tube that widens monotonically. Figure 3 visualizes this difference directly. 3.1.1 Contraction and Bounded Reachability At each step, two forces act on the interval width. The state transition A projects the current width by up to ‖A‖∞ A _∞, and the new input perturbation δ adds a fixed width of up to δ‖B‖∞δ B _∞. Clearly, if ‖A‖∞<1 A _∞<1, each step shrinks the existing width before the new perturbation arrives, and the width converges to a finite steady-state. If ‖A‖∞≥1 A _∞≥ 1, the existing width never contracts, so accumulating perturbations and the width grows unbounded as the sequence lengthens. This geometric intuition is made more precise in the following proposition. Proposition 1 (Contraction ⟹ Bounded Reachability; Separation ⟹ Certification). Let rt=‖xth−tl‖∞/2r_t= x_t^h--_t^l _∞/2 be the state box half-width at time-step t. (i) If ‖A‖∞<1 A _∞<1, then for all t≥0t≥ 0: rt≤‖A‖∞t⋅r0+δ‖B‖∞⋅1−‖A‖∞t1−‖A‖∞r_t≤ A _∞^t· r_0+δ B _∞· 1- A _∞^t1- A _∞ (5) In particular, the steady-state bound is r∞≤δ‖B‖∞/(1−‖A‖∞)r_∞≤δ B _∞/(1- A _∞). If ‖A‖∞≥1 A _∞≥ 1, then rtr_t grows at least as fast as ‖A‖∞t A _∞^t and is unbounded as T→∞T→∞. (i) An example is certified (Definition 2.2) if and only if the reachable output interval at horizon T satisfies the separation criterion: sTl>0s_T^l>0 for y=1y=1, or sTh<0s_T^h<0 for y=0y=0. The width of the output interval satisfies sTh−sTl≤2(‖C‖∞⋅rT+δ‖D‖∞)s_T^h-s_T^l≤ 2( C _∞· r_T+δ D _∞). A sketch of proof for Proposition 1 is provided in Appendix A.1. Proposition 1 provides binary certification decisions. The output interval width is bounded by 2(‖C‖∞rT+δ‖D‖∞)2( C _∞r_T+δ D _∞), so tighter contraction produces a narrower output interval. An example with logit sTnom=+0.5s_T^nom=+0.5 is certified at δ if the interval half-width is less than 0.5, otherwise the interval may straddle zero. In practice, the interval width and the logit margin are on the same order, which is the primary certification bottleneck, not the bound’s tightness. Note that Proposition 1 says nothing about how large the margins will be, as that depends on the learning objective and training data. 3.1.2 Contraction Regularization Rather than constraining A directly, we can add a soft hinge penalty to the training loss that pushes ‖A‖∞ A _∞ below 11 while leaving the update rule unconstrained: ℒ=ℒBCE+λ⋅relu(‖A‖∞−1) L= L_BCE+λ· relu ( A _∞-1 ) (6) where ‖A‖∞=max∑ji|Aij| A _∞= _i _j A_ij . This induces a gradient ∂ℒ/∂Aij=λ⋅sign(Aij)∂ L/∂ A_ij=λ·sign(A_ij) on the row that achieves the maximum row-sum when ‖A‖∞>1 A _∞>1, pulling that row inward. When ‖A‖∞<1 A _∞<1, the gradient is exactly zero—the penalty does not interfere with the binary cross-entropy (BCE) loss in the contracting regime. The hyperparameter λ controls the rate of push (a sweep over 12 values shows monotone improvement in certified fraction in Appendix B). 3.2 S4 Safety Head for Jailbreak Detection The S4 safety head takes a sequence of token embeddings as input and outputs a single scalar score indicating whether the prompt is a jailbreak attempt. It reads the token embeddings one by one, maintaining a 6464-dimensional hidden state that accumulates information about the prompt as tokens arrive. After all tokens are processed, it averages the per-token hidden states (weighted by which positions are real tokens, not padding). Then it passes the average through a linear layer to produce the final score. This prompt-only design is deliberate. Response-level judges, systems that examine whether the model’s reply contains refusal phrases, require a model response and fail silently on base models without safety training, producing no structured refusals. Operating on the prompt alone avoids this dependency and allows detection before any generation occurs. On the other hand, the S4 head is lightweight and is attached to, not built into, the base model. Token embeddings are extracted directly from the Mamba-130M embedding table, so inference runs in ∼ 5 ms on CPU and the head adds ∼ 1 MB of parameters on top of the frozen model. This S4 head utilizes a single lookup without any forward pass through any backbone layers of the Mamba-130M model. Formally, the head applies a two-layer diagonal S4 architecture (Gu et al. (2022)). Each S4 layer uses the zero-order hold (ZOH) discretization: A¯i=exp(Ai⋅ΔavgCLOSE A_i= (A_i· _avg, Bi=((A¯i−1)/Ai)⋅BiB_i=(( A_i-1)/A_i)· B_i, with Δavg _avg the mean step size. The contraction penalty in (6) is applied to the l∞l_∞-norm of the diagonal discretized state matrix, |A¯|max=maxi|A¯i| A _ = _i A_i . The full pipeline is: Linear(768→ 64) → S4 layer × 2 (d=64,N=16d=64,N=16) → LayerNorm → masked-mean pooling → Linear(64→ 1). Masked-mean pooling averages the per-token outputs over real (non-padding) positions only, producing a fixed-size representation regardless of prompt length. As a result, the total number of parameters in our experiment are 54,049. Remark 3.1 (Score is a linear function of the mean-pooled SSM state). The SSM head final score is s=wTh+bs=w^Th+b, where h=∑tmtzt/∑tmth= _tm_tz_t/ _tm_t is the masked mean of per-token SSM outputs zt∈ℝ64z_t ^64. By linearity, two input sequences that produce the same mean-pooled state vector h receive identical scores. The two S4 layers compute ztz_t recurrently, so the intermediate representations are order-dependent. Whether two surface reformulations of the same request produce the same pooled h depends on the learned dynamics and cannot be derived analytically. 4 Certifiable Jailbreak Detection In this Section we evaluate our methodology on JailbreakBench (JBB), a benchmark of 100 harmful behaviors across 10 harm categories, by Chao et al. (2024). Primary evaluation uses the full test split (100 behaviors) across all six attack families. The original dev split (50 behaviors) is preserved for comparison with the preceding sections. The benign evaluation set is expanded to 80 prompts: 30 clearly benign prompts (recipes, trivia, science questions) and 50 hard-negative prompts (dual-use queries on security, chemistry, medicine, and historical events). The base model is a Mamba model with 130M parameters, with no safety training. We compare against StringJudge and KeywordJudge as protocol reference points (Chao et al. (2024)), with both judges operate on model responses, not prompts. StringJudge detects refusal phrases in the output, and KeywordJudge detects compliance-indicating phrases. The base Mamba-130M has no safety fine-tuning and produces no structured refusals, so both judges return JDR=0.0 (this is a protocol-mismatch result, not a meaningful performance comparison). The SSM-head is then trained on 770 examples: 100 JBB goals, 100 JBB targets, 200 template-attacked goals, and 172 benign prompts with scripted refusals. Evaluation uses prompt-only mode (empty response strings) and the head classifies on embedding-level intent. We first use a static threat model, where attack templates are fixed in advance and do not adapt to the detector. This is the primary evaluation regime for JBB-style benchmarks. An adaptive threat model, where the attacker observes detector scores and iterates, is addressed in later section. Jailbreaking results for static threat model do not guarantee robustness against adaptive adversaries. 4.1 Static Threat Model 4.1.1 Detection Walkthrough For direct-attack prompts, the score rises within the first 10–20 tokens as the embedding captures the harmful framing. Template-wrapped attacks (DAN, roleplay) show a delayed rise, as the wrapper text is processed first and contributes low-harm signal, then the actual harmful content raises the score above τ=0τ=0. Benign prompts stay near or below τ=0τ=0 throughout. This trajectory pattern is consistent with the model operating on embedding-level content rather than surface phrasing, shown in Fig. 1. All four harmful examples are correctly detected (score >0>0) and both benign examples are correctly passed (score <0<0). Figure 1: SSM-head detection walkthroughs for six representative prompts. Each panel shows the running logit score as tokens are added (top) and the full prompt text (bottom, colored border). Red border = harmful ground truth; green border = benign. Figure 12 supports the claim that SSM-head operates semantically rather than syntactically. Different attack formulations (template injection, roleplay framing, developer mode) all map to similar score values. This is consistent with the prompt-level embedding mode: the Mamba-130m embedding table encodes token identity rather than surface template structure. The benign distribution (bottom row, mean μ≈−1.5μ≈-1.5) is well-separated from all harmful families, explaining the high AUROC at the default threshold. Three conclusions follow consistently from the cross-benchmark evaluation. 1) The linear probe is the stronger detector: LR-mean-pool achieves AUROC=1.0 on all three benchmarks, while SSM-head achieves 0.993, 0.981, and 0.947 on JBB, AdvBench, and HarmBench respectively. The HarmBench gap of 5.3 AUROC points is concentrated in two semantic categories (Table X), and adding MLP capacity does not close it; the MLP also reaches AUROC=1.0. 2) SSM-head’s failure modes are localized and mechanistically attributable to distribution mismatch: missed detections cluster in copyright and harassment/bullying categories, where JBB training distribution diverges from HarmBench vocabulary. The probe’s advantage in precisely these categories suggests that mean-pooled embeddings capture a broader distributional signal than SSM-head’s sequential recurrence at this training scale. 3) SSM-head’s contribution is formal certifiability under embedding perturbation. Neither the probe nor the MLP can provide this guarantee at any perturbation radius. Contraction regularization improves certified fraction from 41% to 68% while leaving accuracy essentially flat, confirming that certifiability and discrimination are orthogonal properties in this setting. 4.2 Adaptive Threat Model Proxy-Gradient Adaptive Attack We evaluate an embedding-space gradient attack as a proxy for white-box adversarial robustness. The attack performs gradient ascent on the input embedding (100 steps, l2l_2-ball ε= =0.5, Adam lr=0.01) to minimize the SSM-head score. Figure 2 shows the score trajectory and pre/post-attack score distributions for 20 direct-attack prompts. JDR=1.0 before and after attack: the mean score declines from 3.681 to 3.675 over 100 steps, and no example crosses the τ=τ=0 boundary. The attack fails to push any score below zero because the margin (mean score ≈ 3.7) far exceeds the perturbation budget (ε= =0.5 in l2l_2 over the full sequence embedding), and the SSM’s temporal integration further smooths local perturbations. Figure 2: Proxy-gradient attack: mean score trajectory (left, 3.681→ 3.675) and score distribution before vs. after (right). JDR=1.0 before and after; this is a continuous embedding-space relaxation, not a discrete-token attack. Black-Box and White-Box Adaptive Attacks We evaluate two adaptive attacks of increasing adversarial strength against SSM-head, comparing a gradient-free black-box strategy and a gradient-guided white-box strategy. Both operate at the discrete token level and require no access beyond what is stated in each threat model. The black-box attacker does not observe model gradients; it can only query the detector score (see Fig. 17). At each step, the attacker does the following. 1) leave-one-out sensitivity identifies the most-score-affecting token position; 2) the top-15 vocabulary tokens by cosine similarity to the current token are evaluated as discrete replacements; and 3) the best-reducing replacement is accepted if it lowers the score. The white-box attacker on the other hand has full access to model weights and computes exact gradients (see Fig. 18). At each step the attacker does the following. 1) a forward pass + backward pass yields ∂s/∂et∂ s/∂ e_t for every token position simultaneously; 2) for each position t, every vocabulary token j is ranked by the linear score approximation Δsapprox(t,j)=(ej−etcurr)⋅(∂s/∂et) s_approx(t,j)=(e_j-e^curr_t)·(∂ s/∂ e_t); 3) the top-K positions with the largest predicted score reduction are evaluated as discrete replacements; and 4) the globally best replacement is accepted if it reduces the score. This is a standard HotFlip-style gradient attack (Ebrahimi et al. (2018)) applied to the SSM-head scorer. 4.3 Cross-Benchmark Transfer AdvBench transfer (DR=0.994, AUROC=0.981). SSM-head detects 517/520 AdvBench harmful instructions at τ=0τ=0 (0.6% miss rate). The linear probe detects all 520 (DR=1.000, AUROC=1.000). Failure-mode analysis shows no consistent pattern for the 3 missed prompts, all have slightly lower embedding norms than the detected set. HarmBench transfer (DR=0.988, AUROC=0.947). SSM-head misses 5/400 behaviors (1.25% miss rate); the linear probe misses 2/400 (DR=0.995, AUROC=1.000). The SSM-head miss rate is highest in harassment_bullying (4.0%, 1/25) and copyright (3.0%, 3/100); all other categories achieve miss rate ≤ 1.5%. The copyright category uses vocabulary that diverges from JBB training, lacking the direct harmful-action verbs (”make”, ”create”), characterizing most JBB and AdvBench goals. The AUROC gap (SSM-head 0.947 vs. probe 1.0) is concentrated in these two categories only. The misses are semantically localized. Copyright prompts use vocabulary that diverges from JBB training data (no direct harm-action verbs), and harassment/bullying prompts are phrased in a conversational register underrepresented in JBB. The probe’s advantage in these categories suggests that the mean-pooled embedding captures a broader distributional signal than SSM-head’s sequential recurrence learns to use at this training scale. The baseline comparisons confirm that the harmful signal is already linearly separable in the Mamba embedding space. The failure modes are a distribution-mismatch artifact, and not an architectural limitation of SSM-head per se. 5 Conclusion This work proves that contraction of the state transition matrix is sufficient and necessary for bounded reachability in a toy LTI SSM classifier, and applies a contraction-regularized S4 safety head to jailbreak detection on JailbreakBench. On the full JBB test split for a static thread model, SSM-head achieves JDR=1.0 across all six attack families under a static threat model, and AUROC=0.994. It generalizes across obfuscation styles, multi-turn injection positions, and held-out attack families. Under adaptive threat models, SSM-head is vulnerable on direct-attack prompts: a black-box greedy substitution attack achieves ASR=0.933, and a white-box gradient-guided attack achieves ASR=1.0. A logistic regression on mean-pooled Mamba-130M embeddings achieves AUROC=1.0 on all three benchmarks, exceeding SSM-head on every discrimination metric. The embedding space drives the transfer, and the SSM-head’s recurrent structure is not necessary for discrimination. Its contribution is formal certifiability, the ability to verify a classification decision at radius δ. Primary future works include: (i) adversarial training on token-substitution examples to close the direct-attack vulnerability, (i) fine-tuning or retraining on copyright and harassment/bullying behaviors to close the HarmBench AUROC gap, and (i) extending box propagation to nonlinear SSM architectures. References Borkan et al. (2019) D. Borkan, L. Dixon, J. Sorensen, N. Thain, and L. Vasserman Nuanced metrics for measuring unintended bias with real data for text classification. In Companion proceedings of the 2019 world wide web conference, p. 491–500. Cited by: Appendix B. Chao et al. (2024) P. Chao, E. Debenedetti, A. Robey, M. Andriushchenko, F. Croce, V. Sehwag, E. Dobriban, N. Flammarion, G. J. Pappas, F. Tramer, et al. Jailbreakbench: an open robustness benchmark for jailbreaking large language models. Advances in Neural Information Processing Systems 37, p. 55005–55029. Cited by: §A.1, §1, §4, §4. Cohen et al. (2019) J. Cohen, E. Rosenfeld, and Z. Kolter Certified adversarial robustness via randomized smoothing. In international conference on machine learning, p. 1310–1320. Cited by: §2. Ebrahimi et al. (2018) J. Ebrahimi, A. Rao, D. Lowd, and D. Dou Hotflip: white-box adversarial examples for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), p. 31–36. Cited by: §4.2. Gowal et al. (2018) S. Gowal, K. Dvijotham, R. Stanforth, R. Bunel, C. Qin, J. Uesato, R. Arandjelovic, T. Mann, and P. Kohli On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprint arXiv:1810.12715. Cited by: §A.1, §2. Gu and Dao (2024) A. Gu and T. Dao Mamba: linear-time sequence modeling with selective state spaces. In First conference on language modeling, Cited by: §A.1, §1. Gu et al. (2022) A. Gu, K. Goel, A. Gupta, and C. Ré On the parameterization and initialization of diagonal state space models. Advances in neural information processing systems 35, p. 35971–35983. Cited by: §A.1, §3.2. Gu et al. (2021) A. Gu, K. Goel, and C. Ré Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396. Cited by: §A.1, §2.1. Inan et al. (2023) H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y. Mao, M. Tontchev, Q. Hu, B. Fuller, D. Testuggine, et al. Llama guard: llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674. Cited by: §A.1. Jacoby et al. (2020) Y. Jacoby, C. Barrett, and G. Katz Verifying recurrent neural networks using invariant inference. In International Symposium on Automated Technology for Verification and Analysis, p. 57–74. Cited by: §A.1, §2. Jia et al. (2019) R. Jia, A. Raghunathan, K. Göksel, and P. Liang Certified robustness to adversarial word substitutions. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), p. 4129–4142. Cited by: §2.2. Lieber et al. (2024) O. Lieber, B. Lenz, H. Bata, G. Cohen, J. Osin, I. Dalmedigos, E. Safahi, S. Meirom, Y. Belinkov, S. Shalev-Shwartz, et al. Jamba: a hybrid transformer-mamba language model. arXiv preprint arXiv:2403.19887. Cited by: §1. Zhang et al. (2018) H. Zhang, T. Weng, P. Chen, C. Hsieh, and L. Daniel Efficient neural network robustness certification with general activation functions. Advances in neural information processing systems 31. Cited by: §A.1, §A.3, §2. Zou et al. (2023) A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043. Cited by: §A.1, §A.1. Appendix A Appendix A.1 Related Work State Space Models Structured State Space Models (S4) Gu et al. (2021) and Mamba Gu and Dao (2024) demonstrate competitive performance with Transformers on long sequences. The diagonal-real S4 parameterization Gu et al. (2022) and ZOH discretization we use connect directly to classical LTI stability theory, making formal analysis tractable. LLM Safety and Jailbreaking JailbreakBench Chao et al. (2024) provides a standardized framework for evaluating jailbreak attacks. Response-level judges (LlamaGuard Inan et al. (2023), GPT-4 graders) rely on model outputs; string/keyword heuristics Zou et al. (2023) are cheaper but brittle and response-dependent. Our head classifies at the prompt embedding level, avoiding response-level dependencies entirely. Formal Robustness Certification Interval bound propagation (IBP, Gowal et al. (2018)) and CROWN (Zhang et al. (2018)) certify feed-forward networks under lpl_p perturbations. For linear networks, IBP is exact; for nonlinear networks, it introduces relaxation error. Recurrent network certification Jacoby et al. (2020) is less developed. For LTI systems, the A+A^+/A−A^- box propagation is equivalent to exact IBP; the nonlinear activations that require relaxation in general networks are absent. The contraction condition is the LTI analog of a 1-Lipschitz constraint on a feedforward network in that they both ensure that perturbation magnitude is not amplified through the computation graph. Representation-Based Safety Representation Engineering Zou et al. (2023) trains lightweight classifiers on LLM hidden states for alignment tasks, without formal robustness analysis. The linear probe baseline in Section 4 is directly analogous to this line of work; its competitive detection performance is consistent with prior evidence that harmful intent is linearly encoded in large pretrained models. Our contribution is a certification framework for the classifier itself, not a claim that the safety signal is absent from simpler baselines. A.2 Contributions 1. Exact box propagation framework for LTI safety heads The A+A^+/A−A^- splitting and the resulting per-step recursion implement exact interval arithmetic for LTI systems. These are exact in the sense that the computed interval [sTl,sTh][s_T^l,s_T^h] is the tightest possible axis-aligned bounding interval for the reachable output set under input perturbations. This contrasts with nonlinear networks where IBP is only an over-approximation. The implementation (BoxReachSSM) exposes both a final-step API (propagate) and a trajectory API (propagate_trajectory), enabling both scalar certification and reach tube visualization. 2. Hinge penalty training procedure for soft contraction enforcement The training-time hinge penalty is a practically important contribution because it avoids constrained optimization (projection onto the set A:‖A‖∞≤1A: A _∞≤ 1) while achieving the same effect during the natural gradient descent trajectory. This is useful because: (a) it is trivially compatible with any standard optimizers like Adam and SGD, without projection steps; (b) it allows ‖A‖∞ A _∞ to transiently exceed 1 during optimization and recover, which is important for escaping local minima near the boundary; and (c) the hyperparameter λ provides a continuous knob between pure classification and pure contraction. 3. Prompt-only jailbreak detection without response signal Existing jailbreak detection benchmarks largely evaluate models that can observe the LM’s response (StringJudge, GPT-4 graders). Using only the token embedding table of the base model to extract prompt features makes the SSM-head response-agnostic. The advantage of the sequential SSM-head formulation is that it opens the door to formal certification 4. Reachability as Safety head Diagnostic Tool The reach tube figures are a contribution to interpretability for certified models. Rather than reporting a single scalar (certified fraction at a given perturbation radius), the tube shows the full temporal geometry of the reachable output set. These are diagnostics not available from scalar metrics, and they provide intuition for why certain examples are certifiable and others are not. The reachability tool can be applied to any LTI SSM safety head without modification. A.3 Limitations 1. The box-propagation results in (3), (4) are exact only for the LTI model (N=8, d=32, T=64 for our examples). They do not apply to SSMs with nonlinear gating, selection mechanisms, or attention. Extending interval propagation to those settings requires bounding nonlinear activations, which introduces relaxation error as in CROWN Zhang et al. (2018). 2. Certification rates are bounded by logit margins, not by the bound’s looseness. The toy classifier’s mean clean logit on correctly classified toxic examples is 0.14. By Proposition 1, certification at radius δ requires the output interval half-width to be smaller than this margin. At δ=10−3δ=10^-3, the half-width under the constrained model is 0.11-0.18 for most examples, comparable to the margin itself. More data, a larger model, or a margin-maximizing loss would increase margins and improve certification rates at the same δ, and the bound would still be tight. 3. The δ-ball is defined in token embedding space, not character or token space. A formal bridge would require composing the IBP interval with the tokenizer’s discrete-to-continuous map, which is outside the scope of this work. 4. The certified set is a conservative subset of the true robust set. A non-certified example may still be correctly classified under all perturbations in the ball, but the box propagation cannot prove it. Tighter bounds, such as the LP relaxations in the style of CROWN, could certify more examples at the same δ but would be more expensive to compute. A.4 Sketch of proof for Proposition 1 Sketch of proof. [Proposition 1.] (i) We define the box half-width vector wt=(xth−xtl)/2≥0w_t=(x_t^h-x_t^l)/2≥ 0. From (3)-(4), the half-width satisfies the recursion wt=|A|wt−1+δ|B|1→wt= A w_t-1+δ B 1 where |A|ij=|Aij| A _ij= A_ij and |B|ij=|Bij| B _ij= B_ij . Since ‖|A|‖∞=‖A‖∞ A _∞= A _∞, applying the recursion t times gives wt≤‖A‖∞tw0+δ‖B‖∞(I+|A|+⋯+|A|t−1)1→w_t≤ A _∞^tw_0+δ B _∞(I+ A +·s+ A ^t-1) 1. Taking the ∞-norm and summing the geometric series (convergent iff ‖A‖∞<1 A _∞<1) yields (5). Further, when ‖A‖∞≥01 A _∞≥ 01 and r0>0r_0>0, the geometric series diverges and rt→∞r_t→∞. (i) (Certification criterion.) By Definition 2.2, an input x is certified at radius δ iff the classifier assigns the correct label y to every x~ x satisfying ‖x~−x‖∞≤δ x-x _∞≤δ. The SSM classifier assigns label 11 iff sT>0s_T>0 and label 00 iff sT≤0s_T≤ 0. Box propagation computes the exact reachable output interval [sTl,sTh][s_T^l,s_T^h] for all x~ x in the δ-ball, and no relaxation is introduced since the system is LTI. The classifier is correct for all such x~ x iff the interval lies entirely on the correct side: sTl>0s_T^l>0 when y=1y=1 (entire interval is positive), or sTh<0s_T^h<0 when y=0y=0 (entire interval is negative). This equivalence holds exactly because box propagation is tight for LTI systems. (Output interval width bound.) The output equation is sT=CxT+DuTs_T=Cx_T+Du_T. Applying the same IBP argument as above (but at the output map rather than the state transition) the output interval half-width satisfies (sTh−sTl)/2≤‖C‖∞⋅‖(xTh−xTl)/2‖∞+δ‖D‖∞=‖C‖∞⋅rT+δ‖D‖∞(s_T^h-s_T^l)/2≤ C _∞· (x_T^h-x_T^l)/2 _∞+δ D _∞= C _∞· r_T+δ D _∞. Multiplying by 2 yields the stated bound. ∎ Appendix B A Toy Example: Empirical Results To begin with, we use a small LTI classifier to validate our method. The aim of such a ToySSMClassifier is to test and visualize the formal results from proposed methods in Section 3, not to build a competitive toxicity detector. The ToySSMClassifier uses a word-level tokenizer (vocabulary 3,000), 32-dimensional embedding, 8-state LTI SSM (full A∈ℝ8×8A ^8× 8), max length T=64. The training data consists of 10,000 toxic conversation examples drawn from the Civil Comments/Jigsaw Unintended Bias dataset from Borkan et al. (2019). The certification set consists of 600 held-out examples. We utilize two models for a running comparison: a constrained (λ=0.10λ=0.10, which drives ‖A‖∞ A _∞ below 1) and an unconstrained model (λ=0λ=0, which leaves ‖A‖∞=1.091 A _∞=1.091). B.1 Geometric Structure of the Reach Tube Figure 3 shows the conceptual geometric core of the paper. It shows the output reach tube for three examples under both models, making Proposition 1’s prediction directly visible. For the constrained model, the reachable tube converges, while for the unconstrained model’s tube grows with the sequence. In Fig. 3, the first row shows certified toxic examples. The constrained model’s tube stays above zero (stl>0s_t^l>0); the unconstrained model’s tube crosses zero (uncertain). Row 2 shows uncertain toxic example, where both models’ tubes straddle zero, but the constrained tube is substantially narrower. The third row 3 shows certified non-toxic example, where the constrained model’s tube stays below zero (sth<0s_t^h<0). Figure 3: Output reach tube [stl,sth][s_t^l,s_t^h] (shaded) and nominal trajectory stnoms_t^nom (solid) over time steps t=1,⋯,64t=1,·s,64 for three representative examples under the constrained (λ=0.10λ=0.10, left) and unconstrained (λ=0λ=0, right) models. The ‖A‖∞ A _∞ per panel in Fig. 3 shows that the constrained model (0.998) keeps the tube bounded while the unconstrained model (1.091) allows it to widen over time. Under the constrained model, the shaded band stabilizes well before T=64T=64, the tube converges to its steady-state width, consistent with (5). Under the unconstrained model, the band widens monotonically and eventually straddles the decision boundary at s=0 for examples with small margins. The tube width is not a numerical artifact; it is a direct consequence of ‖A‖∞ A _∞ crossing 1. B.2 Certification vs. Perturbation Radius Figure 4 and Table 1 quantify how the certified fraction depends on δ. The key comparison is not just the fraction at a single radius, but whether certification remains non-zero as δ grows (which it does for the constrained model and does not for the unconstrained model). The min–max band over 3 random seeds is shown on the overall panel for the constrained model in Fig. 4, and the curves are Gaussian-smoothed for readability (σ=1.5σ=1.5). Figure 4: Certified fraction vs. perturbation radius δ (log scale, 50 log -spaced points in [10−5,3×10−2][10^-5,3× 10^-2]) for constrained (blue) and unconstrained (red) models. Left: toxic examples only. Center: non-toxic examples only. Right: overall (all correctly classified). δ Con. Toxic Con. Non-tox Con. Overall Unc. Toxic Unc. Non-tox Unc. Overall 1e−41e-4 58.8% 59.3% 59.2% (± 0.24) 24.2% 43.1% 41.0% 2e−42e-4 44.1% 49.2% 48.6% (± 0.26) 9.1% 33.7% 31.0% 5e−45e-4 23.5% 38.4% 36.9% (± 0.24) 6.1% 10.1% 9.7% 1e−31e-3 11.8% 27.3% 25.7% (± 0.16) 0% 0.4% 0.3% 2e−32e-3 5.9% 13.5% 12.7% (± 0.07) 0% 0% 0% Table 1: Certified fractions at representative δ values. Constrained (‖A‖∞=0.999 A _∞=0.999) vs. unconstrained (‖A‖∞=1.091 A _∞=1.091). Overall column: mean ± std across 3 seeds. At δ=10−4δ=10^-4, the constrained model certifies 59% of examples versus 41% for the unconstrained model (a +44% relative gain; see Table 1). The gap is observed to be sharper at larger radii. The constrained model retains 12% certification at δ=2×10−3δ=2× 10^-3, while the unconstrained model certifies zero examples beyond δ=5×10−4δ=5× 10^-4. Seed variance is moderate at intermediate δ (std ≈ 0.16-0.26), reflecting sensitivity to the checkpoint and the small number of correctly classified toxic examples. Non-toxic examples certify at consistently higher rates, consistent with the class-imbalance correction (pos_weight) compressing the toxic logit distribution. B.2.1 Interpreting the Perturbation Radius So far in the toy example, the perturbations to the latent state xtx_t are shown to hold no value for real lexical data. In this section, we translate the abstract δ-ball in embedding space to concrete text-edit magnitudes by measuring the l∞l_∞ distance between word embedding vectors for three pair types: character-drop typos, synonym pairs, and antonym pairs (see Fig. 5. Random word pairs are included as an upper reference. Figure 5: δ calibration to text-edit distances. Left: Violin plot of l∞l_∞ embedding distances by pair type. Gray dotted lines mark the reference δ values used in the experiments. Right: Distribution histograms of pair distances with the same δ reference lines. For the particular ToySSMClassifier experiments, δ=10−4δ=10^-4 falls well below the mean typo-pair distance, meaning the certified perturbation radius is sub-typo-level. At δ=10−3δ=10^-3, coverage reaches the lower end of the synonym pair distribution. And at δ=5×10−3δ=5× 10^-3, the delta value overlaps the synonym distribution for the Civil Comments / Jigsaw Unintended Bias dataset. Note that this mapping is a heuristic proxy, not a formal guarantee. Embedding l∞l_∞ distance between isolated word vectors and token-substitution attack difficulty are correlated but not equivalent. (i) A token substitution changes one position in a sequence, while the δ-ball perturbs all positions simultaneously, (i) The S4 recurrent layers introduce temporal dependencies absent in the static pair-distance measure, and (i) the calibration uses a toy vocabulary (word-level tokenizer, vocabulary ∼ 10K), not the Mamba-130M BPE tokenizer used in Section 4. Therefore, the δ calibration should be read as providing order-of-magnitude intuition for what δ values mean, not as a certificate against concrete token-level attacks. ToySSMClassifier results gallery at δ=3×10−4δ=3× 10^-4, grouped by certification and classification outcome is shown in Fig. 6 Figure 6: Results Gallery for ToySSMClassifier: each cell shows the text, clean logit, and output interval [sl,sh][s^l,s^h]. Border color encodes outcome: red = certified toxic, blue = certified non-toxic, orange = uncertain correct, gray = misclassified. The growth-rate figure closes the loop between theory and experiment. Proposition 1 predicts that for ‖A‖∞<1 A _∞<1, the steady-state interval width is bounded by δ‖B‖∞/(1−‖A‖∞)δ B _∞/(1- A _∞), which diverges as ‖A‖∞→1− A _∞→ 1^-. The empirical sweep (10 trained models at different λ values) confirms this: interval width increases sharply near the phase boundary, and at ‖A‖∞>1 A _∞>1 certification collapses entirely for δ10−3δ 10^-3. Empirical interval width matches the analytic bound near ‖A‖∞=1 A _∞=1 and certified fraction collapses to 0 above it. This validates the analytic prediction as an empirically tight bound at this scale. Figure 7: Growth-rate sweep across 10 λ values: ‖A‖∞ A _∞ vs.λ (left), certified fraction at two δ values (center), and mean output interval width vs. ‖A‖∞ A _∞ with the analytic steady-state bound overlaid (right). B.3 λ and horizon T Sensitivity Figure. 8 shows how the penalty strength λ controls the contraction tradeoff. The phase transition at ‖A‖∞=1 A _∞=1 appears as a clean step change in certification at δ=10−3δ=10^-3, consistent with Proposition 1’s sharpness at this threshold. Accuracy is flat or slightly increasing, confirming that the hinge penalty does not degrade classification performance at this scale. Figure 8: Effect of contraction penalty λ on ‖A‖∞ A _∞ (left), accuracy and recall (center), and certified fraction at δ=10−4δ=10^-4 and δ=10−3δ=10^-3 (right) across 12 λ values. Figure 9 shows certified fraction vs. sequence horizon T∈8,16,24,32,40,48,56,64T∈\8,16,24,32,40,48,56,64\ at δ=10−4δ=10^-4. The constrained model’s cert rate is nearly flat (small variation from rt stabilizing quickly); the unconstrained model’s cert rate falls to zero by T=32 as the tube widens beyond the logit margin. The secondary y-axis shows the analytic upper bound ‖A‖∞T⋅ A _∞^T· : at T=64T=64 is 1.0916410−4≈10^-4 0.104 vs. 0.99964·10−4≈10^-4 9.4×10−5× 10^-5, a 4-order-of-magnitude difference in the perturbation amplification factor. Figure 9: Certified fraction at δ=10−4δ=10^-4 for various T values (primary y-axis) and analytic bound (log scale, right axis). The constrained model’s cert rate is nearly flat; the unconstrained model reaches zero by T=32T=32. B.4 Certification Boundary Analysis Figure A3 (Appendix) shows the 2D scatter of clean logit vs. output interval half-width for each example, colored by certification status. The certification boundary is implicitly defined by: |sTnom|>(sTh−sTl)/2 s_T^nom >(s_T^h-s_T^l)/2. Examples with large positive or negative logits are certified; those near the decision boundary are uncertain regardless of the model’s contraction status, which is a fundamental limitation. Figure 10: Clean logit sTnoms_T^nom vs. output interval half-width scatter at delta=5×10−5delta=5× 10^-5 (left) and δ=2×10−3δ=2× 10^-3 (right); colors encode certification status (blue = certified toxic, light blue = certified non-toxic, orange = uncertain, gray = misclassified). The mean clean logit for toxic examples is ∼ 0.14, meaning many examples sit in the margin zone where the bound nearly ties the logit. The uncertified-but-correct population (uncertain, correct class) is large precisely because of this margin sensitivity. This finding implies that the primary lever for increasing certification is not better regularization of A, it is improving the classification margin, through better features, more training data, or a margin-aware loss function. The bound itself is tight for the constrained model, and the bottleneck is the numerator of the certification criterion, not the denominator. B.5 Horizon Sweep T∈[4,128]T∈[4,128] Figure 11 (left) shows certified fraction at δ=10−4δ=10^-4 for both the constrained (‖A‖∞=0.999 A _∞=0.999) and unconstrained (‖A‖∞=1.091 A _∞=1.091) models. The constrained model degrades gracefully from 0.996 at T=4T=4 to 0.629 at T=64T=64, then plateaus beyond T=64T=64 — a consequence of the bounded reachable set: the horizon no longer matters once the state has converged to the steady-state box. The unconstrained model degrades more steeply (0.992 at T=4 to 0.421 at T=64) and also plateaus, indicating the test set’s margin distribution sets the floor rather than further divergence. The analytic overlay (Fig. 11, right) contrasts the theoretical bound behavior: the steady-state formula δ‖B‖∞/(1−‖A‖∞)δ B _∞/(1- A _∞) for the constrained model is constant across T, while the unconstrained bound ‖A‖∞T⋅δ A _∞T·δ grows exponentially. The plateau observed in empirical certified fractions at T>64T>64 confirms that the constrained model reaches its steady-state before T=64T=64, consistent with the predicted convergence rate. Figure 11: Horizon sensitivity sweep (16 points, T∈[4,128]T∈[4,128]): empirical certified fraction at δ=10−4δ=10^-4 for constrained vs. unconstrained models (left) and analytic bound overlay (right). Appendix C Jailbreaking Example: Supplementary Results The score KDE figure (Fig. 12) is a significant finding here. All six attack families—direct, prefix injection, DAN, developer mode, roleplay, completion—produce score distributions concentrated at logit values +2 to +5, with nearly identical shapes and means. The benign distribution concentrates near -1.5. This similarity across radically different surface forms (DAN uses ”Do Anything Now” framing; completion uses fill-in-the-blank templates; roleplay uses character-based framing) indicates the head is responding to something in the embedding space that is shared across all harmful formulations. In the Mamba-130m embedding table, this shared signal likely corresponds to the semantic proximity of harmful-intent tokens to one another, independent of the surrounding template syntax. Figure 12: SSM-head logit score KDEs by attack family and benign prompts; dashed line = τ=0τ=0. All attack families concentrate above zero; benign prompts cluster below zero, showing consistent score separation regardless of attack template. C.1 Static Threat Model Per-token heatmap The token heatmap provides evidence that SSM-head’s classification is driven by semantically harmful content rather than surface syntax. Template tokens in DAN and roleplay attacks (the wrapper text) receive lower attributions than the underlying harmful request tokens, consistent with the embedding table encoding token identity rather than positional context from generation. Figure 13: Marginal risk contribution per token: each cell is one sub-word token; color intensity = |Δscore| when that token is added (red = positive contribution = raises harm score; blue = negative contribution = lowers harm score). Rows: three direct attack variants, two DAN variants, and four benign prompts. All cells are normalized to the same colormap range per figure. Harmful tokens (verbs like ”malware”, ”weapon”) consistently receive high positive attribution; DAN preamble tokens (”from now on”, ”act as”) receive lower or negative attribution Embedding Space Separation and Reachable Neighborhoods SSM-head operates on mean-pooled Mamba-130M embeddings, a 768-dimensional vector per prompt. The formal reachability analysis and certification of Section. 3 are derived for the toy LTI classifier, and do not directly carry over to SSM-head’s non-linear Mamba backbone. Instead, the subsequent Figures provide an empirical latent-space view: . The true object of interest is the reachable region in 768-dimensional space, but we visualize two projections of it. Figure 15 shows the global structure (2D PCA of all prompts’ nominal embeddings), and Figure 14 zooms into local reachable neighbourhoods; projections of the cloud of embeddings reachable from a single prompt under embedding-space perturbation radius. Neither figure constitutes a formal certified bound, and are illustrative projections of high-dimensional geometry. Figure 14: 2D PCA projection of mean-pooled Mamba-130M embeddings (768→ 2 dims; PC1=9.3%, PC2=5.0%) with harmful JBB (red, n=100), easy-benign (green, n=30), and hard-negative (orange, n=70) prompts; two starred points show l∞l_∞(0.8) perturbation halos (right: 1D projection onto probe normal, AUROC=1.0). Hard-negative prompts straddle the dashed probe boundary, consistent with FPR=0.557 at τ=0τ=0. Figure 15: Projected l∞l_∞ (0.8) perturbation clouds (200 samples each) for four prompts in the same 2D PCA subspace as Fig. 14: harmful high-confidence (cloud fully in detected region), easy-benign (cloud fully in safe region), hard-negative near-boundary (cloud straddles the probe boundary), and harmful near-boundary (cloud partially crosses safe region). Representation Ablation for Mean Pooling To determine whether the linear probe’s strength is specific to mean pooling or a more general property of the Mamba embeddings, we train a logistic regression probe on three different fixed summaries of the 768-dimensional token embedding sequence: (a) the token-wise mean over all non-padding positions (mean pool), (b) the final non-padding token’s embedding (final token), and (c) the first token’s embedding (first token). All variants use the same evaluation set (100 harmful JBB + 100 benign), evaluated via 5-fold cross-validation to avoid train-test leakage. Pooling strategy AUROC (5-fold CV) σ Remark Mean pool (all tokens) 0.998 0.002 Aggregates full prompt context Final non-pad token 0.718 0.029 Much weaker: last token carries little signal alone First token (position 0) 0.515 0.025 Near chance: first token is almost always punctuation/BOS Table 2: Representation ablation: LR probe AUROC (5-fold CV ±σ±σ) across three token-embedding pooling strategies, on 100 harmful JBB + 100 benign prompts. The harmful/benign signal is distributed across the full token sequence, not concentrated at any single position. Mean pooling aggregates this signal effectively; single-token representations lose almost all of it. This confirms that SSM-head’s sequential processing, which also aggregates over the full sequence, has access to the same signal that makes mean-pooled probes strong. The difference is architectural, and the SSM-head can be formally certified, while the mean-pooled probe cannot. Prompt Length Stress Test Figure. 16 shows SSM-head logit score vs. prompt length (5–120 tokens, 11 lengths) for direct attacks, DAN attacks, and benign prompts. Figure 16: Mean ± 10–90th percentile (left) and within-prompt score std across lengths (right). Harmful prompts remain above τ=0τ=0 at all lengths tested; score instability is highest at 5-15 tokens. C.2 Adaptive Threat Model Under adaptive threat models, SSM-head is vulnerable on direct-attack prompts: a black-box greedy substitution attack achieves ASR=0.933, and a white-box gradient-guided attack achieves ASR=1.0. Template-wrapped prompts are unaffected by both attacks (ASR=0.000), because template redundancy distributes the harmful signal across many positions simultaneously. Hard-negative FPR=0.557 at τ=0τ=0 is partially addressable via threshold tuning: τ=2.46τ=2.46 reduces FPR to 0.010 at JDR=0.990. Figure 17: Black-box greedy token substitution attack (10 steps, 30 direct-attack prompts): score trajectories (blue = not evaded, orange = evaded, ASR=0.933) at left, score before vs. after at center, and per-family JDR before vs. after at right. Template attacks maintain JDR=1.0; direct attacks drop to JDR=0.067 after 10 steps. Figure 18: White-box HotFlip-style attack (10 steps, top-5 candidate positions): ASR comparison between black-box (ASR=0.933) and white-box (ASR=1.0) on direct attacks, both ASR=0.0 on DAN/roleplay (left); score trajectories for 30 direct-attack prompts (center); semantic similarity vs. token change rate showing evasion does not require large surface changes (right).