Paper deep dive
Learning to reconstruct from saturated data: audio declipping and high-dynamic range imaging
Victor Sechaud, Laurent Jacques, Patrice Abry, Julián Tachella
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 7/20/2026, 12:13:44 PM
Summary
This paper proposes a self-supervised learning framework for solving non-linear inverse problems, specifically audio declipping and high-dynamic range (HDR) imaging, where ground truth data is unavailable. The method leverages the assumption that signal distributions are invariant to amplitude changes (scale invariance). The authors provide theoretical conditions for model identification and signal recovery from saturated measurements and demonstrate that their approach achieves performance comparable to fully supervised methods using only clipped data for training.
Entities (7)
Relation Signals (5)
Self-supervised learning → appliedto → Audio declipping
confidence 95% · This work extends self-supervised learning to the non-linear problem of recovering audio and images from clipped measurements
Self-supervised learning → appliedto → High-dynamic range imaging
confidence 95% · This work extends self-supervised learning to the non-linear problem of recovering audio and images from clipped measurements
Scale Invariance → assumptionfor → Self-supervised learning
confidence 93% · by assuming that the signal distribution is approximately invariant to changes in amplitude.
Self-supervised learning → overcomeslimitationof → Lack of ground truth
confidence 92% · avoiding the need for ground truth.
Clipping operator → causes → Saturated measurements
confidence 90% · recovering audio and images from clipped measurements
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Learning based methods are now ubiquitous for solving inverse problems, but their deployment in real-world applications is often hindered by the lack of ground truth references for training. Recent self-supervised learning strategies offer a promising alternative, avoiding the need for ground truth. However, most existing methods are limited to linear inverse problems. This work extends self-supervised learning to the non-linear problem of recovering audio and images from clipped measurements, by assuming that the signal distribution is approximately invariant to changes in amplitude. We provide sufficient conditions for learning to reconstruct from saturated signals alone and a self-supervised loss that can be used to train reconstruction networks. Experiments on both audio and image data show that the proposed approach is almost as effective as fully supervised approaches, despite relying solely on clipped measurements for training.
Tags
Links
- Source: https://arxiv.org/abs/2602.22279v1
- Canonical: https://arxiv.org/abs/2602.22279v1
Trouble viewing inline? Open PDF directly →
Full Text
83,411 characters extracted from source content.
Expand or collapse full text
Learning to reconstruct from saturated data: audio declipping and high-dynamic range imaging Victor Sechaud victor.sechaud@ens-lyon.fr CNRS and ENS de Lyon, Laboratoire de physique, Lyon, France Laurent Jacques laurent.jacques@uclouvain.be UCLouvain, ICTEAM, Louvain-la-Neuve, Belgium Patrice Abry patrice.abry@ens-lyon.fr CNRS and ENS de Lyon, Laboratoire de physique, Lyon, France CIFAR Fellow Julián Tachella julian.tachella@ens-lyon.fr CNRS and ENS de Lyon, Laboratoire de physique, Lyon, France Abstract Learning based methods are now ubiquitous for solving inverse problems, but their deployment in real-world applications is often hindered by the lack of ground truth references for training. Recent self-supervised learning strategies offer a promising alternative, avoiding the need for ground truth. However, most existing methods are limited to linear inverse problems. This work extends self-supervised learning to the non-linear problem of recovering audio and images from clipped measurements, by assuming that the signal distribution is approximately invariant to changes in amplitude. We provide sufficient conditions for learning to reconstruct from saturated signals alone and a self-supervised loss that can be used to train reconstruction networks. Experiments on both audio and image data show that the proposed approach is almost as effective as fully supervised approaches, despite relying solely on clipped measurements for training. 1 Introduction Inverse problems appear in various engineering and science applications, such as tomography (Jin et al., 2016), MRI (Lustig et al., 2007) or phase retrieval (Shechtman et al., 2015). They are described by the following forward equation: =h()+ϵ, y=h( x)+ ε, (1) where the aim is to recover the ground-truth ∈⊆ℝn x ^n from measurements ∈⊆ℝm y ^m. The set X defines the support of the of the signal distribution p()p( x), which we refer to as the “signal set”. The function h:ℝn↦ℝmh:R^n ^m represents the forward operator and ϵ∈ℝm ε ^m the noise. These problems are often ill-posed due to potential information loss induced by h (for example, if m<nm<n) or the existence of an infinite number of solutions. One way of solving this is to impose prior information on x, such as piecewise constant assumptions (Rudin et al., 1992). However, choosing the right prior can be challenging and misspecified priors can introduce biases or give poor approximations of the underlying signals. This problem can be mitigated by learning the prior or reconstruction function directly from data. Most learning based methods learn an inverse operator of h on X, f:↦\ f: y x from a training dataset containing ground truth with their associated measurements (i,i)i=1N \( x_i, y_i) \^N_i=1 and a neural network f_ θ that parameterizes the inverse function. The standard approach computes the parameters ∈ℝp θ ^p by minimizing the mean square error (MSE): ^=argmin∑i=1N‖f(i)−i‖2. θ= * \,min _ θΣ ^N_i=1\|f_ θ( y_i)- x_i\|^2. (2) Despite the good performance in some applications, this approach suffers from two main drawbacks: (i) ground-truth images can be particularly difficult or impossible to obtain, e.g., in scientific imaging applications (Belthangady and Royer, 2019), and (i) even when we have access to a ground truth dataset, there might be a significant distribution shift between training and testing. Self-supervised learning presents an alternative that circumvents some of these limitations (Belthangady and Royer, 2019). This approach operates without the need for ground-truth training data, making it especially advantageous in scenarios where obtaining such datasets is expensive or impractical. Furthermore, self-supervised learning can be trained on measurement data directly, thus mitigating the challenge of distribution shift. Learning from measurement data only is particularly challenging when the forward operator is not invertible (Tachella et al., 2023). If the forward operator is linear, self-supervised learning is possible by assuming that the underlying set of clean signals is invariant to a set of transformations, such as rotations or translations (Chen et al., 2021). This also holds for the non-linear problem of reconstructing heavily quantized signals (11-bit measurements) (Tachella and Jacques, 2023). Here we show that invariance to rotations or translations is not enough for learning from clipped measurement data alone, and instead prove that invariance to amplitude is sufficient for fully self-supervised learning. We propose a self-supervised loss that enforces equivariance to changes in amplitude, and show throughout a series of experiments that our method can achieve performance comparable to fully supervised methods. A preliminary version of this work was presented in Sechaud et al. (2024), and here we generalize to a larger class of signals such as images, and propose a theoretical framework showing sufficient conditions for learning to recover signals from saturated measurements alone. In summary, the contributions of this work are threefold: • We provide necessary conditions for signal recovery under a known signal set, and also demonstrate, under an invariance assumption, that the signal set can be identified with a sufficient number of observed measurements; • We propose a self-supervised loss that can be used to train neural networks for audio declipping and High-Dynamic Range (HDR) imaging; • We show that this method can achieve comparable performance to fully supervised methods on both audio and image data. The rest of the paper is structured as follows: Section˜2 reviews related work concerning self-supervised learning and state-of-the-art declipping methods. Section˜3 introduces our theoretical framework, focusing on model identification—i.e., determining underlying signal set from observations—and signal recovery. From there, in Section˜4 we describe how one can practically define and minimize a self-supervised cost function with scale invariance. Section˜5 presents experimental results that validate our approach. 2 Related work 2.1 Self-supervised learning for inverse problems Self-supervised methods allow to train reconstruction networks using only measurement data (Chen et al., 2021). Existing methods can be separated into two categories: the ones that handle the noise of the problem, e.g., Noise2X methods (Lehtinen et al., 2018; Krull et al., 2019), and the others that handle non-invertible forward operators (Tachella et al., 2023). The last set of methods mostly focuses on the case a linear operator where the loss of information is associated to its nullspace. It is possible to overcome this missing information by (i) accessing measurements from several operators with different nullspaces (Daras et al., 2023; Tachella et al., 2023), or (i) by assuming that the signal distribution is invariant to a set of transformations such as translations or rotations, a method known as equivariant imaging (EI) (Chen et al., 2021). The EI framework is well studied for linear inverse problems, both in terms of applications (Scanvic et al., 2024; Chen et al., 2021) and theory (Tachella et al., 2023). To the best of our knowledge, the only case of nonlinear inverse problems analyzed using the invariance approach is one-bit compressed sensing (Tachella and Jacques, 2023). 2.2 Audio declipping Several unsupervised techniques for audio declipping have been proposed (Zaviska et al., 2021; Kitic et al., 2013; Gaultier et al., 2021), many of which make use of variational methods with priors such as sparsity in certain bases (e.g. Fourier domain). Both ℓ0 _0 and ℓ1 _1-minimization approaches are commonly employed, often solved using dedicated optimization algorithms, with social sparsity as one of the best performing (Siedenburg et al., 2014). Learning-based techniques, such as dictionary learning, have also shown to be effective, where a dictionary is trained from time windows of the clipped signal to represent it as a sparse vector. Additionally, methods incorporating prior knowledge based on human perception (Defraene et al., 2013), or using multichannel data (Ozerov et al., 2016; Gaultier et al., 2018), have been used to improve performance. Diffusion models trained on a supervised way (i.e., with ground-truth references) have also been studied in the context of declipping (Moliner et al., 2023). 2.3 High Dynamic Range images The dynamic range in images refers to the difference between the brightest and darkest values. Since camera sensors and displays have limitations in capturing and showing both dark and bright regions, this often results in lost details in these zones. To address this, various techniques are employed to extend the dynamic range of images. One common method is exposure bracketing (Mertens et al., 2007), which combines multiple photos at different exposures for higher quality but requires long exposures and a stationary target to avoid misalignment. Another approach is modulo sensing (Contreras et al., 2024), a promising technique, though it is still limited by the unavailability of commercial hardware. Finally, supervised deep learning methods, such as the ones proposed by Eilertsen et al. (2017), utilize a single image to estimate a high dynamic range. However, these methods require a large dataset of ground truths to train the model effectively, which can be a significant limitation. For example, in ultra-high-speed imaging (thousands of frames per second), sensors must trade off dynamic range for acquisition speed. As a result, high-contrast and transient phenomena—such as combustion in rocket launch site monitoring—remain challenging to capture accurately, even with the most high-end cameras (Giassi et al., 2015; Tao et al., 2025). 2.4 Signal recovery guarantees for saturated measurements Some theoretical works have analysed the feasibility of the signal recovery problem from saturated observations. Following a compressive sensing approach (Candes and Wakin, 2008), Foucart and Needham (2016) present unique recovery guarantees for the case of Gaussian measurements and using an ℓ1 _1 minimization approach, showing that sparse vectors with low magnitude can be uniquely recovered with high probability. 3 Analysis for model identification and signal recovery 3.1 Problem formulation We consider the forward formulation =η() y=η( x), where η:ℝ→ℝη:R is the element-wise clipping operator with threshold levels μ1,μ2 _1, _2: η(u)=μ1 if u≤μ1,u if u∈(μ1,μ2),μ2 if u≥μ2.η(u)= \ array[]rcl _1& if &u≤ _1,\\ u& if &u∈( _1, _2),\\ _2& if &u≥ _2. array . For our examples with audio and synthetic signals, we use a symmetric threshold μ1=−μ2=μ _1=- _2=μ. For images, we only consider an upper threshold, where μ1=0 _1=0 and μ2=1 _2=1, assuming positive signals x, as camera sensors do not suffer saturation from below. Indeed, under low-light conditions, the sensor produces a weaker signal, which may lead to noise or reduced image quality, but not to saturation. We assume that μ1 _1 and μ2 _2 are known and that we have a dataset of saturated signals ii=1N \ y_i \^N_i=1. We aim to learn the reconstruction function f_ θ using the measurement dataset alone. We will therefore consider the following two theoretical questions: (i) model identification: for a forward operator η and measurement sets Y, does a unique signal set X exist satisfying some mild priors and =η()Y=η(X)? In other words, we want to know if it is possible to find the support of the signal distribution X from measurement data ii=1N \ y_i \^N_i=1 alone as N→∞N→∞, and some mild priors on X (e.g. invariance to transformations). If the set X can be identified, we could then hope to recover the signal x from a single observation y via the following program: ^∈argmin∈‖h()−‖2. x∈ * \,min _ x \|h( x)- y\|^2. (3) (i) Signal recovery: is there a unique solution to the problem in (3)? In other words, can we uniquely recover the signal x from observations y knowing the set X? There may be a unique solution for one, both, or neither of the problems (Tachella et al., 2023). Blind compressed sensing is an example where we can have signal recovery if the dictionary is known, but we cannot identify the dictionary from measurement data alone (Gleichman and Eldar, 2011). On the contrary, we can identify the signal set from rank-one measurements without being able to recover the signals associated with each measurement (Chen et al., 2015). Solving these two theoretical problems allows us to assess the feasibility of learning to reconstruct from measurement data alone, using the self-supervised loss proposed in Section˜4. In the following, we will use “identify” only for finding the set X given a measurement set Y, and “recovery” for finding a vector x given a measurement y. 3.2 Model identification. In this section, we will focus on the case where the full set of measurements Y is observed, although in practice we only observe ii=1N\ y_i\^N_i=1. We thus study the conditions for identifying X from the set of saturated signals =η()Y=η(X). It may be interesting to note that in some situations we can easily identify the signal set. This happens in the trivial case where the whole signal set is below the saturation threshold. Example 1. In the symmetric case with threshold level μ and no additional priors on X, if ‖∞<μ\| y\|_∞<μ for all ∈ y , then =X=Y. However, in general without any assumptions on X neither Y, we cannot hope to guarantee identification, as shown in Remark˜1. We thus need to consider some assumptions on X to identify the signal set from Y. As done in equivariant imaging (Chen et al., 2021), we can consider the group of transformations G under which the signal set is invariant. The reason is that the invariance to transformations gg∈G\ T_g\_g∈ G gives us access to the sets g=η(g)Y_g=η( T_gX) for all g∈Gg∈ G. Indeed, we can see Y as a set of measurements associated with the set X for different forward operators η(g⋅)η( T_g·), as we have that: g=η(g)=η()=if g=.Y_g=η( T_gX)=η(X)=Y if T_gX=X. We then need to identify which groups of transformations can help in identifying X. We cannot use the classical groups considered in the linear framework (shift or rotation) as these transformations commute with the operator η. As shown in the following proposition, if the transformations commute with η, they do not help to identify the signal set. Proposition 1. Let G be a group, if we cannot identify X from =η()Y=η(X) and if for all g∈Gg∈ G, the transformations g T_g commute with η(⋅)η(·), then we cannot identify X from g=η(g)Y_g=η( T_gX) for all g∈Gg∈ G. Proof. Let G such that for all g∈G,gη(⋅)=η(g⋅)g∈ G,\; T_gη(·)=η( T_g·). We have access to all measurements associated with the different transformations of the signals: ⋃g∈Gg=yg=η(Tg):∀g∈G,∀∈. _g∈ GY_g= \y_g=η(T_g x):∀ g∈ G,\ ∀ x \. As g T_g and η commute, we have ⋃g∈Gg=yg=gη():∀g∈G,∀∈=⋃g∈Ggη(), _g∈ GY_g= \y_g= T_gη( x):∀ g∈ G,\ ∀ x \= _g∈ G T_gη(X), and so we cannot identify X from that set since it doesn’t bring us more information than η()η(X); the set of measurements is a function of η()η(X). ∎ Example˜1 shows us that the norm of x has an impact on the feasibility of reconstructing the signal, which gets us to consider the group action of scaling. Definition 1 (Scale invariance). We say that X is scale invariant if g= for all g∈ℝ+∗.gX=X for all g ^*_+. (4) A set that is scale-invariant is called a cone (Boyd and Vandenberghe, 2004). Proposition 2. For any two distinct conic sets ≠′X , we have that η()≠η(′)η(X)≠η(X ). Proof. Let the open set μ=∈ℝn:‖∞<μB_μ= \ u ^n\ :\ \| u\|_∞<μ \ with radius μ>0μ>0. We recall that we note Y the set of measurement =η()Y=η(X). Taking the set of unsaturated signals in X, that is ∩μX _μ, the operator η has no impact on them and therefore ∩μ=η(∩μ).X _μ=η(X _μ). Moreover, if x is in μB_μ, then |η()|<μ|η( x)|<μ, therefore η(∩μ)⊂∩μ.η(X _μ) _μ. But if y is in η()∩μη(X) _μ, then y never reaches the threshold μ or −μ-μ, and there is thus an x in ∩μX _μ such that =η()= y=η( x)= x, i.e., (η()∩μ)⊂(∩μ)(η(X) _μ)⊂(X _μ) showing finally that ∩μ=∩μ.Y _μ=X _μ. Defining ^=g:g∈ℝ+∗,∈∩μ X= \g y\ :\ g ^*_+,\ y _μ \ as the conic extension of ∩μY _μ. We will show that ^= X=X. Inclusion ⊂^:X⊂ X: Let ∈ x . Then, there exists c>0c>0 such that =c y=c x belongs to ∩μX _μ. Therefore, =1c x= 1c y belongs to X, which implies ⊂^X⊂ X. Inclusion ^⊂: X : Let ^∈ x∈ X. By definition, there exist g∈ℝ+∗g _+^* and ∈∩μ y _μ such that ^=g.\ x=g y. Since ∩μ=∩μY _μ=X _μ, we have ∈ y . As X is a cone, it follows that ^=g∈ x=g y . Thus, we conclude that ^⊂ X , completing the proof. ∎ Remark 1. For asymmetric clipping, the same result can be obtained for μ1≤0<μ2 _1≤ 0 [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0< _2 and ⊂ℝ+nX _+^n. However, we can find a 22-dimensional counter example for 0<μ1<μ20< _1< _2: we choose 1=a1:a∈ℝ+X_1= \a x_1\ :\ a ^+ \ and 2=a2:a∈ℝ+X_2= \a x_2\ :\ a ^+ \ with 1=(μ2,μ12)⊤ x_1=( _2, _12) and 2=(μ2,μ13)⊤ x_2=( _2, _13) . Then 1X_1 and 2X_2 have the same measurement set: =(t,μ1):t∈(μ1,μ2)∪(μ2,t):t∈(mu1,μ2).Y= \(t, _1):t∈( _1, _2) \∪ \( _2,t):t∈(mu_1, _2) \. 3.3 Signal recovery Sufficient conditions for signal recovery generally depend on the dimension of X (Sauer et al., 1991). Following previous work in compressed sensing (Sauer et al., 1991; Falconer, 2013), we use the box-counting dimension, a measure of the complexity of a set which generalizes several notions of dimension. Definition 2. The upper box-counting dimension of a compact subset X is dim()=lim supϵ→0log[N(ϵ)]−log(ϵ), (X)= _ε→ 0 [N_X(ε) ]- (ε), where N(ϵ)N_X(ε) is the covering number, i.e., the minimum number of closed balls of radius ϵε (with respect to the norm ∥⋅∥2\|·\|_2) with centres in X needed to cover X. It can be applied to various structures: • Vector spaces: The box-counting dimension of a space ℝkR^k intersected with the unit ball is k, corresponding to the classical geometrical dimension. • Manifolds: For a manifold of dimension k, the box-counting dimension is k, consistent with its geometric dimension. • Bounded k-sparse sets: In high dimensional spaces, the box-counting dimension reflects the effective dimension k rather than the ambient dimension n. Thus, the box-counting dimension uniquely determines the dimension of these various objects, providing a flexible framework for measuring the dimension of objects that can appear in compressive sensing or inverse problems. We approach the problem of signal recovery by considering, for a conic signal set X, its normalized version: S=‖2:∈S_X= \ x\| x\|_2\ :\ x \ and the open bounded set R=∈:‖2<R.X_R= \ x \ :\ \| x\|_2<R \. When studying the signal recovery problem, one observes that pathological cases where η(1)=η(2)η( x_1)=η( x_2) for 1≠2 x_1≠ x_2 may arise even when dim() (X) is small, as illustrated in Figure˜1. As the operator η acts as a projection onto the cube μB_μ if the signal set is orthogonal to one of the cube’s faces, it becomes impossible to recover a signal once it reaches that face. However, we expect such cases to be relatively rare, and therefore we study the problem by randomizing the orientation. To this end, as commonly employed in signal recovery analyses (Ahmed et al., 2013), we consider the operator η(⋅)η( A·), where ∈ℝm×n A ^m× n is a random Gaussian matrix, which acts approximaly as a random rotation of the signal set. An illustration is provided in Figure˜1. The following theorem provides sufficient conditions on the number of measurements and the maximum radius R to ensure, with high probability, that distances between points in the set RX_R are approximately preserved after applying η(⋅)η( A·). Section˜5.2.1 provides a simplified example demonstrating how this matrix effectively addresses degenerate cases. Figure 1: Example in 3 dimensions illustrating Theorem˜1. The black set presents a particularly challenging scenario in which all saturated signals are projected to the same point. In contrast, with high probability on A, the colored set enables recovery of more signals with moderate norms, as points in X beyond a certain radius— which is connected to the signal norm when A is Gaussian—are all projected onto a single corner, showing the non-injectivity beyond this radius. Theorem 1. Let us assume that the normalized set S_X has a finite upper box-counting dimension dim(S)<k (S_X)<k, such that N(ϵ)≤ϵ−kN_X(ε)≤ε^-k for all ϵ<ϵ∗ε<ε^* with ϵ∗∈(0,1/2)ε^*∈(0,1/2). For m,n>0m,n>0, given a random matrix ∈ℝm×n A ^m× n with i.i.d. entries Aij∼(0,1)A_ij (0,1 ), a threshold μ>0μ>0, and a radius R>0R>0, there exists an absolute constant C>0C>0 such that, if m≥klog(1/ϵ∗)andR<μ(12−k+1m), m≥ k (1/ε^*)\ and\ R<μ( 12- k+1m), then ℙ(η()=η() for some ≠∈R)≤12e−Cm.P (η( A x)=η( A u) for some x≠ u _R )≤ 12e^-Cm. (5) In other words, we measure the probability that the mapping η∘η A is not injective (and therefore impossible to invert) over RX_R. Theorem 1 shows that the event of picking two vectors of RX_R with identical image through η∘η A has a probability that decays exponentially fast with the dimension m if this dimension is large compared to the dimension k of the normalized set S_X, and the radius is not bigger than μ(12−k+1m)μ( 12- k+1m). This is quite an interesting effect considering that that the fraction of components of A x or A u that are impacted by the saturation η is constant (in average) when m increases. Indeed, for ∈R x _R, each entry of A x is distributed as a Gaussian random variable with zero mean and standard deviation ‖\| x\|. Therefore, for g∼(0,1)g (0,1) and if m is large enough, the fraction of saturated projections m−1|1≤i≤m:|()i|≥μ|m^-1|\1≤ i≤ m:|( A x)_i|≥μ\| is close to m−1∑i=1mℙ(|()i|≥μ)=ℙ(|g|‖≥μ)m^-1 _i=1^mP(|( A x)_i|≥μ)=P(|g|\| x\|≥μ), which does not depend on m. To consider the initial problem, that is, the forward operator η(⋅)η(·), we may introduce an assumption on X by considering it as a cone of the form =:∈,X= \ A z: z \, where Z is a cone with S_Z of box-dimension k and ∈ℝn×n A ^n× n is such that Aij∼(0,1)A_ij (0, [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,01 ). We can apply Theorem˜1 in the case of a random rotation m=nm=n, to show that a signal x can be recovered with high probability if ‖2<‖R\| x\|_2<\| A\|R, where ‖\| A\| denotes the norm of the operator A, and in the case of Gaussian square matrix of variance 1 [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,01, it is close to 2n2 [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0 n (Vershynin, 2018). The proof of the theorem is provided in the Appendix˜A, and sketched in Figure˜1. It consists in showing that the same measurement vector cannot be associated with two distinct signals, with high probability. The core idea of the proof is to decompose the probability that the mapping is not injective into two parts: Given two distinct signals, we control, the probability that the number of common unsaturated measurements is small (thus making it impossible to differentiate between 2 signals). On the other hand, we control the probability that two signals have the same common unsaturated measurements given that the number of saturated measurements is low and so it is acceptable to ignore them. Let us stress that in Theorem˜1 we cannot set an upper bound on the probability in (5) for all signals in X and need to limit this space to bounded signals from RX_R. Some signals in X with an excessive norm cannot be recovered as they have too numerous saturated measurements. In Figure˜1, this is illustrated by the accumulation, after saturation, of signals in the corners of the cube, particularly when signals have high norms, i.e., far from the cube’s center. When all measurements are saturated, or equivalently when μ tends to 0, we expect to recover the behavior of 11-bit compressive sensing (Tachella and Jacques, 2023), where x can only be estimated up to a non-zero error. This is why we focus on signals with low norms. Theorem˜1 can be seen as a generalization of Foucart and Needham (2016, Theorem 1), to more general low-dimensional sets, going beyond sets of k-sparse signals. 4 Self-supervised learning approach Given the positive theoretical results regarding model identification and signal recovery of the previous section, we now propose a method for learning from saturated signals i=η(i)i=1N \ y_i=η( x_i) \_i=1^N. We first introduce the loss functions used to train the network, then describe the network architecture and implementation details used for the reconstruction. 4.1 Loss functions Measurement consistency (MC) loss: in the absence of noise, a good reconstruction method should provide an estimate that is consistent with the observations, i.e., for all measurements y, we should have η(f())=.η(f_ θ( y))= y. Measurement consistency means that the reconstructed signal produces the same measurements as those that were originally observed. Ensuring MC is crucial, as it guarantees that estimates adhere to observations. However, it does not necessarily mean that the solution is unique since multiple solutions could be consistent with the same measurements. A straightforward loss function for imposing measurement consistency is ℒNMC()=∑i=1N‖i−η(f(i))‖2,L_ NMC( θ)=Σ ^N_i=1\| y_i-η (f_ θ( y_i) )\|^2, (6) which we will call the naive measurement consistency loss. If at the first training steps, the network predicts a value x^j x_j above the threshold, the gradient will be zero due to the η operator and thus get stuck on this prediction, preventing the network from learning. This is particularly problematic if yjy_j is below the threshold, since we know that the true value is xj=yjx_j=y_j. We therefore choose to use the MSE where the signal is not saturated according to the following loss: ℒMC()=∑i=1N‖ρ(f(i),i)‖2L_ MC( θ)=Σ ^N_i=1 \|ρ(f_ θ( y_i), y_i) \|^2 (7) where ρ is applied element-wise as: ρ(a,b)=|b|<μ|b−a|+b=μ(b−a)++b=−μ(a−b)+,ρ(a,b)= 1_\|b|<μ\|b-a|+ 1_\b=μ\(b-a)_++ 1_\b=-μ\(a-b)_+, where (⋅) 1(·) the indicator function and (⋅)+(·)_+ the positive function. See Figure˜2 for a comparison of these functions. Figure 2: ℒNMCL_ NMC compared to ℒMCL_ MC. On the left: Curve of the ℒNMCL_ NMC in one dimension for two examples in both cases saturated and not. On the right: Curve of the new ℒMCL_ MC for the same two examples. However, even with this new loss, we cannot hope to learn the correct reconstruction function. Minimizing ℒMCL_ MC allows us to learn a measurement consistent solution f_ θ, but this solution could be of the form: f()j=yj if |yj|≤μv(yj) if |yj|=μf_ θ( y)_j= \ array[]cy_j& if &|y_j|≤μ\\ v(y_j)& if &|y_j|=μ array . (8) with v being any function lying beyond the threshold level μ i.e., v(yj)v(y_j) belongs to the preimage η−1(−μ,μ)η^-1(\-μ,μ\). For example, the function f()=f_ θ( y)= y is a minimizer of (7). Equivariance loss: To effectively learn in the saturated region, we use the amplitude invariance assumption of the signal set defined above, going beyond the limitations imposed by measurement consistency alone. To enforce this invariance property on the reconstruction network, we note that since g∈g x , the function f_ θ must be capable of reconstructing both x and gg x for all x in X and g∈ℝ+g _+, that is: f(η(g))=gandf(η())=.f_ θ (η(g x) )=g x f_ θ(η( x))= x. (9) Thus, we can conclude that: f(η(g))=gf(η()) for all ∈, and g∈ℝ+.f_ θ (η(g x) )=gf_ θ (η( x) ) for all x , and g _+. (10) Hence, the composition of f_ θ with η should be equivariant with respect to the multiplicative group (ℝ+,⋅)(R_+,·). To ensure this, we propose a loss function that enforces this equivariance property: ℒEI()=∑i=1Ng∼pg[‖gf(i)−f(η(gf(i)))‖2].L_ EI( θ)= _i=1^NE_g p_g [ \|gf_ θ ( y_i )-f_ θ (η (gf_ θ ( y_i ) ) ) \|^2 ]. (11) We can observe a connection with Proposition˜2: for N→∞N→∞, after rescaling, the sum becomes an expectation over a distribution supported on Y, and the expectation over g basically acts as a (restricted) conic extension of that support. The expectation is evaluated with respect to a specific distribution pgp_g over the multiplicative group, which, in principle, could be any unbounded distribution over ℝ+R_+. However, in practical scenarios, signals are typically limited in amplitude, making it sufficient to use a distribution defined over a finite range [gmin,gmax][g_ min,g_ max] (not necessarily uniform). The parameters (gmin,gmax)(g_ min,g_ max) were determined empirically: setting gmaxg_ max too low prevents the network from learning effectively, while excessively large values cause instability. We could also extend (11) to include transformations not limited to scale invariance, such as affine group transformations of the form →a+b x→ a x+b 1, where (a,b)∈ℝ+×ℝ(a,b) _+×R and =(1,…,1)⊤∈ℝn 1=(1,…,1) ^n. The final loss function is a combination of the measurement consistency and equivariance losses: ℒ()=ℒMC()+λℒEI().L( θ)=L_ MC( θ)+ _ EI( θ). (12) The parameter λ is a positive hyperparameter that controls the trade-off between the two losses. 4.2 Network architecture 4.2.1 Core architecture The proposed self-supervised loss can be used with any network architecture f_ θ. For most of the experiments reported here, we choose the well-known U-Net neural network for the reconstruction function f_ θ (Ronneberger et al., 2015), which is well-suited for capturing the relevant (temporal or spatial) correlations in natural signals. The details of the architecture vary depending on the type of data, audio or image, and are explained in the experiment section. 4.2.2 Bias-free neural network We choose a bias-free version of the U-Net, to preserve the scale invariance assumption (Mohan et al., 2020). This is because a feedforward neural network with ReLU activation functions and without bias defines a homogeneous function: f(g)=gf()∀g∈ℝ+.f_ θ(g y)=gf_ θ( y) ∀ g _+. This gives f(g)⟶g→00f_ θ(g y) g→ 0 0, which is particularly necessary for unsaturated signals. Figure 3: An example where the reconstruction is learned through the bias (third column) which prevents the reconstruction for low amplitude measurements. We thus observe that removing the bias makes the network naturally homogeneous and improves the reconstruction. On the first row, we consider a signal x, on the second row 10 x10. Figure˜3 shows an example where a biased network cannot reconstruct an unsaturated signal. 4.3 Masking A key aspect of the threshold operator is that, by knowing the threshold levels, we can identify the unsaturated, and therefore non-degraded, portion of the signal. While the measurement consistency loss encourages the network to preserve the unsaturated part, it does not guarantee strict adherence to this constraint. To address this limitation, we can enforce measurement consistency more rigorously by employing the following masking method, comparable to a more structured skip connection architecture. The estimated signals x are computed using an element-wise linear blending between the outputs of the network f()f_ θ( y) and the measurements, that is x^j=(1−bj)yj+bjf()j. x_j=(1-b_j)y_j+b_jf_ θ( y)_j. (13) Figure 4: An image with its associated blending mask. The mask is white where a channel is saturated and black when none is saturated. By choosing an appropriate blend value bjb_j for all sample points or pixels, the blend works as a mask that modifies only saturated measurements, avoiding artifacts in the unsaturated parts. bj=0if |yj|<μ1if |yj|=μb_j= \ array[]c0& if |y_j|<μ\\ 1& if |y_j|=μ array . An example of an image and its associated mask is shown in Figure˜4. 5 Experiments We begin by presenting experiments on toy datasets, where we can control the properties of the signal set. The first experiment uses a dataset associated to a one-dimensional vector space that is invariant to amplitude. The second experiment is performed using various datasets generated by sampling vectors from random k-dimensional subspaces. To generate these datasets, we first control the dimension of the vector space, followed by the saturation proportion for each sampled vector. The reconstruction is then evaluated based on these varying parameters. Next, we conduct experiments on real-world music data, and we compare our method to several state-of-the-art techniques. Finally, we conclude with experiments on two-dimensional signals, where we aim to reconstruct HDR images. 5.1 Metrics Two common metrics for evaluating the quality of reconstructions are the Signal-to-Distortion Ratio (SDR) and Peak Signal-to-Noise Ratio (PSNR). PSNR is more suitable for image quality assessment, as it incorporates a maximum possible pixel value, allowing for a clearer comparison of signal fidelity. SDR, on the other hand, is generally applied to audio signals because it does not require knowledge of the maximum possible value. Although we do not know this value for HDR images, we still use PSNR to provide a more relevant comparison with other works. The PSNR and SDR metrics are defined as follows: PSNR(,^)=20log10(1‖−^‖2), PSNR( x, x)=20 _10 ( 1\| x- x\|_2 ), (14) SDR(,^)=20log10(‖2‖−^‖2). SDR( x, x)=20 _10 ( \| x\|_2\| x- x\|_2 ). (15) Both metrics are referred to as distortion metrics since they assess the fidelity of the reconstruction process. In addition to distortion metrics, perception metrics could be used to evaluate the quality of the reconstruction, offering a more accurate measure of how changes to a signal or image affect human perception. In this work, we employ the widely adopted metric Perceptual Evaluation of Speech Quality (PESQ) to evaluate audio signals. For image, we use the Natural Image Quality Evaluator (NIQE), a no-reference (no ground-truth needed) metric that estimates image quality. While perceptual metrics are particularly relevant for applications like music audio, where subjective experience plays a key role, they are less suitable for signals where preserving critical information is essential, as they tend to permit greater hallucination - introducing elements that were not part of the original signal, potentially compromising the accuracy of the reconstruction. Blau and Michaeli (2018) study the impossibility of algorithms to optimize both perception and distortion metrics, known as the perception-distortion tradeoff. 5.2 Toy datasets 5.2.1 MNIST experiment In this example, we use a toy signal set, generated from a single signal and augmented with its scaled versions, i.e. =e0:e∈ℝ+∗X= \e x_0:e ^*_+ \ for 0 x_0 fixed. In practice, we take an image x from MNIST and create the dataset =ei0i=1ND= \e_i x_0 \^N_i=1 with eie_i a realization of a random variable with exponential distribution of mean 22, for i∈1,…,Ni∈ \1,…,N \. This dataset has two interesting properties: it is scale invariant, and we cannot have unique signal recovery as shown in Figure˜5. Figure 5: Example where two different signals 1,2 x_1, x_2 have the same measurement η(1),η(2)η( x_1),η( x_2) and so the network f_ θ fails to recover them both (the second and fourth columns). Adding randomness implies that the two measurements are not equal anymore, and so a network f′f^ _ θ can reconstruct the original signal (the third and fifth columns). Measurements are obtained by applying the forward function η(⋅)η( A·) with A an orthogonal matrix and threshold levels μ1=0,μ2=0.4 _1=0,\ _2=0.4. The matrix A is drawn randomly with respect to the Haar measure (Mezzadri, 2007). While Figure˜5 shows that it is impossible without A to recover the original signals, adding randomness in the forward operator solves this issue, as illustrated in Figure˜1. Figure˜6 shows that the neural network trained using the loss function ℒMC+ℒEIL_ MC+L_ EI successfully reconstructs images with a dynamic range similar to the original images, whereas the network trained with ℒNMC+ℒEIL_ NMC+L_ EI as the loss function does not achieve this. However, both fail when the dynamic range of the original signal becomes excessively high. For signals with high dynamic range, characterized by a large norm ∥⋅∥2\|·\|_2, the network is unable to recover them as indicated in Theorem˜1. Figure 6: Plot of the dynamic range of f()f_ θ( y) depending on dynamic range of x, where =η() y=η( A x). Left: The training is done with the losses ℒNMC+ℒEIL_ NMC+L_ EI. Right: The training is done with loss ℒMC+ℒEIL_ MC+L_ EI. A perfect plot should be the identity line. 5.2.2 Synthetic dataset First, we generate a random subspace of ℝ100R^100 with dimension k∈1,…,100k∈\1,…,100\ using basis vectors whose coordinates are drawn from a standard normal distribution. Next, we generate N=1000N=1000 vectors ii=1N\ x_i\_i=1^N within this subspace, rescaling each vector such that a proportion v∈(0,1)v∈(0,1) of their components are clipped with the threshold μ set to 1. We evaluate the recovery performance as a function of the parameters k and v. Figure˜7 illustrates the network’s ability to reconstruct unsaturated signals as these parameters vary. The performance decreases as the subspace dimension grows or the proportion of saturated samples increases. Indeed, signals from higher-dimensional subspaces are more challenging to reconstruct, which is consistent with the result of Theorem˜1. Figure 7: Average reconstruction performance as a function of saturated part v and signal set dimension k for both supervised (left) and self-supervised (right) methods. The value indicated corresponds to the mean SDR over the test dataset. 5.3 Audio We evaluate the method on a real audio dataset. Specifically, we use the GTZAN dataset, which contains 10 genres (rock, classical, jazz, etc.), each represented by 100 audio files, all 30 seconds long with a sampling rate of 22,050 Hz. The audio signals are split into 30 segments of 1-second each and then saturated with a threshold level μ=0.1μ=0.1. Measurements y without any saturated entries are discarded. This process results in a training dataset of 21898 audio samples and a testing dataset of 100 samples. The distribution of pgp_g is set uniformly on the interval [0.1,2][0.1,2]. Results are compared with a state-of-the-art variational method called social sparsity (Zaviska et al., 2021; Siedenburg et al., 2014) and supervised methods. Both the supervised and self-supervised approaches use the same network architecture for training. We refer to the method trained using only 5% of the whole training dataset as “Supervised 5%”, which is used to compare the performance of the both approaches under limited ground-truth data acquisition. We also compare our method with the following methods: self-supervised with loss ℒMCL_ MC alone, supervised with MSE + ℒEIL_ EI. As shown in Table˜1, the proposed self-supervised method achieves performance comparable to the fully supervised approach, although the latter requires significantly less training time: 7 hour 46 minutes compared to 13 hours 52 minutes for the self-supervised method. The results further confirm that training with the MC loss alone does not yield additional information beyond the measurements themselves, as anticipated in Section˜4. Similarly, the proposed method with a biased network does not perform as well, illustrating the importance of using a bias-free architecture as seen in Section˜4.2.2. In addition, the supervised method with equivariance does not demonstrate any significant improvement over the standard supervised approach. Table 1: Reconstruction performance, SDR and PESQ are averaged over all music test dataset. Methods SDR ↑ PESQ ↓ Identity 4.84±2.124.84± 2.12 2.7±0.652.7± 0.65 Social Sparsity 9.92±4.469.92± 4.46 2.09±0.952.09± 0.95 Supervised 11.69±2.2511.69± 2.25 1.94±0.751.94± 0.75 Supervised 5% 9.79±1.599.79± 1.59 2.28±0.782.28± 0.78 Supervised + ℒEIL_ EI 11.72±2.2311.72± 2.23 1.92±0.741.92± 0.74 Proposed self-supervised 10.48±2.2010.48± 2.20 2.20±0.802.20± 0.80 Unsupervised with ℒMCL_ MC alone 4.84±2.124.84± 2.12 2.68±0.652.68± 0.65 Proposed self-supervised with bias 9.27±3.229.27± 3.22 2.43±0.902.43± 0.90 To demonstrate that a supervised training dataset may not generalize well to a different test set, we conduct the following experiment: we create a supervised training dataset consisting of only music, while the test dataset includes both music and voice recordings (Vryzas et al., 2018a; b), with only measurement data available (no ground truth). The supervised method, trained using the MSE loss (2), learns solely from the training dataset, whereas the self-supervised method is trained on both the training and test datasets since it does not require ground truth. Both methods are then evaluated on the test dataset. The results, shown in Table˜2, demonstrate that the self-supervised method is more robust when the training and test datasets differ. Table 2: Average SDR performance on the test dataset, which includes both music and voice recordings. Methods SDR (dB) Identity 6.54±2.346.54± 2.34 Supervised (trained on music) 10.94±2.0010.94± 2.00 Proposed self-supervised (trained on music and voice) 11.92±2.4611.92± 2.46 Table 3: Average SDR and PESQ performance on the test MAESTRO dataset. Methods SDR ↑ PESQ ↓ Identity 5.00±05.00± 0 2.9±0.732.9± 0.73 Supervised 6.70±1.696.70± 1.69 2.22±0.522.22± 0.52 Proposed self-supervised 7.78±8.647.78± 8.64 2.47±1.122.47± 1.12 We also compare the self-supervised learning method to a supervised learning approach using diffusion models (Moliner et al., 2023). The comparison is conducted on a portion of the MAESTRO dataset (Hawthorne et al., 2019), used in Moliner’s work, which consists of 200 hours of classical solo piano recordings. The measurement signals from Moliner et al. (2023) are used as a benchmark; although the saturation threshold varies for each signal, it consistently corresponds to an SDR value of 55. Each signal is then normalized to yield a saturation threshold of 0.1, while preserving a constant SDR of 55. This approach enables comparison between the proposed method, based on a fixed saturation threshold, and Moliner’s trained method. Results demonstrate that the self-supervised method outperforms the supervised one Table˜3. It is important to note that the proposed self-supervised method focuses on optimizing the distortion metric SDR, whereas this supervised method aims to optimize a perceptual metric PESQ. Results are then consistent with the perception-distortion tradeoff seen in Section˜5.1. 5.4 HDR Figure 8: Image reconstruction on the test dataset. An exposition correction (Wikipedia, ) is applied to see details better. The HDR experiment is inspired by Eilertsen et al. (2017), and its goal is to compare supervised and self-supervised methods. Results are summarized in Table˜4 and in Figure˜8. The dataset is composed of 10431043 HDR images noted u (Le et al., 2023), which are considered as real scenes. Photos are taken with a virtual camera calibrated by a camera curve and exposure. For each image, we choose the exposure time to saturate between 5% and 15% of the pixels. We divide the image by the quantile qvq_v where v is chosen uniformly in (0.85, 0.95) (0.85\ ,\ 0.95 ). Then we have ℙ(qv>1)=1−vP ( uq_v>1 )=1-v if ℙP is the cumulative histogram of u. The camera curve is defined as ω()j=(1+σ)ujβuiβ+σ,ω( u)_j=(1+σ) u_j^βu_i^β+σ, We use these functions to fit the database of real camera curves collected by Grossberg and Nayar (2003). We choose β∼(0.9,0.1)β (0.9,0.1) and σ∼(0.6,0.1)σ (0.6,0.1). The images =ω(qv) x=ω ( uq_v ) represent the ground truth dataset. They are quantized and clipped to create the measurement dataset. This step is performed with: yj=⌊255min(1,xj)+0.5⌋255.y_j= 255 (1,x_j)+0.5 255. (16) We use an unfolded network with a U-Net architecture. It combines traditional iterative optimization with neural network learning. For each iteration, we replace a part of the optimization step with a neural network. We choose the Half-Quadratic Splitting (HQS) (Aggarwal et al., 2018) algorithm which aims to minimize φ(x)+λψ(x) (x)+λψ(x), for two closed proper convex functions φ and ψ. The iteration step is given by: uk u_k =proxγφ(xk), =prox_γ (x_k), xk+1 x_k+1 =proxσλψ(uk). =prox_σλψ(u_k). In our setting, φ corresponds to the measurement consistency term φ()=‖ρ(,)‖2 ( x)=\|ρ( x, y)\|^2 and proxσλψ(uk)prox_σλψ(u_k) is replaced by a neural network. The proximal operator of the measurement consistency term is given by: proxγφ()j=xj+γyj1+γif|xj|≤μ,xjotherwiseprox_γ ( x)_j= \ array[]l x_j+γ y_j1+γ |x_j|≤μ,\\ x_j array . for j∈1,…,nj∈\1,…,n\. Table 4: Reconstruction performance, PSNR is averaged over all test dataset images. Methods PSNR ↑ NIQE ↓ Identity 29.54±6.9529.54± 6.95 4.21±1.964.21± 1.96 PnP DPIR 32.11±5.8032.11± 5.80 4.06±1.934.06± 1.93 Supervised 36.67±6.1936.67± 6.19 4.04±1.744.04± 1.74 Proposed self-supervised 35.04±6.3335.04± 6.33 4.10±1.854.10± 1.85 In Table˜4, we compare our method against two reference approaches: the supervised method from Eilertsen et al. (2017) and the Plug-and-Play (PnP) method (Venkatakrishnan et al., 2013). The PnP method is an iterative technique that integrates a denoising module into the optimization process, replacing the proximal operator. We used the Deep Plug-and-Play Image Restoration (DPIR) method (Zhang et al., 2021), a PnP approach based on HQS in which the denoising network is a pre-trained DRUNet, and the noise level for the denoiser is adapted at each iteration. For better results, we modified the initialization of the algorithm – which is generally set to y – by adding a white Gaussian noise of standard deviation σ=0.18σ=0.18, which we found to better avoid local minima close to y. In this way, DPIR outperforms the other PnP methods tested. Results in Table˜4, demonstrate that the self-supervised method performs on par with the supervised method and significantly outperforms the DPIR approach. As illustrated in Figure˜8, both the supervised and self-supervised methods effectively recover details in the images. 6 Conclusion In this study, we propose a self-supervised method that uses amplitude invariance to address the nonlinear declipping problem. We provide a theoretical framework with guarantees for model identification of scale invariant signal models and unique recovery of the unsaturated signals when the operator involves random matrices with coefficients following a Gaussian distribution. Experimental results, carried out on audio and images, indicate that this method can perform on par with the supervised approach, and surpasses variational methods. This work opens up new possibilities for learning-based methods in nonlinear inverse problems. 7 Acknowledgment Victor Sechaud and Julian Tachella are supported by the ANR grant UNLIP (ANR-23-CE23-0013). The reserch of Laurent Jacques is partly funded by FRS-FNRS (QuadSense, T.0160.24). Part of this research was carried out when LJ was supported by the OCKHAM team at ENS Lyon (France), with an INRIA Chair at the Institute for Advanced Study, Collegium of Lyon. References H. K. Aggarwal, M. P. Mani, and M. Jacob (2018) MoDL: Model-based deep learning architecture for inverse problems. IEEE transactions on medical imaging 38 (2), p. 394–405. Cited by: §5.4. A. Ahmed, B. Recht, and J. Romberg (2013) Blind deconvolution using convex programming. IEEE Transactions on Information Theory 60 (3), p. 1711–1732. Cited by: §3.3. C. Belthangady and L. A. Royer (2019) Applications, Promises, and Pitfalls of Deep Learning for Fluorescence Image Reconstruction. Vol. 16, Nature Publishing Group US New York (en). Cited by: §1, §1. Y. Blau and T. Michaeli (2018) The Perception-Distortion Tradeoff. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 6228–6237. External Links: 1711.06077 Cited by: §5.1. S. P. Boyd and L. Vandenberghe (2004) Convex optimization. Cambridge university press. Cited by: Definition 1. E.J. Candes and M.B. Wakin (2008) An Introduction To Compressive Sampling. IEEE Signal Processing Magazine 25 (2), p. 21–30 (en). External Links: ISSN 1053-5888 Cited by: §2.4. D. Chen, J. Tachella, and M. E. Davies (2021) Equivariant Imaging: Learning Beyond the Range Space. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 4379–4388 (en). Cited by: §1, §2.1, §3.2. Y. Chen, Y. Chi, and A. J. Goldsmith (2015) Exact and Stable Covariance Estimation from Quadratic Sampling via Convex Programming. IEEE Transactions on Information Theory 61 (7), p. 4034–4059. Cited by: §3.1. K. Contreras, B. Monroy, and J. L. Bacca (2024) High dynamic range modulo imaging for robust object detection in autonomous driving. In ROAM ECCV 2024, Cited by: §2.3. G. Daras, K. Shah, Y. Dagan, A. Gollakota, A. Dimakis, and A. Klivans (2023) Ambient diffusion: learning clean distributions from corrupted data. Advances in Neural Information Processing Systems 36, p. 288–313. Cited by: §2.1. B. Defraene, N. Mansour, S. De Hertogh, T. Van Waterschoot, M. Diehl, and M. Moonen (2013) Declipping of audio signals using perceptual compressed sensing. IEEE Transactions on Audio, Speech, and Language Processing 21 (12), p. 2627–2637. Cited by: §2.2. G. Eilertsen, J. Kronander, G. Denes, R. K. Mantiuk, and J. Unger (2017) HDR image reconstruction from a single exposure using deep CNNs. ACM Transactions on Graphics 36 (6), p. 1–15 (en). External Links: ISSN 0730-0301, 1557-7368 Cited by: §2.3, §5.4, §5.4. K. Falconer (2013) Fractal geometry: mathematical foundations and applications. John Wiley & Sons. Cited by: §3.3. S. Foucart and T. Needham (2016) Sparse recovery from saturated measurements. Information and Inference, p. iaw020 (en). External Links: ISSN 2049-8764, 2049-8772 Cited by: §2.4, §3.3. C. Gaultier, N. Bertin, and R. Gribonval (2018) Cascade: Channel-aware structured cosparse audio declipper. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 571–575. Cited by: §2.2. C. Gaultier, S. Kitić, R. Gribonval, and N. Bertin (2021) Sparsity-based audio declipping methods: Selected overview, new algorithms, and large-scale evaluation. IEEE/ACM Transactions on Audio, Speech, and Language Processing 29, p. 1174–1187. Cited by: §2.2. D. Giassi, B. Liu, and M. B. Long (2015) Use of high dynamic range imaging for quantitative combustion diagnostics. Applied optics 54 (14), p. 4580–4588. Cited by: §2.3. S. Gleichman and Y. C. Eldar (2011) Blind compressed sensing. IEEE Transactions on Information Theory 57 (10), p. 6958–6975. Cited by: §3.1. M. Grossberg and S.K. Nayar (2003) What is the Space of Camera Response Functions?. In 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., Vol. 2, p. I – 602. Cited by: §5.4. C. Hawthorne, A. Stasyuk, A. Roberts, I. Simon, C. A. Huang, S. Dieleman, E. Elsen, J. Engel, and D. Eck (2019) Enabling factorized piano music modeling and generation with the MAESTRO dataset. In International Conference on Learning Representations, Cited by: §5.3. K. H. Jin, M. T. McCann, and M. Unser (2016) BPConvNet for compressed sensing recovery in bioimaging. Cited by: §1. S. Kitic, L. Jacques, N. Madhu, M. P. Hopwood, A. Spriet, and C. De Vleeschouwer (2013) Consistent Iterative Hard Thresholding for Signal Declipping. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, (Anglais). Cited by: §2.2. A. Krull, T. Buchholz, and F. Jug (2019) Noise2void-learning denoising from single noisy images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 2129–2137. Cited by: §2.1. P. Le, Q. Le, R. Nguyen, and B. Hua (2023) Single-Image HDR Reconstruction by Multi-Exposure Generation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Cited by: §5.4. J. Lehtinen, J. Munkberg, J. Hasselgren, S. Laine, T. Karras, M. Aittala, and T. Aila (2018) Noise2Noise: Learning image restoration without clean data. International Conference on Machine Learning. Cited by: §2.1. M. Lustig, D. Donoho, and J. M. Pauly (2007) Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine 58 (6), p. 1182–1195. Cited by: §1. T. Mertens, J. Kautz, and F. Van Reeth (2007) Exposure Fusion. In 15th Pacific Conference on Computer Graphics and Applications (PG’07), Maui, HI, p. 382–390 (en). External Links: ISBN 978-0-7695-3009-3 Cited by: §2.3. F. Mezzadri (2007) How to generate random matrices from the classical compact groups. Vol. 54, p. 592 – 604 (en). Cited by: §5.2.1. S. Mohan, Z. Kadkhodaie, E. P. Simoncelli, and C. Fernandez-Granda (2020) Robust and interpretable blind image denoising via bias-free convolutional neural networks. In International Conference on Learning Representations, (en). Cited by: §4.2.2. E. Moliner, J. Lehtinen, and V. Välimäki (2023) Solving Audio Inverse Problems with a Diffusion Model. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (en). Cited by: §2.2, §5.3. A. Ozerov, Ç. Bilen, and P. Pérez (2016) Multichannel audio declipping. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 659–663. Cited by: §2.2. G. Puy, M. E. Davies, and R. Gribonval (2017) Recipes for Stable Linear Embeddings From Hilbert Spaces to ℝmR^m. IEEE Transactions on Information Theory 63 (4), p. 2171–2187 (en). External Links: ISSN 0018-9448, 1557-9654 Cited by: Appendix A, Theorem 2. J. C. Robinson (2010) Dimensions, embeddings, and attractors. Vol. 186, Cambridge University Press. Cited by: Appendix A, Theorem 3. O. Ronneberger, P. Fischer, and T. Brox (2015) U-net: convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part I 18, p. 234–241. Cited by: §4.2.1. L. I. Rudin, S. Osher, and E. Fatemi (1992) Nonlinear total variation based noise removal algorithms. Physica D: nonlinear phenomena 60 (1-4), p. 259–268. Cited by: §1. T. Sauer, J. A. Yorke, and M. Casdagli (1991) Embedology. Journal of Statistical Physics 65 (3-4), p. 579–616. Cited by: §3.3. J. Scanvic, M. Davies, P. Abry, and J. Tachella (2024) Self-Supervised Learning for Image Super-Resolution and Deblurring. arXiv (en). Cited by: §2.1. V. Sechaud, L. Jacques, P. Abry, and J. Tachella (2024) Equivariance-based self-supervised learning for audio signal recovery from clipped measurements. In 2024 32nd European Signal Processing Conference (EUSIPCO), p. 852–856. Cited by: §1. Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev (2015) Phase Retrieval with Application to Optical Imaging: A contemporary overview. IEEE Signal Processing Magazine 32 (3), p. 87–109 (en). External Links: ISSN 1053-5888 Cited by: §1. K. Siedenburg, M. Kowalski, and M. Dorfler (2014) Audio declipping with social sparsity. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy, p. 1577–1581 (en). External Links: ISBN 978-1-4799-2893-4 Cited by: §2.2, §5.3. J. Tachella, D. Chen, and M. Davies (2023) Sensing theorems for unsupervised learning in linear inverse problems. Journal of Machine Learning Research 24 (39), p. 1–45. Cited by: §1, §2.1, §3.1. J. Tachella and L. Jacques (2023) Learning to reconstruct signals from binary measurements. In Fourteenth International Conference on Sampling Theory and Applications, Cited by: §1, §2.1, §3.3. J. Tao, Y. Zong, B. Guan, P. Sun, T. Lei, Y. Shang, and Q. Yu (2025) Perceptual region-driven infrared-visible co-fusion for extreme scene enhancement. Optics & Laser Technology 192, p. 113870. Cited by: §2.3. S. V. Venkatakrishnan, C. A. Bouman, and B. Wohlberg (2013) Plug-and-play priors for model based reconstruction. In 2013 IEEE global conference on signal and information processing, p. 945–948. Cited by: §5.4. R. Vershynin (2018) High-dimensional probability: An introduction with applications in data science. Vol. 47, Cambridge university press. Cited by: Appendix A, §3.3. N. Vryzas, R. Kotsakis, A. Liatsou, C. A. Dimoulas, and G. Kalliris (2018a) Speech emotion recognition for performance interaction. Journal of the Audio Engineering Society 66 (6), p. 457–467. Cited by: §5.3. N. Vryzas, M. Matsiola, R. Kotsakis, C. Dimoulas, and G. Kalliris (2018b) Subjective evaluation of a speech emotion recognition interaction framework. In Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion, p. 1–7. Cited by: §5.3. [48] Wikipedia Exposure value. Note: https://en.wikipedia.org/wiki/Exposure_value Cited by: Figure 8. P. Zaviska, P. Rajmic, A. Ozerov, and L. Rencker (2021) A Survey and an Extensive Evaluation of Popular Audio Declipping Methods. IEEE Journal of Selected Topics in Signal Processing 15 (1), p. 5–24 (en). External Links: ISSN 1932-4553, 1941-0484 Cited by: §2.2, §5.3. K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool, and R. Timofte (2021) Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (10), p. 6360–6376. Cited by: §5.4. Appendix A Appendix: Proof of Theorem 1 We recall that the idea of the proof is to upper-bound two probabilities: (i) the probability that the number of common unsaturated measurements is small; (i) the probability that two signals have the same common unsaturated measurements given that the number of saturated measurements is low. Before we begin, we need to introduce some tools of linear embedding. Theorem˜2 allows us to prove Corollary˜2.1 used to control the first targeted probability. Theorem˜3 is used to control the second one. We also recall the Hoeffding inequality for sub-Gaussian random variables, which we will use to verify an assumption for Theorem˜2. Lemma 1. Hoeffding inequality. Let X1,…,XmX_1,…,X_m be independent random variables, where XiX_i has mean μi _i and is σi _i-sub-Gaussian. Then for all t≥0t≥ 0 we have ℙ(|∑i=0mXi−μi|≥t)≤2exp(−t22∑i=0mσi2)P ( |Σ^m_i=0X_i- _i |≥ t )≤ 2 (- t^22Σ ^m_i=0σ^2_i ) Theorem 2. This theorem is adapted from Puy et al. (2017, Theorem I.2). Let us assume that the normalized set S_X has a finite upper box-counting dimension dim(S) (S_X) which is strictly bounded by k≥1k≥ 1 i.e. dim(S)<k (S_X)<k, and so there exists a signal set dependent constant ϵ∗∈(0,12)ε^*∈ (0, 12 ) such that NS(ϵ)≤ϵ−kN_S_X(ε)≤ε^-k for all ϵ≤ϵ∗ε≤ε^*. Let :Rn→ℝm A:R^n ^m such that Aij∼(0,1m)A_ij (0, 1m ). Then there exists c1>0c_1>0 such that, for any ξ,δ∈(0,1)ξ,δ∈(0,1), we have sup∈S|c‖‖1−1|≤δ _ x∈ S_X |c\| A x\|_1-1 |≤δ with probability at least 1−ξ1-ξ provided that m≥C1c1δ2max(klog(1ϵ∗),log(6ξ)),m≥ C_1c_1δ^2 (k ( 1ε^* ), ( 6ξ ) ), where C1C_1 is an absolute constant. The proof of Theorem˜2 closely follows that of Puy et al. (2017, Theorem I.2). It consists in using a general chaining argument (see, e.g., Vershynin (2018, Chapter 8)) on the function ∈ℝn↦h():=c‖‖1−‖2 x ^n h( x):=c\| A x\|_1-\| x\|_2 provided that, for any fixed vectors ,∈S∪ u, v∈ S_X∪ \ 0 \ and with high probability, the absolute difference |h()−h()||h( u)-h( v)| does not deviate too much from the distance ‖−‖\| u- v\|, a fact that we prove the following lemma. Lemma 2. Let :ℝn→ℝm A:R^n ^m such that Aij∼(0,1m)A_ij (0, 1m ), c>0c>0, and h be defined as h:ℝn→ℝ↦c‖‖1−‖2. array[]ccclh:&R^n&→&R\\ & x& &c\| A x\|_1-\| x\|_2. array Then there exist a constant c1∈ℝ+c_1 ^+ such that for any fixed ,∈S∪ u, v∈ S_X∪ \ 0 \, ℙ(|h()−h()|≥λ‖−‖)≤2e−c1mλ2∀λ∈ℝ+. (|h( u)-h( v)|≥λ\| u- v\| )≤ 2e^-c_1mλ^2 ∀λ ^+. Proof. We consider h:↦c‖‖1−‖2h: x c\| A x\|_1-\| x\|_2 with c>0c>0. Restricted to S_X, we have h:↦c‖‖1−1h: x c\| A x\|_1-1. ℙ(|h()−h()|≥λ‖−‖) 14.0ptP (|h( u)-h( v)|≥λ\| u- v\| ) (17) =ℙ(c|‖‖1−‖‖1|≥λ‖−‖) =P(c|\| A u\|_1-\| A v\|_1|≥λ\| u- v\|) =ℙ(c|∑i=0m|i⊤|−|i⊤||≥λ‖−‖). =P (c |Σ ^m_i=0| a _i u|-| a _i v| |≥λ\| u- v\| ). (18) We note Xi=|i⊤|−|i⊤|X_i=| a _i u|-| a _i v|. Let’s show that XiX_i is sub-Gaussian: For all q∈ℕq and for all i∈1,…,mi∈ \1,…,m \, [|Xi|q]=E[||i⊤|−|i⊤||q]≤E[|i⊤(−)|q]E [|X_i|^q ]=E [ || a _i u|-| a _i v| |^q ]≤ E[| a _i( u- v)|^q] by the reverse triangle inequality. As i⊤(−)∼(0,‖−‖22m), a _i( u- v) (0, \| u- v\|^2_2m ), i⊤(−) a _i( u- v) is Gaussian and thus XiX_i is also sub-Gaussian with parameter σi2≤‖−‖22m.σ^2_i≤ \| u- v\|^2_2m. We can therefore apply the Lemma˜1: ℙ(c|∑i=0mXi|>t)≤2exp(−t22c2∑i=0mσi2)P (c |Σ ^m_i=0X_i |>t )≤ 2 (- t^22c^2Σ _i=0^mσ^2_i ) and thus give an upper bound of (18): ℙ(|h()− (|h( u)- h()|≥λ∥−∥) h( v)|≥λ\| u- v\|) ≤2exp(−(λ‖−‖)22c2∑i=0mσi2) ≤ 2 (- (λ\| u- v\|)^22c^2Σ _i=0^mσ^2_i ) ≤2exp(−(λ‖−‖)22c2∑i=0m‖−‖22m) ≤ 2 (- (λ\| u- v\|)^22c^2Σ _i=0^m \| u- v\|^2_2m ) ≤2exp(−λ22c2) ≤ 2 (- λ^22c^2 ) =2exp(−mc1λ2). =2 (-mc_1λ^2 ). We conclude the proof with c1=12mc2c_1= 12mc^2. ∎ Corollary 2.1. Under assumptions of Theorem˜2, we have for all m≥klog(1ϵ∗)m≥ k ( 1ε^* ): ℙ(∀∈:∥1≤m∥2)>1−6e−Cm.P (∀ x :\| A x\|_1≤ m\| x\|_2 )>1-6e^-Cm. Proof. By applying Theorem˜2 for c=2mc= 2 m we deduce that with c1=12mc2=18c_1= 12mc^2= 18, for any ξ,δ∈(0,1)ξ,δ∈(0,1), we have with probability 1−ξ1-ξ, ∀∈S,|2m‖‖1−1|≤δ∀ z∈ S_X, | 2 m\| A z\|_1-1 |≤δ (19) as long as m≥C1c1δ2max(klog(1ϵ∗),log(6ξ)).m≥ C_1c_1δ^2 (k ( 1ε^* ), ( 6ξ ) ). For all ∈ x , we obtain by replacing z by ‖ x\| x\| in (19), ‖‖1≤(δ+1)m2‖2\| A x\|_1≤(δ+1) m2\| x\|_2 (20) provided that m≥8C1δ2max(klog(1ϵ∗),log(6ξ)).m≥ 8C_1δ^2 (k ( 1ε^* ), ( 6ξ ) ). By setting C=18C1C= 18C_1, δ=1δ=1 and ξ=6exp(−Cm)ξ=6 (-Cm ), we can verify that the last inequality holds for all m≥klog(1ϵ∗)m≥ k ( 1ε^* ) and thus: ℙ(∀∈:∥1≤m∥2)>1−6e−Cm.P (∀ x :\| A x\|_1≤ m\| x\|_2 )>1-6e^-Cm. ∎ Theorem 3. Adapted from Robinson (2010, Thm. 4.3) Let a compact set X with finite upper box-counting dimension dim()<k (X)<k and ∈ℝm×n A ^m× n such that Aij∼(0,1m)A_ij (0, 1m ). If m>2km>2k then ℙ(= for some ≠∈)=0P ( A x= A u for some x≠ u )=0 Note that Robinson’s theorem states that almost every linear map :ℝN→ℝk L ^N ^k (for m>2km>2k) embeds X injectively with Hölder distortion ‖−‖≤C‖−‖ρ\| x- y\|≤ C\| L x- L y\|^ρ, for some ρ∈(0,1)ρ∈(0,1). By considering the definition of “almost every” in the sense of measure, we can say that the set of non-injective linear maps has zero measure, and therefore zero probability. For the rest of the proof, we will consider R=∩∥.∥2(0,mμα)X_R=X _\|.\|_2 (0, mμα ) with R=mμαR= mμα and α>0α>0 to be define latter. We define ℐsat()=j:|yj|≥μ _ sat( y)= \j:|y_j|≥μ \ ℐsat¯()=j:|yj|<μ I_ sat( y)= \j:|y_j|<μ \ as the sets of the index of respectively unsaturated and saturated entries of y. We note ℐ A^I the matrix represents the row-submatrix of A where only the rows indexed by ℐI are selected. Let ℐ,=ℐsat¯()∩ℐsat¯()I A_ x, u= I_ sat( A x)∩ I_ sat( A u) be the intersection of the unsaturated index sets of A x and A u. Using the law of total probability we can decompose the probability of Theorem˜1 into the two target probabilities. By considering: =∃,∈R,≠:ℐ,=ℐ, = \∃ x, u _R, x≠ u: A^I A_ x, u x= A^I A_ x, u u \ ℬ=∀,∈R,≠:|ℐ,|>(1−2α)m = \∀ x, u _R, x≠ u: |I A_ x, u |>(1-2α)m \ where A is the set of events where there exists two signals having the same common unsaturated measurements, ℬB is the set of events where the number of common unsaturated measurements is greater than (1−2α)m(1-2α)m for all couple of signals. And taking it into the law of total probability: ℙ() (A ) =ℙ(∩ℬ)+ℙ(∩ℬ¯) =P (A )+P (A∩ B ) =ℙ(∣ℬ)×ℙ(ℬ)+ℙ(∣ℬ¯)×ℙ(ℬ¯) =P (A )×P (B )+P (A B )×P ( B ) ≤ℙ(∣ℬ)+ℙ(ℬ¯) (A )+P ( B ) (21) as we have that ℙ(ℬ)<1P (B )<1 and ℙ(|ℬ¯)<1P (A| B )<1. ℙ(∣ℬ)P (A ) and ℙ(ℬ¯)P ( B ) are respectively the probability that two signals have the same common unsaturated measurements given that the number of saturated measurements is low, and the probability there exist two distinct signals whose the number of common unsaturated measurements is low. Next, we upper-bound separately the two terms ℙ(∣ℬ)P (A ) and ℙ(ℬ¯)P ( B ). We start with the first one: ℙ(∣ℬ) (A ) =ℙ(∃,∈R,≠:ℐ,=ℐ,|∀,∈R,≠:|ℐ,|>(1−2α)m) =P ( \∃ x, u _R, x≠ u: A^I A_ x, u x= A^I A_ x, u u \ | \∀ x, u _R, x≠ u:|I A_ x, u|>(1-2α)m \ ) ≤ℙ(∃ℐ⊂⟦0:m⟧,|ℐ|>(1−2α)m,∃,∈R,≠:ℐ=ℐ). (∃~I⊂ 0:m ,\,|I|>(1-2α)m,\ ∃ x, u _R, x≠ u: A^I x= A^I u ). And because of X is a cone, we have (Robinson, 2010): dim(R)≤dim(∩)+1=dim(S)+1<k+1 (X_R)≤ (X )+1= (S_X)+1<k+1 We thus have, as long as (1−2α)m>2(k+1)(1-2α)m>2(k+1), and thanks to Theorem˜3: ℙ(∃ℐ⊂⟦0:m⟧,|ℐ|>(1−2α)m,∃,∈R,≠:ℐ=ℐ)=0P (∃~I⊂ 0:m ,|I|>(1-2α)m,\,∃ x, u _R, x≠ u: A^I x= A^I u )=0 (22) And so the first right-hand side term of (21) is equals 0. From (1−2α)m>2(k+1)(1-2α)m>2(k+1) which imposes α<12−k+1mα< 12- k+1m, we can fix α and therefore fix R<mμ(12−k+1m)R< mμ ( 12- k+1m ). We also note as α>0α>0, we necessarily have 2(k+1)<m2(k+1)<m. We can now upper-bound the second term appearing in (21): ℙ(∃,∈R,≠:|ℐ,|≤(1−2α)m). (∃ x, u _R, x≠ u: |I A_ x, u |≤(1-2α)m ). By observing that for two set I and J of 1,…,m \1,…,m \ with |I∩J|≤(1−2α)m|I∩ J|≤(1-2α)m, this means that |I¯∪J¯|>2αm | I∪ J |>2α m, which implies that either |I¯|>αm or |J¯|>αm | I |>α m or | J |>α m (since otherwise the union has size <2αm<2α m), therefore |I|≤(1−α)m or |J|≤(1−α)m|I|≤(1-α)m or |J|≤(1-α)m, we can deduce: |ℐ,|≤(1−2α)m |I A_ x, u |≤(1-2α)m ⇔ |ℐsat¯()∩ℐsat¯()|≤(1−2α)m | I_ sat( A x)∩ I_ sat( A u) |≤(1-2α)m ⇒ or|ℐsat¯()|≤⌊m+(1−2α)m2⌋≤(1−α)m|ℐsat¯()|≤⌊m+(1−2α)m2⌋≤(1−α)m \ or array[]l | I_ sat( A x) |≤ m+(1-2α)m2 ≤(1-α)m\\ \\ | I_ sat( A u) |≤ m+(1-2α)m2 ≤(1-α)m array . we have ℙ(∃,∈R,≠:|ℐ,|≤(1−2α)m) (∃ x, u _R, x≠ u:|I A_ x, u|≤(1-2α)m ) ≤ℙ(∃∈R:|ℐsat¯()|≤(1−α)m∪∃∈R:|ℐsat¯()|≤(1−α)m) ( \∃ x _R: | I_ sat( A x) |≤(1-α)m \∪ \∃ u _R: | I_ sat( A u) |≤(1-α)m \ ) ≤2ℙ(∃∈R:|ℐsat¯()|≤(1−α)m) ≤ 2P (∃ x _R: | I_ sat( A x) |≤(1-α)m ) and if we remark that ‖‖1=∑i=1m|i⊤|≥∑i∈ℐsat()|i⊤|≥∑i∈ℐsat()μ\| A x\|_1=Σ ^m_i=1| a _i x|≥Σ _i _ sat( A x)| a _i x|≥Σ _i _ sat( A x)μ we obtain ‖‖1≥|ℐsat()|μ.\| A x\|_1≥ |I_ sat( A x) |μ. Thanks to Corollary˜2.1 we know that with probability at least 1−6e−Cm1-6e^-Cm, ‖‖1≤m‖2≤mαμ.\| A x\|_1≤ m\| x\|_2≤ mαμ. We deduce from these two last inequalities |ℐsat()|μ≤mαμ |I_ sat( A x) |μ≤ mαμ m−|ℐsat¯()|≤mα m- | I_ sat( A x) |≤ mα (1−α)m≤|ℐsat¯()|. (1-α)m≤ | I_ sat( A x) |. And thus ℙ(∃∈R:|ℐsat¯()|≤(1−α)m)≤6e−CmP (∃ x _R: | I_ sat( A x) |≤(1-α)m )≤ 6e^-Cm as long as m≥klog(1ϵ∗).m≥ k ( 1ε^* ). Finally, ℙ(ℬ¯)≤12e−Cm.P ( B )≤ 12e^-Cm. (23) we conclude by adding the upper-bound obtain in (22) and (23): ℙ(∃,∈R,≠:η()=η()) (∃ x, u _R, x≠ u:η( A x)=η ( A u ) ) ≤ℙ(∃,∈R,≠:ℐ,=ℐ,) (∃ x, u _R, x≠ u: A^I A_ x, u x= A^I A_ x, u u ) ≤ℙ(∣ℬ)+ℙ(ℬ¯) (A )+P ( B ) ≤0+12e−Cm. ≤ 0+12e^-Cm. Finally, (5) follows from a simple rescaling of the radius R and the vectors x and u by a factor m m, and an opposite rescaling of the entries AijA_ij of A by 1/m1/ m so that Aij∼(0,1)A_ij (0,1). Appendix B Appendix: Hyperparameters, training details and images reconstruction Table 5: Hyperparameters for different experiments. Section λ pgp_g Architecture Learning rate Epochs Batch size Section˜5.2.1 11 (0.1,2)U(0.1,2) Bias-free U-Net, 4 down + 4 up blocks 5e−45e^-4 300300 5050 Section˜5.2.2 11 (0.5,1.5)U(0.5,1.5) MLP, input size 100, hidden size 100, depth 5 1e−41e^-4 300300 100100 Section˜5.3 0.10.1 (0.1,2.0)U(0.1,2.0) Bias-free U-Net, 5 down + 5 up blocks 5e−45e^-4 100100 3535 Section˜5.4 0.10.1 (0.2,1.5)U(0.2,1.5) Bias-free U-Net, 4 down + 4 up blocks 5e−55e^-5 400400 1212 Figure 9: SDR performance on the evaluation dataset during training for supervised and self-supervised methods. Figure 10: Image reconstruction on the test dataset. Artifacts can appear in saturation zone.