Paper deep dive
Neural Dynamic GI: Random-Access Neural Compression for Temporal Lightmaps in Dynamic Lighting Environments
Jianhui Wu, Jian Zhou, Zhi Zhou, Zhangjin Huang, Chao Li
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 4/15/2026, 1:41:20 AM
Summary
Neural Dynamic GI (NDGI) is a novel neural compression framework for temporal lightmap sets in dynamic lighting environments. It utilizes a hybrid feature map structure (2D and 3D) combined with a lightweight MLP decoder to represent temporal illumination compactly. The method incorporates a block compression (BC) simulation strategy during training to ensure compatibility with standard GPU hardware formats, and integrates with a virtual texturing (VT) system to enable efficient, on-demand real-time decompression.
Entities (5)
Relation Signals (3)
Neural Dynamic GI → compresses → Temporal Lightmap Sets
confidence 98% · NDGI, a novel compression technique specifically designed for temporal lightmap sets.
Neural Dynamic GI → integrates → Virtual Texturing
confidence 95% · we also integrate a virtual texturing (VT) system with our neural representation.
Neural Dynamic GI → uses → Block Compression
confidence 95% · we introduce a block compression (BC) simulation strategy during the training process
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:High-quality global illumination (GI) in real-time rendering is commonly achieved using precomputed lighting techniques, with lightmap as the standard choice. To support GI for static objects in dynamic lighting environments, multiple lightmaps at different lighting conditions need to be precomputed, which incurs substantial storage and memory overhead. To overcome this limitation, we propose Neural Dynamic GI (NDGI), a novel compression technique specifically designed for temporal lightmap sets. Our method utilizes multi-dimensional feature maps and lightweight neural networks to integrate the temporal information instead of storing multiple sets explicitly, which significantly reduces the storage size of lightmaps. Additionally, we introduce a block compression (BC) simulation strategy during the training process, which enables BC compression on the final generated feature maps and further improves the compression ratio. To enable efficient real-time decompression, we also integrate a virtual texturing (VT) system with our neural representation. Compared with prior methods, our approach achieves high-quality dynamic GI while maintaining remarkably low storage and memory requirements, with only modest real-time decompression overhead. To facilitate further research in this direction, we will release our temporal lightmap dataset precomputed in multiple scenes featuring diverse temporal variations.
Tags
Links
- Source: https://arxiv.org/abs/2604.12625v1
- Canonical: https://arxiv.org/abs/2604.12625v1
Trouble viewing inline? Open PDF directly →
Full Text
52,650 characters extracted from source content.
Expand or collapse full text
Neural Dynamic GI: Random-Access Neural Compression for Temporal Lightmaps in Dynamic Lighting Environments Jianhui Wu1 Jian Zhou1 Zhi Zhou1 Zhangjin Huang1 Chao Li211footnotemark: 1 1University of Science and Technology of China 2Zhejiang University https://magicdawnlab.github.io/ Corresponding authors. Abstract High-quality global illumination (GI) in real-time rendering is commonly achieved using precomputed lighting techniques, with lightmap as the standard choice. To support GI for static objects in dynamic lighting environments, multiple lightmaps at different lighting conditions need to be precomputed, which incurs substantial storage and memory overhead. To overcome this limitation, we propose Neural Dynamic GI (NDGI), a novel compression technique specifically designed for temporal lightmap sets. Our method utilizes multi-dimensional feature maps and lightweight neural networks to integrate the temporal information instead of storing multiple sets explicitly, which significantly reduces the storage size of lightmaps. Additionally, we introduce a block compression (BC) simulation strategy during the training process, which enables BC compression on the final generated feature maps and further improves the compression ratio. To enable efficient real-time decompression, we also integrate a virtual texturing (VT) system with our neural representation. Compared with prior methods, our approach achieves high-quality dynamic GI while maintaining remarkably low storage and memory requirements, with only modest real-time decompression overhead. To facilitate further research in this direction, we will release our temporal lightmap dataset precomputed in multiple scenes featuring diverse temporal variations. Figure 1: We evaluate performance in the “FarmLand” scene, which is derived from a real video game. The top row shows GI effects at different times. The bottom row presents lighting reconstructed by PRT [23], NTC [27], and our method. To enable a clearer comparison, textures are removed to emphasize illumination effects. The top-left part of the image shows the error relative to the reference. We report bits per pixel (BPP) and compression ratio as evaluation metrics. Compared with PRT and NTC, our method achieves a higher compression ratio and superior lighting quality, while incurring lower real-time computational overhead and less noise than NTC. 1 Introduction Real-time rendering quality has advanced rapidly in recent years, driven by progress in global illumination (GI) algorithms including voxel-based approaches [28], real-time ray tracing [26, 18], and denoising methods [32]. Among various real-time GI solutions, precomputed lighting [21] remains widely adopted due to its stable performance and scalability across hardware. In this paradigm, indirect illumination, static shadows, and selected direct lighting components are precomputed for static geometry and stored in structured forms such as lightmaps or spherical harmonic (SH) probes. At runtime, these precomputed results enable efficient reproduction of complex lighting with minimal computational cost. In particular, lightmap-based techniques [22, 6] offer higher spatial fidelity than probe-based methods [7], capturing detailed surface illumination at fine resolutions. However, extending lightmap-based techniques to dynamic lighting environments remains challenging. Realistic time-of-day (TOD) effects or dynamic light changes require precomputing multiple sets of lightmaps corresponding to different lighting conditions (Figure 2), followed by real-time interpolation. This design leads to prohibitively large storage and memory consumption, making it unsuitable for large-scale scenes. As scene complexity and texture resolution continue to grow, efficient compression becomes crucial to enabling dynamic GI at scale. Conventional GPU texture compression methods [17, 15] are hardware efficient and support random access. However, maintaining quality often requires relatively high bitrates and leads to block artifacts, and processing textures independently fails to exploit temporal redundancy across multiple lightmap sets. Recent advances in neural compression [27] improve compression ratios and reconstruction quality, but most approaches rely on large decoders that hinder real-time performance. To our knowledge, no prior work addresses compression specifically for multiple lightmap sets, nor applies such compression to dynamic GI. To address these challenges, we propose Neural Dynamic GI (NDGI)—a neural compression framework specifically designed for temporal lightmap sets. Instead of storing multiple lightmap sets explicitly, NDGI encodes temporal illumination into hybrid feature maps and reconstructs lighting values on-demand through a lightweight neural decoder. During training, we introduce a block compression (BC) simulation strategy to ensure that the learned feature maps are compatible with standard GPU BC formats, enabling further compression without visible quality degradation. To make the system practical for real-time rendering, we adopt a tile-based training paradigm that integrates NDGI with a virtual texturing (VT) [8] system. The VT framework manages texture streaming and caching at the tile level, enabling on-demand loading and high-speed decompression during rendering. This design reduces both storage and memory simultaneously while maintaining high-quality dynamic GI with negligible runtime overhead. Moreover, to support future research in this direction, we construct and release a temporal lightmap dataset precomputed across multiple scenes. These scenes include various temporal variations, such as gradual changes in sunlight and skylight, as well as the switching of local light sources. The dataset provides a practical benchmark for studying dynamic illumination compression and serves as a valuable resource for developing future neural rendering systems. In summary, our main contributions are: • We propose Neural Dynamic GI (NDGI), the first neural compression framework tailored for temporal lightmap sets, achieving high-quality dynamic GI with extremely low storage and memory costs. • We introduce a BC simulation strategy that ensures compatibility with standard BC formats, enabling further compression without sacrificing quality. • We design a virtual texturing-integrated neural decoding system that supports on-demand decompression for real-time rendering. • We provide a temporal lightmap dataset comprising multiple scenes with diverse temporal variations to support future research in dynamic GI compression. morning noon dusk night Figure 2: Examples of lightmaps under different lighting conditions. During night scenes, certain local light sources are enabled, resulting in brighter illumination recorded in the lightmap. 2 Previous Work 2.1 Precomputed Lighting Precomputed lighting computes illumination in an offline stage and stores it in compact representations for efficient reuse. During rendering, lighting reconstruction requires only sampling and lightweight computations, which makes this approach widely adopted in real-time applications. Lightmaps are one of the most common carriers for precomputed lighting. Research on lightmaps primarily focuses on packing, and compression [13] in order to improve storage efficiency and reduce runtime memory usage. The Directional Lightmap technique [22] extends the conventional formulation by storing the dominant incident lighting direction alongside irradiance values, which introduces view-dependent effects and increases storage cost. Precomputed Radiance Transfer (PRT) methods [23] enable dynamic lighting by storing transfer data that describe how outgoing light responds to incident light. Such transfer data can be encoded into lightmaps or represented within probes depending on the desired trade-off between accuracy and storage efficiency. 2.2 Traditional Texture Compression Traditional texture compression primarily relies on block-based encoding. The idea dates back to Block Truncation Coding (BTC) [4] for grayscale images, followed by extensions that support color images and efficient hardware decoding. For example, Campbell et al. [3] proposed Color Cell Compression (C), and Knittel et al. [12] enhanced it with GPU‑accelerated decompression. DirectX Texture Compression (DXTC) [15] spans BC1 through BC7 and stores quantized color endpoints with per block interpolation weights, providing a practical balance between quality, bitrate, and decoding efficiency, with variants such as BC5 for normal maps and BC6H for high dynamic range (HDR) content. Adaptive Scalable Texture Compression (ASTC) [17] offers flexible bitrate configurations and broad texture support including 3D and HDR textures and is widely adopted on modern mobile platforms. The Ericsson Texture Compression family provides another mobile‑oriented branch, where ETC1 [25] and ETC2 [24] achieve efficient compression with backward compatibility and low decoding cost. Despite their widespread use, these block‑based approaches yield modest compression ratios and lack mechanisms for multi‑lightmap or temporally varying data required by dynamic global illumination. Our method addresses these limitations by enabling high‑quality, temporally coherent lightmap compression with significantly reduced storage requirements. 2.3 Neural Texture Compression Neural texture compression methods have emerged as promising alternatives to traditional compression formats. These approaches provide a flexible trade-off among reconstruction quality, storage cost, and computational overhead. Instant NGP [16] introduced hash encoding that uses multi-resolution grids to index hash tables for feature retrieval and has been adapted for large-scale image compression to achieve very low bitrates. Another line of work [27] employs feature pyramid to encode multi-channel textures across multiple mipmap levels. This approach supports random access at arbitrary positions and resolutions and also enables real-time decompression. Furthermore, Belcour [14] and Weinreich [31] integrate traditional block compression into feature maps. By directly optimizing endpoints and weights, they achieve additional bitrate reduction with minimal quality loss. However, neural network based approaches often incur substantial decoding costs that hinder runtime performance, while our method attains high‑fidelity compression under minimal runtime overhead. Figure 3: Method overview. Our method samples feature maps of different structures and feeds the features together with encoded time into an MLP to reconstruct the corresponding lighting (Section 3.3). During training, we introduce random noise to simulate the quantization and use weights and endpoints to emulate BC compression (Section 3.4). This approach allows us to further compress the feature map using quantization and BC7 [15]. The compressed feature maps and the MLP serve as the final parameters of our model. At runtime, they are streamed and decompressed on demand via a virtual texturing system to deliver high-quality dynamic global illumination (Section 3.5). 3 Method 3.1 Problem Statement Our compression target consists of multiple sets of lightmaps, which can be represented as: L=Li|i=0,1,2…,n−1,L=\L_i|i=0,1,2...,n-1\, (1) where n refers to the number of lightmap sets, and each LiL_i is a multi‑channel lightmap of size (H,W,C)(H,W,C), representing lighting information at time tit_i. For simplicity, we will assume there is only one lightmap in each set. Specifically, depending on the requirements of the scene, the distribution of time points tit_i may be non-uniform. For example, when a scene requires a rapid change in lighting at a certain moment (a pre-set switching event of a light source), multiple sets of lightmap may be computed around that time. In traditional methods, dynamic GI is achieved via runtime interpolation. The lighting I at a specific texture coordinates (u,v)(u,v) and time t∈(ti−1,ti)t∈(t_i-1,t_i) can be obtained as: I(u,v,t)=ℐ[ti−1,ti][Li−1(u,v),Li(u,v)](t).I(u,v,t)=I_[t_i-1,t_i][\,L_i-1(u,v),L_i(u,v)\,](t). (2) Here, ℐI denotes a general interpolation operator that estimates I(u,v,t)I(u,v,t) based on Li−1(u,v)L_i-1(u,v) and Li(u,v)L_i(u,v). Depending on the specific configuration, ℐI can represent different interpolation schemes (e.g. linear, cubic, or nearest sampling). To implement the above interpolation algorithm, existing methods require substantial disk space and runtime memory to store the lightmap sets L, which is prohibitively expensive for performance-critical video games. To address this issue, our approach avoids directly storing L by representing it as compact model H with parameter Θ , such that the lighting can be directly represented as: I(u,v,t)=Θ(u,v,t),I(u,v,t)=H_ (u,v,t), (3) Furthermore, we employ several optimization techniques to achieve a balance among storage space, runtime memory, and real-time decompression performance, thereby enabling practical dynamic GI in real-time rendering. The details of our method will be introduced in subsequent sections. 3.2 Method Overview Figure 3 illustrates the overall framework of our approach. Given a set of precomputed lightmaps under varying lighting conditions, our method compresses them into a compact set of feature maps and a lightweight MLP for efficient decoding. At runtime, given specific texture coordinates and time (u,v,t)(u,v,t), the feature maps are sampled to obtain a latent vector, which is concatenated with a position-encoded time embedding and fed into the MLP to reconstruct lighting. The reconstructed results are compared with the original lightmaps during training to jointly optimize both the feature maps and the network. To further enhance storage efficiency, we incorporate a block compression (BC) simulation during training, enabling compatibility with hardware-supported BC formats. Noise is added to the latent vector to mitigate the impact of subsequent feature map quantization. In addition, a virtual texturing (VT) [8] system is integrated at runtime to progressively stream lightmap regions on demand, significantly reducing decoding overhead and memory usage. Table 1: Compressed representation of feature maps for a specific lightmap set comprising 26 lightmaps at 128×128128× 128 resolution. Feature Map Resolution Channels uv2DF^2D_uv 128×128128× 128 4 ut2DF^2D_ut 64×2464× 24 2 vt2DF^2D_vt 64×2464× 24 2 uvt3DF^3D_uvt 32×32×1232× 32× 12 4 3.3 Hybrid Feature Map Latent feature maps are commonly used in neural compression to store implicit information. To preserve the common spatial signal across the target lightmap set, we employ a 2D feature map uv2DF_uv^2D that has the same size as the target lightmap. Moreover, although lightmaps at different time points exhibit correlation, the variation is high frequency and more complex than those in material textures. As a result, a single 2D feature map is insufficient to represent temporal variation and leaves preserving temporal information completely to the lightweight MLP decoder. To address this challenge, we propose a hybrid feature map structure. We introduce an additional 3D feature map uvt3DF^3D_uvt at a lower resolution to capture the luminance variations of the lightmap over time. Moreover, we extend the 2D feature map into triplane feature maps, denoted as uv2DF^2D_uv, ut2DF^2D_ut and vt2DF^2D_vt, to better capture fine details. During inference, we first sample uvt3DF^3D_uvt using (u,v,t)(u,v,t) coordinates to get vector VuvtV_uvt. Then we project the coordinates onto three distinct planes, sample accordingly to get three feature vectors Vuv,Vut,VvtV_uv,V_ut,V_vt. Finally, we concatenate the four vectors with positional encoded time γ(t)γ(t) and input them into the decoder network GΦG_ , with parameter Φ , to obtain the decompressed result I(u,v,t)I(u,v,t): γ(t) γ(t) =[sin(20πt),cos(20πt),sin(21πt),cos(21πt)], =[\, (2^0π t),\ (2^0π t),\ (2^1π t),\ (2^1π t)\,], (4) I(u, I(u, v,t)=GΦ(Vuvt,Vuv,Vut,Vvt,γ(t)). v,t)=G_ (V_uvt,V_uv,V_ut,V_vt,γ(t)). Decoder network GΦG_ is a lightweight MLP, whose size can be adjusted to meet practical requirements and accommodate different hardware capabilities. In summary, the original lightmap set L is compressed into the following parameter set Θ : L→Θ=uvt3D,uv2D,ut2D,vt2D,Φ.L→ =\F^3D_uvt,F^2D_uv,F^2D_ut,F^2D_vt, \. (5) This design enables the model to capture high-frequency information simultaneously in both the spatial and temporal domains. Table 1 illustrates the feature maps resolutions for a specific lightmap set comprising 26 lightmaps at 128×128128× 128 resolution, baked under varying lighting conditions. The resolution of these feature maps is adjustable and only needs to be compatible with the BC representation that will be introduced later (Section 3.4). Furthermore, our method supports random access, enabling on demand decoding of individual regions without processing the rest. Figure 4: Each 4×44× 4 block is directly generated by per pixel interpolation between a pair of endpoints using a set of weights. 3.4 Feature Map Compression As shown in Figure 3, we adopt two strategies to further compress the four feature maps. For ut2DF^2D_ut and vt2DF^2D_vt, we apply 8-bit post-training quantization and emulate quantization during training by injecting uniform noise [2, 27]. Concretely, sampling at (u,v,t)(u,v,t) yields latent vectors VutV_ut and VvtV_vt, to which we add noise as follows: Vut′ V _ut =Vut+(−0.5,0.5)⋅αut, =V_ut+U(-5,5)· _ut, (6) Vvt′ V _vt =Vvt+(−0.5,0.5)⋅αvt. =V_vt+U(-5,5)· _vt. Here, αut _ut and αvt _vt are set to be 1256 1256 in all our experiments, which corresponds to the 8-bit representation. For uvt3DF^3D_uvt and uv2DF^2D_uv, unlike ut2DF^2D_ut and vt2DF^2D_vt, these two feature maps are much larger in size. Therefore, after training, we not only quantize them to 8-bit, but also apply the BC7 compression algorithm, which encodes each 4×44× 4 texel block using multiple modes to the quantized results. For the 3D texture uvt3DF^3D_uvt, we slice it along the temporal dimension and compress each 2D slice independently. Table 2: The impact of quantization and BC compression on the BPP and compression ratio of our method. Compression Strategy BPP Compression Ratio Original 4.62 4.8% 8-bit Quant. 2.32 2.4% 8-bit Quant. & BC 0.68 0.7% However, when applying the BC algorithm to trained feature maps, significant information loss may occur. Inspired by previous work [4, 14], we address this issue by adopting a BC-simulation strategy during the training of uvt3DF^3D_uvt and uv2DF^2D_uv, as illustrated in Figure 4. Instead of directly updating their parameters, we divide each feature map into 4×44× 4 blocks, where the feature values within each block are represented by a set of endpoints E and weights W, as formulated below: =e1,e2∈[0,1]k,=w1,…,w16,wp∈[0,1]. \ array[]lE=\e_1,e_2∈[0,1]^k\,\\ W=\w_1,…,w_16\, w_p∈[0,1]. array . (7) Each pixel p in the 4×44× 4 block is then reconstructed by linear interpolation between the two endpoints: fp=(1−wp)e1+wpe2,p=1,…,16,f_p\;=\;(1-w_p)\,e_1\;+\;w_p\,e_2, p=1,…,16, (8) where fp∈[0,1]kf_p∈[0,1]^k is the k channels feature of pixel p. During training, we update the endpoints and weights of each block, use them to reconstruct uvt3DF^3D_uvt and uv2DF^2D_uv, and perform sampling on the reconstructed feature maps. Noise is then added to the sampled vector VuvtV_uvt, VuvV_uv to obtain the feature vectors Vuvt′V _uvt, Vuv′V _uv, which are concatenated with Vut′V _ut, Vvt′V _vt, and γ(t)γ(t) before being fed into the network. After training converges, we use the learned endpoints and weights to reconstruct uvt3DF^3D_uvt and uv2DF^2D_uv once again, followed by quantization and BC7 compression to produce the final compressed results. Table 2 illustrates the impact of our quantization compression on the bitrates. 3.5 Runtime Decompression Since lighting varies with scene time, decompressing all lightmaps at load time is impractical. Decoding must be performed online. However, per frame per pixel decoding imposes a heavy computational burden, and adjacent frames typically change slowly, leading to redundant inference. To address this problem, we propose a method that leverages the caching mechanism of virtual texturing (VT) [8]. In the VT system, each lightmap is partitioned into fixed-size tiles, and at render time only the required tiles are streamed and written into a physical texture. To align training with runtime, we tile the lightmaps and train a separate model per tile. As illustrated in Figure 3, on each frame we fetch the parameters of the required tiles on demand, decompress them with a compute shader, and write the results to the physical texture. During shading, we first sample the page table to locate each tile within the physical texture, then sample the physical texture to obtain the final lighting. For tiles already resident in the physical texture, because lighting changes are negligible over short timescales, we can reuse the cached content for a period without redundant inference. This design substantially reduces online decoding cost, avoids redundant computation, and improves real-time performance. 4 Implementation Our neural compression model is trained separately for each lightmap set using the PyTorch framework [19]. Before training, gamma correction is applied to enhance details in dark regions, and per channel mean normalization is performed at each time step. These mean values are later used during rendering to restore the original lightmap data. The trained model outputs are floating-point RGB values, which are quantized into an 8-bit 4-channel format to reduce GPU memory usage. In large-scale scenes, thousands of independent NDGI models may need to be trained. To improve efficiency, we employ PyTorch’s batched matrix operations [20] for parallel training. The tile-based structure of our approach also enables straightforward scaling to distributed training. The MLP has We use the Adam optimizer [11] with an initial learning rate of 10−310^-3, a batch size of 2122^12, and use L2 loss for the loss function. The MLP network consists of two hidden layers with variable widths. GELU activation [9] is applied to hidden layers, with no activation at the output. In the final training stage, we freeze the feature maps and fine-tune the MLP under simulated quantization and BC compression. For real-time rendering, we utilize the virtual texturing system and compute shader in Unreal Engine [5] to enable efficient lightmap reconstruction with minimal runtime overhead. Table 3: NDGI profiles with different BPP and decoder size. Profile uvt3DF^3D_uvt Resolution Hidden Size BPP NDGI L. 16×16×12×416× 16× 12× 4 16 0.50 NDGI M. 32×32×12×432× 32× 12× 4 16 0.68 NDGI H. 64×64×12×464× 64× 12× 4 16 1.39 NDGI M.64 32×32×12×432× 32× 12× 4 64 0.86 5 Experiment (a) FarmLand (b) Room (c) City (d) Yard Figure 5: Representative examples from our datasets. (a) FarmLand, sourced from “Arena Breakout: Infinite”. (b) An indoor environment with time-varying skylight and sunlight. (c) A city scene where streetlights toggle on and off at specific times. (d) An yard with multiple local light sources. Table 4: Average PSNR, 1-SSIM, and LPIPS values over the evaluation data set for the methods used in our comparison. “L.”, “M.” and “H.” is short for “Low”, “Medium” and “High”. Our method achieves better compression quality while maintaining low bitrates. Low Bitrates ( < 1.0 BPP) High Bitrates ( > 1.0 BPP) NDGI L. NDGI M. NTC L. NDGI M.64 ASTC 12×1212× 12 NTC M. ASTC 10×1010× 10 NDGI H. NTC H. BC6H BC7 BPP (Mean) 0.50 0.68 0.78 0.86 0.89 1.10 1.28 1.39 1.78 8 8 PSNR (↑ ) 45.96 46.69 43.61 47.42 32.50 44.26 34.52 48.68 45.77 44.31 42.27 1 - SSIM (↓ ) 0.014 0.012 0.026 0.012 0.069 0.022 0.051 0.009 0.016 0.026 0.039 LPIPS (↓ ) 0.007 0.006 0.010 0.007 0.057 0.008 0.043 0.004 0.007 0.010 0.016 NDGI M. (Ours) NTC L. ASTC 12×1212× 12 BC7 Reference 1.10 MB 1.26 MB 1.44 MB 13 MB 156 MB PSNR (↑ ) 49.31 dB 45.14 dB 33.78 dB 44.29 dB PSNR (↑ ) 45.73 dB 41.69 dB 31.13 dB 42.49 dB Figure 6: Quality and storage comparison showing PSNR scores for lightmap tiles in the “FarmLand” scene. Compared with traditional texture compression methods, our approach achieves higher compression ratios and better reconstruction quality. Compared to NTC, NDGI yields obviously less noise with a much smaller decoder, leading to superior decompression performance. Lit Scene PRT ASTC 12×1212× 12 NTC L. NDGI M. (Ours) Reference BPP(↓ ), PSNR(↑ ), SSIM(↑ ) 0.92, 35.59 dB, 0.993 0.89, 42.09 dB, 0.998 0.78, 43.48 dB, 0.998 0.67, 45.75 dB, 0.999 FarmLand BPP(↓ ), PSNR(↑ ), SSIM(↑ ) 1.00, 20.85 dB, 0.935 0.89, 31.43 dB, 0.967 0.78, 42.28 dB, 0.997 0.71, 43.31 dB, 0.997 Yard Figure 7: Comparison of rendered results. The “Lit Scene” displays the final rendered image. Notably, PRT exhibits color tone deviation, ASTC displays obvious blocky artifacts, and NTC produces noticeable noise. Compared with previous methods, NDGI delivers higher-quality GI effects. Our lightmap compression framework supports flexible configurations, allowing users to balance compression quality, storage overhead, and inference performance. We define four compression configurations, as detailed in Table 3, each corresponding to varying network costs and bits per pixel (BPP). These configurations are used in our experiments to provide comprehensive comparisons across different trade-offs between quality and performance. More detailed experiment results can be found in supplementary materials. 5.1 Evaluation Dataset We evaluate our method on lightmap datasets precomputed from multiple real game scenes, covering diverse indoor and outdoor environments. The lighting conditions vary over scene time, including changes in skylight, directional light, and local light sources. In particular, switching of local light sources introduces high-frequency variations on the lightmap. Representative scene examples are shown in Figure 5. 5.2 Compared Methods We compare our method against both traditional GPU texture compression and neural compression approaches to evaluate compression performance and rendering quality. For traditional methods, GPU texture compression formats such as BC6H, BC7, and ASTC [17, 15] are commonly adopted in production, typically chosen according to texture type rather than a unified standard. We evaluate two BC-based configurations: directly applying BC6H to HDR lightmaps, and converting HDR lightmaps to 4-channel 8-bit before applying BC7 compression. In addition, we test ASTC under multiple bit-per-pixel settings for comparison. For neural methods, we include NTC [27], a representative neural texture compression approach. However, NTC supports at most 16 channels per texture set, which limits its ability to handle our temporal lightmap data. To ensure a fair comparison, we partition our dataset into consecutive temporal segments, each compressed independently within this constraint. We also compare with Precomputed Radiance Transfer (PRT) [23], which supports dynamic lighting through precomputed probe data. We exclude general image compression algorithms such as JPEG [29, 1], as they do not support random access and are unsuitable for real-time rendering. 5.3 Lightmap Quality Evaluation Figure 8: Lightmap quality across different BPP. NDGI consistently outperforms other methods across all compression ratios. Table 4 summarizes the compression and reconstruction results in our test scene. Overall, NDGI achieves significantly higher lightmap reconstruction quality compared with other methods under similar bitrates. Although our training objective does not directly optimize for perceptual fidelity, our method still consistently outperforms conventional GPU and neural compression techniques under these perceptual metrics. Additionally, as illustrated in Figure 8, NDGI maintains high-fidelity reconstructions across a wide range of compression rates. Notably, even with an extremely lightweight decoder containing only a few hundred parameters, our method preserves fine illumination details and avoids the artifacts commonly observed in other approaches. We attribute this robustness to the efficient representation of spatial-temporal lightmap features learned by our network. 5.4 Rendered Quality Evaluation Table 5: Decompression time for a 1024×10241024× 1024 lightmap. Performance is similar across all lightmaps for a given profile. NDGI L. NDGI M. NDGI M.64 NTC L. NTC M. 0.201ms 0.203ms 0.314ms 0.886ms 0.918ms For rendered quality evaluation, we take the original lightmap rendering results as the ground truth and compare our method against representative alternatives, including NTC [27] and PRT [23]. To ensure a fair and accurate comparison, we directly measure the differences between the reconstructed lighting and the ground-truth lighting, rather than comparing the final rendered images. This design eliminates potential variations introduced by the rendering pipeline and focuses solely on lighting reconstruction quality. As illustrated in Figure 1 and Figure 11, NDGI achieves higher-fidelity lighting reconstruction across multiple scenes. Furthermore, our method exhibits stronger temporal consistency and more accurate reproduction of lighting transitions over time, substantially outperforming PRT in dynamic illumination scenarios. 5.5 Decompression Performance We evaluate real-time performance by decoding a 1024×10241024× 1024 lightmap using a CUDA implementation accelerated by Tensor Cores. Table 5 reports the decoding latency measured on an NVIDIA RTX 4060 GPU. Our method achieves significantly lower latency than NTC [27], primarily due to its compact decoder network and reduced feature dimensionality, which effectively minimize sampling overhead. Although NDGI involves additional computations compared to standard lightmaps, the results demonstrate that our approach remains highly efficient and well suited for real-time rendering scenarios. 6 Discussion and Conclusion In temporally varying scenes, our method supports random access and enables smooth temporal transitions while maintaining a single parameter set in memory. However, abrupt illumination changes cannot be reflected within a single frame and typically exhibit a short latency at common update rates. In addition, the current training process in PyTorch [19] is also relatively slow. We plan to develop a dedicated training framework to improve throughput and to integrate the method more tightly into real-time pipelines through asynchronous execution and better scheduling. In conclusion, we present a compact and efficient compression framework for temporal lightmap sets that preserves random access across both space and time with minimal runtime cost. Our method achieves high-quality reconstructions at extremely low bitrates and supports fast online decompression suitable for real-time dynamic lighting. We believe this approach opens new possibilities for neural rendering and we will release our temporal lightmap dataset to encourage further research in this direction. References Alakuijala et al. [2019] Jyrki Alakuijala, Ruud Van Asseldonk, Sami Boukortt, Martin Bruse, Iulia-Maria Comșa, Moritz Firsching, Thomas Fischbacher, Evgenii Kliuchnikov, Sebastian Gomez, Robert Obryk, et al. Jpeg xl next-generation image compression architecture and coding tools. In Applications of digital image processing XLII, pages 112–124. SPIE, 2019. Ballé et al. [2016] Johannes Ballé, Valero Laparra, and Eero P Simoncelli. End-to-end optimized image compression. arXiv preprint arXiv:1611.01704, 2016. Campbell et al. [1986] Graham Campbell, Thomas A. DeFanti, Jeff Frederiksen, Stephen A. Joyce, Lawrence A. Leske, John A. Lindberg, and Daniel J. Sandin. Two bit/pixel full color encoding. SIGGRAPH Comput. Graph., 20(4):215–223, 1986. Delp and Mitchell [1979] E. Delp and O. Mitchell. Image compression using block truncation coding. IEEE Transactions on Communications, 27(9):1335–1342, 1979. Games [2025a] Epic Games. Unreal engine. https://w.unrealengine.com/en-US, 2025a. Accessed: October 25, 2025. Games [2025b] Epic Games. Understanding lightmapping in unreal engine. https://dev.epicgames.com/documentation/en-us/unreal-engine/understanding-lightmapping-in-unreal-engine, 2025b. Accessed: October 25, 2025. Games [2025c] Epic Games. Volumetric lightmaps in unreal engine. https://dev.epicgames.com/documentation/en-us/unreal-engine/volumetric-lightmaps-in-unreal-engine, 2025c. Accessed: October 25, 2025. Games [2025d] Epic Games. Virtual texturing. https://dev.epicgames.com/documentation/en-us/unreal-engine/virtual-texturing-in-unreal-engine, 2025d. Accessed: October 25, 2025. Hendrycks [2016] D Hendrycks. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016. Kajiya [1986] James T. Kajiya. The rendering equation. SIGGRAPH Comput. Graph., 20(4):143–150, 1986. Kingma [2014] Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. Knittel et al. [1996] Günter Knittel, Andreas Schilling, Anders Kugler, and Wolfgang Straßer. Hardware for superior texture performance. Computers & Graphics, 20(4):475–481, 1996. Knodt et al. [2023] Julian Knodt, Zherong Pan, Kui Wu, and Xifeng Gao. Joint uv optimization and texture baking. ACM Trans. Graph., 43(1), 2023. Laurent and Anis [2025] Belcour Laurent and Benyoub Anis. Hardware accelerated neural block texture compression with cooperative vectors. arXiv preprint arXiv:2506.06040, 2025. Microsoft [2025] Microsoft. Texture block compression in direct3d 11. https://learn.microsoft.com/en-us/windows/win32/direct3d11/texture-block-compression-in-direct3d-11, 2025. Accessed: October 25, 2025. Müller et al. [2022] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph., 41(4), 2022. Nystad et al. [2012] J. Nystad, A. Lassen, A. Pomianowski, S. Ellis, and T. Olson. Adaptive scalable texture compression. In Proceedings of the Fourth ACM SIGGRAPH / Eurographics Conference on High-Performance Graphics, page 105–114, Goslar, DEU, 2012. Eurographics Association. Ouyang et al. [2021] Yaobin Ouyang, Shiqiu Liu, Markus Kettunen, Matt Pharr, and Jacopo Pantaleoni. Restir gi: Path resampling for real-time path tracing. In Computer Graphics Forum, pages 17–29. Wiley Online Library, 2021. Paszke et al. [2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019. Pytorch [2025] Pytorch. torch.baddbmm. https://docs.pytorch.org/docs/stable/generated/torch.baddbmm.html, 2025. Accessed: October 25, 2025. Ritschel et al. [2012] Tobias Ritschel, Carsten Dachsbacher, Thorsten Grosch, and Jan Kautz. The state of the art in interactive global illumination. Comput. Graph. Forum, 31(1):160–188, 2012. Sloan and Silvennoinen [2018] Peter-Pike Sloan and Ari Silvennoinen. Directional lightmap encoding insights. In SIGGRAPH Asia 2018 Technical Briefs, New York, NY, USA, 2018. Association for Computing Machinery. Sloan et al. [2023] Peter-Pike Sloan, Jan Kautz, and John Snyder. Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 339–348. 2023. Stroem and Pettersson [2007] Jacob Stroem and Martin Pettersson. ETC2: Texture Compression using Invalid Combinations. In SIGGRAPH/Eurographics Workshop on Graphics Hardware. The Eurographics Association, 2007. Ström and Akenine-Möller [2005] Jacob Ström and Tomas Akenine-Möller. ipackman: high-quality, low-complexity texture compression for mobile phones. In Proceedings of the ACM SIGGRAPH/EUROGRAPHICS Conference on Graphics Hardware, page 63–70, New York, NY, USA, 2005. Association for Computing Machinery. Tatarchuk et al. [2022] Natalya Tatarchuk, Jonathan Dupuy, Thomas Deliot, Daniel Wright, Krzysztof Narkowicz, Patrick Kelly, Aleksander Netzel, and Tiago Costa. Advances in real-time rendering in games: part i. In ACM SIGGRAPH 2022 Courses, New York, NY, USA, 2022. Association for Computing Machinery. Vaidyanathan et al. [2023] Karthik Vaidyanathan, Marco Salvi, Bartlomiej Wronski, Tomas Akenine-Moller, Pontus Ebelin, and Aaron Lefohn. Random-access neural compression of material textures. ACM Transactions on Graphics, 42(4):1–25, 2023. Villegas and Ramírez [2016] José Villegas and Esmitt Ramírez. Deferred voxel shading for real-time global illumination. In 2016 XLII Latin American Computing Conference (CLEI), pages 1–11. IEEE, 2016. Wallace [1991] Gregory K Wallace. The jpeg still picture compression standard. Communications of the ACM, 34(4):30–44, 1991. Wang et al. [2004] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. Weinreich et al. [2024] Clément Weinreich, Louis De Oliveira, Antoine Houdard, and Georges Nader. Real-time neural materials using block-compressed features. In Computer Graphics Forum, page e15013. Wiley Online Library, 2024. Xing and Chen [2020] Qiwei Xing and Chunyi Chen. Path tracing denoising based on sure adaptive sampling and neural network. IEEE access, 8:116336–116349, 2020. Zhang et al. [2018] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018. Supplementary Material 7 Lightmap At a high level, real-time rendering first identifies visible surface points (e.g. via rasterization) and then shades each fragment by combining emitted radiance with reflected radiance integrated over the incident hemisphere. Formally, the rendering equation [10] relates outgoing radiance to emission and reflection: Lo(,)=Le(,)+∫Ωf(,)Li(,)(⋅)+.L_o(p,v)=L_e(p,v)+ _ f(l,v)\,L_i(p,l)\,(n\!·\!l)^+\,dl. (9) Here, Lo(,)L_o(p,v) is the outgoing radiance at point p toward view v. Le(,)L_e(p,v) denotes self emission from p toward v. Ω is the visible hemisphere. f(,)f(l,v) is the bidirectional reflectance distribution function (BRDF), and Li(,)L_i(p,l) is the incident radiance from direction l. The reflected term is often decomposed as f=fdiff+fspecf=f_diff+f_spec, where the diffuse term is direction independent. This yields: Lodiff() L_o^diff(p) =fdiff∫ΩLi(,)(⋅)+, =f_diff _ L_i(p,l)\,(n\!·\!l)^+\,dl, (10) Lospec(,) L_o^spec(p,v) =∫Ωfspec(,)Li(,)(⋅)+, = _ f_spec(l,v)\,L_i(p,l)\,(n\!·\!l)^+\,dl, and thus: Lo(,)≈Le(,)+Lodiff()+Lospec(,). L_o(p,v)≈ L_e(p,v)+L_o^diff(p)+L_o^spec(p,v). (11) Lightmap baking targets the view independent diffuse component. Concretely, we store the lighting information in lightmaps: LLM()=∫ΩLi(,)(⋅)+.L_LM(p)= _ L_i(p,l)\,(n\!·\!l)^+\,dl. (12) A high-quality lightmap typically includes direct illumination as well as multi-bounce indirect illumination. At runtime, we fetch the baked diffuse term using fragment UVs and apply it during shading. Generating lightmaps requires UVs. Only texels that map to visible surfaces are valid, so we use a binary mask to flag valid regions. Resolution sets the trade-off between detail and memory. 8 Block Compression Block compression (BC) is a family of fixed-rate, lossy texture compression formats designed for real-time GPU decoding. The core idea originates from Block Truncation Coding (BTC) [4]: the image is partitioned into small blocks (e.g. 4×44× 4 texels), and the color values within each block are approximated by a compact set of representative colors together with per texel selection indices. In the simplest and most representative case [15], each 4×44× 4 block stores two color endpoints e1,e2e_1,e_2 and a set of per texel interpolation weights wp\w_p\. At decode time, the color of each texel p is reconstructed by linearly interpolating between the endpoints: cp=(1−wp)e1+wpe2,p=1,…,16.c_p=(1-w_p)\,e_1+w_p\,e_2, 28.80008ptp=1,…,16. (13) More advanced variants (e.g. BC7) extend this scheme by supporting multiple partitions within a single block, each with its own pair of endpoints, thereby improving quality at the cost of a more complex encoding. This design enables constant-time random access to any texel, which is critical for GPU texture sampling. Different BC variants target different use cases. Adaptive Scalable Texture Compression (ASTC) [17] generalizes this framework with flexible block sizes, which provides a continuous trade-off between quality and bitrate. ASTC supports LDR, HDR, and 3D textures and is widely adopted on modern mobile platforms. A key advantage of block compression is that decoding is performed entirely in hardware during texture sampling, imposing small computational overhead on the shader pipeline. In NDGI, we leverage this property by applying BC7 to our learned feature maps uv2DF^2D_uv and uvt3DF^3D_uvt. 9 Virtual Texturing Virtual texturing (VT) [8], also known as megatexture or sparse virtual texturing, is a streaming technique that decouples the logical texture space from the physical GPU memory. It enables applications to reference texture data far exceeding the available video memory, loading only the portions that are actually visible. The key data structures of a VT system are: • Virtual texture. The entire logical texture, which may be many gigabytes and is divided into fixed-size tiles (e.g. 128×128128× 128 or 256×256256× 256 texels). Tiles are the atomic unit of streaming. • Physical texture. A GPU-resident texture atlas that holds only the currently needed tiles. Its capacity is bounded by available video memory. • Page table. An indirection texture that maps each virtual tile to its location in the physical texture. During shading, the shader first samples the page table with the fragment’s UV coordinates to obtain a physical tile address, and then samples the physical texture to retrieve the actual texel data. At runtime, the VT system operates in a feedback-driven loop. First, the GPU renders a low-resolution feedback buffer that records which virtual tiles are required for the current view. The CPU then reads back the feedback, identifies missing tiles, and transfers them from disk into the physical texture. Finally, the page table is updated to reflect the new mapping. Tiles that have not been referenced within a interval may be evicted to reclaim capacity for newly requested tiles. This architecture benefits NDGI in two ways. First, only the tiles visible in the current frame need to be decoded, reducing runtime decompression cost. Second, once decoded, a tile remains cached in the physical texture across frames until the lighting state changes, avoiding repeated inference. In our pipeline, each lightmap is partitioned into fixed-size tiles with a dedicated NDGI model per tile. The VT system identifies required tiles each frame and invokes neural decompression via a compute shader. Decoded results are written into the physical texture and invalidated only when re-decoding is needed. To support hardware texture filtering, each tile is stored with a small border (e.g. 4 pixels per edge). We apply the same mirrored padding during training, ensuring seamless tile boundaries. 10 Dataset 10.1 Dataset Description Our dataset comprises baked lightmap data across multiple scenes. For each scene, we provide two types of files: lightmap data files and mask files. The lightmap file stores 3-channel (RGB) lighting textures. We bake 24 sets per day at hourly intervals aligned to the top of the hour. For scenes that exhibit light-switching behavior, we additionally bake two extra sets around the on/off transition times to better capture fast lighting changes. The mask is a single-channel image with the same spatial resolution as the lightmaps. Each texel indicates whether the corresponding lightmap texel is valid. In real-time rendering, only valid regions are sampled, and compression methods can leverage this information to further improve compression ratio. We also provide a per scene configuration file that records the spatial resolution of each lightmap set and the correspondence among lightmaps, masks, and time indices. Unless otherwise noted, all lightmap values are treated as linear HDR intensities for evaluation and visualization. 10.2 Dataset Evaluation We evaluate lightmap reconstruction quality using PSNR, SSIM [30], and LPIPS [33]. Metrics are computed in a tiled manner by partitioning each lightmap into non-overlapping 128×128128× 128 tiles. For each tile, we compute the metric within the valid region defined by the mask and report the result by averaging over all tiles across all lightmaps of that scene. Because the data are HDR, we determine the dynamic range per tile: the minimum and maximum values within the tile’s valid region set the range (e.g. the peak value for PSNR), rather than adopting fixed global constants. Unless specified otherwise, metrics are computed in linear RGB. This tiled, mask-aware protocol yields stable and comparable scores across scenes with different UV packings and sparsity patterns. 11 Ablation Study Reference 2D Feature Map Hybrid Feature Map PSNR(↑ ) 41.12 dB 44.77 dB Figure 9: At comparable bitrates and with the same decoder, our hybrid feature maps outperform 2D-only feature maps, producing less noise. Reference w/o BC simulation w/ BC simulation PSNR(↑ ) 39.82 dB 44.92 dB Figure 10: We compare variants with and without the BC simulation strategy and report PSNR. Parameterizing the feature map with BC endpoints and weights yields less noisy final reconstructions than directly optimizing feature parameters. Table 6: PSNR comparison for feature maps with different structures at comparable bitrates. Feature Map Structure BPP(↓ ) PSNR(↑ ) 2D Feature Maps 0.73 42.1 dB 3D Feature Maps 0.69 36.6 dB Hybrid Feature Maps 0.67 44.9 dB Table 7: PSNR comparison for modeling feature maps with endpoints and weights or not. Feature Map Strategy PSNR(↑ ) SSIM(↑ ) LPIPS(↓ ) w/o BC Simulation 37.96 dB 0.975 0.013 w/ BC Simulation 44.20 dB 0.992 0.004 We validate the effectiveness of our hybrid feature representation. Figure 9 compares a baseline that uses only 2D feature maps with our hybrid features at comparable bitrates under the same decoder configuration. The hybrid representation captures lightmap content more faithfully, yielding smoother shading transitions and less aliasing, and it consistently improves reconstruction quality across viewpoints. Table 6 reports PSNR on a test scene across several feature map architectures at matched bitrates, showing that the hybrid representation attains higher accuracy with similar storage. We also evaluate a BC simulation strategy and observe clear gains across test scenes and metrics. The results are shown in Figure 10 and Table 7. To ensure a fair comparison, both variants apply BC compression after training, while they differ in how the feature map is parameterized during optimization. In the first variant, we model the feature map with BC endpoints and weights during training, which better matches the target representation, encourages piecewise linear structure, and improves rate distortion behavior. In the second variant, we directly optimize the full feature map and compress it with BC at the end, which creates a mismatch with the BC quantization grid and introduces high frequency noise and block inconsistency. Empirically, the endpoints and weights modeling yields consistently higher fidelity and fewer artifacts, whereas direct optimization introduces substantial noise that degrades the final quality. Table 8: Tri-plane vs. 8-channel uv2DF^2D_uv on NDGI.M. Tri-plane yields smaller storage at higher quality. Method PSNR ↑ BPP ↓ Size (MB) ↓ 8-channel uv2DF^2D_uv 41.32 0.86 358 Tri-Plane 41.50 0.67 279 We further compare our tri-plane hybrid feature representation against a single 8-channel uv2DF^2D_uv baseline on NDGI.M. As shown in Table 8, the tri-plane design achieves comparable or higher PSNR at a noticeably lower bitrate and smaller storage footprint. We attribute this to the additional ut2DF^2D_ut and vt2DF^2D_vt planes, which more effectively capture higher-frequency temporal variations that a single spatial feature map cannot represent as compactly. 12 Rendered Results We provide additional qualitative comparisons of rendered results in the supplementary. Under matched bitrates, Figure 11 compares PRT [23], NTC L. [27] and our NDGI M. (Ours) against the reference. Our method delivers cleaner global illumination, fewer color shifts and less noise, and better preservation of fine details, which leads to higher visual fidelity relative to the reference. Row annotations report BPP, PSNR, and SSIM, and our method consistently achieves a better trade-off between quality and bitrate across scenes. Lit Scene PRT NTC L. NDGI M. (Ours) Reference BPP(↓ ), PSNR(↑ ), SSIM(↑ ) 0.92, 26.09 dB, 0.921 0.78, 45.56 dB, 0.998 0.67, 47.53 dB, 0.999 FarmLand BPP(↓ ), PSNR(↑ ), SSIM(↑ ) 0.92, 24.39 dB, 0.971 0.78, 46.28 dB, 0.998 0.67, 47.55 dB, 0.999 FarmLand BPP(↓ ), PSNR(↑ ), SSIM(↑ ) 1.00, 17.45 dB, 0.719 0.78, 45.41 dB, 0.997 0.71, 47.42 dB, 0.998 Yard BPP(↓ ), PSNR(↑ ), SSIM(↑ ) 1.00, 15.18 dB, 0.667 0.78, 38.50 dB, 0.998 0.71, 41.07 dB, 0.999 Room BPP(↓ ), PSNR(↑ ), SSIM(↑ ) 1.00, 10.37 dB, 0.688 0.78, 39.97 dB, 0.999 0.71, 42.25 dB, 0.999 Room Figure 11: Additional comparisons of rendering quality. Compared to PRT [23], our method better captures global illumination tone variations, such as multi-bounce interreflections. Compared to NTC [27], it reconstructs lightmaps with higher fidelity and noticeably less noise.