Paper deep dive
Physically Real-time Infrared Attack against Optical Flow Estimation Networks
Shen You, Wei Jiang, Jiarui Liu, Yijian Ye, Qiuzhen Lin, Xiangtao Li, Ka-Chun Wong
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/4/2026, 10:56:00 AM
Summary
This paper proposes a physically real-time infrared attack method against Optical Flow Estimation Networks (OFENs) to impair their ability to estimate motion. The approach uses infrared lights attached to target objects to display adversarial examples (AEs) in real-time, leveraging the stealth of infrared light and a two-stage process involving Genetic Algorithm (GA) training and an Adversarial Generative Network (AGN) for real-time inference. The attack aims to break the brightness constancy assumption of OFENs, causing erroneous optical flow estimates without modifying the victim system.
Entities (10)
Relation Signals (9)
RILA → targets → Optical Flow Estimation Networks
confidence 98% · Physically Real-time Infrared Attack against Optical Flow Estimation Networks
Infrared Lights → displays → Adversarial Examples
confidence 95% · dynamically displays them [AEs]
RILA → uses → Infrared Lights
confidence 95% · we propose a physically real-time infrared light attack method... leveraging infrared lights for their stealthiness.
RILA → uses → Adversarial Generative Network
confidence 93% · Adversarial Generative Network (AGN) is used to generate AEs in real-time.
RILA → breaks → Brightness Constancy
confidence 92% · our attack primarily aims to break the brightness constancy assumption
RILA → uses → Genetic Algorithm
confidence 92% · we use the GA algorithm to generate a large number of adversarial examples as training data.
Optical Flow Estimation Networks → assumes → Brightness Constancy
confidence 90% · Brightness Constancy: In adjacent frames, the brightness of the same object remains unchanged.
Adversarial Examples → generatedby → Adversarial Generative Network
confidence 90% · AGN is used to generate AEs in real-time.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:With the promising performance of deep neural networks on image-based tasks, different real-world applications such as autonomous driving and motion detection have become increasingly mature and relevant to human lives. In particular, Optical Flow Estimation Networks (OFENs), as upstream models, play a critical role in different domains. Its outputs are heavily assumed and adopted for different downstream tasks, and it is essential to test its robustness to prevent safety accidents. We present an approach for real-time attacks on OFENs in the physical world, leveraging infrared lights for their stealthiness. By generating a large number of Adversarial Examples in advance, our approach computes AEs in real time and dynamically displays them, which allows our method to facilitate precise and targeted attacks without modifying the victim system. Unlike previous digital-to-physical attack techniques, our method directly attacks victim models within the physical world, thereby overcoming the limitations associated with the ineffectiveness of AEs. Experimental results demonstrate the efficacy of our approach in compromising OFENs across diverse lighting conditions, varying object motion velocities, and different object placements, ultimately impairing the network's ability to accurately estimate optical flow.
Tags
Links
- Source: https://arxiv.org/abs/2607.26651v1
- Canonical: https://arxiv.org/abs/2607.26651v1
Trouble viewing inline? Open PDF directly →
Full Text
41,593 characters extracted from source content.
Expand or collapse full text
Physically Real-time Infrared Attack against Optical Flow Estimation Networks Shen You City University of Hong Kong Tat Chee Avenue, Kowloon, Hong Kong SAR shenyou2-c@my.cityu.edu.hk Wei Jiang University of Electronic Science and Technology Chengdu, Sichuan, China weijiang@uestc.edu.cn Jiarui Liu University of Electronic Science and Technology Chengdu,Sichuan,China jiarui.liu@std.uestc.edu.cn Yijian Ye University of Electronic Science and Technology Chengdu,Sichuan,China 202321090116@std.uestc.edu.cn Qiuzhen Lin Shenzhen University Shenzhen,Guangdong,China qiuzhlin@szu.edu.cn Xiangtao Li Jilin University Changchun,Jilin,China lixt314@jlu.edu.cn Ka-Chun Wong* City University of Hong Kong Tat Chee Avenue, Kowloon, Hong Kong SAR kc.w@cityu.edu.hk Abstract With the promising performance of deep neural networks on image-based tasks, different real-world applications such as autonomous driving and motion detection have become increasingly mature and relevant to human lives. In particular, Optical Flow Estimation Networks (OFENs), as upstream models, play a critical role in different domains. Its outputs are heavily assumed and adopted for different downstream tasks, and it is essential to test its robustness to prevent safety accidents. We present an approach for real-time attacks on OFENs in the physical world, leveraging infrared lights for their stealthiness. By generating a large number of Adversarial Examples (AEs) in advance, our approach computes AEs in real time and dynamically displays them, which allows our method to facilitate precise and targeted attacks without modifying the victim system. Unlike previous digital-to-physical attack techniques, our method directly attacks victim models within the physical world, thereby overcoming the limitations associated with the ineffectiveness of AEs. Experimental results demonstrate the efficacy of our approach in compromising OFENs across diverse lighting conditions, varying object motion velocities, and different object placements, ultimately impairing the network’s ability to accurately estimate optical flow. 1 Introduction Optical Flow Estimation Networks (OFENs) are proposed to estimate the motion between two image frames. It has the capability to identify and track moving objects under a static background by analyzing the changes in optical flow. These capabilities have enabled OFENs to be widely adopted in various applications, such as motion recognition [4], behavior analysis [17], vehicle tracking [11] and obstacle avoidance [18]. For example, Capito et al. [3] propose an artificial potential field, known as a visual potential field, from a sequence of images using sparse optical flow. This field is then utilized in conjunction with a gradient tracking sliding mode controller to navigate the vehicle to its destination while avoiding obstacles. Typically, OFENs are considered an upstream task model where its outputs serve as the inputs for other downstream models. Consequently, it significantly influences the performance of downstream models. Therefore, the robustness of OFENs in autonomous driving has garnered considerable attention. However, as a type of Deep Neural Networks (DNNs), OFENs inherit similar vulnerabilities to adversarial perturbations [15, 5], which can significantly disrupt model outputs. This poses a unique challenge in autonomous driving as erroneous predictions can lead to severe consequences for the safety of drivers and passengers, where OFENs’ outputs are critical for downstream decision-making systems. Figure 1: The application infrared attack. Despite the extensive research on Adversarial Examples (AEs) in the physical world [22, 8, 23, 1], there are several critical limitations. First, most studies evaluate the performance in the digital domain, which limits their practicality due to the complex transformations required for transferring these attacks to the physical world. Second, some physical attacks [14, 21] do not account for the differences between the digital and physical domains; they simply print AEs generated in the digital world. However, cameras inevitably capture noise, which can degrade the effectiveness of these AEs. Third, they do not consider the movement of target object [2, 10], which leads to failures when the target object changes its angle or position, particularly in the context of attacking OFENs. Finally, most attacks rely on visible patches or lights [12, 13, 7], which can easily be noticed by humans, thus violating the principle of imperceptibility in AEs. To address the aforementioned limitations, we propose a physically real-time infrared light attack method. Figure 1 shows the application of our method. The left figure shows the attacker paste the infrared lights on the surface of car, allowing it been monitored and escape from traffic police. The right figure shows a thief wants steal jewelry and leverage infrared lights to spoof OFEN. Since most cameras are designed to work in low-light environments, they are usually not equipped with infrared filters, which gives good physical conditions for our attacks. Our contributions can be summarized as follows: • A real-time adversarial attack method is proposed to shift the attack paradigm from digital-attack and physical-deploy to physical-train and physical-deploy. This approach significantly reduces the likelihood of AE failures. • Infrared lights are proposed to reduce the probability of AEs being detected because of the invisibility to the human eyes, enabling our attacks to be stealthy and imperceptible. • Three losses are introduced to effectively guide the generation of AEs for attacking OFENs. Experimental results show that our method achieves robust attack performance across multiple scenarios. 2 Related Works In the following, we mainly focus on the development of OFENs and related attack work in this field. 2.1 Optical Flow Estimation The OFENs are used to track the motion of the target object, which is based on the following three assumptions: • Brightness Constancy: In adjacent frames, the brightness of the same object remains unchanged. • Time Continuity Assumption: The interval between adjacent video frames is very small, or the motion of objects between adjacent frames is relatively small. This implies that the motion of the object is continuous between adjacent frames without sudden jumps. • Spatial Coherence: Pixels in the same sub-image have the same motion. This means that the motion of pixels in the same sub-image between adjacent frames is coordinated and consistent. The development of deep learning on optical flow estimation networks can be categorized into three stages as follows: In 2015, Dosovitskiy et al. [6] introduced deep learning methods to estimate optical flow, constructing convolutional neural networks capable of solving the optical flow estimation problem as a supervised learning task. They proposed two main network architectures: FlowNetSimple, which features a straightforward encoder-decoder structure, and FlowNetCorr, which first extracts features for each frame and then uses a correlation layer to compute the similarity between the features of the two frames. In 2018, Sun et al. [19] introduced PWC-Net, utilizes three key components: an image pyramid, warping, and a matching cost volume (similar to the correlation computation in FlowNet). The warping and matching cost volume computations do not require training parameters, thereby reducing the overall number of model parameters. PWC-Net achieves real-time optical flow estimation on 1024×436 resolution videos, processing up to 35 frames per second with only 0.06 times the number of parameters compared to FlowNet2.0. Subsequently, Teed et al. [20] introduced Recurrent All-Pairs Field Transforms (RAFT) in 2020, a novel deep network architecture for optical flow estimation. RAFT extracts per-pixel features, constructs multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates the flow field through a recurrent unit that performs lookups on these correlation volumes. (a) Timeline (b) Sequence Chart Figure 2: (a) Timeline and (b) Sequence Chart of the OFEN attacking process with AEs in the physical world. 2.2 Physical Attacks for OFENs In recent years, there are some research on attacking OFENs. Ranjan et al. [16] extended adversarial patch attacks to optical flow networks and found that corrupting a small patch, covering less than 1% of the image size, can significantly affect optical flow estimates. Abhiram et al. [9] propose an adversarial attack in the physical world aimed at fooling image classifiers without physically touching the objects. Their method uses a low-cost projector to alter the appearance of the target objects. Guo et al. [10] present an attack that uses light-emitting diodes and exploits the camera’s rolling shutter effect to create adversarial stripes in the captured images to mislead traffic sign recognition. Although the existing research achieved remarkable success, it still has some fatal problems. The methods used in the digital domain are often difficult to directly migrate and apply to the physical world and different environmental conditions such as lighting, distance, and sensor viewing angle, the image captured by the sensor and input into the model will differ from the original image. Additionally, fabrication errors pose a significant challenge. To fabricate perturbations based on theoretical calculations, every pixel must be accurately printed using existing equipment, such as adversarial patches. Due to the limitations of the color gamut of modern printing equipment, some theoretically calculated colors may not be fully or accurately reproduced. 3 Problem Formulation Optical flow ∈ℝH×W×2O ^H× W× 2 indicates the motion of target object where (x,y)=[Δx,Δy]TO(x,y)=[ x, y]^T. For a real-time frame flow i∈ℝH×W×CI^i ^H× W× C with three channels (rgb) at time t=it=i, OFEN ℱ(⋅,⋅)F(·,·) takes the previous frame i−1I^i-1 and current frame iI^i to estimate optical flow ∈ℝH×W×2O ^H× W× 2. The relationship between optical flow O and frame I is shown as Eq. 1 ∂xΔx+∂yΔy+∂t=0 ∂ x x+ ∂ y y+ ∂ t=0 (1) This differential form derives from the first-order taylor expansion under brightness constancy assumption. x and y represent the specific pixel coordinates in the frame, while H and W denote the height and width of the input frame, respectively. The optical flow O has two dimensions (Δx,Δy)( x, y) in each position, corresponding to the horizontal and vertical components of the flow. Next, we need to define the perturbation for the infrared attack. Consider a scenario where n infrared lights are attached to the surface of the target object, and each light has a normalized intensity range from 0 to 1. The perturbation ∈ℝnP ^n can be encoded as a vector with n elements, where each element in P represents the intensity of the corresponding infrared light. Next, we define a non-derivative conversion function (⋅)C(·), which transforms the perturbation into the physical world. Specifically, Eq. 2 shows the attacked optical flow at time t=it=i. ∗i=ℱ((i−1,∗i−1),(i,∗i))O^i_*=F(C(P^i-1,I^i-1_*),C(P^i,I^i_*)) (2) Then, we formulate the attack problem as follows: max|∗i−i|s.t.i∈[0,1],ti−ti−1≤ϵ split & |O^i_*-O^i|\\ s.t. &P^i∈[0,1], t_i-t_i-1≤ε split (3) Here, ϵε represents the real-time constraint and the objective is to make the perturbed optical flow as different as possible from the original optical flow. 4 Methodology In this section, we introduce a Real-time Infrared Light Attack (RILA). Our goal is to deploy a real-time attack to make sure the target object escapes being tracked by optical flow estimation network. Considering that the tracking camera captures frames every 30ms or less, making it necessary for the infrared light to react in a shorter period of time, we divide the whole attack process into two stages. Within the first phase, we build a training environment, which is the same as the real environment, and we use the GA algorithm to generate a large number of adversarial examples as training data. Within the second stage, we use the training data to train a perturbation generation network, which can generate adversarial examples in time. • Training: An evolutionary algorithm is adopted to generate a large number of AEs as training data. • Attacking: Adversarial Generative Network (AGN) is used to generate AEs in real-time. 4.1 Overall Framework Fig. 2 (a) illustrates the timeline and sequence chart of the attack process. The arrows above the timeline represent the capture times of the camera, while the arrows below the timeline represent the actions of our attack. • At t1t_1, the camera captures frame 1I_1. • At t2t_2, we turn off the infrared lights, calculate the corresponding adversarial example and send it to the hardware controller. • At t3t_3, the hardware controller displays the AE. • At t4t_4, the camera captures the frame 2I_2 and uses frames 1I_1 and 2I_2 to estimate the optical flow. In the above analysis, there are some relationships between those times. Firstly, 1(t4−t1)= 1(t_4-t_1)=R means the frame rate of the camera, which is normally equal to 30 or 60. Besides, in the train stage, t3−t2t_3-t_2 can be longer because we can fix the position of the target object to provide sufficient time for the search process. In Attacking stage, t3−t2t_3-t_2 must be very short to ensure real-time performance. Fig. 2 (b) shows the sequence chart of our attack process and how the AE disrupts the OFEN. We use GA (in Training stage) or AGN (in Attacking stage) to generate AE and use the hardware to display it. Once the AEs are presented, the attack at the current position is complete. The perturbed frame is then captured by the camera and fed into the target model, leading to the generation of an erroneous optical flow. 4.2 GA for Non-Real Time Training Owing to the convert function (⋅)C(·) is a physical process and is non-derivative, we cannot use the gradient-based optimization algorithm, so we use a genetic algorithm, which does not need gradients to generate AEs. The goal of GA is to generate an optimal light strength array P that constructs a perturbed frame according to Eq. 2 to satisfy the objective defined in Eq. 3. Figure 3: The flowchart of AE generation with GA Fig. 3 shows how we use GA to generate AEs. Firstly, we initialize a perturbation set, and then for each perturbation, we send it to the hardware controller. The hardware controller will set each infrared light at specific power to get the dirty frame, then the previous frame and the current dirty frame will be sent to OFEN to get the fitness. After we get the fitness of each perturbation, we need to construct the next generation of perturbations. This involves selection, crossover, and mutation. By iteratively applying these steps, the GA evolves the perturbations, gradually improving their ability to disrupt the OFEN. This process continues until the desired level of attack performance is achieved. 4.2.1 Initial Perturbation Strategies Unlike other methods that use uniform distributions to initialize perturbations, we employ two preset initial plans based on the specific situation according to the feature of OFEN. We provide an example to explain how it works, considering the frames at t1t_1 and t2t_2. Since our attack primarily aims to break the brightness constancy assumption that the brightness of the same object remains unchanged in adjacent frames, we need to make the brightness between adjacent frames more different to break this assumption. Specifically, we need to increase the brightness in the current frame and then decrease it in the next frame. To control this process, we introduce a parameter γ. If we are attacking odd frames, γ is set to 0.2; otherwise, γ is set to 0.8. The initial perturbation iP_i for each infrared light is given by: i=0.1⋅()+γP_i=0.1·U()+γ (4) Here, ()U() will return a uniformly distributed random value between 0 and 1. 4.2.2 Fitness Design The fitness function is crucial for guiding the evolution process and significantly impacts the performance of the evolutionary algorithm. Although Eq. 3 requires maximizing the difference between the optical flow and perturbed optical flow, we choose the approximate approach for convenience, which is donated by these three sub-fitness terms. • Invisibility (Vis L_Vis): This term is defined as Vis=|i|H×W×C∈[0,1] L_Vis= |O^i|H× W× C∈[0,1], measures the invisibility of the perturbation. When Vis L_Vis tends to one, it indicates that the optical flow is blank, meaning no moving object will be detected. • Imperceptibility (Per L_Per): This term, defined as Per∈[0,1] L_Per∈[0,1], it equals one minus the average variance of each sliding window, like in the convolution operation 1−(i)¯1- V(i), i means the sliding window. It measures the imperceptibility of the perturbation. A higher Per L_Per value indicates that the optical flow is more chaos and unable to accurately perceive the motion, making it difficult for the model to identify the true movement. • Flash Loss (Flash L_Flash): Based on previous experience, we introduce a flash loss term, Flash=|i−i−1|n∈[0,1] L_Flash= |P^i-P^i-1|n∈[0,1]. A large Flash L_Flash value indicates that brightness in adjacent frames is more different, which can decrease the victim model’s performance. Given these three sub-fitness terms, the overall fitness function is a typical multi-objective optimization problem and is defined as: Fitness(Pi)=α×Vis+β×Per+FlashFitness(P^i)=α× L_Vis+β× L_Per+ L_Flash (5) where α, β, are weighting factors. 4.2.3 Perturbations Update The update of perturbations in our evolutionary algorithm consists of three main genetic operators: selection, crossover, and mutation. We use those operators to generate the next generation. Selection Operator A selection operator is used to retain domain genes into the next population. We adopt a tournament selection strategy similar to that used in NSGA-I. The process is divided into the following steps: Firstly, we randomly select a fixed number of perturbations from the current population. Then, among the selected perturbations, choose the one with the highest fitness score. Next, the individual with the best performance is designated as the ’winner’ and may become part of the new generation. Finally, the above steps are repeated multiple times until the required number of perturbations is obtained. Crossover Operator In the crossover operator, we combine two parent perturbations to create offspring. We select multiple fixed crossover points and exchange the perturbation values between the parents. The resulting offspring inherit a combination of traits from both parents, which can lead to new and potentially better solutions. Mutation Operator In the mutation operator, we introduce small random changes to the perturbation values of the selected individuals. This helps to maintain diversity in the population and prevents premature convergence. Specifically, for each element in the perturbation vector P, we apply a small random perturbation drawn from a uniform distribution. This ensures that the perturbation remains within the valid range [0, 1]. i′=()P_i =U(), where i′P_i is the mutated perturbation value. 4.3 AGN for Real Time Attacking Since the time-costing nature of GA is not suitable for real-time attacks, we design a generative network to generate AEs using the last perturbation to generate the next perturbation. In order to fulfill the real-time requirement, the AGN network must generate perturbations in a short period of time, so we use a simple network with a fully connected layer for perturbation generation. This model is also easy to deploy at an embedded system and the input of it is i−1P^i-1 and output is iP^i. 4.4 Pseudocode of Real-time Physical Attack Algorithm 1 Pseudocode of RILA Input: The attack parameters. Output: The dirty optical flow. Generate Training Dateset 1: while Target object is on the track do 2: Capture −I^i-1 from camera for time i-1. 3: Set move instruction to hardware controller. 4: P^i = GA(i−1)GA(I^i-1). 5: Add (−,)(P^i-1,P^i) as train data. 6: Send iP^i to hardware controller and display it 7: end while 8: Using data to train AGNAGN Physical Real-Time Attack 1: while Target object is under tracking do 2: iP^i = AGN(i−1)AGN(P^i-1). 3: Send iP^i to hardware controller and display it. 4: Wait for R millisecond. 5: end while Alg. 1 shows the pseudocode for the real-time physical attack. In training stage, we mainly focus on the data collection. While the target object is still on the track, we capture the initial frame 1I_1 at time i−1i-1. Then, We send a move instruction to the hardware controller. By using the GA, we generate the optimal perturbation iP^i, and we add the pair (−,)(P^i-1,P^i) to the training dataset. Next, we send the perturbation iP^i to the hardware controller and display it. Finally, after collecting sufficient training data, we use this data to train the AGN. In real-time attacking stage, we deploy a real-time attack. If the target object is still under monitoring, we generate the perturbation iP^i from Pi−1P^i-1 at time i, and we send the perturbation iP^i to the hardware controller and display it. Next, we wait for ℝR milliseconds to wait for the next iteration. 5 Experiments In this section, experiments are conducted to evaluate the effectiveness of the proposed method. All experiments have been performed on an Intel(R) Xeon(R) Gold 5218 CPU at 2.30GHz, one GPU of NVIDIA GeForce RTX 2080Ti, nine 840nm infrared lights, and an ESP32 controller. 5.1 Experiment Setup 5.1.1 Physical Test Environment To test RILA, we constructed a physical test environment consisting of five main components: a track, ambient lighting, attached infrared lights, camera, and a controller. • Track: The track is used to control the movement of the target object, allowing us to easily stop the object and wait for the GA to compute the AEs. • Ambient lighting: This component helps us evaluate the performance of RILA under different lighting conditions. • Attached infrared lights: Infrared lights are attached to the surface of the target object and dynamically display the AEs. • Camera: The camera captures 30 frames per second, which is used for estimating the optical flow. • Controller: The controller manages the track, ambient lighting, and attached infrared lights. Figure 4: The experiment environments. The physical test environment is shown in Figure 4. Considering that OFEN only detects the movement of pixels, not the change of shape and color of objects. So we embedded an infrared light on a plastic plate instead of a small car or a human body. 5.1.2 Victim Model As for victim model, we choose the existing SOTA model RAFT [20] and another milestone model PWC-Net [19]. As for comparison methods, because there are not any methods with the same physical test environments and we do not have the digital attack stage, we only test our methods in this work. 5.1.3 Performance Metrics Two performance metrics are computed to evaluate the performance of RILA: Average Invisibility (AIV) and Average Imperceptibility (AIP). Higher AIV means the lower probability to be detected by downstream model. Higher AIP means it is hard to get the true motion of the output optical flow. 5.2 Ablation Study In this section, we mainly focus on the validation of loss item, We test a version with all loss item against versions without individual loss item. Table 1: Ablation Study Results. Dis (cm) RAFT [20] PWC-Net [19] AIV AIP AIV AIP N-Vis .887 ± .007 .974 ± .004 .901 ± .011 .980 ± .004 N-Per .888 ± .027 .976 ± .010 .894 ± .007 .977 ± .002 N-Flash .869 ± .022 .969 ± .008 .882 ± .022 .972 ± .008 FULL .900 ± .011 .979 ± .004 .907 ± .013 .980 ± .004 From Table 1, Fig. 5 6 shows the results, we observe the following effects when individual loss functions are removed: without invisibility loss, we could notice the AIV and AIP decreased by 1.16%, 0.26%. This indicates that the output optical flow has difficulty in accurately capturing the true motion. As shown in Fig. 5(a), the optical flow appears chaotic and disorganized. Without imperceptibility loss, The AIV and AIV decreased by 1.70%, 0.38%. The optical flow tends to become blank, as seen in Fig. 5(b). This makes it challenging to locate and track the target object, as the flow lacks the necessary details to represent the object’s motion accurately. Without flash loss, both the AIV and AIP decreased by 3.22%, 0.53%. This reduction occurs because the absence of the flash loss diminishes the ability of the infrared lights to alter the lighting on the object’s surface across multiple frames, so the general outline of the object remains visible, as shown in Fig. 5(c). It can be observed that the full version of the method, which includes all three loss items, performs better than any of the incomplete versions. This indicates that each loss function contributes uniquely to achieving a stronger and more effective attack, and the flash loss is the most important item. (a) Without Vis L_Vis (b) Without Per L_Per (c) Without Flash L_Flash Figure 5: Disable one loss item when attacking RAFT. Figure 6: The target object is moving to the left from right. The original optical flow (up) and the attack results with full loss item (down) when attacking RAFT. 5.3 Attack performance In this section, we test RILA under different physical conditions compared with Blank methods. 5.3.1 Ambient Lighting OFENs rely on the light reflected from the surface of the target object, and our method attacks on this principle. Therefore, ambient lighting conditions are crucial, as it can interfere with the infrared light. This is a common scenario due to the natural variation in ambient lighting, such as the dimmer conditions in the evening and the brighter conditions at midday. To control these variables, we conducted our tests in a black box environment with four adjustable LED lights to simulate different ambient light conditions. (a) Without Vis L_Vis (b) Without Vis L_Vis Figure 7: The invisibility of RILA in different ambient lights. Figs. 7 show the AIV and AIP of RILA under different ambient lighting conditions. As the ambient light intensity increases, the original optical flow becomes clearer. When the ambient light intensity is below 0.3, the original optical flow is poorly estimated because OFENs struggle to track the light on the target object. As a result, the attacked flow becomes random, leading to a decline in invisibility and imperceptibility. In the range of 0.3 to 0.7 ambient light intensity, OFENs can accurately track the motion, and RILA effectively manipulates the OFENs to output a blank optical flow rather than a random flow, thus increasing the invisibility and imperceptibility. However, when the ambient light intensity exceeds 0.7, the enhanced natural light reduces the relative contribution of the infrared light on the object’s surface, making the attack more challenging. Although the ambient lighting will affect our approach, the results indicate that RILA performs well across a wide range of lighting conditions. Table 2: Performance at different object motion speeds. Speed (cm/s) RAFT[20] PWC-Net[19] AIV AIP AIV AIP 6 .929 ± .019 .989 ± .005 .932 ± .019 .990 ± .005 9 .908 ± .007 .982 ± .005 .911 ± .002 .985 ± .008 12 .899 ± .014 .980 ± .005 .888 ± .002 .980 ± .006 15 .868 ± .018 .963 ± .005 .879 ± .009 .965 ± .001 18 .859 ± .021 .963 ± .013 .873 ± .007 .967 ± .008 21 .866 ± .017 .964 ± .010 .863 ± .017 .964 ± .010 24 .859 ± .016 .960 ± .012 .872 ± .005 .968 ± .003 27 .856 ± .028 .960 ± .016 .875 ± .008 .953 ± .012 30 .858 ± .006 .962 ± .005 .859 ± .009 .963 ± .007 33 .838 ± .002 .955 ± .010 .842 ± .005 .957 ± .010 5.3.2 Object Speed Considering the target object could have different speed in real world, and OFENs are based on the change of position of each pixel, It is more likely to track the optical flow when the change is huge and ignore the small motion, so we test our methods in different object speed. Table 3: Performance on different camera distances Dis (cm) RAFT[20] PWC-Net[19] AIV AIP AIV AIP 30 .896 ± .013 .978 ± .005 .900 ± .013 .982 ± .005 40 .892 ± .025 .977 ± .008 .909 ± .004 .987 ± .008 50 .867 ± .009 .970 ± .004 .871 ± .009 .975 ± .003 60 .858 ± .005 .967 ± .002 .860 ± .006 .969 ± .004 70 .857 ± .011 .970 ± .003 .866 ± .003 .971 ± .002 80 .880 ± .034 .976 ± .012 .899 ± .013 .982 ± .005 90 .859 ± .025 .967 ± .010 .858 ± .025 .971 ± .010 100 .881 ± .007 .975 ± .002 .885 ± .004 .976 ± .001 110 .872 ± .010 .973 ± .003 .874 ± .011 .972 ± .001 120 .853 ± .024 .964 ± .005 .868 ± .012 .968 ± .005 Table 2 shows the performance of our methods when attacking target objects moving at different speeds. We can observe that the AIV and AIP gradually decrease as the speed of the target object increases. This trend indicates that our method performs more effectively under lower object speeds. 5.3.3 Camera Distance Given that the distance between the target object and the camera varies in typical scenarios, it can affect the size of the target object in the input frame. Since OFENs are generally more adept at tracking large objects rather than small ones, we tested RILA at different camera distances. Due to the limitations of our testing environment, we selected eleven close distances, and the results are shown in Table 3. We observe that the AIV and AIP do not change a lot with the increase of camera distance when attacking both models. This may be because the size of the area affected by the infrared light is fixed. Even if the object is far or near the camera, the infrared lights still fully cover the object’s surface; this feature makes our methods could be used in wide environments. 5.3.4 Iteration number in RAFT Unlike traditional OFENs, RAFT employs an iterative Long Short-Term Memory (LSTM) module for decoding the optical flow, which typically results in improved performance as the Number Of Iterations (NOI) increases. To evaluate the robustness of our attack method, we conducted experiments across various iteration numbers. The results are summarized in Table 4. Table 4: The performance in differ iteration number of RAFT. NOI Under Attacking Original AIV AIP AIV AIP 5 .935 ± .004 .990 ± .009 .900 ± .019 .971 ± .012 6 .925 ± .001 .987 ± .003 .854 ± .013 .953 ± .016 7 .933 ± .016 .990 ± .003 .841 ± .016 .941 ± .003 8 .941 ± .014 .990 ± .005 .839 ± .012 .938 ± .018 9 .935 ± .017 .988 ± .005 .813 ± .006 .919 ± .007 10 .919 ± .015 .984 ± .004 .828 ± .011 .930 ± .003 11 .923 ± .016 .986 ± .015 .819 ± .015 .922 ± .011 12 .919 ± .014 .982 ± .004 .820 ± .007 .924 ± .018 13 .916 ± .003 .980 ± .018 .815 ± .016 .921 ± .018 14 .915 ± .002 .987 ± .014 .819 ± .002 .921 ± .012 15 .912 ± .014 .989 ± .005 .814 ± .015 .918 ± .018 In the context of the original optical flow, we observed a significant decline in both AIV and AIP when the iteration count increased from 5 to 9. However, beyond 9 iterations, the AIV and AIP metrics remained relatively stable, indicating that additional iterations did not substantially improve performance. When the victim model was subjected to our attack, a similar trend was noted: a notable drop in AIV and AIP occurred between 5 and 9 iterations, after which the metrics stabilized. Specifically, under attack, the AIV and AIP metrics decreased by only 2.48% and 0.94%, respectively, compared to a 9.52% and 5.84% reduction in the original optical flow. This suggests that our method maintains its effectiveness across different iteration numbers, demonstrating its robustness against the varying computational complexity of the RAFT model. 5.4 Discussion From the above experiments, the proposed method has achieved high attack performance across different situations. It can be attributed to designing several bespoke loss functions to guide the generation of adversarial perturbations: Invisibility loss Vis L_Vis reduces the probability that the target object is detected. By maximizing it, the optical flow becomes blank, making it difficult for the system to detect any moving objects. Imperceptibility loss Per L_Per ensures that the downstream tasks cannot correctly recognize the output of OFENs. A high Per L_Per value indicates that the optical flow is unable to accurately perceive the motion, thereby degrading the performance of subsequent tasks. Flash loss Flash L_Flash leverages prior knowledge to maximize the difference between pixel points in consecutive frames. By maximizing Flash L_Flash, the generated AEs create a significant gap between the pixel values of the current and previous frames, disrupting the optical flow estimation further. 6 Conclusion and Future Work In this work, we have developed a practical real-time infrared light attack method in the physical world. Firstly, to ensure that the AEs are not noticeable by humans, we used infrared lights as the attack tool. Humans are not sensitive to infrared light, making this method highly stealthy and imperceptible. Considering physical attack is quite different from the digital world, instead of adopting simulation experiments, we directly attack the target model using GA which do not require gradient information in the physical environment, ensuring that the generated AEs are robust to unexpected noise. To address the time-consuming nature of query-based methods, we designed a generative network that can produce AEs in real-time. Finally, we established a physical test environment consisting of tracks and controllers to simulate the movement of target object, ensuring that the AEs are tested and validated in a realistic setting. Experiments demonstrate that our methods successfully attack the target system in complex environments, including varying object speeds, different ambient lighting conditions, and varying camera distances. This research shows the infrared lights attack has a great adaptive capacity to disturb the OFEN at different environment, which shows a heavy threaten to the current OFEN-driven security system. For future works, considering that autonomous driving systems rely not only on optical flow estimation but also on other tasks such as object detection, we plan to extend our work to attack models in these additional tasks. This will enable us to achieve a more comprehensive and robust multimodal attack, further enhancing the effectiveness of our approach. By addressing these areas, we aim to improve the efficiency of our adversarial attack methods, making them more applicable to a wide range of real-world scenarios. References [1] S. H. V. Bhupathiraju, T. Sugawara, T. Sato, Q. A. Chen, M. Clifford, and S. Rampazzi (2024) On the vulnerability of traffic light recognition systems to laser illumination attacks. In ISOC Symposium on Vehicle Security and Privacy (VehicleSec). ISOC, San Diego, CA, USA. https://doi. org/10, Vol. 14722. Cited by: §1. [2] T. B. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer (2017) Adversarial patch. arXiv preprint arXiv:1712.09665. Cited by: §1. [3] L. Capito, U. Ozguner, and K. Redmill (2020) Optical flow based visual potential field for autonomous driving. In 2020 IEEE Intelligent Vehicles Symposium (IV), p. 885–891. Cited by: §1. [4] J. Chamorro-Martinez and J. Fernández-Valdivia (2006) A new approach to motion pattern recognition and its application to optical flow estimation. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 37 (1), p. 39–51. Cited by: §1. [5] F. Croce and M. Hein (2019) Sparse and imperceivable adversarial attacks. In Proceedings of the IEEE/CVF international conference on computer vision, p. 4724–4732. Cited by: §1. [6] A. Dosovitskiy, P. Fischer, E. Ilg, P. Hausser, C. Hazirbas, V. Golkov, P. Van Der Smagt, D. Cremers, and T. Brox (2015) Flownet: learning optical flow with convolutional networks. In Proceedings of the IEEE international conference on computer vision, p. 2758–2766. Cited by: §2.1. [7] R. Duan, X. Mao, A. K. Qin, Y. Chen, S. Ye, Y. He, and Y. Yang (2021) Adversarial laser beam: effective physical-world attack to dnns in a blink. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 16062–16071. Cited by: §1. [8] K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, A. Prakash, T. Kohno, and D. Song (2018) Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 1625–1634. Cited by: §1. [9] A. Gnanasambandam, A. M. Sherman, and S. H. Chan (2021) Optical adversarial attack. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 92–101. Cited by: §2.2. [10] D. Guo, Y. Wu, Y. Dai, P. Zhou, X. Lou, and R. Tan (2024) Invisible optical adversarial stripes on traffic sign against autonomous vehicles. In Proceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services, p. 534–546. Cited by: §1, §2.2. [11] M. Haag and H. Nagel (1999) Combination of edge element and optical flow estimates for 3d-model-based vehicle tracking in traffic image sequences. International Journal of Computer Vision 35, p. 295–319. Cited by: §1. [12] Y. Hu and e. al. Kung (2021) Naturalistic physical adversarial patch for object detectors. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 7848–7857. Cited by: §1. [13] M. Lee and Z. Kolter (2019) On physical adversarial patches for object detection. arXiv preprint arXiv:1906.11897. Cited by: §1. [14] A. Liu, X. Liu, J. Fan, Y. Ma, A. Zhang, H. Xie, and D. Tao (2019) Perceptual-sensitive gan for generating adversarial patches. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33, p. 1028–1035. Cited by: §1. [15] N. Narodytska and S. P. Kasiviswanathan (2017) Simple black-box adversarial attacks on deep neural networks.. In CVPR Workshops, Vol. 2. Cited by: §1. [16] A. Ranjan, J. Janai, A. Geiger, and M. J. Black (2019) Attacking optical flow. In Proceedings of the IEEE/CVF international conference on computer vision, p. 2404–2413. Cited by: §2.2. [17] S. Sivaraman and M. M. Trivedi (2013) Looking at vehicles on the road: a survey of vision-based vehicle detection, tracking, and behavior analysis. IEEE transactions on intelligent transportation systems 14 (4), p. 1773–1795. Cited by: §1. [18] K. Song and J. Huang (2001) Fast optical flow estimation and its application to real-time obstacle avoidance. In Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No. 01CH37164), Vol. 3, p. 2891–2896. Cited by: §1. [19] D. Sun, X. Yang, M. Liu, and J. Kautz (2018) Pwc-net: cnns for optical flow using pyramid, warping, and cost volume. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 8934–8943. Cited by: §2.1, §5.1.2, Table 1, Table 2, Table 3. [20] Z. Teed and J. Deng (2020) Raft: recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16, p. 402–419. Cited by: §2.1, §5.1.2, Table 1, Table 2, Table 3. [21] D. Wang, W. Yao, T. Jiang, C. Li, and X. Chen (2023) Rfla: a stealthy reflected light adversarial attack in the physical world. In Proceedings of the IEEE/CVF international conference on computer vision, p. 4455–4465. Cited by: §1. [22] W. Wang, Y. Yao, X. Liu, X. Li, P. Hao, and T. Zhu (2021) I can see the light: attacks on autonomous vehicles using invisible lights. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, p. 1930–1944. Cited by: §1. [23] X. Zheng, Y. Fan, B. Wu, Y. Zhang, J. Wang, and S. Pan (2023) Robust physical-world attacks on face recognition. Pattern Recognition 133, p. 109009. Cited by: §1.