Paper deep dive
Adaptive Pilot Selection for Unified Semantic Communication and Semantic Sensing in ISAC
Muhammad Abubakar Rashid, Muhammad Hannan Akram, Haejoon Jung, Syed Ali Hassan
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/28/2026, 3:50:31 AM
Summary
The paper proposes SemISAC, a unified framework for Integrated Sensing and Communication (ISAC) that combines Semantic Communication (SemCom) and Semantic Sensing (SemSens) within a single dual-function waveform. It utilizes a joint semantic encoder to extract task-specific information for both functions and employs an adaptive pilot selection strategy based on channel conditions to optimize the trade-off between communication and sensing performance. The system is evaluated in a vehicular scenario, demonstrating improved target recognition and range estimation compared to baselines while maintaining segmentation accuracy.
Entities (8)
Relation Signals (7)
SemISAC → implements → Semantic Sensing
confidence 95% · SemISAC, which performs both SemCom and semantic sensing within a single dual-function waveform.
SemISAC → implements → Semantic Communication
confidence 95% · SemISAC, which performs both SemCom and semantic sensing within a single dual-function waveform.
SemISAC → uses → Joint Semantic Encoder
confidence 93% · SemISAC uses a joint semantic encoder that extracts task-specific information for both communication and sensing.
Joint Semantic Encoder → extracts → Task-specific Information
confidence 90% · joint semantic encoder that extracts task-specific information for both communication and sensing.
SemISAC → optimizes → Adaptive Pilot Selection
confidence 90% · The pilot configuration is adaptively optimized based on the channel conditions to balance communication and sensing requirements.
SemISAC → evaluatedin → Vehicular Scenario
confidence 88% · We evaluate SemISAC in a vehicular scenario in which vehicles share pixel-wise segmentation of the road environment
Semantic Communication → partof → ISAC
confidence 85% · Existing studies have applied semantic technology to either the communication module or the sensing module of ISAC.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Semantic communication (SemCom) and integrated sensing and communication (ISAC) are promising technologies for future 6G wireless networks. Existing studies have applied semantic technology to either the communication module or the sensing module of ISAC. In this work, we propose SemISAC, which performs both SemCom and semantic sensing within a single dual-function waveform. SemISAC uses a joint semantic encoder that extracts task-specific information for both communication and sensing. We evaluate SemISAC in a vehicular scenario in which vehicles share pixel-wise segmentation of the road environment and, through sensing, classify surrounding objects and estimate their ranges. On the transmitter side, a deep learning encoder converts the input road-scene image into semantic symbols and places them on the data cells of an OFDM grid, while the remaining cells serve as pilots for channel state information estimation and sensing. The pilot configuration is adaptively optimized based on the channel conditions to balance communication and sensing requirements. At the receiver, a deep learning model reconstructs the segmentation from the received waveform, while the transmitting vehicle captures the reflected waveforms from surrounding objects and uses task-specific deep learning decoders for target recognition and range estimation. Simulation results show that SemISAC achieves a segmentation accuracy close to that of the dedicated SemCom module while outperforming both conventional and semantic baselines in target recognition and range estimation.
Tags
Links
- Source: https://arxiv.org/abs/2609.30891v1
- Canonical: https://arxiv.org/abs/2609.30891v1
Trouble viewing inline? Open PDF directly →
Full Text
72,334 characters extracted from source content.
Expand or collapse full text
Adaptive Pilot Selection for Unified Semantic Communication and Semantic Sensing in ISACThanks: Muhammad Abubakar Rashid, Muhammad Hannan Akram and Syed Ali Hassan are with the School of Electrical Engineering and Computer Science, National University of Sciences and Technology, Islamabad, Pakistan (e-mails: mrashid.bsds23seecs, makram.bsds23seecs, ali.hassan@seecs.edu.pk)Thanks: Haejoon Jung is with the Department of Electronic Engineering, Kyung Hee University, Yongin 17104, Republic of Korea (e-mail: haejoonjung@khu.ac.kr). Muhammad Abubakar Rashid Muhammad Hannan Akram Affiliation: Haejoon Jung, and Syed Ali Hassan, Abstract Semantic communication (SemCom) and integrated sensing and communication (ISAC) are promising technologies for future 6G wireless networks. Existing studies have applied semantic technology to either the communication module or the sensing module of ISAC. In this work, we propose SemISAC, which performs both SemCom and semantic sensing within a single dual-function waveform. SemISAC uses a joint semantic encoder that extracts task-specific information for both communication and sensing. We evaluate SemISAC in a vehicular scenario in which vehicles share pixel-wise segmentation of the road environment and, through sensing, classify surrounding objects and estimate their ranges. On the transmitter side, a deep learning encoder converts the input road-scene image into semantic symbols and places them on the data cells of an OFDM grid, while the remaining cells serve as pilots for channel state information estimation and sensing. The pilot configuration is adaptively optimized based on the channel conditions to balance communication and sensing requirements. At the receiver, a deep learning model reconstructs the segmentation from the received waveform, while the transmitting vehicle captures the reflected waveforms from surrounding objects and uses task-specific deep learning decoders for target recognition and range estimation. Simulation results show that SemISAC achieves a segmentation accuracy close to that of the dedicated SemCom module while outperforming both conventional and semantic baselines in target recognition and range estimation. I Introduction Integrated sensing and communication (ISAC) combines ubiquitous connectivity and advanced sensing capabilities in a unified framework and is critical to autonomous vehicles, smart cities, non-terrestrial networks (NTNs), and industrial automation, thereby emerging as a key enabler for future 6G networks [1], [2]. In addition to ISAC, semantic communication (SemCom) has also attracted significant interest as a fundamental component of 6G networks, as it shifts the design objective from bit transmission to meaning exchange, facilitating the introduction of high-level semantics in wireless networks [3], [4]. When combined, these paradigms enable wireless systems to not only sense and communicate information but also understand and extract its underlying meaning. Systems incorporating SemCom into ISAC systems have attracted significant attention in the recent literature, focusing primarily on conveying semantic features learned from the data using deep learning (DL) techniques. By doing so, and by learning the properties of the wireless channel, SemCom enables robust transmissions and reliable task execution. Moreover, the tradeoff between sensing and communication is also mitigated in SemCom ISAC systems, which maintain downstream task accuracy comparable to that of conventional systems while improving sensing performance. Studies such as [5], [6], [7], [9], [10], and [11] explore the use of SemCom for different purposes in ISAC systems. Reference [5] presents a framework that combines SemCom with multimodal radar and visual sensing in a unified ISAC system. A semantic-based channel state information (CSI) feedback scheme was proposed in [6], which replaces full channel data transmission with lightweight semantic database labels to reduce the feedback burden for autonomous-aerial-vehicle-assisted ISAC tasks. In [7], a framework that jointly optimizes semantic symbol quantization, modulation order, power allocation, and dual-function radar-communication beamforming is proposed for image-based SemCom. Reference [9] proposes a semantic-importance-guided deep reinforcement learning framework for multimodal SemCom in UAV-based ISAC, while [11] incorporates SemCom into UAV-RIS-assisted ISAC by optimizing the transmission of task-relevant semantic symbols under semantic accuracy and secrecy constraints. The work in [10] focuses on security-aware semantic ISAC, utilizing paired adversarial residual networks to protect semantic information from eavesdropping. In addition to the rapid evolution of SemCom, enabling the same level of intelligence in the sensing domain is also an active area of research. The idea of semantic sensing (SemSens) has its theoretical foundations in cognitive radar [12], which established a closed feedback loop between the transmitter and receiver. Subsequently, many studies were conducted to adjust sensing strategies based on environmental knowledge, such as adaptive waveform design and predictive beamforming [16]. In [13], the authors developed a sensing-assisted beamforming technique that employs particle filtering and particle swarm optimization (PSO) and utilizes multipath echoes for vehicle tracking and beamforming optimization. Similarly, an adaptive waveform design method was proposed in [14], which maximized the signal-to-clutter-plus-noise ratio (SCNR) under a constant-modulus constraint and optimized target detection performance through energy allocation. Reference [15] addressed the joint optimization of bandwidth allocation for sensing and data transmission in SemCom systems by applying the projected gradient method to maximize semantic spectrum efficiency. Although sensing strategies have advanced noticeably, the optimization goals have, for the most part, been confined to low-level statistical signal metrics such as normalized mean-squared error (NMSE) and peak signal-to-noise ratio (PSNR). Motivated by the need for high-level semantics in intelligent systems, there has been a gradual shift in the focus of radar sensing from low-level sensing metrics to task-oriented design. A framework leveraging graph structures and temporal features was introduced in [17] to perform semantic segmentation on sparse sequential point clouds from millimeter-wave (mmWave) radar, while Reference [18] proposed a semantic simultaneous localization and mapping (SLAM) algorithm using radar-vision fusion to generate semantic grid maps. Likewise, micro-Doppler signatures have been used to perform human activity recognition (HAR) in various scenarios [19], [20]. Reference [21] introduced the SemSens paradigm based on the information bottleneck principle, in which the authors jointly optimized transmit waveform parameters and receiver representations using a deep learning framework for task-oriented design. SemSens considers a task-oriented representation of the wireless propagation environment by transforming a high-dimensional physical channel observation into a semantic representation [22]. Conventional sensing models explicitly characterize all multipath components through parameters such as delay, Doppler, angle, amplitude, and phase. On the other hand, SemSens retains only those attributes that are relevant to the task. Therefore, the resulting semantic channel can be considered a generalized mapping between the transmitted waveform and the task-relevant representation of the environment, such that irrelevant propagation components are treated as nuisance terms. This semantic extraction forms a many-to-one dimensionality reduction of the physical channel and can eliminate irrelevant channel information while retaining the information required for the sensing task [23]. From an information-theoretic perspective, the data-processing inequality given in [21] implies that the semantic representation cannot contain more information about the underlying environmental state than the original physical observation. However, the discarded information does not compromise task performance when the extracted information is sufficient for the task. Therefore, SemSens shifts the objective from reconstructing the complete physical channel to extracting a task-relevant representation of the environment. Despite advances in SemCom and SemSens, existing studies treat them separately in ISAC systems. Semantic processing of communication information and semantic extraction of sensing information have largely been independent processes, even though both tasks share the same propagation environment, waveform, and limited resources. This separation prevents exploiting their inherent interaction at the semantic level. SemCom discards information irrelevant to communication, while SemSens eliminates information unnecessary for sensing. When both coexist, independently determining task-relevant information leads to redundant processing and fails to exploit semantic information shared across tasks. A unified semantic ISAC framework would instead treat communication and sensing as interdependent semantic tasks, jointly determining task-relevant information. This could establish a common semantic representation of the wireless environment and transmitted information, selectively preserving relevant information while discarding the irrelevant. Since the semantic representations required by communication and sensing may overlap significantly, information extracted for one task can benefit the other, improving overall ISAC efficiency. This motivates a unified framework performing both communication and sensing at the semantic level, rather than incorporating semantic processing into only one functionality. This gap motivates a unified semantic ISAC framework in which SemCom and SemSens are considered jointly. We introduce a semantic ISAC system that simultaneously extracts task-relevant information for both communication and sensing, using a combined encoder for SemCom and SemSens along with adaptive pilot placement and waveform design. We mathematically formulate the joint representation of SemCom and SemSens and propose a pilot selection strategy that facilitates both tasks. Our main contributions are summarized as follows. • We propose SemISAC, a fully semantic approach that unifies SemSens and SemCom and shifts the design objective to task-oriented communication and sensing. Specifically, we formulate the unified SemISAC problem mathematically and establish the joint encoder architecture. In addition, our framework employs joint optimization for the entire pipeline to address the interrelated SemSens and SemCom losses. • We present an adaptive pilot strategy in order to maximize the performance of the SemISAC system. For this purpose, we frame the pilot selection mathematically and adopt a straight-through estimator strategy to facilitate the flow of gradients and train the adaptive pilot selection network. • We instantiate the proposed SemISAC framework and evaluate its performance across multiple communication and sensing tasks. We validate SemCom and SemSens separately with communication and sensing baselines and compare the performance of the proposed approach across different power and subcarrier resources. Fig. 1: Proposed SemISAC system. I Problem Formulation and Design Principle This section formulates the proposed semantic ISAC framework as a joint optimization problem involving SemCom, SemSens, and resource allocation. First, the joint SemCom-SemSens system model is established, where a common semantic representation is designed to support both communication and sensing tasks. The pilot selection problem is then formulated to characterize the tradeoff between sensing performance and the communication resources available in the TF grid, motivating an adaptive allocation strategy. A joint representation learning principle is then developed in which the semantic representation and resource allocation are optimized according to their collective contribution to downstream task performance. Together, these formulations provide the theoretical foundation for the SemISAC system with adaptive pilot selection developed in the subsequent sections. I-A Joint Semantic ISAC Problem We consider an ISAC system designed to jointly support both SemCom and SemSens, rather than being dedicated to a single functionality. The proposed framework consists of a joint semantic encoder, a channel model, and separate decoders for SemCom and SemSens. Let x represent the real-valued source data or observation that needs to be communicated and z denote the environmental state relevant to sensing. The joint encoder is used to generate the complex-valued semantic representation of the source information =fj(),s=f_j(x), (1) where fj(⋅)f_j(·) represents the joint encoder. This representation serves as the common transmitted representation for both communication and sensing and is processed through the relevant channels to produce the observations YcY_c and YsY_s: Yc=Hc(),Ys=Hs(,),Y_c=H_c(s), Y_s=H_s(s,z), (2) where Hc(⋅)H_c(·) and Hs(⋅)H_s(·) represent the communication and sensing channels, respectively. After being passed through the communication decoder fc(⋅)f_c(·) and the sensing decoder fs(⋅)f_s(·), respectively, the task outputs are ^=fc(Yc),^=fs(Ys), x=f_c(Y_c), z=f_s(Y_s), (3) where x and z are the recovered task-relevant SemCom and SemSens information, respectively. A key distinction of this framework is that neither task requires the reconstruction of the complete transmitted waveform or channel. Instead, each decoder is optimized to extract the semantic information relevant to its task. Consequently, the fundamental objective is to design a unified transmitted representation that simultaneously preserves the semantic information required for communication and the task-oriented physical information required for sensing. This formulation highlights the central challenge of semantic ISAC: learning a common representation that effectively captures and balances the information requirements of both SemCom and SemSens while avoiding unnecessary transmission of task-irrelevant information. I-B Pilot Selection In ISAC systems, sensing relies on reference signals, commonly referred to as pilot signals, whose known waveforms and transmission parameters allow the receiver to estimate the wireless channel. In SemSens, the role of pilot signals is even more important because the amount of information extracted from the received signal is deliberately constrained by the semantic objective. Therefore, pilot placement must be designed to provide the SemSens decoder with the information most relevant to the task, while suppressing information arising from irrelevant channel variation and noise. Since the total number of resource elements (REs) in a single waveform is limited, allocating more pilots for sensing necessarily reduces the REs available for communication, creating a resource-allocation problem. Let Pi∈0,1P_i∈\0,1\ denote the allocation of a resource element indexed by resource unit i, where Pi=1, allocated for sensing,0, allocated for communication,P_i= cases1,& allocated for sensing,\\ 0,& allocated for communication, cases (4) Accordingly, the total number of resources allocated to sensing can be expressed as Np()=∑iPi,N_p(P)= _iP_i, (5) while the remaining resources are available for communication Nd()=∑i(1−Pi).N_d(P)= _i(1-P_i). (6) Thus the pilot placement matrix P directly determines the division of available time frequency resources between sensing and communication. To capture the fact that the ultimate objective is task performance rather than conventional physical layer accuracy, we define task oriented quality measures for communication and sensing. In particular, let Qc()=−ℒc(),Q_c(P)=-L_c(P), (7) where ℒcL_c denotes the communication related semantic loss such as the error associated with semantic reconstruction or decoding. Similarly, Qs()=−ℒs(),Q_s(P)=-L_s(P), (8) where ℒsL_s represents the sensing loss which may quantify errors in sensing tasks such as object classification, localization or range estimation. Defining the quality measures in terms of downstream task losses allows the value of a pilot resource to be evaluated according to the information it provides for the intended SemSens and SemCom tasks. The optimal pilot allocation can therefore be formulated abstractly as ⋆=argmax(Qc(),Qs()),P = _PU (Q_c(P),Q_s(P) ), (9) subject to the available resource and power constraints. A simple realization of the utility function is a weighted combination of the two task oriented quality measures, =αQc+(1−α)Qs,0≤α≤1,U=α Q_c+(1-α)Q_s, 0≤α≤ 1, (10) where α determines the relative importance of SemCom and sensing. More generally, (⋅)U(·) can be selected to represent application specific priorities or quality of service requirements. The above formulation highlights an important property of the pilot placement problem that the optimal allocation is not necessarily fixed across different channel realizations or task requirements. The usefulness of a particular resource element for sensing depends on the underlying channel and propagation conditions, while its usefulness for SemCom depends on the information required by the downstream semantic task. Consequently, the optimal allocation can be expressed as a function of the current system state, ⋆=⋆(,)P =P (x,z) (11) Therefore, a fixed pilot pattern designed independently of the instantaneous channel and task conditions cannot, in general, achieve the optimal trade off between sensing and SemCom. Instead, the pilot configuration should adapt to the current system state so that sensing resources are concentrated where they provide the greatest task oriented benefit, while unnecessary pilot resources are avoided. This state dependence can be made explicit by considering the effect of the instantaneous SNR on the sensing-communication resource tradeoff. Assumption 1 (Noise limited pilot count model). In the noise-limited regime, the sensing distortion decreases proportionally with the effective pilot SNR resource NpγN_pγ, while the communication distortion decreases proportionally with the rate log(1+γ) (1+γ) available to the remaining Nc−NpN_c-N_p symbols. Specifically, there exist constants a,c>0a,c>0, independent of NpN_p and γ, such that Ds(Np,γ)=aNpγ,Dc(Np,γ)=c(Nc−Np)log(1+γ).D_s(N_p,γ)= aN_pγ, D_c(N_p,γ)= c(N_c-N_p) (1+γ). (12) Under Assumption 1, the sensing distortion decreases proportionally to 1/γ1/γ while the communication distortion decreases more slowly, proportionally to 1/log(1+γ)1/ (1+γ). Consequently, the marginal benefit of allocating an additional resource element to sensing changes with SNR, and so does the optimal sensing resource count. Proposition 1 (Optimal pilot count decreases with SNR). Under Assumption 1, for any w∈(0,1)w∈(0,1) the weighted distortion J(Np)=wDs(Np,γ)+(1−w)Dc(Np,γ)J(N_p)=wD_s(N_p,γ)+(1-w)D_c(N_p,γ) has a unique minimizer Np⋆(γ)=Ncwa/γwa/γ+(1−w)c/log(1+γ),N_p (γ)=N_c\, wa/γ wa/γ+ (1-w)c/ (1+γ), (13) which is strictly decreasing in γ, with Np⋆(γ)→0N_p (γ)→ 0 as γ→∞γ→∞. The proof is given in Appendix B. This result provides a theoretical justification for making pilot allocation SNR-aware. As the channel becomes less noise-limited, fewer pilot resources are required to meet the sensing objective, leaving more resource elements available for SemCom. However, directly solving the above binary optimization problem for every channel realization can be computationally expensive, particularly when the number of available TF resources is large. The monotonic SNR dependence established by Proposition 1 further suggests that the pilot allocation policy should explicitly account for the operating SNR rather than relying on a fixed pilot count. This motivates a learning based approach in which a neural network learns an adaptive pilot allocation policy. Specifically, the allocation can be represented as =f(,),P=f_ θ(x,z), (14) where f(⋅)f_ θ(·) denotes the proposed neural network based allocation function parameterized by θ. The network therefore learns to map the observed system state to an appropriate pilot configuration, f(,)≈⋆(,).f_ θ(x,z) (x,z). (15) In this way, the neural network does not replace the underlying resource-allocation objective rather it provides a computationally efficient approximation to the optimal adaptive policy. The resulting pilot placement mechanism can dynamically adjust the sensing resources according to channel conditions and the relative requirements of the sensing and SemCom tasks, thereby providing the theoretical basis for the proposed adaptive pilot selection framework. I-C Joint Representation Principle The preceding formulations establish that the transmitted signal must preserve the semantic information required for communication while simultaneously preserving sufficient task-relevant information for sensing. These objectives are coupled through the finite TF resources available for transmission. Consequently, the central design problem is not simply to construct an encoder for each task independently, but to learn a unified representation that jointly serves the heterogeneous requirements of SemCom and SemSens. Let s denote the latent semantic representation extracted from the source data, as in (1), and let P be a matrix that denotes the sensing-resource allocation pattern over the available TF resource elements as before. The resulting transmitted TF representation X can be abstractly written as =(,),X=A(s,P), (16) where (⋅)A(·) represents the mechanism that maps the semantic representation and the selected sensing resources onto the transmitted TF grid. Here, s captures the information that should be conveyed through the waveform, whereas P determines where explicit sensing resources are introduced. The joint representation principle can therefore be formulated as a multi-objective optimization problem. Specifically, the parameters of the semantic encoder and task-specific decoders, together with the sensing-resource allocation, are jointly optimized according to min,ϕ,,λcℒc()+λsℒs()+λrℛ(), _ θ, φ, ψ,P _cL_c(P)+ _sL_s(P)+ _rR(P), (17) where θ, ϕ φ, and ψ denote the parameters associated with the joint semantic encoder, SemCom decoder, and SemSens decoder, respectively. The terms ℒcL_c and ℒsL_s denote the SemCom and SemSens losses, while ℛ()R(P) represents the cost associated with allocating sensing resources. The coefficients λc _c, λs _s, and λr _r control the relative importance of communication performance, sensing performance, and resource efficiency. This formulation establishes the fundamental tradeoff underlying the proposed semantic ISAC framework. A representation optimized solely for ℒcL_c may discard physical information that is useful for sensing, whereas a representation optimized solely for ℒsL_s may allocate excessive resources to sensing or preserve information that is unnecessary for communication. Joint optimization instead encourages the learned representation to retain information that is simultaneously valuable to both tasks while avoiding task-irrelevant information and unnecessary sensing overhead. The proposed architecture can then be interpreted as one realization of the optimization problem in (17). The joint semantic encoder generates the common representation s from the source information, as in (1), while the adaptive pilot selection mechanism determines the sensing resource configuration as in (14). These components jointly determine the transmitted TF representation =(fj(),f(,))=(,).X=A (f_j(x),f_ θ(x,z) )=A (s,P ). (18) After propagation through the communication and sensing channels, task-specific decoders extract the required semantic information from the received observations. Thus, the proposed framework constitutes a joint representation learning problem in which the semantic representation and sensing-resource allocation are optimized according to their collective contribution to downstream task performance. The TF waveform consequently becomes a shared information-bearing structure. The data REs convey semantic information for communication, while its sensing REs provide additional physical information required for environmental inference. Consequently, the optimal representation is inherently task- and resource-dependent. In general, there is no universally optimal semantic representation or fixed pilot pattern that is simultaneously optimal for all communication and sensing conditions. The desired representation depends on the source information, environmental state, channel conditions, and relative importance assigned to the two tasks. The joint objective in (17) therefore provides a general mathematical principle for learning representations that adapt to these requirements. Accordingly, the key principle of the proposed semantic ISAC framework is that the same transmitted TF representation should be jointly optimized to preserve communication-relevant semantics and sensing-relevant physical information, subject to the finite sensing and communication resources available in the waveform. The neural architecture and adaptive pilot selection mechanism introduced in the subsequent sections provide a practical realization of this principle. Fig. 2: System model. I System Model In this study, we consider the vehicular ISAC scenario as illustrated in Figure 2. The scenario uses an autonomous car equipped with an ISAC transceiver, driving through an urban road environment. The car perceives the surrounding road scene and, to support cooperative and safe driving, it must both (i) share this perception with the nearby neighboring vehicles and (i) remain aware of the potential obstacles in its vicinity, such as pedestrians, cars, and drones. To perform these functions, the vehicle emits a single dual-function OFDM waveform. The waveform is structured as a time-frequency (TF) resource grid, and its cells are divided into two categories: pilot cells and data cells. This OFDM waveform simultaneously performs two tasks: 1. SemCom: it transmits a semantic description of the road scene to the neighboring vehicles, which recover pixel-wise segmentation of the image rather than the original image. 2. SemSens: the backscattered signal of the transmitted waveform is processed at the transmitting vehicle to classify the object (pedestrian, car, or drone) and to estimate its distance. I-A Dual-Function OFDM Frame and Power Allocation I-A1 Time–frequency grid The dual-function waveform is an OFDM frame whose resources are naturally organized on a two-dimensional time–frequency (TF) grid. The frame consists of M orthogonal subcarriers along the frequency axis and N OFDM symbols along the slow-time axis, giving a total of Nc=N×MN_c=N× M (19) resource cells. Each cell, indexed by (n,m)(n,m) with n∈0,…,N−1n∈\0,…,N-1\ and m∈0,…,M−1m∈\0,…,M-1\, carries a single complex symbol on the m-th subcarrier of the n-th OFDM symbol. The subcarriers are separated by Δf=B/M f=B/M, where B is the total occupied bandwidth and the OFDM symbols are transmitted at a slow-time interval T. The simulation operates at carrier frequency fcf_c with wavelength λ=c/fcλ=c/f_c, where c is the speed of light. The numerical values of these parameters used in the study are listed in Table I. The two axes of the time-frequency grid encode complementary parameters that can be utilized by the sensing receiver to classify the target. Consider a single reflecting path with round-trip delay τ and Doppler shift ν. The delay τ produces a phase that linearly changes along the subcarriers (frequency axis), e−j2πmΔfτe^-j2π m\, f\,τ. Similarly, the Doppler shift ν produces a phase that varies linearly along the slow-time axis ej2πνnTe^\,j2πν\,nT. The target distance R=cτ/2R=cτ/2 is estimated along the frequency axis, whereas its velocity v=λν/2v=λν/2 and the micro-Doppler modulation of its moving parts (if any) are estimated along the slow-time axis n. This delay–Doppler separation allows the pilot cells to be utilized for sensing while the data cells convey the semantic data. I-A2 Pilot–data partitioning We introduce deep-learning-based pilot selection using a binary pilot-placement mask ∈0,1N×MP∈\0,1\^N× M. The mask is not fixed; instead, a deep learning encoder assigns a selection score to each cell and then compares the score against a threshold. Cells whose scores exceed the threshold are selected as pilot cells []n,m=1[P]_n,m=1, and the rest function as data cells []n,m=0[P]_n,m=0. Importantly, the threshold is a function of SNR, so the resulting number of pilots NpN_p adapts to the channel conditions. I-A3 Power allocation and transmit grid The transmitting vehicle operates under a fixed total transmit power PtP_t per OFDM frame, which is shared between the pilot cells and data cells through a single power-allocation coefficient ρ∈(0,1)ρ∈(0,1). This ρ is the fraction of the total power assigned to the pilots. The resulting per-cell transmit energy is given by Ep=ρPtNp,Ed=(1−ρ)PtNd,E_p= ρ\,P_tN_p, E_d= (1-ρ)\,P_tN_d, (20) such that the pilot cells altogether receive a power budget of EpNp=ρPtE_pN_p=ρ P_t and the data cells EdNd=(1−ρ)PtE_dN_d=(1-ρ)P_t. Consequently, the total power budget constraint EpNp+EdNd=PtE_pN_p+E_dN_d=P_t (21) holds for any value of ρ and any number of pilot cells NpN_p. The coefficient ρ therefore controls the communication-sensing power tradeoff in the system, where a larger value of ρ allocates more power to the pilot cells, improving sensing and channel estimation at the cost of weaker semantic data symbols, and vice versa. Under the noise-limited regime, this tradeoff admits a closed-form characterization. Assumption 2 (Noise-limited power split model). In the noise-limited regime, the sensing and communication task distortions are inversely proportional to their respective allocated energies. Specifically, there exist constants a,b>0a,b>0, independent of ρ, such that Ds(ρ)=aρ,Dc(ρ)=b1−ρ,0<ρ<1.D_s(ρ)= aρ, D_c(ρ)= b1-ρ, 0<ρ<1. (22) Remark 1. Assumption 2 follows from (20). The pilot estimation error variance scales as σ2/(EpNp)=σ2/(ρPt)σ^2/(E_pN_p)=σ^2/(ρ P_t), giving a=κsσ2/Pta= _sσ^2/P_t. Similarly, the per-symbol data distortion scales as σ2/Ed=Ndσ2/((1−ρ)Pt)σ^2/E_d=N_dσ^2/((1-ρ)P_t), giving b=κcNdσ2/Ptb= _cN_dσ^2/P_t, with task-dependent κs,κc>0 _s, _c>0. Lemma 1 (Unique Pareto-optimal power split). Under Assumption 2, let w∈(0,1)w∈(0,1) denote the priority assigned to the sensing task. The weighted distortion J(ρ)=wDs(ρ)+(1−w)Dc(ρ)J(ρ)=wD_s(ρ)+(1-w)D_c(ρ) is strictly convex on (0,1)(0,1) and attains a unique minimizer ρ⋆(w)=wawa+(1−w)b.ρ (w)= wa wa+ (1-w)b. (23) Moreover, ρ⋆ρ is strictly increasing in w, with ρ⋆→0ρ → 0 as w→0w→ 0 and ρ⋆→1ρ → 1 as w→1w→ 1. The proof is given in Appendix A. A larger value of ρ therefore allocates more power to the pilot cells, improving sensing and channel estimation at the cost of weaker semantic data symbols, and vice versa. Combining the pilot placement P with the power allocation budget, the transmitted grid ∈ℂN×MX ^N× M is structured as follows: =Epsp⏟pilots (sensing)+Ed(−)⊙⏟data (communication),X= E_p\,s_p\,P_pilots (sensing)+ E_d\,(1-P) _data (communication), (24) where sps_p represents the known pilot symbol with unit magnitude (|sp|=1|s_p|=1), and 1 is the all-ones matrix. ∈ℂN×MS ^N× M is the grid of SemCom symbols produced by the deep learning encoder. I-B Semantic Communication Model To convey the road scene efficiently, we use SemCom instead of sharing raw images. The transmitting vehicle encodes only the semantic description required by the neighboring vehicle to reproduce a pixel-wise segmentation of the scene. I-B1 Semantic symbol generation The transmitting vehicle perceives the roadside environment through an image ∈ℝ3×H×Wx ^3× H× W. A Swin-Transformer is used as a semantic encoder to convert the image into semantic symbols, which are complex numbers carrying the task-relevant semantics of the scene. =ℰ()∈ℂN×M.S=E_ θ(x) ^N× M. (25) Because the encoder’s output has a nonuniform magnitude, it is rescaled to unit average power, 1Nc∑n,m|[]n,m|2=1 1N_c _n,m|[S]_n,m|^2=1. After this normalization, the average power of each data cell equals EdE_d, such that data-cell transmit power is determined solely by the coefficient EdE_d rather than by the image-dependent symbol magnitudes. I-B2 Communication channel These semantic symbols are then transmitted to the neighboring vehicle, which receives the data from a direct (line-of-sight, LOS) path and several reflected (non-line-of-sight, NLOS) paths that bounce off the surrounding objects. Their combination is modeled as a frequency-selective Rician channel. c=K+1LOS+1K+1NLOSH^c= KK+1\,H^LOS+ 1K+1\,H^NLOS (26) where the Rician factor K is used to control power in the direct path relative to the reflected ones. The direct path travels straight from the transmitting vehicle to the neighboring vehicle without reflecting off anything [LOS]n,m=ejϕ0ej2πν0nT,[H^LOS]_n,m=e^\,j _0\,e^\,j2π _0nT, (27) where ejϕ0e^\,j _0 is a constant random phase and ej2πν0nTe^\,j2π _0nT is a slow phase rotation along the slow-time axis caused by the bulk-Doppler shift ν0 _0 from the relative motion of two vehicles. The reflected signals are collected and summed before reaching the neighboring vehicle n,m _n,m =∑l=1Lcale−j2πmΔfτlej2πνlnT, = _l=1^L_ca_l\,e^-j2π m\, f\, _l\,e^\,j2π _lnT, (28) al∼(0, 1/Lc). a_l (0,\,1/L_c). Each path l has a complex gain ala_l (a random amplitude and phase), a delay τl _l that produces a linear phase change across subcarriers, and a Doppler νl _l that produces a phase variation across the slow-time axis. Overall, a large K means that the direct path dominates and the channel is smooth and clean, whereas a small K means that the reflected signals dominate. I-B3 Semantic decoding The signal received by the neighboring vehicle is c=c⊙+c,Y^c=H^c +N^c, (29) where the entries of cN^c are independent and identically distributed additive white Gaussian noise samples. Since the pilot and data cells experience the same channel within the grid, channel state information (CSI) can be estimated from the pilot cells. The received signal c⊙(−)Y^c (1-P), along with the channel estimate ^c H^c, is passed to a semantic decoder ϕ(⋅)D_ φ(·) which recovers the segmentation of the road scene. ^=ϕ(c,^c,)∈1,…,OsH×W. m=D_ φ (Y^c, H^c,P )∈\1,…,O_s\^H× W. (30) Here, OsO_s is the number of semantic classes on which the model is trained, and each pixel of m is assigned one class label. This segmentation can be used by the vehicle to interpret the surrounding obstacles from another perspective. I-C Semantic Sensing Model For the sensing part of semantic ISAC, the transmitting vehicle processes the echo of its own waveform to identify surrounding objects and measure their distances. I-C1 Sensing channel The transmitted OFDM grid reflects off moving targets and returns to the transmitting vehicle. The target obstacle lies at a range of R and moves with radial velocity v. Instead of acting as a single point, the target reflects the signal from L different points on its body. The target as a whole moves at a certain velocity; however, its own parts move independently, like a person’s swinging arms and legs or a drone’s spinning blades. This extra motion of the parts adds a small time-varying pattern to the reflected signal called micro-Doppler. The sensing channel is [s]n,m=∑l=1Lαle−j2πmΔfτej2πνnTejβlsin(2πfμ,lnT+φl),[H^s]_n,m= _l=1^L _l\,e^-j2π m\, f\,τ\,e^\,j2πν\,nT\,e^\,j _l (2π f_μ,l\,nT+ _l), (31) Each of the exponential terms plays a different role. The delay term e−j2πmΔfτe^-j2π m\, f\,τ which changes linearly across the subcarriers, encodes the range of the target, ej2πνnTe^\,j2πν\,nT changes linearly across the slow-time axis and shows the velocity of the target, and ejβlsin(2πfμ,lnT+φl)e^\,j _l (2π f_μ,l\,nT+ _l) gives the micro-Doppler along the slow-time axis. τ=2R/cτ=2R/c is the round-trip delay and ν=2v/λν=2v/λ is the bulk Doppler caused by the velocity v of the target. These two are the same for all L points. The term αl∼(0,1/L) _l (0,1/L) is a complex gain that indicates the reflection strength of point l, while βl _l, fμ,lf_μ,l, and φl _l give the strength, frequency, and phase of its micro-Doppler, respectively. TABLE I: Simulation and model parameters. Symbol Parameter Value Grid & Channel N OFDM symbols 32 M Subcarriers 64 NcN_c Total cells 2048 B Bandwidth 30 MHz Δf f Subcarrier spacing 468.75 kHz fcf_c Carrier frequency 3 GHz λ Wavelength 0.1 m T Repetition interval 1 ms L Paths/target 3 K Rician factor 6 dB LcL_c Comm. reflected paths 3 OsO_s Segmentation classes 11 H×WH× W Image resolution 256×256256× 256 Optimization ξ Learning rate 10−310^-3 NbN_b Batch size 16 NeN_e Epochs 40 κ Gradient clip 1.0 γtr _ tr Training SNR [−6,20][-6,20] dB γev _ ev Evaluation SNR [−9,20][-9,20] dB Architecture dfd_f Feature dimension 8192 Q Range bins 256 NrN_r Ranging projections 4 ω Gate temperature 0.1 Objective γcls _ cls Recognition penalty 0.10 γrng _ rng Ranging penalty 0.05 σR _R Range normalization 57.7 m sis_i Task loss weights Learned Sensing / Signal RmaxR_ Max. range 200 m β Micro-Doppler depth 2.0 rad NtN_t Targets/frame 1 NpN_p Pilot budget 256 sps_p Pilot symbol 1 Ep,EdE_p,E_d Pilot/data energy 1 ρ Power split (evaluation) [0.05,0.95][0.05,0.95] I-C2 Echo processing and target recognition The reflected signal is received by the transmitting vehicle s=s⊙+s.Y^s=H^s +N^s. (32) Only pilot signals are utilized for SemSens and data cells are discarded using the mask P. The receiver measures how the target changed the pilots by dividing the received pilot signals by the original ones. ^s=(s⊘(Epsp))⊙. H^s= (Y^s ( E_p\,s_p) ) . (33) Conventional radars reconstruct the target’s description; however, we perform task-oriented SemSens, where two task-oriented deep learning decoders map the pilot observation directly to the information needed by the task c^=cls(^s),R^=rng(^s), c=G_cls( H^s), R=G_rng( H^s), (34) where c^∈pedestrian,car,drone c∈\pedestrian,car,drone\ is the category of the target object and R R is the distance from the receiver to that object. The semantic decoders only capture the meaning of the reflection, such as “a pedestrian 3030 m ahead,” rather than reconstructing a full physical description of it. This makes the sensing more efficient as no effort is spent on recovering unnecessary details that the task does not need. IV Proposed Method In our proposed approach, the following components are jointly learned: the semantic encoder ℰE_ θ that produces semantic data symbols, the pilot-selection mask that decides which cells will act as pilot cells, the communication decoder ϕD_ φ that reconstructs the segmentation of the road environment and the two sensing decoders clsG_cls and rngG_rng that classify the target and estimate its range. IV-A Training and Evaluation Dataset These road scenes are taken from the Cambridge-driving Labeled Video Database (CamVid) [24]. It consists of images captured from the viewpoint of a driving vehicle, as illustrated in Fig. 5(a). Each image has annotated labels, which provide a pixel-wise segmentation of different objects present in the image, as illustrated in Fig. 5(b). Following common practice, we use the Os=11O_s=11 semantic classes (such as road, building, car, pedestrian, sign, and pavement), with the remaining pixels marked as unlabeled. These images act as the source x for the semantic encoder, which converts them into semantic symbols and whose segmentation is reconstructed by the neighboring vehicles. IV-B Semantic Encoder and Task Decoders A Swin-Transformer (ST) [8] is used as a semantic encoder that extracts semantic features from the road image. For efficient data processing, we use the Swin-Tiny variant with a patch size of 44 and a window size of 77, initialized with weights pretrained on ImageNet. Only the head of the ST learns and changes its weights during the training procedure, and the backbone weights are frozen. The ST processes the image hierarchically: it first divides the 256×256256× 256 image into non-overlapping 4×44× 4 patches and then applies several stages of self-attention and patch-merging layers to gradually reduce the spatial size while increasing the number of feature channels. This produces an 8×88× 8 feature map with 768768 channels. Unlike global attention, the ST computes self-attention only within local windows, which maintains efficiency. Inside a window, it has query, key, and value matrices ,, Q, K, V, and the attention is computed as Attn(,,)=softmax(⊤d+),Attn( Q, K, V)=softmax\! ( Q K d+ B ) V, (35) where d is the dimension of the features and B is the learned bias. At the receiving vehicle, the semantic decoder ϕD_ φ recovers the segmentation of the road. Unlike the semantic encoder, the decoder uses a convolutional encoder–decoder neural network trained from scratch for this task. It is given the received data cells and the channel estimate calculated using the pilot cells. The first convolutional layers compress the received input; then, a sequence of upsampling layers expands it into a segmentation map. In this way, the decoder learns to decode the road segmentation. At the transmitting vehicle, SemSens is carried out using pilot cells only. The decoder clsG_cls receives the pilot observation ^s H^s with its real and imaginary parts and processes it with a convolutional neural network to classify the target. The information it relies upon is the micro-Doppler pattern created by the movement of different parts of the target, such as a pedestrian’s limbs or a drone’s rotating blades. Ranging works in a different way, as each distance imprints a different pattern on the received pilots. We construct a dictionary of Q candidate distances RqR_q and their corresponding patterns beforehand and insert them into the ranging decoder: Dm,q=e−j2πmΔfτq,τq=2Rqc,D_m,q=e^-j2π\,m\, f\, _q, _q= 2R_qc, (36) where τq _q is the delay of the echo coming back from that distance. The decoder compares the received pilots against every pattern in the map and gives each one a match score zqz_q. The closer the match, the higher the score and the more likely it is that the target is at that distance. A small neural network then turns these scores into a single continuous estimate R^=∑q=1Qexp(zq)∑q′exp(zq′)Rq. R= _q=1^Q (z_q) _q (z_q )\,R_q. (37) Only this small network and a short input layer are trained; the distance mappings remain fixed. This method gives accurate distance estimates while needing less data for training. (a) Sensing classification accuracy (b) Sensing classification F1 (c) Sensing ranging MAE Fig. 3: Performance of the sensing module of the SemISAC framework compared with various classification and ranging baselines. IV-C Adaptive Pilot Selection In an OFDM grid, each cell is used as a pilot when its selection score is above a threshold. These scores remain fixed across images; however, the threshold varies. A small network ηφ(γ) _ (γ) considers the current signal-to-noise ratio γ and determines the threshold, so a cell becomes a pilot only when n,m=1,gn,m>ηφ(γ),0,otherwise.P_n,m= cases1,&g_n,m> _ (γ),\\ 0,&otherwise. cases (38) When the channel is poor, the threshold is lowered by the neural network, so more cells are selected as pilot cells and the target is measured more reliably. In contrast, when the channel is good the threshold rises and fewer cells are selected as pilots. Because this is a hard yes-or-no choice, it does not allow any gradients to flow during backpropagation in neural networks. To overcome this, we adopt a straight-through estimator strategy. During the backward pass of training this binary decision is treated as a sigmoid of the difference between the score and the threshold. ~n,m=σ(gn,m−ηφ(γ)ω),σ(u)=11+e−u, P_n,m=σ\! ( g_n,m- _ (γ)ω ), σ(u)= 11+e^-u, (39) This smooth function provides a non-zero gradient, allowing the neural network to learn. In this way, the model can be trained using continuous gradients while the actual transmitted signal still uses exact a binary pilot selection strategy. IV-D Training Objective The communication and sensing pipelines are jointly optimized with respect to three quantities: communication-task accuracy, sensing-task accuracy, and the number of pilots used. The communication quality is measured by the pixel-wise cross-entropy ℒcommL_comm between the predicted and the true road segmentation. The sensing quality is the classification cross-entropy ℒclsL_cls for target recognition; range estimation uses a normalized squared error ℒrng=(R^−RσR)2,σR=Rmax12,L_rng= ( R-R _R )^2, _R= R_ 12, (40) where σR _R is the standard deviation of a range distributed uniformly over [0,Rmax][0,R_ ]. Since the communication and sensing losses differ in scale and behavior, combining them under a single loss with their original scale values would result in the improvement of only one pipeline. Therefore, a parameter sis_i learned by the neural network is used to adjust the weight according to the task. With this, the two losses of a pipeline are combined as ∑i(12e−siℒi+12si) _i ( 12e^-s_iL_i+ 12s_i ). Finally, to stop the network from using more pilots than it needs, we add a cost proportional to the fraction of pilot cells, Np/NcN_p/N_c, so that extra pilots are only kept when they are worth it. Combining these terms, the recognition and ranging pipelines minimize ℒclass _class =∑i∈comm,cls(12e−siℒi+12si)+γclsNpNc, = _i∈\comm,cls\ ( 12e^-s_iL_i+ 12s_i )+ _cls\, N_pN_c, (41) ℒrange _range =∑i∈comm,rng(12e−siℒi+12si)+γrngNpNc. = _i∈\comm,rng\ ( 12e^-s_iL_i+ 12s_i )+ _rng\, N_pN_c. (42) The complete system is trained end-to-end, and the two communication and sensing pipelines are optimized in the same loop, each with its own Adam optimizer and gradient-norm clipping. In each mini-batch of training images, the SNR is drawn uniformly and a new channel is generated, so that the trained model learns to adjust its pilot count across diverse operating conditions rather than at one fixed condition. Throughout the training, the power-allocation coefficient ρ is kept constant and its effect is analyzed separately by changing it at evaluation time. (a) Performance comparison of SemISAC with baselines. (b) Effect of changing the number of subcarriers. Fig. 4: Evaluation of semantic communication in terms of segmentation accuracy. V Simulation Setup & Results We evaluate the proposed approach integrating SemSens and SemCom using different metrics, different signal-to-noise ratios (SNRs), and a range of channels. The time–frequency grid, channel, and model parameters used in the simulations are listed in Table I. Performance is measured with different metrics for the communication and sensing branches. For the communication task, the recovered segmentation is measured by pixel accuracy, mean intersection-over-union (mIoU), and macro-averaged F1 score over the OsO_s road classes. For the sensing task, target recognition is measured by classification accuracy and macro-F1, while range estimation is measured by the mean absolute error (MAE) in meters. The three target classes—pedestrian, car, and drone—and their bulk-velocity and micro-Doppler parameters used in the simulation are summarized in Table I. We compare our proposed approach with several baseline configurations that use different combinations of conventional and semantic modules. To ensure a fair comparison, all baseline methods are reproduced in the same simulation environment and evaluated using identical channel conditions, waveform parameters, target distributions, and SNR levels. TABLE I: Target objects. Class Bulk velocity v (m/s) Micro-Doppler fμf_μ (Hz) Pedestrian 00–22 11–33 Car 1010–2020 00 (rigid) Drone 55–1010 8080–120120 We consider multiple scenarios with different semantic capabilities. We first build a basic ISAC system that serves as the base structure of the framework; the system is then modified by adding semantic modules. Scenario 2 (S2) replaces the conventional communication module of ISAC with SemCom while sensing remains conventional; this scenario is built on the basis of [7]. Scenario 3 (S3) instead replaces conventional sensing with a SemSens module reproduced from the study in [21]. Finally, Scenario 4 (S4) is our proposed system, which integrates both a SemCom module and a SemSens module. This setup lets us analyze the benefit contributed by each semantic module individually—S2 for SemCom and S3 for SemSens—and the importance of combining them into a single unified design. V-A Semantic Communication Based ISAC (S2) S2 adopts the SemCom design of [7], in which a deep joint source–channel coding (DJSC) [25] network jointly performs image compression and channel coding. The semantic symbols are quantized into bits and then mapped to a conventional modulation scheme, such as PSK or QAM. The symbols are also assigned different quantization levels, modulation orders, and transmit powers according to their semantic importance. At the receiver, the DJSC decoder maps the received symbols to the pixel-wise segmentation of the scene, without reconstructing the original image. V-B Semantic Sensing (S3) S3 uses the sensing module design from [21]. It uses a learned pilot-selection mechanism with task-specific sensing decoders. It also uses a dictionary-based ranging head trained end-to-end for the sensing tasks that estimates the delay of the target. To evaluate the sensing task thoroughly, we reproduce the sensing baselines used in this paper. For target recognition: • Full Channel Reconstruction (Perfect CSI): an ideal reference that uses the same backbone neural network as the proposed method but is provided with perfect channel state information during decoding. • DL-MSE: a two-stage scheme that first reconstructs the channel by minimizing the mean-squared error and then classifies the target from the signal reconstruction. • Uniform SVM & Optimized SVM: classical radar-processing pipelines in which handcrafted statistical features of the Doppler spectrum (mean, standard deviation, skewness, and kurtosis) are first extracted and then classified by an SVM model. Uniform SVM uses evenly placed pilots across the grid, and optimized SVM uses a pilot placement optimized for the Doppler pattern. (a) Image (b) Ground truth segmentation (c) SemISAC (d) Simple semantic communication (e) S2 Fig. 5: An instance from the dataset, its ground truth segmentation, and the performance of all approaches. For range estimation: • OMP: orthogonal matching pursuit, which uses a list of possible delays (each delay corresponds to a distance). It goes through that list and picks the delay that best matches the received echo, thereby estimating the target distance. • UP-MUSIC, RP-MUSIC, and OP-MUSIC: three different versions of the multiple signal classification (MUSIC) method, which split the received pilots into a target part and a noise part, scan every possible delay, and retain the delays that match the target part to estimate the target distance. • Blind: a reference baseline that uses no channel state information and predicts the mean range blindly. V-C Performance Evaluation Figure 3(a) illustrates the performance of our proposed methodology in terms of the sensing accuracy of object classification. In addition to the raw accuracy score, we also compare the F1 score in Figure 3(b) for classification to provide a more comprehensive evaluation, as F1 considers both precision and recall. We can observe that the SemSens module of the proposed SemISAC methodology outperforms nearly all other methods across the board, including the simple SemSens methodology and DL-based channel reconstruction. The improved performance stems from the adaptive pilot design, which allows the proposed methodology to identify and favor pilot positions that provide the most informative observations for downstream tasks at test time, allowing the pilot resources to be allocated according to their sensing utility instead of under a fixed scheme, such as a uniform or heuristic allocation. The observed performance improvement over S3 and the DL-based method suggests that adaptive pilot placement is able to extract task-relevant information more effectively than fixed pilot configurations. Additionally, the conventional feature-based methods involving handcrafted features show the most limited performance. Compared to the adaptive learning methods, this underperformance can be attributed to the limitations of the handcrafted features, as they lack the ability to identify and utilize more informative features. The enhanced performance of the proposed methodology and the general trends are visible in both the accuracy and F1 graphs, further substantiating the results. The sensing performance of the proposed methodology is also evaluated in terms of the mean absolute error (MAE) achieved in order to gauge the performance of the proposed method in continuous parameter estimation scenarios. As shown in Figure 3(c), RP-MUSIC and OP-MUSIC exhibit the least error among all the MUSIC variants, whereas S3 consistently shows the largest error. The proposed method performs optimally in this metric as well, recording an MAE of less than 10 m even at an SNR of −9-9 dB. The performance gap between adaptive and non-adaptive pilot designs is much more evident in this experiment, which can be attributed to the higher sensitivity of continuous parameter estimation to small measurement errors. While classification can remain correct despite moderate disturbances in the received signal, range estimation is much more sensitive to the accuracy of the estimated signal parameters. As a result, the improved measurement quality obtained by adaptive pilots is reflected more directly in the reduced MAE, making the advantage more pronounced. Fig. 6: Number of pilots selected at different SNRs. (a) Ranging -9 dB (b) Ranging 0 dB (c) Ranging 20 dB (d) Classification -9 dB (e) Classification 0 dB (f) Classification 20 dB Fig. 7: The adaptive pilot placement of the proposed SemISAC methodology for both ranging and classification sensing scenarios at different SNRs. Figure 4(a) presents the performance of the SemCom module of our proposed SemISAC methodology compared to S2 and simple SemCom. The simple SemCom method serves as a communication-only reference representing the performance achievable in the absence of sensing-related constraints. It provides a practical ceiling for comparison, as it isolates the communication task from the additional requirements imposed by sensing in an ISAC system. As shown by the results, the SemISAC approach achieves performance comparable to simple SemCom despite simultaneously performing sensing tasks. The improvement over S2 is driven by the Swin Transformer architecture used in the backbone of our proposed approach. At extremely low SNRs, the proposed SemISAC approach even outperforms simple SemCom due to its adaptive pilot design, which allows it to adjust its waveform based on channel and environmental conditions. A practical example of the image data, ground truth, and performance of all the approaches is provided in Figure 5, which shows that SemISAC and simple SemCom maintain high segmentation accuracy, whereas S2 introduces significant distortions in the task results. Although the simple SemCom approach also uses the Swin architecture, the performance gains of simple SemCom over SemISAC are not as pronounced as the gain of SemISAC over S2. To further investigate this behavior, we evaluate the proposed approach with different numbers of subcarriers, as shown in Figure 4(b). The results demonstrate diminishing returns after increasing the number of subcarriers beyond 64. Going from 16 to 32 and from 32 to 64 subcarriers leads to a percentage increase of 3.64% and 2.41%, respectively, whereas going beyond 64 subcarriers leads to a percentage increase of less than 1%. This can be linked to the semantic expressive power of the resource elements. With 64 subcarriers, the resource elements are sufficient to encode the maximum amount of semantic information required for successful decoding at the receiver. Therefore, increasing the number of subcarriers beyond 64 does not result in a significant improvement in performance. This explains the minor gap shown by simple SemCom in Figure 4(a), as both approaches are evaluated with 64 subcarriers, and accommodating pilot signals for sensing in SemISAC does not cause a significant drop in the expressive power of the SemISAC encoder. Figure 7 presents the adaptive pilot placement of the proposed SemISAC framework for both classification and ranging tasks at different signal strengths. Within the given N×MN× M grid, the black cells represent the pilot REs and the white cells represent the data-bearing REs. It can be observed that as the SNR improves, the density of pilots decreases under favorable channel conditions. This behavior is shown in Figures 7(a) through 7(c) for ranging and 7(d) through 7(f) for classification and reflects the adaptive nature of the proposed strategy. When the received signal quality is high, reliable sensing information can be obtained from fewer pilots, allowing a larger fraction of the REs to be allocated to data transmission. Conversely, at lower SNRs, additional pilots are required to provide reliable observations. The observed variation and the decreasing trend in pilot density, as shown by both 7 and 6, demonstrate that the proposed methodology dynamically trades pilot density against signal quality instead of relying on a fixed pilot pattern. This observed behavior is consistent with the analytical characterization established in Proposition 1. Specifically, the proposition shows that under the considered distortion model, the optimal pilot count Np⋆(γ)N_p (γ) is strictly decreasing with SNR and approaches zero in the high-SNR regime. The decreasing pilot density observed in the learned masks and in Figure 6 therefore provides empirical evidence that the proposed SNR-conditioned pilot selection captures the qualitative trend predicted by the analytical model. VI Conclusion In this study, we proposed SemISAC, a unified framework that performs both SemCom and SemSens using a single dual-function waveform. SemISAC uses a shared Swin Transformer encoder that converts the input road-scene image into semantic symbols placed on the OFDM grid. An adaptive pilot-selection network, trained with a straight-through estimator, determines how many pilots to use and where to place them based on the SNR. At the receiver, the communication decoder reconstructs the road-scene segmentation from the received data cells, while the channel state information is estimated from the pilot cells. The sensing decoders at the transmitting vehicle process the channel estimate obtained from the pilot cells and output the target class and range. Adaptive pilot selection is a key contributor to the performance gains: the selected pilot count decreases steadily as the SNR improves, allowing the system to use fewer pilots when the signal is strong. The evaluation results show that SemISAC achieves an accuracy close to that of the SemCom-only baseline and clearly outperforms the S2 baseline. It also outperforms the semantic and conventional sensing baselines, including S3, DL-MSE, SVM, OMP, and MUSIC, in classifying pedestrians, cars, and drones and in estimating their ranges. Appendix A Proof of Lemma 1 Proof. Write J(ρ)=waρ+(1−w)b1−ρJ(ρ)= waρ+ (1-w)b1-ρ. Then J′(ρ)=2waρ3+2(1−w)b(1−ρ)3>0J (ρ)= 2waρ^3+ 2(1-w)b(1-ρ)^3>0 on (0,1)(0,1), so J is strictly convex. Since J′(ρ)→−∞J (ρ)→-∞ as ρ→0+ρ→ 0^+ and J′(ρ)→+∞J (ρ)→+∞ as ρ→1−ρ→ 1^-, a unique interior stationary point exists. Setting J′(ρ)=−waρ2+(1−w)b(1−ρ)2=0J (ρ)=- waρ^2+ (1-w)b(1-ρ)^2=0 gives ρ1−ρ=wa(1−w)b ρ1-ρ= wa(1-w)b, and solving yields (23). Writing ρ⋆=(1+b/a(1−w)/w)−1ρ = (1+ b/a\, (1-w)/w )^-1, the factor (1−w)/w (1-w)/w is strictly decreasing in w, so ρ⋆ρ is strictly increasing in w. The stated limiting values follow directly. This concludes the proof. ∎ Appendix B Proof of Proposition 1 Proof. The optimization has the same strictly convex form as in Lemma 1. Specifically, defining A=waγA= waγ, C=(1−w)clog(1+γ)C= (1-w)c (1+γ) gives J(Np)=ANp+CNc−NpJ(N_p)= AN_p+ CN_c-N_p. Hence, by the same allocation argument as in Lemma 1, the unique minimizer is Np⋆=NcA/(A+C)N_p =N_c A/( A+ C), which yields (13). It remains to establish the SNR dependence. Writing Np⋆=Nc(1+C/A)−1N_p =N_c(1+ C/A)^-1, we have CA=(1−w)cwag(γ) CA= (1-w)cwa\,g(γ), where g(γ)=γlog(1+γ)g(γ)= γ (1+γ). Then g′(γ)=[log(1+γ)−γ1+γ]/log2(1+γ)g (γ)=[ (1+γ)- γ1+γ]/ ^2(1+γ). Let h(γ)=log(1+γ)−γ1+γh(γ)= (1+γ)- γ1+γ. Since h(0)=0h(0)=0 and h′(γ)=γ(1+γ)2>0h (γ)= γ(1+γ)^2>0 for γ>0γ>0, we have h(γ)>0h(γ)>0, and therefore g′(γ)>0g (γ)>0. Thus C/AC/A is strictly increasing in γ, implying that Np⋆(γ)N_p (γ) is strictly decreasing. Finally, γlog(1+γ)→∞ γ (1+γ)→∞ as γ→∞γ→∞, so C/A→∞C/A→∞ and, consequently, Np⋆(γ)→0N_p (γ)→ 0. ∎ References [1] X. Zhu et al., “Enabling intelligent connectivity: A survey of secure ISAC in 6G networks,” IEEE Communications Surveys & Tutorials, vol. 27, no. 2, p. 748–781, 2025. [2] F. Liu et al., “Integrated sensing and communications: Toward dual-functional wireless networks for 6G and beyond,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 6, p. 1728–1767, Jun. 2022. [3] X. Luo, H.-H. Chen, and Q. Guo, “Semantic communications: Overview, open issues, and future research directions,” IEEE Wireless Communications, vol. 29, no. 1, p. 210–219, Feb. 2022. [4] S. Guo et al., “A survey on semantic communication networks: Architecture, security, and privacy,” IEEE Communications Surveys & Tutorials, vol. 27, no. 5, p. 2860–2894, Oct. 2025. [5] Y. Peng, L. Xiang, K. Yang, F. Jiang, K. Wang, and D. O. Wu, “SIMAC: A semantic-driven integrated multimodal sensing and communication framework,” IEEE Journal on Selected Areas in Communications, vol. 44, p. 673–688, 2026. [6] G. Zhu et al., “Semantic-based channel state information feedback for AAV-assisted ISAC systems,” IEEE Internet of Things Journal, vol. 12, no. 5, p. 4981–4991, Mar. 2025. [7] O. Huan, C. Liu, N. Yang, Z. Zhang, T. Luo, and M. Chen, “Joint optimization of digital semantic communication and radar sensing for enhanced ISAC,” IEEE Transactions on Wireless Communications, vol. 25, p. 16730–16744, 2026. [8] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin Transformer: Hierarchical vision transformer using shifted windows,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2021, p. 10012–10022. [9] Z. Guo, D. Liu, and Z. Zhang, “Perception enhanced multimodal multitask semantic communication and resource management for UAV-assisted ISAC systems,” IEEE Transactions on Vehicular Technology, vol. 75, no. 6, p. 11051–11065, Jun. 2026. [10] Y. Liu, B. He, and F. Wang, “Security-aware semantic-driven ISAC via paired adversarial residual networks,” IEEE Transactions on Vehicular Technology, vol. 75, no. 8, p. 18529–18534, Aug. 2026. [11] S. Liu, H. Yang, W. Xie, and M. Zheng, “Intelligent semantic communication scheme integrating ISAC for low-altitude intelligent networks,” IEEE Transactions on Communications, vol. 74, p. 3018–3033, 2026. [12] S. Haykin, “Cognitive radar: A way of the future,” IEEE Signal Processing Magazine, vol. 23, no. 1, p. 30–40, Jan. 2006. [13] Y. Zhao, X. Xu, Y. Zeng, F. Liu, Y. Huang, and Y. L. Guan, “Sensing-assisted predictive beamforming with multipath echo signals,” IEEE Transactions on Vehicular Technology, vol. 74, no. 5, p. 7539–7553, May 2025. [14] C. Wu, C. Huang, Y. Li, F. Liu, Y. Liu, and Y. He, “Adaptive waveform design for cognitive radar target detection with constant modulus constraint,” in Proc. IEEE Int. Conf. Signal, Inf. Data Process. (ICSIDP), Zhuhai, China, 2024, doi: 10.1109/ICSIDP62679.2024.10868382. [15] S. Hua et al., “Bandwidth management in semantic communications: A tradeoff between data sensing and transmission,” IEEE Transactions on Vehicular Technology, vol. 75, no. 1, p. 617–630, Jan. 2026. [16] A. F. Martone et al., “Closing the loop on cognitive radar for spectrum sharing,” IEEE Aerospace and Electronic Systems Magazine, vol. 36, no. 9, p. 44–55, Sep. 2021. [17] P. Song, L. Mei, and H. Cheng, “Human semantic segmentation using millimeter-wave radar sparse point clouds,” in Proc. 26th Int. Conf. Comput. Supported Cooperative Work Design (CSCWD), Rio de Janeiro, Brazil, 2023, p. 1275–1280. [18] Y. Wang, Y. Wu, D. Li, and W. Yu, “Millimeter-wave radar and vision fusion-based semantic simultaneous localization and mapping,” IEEE Antennas and Wireless Propagation Letters, vol. 23, no. 11, p. 3977–3981, Nov. 2024. [19] S. Biswas, A. M. Alam, and A. C. Gurbuz, “HRSpecNET: A deep learning-based high-resolution radar micro-Doppler signature reconstruction for improved HAR classification,” IEEE Transactions on Radar Systems, vol. 2, p. 484–497, 2024. [20] M. Gu, Z. Chen, K. Chen, and H. Pan, “IR-ST: A lightweight transformer network for human fall detection based on FMCW radar,” IEEE Sensors Journal, vol. 23, no. 20, p. 25128–25135, Oct. 15, 2023. [21] X. Zhang, J. A. Zhang, C. Liu, W. Yuan, G. Y. Li, and M. G. Amin, “Semantic sensing: A task-oriented paradigm,” arXiv:2603.29102, 2026. [22] Z. Zhang et al., “Channel semantic characterization for integrated sensing and communication scenarios: From measurements to modeling,” IEEE Transactions on Wireless Communications, vol. 25, p. 13073–13089, 2026. [23] Y. Zhang et al., “AI empowered channel semantic acquisition for 6G integrated sensing and communication networks,” IEEE Network, vol. 38, no. 2, p. 45–53, Mar. 2024. [24] G. J. Brostow, J. Fauqueur, and R. Cipolla, “Semantic object classes in video: A high-definition ground truth database,” Pattern Recognition Letters, vol. 30, no. 2, p. 88–97, Jan. 2009. [25] E. Bourtsoulatze, D. Burth Kurka, and D. Gündüz, “Deep joint source-channel coding for wireless image transmission,” IEEE Transactions on Cognitive Communications and Networking, vol. 5, no. 3, p. 567–579, Sep. 2019.