Paper deep dive
PAI: Fast, Accurate, and Full Benchmark Performance Projection with AI
Avery Johnson, Mohammad Majharul Islam, Riad Akram, Abdullah Muzahid
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/23/2026, 12:03:12 PM
Summary
PAI is a novel performance projection technique for SoCs that uses a hierarchical LSTM-based model to predict full benchmark performance metrics (like IPC) without requiring detailed cycle-accurate simulation or instruction-wise encoding. By utilizing microarchitecture-independent features (uAIMs) and hardware configurations, PAI achieves a 9.35% average IPC prediction error on the SPEC CPU 2017 suite while being three orders of magnitude faster than traditional simulators.
Entities (5)
Relation Signals (3)
PAI → evaluatedon → SPEC CPU 2017
confidence 100% · PAI can achieve an average IPC prediction error of 9.35% for SPEC CPU 2017 benchmark suite
PAI → usesmodel → Hierarchical LSTM
confidence 100% · At the heart of PAI is a hierarchical Long Short Term Memory (LSTM)-based model
PAI → collectsdatafrom → Simics
confidence 90% · The features are collected either from real machines or instruction set emulators (such as Simics)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The exponential increase in complex IPs within modern SoCs, driven by Moore's Law, has created a pressing need for fast and accurate hardware-software power-performance analysis. Traditional performance simulators (such as cycle accurate simulators) are often too slow to simulate full benchmarks within a reasonable timeframe; require considerable effort for development, maintenance, and extensions; and are prone to errors, making pre-silicon performance projections and competitive analysis increasingly challenging. Prior attempts in addressing this challenge using machine learning fall short as they are either slow, inaccurate or unable to predict the performance of full benchmarks. To address these limitations, we present PAI, the first technique to accurately predict full benchmark performance without relying on detailed simulation or instruction-wise encoding. At the heart of PAI is a hierarchical Long Short Term Memory (LSTM)-based model that takes a trace of microarchitecture independent features from a program execution and predicts performance metrics. We present the detailed design, implementation and evaluation of PAI. Our initial experiments showed that PAI can achieve an average IPC prediction error of 9.35% for SPEC CPU 2017 benchmark suite while taking only 2 min 57 sec for the entire suite. This prediction error is comparable to prior state-of-the-art techniques while requiring 3 orders of magnitude less time.
Tags
Links
- Source: https://arxiv.org/abs/2603.19330v1
- Canonical: https://arxiv.org/abs/2603.19330v1
Trouble viewing inline? Open PDF directly →
Full Text
22,841 characters extracted from source content.
Expand or collapse full text
1 PAI: Fast, Accurate, and Full Benchmark Performance Projection with AI Avery Johnson, Mohammad Majharul Islam, Riad Akram, and Abdullah Muzahid Abstract—The exponential increase in complex IPs within mod- ern SoCs, driven by Moore’s Law, has created a pressing need for fast and accurate hardware-software power-performance analysis. Traditional performance simulators (such as cycle accurate simulators) are often too slow to simulate full bench- marks within a reasonable timeframe; require considerable effort for development, maintenance, and extensions; and are prone to errors, making pre-silicon performance projections and competitive analysis increasingly challenging. Prior attempts in addressing this challenge using machine learning fall short as they are either slow, inaccurate or unable to predict the performance of full benchmarks. To address these limitations, we present PAI, the first technique to accurately predict full benchmark performance without relying on detailed simulation or instruction-wise encoding. At the heart of PAI is a hierarchical Long Short Term Memory (LSTM)-based model that takes a trace of microarchitecture independent features from a program execution and predicts performance metrics. We present the detailed design, implementation and evaluation of PAI. Our initial experiments showed that PAI can achieve an average IPC prediction error of 9.35% for SPEC CPU 2017 benchmark suite while taking only 2 min 57 sec for the entire suite. This prediction error is comparable to prior state-of-the-art techniques while requiring 3 orders of magnitude less time. I. INTRODUCTION E MERGENCE of Internet-of-Things and smart devices are making System-on-Chips (SoCs) a popular design option for their size, cost, and performance efficiency. Modern SoCs are sophisticated, incorporating a wide range of Intel- lectual Property (IP) blocks such as CPUs, GPUs, memory subsystems, various hardware accelerators, communication interfaces, sensors, etc. Furthermore, the IPs themselves are becoming more and more complex. This necessitates any tool that can make designing SoCs faster and easier to do. Performance modeling is an integral part for designing the next generations SoCs. It is a way to determine what design choices are better and therefore, should be considered at different stages of the design cycle. However, the complexity of modern SoCs make it a challenging task to model or predict performance. Traditionally, performance modeling is done through an architectural performance simulator such as Z-sim [1], Gem5 [2], ChampSim [3], etc. These simulators simulate varying degrees of architectural events in a cycle-by- cycle fashion, eventually leading to a set of desired metrics such as Instruction Per Cycle (IPC), cache misses, branch Manuscript received March 19, 2025; revised June 11, 2025. Avery Johnson and Abdullah Muzahid are with Texas A&M University, College Station, TX 77840 USA. (e-mail: averyjohnson@tamu.edu, abdul- lah.muzahid@tamu.edu). Mohammad Majharul Islam and Riad Akram are with the Intel Corporation, Hillsboro, OR 97124 USA. (e-mail: mohammad.majharul.islam@intel.com, riad.akram@intel.com). mispredictions, power consumption, etc. The simulators, al- though quite accurate, are extremely slow. They operate at Kilo Instructions per Second (KIPS) speed and can take from few hours to days or weeks to simulate a full benchmark. As an example, it takes 8 hours to simulate a single core for one second of execution in Zsim [1]. This exorbitant runtime puts a serious hurdle to the performance estimation step of the SoC design process. As every iteration of the design cycle needs to simulate many benchmarks to evaluate and eventually, improve the design, these simulators impose a significant development cost and delay. Furthermore, even state-of-the-art simulators aren’t perfect; with an average error of 9.7% and 13% respectively for Z-sim and Gem5 [4]. What is needed is a performance modeling/prediction technique that is both fast and accurate. So, we ask ourselves the following question: Can this problem be solved with emergent machine learning models? Given that machine learning models are fast at inference (compared to the time for cycle-accurate simulation), it’s no surprise that it has already been applied in earlier research [5]– [8]. Most recently, Li et al. [5] propose to combine two deep learning models - one to capture program representa- tions by encoding individual instructions and one to capture microarachitectural specifications. The third model combines the output of those two models and predicts performance of a program. As the model needs to encode individual instructions, it’s practicality is limited by how many instructions can be processed before the model starts to exhibit poor prediction accuracy. TAO [6] uses a dataset of microarchitecture in- dependent features from a detailed simulator. This dataset is used to train a multi-head attention network in order to predict IPC, branch mispredictions, etc. TAO requires a detailed simulator for the the dataset to begin with. Therefore, it is inherently limited by the speed and accuracy of detailed simulator. Munigala et al. [7] use a multilayer perceptron to predict the bandwidth and latency of a benchmark based on seven architectural parameters (core count, clock frequency, Last Level Cache (LLC) size, etc.). However, like TAO, the performance metrics are obtained from a detailed simulator and hence, it faces similar issues. Barboza et al. [8] use many performance counters across multiple benchmarks to train a classifier to predict performance bottlenecks. The goal is to find performance degrading components of a mi- croarchitecture. Like earlier works, it also relies on detailed simulation and considers only a single architecture. Besides these techniques, there are earlier attempts to use machine learning for performance prediction [9]–[11]. However, they are program and/or architecture specific, and therefore, lacking arXiv:2603.19330v1 [cs.AR] 18 Mar 2026 2 generality. To overcome the limitations of prior work and propose a general approach capable of full benchmark performance prediction, we start by setting three design goals - (i) detailed simulation independence, (i) microarchitecture independence, and (i) full benchmark prediction capability. With these design goals in mind, we propose PAI. PAI is the first tech- nique to accurately predict full benchmark performance for seen and unseen programs and architectures without relying on detailed simulation or instruction-wise encoding. At the heart of PAI is a deep learning model based on hierarchical LSTM [12]. The model takes a trace of microarchitecture independent features from a program execution. The features are collected either from real machines or instruction set emulators (such as Simics [13] that is orders of magnitude faster than detailed simulation). PAI augments those features with high level microarchitectural specifications and provides performance prediction of the full benchmark both for seen and unseen programs and architectures. We provide a detailed design of PAI’s deep learning model as well as the approach to generate necessary dataset for training the model. Our initial experiments showed that PAI can achieve an average IPC prediction error of 9.35% for SPEC CPU 2017 benchmark suite while taking only 2 min 57 sec for the entire suite. This prediction error is comparable to prior state-of-the-art techniques [5], [6] while requiring 3 orders of magnitude less time. I. MAIN IDEA: PAI A. Overview of PAI At a high level, PAI works by augmenting the traditional design cycle with an additional deep learning-based predictor (as shown in Figure 1) that provides a first-order approx- imation of performance metrics. The intuition is that by approximating the performance potential of different design choices for seen and unseen programs, PAI enables hardware designers to disregard many of those choices and focus on only a few good ones. To achieve this goal, PAI works in two phases. In the training phase, a dataset comprising of hardware configurations, execution traces of microarchitecture independent features, and performance metrics is collected for entire benchmarks using native machines or ISA emulators. Since no detailed simulators are used for this phase, PAI can rely on a dataset large enough to train any deep learning model. In the prediction phase, the trained model is provided with ex- ecution traces and hardware configurations as inputs. The PAI model iterates over a trace of inputs and predicts performance metrics which are then summed up to provide full benchmark performance. By abstracting the hardware configurations and application behavior in a trace, PAI can predict performance for any hardware configuration and program - seen or unseen. B. Dataset Construction For our training dataset, we need to capture the benchmark behavior, hardware configurations, and performance metrics. To capture the benchmark behavior, we use normalized mi- croarchitecture independent metrics (uAIMs). Those can be captured from various hardware performance counters from Performance Metrics Benchmark & HW Configs Detailed Simulator Expert Analysis Fabrication PAI Training Performance Prediction Performance Metrics Training Dataset Deep Learning Model Optimization Full Benchmark Input HW Configs HW Configs Full Benchmark Input Fig. 1. Overview of how PAI works by augmenting the design cycle. PMU/Perf. Counters Processor or ISA Emulator Benchmark Timeline Interval I I+1 < uAIM, HW Config, Perf. Metrics > < uAIM, HW Config, Perf. Metrics > Differential Values Fig. 2. Overview of how PAI collects data for training. native machines or ISA emulators such as Simics [13]. Intel provides a Performance Monitoring Unit (PMU) to capture various core and uncore metrics [14]. Similarly, Simics pro- vides a diverse range of uAIMs to capture the current state of program execution. Examples of uAIMs are number of taken/not taken branches, page faults, memory reads/writes, cache misses at different levels, number of different instruc- tions, etc. In order to capture long running benchmarks or entire benchmarks, a snapshot of uAIMs is recorded at a cer- tain interval (e.g., every 10M instructions). Each such snapshot consists of differential values between the current and previous counters (as shown in Figure 2). Each snapshot is associated with hardware configurations and performance metrics (during the period from the last snapshot to the current one). Examples of hardware configurations are core count, thread count, size of different levels of cache, clock frequency, instruction issue width, ROB size, etc. Examples of performance metrics are Instruction Per Cycle (IPC), number of clock cycles, etc. LSTM LSTM LSTM uAIM HW Configs Perf Metrics Fully Connected Layers LSTM Perf Metrics Fully Connected Layers uAIM & HW Configs Fig. 3. Simple LSTM model. LSTM LSTM LSTM uAIM HW Configs Perf Metrics Fully Connected Layers Fig. 4. Hierarchical LSTM model. C. Model Development Predicting performance metrics such as IPC can be con- sidered as a regression problem, so we started with a linear regression model and a simple multilayer perceptron model. The linear regression model is too simplistic to capture the non-linear relation between the input and the output. Similarly, 3 a multilayer perceptron is not sophisticated enough to achieve a reasonably good accuracy for a dataset that has non-linear and sequential patterns (Section I-C). So, we turned our focus to LSTM that is designed for sequence learning [12]. We experimented with two LSTM-based designs - Sim- ple and Hierarchical. Figure 3 shows the simple model. It combines the uAIMs and hardware configurations to an LSTM layer which produces some latent representation of the inputs. The latent vector is passed through one or more fully connected layers to produce the final IPC prediction. Figure 4 shows the hierarchical LSTM model. One LSTM layer takes on uAIMs whereas another LSTM layer takes on hardware configurations. The resultant latent vectors are processed through a second level LSTM layer to produce the final latent vector. This latent vector is then passed through one or more fully connected layers (just like the simple design) to produce the final output. I. EVALUATION A. Experimental Setup To evaluate PAI we used the SPEC CPU2017 [15] bench- mark suite due to its representativeness. We developed PAI’s training and testing framework using PyTorch and evaluated its performance on an Intel® Xeon® Silver 6336Y 24-core CPU. B. Dataset Preparation To prepare the training and testing datasets, we used Simics to collect 128 uAIMs every 10 million instructions during pro- gram execution. For each interval, we retrieved IPCs from 15 different configurations (SKUs) of Intel® Xeon® 6 processors, capturing performance variations due to different clock speeds, core counts, cache sizes, etc. Each datapoint in the dataset consists of a snapshot of these uAIMs (differential values) and the corresponding IPCs. Overall, the generated dataset consists of 1.3 million datapoints. A breakdown of the different uAIM features is shown in Table I. TABLE I BREAKDOWN OF UAIM FEATURES FeaturesExplanationFeature count Instruction related Instruction count per category, register statistics (e.g., register reuse distance), operand histogram, etc. 61 Memoryaccess related Code and data reuse distance48 Branch relatedCount of branch taken/not-taken and branch entropy [5] 7 Remaining features Interrupts, page faults, etc.12 We experimented with two different dataset splits. In one split, we used an 80/20 training/testing split, meaning 80% of the dataset is used in training and the remaining 20% is used for validation/testing. In other split, we deliberately split the dataset in such a way that it helps evaluating PAI for unseen benchmarks. We reorganized our dataset as follows: as unseen benchmarks, we hold out XZ, WRF, MCF, nab, cactubssn, and xalanbmk for 15 SKUs as our testing set. We selected these three benchmarks to represent core bound, uncore bound, and memory bound behavior. The rest of the benchmarks are used as our training set. C. Model Analysis To develop a suitable model for PAI, we experimented with simple as well as complex model architectures. We started with a simple Linear Regression model followed by a multi- layer perceptron model (MLP). Our initial results for both of these models showed unimpressive results, with our MLP only achieving a 0.45/0.78 train/test MSE. Therefore, we moved on to LSTM-based architectures. We experimented with a simple LSTM-based model first. This model consists of a two-layer LSTM with two fully connected layers. We searched a subset of the hyperparameter space by changing the learning rate and batch size. The results are shown in Figure 5. Each model is trained for 10k epochs. We see that there’s a large spread between the best and worst performing parameter models. Notably, we see that the best performing models all have a batch size of 2048 and the worst performing models all have a batch size of 128. Fig. 5. Hyperparameter tuning of simple LSTM-based model. Our best model achieves a training loss of 0.038 and a test- ing MSE of 0.100. We further analyzed the train/test curve of the best model in order to evaluate whether or not is overfitting. In the train/test curve (Figure 6), we see that the training and testing loss are both consistently improving throughout the entire training process. The lack of any divergence between the training and testing loss indicates that no overfitting has occurred and the model can stand to improve with further training epochs. Fig. 6. Training and testing curve for the best performing model. As we continued with our custom data split to evaluate unseen benchmarks, the simple LSTM-based model failed to converge. So, we developed and experimented with a hierarchical LSTM-based model (Figure 4). After conducting the same hyperparameter tuning process as with the single LSTM, we trained our best model on the custom dataset: achieving a train/test MSE of 0.010/0.029. Figure 7 illustrates the average IPC prediction error of PAI using the hierarchical LSTM model compared to the IPC obtained from silicon. PAI’s predictions have an average absolute error of only 9.35% which demonstrates that our 4 Fig. 7. Full benchmark IPC predictions using hierarchical LSTM model. model is capable of predicting IPC values fairly accurately for entire benchmarks despite being trained on fine-grained intervals. Notably, when just considering seen/unseen bench- mark predictions, we see that our seen benchmarks have an average error of 7.64% and our unseen benchmarks have an average error of 15.5%. In our results, we see a few outliers like gcc, lbm and cactubssn, which have a prediction error significantly over 10%. For cactubssn we observed that with higher core count (64x) IPC went down (0.18x) significantly in Silicon due to high L2 (5x) and LLC (5x) miss latency. It seems PAI’s accuracy degrades in predicting a lower silicon IPC value probably due to not learning the underlying reasons such as the high L2, LLC miss latency. We intend to further study what causes these outliers to have such poor performance and address such causes in our model architecture and training dataset in the future. Thus the results are similar or slightly degraded compared to PerfVec. Despite that, PAI has an edge over PerfVec by not relying on simulation-based data. The improvement in our hierarchical LSTM design over the single LSTM is partly due to the two independent LSTMs for handling uAIMs and hardware configurations. These feature sets display different properties, so allowing the model to train the weights for uAIMs and HW configs separately leads to better accuracy for our 15 SKUs. TABLE I COMPARISON OF PREDICTION TIME. Scheme Prediction time (s) Full benchmarkFull Benchmark suite 10B inst.(3T inst. per BM)(65T inst.) PAI19.91228.91 TAO [6]50761.45×10 6 3.33×10 8 SimNet [4]69481.98×10 6 4.56×10 8 D. Timing Analysis PAI completes inference for the SPEC 2017 benchmark suite in about 2 minutes and 57 seconds, averaging 1 second per 10 billion instructions. This is three orders of magnitude faster than the state-of-the-art simulators like TAO [6] and SimNet [4] which take 5076 seconds and 6948 seconds, respectively. This is because TAO and SimNet process in- structions individually, while PAI operates at the 10 million instruction granularity. PAI’s fast inference speed enables it to run the full benchmark of approximately 3 trillion instructions for multiple iterations, with each iteration taking only about 10 seconds to predict the IPC. For TAO and SimNet, we estimated the prediction time for the full benchmark and the entire benchmark based on their time to predict 10 billion instruc- tions. Table I summarizes this prediction time comparison. Notably, PerfVec outperforms our model in benchmark-wise prediction time; however, the paper doesn’t include enough information to extrapolate this to the 10B instruction or full benchmark suite level. IV. CONCLUSION & FUTURE WORK We started this project by asking if AI/ML can be used to augment the performance estimation step of SoC design cycle by supplementing detailed simulators with a fast and accurate alternative capable of predicting full benchmark performance. Towards that end, we proposed PAI, the first technique to accurately predict full benchmark performance without relying on detailed simulation or instruction-wise encoding. At the heart of PAI is a hierarchical LSTM-based model that takes a trace of uAIMs from a program execution and predicts performance metrics. We presented the detailed design, im- plementation and evaluation of PAI. Our initial experiments showed that PAI can achieve an average IPC prediction error of 9.35% for SPEC CPU 2017 benchmark suite while taking only 2 min 57 sec for the entire suite. This prediction error is comparable to prior state-of-the-art techniques [5], [6] while requiring 3 orders of magnitude less time. Our future work includes generalizing PAI across multiple benchmark suites and processor architectures spanning multiple generations. REFERENCES [1] D. Sanchez and C. Kozyrakis, “ZSim: fast and accurate microarchitec- tural simulation of thousand-core systems,” in ISCA, 2013. [2] N. Binkert, B. Beckmann, G. Black, S. K. Reinhardt, A. Saidi, A. Basu, J. Hestness, D. R. Hower, T. Krishna, S. Sardashti, R. Sen, K. Sewell, M. Shoaib, N. Vaish, M. D. Hill, and D. A. Wood, “The gem5 simulator,” SIGARCH Comput. Archit. News, vol. 39, no. 2, 2011. [3] N. Gober, G. Chacon, L. Wang, P. V. Gratz, D. A. Jim ́ enez, E. Teran, S. Pugsley, and J. Kim, “The championship simulator: Architectural simulation for education and competition,” 2022. [4] L. Li, S. Pandey, T. Flynn, H. Liu, N. Wheeler, and A. Hoisie, “Sim- Net: Accurate and High-Performance Computer Architecture Simulation using Deep Learning,” Proc. ACM MACS, vol. 6, no. 2, 2022. [5] L. Li, T. Flynn, and A. Hoisie, “Learning generalizable program and architecture representations for performance modeling,” in SC, 2024. [6] S. Pandey, A. Yazdanbakhsh, and H. Liu, “TAO: Re-Thinking DL-based Microarchitecture Simulation,” Proc. ACM MACS, June 2024. [7] M. K. Munigala, S. Sood, and K. Madhusudhan., “Novel AI based pre- silicon Performance estimation and validation of complex System-on- Chip,” in CSDE, 2021. [8] E. C. Barboza et al., “ Aiding Microprocessor Performance Validation with Machine Learning ,” in IEEE ISPASS, May 2024, p. 1–9. [9] E. ̈ Ipek, S. A. McKee, R. Caruana, B. R. de Supinski, and M. Schulz, “Efficiently exploring architectural design spaces via predictive model- ing,” in ASPLOS, October 2006. [10] D. Li, S. Yao, Y.-H. Liu, S. Wang, and X.-H. Sun, “Efficient design space exploration via statistical sampling and AdaBoost learning,” in DAC, June 2016. [11] N. Ardalani, C. Lestourgeon, K. Sankaralingam, and X. Zhu, “Cross- architecture performance prediction (XAPP) using CPU code to predict GPU performance,” in MICRO, December 2015. [12] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, p. 1735–1780, Nov. 1997. [13] P. Magnusson, M. Christensson, J. Eskilson, D. Forsgren, G. Hallberg, J. Hogberg, F. Larsson, A. Moestedt, and B. Werner, “Simics: A full system simulation platform,” Computer, vol. 35, no. 2, p. 50–58, 2002. [14] Intel,“IntelPerformanceCounterMonitor- ABetterWaytoMeasureCPUUtilization,” https://w.intel.com/content/w/us/en/developer/articles/tool/performance- counter-monitor.html. [15] J. Bucek, K.-D. Lange, and J. v. Kistowski, “SPEC CPU2017: Next- Generation Compute Benchmark,” in ICPE, April 2018.