Paper deep dive
Beyond Keywords: Leveraging Generative LLMs and Label Aggregation to Classify Economic Policy Uncertainty in News Articles
Paul Trust
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 9/30/2026, 6:37:16 AM
Summary
This paper proposes a weakly supervised framework for classifying Economic Policy Uncertainty (EPU) in news articles using Large Language Models (LLMs). It addresses limitations of keyword-based methods (high false positives) and traditional supervised learning (costly human labeling) by using LLMs to generate synthetic labels via prompting. These noisy labels are aggregated using ensemble methods like Data Programming, Majority Vote, WAWA, and Dawid-Skene to train fine-tuned models (BERT, RoBERTa, LSTM). The study extends beyond binary classification to multi-label and hierarchical classification of specific EPU types.
Entities (11)
Relation Signals (9)
Weak Supervision ā employs ā Large Language Models
confidence 95% Ā· In this study, we propose approaches based on weak supervision techniques, using generative LLMs to create synthetic labels
BERT ā finetunedby ā Weakly Supervised Labels
confidence 95% Ā· The resulting labels are used to fine-tune LLM s: BERT, RoBERTa, and Long Short Term Memory (LSTM)
RoBERTa ā finetunedby ā Weakly Supervised Labels
confidence 95% Ā· The resulting labels are used to fine-tune LLM s: BERT, RoBERTa, and Long Short Term Memory (LSTM)
LSTM ā finetunedby ā Weakly Supervised Labels
confidence 95% Ā· The resulting labels are used to fine-tune LLM s: BERT, RoBERTa, and Long Short Term Memory (LSTM)
Large Language Models ā usedfor ā Economic Policy Uncertainty
confidence 95% Ā· This research describes the adaptation of Large Language Models (LLMs) for economic monitoring... to automatically determine whether an article discusses Economic Policy Uncertanity (EPU)
Data Programming ā usedfor ā Label Aggregation
confidence 95% Ā· refined using aggregation methods such as Data Programming (DP)
Dawid-Skene ā usedfor ā Label Aggregation
confidence 95% Ā· refined using aggregation methods such as ... DawidāSkene
Baker et al. (2016) ā ā
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:This research describes the adaptation of Large Language Models (LLMs) for economic monitoring in the public sector to automatically determine whether an article discusses Economic Policy Uncertanity (EPU) and to identify its specific type. Previous studies either rely on keywords, which often result in a high count of false positives, or use machine learning approaches that require a large number of quality human labeled data that is costly and time consuming to acquire. In this study, we propose approaches based on weak supervision techniques, using generative LLMs to create synthetic labels through prompting, making the approach both cost-effective and scalable. Additionally, we propose methods for for multi-label and hierarchical classification of articles related to EPU.
Tags
Links
- Source: https://arxiv.org/abs/2609.35856v1
- Canonical: https://arxiv.org/abs/2609.35856v1
Trouble viewing inline? Open PDF directly ā
Full Text
54,996 characters extracted from source content.
Expand or collapse full text
Beyond Keywords: Leveraging Generative LLMs and Label Aggregation to Classify Economic Policy Uncertainty in News Articles Paul Trust Email: trustpaul1000@gmail.com Abstract This research describes the adaptation of Large Language Models (LLM s) for economic monitoring in the public sector to automatically determine whether an article discusses Economic Policy Uncertainty (EPU) and to identify its specific type. Previous studies either rely on keywords, which often result in a high count of false positives, or use Machine Learning (ML) approaches that require a large number of quality human labeled data that is costly and time consuming to acquire. In this study, we propose approaches based on weak supervision techniques, using generative LLM s to create synthetic labels through prompting, making the approach both cost-effective and scalable. Additionally, we propose methods for for multi-label and hierarchical classification of articles related to EPU. 1 Introduction The growing availability of digital text, from news media to online platforms and institutional records, has created new opportunities for studying economic and political dynamics at scale. Computational analysis of such unstructured text has become an important tool in disciplines including economics, sociology, and political science Gentzkow et al. (2019); Grimmer and Stewart (2013). In economics, textual indicators have been derived from central bank communications Rybinski and others (2019), corporate earnings calls Keith and Stent (2019), news articles Baker et al. (2016) and social media data Indaco (2020). A well known application is the EPU index by Baker et al. (2016), which measures the frequency of policy- and uncertainty-related terms in news articles Baker et al. (2016). While influential, this keyword-based method is prone to false positives and fails to capture policy context with precision. To overcome these limitations, a range of ML techniques have been explored. Unsupervised approaches such as topic modeling and embedding-based expansions provide improved coverage of relevant content Azqueta-GavaldĆ³n (2017); Azqueta-Gavaldon et al. (2020); Azqueta-GavaldĆ³n et al. (2023); Kaveh-Yazdy and Zarifzadeh (2021); Nyman and Ormerod (2020); Wang et al. (2024); Miranda-Belmonte et al. (2023); Jiang et al. (2023); Xu et al. (2023), but they remain less accurate than supervised methods. Traditional supervised ML and more recent deep learning techniques LoliÄ et al. (2022); Tobback et al. (2018); Sadorsky (2022); Wang et al. (2022); Rao et al. (2024); Chen et al. (2024); Qureshi et al. (2021); Keith et al. (2020); CHANG et al. (2024); Audrino et al. (2024); Yeh et al. (2024); Ito et al. (2025) have improved predictive performance but rely on costly human-labeled datasets. This dependence on manual annotation constrains scalability and limits adaptation to new time periods, countries, or policy domains. The most closely related work is by Paul et al. (2023), which uses labeling functions to create synthetic labels for training LLM s. However, these generated labels are often unstable, and designing high-quality labeling functions requires substantial domain expertise Trust et al. (2023). Furthermore, most prior research has focused only on binary classification, identifying whether an article discusses EPU without distinguishing between specific types of policy uncertainty. This research proposes a weakly supervised framework for EPU classification that addresses these challenges. LLM s are prompted to generate synthetic labels, which are then refined using aggregation methods such as Data Programming (DP), DawidāSkene, Worker Agreement with Aggregate (WAWA), and Majority Vote. The resulting labels are used to fine-tune LLM s: BERT, RoBERTa, and Long Short Term Memory (LSTM), thereby reducing dependence on human annotation. Beyond binary classification, we extend the task to multi-label and hierarchical settings, enabling the assignment of articles to specific policy subcategories once identified as EPU-related. The remainder of this paper is structured as follows: Section 2 reviews related work, Section 3 presents the methodology, Section 4 reports the experimental findings, and Section 5 presents the conclusion of this work. 2 Related Work This section presents related work on EPU classification, categorized into unsupervised ML methods, traditional ML approaches, deep learning techniques, and LLM methods, summarized in Table 1 and discussed next. Table 1: Relevant studies on automated methods for EPU classification Category Relevant Works Unsupervised ML Azqueta-Gavaldon et al. (2020), Xu et al. (2023), Azqueta-GavaldĆ³n (2017), Wang et al. (2024), Miranda-Belmonte et al. (2023), Kaveh-Yazdy and Zarifzadeh (2021), Nyman and Ormerod (2020), Azqueta-GavaldĆ³n et al. (2023), Jiang et al. (2023), Baker et al. (2016) Traditional ML LoliÄ et al. (2022), Tobback et al. (2018), Sadorsky (2022), Wang et al. (2022) Deep Learning & LLM s Rao et al. (2024), Chen et al. (2024), Qureshi et al. (2021), Keith et al. (2020), CHANG et al. (2024), Audrino et al. (2024), Yeh et al. (2024), Ito et al. (2025) Unsupervised ML Approaches: Most EPU detection methods have utilized unsupervised learning techniques to identify news articles related to EPU. The earliest and most widely used approach, introduced by Baker et al. (2016), relies on keyword matching for classification. Building on this, alternative approaches have been proposed to enhance the detection process. For instance, Kaveh et al. (2021) constructed an EPU index by leveraging a word embedding representation space to refine keyword selection for EPU detection in news articles. Nyman and Ormerod (2020) further expanded the original uncertainty keyword list proposed by Baker et al. (2016) using nearest neighbor embeddings and examined the Granger causality between their expanded keyword set and the existing EPU index Nyman and Ormerod (2020). While keyword-based methods offer a simple and interpretable solution, they often suffer from high false positive rates and limited semantic understanding. To overcome these limitations, topic modeling techniques have been proposed as a more robust alternative for uncovering latent themes related to EPU. Relevant studies include: Azqueta et al. (2020), who applied topic models to construct EPU indices for European countries Azqueta-GavaldĆ³n (2017); Azqueta-Gavaldon et al. (2020); Azqueta-GavaldĆ³n et al. (2023). Wang et al. (2024) used Latent Dirichlet Allocation (LDA) to extract themes from newspapers articles and constructed the Chinese EPU index from them Wang et al. (2024). Miranda et al. (2023) implemented topic modeling with semantic clustering based on word embeddings to detect EPU in digital news Miranda-Belmonte et al. (2023). Jiang et al. (2023) applied BERTopic and regularized linear models to predict EPU using social media data from Reddit Jiang et al. (2023). Xu et al. (2023) utilized an unsupervised approach combining principal component analysis and random matrix theory to construct EPU indices Xu et al. (2023). Although topic modeling captures latent semantic structures, its unsupervised nature limits classification accuracy compared to supervised learning. To improve precision and reliability in EPU detection, traditional ML methods using labeled data have been developed. Traditional ML Approaches: Relevant studies in this category include: LoliÄ et al. (2022) who applied ML techniques: linear regression, random forests, Gradient Boosting Machines (GBM), decision trees, extreme gradient boosting, and their ensembles for EPU classification LoliÄ et al. (2022). Tobback et al. (2018) used Support Vector Machines (SVM) to classify news articles related to EPU from Belgian news sources Tobback et al. (2018). Sadorsky (2022) employed various ML models such as random forests, extremely randomized trees, GBM, and NaĆÆve Bayes for EPU classification Sadorsky (2022). Wang et al. (2022) utilized neural networks, GBM, a dictionary-based approach, and ensemble classifiers, to classify EPU in Chinese news articles Wang et al. (2022). The reliance of traditional ML on manual feature engineering and limited capacity to capture complex language patterns has motivated a shift towards deep learning models and LLM s for more advanced text understanding. Deep Learning & LLM Approaches: Representative studies include: Rao et al. (2024) who used LSTM s, GRU s, and multi-layer perceptrons to classify EPU in news articles Rao et al. (2024). Chen et al. (2024) applied word embeddings based on Skip-Gram along with GRU, Convolution Neural Network (CNN), and capsule networks for EPU classification tasks Chen et al. (2024). Qureshi et al. (2021) utilized BERT models, keyword extraction algorithms, and a combination of RoBERTa and sentence transformers to extract and classify EPU related keywords Qureshi et al. (2021). Keith et al. (2020) combined keyword matching, supervised ML techniques (logistic regression with Bag of words (BoW) and BERT embeddings), and prevalence estimation techniques such as probabilistic classify and count, and implicit likelihood to classify news articles related to EPU Keith et al. (2020). Chang et al. (2024) leveraged BERT to classify trade policy uncertainty based on texts from management discussion sections of annual reports from Chinese public firms CHANG et al. (2024). Audrino et al. (2024) adapted various LLM s, including GPT-4, LLAMA, and Zephyr-7B for EPU classification Audrino et al. (2024). Yeh et al. (2024) applied LLM s such as GPT-3.5-Turbo, GPT-4, and Claude3 to generate keywords for EPU classification Yeh et al. (2024). Ito et al. (2024) fine-tuned LLM s to detect EPU, contributing to the development of Japanās monetary policy index for the period between 2015 and 2016 Ito et al. (2025). Paul et al. (2023) proposed to use labeling functions to generate synthentic labels to pre-train LLM s Trust et al. (2023). Although these methods also used LLM s like the approaches proposed in this research, their reliance on costly human-curated labels limits their applicability. Research Gap: Numerous previous studies discussed focused on unsupervised ML approaches, primarily using topic models and keyword matching. While these methods are simple and do not require training, they are less effective than supervised ML techniques. In contrast, supervised ML methods, including traditional ML and deep learning, rely on human-curated datasets, making them less scalable. Constructing a new index for a different year, country, or region requires training new models and generating new labeled datasets, which is resource-intensive. Furthermore, prior research has been limited to binary classification, identifying only whether an article contains EPU or not, with no existing studies attempting to classify specific types of EPU within an article. In this research, we present our proposal to use LLM s for classifying news articles containing EPU. Instead of relying on human-curated datasets, which are costly to obtain, it proposes a weakly supervised learning approach. LLM s are leveraged to generate training labels or directly classify news articles through prompting. The labels obtained via prompting can either serve as the final classification labels or be refined using ensemble aggregation techniques with multiple LLM s to reduce label noise. Furthermore, this research proposes extending beyond binary classification by adapting LLM s for multi-label and hierarchical classification of EPU-related news articles, an aspect not previously explored in existing research. 3 Methodology This section describes the methodology used for analyzing and classifying news articles related to EPU. The process begins with EPU classification, where weakly supervised labels generated by LLM s are compared with human annotated labels. Next, multi-label text classification is applied to identify different types of EPU present in the articles, with labels assigned only to articles classified as EPU-related. Since this classification follows a hierarchical structure, additional experiments are conducted to assess whether incorporating this hierarchy enhances classification performance. 3.1 Weakly supervised EPU classification For the weakly supervised approach used for EPU classification, we consider a dataset DUD^U of n unlabeled data points: DU=xii=1nD^U=\x_i\_i=1^n. The goal of text classification is to assign each data point xix_i to a label yiy_i where each label yiy_i belongs to one of m possible classes: C=c1,ā¦,cmC=\c_1,ā¦,c_m\, resulting in a labeled dataset: D=(xi,yi)i=1nD=\(x_i,y_i)\_i=1^n. Unlike traditional supervised learning, which relies on large volumes of human labeled data, weak supervision relies on synthetic labels generated by LLM s to train the classifiers. However, a human labeled dataset is used during evaluation to accurately assess the modelās performance. We formulate the EPU classification as a binary classification task with two target classes: C=āNo EPU ",āEPU "C=\``No @glossaries@gls@linkacronymepuEPU ",`` @glossaries@gls@linkacronymepuEPU "\. The proposed approach comprises three main stages. In the initial stage, LLM s are used to generate labels via prompting, guided by instructions embedded in the prompt illustrated in Figure 1. This prompt design is inspired by the annotation guidelines developed by Baker et al. (2016) for human annotators Baker et al. (2016). While the work by Paul et al. (2023) Trust et al. (2023), which this research builds on employed multiple labeling functions such as keyword lookups and semantic similarity, the results presented here are based solely on LLM-based prompting. A single prompt (Figure 1) is used to encode the labeling logic, serving as a soft rule-based alternative that simplifies the process and leverages recent advances in LLM capabilities. As a result, a label matrix is automatically constructed, where each news article receives multiple weak labels from k LLM s represented as: Ī»=Ī»1,ā¦,Ī»kĪ»=\ _1,ā¦, _k\. For a given LLM Ī»j _j, the output is Ī»jā(xi)ā0,1 _j(x_i)ā\0,1\, where 11 corresponds to an āEPU " label and 00 corresponds to a āNo EPU " label. Since different LLM s may assign different labels to the same input xix_i, the second stage focuses on aggregating the k generated synthetic labels to produce a less noisy single label. To accomplish this, various ensemble methods are employed which are detailed in Section 3.1.1. Label Generation Prompt You are a classifier that determines whether a given news article or passage discusses Economic Policy Uncertainty (EPU). Your decision process for each passage should be: 1. Check if the article mentions general economic uncertainty. - Examples include: job market fluctuations, inflation concerns, recession fears. - If the article only discusses general economic uncertainty but does not link it to government policy, classify as EPU=0. 2. Check if the article explicitly connects economic uncertainty to government policy actions, decisions, or inactions. - Mentions of policy-related economic uncertainty in the past, present, or future all qualify. - If this condition is met, classify as EPU=1. Further indicators that qualify for EPU=1 classification include uncertainty related to policy decisions in any of the following areas: - Monetary policy (e.g., central bank actions, interest rates, money supply) - Fiscal policy (e.g., government spending, tax reforms) - Regulations (e.g., labor laws, environmental policies, financial regulations) - Trade policy (e.g., tariffs, trade agreements, export/import restrictions) - Government leadership uncertainty (e.g., elections, political conflicts) - Sovereign debt and currency stability (e.g., exchange rate policies, foreign reserves) - National security and terrorism (if linked to economic uncertainty) - Social safety nets and entitlement programs (e.g., welfare, healthcare policies) Output: - If policy-related economic uncertainty is present ā EPU=1. - Otherwise ā EPU=0. Figure 1: EPU label generation prompt. 3.1.1 Weak Label Aggregation In this stage, various aggregation methods including: Data Programming (DP), Majority Vote, Worker Agreement with Aggregate (WAWA), and the Dawid-Skene model, are used to consolidate multiple noisy labels generated by different LLM s into a single label. ā¢ Majority Vote: Majority Vote Penrose (1946) is a simple yet powerful model often used in ensemble learning techniques. Consider a setting with k classifiers in our case the output of k LLM s on the same input text, each providing a vote for one of the possible classes ciāCc_iā C in a classification task. The Majority Vote is determined by aggregating these votes and selecting the class with the highest number of votes. If we let vjāciv_jc_i to be a binary indicator where vjāci=1v_jc_i=1 if classifier Ī»j _j votes for class cic_i, and vjāci=0v_jc_i=0 otherwise. The total number of votes for class cic_i is then given by: Vci=āj=1kvjāciV_c_i= _j=1^kv_jc_i. The predicted class y y is the one with the maximum VciV_c_i shown in Equation 1. y^=argā”maxciā”Vci y= _c_iV_c_i (1) This method is based on the idea that combining outputs from multiple classifiers can produce more accurate results, especially when each classifier brings different and complementary strengths to the task. ā¢ DP: DP Ratner et al. (2016) is an aggregation method that assigns a noisy label to each news article in an unsupervised learning manner. The outputs generated by LLM s through prompting can be viewed as multiple annotators labeling the same article, similar to a crowd-sourcing setup Mishra et al. (2021). These labels may contain conflicts and correlations, even when assigned by domain experts. Instead of a simple Majority Vote, DP employs a probabilistic framework to capture the structure and relationships within the label matrix. The probabilistic model PĪøā(Ī,Y)P_Īø( ,Y) is defined as the joint probability of the LLM sā outputs (label matrix) Ī and the latent (unobserved) true class labels Y. The label matrix is encoded using a factor model with three factor types representing conflicts, correlations, and propensity (instances where LLM s generate valid labels). The generative model is expressed in Equation 2. PĪøā(Ī,Y)=ZĪøā1āeāxāpā(āi=1nĪøTāĻiā(Īi,yi))P_Īø( ,Y)=Z_Īø^-1exp( _i=1^nĪø^T _i( _i,y_i)) (2) where ZĪøZ_Īø is the normalizing constant, Ļiā(Īi,yi) _i( _i,y_i) represents the concatenated feature vector derived from all LLM outputs for a given sample news article xix_i, and Īø denotes the parameter vector. The estimate of the true parameter denoted as Īø Īø, is obtained by minimizing the negative marginal likelihood using the observed label matrix Ī , relying solely on the patterns of agreement and disagreement within Ī since the true labels are unavailable. The estimated parameters are then used to produce probabilistic noisy labels: Y^=PĪø^ā(Yā£Ī) Y=P_ Īø(Y ), which can optionally serve as training labels for downstream models. ā¢ WAWA: The WAWA algorithm Ustalov et al. (2024) aggregates noisy labels through a three-step process. It begins by determining the Majority Vote label for each task. Then, it assesses each workerās skill by measuring the proportion of their responses that align with the Majority Vote. Finally, it computes a weighted Majority Vote, where the weights are derived from these skill estimates. The algorithm is based on the premise that responses agreeing with the Majority Vote are more likely to be accurate, and that a workerās reliability can be inferred from the consistency of their answers with the majority. ā¢ Dawid-Skene: The Dawid-Skene aggregation model Dawid and Skene (1979) is a probabilistic method used to estimate the expertise level of workers (in this case LLM s) in a crowd-sourcing environment. This model evaluates the reliability of each worker by constructing a confusion matrix that captures the probability of correctly or incorrectly labeling. The confusion matrix provides insight into how consistently a worker assigns labels in a classification problem. In addition to individual worker reliability, Dawid-Skene incorporates a prior class probability vector, which represents the likelihood of each class occurring before considering the responses provided by workers. Since the true label of a task is unknown, it is treated as a latent variable that the model seeks to estimate. The probability of a task belonging to a specific class is influenced by the prior probabilities and the observed responses from workers. The optimization of these parameters, including the prior class probabilities, worker confusion matrices, and latent variables representing true labels, is achieved using the expectation maximization algorithm. 3.1.2 Discriminative Model The aggregated labels were used to fine-tune pre-trained LLM s: BERT and RoBERTa, and to train an LSTM model. For pre-trained LLM s, this was achieved by adding a feedforward neural network and a classification head to the last layers while keeping the other pre-trained layers frozen to facilitate adaptation to our downstream EPU classification task. For an LSTM, the news article text was first tokenized and converted into sequences of integers. These sequences were then mapped into dense vector representations using an embedding layer, which captures semantic relationships between words. Initially, these embeddings are randomly initialized with a uniform distribution and are progressively updated during training. The transformed embeddings are then fed into the LSTM layers, which learns to identify sequential dependencies and contextual relationships within the text. For both LLM s and an LSTM, the final layer consists of a dense network with a softmax function, which assigns the text to the appropriate EPU category. The training process optimizes the model by minimizing the expected loss over noisy labels formulated in Equation 3 Ratner et al. (2016). Īø^=argā”minā”āi=1nĪøā”yi^ā¼Y^ā[āā”(hĪøā(xi),yi^)] Īø= _Īø _i=1^nE_ y_i Y[L(h_Īø(x_i), y_i)] (3) where Īø Īø represents the estimated model parameters, n is the number of training examples, E denotes expectation, āL is the expected loss with respect to Y Y, hĪøā(xi)h_Īø(x_i) is the output predicted by the discriminative model, xix_i is a training instance, and yi y_i is the corresponding noisy probabilistic label from Y Y. 3.1.3 Multi-label Classification for EPU Articles For articles positive for EPU, this section discusses the methodology used to identify specific subcategories, with their class distribution detailed in Table 2. In contrast to the binary EPU classification task, where each article receives a single label, the subcategories here may simultaneously belong to multiple classes selected from a predefined set: a,b,ā¦,p\a,b,ā¦,p\. Each article is a multi-hot vector sis_i, indicating the presence or absence of each corresponding subcategory. In this section, experiments relied solely on human labeled data rather than synthetic labels. This decision stemmed from the absence of detailed guidelines in the labeling guide for assigning multiple labels to news articles, unlike the case with EPU classification. Consequently, it was challenging to develop clear and effective instructions for a reliable prompt-based method comparable to that used in EPU classification. We used the same models as in EPU classification: BERT, RoBERTa, and LSTM. The text processing approach remained similar to that used for EPU classification, but the subcategories were converted into multi-hot vectors, denoted as sis_i. Both models were trained using binary CE loss across all subcategories defined in Equation 4. āmulti(i,^i)=ā1nāi=1nāj=1m[siājlog(s^iāj)+(1āsiāj)log(1ās^iāj)]L_multi(s_i, s_i)=- 1n _i=1^n _j=1^m [s_ij ( s_ij)+(1-s_ij) (1- s_ij ) ] (4) where siājs_ij represents the true label for subcategory j of instance i, where 1 indicates that the article belongs to subcategory j and 0 indicates it does not. The term s^iāj s_ij is the predicted probability that the article belongs to subcategory j, outputted by the model. 3.1.4 Hierarchical Classification for EPU Articles Beyond multi-label classification, we performed hierarchical classification, where the classification task follows a structured dependency between an articleās general classification as EPU-related and its assignment to multiple subcategories. Unlike standard multi-label classification, where labels are treated as independent, hierarchical classification ensures that subcategories are only assigned when an article is identified as EPU-related. The dataset is now structured as: (xi,yi,i)i=1n\(x_i,y_i,s_i)\_i=1^n, where xix_i represents a news article, yiā0,1y_iā\0,1\ is a binary label indicating whether the article discusses EPU, and is_i is a multi-hot vector indicating the presence or absence of multiple subcategories. We experimented with various hierarchical classification approaches: basic summation, masking, weighted, and a two-stage, discussed next. ā¢ Basic Summation: A single model is trained to predict both yiy_i and is_i simultaneously. It consists of a shared encoder followed by two classification heads: one for binary EPU classification and another for multi-label subcategory classification. For a given input text xix_i, the model processes it through the shared encoder and applies separate linear classifiers for each prediction. The loss function used for training is defined in Equation 5. ābasicā(yi,y^i,i,^i)=ābinaryā(yi,y^i)+āmultiā(i,^i),L_basic(y_i, y_i,s_i, s_i)=L_binary(y_i, y_i)+L_multi(s_i, s_i), (5) where āmultiL_multi is a mult-label loss, and ābinaryL_binary is a binary CE loss for predicting yiy_i defined in Equation 6. ābinary(yi,y^i)=ā1nāi=1n[yilogy^i+(1āyi)log(1āy^i)]L_binary(y_i, y_i)=- 1n _i=1^n [y_i y_i+(1-y_i) (1- y_i) ] (6) ā¢ Masking: To enforce dependency constraints between binary and multi-label classification, this approach incorporates a masking mechanism. The loss function for subcategory prediction is applied only when yi=1y_i=1, that is when the article is classified as EPU-related. The modified loss function is defined in Equation 7. āmasking(yi,y^i,i,^i)=ābinary(yi,y^i)+[yi=1]ā āmulti(i,^i),L_masking(y_i, y_i,s_i, s_i)=L_binary(y_i, y_i)+ 1[y_i=1]Ā·L_multi(s_i, s_i), (7) where [yi=1] 1[y_i=1] is an indicator function that applies the multi-label loss only to articles classified as EPU. ā¢ Weighted: Building upon the masking-based approach, this method further modifies the multi-label loss by introducing a weighting scheme to handle cases where articles are mistakenly classified as non-EPU but contain some subcategory information. Instead of completely ignoring subcategory loss for yi=0y_i=0, different penalty weights are assigned for false positives and false negatives. The weighted loss function āWL_W is defined in Equation 8. āW(yi,y^i,i,^i)=ābinary(yi,y^i)+Ī»pos[yi=1]āmulti(i,^i)+Ī»neg[yi=0]āmulti(i,^i)L_W(y_i, y_i,s_i, s_i)=L_binary(y_i, y_i)+ _pos 1[y_i=1]L_multi(s_i, s_i)+ _neg 1[y_i=0]L_multi(s_i, s_i) (8) where Ī»pos _pos and Ī»neg _neg are hyperparameters controlling the relative penalty for subcategory errors when yiy_i is 1 versus 0. ā¢ Two-stage: Instead of jointly predicting yiy_i and is_i, this approach separates the task into two stages: binary classification and multi-stage classification, trained separately unlike in the masking approach. Binary Classification Stage: The first stage is a binary classifier trained solely on predicting whether an article discusses EPU. The model is optimized using the CE loss ābinaryā(yi,y^i)L_binary(y_i, y_i) defined in Equation 6. Multi-Label Classification Stage: After training the binary classifier, its predicted labels y^i y_i are used to select articles for the second-stage multi-label classifier, which is applied only to samples where yi^=1 y_i=1. The loss function āmultiā(i,^i)L_multi(s_i, s_i) is the same as in standard multi-label classification but limited to this filtered subset. 3.2 Data We used a dataset of 12,00012,000 news articles provided by Baker et al. (2016), selected from USA news articles that included the terms āeconomyā and āuncertaintyā. Domain experts assigned binary labels to these articles based on whether they discussed EPU. Among these, 49394939 were identified as related to EPU. These articles were then classified into one or more multi-label subcategories, which explains why the total count in the class distribution shown in Table 2 is 60376037 instead of 49394939, as some articles fall into multiple subcategories. The subcategories include: government spending (entitlement programs, social safety nets, welfare programs), trade policy (tariffs, quotas, trade rules), financial regulation (banking and equity markets), energy and environmental regulation (natural resources, commodities), sovereign debt (exchange rate policy, foreign reserves), political conflict and leadership changes (elections, governance crises), and legal policy (judicial decisions, regulatory changes). Further details about the dataset can be obtained from the authorsā website11 1 https://w.policyuncertainty.com Baker et al. (2016). Table 2: Distribution of subcategories for news articles describing EPU Subcategory Count Other policy matters that do not fit into pre-defined categories 1176 Political conflict and leadership changes 827 National security and terrorism 820 Trade policy (tariffs, quota, trade rules) 712 Taxes 659 Monetary policy 372 Government spending 367 Energy & environmental regulation, natural resources and commodities 187 Competition policy 182 Labor regulations 166 Legal policy 164 Sovereign debt, exchange rate policy, foreign reserves 127 Fiscal policy 125 Financial regulation (including banking and equity markets) 100 Entitlement programs, social safety net, welfare programs 46 Health care programs and regulations 7 Grand Total 6037 3.3 Experimental Setup and Evaluation All experiments were conducted in Python with BERT and RoBERTa being implemented using the Huggingface Transformer library 22 2 https://huggingface.co/. LSTM Hochreiter and Schmidhuber (1997) was implemented using the Keras library 33 3 https://keras.io/ Ketkar (2017). For the hyperparameters, the maximum text length was set to 512512 tokens, the Adam optimizer was used, with a learning rate of 1Ć10ā31Ć 10^-3, a batch size of 5050, and training was performed over 1010 epochs. All model experiments were conducted on a single A100 40GB GPU machine. For synthetic label generation, we primarily utilized generative LLM s, specifically open-source models from the LLAMA, Mistral, and Qwen families, using the prompt shown in Figure 1. The selected models represented the SoTA at the time of the experiments, that we could perfom inference on with the hardware that was available to us. The specific versions used in the experiments include: Mistral-24B , Qwen2-7B , LLAMA3-8B , Mistral3-7B , and Qwen2-14B . Binary classification performance was evaluated using accuracy, F1 , Geometric Mean (GM), Area Under the Curve (AUC), and Area Under the Precision Recall Curve (AUPRC). Accuracy represents the overall proportion of correctly classified instances. F1 is the harmonic mean of precision and recall. GM accounts for both sensitivity and specificity, suitable under class imbalance setting. AUC measures the modelās ability to distinguish between classes across different threshold settings, while AUPRC assesses precision-recall trade-offs, making it particularly relevant for imbalanced data. Multi-label classification performance was further assessed using Exact Match Ratio (EMR) and Label Ranking Precision (LRP) Zhang and Zhou (2013). EMR calculates the percentage of instances where the predicted set of labels exactly matches the true set. LRP evaluates how well the model ranks correct labels higher than incorrect ones, focusing on the order of label relevance rather than requiring an exact match. In addition to classification metrics, we used Cohenās Kappa (CK) and Matthews Correlation Coefficient (MCC) to measure the correlation and agreement of different label generators with humans. CK Hsu and Field (2003) evaluates the level of agreement between the predicted and actual labels while accounting for the possibility of random agreement. MCC Chicco and Jurman (2023), on the other hand, provides a balanced measure of correlation between predictions and ground truth, even when class distributions are uneven by considering all elements of the confusion matrix. The code, datasets, and hyperparameters referenced in this research are available in the GitHub repository: https://github.com/TrustPaul/Weak-supervision-in-Economic-Policy-Uncertanity. 4 Results and Discussion This section presents the results of the various experiments for binary EPU classification, as well as multi-label and hierarchical classification. Table 3: Evaluation metrics (CK and MCC) for different label generators for the binary EPU classification task Label Generator CK MCC Majority Vote -0.0121 -0.0139 WAWA -0.0011 -0.0013 Mistral3-7B 0.0460 0.0553 LLAMA3-8B 0.2970 0.3058 Qwen2-14B 0.2849 0.3486 Qwen2-7B 0.2849 0.3486 Mistral-24B 0.3537 0.3766 DP 0.4156 0.4156 Dawid-Skene 0.4813 0.4908 Analysis of Different Label Generators: Table 3 presents evaluation metrics (MCC and CK) comparing the performance of models of different label generators. The evaluation includes individual LLM s such as Mistral-24B, Qwen2-14B, Qwen2-7B, LLAMA3-8B, and Mistral3-7B, as well as aggregation methods like Dawid-Skene, WAWA, and Majority Vote. Dawid-Skene achieved the highest correlation with human labels, with an MCC of 0.4908 and a CK of 0.4813, followed by the DP method, which obtained an MCC and CK of 0.4156. Among individual LLM s, Mistral-24B showed the strongest correlation with human labels (MCC = 0.3766, CK = 0.3537), while Majority Vote had the lowest correlation. These results indicate that proper aggregation or the use of more effective models leads to higher agreement with human labels, and the high values of MCC and CK observed especially by top methods like Dawid-Skene, DP and Mistral-24B suggest that these labels can be reliably used for training other LLM s for this task. Table 4: Performance evaluation metrics (F1, accuracy, and GM) for different label generators for the binary EPU classification task Label Generator Accuracy F1 GM Mistral3-7B 42.06 27.36 41.29 Majority Vote 55.25 46.65 49.44 WAWA 55.41 47.69 49.95 LLAMA3-8B 64.15 42.85 56.09 Qwen2-14B 68.89 61.46 62.85 Qwen2-7B 68.89 61.46 62.85 Dawid-Skene 75.01 49.30 63.12 Mistral-24B 70.72 66.80 66.61 DP 71.83 70.78 70.77 Beyond correlation and agreement metrics, classification performance is of greater interest for this task. Table 4 therefore presents the performance of various data generators without additional training. For LLM s, this corresponds to zero-shot prompting performance. DP achieved the highest performance results, with a GM of 70.77%70.77\% and an F1 score of 70.78%70.78\%, followed by Mistral-24B, which obtained a GM of 66.61%66.61\% and an F1 score of 66.80%66.80\%. The lowest performance was observed with Mistral3-7B and the Majority Vote method. Based on these results, labels generated by DP were selected as synthetic labels for training other models. The findings indicate that ensembling improves performance, but the choice of aggregation method significantly impacts the outcome. The zero-shot performance of LLM s like Mistral-24B suggests that sufficiently large and high-performing models can achieve reasonable classification results through prompting alone. However, even in cases where zero-shot prompting performs well, achieving such results requires models with at least 7 billion parameters, which come with significant inference costs. Therefore, it remains beneficial to use labels generated by these LLM s or aggregation methods in a distillation process to train smaller models, such as BERT, which offer lower training and inference costs. Fine-tuning results for binary classification of EPU: Table 5 presents the performance of different models along with their training data sources, which include human-labeled data and weak supervision (WS) where labels are generated through aggregation. The models compared are LSTM, BERT, and RoBERTa. In general, models perform better when trained on human labeled data. This is particularly evident with BERT, which achieves an F1 score of 74.66%74.66\% and a GM of 73.77%73.77\% when trained on human labeled data, compared to an F1 score of 71.92%71.92\% and a GM of 71.66%71.66\% when trained on synthetic labels. A similar trend is observed with RoBERTa, which performs almost as well as BERT but achieves slightly higher F1 and GM scores when trained on synthetic data. LSTM has the lowest performance on both datasets, showing only minor differences in F1 and GM scores between human labeled and synthetic data. Table 5: Performance comparison of different models and datasets on EPU binary classification. Dataset WS indicates that labels generated by DP were used to train the respective model. Dataset Model Accuracy F1 GM AUC AUPRC Human BERT 75.99 74.66 73.77 0.8423 0.7750 Human RoBERTa 74.97 73.54 72.59 0.8440 0.7833 Human LSTM 63.41 60.79 58.96 0.6452 0.5352 WS DP 71.83 70.78 70.55 0.7338 0.6654 WS LSTM 63.46 62.10 61.66 0.6482 0.5359 WS BERT 71.93 71.33 71.83 0.7859 0.6877 WS RoBERTa 73.03 71.92 71.66 0.7936 0.7114 The results in Table 5 further reveal that models trained on synthetically generated data such as RoBERTa, outperform the zero-shot prompting performance of the LLM used to generate the synthetic data. RoBERTa achieves a higher GM and F1 score than DP and also exceeds the best zero-shot prompting LLM: Mistral-24B, in terms of GM (71.66%71.66\% versus 66.61%66.61\%). Although the performance of zero-shot prompting and aggregation methods is close to that of fine-tuned models, the competitive results are mainly achieved using extremely large LLM s, as previously noted. Despite this, using LLM s to generate synthetic labels for training smaller models remains a promising approach, as human annotation is costly and time-consuming, making synthetic data generation a more efficient and scalable alternative. Figure 2: Receiver Operating Curves (ROC) curves for different models for binary EPU classification task. WS in the case means that labels generated by DP were used to train the respective model. In addition to the classification metrics in Table 5, Figure 2 presents the ROC curves for different models. The trends in the ROC align with those observed in the F1 and GM scores. Models trained on human labeled data generally achieve higher AUC values, with both RoBERTa and BERT reaching approximately 0.840.84. In contrast, LSTM shows consistently lower AUC values, with little difference between models trained on weak supervision and those trained on human labeled data. Table 6: Performance metrics for different models on the multi-label text classification task Model EMR LRP AUPRC AUC LSTM 58.15 75.40 0.0466 0.5931 BERT 61.70 83.70 0.1634 0.7727 RoBERTa 60.43 83.78 0.1676 0.7930 Multi-Label Text Classification: For EPU binary classification, articles identified as positive for EPU were further assigned to 16 multi-label categories, with the class distribution detailed in Table 2. For multi-label classification, only human labeled data was used for fine-tuning. The models evaluated include: RoBERTa, BERT, and LSTM, assessed using multi-label evaluation metrics such as EMR, LRP, AUC, and AUPRC shown in Table 6. In terms of EMR, all three models exhibited comparable performance, with BERT achieving the highest score. LSTM recorded an EMR of 58.15%58.15\%, while BERT and RoBERTa attained 61.70%61.70\% and 60.43%60.43\%, respectively. For the LRP metric, BERT and RoBERTa performed similarly, both reaching approximately 83.70%83.70\%, whereas LSTM had a lower LRP score of 75.40%75.40\%. The AUPRC values were generally low across all models, with RoBERTa obtaining the highest AUPRC and LSTM the lowest. In terms of AUC, RoBERTa achieved the highest value of 0.79300.7930, while LSTM had the lowest at 0.59310.5931. These results indicate that pre-trained LLM s such as BERT and RoBERTa perform better in multi-label classification compared to LSTM. However, their performance is significantly lower than in binary classification, which is expected since multi-label classification is a more complex task. Table 7: Performance evaluation of various models and hierarchical strategies on the binary EPU classification task Model Hierarchical Strategy Accuracy F1 AUC AUPRC GM BERT basic 74.81 74.33 83.86 0.7727 74.54 masking 74.72 74.30 84.16 0.7724 74.58 two-stage 74.64 74.33 83.80 0.7641 74.80 weighted 75.62 74.88 83.93 0.7741 74.90 RoBERTa basic 76.03 75.48 84.68 0.7895 75.60 masking 76.19 75.59 84.81 0.7905 75.66 two-stage 76.15 75.40 84.81 0.7891 75.41 weighted 75.91 75.22 84.15 0.7801 75.25 LSTM basic 61.62 58.66 61.93 0.5069 59.08 masking 62.31 57.93 61.03 0.5023 59.21 two-stage 61.86 57.95 59.18 0.4644 58.89 weighted 62.15 56.15 60.56 0.4967 58.88 Hierarchical EPU Classification: Since the task was hierarchical, where articles classified as EPU could be further assigned to 16 multi-label categories, we investigated multiple strategies for hierarchical text classification. Details on these methods are provided in Section 3.1.4 of the methodology. The strategies include a basic approach, where the model predicts both binary and multi-label classifications simultaneously, a masking approach, where multi-label predictions are made only for articles classified as EPU, a weighted approach, which applies an additional loss penalty for misclassifying articles as not containing EPU, and a two-stage approach, where two models are trained separately, one for binary classification and another for multi-label classification. Table 7 presents the binary EPU classification results for different models across these approaches. Overall, performance differences among the four methods are minimal across all models and evaluation metrics. Among the models, RoBERTa achieved the highest GM of 75.66%75.66\% using the masking approach. BERT performed best with the two-stage approach, reaching a GM of 74.80%74.80\%. For LSTM, the highest GM was obtained using the masking approach. Table 8: Performance evaluation of various models and hierarchical strategies on the multi-label classification task Model Hierarchical Strategy EMR AUPRC AUC LRP BERT basic 59.45 0.0702 0.7080 74.41 masking 59.45 0.0811 0.7189 74.10 two-stage 61.94 0.1974 0.7526 81.21 weighted 59.45 0.0678 0.7046 73.53 RoBERTa basic 59.45 0.0765 0.7070 74.39 masking 59.45 0.0963 0.7219 75.22 two-stage 61.66 0.2231 0.7745 81.02 weighted 59.45 0.0679 0.6999 73.48 LSTM basic 59.45 0.0395 0.5879 68.48 masking 59.45 0.0399 0.5807 67.55 two-stage 59.25 0.0599 0.5919 67.77 weighted 59.45 0.0377 0.5620 66.90 Hierarchical Multi-label Text Classification: Table 8 presents the EMR, LRP, AUPRC, and AUC metrics for the three models across the four hierarchical approaches. The EMR values remain nearly identical across all approaches. For LRP, the best performance is achieved by BERT using the two-stage approach, followed by RoBERTa again under the same approach with an LRP of 81.02%81.02\%. LSTM has consistently lower LRP values overall, but its best result is obtained with the basic approach at 68.48%68.48\%. RoBERTa achieves the highest AUC value of 0.77450.7745 using the two-stage approach. 5 Conclusion This research investigated the application of LLM s for EPU classification, addressing limitations of earlier approaches based on keyword lists, unsupervised ML, or supervised methods dependent on costly human annotations. To reduce reliance on manual labeling, we employed weak supervision by generating labels with LLM s, where large models such as Mistral-24B achieved the strongest performance in zero-shot prompting for label generation, and aggregation methods like DP and DawidāSkene further enhanced label quality. While models trained on human labeled data achieved the highest accuracy, the gap to weakly supervised models was relatively small, indicating that LLM-based labeling offers a practical and scalable alternative. Finally, extending beyond binary detection, we examined multi-label and hierarchical classification, finding that fine-tuned pre-trained LLM s delivered the best performance, with the two-stage strategy most effective for policy subcategory classification, though overall differences across hierarchical methods were minor. ā¦ l References Audrino et al. (2024) F. Audrino, J. Gentner, and S. Stalder Quantifying Uncertainty: A New Era of Measurement through Large Language Models. Technical report Technical Report 24-68, Swiss Finance Institute Research Paper Series, Swiss Finance Institute. External Links: Link Cited by: Ā§1, Table 1, Ā§2. Azqueta-GavaldĆ³n et al. (2023) A. Azqueta-GavaldĆ³n, M. Diakonova, C. Ghirelli, and J. J. PĆ©rez Sources of Economic Policy Uncertainty in the Euro Area: A Ready-to-use Database. Occasional Paper Bank of Spain. External Links: Document, Link Cited by: Ā§1, Table 1, Ā§2. Azqueta-Gavaldon et al. (2020) A. Azqueta-Gavaldon, D. HirschbĆ¼hl, L. Onorante, and L. Saiz Economic Policy Uncertainty in the Euro Area: an Unsupervised Machine Learning Approach. Working Paper Series Technical Report 2359, Vol. 158, European Central Bank. External Links: Link Cited by: Ā§1, Table 1, Ā§2. Azqueta-GavaldĆ³n (2017) A. Azqueta-GavaldĆ³n Developing News-Based Economic Policy Uncertainty Index with Unsupervised Machine Learning. Economics Letters 158, p. 47ā50. External Links: Link Cited by: Ā§1, Table 1, Ā§2. Baker et al. (2016) S. R. Baker, N. Bloom, and S. J. Davis Measuring Economic Policy Uncertainty. The Quarterly Journal of Economics 131, p. 1593ā1636. Cited by: Ā§1, Table 1, Ā§3.1, Ā§3.2. CHANG et al. (2024) D. CHANG, N. HU, P. LIANG, and M. SWINK Understanding the Impact of Trade Policy Effect Uncertainty on Firm-Level Innovation Investment: A Deep Learning Approach. Journal of Operations Management 70, p. 316ā340. Cited by: Ā§1, Table 1, Ā§2. Chen et al. (2024) C. Chen, Y. Huang, and F. Yang Semantics Matter: An Empirical Study on Economic Policy Uncertainty Index. International Review of Economics & Finance 89, p. 1286ā1302. Cited by: Ā§1, Table 1, Ā§2. Chicco and Jurman (2023) D. Chicco and G. Jurman The Matthews Correlation Coefficient (MCC) Should Replace the ROC AUC as the Standard Metric for Assessing Binary Classification. Bio Data Mining 16, p. 4. Cited by: Ā§3.3. Dawid and Skene (1979) A. P. Dawid and A. M. Skene Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm. Journal of the Royal Statistical Society: Applied Statistics 28, p. 20ā28. Cited by: 4th item. Gentzkow et al. (2019) M. Gentzkow, B. Kelly, and M. Taddy Text as Data. Journal of Economic Literature 57, p. 535ā74. Cited by: Ā§1. Grimmer and Stewart (2013) J. Grimmer and B. M. Stewart Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts. Political Analysis 21, p. 267ā297. Cited by: Ā§1. Hochreiter and Schmidhuber (1997) S. Hochreiter and J. Schmidhuber Long Short-Term Memory. Neural Computation 9, p. 1735ā1780. Cited by: Ā§3.3. Hsu and Field (2003) L. M. Hsu and R. Field Interrater Agreement Measures: Comments on Kappan, Cohenās Īŗ, Scottās Ļ, and Aickinās Ī±. Understanding Statistics 2, p. 205ā219. Cited by: Ā§3.3. Indaco (2020) A. Indaco From Twitter to GDP: Estimating Economic Activity from Social Media. Regional Science and Urban Economics 85, p. 1ā32. Cited by: Ā§1. Ito et al. (2025) A. Ito, M. Sato, and R. Ota Content-based Metric on Monetary Policy Uncertainty by using Large Language Models. Finance Research Letters 75, p. 106832. External Links: ISSN 1544-6123, Document, Link Cited by: Ā§1, Table 1, Ā§2. Jiang et al. (2023) J. Jiang, M. Feng, and K. Srinivasan Can Conversations on Reddit Forecast Future Economic Uncertainty? An Interpretable Machine Learning Approach. In Proceedings of the 44th International Conference on Information Systems (ICIS 2023), Hyderabad, India, p. 1975ā1990. External Links: Link Cited by: Ā§1, Table 1, Ā§2. Kaveh-Yazdy and Zarifzadeh (2021) F. Kaveh-Yazdy and S. Zarifzadeh Measuring Economic Policy Uncertainty Using an Unsupervised Word Embedding-Based Method. ArXiv Preprint arXiv:2105.04631. Cited by: Ā§1, Table 1. Keith et al. (2020) K. A. Keith, C. Teichmann, B. OāConnor, and E. Meij Uncertainty over Uncertainty: Investigating the Assumptions, Annotations, and Text Measurements of Economic Policy Uncertainty. In Proceedings of the Fourth Workshop on Natural Language Processing and Computational Social Science, p. 116ā131. External Links: Link, Document Cited by: Ā§1, Table 1, Ā§2. Keith and Stent (2019) K. Keith and A. Stent Modeling Financial Analystsā Decision Making via the Pragmatics and Semantics of Earnings Calls. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, p. 493ā503. External Links: Link, Document Cited by: Ā§1. Ketkar (2017) N. Ketkar Introduction to Keras. In Deep Learning with Python: A Hands-on Introduction, p. 97ā111. Cited by: Ā§3.3. LoliÄ et al. (2022) I. LoliÄ, P. SoriÄ, and M. LogaruÅ”iÄ Economic Policy Uncertainty Index Meets Ensemble Learning. Computational Economics 60, p. 401ā437. Cited by: Ā§1, Table 1, Ā§2. Miranda-Belmonte et al. (2023) H. U. Miranda-Belmonte, V. MuƱiz-SĆ”nchez, and F. Corona Word Embeddings for Topic Modeling: An Application to the Estimation of the Economic Policy Uncertainty Index. Expert Systems with Applications 211, p. 118499. Cited by: Ā§1, Table 1, Ā§2. Mishra et al. (2021) S. Mishra, D. Khashabi, C. Baral, and H. Hajishirzi Cross-task Generalization via Natural Language Crowdsourcing Instructions. ArXiv Preprint arXiv:2104.08773. Cited by: 2nd item. Nyman and Ormerod (2020) R. Nyman and P. Ormerod Text as Data: A Machine Learning-based Approach to Measuring Uncertainty. ArXiv Preprint arXiv:2006.06457. Cited by: Ā§1, Table 1, Ā§2. Penrose (1946) L. S. Penrose The Elementary Statistics of Majority Voting. Journal of the Royal Statistical Society 109, p. 53ā57. Cited by: 1st item. Qureshi et al. (2021) S. Qureshi, B. Chu, F. S. Demers, and M. Demers Using Natural Language Processing to Measure COVID-19-Induced Economic Policy Uncertainty for Canada and the USA. In International Conference on Time Series and Forecasting, p. 107ā121. Cited by: Ā§1, Table 1, Ā§2. Rao et al. (2024) A. Rao, M. Tedeschi, K. S. Mohammed, and U. Shahzad Role of Economic Policy Uncertainty in Energy Commodities Prices Forecasting: Evidence from a Hybrid Deep Learning Approach. Computational Economics 64, p. 3295ā3315. Cited by: Ā§1, Table 1, Ā§2. Ratner et al. (2016) A. J. Ratner, C. M. De Sa, S. Wu, D. Selsam, and C. RĆ© Data Programming: Creating Large Training Sets, Quickly. In Advances in Neural Information Processing Systems, Vol. 29, p. 1ā9. Cited by: 2nd item, Ā§3.1.2. Rybinski et al. (2019) K. Rybinski et al. A Machine Learning Framework for Automated Analysis of Central Bank Communication and Media Discourse: The Case of Narodowy Bank Polski. Bank i Kredyt 1, p. 1ā20. Cited by: Ā§1. Sadorsky (2022) P. Sadorsky Using Machine Learning to Predict Clean Energy Stock Prices: How Important are Market Volatility and Economic Policy Uncertainty?. Journal of Climate Finance 1, p. 100002. Cited by: Ā§1, Table 1, Ā§2. Tobback et al. (2018) E. Tobback, H. Naudts, W. Daelemans, E. J. de Fortuny, and D. Martens Belgian Economic Policy Uncertainty Index: Improvement through Text Mining. International Journal of Forecasting 34, p. 355ā365. Cited by: Ā§1, Table 1, Ā§2. Trust et al. (2023) P. Trust, A. Zahran, and R. Minghim Understanding the Influence of News on Society Decision Making: Application to Economic Policy Uncertainty. Neural Computing and Applications 35, p. 14929ā14945. Cited by: Ā§1, Ā§2, Ā§3.1. Ustalov et al. (2024) D. Ustalov, N. Pavlichenko, and B. Tseitlin Learning from Crowds with Crowd-Kit. Journal of Open Source Software 9, p. 6227ā6235. External Links: Document, Link Cited by: 3rd item. Wang et al. (2022) M. D. Wang, J. Lou, D. Zhang, and C. S. Fan Measuring Political and Economic Uncertainty: a Supervised Computational Linguistic Approach. SN Business & Economics 2, p. 1ā17. Cited by: Ā§1, Table 1, Ā§2. Wang et al. (2024) S. Wang, X. Yang, X. Zhu, and X. Ge The Impacts of Economy Policy Uncertainty on Peer Effects of Firms R&D Investment: Based on LDA Machine Learning and Regression Statistical Modeling Approach. PLOS ONE 19, p. 1ā23. External Links: Document, Link Cited by: Ā§1, Table 1, Ā§2. Xu et al. (2023) W. Xu, W. Rao, L. Wei, and Q. Wang A Normalized Global Economic Policy Uncertainty Index From Unsupervised Machine Learning. Mathematics 11, p. 3268. Cited by: Ā§1, Table 1, Ā§2. Yeh et al. (2024) H. Yeh, Y. Huang, Z. Park, and C. Chen Automation of Text-Based Economic Indicator Construction: A Pilot Exploration on Economic Policy Uncertainty Index. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, p. 4213ā4217. Cited by: Ā§1, Table 1, Ā§2. Zhang and Zhou (2013) M. Zhang and Z. Zhou A Review on Multi-label Learning Algorithms. IEEE Transactions on Knowledge and Data Engineering 26, p. 1819ā1837. Cited by: Ā§3.3.