Paper deep dive

LLM-Guided Contextual Action Evaluation for Operational Decisions in Industrial Processes

Youcheng Zong, Runda Jia, Dakuo He

Year: 2026Venue: arXiv preprintArea: eess.SYType: PreprintEmbeddings: 48

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 8/29/2026, 4:05:46 AM

Summary

The paper introduces LLM-Guided Contextual Action Evaluation (LCAE), a method for industrial process control that integrates fixed industrial document semantics with real-time numerical history. An LLM pre-processes documents into frozen action-observation-direction-delay relation cards, which form a semantic basis. A numerical actor-critic system then uses recent action-response history to modulate the strength of these relations, creating a state-conditioned action-effect field. This approach allows the critic to evaluate actions based on documented causal structures and current context, while the actor generates actions using the same relation gains, improving sample efficiency and decision bias without online LLM inference.

Entities (6)

LCAE · method · 98%Actor-Critic · algorithm · 95%Industrial Process Control · application-domain · 93%LLM · model · 92%Relation Cards · data-structure · 90%Action-Effect Field · concept · 88%

Relation Signals (5)

LCAE → uses → LLM

confidence 95% · This article presents LLM-Guided Contextual Action Evaluation... which uses a large language model before training to normalize fixed documents...

LCAE → appliesto → Industrial Process Control

confidence 93% · This article presents LLM-Guided Contextual Action Evaluation for Operational Decisions in Industrial Processes...

LLM → generates → Relation Cards

confidence 92% · The LLM runs only before policy training and produces a set of relation cards... Each card specifies an action field, an affected visible observation...

Actor-Critic → isimprovedby → LCAE

confidence 90% · LCAE builds on this interface: it retains maximum-entropy actor–critic learning but changes the action representation received by the critic...

Relation Cards → form → Action-Effect Field

confidence 88% · The current action and these gains form a state-conditioned action-effect field in the frozen relation basis.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Industrial actor--critic methods usually represent continuous actions as anonymous numerical coordinates. They must therefore learn from limited interactions which process variables each action affects, in which direction, and after what delay. Fixed industrial documents already describe part of these relations, but their open-text statements neither represent the current operating condition nor directly fit a numerical policy. This article presents LLM-Guided Contextual Action Evaluation for Operational Decisions in Industrial Processes (LCAE), which uses a large language model before training to normalize fixed documents into a frozen action--observation--direction--delay relation basis. Recent numerical action--response history then modulates the current strength of each relation, while the evaluated action forms a state-conditioned nonlinear action-effect field in the same basis. The critic evaluates actions through this field, and the actor uses the same relation gains to generate actions, making document semantics part of maximum-entropy policy learning. Neither the LLM nor the embedding model runs online during training or deployment; the deployed policy uses only frozen semantic artifacts and visible numerical history. The method states a falsifiable hypothesis: when documented relations are correct and recent history reflects their contextual strength, this action representation should provide a more useful decision bias than raw action coordinates.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

44,856 characters extracted from source content.

Expand or collapse full text

LLM-Guided Contextual Action Evaluation for Operational Decisions in Industrial Processes Youcheng Zong Runda Jia and Dakuo He †thanks: This work was supported by the Fundamental Research Funds for the Central Universities, China (N26GFZ006). (Corresponding author: Runda Jia.)†thanks: Youcheng Zong, Runda Jia, and Dakuo He are with the College of Information Science and Engineering, Northeastern University, Shenyang 110004, China (e-mail: youchengzong@stumail.neu.edu.cn; jiarunda@ise.neu.edu.cn; hedakuo@ise.neu.edu.cn). Abstract Industrial actor–critic methods usually represent continuous actions as anonymous numerical coordinates. They must therefore learn from limited interactions which process variables each action affects, in which direction, and after what delay. Fixed industrial documents already describe part of these relations, but their open-text statements neither represent the current operating condition nor directly fit a numerical policy. This article presents LLM-Guided Contextual Action Evaluation for Operational Decisions in Industrial Processes (LCAE), which uses a large language model before training to normalize fixed documents into a frozen action–observation–direction–delay relation basis. Recent numerical action–response history then modulates the current strength of each relation, while the evaluated action forms a state-conditioned nonlinear action-effect field in the same basis. The critic evaluates actions through this field, and the actor uses the same relation gains to generate actions, making document semantics part of maximum-entropy policy learning. Neither the LLM nor the embedding model runs online during training or deployment; the deployed policy uses only frozen semantic artifacts and visible numerical history. The method states a falsifiable hypothesis: when documented relations are correct and recent history reflects their contextual strength, this action representation should provide a more useful decision bias than raw action coordinates. Index Terms: large language models, reinforcement learning, industrial process decision-making, action representation, actor–critic. I Introduction Operational decision-making in process industries adjusts manipulated variables from continuous observations and improves long-term operating objectives through delayed process responses. These tasks are usually partially observable because available measurements provide an incomplete view of the internal process state, and an action may affect several later sampling instants [1]. Reinforcement learning provides a common interface for such closed-loop sequential decisions, and deep actor–critic methods have been used for continuous control and industrial process control [2, 3, 4]. These studies show that numerical policies can learn complex control laws from interaction, but their action interfaces still usually treat each manipulated variable as an anonymous coordinate. Anonymous action coordinates leave the critic with a compound learning problem. To estimate action value, the critic must learn how the state evolves and rediscover which observations each action affects, in which direction, and after what delay. Twin critics, target networks, and maximum-entropy objectives can improve numerical optimization [5, 3], but they do not explain the process meaning of an action coordinate. In industrial environments with costly interaction, coupled actions, or slow responses, rediscovering these relations consumes samples that could otherwise be used to learn the policy. Industrial numerical modeling has extracted state and prediction information from numerical trajectories [6, 7, 8, 9], but industrial systems also contain information outside those trajectories. Equipment descriptions, variable tables, operating procedures, and environment descriptions often state which visible variables an action affects through process paths, the typical direction of the effect, and the delay range in which a response may occur. These relations may span several paragraphs, use inconsistent terminology, and mix structured fields with open text. An LLM can use open knowledge and language reasoning to normalize these descriptions into consistent relations, but the documents themselves remain static. A document can describe a basic effect path, but it cannot state how strong that path is in the current operating condition or directly specify the optimal action. Variable-name matching or fixed rules alone are often insufficient for this conversion. The same manipulated variable may appear as an equipment tag, a control-loop name, or a process term, while direction and delay statements may be distributed across different document locations. The necessary role of the LLM in this work is to resolve such open vocabulary and cross-sentence relations and then restrict the result to fields allowed by the task schema. This role differs from replacing the controller with a language model: the LLM constructs the relation interface, while the numerical policy still learns current decisions from interaction. This distinction determines how this work separates language and numerical information. Fixed documents define action-relation coordinates that are expected to remain stable across operating conditions, while recent action–response history estimates their relative strength in the current state. The evaluated action must also retain its own amplitude because different actions cannot have the same value under identical relation strengths. The required action representation must therefore depend jointly on fixed relation identity, recent numerical context, and the current candidate action. This article presents LLM-Guided Contextual Action Evaluation (LCAE) to implement this division of roles. Before training, an LLM reads only fixed environment documents and a task schema and produces action–observation–direction–delay relation cards. A fixed renderer and embedding model convert the cards into a frozen semantic basis, while the relation fields determine action selection, observation selection, and valid delay bands. An online numerical module extracts relation-wise action–response events from recent history and computes relation gains centered at one. The current action and these gains form a state-conditioned action-effect field in the frozen relation basis. The critic uses this field to estimate long-term value, and the actor uses the same gains to generate the action that will be executed. LCAE does not place an LLM in the control loop. The LLM does not read numerical trajectories, rewards, hidden simulator states, optimal actions, or test information, and it does not generate, rank, or validate online actions. The embedding model also runs only before training. Closed-loop decisions during training and deployment are made entirely by a compact numerical actor, separating semantic construction cost from per-step control latency. This work focuses on a scientific hypothesis rather than presuming a performance improvement. If fixed documents provide stable action paths and recent history contains their contextual strength, LCAE should improve policy learning and closed-loop return under limited interaction or changing action effectiveness. Conversely, if a matched-budget raw-action baseline reaches the same performance, or shuffling action–effect correspondences does not change the result, then the documented relations do not provide the expected decision bias. This falsifiable boundary limits the claim to contextual action evaluation rather than causal-effect identification, fault diagnosis, or safety control. The main contributions are as follows. • We reformulate action learning in industrial actor–critic as contextual action evaluation: fixed documents define relation identity, recent numerical history defines current relation strength, and the candidate action defines relation amplitude. • We introduce a state-conditioned action-effect field that combines relation semantics, historical gains, and continuous actions through a bounded nonlinearity, with explicit empty-history fallback, local sensitivity, and action-coverage conditions. • We give a complete maximum-entropy actor–critic interface, training objective, and deployment boundary so that frozen offline semantic relations organize both action generation and value learning without an online LLM. The remainder of this article is organized as follows. Section I reviews action representation, language-informed reinforcement learning, and knowledge-guided industrial control. Section I defines LCAE, and Section IV summarizes its claims and boundaries. I Related Work I-A From Continuous Control to Industrial Actor–Critic Reinforcement learning formulates control as maximizing long-term return through environment interaction [10]. DDPG extends deterministic policy gradients to deep continuous control, TD3 reduces function-approximation error through twin critics and delayed updates, and SAC uses a stochastic policy and maximum-entropy objective to improve exploration and training stability [2, 5, 3]. Industrial studies have applied deep reinforcement learning to process control and discussed sample efficiency, constraints, stability, and deployment risk [11, 4]. A continuous polymerization case further shows that a numerical actor can directly generate manipulated actions [12]. These routes mainly improve how policies and value functions are optimized, while the action usually still enters the critic as a raw numerical vector. LCAE builds on this interface: it retains maximum-entropy actor–critic learning but changes the action representation received by the critic and the relation context visible to the actor. This position also separates the roles of numerical state and action representations. The history encoder must still summarize load, inventory, setpoints, and other operating conditions, while the action-effect field specifically determines how a candidate action should enter the value function under that history. LCAE therefore does not assume that semantic relations replace dynamic state modeling, and it does not change the environment’s reward or transition definition. I-B From Action Identity to Action Relations Action-representation studies have shown that structure among actions can improve generalization across actions. Act2Vec extracts action relations from demonstration contexts and uses the learned vectors for state augmentation and value-function approximation [13]. Another approach jointly learns a low-dimensional action representation and its mapping to actual actions, allowing outcomes of similar actions to be reused in large finite action sets [14]. Know Your Action Set explicitly learns relations in variable action sets so that a policy can handle action combinations that were not jointly available during training [15]. Action-adaptive policies can also infer action impact from recent state changes and adjust when actions are missing or their effects are perturbed [16]. These studies learn action structure from demonstrations, action sets, or online state changes, but they do not combine action–observation–direction–delay relations from fixed industrial documents with the current numerical amplitude of a continuous action. LCAE retains both stable relation identity from documents and contextual strength from history and evaluates continuous actions in the same effect field. Compared with discrete action embeddings, continuous industrial actions also require amplitude and sign to be preserved. Mapping an increase and a decrease in valve opening to one static action vector would lose control meaning, while learning only local action impact would discard cross-variable paths stated in documents. The relation-wise nonlinearity in LCAE preserves both elements and allows one action coordinate to correspond to several observation paths and delay bands. I-C From Language Knowledge to Decision Interfaces Natural language can provide goals, policy advice, reward structure, and environment knowledge to reinforcement learning, and these interfaces have been summarized in prior work [17]. EMMA grounds entities and dynamics from free-form manuals to observations to support policy generalization to unseen tasks [18]. RLang uses a formal declarative language to describe partial knowledge about all components of an MDP and grounds it into an algorithm-independent partial world model and policy [19]. Reward machines represent task-reward structure as automata that reinforcement-learning algorithms can exploit [20]. These methods show that language or symbolic knowledge can change the information available to decision learning rather than serving only as a prediction label. Foundation models can also transfer pretrained representations to zero-shot domain tasks [21], while LLMs further provide the ability to propose goals or construct policy priors from open descriptions. ELLM rewards pretraining behaviors with LLM-suggested goals, while GLAM updates an LLM policy through online reinforcement learning [22, 23]. Both routes place the language model closer to the online decision process. LCAE extends the offline semantic-interface principle from industrial forecasting [24] to action evaluation: the LLM only normalizes fixed industrial relations before training, while the online actor and critic remain numerical models. LCAE therefore studies how language knowledge defines action-evaluation coordinates that numerical history can activate, rather than studying a language policy itself. The offline interface also allows relation artifacts to be inspected manually or programmatically before training. Compared with online free-text prompting, this interface can freeze document versions, relation fields, rendered text, and embeddings, ensuring that different training runs receive exactly the same semantic input. Auditability does not guarantee that a relation is correct, but it makes erroneous relations identifiable and removable rather than hiding them inside per-step language-model calls. I-D From Industrial Knowledge Guidance to Contextual Action Evaluation Industrial reinforcement learning has used rules, simulators, and process knowledge to narrow the search space or constrain policies. Chemical-operation procedure synthesis combines external knowledge, dynamic simulation, and deep reinforcement learning to search for interpretable operating steps [25]. Knowledge–data-guided methods use process-causality structure to organize industrial reinforcement learning, and multi-objective graph reinforcement learning uses expert prior graphs to guide fermentation optimization [26, 27]. These studies show that industrial knowledge can provide structure beyond numerical interaction, but they usually require manual rules, explicit graphs, or task-specific knowledge encoding. Another route places LLM agents directly in industrial decisions. Systems have used LLMs to interpret real-time events, generate production plans, and control automation operations [28], and other work has used them for human–AI collaborative industrial decision support [29]. Agent frameworks for fault-tolerant control also generate recovery actions and validate them with digital twins, knowledge graphs, and deterministic constraints [30]. These systems target planning, recovery, or human-decision layers, whose time scales and safety interfaces differ from a closed-loop policy that produces continuous actions with a compact actor at every sampling instant. LCAE therefore restricts the LLM to offline relation construction and focuses on contextual action evaluation jointly defined by document relations and recent numerical history. This distinction also limits the deployment claim. LCAE provides no safety guarantee for LLM-generated actions and does not place language-reasoning latency inside the real-time control budget because the deployed policy never calls the LLM. Its required engineering checks are offline relation correctness, action-space coverage, and standard validation of the online numerical module. I Method I-A Problem Formulation and Decision Coordinates Consider a partially observable industrial decision process in environment e. Let otphy∈ephyo_t^phy _e^phy and atphy∈ephya_t^phy _e^phy denote the physical observation and physical action, respectively. Fixed elementwise strictly increasing affine bijections eoN_e^o and eaN_e^a define the internal decision coordinates: ot=eo(otphy)∈ℝdo,at=ea(atphy)∈e⊆ℝda.o_t=N_e^o\! (o_t^phy ) ^d_o, a_t=N_e^a\! (a_t^phy ) _e ^d_a. (1) Strict monotonicity preserves the signs of action and response directions stated in the documents. Except for quantities carrying the phyphy superscript at the environment interface, all observations, actions, histories, and policy distributions below use these decision coordinates. The policy-visible history at decision instant t is ℋt=(ot−L:t,at−L:t−1,Δt−L:t−1),H_t= (o_t-L:t,a_t-L:t-1, t_t-L:t-1 ), (2) where Δti t_i is the physical time elapsed after executing action aia_i. The policy generates ata_t, the environment executes atphy=(ea)−1(at)a_t^phy=(N_e^a)^-1(a_t), and then returns reward rtr_t and the next observation ot+1o_t+1. This history definition covers both fixed-step and irregularly timed decision processes. For a maximum-entropy policy π, the action value with continuous-time discounting is Qπ(ℋt,at)=π[ Q^π(H_t,a_t)=E_π\! [ rt+∑k=1∞exp(−λ∑j=0k−1Δt+j) r_t+ _k=1^∞ \! (-λ _j=0^k-1 t_t+j ) ⋅(rt+k−αlogπ(at+k∣ℋt+k))|ℋt,at], · (r_t+k-α π(a_t+k _t+k) )\, |\,H_t,a_t ], (3) where λ>0λ>0 is the temporal discount rate and α>0α>0 is the entropy temperature. When every Δt t_t is fixed, exp⁡(−λΔt) (-λ t_t) reduces to the standard discrete discount factor γ. LCAE retains this decision objective but does not use raw action a directly as the critic’s action input. I-B Offline Action-Effect Relation Basis For environment e, let DeD_e and eS_e denote fixed industrial documents and the task schema. The LLM runs only before policy training and produces a set of relation cards e=FLLM(De,e)=cpp=1Pe.K_e=F_LLM(D_e,S_e)=\c_p\_p=1^P_e. (4) Each card cpc_p specifies an action field, an affected visible observation, a typical effect direction, and a physical response-delay band. A relation card may reference only actions and visible observations declared in the task schema. It contains no reward, hidden state, future measurement, test information, or recommended action. Each relation card should also retain its document-evidence location and schema version so that the relation artifact can be traced to fixed inputs. Structural validation rejects unknown fields, invisible observations, undeclared actions, empty delay bands, and relations outside the allowed scope. Semantic validation checks whether direction and delay are supported by the recorded document span rather than freely completed by the model. These checks occur before embedding so that invalid cards cannot enter the frozen relation basis. A fixed renderer ℛR turns each card into canonical text, and a fixed embedding model EψE_ψ then produces a unit semantic direction: vp=Eψ(ℛ⁡(cp))‖Eψ(ℛ⁡(cp))‖2∈ℝk,Ve=[v1⊤vPe⊤]∈ℝPe×k.v_p= E_ψ(R(c_p)) \|E_ψ(R(c_p)) \|_2 ^k, V_e= bmatrixv_1 \\ \\ v_P_e bmatrix ^P_e× k. (5) Matrix VeV_e remains frozen during policy training and deployment. The unit normalization controls only embedding scale and does not change the field semantics of a relation card. The cards also determine an action-selection matrix Mea∈0,1Pe×daM_e^a∈\0,1\^P_e× d_a, an observation-selection matrix Meo∈0,1Pe×doM_e^o∈\0,1\^P_e× d_o, and delay bands ℒpp=1Pe\L_p\_p=1^P_e. Row p selects the action and visible response for relation p, while ℒpL_p gives the physical-time range in which a response may be collected after the action. The effect direction remains in the full relation card and its semantic direction vpv_p, while the selection matrices perform only field routing. Together, these offline objects define the basic action-relation coordinates that remain fixed across episodes. One action may correspond to several cards, and one observation may receive several action paths, but duplicate cards should be merged before freezing. Every action coordinate must be covered by at least one relation, and the final relation order must remain identical across caches, matrices, and training runs. Thus, eK_e, VeV_e, the two selection matrices, and the delay bands form one versioned offline artifact rather than a stochastic prompt result regenerated between episodes. I-C History-Aligned Relation Strength A numerical history encoder first produces the state representation ht=Eθ(ℋt)∈ℝdh.h_t=E_θ(H_t) ^d_h. (6) This representation summarizes the overall history needed by the policy, while the relation-strength module explicitly preserves action–response events aligned with documented relations. To handle irregular sampling, set τt−L=0 _t-L=0 and τi+1=τi+Δti _i+1= _i+ t_i, and let Tt=τt−τt−LT_t= _t- _t-L denote the window duration. Let Me,paM_e,p^a and Me,poM_e,p^o denote row p of the two selection matrices. The valid action–response index set for relation p is ℐt,p=(i,j):t−L≤i<j≤t,τj−τi∈ℒp.I_t,p= \(i,j):t-L≤ i<j≤ t,\; _j- _i _p \. (7) This set permits only responses that are already visible at the current instant and satisfy the documented delay. For every (i,j)∈ℐt,p(i,j) _t,p, define the relation-aligned event zt,ij(p)=[Me,paai,Me,po(oj−oi),τj−τiTt,τt−τjTt]∈ℝ4.z_t,ij^(p)= [M_e,p^aa_i,\;M_e,p^o(o_j-o_i),\; _j- _iT_t,\; _t- _jT_t ] ^4. (8) The first two components are the corresponding historical action amplitude and visible response change, and the last two are normalized response delay and response age. An event describes observed action–response alignment and is not interpreted as an identified causal effect. Using change oj−oio_j-o_i rather than an absolute observation makes the event focus on the local response after an action, while hth_t retains responsibility for modeling the overall operating condition. Explicit physical time rather than index distance allows one documented delay to be used with irregular sampling. Response age distinguishes newly observed evidence from older evidence near the start of the window, while the shared map learns how to attenuate or retain this information. One shared numerical map Φξ:ℝ4→ℝdr _ξ:R^4 ^d_r encodes events from every relation, and mean aggregation forms the relation-wise representation: Rt,p=Mean(i,j)∈ℐt,pΦξ(zt,ij(p))∈ℝdr,Mean∅=0.R_t,p= (i,j) _t,pMean _ξ\! (z_t,ij^(p) ) ^d_r, _ =0. (9) Using a shared map avoids assigning an independent event network to every documented relation; relation differences are expressed by field selection, delay, and the semantic basis. Stacking all relation representations gives Rt=[Rt,1⊤Rt,Pe⊤]=e,ξ(ℋt,Mea,Meo,ℒp)∈ℝPe×dr.R_t= bmatrixR_t,1 \\ \\ R_t,P_e bmatrix=T_e,ξ\! (H_t;M_e^a,M_e^o,\L_p\ ) ^P_e× d_r. (10) Numerical events and frozen semantic directions are projected into the same q-dimensional matching space. The relation-wise matching score is ℓt=diag⁡[(RtWr)(VeWv)⊤]∈ℝPe, _t=diag\! [(R_tW_r)(V_eW_v) ] ^P_e, (11) where Wr∈ℝdr×qW_r ^d_r× q and Wv∈ℝk×qW_v ^k× q are trainable projections. The diagonal operator matches numerical events for relation p only with documented relation p, avoiding unconstrained cross-relation mixing. The matching score is converted into relation gains through the centered map gt=2σ(ℓt)∈(0,2)Pe,g_t=2σ( _t)∈(0,2)^P_e, (12) where σ is the logistic function. gt,p=1g_t,p=1 denotes neutral documented strength, while gt,p<1g_t,p<1 and gt,p>1g_t,p>1 denote suppression and enhancement of relation p by recent history. When a relation has no valid historical event, Rt,p=0R_t,p=0, so ℓt,p=0 _t,p=0 and gt,p=1g_t,p=1. This default preserves the basic documented relation rather than deleting an action path because recent evidence is unavailable. The neutral fallback also avoids equating “no valid event has been observed” with “the relation is currently ineffective.” The former is missing evidence, whereas the latter requires recent events that support a smaller gain. This distinction prevents cold-start windows, long-delay relations, and temporary missing observations from automatically creating zeroed action channels. I-D State-Conditioned Action-Effect Field For any action a∈ea _e currently evaluated by the critic, the relation-level action amplitude is ue(a)=Meaa∈ℝPe.u_e(a)=M_e^aa ^P_e. (13) The same action coordinate may appear in several relations because one manipulated variable may affect several process observations or delay paths. Relation gains and the current action are jointly activated by the same bounded nonlinearity: ηt(a)=tanh⁡(gt⊙ue(a))∈(−1,1)Pe. _t(a)= \! (g_t u_e(a) )∈(-1,1)^P_e. (14) This operation preserves the action sign, allows relation gains to continuously modulate action sensitivity, and bounds the relation amplitude produced by large normalized actions. The LCAE state-conditioned action-effect field is defined as ft(a)=ηt(a)⊤Ve=tanh⁡(gt⊙Meaa)⊤Ve∈ℝk. f_t(a)= _t(a) V_e= \! (g_t M_e^aa ) V_e ^k. (15) This field combines three sources with distinct roles: VeV_e gives relation identity, gtg_t gives history-conditioned strength, and MeaaM_e^aa gives the amplitude of the currently evaluated action. The same action therefore usually has different effect fields under different histories, while different actions remain distinct under the same history. This is not ordinary concatenation of a document embedding, history vector, and action vector because the three jointly determine the critic’s action input through a relation-wise nonlinearity. The local sensitivity of the gain-modulated activation to relation-level action amplitude is ∂ηt,p(a)∂ue,p(a)=gt,p(1−ηt,p(a)2)>0. ∂ _t,p(a)∂ u_e,p(a)=g_t,p (1- _t,p(a)^2 )>0. (16) Because this derivative is strictly positive, the gain does not reverse the action direction in decision coordinates. It changes local sensitivity, while tanh gradually reduces sensitivity in large-action regions. Relation direction is retained by the frozen semantic basis, and numerical history modulates only its current strength. To prevent the effect field from collapsing valid action directions, its Jacobian with respect to the original decision action should have full column rank: Jt(a) J_t(a) =∂ft(a)∂a = ∂ f_t(a)∂ a =Ve⊤Diag⁡[gt⊙(1−ηt(a)2)]Mea,rank⁡Jt(a)=da. =V_e Diag\! [g_t (1- _t(a)^2 ) ]M_e^a, _t(a)=d_a. (17) Because every diagonal term is strictly positive, a sufficient structural condition is rank⁡(Mea)=darank(M_e^a)=d_a and rank⁡(Ve)=Perank(V_e)=P_e. The first condition requires every valid action direction to be covered by an independent set of relations, and the second requires the semantic relation directions to have no row-rank degeneration. These conditions should be checked when building the relation basis; if they fail, relation cards must be corrected, merged, or added rather than allowing action-direction loss to remain hidden during training. I-E Decision Backbone and Training Objective LCAE defines twin critics from the history representation and action-effect field: Qm(ℋt,a,Ve)=Cϕm(ht,ft(a)),m∈1,2.Q_m(H_t,a;V_e)=C_ _m\! (h_t,f_t(a) ), m∈\1,2\. (18) The critic receives no raw-action bypass, so its action evaluation must pass through the relation field. The environment-specific actor is at∼πω(⋅∣ℋt):=πω(⋅∣ht,gt).a_t _ω(· _t):= _ω(· h_t,g_t). (19) The actor uses the overall history representation hth_t and relation-wise gains gtg_t but does not receive an arbitrary concatenation of frozen semantic vectors. Relation identity enters policy learning through the gain indices and the critic’s action field. For replay transition (ℋt,at,rt,Δt,ℋt+1,dt)(H_t,a_t,r_t, t_t,H_t+1,d_t), where dtd_t is the termination indicator, the maximum-entropy Bellman target is yt=rt y_t=r_t +(1−dt)exp⁡(−λΔt) +(1-d_t) (-λ t_t) ⋅a′∼πω(⋅∣ℋt+1)[minmQ¯m(ℋt+1,a′;Ve)−αlogπω(a′∣ℋt+1)], ·E_a _ω(· _t+1) [ _m Q_m(H_t+1,a ;V_e)-α _ω(a _t+1) ], (20) where Q¯m Q_m denotes a target critic. The temporal discount directly uses the realized Δt t_t in the transition, keeping different action durations consistent in the value target. The twin critics are trained by minimizing ℒQ=⁡[∑m=12(Qm(ℋt,at,Ve)−yt)2].L_Q=E\! [ _m=1^2 (Q_m(H_t,a_t;V_e)-y_t )^2 ]. (21) The reparameterized actor objective is ℒπ=a∼πω(⋅∣ℋt)[αlogπω(a∣ℋt)−minmQm(ℋt,a;Ve)].L_π=E_a _ω(· _t)\! [α _ω(a _t)- _mQ_m(H_t,a;V_e) ]. (22) The actor gradient passes through the critic that depends on ft(a)f_t(a), so the relation basis does more than provide extra context: it directly shapes action updates. At the same time, the actor explicitly receives gtg_t, allowing it to use the same history-conditioned relation strengths before generating an action. The parameter sets EθE_θ, Φξ _ξ, WrW_r, WvW_v, Cϕ1C_ _1, Cϕ2C_ _2, and πω _ω are trained by the decision losses above. FLLMF_LLM, EψE_ψ, VeV_e, MeaM_e^a, MeoM_e^o, and ℒp\L_p\ remain frozen during policy training. This separation prevents the policy loss from rewriting offline document relations into unauditable online semantic objects. It also allows the same relation artifacts to be reused across random seeds and matched baselines under a fixed experimental protocol. To separate method gains from extra model capacity, a raw-action baseline should share the history encoder, actor and critic widths, optimizer, replay data, environment-interaction budget, and seed path. Because LCAE adds relation-event and projection parameters, matching hidden widths alone does not constitute strict capacity matching. A parameter-count-matched numerical baseline without semantic content is therefore also required, together with ablations that keep the architecture fixed while shuffling or removing relation content. These comparisons are not part of the method definition itself, but they are necessary interfaces for testing whether the action-effect field actually uses documented relations. I-F Deployment and Claim Boundaries At deployment, the system first computes hth_t, RtR_t, and gtg_t from visible numerical history, after which the actor generates decision action ata_t. The action is mapped back to physical coordinates through (ea)−1(N_e^a)^-1 and sent to the environment. The critic, LLM, and embedding model do not participate in per-step online execution. Online computation contains only history encoding, relation-event aggregation, fixed matrix operations, and one actor forward pass. Different industrial environments share the relation-card schema, action-effect-field definition, and training interface, but they do not share one controller. Each environment retains its own observation space, action space, reward, coordinate transforms, history length, delay bands, encoder, actor, and critic parameters. This work therefore does not claim direct policy transfer across environments without retraining. Relation gain gtg_t is a history-conditioned action-evaluation quantity, not a physical gain or a strict causal coefficient. Unobserved disturbances, closed-loop feedback, and sensor errors may all affect observation changes after an action. LCAE does not perform fault diagnosis, measurement correction, safety shielding, or causal identification; these capabilities require separate evidence and dedicated mechanisms. The method also assumes that fixed documents are sufficiently accurate and that relation cards cover every valid action direction. When documents are outdated, relations conflict, or action coverage is insufficient, the corresponding relation artifacts should be withheld and revalidated before deployment rather than left for the online policy to repair automatically. The deployed system should also bind an offline-artifact hash to the policy checkpoint so that the controller cannot load a mismatched relation order or coordinate definition. If a history window is incomplete, the system may use the defined neutral fallback; if the schema or coordinate transform does not match, execution must be rejected rather than silently degraded. This distinction separates recoverable data sparsity from configuration errors that would change action meaning. IV Conclusion This article introduced LCAE, which replaces anonymous continuous actions in industrial actor–critic learning with a state-conditioned action-effect field. Fixed industrial documents define action–observation–direction–delay relations through an offline LLM, recent numerical history modulates relation strength, and the current action provides the relation amplitude being evaluated. The same relation gains inform action generation, and the same semantic basis supports long-term value learning, so document knowledge enters policy optimization while the deployed loop remains free of LLM calls. The central claim is a testable decision bias rather than an unmeasured performance advantage. When documented relations are correct and recent action–response history reflects contextual strength, LCAE should improve policy learning under limited interaction and changing action effectiveness; matched baselines and shuffled-relation ablations can directly falsify this hypothesis. Relation gains should not be interpreted as identifiable causal coefficients, and different environments still require their own controllers, rewards, and training processes. Subsequent validation should first examine relation-card correctness, action coverage, non-semantic baselines with matched information and capacity, and closed-loop outcomes after documented relations are removed or shuffled. References [1] L. P. Kaelbling, M. L. Littman, and A. R. Cassandra (1998) Planning and acting in partially observable stochastic domains. Artificial Intelligence 101 (1–2), p. 99–134. External Links: Document, Link Cited by: §I. [2] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra (2016) Continuous control with deep reinforcement learning. In Proceedings of the 4th International Conference on Learning Representations, Note: OpenReview publication; no page range assigned External Links: Link Cited by: §I, §I-A. [3] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine (2018) Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proceedings of the 35th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 80, p. 1861–1870. External Links: Link Cited by: §I, §I, §I-A. [4] R. Nian, J. Liu, and B. Huang (2020) A review on reinforcement learning: introduction and applications in industrial process control. Computers & Chemical Engineering 139, p. 106886. External Links: Document, Link Cited by: §I, §I-A. [5] S. Fujimoto, H. van Hoof, and D. Meger (2018) Addressing function approximation error in actor-critic methods. In Proceedings of the 35th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 80, p. 1587–1596. External Links: Link Cited by: §I, §I-A. [6] Y. Zong, S. Hu, D. Qin, Z. Wang, C. Zhang, J. Chu, and L. Zhang (2023) Iron-tapping state recognition of blast furnace based on Bi-GRU composite model and post-processing classifier. IEEE Sensors Journal 23 (18), p. 22006–22018. External Links: Document, Link Cited by: §I. [7] Y. Zong, Z. Wang, X. Liu, Y. Nian, J. Pan, C. Zhang, Y. Wang, J. Chu, and L. Zhang (2023) Judgment of blast furnace iron-tapping status based on data differential processing and dynamic window analysis algorithm. Progress in Natural Science: Materials International 33 (4), p. 450–457. External Links: Document, Link Cited by: §I. [8] Y. Zong, Y. Nian, C. Zhang, X. Tang, L. Wang, and L. Zhang (2025) Hybrid grid search and Bayesian optimization-based random forest regression for predicting material compression pressure in manufacturing processes. Engineering Applications of Artificial Intelligence 141, p. 109580. External Links: Document, Link Cited by: §I. [9] Y. Zong, R. Jia, S. Wu, L. Zhang, and D. He (2025) A meta-contrastive learning hybrid model for adaptive temperature trend prediction in variable ladle preheating. Engineering Applications of Artificial Intelligence 162, p. 112750. External Links: Document, Link Cited by: §I. [10] R. S. Sutton and A. G. Barto (2018) Reinforcement learning: an introduction. 2 edition, MIT Press, Cambridge, MA, USA. External Links: ISBN 978-0-262-03924-6, Link Cited by: §I-A. [11] S. Spielberg, A. Tulsyan, N. P. Lawrence, P. D. Loewen, and R. B. Gopaluni (2019) Toward self-driving processes: a deep reinforcement learning approach to control. AIChE Journal 65 (10), p. e16689. External Links: Document, Link Cited by: §I-A. [12] Y. Ma, W. Zhu, M. G. Benton, and J. Romagnoli (2019) Continuous control of a polymerization system with deep reinforcement learning. Journal of Process Control 75, p. 40–47. External Links: Document, Link Cited by: §I-A. [13] G. Tennenholtz and S. Mannor (2019) The natural language of actions. In Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 97, p. 6196–6205. External Links: Link Cited by: §I-B. [14] Y. Chandak, G. Theocharous, J. Kostas, S. Jordan, and P. Thomas (2019) Learning action representations for reinforcement learning. In Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 97, p. 941–950. External Links: Link Cited by: §I-B. [15] A. Jain, N. Kosaka, K. Kim, and J. J. Lim (2022) Know your action set: learning action relations for reinforcement learning. In Proceedings of the 10th International Conference on Learning Representations, Note: OpenReview publication; no page range assigned External Links: Link Cited by: §I-B. [16] K. Zeng, L. Weihs, R. Mottaghi, and A. Farhadi (2023) Moving forward by moving backward: embedding action impact over action semantics. In Proceedings of the 11th International Conference on Learning Representations, Note: OpenReview publication; no page range assigned External Links: Link Cited by: §I-B. [17] J. Luketina, N. Nardelli, G. Farquhar, J. Foerster, J. Andreas, E. Grefenstette, S. Whiteson, and T. Rocktäschel (2019) A survey of reinforcement learning informed by natural language. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, p. 6309–6317. External Links: Document, Link Cited by: §I-C. [18] A. W. Hanjie, V. Y. Zhong, and K. Narasimhan (2021) Grounding language to entities and dynamics for generalization in reinforcement learning. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, p. 4051–4062. External Links: Link Cited by: §I-C. [19] R. Rodriguez-Sanchez, B. A. Spiegel, J. Wang, R. Patel, S. Tellex, and G. Konidaris (2023) RLang: a declarative language for describing partial world knowledge to reinforcement learning agents. In Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 202, p. 29161–29178. External Links: Link Cited by: §I-C. [20] R. Toro Icarte, T. Q. Klassen, R. Valenzano, and S. A. McIlraith (2022) Reward machines: exploiting reward function structure in reinforcement learning. Journal of Artificial Intelligence Research 73, p. 173–208. External Links: Document, Link Cited by: §I-C. [21] W. Su, Y. Zong, R. Jia, J. Qin, and M. Li (2026) Zero-shot capillary segmentation in dermoscopy images via SAM2: a case study on oral mucosa. IEEE Journal of Biomedical and Health Informatics 30 (5), p. 4376–4387. External Links: Document, Link Cited by: §I-C. [22] Y. Du, O. Watkins, Z. Wang, C. Colas, T. Darrell, P. Abbeel, A. Gupta, and J. Andreas (2023) Guiding pretraining in reinforcement learning with large language models. In Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 202, p. 8657–8677. External Links: Link Cited by: §I-C. [23] T. Carta, C. Romac, T. Wolf, S. Lamprier, O. Sigaud, and P. Oudeyer (2023) Grounding large language models in interactive environments with online reinforcement learning. In Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 202, p. 3676–3713. External Links: Link Cited by: §I-C. [24] Y. Zong, R. Jia, M. Ren, and D. He (2026) LLM-guided task-semantic field factorization for industrial process forecasting. Note: arXiv preprint arXiv:2607.06623 External Links: 2607.06623, Document, Link Cited by: §I-C. [25] S. Kubosawa, T. Onishi, and Y. Tsuruoka (2018) Synthesizing chemical plant operation procedures using knowledge, dynamic simulation and deep reinforcement learning. In Proceedings of the SICE Annual Conference, p. 1376–1379. External Links: Link Cited by: §I-D. [26] X. Zhang, C. Song, J. Zhao, Z. Xu, and X. Deng (2024) Spatial-temporal causality modeling for industrial processes with a knowledge-data guided reinforcement learning. IEEE Transactions on Industrial Informatics 20 (4), p. 5634–5646. External Links: Document, Link Cited by: §I-D. [27] D. Li and Z. Yang (2025) Prior-informed adaptive multi-objective graph reinforcement learning for lysine fed-batch fermentation process. Journal of Process Control 156, p. 103578. External Links: Document, Link Cited by: §I-D. [28] Y. Xia, N. Jazdi, J. Zhang, C. Shah, and M. Weyrich (2025) Control industrial automation system with large language model agents. In Proceedings of the 30th IEEE International Conference on Emerging Technologies and Factory Automation, p. 1–8. External Links: Document, Link Cited by: §I-D. [29] Y. Zong, R. Jia, K. Li, D. Xue, L. Zhang, and D. He (2026) LLM-driven human–AI collaborative decision support system for complex industrial processes: a case study in metallurgy. Neural Networks 202, p. 109055. External Links: Document, Link Cited by: §I-D. [30] J. Vyas, M. S. Gill, A. Markaj, F. Gehlhoff, and M. Mercangöz (2026) From detection to action: using LLM agents for fault-tolerant control. External Links: 2606.28011, Document, Link Cited by: §I-D.

Instant research discovery