Two years ago, I was offered an opportunity to contribute a chapter to the Encyclopedia of Systems and Control Engineering. The chapter I ended up submitting was on Machine Learning for Industrial Process Monitoring - ScienceDirect – an overview of many of the concepts that I’ve encountered during my academic and industry positions, especially some of the tricky concepts that limit successful transfer of academic research to practical industrial solutions.
Key topics covered included:
- An overview of the context of industrial process monitoring, including a summary of process characteristics which make data-driven process monitoring challenging.
- A brief explanation of the different machine learning tasks, typical machine learning models, as well as the requirements for practical implementation of machine learning models in industry.
- An overview of process monitoring tasks and how different machine learning approaches map to these tasks.
- A survey of a selection of unsupervised and semi-supervised learning techniques applied to process monitoring.
- A brief discussion on the current challenges and prospects of machine learning in industrial process monitoring.
In this post, I highlight one or two interesting ideas around the first three and last points (all figures were created by me and used in the chapter mentioned). For a detailed review of different machine learning techniques applied to process monitoring, have a look at the chapter!
Process monitoring: Fault diagnosis and process optimization
In this context, process monitoring refers to fault diagnosis (abnormal event detection) and process optimization (identifying opportunities to improve process performance) for industrial continuous processes (e.g., mineral processing plants). The outputs generated by such process monitoring activities can be soft sensors (predicting hard-to-measure key performance indicators), alarms (e.g., ore oversize alarm that can be used as an interlock to a feeder system), notifications (e.g., undesirable process condition notification with summary statistics and suggested action) or offline investigative reports.
Figure 1. (Source: Auret, 2026)
Challenging process data characteristics
Most industrial process monitoring applications make use of tabular time series data, while computer vision applications are a growing and robust addition to process monitoring approaches. If we focus on time series data, the characteristics that make data-driven process monitoring challenging can be structured according to temporal, distribution, causality and representation issues.
I go into some depth on each of these challenging characteristics in the mentioned chapter, including some strategies (preprocessing or model adaptations) that may be required to improve process monitoring applications. In my opinion, the most challenging process data characteristics that limit practical industrial applications are time-variance, control interaction and lack of observability. Time-variant systems imply that the causal relationships between process variables change over time.
For example, in a state-space representation, the parameters $A(t)$, $B(t)$, $C(t)$, and $D(t)$ are functions of time, where $x(t)$, $u(t)$, and $y(t)$ represent the system states, independent inputs, and measurements:
$\dot{x}\left(t\right)=Ax\left(t\right)+Bu\left(t\right)$
$y\left(t\right)=Cx\left(t\right)+Du(t)$
Figure 2. (Source: Auret, 2026)
As an example: a mineral processing concentrator exhibits time-variance through changing feed ore properties (as the mine plan evolves over time), as equipment wears and is upgraded, as new control philosophies are implemented, as expansion projects are undertaken, as new operating philosophies are enacted. This makes it challenging for a data-driven approach trained on historical data to stay relevant, requiring constant model adaptation interventions.
The issue of control interaction is rooted in the automation hierarchy, and how the various layers of control (and the varying active status of these layers) creates artefacts in process data, limiting the accuracy of estimated causality between measured variables, and therefore the generation of useful actionable advisories. For example, in an open-loop response, a manipulated variable (MV) might have a positive gain effect on a controlled variable (CV), represented as MV -> CV (+). In the close-loop case, a feedback controller would cause a response in the MV due to a deviation of CV from its setpoint (e.g., due to a setpoint change), e.g., CV -> MV (-) and MV -> CV (+). Such bidirectional causality is difficult to extract from noisy data, and also requires data on whether the specific controller is active. Saturation of the MV will also (temporarily) break the correlation between MV and CV.
Finally, and most challenging, is the issue of unobservability. Industrial processes are complex, with key performance indicators influenced by many factors: raw materials, equipment efficiency, operational philosophies, control structures, etc. Back to a mineral processing concentrator as an example: although elemental grade and particle size estimates can be measured online to some extent, other key factors in flotation performance (e.g., mineral liberation, reagent strength and dosages, process water quality) are not available online. Many operational factors (e.g., manually reconfiguration of circuits, equipment modifications, automation equipment and algorithm changes, etc.) are also not captured in typical time series tables considered for analysis. With limited observability on the process to be modelled, the expectation of data-driven model prediction accuracy should be tempered.
It is not all doom and gloom for data-driven approaches: understanding challenging process data characteristics allows one to critically assess the appropriate scope and potential benefit of a specific application, to select appropriate data sources, and to motivate the use of as much additional context as possible (e.g., operating and control philosophies, first-principle and phenomenological models, other data sources such as maintenance logs), and at the very least to manage expectations around data collection requirements and the need for ongoing maintenance of data-driven solutions.
Framework for industrial process monitoring development
A framework that I find useful for understanding the practical requirements of industrial application of process monitoring algorithms is CRISP-DM (cross-industry process for data mining). The following figure gives an overview of the various (and iterative!) phases.
Figure 3. (Source: Auret, 2026)
Process monitoring algorithm training and testing
During the modelling phase (which receives the most academic interest), several data-driven models may be constructed. Models f and their associated parameters 0 are fitted during training, with hyperparameter optimization done during validation (to select the best model structure f* with 0* to optimize a process monitoring objective function J(f,0).
Generalization performance of the optimized workflow should be tested on unseen testing data (due to the time-variance challenges of industrial processes, the unseen testing data should be sampled from time periods occurring after the training and validation data, in order to critically and objectively assess performance in future).
An important sanity check is to compare any new approach to a simple baseline, e.g., linear approaches with simple structures, especially compared to the minimum performance required based on the business case of the process monitoring solution. If a simple approach can provide the minimum necessary benefit, then it is a better investment of resources to focus on building the deployment pipeline (including exception handling, change management, etc.) to get value to the process as soon as possible. This can be followed with rounds of iteration to add complexity to algorithms as is merited by the end-use.
Figure 4. (Source: Auret, 2026)
Challenges and prospects for machine learning in process monitoring
Process monitoring with machine learning is challenging due to time-variance and lack of observability of industrial processes, as well as the requirement of human interaction to provide the final benefit: information and advisories generated by algorithms must be interrogated and actioned by operators, control engineers or others before any value can be realised. Academic research typically does not consider these complexities (lack of large industrial datasets and sufficiently complex simulations contribute to this issue, as well as inappropriate performance metrics).
Given the proliferation of advanced sensing and computational technologies, process monitoring approaches that exploit heterogenous data (e.g., time series tables, images, videos, spectra, text, etc.) show promise in chipping away at the lack of observability challenges. Process knowledge (in the form of process topology, physics-inspired models, operational context) is essential to explicitly incorporate in data-driven methods. Design and testing of end-to-end process monitoring solutions in sufficiently complex simulations (that include process, actuation, sensing, fault, disturbance, control, monitoring and operator intervention modules) will also allow more nuanced and robust solutions. With careful consideration of its limitations, large language models may have some use in capturing and succinctly representing unstructured process context (given that the necessary guardrails against hallucinations are built in).
Figure 3. (Source: Auret, 2026)
Overall, a wonderful time to bring deep domain expertise, practical delivery requirements and powerful tools together!
