Sensor History as Training Data: Lessons from First Deployments

Before we started building AMI Labs, we had a reasonable mental model of what connecting to a process historian would look like. You point the connector at the server, specify a tag list and a time range, pull the data, clean it up, and hand it to the learning pipeline. The reality of every early deployment taught us that this picture was missing a few fairly important dimensions.

This is not a complaint about historian software or about plant data practices. Every one of the issues we encountered was rational from the perspective of how these systems were designed and maintained. But if you are trying to train a behavioral model from that data, you need to understand what you are actually working with before you commit to a training approach.

The first thing you notice: compression artifacts everywhere

Most historians use exception reporting or swinging door compression to reduce storage requirements. Instead of recording every sample at the configured scan rate, the system only writes a new value when the measured value changes by more than a deadband threshold, or when the trend would deviate from a linear interpolation by more than a configurable percentage. For a temperature sensor with a deadband of 0.5 degrees Celsius, the historian might not write a new value for twenty minutes if the process is stable.

When you read this data back at a fixed time step, the historian client interpolates between the recorded exceptions. The resulting time series looks smooth and continuous. But that smoothness is partly real and partly an artifact of the interpolation algorithm. When you are trying to train a model on dynamic response patterns, particularly for sensors with faster dynamics, the interpolated values can mask real transients or introduce apparent dynamics that are not physically real.

Our approach is to always request the raw archived data before interpolation, not the interpolated output, and to treat regions of constant interpolated values as potentially compressed periods. For a training dataset, this means marking those periods differently from genuinely steady-state periods rather than treating all flat segments the same way.

Tag metadata is often incomplete or inconsistent

A historian tag is usually identified by a name like UNIT_02.T-1042.PV, which tells you the unit, the instrument tag number, and the value type. What it often does not tell you, or tells you inconsistently across the tag database, is the engineering unit of the value, the full-scale range of the transmitter, whether the value is the raw input or a processed value downstream of a calculation block, and whether the instrument was recently recalibrated or replaced.

We have seen cases where two temperature tags measuring the same stream were in different units after a calibration contractor updated the historian configuration without changing the tag metadata. The values looked plausible in their respective scales. The discrepancy only appeared when we cross-correlated the two tags and found the correlation coefficient was negative when it should have been strongly positive.

The practical implication is that tag metadata cannot be trusted without cross-validation. For every sensor that matters to the model, we verify the engineering units against a known reference period, check that the value range is physically plausible, and flag any tag where the apparent value suddenly shifts by more than the normal operating range. These flags do not disqualify the data; they identify periods that require context from the process team before they can be used for training.

Maintenance periods are not labeled

During a scheduled turnaround or a pump replacement, the historian continues recording. The sensor readings during that period may look like a valid process state, or they may show values that are physically impossible during normal operation. In either case, without a label or annotation that says "this was a maintenance window," the learning algorithm cannot distinguish between unusual operating conditions and periods when the process was not running at all.

This matters for training because a world model trained on maintenance-period data will include in its behavioral representation things like: the reactor temperature slowly equalizing to ambient over eight hours after a hot shutdown, the pressure dropping to atmospheric as the system is depressurized, and the flow rates going to zero. If the model then encounters a slow temperature decay in normal operation, it may conflate that pattern with the shutdown trajectory rather than recognizing it as an unusual steady-state deviation.

The solution is not to filter out all periods with unusual values, because some of the most informative operating periods look unusual. The solution is to request a maintenance log from the plant team and use it to explicitly label those windows in the training dataset. This is a manual step that cannot be fully automated from the historian data alone, and it is one of the first things we ask for during a new deployment.

Setpoint changes are often not explicitly tagged

To train a causal model, you need to know when control variables changed and by how much. The best source for this is a separate setpoint historian that logs operator actions with timestamps. Some plants have this configured; many do not, or have it configured only for some control loops.

Where a setpoint historian is unavailable, you can infer setpoint changes by looking at the controller output signal and identifying step changes that correlate with a shift in the process variable trajectory. This is workable but introduces uncertainty: a step change in controller output might be a manual setpoint adjustment or an automatic response from a higher-level optimizer. Those two situations call for different treatment in the training data.

We learned early that trying to infer setpoint actions purely from process variable behavior is too unreliable for training a causal model. The model needs to know what was commanded, not just what happened. If a plant does not have a setpoint historian, establishing one, even a simple log of operator-initiated changes from the DCS, is one of the most valuable infrastructure investments they can make before attempting this kind of modeling.

Long-horizon drift confounds steady-state identification

Heat exchanger fouling, catalyst aging, and filter loading all cause slow drifts in process behavior over weeks and months. A reactor that required a feed temperature of 165 degrees Celsius to achieve target yield at the start of a catalyst cycle might require 172 degrees at the end of the same cycle. If you try to identify steady-state operating periods for training, and you treat all periods where the setpoints were constant as equivalent steady-state conditions, you are pooling together operating windows that reflect very different machine states.

This is one of the places where a world model architecture has a concrete advantage over a stationary regression approach. The model can be trained to include a state representation that evolves over time and affects the relationship between inputs and outputs. That state representation can capture fouling-related drift without requiring the operator to explicitly measure or log the fouling level. But only if the training pipeline accounts for the fact that identical setpoints in January and in August may correspond to quite different machine behavior. Segmenting the training data by catalyst cycle or maintenance interval and treating each segment as a semi-independent operating regime is a practical way to handle this.

What we now do differently from how we started

The most important change in our approach has been front-loading the data characterization step. Before we run any training, we now spend time producing a data quality report for the historian pull: compression artifact density by tag, engineering unit verification, identification of apparent maintenance windows, setpoint action availability, and drift analysis over the full training window. This report goes back to the process team for review before training begins. Things that look like data quality issues in the report are often explained by known plant events that simply were not in the historian metadata.

This step feels like overhead when you are trying to get to a live model quickly. But we have never regretted the time spent on it. The alternative, training a model on data you have not fully characterized and then trying to understand why its predictions do not hold up on certain operating regimes, takes considerably longer than the upfront characterization.

The historian is full of information about how your machine behaves. Getting that information into a useful form requires understanding the historian's own behavior first. That is the part nobody wrote a manual for.