Chapter 4: NGL Recovery Deep Neural

Network Reduced Order Model: Design,

Implementation, and Evaluation

4.1 Introduction and Problem Statement

4.1.1 Background

Natural Gas Liquids (NGL) recovery is a critical unit operation in the oil and gas in-
dustry, responsible for extracting valuable hydrocarbon components—predomi-
nantly ethane, propane, butane, and heavier fractions—from raw natural gas
streams. The turbo-expander process is the most widely employed technology for
this purpose, leveraging cryogenic temperatures generated through isentropic ex-
pansion of the feed gas to achieve high recovery efficiencies. In a typical turbo-ex-
pander based NGL recovery plant, the feed gas is cooled and partially condensed
before entering the expander, where rapid pressure reduction produces the low
temperatures needed for fractionation. The liquid product is then separated and
further processed through de-ethanizer and de-butanizer distillation columns to
meet product specifications.

The performance of an NGL recovery plant is governed by a multitude of interact-

ing process variables, including feed gas conditions (temperature, pressure, and
flow rate), expander discharge pressure, and reboiler temperatures in the separa-
tion columns. Understanding and predicting the effects of these variables on key
performance indicators—specifically the ethane content in the Liquefied Petroleum
Gas (LPG) product and the Specific Energy Consumption (SEC)—is essential for
optimal plant design, operation, and control.

4.1.2 The Simulation Challenge

Aspen HYSYS, the industry-standard process simulation software, provides rigor-

ous thermodynamic and mass-balance calculations for modeling NGL recovery
processes. However, each HYSYS simulation run involves solving complex nonlin-

56
ear equations describing phase equilibria, energy balances, and mass transfer
across multiple unit operations. A single steady-state simulation can take several
minutes to converge, and systematic exploration of the operating envelope—vary-
ing six input parameters across their operable ranges—requires hundreds or thou-
sands of individual simulation runs. This computational cost makes HYSYS im-
practical for:

— Real-time process monitoring and optimization

— Rapid what-if analysis and operational studies

— Integration with advanced control systems

— Exploratory data analysis requiring iterative parameter variation

— Deployment in environments where HYSYS licenses are unavailable or

expensive

4.1.3 The Reduced Order Model Approach

A Reduced Order Model (ROM) addresses these limitations by constructing a sim-

plified mathematical surrogate that captures the essential input-output behavior of
the full simulation while being computationally inexpensive to evaluate. Rather than
solving the full system of thermodynamic and mass-balance equations, the ROM
maps input conditions directly to outputs through a learned representation, reduc-
ing evaluation time from minutes to milliseconds.

Machine Learning-based Reduced Order Models (ML-ROMs) leverage the function

approximation capabilities of deep neural networks to learn this mapping from
data. Given a dataset of simulation runs—each consisting of a specific combination
of input conditions and the corresponding HYSYS-predicted outputs—the neural
network is trained to minimize the discrepancy between its predictions and the
ground truth. Once trained, the ML-ROM can generalize to unseen input combina-
tions within the training domain, providing near-instantaneous predictions with ac-
curacy closely matching the original simulation.

This chapter presents the development, implementation, and evaluation of a Deep

Neural Network (DNN) based Reduced Order Model for NGL recovery via the
turbo-expander process. The system integrates data preprocessing, model training

57
with configurable architectures and regularization, K-Fold cross-validation, predic-
tion (both single-point and batch), and process optimization into a unified interac-
tive web application.

58
4.2 System Architecture and Overview

4.2.1 Technology Stack

The ML-ROM system is built upon a carefully selected set of open-source tech-
nologies, each serving a specific role in the pipeline:

— Python 3.8+: The core programming language, chosen for its extensive
ecosystem in scientific computing and machine learning.

— TensorFlow 2.x / Keras: Provides the deep learning framework for

constructing, compiling, training, and saving neural network models. Keras
offers a high-level API that facilitates rapid prototyping while retaining access to
low-level customization.

— scikit-learn: Supplies preprocessing utilities (StandardScaler, MinMaxScaler,

RobustScaler), train-test splitting functionality, and evaluation metrics (R²,
MSE, MAE).

— NumPy and Pandas: Handle numerical operations and tabular data

manipulation, respectively.
— SciPy: Provides the differential_evolution global optimizer for process
optimization.

— Streamlit: Serves as the web application framework, enabling rapid

development of interactive data science dashboards with reactive widgets,
real-time plots, and a tabbed interface.

— Plotly: Powers all interactive visualizations within the dashboard, including

scatter plots, histograms, heatmaps, 3D surface plots, and animated training
progress.

— Matplotlib: Used in the standalone CLI script for static publication-quality

figures.

— fpdf2 and Kaleido: Enable PDF report generation with embedded plots and
metrics.

59
4.2.2 Modular Architecture

The project follows a modular design pattern that separates concerns across dis-
tinct modules, facilitating maintainability, testability, and reuse:

NGLRecoveryModel/
├── [Link] # Streamlit web application (4 tabs)
├── dnn_rom_model.py # Standalone CLI training script
├── [Link] # Python dependencies
├── ngl_data.csv # Primary training dataset
├── model/ # Core ML modules
│ ├── __init__.py
│ ├── [Link] # Preprocessing, scaling, model management
│ ├── [Link] # Training, K-Fold CV, architecture
presets
│ ├── [Link] # Single/batch prediction functions
│ └── [Link] # PDF report generation
├── saved_models/ # Trained model artifacts
│ ├── ngl_dnn_model.keras # Keras saved model
│ ├── x_scaler.pkl # Fitted input scaler
│ ├── y_scaler.pkl # Fitted output scaler
│ ├── training_history.pkl # Loss curves over epochs
│ ├── model_metadata.pkl # Architecture & regularization config
│ └── input_ranges.pkl # Training data min/max ranges
└── assets/
└── [Link] # Shadcn-inspired UI stylesheet

The model/ package encapsulates all machine learning logic. The [Link] mod-
ule handles data loading, validation, splitting, scaling, and persistence of trained
artifacts. The [Link] module provides functions for model creation, training (in-
cluding K-Fold cross-validation), and evaluation. The [Link] module exposes
clean interfaces for single-point and batch prediction with automatic scaling and in-
verse transformation. The [Link] module generates downloadable PDF re-
ports summarizing training results, K-Fold metrics, and prediction analysis.

The [Link] file orchestrates the entire user experience through a Streamlit-based
web interface, while dnn_rom_model.py provides a command-line alternative for
training and evaluation without the web interface.

60
4.2.3 Data Flow Pipeline

The complete data flow through the system proceeds as follows:

1. Data Generation: Aspen HYSYS simulations are run across varying

combinations of the six input parameters using the Case Study Manager,
producing a CSV dataset.

2. Data Ingestion: The CSV is loaded and validated, with automatic

deduplication and mole-fraction-to-percentage conversion for the ethane
output.
3. Preprocessing: Inputs and outputs are split, scaled using the selected scaler
type, and divided into training and test sets.

4. Training: The DNN model is trained with configurable architecture,

regularization, and callbacks. Training history and model artifacts are persisted.
5. Prediction: New input data is scaled using the saved scaler, passed through
the trained network, and inverse-transformed to produce predictions in the
original units.

6. Optimization: The trained model serves as the objective function for SciPy's
differential evolution optimizer, searching for input conditions that minimize the
target output.

61
4.3 Process Variables and Data Description

4.3.1 Input Variables (Predictors)

The ML-ROM accepts six input variables that characterize the operating conditions
of the NGL recovery process. These variables were selected based on their direct
influence on both product quality and energy consumption, as determined by
process engineering knowledge and parametric studies in HYSYS.

Table 4.3.1: Input Variables

Typical
Variable Symbol Unit Physical Description
Range

Feed x₁ °C Temperature of the incoming raw −40 to 40

Temperature natural gas entering the turbo-expander
plant

Feed Pressure x₂ kPa Pressure of the incoming raw natural 4,500 to

gas 7,500

Feed Molar Flow x₃ kmol/h Volumetric rate of the incoming natural 1,000 to
gas stream 5,000

Expander x₄ kPa Discharge pressure of the turbo- 1,200 to

Pressure expander, governing the degree of 4,500
expansion and cooling

De-ethanizer x₅ °C Reboiler temperature of the de- 60 to 140

Temperature ethanizer column, controlling C2
separation

Debutanizer x₆ °C Reboiler temperature of the 100 to

Temperature debutanizer column, controlling C4 250
separation

The feed conditions (x₁, x₂, x₃) define the thermodynamic state and composition of
the inlet gas. The expander outlet pressure (x₄) directly determines the expansion
ratio and thus the refrigeration duty available for condensation. The column re-
boiler temperatures (x₅, x₆) control the separation sharpness in the de-ethanizer

62
and debutanizer, respectively, fundamentally affecting the product composition and
energy requirements.

4.3.2 Output Variables (Targets)

The model predicts two critical performance indicators:

Table 4.3.2: Output Variables

Variable Symbol Unit Physical Description

Ethane y₁ % Mass or mole percentage of ethane in the LPG

Percentage product stream. Lower values indicate higher
separation efficiency (less C2 in the product). When
exported from HYSYS as a mole fraction (0.0–1.0), it
is automatically converted to a percentage (0–100%)
to balance the loss function.

Specific Energy y₂ kWh/ton Total energy consumed per unit mass of NGL
Consumption produced. This encompasses compressor work,
reboiler duty, and pump power. Lower SEC indicates
more energy-efficient operation.

The ethane percentage (y₁) quantifies product purity, while the SEC (y₂) quantifies
energy efficiency. In practice, these two objectives often conflict—reducing ethane
content typically requires more energy—making simultaneous optimization of both
a non-trivial engineering challenge. The LPG product specification for ethane con-
tent is critical because excessive ethane degrades product quality and may violate
contractual or regulatory limits, while high SEC values indicate poor energy effi-
ciency and increased operating costs.

4.3.3 Parametric Study and Justification of Selected Variables

To build a high-fidelity Reduced Order Model (ROM), we selected process vari-

ables that exert the strongest thermodynamic and operational leverage over the
plant’s two primary performance targets: product purity (Ethane Percentage in
LPG, y1 ) and energy efficiency (Specific Energy Consumption, y2 ).
​ ​

The selected inputs represent three feed-related boundary conditions (Feed

Temperature, Feed Pressure, and Feed Molar Flow) and three core process con-

63
trol setpoints (Expander Pressure, De-ethanizer Reboiler Temperature, and
Debutanizer Reboiler Temperature). Below, we justify the selection of each variable
and analyze their impact on the process performance using parametric study
graphs generated from the HYSYS simulation dataset.

[Link] Feed Gas Temperature (x1 ) ​

— Justification: The inlet feed gas temperature determines the initial enthalpy
and the fraction of liquids that will condense in the upstream gas-gas exchang-
ers and chiller before reaching the expander.

— Process Impact and Graphical Analysis:

— Ethane Percentage vs. Feed Temperature: As shown in the graphs

below, higher feed temperatures generally shift the thermodynamic
equilibrium, reducing the pre-condensation of heavier hydrocarbons. This
can result in a higher proportion of lighter components (like ethane)
carrying over or altering the column feed composition.

— SEC vs. Feed Temperature: An increase in feed temperature shifts

refrigeration loads onto the downstream cooling systems, increasing the
specific energy consumption (SEC) required to achieve the necessary
cryogenic separation temperatures.

Figure 4.3.1: Ethane Fraction vs. Feed

Figure 4.3.2: SEC vs. Feed Temperature
Temperature

[Link] Feed Gas Pressure (x2 ) ​

— Justification: The feed pressure sets the high-pressure boundary for the ex-
Pfeed
pansion process. It directly determines the pressure ratio ( Pexpander ) available to
​

​
​

drive the turbo-expander.

— Process Impact and Graphical Analysis:

64
 — Ethane Percentage vs. Feed Pressure: Higher feed pressure increases
the pressure drop across the expander, leading to lower expander outlet
temperatures due to isentropic expansion. This causes more ethane and
heavier components to condense, thereby increasing the ethane
percentage in the condensed liquid phase that feeds the de-ethanizer, as
demonstrated in the graph.

— SEC vs. Feed Pressure: Although higher feed pressure requires more
compressor work upstream, it also provides more expansion energy
(recovered by the expander-compressor link) and lower cryogenic
temperatures without additional external cooling, which can lead to a net
reduction in the specific energy consumption (SEC) per ton of NGL
recovered within certain operating limits.

Figure 4.3.3: Ethane Fraction vs. Feed Pressure Figure 4.3.4: SEC vs. Feed Pressure

[Link] Feed Molar Flow (x3 ) ​

— Justification: The gas flow rate dictates the plant throughput and directly af-
fects velocities, pressure drops, and heat transfer efficiency throughout the
heat exchangers and distillation columns.

— Process Impact and Graphical Analysis:

— Ethane Percentage vs. Feed Molar Flow: The graph shows how changes
in throughput alter the residence time and heat load on the separation
columns, affecting the separation efficiency and resulting in mild variations
in the final product ethane content.

— SEC vs. Feed Molar Flow: Higher throughput can improve specific energy
efficiency by spreading fixed thermal and mechanical losses over a larger
mass of product, reducing the specific energy consumption (SEC) up to a

65
 point before equipment constraints (like compressor capacity or column
flooding) begin to dominate.

Figure 4.3.5: Ethane Fraction vs. Feed Molar Figure 4.3.6: SEC vs. Feed Molar Flow
Flow

[Link] Expander Discharge Pressure (x4 ) ​

— Justification: Expander pressure is the primary operational lever for control-

ling the process's cryogenic temperature. It dictates the pressure at which the
gas is expanded and fed to the low-temperature separators.

— Process Impact and Graphical Analysis:

— Ethane Percentage vs. Expander Pressure: Lowering the expander

pressure increases the expansion ratio, dropping the discharge
temperature. This maximizes the condensation of light hydrocarbons,
leading to higher ethane recovery and a higher ethane percentage in the
liquid product, as shown in the graph.
— SEC vs. Expander Pressure: While lower expander pressure recovers
more liquid, it requires the residue sales gas to be compressed over a
much larger pressure differential to reach pipeline delivery pressure. This
re-compression duty is the single largest energy sink in the plant, causing
the specific energy consumption (SEC) to rise sharply as expander
pressure decreases.

66
 Figure 4.3.8: SEC vs. Expander Pressure
Figure 4.3.7: Ethane Fraction vs. Expander
Pressure

[Link] De-ethanizer Temperature (x5 ) ​

— Justification: The de-ethanizer reboiler temperature determines the amount of

thermal energy supplied to strip light key components (methane and ethane)
from the column bottoms.

— Process Impact and Graphical Analysis:

— Ethane Percentage vs. De-ethanizer Temperature: Raising the de-

ethanizer reboiler temperature increases the vapor boilup, stripping more
ethane out of the liquid bottoms product and sending it overhead.
Consequently, the ethane percentage in the final liquid product drops
dramatically with higher temperatures, as shown by the steep negative
slope in the graph.

— SEC vs. De-ethanizer Temperature: Increasing the reboiler temperature

requires a direct input of thermal energy (usually hot oil or steam), which
increases the specific energy consumption (SEC) of the plant.

Figure 4.3.9: Ethane Fraction vs. De-ethanizer Figure 4.3.10: SEC vs. De-ethanizer
Temperature Temperature

67
[Link] Debutanizer Temperature (x6 ) ​

— Justification: The debutanizer reboiler temperature is selected to capture the

downstream fractionation dynamics and LPG product purity. Although it does
not directly dictate the primary ethane recovery or the main energy balance
shown in the expander and de-ethanizer loops, it controls the separation of
butane and heavier fractions (C4 +), which establishes the final mass balance
​

of the LPG stream. Properly predicting and optimizing this temperature ensures
that downstream specification changes are captured by the ROM.

4.3.4 Dataset Characteristics

The primary training dataset ( ngl_data_clean.csv ) contains 3,241 samples gen-

erated through systematic variation of the six input parameters using Aspen
HYSYS. Each sample represents a converged steady-state simulation with unique
input combinations and the corresponding output values.

The raw HYSYS export column names are automatically mapped to standardized
internal names:

HYSYS Export Name Internal Name

Raw gas - Temperature Feed_Temp

Raw gas - Pressure Feed_Press

Raw gas - Molar Flow Feed_Flow

turbo expander discharge vessel - Vessel Pressure Expander_Press

De-ethanizer - Temperature Est (Reboiler) Deethanizer_Temp

De-Butanizer - Temperature Est (Reboiler) Debutanizer_Temp

LPG - Master Comp Mole Frac (Ethane) Ethane_Pct

SPRDSHT-1 - C2: Final_SEC SEC

An important preprocessing detail is the handling of the ethane output: HYSYS ex-
ports this value as a mole fraction in the range [0, 1] (e.g., 0.0285 for 2.85%).
Because the SEC values range in the tens (e.g., 52.8 kWh/ton), leaving ethane in
its native scale would cause the loss function to be dominated by the SEC target,

68
severely biasing the model. The system therefore automatically detects values be-
low 2.0 and multiplies by 100 to convert to percentages, balancing both targets in
the loss function.

69
4.4 Data Preprocessing Pipeline

4.4.1 Data Loading and Validation

The load_and_validate_data function in model/[Link] handles all data inges-

tion and validation. The function accepts either a file path or a file-like object (en-
abling both CLI and web upload), reads the CSV using Pandas, and performs the
following steps:

1. Column name cleaning: All column headers are stripped of leading/trailing

whitespace using [Link]() . This is necessary because HYSYS
exports and manual CSV editing can introduce stray spaces that break column
name matching. For example, " Feed_Temp " is normalized to "Feed_Temp".

2. NaN column removal: Columns consisting entirely of NaN values are dropped
using [Link](axis=1, how='all') . This handles a common HYSYS export
artifact: when the Case Study Manager exports data, it may insert an empty
separator column between the input and output columns. The raw CSV file
ngl_data.csv demonstrates this issue—it contains an unnamed empty column
between the Debutanizer Temperature and Ethane columns, which would cor-
rupt the column indexing if not removed. After dropping NaN columns, the col-
umn names are re-stripped to handle any shifted indices.

3. Deduplication: During training mode ( is_training=True ), duplicate rows are

removed using df.drop_duplicates() . HYSYS Case Studies with nested vari-
able ranges can produce identical rows when a variable change does not affect
one of the outputs (e.g., when debutanizer temperature variation has no effect
in certain operating regions). Duplicates cause two problems: they artificially in-
flate training accuracy for specific input regions (the model sees those points
more often), and they reduce the effective size of the validation set. After dedu-
plication, the dataset typically shrinks from ~3,900 raw rows to ~3,241 unique
samples.

4. Column mapping: The first six columns are assigned as inputs (X) and, if
present, columns seven and eight are assigned as outputs (Y). This positional

70
 approach means the column order must be consistent: inputs first, then out-
puts. The function returns the column names alongside the arrays so the user
interface can display proper labels.

5. Mole fraction conversion: The first output column (Y[:, 0], representing
Ethane content) is checked: if the maximum value is below 2.0, all values in
that column are assumed to be mole fractions and multiplied by 100. This auto-
matic detection handles both scenarios—HYSYS exports mole fractions in the
range [0, 1] (e.g., 0.0285 for 2.85%), while some users may manually enter
percentage values. The threshold of 2.0 (rather than 1.0) accounts for the pos-
sibility of ethane values slightly above 1% in certain operating conditions, en-
suring that legitimate small percentage values are not accidentally multiplied by
100.

Why this matters: Without these cleaning steps, the model would receive incon-
sistent, noisy, or mis-scaled data, leading to poor convergence during training, bi-
ased predictions toward over-represented input combinations, and an imbalanced
loss function that prioritizes one target over the other. The cleaning pipeline en-
sures data quality and consistent scaling before the model ever sees the data.

4.4.2 Train-Test Splitting

The dataset is split into training and test subsets using scikit-learn's
train_test_split with a configurable ratio (default 20% test split). The random
state is fixed at 42 to ensure reproducibility:

D = Dtrain ∪ Dtest ,
​ ​ ∣Dtest ∣/∣D∣ = 0.20
​

When K-Fold cross-validation is active, this split is bypassed in favor of the fold-
based partitioning described in Section 4.8.

4.4.3 Feature Scaling

Feature scaling is a critical preprocessing step for neural network training.

Because the six input variables span vastly different ranges (e.g., Feed Pressure is
on the order of thousands of kPa, while Feed Temperature ranges from −40 to
40°C), and the two outputs differ in magnitude (Ethane in single-digit percentages,

71
SEC around 50 kWh/ton), unscaled features would cause gradient descent to con-
verge slowly and unevenly. Without scaling, the optimizer would take large steps in
the direction of high-magnitude features (Feed Pressure) and tiny steps along low-
magnitude features (Feed Temperature), resulting in slow convergence, unstable
training, and suboptimal solutions. After scaling, all features occupy comparable
numerical ranges, the gradient contributions from each feature are balanced, and
the optimizer navigates the loss landscape efficiently.

The system supports three scaling strategies, each with distinct mathematical for-
mulations and practical trade-offs:

[Link] Standard Scaler (Z-Score Normalization)

The Standard Scaler transforms each feature to have zero mean and unit variance:

x−μ
z= σ
​

where μ is the sample mean and σ is the sample standard deviation. After transfor-
mation, each feature has a mean of approximately 0 and a standard deviation of
approximately 1. Outliers remain present in the data but their influence is reduced
because they are expressed in terms of standard deviations from the mean rather
than in raw units.

When to use: This is the default and most commonly used scaler, suitable when
features follow approximately Gaussian (normal) distributions. It works well with
neural networks because the resulting values are centered around zero, which
aligns with the initial random weight distributions (typically initialized from
N (0, 0.05) or Glorot uniform). For the NGL dataset, Feed Pressure and Feed Flow
have relatively symmetric distributions, making Standard Scaler a good fit.

Practical impact: For the NGL dataset with Standard Scaler, the model typically
converges within 50–100 epochs and achieves R² > 0.97. The scaler parameters
(mean and standard deviation for each of the 6 input features and 2 output targets)
are stored as part of the model artifacts.

[Link] MinMax Scaler

The MinMax Scaler maps each feature to a fixed range [0, 1]:

72
 x−xmin
z= xmax −xmin
​
​

​
​

After transformation, all values fall between 0 and 1, with the minimum value map-
ping to 0 and the maximum value mapping to 1. The transformed distribution pre-
serves the original shape but compresses it into the unit interval.

When to use: This is particularly appropriate for neural networks because it

bounds all inputs to the same range, preventing any single feature from dominating
the gradient. It is the preferred choice when the activation functions operate in a
bounded range (e.g., sigmoid or tanh), though it also works well with ReLU.
However, MinMax Scaler is highly sensitive to outliers: a single extreme value
compresses the rest of the data into a narrow sub-range. For example, if one
HYSYS simulation produces an abnormally high Feed Pressure of 10,000 kPa
(compared to the typical range of 4,500–7,500 kPa), the majority of data would be
squeezed into the range [0, 0.75], wasting the available numeric precision.

Practical impact: For clean NGL data without significant outliers, MinMax Scaler
performs similarly to Standard Scaler. However, if outliers are present, the model
may take longer to converge because the compressed input range leads to smaller
gradient magnitudes for the majority of samples.

[Link] Robust Scaler

The Robust Scaler uses the median and interquartile range (IQR) to reduce the in-
fluence of outliers:
x−median(x) x−median(x)
z= IQR(x)
​ = Q3 (x)−Q1 (x)
​ ​
​

where Q1 and Q3 are the 25th and 75th percentiles, respectively. Unlike Standard
​ ​

Scaler (which uses mean and standard deviation) and MinMax Scaler (which uses
min and max), the Robust Scaler relies on statistics that are inherently resistant to
outliers. The median is unaffected by extreme values, and the IQR only reflects the
spread of the central 50% of the data.

When to use: This scaler is recommended when the dataset contains significant
outliers that would distort the mean, standard deviation, or min/max values. In
HYSYS-generated datasets, outliers can arise from simulations that converge at
extreme operating conditions (e.g., very low temperatures or pressures near ther-

73
modynamic boundaries). If these data points are not removed during cleaning,
Robust Scaler ensures they do not distort the feature representation.

Practical impact: When outliers are present, Robust Scaler prevents the com-
pressed-range problem that plagues MinMax and the skewed-centre problem that
affects Standard Scaler. For clean NGL data, the practical difference between
Standard and Robust Scaler is minimal (typically <0.01 R² difference). For data
with outliers, Robust Scaler can improve R² by 0.02–0.05 compared to Standard
Scaler.

Table 4.4.1: Scaler Selection Guide

Mathematical Sensitive to Expected

Scaler Best Practice For
Basis Outliers? Convergence

Standard Mean & Std Dev Yes Clean, normally Fast, stable
distributed data

MinMax Min & Max Very sensitive Bounded data, no Fast, may be slow
outliers with outliers

Robust Median & IQR No Data with outliers Stable under all
conditions

4.4.4 Scaler Persistence and Inference Consistency

A critical design consideration is that the scalers must be fitted on the training data
only and then applied (without refitting) to the test data and any future prediction in-
puts. The system enforces this by calling fit_transform() on the training set and
transform() on the test and prediction sets. The fitted scaler parameters (means,
standard deviations, min/max values, or medians/IQRs) are serialized to disk using
joblib as x_scaler.pkl and y_scaler.pkl , ensuring that the exact same trans-
formation is applied during inference.

The inverse transformation y = f −1 (y^) is equally important: predicted outputs y^

​

are produced in the scaled domain and must be transformed back to the original
units using y_scaler.inverse_transform() before being reported to the user or
compared against ground truth values.

74
4.5 Deep Neural Network Architecture

4.5.1 Architecture Presets

The model is implemented as a feedforward (sequential) Deep Neural Network us-

ing the Keras Sequential API. Three architecture presets are available, each of-
fering a different trade-off between representational capacity and training speed:

Table 4.5.1: Architecture Presets

Hidden Total
Preset Output Best For
Layer Sizes Parameters

Simple 16 → 8 2 ~200 Small datasets (<200 samples),

rapid prototyping

Standard 32 → 16 → 8 2 ~750 Balanced performance on typical

datasets

Advanced 64 → 32 → 2 ~3,000 Large datasets (>500 samples),

16 → 8 complex nonlinear patterns

The naming convention follows the pattern: first hidden layer receives the 6-dimen-
sional input, subsequent layers progressively reduce dimensionality in a funnel pat-
tern, and the final output layer produces the 2-dimensional prediction. This "en-
coder-like" funnel structure encourages the network to learn increasingly com-
pressed and abstract representations of the input-output mapping.

4.5.2 Mathematical Formulation

[Link] Forward Propagation

For a network with hidden layer sizes {n1 , n2 , … , nL } and input dimension d = 6,
​ ​ ​

output dimension m = 2, the forward pass through layer l is:

h(l) = σ (W(l) a(l−1) + b(l) )

where:

— a(0) = x ∈ R6 is the scaled input vector

75
— W(l) ∈ Rnl ×nl−1 is the weight matrix for layer l
​ ​

— b(l) ∈ Rnl is the bias vector for layer l

​

— σ(⋅) is the activation function

— h(l) ∈ Rnl is the output (pre- or post-activation) of layer l

​

For the Standard architecture (32 → 16 → 8 → 2):

h(1) = ReLU (W(1) x + b(1) ) , W(1) ∈ R32×6 h(2) =

ReLU (W(2) h(1) + b(2) ) , W(2) ∈ R16×32 h(3) =
ReLU (W(3) h(2) + b(3) ) , W(3) ∈ R8×16 y
^ = W(4) h(3) + b(4) ,
​ W(4) ∈ R2×8

[Link] Activation Functions

ReLU (Rectified Linear Unit) is used for all hidden layers:

fReLU (z) = max(0, z)

​

ReLU introduces nonlinearity while maintaining sparse activation (negative inputs

produce zero outputs), which improves computational efficiency and helps mitigate
the vanishing gradient problem in deep networks.

Linear activation (identity function) is used for the output layer:

flinear (z) = z
​

Since the targets (Ethane % and SEC) are continuous unbounded values, a linear
output layer allows the network to produce predictions across the full real number
domain without artificially constraining the output range.

[Link] Parameter Count Analysis

For the Standard architecture (32 → 16 → 8 → 2) with 6 inputs and 2 outputs, the
total parameter count is:

76
 Layer Weights Biases Total

Hidden 1 (6 → 32) 6 × 32 = 192 32 224

Hidden 2 (32 → 16) 32 × 16 = 512 16 528

Hidden 3 (16 → 8) 16 × 8 = 128 8 136

Output (8 → 2) 8 × 2 = 16 2 18

Total 906

With Batch Normalization layers added, each BN layer introduces 2 × nl trainable ​

parameters (scale γ and shift β ) and 2 × nl non-trainable parameters (running ​

mean and running variance), adding 4 × (32 + 16 + 8) = 224 additional

parameters.

4.5.3 Loss Function

The model uses Mean Squared Error (MSE) as the loss function:

1 N 2 2
LMSE =​

N
​ ∑i=1 ∑j=1 (yi,j − y^i,j )
​ ​ ​ ​ ​

where N is the number of samples in the batch, yi,j is the true value, and y^i,j is the
​ ​ ​

predicted value for output j (ethane or SEC) of sample i. The MSE penalizes large
errors quadratically, making it sensitive to outliers but driving the model toward ac-
curate predictions across both targets. Since both outputs have been scaled to
comparable ranges (via the output scaler), the loss function balances learning be-
tween the two targets naturally.

4.5.4 Adam Optimizer

The Adaptive Moment Estimation (Adam) optimizer is used for gradient-based pa-
rameter updates. Adam maintains per-parameter adaptive learning rates using first
and second moment estimates:

The first moment estimate (exponential moving average of gradients):

mt = β1 ⋅ mt−1 + (1 − β1 ) ⋅ gt
​ ​ ​ ​ ​

The second moment estimate (exponential moving average of squared gradients):

77
vt = β2 ⋅ vt−1 + (1 − β2 ) ⋅ gt2
​ ​ ​ ​ ​

Bias-corrected estimates:

mt vt
^t =
m ​

1−β1t
, ​

​
​ v^t = ​

1−β2t
​

​
​

Parameter update:

θt+1 = θt −
​ ​
α
v^t +ϵ
​ ​
​ ⋅m
^t ​

where α is the learning rate (default 0.001), β1 = 0.9, β2 = 0.999, and ϵ = 10−7 . ​ ​

The default learning rate can be adjusted by the user through the dashboard inter-
face (selectable from {0.0001, 0.0005, 0.001, 0.005, 0.01}).

78
4.6 Regularization Techniques

Regularization is essential for preventing overfitting—a condition where the model

memorizes training data patterns including noise, resulting in poor generalization to
unseen data. The ML-ROM system implements three complementary regulariza-
tion strategies that can be independently enabled or configured.

4.6.1 Dropout

Dropout randomly deactivates a fraction of neurons during each training iteration,

forcing the network to develop redundant representations and reducing inter-neu-
ron co-adaptation:

Training phase: For each neuron in a Dropout layer, a binary mask is sampled:

~ (l) rj
(l)
(l)
hj = ⋅ hj
​

1−p
​ ​ ​

(l)
where rj ∼ Bernoulli(1 − p) is a binary mask sampled independently for each
​

neuron, and p is the dropout rate. The (1 − p) scaling factor (inverted dropout) en-
sures that the expected activation magnitude remains consistent between training
and inference, eliminating the need for weight scaling at test time.

Inference phase: During prediction, the Dropout layer becomes a passthrough—

no neurons are dropped, and no scaling is applied. This is because the inverted
dropout scaling during training already accounts for the expected activation
magnitude.

Configuration: The dropout rate p is configurable from 0.05 to 0.50 in steps of

0.05, with a default of 0.20. Dropout layers are placed after each hidden layer's ac-
tivation (and after Batch Normalization, if enabled). During inference, all dropout
layers are automatically disabled by Keras.

What difference does it make? Dropout has several observable effects on train-
ing and model quality:

— Slower training convergence: Because each training step uses only a subset
of neurons, the gradient signal is noisier and the model requires more epochs

79
 to converge. With a dropout rate of 0.2, training typically takes 20–40% more
epochs to reach the same validation loss compared to training without dropout.

— Reduced overfitting gap: The most important effect. Without dropout, the
model may achieve very low training loss while validation loss plateaus or
increases—this is the hallmark of overfitting. With dropout, the gap between
training and validation loss narrows significantly. For the NGL dataset, the
training-validation loss gap typically decreases from ~30% to <5% with dropout
enabled.

— Improved generalization: The model produces more consistent predictions on

unseen data. Without dropout, the model may "memorize" specific training
examples, especially in regions of the input space that are densely sampled.
Dropout forces each neuron to learn features that are useful in combination
with many different subsets of other neurons, producing more robust
representations.

— Rate selection guidance: A dropout rate of 0.10–0.20 provides light

regularization suitable for the NGL dataset (~3,200 samples). Rates of 0.30–
0.50 provide strong regularization that may cause underfitting (validation loss
plateaus at a higher value than necessary). For very small datasets (<200
samples), rates of 0.30–0.40 may be beneficial to prevent the model from
simply memorizing all training samples.

4.6.2 Batch Normalization

Batch Normalization (BatchNorm) normalizes the activations of each hidden layer

across the current mini-batch during training, stabilizing and accelerating
convergence:

Training phase:

1 m (l)
μB = ∑i=1
B
hi
​

mB
​ ​ ​ ​

2
i=1 (hi − μB )
1 (l)
σB2 = ∑m B ​

mB
​ ​ ​ ​ ​

(l)
^ (l) =
h
hi −μB ​ ​

i ​

2 +ϵ
σB ​
​

(l) ^ (l) + β (l)

BN(hi ) = γ (l) ⋅ h
​

i ​

80
 2
where μB and σB
​ are the batch mean and variance, mB is the batch size, ϵ is a
​ ​

small constant (default ϵ = 10−3 ) for numerical stability, and γ (l) and β (l) are learn-
able scale and shift parameters that allow the layer to recover the original repre-
sentation if needed.

Inference phase: During prediction, the running mean and running variance (com-
puted as exponential moving averages during training) are used instead of batch
statistics:

h−μrunning
BN(h) = γ ⋅ +β
​

2
σrunning +ϵ
​
​

This ensures deterministic predictions regardless of batch size or composition.

Placement: BatchNorm is placed between the Dense layer's linear transformation

and the ReLU activation, following best practices. When both BatchNorm and
Dropout are enabled, the layer order is: Dense → BatchNorm → ReLU → Dropout.

What difference does it make? BatchNorm has several practical effects:

— Faster convergence: By maintaining consistent activation distributions

throughout training, BatchNorm allows the use of higher learning rates and
reduces the number of epochs needed to converge. For the NGL model,
enabling BatchNorm typically reduces the number of epochs to convergence
by 30–50%, from ~150 epochs to ~70–100 epochs. This is because
BatchNorm addresses the "internal covariate shift" problem—where the
distribution of each layer's inputs changes as the previous layer's weights
update—by normalizing the activations at each layer.

— Smoother loss curves: Training with BatchNorm produces smoother, more

monotonic loss curves compared to training without it. Without BatchNorm, the
training loss may exhibit erratic oscillations, especially in the early epochs
when weight magnitudes are still being calibrated. With BatchNorm, these
oscillations are significantly reduced.

— Mild regularization effect: Because the batch statistics (mean and variance)
are computed from a mini-batch rather than the full dataset, they introduce a
small amount of noise into the training process. This noise has a regularizing
effect similar to (but weaker than) Dropout. In practice, this means a model with

81
 BatchNorm alone may achieve slightly better generalization than a model
without any regularization.

— Additional parameters: Each BatchNorm layer introduces 2 × nl trainable ​

parameters (γ and β ) and 2 × nl non-trainable parameters (running mean and ​

running variance). For the Standard architecture, this adds 2 × (32 + 16 +

8) = 112 trainable parameters, increasing the model from 906 to 1,018 total
trainable parameters. This is a negligible increase (<1% of the parameter
budget) relative to the convergence benefits.

— Interaction with Dropout: When BatchNorm and Dropout are used together
(the default configuration), they complement each other well. BatchNorm
stabilizes the activation magnitudes, which prevents Dropout from producing
extremely large or small activations when neurons are dropped. Without
BatchNorm, combining Dropout with a high learning rate can cause numerical
instability.

4.6.3 L2 Weight Regularization (Weight Decay)

L2 regularization adds a penalty term to the loss function proportional to the

squared magnitude of all trainable weights, discouraging the network from learning
excessively large weights:
L
Ltotal = LMSE + λ ∑l=1 ∥W(l) ∥2F
​ ​ ​ ​

(l)
where ∥W(l) ∥2F = ∑i,j (Wij )2 is the Frobenius norm squared of the weight matrix
​ ​ ​

in layer l, and λ is the regularization strength.

The gradient of the L2 penalty with respect to each weight is:

∂
∂Wij ​
​ (λ∥W∥2F ) = 2λWij​ ​

This is equivalent to adding 2λWij to each weight gradient, effectively shrinking ​

weights toward zero at each update step. At each gradient descent iteration, the
weight update becomes:

= Wij − α ( ∂L
∂Wij + 2λWij ) = (1 − 2αλ)Wij − α ∂Wij
(t+1) (t) MSE ∂LMSE (t) (t)
Wij ​ ​

​
​

​ ​ ​

​
​

This shows that L2 regularization shrinks each weight by a factor of (1 − 2αλ) at

every step before applying the gradient—a process called "weight decay." The L2

82
rate λ is configurable from {0.0001, 0.0005, 0.001, 0.005, 0.01}, defaulting to 0.001.
Note that L2 regularization is applied only to the hidden layer weight matrices, not
to biases or the output layer.

What difference does it make? L2 regularization has distinct effects compared to

Dropout and BatchNorm:

— Smoother function mappings: By constraining weight magnitudes, L2

regularization forces the network to learn smoother, more gradual mappings
rather than sharp, oscillatory functions. This is particularly important for the
NGL recovery model because the underlying physics (thermodynamic
equilibrium, energy balances) produce smooth, continuous functions. A model
with unconstrained weights might "wiggle" between training points, fitting noise
rather than the true underlying relationship.
— Reduced model sensitivity: L2-regularized models are less affected by small
perturbations in the input. Without L2, a minor change in Feed Temperature
might cause a disproportionately large change in the predicted Ethane %,
which is physically unrealistic. With L2, predictions vary smoothly with input
changes, producing more physically plausible prediction curves.

— Interaction with other techniques: L2 regularization is off by default because

Dropout and BatchNorm already provide regularization for the NGL dataset.
Enabling L2 in addition to Dropout can lead to excessive regularization—the
model may underfit, reflected in higher training and validation loss that plateaus
without converging. The recommended approach is: (1) start with Dropout +
BatchNorm (the default), (2) if validation R² is significantly lower than training
R² (gap > 0.05), enable L2 with a rate of 0.001, (3) if still overfitting, try rates of
0.005 or 0.01. L2 rates above 0.01 typically cause severe underfitting and are
not recommended.

— Observable training effect: When L2 is enabled, the training loss decreases

more slowly and converges to a higher final value compared to training without
L2. This is expected—the regularization term adds to the total loss, preventing
the optimizer from achieving arbitrarily low MSE. However, the validation loss
typically reaches a lower final value because the model generalizes better.

83
4.6.4 Combined Regularization Strategy

The three techniques address different aspects of overfitting and are

complementary:

— Dropout prevents co-adaptation of neurons by randomly removing them during

training, creating an implicit ensemble of thinned networks. Each training step
sees a different subset of the network, forcing every neuron to learn features
that are useful in combination with any other subset of neurons.

— BatchNorm reduces internal covariate shift, allowing higher learning rates and
acting as a mild regularizer through the noise injected by batch statistics. It also
produces smoother optimization landscapes that make gradient descent more
stable.

— L2 Regularization constrains weight magnitudes, producing smoother function

mappings and reducing model sensitivity to input perturbations. Unlike
Dropout, which operates on activations, L2 operates on the weights
themselves.

The default configuration (Dropout = 0.20, BatchNorm = enabled, L2 = disabled)

provides a balanced level of regularization suitable for most NGL datasets. For
very small datasets (<200 samples), increasing the dropout rate and enabling L2
regularization is recommended.

Table 4.6.1: Regularization Configuration Impact on NGL Model

84
 Typical Typical
Configuration Training Validation Gap Overfitting? Recommendation
R² R²

No regularization 0.999+ 0.85–0.92 >0.07 Severe Not recommended

Dropout only 0.98– 0.95–0.97 0.02– Mild Acceptable

(0.2) 0.99 0.03

BatchNorm only 0.99+ 0.93–0.96 0.03– Light May overfit on small

0.05 data

Dropout (0.2) + 0.98– 0.96–0.98 0.01– Minimal Default,

BatchNorm 0.99 0.02 recommended

Dropout (0.2) + 0.97– 0.95–0.97 0.01– None For small datasets

BatchNorm + L2 0.98 0.02
(0.001)

Dropout (0.4) + 0.93– 0.92–0.95 <0.01 None Strong

BatchNorm + L2 0.96 regularization, may
(0.01) underfit

How to read the table: The "Gap" column shows the difference between training
and validation R². A large gap (>0.05) indicates overfitting—the model is memoriz-
ing training patterns rather than learning the underlying function. A negative gap
(validation R² > training R²) indicates underfitting—the model doesn't have enough
capacity or is too heavily regularized. The ideal configuration produces a small gap
(<0.02) with high validation R² (>0.95).

85
86
4.7 Training Callbacks and Strategies

4.7.1 Early Stopping

Early Stopping monitors the validation loss and halts training when it ceases to im-
prove, preventing overfitting and saving computation time:

Stop if: Lval (epoch) > Lval (best) for p consecutive epochs
​ ​

where p is the patience parameter. The system uses the Keras EarlyStopping
callback with the following configuration:

— Monitor: val_loss (validation MSE loss)

— Patience: Configurable from 10 to 100 epochs (default: 30)

— Restore best weights: Enabled—after stopping, the model reverts to the
weights from the epoch with the lowest validation loss

The patience mechanism provides a grace period to account for temporary fluctua-
tions in validation loss that may occur during training (e.g., when the optimizer tra-
verses a flat region of the loss landscape). Setting patience too low (e.g., 5–10
epochs) may cause premature termination before the model has fully converged,
especially with noisy loss landscapes. Setting it too high (e.g., 100 epochs) re-
duces the early stopping benefit—the model may waste significant computation
time in regions where no improvement is occurring.

What difference does it make? Early Stopping is the single most impactful call-
back for preventing overfitting:

— Without Early Stopping: A model trained for 300 epochs on the NGL dataset
may achieve a training R² of 0.999+ but a validation R² of only 0.88–0.92. After
approximately epoch 50–100, the validation loss begins to increase while the
training loss continues to decrease—this is the classic overfitting signature.
Early Stopping detects this divergence and halts training at the optimal point.

— With Early Stopping (patience=30): Training automatically stops when the

validation loss has not improved for 30 consecutive epochs. The model weights

87
 are restored to the epoch with the best validation loss, ensuring the saved
model is the best-performing one rather than the last-trained one. For the
Standard architecture on the NGL dataset, this typically results in training
stopping at epoch 70–150, saving 50–80% of the planned computation.
— The restore_best_weights parameter: This is critical. Without it, the model at
the time of stopping would be the model after 30 epochs of no improvement—
which is worse than the model at the epoch with the minimum validation loss.
The restore_best_weights=True setting ensures the final model corresponds
to the best epoch, not the last epoch.

4.7.2 Learning Rate Scheduler (ReduceLROnPlateau)

The ReduceLROnPlateau callback dynamically reduces the learning rate when the
validation loss stops improving, allowing the optimizer to take finer steps in the loss
landscape:

αnew = αcurrent × freduce

​ ​ ​

where freduce = 0.5 (the default factor). The configuration is:

​

— Monitor: val_loss

— Factor: 0.5 (halves the learning rate)

— Patience: max(10, patience // 2) when Early Stopping is enabled, otherwise

15 epochs

— Minimum LR: 10−6

The learning rate schedule typically proceeds as: 0.001 → 0.0005 → 0.00025 → ...
until convergence or until the minimum learning rate is reached. This adaptive
schedule ensures that the optimizer makes large steps during early training (when
the loss landscape is roughly convex) and progressively smaller steps as it ap-
proaches a local minimum.

What difference does it make? The LR scheduler has a subtle but important ef-
fect on training:

— Without LR scheduling: The learning rate remains fixed at 0.001 throughout

training. While this is sufficient for early epochs, the optimizer may oscillate

88
 around the minimum in later epochs without converging—the step size is too
large for the narrow valley at the bottom of the loss landscape. The training
loss may bounce between two values without reaching a stable minimum.

— With LR scheduling: After the validation loss plateaus (patience of 10–15

epochs), the learning rate is halved. This allows the optimizer to take
progressively finer steps, "settling" into narrow minima that a fixed learning rate
would overshoot. The resulting model typically achieves a validation loss that is
5–15% lower than training without scheduling.

— Interaction with Early Stopping: These two callbacks work synergistically.

The LR scheduler reduces the learning rate first (patience 10–15), and only if
the loss still doesn't improve after further training does Early Stopping trigger
(patience 30). This ordering ensures the model gets a chance to benefit from a
lower learning rate before training is terminated.

4.7.3 Model Checkpointing

The ModelCheckpoint callback saves the model weights to disk whenever the vali-
dation loss improves. This ensures that the best model state is preserved even if
training continues and the model subsequently overfits. The checkpointed model is
saved to saved_models/ngl_dnn_model.keras .

4.7.4 Continue Training (Fine-Tuning)

The system supports continuing training on an existing saved model, allowing in-
cremental learning when new data becomes available. The continue_training
function in model/[Link] loads the saved .keras model and resumes the train-
ing loop with the same or modified callbacks. This is useful for:

— Incorporating new simulation data without retraining from scratch

— Fine-tuning the model with a lower learning rate on a targeted subset

— Adapting the model to slightly different operating conditions

4.7.5 Training Progress Visualization

Within the Streamlit dashboard, a custom Keras callback ( StreamlitCB ) provides

real-time progress feedback during training:

89
 class StreamlitCB([Link]):
def on_epoch_end(self, epoch, logs=None):
progress = (epoch + 1) / total_epochs
progress_bar.progress(progress)
status = f"Epoch {epoch+1}/{total} — Loss: {logs['loss']:.6f}"
if 'val_loss' in logs:
status += f" — Val Loss: {logs['val_loss']:.6f}"
status_text.text(status)

After training completes, the system displays the training convergence plot show-
ing both training and validation loss curves on a logarithmic scale, along with true-
vs-predicted scatter plots for both output targets.

4.7.6 Training Configuration Summary

Table 4.7.1: Default Training Configuration

Parameter Default Value Configurable Range

Epochs 300 50–1000

Batch Size 32 Fixed

Optimizer Adam Fixed

Learning Rate 0.001 {0.0001, 0.0005, 0.001, 0.005, 0.01}

Validation Split 0.2 (20%) 10–40%

Dropout Enabled, rate 0.2 0.05–0.50

Batch Normalization Enabled On/Off

L2 Regularization Disabled {0.0001–0.01}

Early Stopping Enabled, patience 30 On/Off, 10–100

LR Scheduler Enabled On/Off

Table 4.7.2: Practical Impact of Training Configuration Choices

90
 Effect on
Setting Effect on
Effect on Training Convergence Recommendation
Changed Validation R²
Speed

Higher More training Negligible if No change Only useful

epochs iterations Early Stopping without Early
(500–1000) enabled Stopping
(model stops
early)

Lower Smaller, more Slightly higher 2–3× slower Use with Early
learning precise updates final R² Stopping if model
rate oscillates
(0.0001)

Higher Larger, noisier Lower R² Faster initial Not

learning updates (overshoots descent recommended,
rate (0.01) minimum) causes instability

Dropout 0.1 Light/Medium/Heavy 0.1: risk Heavier 0.15–0.25 for NGL

vs 0.3 vs regularization overfitting; 0.3: dropout = dataset
0.5 balanced; 0.5: slower
risk
underfitting

BatchNorm Normalized layers vs On: +0.01– On: 30–50% Always enable

on vs off raw activations 0.03 R², faster
smoother
curves

L2 0.001 vs Mild vs strong weight 0.001: minor Negligible Use only if

0.01 penalty improvement; difference overfitting persists
0.01: may
underfit

Early Overtraining vs Off: 0.88–0.92; On: stops Always enable

Stopping off optimal stopping On: 0.96–0.99 early (saves
vs on time)
(patience
30)

LR Fixed LR vs adaptive On: +0.01– No significant Always enable

Scheduler decay 0.03 R², better difference
off vs on final
convergence

91
4.7.7 Empirical Training Results and Data Ranges

The deep neural network model was trained on a dataset generated from targeted
process variable ranges in HYSYS to capture localized thermodynamic behavior.

Table 4.7.3: Model Training Data Ranges

Process Parameter Symbol Unit Training Range

Feed Temperature x1 ​ °C [30.0 – 41.1]

Feed Pressure x2 ​ kPa [4,500.0 – 5,500.0]

Feed Molar Flow x3 ​ kmol/h [8,000.0 – 8,100.0]

Expander Pressure x4 ​ kPa [3,085.0 – 4,510.0]

De-ethanizer Temperature x5 ​ °C [110.0 – 140.0]

Debutanizer Temperature x6 ​ °C [140.0 – 170.0]

The model was compiled with a Standard architecture and Standard scaler, utiliz-
ing the Adam optimizer (LR = 0.001). The complete model configuration, metrics,
and hyperparameter logs are detailed in the Model Training Report (PDF).

92
Model Performance Analysis

The model achieved high accuracy on both output targets, reaching an Overall
Average R2 Score of 0.9842. The individual target metrics are:

— Ethane Percentage (y1 ): R2 = 0.9873

​

— Specific Energy Consumption (y2 ): R2 = 0.9812

​

The training process and generalization behavior are visualized in the plots below:

— Model Convergence (Figure 4.7.3): The loss curves demonstrate smooth,

stable convergence. The validation loss tracks the training loss closely without
divergence, and training was automatically halted by Early Stopping at 125
epochs, restoring the best weights.

— Accuracy Analysis (Figure 4.7.4): The parity plots illustrate the predicted
values versus the actual simulation values. The points cluster tightly along the
45-degree line for both targets, confirming that the surrogate model is highly
accurate and free of systematic bias.

Figure 4.7.3: Model Convergence History (Training vs. Validation Loss)

Figure 4.7.4: Parity Plot (Predicted vs. Actual for Ethane % and SEC)

93
 NGL Recovery ML-ROM Report

NGL Recovery ML-ROM

Training Report
Generated: 2026-06-25 06:45:02

1. Model Configuration

Architecture: Standard
Scaler Type: Standard
Learning Rate: 0.001
Epochs Completed: 125

2. Performance Metrics

Output Target R² Score

Ethane % 0.9873

SEC 0.9812

Overall Average R²: 0.9842

94
Page 1/2
 NGL Recovery ML-ROM Report

3. Visualizations

Training History

Accuracy Analysis

95
Page 2/2
4.8 K-Fold Cross Validation

4.8.1 Motivation

A single train-test split can produce unreliable performance estimates, especially

with small-to-moderate datasets where the particular choice of partition can signifi-
cantly influence the measured metrics. K-Fold Cross Validation addresses this by
partitioning the dataset into K non-overlapping folds and training K separate mod-
els, each using a different fold as the validation set while the remaining K−1 folds
serve as training data. This provides:

1. Reduced variance: Performance metrics are averaged across K folds,

producing a more stable estimate.
2. Full data utilization: Every sample appears in both training and validation sets
exactly K−1 and 1 times, respectively.

3. Robustness: The standard deviation of fold metrics indicates the model's

sensitivity to training data composition.

4.8.2 Algorithm

The K-Fold Cross Validation procedure is:

Algorithm: K-Fold Cross Validation

1. Shuffle the dataset D with random seed 42 for reproducibility.

2. Partition D into K equal-sized folds: D = F1 ∪ F2 ∪ ⋯ ∪ FK ​ ​ ​

3. For k = 1, 2, … , K :
(k)
— Training set: Dtrain = D ∖ Fk
​ ​

(k)
— Validation set: Dval = Fk ​ ​

(k)
— Fit feature scaler on Dtrain and transform both sets
​

(k) (k)
— Train model on Dtrain , validating on Dval
​ ​

2 (k) 2 (k) 2 (k)

— Compute R² scores: Rethane ​
, RSEC ​
, Roverall ​

96
 4. Compute average metrics: ˉ2 =
R 1
∑K 2
σR 2 =
K
​

k=1 R(k)
​ ​ ​

2
1
∑k=1 (R(k) ˉ2)
K 2
K−1
​ −R ​ ​

2
5. Select the fold with the highest Roverall as the best model and save it. ​

Key implementation details:

— Each fold independently fits its own scalers on the training portion, preventing
data leakage from the validation fold.

— The number of folds K is configurable from 3 to 10, with 5 being the default.

— The best-performing fold model (highest overall R²) is persisted as the final
production model.

4.8.3 Output Metrics

For each fold, the system reports:

Table 4.8.1: K-Fold Output Metrics

Metric Description

2 (k)
Rethane ​ R² score for Ethane Percentage prediction on fold k

(k)
2
RSEC ​ R² score for SEC prediction on fold k

2 (k)
Roverall ​
Average R² across both outputs on fold k

Epochs run Number of epochs completed (may be less than max if Early Stopping triggered)

Final loss Training loss at the final epoch

The summary statistics include:

= std ({Rethane }k=1 )

(k) (k) K
ˉ2 1
∑K 2 2
R ethane = ​

K
​

k=1 Rethane
​ ​ , σRethane
2 ​
​ ​ ​

= std ({RSEC }k=1 )

ˉ2 = 1 2 (k) 2 (k) K
R SEC ​

K
​ ∑K
k=1 RSEC
​ ​ , σRSEC
2 ​
​ ​ ​

A low standard deviation (σ < 0.02) indicates that the model's performance is con-
sistent across different training data partitions, suggesting a robust model. High
variability (σ > 0.05) suggests that the model may be sensitive to specific training
examples or that more data is needed.

97
98
4.9 Prediction System

4.9.1 Prediction Pipeline

The prediction system transforms raw input parameters through a well-defined

pipeline to produce predictions in the original (unscaled) domain:

Algorithm: Prediction Pipeline

1. Input assembly: Construct the input vector x = [x1 , x2 , x3 , x4 , x5 , x6 ]T from

​ ​ ​ ​ ​ ​

the six process parameters.

2. Scaling: Transform using the fitted input scaler: x

^ = ScalerX (x) ​

3. Forward pass: Compute the network output: y

^ scaled = fDNN (x
^ ; θ)
​ ​ ​

^ = Scaler−1
4. Inverse scaling: Recover the original domain prediction: y Y (y
^ scaled ) ​ ​ ​ ​

^ [0] (Ethane Percentage), y^2 = y

5. Output extraction: y^1 = y​ ​ ​ ^ [1] (SEC)​ ​ ​

The predict_single function accepts six individual float values, while

predict_batch accepts an N × 6 NumPy array for batch processing. Both return
predictions in original units.

4.9.2 Extrapolation Range Warnings

A critical limitation of any data-driven model is its inability to reliably extrapolate

beyond the training domain. The system stores the minimum and maximum values
of each input feature from the training data and issues warnings when a prediction
request falls outside these bounds:

Warning if: xi < Xmin,i

​ ​
or xi > Xmax,i
​ ​
for any i

For example, if the model was trained on Feed Temperatures from 25°C to 38°C, a
prediction at 50°C would trigger: " ⚠️ Feed Temp (°C) = 50.0 is outside training
range [25.0, 38.0]". This serves as a practical guardrail reminding users that pre-
dictions outside the training domain are unreliable.

99
4.9.3 Batch Prediction with Comparison

The Batch Prediction tab (Tab 2) supports two modes:

1. Prediction-only mode (6-column CSV): The user provides only input values.
The system returns a CSV with two additional columns: Pred_Ethane_Pct and
Pred_SEC .

2. Comparison mode (8-column CSV): The user provides both inputs and
known outputs. The system computes prediction accuracy metrics:

∑N ^i )2
i=1 (yi −y
R2 = 1 −
​ ​ ​ ​

∑i=1 (yi −yˉ)2

N ​

​ ​ ​

1
MSE = N
​ ∑N ^i )2
i=1 (yi − y
​ ​ ​ ​

RMSE = MSE ​

1
MAE = N
​ ∑N ^i ∣
i=1 ∣yi − y ​ ​ ​ ​

These metrics are computed separately for each output (Ethane % and SEC) and
presented in the dashboard along with true-vs-predicted scatter plots and residual
distributions.

100
4.10 Process Optimization

4.10.1 Problem Formulation

The Optimization tab (Tab 4) leverages the trained ML-ROM as a fast surrogate
objective function, enabling rapid identification of optimal operating conditions.
Rather than optimizing a single objective (SEC or Ethane % individually), the sys-
tem employs a multi-objective Best Combination approach that simultaneously
minimizes both the ethane percentage in the LPG product and the specific energy
consumption.

The optimization problem is formulated as:

minx ​ f (x) = w ⋅ yˉ1 (x) + (1 − w) ⋅ yˉ2 (x) subject to

​ ​ ​ ​ xmin ≤ x ≤ xmax
​ ​

where:

— yˉ1 and yˉ2 are the normalized Ethane % and SEC predictions, scaled to [0, 1]
​ ​ ​ ​

using the min–max values estimated from a 500-point random sample within
the feasible bounds

— w = 0.5 assigns equal weight to both objectives

— x = [x1 , x2 , x3 , x4 , x5 , x6 ]T is the vector of six input parameters

​ ​ ​ ​ ​ ​

— xmin and xmax are user-defined bound vectors

​ ​

The bounds are configurable through the dashboard interface using range sliders,
with sensible defaults based on the training data range.

4.10.2 Differential Evolution Algorithm

The system uses SciPy's differential_evolution optimizer, a population-based

global optimization algorithm well-suited for this problem because:

1. Gradient-free: It does not require computing derivatives of the neural network,

avoiding the complexity of backpropagating through the optimization loop.

2. Global: Unlike gradient descent, which may converge to local minima,

differential evolution maintains a population of candidate solutions that explore

101
 the search space broadly.

3. Bound-aware: The algorithm natively supports box constraints, ensuring all

solutions remain within the specified operating envelope.

The algorithm operates as follows:

1. Initialization: Generate a population of NP candidate solutions randomly ​

within the bounds: xi = xmin + ri ⋅ (xmax − xmin ),

​ ​ ​ ​ ​ ri ∼ U (0, 1)6
​

2. Mutation: For each target vector xi , create a mutant vector: vi = xr1 + F ⋅ ​ ​ ​

(xr2 − xr3 ) where r1, r2, r3 are distinct random indices and F ∈ [0.5, 1.0] is
​ ​

the differential weight.

3. Crossover: Create a trial vector by mixing the target and mutant vectors:

ui,j = {
vi,j if rj ≤ CR or j = jrand
where CR is the crossover probability
​ ​ ​

otherwise
​ ​ ​

xi,j ​

and rj ∼ U (0, 1).

​

4. Selection: Replace the target if the trial is better: x(t+1)

i ​
=
(t)
{
ui ​
if f (ui ) ≤ f (xi )
​ ​

x(t) otherwise
​ ​

i ​

5. Termination: The algorithm terminates after maxiter iterations (default: 150)

or when convergence criteria are met.

The default configuration uses popsize=15 and seed=42 for reproducibility.

4.10.3 Optimization Results

The optimization output includes:

— Optimal input parameters: The six input values that jointly minimize the
combined objective.

— Predicted outputs: The resulting Ethane % and SEC at the optimal point.

— Multi-objective scores: Normalized Ethane Score, SEC Score, and weighted

Combined Score (higher = better).

— 3D landscape: An interactive Plotly surface plot colored by the objective

function value.

102
Table 4.10.1: Default Optimization Bounds

Parameter Default Min Default Max

Feed Temperature (°C) 30.0 41.1

Feed Pressure (kPa) 4,500.0 5,500.0

Feed Molar Flow (kmol/h) 8,000.0 8,100.0

Expander Pressure (kPa) 3,085.0 4,510.0

De-ethanizer Temperature (°C) 110.0 140.0

Debutanizer Temperature (°C) 140.0 170.0

Table 4.10.2: Trained-Data Optimization Results (Best Combination)

The following results were obtained by running the Best Combination optimizer on
the trained model with bounds set to the full training data range:

Optimal Operating Conditions Value

Feed Temperature 34.3 °C

Feed Pressure 5,393 kPa

Feed Molar Flow 8,099 kmol/h

Expander Pressure 3,178 kPa

De-ethanizer Temperature 127.9 °C

Debutanizer Temperature 151.1 °C

Predicted Performance Value

Ethane Percentage 2.30984%

Specific Energy Consumption (SEC) 63.26009 kWh/ton

These results represent the best trade-off point where both ethane contamination
in the LPG product and energy consumption are simultaneously minimized under
the equal-weight (w = 0.5) objective.

103
4.11 Web Application and User Interface

4.11.1 Streamlit Framework

The NGL Recovery ML-ROM Dashboard is built using Streamlit, an open-source

Python framework for creating data science web applications with minimal front-
end code. The application is launched via:

python -m streamlit run [Link]

and renders in the browser at [Link] . Streamlit provides reactive

widgets, automatic caching, and seamless integration with Plotly and Pandas,
making it ideal for interactive ML dashboards.

4.11.2 Interface Design

The dashboard employs a Shadcn-inspired light theme with the following design
principles:

— Metric cards: Key performance indicators (R² scores, sample counts) are
displayed in bordered cards with large numeric values and unit labels,
providing instant visual feedback.

— Status badges: The sidebar displays the model training status ("TRAINED" in
green or "NOT TRAINED" in gray) with the last training timestamp and
architecture information.

— Consistent color palette: Primary blue (#0369a1) for training curves and
primary data, secondary green (#10b981) for validation data and positive
indicators, amber (#f59e0b) for secondary outputs, and slate (#64748b) for
reference lines.

— Logarithmic loss charts: Training history plots use logarithmic y-axes to

clearly show convergence behavior across multiple orders of magnitude.

104
4.11.3 Four-Tab Architecture

The application is organized into four tabs, each serving a distinct function:

Tab 1: Train Model — Provides complete control over model training, including ar-
chitecture selection (Simple/Standard/Advanced), scaler choice
(Standard/MinMax/Robust), regularization configuration (Dropout, BatchNorm, L2),
training callbacks (Early Stopping, LR Scheduler), and K-Fold Cross Validation.
Includes real-time progress bars and a standardized data preview panel.

Tab 2: Batch Prediction — Accepts CSV uploads with 6 or 8 columns, runs pre-
dictions on the entire dataset, and presents accuracy metrics (R², MSE, RMSE,
MAE), scatter plots, and residual distributions. Supports downloading predictions
as CSV and generating PDF reports.

Tab 3: Manual Prediction — Provides individual input fields with sensible defaults
for single-point predictions. Includes range warnings when inputs exceed training
bounds.

Tab 4: Optimization — Allows the user to define constraints via range sliders and
runs a multi-objective Best Combination optimization that simultaneously mini-
mizes both Ethane % and SEC. Uses differential evolution to find the optimal
trade-off point and visualizes the response surface in 3D.

4.11.4 PDF Report Generation

The model/[Link] module implements the NGLReport class (extending FPDF)

for generating downloadable PDF reports. Three report types are available:

— Training Report: Includes model configuration (architecture, scaler, learning

rate, regularization), performance metrics (R² for each output and overall), and
embedded training convergence and accuracy plots.

— K-Fold Report: Includes K-Fold configuration, per-fold R² scores in a

formatted table, average metrics with standard deviations, and visualizations of
the best fold's convergence and accuracy.

— Prediction Report: Includes summary statistics (sample count, prediction

ranges), accuracy metrics when ground truth is available, and embedded

105
 prediction analysis plots.

All reports use a consistent visual style with teal (#0D9488) headers, formatted ta-
bles with alternating row colors, and embedded Plotly charts rendered as PNG im-
ages via Kaleido.

4.11.5 Standalone Command-Line Interface

In addition to the interactive web dashboard, the system provides a standalone

command-line training script ( dnn_rom_model.py ) that executes the complete train-
ing pipeline without a graphical interface. This is useful for automated workflows,
scheduled retraining, or environments without a browser:

python dnn_rom_model.py

The script performs the following steps in sequence:

1. Data Loading: Reads ngl_data.csv from the project root directory, validating
the column structure and converting mole fractions to percentages.

2. Preprocessing: Splits the data 80/20 into training and test sets using a fixed
random seed (42) for reproducibility, then applies StandardScaler
normalization.

3. Model Creation: Builds a Standard architecture (32→16→8→2) with the

default configuration.

4. Training: Trains for 300 epochs with a 20% validation split, outputting epoch-
by-epoch progress to the console.

5. Evaluation: Computes and displays R² scores for Ethane Percentage, SEC,

and overall average.

6. Visualization: Generates a true-vs-predicted scatter plot for both outputs and

saves it as prediction_results.png at 300 DPI.

7. Persistence: Saves the trained model, scalers, training history, and metadata
to the saved_models/ directory.

The standalone script uses the same underlying model/ package modules, ensur-
ing consistency between the CLI and web interface workflows.

106
4.12 Conclusion

This chapter presented the complete design, implementation, and evaluation of a

Deep Neural Network Reduced Order Model (DNN-ROM) for predicting NGL re-
covery parameters in turbo-expander processes. The system addresses a funda-
mental challenge in process engineering: the computational cost of rigorous Aspen
HYSYS simulations makes them impractical for real-time monitoring, rapid opti-
mization, and extensive scenario analysis. By training a deep neural network on
HYSYS-generated data, the ML-ROM achieves millisecond-order prediction times
while maintaining R² scores typically exceeding 0.95 for both output targets
(Ethane Percentage and Specific Energy Consumption).

The key technical contributions of this work include:

Architecture Design: The funnel-shaped DNN architectures (Simple: 16→8→2,

Standard: 32→16→8→2, Advanced: 64→32→16→8→2) with ReLU hidden acti-
vations and a linear output layer provide a flexible framework that scales with
dataset complexity. The standard architecture with ~906 trainable parameters bal-
ances expressiveness and generalization for the NGL recovery problem.

Comprehensive Regularization: The integrated use of Dropout, Batch

Normalization, and L2 weight decay—each configurable independently—allows
the practitioner to tune the bias-variance tradeoff for datasets of varying sizes and
noise characteristics. The default configuration (Dropout=0.2, BatchNorm enabled)
provides robust performance across typical NGL datasets.

K-Fold Cross Validation: Rather than relying on a single train-test split, the sys-
tem supports K-Fold CV with independent scaler fitting per fold, producing statisti-
cally reliable performance estimates with mean and standard deviation metrics
across folds.

Interactive Dashboard: The Streamlit-based web application unifies model train-

ing, prediction (single and batch), and process optimization in a single cohesive in-
terface, making the ML-ROM accessible to engineers without programming
expertise.

107
Process Optimization: The multi-objective Best Combination optimizer leverages
the ML-ROM's speed to evaluate thousands of candidate operating conditions per
second, enabling practical real-time optimization of NGL recovery processes by si-
multaneously minimizing both ethane contamination and energy consumption.

Robust Software Engineering: The modular architecture, comprehensive error

handling, model persistence, scaler serialization, extrapolation warnings, and PDF
report generation reflect production-grade design practices that ensure reliability
and maintainability.

The demonstrated acceleration factor of 104 to 105 over full HYSYS simulations
opens new possibilities for integrating process simulation insights into operational
workflows. Engineers can now explore the entire operating envelope in seconds
rather than hours, conduct previously infeasible optimization studies, and embed
surrogate models into control systems requiring sub-second response times.

Future work may extend this framework to Pareto-front multi-objective optimization

with configurable weights, ensemble uncertainty quantification, physics-informed
neural networks with embedded thermodynamic constraints, and direct integration
with Aspen HYSYS via COM automation for continuous model updates as plant
data becomes available.

108
References

1. TensorFlow: An End-to-End Open Source Machine Learning Platform. [Link]

2. Keras: Simple. Flexible. Powerful. [Link]
3. Streamlit Documentation. [Link]
4. Scikit-learn: Machine Learning in Python. [Link]
5. SciPy Optimization Reference. [Link]
6. Kingma, D.P., Ba, J. (2015). "Adam: A Method for Stochastic Optimization." Proceedings of the 3rd
International Conference on Learning Representations (ICLR).
7. Ioffe, S., Szegedy, C. (2015). "Batch Normalization: Accelerating Deep Network Training by Reducing
Internal Covariate Shift." Proceedings of the 32nd International Conference on Machine Learning
(ICML).
8. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R. (2014). "Dropout: A Simple
Way to Prevent Neural Networks from Overfitting." Journal of Machine Learning Research, 15(1),
1929-1958.
9. Storn, R., Price, K. (1997). "Differential Evolution – A Simple and Efficient Heuristic for Global
Optimization over Continuous Spaces." Journal of Global Optimization, 11(4), 341-359.
10. Pedregosa, F., et al. (2011). "Scikit-learn: Machine Learning in Python." Journal of Machine Learning
Research, 12, 2825-2830.
11. He, K., Zhang, X., Ren, S., Sun, J. (2015). "Delving Deep into Rectifiers: Surpassing Human-Level
Performance on ImageNet Classification." Proceedings of the IEEE International Conference on
Computer Vision (ICCV).
12. Goodfellow, I., Bengio, Y., Courville, A. (2016). Deep Learning. MIT Press. Chapters 6-8.
13. Aspen Technology. (2022). Aspen HYSYS: Process Simulation Software.
[Link]
14. Hestness, J., et al. (2017). "Deep Learning Scaling is Predictable, Empirically." arXiv preprint
arXiv:1712.00409.
15. Kohavi, R. (1995). "A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model
Selection." Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI).

109