Optimised IES-VE model calibration methodology integrating IoT,

smart meter and BMS data

Adalberto Guerra Cabrera1 , Ruth Kerrigan2

1
Integrated Environmental Solutions Limited, Glasgow, United Kingdom
2
IES R&D Ltd, Dublin, Ireland

Abstract
In this paper, we present a model-agnostic calibra-
tion methodology derived from IES’ best practices
and calibration guidelines. The calibration method-
ology relies on a 3-stage process that consist of (1)
checking input priority matrix and SA results, (2)
creating data-driven profiles for high priority inputs,
and (3) determining and deriving high-priority pa-
rameters. The process uses data analysis techniques, Figure 1: Example of three types of energy use met-
Sensitivity Analisys (SA) and optimisation tools to rics: baseline, current and target. Baseline and tar-
maximise model accuracy and minimise calibration get metrics (marked with *) are calculated using cal-
efforts. ibrated models. Source: Adapted from ISO (2016).
IES headquarters, an office building in the UK, is tionally used during the design stage to inform design
presented as case study. A model of this building decisions (Manz et al., 2018), ensure occupants’ com-
used for ongoing commissioning has been calibrated fort (ASHRAE Standard 55-2013, 2013), ensure com-
at hourly level. Internal gains (i.e. lighting, equip- pliance with energy efficiency codes (Pacific North-
ment and occupancy) are derived from IoT sensors west National Laboratory, 2016), gain credits in
and included in the simulation as Free-From-Profiles building rating systems, such as LEED and BREAM
(FFPs). Room heating setpoints are included in the (Kestner et al., 2010), and determine building perfor-
simulation as Parametric Profiles (PPs). Sensitivity mance related to a regulation or standard, e.g. En-
analysis and Optimisation-based parameter search is ergy Performance Certificates (EPCs).
done by grouping spaces with similar-end use to min-
imise the number of parameters. Electricity and air During the operational stage, a calibrated model can
temperature calibrated to match utility data achiev- be used for establishing a baseline and target en-
ing a NMBE, CVRMSE and RMSE within recom- ergy use (see Figure 1), energy savings estimation
mended thresholds. Prediction error for air tempera- (M&V) (EVO, 2002), fault detection and diagnostics
ture outputs are minimised simultaneously to ensure (FDD) (Zimmermann et al., 2011), enhance building
that the model represents the building at space level. intelligence through model predictive control (MPC)
We show that use of metered data and automated (Aftab et al., 2017), enabling model-free control tech-
tools can improve the quality of the model outputs at niques such as Reinforcement Learning (RL) (Brandi
energy and space level with lower consultancy efforts. et al., 2020), evaluation of demand-response (DR)
technologies (Seitenfus et al., 2019) and prediction
Introduction of future savings as a result of retro-commissioning
activities or large renovations (Bande et al., 2019).
Buildings account for 40% of the global CO2 emis-
sions (Ahmad et al., 2016). In the UK, buildings In Measurement and Verification (M&V), which can
account for 37% of the total annual greenhouse gas be defined as the process of determining the amount
emissions (Committee on Climate Change, 2013). For of savings obtained through an Energy Conservation
the case of office buildings, it is expected that 20% of Measures (ECMs), calibrated simulation models are
savings could be achieved by implementing minor en- used to (1) establish a baseline energy consumption in
ergy conservation measures (IPCC, 2014). Additional order to compare to energy consumption before and
major cost-effective actions could reduce energy de- after implementing an ECM and eventually estimate
mand up to 40% (Lewry, 2016). savings, (2) adjust the baseline energy consumption
in case a change occurs, such as occupancy hours or
Building Energy Simulations (BES) have been tradi-
addition of new equipment, (3) assess the impact of
a single ECM when multiple ones have been imple- iterative model evaluation. The guideline also recom-
mented at the same time and no individual measure- mends cross-checking space level simulation outputs
ment is in place and (4) reduce monitoring waiting such as air temperature and CO2 concentration to
time after ECM for early savings estimation. ensure that there is a match between the measured
The use of BES during the operational stage is sub- and simulated data at space level.
ject to a successful model calibration effort which can Additionally, the recent cost reduction in sensors
be defined as “the process of reducing the uncertainty from buildings coming from Smart Meters (AMR),
of a model by comparing the predicted output of the Building Management Systems (BMS) and Internet
model under a specific set of conditions to the ac- of Things (IoT) can improve the quality of the cal-
tual measured data for the same set of conditions”, ibrated model. Also, richer input data can reduce
(ASHRAE, 2002). Without calibration, is not pos- the effort required during the calibration process by
sible to determine the degree of uncertainty of the reducing the number of assumptions made to the
model with respect to the studied building. model, e.g. actual start/stop times of heating/cooling
However, model calibration is widely recognised as a plants, temperature setpoints and occupancy profiles.
time consuming activity, exclusive for scientific re- In this work, we present an iterative calibration
search and it often requires experienced engineers methodology and tools that not only supports but
with knowledge about the building and its charac- also complements the CIBSE TM63 guideline. This
teristics, The most common challenges are (Fabrizio methodology is aimed for buildings with available
and Monetti, 2015): sensor data consisting of three high-level stages: (1)
check input priority matrix and SA results, (2) cre-
• Lack of standardisation: The workflow carried
ate data-driven profiles for high priority inputs, and
out is reliant on the users’ own knowledge and
(3) determine and fine-tune high-priority parameters.
experience, which can differ vastly between con-
The process is repeated until the calibration metrics
sultants. Additionally, there is a general consen-
are achieved, i.e. uncertainty of the building is min-
sus about the lack of defined methodology that
imised to acceptable levels. An office building located
ensures completeness, consistency and trans-
in Scotland is used as case study and calibration re-
parency (Raftery et al., 2011)
sults are presented according to recommended guide-
• Costs: Due to the lack of standardisation and
lines.
the complexity involved in creating a calibrated
model, the modelling process is significantly Methodology
more time intensive than standard compliance
models. The calibration methodology presented in this work
exploits the relevant available data in the studied
• Model input data: Calibrated models require
building, machine learning regression models, SA and
large amounts of input data to complete the pro-
optimisation tools with the objective of reducing cal-
cess, where models of greater complexity require
ibration efforts and increasing model accuracy both
greater amounts of data. As the process is user
at room level (e.g. air temperature) and energy level
determined, it can be difficult to identify the
(e.g. electricity and cooling/heating demand).
most important datasets to focus on to achieve
The first sub-section introduces the input priority ma-
the calibration metrics at least effort. This fur-
trix concept, which is intended to work as a look up
ther increases the time and cost requirements of
table indicating the data required and its relative im-
the calibration process (Wang et al., 2019).
portance for the model, based on its end-use. The
• Uncertainty: During manual calibration, a de-
process can be complemented with a SA outputs anal-
terministic approach is typically adopted. How-
ysis if a model is available at this stage. In step 2,
ever, not all data will impact the results of the
we present two methods for creating data-driven in-
model to the same degree. For this reason, it
put parameters, the Free-Form-Profiles (FFPs) and
is important to identify, throughout a screening
the Parametric Profiles (PPs) which are used as in-
analysis, the parameters that influence the most
puts for the model. Step 3 consists of determining the
the building model, such as occupancy, weather,
high priority parameters by doing a Sensitivity Anal-
sensors’ accuracy an others; and define their level
ysis (SA) followed by a robust optimisation-based
of uncertainty.
tuning approach that minimises calibration metrics
• Lack of automation: The calibration process is
to recommended values in existing guidelines. These
a manual one which relies on the experience and
steps are model-agnostic but depend on the existence
skills of the modeller.
of measured data from the building especially from
Recently, the guideline CIBSE TM63 (CIBSE, 2020) IoT sensors, BMS and AMR. Figure 2 shows a dia-
introduces a whole building calibration and has been gram with the presented methodology. Notice that
widely tested and proven for building energy model this work describes the steps within the calibration
calibration. The process relies on Sensitivity Analysis methodology box.
(SA), Optimisation-based parameter derivation and
Figure 2: Model calibration workflow introduced in
this work.
Step 1: Check input priority matrix and SA
results
One of the most important steps when calibrating a
model is to have a clear idea of its end use. Know-
ing the purpose of the calibrated model will allow the
modeller to have a clear idea of the most critical pa-
rameters in a model, which potentially has thousands
of them. Once the end-use of the model has been
agreed, the use of input priority matrix provides the
relative importance of model parameters and mea-
Figure 3: Example of relative importance of parame-
sured data. This matrix has been created based on
ters and measured energy data for calibrating a model
a collection of IES’s best practices for model calibra-
intended for retro-commissioning.
tion. The full explanation of the matrix is out of the
scope of this paper, but the model applications con-
sidered in this matrix are:
• Long-term assessment
• Urban energy simulations
• Model-based commissioning
• Pre-occupancy commissioning
• Post-construction commissioning
• Retro-commissioning
• Measurement and verification
Figure 4: Example of SA results for an early-stage
Parameters are divided into static (e.g. U-Values)
warehouse model for Electricity and Gas consump-
and dynamic (e.g. setpoints, internal gains, weather
tion. Parameters with higher relative impact in the
data), as well as required measured data for calibra-
selected model output are sorted in descendent order.
tion (e.g. monthly, hourly, sub-hourly). An example
of the relative importance of parameters and energy been successfully used for model calibration and is
data for a model that is intended to be used for retro- preferred due to its relative low-computational costs
commissioning is presented in Figure 3. and results explainability (Kristensen and Petersen,
A follow-up work will describe in more detail each of 2016). An example of a SA study of an early stage
the model applications, including the justification of model of a warehouse is presented in Figure 4. Ac-
the importance assessment. However, the intent is cording to these results, equipment gains in the sales
that a model should be as reliable and accurate as space is the parameter that impact the most the to-
needed, not as possible, while satisfying the require- tal electricity demand and the heating setpoint on the
ments of calibration standards. same space has the largest impact on gas demand.
The CIBSE TM63 guideline suggests the use of a SA However, at this stage we consider the that SA results
study to determine the relevant inputs. This step is have complementary role to the matrix. While a SA
also considered in the present methodology and pro- study is a more objective method to determine pa-
posed the use of a SA solution based on the Morris rameter importance compared to best-practice crite-
method (Campolongo et al., 2007). This method cal- ria, it is difficult to carry out a study to determine rel-
culates the elementary effects (EE) to identity im- ative importance between dynamic (time-series) and
portant inputs in large models. This method has static inputs. Also, a SA study done without any en-
 Step 3: Determine and fine-tune high-priority
parameters
A BES model has thousands of inputs and it is not
possible to determine all of them by direct measure-
ments due to resource constraints, hence most of the
time is imperative to derive inputs with high impor-
tance for the simulation in lieu of direct measure-
ments. This step is subdivided in (1) determine the
impact due to the uncertainty of high priority param-
eters by doing a SA study and (2) fine-tune shortlisted
Figure 5: Example of observed trends from motion parameters. Calibration metrics are the main driver
sensors in a building. to determine if the uncertainty of an input is relevant
and whether the parameter fine-tuning is the optimal.
gineering criteria will likely have an excessive number
of parameters, specially when time-series inputs are Calibration metrics for energy outputs are the Nor-
considered. Hence, the list of inputs is the result of malised Mean Bias Error (NMBE), defined in Equa-
both the priority matrix and SA results. tion 1, and the Coefficient of Variation of the Root
Mean Squared Error (CVRMSE), defined in Equa-
Step 2: Data-driven time-series input data
tion 2.
In this step, the aim is to take advantage of existing PN
time-series data and incorporate it into the model in (yt − ŷt )
N M BE(%) = t=1 × 100% (1)
the form of Free-From-Profiles (FFPs) or Parametric (N − P ) × µ
Profiles (PPs). In this sub section, we describe them
in detail and provide example of the kind of situation rP
N
(yt −ŷt )2
where each alternative is more adequate. t=1
N −P −1
FFPs are used to import data coming directly from CV RM SE(%) = × 100% (2)
µ
a third-party source, e.g. a machine learning (ML)
model. An application of an ML model in the con- Following the CIBSE TM63 guideline, the recom-
text of model calibration is the Trended daily profiles, mended calibration metrics for non-energy measure-
a data driven adaptation of a technique defined as ments, such as air temperature and CO2 concentra-
“day-typing” by Reddy (2006). ML regression models tion include the Root Mean Squared Error (RMSE),
are used to generate a typical day for main occupied defined in Equation 3, and the Mean Absolute Error
spaces for each day of the week. Hour, Day of the (MAE), defined in Equation 4.
week, holidays and daylight saving time behaviours s
can be captured in the model, making it possible to PN
− ŷt )2
t=1 (yt
account for typical occupancy behaviours not only RM SE(unit) = (3)
during the calibration stage but also during model N −1
deployment. An example of observed motion trends PN
is presented in Figure 5. Trended profiles can then be t=1 |yt− ŷt |
M AE(unit) = (4)
extrapolated for the rest of the year and also to fill in N −1
gaps in the datasets. These profiles are then imported These metrics have thresholds depending on whether
to the model created in IES-VE as FFPs. Additional the simulation is calibrated at monthly or hourly in-
trended inputs may include lighting usage and small tervals, as presented in Table 1. However, for time-
equipment usage patterns that can be obtained from series sensor data the monthly aggregation does not
short-term measurements and extrapolated over the apply.
calibration period.
PPs are useful to characterise dynamic inputs that Table 1: NMBE and CVRMSE calibration thresh-
can be largely explained by a set operation rules. PPs olds for monthly and hourly intervals according to
can be generated using syntax expressions based on ASHRAE (2002) and CIBSE (2020).
current hour of the day, outdoor temperatures, occu- Monthly Hourly
pancy levels and can be assigned to specific days. E.g. NMBE 5% 10%
Weekends and weekday setpoint PPs may be enough CVRMSE 15% 30%
in some cases to describe the whole-year heating pro- RMSEtemp N/A 1.5 ◦ C
files. Additional inputs that can be described with MAEtemp N/A 1.5 ◦ C
PPs may include window-usage trends, HVAC on/off
schedules, or ventilation rates. It is possible also to SA study for calibration metrics
combine Trended profiles with parametric profiles, to
describe a building with or without COVID-19 social To determine the impact that missing or uncertain
distancing measures. parameters have on the calibration metrics, a SA
 data, as defined in Equation 5. RNRMSE can provide
a more meaningful representation of how the model
fits all the measured data regardless its scale, which
translates in a more robust optimisation algorithm.

rP
N
(yt −ŷt )2
t=1
N −1
RN RM SE(%) = × 100% (5)
range(y)
The optimisation problem is then defined as:
Figure 6: SA for calibration metrics outputs for elec-
tricity use. min f (x)
Px
study based on the Morris method similar to the pre- where x = RN RM SEoutput
vious step is required. The difference in this step
with respect to the previous SA, is that the elemen- Case study: the Helix building
tary effects are not in terms of model outputs but in
The helix building is an office type building with a
calibration metrics. The outputs of the SA are use-
floor area of 2,900 m2 and it was constructed post
ful to determine which variables can be shortlisted
2000s. The building has natural ventilation and heat-
in the next sub step. An example of the SA out-
ing is provided by a biomass (main) and natural gas
puts for calibration metrics in a simple Warehouse
(backup) boilers. The building accommodates around
model are presented in Figure 6. According to these
180 people. The building is controlled by two ther-
results, equipment gains in the sales space is the pa-
mostats. The typical temperature setpoint is 23 ◦ C,
rameter that impact the most the calibration metrics
with a night setback of 15 ◦ C for both of them. The
(i.e. CVRMSE and NMBE) for electricity use.
thermostat has two typical days: working and week-
end day. Additionally, a server room is equipped with
Fine-tune shortlisted parameters a cooling unit (mini-split) that keeps the equipment
at the correct operational temperature. Currently,
Shortlisted parameters can determined by using a hi- the building has 14 indoor environmental sensors at
erarchy of information sources and by optimisation- desk level, plus 5 roof-level additional sensors. These
based parameter search. Raftery et al. (2011) sug- sensors communicate to a gateway every 10 minutes
gests a hierarchy of information sources. In order of with a 5-minute data sample interval. Additional
priority, preferred sources are: measurements include relative humidity, CO2 , mo-
1. Data logged measurements tion and lighting levels (lux).
2. Short-term measurements An existing IES-VE model was updated with the cur-
3. Observation from site survey rent building layout, constructions, orientation and
4. Interview to operator external shading elements. A 3D view of the model is
presented in Figure 7. The heating system has been
5. Operation manuals
modelled using Apache HVAC, an IES-VE module
6. Commissioning documents that allows detailed dynamic modeling of systems,
7. Benchmarks equipment, and controls (IES-VE, 2012). Using this
8. Standards, specifications and guidelines functionality, steel horizontal radiators with a 20 ◦ C
9. Design stage information reference temperature difference have been included
in all required rooms. Heat output is 1 kW per unit,
In order to fine-tune the final parameter values, an
similar to the manufacture’s design value. Weight of
optimisation-based method using the Ant Colony Op-
the radiator was estimated to be 40 kg with a capac-
timisation (ACO) algorithm is used (Dorigo et al.,
ity of 9.9 liters, i.e. the simulation takes into account
2006). The cost function is designed to minimise all
the mass of water that needs to be heated within the
the calibration metrics errors simultaneously. The
radiator before the room starts heating up and the
goal of the optimisation function consists of minimis-
heating inertia after the system switches off. Addi-
ing the average of the Range-Normalised Root Mean
tionally, radiators have a local proportional controller
Squared Error (RNRMSE) for all outputs that have
to simulate the installed thermostatic radiator valves
metered data simultaneously. The intention of this
(TRV). The server room is conditioned with a simpli-
approach, is to ensure that space level outputs are
fied mini-split HVAC system.
represented during the parameter search so that the
algorithm receives a penalty if the parameters devi- Case study: Input priority matrix for ongoing
ate from space level measurements. Chakraborty and commissioning
Elzarka (2018) proposed RNRMSE as an alternative The calibration period is from the 01-October-2019 to
metric that normalises the RMSE by the range of the 01-November-2019. The model will be calibrated for
 Figure 9: Setpoint estimation for weekdays. Red re-
Figure 7: IES-VE model of the Helix building. gion shows an overlay view of air temperatures across
weekdays, green line represents the setpoint which was
exported as a PP to the IES-VE model.

Figure 10: Setpoint estimation for weekends. Red re-

Figure 8: Motion trends for three meeting rooms in gion shows an overlay view of air temperatures across
the Helix building, where blue means no motion and weekends, green line represents the setpoint which was
red means high motion periods. The motion is used exported as a PP to the IES-VE model.
as a predictor of occupancy and equipment gains mod-
(Figure 10). Using overlay plots, it is possible to visu-
ulation.
ally determine if a temperature setpoint has occurred
electricity use and space air temperatures at hourly during a given month or week. A PP of the setpoint
intervals with a 2-minute simulation timestep. The for each relevant room of the building was created
application of the model is ongoing commissioning. and assigned to the model.
This is a typical application of a calibrated model
Case study: Determine and fine-tune high-
that can be used to constantly interrogate measured
priority parameters
variables in a building and spot inefficient operations
as they occur. According to the input priority ma- For this model, weather data is obtained from a
trix, fixed parameters with high importance include weather station located in the rooftop complemented
HVAC layout and schematics and HVAC equipment with METeorological Aerodrome Reports (METAR)
and specifications. Relevant dynamic parameters are: data from a station located at Paisley Airport in Glas-
room setpoints, site weather data and schedules (e.g. gow. Fixed parameters, such as U-values and HVAC
lighting, equipment, system and occupancy). specifications, are calculated using as-built drawings
and energy audit information. A summary of the ini-
Case study: Data-driven profiles for main oc-
tial high-priority inputs are explained in the following
cupied rooms
list. Notice that uncertain parameters are highlighted
Motion detection in various areas of the buildings are in bold.
used as a proxy value to determine occupancy,lighting
and equipment profiles, see Figure 8. These trends • Trees added as local shading
are generated using a machine learning regression • Ground temperature based on OAT 30-day mov-
model for the occupied areas under the assumption ing average
that motion is a significant predictor of internal gains, • U-Values for Roof 0.155 W/m2 K Walls 0.26
and that it can be correlated with independent vari- W/m2 K
ables such as day-of-week, time-of-day, holiday calen- • Initial estimated infiltration rate of 0.25
dar and weather variables. The occupancy trends are ACH
exported to the models via FFPs. • Initial estimated lighting density 8 W/m2
Heating setpoints are derived from air temperature • Initial estimated equipment 12 W/m2
readings for both weekday (Figure 9) and weekends • Initial server room cooling sp 23 ◦ C
Figure 11: EE for electricity in terms of CVRMSE Figure 12: SA parameters for air temperature in an
and NMBE. meeting room in terms of RMSE and MAE.

Once FFPs for internal gains, PPs for setpoints, static Table 3: Calibration metrics for hourly electricity use.
inputs and weather data have been assigned to the CVRMSE before CVRMSE after
model, the uncertain parameters are ranked in terms Electricity 70.35% ◦ C 15.89%
of relative impact to the calibration metrics. To NMBE before NMBE after
avoid an excessive number of parameters, spaces were Electricity -40.48% ◦ C -7.09%
grouped by activity: Meeting rooms, open plan office, Results are not as straightforward in this case. The
and server room. The SA algorithm requires a range main parameter that drives the RMSE for air tem-
of values for each of the input parameters. Using peratures are the infiltration rates from the open ar-
the SA IES tool, it is possible to change the value of [Link] the infiltration rate of the meeting rooms
a space or a group of spaces via a scaling factor or is the main parameter that drives changes in MAE.
absolute values. In this case, all parameters that rep- On the other hand, there is an agreement that pa-
resent internal gains have a range based on a scaling rameters related to the server room are not relevant
factor between 0.7 to 1.3, which represents the range for this meeting room.
of the uncertainty. The cooling setpoint temperature Based on the SA results and the number of model
of the server room, on the other hand, is represented outputs that can be used to calibrating the model
as a range of absolute values. All the parameters and (eleven), it was decided to leave the eight parameters
ranges are presented in Table 2. for the actual calibration process. The total number
of simulation was 500, this number is product of 50
Table 2: Full list of parameters that may need to be iterations and a population number of 10. The pro-
fine-tuned. Shortlisted parameters for optimisation cess was calculated on an 16-core 32 GB RAM vir-
are marked with a star character *. We selected all tual machine and the total calculation time was 4394
parameters due to the large number of measurements [Link] 3 shows the calibration metrics before
available for calibration. and after for electricity demand. Notice that the new
Parameter Abs/Scale Range metrics are below the calibration thresholds.
Open plan offices - Equip* Scale 0.7-1.3 Table 4 shows the calibration metrics before and after
Open plan offices - Lighting* Scale 0.7-1.3
for air temperature in selected spaces. Even though
Open plan offices - Inf* Scale 0.7-1.3
the calibration metrics for air temperature were al-
Meeting rooms - Equip* Scale 0.7-1.3
ready below the recommended threshold, most of
Meeting rooms - Lighting* Scale 0.7-1.3
them showed an additional improvement. These re-
Meeting rooms - Inf* Scale 0.7-1.3
sults suggest that no compromise was required to
Server room - cooling SP* Abs 21-24
meet the calibration metrics for electricity.
Server room - Equip* Scale 0.5-2
Case study: Results comparison
The SA for calibration metrics both for electricity,
in terms of CVRMSE and NMBE, and air tempera- After optimisation is completed, it is possible to re-
tures, in terms of RMSE and MAE, are calculated a cover the model parameters with their original units.
number of levels 4 and 30 samples, which translates Parameters before and after optimisation are pre-
into 350 total simulations. The process was calcu-
lated using an 16-core and 32 GB RAM virtual ma-
Table 4: Calibration metrics for air temperatures in
chine. The total calculation time was 3352 seconds.
selected rooms.
Figure 11 shows the EE for electricity demand during Space (Group) RMSE before RMSE after
the calibration period. Equipment gains in the server Nevis (Meeting room) 1.42 ◦ C 1.24 ◦ C
room are the main single relevant parameter that im- Cuillings (Meeting Room) 0.98 ◦ C 0.79 ◦ C
pacts the calibration metrics for electricity. On the Grampian (Meeting Room) 1.22 ◦ C 1.36 ◦ C
other hand, infiltration values have negligible impact Consultancy (Open area) 0.97 ◦ C 0.90 ◦ C
Support (Open Area) 1.07 ◦ C 1.06 ◦ C
on electricity, since heating is provided by a biomass
Server room 0.72 ◦ C 0.47 ◦ C
boiler, however, the value is not zero probably due to Torridon (Meeting Room) 1.14 ◦ C 0.91 ◦ C
the impact on cooling loads. Training (Meeting Room) 1.32 ◦ C 1.29 ◦ C
Figure 12 shows the EE for air temperature in a se- Knoydart (Meeting Room) 1.26 ◦ C 1.24 ◦ C
Developers (Open Area) 1.22 ◦ C 1.08 ◦ C
lected meeting room during the calibration period.
Average 1.13 ◦ C 1.06 ◦ C
Table 5: Parameters before and after calibration. Av-
erage value of the spaces on each group.
Parameter Updated
Open plan offices - Lighting 6.81 W/m2
Open plan offices - Equip 9.56 W/m2
Open plan offices - Inf 0.24 ACH
Meeting rooms - Equip 9.18 W/m2
Meeting rooms - Lighting 10.01 W/m2
Meeting rooms - Inf 0.55 ACH Figure 14: Measured (solid line) vs simulated (dashed
Server room - Equip 39.78 W/m2 line) electricity data for the ground used for electricity
Server room - cooling SP 23.63 ◦ C calibration of the model. Notice that periods when EV
cars are being charged are not being simulated by the
model.

and (3) determine and fine-tune high-priority param-

eters. The process relies on the use of specialised tools
for Sensitivity Analysis, Optimisation-based parame-
ter derivation and calibration metrics computation.
Using an office building in Scotland as a case study,
Figure 13: Simulated vs measured air temperature re-
we did an hourly level calibration of a model intended
sults during a week.
for ongoing commissioning. Internal gains modula-
sented in Table 5. tion (i.e. lighting, equipment and occupancy) are in-
cluded in the model as FFPs derived from time-series
The average RMSE has improved from 1.13 ◦ C to data from sensors using a machine learning regression
1.06 ◦ C, both are well below the values recommended model. Lighting and equipment intensity, infiltration
in CIBSE TM63. Air temperature results during a and setpoints were determined for spaces grouped by
week within the calibration period are presented in activity, e.g. open office areas and meeting rooms.
Figure 13. In this plot, it can be noticed that there A SA determined the most relevant variables for the
seems to be a close match between the simulated and optimisation-based parameter search, and an ACO
metered temperatures, especially during the warm up algorithm was used to fine-tune the final gains, infil-
and cool down times and the weekend. tration rates and the server room setpoint.
On the energy side, electricity values are calculated Electricity calibration achieves a NMBE -7.09% and
when compared to sub-metered electricity readings CVRMSE of 15.89%. Most of the calibration metrics
covering the ground floor. These have been adjusted for air temperatures were improved in average 6%
to remove the effect of a two-car electric vehicle (EV) and there were kept within the 1.5 ◦ C recommended
charging unit connected to the building, due to the threshold. Based on these findings, we provide evi-
reason that EV charging station is outside the scope dence that metered data can benefit the model pre-
of the model, Figure 14. diction outputs both at the energy and space level
It is worth noticing that the time taken to calibrate and that SA and optimisation tools can speed up the
the model is now significantly reduced as a result calibration process.
of the automated SA and Optimisation. In total,
around 860 simulations were required to achieve these Acknowledgments
metrics from which 850 were part of an automated This paper has received funding from the European
process. The process was completed in less than 7 Union’s Horizon 2020 research and innovation pro-
hours. Previous attempts to manually fine-tune un- gramme under grant agreement No 847053. Shane
certain parameters resulted in higher calibration met- Campbell, Giulia Barbano and Ricardo Filho con-
rics, e.g. CVRMSE > 25%; substantially higher con- tributed to this paper as technical reviewers. Moham-
sultancy time, e.g. three working days; with only mad Eskandari, Giovanni Tardioli and Dereje Workie
visual estimations of the uncertainty at space level. worked on the development of the SA and optimisa-
Conclusions tion tools.

In this paper, we present a calibration methodol- References

ogy derived from IES’ best practices and calibration Aftab, M., C. Chen, C. K. Chau, and T. Rah-
guidelines. The calibration methodology is model- wan (2017). Automatic HVAC control with real-
agnostic and relies on a 3-step process that consist time occupancy recognition and simulation-guided
of (1) checking input priority matrix and SA results, model predictive control in low-cost embedded sys-
(2) create data-driven profiles for high priority inputs, tem. Energy and Buildings.
Ahmad, M. W., M. Mourshed, D. Mundow, IPCC (2014). Climate Change 2014: Synthesis Re-
M. Sisinni, and Y. Rezgui (2016). Building energy port. Contribution of Working Groups I, II and III
metering and environmental monitoring - A state- to the Fifth Assessment Report of the Intergovern-
of-the-art review and directions for future research. mental Panel on Climate Change.

ASHRAE (2002). ASHRAE Guideline 14 -2002: Mea- ISO (2016). ISO 17741 - General technical rules for
surement of Energy and Demand Savings. Ameri- measurement, calculation and verification of energy
can Society of Heating, Ventilating, and Air Con- savings of projects. 0 (0).
ditioning.
Kestner, D. M., J. Goupil, E. Lorenz, and M. D. Web-
ASHRAE Standard 55-2013 (2013). ANSI/ASHRAE ster (2010). LEED Rating System. In Sustainability
Standard 55-2013: Thermal Environmental Condi- Guidelines for the Structural Engineer.
tions for Human Occupancy. ASHRAE Standard .
Kristensen, M. H. and S. Petersen (2016, Octo-
Bande, L., A. G. Cabrera, Y. K. Kim, A. Afshari, ber). Choosing the appropriate sensitivity analysis
M. F. Ragusini, and M. G. Cooke (2019, dec). A method for building energy model-based investiga-
Building Retrofit and Sensitivity Analysis in an tions. Energy and Buildings 130, 166–176.
Automatically Calibrated Model Considering the
Lewry, A. (2016). Energy management and building
Urban Heat Island Effect in Abu Dhabi, UAE. Sus-
controls. pp. 8.
tainability 11 (24), 6905.
Manz, H., D. Micallef, S. P. Borg, and V. Buhagiar
Brandi, S., M. S. Piscitelli, M. Martellacci, and
(2018). A parametric building energy simulation
A. Capozzoli (2020, October). Deep reinforcement
case study on the potential and limitations of pas-
learning to optimise indoor temperature control
sive design in the Mediterranean climate of Malta.
and heating energy consumption in buildings. En-
Sustainable Buildings.
ergy and Buildings 224, 110225.
Pacific Northwest National Laboratory (2016). AN-
Campolongo, F., J. Cariboni, and A. Saltelli (2007, SI/ASHRAE/IES Standard 90.1-2010 Performance
October). An effective screening design for sensitiv- Rating Method Reference Manual. Pnnl-255130 .
ity analysis of large models. Environmental Mod-
elling & Software 22 (10), 1509–1518. Raftery, P., M. Keane, and J. O’Donnell (2011, sep).
Calibrating whole building energy models: An
Chakraborty, D. and H. Elzarka (2018, July). Per- evidence-based methodology. Energy and Build-
formance testing of energy models: are we using ings 43 (9), 2356–2364.
the right statistical metrics? Journal of Building
Performance Simulation 11 (4), 433–448. Reddy, T. A. (2006). Literature review on cali-
bration of building energy simulation programs:
CIBSE (2020, August). TM63: Operational perfor- Uses, problems, procedure, uncertainty, and tools.
mance: Building performance modelling (2020). ASHRAE Transactions 112 PART 1, 226–240.
Committee on Climate Change (2013). Progress re- Seitenfus, F., C. H. Barriquello, L. Neves Canha,
ducing emissions from buildings. In Meeting Car- A. Pedretti, T. A. Silva Santana, and Z. Nadal
bon Budgets – 2014 Progress Report to Parliament. (2019, September). Simulation and analysis of Ope-
nADR agents using VOLTTRON platform. In 2019
Dorigo, M., M. Birattari, and T. Stutzle (2006,
IEEE PES Innovative Smart Grid Technologies
November). Ant colony optimization. IEEE Com-
Conference - Latin America (ISGT Latin Amer-
putational Intelligence Magazine 1 (4), 28–39. Con-
ica), pp. 1–6. ISSN: 2643-8798.
ference Name: IEEE Computational Intelligence
Magazine. Wang, L., G. Mcmorrow, X. Zhou, and Z. D. O’Neill
(2019). Assessing the validity, reliability, and prac-
EVO (2002). International Performance Measure- ticality of ashrae’s performance measurement pro-
ment & Verification Protocol: Concepts and Op- tocols (ashrae research project 1702). Science and
tions for Determining energy and Water Savings, Technology for the Built Environment 25 (4), 464–
Volume 1. 487.
Fabrizio, E. and V. Monetti (2015, April). Method- Zimmermann, G., Y. Lu, and G. Lo (2011). A simula-
ologies and Advancements in the Calibration of tion based fault diagnosis strategy using extended
Building Energy Models. Energies 8 (4), 2548– heat flow models (HFM). In Proceedings of Build-
2574. Number: 4 Publisher: Multidisciplinary Dig- ing Simulation 2011: 12th Conference of Interna-
ital Publishing Institute. tional Building Performance Simulation Associa-
IES-VE (2012). ApacheHVAC User Guide. tion, pp. 405–412.