J. Dairy Sci.

102:7948–7960
[Link]
© American Dairy Science Association®, 2019.

Updating predictions of dry matter intake of lactating dairy cows

R. A. de Souza, R. J. Tempelman, M. S. Allen, and M. J. VandeHaar*
Department of Animal Science, Michigan State University, East Lansing 48824

ABSTRACT evaluation were the mean bias, slope bias, concordance

correlation coefficient (CCC), and root mean squared
Our objective was to model dry matter intake (DMI) error of prediction (RMSEP). The proposed prediction
by Holstein dairy cows based on milk energy (MilkE), equation was DMI (kg/d) = [(3.7 + parity × 5.7) +
body weight (BW), change in BW (ΔBW), body con- 0.305 × MilkE (Mcal/d) + 0.022 × BW (kg) + (−0.689
dition score (BCS), height, days in milk (DIM), and + parity × −1.87) × BCS] × [1 – (0.212 + parity ×
parity (primiparous and multiparous). Our database 0.136) × exp(−0.053 × DIM)] (mean bias = 0.021 kg, slope
included 31,631 weekly observations on 2,791 cows en- bias = 0.059, CCC = 0.72, and RMSEP = 2.89 kg),
rolled in 52 studies from 8 states of the United States, where parity is equal to 1 if the animal is multipa-
mostly in the Upper Midwest. The means ± standard rous and 0 otherwise. Finally, the proposed model was
deviations of these variables were 24 ± 5 kg of DMI, compared against the Nutrient Requirements of Dairy
30 ± 6 Mcal of MilkE/d, 624 ± 83 kg of BW, 0.24 ± Cattle (2001) prediction equation for DMI using an
1.50 kg of ΔBW/d, 3.0 ± 0.5 BCS, 149 ± 6 cm height, independent data set of 9,050 weekly observations on
and 102 ± 45 DIM. Data analysis was performed us- 1,804 Holstein cows. The proposed model had smaller
ing a mixed-effects model containing location, study mean bias and RMSEP and higher CCC than the Nu-
within location, diet within study, and location and trient Requirements of Dairy Cattle equation to predict
cow within study as random effects, whereas the fixed DMI and has potential to improve diet formulation for
effects included the linear effects of the covariates de- lactating dairy cows.
scribed previously and all possible 2-way interactions Key words: feed intake, model, lactation
between parity and the other covariates. A nonlinear
(NLIN) mixed model analysis was developed using a
INTRODUCTION
2-step approach for computational tractability. In the
first step, we used a linear (LIN) model component Actual or accurately estimated DMI is essential for
of the NLIN model to predict DMI using only data the formulation of diets to prevent underfeeding or
from mid-lactation dairy cows (76–175 DIM) without overfeeding of nutrients and to promote efficient nutri-
including information on DIM. In the second step, a ent use (NRC, 2001). Current equations used to pre-
nonlinear adjustment for DIM using all data from 0 to dict DMI of lactating dairy cows are based on existing
368 DIM was estimated. Additionally, this NLIN model databases that contain observed intake and predictor
was compared with an LIN model containing a fourth- variables such as milk energy (MilkE), BW, and DIM.
order polynomial for DIM using data throughout the Consequently, the accuracy and precision of these pre-
entire lactation (0–368 DIM) to assess the utility of dictions are dependent on the quality of the database
an NLIN model for the prediction of DMI. In sum- and the use of appropriate statistical analysis to derive
mary, a total of 8 candidate models were evaluated as the prediction equations.
follows: 4 ways to express energy required for main- Traditionally, the prediction of DMI in lactating
tenance (BW, BW0.75, BW adjusted for a BCS of 3, North American dairy cows is based on milk produc-
and BW0.75 adjusted for a BCS of 3) × 2 modeling tion and the energy required for maintenance, which
strategies (LIN vs. NLIN). The candidate models were together represent the major energy expenditures of the
compared using a 5-fold across-studies cross-validation lactating dairy cow (Tempelman et al., 2015; Vande-
approach repeated 20 times with the best-fitting model Haar et al., 2016). Furthermore, due to the peculiarities
chosen as the proposed model. The metrics used for of each stage of lactation (i.e., transition and early-
lactation dairy cows; Allen et al., 2009), DIM must also
be considered.
Received December 18, 2018.
Accepted April 25, 2019. The most recent edition of Nutrient Requirements of
*Corresponding author: mikevh@​msu​.edu Dairy Cattle (NRC, 2001) includes an empirical equa-

7948
 PREDICTIONS OF DRY MATTER INTAKE OF LACTATING DAIRY COWS 7949

tion to estimate DMI of lactating Holstein cows based especially important in formulating diets. Our overall
on animal factors that could be easily measured or idea is that the starting prediction for DMI when for-
known, as follows: mulating diets should first be a prediction based on
animal factors or on measured values, and then feed
DMI (kg/d) = (0.372 × FCM + 0.0968 × BW0.75) factors should be used to assess the effect of ration
changes on DMI during the ration formulation process.
× {1 − e[−0.192 × (WOL + 3.67)]}, A new equation to predict changes in DMI based on
the filling effects of rations is described in a companion
where FCM = 4% FCM (kg/d), BW is given in kilo- paper (Allen et al., 2019).
grams, and WOL = week of lactation (NRC, 2001). The objective of this study was to derive a prediction
This prediction equation was based on data collected equation for DMI based on animal factors that include
on 1,284 Holstein cows using the equation proposed not only BW, MY, milk composition, DIM, and parity
by Rayburn and Fox (1993) and further modified by as in NRC (2001) but also BCS and change in BW,
Fox et al. (1999). Also, the nonlinear effect of week of which were not available for NRC (2001). Our data
lactation was based on Roseler et al. (1997). comprised an updated version of the database used
A more recent and much larger database of weekly in Tempelman et al. (2015) that included 2,791 cows
DMI data from many research stations was used by between 1 and 368 DIM. We expect that this data-
Lu et al. (2017) to estimate the genetic and nonge- base would better represent the current genetics and
netic components of regression of DMI on MilkE and management practices of the North American dairy
maintenance as a function of various management and industry, although we recognize that it contains mostly
environmental factors. Similarly, Tempelman et al. cows from the Upper Midwest. We hypothesized that
(2015) developed station-specific prediction equations an equation derived from this larger and newer data-
for DMI but without distinguishing genetic from non- base would increase the accuracy and precision of DMI
genetic components as in Lu et al. (2017). Nevertheless, predictions based on animal factors compared with the
both studies used only data between 50 and 200 DIM equation provided in NRC (2001) when tested in an
(Tempelman et al., 2015; Lu et al., 2017) and therefore independent data set.
might not be valid for predicting DMI outside that
range of DIM. Additional data were available in those
MATERIALS AND METHODS
studies that could be used to better develop intake
predictions across the lactation. In addition, many of Data
the cows in these data sets also had measures of BCS.
Because body fatness alters the concentration of circu- The database consisted of individual weekly values
lating leptin, which in turn alters appetite, the inclu- of DMI, MilkE, BW, metabolic BW (BW0.75), BCS,
sion of BCS should increase the accuracy of predicting change in BW (ΔBW), height, DIM, and parity (pri-
voluntary DMI (Block et al., 2003; Allen, 2014). miparous and multiparous) from 10 research stations
In addition to models based only on animal character- across the eastern half of the United States. We calcu-
istics, several models (Arnedal, 2005; Huhtanen et al., lated BW0.75 as BW to the power of 0.75 and ΔBW as the
2011; Zom et al., 2012; Krizsan et al., 2014) have been final BW minus the initial BW through the experimetal
proposed to predict DMI based on both animal and diet period divided by number of days. Additionally, we cal-
characteristics. Jensen et al. (2015) evaluated 5 DMI culated BW adjusted to a BCS of 3 (BWBCS3; kg)
prediction models and showed that models containing and BW0.75 adjusted to a BCS of 3 (BW0.75BCS3; kg)
both animal and nutritional effects would yield better as a function of BW and BCS: BWBCS3 = BW + BW
predictions for DMI than models based only on animal × 0.084 × (3 − BCS), and BW0.75BCS3 = BWBCS30.75.
effect. Although DMI prediction may be improved by To calculate the BWBCS3, it was assumed that a
using both animal and dietary factors as covariates, 1-unit shift in BCS causes a 8.4% change in BW, as
in this paper we focused only on animal characteris- determined by de Souza and Vandehaar (2018) using
tics. One reason to use only animal characteristics is a database comprising 2,181 Holstein cows. The MilkE
that predictions of intake are sometimes desired when was calculated according to NRC (2001).
diet composition is unknown, and the effects of feed Recording frequencies of the variables included in the
factors are largely accounted for when milk yield and database varied from daily, thrice weekly, twice weekly,
BW change are used in empirical predictions of intake. weekly, and biweekly depending on the variable and
In addition, effects of feed factors differ depending on the research station. To standardize variables as weekly
stage of lactation (Allen, 2014). Predictions of DMI are means across all studies, each variable (i.e., MilkE, BW,

Journal of Dairy Science Vol. 102 No. 9, 2019

7950 DE SOUZA ET AL.

BCS, ΔBW, and height) on each animal on the same observations from 3,143 lactations (1,462 primiparous
treatment for each study was analyzed using a fifth- and 1,681 multiparous) on 2,791 cows, was used in the
order polynomial linear regression model on days since model development process (referred to as the develop-
the start of that particular treatment. Subsequently, ment dataset), whereas the second data set, consisting
the predicted values from these animal-specific analyses of 9,050 weekly observations from 1,968 lactations (908
were used to calculate weekly means for each animal on primiparous and 1,060 multiparous) on 1,804 cows, was
a particular treatment. used in the independent evaluation process. The distri-
The final database comprised 40,681 weekly observa- bution of the number of observations in each data set
tions on 4,785 Holstein dairy cows from 105 studies throughout the lactation is presented in Figure 1, and
(Table 1). About 54% (22,045 weekly observations) of their respective means, medians, quartiles, maximums,
these data was previously described by Tempelman et and minimums are presented in Figure 2. The criteria
al. (2015), but we used only their data from US research used to determine the separation of the database into 2
stations (University of Florida, Gainesville; Iowa State data sets (model development and independent evalu-
University, Ames; Michigan State University, East Lan- ation data sets) were based on the number of weekly
sing; University of Wisconsin, Madison; United States observations per cow within a study. Studies that
Dairy Forage Research Center, Madison, WI; and provided 7 or more weekly observations per cow con-
USDA Animal Genomics and Improvement Laboratory, stituted the model development data set, whereas the
Beltsville, MD) to be consistent with other data added remaining smaller studies constituted the independent
to this database. The remaining 46% (18,636 weekly evaluation data set. Basic summary statistics of the
observations) of the data came either from locations raw weekly data for each of the 2 data sets are provided
already used in Tempelman et al. (2015) but collected in Table 2 and shown in Figure 2.
in more recent studies or collected at DIM less than 50
or greater than 200 (50% of all data) at these same sta- Statistical Analyses
tions or from new locations (14% of the data) includ-
ing the Purina Animal Nutrition Center (Gray Sum- All data analysis was performed using SAS version
mit, MO), Cargill Research and Development Center 9.4 (SAS Institute Inc., Cary, NC). For numerical sta-
(Minneapolis, MN), Miner Institute (Chazy, NY), and bility, all covariates were standardized to have a mean
Virginia Tech University (Blacksburg, VA). Our final of zero and a standard deviation of 1 using PROC
database thereby comprised records from cows between STANDARD and the functions MEAN = 0 and STD =
1 and 368 DIM and collected in research stations across 1; nevertheless, all reported prediction equations in this
the United States (Florida, Iowa, Maryland, Michigan, paper are provided on the actual observed or measured
Missouri, New York, Virginia, and Wisconsin) from scale. All covariates were jointly checked with PROC
2007 to 2016. All the data were collected from lactating REG for multicollinearity using variance inflation fac-
Holstein dairy cows offered a TMR once per day and tors (multicollinearity analysis).
milked 2 or 3 times per day. In this paper, we refer to the models that initiated the
The final database was subdivided into 2 independent modeling processes as the starting models, the models
data sets. The first data set, consisting of 31,631 weekly that were selected by the modeling processes and were

Table 1. Frequency of number of studies, diets, cows, and lactations by state

No. of No. of No. of No. of No. of weekly Average no. of weekly

Station1 studies diets cows lactations records records per lactation
AGIL 2 3 419 586 5,431 9.3
UF 11 36 366 420 3,326 7.9
ISU 3 4 952 1,008 10,335 10.3
MSU 21 80 339 540 5,689 10.5
CRDC 1 22 237 361 1,331 3.68
PANC 1 3 67 67 961 14.3
Miner 1 4 58 58 340 5.9
VT 6 22 110 111 842 7.6
UW 33 95 1,088 1,183 12,106 10.5
Total 79 269 3,636 4,334 40,681 9.4
1
AGIL = USDA Animal Genomics Improvement Laboratory, Beltsville, Maryland; UF = University of Florida,
Gainesville; ISU = Iowa State University, Ames; MSU = Michigan State University, East Lansing; CRDC =
Cargill Research and Development Center, Minneapolis, Minnesota; PANC = Purina Animal Nutrition Center,
Gray Summit, Missouri; Miner = Miner Institute, Chazy, New York; VT = Virginia Tech, Blacksburg; UW =
University of Wisconsin–Madison.

Journal of Dairy Science Vol. 102 No. 9, 2019

 PREDICTIONS OF DRY MATTER INTAKE OF LACTATING DAIRY COWS 7951

used in cross-validation as the candidate models, and DMIpcdsl = β0 + u0,d + (β1 + u1,d) × MilkE
the model selected for best fit based on cross-validation
+ (β2 + u2,d) × body weight + β3 × ΔBW + β4
and further evaluated against the NRC (2001) DMI
model as the proposed model. These models are more × BCS + β5 × Parityp + β6 × MilkE × Parityp + β7
carefully explained in the subsequent sections.
× body weight × Parityp + β8 × ΔBW × Parityp
Modeling Process + β9 × BCS × Parityp + Cowc(Studys × Locationl)
The nonlinear (NLIN) modeling process was ac- + Studys(Locationl) + εpcdsl, [1]
complished in 2 major phases. The first phase was the
development of a linear (LIN) model to predict DMI in where DMIpcdsl (p = parity, c = cow, d = diet, s = study,
mid-lactation cows (76–175 DIM) without the inclusion l = location) is the observed DMI; β0 + u0,d is the inter-
of DIM as a covariate. The second phase was the inclu- cept specific to diet d (d = 1–153); β1 + u1,d is the
sion of the nonlinear effect of DIM on DMI across the partial regression coefficient of DMI on MilkE specific
entire modeling data set, including early (1–75 DIM), to diet d; β2 + u2,d is the partial regression coefficient of
mid (76–175 DIM), and late (176–368 DIM) lactation. DMI on body weight (BW, BW0.75, BWBCS3, or
For the first phase of the modeling process, we speci- BW0.75BCS3) specific to diet d; β3 is the partial regres-
fied a starting model that contained the fixed effects of sion coefficient of DMI on ΔBW; β4 is the partial re-
partial linear regressions on MilkE, the corresponding gression coefficient of DMI on BCS; β5 is the partial
BW measure (BW, BW0.75, BWBCS3, or BW0.75BCS3), regression coefficient of DMI on parity (p = primipa-
ΔBW, BCS, parity (primiparous and multiparous), and rous or multiparous); and β6, β7, β8, and β9 are the ef-
all possible 2-way interactions between parity and the fects for the 2-way interactions between the correspond-
other covariates. Location, study within location, diet ing covariates and parity. All other terms were random
within study and location, and cow within study and effects, including Cowc ~ NIID (0, σcow
2
) for cows c = 1 to
location were treated as random effects. Furthermore, 2,805 and Studys ~ NIID (0, σstudy
2
) for studies s = 1 to
the model also included diet-specific partial regressions
of DMI on MilkE and on BW, recognizing that the 57 across the various locations, with ~ NIID (0, σcow
2
) be-
partial regressions of DMI on MilkE and on BW could ing the error term assuming independent and identi-
depend on diet compositions. The generic starting cally distributed random variables with a mean of 0,
model is presented in Equation 1: and variance σ 2. We assumed a multivariate normal

Figure 1. Distribution of the number of observations by DIM in the development (Dev) and evaluation (Eval) data sets.

Journal of Dairy Science Vol. 102 No. 9, 2019

7952 DE SOUZA ET AL.

Figure 2. Means (×), medians (solid lines), quartiles (middle 2 quartiles in shaded boxes), and accepted ranges (whiskers) for DMI, milk
energy (MilkE) output, BW, metabolic BW (MBW), BW change, BCS, height, and DIM for model development (Dev; n = 31,631) and evalu-
ation (Eval; n = 9,050). The development data set is in blue, and the evaluation data set is in red. Whiskers extend up from the top of the box
to the largest data element that is less than 1.5 times the interquartile range (IQR) and down from the bottom of the box to the smallest data
element that is larger than 1.5 times the IQR. Anything outside of this range is considered to be an outlier (blue dots).

distribution for diet effects and diet-specific partial re- in 2 phases. During the first phase, each of the full
gressions on MilkE and BW as follows: models was subjected to a backward model selection
method using PROC GLMSELECT in SAS with the
u    σ2 σu0 ,u1 σu0 ,u2   option HIERARCHY = SINGLE, such that the model
 0  u0
 0,d  
u  ~ N    , G =    was forced to include the corresponding main effects
 1,d   0 σu ,u σu21 σu1 ,u2   . [2] if the interaction between any 2 effects was deemed to
      0 1  
 2  be significant. Because PROC GLMSELECT does not
u2,d 
   0 σu ,u σu1 ,u2 σu2  
  1 2  allow for the specification of random effects, the model
was limited to the specification of fixed effects only in
Note that the specification in Equation 2 also allowed this first stage of model selection. During the second
for nonzero covariances, σu0 ,u1 , σu0 ,u2 , and σu1,u2 (i.e., off- phase of model selection, the model chosen by PROC
diagonal values of the variance-covariance matrix G) GLMSELECT was analyzed using PROC HPMIXED,
between diet-specific intercepts and slopes for MilkE, in which the above-mentioned random effects were in-
between diet-specific intercepts and slopes for BW, and cluded. The fixed effects with the highest P-values were
between diet-specific slopes for MilkE and BW, respec- successively removed until only significant fixed effects
tively. (P < 0.05) and all specified random effects remained
The starting models described above Equation 1 were in the model, yielding a linear mixed model as a can-
subjected to model selection, which was accomplished didate model to predict DMI for mid-lactation dairy

Journal of Dairy Science Vol. 102 No. 9, 2019

 PREDICTIONS OF DRY MATTER INTAKE OF LACTATING DAIRY COWS 7953
Table 2. Number of observations, mean, and standard deviation of the covariates used to develop (development
data set) and evaluate (evaluation data set) the proposed equations for DMI

Development data set Evaluation data set

No. of No. of
Variable1 observations Mean SD observations Mean SD
DMI, kg/d 31,631 24.4 4.56 9,050 24.0 4.58
MilkE, Mcal/d 31,631 29.8 6.28 9,050 28.9 6.72
BW, kg 31,631 624 79.4 9,050 639 88.2
BW0.75, kg0.75 31,631 125 11.9 9,050 127 13.1
ΔBW, kg/d 28,729 0.237 1.24 7,309 −0.003 2.08
BCS2 31,631 3.04 0.457 9,050 2.99 0.429
Height, cm 7,691 149 5.28 4,129 150 4.44
DIM 31,631 108 46.7 9,050 114 65.6
1
MilkE = milk energy; BW0.75 = metabolic BW; ΔBW = change in BW.
2
Scale of 1 to 5.

cows. With this model selection process, we created 4 To verify the importance of using a nonlinear adjust-
candidate models (i.e., 4 different ways to express the ment for DIM in the prediction of DMI, we refitted
covariate BW; namely, BW, BW0.75, BWBCS3, and the 4 linear candidate models (based on model selec-
BW0.75BCS3). tion) to the entire development data set by adding a
The NLIN model involved the estimation of the non- fourth-order polynomial for DIM and all possible 2-way
linear effect of DIM on DMI using PROC NLMIXED in interactions with parity as fixed effects as part of an
SAS. The nonlinear adjustment was based on the ad- LIN modeling. Hence, our model comparison involved
justment factor proposed by Roseler et al. (1997); that a total of 8 candidate models [4 ways to express BW ×
−(β −β × PMM)× WOL +β3  model (NLIN vs. LIN)].
is, 1 − exp  1 2 , where PMM is peak
milk month and WOL is week of lactation, with further
modifications as follows: (1) effect of parity was in- Cross-Validation
cluded to allow for parity specific effects, (2) PMM was
removed because information was not available in our The 8 candidate models were formally compared using
data set, and (3) a term was included that determines a 5-fold across-study cross-validation repeated 20 times
the maximum possible discount (β1 in Equation 3). The for a total of 100-fold replicates using the development
generic model used during the second phase is pre- data set of 31,631 weekly observations (development
sented in Equation 3: data set described in the “Data” section). That is, for
each of the 20 cross-validation replicates, the data were
partitioned across rather than within studies into 5
DMI =
almost-equal-sized subsets, each involving all data from
DMI { 1 2 }
 c × 1 − [ β + β × Parity ]× exp (β3 + β4×Parity)×DIM + ε ,
c
20% of the studies, such that for 1 fold, 4 of these
subsets were used for development and the remaining
[3] subset was used for evaluation. Hence, across 5 folds,
each subset took a turn being the evaluation data set.
where DMI is the observed DMI, DMI  c is the predicted
The candidate models were compared for fit criteria
DMI from candidate linear submodel c (c = 1–4) based according to Tedeschi (2006), including mean bias,
on the 4 different ways to express the covariate BW, β1 slope bias, concordance correlation coefficient (CCC),
is the coefficient that determines the maximum possible mean squared error of prediction (MSEP), root MSEP
discount for DIM for primiparous cows, β2 specifies the (RMSEP), and the decomposition of MSEP in mean
difference for this discount between multiparous and bias, slope bias, and random error (as percentage of
primiparous cows, β3 is the nonlinear regression coeffi- MSEP). To determine our proposed model, the fit cri-
cient on DIM for primiparous cows, and β4 specifies the teria were analyzed using PROC GLIMMIX of SAS
difference for this coefficient between multiparous and version 9.4 according to the following model (Equation
primiparous cows. Furthermore, parity is the dummy 4):
variable (0 for primiparous, 1 for multiparous) for par-
ity in Equation 3, whereas ~ NIID (0, σe2 ) is the error Yijl = µ + ri + BWj + Modelm + BW
term. × Modeljm + eijl, [4]

Journal of Dairy Science Vol. 102 No. 9, 2019

7954 DE SOUZA ET AL.

where Yijl is one of the fit statistics of interest; µ is the Stagek is the fixed effect of stage of lactation (k = early,
overall mean; ri is the random effect of fold (i = 1–100); mid, or late), Model × Stagejk is the fixed effect of the
BWj is the fixed effect of corresponding BW measure (j interaction between model and stage, and eijk is the
= BW, BW0.75, BWBCS3, or BW0.75BCS3), Modelm is residual error.
the fixed effect of model (m = LIN and NLIN), BW ×
Modeljm is the interaction between corresponding BW RESULTS
and model, and eijl is the residual error. The candidate
model that consistently had the best goodness of fit was Regarding the 4 different measures of expressing
considered the best-fitting model and, therefore, our BW (BW, BW0.75, BWBCS3, or BW0.75BCS3), we did
proposed model. A generic SAS code for model evalua- not observe any difference in their ability to predict
tion (using only linear models) and a short explanation DMI based on the cross-validation (P > 0.9; results
are available in Supplemental Material S1 (https:​/​/​doi​ not shown). We did not observe differences between the
.org/​10​.3168/​jds​.2018​-16176). 4 ways to express BW, possibly because all measures
were highly correlated with each other (r = 0.96–0.99)
Independent Evaluation Process and all cows in our data set were Holsteins. Thus, for
the remainder of this paper, we show and discuss only
As an independent evaluation process beyond the data for the models developed using BW due to its
cross-validation strategy described previously, the pro- simplicity and applicability.
posed model was compared with the DMI prediction During the model development process, the main ef-
equation proposed by NRC (2001) using the indepen- fect of ΔBW and the interactions of parity with MilkE,
dent evaluation data set of 9,050 weekly observations BW, and ΔBW were removed from the starting model
involving smaller studies with 6 or fewer weekly ob- because they were not significant (P > 0.05). With
servations per cow. That is, for each of 20 evaluation respect to the nonlinear effect of DIM in the NLIN
replicates, the data were partitioned across studies model, parity was an important (P < 0.01) source of
into 5 almost-equal-sized subsets, each subset involv- variation for the maximum discount on DMI due to
ing all data from 20% of the studies. For both models DIM (i.e., β2 in Equation 3), but the effect of parity
(proposed and NRC), the predictions of DMI were on the estimated nonlinear coefficient for DIM (i.e.,
computed using the same random effects model. For β4 in Equation 3) was not significant (P = 0.68) and
each fold, the fit statistics (CCC, MSEP, RMSEP, and therefore was removed.
decomposition of MSEP) were computed as described The NLIN model had better goodness-of-fit proper-
previously. The major difference between the previously ties than the LIN model for all fit statistics, and there-
described cross-validation study and this independent fore we selected a nonlinear relationship for the effect
evaluation study was that the same entire development of DIM on DMI (Table 3). Our proposed NLIN model
data set of 31,631 weekly observations involving all of is shown in Equation 6:
the larger studies (≥7 records per cow) was used for
each evaluation subset such that predictions for each DMI (kg/d) = [(3.7 + Parity × 5.7) + 0.305
evaluation subset were based on the same fixed effects × MilkE (Mcal/d) + 0.022 × BW (kg) + (−0.689
and variance components as based on the proposed
model. Furthermore, to facilitate stage-specific compar- + Parity × −1.87) × BCS] × [1 − (0.212 + Parity
isons, the evaluation data set was partitioned further × 0.136) × e(−0.053 × DIM)], [6]
according to the stage of lactation (early = 1–75 DIM;
mid = 76–175 DIM; late = 176–368 DIM). The various where Parity is equal to 1 if the animal is multiparous
fit statistics were compared using PROC GLIMMIX and 0 if primiparous.
of SAS version 9.4 according to the following model The diagonal values of the estimated G matrix,
(Equation 5): which represent the estimated variance components for
the diet-specific intercept, MilkE, and BW were σ̂u20 =
Yijk = µ + ri + Modelj + Stagek + Model 0.56, σ̂u21 = 0.056, and σ̂u22 = 0.000247, respectively. The
× Stagejk + eijk, [5] estimated DMI for multiparous animals adjusted to the
mean values of MilkE, BW, BCS, and DIM as reported
where Yijk is the fit statistic of interest; µ is the overall in Table 2 and using Equation 6 was 24.5 kg. Based on
mean; ri is the random effect of fold (i = 1–100); Modelj the multivariate normality assumption as invoked in
is the fixed effect of model (j = proposed or NRC), Equation 3, this implies that roughly 95% of the diets

Journal of Dairy Science Vol. 102 No. 9, 2019

 PREDICTIONS OF DRY MATTER INTAKE OF LACTATING DAIRY COWS 7955
Table 3. Fit statistics for the cross-validation across studies of the candidate models (linear and nonlinear)
using the modeling data set

Candidate model2

Fit statistic1 LIN NLIN SEM3 P-value4

CCC 0.751 0.800 0.006 <0.01
Mean bias −0.53 0.008NS 0.059 <0.01
Slope bias −0.050NS −0.033NS 0.009 0.03
MSEP 10.9 6.83 0.143 <0.01
RMSEP 3.20 2.61 0.073 <0.01
Decomposition of MSEP, %
Mean bias 13.6 6.90 0.616 <0.01
Slope bias 1.96 2.28 0.184 0.09
Random error 84.4 90.8 0.647 <0.01
1
CCC = concordance correlation coefficient; MSEP = mean squared error of prediction; RMSEP = root MSEP.
2
The 2 modeling strategies, where the LIN model is a linear model including a fourth-order polynomial of DIM
and the NLIN model contains the nonlinear effect of DIM. NS = not significantly different from zero.
3
SEM common to both LIN and NLIN.
4
For the mean comparison of LIN and NLIN.

between model and stage of lactation was significant,

lead to a mean DMI within 24.5 ± 2 × σˆu20 (or 23.0–26.0
where the goodness-of-fit properties were slightly better
kg) for multiparous animals adjusted to mean values of for our proposed model than the NRC (2001) model
MilkE, BW, BCS, and DIM. Similarly, 95% of the diets during early lactation but much better during mid and
would have a partial regression of DMI on MilkE of late lactation (Table 4). The predicted versus observed
0.305 ± 2 × σˆu21 (or −0.17 to 0.78 kg/Mcal; value of plot for each stage of lactation is presented in Figures
0.305 from Equation 6), whereas 95% of the diets would 3, 4, and 5, with data points for weekly DMI shown as
have a partial regression of DMI on BW of measured and after adjustment for all random effects.
The proposed model had slightly less mean slope bias
0.022 ± 2 × σˆu22 (or −0.01 to 0.053 kg/kg; value of 0.022
than did the NRC model, but both models tended to
from Equation 6). This variability across diets natu-
overpredict intakes for cows at high intakes.
rally reflects, for example, differences in the filling ef-
On dairy farms, diets are fed to groups of animals that
fects and energy density across different diets whereas
contain a mix of primiparous and multiparous cows. In
negative lower bounds likely reflect that these diet ef-
this case, Equation 6 can be modified to represent the
fects are either not precisely normally distributed or
proportion between primiparous and multiparous in the
based on relatively small numbers of observations. The
group. For example, in our development data set the
remaining variance component estimates were 3.39 for
2 2 ratio between primiparous and multiparous was 44:66;
σcow , 4.41 for σstudy ,and 3.24 for σe2, implying that these therefore, to use the equation for the entire develop-
were even larger sources of variability relative to diets ment data set, the estimated coefficient for parity will
within studies. Thus, as a proportion of the total vari- be multiplied by 0.66 (proportion of multiparous cows
ance, cow, diet within study, study, and residual vari- in the data set). As a result, the prediction equation for
ance accounted for 29, 5, 38, and 28% of the total DMI for the average cow in the development data set
variation, respectively, relative to cows of average MilkE is presented in Equation 7. Equation 7 was named the
and BW. simplified proposed model:
For the comparison of the proposed model (Equation
6) and the NRC (2001) DMI prediction equation, the
 = (6.89 + 0.305 × MilkE + 0.022 × BW − 1.74 × BCS)
DMI
effects of model, stage, and interaction between model
and stage were highly significant (P < 0.01; Table 4)
× 1 − 0.288 ×e(
−0.053× DIM) 
for all fit statistics. First, related to the effect of model,   .
the proposed NLIN model had better goodness of fit [7]
than the NRC (2001) model for all fit statistics (Table
4). Regarding the effect of stage, the prediction equa- The difference in the prediction of DMI using the pro-
tions performed better for cows during mid lactation posed model (Equation 6) and the simplified proposed
(CCC, slope bias, MSEP, RMSEP), followed by early model (Equation 7) is presented in Table 5. We recom-
lactation and last by late lactation. The interaction mend using Equation 6 rather than Equation 7. How-

Journal of Dairy Science Vol. 102 No. 9, 2019

7956 DE SOUZA ET AL.

Table 4. Fit statistics for the across-study stage of lactation1 (early, mid, and late)-specific cross-validation performance of the proposed and
NRC (2001) models using the independent evaluation data set

Proposed model3 NRC (2001) model P-value

Model
Fit statistic2 Early Mid Late Early Mid Late SEM4 Model Stage × stage
CCC 0.717A,a 0.735A,a 0.688A,b 0.702B,a 0.681B,a 0.645B,b 0.010 <0.01 <0.01 0.01
Mean bias −0.281A,a 0.058A,b,NS 0.293A,a −0.801B,a 1.54B,b 1.45B,a 0.012 <0.01 <0.01 <0.01
Slope bias5 −0.131A,a −0.028A,b,NS −0.068A,c −0.137B,a −0.117B,b −0.116B,c 0.014 <0.01 <0.01 <0.01
MSEP 10.2A,c 7.13A,a 8.42A,b 11.8B,c 10.5B,a 11.3B,b 0.339 <0.01 <0.01 <0.01
RMSEP 3.19A,c 2.67A,a 2.90A,b 3.42B,c 3.23B,a 3.32B,b 0.048 <0.01 <0.01 <0.01
Decomposition of MSEP, %
Mean bias 7.15A,a 3.45A,b 8.02A,b 12.0B,a 25.9B,b 24.2B,b 1.57 <0.01 <0.01 <0.01
Slope bias 2.73c 0.74a 3.91b 5.24c 2.28a 1.32b 0.564 0.13 <0.01 <0.01
Random error 90.2A,a 95.8A,b 88.1A,c 82.7B,a 71.8B,b 74.5B,c 1.60 <0.01 <0.01 <0.01
A,B
Different uppercase superscripts within fit statistic represent statistical differences for model. P < 0.05.
a–c
Different lowercase superscripts within fit statistic represent statistical differences for stage. P < 0.05.
1
Early = 1–75 DIM; mid = 76–175 DIM; late = 176–368 DIM.
2
CCC = concordance correlation coefficient; MSEP = mean squared error of prediction; RMSEP = root MSEP.
3
NS = not significantly different from zero.
4
SEM common to all models.
5
Slope of the residuals for observed versus predicted intake.

ever, when parity information is not known, Equation 7 analysis, especially during mid and late lactation. We
could be used as an alternative. suspect that this is because we had data from more
cows and included parity and BCS in our prediction.
DISCUSSION As can be seen in Table 4 and in Figures 3, 4, and 5,
both the NRC and proposed models show some slight
This study shows that the NRC (2001) model does systematic error with negative slope bias. Negative
a reasonably good job of predicting intake in Holstein slope bias was worst in early lactation and indicates
cows, but the proposed model had better goodness of that the model overpredicts intake for cows with the
fit than the NRC model in the independent evaluation greatest intakes, possibly because the filling effects of

Figure 3. Plots of the observed weekly DMI of cows in early lactation (1–75 DIM) versus that predicted by NRC (2001) (blue dots, dotted
line) and the proposed model (red triangles, dashed line) with individual weekly DMI points as measured (a) and after adjustment for random
effects (b). Systematic prediction error can be observed by comparing models with the line of equivalence (solid line).

Journal of Dairy Science Vol. 102 No. 9, 2019

 PREDICTIONS OF DRY MATTER INTAKE OF LACTATING DAIRY COWS 7957

Figure 4. Plots of the observed weekly DMI of cows in mid lactation (76–175 DIM) versus that predicted by NRC (2001) (blue dots, dotted
line) and the proposed model (red triangles, dashed line) with individual weekly DMI points as measured (a) and after adjustment for random
effects (b). Systematic prediction error can be observed by comparing models with the line of equivalence (solid line).

diets limit intake most in those cows with the greatest important after 70 DIM (Table 6). Multiparous animals
requirements (Jensen et al., 2015; Allen et al., 2019). had proportionately greater deviance in the effect of
The DIM adjustment for the prediction of DMI is DIM on DMI than primiparous cows in early lactation
necessary due to the peculiarities of different postpar- (Table 6). As an example, at the onset of lactation
tum stages over and beyond the effects of MilkE or (1 DIM), multiparous cows had a discount of 21% on
BW (Dann et al., 1999; Drackley, 1999; Chan et al., the predicted DMI based on MilkE, BW, and BCS,
2006; Janovick et al., 2011; Allen, 2014). For the pro- whereas primiparous cows had a discount of 13%.
posed model, as in Roseler et al. (1997), the greatest The NRC (2001) applies a more aggressive discount
discounts in DMI are early in lactation and are not for DIM on the DMI than the proposed model as shown

Figure 5. Plots of the observed weekly DMI of cows in late lactation (>175 DIM) versus that predicted by NRC (2001) (blue dots, dotted
line) and the proposed model (red triangles, dashed line) with individual weekly DMI points as measured (a) and after adjustment for random
effects (b). Systematic prediction error can be observed by comparing models with the line of equivalence (solid line).

Journal of Dairy Science Vol. 102 No. 9, 2019

7958 DE SOUZA ET AL.

Table 5. Fit statistics for the cross-validation across studies of the proposed model and simplified model using
the evaluation data set

Model2

Fit statistic1 Proposed Simplified SEM3 P-value4

CCC 0.721 0.714 0.006 0.52
Mean bias 0.061 0.065 0.004 0.47
Slope bias −0.039NS −0.042NS 0.009 0.01
MSEP 8.36 9.05 0.241 0.01
RMSEP 2.89 3.01 0.038 0.01
Decomposition of MSEP, %
Mean bias 5.70 6.32 0.667 0.02
Slope bias 2.11 2.15 0.218 0.85
Random error 92.2 91.5 0.646 0.02
1
CCC = concordance correlation coefficient; MSEP = mean squared error of prediction; RMSEP = root MSEP.
2
The proposed model with parity effect and the simplified model using the proportion of the primiparous:​
multiparous (44:56) cows in the development data set. NS = not significantly different from zero.
3
SEM common to both proposed and simplified.
4
For the mean comparison of proposed and simplified.

in Table 6. The difference in the effect of DIM on DMI Dry matter intake changes less per unit change in
observed between the 2 equations may be attributable MilkE partly because the proposed equation has an
to differences in the covariates used to derive the equa- intercept of approximately 1.7 kg/d and is adjusted
tion. That is, whereas the proposed model included the for BCS, with low BCS increasing intake, especially
effect of parity, BCS, and DIM expressed as days, the for multiparous cows. Low BCS generally occurs when
NRC model does not account for differences in parity multiparous cows are at peak milk production, so this
or BCS and uses the week of lactation. Furthermore, might decrease the coefficient for milk. In addition, our
the NRC prediction equation for DMI was derived database includes only cows from the past approxi-
mostly from data collected in the 1990s, such that it mately 10 yr, and they had greater milk production
is likely that the data were collected on animals with that those of the NRC (2001). These more recent cows
lower milk production and lower body size than the also may have been fed diets that were higher in con-
animals in our database (mostly data collected since centrates than the diets fed to the cows used in the
2007). Additionally, since 2000, many researchers have NRC model so that less DMI was needed per unit of
focused on better understanding the mechanisms that milk. Cows in our database also were of more modern
control feed intake and the development of strategies to genetics. Moraes et al. (2015) showed that maintenance
maximize intake during early lactation (Drackley, 1999;
Allen, 2000; Grummer et al., 2004; Allen et al., 2009;
Allen, 2014). Although our proposed model accounts Table 6. Nonlinear adjustment of DIM on the predicted DMI
only for animal effects, we speculate that differences in estimated by the proposed (primiparous and multiparous) and NRC
(2001) models
management, environment, and diet composition (in-
cluded as random effects in our model) could explain Proposed model
the difference observed between our proposed model NRC
DIM Primiparous Multiparous model
and the NRC model with respect to the effect of DIM.
The estimated partial regression coefficients of DMI 10 0.875 0.795 0.624
on milk production and energy required for maintenance 20 0.927 0.880 0.714
30 0.957 0.929 0.783
based on the proposed model (0.305 kg of DMI/Mcal 40 0.975 0.958 0.835
of MilkE and 0.022 kg of DMI/kg of BW, respectively) 50 0.985 0.976 0.875
differed from those of the NRC (2001) model. For the 60 0.991 0.986 0.905
70 0.995 0.992 0.928
NRC equation, DMI increased 0.372 kg of DMI/kg of 80 0.997 0.995 0.945
FCM and 0.0968 kg of DMI/kg0.75 of BW, which con- 90 0.998 0.997 0.958
vert to 0.330 kg of DMI/Mcal of MilkE and 0.014 kg of 100 0.999 0.998 0.968
110 0.999 0.999 0.976
DMI/kg of BW for cows in the range of 500 to 700 kg 120 1.000 0.999 0.982
of BW. Thus, in our proposed model, changes in DMI 130 1.000 1.000 0.986
are less per unit change in MilkE output and more per 140 1.000 1.000 0.989
150 1.000 1.000 0.992
unit change in BW than in the NRC model.

Journal of Dairy Science Vol. 102 No. 9, 2019

 PREDICTIONS OF DRY MATTER INTAKE OF LACTATING DAIRY COWS 7959

Figure 6. Lactation curve for the average Holstein dairy cow in the data set. The solid lines represent the observed milk energy (MilkE; blue
with circles), observed DMI (red with diamonds), and observed BW (green with ×), and the dashed lines represent the DMI predicted by the
NRC (2001) (dotted line) and proposed (dashed line) models.

requirements have been increasing over the past 50 yr, CONCLUSIONS

and this may be part of the reason why the coefficient
for BW is greater. The proposed model to predict DMI contains the ef-
A substantial difference between the proposed model fect of MilkE, BW, BCS, parity, and DIM. This model
and the NRC (2001) DMI prediction equation is that differs from the NRC (2001) DMI prediction equation
our model includes the effect of BCS. The effect of BCS in that the NRC model contains only FCM, metabolic
was highly significant (P < 0.01) and interacted with BW, and week of lactation. Using an independent eval-
partiy, so that for each 1-unit decrease in BCS, intake uation data set, both models were similar for predict-
was increased 0.7 kg/d for primiparous cows and 2.6 ing DMI during early lactation (1–75 DIM), and the
kg/d for multiparous cows. These results are consistent proposed model outperformed the NRC model during
with the idea that thinner animals have greater ap- mid and late lactation (76–368 DIM). Finally, in regard
petites and consume more feed than fatter animals, at to the estimated coefficients for milk and BW, the pro-
least partly because they have less circulating leptin posed model suggests less change in DMI for changes
(Block et al., 2003; Allen, 2014). The reason why BCS in milk production but greater changes in DMI for
had less effect on DMI in primiparous cows may be changes in cow BW. We recongnize that our equation
that the cows are still growing and have flatter lacta- is based on cows mostly from the eastern United States,
tion curves than multiparous cows. We derived Equa- but we suggest that our new equation can serve as a
tion 7 to illustrate how the proposed model (Equation base prediction of voluntary DMI for lactating cows,
6) would be applied to a group of cows in which the especially when diet composition is unknown, and that
proportion of primiparous and multiparous animals is feed factors can be used to adjust the predicted DMI
known. Figure 6 illustrates the behavior of the pro- during ration formulation, as described in Allen et al.
posed and NRC (2001) models over a lactation curve of (2019).
a dairy cow using the observed DMI, MilkE, and BW of
the average cow in the evaluation data set. ACKNOWLEDGMENTS
Finally, we found important variability across diets
in these partial coefficients. This is not surprising based This project was supported by Agriculture and Food
on Jensen et al. (2015) and suggests that dietary charac- Research Initiative Competitive Grant no. 2011-68004-
teristics can enhance the prediction of DMI when they 30340 from the USDA National Institute of Food and
are known. Dietary characteristics were not known for Agriculture (Washington, DC) and by Michigan State
all cows in the current database, but the combination University AgBioResearch (East Lansing, MI). Rodrigo
of diet and animal factors is addressed in a companion Araujo de Souza was supported by a PhD fellowship
paper (Allen et al., 2019). from National Council for Scientific and Technological

Journal of Dairy Science Vol. 102 No. 9, 2019

7960 DE SOUZA ET AL.

Development (CNPq) from the Brazilian Ministry of Huhtanen, P., M. Rinne, P. Mäntysaari, and J. Nousiainen. 2011. In-
tegration of the effects of animal and dietary factors on total dry
Education (Brasilia, DF, Brazil). The authors thank matter intake of dairy cows fed silage-based diets. Animal 5:691–
L. Armentano, K. Weigel, D. Spurlock, C. Staples, E. 702. https:​/​/​doi​.org/​10​.1017/​S1751731110002363.
Connor, H. Dann, W. Weiss, and M. Hanigan for pro- Janovick, N. A., Y. R. Boisclair, and L. K. Drackley. 2011. Prepartum
dietary energy intake affects metabolism and health during the pe-
viding some of the data used in this analysis. riparturient period in primiparous and multiparous Holstein cows.
J. Dairy Sci. 94:1385–1400.
Jensen, L. M., N. I. Nielsen, E. Nadeau, B. Markussen, and P. Nør-
REFERENCES gaard. 2015. Evaluation of five models predicting feed intake by
dairy cows fed total mixed rations. Livest. Sci. 176:91–103.
Allen, M. S. 2000. Effects of diet on short-term regulation of feed in-
Krizsan, S. J., A. Sairanen, A. Höjer, and P. Huhtanen. 2014. Evalu-
take by lactating dairy cattle. J. Dairy Sci. 83:1598–1624.
ation of different feed intake models for dairy cows. J. Dairy Sci.
Allen, M. S. 2014. Drives and limits to feed intake in ruminants. Anim.
97:2387–2397.
Prod. Sci. 54:1513–1524.
Lu, Y., M. J. Vandehaar, D. M. Spurlock, K. A. Weigel, L. E. Ar-
Allen, M. S., B. J. Bradford, and M. Oba. 2009. Board-invited review:
mentano, C. R. Staples, E. E. Connor, Z. Wang, M. Coffey, R.
The hepatic oxidation theory of the control of feed intake and its
F. Veerkamp, Y. de Haas, and R. J. Tempelman. 2017. Modeling
application to ruminants. J. Anim. Sci. 87:3317–3334. https:​/​/​doi​
genetic and nongenetic variation of feed efficiency and its partial
.org/​10​.2527/​jas​.2009​-1779.
relationships between component traits as a function of manage-
Allen, M. S., D. O. Sousa, and M. J. VandeHaar. 2019. Equation to
ment and environmental factors. J. Dairy Sci. 100:412–427.
predict feed intake response by lactating cows to factors related to
Moraes, L. E., E. Kebreab, A. B. Strathe, J. Dijkstra, J. France, D. P.
the filling effect of rations. J. Dairy Sci. 102:7961–7969. https:​/​/​
Casper, and J. G. Fadel. 2015. Multivariate and univariate analy-
doi​.org/​10​.3168/​jds​.2018​-16166.
sis of energy balance data from lactating dairy cows. J. Dairy Sci.
Arnedal, S. 2005. Predictions for voluntary dry matter intake in dairy
98:4012–4029. https:​/​/​doi​.org/​10​.3168/​jds​.2014​-8995.
cows. PhD Diss. Department of Animal Nutrition and Manage-
NRC. 2001. Nutrient Requirements of Dairy Cattle. 7th rev. ed. Natl.
ment, Swedish University of Agricultural Sciences, Uppsala, Swe-
Acad. Press, Washington, DC.
den.
Rayburn, E. B., and D. G. Fox. 1993. Variation in neutral detergent
Block, S. S., J. M. Smith, R. A. Ehrhardt, M. C. Diaz, R. P. Rhoads,
fiber intake of Holstein cows. J. Dairy Sci. 76:544–554.
M. E. Van Amburgh, and Y. R. Boisclair. 2003. Nutritional and
Roseler, D. K., D. G. Fox, L. E. Chase, A. N. Pell, and W. C. Stone.
developmental regulation of plasma leptin in dairy cattle. J. Dairy
1997. Development and evaluation of equations for prediction of
Sci. 86:3206–3214.
feed intake for lactating Holstein dairy cows. J. Dairy Sci. 80:878–
Chan, P. S., J. W. West, and J. K. Bernard. 2006. Effect of prepartum
893.
dietary calcium on intake and serum and urinary mineral concen-
Tedeschi, L. O. 2006. Assessment of the adequacy of mathematical
trations of cows. J. Dairy Sci. 89:704–713.
models. Agric. Syst. 89:225–247.
Dann, H. M., G. A. Varga, and D. E. Putnam. 1999. Improving energy
Tempelman, R. J., D. M. Spurlock, M. Coffey, R. F. Veerkamp, L. E.
supply to late gestation and early postpartum dairy cows. J. Dairy
Armentano, K. A. Weigel, Y. de Haas, C. R. Stamples, E. E. Con-
Sci. 82:1765–1778.
nor, Y. Lu, and M. J. VandeHaar. 2015. Heterogeneity in genetic
de Souza, R. A., and M. J. VandeHaar. 2018. Determining the change
and nongenetic variation and energy sink relationships for residual
in body weight per unit of body condition score in Holstein cows.
feed intake across research stations and countries. J. Dairy Sci.
J. Dairy Sci. 101(Suppl. 2):153. (Abstr.)
98:2013–2026.
Drackley, J. K. 1999. Biology of dairy cows during the transition pe-
VandeHaar, M. J., L. E. Armentano, K. Weigel, D. M. Spurlock, R.
riod: The final frontier? J. Dairy Sci. 82:2259–2273.
J. Tempelman, and R. Veerkamp. 2016. Harnessing the genetics of
Fox, D. G., M. E. Van Amburgh, and T. P. Tylutki. 1999. Predict-
the modern dairy cow to continue improvements in feed efficiency.
ing requirements for growth, maturity, and body reserves in dairy
J. Dairy Sci. 99:4941–4954.
cattle. J. Dairy Sci. 82:1968–1977.
Zom, R. L. G., G. Andre, and A. M. van Vuuren. 2012. Development
Grummer, R. R., D. G. Mashek, and A. Hayirli. 2004. Dry matter in-
of a model for the prediction of feed intake by dairy cows: 1. Pre-
take and energy balance in the transition period. Vet. Clin. North
diction of feed intake. Livest. Sci. 143:43–45. https:​/​/​doi​.org/​10​
Am. Food Anim. Pract. 20:447–470. https:​/​/​doi​.org/​10​.1016/​j​
.1016/​j​.livsci​.2011​.08​.014.
.cvfa​.2004​.06​.013.

Journal of Dairy Science Vol. 102 No. 9, 2019