Aceituno et al.

BMC Public Health (2017) 17:911

DOI 10.1186/s12889-017-4904-5

RESEARCH ARTICLE Open Access

Using a monitoring and evaluation

framework to improve study efficiency and
quality during a prospective cohort study
in infants receiving rotavirus vaccination in
El Alto, Bolivia: the Infant Nutrition,
Inflammation, and Diarrheal Illness (NIDI)
study
Anna M. Aceituno1,2*†, Kaitlyn K. Stanhope3†, Paulina A. Rebolledo2,4, Rachel M. Burke3, Rita Revollo5,
Volga Iñiguez6, Parminder S. Suchdev2,4,7 and Juan S. Leon2

Abstract
Background: Implementing rigorous epidemiologic studies in low-resource settings involves challenges in participant
recruitment and follow-up (e.g., mobile populations, distrust), biological sample collection (e.g., cold-chain, laboratory
equipment scarcity) and data collection (e.g., literacy, staff training, and infrastructure). This article describes the use of a
monitoring and evaluation (M&E) framework to improve study efficiency and quality during participant engagement,
and biological sample and data collection in a longitudinal cohort study of Bolivian infants.
Methods: The study occurred between 2013 and 2015 in El Alto, Bolivia, a high-altitude, urban, low-resource
community. The study’s M&E framework included indicators for participant engagement (e.g., recruitment, retention,
safety), biological sample (e.g., stool and blood), and data (e.g., anthropometry, questionnaires) collection and quality.
Monitoring indicators were measured regularly throughout the study and used for course correction, communication,
and staff retraining.
(Continued on next page)

* Correspondence: aaceituno@[Link]
†
Equal contributors
1
RTI International, 3040 E. Cornwallis Road, PO Box 12194, Research Triangle
Park, NC 27709, USA
2
Hubert Department of Global Health, Rollins School of Public Health, Emory
University, Atlanta, GA, USA
Full list of author information is available at the end of the article

© The Author(s). 2017 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License ([Link] which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
([Link] applies to the data made available in this article, unless otherwise stated.
Aceituno et al. BMC Public Health (2017) 17:911 Page 2 of 12

(Continued from previous page)

Results: Participant engagement indicators suggested that enrollment objectives were met (461 infants), but
15% loss-to-follow-up resulted in only 364 infants completing the study. Over the course of the study, there
were four study-related adverse events (minor swelling and bruising related to a blood draw) and five severe
adverse events (infant deaths) not related to study participation. Biological sample indicators demonstrated
two blood samples collected from 95% (333 of 350 required) infants and stool collected for 61% of reported
infant diarrhea episodes. Anthropometry data quality indicators were extremely high (median SDs for weight-
for-length, length-for-age and weight-for-age z-scores 1.01, 0.98, and 1.03, respectively), likely due to extensive
training, standardization, and monitoring efforts.
Conclusions: Conducting human subjects research studies in low-resource settings often presents unique
logistical difficulties, and collecting high-quality data is often a challenge. Investing in comprehensive M&E is
important to improve participant recruitment, retention and safety, and sample and data quality. The M&E
framework from this study can be applied to other longitudinal studies.

Background inflammation due to co-infection with other enteric patho-

The challenges in implementing epidemiologic studies in gens [16–25]. Few studies have tested hypothesized causes
low-resource settings, particularly with infants, include ef- for reduced RV vaccine effectiveness in low-resource set-
fectively engaging with mobile communities [1], recruiting tings [18]. Testing these hypotheses requires human sub-
and training qualified staff [2], avoiding cold-chain or lab jects’ studies in low-resource settings, where the vaccines
supply disruption [3], ensuring quality equipment for are least effective. The additional logistical challenges of
sample processing, minimizing burden to participants, conducting human subjects research in low-resource set-
and building relationships and communicating between tings with vulnerable populations (infants) could interfere
multi-national study teams [4]. Though the methods of with data quality and participant safety. A robust M&E
such studies are sometimes reported [5–7], strategies and framework may be helpful to continuously monitor and im-
systems for maintaining data quality and knowledge prove participant engagement, biological sample, and data
exchange with local partners are not well documented. quality and collection.
Monitoring and evaluation (M&E) frameworks offer This paper describes the use of a monitoring and
a metric for tracking progress towards project goals evaluation (M&E) framework to improve study efficiency
through a logical framework documenting intermedi- and quality during participant engagement, and
ate and long-term measurable objectives [8, 9]. Inter- biological sample and data collection in human subjects
mediate objectives are ways to check on individual research in a low-resource setting. We use as a case
pieces of a study and are measured once or more at study an observational prospective cohort study to
pre-set intervals during implementation, allowing evaluate the effect of infant chronic undernutrition on
study staff to reassess implementation strategy if ob- infant RV-specific immunogenicity of 364 infants in El
jectives are not met. Long-term objectives clarify Alto, Bolivia.
plans for the study [8]. M&E frameworks facilitate
adaptation to real-world conditions during study im- Methods
plementation, improving data quality, participant Study protocol
safety, and study efficiency. Participant engagement The primary objective of the Infant Nutrition, Immun-
(e.g., recruitment, retention, safety), biological sample ology, and Diarrheal Illness study (Nutrición, Inmunología
(e.g., stool and blood), and data (e.g., anthropometry, y Diarrea Infantil - NIDI study) was to evaluate the effect
questionnaires) collection and quality during human of infant chronic undernutrition (length-for-age z-score
subjects research with vulnerable populations or in (LAZ) < −2 or weight-for-length z-score (WLZ) < −2 [26])
low-resource settings. on infant RV-specific immunogenicity of infants in El
Rotavirus (RV) is the most common cause of severe diar- Alto, Bolivia. Secondary objectives were to evaluate the ef-
rhea in infants worldwide [10, 11], despite widespread im- fects of (1) maternal and infant micronutrient deficiency,
plementation of anti-RV oral vaccines Rotarix® and (2) maternal RV-specific immunity, and (3) early-life
Rotateq®. These vaccines are least efficacious and effective enteric infection and inflammation on infant RV-specific
in developing countries where morbidity and mortality immunogenicity. A tertiary objective was to evaluate the
from RV are high [12–15]. Proposed hypotheses for re- effects of early-life enteric infection, subclinical inflamma-
duced oral vaccine effectiveness include growth impair- tion, nutritional status, and post-vaccination RV-specific
ment, micronutrient deficiency in infants and mothers, and immunity on malnutrition at one year of age.
Aceituno et al. BMC Public Health (2017) 17:911 Page 3 of 12

Bolivia was selected because it has a high prevalence up based on previous pilot studies at the hospitals, esti-
of infant and maternal anemia and undernutrition, high mated non-compliance with sample collection. Rigorous
incidence of diarrheal illness, high under-five mortality participant safety M&E indicators focused on adverse
rate [27], and because it provides, as part of state- events (AE) related to the infant blood draws and on in-
sponsored programming, the Rotarix® vaccine free of fant mortality (a severe adverse event (SAE) not ex-
charge during well-child visits. Following a pilot study in pected to be related to the study) because of the high
three hospitals, the city of El Alto was selected as a loca- infant mortality rate in Bolivia [30]. This AE and SAE
tion with proximity to a partner laboratory and home to monitoring was not required by Emory Institutional Re-
a marginalized indigenous population with chronic view Board or the Bolivian National Bioethics Commit-
undernutrition. tee, as this study was not an interventional trial, but was
A sample size of 350 sets of maternal-infant data was done to ensure that any unforeseen risks to participants
calculated to provide 80% power (2-sided α ≤ =0.05), based would be identified quickly during data collection, so
on 1) an 18% difference between rates of RV-vaccine anti- they could be addressed in real time. Biological sample
body seroconversion among children with vs. without M&E indicators were determined in collaboration with
chronic undernutrition [12–14, 28], and 2) an estimated the laboratories that analyzed the resulting samples, fo-
prevalence of 29–38% chronic undernutrition in mothers cusing on the factors most likely to affect stool or
and infants. Unpublished pilot data collected 2010–2011 plasma quality and analysis. Anthropometric M&E indi-
at the study hospitals found that 29% of children less than cators were developed by the expert anthropometrist
four months of age had moderate to severe chronic under- (PSS) based on World Health Organization (WHO) best
nutrition undernutrition (LAZ < −2 or WLZ < −2 [26]) practice [31, 32]. Anthropometric measurements were
and the 2008 Bolivia National Demographic and Health monitored closely using multiple indicators (Table 3) be-
Survey found that 38% of women had anemia cause of the difficulty of precise and accurate anthropo-
(Hemoglobin (Hb) <14.7 g/dl) [27]. Based on an estimated metric data collection in infants and because these
22% loss to follow-up (LTFU) or failed blood sample col- measurements directly affected the primary study out-
lection (unpublished pilot data), the recruitment objective come. Because the study staff were collecting the infants’
was set at 450 mother-infant (M-I) pairs. anthropometric data for their well-child visits (rather
Data were collected June 2013 – March 2015. There than the hospital nurses), an additional participant safety
were up to ten study visits; seven or eight scheduled M&E indicator included the percentage of infants with
hospital visits at well-child visits over 12–18 months, chronic or acute malnutrition that were identified as
and one or two at home 4 and/or 7 days after the first such upon anthropometric measurement, to ensure that
dose of the RV vaccine (Table 1). Some infants returned study participants were appropriately flagged for evalu-
to the hospital for an eighth visit for a third blood sam- ation by their pediatricians during the well-child visits
ple collection if it was not collected at the seventh well- that followed the study visits.
child hospital visit at one year of age. The first two study
visits were prior to the initial dose of the RV vaccine and Staff training
served as baseline data. Please see the first study visit NIDI staff underwent a 10-day training on the Bolivian
questionnaire provided as an Additional file 1 (NIDI_S- health system, interview protocols, informed consent,
tudy_Questionnaire_Visit_1.pdf ) for complete baseline anthropometry, clinical sample collection and process-
information collected. ing, treatment referrals, and AE. An experienced anthro-
pometrist (PSS) conducted anthropometry training using
Development of the monitoring and evaluation methods developed by the WHO [33, 34]. During the
framework initial training and bi-annual standardizations, staff mea-
The NIDI study M&E framework was developed using a sured the recumbent length of 10 children under two
formal M&E logical framework [29] based on the overall years old twice, and continued until they produced mea-
study goal and specific objectives and included a com- surements within 0.5 cm of repeat measurements and of
prehensive data management plan and objectives in the the expert anthropometrist’s measurement in order to
areas of participant engagement (e.g., recruitment, reten- optimize precision and accuracy of measurements [33].
tion, safety), biological sample (e.g., stool and blood), Staff were trained in human subjects research ethics
and data (e.g., anthropometry, questionnaires) collection using the FHI 360 Research Ethics Training Curriculum
and quality (Tables 2 and 3). (FHI 360, Durham, NC) and passed the corresponding
Participant engagement (e.g., recruitment, retention, evaluation [35]. Phlebotomy staff at study hospitals were
safety) M&E indicators were determined based on the trained on study protocols for blood collection, process-
statistical power needed to answer the primary and ing, and cold chain following Centers for Disease Con-
secondary study objectives, the estimated loss-to-follow trol and Prevention (CDC) guidelines [36]. During
Aceituno et al. BMC Public Health (2017) 17:911 Page 4 of 12

Table 1 The NIDI study visit and data collection schedule 2012–2015
Measure Study Visit (Infant Age in Months)
Target visit schedule 1 (1) 2 (2) 3+ (2) 4 (3) 5 (4) 6 (6–8) 7 (9) 8+
(12–18)
RVa vaccine criteria – – 4 and/or 7 days post – – 60 days post – –
RV dose 1 RV dose 2
Maternal Characteristics
Weight and height X X X X X X X
Clinical and prenatal history X X
Prenatal and postpartum micronutrient supplementation X X
Two-week morbidity X X X X X X X X
b
Plasma anti-RV IgG X
Plasma ferritin, sTfRc, CRPd, AGPe X X
Maternal/Infant SES
Sociodemographic X X
Household characteristics X X
Infant feeding practices, iron supplementation X X X X X X X X
Infant Nutritional & Inflammation Status
Anthropometry X X X X X X X
Vitamin A, zinc, micronutrient sprinkles supplementation X X
Plasma ferritin, sTfRb, zinc, CRPd, AGPe X X X
Infant Infection and Morbidity
Two-week morbidity and diarrhea recall X X X X X X X X
Vaccines received X X
Fecal RV X X X
Fecal RV and enteric co-pathogens When diarrhea
reported
Outcome: Infant RV Vaccine Immunogenicity
Plasma anti-RV IgG X X X
Plasma anti-RV IgA X X X
a
RV = Rotavirus
b
IgG = Immunoglobulin
c
sTfR = Soluble transferrin receptor
d
CRP = C-reactive protein
e
AGP=α 1-acid glycoprotein

training, indicators of participant’s engagement, bio- age. Infants were eligible if they were ≥14 days old, in
logical sample and data collection and quality to be mea- general good health, and had not yet been vaccinated
sured during the study were shared with staff. with Rotarix®. Infants were ineligible if they had fever,
diarrhea, frequent cough, respiratory infection, or
Participant’s engagement and indicators hospitalization in the past week, had been diagnosed
Enrollment and retention with a birth defect, chromosomal disorder affecting
Potential study participants were identified in the out- growth, or immunodeficiency disorder. Eligible
patient clinic waiting rooms of Hospital Municipal mothers were given information sheets and invited to
Corea or Hospital Los Andes. Mothers of young in- enroll that day or at their next well-child visit (Fig. 1).
fants were approached, and if interested, their eligibil- The NIDI study logo (Fig. 2b) was on all study docu-
ity was assessed via a screening questionnaire. ments, and staff wore study clothing and photo iden-
Mothers were eligible if they were ≥15 years, in gen- tification cards bearing the logo so that mothers
eral good health, the mother of an eligible infant, and could easily identify study staff in the hospitals.
were willing to bring their infant to the study hospital Mothers were compensated for their time at each
for well-child and study visits through one year of visit with food staples (e.g., 1 pound fortified rice).
Aceituno et al. BMC Public Health (2017) 17:911 Page 5 of 12

Table 2 Example Evaluation Indicators for Participant Enrollment, Follow-Up, Safety and Biological Samples, NIDI Study, 2013–2015
Study Area Indicator Data Source for Indicator Results
Enrollment All eligible M-I pairs at each hospital are Clinical records review 2331 charts screened; 2203 age-eligible infants
screened identified
Enrollment of 422 M-I pairs in 3 months Enrollment data (weekly Enrollment of 456 M-I pairs in 9 months
(3–4 pairs/day/hospital) monitoring throughout study)
100% of M-I pairs have properly Paper copies of consent forms Successful consent collection from 100% of pairs
documented consent forms
Follow-up 10% LTFU (8–9 pairs lost/visit, 380 pairs Enrollment data (weekly 15% LTFU (364 infants completed study)
complete study) monitoring throughout study)
Safety 0 adverse events and 0 study-related Documentation and follow up 4 adverse eventsa; 5 severe adverse events, 0
severe adverse events of all adverse events (with study-related severe adverse events
standard form)
Infant and maternal death rates below Documentation and follow up Among infants enrolled in the NIDI study, there
DHS rates for Bolivia of all deaths were 0 maternal deaths and 5 infant deaths,
equivalent to 10.8 per 1000, less than half of
Bolivia’s infant mortality rateb.
Monthly reporting of AE/SAE, details Documentation of AE/SAE, 100% of withdrawals, deaths, AE/SAE
of any deaths, reasons for all withdrawals deaths and reasons for withdrawal documented
using standard forms
Biological Samples Successful collection of both infant Documentation of blood sample Successful collection of first two infant blood
blood samples from 82% of M-I pairs collection through survey and samples from 327 infants (75%)
laboratory data
Successful collection of shedding stool Documentation of stool collection Successful collection of shedding stool samples
samples from 50% of infants through survey and laboratory data from 75% of infants
Successful collection of 25% of diarrhea Documentation of stool collection Successful collection of 61% samples of reported
samples (50% loss to non-reporting, through survey and laboratory data diarrhea cases.
collect 50% reported samples) Comparison to Bolivia DHS data
a
Adverse events included minor bruising and swelling from blood draws
b
In Bolivia, there were 23 infant deaths per 1000 post-neonatal infants; 34.7 maternal deaths per 100,000 live births (DHS, 2008)

Table 3 Example Monitoring Indicators for Data Collection, NIDI Study, 2013–2015
Study Area Indicator Data Source for Indicatora Strategies Taken If Indicator Criteria Not Met
(Examples)
Digit Preference (head Percent of measurements at each Infant length, maternal - Re-training of interviewers
circumference, weight, and digit (0.0–0.9) between 8 and 12% height, and weight data - Communication of finding in monthly
height or length) meetings
Percent Missing Data (age, No missing data Survey data - Explanation of importance of complete
weight, height or length, data collection
and gender) - Incentives for months with no missing
data
Standard Deviation of LAZ: 1.10–1.30 Infant length, maternal - Communication in monthly meetings
Z-Scores (WLZ, LAZ, WAZ) WAZ: 1.00–1.20 height, and weight data - Recognition of interview staff for meeting
WLZ: 0.85–1.10[53] of goals
Discrepancies in calculated Discrepancies ≥15 days flagged Survey data - Use of calculated age in analysis
v. reported age (in days) - Implementing data check of surveys at
data entry point
Age range for given visit, 0 infants outside of specified Survey data - Review of protocols for recruitment
number of infants outside range - Reminder calls to mothers for visits
specified range
Referrals for stunting, 100% of infants correctly referred Survey data; referral - Retraining on referral protocol
wasting and those with forms - Recognizing staff members in monthly
ongoing diarrhea meeting for correctly referring study
participants
a
All indicators were monitored on an ongoing basis and presented in a monthly report to Bolivia and Emory University staff
Aceituno et al. BMC Public Health (2017) 17:911 Page 6 of 12

Fig. 1 The NIDI Study Enrollment and Loss to Follow-up, 2013–2015. * Five twin pairs were enrolled. ¥ The third infant blood draw was at 12–18 months
of age due to new funding and new research questions, after many pairs had already completed the final study visit, resulting in a smaller sample size

Fig. 2 NIDI Study Documents. a An example of the growth charts included in the handout mothers were given at the end of the study. See
Appendix A for SAS program code used to create the growth chart, [Link]. b The NIDI study logo (pending copyright),
featuring an indigenous Aymara mother and infant. The logo was used on study documents, staff clothing and identification cards so that study
participants could quickly find and identify staff. c An example of a figure from the monthly monitoring and evaluation reports. Identifying any
digit preferences in anthropometric measurements was one method used to monitor measurement quality
Aceituno et al. BMC Public Health (2017) 17:911 Page 7 of 12

Enrollment and retention indicators included weekly stool was available in the soiled diaper, the soiled area
and monthly enrollment numbers and percent lost to of the diaper was eluted in 500 ul of PBS. Stool was
follow up by visit. also tested for ETEC, EPEC, and EAEC as described
in Gonzales et al., 2013 [39]. Stool collection and
Participant safety quality indicators included the percent of samples
RV vaccination was conducted as part of the Bolivian successfully collected following maternal report of in-
national vaccination schedule following the blood draw fant diarrhea.
at the second study visit which coincided with the in- Venous blood was collected from mothers approxi-
fants’ 2-month well-child visits. The blood draw was mately one and 6–8 months postpartum, and from infants
considered an intervention in this study. RV vaccination at two, 6–8, and 12–18 months of age by the phlebotomy
was not a study intervention as it was administered as staff at study hospital laboratories (Fig. 1). Blood was col-
part of standard well-child care by hospital staff follow- lected by trained hospital phlebotomists and processed by
ing the Bolivian national vaccination schedule. Thus, study staff using sterile, disposable equipment. Blood was
vaccination-related AE would have been recorded but drawn using Safety-Lok™ 23-gauge winged needles and
not considered study-related AE. Expected related AE trace-metal-free EDTA Vacutainers® (BD, Franklin Lakes,
were blood draw complications including bruising, infec- NJ). An aliquot of whole blood from the needle was ana-
tion, or excessive bleeding within 48 h of blood collec- lyzed for Hb using the HemoCue® Hb 201+ System
tion. No study-related SAE were expected; expected (HemoCue AB, Ängelholm, Sweden); anemia was defined
unrelated SAE included hospitalization, disability, or using cut-offs adjusted for altitude (3500 m) and preg-
death. All AE and SAE were reviewed by a pediatrician nancy [40]. The remaining sample was processed to
to determine severity and relation to the study within plasma as per Vacutainer® instructions: the vial was
48 h. AE and SAE indicators included review and inverted 8 times, then centrifuged for 15 min at 1300
follow-up of all AE and SAE reported by participants, relative centrifugal force (RCF) (LW Scientific USA,
number of study related AE and number and follow-up Lawrenceville, GA) to separate the blood. The plasma
of study and non-study related SAE. supernatant was then transferred to microcentrifuge tubes
Mothers were referred to treating physicians for a and centrifuged for 3 min at 2200 RCF (Cole Parmer,
body-mass index <18.5 kg/m2 (or mid-upper arm cir- Vernon Hills, IL) to remove any remaining precipitates.
cumference < 21.0 cm if pregnant) or Hb <14.7 g/dl (Hb The supernatant was stored in screw-cap microtubes at
< 13.7 g/dl if pregnant) [37]. Infants were referred to 2–8 °C (up to 24 h) before being transported on ice to the
treating physicians for moderate or severe undernutri- UMSA-IBMB laboratory, where the tubes were stored at
tion (LAZ < −2 or WLZ < −2 [26]), anemia (Hb < 10.9 g/ −70 °C until being transported on dry ice to Emory Uni-
dl as per Bolivian national guidelines [38]), for any signs versity laboratories following International Air Transport
of severe illness, or if mothers reported diarrhea. If Association and CDC regulations. There plasma was ali-
mothers reported diarrhea between visits, staff would quoted and shipped on dry ice to final analysis locations.
collect a stool sample, test for RV, enterotoxigenic E. coli Blood quality indicators included percent of samples with
(ETEC), enteropathogenic E. coli (EPEC), enteroaggres- hemolysis, storage temperature and storage time.
sive E. coli (EAEC), and shared results with the treating Plasma was analyzed for RV-specific Immunoglobu-
physician. Maternal or infant morbidity indictors related lins A (IgA) and G (IgG) by Enzyme-linked immu-
to participant safety included referral of each malnutri- nosorbent assay (ELISA) by the Gastroenteritis &
tion case to appropriate physicians for treatment and Respiratory Viruses Laboratory, Division of Viral Dis-
follow-up of all malnutrition cases at subsequent visits. eases, CDC as described in Moon et al., 2010 [41].
Plasma samples were analyzed for retinol binding pro-
Biological sample collection, quality, and indicators tein (RBP), ferritin, soluble transferrin receptor (sTfR),
At each visit, mothers provided their infant’s diaper alpha(1)-acid-glycoprotein (AGP), and C-reactive pro-
with stool, if available. Study staff provided a clean tein (CRP) by the VitMin Laboratory in Willstaett,
diaper and transferred the stool to a sample container Germany, as described in Erhardt et al., 2004 [42],
and a portion to Cary-Blair transport medium. Sam- and for plasma zinc using inductively-coupled plasma
ples were stored at 2–8 degrees Centigrade (°C) and optical emission spectrometry by the Children’s Hos-
transported on ice within 24 h to the Universidad pital of Oakland Research Institute Elemental Analysis
Mayor de San Andrés Instituto de Biología Molecular Facility as described in Engle-Stone et al. 2014 [43].
y Biotecnología (UMSA-IBMB) laboratory for process- A subsample (10%) were also tested for retinol; re-
ing. Stool samples were analyzed for RV with the sults were used to validate RBP cutoffs for vitamin A
ProSpecT RV Enzyme-linked immunosorbent assay deficiency since the molar ratio of retinol-to-RBP is
(ELISA) kit (Oxoid, Basingstoke, UK). If no formed not always 1:1 [44]. All micronutrient biomarkers
Aceituno et al. BMC Public Health (2017) 17:911 Page 8 of 12

were adjusted for the effect of inflammation using a quality targets. All study components were monitored
novel regression correction approach [45–47] . in-person by principal investigators or project managers
2–3 times a year.
Data collection, quality, and indicators
Vaccination data for the infants was collected from the Results
infants’ clinical vaccination records, provided by the While indicator targets for participant enrollment were
mothers at each study visit and was recorded by study met, the number of infants lost to follow-up was greater
staff as part of each study visit’s data collection. Sociode- than expected. Of 2331 M-I pairs initially screened, 1336
mographic data were collected at the first and last visit, infants were eligible, and 456 mothers and 461 infants
morbidity and clinical data at all visits. Mothers were (including 5 twins) were enrolled in the study June
given a wall calendar and stickers of images indicating 2013–April 2014, exceeding the target of 422 M-I pairs.
morbidities (diarrhea, fever, cough, hospitalization) as a Six percent (27/461) of infants were excluded as ineli-
memory aid to record infant morbidities. Mothers were gible, primarily due to having received the RV vaccine
asked about infant feeding practices, including breast- prior to the first infant blood-draw. Fifteen percent (70/
feeding. Infant nutritional status was determined by 461) of infants were lost to follow-up, which did not
LAZ, WLZ, and head circumference at each visit by meet the target of <10% LTFU (Fig. 1). However, 91%
trained interviewers using standardized wooden boards (364/400) of infants that completed the first two visits
to measure supine length (Shorrboards, Olney, MD) and completed the study. In total, 364 infants completed the
an electronic M-I floor scale to measure weight (SECA study through the final visit.
scales, Hanover, MD). Measurements were placed in the There were four study-related AE (all minor swelling
infant’s clinical chart for their corresponding well-child and bruising related to the blood draw), no study-related
visit, and mothers were given a handout at the end of SAE, and five infant deaths (SAE) unrelated to the study.
the study documenting their child’s growth throughout SAE included hypoxic-ischemic encephalopathy (age
the study (Fig. 2a). The SAS program code used to cre- 4 months), pneumonia (two at ages 2 and 3 months),
ate the growth chart (NIDI_GrowthCurveMacro.sas) is diarrhea (age 2 months), and sudden infant death syn-
provided as an Additional file 2. Maternal height and drome (age 3 months). After review of death certificates
weight were collected at each visit to calculate body and conducting verbal autopsies when necessary, a
mass index. Data were collected on paper forms and pediatrician determined each death was unrelated to the
double-data entered using the web-based research elec- study due to its medical cause. This mortality rate of in-
tronic data capture (REDCap) system hosted at Emory fants in the study, equivalent to 10.8 per 1000 enrolled
University (UL1 TR000421) [48]. Discrepancies between infants, is less than half the infant mortality rate in
entries were reconciled within a week. Any biologically Bolivia (23 per 1000 post-neonatal infants) [27].
implausible data points were corrected or set to a Biological sample indicator targets were met for stool
missing value. Sample processing and cold chain data sample collection but not blood sample collection
were double-entered and reconciled in Microsoft Excel (Table 2). There were 389 successful infant blood draws
(Seattle, WA). at 1–2 months (baseline), but only 333 (95% of the re-
Data collection and quality indicators included digit quired 350 blood sample size) also had a 6–8 month
preference for anthropometry measures (height, weight sample (Fig. 1). Hemolysis occurred frequently during
and length), standard deviations of weight-for-length, blood draws, despite proper collection and processing
length-for-age and weight-for-age z-scores and percent technique, but did not affect sample analysis. Stool sam-
missing anthropometric data for each visit. ples to assess post-vaccination RV viral shedding were
collected from 75% of infants, while 61% of reported
Reporting and application of M&E indicators diarrhea cases had associated stool samples collected.
Emory University staff created monthly reports of Anthropometry data quality indicators were monitored
monitoring indicators including measures of participant throughout the study (Table 3). The medians of monthly
retention, biological sample collection (e.g., hemolysis, standard deviations for weight-for-length, length-for-age,
storage temperature, and storage time for blood sam- and weight-for-age z-scores were 1.01, 0.98, and 1.03, re-
ples), and data quality. LTFU and reasons for withdrawal spectively; close to the expected value of 1.0 for a refer-
were presented by visit. For anthropometry data quality ence distribution. Standard deviations for z-scores varied
indicators (Table 3), digit preferences in measurements month-to-month, but never reached the WHO thresh-
were visualized in a graph (Fig. 2c). Reports were shared olds for measurement error or incorrect age reporting
with study staff during monthly meetings, strategies [49]. However, only 61% of measurements indicating
were developed to improve data quality, goals were set stunting and only 56% of measurements indicating wast-
for the next month, and rewards were given for meeting ing resulted in an immediate referral for treatment.
Aceituno et al. BMC Public Health (2017) 17:911 Page 9 of 12

Referral practices improved during the second year of Anthropometry data quality was extremely good,
data collection: 87% of measurements indicating stunt- likely due to extensive training, standardization, and
ing and 91% of measurements indicating wasting re- monitoring efforts. However, many children were not
sulted in an immediate referral. Throughout the study, immediately referred for treatment when they were
these indicators were presented to study staff each measured as stunted or wasted. Each month, this in-
month to encourage improvement and follow-up. Thus, dicator was measured and presented to study staff to
all stunted and wasted children who had not initially encourage improvement and follow-up. Thus, all
been referred were followed-up to ensure they received stunted and wasted children who had not initially
treatment. been referred were followed-up to ensure they re-
ceived treatment. Poor referral practices were likely
Discussion the result of failure to identify stunting and wasted
This paper describes the data and participant safety infants using the WHO growth charts, especially in
M&E framework of an observational cohort study of the youngest infants and early in the study, when
mothers and their infants and challenges encountered staff burden was highest. In addition, study staff
in the implementation of this study in El Alto, stopped referring children measured as stunted at re-
Bolivia. The M&E framework allowed staff to respond peated visits, as they were already being monitored by
both to predicted and unforeseen challenges in study their physicians.
implementation. The monitoring plan, based on the M&E framework
The study met its enrollment objective (450 M-I indicators, successfully allowed course-correction and
pairs). However, it collected 95% of the required first quality improvement throughout the study. Well-
two infant blood samples from 350 infants which meant trained local staff, a clear protocol for AE, and a
it did not meet its sample size objectives (350 infants), comprehensive data management plan facilitated the
although 364 infants completed the final study visit. success of the study. Having well-trained staff who
Follow-up was hindered by unexpected and rapid policy spoke local languages and understood the political
changes to the Bolivian health system in December and cultural landscape allowed the study to continue
2013, which shifted well-child visits to primary care despite local political unrest and health policy
clinics during data collection [50, 51]. Though partici- changes. Preparation for, and management of, AE
pants could continue well-child visits at the study took considerable time and resources. Although this
hospital, conflicting national messages and the conve- was an observational study, this effort was warranted
nience of primary care clinics reduced follow-up. given cultural sensitivity to blood collection and the
Reported AE were minor (local bruising and swell- vulnerable population (infants as young as two
ing related to the blood draw). All infant deaths were months). Having a comprehensive data management
determined to be unrelated to the study after verbal plan allowed for high quality data control. Monthly
autopsy and review of the death certificate by the monitoring meetings in which the M&E framework
study pediatrician. Though the target for this indica- indicators were reviewed and discussed with in-
tor was no SAE during the course of the study, infant country staff maintained a sense of accountability and
mortality in the study was less than half of the post- ownership, and allowed for open discussion of study
neonatal infant mortality rate in Bolivia [27]. This challenges and potential solutions.
could be due to differences between the national Some aspects of development, implementation and
population and the populations attending the two adaptation of the M&E framework could be improved
study hospitals. in future studies. In the development of the frame-
The study collected samples from only 18% of the pre- work, indicators for staff satisfaction and retention
dicted diarrhea case count, but collected a sample 61% were not considered, although staff data collection
of the time when diarrhea was reported. As the monthly burden was monitored. As training and management
monitoring indicator was the percent of reported diar- of local staff took considerable time and resources,
rhea that resulted in sample collection, no changes were monitoring of staff satisfaction may have allowed
made in study protocol to collect more samples. Diar- earlier course correction in this area. In the second
rhea may have been under-reported, but it is also pos- year of the study, staff satisfaction surveys were
sible that the population simply had fewer cases of implemented, which allowed in-country and Emory
diarrhea during the first year of life than were predicted University project managers to improve staff satisfac-
based on pilot data. Post-RV vaccination stool samples tion and provide additional resources to in-country
collection (75%) exceeded the goal of 50% of enrolled in- staff as needed. In the implementation of the
fants, likely due to the convenience of the home visit(s) framework, specific aspects of how indicators would
for sample collection. be analyzed and communicated to all study team
Aceituno et al. BMC Public Health (2017) 17:911 Page 10 of 12

members were not considered until data collection + Children’s Pediatric Center Seed Grant Program (received by PS), the Public
was underway. Including a detailed communication Health Service Grant UL1 TR000454 from the NIH Clinical and Translational
Science Award Program (received by JL), and the International Collaborative
structure in the framework would have ensured uni- Award for Research from the International Pediatric Research Foundation (re-
formity and continuity of communication from the ceived by PS). RB’s time and travel for data collection and analysis were sup-
beginning. Finally, the study design and M&E frame- ported by the Laney Graduate School of Emory University, a NIH T32
Institutional National Research Service Award training grant in Reproductive,
work were not revised following an unexpected and Pediatric and Perinatal Epidemiology (HD052460–01), and the Burroughs
rapid policy change in the Bolivian health system that Wellcome Fund’s Molecules to Mankind Program (M2 M). PR’s time and
influenced enrollment and retention [50, 51]. Allow- travel for data collection and analysis were supported by the Thrasher Re-
search Fund 2015 Early Career Award and the NIH Vaccinology Training Pro-
ing the study design and M&E framework to adapt to gram award T32AI074492. KS’s time for analysis and interpretation of data
external features such as changes in the health system and writing the manuscript was supported by the Laney Graduate School of
would have improved implementation. Emory University. AA’s time for analysis and interpretation of data and writ-
ing the manuscript was supported by RTI International.

Conclusions Availability of data and materials

The data and participant safety M&E framework de- The datasets generated and/or analyzed during the current study are not
scribed here and lessons learned from this study can publicly available due to ongoing analysis and publication but will be
available from the corresponding author on reasonable request once data
be applied to other observational studies to improve analysis and publication is complete.
the quality and comparability of data. The data and
current [46, 52] and forthcoming publications will ad-
Authors’ contributions
vance the base of knowledge on oral vaccine under- AA contributed to the design and execution of the study, and drafted the
performance and infant undernutrition in developing methods and discussion sections. KS contributed to the execution of the
countries. study and drafted the background and results sections. RB contributed to
the design and execution of the study, and cleaned and analyzed the data.
JL, PS, and PR designed and conceptualized the study, oversaw the research,
Additional files. and contributed to the manuscript. RR and VI contributed to the design and
conceptualization of the study, and provided critical input and oversight of
field work. All authors critically reviewed drafts and approved the final
Additional file 1: NIDI_Study_Questionnaire_Visit_1.pdf. NIDI Study manuscript.
Questionnaire - visit one. NIDI study questionnaire for visit one – baseline
data collection. Includes informed consent. Note: the questionnaire was
developed in Spanish and is presented here in Spanish (SAS 7 kb) Ethics approval and consent to participate
Additional file 2: NIDI_GrowthCurveMacro.sas. SAS Program Code for The study protocol, consent forms, documents, annual reports, and
Infant Growth Chart. SAS Macro to create graphs that show individual amendments were reviewed and approved by the Bolivian National
child growth over time compared to a WHO reference (PDF 882 kb) Bioethics Committee, Research Ethics Commission (Comisión de Ética de la
Investigación - Comité Nacional de Bioética) and by Emory University’s
Institutional Review Board (IRB00056127). The study convened a Bolivian
Abbreviations scientific advisory board, composed of three Bolivian pediatricians, to
AE: Adverse events; AGP: Alpha(1)-acid-glycoprotein; CRP: C-reactive protein; provide independent oversight and advocate for study participants. The
EAEC: Enteroaggressive E. coli; ELISA: Enzyme-linked immunosorbent assay; scientific advisory board met annually 2012–2015 and reviewed all study
EPEC: Enteropathegenic E. coli; ETEC: Enterotoxigenic E. coli; Hb: Hemoglobin; data, protocol changes, and adverse event and severe adverse event reports.
IgA: Immunoglobulin A; IgG: Immunoglobulin G; LAZ: Length-for-age z-score; Mothers provided written consent in Spanish or Aymara after explaining the
LTFU: Loss to follow-up; M&E: Monitoring and evaluation; M-I: Mother-infant; purpose of the study in their own words. The 42 mothers who were minors
NIDI: Nutrition, Immunology, and Diarrheal Illness or Nutrición, Inmunología also needed their parent’s consent.
y Diarrea Infantil; RBP: Retinol binding protein; RCF: Relative centrifugal force;
REDCap: Research electronic data capture; RV: Rotavirus; SAE: Severe adverse
events; sTfR: Soluble transferrin receptor; UMSA-IBMB: Universidad Mayor de Consent for publication
San Andrés Instituto de Biología Molecular y Biotecnología; WLZ: Weight-for- Not applicable.
length z-score

Acknowledgements Competing interests

First, we thank our study participants and their families. We also thank our The authors declare they have no competing interests.
study personnel, colleagues at the Universidad Mayor de San Andrés and
Centro de Atención Integral para Adolecentes, and participating hospitals
“Infantil Los Andes” and “Modelo Corea” in La Paz and El Alto, Bolivia. We are Publisher’s Note
also grateful to Drs. Donnie Whitehead and Juergen Erhardt for their Springer Nature remains neutral with regard to jurisdictional claims in
assistance with the biological samples, and to Ms. Janet Figueroa for her published maps and institutional affiliations.
assistance with M&E activities. Thank you to Dr. Emily R. Smith for the initial
consultation of an M&E framework in the study design of a low-resource set- Author details
ting. We are grateful to the Emory University Global Health Institute and 1
RTI International, 3040 E. Cornwallis Road, PO Box 12194, Research Triangle
Emory University Rollins School of Global Public Health Global Field Experi- Park, NC 27709, USA. 2Hubert Department of Global Health, Rollins School of
ence Fund for travel support of several Emory graduate students who partici- Public Health, Emory University, Atlanta, GA, USA. 3Department of
pated in this research. Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA,
USA. 4Emory School of Medicine, Atlanta, GA, USA. 5Servicio Departamental
Funding de Salud, La Paz, Bolivia. 6Instituto de Biotecnología y Microbiología,
Study design, data collection, and data analysis were supported by NIH- Universidad Mayor de San Andrés, La Paz, Bolivia. 7Nutrition Branch, Centers
NIAID grants 1K01AI087724–01 and U19-AI057266 (received by JL), the Emory for Disease Control and Prevention, Atlanta, GA, USA.
Aceituno et al. BMC Public Health (2017) 17:911 Page 11 of 12

Received: 23 March 2017 Accepted: 12 November 2017 21. Caulfield LE, Richard SA, Rivera JA, Musgrove P, Black RE. Stunting, Wasting,
and Micronutrient Deficiency Disorders. In: Disease Control Priorities in
Developing Countries. edn. Edited by Jamison DT, Breman JG, Measham AR,
Alleyne G, Claeson M, Evans DB, Jha P, Mills A, Musgrove P. Washington
References (DC): World BankThe International Bank for Reconstruction and
1. Tindana PO, Singh JA, Tracy CS, Upshur REG, Daar AS, Singer PA, Frohlich J, Development/The World Bank Group; 2006.
Lavery JV. Grand Challenges in Global Health: Community Engagement in 22. Fischer Walker CL, Ezzati M, Black RE. Global and regional child mortality
Research in Developing Countries. PLoS Med. 2007;4(9):e273. and burden of disease attributable to zinc deficiency. Eur J Clin Nutr. 2009;
2. Sitthi-Amorn C, Somrongthong R. Strengthening health research capacity in 63(5):591–7.
developing countries: a critical element for achieving health equity. BMJ. 23. Guerrant RL, Oria RB, Moore SR, Oria MO, Lima AA. Malnutrition as an
2000;321(7264):813–7. enteric infectious disease with long-term effects on child development. Nutr
3. Matthias DM, Robertson J, Garrison MM, Newland S, Nelson C. Freezing Rev. 2008;66(9):487–505.
temperatures in the vaccine cold chain: a systematic literature review.
24. Ngure F, Reid B, Humphrey J, Mbuya M, Pelto G, Stoltzfus R. Water,
Vaccine. 2007;25(20):3980–6.
sanitation, and hygiene (WASH), environmental enteropathy, nutrition,
4. White F. Capacity-building for health research in developing countries: a
and early child development: making the links. Ann N Y Acad Sci. 2014;
manager's approach. Rev Panam Salud Publica. 2002;12:165–72.
1308:118–28.
5. Murray-Kolb LE, Rasmussen ZA, Scharf RJ, Rasheed MA, Svensen E, Seidman
25. Rytter MJ, Kolte L, Briend A, Friis H, Christensen VB. The immune
JC, Tofail F, Koshy B, Shrestha R, Maphula A, et al. The MAL-ED cohort study:
system in children with malnutrition–a systematic review. PLoS ONE.
methods and lessons learned when assessing early child development and
2014;9(8):e105017.
caregiving mediators in infants and young children in 8 low- and middle-
26. WHO Growth Charts [[Link]
income countries. Clin Infect Dis. 2014;59(Suppl 4):S261–72.
htm#The WHO Growth Charts]
6. Suchdev PS, Ruth L, Obure A, Were V, Ochieng C, Ogange L, Owuor M,
27. Coa R, Ochoa L, MEASURE DHS, Macro International Inc.: Bolivia: Encuesta
Ngure F, Quick R, Juliao P, et al. Monitoring the marketing, distribution, and
nacional de demografia y salud 2008. In: DHS Final Reports. Edited by DHS.
use of Sprinkles micronutrient powders in rural western Kenya. Food Nutr
Calverton, MD; 2008.
Bull. 2010;31(2 Suppl):S168–78.
28. Vesikari T, Itzler R, Matson DO, Santosham M, Christie CD, Coia M, Cook JR,
7. Kirkpatrick BD, Colgate ER, Mychaleckyj JC, Haque R, Dickson DM, Carmolli
Koch G, Heaton P. Efficacy of a pentavalent rotavirus vaccine in reducing
MP, Nayak U, Taniuchi M, Naylor C, Qadri F, et al. The "Performance of
rotavirus-associated health care utilization across three regions (11
Rotavirus and Oral Polio Vaccines in Developing Countries" (PROVIDE) study:
countries). Int J Infect Dis. 2007;11(Suppl 2):S29–35.
description of methods of an interventional study designed to explore
29. Jensen G. The logical framework approach. Bond for International
complex biologic problems. Am J Trop Med Hyg. 2015;92(4):744–51.
Development. 2010;4
8. UNICEF, World Health Organization: Framework for operations and
implementation research in health and disease control programs. 2008. 30. Sardan MG, Ochoa LH, Guerra WC. Bolivia - Encuesta Nacional de Demografia y
9. Crawford P, Bryce P. Project monitoring and evaluation: a method for Salud 2003. In: MEASURE DHS. Edited by DHS Final Reports. Instituto Nacional
enhancing the efficiency and effectiveness of aid project implementation. de Estadistica, Ministerio de Salud y Deportes: Calverton, MD; 2004.
International Journal of Project Management. 2003;21(5):363–73. 31. Mei Z, Grummer-Strawn LM. Standard deviation of anthropometric Z-scores
10. Kotloff KL, Nataro JP, Blackwelder WC, Nasrin D, Farag TH, Panchalingam S, as a data quality assessment tool using the 2006 WHO growth standards: a
Wu Y, Sow SO, Sur D, Breiman RF, et al. Burden and aetiology of diarrhoeal cross country analysis. Bull World Health Organ. 2007;85(6):441–8.
disease in infants and young children in developing countries (the Global 32. Wit JM, Himes JH, van Buuren S, Denno DM. Suchdev PS: Practical
Enteric Multicenter Study, GEMS): a prospective, case-control study. Lancet. Application of Linear Growth Measurements in Clinical Research in Low-and
2013;382(9888):209–22. Middle-Income Countries. Hormone Research in Paediatrics. 2017;
11. Rotavirus [[Link] 33. Nutrition Survey Toolkit [[Link]
12. Zaman K, Anh DD, Victor JC, Shin S, Yunus M, Dallas MJ, Podder G, Thiem [Link]]
VD, Luby SP, Coia ML. Efficacy of pentavalent rotavirus vaccine against 34. Onis M. WHO Child Growth Standards based on length/height, weight and
severe rotavirus gastroenteritis in infants in developing countries in Asia: a age. Acta Paediatr. 2006;95(S450):76–85.
randomised, double-blind, placebo-controlled trial. The Lancet. 2010; 35. Research Ethics Training Curriculum [[Link]
376(9741):615–23. libraries/webpages/fhi-retc2/]
13. Madhi SA, Cunliffe NA, Steele D, Witte D, Kirsten M, Louw C, Ngwira B, 36. Guidelines for Collecting and Testing Blood Samples for Micronutrients
Victor JC, Gillard PH, Cheuvart BB, et al. Effect of Human Rotavirus Vaccine [[Link]
on Severe Diarrhea in African Infants. N Engl J Med. 2010;362(4):289–98. 37. Cohen JH, Haas JD. Hemoglobin correction factors for estimating the
14. Armah GE, Sow SO, Breiman RF, Dallas MJ, Tapia MD, Feikin DR, Binka FN, prevalence of iron deficiency anemia in pregnant women residing at high
Steele AD, Laserson KF, Ansah NA. Efficacy of pentavalent rotavirus vaccine altitudes in Bolivia. Rev Panam Salud Publica. 1999;6(6):392–9.
against severe rotavirus gastroenteritis in infants in developing countries in 38. Ministerio de Salud y Deportes de Bolivia: Programa de "Desnutrición Cero"
sub-Saharan Africa: a randomised, double-blind, placebo-controlled trial. The Niñas y niños bien nutridos, un compromiso de todos. In., vol. 14, Primero
Lancet. 2010;376(9741):606–14. edición edn. La Paz, Bolivia: Serie Documentos Técnicos Normativos; 2007.
15. Bar-Zeev N, Jere KC, Bennett A, Pollock L, Tate JE, Nakagomi O, Iturriza- 39. Gonzales L, Joffre E, Rivera R, Sjoling A, Svennerholm AM, Iniguez V.
Gomara M, Costello A, Mwansambo C, Parashar UD, et al. Population Impact Prevalence, seasonality and severity of disease caused by pathogenic
and Effectiveness of Monovalent Rotavirus Vaccination in Urban Malawian Escherichia coli in children with diarrhoea in Bolivia. J Med Microbiol. 2013;
Children 3 Years After Vaccine Introduction: Ecological and Case-Control 40. World Health Organization: Haemoglobin concentrations for the diagnosis
Analyses. Clin Infect Dis. 2016;62(Suppl 2):S213–9. of anaemia and assessment of severity. In: Vitamin and Mineral Nutrition
16. Patel M, Shane AL, Parashar UD, Jiang B, Gentsch JR, Glass RI: Oral rotavirus Information System. Geneva: World Health Organization (WHO/NMH/NHD/
vaccines: how well will they work where they are needed most? J Infect Dis MNM/11.1); 2011.
2009, 200(Supplement 1):S39-S48. 41. Moon SS, Wang Y, Shane AL, Nguyen T, Ray P, Dennehy P, Baek LJ, Parashar
17. Savy M, Edmond K, Fine PE, Hall A, Hennig BJ, Moore SE, Mulholland K, U, Glass RI, Jiang B. Inhibitory effect of breast milk on infectivity of live oral
Schaible U, Prentice AM. Landscape analysis of interactions between rotavirus vaccines. Pediatr Infect Dis J. 2010;29(10):919–23.
nutrition and vaccine responses in children. J Nutr. 2009;139(11):2154S–218S. 42. Erhardt JG, Estes JE, Pfeiffer CM, Biesalski HK, Craft NE. Combined measurement
18. Lopman BA, Pitzer VE, Sarkar R, Gladstone B, Patel M, Glasser J, Gambhir M, of ferritin, soluble transferrin receptor, retinol binding protein, and C-reactive
Atchison C, Grenfell BT, Edmunds WJ. Understanding Reduced Rotavirus protein by an inexpensive, sensitive, and simple sandwich enzyme-linked
Vaccine Efficacy in Low Socio-Economic Settings. PLoS ONE. 2012;7(8):e41720. immunosorbent assay technique. J Nutr. 2004;134(11):3127–32.
19. Bailey RL, West KP Jr, Black RE. The Epidemiology of Global Micronutrient 43. Engle-Stone R, Ndjebayi AO, Nankap M, Killilea DW, Brown KH. Stunting
Deficiencies. Ann Nutr Metab. 2015;66(Suppl. 2):22–33. prevalence, plasma zinc concentrations, and dietary zinc intakes in a nationally
20. Burke RM, Leon JS, Suchdev PS. Identification, prevention and treatment of representative sample suggest a high risk of zinc deficiency among women
iron deficiency during the first 1000 days. Nutrients. 2014;6(10):4093–114. and young children in Cameroon. J Nutr. 2014;144(3):382–91.
Aceituno et al. BMC Public Health (2017) 17:911 Page 12 of 12

44. Engle-Stone R, Haskell MJ, Ndjebayi AO, Nankap M, Erhardt JG, Gimou M-M,
Brown KH. Plasma retinol-binding protein predicts plasma retinol
concentration in both infected and uninfected Cameroonian women and
children. The Journal of nutrition. 2011;141(12):2233–41.
45. Suchdev PS, Namaste SM, Aaron GJ, Raiten DJ, Brown KH, Flores-Ayala R.
BRINDA Working Group: Overview of the Biomarkers Reflecting
Inflammation and Nutritional Determinants of Anemia (BRINDA) Project.
Advances in Nutrition: An International Review Journal. 2016;7(2):349–56.
46. Burke RM, Suchdev PS, Rebolledo PA, de Aceituno AM, Revollo R, Iniguez V,
Klein M, Drews-Botsch C, Leon JS. Predictors of Inflammation in a Cohort of
Bolivian Infants and Toddlers. Am J Trop Med Hyg. 2016;95(4):954–63.
47. Namaste SM, Aaron GJ, Varadhan R, Peerson JM, Suchdev PS: Methodologic
approach for the Biomarkers Reflecting Inflammation and Nutritional Determinants
of Anemia (BRINDA) project. Am J Clin Nutr 2017, 106(Suppl 1):333s-347s.
48. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research
electronic data capture (REDCap)—a metadata-driven methodology and
workflow process for providing translational research informatics support. J
Biomed Inform. 2009;42(2):377–81.
49. World Health Organization: Physical status: The use of and interpretation of
anthropometry, Report of a WHO Expert Committee. In. Geneva,
Switzerland: World Health Organization; 1995.
50. Sistema de salud rechaza Ley 475. In: El Diario. Bolivia; 2014.
51. Ley de Prestaciones de Servicios de Salud Integral del Estado Plurinacional
de Bolivia. In: 475. Edited by Plurinacional lAL. Gaceta Oficial del Estado
Plurinacional de Bolivia: Bolivia; 2013.
52. Burke RM, Rebolledo PA, Fabiszewski de Aceituno AM, Revollo R, Iniguez V,
Klein M, Drews-Botsch C, Leon JS, Suchdev PS. Early deterioration of iron
status among a cohort of Bolivian infants. Matern Child Nutr. 2016;
53. World Health Organization: Global Database on Child Growth and
Malnutrition. In. Geneva, Switzerland: World Health Organization; 2017.

Submit your next manuscript to BioMed Central

and we will help you at every step:
• We accept pre-submission inquiries
• Our selector tool helps you to find the most relevant journal
• We provide round the clock customer support
• Convenient online submission
• Thorough peer review
• Inclusion in PubMed and all major indexing services
• Maximum visibility for your research

Submit your manuscript at

[Link]/submit