JOURNAL OF MEDICAL INTERNET RESEARCH Stellefson et al

Review

eHealth Literacy Among College Students: A Systematic Review

With Implications for eHealth Education

Michael Stellefson1, PhD; Bruce Hanik2, PhD; Beth Chaney1, PhD; Don Chaney1, PhD; Bethany Tennant1, MS;
Enmanuel Antonio Chavarria1, MS
1
Department of Health Education and Behavior, University of Florida, Gainesville, FL, United States
2
Department of Health and Kinesiology, Texas A&M University, College Station, TX, United States

Corresponding Author:
Michael Stellefson, PhD
Department of Health Education and Behavior
University of Florida
PO Box 118210
Gainesville, FL, 32611
United States
Phone: 1 352 294 1805
Fax: 1 352 392 1909
Email: mstellefson@[Link]

Abstract
Background: eHealth literacy refers to the ability of individuals to seek, find, understand, and appraise health information from
electronic resources and apply such knowledge to addressing or solving a health problem. While the current generation of college
students has access to a multitude of health information on the Internet, access alone does not ensure that students are skilled at
conducting Internet searches for health information. Ensuring that college students have the knowledge and skills necessary to
conduct advanced eHealth searches is an important responsibility particularly for the medical education community. It is unclear
if college students, especially those in the medical and health professions, need customized eHealth literacy training for finding,
interpreting, and evaluating health- and medical-related information available on the Internet.
Objective: The objective of our review was to summarize and critically evaluate the evidence from existing research on eHealth
literacy levels among college students between the ages of 17 and 26 years attending various 4-year colleges and universities
located around the world.
Methods: We conducted a systematic literature review on numerous scholarly databases using various combinations of relevant
search terms and Boolean operators. The records were screened and assessed for inclusion in the review based on preestablished
criteria. Findings from each study that met inclusion criteria were synthesized and summarized into emergent themes.
Results: In the final review we analyzed 6 peer-reviewed articles and 1 doctoral dissertation that satisfied the inclusion criteria.
The number of participants in each reviewed study varied widely (from 34 to 5030). The representativeness of the results from
smaller studies is questionable. All studies measured knowledge and/or behaviors related to college student ability to locate, use,
and evaluate eHealth information. These studies indicated that many college students lack eHealth literacy skills, suggesting that
there is significant room for improvement in college students’ ability to obtain and evaluate eHealth information.
Conclusion: Although college students are highly connected to, and feel comfortable with, using the Internet to find health
information, their eHealth literacy skills are generally sub par. College students, especially in the health and medical professions,
would be well served to receive more customized college-level instruction that improves general eHealth literacy.

(J Med Internet Res 2011;13(4):e102) doi: 10.2196/jmir.1703

KEYWORDS
eHealth literacy; college students; health occupations; professional preparation

[Link] J Med Internet Res 2011 | vol. 13 | iss. 4 | e102 | p. 1

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Stellefson et al

exists as an important skill set for health professionals tasked

Introduction with seeking valid and reliable health information in a
Electronic resources increasingly play a major role in consumer Web-based environment. However, most studies on literacy
health, with the Internet being the preferred primary and health, such as the US Institute of Medicine’s report titled
telecommunications vehicle for seekers of novel and germane Health Literacy: A Prescription to End Confusion, exclusively
health information. Although now widely relevant, the term examine the relationship between health outcomes and literacy
electronic health information, also called eHealth, first appeared in the context of paper-based resources, not literacy in electronic
in 2000 to describe where health informatics, public health, environments [11,13]. Therefore, eHealth literacy is still a novel
health services, and information transmission processes concept with varied definitions and models.
intersected, primarily through Web-based applications [1,2]. eHealth literacy refers to the ability of individuals to seek, find,
Health information is one of the most investigated topics online understand, and appraise health information from electronic
[3]: 8 out of 10 Internet users report that they have at least once resources and apply such knowledge to addressing or solving
looked online for health information, making it the third most a health problem [13]. eHealth literacy combines six core skills
popular Web activity next to checking email and using search or types of literacy: traditional literacy, health literacy,
engines in terms of activities that almost everybody has done information literacy, scientific literacy, media literacy, and
[4]. The importance of the Internet to acquire health information computer literacy [13,14]. Table 1 [12,13,15,16] provides
has spurred the creation of numerous eHealth information definitions of each type of literacy considered within the scope
resources that assist consumers in discovering knowledge that of eHealth literacy. These six facets have been developed by
can help promote and sustain personal health. Subsequent studies Norman and Skinner and have been depicted as the eHealth
examining the effectiveness of eHealth interventions have Literacy Lily Model, characterizing the six types of literacy as
proposed many definitions for eHealth [1,2,5]. Broadly stated, forming overlapping lily petals that feed into the overall eHealth
eHealth can also be thought of as the field where information literacy “pistil” (ie, center of the model). More specifically, the
and communication technology design enables the delivery of lily model categorizes the six core literacies into two primary
health-related and medical information [6]. While eHealth can types: analytic (ie, traditional, media, and information) and
potentially revolutionize medical and public health practice [7], context- specific (ie, health, scientific, and computer). Analytic
numerous human resource, organizational, and cultural changes literacies refer to a set of skills that can be applied to an array
are still necessary to enable mainstream adoption of eHealth of information sources, whereas context-specific literacies
strategies for retrieving good-quality health information [1,8,9]. involve skills that are specific to a certain problem or situation.
eHealth and the topic of health literacy are closely connected eHealth literacy, as the composite of both analytic and
in public health. Health literacy is defined as “the degree to context-specific skills, requires the behavioral capability to do
which individuals have the capacity to obtain, process, and the following: work with technology, critically think about
understand basic health information and services needed to issues of media and science, and navigate through the vast array
make appropriate health decisions” [10]. It has been identified of eHealth decision-making resources. A variety of
as a public health goal for the 21st century and stands as a competencies are associated with obtaining eHealth information,
significant challenge facing health care globally [11,12]. including the knowledge, skills, abilities, and other attributes
According to Norman and Skinner, the articulation of health necessary to (1) conduct basic and advanced information
literacy “underscores the importance of contextual factors that searches, (2) apply Boolean operators to limit searches, (3)
mediate health information and the need to consider health differentiate between scholarly documents, authoritative sources,
literacy in relation to the medium by which health resources are periodicals, and primary sources of information, and (4)
presented” [13]. The pervasiveness of the Internet has made understand sometimes ambiguous eHealth terminology. Specific
obtaining, processing, and understanding health information techniques using these proficiencies are necessary to find
using Web-based technologies a critical competency area for documents on the Web such as abstracts, bibliographies,
medical professionals. With the emergence of electronic medical research articles, and government reports. To ensure that
and health records, medical mobile apps, and other related health individuals are optimally making use of available eHealth
informatics technologies, medical professionals are increasingly access, it is important that appropriate search-related practices
responsible for finding and evaluating health information and procedures be used to retrieve and assess the eHealth
resources electronically. In light of this, eHealth literacy now information that is located.

[Link] J Med Internet Res 2011 | vol. 13 | iss. 4 | e102 | p. 2

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Stellefson et al

Table 1. Six components of eHealth literacy[13]

Type of literacy Definition
Traditional literacy Involves basic literacy skills, such as reading text, understanding written passages, and coherently speaking and writing
a language [15].
Information literacy According to the American Library Association, involves a person knowing “how knowledge is organized, how to find
information, and how to use information in such a way that others can learn from them” [16].
Media literacy Involves the ability to critically think about media content, and “enables people to place information in a social and
political context and to consider issues such as the marketplace, audience relations, and how media forms in themselves
shape the message that gets conveyed” [13].
Health literacy Defined by the American Medical Association as a person’s capability to “perform basic reading and numerical tasks
required to function in the health care environment. Patients with adequate health literacy can read, understand, and act
on health care information” [12].
Computer literacy Involves the ability to use computers to solve problems. According to Norman and Skinner, “computer literacy includes
the ability to adapt to new technologies and software and includes both absolute and relative access to eHealth resources”
[13].
Scientific literacy Involves an “understanding of the nature, aims, methods, applications, limitations, and politics of creating knowledge
in a systematic manner” [13]. Allows health research findings to be placed in the appropriate context and requires the
understanding of the discovery process.

Access to eHealth information is ubiquitous now for many who Literacy Competency Standards for Higher Education,
have broadband Internet; however, access to eHealth resources developed by the Association of College and Research Libraries
does not inevitably assure acuity in discerning good-quality [18,19], to create the Research Readiness Self-Assessment
health information from quackery on the Internet. The ability (RRSA). The RRSA measures basic information literacy skills
to diagnose and engage useful eHealth information from related to research ability. Specific to measuring information
reputable medical sources, such as governmental agencies (eg, literacy, the competency model assesses knowledge and skill
National Institutes of Health, Centers for Disease Control and sets necessary to locate good-quality information on a specific
Prevention, Health Canada) and medical establishments (eg, health topic. These competencies verify abilities to determine
Mayo Clinic, WebMD, Canadian Medical Association) as possible sources of good-quality health information, conduct
compared with opinion or advertisements from so-called experts health information searches, evaluate the quality of the
such as private sector marketers and nonverified public information, and appropriately use the information. Declarative
commentators, is becoming increasingly important. With the knowledge, such as knowledge of plagiarism, health-related
wealth of health information that exists on the Internet, this information sources, and research terminology, consists of
complex task requires far more interpretive and demonstrative typical knowledge variables measured in this competency-based
skill than simply being able to enter a medical condition or term approach. In addition, procedural knowledge, which involves
into an Internet search engine such as Google or Bing. For skills and problem solving, includes knowledge of the
example, when using the Internet as a medical education procedures used to complete an information-seeking task
resource, consumers should know how to critically examine electronically (ie, database navigation). Both types of knowledge
and discriminate between primary and secondary sources of are important for assessing the behavior and eHealth literacy
health information posted on a website [13]. of health information consumers [19].
Implementing effective Internet searches to locate health In summary, there is a growing interest in eHealth literacy as
information is especially important for college students, as the an essential skill for students, especially those in the medical
Internet is now a favorite resource for information gathering and health professions, and it is unclear whether the current
among the “Millennial” generation. For the Millennial level of eHealth literacy is sufficient, or whether customized
generation of college students, the Internet is a preferred source eHealth literacy training for finding, interpreting, and evaluating
of health information [17]. While it may be safe to assume that health- and medical-related information available on the Internet
college students have ample access to Web-based portals leading at the college level would be required to nurture these skills.
to eHealth information, it is important to be cognizant that access The purpose of this systematic review is to evaluate the current
alone does not ensure that college students are adroit at literature to determine whether college students can generally
searching for, locating, and evaluating health information. be considered an “eHealth literate” population.
Ensuring that college students have the knowledge and skills
necessary to conduct advanced eHealth searches is an important Methods
responsibility particularly for the medical education community.
Search Procedures
To determine eHealth literacy among college students, it is first
important to define the specific knowledge, skills, abilities, and This review adopted the widely accepted definition of eHealth
other user attributes that have been considered in previous literacy as the ability of individuals to seek, find, understand,
eHealth literacy research. Some of these attributes have been and appraise health information from electronic sources and
investigated by Ivanitskaya and Casey, who used the Information apply such information to addressing or solving a health problem
[13]. For this review, the experimental units of analysis for

[Link] J Med Internet Res 2011 | vol. 13 | iss. 4 | e102 | p. 3

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Stellefson et al

inclusion were peer-reviewed articles evaluating eHealth literacy abstracts. We initially excluded studies that did not survey
(ie, seeking, finding, understanding, and appraising health 4-year college student populations between the ages of 17 and
information among electronic sources, primarily the Internet) 26 years, and eliminated those that did not measure knowledge,
exclusively among college students. The scope of the review skills, abilities or other attributes associated with eHealth
was male and female college students between the ages of 17 literacy. In addition to the 37 papers that remained after the
and 26 years attending various 4-year colleges and universities initial exclusion, we identified 5 other articles by hand searches
located around the world. To generate a sample of empirical after scanning the reference section of each database-identified
studies, we conducted an exhaustive search of electronic article to enhance the breadth of the examination. This hand
databases. Due to the relatively recent emergence of eHealth in search resulted in the addition of 5 other articles meeting criteria
the 21st century, only articles published from 2000 to the present for a full-text assessment. Overall, 42 papers were included in
day were eligible for inclusion. The actual search of all relevant this full-text assessment, of which 35 were excluded for a variety
literature took place during the spring of 2011. The searched of reasons, including (1) being secondary sources of information
databases were ERIC, PsycINFO, HealthSource, Medline, (n = 5), or purely conceptual or theoretical in scope (n = 3), (2)
MasterFILE Premier, Academic Search Complete, CINAHL acting as opinion or editorial pieces (n = 2), (3) including
Plus with Full Text, Health Source: Nursing/Academic Edition, populations other than college students (n = 14), (4) not
Psychology & Behavioral Sciences Collection, Applied Social explicitly measuring and reporting students’ ability to seek,
Sciences Index and Abstracts, and CSA. The following key find, or evaluate electronic sources of health information (n =
terms were entered in various combinations with multiple 8), or (5) reporting studies that assigned participants to an
Boolean operators: eHealth, electronic health, eHealth literacy, “Internet” treatment group within an intervention or trial (n =
electronic health literacy, health literacy, Internet literacy, 3). After accounting for conditions outlined by the above
Internet health, electronic literacy, college students, university exclusion criteria, we were left with 28 articles out of the review,
students, and literature review. leaving 7 articles that were empirical studies assessing eHealth
literacy among college students. Figure 1 presents a flow
All articles gathered through this initial search and screen
diagram of the systematic literature review search process
process (n = 135) were evaluated for inclusion in the sample
described above. Of the final 7 articles, 6 studies were carried
pool. We excluded 98 records after the screen of titles and
out in the United States and 1 in Finland.

[Link] J Med Internet Res 2011 | vol. 13 | iss. 4 | e102 | p. 4

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Stellefson et al

Figure 1. Flow diagram of the article selection process.

articles), Journal of American College Health (2), BMC Medical

Methodological Data Analysis Informatics and Decision Making, and the Californian Journal
To evaluate the methodological quality of each retained study, of Health Promotion. A total of 4 articles [19,21-23] explicitly
we used a modified version of criteria established by Nagel defined eHealth literacy, but all explored at least one aspect of
Bernstein and Freeman [20] to develop a methodological data eHealth literacy accounted for in the Norman and Skinner [13]
score (MDS) for each article ranging from 0 (low) to 4 (high). definition used within this review. For example, Nsuangani and
If a study used multivariate procedures such as discriminant Perez [21] asked specific questions about Internet use tendencies
analysis, factor analysis, cluster analysis, hierarchical regression, to find health information, while the RRSA, administered in 3
or multivariate analysis of variance, then it received a score of studies [19,25,26], sought to evaluate all dimensions of eHealth
4. Articles reporting descriptive statistics, univariate regressions, literacy.
or nonparametric tests such as chi-square were assigned a 3.
Those reporting strictly qualitative data received a score of 2, The studies included in this systematic review were generally
and purely narrative descriptions or written observations exploratory in nature. Many used demographic variables to
received a score of 1. When studies did not report any statistical group students in a nonexperimental fashion and then explored
analysis procedures, then no points were awarded. differences in relationships. The independent variables used in
all studies were unique and directly related to the study purpose;
Results however, the most common independent variables were users
and nonusers of the Internet, gender, student classification, and
Study Characteristics race. The dependent variables included self-reported use of
Web-based health advice services [22]; perceptions of the
Although eHealth has been a topic of interest since the turn of
accuracy of health information on the Internet [21]; perceptions
the century, this systematic search identified only 6
of the privacy of health information on the Internet [21];
peer-reviewed articles [19,21-25] and 1 doctoral dissertation
frequency of Internet use for seeking health information
[25], published between 2005 and 2010. Four different journals
[20,22,23]; quality of health-related websites [19,25,26];
published the articles: Journal of Medical Internet Research (2
[Link] J Med Internet Res 2011 | vol. 13 | iss. 4 | e102 | p. 5
(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Stellefson et al

attitudes and beliefs about using the Internet for finding health variables, and 1 study [24] used a mixed-methods approach.
information (eg, beliefs that open-access Internet and search Only 2 studies [21,26] used validated surveys containing
engines are always the best sources of information) [25]; ability reliability estimates for the data collected, while 3 studies
to find electronic health information [19,25,26]; ability to [22,25,26] did not explicitly report this information. A total of
evaluate electronic health information [19,25,26]; perceived 3 studies [21-23] used chi-square as the analysis of choice to
research skills (ie, self-reported subjective beliefs about one’s explore differences in patterns between groups, and the
own skills) [19,25,26]; ability to judge the trustworthiness of remaining studies [19,24-26] simply reported descriptive
Internet pharmacies [25]; and number of correct answers to statistics. Redmond [26] used multiple t tests to determine
sexual health questions following the conduct of Internet differences in eHealth literacy skills between rural and nonrural
searches [24]. college students. Ivanitskaya and Brookins-Fisher [25]
performed multiple independent t tests to assess whether
Results from the methodological assessment described above
differences in critical judgment existed among students who
indicated that the reviewed articles had similar degrees of
either did or did not use the Internet for health decision making.
analytic rigor. That is, 2 studies randomly assigned participants
The mean MDS for the reviewed studies was 3.14 (SD 0.38),
to treatment groups [21,22], while the others used convenience
with 6 of the 7 studies (86%) scoring 3. Table 2 describes the
[19,23,25,26] and purposive [24] sampling techniques; 6 of the
basic design, measurement, and analysis of each study
studies [19,21-23,25,26] used a quantitative paradigm to
accompanied by each study’s individual MDS.
determine patterns between independent and dependent

Table 2. Design, measurement, analysis, and methodological data score (MDS) of selected studies
First author Sample Design Instrument Instrument Instrument reliability Analysis MDS
(year) size validity
Ivanitskaya 308 Nonexperi- RRSAa Face, con- Yes, but no value reported Descriptive statistics, multiple 3
[19] (2006) mental tent regression
Nsuangani 136 Experimen- Ad hoc sur- Face (ex- Pre–post, κ = 0.41 for items re- Frequency distributions; cross- 3
[21] (2006) tal vey pert panel) tained for analysis tabulations; chi-square
Castren 5030 Experimen- Student Not report- Not reported Frequency distributions; cross- 3
[22] (2008) tal Health Sur- ed tabulations; chi-square
vey 2004
(Finland)
Escoffery 743 Nonexperi- Ad hoc sur- Not report- Not reported Descriptive statistics; chi-square 3
[23] (2005) mental vey ed
Buhi [24] 34 Nonexperi- Ad hoc Content Not reported Descriptive statistics 3
(2009) mental (implied)
Ivanitskaya 1914 Nonexperi- RRSA Face, con- Not reported Descriptive statistics; probabili- 4
[25] (2010) mental tent ties; t tests; hierarchical regres-
sion analysis
Redmond 243 Nonexperi- RRSA Face, con- Ability to obtain health informa- Descriptive statistics; t tests; Co- 3
[26] (2007) mental tent tion, alpha = .69; ability to evalu- hen d
ate electronic health information,
alpha = .65; overall health infor-
mation competency, alpha = .77

a
Research Readiness Self-Assessment.

conditions using the Internet [22]. Based on this evidence, it

Demographics appears that females used the Internet more for health
The number of participants in each reviewed study varied widely information and diagnostic purposes, while males were more
(ranging from 34 to 5030), which calls into question the likely to use Internet for consumer health products and services.
representativeness of the results from those studies with smaller Also, male and female college students did not differ
samples. Additionally, the research findings related to gender significantly as to whether they expressed concern regarding
varied within 5 studies [19,21-23,26]. For example, Nsuangani the accuracy of health information found on the Internet [21].
and Perez [21] found that male college students were more likely We found no statistically significant differences on any eHealth
to use the Internet to buy pharmaceutical products and locate literacy outcome when considering race or ethnicity.
consumer health information, whereas female students were
more likely to obtain general health- and medical-related Obtaining Health Information Using the Internet
information online. Interestingly, this finding was supported in We found that in 3 studies performed in the United States
2 other studies as well [22,23]. Males were more likely to seek exploring the percentage of college students using the Internet
out medical consultations using the Internet [21], while females to acquire health information, 91 of 136 (67%), 549 of 743
were more likely to self-report diagnosing chronic health (73.9%), and 24 of 34 (71%) college students surveyed had ever

[Link] J Med Internet Res 2011 | vol. 13 | iss. 4 | e102 | p. 6

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Stellefson et al

used the Internet to search for health information. In 1 study, that measure procedural knowledge; (3) demographic questions;
111 of 743 (14.9%) college students reported using the Internet and (4) a question that asks for a self-report about the level of
to locate health information in the past day or week, with less the respondent’s research skills” [19]. For example, a
than one-third reporting doing so in the past month [23]. In the knowledge-based item in the survey asks respondents to indicate
same study, 539 of 743 (72.5%) students reported being averse which Boolean operator (eg, “and,” “or,” or “not”) produces
to logging onto a health program delivered over the Internet, the most Internet search results (answer: or). An example of a
and only 204 of 743 (27.5%) students surveyed reported skill-based survey item asks respondents to determine which
willingness to participate in a health program on the Internet. Boolean operator is appropriate for a particular search situation,
Another study reported that participants were reluctant to use then requests that the respondent perform an Internet search
the Internet for interactive health purposes, with 119 of 136 using that particular Boolean operator, followed by reporting
surveyed students (87.5%) reporting an unwillingness to use back the number of Web hits generated by the search [27]. There
online medical discussion applications [21]. Another study are two subscale measurements within the RRSA: Actual Ability
conducted in Finland corroborated this reticence to participate to Obtain (AAO) eHealth information and Actual Ability to
in online health programming, finding that only 370 of 3153 Evaluate (AAE) eHealth information. The AAO subscale
(11.7%) Finnish undergraduate students had ever used a comprises 11 multiple choice items where total scores can range
Web-based health advice service offered to them through their from 0 to 16. The AAE subscale comprises 13 multiple choice
student health services department [22]. items where total scores can range from 0 to 23. A higher score
on both subscales indicates better actual ability. One study
While 1 study [23] suggested that 393 of 743 (52.9%) college
within this review [19] demonstrated that the data derived from
students surveyed would like to individually retrieve health
the RRSA possessed satisfactory internal reliability (alpha =
information on the Internet, several studies indicated college
.78). Ivanitskaya et al [19] found that 258 of 306 (84%) college
students self-reported a lack of skills necessary to execute
students surveyed perceived their eHealth literacy skills as
successful health-related searches on the Internet [19,23,25,26].
“good,” “very good,” or “excellent,” yet students’ scores on a
Escoffery et al [23] noted that 661 of 743 (89.0%) college
56-item scale evaluating their actual eHealth literacy skills were
students surveyed did not always find their desired eHealth
very poor (mean 37%, SD 6.4%).
information. Among those, 82 (11%) students surveyed did feel
that they were capable of finding health information on the Also, it was found that within each perceived skill category (eg,
Internet, whereas only slightly more than half reported success perceived ability to find health information and perceived ability
“most of the time.” Two studies [19,26] determined that many to judge the quality of health information), the actual overall
college students are rather unsophisticated health information competency scores of college students varied greatly.
seekers when using the Internet. Another study noted that Specifically, the ability of college students to evaluate their own
college students were unable to critically evaluate health competency was inconsistent with their actual eHealth literacy.
information found on the Internet [25]. Students were also Redmond [26] found that nonrural college students were better
unaware of the difference between a primary and secondary able to obtain eHealth information than were rural college
source of data when attempting to locate online journal articles students, but there were no statistically significant differences
in the health-related fields [19,26]. Finally, students who used in the ability to evaluate eHealth information between the two
eHealth information to help make health decisions had lower groups. Escoffery et al [23] found that 260 of 743 (35%) college
overall critical judgment ability than those who used students surveyed expressed “serious concern” about their ability
nonelectronic sources of information for the same purpose [25]. to find good-quality health information using the Internet, while
only a small proportion, 52 of 743 (7%), expressed “no concern”
Perceived versus Actual eHealth Literacy regarding the accuracy of health information they acquired on
Ivanitskaya et al [19] and Redmond [26] assessed (1) how the Internet. Despite the relatively higher level of apprehension
students felt about their own level of eHealth literacy, (2) how regarding ability to find eHealth information, 204 of 514
proficient students were at searching for and evaluating eHealth (39.7%) college students who reported seeking health
information, and (3) how well students understood the difference information online believed that being able to retrieve health
between peer-reviewed scholarly resources and opinion pieces information online improved the way they took care of their
or sales pitches. Both studies used the RRSA online assessment health “some” or “a lot.”
tool, which evaluated perceived and actual knowledge of student
ability in browsing the Internet and researching health topics In light of these findings, all studies tended to agree that college
given various search scenarios. The RRSA, based on the students in general [19,21,23-26], and those in health and
Information Literacy Competency Standards for Higher medical professional programs specifically [19,26], should
Education, assesses knowledge and skills related to locating, further develop their proficiency in appraising, using, and
evaluating, and using good-quality sources of eHealth evaluating health information found on the Internet. Table 3
information. Specifically, the RRSA contains the following describes the primary findings gathered from the research
items: “(1) multiple choice or true/false questions that measure questions posed in each study.
declarative knowledge; (2) interactive, problem-based exercises

[Link] J Med Internet Res 2011 | vol. 13 | iss. 4 | e102 | p. 7

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Stellefson et al

Table 3. Primary findings from research questions

First author Research question(s) Findings
Ivanitskaya [19] How proficient are university students at Students are not proficient at advanced health information searches.
searching for health-related information?
How proficient are university students at Students have mixed proficiency at evaluating health-related information.
evaluating health-related information?
How well do university students understand Students are deficient in discriminating between different types information
the difference between peer-reviewed sources.
scholarly resources, opinion pieces, and
sales pitches?
How aware are university students of their Undergraduate students are inaccurate judges of their own health information
own level of health information competen- competencies. Self-reports may not accurately predict students’ actual health in-
cies? formation competencies.
Nsuangani [21] Do male and female college students differ Males more likely than females to report online medical consultation. Males are
in their Internet behaviors related to health? more likely to buy pharmaceuticals online. More males use email to communicate
with a health care provider.
Castren [22] Does self-reporting of chronic conditions Male users of a health advice service had a higher rate of self-reported chronic
differ between users and nonusers of a Web- conditions than male nonusers; female users of a health advice service had a
based health advice service? higher rate of a reported chronic condition than female nonusers.
Escoffery [23] Are there differences in Internet use for There is no difference.
health information by level of Internet expe-
rience?
Are there differences in Internet use for Significantly more female than male students obtain health information online.
health information by gender?
Are there differences in Internet use for There is no difference.
health information by level in college?
Buhi [24] When asked questions about sexual health, For 12 of the 13 questions asked, at least 24 of 34 (71%) students answered the
do college students find accurate answers questions correctly. Of 34 students surveyed, 17 (50%) correctly answered the
online? question that asked them to locate an anonymous HIV test in the local area.
Ivanitskaya [25] To what degree are college-educated infor- How college students rate trustworthiness of online pharmacies varies substantially.
mation seekers able to determine trustwor- Only 593 of 1914 (31.0%) respondents gave low ratings to untrustworthy online
thiness of online pharmacies? pharmacies.
Do those who used information to make Respondents using online health information for decision-making have significantly
health decisions have better judgment skills? worse judgment than those not using online health information for decision-
making.
Redmond [26] Do rural and nonrural freshmen differ in A statistically significant difference exists, with nonrural students performing
their ability to obtain health information? higher than rural students, t241 = 2.23, P = .03, Cohen d = .29.

Do rural and nonrural freshmen differ in No difference exists, t241 = –.14, P = .89, Cohen d = .02.
overall health information competency?
Do rural and nonrural freshmen differ in No difference exists, t241 = 1.34, P = .18, Cohen d = .18.
their ability to evaluate health information?

own ability to successfully locate and evaluate eHealth

Discussion information. They may (or may not) hold an overly optimistic
Main Findings view of their ability to do Internet research on health-related
topics. While it is clearly too early in this field of investigation
The main conclusion of this systematic review was that college to state definitively that there is a gap between perceived and
students may lack important skills for seeking and evaluating actual eHealth literacy among college students, the trend noted
health information available on the Internet. While college in this systematic review provides impetus for future research
students, for the most part, have convenient access to health to either support or disconfirm whether this phenomenon may
information on the Internet, this systematic review indicated truly exist.
that many students possess weak eHealth literacy skills related
to searching for, retrieving, using, and evaluating sources of Regardless of whether a discord exists between perceived and
eHealth information. Furthermore, 3 studies [19,25,26] noted actual eHealth literacy among college students, there
that the subjective self-perceptions of college students regarding nevertheless is an invaluable opportunity to build medical
their ability to use eHealth information sources were incongruent education competencies among college-age students, especially
with their demonstrated eHealth literacy skills. Therefore, it is those seeking degrees in the medical and health professions.
possible that college students may be mistakenly judging their College students surveyed in the reviewed studies did not

[Link] J Med Internet Res 2011 | vol. 13 | iss. 4 | e102 | p. 8

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Stellefson et al

achieve satisfactory levels of eHealth literacy; thus, we should to search for general information on illness, disease, or disease
perhaps reexamine the standards that are being used to measure prevention using medical reference websites. Female college
eHealth literacy among this diverse audience. While valid and students, on the other hand, were generally more likely to
reliable health literacy measures have been widely established, conduct these types of general health or medical searches and
far fewer instruments are universally accepted as accurately and were less likely to obtain health services over the Internet (eg,
appropriately assessing eHealth literacy. It is possible that the accessing primary care physicians’ Web portals, communicating
current standard being promulgated might be appropriate only by email with health care providers). In light of this interesting
for technologically elite audiences. Supposing that consumers preliminary trend, future research would benefit from further
will meet eHealth literacy standards set by technicians is study regarding what particular Internet search and retrieval
probably unrealistic. High-stakes measures used in the current characteristics can be attributed to male or female college
studies may have attempted to assess skill navigating the Internet students. Unique search propensities could speak to various
to locate health information, but these measures may do so in developmental issues of marketing pressures, peer influences,
a manner less applicable to a wide-ranging audience of future and even health privacy concerns.
public health professionals. Future professionals, especially in
a health-related field. will undoubtedly be using the Internet
Obtaining Health Information Using the Internet
and related health informatics technologies to gather, manage, While the literature supports college students wanting to use
and deliver health information; however, we have yet to fully the Internet to seek out general health information, there is little
understand the context of the interactions occurring between evidence to suggest that students care to discuss their own health
diverse users and health informatics technologies (given that problems or obtain personalized medical advice over the
the consumer health informatics field is still in its infancy). Internet. College students reported reluctance to using interactive
There is a strong possibility that the broad-based, Internet applications for health communication purposes (ie,
multidimensional definition of eHealth literacy is overly electronic communication with health care providers). This
ambitious, even for individuals seeking an advanced degree in finding revealed itself not only in the United States, but also in
a health-related field. 1 Finnish study that we reviewed. Among college students, the
convenience of using the Internet for seeking personal health
While measurement issues are important to consider, it is also information may be valued more so than the prospect of
important to recognize that future inquiries should avoid receiving individualized feedback on personal health concerns
reporting purely descriptive self-report data on frequency of or problems via interaction with a qualified medical professional.
use and self-efficacy for using the Internet to find health This could be the result of contextual Web security issues
information. Data from self-reports depicting college students affecting confidentiality. The issue of trust when using the
as both frequent and confident eHealth users may be more Internet to seek and share medical information is an important
assumptive than truly substantive, especially considering the one to consider, especially with the emergence of peer-to-peer
current research. As explained by Bandura within self-efficacy or horizontal health communication among college students.
theory, “expectation alone will not produce desired performance More research should be done to discover what particular
if the [individual’s] component capabilities are lacking” [28]. sources of Web-based health information college students are
Thus, what might be more important is testing of relevant skills consulting and which cause uneasy feelings originating from
in this new area of inquiry. Thus, more research should evaluate potential threats to data security and privacy.
the most effective instructional strategies for molding
able-bodied “eHealth educators” within a variety of medical Limitations
and allied health professional preparation programs. Planned This systematic review had several limitations. Although we
instructional experiences must consider the unique eHealth conducted a comprehensive literature search on numerous
literacy competencies that are expected of college students databases using a variety of pertinent search terms, certain
studying to become health professionals, a distinction that places studies may have been overlooked due to lack of indexing in
them in a unique position as compared with the general public. searched databases. In addition, all studies were carried out in
These students are expected to gravitate toward evidence-based either the United States or Finland, which are both highly
practice and to critically appraise qualified sources of health technologically savvy countries. Also, Finland is regarded as
information using specific resources such as The Cochrane one of the world’s most literate societies, with high levels of
Collaboration [29] or the Guide to Community Preventive educational attainment [31]. Another noted limitation is that
Services [30]. More studies of college students at varied one standard definition of eHealth does not exist, which limits
institutions, majoring in a variety of health and medical the ability of researchers to find all articles examining eHealth
programs, will enable the eHealth literacy research community literacy within a single literature review. Another limitation
to ask and answer more targeted research questions with more involves the number of articles included in the review. Although
specific audiences. the studies reached similar conclusions in selected instances,
Demographics the small sample of studies (n = 7) may not truly reflect the
population’s (ie, college students) true eHealth literacy levels.
The literature also indicated a tendency for male college students In addition, most studies used convenience sampling techniques,
to be more likely to use the Internet to locate and acquire which can result in findings not being reflective of the true
consumer health products (eg, pharmaceuticals, dietary/sports populations of interest. As well, most studies in this review (n
supplements, vitamins, performance-enhancing substances) and
services (eg, Web-based medical consultations) and less likely
[Link] J Med Internet Res 2011 | vol. 13 | iss. 4 | e102 | p. 9
(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Stellefson et al

= 4) collected self-report data and did not test actual eHealth professionals. It is important for education administrators to
literacy skills to complement students’ self-perceptions. determine (1) what list of eHealth topics should be covered, (2)
what types of courses and materials can address the needed
The marketplace penetration of information technology into
competencies, (3) how many hours of subject matter instruction
college students’ lives and educational settings is shifting rapidly
might be necessary for eHealth literacy skill development, and
(eg, smartphones, social networking websites, iPads). The
(4) whether eHealth warrants a specific emphasis area or track
reviewed studies of eHealth literacy among college students
within professional preparation programs. Creating mission and
did not distinguish these emerging applications among the many
policy statements that give attention to these relevant aspects
alternative electronic sources of information, which may not
of eHealth literacy instruction will help improve student
truly reflect current search tendencies of college students. These
outcomes.
types of applications conducive to mobile health information
searches have spawned the new field of mHealth, which may Conclusions
suggest broadening or revising the study of eHealth literacy Evidence from this systematic review suggests that future health
among college students. Finally, while the mean MDSs for the professionals need professional preparatory experiences that
studies in this systematic review were quite good, few reported help build their eHealth literacy proficiencies. Enhanced skills
sufficient validity and reliability measures for data collected development will likely develop as a product of practical
with survey or testing instruments, and almost all data analyses medical Internet research opportunities that encourage critical
were univariate versus multivariate. thinking among students. As suggested by Escoffery et al [23],
Comparison With Prior Works and supported by this systematic review, more needs to be done
to inform the training of students in the health and medical
Even where access to basic Internet infrastructure exists or is
professions, to “search the Internet for health information and
provided, optimal Internet use is often limited by other factors,
to evaluate health information on Web sites.” Because of this,
such as human interface. To some extent, human interface
two important research questions should continue to be
encompasses issues commonly considered when assessing
investigated in medical education. First, do professionally
usability. Usability of an eHealth information source typically
prepared college students in the health professions have the
refers to the quality of a user experience when interacting with
skills to navigate electronic resources to retrieve evidence-based
the resource, with an emphasis on behavior rather than opinion
health information? Second, do college students studying to be
or recollection [32,33]. The construct measures learnability,
health professionals have an inflated sense of self-efficacy
memorability, efficiency, frequency, and severity of errors. All
regarding their actual ability to locate and evaluate good-quality
of these aspects are affected by human limitations, such as
health information on the Internet?
literacy, and by health website quality criteria, such as accuracy,
completeness, readability, and design. Thus, the construct of Given that governmental and advisory agencies have designated
usability is inextricably linked with eHealth literacy. There are eHealth literacy as paramount to improving societal health in
varying levels of usability among eHealth resources, so it would both Canada [36] and the United States [37], it is important that
be useful to determine whether the perceived usability of future eHealth educators be provided with planned learning
resources is related to eHealth literacy outcomes [34,35]. An experiences in this growing field. Several health and medical
analysis that assesses individual perceptions of eHealth usability disciplines have recognized this need area and have incorporated
in relation to overall behavioral capability to locate and evaluate formal professional responsibilities related to eHealth literacy
eHealth information is vital for future eHealth literacy research into core competency development models. For example, future
[33,35]. Studying consumer health informatics (ie, analyzing professionals in the field of health education are expected to
consumer needs for acquiring and using information retrieved find valid health information resources electronically and
using technology) in conjunction with eHealth literacy [34,35] evaluate the usefulness of such information [38]. It is important
can further develop methods that pave the way for providing that health and medical education programs develop these types
health care service in the information age. of proficiencies among future health professionals. Both current
and future college students, especially those in the medical and
Consequently, collegiate degree programs for those entering
health professions, need customized eHealth literacy training
the medical and allied health fields are uniquely positioned to
for finding, interpreting, and evaluating health- and
nurture and develop eHealth competencies among future health
medical-related information available on the Internet.

Conflicts of Interest
None declared

References
1. Pagliari C, Sloan D, Gregor P, Sullivan F, Detmer D, Kahan JP, et al. What is eHealth (4): a scoping exercise to map the
field. J Med Internet Res 2005;7(1):e9 [FREE Full text] [doi: 10.2196/jmir.7.1.e9] [Medline: 15829481]
2. Oh H, Rizo C, Enkin M, Jadad A. What is eHealth (3): a systematic review of published definitions. J Med Internet Res
2005;7(1):e1 [FREE Full text] [doi: 10.2196/jmir.7.1.e1] [Medline: 15829471]

[Link] J Med Internet Res 2011 | vol. 13 | iss. 4 | e102 | p. 10

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Stellefson et al

3. Taylor H, Leitman R. Healthcare News;1(13):1-3. 2001 Apr 23. eHealth traffic critically dependent on search engines and
portals URL: [Link] [accessed 2011-05-27] [WebCite
Cache ID 5z0FCTVgX]
4. Fox S. Health topics: 80% of Internet users look for health information online. Washington, DC: Pew Internet & American
Life Project; 2011 Feb 01. URL: [Link] [accessed
2011-10-05] [WebCite Cache ID 62D0se6Gu]
5. Jones S, Johnson-Yale C, Millermaier S, Perez FS. First Monday(10). 2009. Everyday life, online: U.S. college students'
use of the Internet URL: [Link] [accessed
2011-10-10] [WebCite Cache ID 62L2Sp9eS]
6. Eysenbach G. What is e-health? J Med Internet Res 2001;3(2):E20 [FREE Full text] [doi: 10.2196/jmir.3.2.e20] [Medline:
11720962]
7. Kwankam SY. What e-Health can offer. Bull World Health Organ 2004 Oct;82(10):800-802. [Medline: 15643805]
8. Kreps L, Neuhauser L. Ehealth and the delivery of health care. J Comput Mediat Commun 2010;15(3):364-366. [doi:
10.1111/j.1083-6101.2010.01524.x]
9. Kelley H, Chlasson M, Downey A, Pacaud D. The clinical impact of eHealth on the self-management of diabetes: a double
adoption perspective. J Assoc Inf Syst 2011;12(3):208-234.
10. US Department of Health and Human Services. US Government Printing Office. Healthy People 2010: Understanding and
Improving Health. 2nd edition URL: [Link]
[Link] [accessed 2011-07-22] [WebCite Cache ID 60NFpVRES]
11. Nielsen-Bohlman L, Panzer AM, Kindig DA. Health Literacy: A Prescription to End Confusion. Washington, DC: National
Academies Press; 2004.
12. Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs. Health literacy: report of the Council on
Scientific Affairs. Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association.
JAMA 1999 Feb 10;281(6):552-557. [Medline: 10022112]
13. Norman CD, Skinner HA. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res
2006;8(2):e9 [FREE Full text] [doi: 10.2196/jmir.8.2.e9] [Medline: 16867972]
14. Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. J Med Internet Res 2006;8(4):e27 [FREE Full text] [doi:
10.2196/jmir.8.4.e27] [Medline: 17213046]
15. Tyner K. Literacy in a Digital World: Teaching and Learning in the Age of Information. Mahwah, NJ: Erlbaum; 1998.
16. Presidential Committee on Information Literacy. American Library Association. 1989 Jan 10. Final Report URL: http:/
/[Link]/ala/mgrps/divs/acrl/publications/whitepapers/[Link] [accessed 2011-05-27] [WebCite Cache ID
5z0FaJ7f9]
17. Heuberger RA, Ivanitskaya L. Preferred sources of nutrition information: contrasts between younger and older adults. J
Intergener Relatsh 2011;9(2):176-190. [doi: 10.1080/15350770.2011.567919]
18. Association of College & Research Libraries. American Library Association. 2011. ACRL Publications URL: [Link]
[Link]/ala/mgrps/divs/acrl/publications/[Link] [accessed 2011-05-27] [WebCite Cache ID 5z0GWX4Jv]
19. Ivanitskaya L, O'Boyle I, Casey AM. Health information literacy and competencies of information age students: results
from the interactive online Research Readiness Self-Assessment (RRSA). J Med Internet Res 2006;8(2):e6 [FREE Full
text] [doi: 10.2196/jmir.8.2.e6] [Medline: 16867969]
20. Nagel Bernstein I, Freeman HE. Academic and Entrepreneurial Research: The Consequences of Diversity in Federal
Evaluation Studies. New York, NY: Russell Sage Foundation; 1975.
21. Nsuangani NM, Pérez MA. Accessing web-based health related information by college students: An exploratory study.
Californian J Health Promot 2006;4(1):64-74 [FREE Full text]
22. Castrén J, Huttunen T, Kunttu K. Users and non-users of web-based health advice service among Finnish university students
- chronic conditions and self-reported health status (a cross-sectional study). BMC Med Inform Decis Mak 2008;8:8 [FREE
Full text] [doi: 10.1186/1472-6947-8-8] [Medline: 18237414]
23. Escoffery C, Miner KR, Adame DD, Butler S, McCormick L, Mendell E. Internet use for health information among college
students. J Am Coll Health 2005;53(4):183-188. [doi: 10.3200/JACH.53.4.183-188] [Medline: 15663067]
24. Buhi ER, Daley EM, Fuhrmann HJ, Smith SA. An observational study of how young people search for online sexual health
information. J Am Coll Health 2009;58(2):101-111. [doi: 10.1080/07448480903221236] [Medline: 19892646]
25. Ivanitskaya L, Brookins-Fisher J, O Boyle I, Vibbert D, Erofeev D, Fulton L. Dirt cheap and without prescription: how
susceptible are young US consumers to purchasing drugs from rogue internet pharmacies? J Med Internet Res 2010;12(2):e11
[FREE Full text] [doi: 10.2196/jmir.1520] [Medline: 20439253]
26. Redmond TL. Electronic (Digital) Health Information Competency: A Comparative Analysis of Knowledge and Skills of
Rural and Non-rural Freshman College Students dissertation. Mount Pleasant, MI: Central Michigan University; 2007.
27. Hanik B, Stellefson M. E-health literacy competencies among undergraduate health education students: A preliminary
study. Int Electronic J Health Educ 2011;14:46-58 [FREE Full text]
28. Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev 1977 Mar;84(2):191-215. [Medline:
847061]

[Link] J Med Internet Res 2011 | vol. 13 | iss. 4 | e102 | p. 11

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Stellefson et al

29. The Cochrane Collaboration. 2010. The Cochrane Library Database URL: [Link] [accessed 2011-07-09]
[WebCite Cache ID 603vpBIuY]
30. Guide to Community Preventive Services. The Community Guide Branch, Epidemiology Analysis Program Office (EAPO),
Office of Surveillance, Epidemiology, and Laboratory Services (OSELS), Centers for Disease Control and Prevention.
2011 Jun 30. The Community Guide URL: [Link] [accessed 2011-07-09] [WebCite
Cache ID 603wd3REz]
31. Darling-Hammond L, Rothman R. Alliance for Excellent Education and Stanford Center for Opportunity Policy in Education.
editors. Teacher and Leader Effectiveness in High-Performing Education Systems URL: [Link]
[Link] [accessed 2011-07-22] [WebCite Cache ID 60NKrhsye]
32. Atkinson NL, Gold RS. The promise and challenge of eHealth interventions. Am J Health Behav 2002;26(6):494-503.
[Medline: 12437024]
33. Glasgow RE. eHealth evaluation and dissemination research. Am J Prev Med 2007 May;32(5 Suppl):S119-S126. [doi:
10.1016/[Link].2007.01.023] [Medline: 17466816]
34. Stellefson M, Chaney B, Chaney D. The digital divide in health education: myth or reality? Am J Health Educ
2008;39(2):106-112.
35. Korp P. Health on the Internet: implications for health promotion. Health Educ Res 2006 Feb;21(1):78-86 [FREE Full text]
[doi: 10.1093/her/cyh043] [Medline: 15994845]
36. Canadian Council on Learning. 2008. Health Literacy in Canada: A Healthy Understanding URL: [Link]
pdfs/HealthLiteracy/[Link] [accessed 2011-07-22] [WebCite Cache ID 60NNZEBdF]
37. Anonymous. US Department of Health and Human Services, Office of Disease Prevention and Health Promotion. 2010.
National Action Plan to Improve Health Literacy URL: [Link]
Health_Literacy_Action_Plan.pdf [accessed 2011-07-22] [WebCite Cache ID 60NO2GQ4U]
38. National Commission for Health Education Credentialing, American Association for Health Education, Society for Public
Health Education. A Competency-Based Framework for Health Educators--2006. Whitehall, PA: National Commission
for Health Education Credentialing; 2006.

Abbreviations
AAE: Actual Ability to Evaluate
AAO: Actual Ability to Obtain

Edited by G Eysenbach; submitted 02.12.10; peer-reviewed by L Ivanitskaya, E Neter, T Matingwina, M Stephens; comments to author
01.04.11; revised version received 02.06.11; accepted 19.07.11; published 01.12.11
Please cite as:
Stellefson M, Hanik B, Chaney B, Chaney D, Tennant B, Chavarria EA
eHealth Literacy Among College Students: A Systematic Review With Implications for eHealth Education
J Med Internet Res 2011;13(4):e102
URL: [Link]
doi: 10.2196/jmir.1703
PMID: 22155629

©Michael Stellefson, Bruce Hanik, Beth Chaney, Don Chaney, Bethany Tennant, Enmanuel Antonio Chavarria. Originally
published in the Journal of Medical Internet Research ([Link] 01.12.2011. This is an open-access article distributed
under the terms of the Creative Commons Attribution License ([Link] which permits
unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of
Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on
[Link] as well as this copyright and license information must be included.

[Link] J Med Internet Res 2011 | vol. 13 | iss. 4 | e102 | p. 12

(page number not for citation purposes)
XSL• FO
RenderX