Chapter 3: Comparative Method Studies

- Comparative method studies aim to determine the most effective type of language
instruction among two or more options, with the goal of improving teaching
effectiveness
+ assume that it's possible to identify a specific type of instruction that works more
effectively than others
- Two main categories of comparative method studies:
+ global studies: examine long-term methods or approaches for teaching a second language
and assess learning outcomes in terms of general lang. proficiency or achievement.
+ local studies: investigate methods over a short period of time and focus on specific
linguistic features to assess learning outcomes.
- The chapter primarily focuses on global studies
- "method": a way of teaching lang. based on systematic principles & procedures (an
application of views of how a lang. is best taught & learned)
+ various methods or approaches: the Audiolingual Method, Total Physical Response,
various humanistic methods (e.g. community lang. learning) and communicative language
teaching.
+ other methods: the Lexical Approach and Competency-Based Lang. Teaching
+ methods have proliferated over the years
- The comparative method studies have examined relatively few of the diff. methods
=> method is a pedagogic construct enshrined in a set of principles and techniques that
specify how specific acts of teaching are to be performed
- However, the concept of "method" presents challenges, as it often treats teaching as an
external process separate from the actual act of teaching. This approach positions
teachers as implementers of a method's prescribed script, which can limit their
autonomy and creativity in the classroom.
-> This gap between theory and practice (and between theorists & practitioners) is a central
issue in language teaching research.
- The second issue with the concept of teaching methods, related to the first, is that it
defines teaching in broad terms which ignores the fact that “lang. pedagogy to be
relevant, must be sensitive to a particular group of teachers, teaching a particular group
of learners, pursuing a particular set of goals, within a particular institutional context,
embedded in a particular sociocultural environment”.
-> The assumption that there is a "best" way to teach language is mistaken. What may be
the best approach for one group of learners in a specific context might not be suitable for a
different group in a diff. context.
- Comparative method studies have largely ignored these fundamental issues. They have
been grounded in a perspective that views teachers as implementers of externally
 defined set of specifications about how to teach, assuming that these specifications are
broadly applicable to all (or a wide range) of classroom contexts. These issues should be
kept in mind when examining research on the effectiveness of different teaching
methods.
=> in this chapter, the author:
+ reviews a number of early and later global method studies
+ presents the methodology employed & the results obtained y each study
+ submits each to a critical evaluation
+ consider whether there is still any value in conducting such studies
+ prepare for the alternative ‘post-method’ view of lang. pedagogy that informs how lang.
teaching has been investigated
1. Early Comparative Method Studies
- A feature of many early method studies: carried out by teachers in their classrooms
+ the studies these teachers conducted were methodologically flawed
>< should not be seen as criticism as methodological preciseness in experimental research at
the time of these studies was not strongly established in classroom research
2 examples:
- In an early study by Bennett (1917), he compared the German vocabulary learned by
groups of junior and senior high school students in the USA.
+ the students were taught using either ‘translation study’ (translating from German to
English) or ‘immediate study’ (memorizing a passage in German to later translate it into
English).
+ Bennett's rationale was that 'immediate study' involved more attention to German words
as carriers of meaning.
+ while descriptive results were presented for both treatments, there was no test of
statistical significance.
-> The 'immediate study' approach resulted in more words learned - Bennett suggested it
was because it demanded greater cognitive effort.
- In a study by Beck (1951), he compared students taught with a traditional approach
(grammar translation) and an 'oral-cultural approach' aimed at developing oral skills.
+ the oral-cultural treatment involved listening to spoken language, repetition, reading
aloud, and conversation practice.
+ testing was challenging as no oral development measures were available -> reading,
vocabulary, and grammar were assessed.
-> The results favored the university students who received the oral-cultural treatment -
Beck provided no discussion of the results.
=> Neither of the studies satisfies the criterion of replicability
+ both studies have unclear descriptions of the methods, lack of clear group equivalence,
limited information about the tests used, and a lack of statistical significance testing.
+ despite their shortcomings, these studies are interesting as Bennett's study focused on the
role of cognitive effort in learning >< Beck’s study aimed to enhance oral proficiency, recognizing
the limitations of grammar translation.
+ these issues remain relevant in language teaching today, emphasizing that early researchers
were already considering essential aspects of language pedagogy
2. Large-Scale Comparative Method Studies in the 1960s
- Early studies: all small-scale
>< In the 1960s, a number of large-scale global method studies was conducted – often cited
by lang. teaching researchers to illustrate the problems of such method comparisons
- This period in the 1960s was marked by what Diller (1978) referred to as 'the language
teaching controversy,' which pitted rationalist approaches to language teaching against
empiricist approaches.
- Rationalist methods, such as grammar translation and the Cognitive-Code Method,
emphasized explicit knowledge acquisition through rule explanation >< Empiricist or
'functional' methods, such as the Audiolingual Method and the Oral Approach,
emphasized inductive rule learning through extensive oral practice.
- At the time, there was a desire to investigate which of these methods produced better
results.
+ both rationalist and empiricist methods focused on linguistic form, primarily grammatical
form, and aimed to help learners achieve accuracy in a second language.
- Three significant large-scale studies from this era are discussed: Scherer and Wertheimer
(1964), Smith (1970), and Levin (1972). These projects have become known by their
names - Colorado Project, Pennsylvania Project and GUME
- The author provides a brief description of each project -> review the various issues that
these projects brought to the fore
a. The Colorado Project
- The first large comparative method study, comparing the Grammar-Translation Method
and the Audiolingual Method.
+ the researchers observed the progress of beginner-level college students learning L2
German through each method and assessed their performance at the end of the first and
second years of study.
-> at the end of the first year, the results favored the Audiolingual Method in all tests except
for translation >< at the end of the second year, no significant overall difference was found
between the two methods.
+ the Audiolingual Method group performed better in speaking
+ the Grammar-Translation Method group did better in reading, writing, and translation.
 - Essentially, each method led to learning outcomes that reflected its instructional focus.
It's worth noting that the finding regarding the Audiolingual Method's advantage in speaking
may not be universally applicable to language pedagogy, as students needed special
inducement to engage in speaking practice.
=> Although the Colorado study didn't produce a definitive conclusion on the superiority of one
method over the other, it led to a critical examination of the claims of the Audiolingual Method
b. The Pennsylvania Project
- The Pennsylvania Project, led by Smith in 1970, conducted a study comparing the effects
of three different teaching methods in beginning and intermediate high school-level
French and German classes. The three methods:
+ 'Traditional' method (grammar translation)
+ 'Functional skills' method (essentially the Audiolingual Method)
+ 'Functional skills plus grammar' method.
- The study assessed student achievement in listening comprehension, speaking, reading,
and writing at both mid-year and the end of the year using standardized tests.
-> the results, in general, showed no significant differences between the three methods.
>< the 'traditional' group performed better than the other two groups in two of the reading
tests.
+ after two years, the 'traditional' group again excelled in reading ability but did significantly
worse on a test of oral mimicry - no notable differences were found in the students'
performance in the other tests.
- This project faced criticism from Clark (1969) for several reasons
+ the project did not ensure that the teachers of the two programs had equivalent
experience
+ although the project included classroom observations to verify if the methods were
implemented as specified, the use of different rating scales made it difficult to determine if
the two methods were in fact distinct.
c. GUME (Gothenburg/Teaching/Methods/English)
- Conducted six studies to compare two different methods, both of which were related to
audiolingual approaches but differed in the provision of explicit grammar explanations.
- Five of these studies involved comprehensive school students, while one focused on
adult learners. The two methods compared were:
+ The Implicit Method, corresponding to an inductive-oriented audiolingual method.
+ The Explicit Method, corresponding to a deductive-oriented audiolingual method with
grammatical explanations, provided either in L2 English or L2 Swedish, depending on the
specific study.
- The instruction in most experiments concentrated on teaching specific grammatical
structures but with one experiment exposed students to various grammatical features.
+ The studies used various types of tests, primarily selected response (multiple choice
questions tests) or constrained constructed response (sentence completion) tests.
-> the results generally showed no significant differences between the implicit and explicit
school groups.
+ in some groups very little learning was evident but even in cases where learning occurred,
the type of instruction did not make a significant difference.
>< the overall pattern of results indicated that, in terms of learning outcomes, the Explicit
Method with L1 grammar explanations was the most effective, followed by the Implicit
Method, and then the Explicit Method with L2 grammar explanations.
+ a group of older high school students showed a clearer advantage for the Explicit Method,
especially when grammar explanations were provided in their L1 (Swedish) - similarly, adult
learners benefited the most from the Explicit Method when grammar explanations were
offered in their L1.
=> Levin concluded that the main results tended to support the cognitive-code learning
theory, but only at the upper secondary school and adult learner levels.

- These studies point to the inherent difficulties of global method comparisons – carefully
designed studies >< failed to provide a clear demonstration that 1 method was superior
to another in terms of overall lang. proficiency
+ learner age and context: The age of the learners and the instructional context appeared to
act as mediating variables, influencing the effectiveness of different instructional methods.
e.g. the Pennsylvania Project, which focused on school-level learners, found no significant
impact of the method chosen on these learners >< GUME, which involved adult learners,
showed that the Explicit Method was more effective than the Implicit Method
-> age or the specific context in which instruction takes place can affect the relative
effectiveness of teaching methods.
+ other mediating factors: while age and context seemed to be influential, there might be
other mediating factors such as language aptitude and motivation >< GUME (which
investigated these) did not report any clear interaction effects between these variables and
the choice of teaching method
+ lack of detailed information: there was a gap in all three projects - detailed information
about the instructional processes -> difficult to determine whether the external differences
between teaching methods were reflected in the way instruction was actually conducted.
+ testing instruments: a significant limitation was the nature of the testing instruments used
to measure learning - these tests primarily relied on discrete-item constrained-response
type. Such testing instruments tend to favor methods that emphasize explicit knowledge of
the second language (L2) - if these studies had included tests of free oral production, it's
possible that clearer differences, potentially in favor of the Audiolingual Method, might have
been evident.

3. Comparative Studies investigating Superlearning

- In the 1970s, there was a growing interest in humanistic language teaching methods -
one of these methods is known as Suggestopedia and its North American adaptation
called the System for Accelerative Learning and Teaching (SALT), has been the object of a
number of comparative method studies.
+ Suggestopedia (Lozanov) aimed to enhance language learning (especially vocabulary
learning) by inducing a relaxed state in learners and using suggestive techniques for
subliminal learning (include relaxation training, synchronized breathing, music, suggestion
(learning an L2 is easy), visualization, vivid presentation of target items, passive review, and
dramatic enactment).
-> Lozanov claimed that these techniques could result in a 100 percent increase in learning.
- Wagner and Tilney (1983) conducted a study comparing the superlearning method and a
traditional rote-learning method on the acquisition of 300 German words by 21 adult
English-speaking learners divided into 3 groups.
+ two groups were taught using superlearning techniques, one with relaxation training and
Baroque classical music and the other with just relaxation training.
+ the third group participated in traditional rote classroom vocabulary sessions
+ learning was assessed through an aural translation test (L1 to L2) and the study also
measured alpha brain waves.
-> the results showed no significant differences in alpha brain waves between the
superlearning groups and the traditional group >< the traditional method proved to be
superior in the translation test.
-> Wagner and Tilney concluded that the combination of relaxation, special breathing,
intonation, and music was not enough to produce "super" results.
- Several other studies investigating superlearning techniques have been published, as
surveyed by Dipamo and Job (1991):
+ they noted that many of these studies had significant methodological weaknesses and did
not provide clear evidence to support the idea that superlearning techniques are superior to
more traditional teaching methods.
+ they suggested that studies showing an advantage for superlearning often included
visualization techniques, which have been consistently shown to be effective in other
vocabulary learning research.
4. Comparative Studies investigating comprehension-based lang. teaching
 - Around the same time as the Colorado, Pennsylvania and GUME projects, some lang.
educators were investigating an approach to lang. teaching that differed quite markedly
from both the inductive & deductive methods (the focus of the 3 projects)
- The approach entailed teaching grammar through comprehension-based instruction
rather than through production
- The theoretical rationale for such an approach was given by Gary:
+ the comprehension was prior to production in lang. acquisition:
 forcing learners to produce in the L2 overloaded short-term memory & could have
deleterious effect on acquisition
 teaching through comprehension reduced learner anxiety
 it exposed learner to more target lang.
- The key figure in researching comprehension-based lang. teaching is James Asher – he
and his associates conducted studies to investigate the effectiveness of Total Physical
Response (TPR) compared to other language teaching methods, particularly the
Audiolingual Method.
+ TPR emphasizes:
 delaying speaking until learners are ready
 maximizing exposure to the language through commands
 postponing abstract language until learners have advanced
e.g. of Asher's study compared TPR and the Audiolingual Method's effects on immigrants
learning English as a second language (ESL)
+ the Fleming Test (a 40-item test design to measure reading ability) was used to measure
learning
+ learners taught using the Audiolingual Method scored 24% after 96 hours of instruction, while
TPR learners at the same level scored 52% after 120 hours of training, even outperforming
students in more advanced Audiolingual classes
–> Asher suggested that for children & adults learning lang. such as English, German, French,
Japanese, Russian, Spanish – the results were accelerated stress-free acquisition that had long-
term retention -> suggests an advantage for TPR, especially in the early stages of L2 learning
- However, the interpretation of Asher's studies by Krashen (the TPR results are clear and
consistent, and the magnitude of superiority of TPR is quite striking) has been critiqued.
+ the tests used to measure proficiency in his studies appeared to favor TPR
+ the instruction period was relatively short -> there is a question about the long-term
effectiveness of production-based methods
+ TPR primarily focused on beginner learners, and comprehension-based instruction may be
less effective with learners who have developed some proficiency in the L2
+ a pre-test was not included in the study
>< the TPR studies stand out in comparative method studies as providing evidence that
support the superiority of a particular method (at least in the early stages of L2 learning)
- Advocates of comprehension-based instruction do not propose the abandonment of
methods based on production >< practice in oral production should be delayed & early
lang. instruction should emphasize developing substantial competence in
comprehension before requiring learners to talk in the new lang.
- Another study by Lightbown (1992) in New Brunswick involved Canadian French children
in grades 3–6 learning English through listening to tapes and following written text,
compared with an ESL program focusing on oral production practice and grammar
explanation
+ a strength of this study: it measured learning using tests that were matched to each
method -> avoid the bias problem
+ results showed that the comprehension-based program was at least as effective as the
production-based program, and students in both programs performed similarly in speaking
English
+ however, they performed differently in written production – the students in the regular
ESL program produced better results
- Shintani and Ellis (2010) conducted a study on English plural –s, comparing
comprehension-based and production-based instruction
+ this study had two notable differences from Lightbown's studies: it was relatively short-
term, involving only three lessons and it centered on the acquisition of plural -s and
investigated incidental learning (there was no explicit focus on the target structure)
+ the comprehension-based instruction comprised listening tasks that emphasized
understanding the meaning, while the production-based instruction followed a PPP
approach. + the study measured the acquisition of plural -s through comprehension and
production tests -> the results showed that both the comprehension-based and production-
based groups outperformed a control group -> only completed the tests but there were no
statistically significant differences between the two experimental groups in general.
>< the comprehension-based instruction appeared advantageous for complete beginners –
this can be explained by qualitative diff. in the kind of feedback the 2 groups experienced
+ the feedback in comprehension-based instruction enabled learners to distinguish
meanings of plural & singular N, whereas production-based instruction mainly provided
corrective feedback (focused on whether the learners had produced the correct noun form)
-> Shintani conducted a follow-up analysis of the instructional processes, emphasizing how
the two types of instruction differed markedly in opportunities for learners to seek meaning
and in who controlled the discourse (whether it was the teacher or the learners themselves)
=> Comparative method studies investigating comprehension-based and production-based
language instruction have provided clearer results compared to earlier studies that focused
on inductive vs. deductive instruction, all of which were primarily production-based. T
+ there is fairly convincing evidence that comprehension-based instruction is as effective as
production-based instruction, and in some cases, found to be superior for beginner learners.
The evidence supporting this conclusion comes from studies conducted by Asher,
Lightbown's New Brunswick project, and the research by Shintani and Ellis.
>< There is no evidence to show that the comprehension-based instruction is more effective
with more advanced learners & some evidence that it may be less effective than production-
based instruction in the long term
5. Comparative Studies investigating Communicative Approaches
- In the 1970s, the concept of communicative language teaching emerged as a theoretical
approach
- This approach (often referred to as the "communicative approach") emphasized
teaching language not as a set of formal rules but as a tool for making meaning, focusing
on language "use" rather than "usage."
+ the foundation of this approach drew from functional accounts of language, theories of
communicative competence, and nativist theories of language learning.
- Two versions of the communicative approach were identified:
+ weak version: teaching content was defined in terms of linguistic realizations of notions
and functions, with a methodology similar to the Audiolingual Method - involving controlled
practice and was accuracy-oriented
+ strong version: specified content in terms of tasks for learners and prioritized fluency,
encouraging learners to communicate using available linguistic and non-linguistic resources.
- Over time, the strong version evolved into "task-based language teaching" (TBLT), and
the term "communicative" was sometimes applied to courses that followed a traditional
structural syllabus, leading to a diverse range of conceptualizations of language teaching
labeled as "communicative"
- This diverse usage of the term "communicative" should be considered when evaluating
various comparative method studies that claim to investigate this type of teaching
Early studies investigating communicative lang. teaching with traditional teaching:
- Savignon (1972) studied university classes of French as a foreign language.
+ all three groups received traditional instruction, but one group had an additional
'communicative hour,' another had an extra 'cultural hour,' and the third group used a
language laboratory.
+ while there were no differences in grammatical proficiency among the groups, the
communicative group outperformed the others in terms of communicative ability.
-> This study suggests that a combination of traditional and communicative instruction is
more effective than purely traditional instruction.
- Palmer (1979) compared 'traditional' and 'communicative' instruction for Thai university
students.
+ the traditional group focused on mechanical and meaningful drills for specific linguistic
features >< while the communicative group also followed a structural syllabus but engaged
in communicative activities (an exchange of information)
+ the communicative group showed significant positive correlations between
communicative and achievement measures, whereas the traditional group did not.
>< Krashen (1981) noted that there were no significant differences between the groups on
communicative measures, which he explained in terms of access to comprehensible input -
the teacher-talk in the traditional group was in the target language, while it was in the
students' L1 in the communicative group.
- Hammond (1988) compared Spanish language programs at two universities
+ experimental groups were taught using the Natural Approach (a communicative method)
>< while comparison groups were taught using the Grammar-Translation Method (a
traditional method emphasizing deductive grammar learning)
+ while the experimental groups generally outperformed the comparison groups in mid-
term and final examinations, many of the differences were not statistically significant.
>< the study did demonstrate that students in communicative classrooms did not perform
worse in learning grammar compared to those in traditional programs.
- In a study by Prabhu (1987), the Communicational Teaching Project in India was
evaluated using a task-based approach (a strong form of the communicative teaching
method)
+ the learners engaged in a series of communicative tasks, initially with the teacher and then
individually, without explicit presentation or practice of specific linguistic items.
+ Beretta and Davies (1985) compared the learning outcomes of the students in the
Communicational Teaching Project (the experimental group) with those taught using a
'traditional' approach based on the Structural-Oral-Situational Method (SOS), which
followed a structural syllabus and involved pattern practice exercises
+ to minimize testing bias, Beretta and Davies designed a battery of tests, including one that
favored the experimental group (a task-based test) and another that favored the control
group (a structural test). Additionally, three 'neutral' tests (contextualized grammar,
dictation, and listening/reading comprehension tests) were administered
-> The results of the study provided general support for the effectiveness of task-based
teaching.
 + in the neutral tests, the experimental group outperformed the control group >< in the
group-biased tests, the experimental group scored higher on the task-based test, while the
control group performed better on the structural test
=> Beretta and Davies concluded that the results of the evaluation suggested that task-
based instruction leads to significantly different learning outcomes compared to traditional,
form-focused instruction.
+ task-based learners demonstrated superior acquisition of structures not explicitly taught
and were more proficient in applying what they had learned
- In a study by Allen et al. (1990), a different approach to comparative method studies was
adopted - rather than externally defining "method" in terms of specific teaching
prescriptions, the study was based on Stern's distinction between "experiential" and
"analytic" teaching strategies (communicative & traditional teaching approaches)
+ they conducted observations of actual lessons using an interaction analysis schedule
known as the Communicative Orientation of Language Teaching (COLT) - these observations
ranked eight Grade 11 French classes in Toronto according to how experiential or analytic
they were in practice.
+ the study hypothesized that analytic classes would perform better on measures of written
and grammatical accuracy, while experiential classes would excel in sociolinguistic and
discourse competence >< when comparing pre- and post-test measures, the study found
that there were few statistically significant differences between the two most experiential
and the two most analytic classes. Allen et al. admitted that the results were somewhat
disappointing, as they did not align with the initially hypothesized outcomes.
- Recent studies have continued to explore the comparison between communicative and
non-communicative language teaching approaches. A shift in theoretical orientation is
noted, as Long (1991) introduced the distinction between "focus on forms" and "focus
on form":
+ "focus on forms" relates to traditional language teaching, which involves presenting and
practicing linguistic items based on a structural syllabus
+ "focus on form" utilizes various teaching strategies (corrective feedback) to direct learners'
attention to linguistic form while they engage in communicative tasks.
-> Both approaches aim to teach linguistic forms, differing in whether this occurs in a
communicative or non-communicative context - 2 studies have been conducted to compare
the effects of these approaches on second language acquisition

1, study conducted by R. Sheen in 2006, the effects of focus on form and focus-on-forms
instruction were compared in the context of acquiring two grammatical structures: WH
interrogatives and adverb placement - lasted 7 months and involved Grade 6 elementary
students aged between 11 and 12
 + the focus-on-forms instruction involved explicit descriptions of the target structures,
controlled practice exercises, and opportunities for free production (i.e., presentation,
practice, production) >< the focus-on-form instruction utilized various communicative
activities, with the teacher providing feedback when errors in the production of the target
structures occurred.
+ three types of tests were used to measure progress, including an aural written
comprehension test, an oral interview (scored for correct use of the target structures), and a
grammaticality judgment test
-> none of these tests assessed the learners' ability to produce the target structures in
spontaneous communicative speech
+ results indicated that in the oral production test, the focus-on-forms group demonstrated
increased accuracy over time (with a decline in the delayed post-test), while the focus-on-
form group showed no improvement
+ there were no statistically significant differences in the pre-test scores between the two
groups, but in all the post-tests, the focus-on-forms group outperformed the focus-on-form
group
+ in the grammaticality judgment test, administered only at the end of the study, the focus-
on-forms group also displayed superiority
>< the study's validity was questioned, as Sheen noted that there were few opportunities for
corrective feedback in the focus-on-form group -> this type of instruction might not have been
effectively implemented
+ the testing approach was seen as biased in favor of the focus-on-forms group
-> this study raises doubts about the clear superiority of either instructional method & it seems
challenging to reach definitive conclusions based on the study's limitations

2, study conducted by Laufer in 2006 that compared focus on form and focus on forms, the
focus was on vocabulary acquisition - the participants included 158 high school students in
Israel.
+ focus on form was operationalized as the incidental acquisition of English L2 target words
(learners were exposed to the target words in a reading task) - they read a text, answered
comprehension questions, and were allowed to use a bilingual dictionary when needed
+ focus on forms involved intentional learning of the words - learners received a list of the
words with their Hebrew translations and explanations in English, followed by two word-
focused exercises
+ both groups were tested by requiring them to provide translations or explanations of the
target words.
-> the results indicated that the focus-on-forms group outperformed the focus-on-form group,
and this difference was statistically significant
+ in a follow-up study, both groups engaged in intentional learning of the words - they received
a list of target words with definitions, examples, and translations and spent 15 minutes
memorizing them -> the subsequent testing, which included translating the words from L1 to L2
and in a delayed post-test from L2 to L1, revealed no significant differences between the two
groups.
=> Laufer's conclusion was that in the context of vocabulary learning, focus on forms
(intentional learning) appeared to be superior >< it should be noted that the testing approach
used was somewhat biased in favor of the focus-on-forms group, as it closely matched the types
of activities that this group had engaged in during instruction.

Author’s comment: The studies comparing communicative language teaching (CLT) and
traditional language teaching have produced mixed results, and reaching clear conclusions is
challenging. Out of the seven studies discussed:
+ 3 studies (Allen et al., 1990; Hammond, 1988; Palmer, 1979) found no clear differences in
learning outcomes between CLT and traditional language teaching, which typically involved
some form of present-practice-produce approach.
+ 2 studies (Beretta and Davies, 1985; Savignon, 1972) generally favored a task-based approach
over traditional methods.
+ 2 studies (Laufer, 2006; Sheen, 2006) that compared focus on form and focus on forms
instruction found traditional language teaching to be more effective.
>< all of these evaluation studies had methodological issues, including problems with the
assessment of learning outcomes -> to draw valid conclusions, it's essential to ensure that the
assessment methods used are not biased towards any particular teaching approach
6. Summary
- The studies comparing different language teaching methods have several
methodological problems, which can affect the validity and reliability of their findings.
These issues include:
1, lack of equivalence among learner groups - some studies compared groups of learners that
were not equivalent, such as school students and university students, without administering a
pre-test to ensure equivalence within the groups.
2, diff. instructors - in many studies, the groups being compared were taught by different
teachers -> makes it difficult to isolate the effects of the teaching method, as differences in
instructors' teaching skills could have influenced the outcomes.
3, lack of adherence to method descriptions - many studies failed to establish that the
instructional treatments were carried out as described in the external descriptions of the
teaching methods -> not clear that the methods actually differed in terms of classroom
processes.
4, ignoring individual learner differences - most studies did not consider the possibility that
different learners may benefit from different types of instruction
+ only the GUME project investigated the interactions between individual learner differences
and instructional outcomes.
5, absence of control groups - many studies did not include control groups, which are essential
for determining whether improvements in test scores result from the instructional methods or
simply from the experience of taking the tests multiple times.
6, lack of test bias control - only 4 of the studies attempted to address test bias by ensuring that
the testing battery included tests aligned with the methods being compared or by administering
"neutral" tests that were not biased toward any particular approach.
=> highlight the need for well-designed and controlled studies to provide more reliable insights
into the effectiveness of different language teaching methods.
- Drawing general conclusions from the studies comparing language teaching methods is
challenging due to the heterogeneity of the studies and various methodological
problems.
+ early studies - the very early studies had major methodological problems, making it
difficult to draw meaningful conclusions from them.
+ large-scale studies in the 1970s: these studies did not demonstrate the superiority of any
of the methods they investigated, leading to a shift in focus towards direct observations of
classroom processes.
+ comprehension-based instruction: some studies suggest that comprehension-based
instruction may be more effective than other approaches, particularly for beginner learners:
Asher's Total Physical Response (TPR) demonstrate the superiority of TPR over the
Audiolingual or Grammar-Translation Methods, especially for beginners - Shintani and Ellis's
study also favored comprehension-based instruction over PPP for beginners.
+ task-based instruction: Beretta and Davies' evaluation of Prabhu's task-based approach
indicated the effectiveness of comprehension-based instruction but this was for learners at
an elementary level of L2 proficiency
+ comprehension-based instruction differences: the effectiveness of comprehension-based
instruction may depend on whether it is teacher-centered and directed at beginner learners
- studies have shown varying results and the advantage may not hold for learners at more
advanced stages of proficiency.
-> a tentative conclusion: when comprehension-based instruction is teacher-centered &
directed at beginner learners it is more effective than methods seeking to elicit production
from such learners
+ communicative vs. explicit instruction: there is little evidence to show significant
differences in the learning outcomes of communicative approaches and more explicit forms
 of instruction. Savignon's study suggested that a combination of form- and meaning-based
instruction was superior in developing communicative abilities.
+ focus-on-forms instruction: 2 studies by Sheen and Laufer claimed the superiority of focus-
on-forms instruction but had biases in testing in favor of explicit instruction -> these findings
should be interpreted with caution.
=> while some studies hint at the potential advantages of comprehension-based instruction for
beginners, the effectiveness of different teaching methods depends on several factors, including
learner proficiency levels and the specific teaching context -> further research with improved
methodology is needed to provide more conclusive findings