QALMRI Guide | PHIL/COGS 2160 | Prof.

Lande
Details: Explain and analyze the target paper in your own words, by completing each of the compo-
nents in the rubric below. (1) Papers should be written in prose. (2) The discussion should be broken
into sections according to the headings in the left column of the rubric. (3) It must be double-spaced
and no longer than 1000 words. (4) Papers must be uploaded through Turn-It-In.
Late policy: You have 5 extension days for the term. Once exhausted, late papers will not be accepted.
Academic honesty: You are expected to communicate your own understanding of the material. If your
work substantially channels someone else’s understanding (without attribution) or fails to make appropriate
use of quotation marks, then this could constitute academic dishonesty. Academic dishonesty (whether
intentional or not) will be taken extremely seriously, involves prolonged bureaucratic procedures, and will
likely result in severe grade penalties. Familiarize yourself with the Academic Integrity Tutorial and the
Academic Integrity Checklist. If you have any questions at all, reach out to your instructors.
QALMRI Rubric (100 pts):
Question • [4 pts] What was the broad question being asked by this research project?
(10pts) • [4 pts] What was the specific question being asked in this study?
• [2 pts] Well-written (full paragraphs, full sentences, no grammar/spelling errors, etc.)
Alternatives • [4 pts] What were the competing hypotheses (state at least two)
(10pts) • [2 pts] The hypotheses must pertain to the specific question
• [2 pts] Motivate each hypothesis: why one might expect it to be true
• [2 pts] Well-written
Logic • [4 pts] Clearly state: If hypothesis X is true, then we should expect to observe Y.
(10pts) • [4 pts] Make sure an “If. . . then” statement is given for at least two hypotheses.
• [2 pts] Well-written.
Methods • [5 pts] Clearly explain procedures of the study, in everyday terms.
(20pts) • [5 pts] Correctly describe the independent variables.
• [5 pts] Correctly describe the dependent variable.
• [3 pts] Identify whether within- or between-subjects design.
• [2 pts] Well-written.
Results • [8 pts] Clearly describe the pattern of results (how manipulating the independent
(10pts) variables affected the dependent variable).
• [2 pts] Well-written.
Inferences • [5 pts] Clearly describe which hypothesis the results support.
(30pts) • [5 pts] Describe a criticism of the experiment that explains how the results could
obtain without the hypothesis being true AND/OR describe a follow-up study that
addresses open questions.
• [5 pts] Describe criticism/follow-up study in detail.
• [5 pts] Give a persuasive, feasible argument for criticism/follow-up study.
• [5 pts] The criticism/follow-up study should show originality, creativity, or insight.
• [5 pts] Well-written.
Global • Stay within word limit.
Properties • Not too far under word limit.
(10pts) • Points are made in the appropriate sections.
• Not redundant.
• Doesn’t rely overly much on quotation.
• Focuses more on important details than unimportant ones.
Contents

1 Writing Plan (see below for visual guide) 2

2 A Guide to QALMRI 2

3 Final words of advice 5

4 Visual Guide to Writing Plan 6

5 QALMRI Worksheet 9

6 Academic Integrity Checklist 10

Step 2 (Read): Read the target article, taking notes. Write “BQ” in the margin for any-
thing that might be related to a “broad question.” Likewise, “SQ” for specific question,
“A1” for one alternative hypothesis, “A2” for another, “L” for a logic statement, etc.

Step 3 (Plan): Open your QALMRI worksheet (see below or on website). Referring to
the QALMRI rubric above and looking at your annotations on the article, jot down
initial responses to each of the parts of the QALMRI. The target article may not be
ideally clear. E.g., it may not adequately distinguish the broad question from the
specific question. You will likely have to do some work to put some order to it all.

Step 4 (Draft): Set the target article aside. Using the worksheet from Step 3 as reference,
begin writing the paper. In non-technical language, using full sentences and para-
graphs, try to explain clearly your responses to each part of the QALMRI. If you are
confused, you can look back at the target article; but you should try to do as much as
possible just by reconstructing the reasoning of the article from what you have in your
head and in your notes.

Step 5 (Revise): Now that you’ve written a first draft, compare it with the target article
and correct any errors you find in content, exposition, and grammar/spelling.

Step 6 (Check): Re-read your new draft with the QALMRI rubric in hand. Ensure you
have addressed every component and haven’t left out something (e.g. explicitly identi-
fying the independent variables), for which you will needlessly lose points.

Ask: If you have questions, don’t hesitate to ask your instructors. We are here to help!

2 A Guide to QALMRI
The QALMRI method provides a means for critically evaluating experiments. It helps you
to find connections between theory and data by making explicit the question being asked,

2
the approach used to answer it, and the implications of the answer.1

Q stands for Questions All research begins with a question, and the point of the
research is to answer it. For example, we can ask whether a placebo is better than no action
in alleviating depression. For most journal articles, the General Introduction should tell the
reader what question the article is addressing, and why it is important enough that anyone
should care about the answer. Questions fall into two categories: broad and specific. In
your QALMRI, state both the broad and the specific questions being asked.
Broad questions are often too general to answer in a single study, although one should
view the study as one step on a journey to answer the broad question. An example of a
broad question might be, “How do we perceive three-dimensional shapes?” or “Does language
influence perception?” This sort of question provides the general topic of the paper, and can
only be answered through compiling many results.
In contrast, the specific questions can typically be addressed in a single study or set
of studies. A specific question might be, “How accurately do subjects perceive the three-
dimensional shapes of objects under certain conditions?” or “If one language has a specific
term for one colour, and another language does not have any term for that colour, will
speakers of the two languages respond differently on tasks involving that colour?” Again, be
sure to identify the broad and specific question from the article.
A stands for Alternatives Good studies consider at least two possible, alternative
answers to a specific question. They might advocate for one answer over the other. But
they will explain why both are plausible (why a reasonable person could believe either one).
For example, the hypothesis that shape perception is accurate under certain conditions is
plausible given that we seem to be able to identify the same shapes under many different
conditions. The alternative hypothesis, that shape experiences are not accurate under those
conditions, might also be plausible because prior studies have found shape illusions.
When describing an article, you should identify the different hypotheses that the
authors considered, including the hypotheses they themselves advocate as well as any com-
peting hypotheses they consider. There are always at least two hypotheses. For any
hypothesis (e.g. that factor X will show an effect), there is always at least one competing
hypothesis (that factor X will not show an effect; that a null result will be obtained). If
possible identify other alternative patterns as well.
L stands for Logic The logic of the study identifies how the study’s design will
allow the researcher experimentally to distinguish among the different hypotheses. The logic
is typically explained towards the end of the study’s introduction, and has the following
structure: If hypothesis 1 (and not hypothesis 2) is correct, then when a particular
variable is manipulated, the participants’ behaviour should change in a certain
way. For example, the logic of a shape experiment would be: If shape perception is accurate
under these conditions, then: when asked to name the shape of an object (“cylinder,” “cube,”
etc.) viewed under specified conditions, subjects’ responses should correlate significantly
with the actual shape of the object. Alternatively, if shape perception is not accurate
1
Adapted from: Kosslyn, S.M. & Rosenberg, R.S. (2001). Psychology: The Brain, The Person, The
World. Boston: Allyn & Bacon.

3
under specified conditions, subjects’ responses should not correlate significantly with the
actual shapes of the objects they are viewing. Note that the logic of the study is integrally
connected to the hypotheses you stated in the last section.
This section should include “If. . . then. . . ” statements in which you restate
each hypothesis you described earlier (“If X. . . ”), and then state what pattern of
observed data would support that hypothesis (“. . . then Y”). You should therefore
have equal numbers of hypotheses and “If . . . then . . . ” statements.
M stands for Methods This section identifies the procedures that will be used to
implement the logic of the study. It should state the independent variables in the study.
These are the factors that are manipulated in each condition of the study. For example, in
the shape experiment, the independent variables include: the actual shapes of the viewed
objects and the viewing conditions (e.g. the angle form which the objects are viewed). You
should also state the dependent variable of the study: the observed factor or response that
is hypothesized to be affected by one or other of the independent variables. For example,
in the shape study, the dependent variable is the label that the participant provides when
asked to name a given shape. Also describe the subjects, the materials used to conduct the
study, and the experimental stimuli. Convey an idea of what it was actually like to conduct
or participate in the experiment. What did the subjects see? What were they asked to do?
Make sure to describe whether the study was a within-subjects or between-subjects de-
sign. In a within-subjects design, each participant undergoes every experimental condi-
tion/manipulation (every combination of independent variables). For example, each partic-
ipant might have to respond to every shape stimulus under every viewing condition being
studied. In a between-subjects design, different groups of subjects are given different
experimental conditions/manipulations (different independent variables). In order to com-
pare the efficacy of different drugs on depression, for example, one would likely conduct a
between-subjects study: different groups will receive different drugs. A within-subject study,
in that case, would mean giving different drugs to the same person at different times!
R stands for Results What was the outcome of the study? Describe the results
of the primary measures of interest. For example, did different subject groups yield
different group means? What were these means? Or did the entire subject population
produce a distinctive pattern of responses? Describe that pattern. For this section, it is
often a good idea to refer to graphs or tables that illustrate the pattern of data, although
you should describe the results in words to show that you understand them. (If you do make
reference to a graph or table, you can copy and paste it from the article you read into your
own paper, or just refer to it in your paper with a citation.)
I stands for Inferences What can the results of the study tell us about the hypothe-
ses? If the study was well designed, the results should allow you to eliminate at least
one of the possible hypotheses. For example, if subjects’ responses correlate highly with
the actual shapes of the objects they viewed, then the experiment supports the view that
shape perception is not inaccurate under the conditions investigated. Or, if a language lacks
a colour word but the speakers of that language respond to the colour no differently than
speakers of a language lacking a term for the colour, then the experiment supports the view
that language does not influence colour perception.
Now, take a step back and think about potential problems with the study that

4
could have led to the pattern of results you obtained. Were there confounds—
other factors that could have caused the results but which were not studied? For example,
maybe the shapes happened to be different sizes and while subjects were not able to reliably
perceive the shapes themselves, they could make an informed guess on the basis of their sizes.
It is not enough here to point out minor weaknesses in the study (e.g., “There were only
n subjects”) or non-specific possibilities (“The experimenters might have been biased”). A
good criticism shows how someone who rejects the hypothesis that the experiment seemed to
support can nevertheless explain the experiment’s findings. In addition, this is the section in
which to consider the hypothetical next step in answering the broad question. If you were
to conduct a follow-up study, what would it be (hint: think of questions that remain
unanswered by the present results, and sketch a study that could bear on one or more of
those questions)? What questions do the results raise? In this section, you should at
least raise one problem with the study or describe one follow-up study.
3 Final words of advice
• Follow the format and instructions in this document as closely as possible. Contact
your instructors if you are at all confused or have any questions.
• Carefully read and annotate article a couple of times. Before filling in the QALMRI
worksheet, form in your a head a big picture idea of what is happening in the article.
• Begin drafting the paper itself a day or more after having read the target article,
when the exact wording of it is not fresh in your mind. This will help you begin to
communicate the paper in your own words as you’ve understood it. It will also allow
you to check the accuracy of your understanding.
• Write a draft. Put it down for a day or two. Come back and revise with fresh eyes.
• Try to write so clearly that someone with no background in cognitive science can
understand you. This means that you should explain any technical terms you use.
• Proof read! Spelling and grammar mistakes are to be avoided.
• To stay within the word count, you may need to be selective about which parts of
the experiment you describe, especially in the methods section. Include only the most
relevant aspects. You don’t need to describe the statistics used to calculate the results.
• The Inferences section should be the longest. This is your opportunity to distinguish
your paper by showing that you’ve thought critically about the study.
• Use your own words and try to avoid just quoting the source paper. Communicate the
study as you’ve understood it. Rephrase things entirely in your own words, based on
your understanding. Do not just copy a passage from the source paper and altering
it slightly by use of synonyms and grammatical transformations—this sort of close
paraphrasing is a potential breach of academic integrity.
• If you find it absolutely necessary to appeal to someone else’s wording, you should be
sure to enclose the passage in quotation marks and attribute it to its source.

5
4 Visual Guide to Writing Plan
Step 1 (Prepare): (Re-)read this document thoroughly.

Step 2 (Read): Read and annotate article. & Step 3 (Plan): Fill out QALMRI Worksheet.
 Step 4 (Draft): Using the worksheet as reference, begin writing the paper.

Step 5 (Revise): Compare first draft with the target article and correct any errors in
content, exposition, and grammar/spelling.
Step 6 (Check): Re-read your new draft with the QALMRI rubric in hand. Ensure you
have addressed every component and haven’t left out something
5 QALMRI Worksheet

QALMRI Worksheet
COGS/PHIL 2160

Questions
[Broad question]
[Narrow question]
[Check that written clearly, in full paragraphs, full sentences, no grammar/spelling errors, with
non-technical prose]

Alternatives
[Alternative hypotheses (at least two)]
[How they pertain to the specific question]
[Motivate each alternative]
[Check that written clearly, in full sentences, with non-technical prose]

Logic
[If hypothesis 1 is true, then we should expect to observe…]
[If hypothesis 2 is true, then we should expect to observe…]
[So on for remaining hypotheses]
[Check that written clearly, in full sentences, with non-technical prose]

Methods
[Explain design, stimuli, procedures, etc.]
[What are the independent variables?]
[What are the dependent variables?]
[Within-subject or between-subject?]
[Check that written clearly, in full sentences, with non-technical prose]

Results
[Main results: how dependent variable changed as a function of independent variables]
[Check that written clearly, in full sentences, with non-technical prose]

Inferences
[Which hypothesis is supported by results?]
[How to explain results otherwise AND/OR what follow-up studies to address open questions]
[Details of criticism/follow-up]
[Argument for criticism/follow-up: why is it plausible? Why is it important?]
[Check that written clearly, in full sentences, with non-technical prose]

References:
[Citation info for referenced articles, including target article, in whichever format you prefer.
E.g., APA style]
6 Academic Integrity Checklist

Academic Integrity Checklist

❏ I fully understand my instructor's requirements for group and individual assignments (when in doubt
always ask).

❏ There is no section or component of my assignment, paper or research report that I do not

❏ I have kept my research notes, lab-sheets, outlines and early drafts (you may be asked to provide
these to your instructor).

❏ My direct quotations have quotation marks (both opening and closing) and are fully referenced.

❏ All ideas used from other sources include citations.

❏ The reference entries in my bibliography/works cited/reference list corresponds with every idea/quote
I have used.

❏ I have checked my work against my notes to be sure I have correctly referenced all direct quotes or
borrowed ideas.

❏ Apart from material that is a direct quotation, everything else is in my own words.

❏ In paraphrasing the work from others I put the complete idea into my own words and did not just
change a few words.

❏ I have checked all citations for accuracy (i.e., page numbers, dates, web page addresses).

❏ I have asked an instructor or librarian about how to include any material I was unsure how to
document.

❏ I have used one recognized Reference Style (i.e., APA, MLA, Chicago, etc.) consistently throughout
my paper/work.

❏ I have accurately and completely described any data or evidence I have collected.

❏ I did not share my work/answers/content with others in ways not explicitly authorized by my
instructor.

❏ This is the first time I have submitted this material for a course.

Name Date

SPARK content is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.