VIDEO TRANSCRIPT PSY4062 [1]
Research Methods and Theory
Video 2 ‘Designing your Project’
Slide 1
Hi and welcome to the second video in this series - designing your project. In this video
we’re going to explore research designs so that you can begin to develop your own research
question for this project.
Slide 2
Let’s look over the entire research process, from beginning to end. You may have seen this
represented in similar ways in the readings you have done, but the 5 main steps are
depicted here:
1. Developing the research question
2. Research design
3. Conduct the study and collect data
4. Analyse the data
5. Interpret and report findings
Notice that research design and statistical analysis are not the same thing, and in fact they
are far separated in the research process (more on this in a few slides).
For now, we will mostly focus on the second stage in this video, but before we can, we need
to, of course, develop a research question. Research questions are crucial because they
help scientists focus their efforts by providing a path through the research and writing
process.
Most researchers begin by looking at a research area or topic that they are genuinely
curious about. They then gauge the lay of the land, so to speak, by looking at the published
literature on the topic, to assess what has been done before, and what hasn’t… Or perhaps,
what has not been done well… This allows the identification of gaps in previous literature.
Now remember (and this is a really crucial point) - just because something has not been
researched before does not immediately mean that it is a gap, nor that it should be
researched. What is needed here is a rationale or a reason or a need for doing the proposed
research. How would this benefit the field? How will it help advance the current knowledge
and understanding? How will it benefit society more broadly? All these questions are crucial
to consider when settling on a research question, and it is a process you will need to go
through when picking yours.
But let’s assume that you have identified a worthwhile gap, and a rationale for addressing it.
In that case, each of these gaps can be stated as a research aim. From prevailing current
theories and findings, you can also predict what you expect to happen if you design a study
to test that aim.
So focusing now on the second step of design, you need to identify your main variables,
operationalise them (that is, define how we’ll measure them), state your prediction as a
hypothesis and then determine which statistical test will be suitable to test this hypothesis.
Slide 3
Let’s revise a couple of the main research designs you’ve learned from previous units in
psychology. One of the most powerful is the experimental/interventional design. In these
studies, participants are randomly allocated to conditions in which some condition of interest
is manipulated. They may be in a treatment group, which receives an intervention, or a
control group, which does not. It’s a powerful design because we can conclude that any
differences between groups are likely to have been caused by the intervention - that is,
these types of designs, when designed properly, allow us to infer causality. To truly infer
causality, however, a pre and a post test is useful so that changes can be confirmed to be a
result of the intervention rather than any pre-existing difference between the groups.
Quasi--experimental designs are similar but not as powerful as participants are not able to
be randomly allocated to groups.
Slide 4
Observational studies (of which there is a number of types), are often placed in contrast to
experimental studies because they do not include any intervention or manipulation - only
observation/measurement. One popular type of observational design, which you have come
across before, is called the cross-sectional design. This type of design involves
testing/taking measurements at only one time-point (as opposed to a longitudinal design,
which takes measurements at multiple time points) and it shares the same characteristics as
all other observational designs - there is no intervention, no manipulation and no
randomisation of participants. Given this, causality most often cannot be inferred from any
results in these studies, and this is an important difference to always keep in mind.
Slide 5
So how does this all relate to statistical analyses? Well, the main types of statistical analyses
you’ve learned so far fall into two broad types. Correlational -consisting of, for example,
correlations and regressions and tests of differences -like t--tests and ANOVAs. What is
crucial to understand is that both types of analyses can be done on both types of
design. That is, you are able to look at differences or associations in experimental studies,
you are able to look at both differences and associations in quasi-experimental studies, you
can look at both differences and associations in longitudinal studies, and you can look at
both differences and associations in cross--sectional studies - it all depends on the research
question you are asking, and the hypotheses you are proposing. If you got confused over the
last 15 seconds, please go back and listen to that again - research design and statistical
analyses are not the same thing, and analysing both differences and associations is
appropriate for the different research designs. What is crucial is your research question.
Slide 6
Let’s now consider the research project we will be using, which is the Emerging Adulthood
Measured at Multiple Institutions project made available on the Open Science Framework.
The important thing to note with this data is that it was measured at one time-point only and
there was no random allocation and no intervention. This means that the study design is a
cross-sectional design. Now depending on your aim and hypothesis you’ll choose either an
association/correlation or difference analysis.
Slide 7
However, just because you can compare differences using this data it doesn’t mean that you
can infer causality. Statistical analyses themselves don’t allow a researcher to infer
causality, it’s the research design that allows a researcher to do this and so because it’s a
cross--sectional design you’ll need to be careful not to infer causality when you discuss your
findings, even if you look at differences.
Slide 8
Given this is quite a limitation, why do we even do cross--sectional studies in psychology?
There are a number of good reasons. First, we might perform cross--sectional studies as a
first step to see if any associations between variables of interest exist, and which
relationships are the strongest. These can be then more selectively followed up in the often
more time consuming and more expensive experimental studies. Second, sometimes we
simply can’t manipulate a variable or randomly allocate people to groups, for instance, when
looking at the link between child abuse and educational attainment - it would not be ethical to
place random participants into the “child abuse” group and expect them to abuse their
children for the sake of the study. Clearly this is an absurd example, but hopefully you
understand the underlying point - sometimes manipulation is not possible or desirable. Third,
often non-experimental studies are more naturalistic. They’re not in a lab and they’re a
snapshot of natural activities rather than activities manipulated by experimental instructions.
Slide 9
Ok - it is finally time to look at the list of variables. And so here it is - as you can see, they
can be broken down into three categories which are represented by the three colours on this
slide: Emerging Adulthood variables, “Other” variables, and Demographic variables.
Slide 10
Let’s look at the Emerging adulthood variables first. These can are made up of two surveys:
the Markers of Emerging Adulthood (MOA), and the Inventory of Dimensions of Emerging
Adulthood. And each of these is made up of 4 subscales. There is no need to go into detail
here on what these represent as you will have an opportunity to look at each survey and
variable in much more detail as part of the activities provided to you on Moodle. For now, the
aim is to get a general picture.
Slide 11 and 12
On these next two slides, you can see a clearer representation of the “other” variables that
are available. These include mindfulness, belonging, self-efficacy, stress, wellbeing, and so
on.
Slide 12
Continuing on, we see there are also variables representing attitudes towards marriage,
social media use, and narcissism, all of which are comprised of subscales and measures.
Slide 13
Finally, on this slide you can see the list of demographic variables: gender, education,
siblings, ethnicity, and household income.
Slide 14
So now the question is, how are you going to pick which variables to include? Well, the truth
is, it is entirely up to you - remember, the point of having such a big list is to give you an
opportunity to follow your interests as much as allowable given the time constraints of of an
intensive deliver program such as the GDP.
There is only one specific (and very crucial!) requirement: to fulfill the assignment
requirements your research question you will need to have at least one emerging adulthood
variable and at least one other variable. You must also ensure that you only have one
outcome or dependent variable in your research question, and this is because we don’t
cover the statistical analyses for complex designs that have multiple dependent variables.
Finally, and this probably goes without saying, but the research question you propose needs
to be answerable using the research data we have available from the Emerging Adulthood
study and using the statistical analyses that we cover in this unit.
So the next question you probably have is how do you go about selecting the appropriate
statistical analysis? Well again, that all depends on the type of research question you ask,
and the hypotheses you propose. If you were interested in correlational research you might
select one of the emerging adulthood variables and look at whether a participant’s score can
be predicted by other variables. For example, whether a person’s degree of relational
maturity can be predicted from narcissism and interpersonal exploitativeness. This research
question fulfils the assignment requirements because it has at least one emerging adulthood
variable and two independent/predictor variables. Because we are looking at relationships
among variables, that is whether scores on one variable can be predicted from scores on
other variables, it makes sense to use a correlational analysis like multiple regression.
However, if you were interested in whether ‘is there a difference in scores on relational
maturity for narcissistic versus non narcissistic people’, then a difference test would be more
appropriate. But to meet the assignment requirements you need to have more than one
independent variable and so you might consider including gender as a moderating variable.
For example, by asking whether differences in relational maturity for narcissistic versus non
narcissistic people change depending on whether the person is male, female, or other. In
this example you are interested in interactions between narcissism and gender for
differences in relational maturity. If this is your research question, then the most appropriate
analysis would be a two--way independent ANOVA. Two--way because we have two factors
or two independent variables, one is narcissism and the other is gender. And independent
because people within each one of the resulting four groups (or cells as they are also called)
are all different. There are no repeated measures.
Slide 15
If you’re still having trouble working out which statistical test you should use, there are a
number of useful decision trees in textbooks,- for instance, in Andy Field’s textbook - this is
one version he has used. Yes, it does seem quite overwhelming when you first look at it, but
we can clean it up quite nicely, quite quickly, First of all, we only need to look at the part with
a single outcome/DV variable, so the bottom part can go…
Slide 16
Remember also that we need to also have at least 2 predictor/IVs, and so the parts that talk
about only 1 predictor can be removed too…
Slide 17
A quick clean-up and reformatting of what is left, and we get this…
Slide 18
Ok, much better! These are the statistical tests available to you - note that even in this table,
there are still a few analyses that we do not cover in this unit, and so it is advisable to stay
clear of them (you simply won’t have time to learn everything you need as part of the unit as
analyses that are outside the syllabus). Try to keep it as simple as allowable.
Looking closer at this table, you can only see that you first need to choose whether that
outcome or dependent variable is continuous or categorical. And then you need to work
whether the predictor or independent variables you have are continuous or categorical.
Finally you need to choose whether the groups are independent or repeated measures (or
both), - that is, are they the same people or different people in each of your groups (or a mix
of each). This will lead you to a decision on the type of analysis required. Don’t worry if this
is not immediately clear - you’ll be exploring and practising these sorts of designs and
decisions in your classes in the weeks to come.
Category Variables/Subscales Measured By
(Survey Name)
Emerging Role Transition Markers of Emerging Adulthood (MOA).
Adulthood
Items Derived from Arnett, 1997, 2001.
Normative Compliance Subscales as determined by Fosse and
Toyokawa (2016).
Relational Maturity
Independence
Experimentation/possibilit Inventory of Dimensions of Emerging
ies Adulthood- Short Form (IDEA-8) (Baggio,
Iglesias, Studer, & Gmel, 2015).
Negativity/instability
Identity exploration
Feeling “in-between”
Subjective Well- Total subjective well- Diener, E. (2000). Subjective well-being: The
being being score science of happiness and a proposal for a
national index. American Psychologist, 55(1),
34-43. doi:10.1037/0003-066X.55.1.34
Mindfulness Total mindfulness score Brown & Ryan (2003). The benefits of being
present: Mindfulness and its role in
psychological well-being. Journal of
Personality and Social Psychology, (84), 822-
848.
Belonging Total belonging score 10 items. Leary, M. R., Kelly, K. M., Cottrell,
C. A., & Schreindorfer, L. S. (2013). Individual
differences in the need to belong: Mapping
the nomological network. Journal of
Personality Assessment, 95(6), 610-624.
Self-efficacy Total self-efficacy score Generalized Self-efficacy (10 items;
Schwarzer & Jerusalem, 1995)
Perceived social Total perceived social 11 items
support support score
Social Media Use Maintaining existing Most items were adapted from Yang and
Scale connections Brown (2013, 2015)
Yang, C.-c., & Brown, B. B. (2013). Motives
for using Facebook, patterns of Facebook
activities, and late adolescents’ social
adjustment to college. Journal of Youth and
Adolescence, 42, 403-416.
Making new connections doi:10.1007/s10964-012-9836-x
Yang, C.-c., & Brown, B. B. (2015). Factors
involved in associations between Facebook
use and college adjustment: Social
competence, perceived usefulness, and use
patterns. Computers in Human Behavior, 46,
245-253. doi: 10.1016/[Link].2015.01.015
Information
Interpersonal Interpersonal These items were exploratory in nature and
Transgressions transgressions newly created for the EAMMI-2 study to
assess participants’ perceptions of the
frequency and perception of interpersonal
transgression they commit in their daily life.
Some of the items were adapted from items
used by McCullough, Emmons, Kilpatrick, and
Mooney (2003).
Narcissism Total Narcissism Score Narcissistic Personality Inventory-13
Gentile, B., Miller, J. D., Hoffman, B. J., Reidy,
D. E., Zeichner, A., & Campbell, W. K. (2013).
LA- leadership/authority A test of two brief measures of grandiose
narcissism: The Narcissistic Personality
subscale
Inventory–13 and the Narcissistic Personality
Inventory-16. Psychological Assessment,
25(4), 1120–1136.
GE-grandiose
exhibitionism subscale
EE-
entitlement/exploitativene
ss subscale
Interpersonal Total interpersonal Interpersonal Exploitativeness Scale (IES).
Exploitativeness exploitativeness score Brunell, A. B., Davis, M. S., Schley, D. R.,
Eng, A. L., van Dulmen, M. H. M., Wester, K.
L., & Flannery, D. J. (2013). A new measure
of interpersonal exploitativeness. Frontiers in
Psychology, 4.
Disability Identity Disability and Personal Opinions Questionnaire (POQ);
intrapersonal Bolton, B., & Brookings, J. (1998).
empowerment (11 items) Development of a measure of intrapersonal
empowerment. Rehabilitation Psychology,
43(2), 131-142. doi:10.1037/0090-
5550.43.2.131
Disability type
Health Somatic Total health score The Patient Health Questionnaire. Kroenke,
Symptoms K., Spitzer, R. L., Williams, J. B. W., & Lowe,
B. (2010). The patient health questionnaire
somatic, anxiety, and depressive symptoms
scale: a systematic review. General Hospital
Psychiatry, 32, 345-359. Doi:
10.1016/[Link].2010.03.006
Perceived Stress Total perceived stress Perceived stress scale. Cohen, S., Kamarck,
score T., Mermelstein, R. (1983). A global measure
of perceived stress. Journal of Health and
Social Behavior, 24, 385-396.
Marriage Central identities This item can be used in multiple ways. Any of
the four responses (each response is a
percentage when, added to the other three
percentages, equals 100%) can be used as a
continuous variable ranging from 0 to 100.
This strategy has been used for measuring
"marital centrality" by using the percentage
assigned to "marriage" as an isolated
variable. See: Willoughby, B. J., Hall, S. S., &
Goff, S. (2015). Marriage matters but how
much? Marital centrality among young adults.
The Journal of Psychology, 149, 796-817.
Marital Salience This item measures the perceived importance
of getting married. This is a single item that
was adapted from a longer scale (the item
was used for the sake of brevity given the
length of the overall EAMMI survey). For use
of full scale, see: Willoughby, B. J., Hall, S. S.,
& Goff, S. (2015). Marriage matters but how
much? Marital centrality among young adults.
The Journal of Psychology, 149, 796-817.
Marital Timing This item simply captures the age at which
one thinks people should marry. Besides age,
response options include "never" and "age
has no effect on marriage." See similar item
in: Willoughby, B. J., Hall, S. S., & Goff, S.
(2015). Marriage matters but how much?
Marital centrality among young adults. The
Journal of Psychology, 149, 796-817.
Marital Permanence This item measures the perceived inherent
permanence of marriage. This is a single item
that was adapted from a longer scale. See:
Willoughby, B. J., Hall, S. S., & Goff, S.
(2015). Marriage matters but how much?
Marital centrality among young adults. The
Journal of Psychology, 149, 796-817.
Demographic Gender -
variables
Level of educational -
attainment
Siblings -
Ethnicity -
Current household -
income