0% found this document useful (0 votes)
2 views11 pages

Reliability, Validity, and SPSS Analysis

The document outlines various assignments related to reliability, validity, moderating and mediating effects in research, including methods for calculating these effects using SPSS and AMOS. It explains concepts such as construct reliability, types of validity, and the significance of moderating and mediating effects in management. Additionally, it discusses the importance of understanding level issues in research theory, measurement, and analysis.
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
2 views11 pages

Reliability, Validity, and SPSS Analysis

The document outlines various assignments related to reliability, validity, moderating and mediating effects in research, including methods for calculating these effects using SPSS and AMOS. It explains concepts such as construct reliability, types of validity, and the significance of moderating and mediating effects in management. Additionally, it discusses the importance of understanding level issues in research theory, measurement, and analysis.
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

MID TERM AND FINAL TERM ASSIGNMENT

Nurmirzaev Shohruh Mirzo Sobirjon ugli (22450132)


Assignment 1: Explain reliability and validity
1. Explain the meaning of reliability

- Construct reliability refers to consistency of results. For example, if we conduct a research


and use similar measurement (instrument) on the same subjects under same conditions, and
we get similar results, which means that our instrument is consistent.

2. Give a method for calculating the reliability

- One of the common method for establishing reliability of the constructs is Cronbach’s Alpha
using SPSS. Alpha reliability tells us that the instrument we used in our research is
consistent. In SPSS, we conduct a reliability analysis going to SPSS menu and using the path
(analyze scale reliability analysis) and moving the items of one variable into ‘Items’
box.

- Interpretation of the Cronbach’s Alpha is accordingly:


α>0.9: Excellent
α>0.8: Very good
α>0.7: Good
α>0.6: Fair
α<0.6: Reliability issues to be solved

3. Explain the meaning of validity

- Validity refers to the extent to which a measurement tool or instrument accurately measures
what it is intended to measure. It is a crucial concept in research to make us sure that the
conclusions drawn are meaningful and accurate.

4. Explain each type of validity

There are several types of validity that help us to assess the accuracy and relevance of a
measurement instrument.
- Content validity (assessing if an instrument fully represents the construct it is intended to
measure. It ensures that all facets of the construct are covered)
- Criterion validity (assessing if a test correlates with an external criterion that it is supposed to
be related to)
- Construct validity (checking if a test measures the theoretical construct it is intended to
measure)
- Face validity (checking whether a test "appears" to measure what it claims to measure)

5. Explain each type of construct validity and the way to check each type

Types of Construct validity are:


- Convergent validity;
- Discriminant validity;
- Convergent validity refers to how well a target test matches tests of theoretically related
constructs. It is assessed through AVE (average variance extracted). It means how much
variance in one variable is extracted through with its indicators. If AVE value is greater
than .50, it means that convergent validity of a particular variable is supported.

- Discriminant validity refers to assessing whether a target test produces results that differ from
tests of unrelated constructs. It can be supported through Fornell & Larcker Criterion, Cross
Loadings, and HTMT (Heterotrait-Monotrait Ratio).

Most commonly used:


- Fornell & Larcker Criterion – square root of AVE value of one variable should be greater
than its correlation with any other variables.
- Cross Loadings – using EFA or CFA and comparing the factor loadings of each item with
higher loadings with their corresponding construct and low loadings with unrelated construct.

Assignment 2: Calculating moderating effect with SPSS


1. Explain the meaning of moderating effect and the practical utility in the actual
management.

- In research, we test a moderating effect when the relationship between independent and
dependent variables strengthens, weakens or overall changes with the presence of a third
variable, called the moderator.

- Understanding a moderating effect in real-life practice can help managers and decision-
makers to come up with practical improvements in management. For example, it helps
managers understand that in management, the relationships are not always simple or
straightforward, rather complex. Moreover, in order to improve employee development or
factors for organizational success, managers should take care of and understand the actual
meaning of the moderating effect.

2. Moderation analysis

Following Aiken & West’s(1991) ways for the analysis of moderating effects, we conducted
the analysis of moderating effect of Leaders’ Ethical Leadership in the relationship between
Leaders’ Innovative Behavior and Followers’ Innovative Behavior. Gender and age were
taken as control variables. The original data was analyzed by mean centering. In order to
conduct a moderation analysis using SPSS, from menu we followed the path
(analyzeregressionlinear) and entered DV into Dependent variable and control variables,
mean centered IV and mean centered MV with interaction term of them into Independent(s)
box.
Coefficientsa
Unstandardized Standardized
Coefficients Coefficients Collinearity Statistics

Model B Std. Error Beta t Sig. Tolerance VIF


1 (Constant)
3.692 .110 33.467 .000

gender_dummy_variabl -.003 .084 -.002 -.039 .969 .926 1.079


e
age_variable
.024 .034 .038 .717 .474 .903 1.107

Mean_Cen_ld_ib
.269 .081 .314 3.316 .001 .288 3.477

Mean_Cen_ld_el
.316 .082 .355 3.850 .000 .303 3.298

Interaction_LDIBxLDEL
.189 .043 .245 4.433 .000 .846 1.182

a. Dependent Variable: followers' innovative behavior which were rated by followers themselves

As shown in the table below, the result of interaction term is significant and positive (b=.189,
p<.001). This finding indicates that the relationship between Leaders’ Innovative Behavior
and Followers’ Innovative Behavior was influenced by Leaders’ Ethical Leadership. And the
multicollinearity issue is not a problem in our analysis since the results in the table show VIF
values much below than 10.

According to simple slope analysis below, Leaders’ Ethical Leadership strengthens the
positive relationship between Leaders’ Innovative Behavior and Followers’ Innovative
Behavior.

Assignment 3: Conducting analysis of the mediating effect with SPSS


1. Explain the meaning of mediating effect and the practical utility in the actual
management.

Mediating effect can be identified when the third variable called mediator (intervening variable)
has the influence in the relationship between independent and dependent variables. It means that
an independent variable has an indirect effect on the dependent variable through a mediator.

The importance of mediating effect in the actual management is that mediator explains the
mechanism of the relationships between variables. The relationships in the management sphere is
much more complex than it seems. Mediating effect can help identify why and how particular
relationships occur in the management, allowing managers and leaders to make more effective
and applicable decisions. For example, in strategy refinement or resource optimization, managers
should pay more attention on the mediating variables to improve organizational outcomes.
2. Mediation analysis

We conducted Baron & Kenny’s (1986) 3-step method in order to analyze mediating effect of
Leaders’ Ethical Leadership (MV) in the relationship between Leaders’ Innovative Behavior (IV)
and Followers’ Innovative Behavior (DV). We took gender and age as control variables in our
study.

In step 1, we examined the direct effect of Leaders’ Innovative Behavior (IV) to Leaders’ Ethical
Leadership (MV). We included control variables and chose Leaders’ Ethical Leadership as a
dependent variable. The results show that Leaders’ Innovative Behavior has positive and
significant effect on the Leaders’ Ethical Leadership (B=.812 p<.001). (Table 1)

In step 2, we examined the link between Leaders’ Innovative Behavior (IV) and Followers’
Innovative Behavior (DV). Here, we chose Followers’ Innovative Behavior as a dependent
variable and put independent and control variables into independent(s) box. The results revealed
significant and positive link between independent and dependent variables (B=.447 p<.001).
(Table 2)

In step 3, we added both independent and mediating variables with control variables into
Independent(s) box while Followers’ Innovative Behavior was chosen as dependent variable. The
results indicated that the value of Leaders’ Innovative Behavior (IV) remains significant, yet it
decreases (B=.208 p<.05) whereas mediating variable’s value continues to remain significant
(B=.295 p<.005), indicating partial mediation by Leaders’ Ethical Leadership (MV). (Table 3)

According to Baron & Kenny (1986), additional steps are necessary to assess the extent and
statistical significance of the mediating effect. They recommend utilizing the Sobel test, which
involves computing both paths of a and b, and then dividing by the standard error of the
intersection to obtain a Z statistic. The results of the Sobel test indicate that the indirect effect of
Leaders’ Ethical Leadership is statistically significant (z=3.43, p<0.01). (Table 4)

(Table 1) Coefficientsa
Unstandardized Standardized
Coefficients Coefficients
Model B Std. Error Beta t Sig.
1 (Constant) .900 .140 6.422 .000
gender_dummy_variable -.035 .064 -.019 -.537 .592
age_variable -.029 .026 -.041 -1.121 .263
leaders' innovative behavior which
.812 .034 .842 23.594 .000
were rated by followers
a. Dependent Variable: leaders' ethical leadership which were rated by followers

(Table 2) Coefficientsa
Unstandardized Standardized
Coefficients Coefficients
Model B Std. Error Beta t Sig.
1 (Constant) 2.166 .192 11.273 .000
gender_dummy_variable -.055 .088 -.034 -.618 .537
age_variable .018 .036 .028 .502 .616
leaders' innovative behavior
.447 .047 .521 9.481 .000
which were rated by followers
a. Dependent Variable: followers' innovative behavior which were rated by followers themselves

(Table 3) Coefficientsa

Unstandardized Standardized
Coefficients Coefficients
Model B Std. Error Beta t Sig.
1 (Constant) 1.901 .203 9.365 .000
gender_dummy_variable -.044 .087 -.028 -.513 .609
age_variable .027 .035 .042 .758 .449
leaders' innovative behavior
.208 .083 .242 2.506 .013
which were rated by followers
leaders' ethical leadership which
were rated by followers .295 .085 .331 3.476 .001

a. Dependent Variable: followers' innovative behavior which were rated by followers themselves

(Table 4) Sobel test results

Assignment 4. Use AMOS to verify the mediating effect


1. Explain the meaning of structural equation model

Structural equation model is a statistical method that examines the relationships among
numerous variables in a simultaneous way and is considered as a family of related
statistical technique.

The usage of structural equation model will often take a confirmatory approach in which
the researcher has proposed a ‘model’ of relationships between variables of interest and
examines whether the observed data will provide evidence of directionality and
significance of the relationships.

2. Mediation analysis with SEM on AMOS

We conducted structural equation model (SEM) in order to examine mediating effect of


Leaders’ Ethical Leadership (MV) in the relationship between Leaders’ Innovative
Behavior (IV) and Followers’ Innovative Behavior (DV) and took gender and age as
control variables.
The indirect effect of Leaders’ Innovative Behavior (IV) and Followers’ Innovative
Behavior (DV) was .240 (a*b (.812*.295)) while the direct effect of IV to DV
showed .208.

Next, we looked at the same output for the confidence intervals generated by our
bootstrap. The lower bound confidence interval was .095 and the upper bound was .396.
Since this confidence interval did not cross zero, we know that the indirect effect is
significant.
Furthermore, we assessed what type of mediation is present, namely full or partial
mediation. To accomplish this, we need to examine the C’ path or the direct path, from
Leaders’ Innovative Behavior (IV) to Followers’ Innovative Behavior (DV) in the
“Estimates” link in the output.

We can see that Leaders’ Innovative Behavior has a significant relationship with
Followers’ Innovative Behavior (b=.208 p<.05). This means that the influence of
Leaders’ Innovative Behavior on Followers’ Innovative Behavior is partially mediated
through the construct of Leaders’ Ethical Leadership.

Finally, we conducted a Sobel test using the analysis results. The results of the Sobel test
indicated that the indirect effect is statistically significant (z=3.47, p<0.01).

Assignment 5. Describe ‘mediated moderating effects’


1. Explain the meaning of ‘mediated moderation effect’

Mediated moderation occurs when the interaction effect of the independent variable and
moderator influences the mediator, which in turn affects the dependent variable. We can
identify whether there is a mediated moderation in a research model through 3 steps:
Step 1: Theorizing about the role of the moderator between independent and dependent
variable
Step 2: Theorizing about the role of the moderator between independent and mediating
variables
Step 3: Theorizing about the effect of interaction term of independent and moderator
variables on dependent variable through a mediator.
The key idea behind mediated moderation is that the interaction effect is mediated by the
mediator.

2. Explain the meaning of ‘moderated mediation effect’

Moderated mediation occurs when the indirect effect of the independent variable on the
dependent variable through the mediator demonstrate different influence by different levels
of the moderator. Here, we are not only trying to determine if mediation is present, but also
we are attempting to see if a moderator is changing the strength of the indirect effect. The key
idea behind moderated mediation effect is that the mediation process is moderated by another
variable.

We can use either Hayes Process Macro or structural equation modeling (SEM) to test both
effects.
The example model of both is below:

Moderator Mediator

Predictor Outcome

3. Explain the meaning of Process Macro

Process is an observed variable OLS (ordinary least square) and logistic regression path
analysis modelling tool. It is developed by Dr. Andrew F. Hayes.

It is widely used through social, business, and health sciences for estimating direct and
indirect effects in single and multiple mediator models (parallel and serial), two and three
way interactions in moderation models along with simple slopes and regions of significance
for probing interactions, and conditional indirect effects in moderated mediation models with
a single or multiple mediators and moderators.

The Process Macro focuses on conditional process, mediation and moderation analysis.

-Conditional process analysis is a modelling strategy undertaken with the goal of describing
the conditional and contingent nature of the mechanisms by which a variable transmits its
effects on another, and testing hypotheses about such contingent effects.
-Mediation analysis (process analysis) is used to quantify and examine the direct and indirect
pathways through which an antecedent variable X transmits its effects on a consequent
variable Y via an intermediary M.

-Moderation analysis used to examine how the effects of an antecedent X on a consequent Y


depends on a third moderating variable M (“interaction”).

Assignment 6. Level issue and HLM


1. Explain the level issue using the terms 'level of theory,' 'level of measure,' and
'level of analysis‘

The issue of levels can arise in a study when there is a mismatch or misalignment
between the level of theory, the level of measurement, and the level of analysis.

Level of Theory
The level of theory refers to the unit of focus at which a theory provides explanations or
makes predictions.

Individual-Level Theory: Focuses on understanding behaviors or outcomes at the


individual level, such as personality traits or motivation.
Group-Level Theory: Examines phenomena occurring within groups or teams, such as
team dynamics or shared culture.
Organizational-Level Theory: Addresses phenomena at the organizational scale, such as
strategy or overall organizational performance.

Level of Measure

The level of measure refers to the unit at which data is gathered or variables are
evaluated.

Individual Measures: Data collected at the individual level, such as employee surveys on
attitudes.
Group Measures: Aggregated data, like the average satisfaction scores of a team.
Organizational Measures: Metrics collected at the organizational level, such as overall
profitability

Level of Analysis

The level of analysis refers to the unit at which data is examined to make conclusions and
should correspond to the level of theory.

For theories focused on individual-level outcomes, the analysis should be conducted


using individual-level data. Similarly, when the theory targets group-level outcomes, the
analysis should involve aggregated or summarized data at the group level.

2. If a variable is at the team level but measured through an individual-level survey,


explain how to make it a team-level variable using the term 'aggregation‘
To make a team-level variable out of data collected through an individual-level survey,
we can use a process called aggregation. This involves combining or summarizing
individual-level responses into a single value that represents the team as a whole.

We have to consider some steps to aggregate individual-level variable into the team-level
one. We have to collect individual-level data through administering the survey to
individuals within each team. Next, we need to justify aggregation and aggregate
responses.

3. Explain how to justify whether aggregation is validated.

To justify whether aggregation is validated, it is required to make both theoretical


justification and empirical evidence.

Theoretical Justification means that we have to argue that the construct being measured
conceptually exists at the team level and should be aggregated.

Empirical Validation can be done through measures of within-team agreement and


within-team variance.

1. Measures of Within-Team Agreement means to assess whether team members’


responses are similar enough to justify aggregation. We can conduct:

rWG (Within-Group Agreement Index): Measures the degree of agreement among team
members on a single variable.
A higher value indicates sufficient agreement within the team to justify aggregation (e.g.,
rWG ≥ 0.70).

Standard Deviation (SD): calculating the SD of individual responses within a team.


Smaller SDs indicate greater agreement.

2. Measures of Between-Team Variance refers to determining whether enough variability


exists between teams (rather than within teams) to treat the construct as a team-level
variable.

Intraclass Correlation Coefficients (ICC):

ICC(1): Reflects the proportion of variance in individual responses that can be attributed
to differences between teams.
A high ICC(1) suggests that team membership has a meaningful effect on individual
responses (e.g., ≥ 0.10).
ICC(2): Measures the reliability of the team-level mean. Higher ICC(2) values indicate
that the team-level measure is stable and reliable (e.g., ≥ 0.70).

One-Way ANOVA:

Tests whether there are significant differences between teams in their mean responses.
If the F-test is significant, it supports the argument that teams differ in the construct being
measured, justifying aggregation.

3. Explain how to conduct cross-level analysis.


Cross-level analysis can be conducted in case of having variables that measured at
different levels of analysis. They can be individual, group or organizational levels.
Firstly, we need to clearly understand our research items and levels of analysis. After
collecting data, we have to aggregate them.
Next, we can use HLM (hierarchical linear modelling) that nested structure of data can be
used. There are techniques like ‘random intercepts model’, ‘random slopes model’ and
cross-level interaction models.
Then, we can utilize tool intraclass correlation coefficients (ICCs) in order to assess how
much variance in the outcomes stands between groups.
Lastly, we should interpret the findings and validate the model.

You might also like