VALIDITY

 Validity is the accuracy and meaningfulness of inferences, which are

based on the research findings
 Or
 It is the degree to which results obtained from the analysis of the data
actually represent the phenomena understudy
 Validity is largely determined by the presence or absence of
systematic/non-random error. This is an error that has a consistent
boosting effect on the measuring instrument

E.g. A faculty weighing scale overestimating subject weight.

This error is non-random because the obtained readings are either always
above or below the true score i.e. always in one direction

There are 3 types of validity in data

i. Construct validity

This is the measure of the degree to which data obtained from an instrument
meaningfully and accurately represents or reflects a theoretical concept.

Example

 Does a score of 90 points on a reading test actually reflect the true

reading ability of a pupil or would score in biology, chem., phyc, and
language reflect ones attitude of being a good doctor or nurse
 This approach is often used where no criteria or domain of content is
generally accepted as an adequate measure of a concept. Concepts
management, intelligence, motivation etc. are all abstract,
hypothetical concepts. They cannot be directly observed but their
effects on the behavior of subjects can be observed
 To measure construct validity, there must be a theoretical framework
regarding the concept to be measured
 The measurements are consistent with theoretical expectations
 If the measurements are consistent with theoretical expectations, then
the data have construct validity and vice versa

To assess construct validity;

 Use two different instruments which measure the same concept

  Compute the validity coefficient

I.e. data is obtained simultaneously from the same sample

ii. Content validity

This is a measure of the degree to which data collected using a particular

instrument represents a specific domain of indicators or content of a
particular concept or it can be defined as the measure of the degree/extent
to which a measure represents all facets of a given construct

Example

 A test of arithmetic for standard 4 pupils would not yield content valid
data if items do not include all four operations : multiplication,
addition, subtraction and division
 The usual procedure in assessing the content validity of a measure is
to ask experts in the particular field

iii. Criterion –related validity

 This refers to the use of a measure in assessing subjects behavior in

specific situations. If an instrument purports to measure performance
in a job, the subjects who score high in the instrument must also
perform well in their jobs
 There are two types of criterion related validity

a) Predictive validity

Refers to the degree to which obtained data predict future behavior of

subjects

Job interviews should have high predictive validity to determine the future
performance of the candidate

b) Concurrent validity

This refers to the degree to which data are able to predict behavior of
subjects in the present but not in the future
Example:

A surgeon might use a measure to determine whether a patient with acute

abdomen is fit for surgery or not

INTERNAL AND EXTERNAL VALIDITY

Def.

Internal validity is the degree to which extraneous variables are controlled

in a study

If extraneous variables have not been controlled for we will not know the
observed effects on the dependent variable due to the independent variable
or to extraneous variables

When extraneous variables are controlled for, the effects can accurately be
attributed to independent variable. Therefore such research study is said to
have high internal validity.

Randomization is an important component of the internal validity of a study

External validity refers to the degree to which research findings can be

generalized to populations and environment outside the experimental
settings

If research findings are only applicable to the sample, the study findings are
not externally valid and hence not generalizable to other populations

Threats to internal validity

1. History

Some researches especially physical and biological extend over a period of

time. During this extended period, events occur which have influence on the
subject such that results may partly be due to extraneous factors such
factors influence research findings and hence internal validity is reduced

Occurrence of events that influence experimental units during the course of

the study is called history

Example
A researcher may want to determine the difference between two methods of
therapy given to patients suffering anxiety, the research is to extend for 3
months. During this period its possible some patients may be involved in
activities such as group support, hobbies, psychology classes, which keep
reduce anxiety. Therefore it will be difficult to determine the true difference
in these methods of therapy.

2. Maturation

This refers to biological or psychological process which occur among

subjects in a relatively short time and which influence research findings. This
can be more intellectually enlightened, motivated, fatigued, irritable or
discouraged

Maturation can be controlled by application of appropriate research designs

3. Instrumentation

This occurs when the measuring instrument is unreliable

It can also occur when research assistants are not consistent among
themselves in observing, scoring or assessing the characteristics under study

To avoid this, the researcher should develop accurate tools and standardize
data collection by holding a training session for all before data collection

4. Pretesting

In a pretest and post test experimental research, pretest sensitizes the

subjects and therefore they tend to perform better in post test. The better
performance in post test is not due to the treatment alone, but the pretest
sensitized them

A reliable method to avoid this problem is to use equivalent form tests

5. Statistical regression

This is a problem where the selection of subjects is based on their

performance in the pretest for example giving a pretest to students and
selecting the top scores to take in a study. On completion they are given a
post test. In this scenario, its likely that their mean score of post test will be
lower than the pretest mean score. If you selected the lower scores and did
the same as top scores, the mean score on post-test for them will be higher
than their mean score in the pretest

The explanation in the above situation is that extreme scores tend to have
the highest errors of measurements.

On retesting, external scores tend to regress towards the mean score

6. Attrition

Attrition or experimental mortality refers to the situation where many

subjects drop out of the study before the study is completed.

It may occur that only subjects with common characteristics are left in the
study. E.g. Less motivated and lower performers drop out and leave well
motivated and higher performers. This has some effect as biased sampling.

Attrition interferes with original randomness of the sample and this leads to
errors.

7. Differential selection

This occurs when subjects are systematically selected into a treatment

group. This selection interferes with randomness of the sample

For example, if subjects are selected for a study because they volunteered,
comparing such a group to a control group of non-volunteers introduces bias
and error in the results. Volunteers are more motivated and are more
interested unlike non-volunteers.

Differential selection is a factor when a researcher uses a whole, in fact

group as a sample

Commonly occurs in studies such as case study or where multi-stage

sampling technique has been used.

THREATS TO EXTERNAL VALIDITY

Various factors pose a threat to external validity or generalization of research

findings

These include:
i. Accessible and target population

Subjects are sampled from the accessible population. If the sample,

accessible and target population are similar on salient characteristics,

then generalization will be made. If accessible and target population are not
similar in salient characteristics, then generalization of the research findings
are only limited to the accessible population.

This occurs when random sampling is not applied and hence, inference to
any population is not possible.

ii. Control of extraneous variables

Research controls the subjects of extraneous variables by including them in

the study. E.g. if gender is a possible extraneous variable in a study, one may
consider only one level of the variable (male or female only). Strong control
may increase the internal validity of a study but it may, at the same time
reduce the generalizability of the results because such control may mean
taking a smaller sample. A balance is needed between internal and external
validity.

iii. Pre-test treatment interaction

In a pre-test and pilot-test study, subjects may score higher on the post- test
due to the treatment given and pre-test sensitization.

Pre-test treatment interaction makes the results only applicable or

generalizable to the target population

iv. Explicit description of the variables

Study variables must specifically be defining both conceptually and

operationally.

To increase the degree to which results are generalizable to other population,

a researcher must operationalize the variables in such a way that the
measures have a meaning outside the setting of the particular study.
Generalization of the research findings is made on the assumption that there
exists a universal language and a universal way of measuring a particular
variable.

v. Non-randomness of the sample

Results from studies, which are done with non-random groups can only be
generalized to those groups because of the unrepresentativeness of the
sample used. Randomness increases representation

vi. Multi-treatment inference

Occurs when two treatments are applied to the same group but at different
times. One drug may have a long duration and therefore its effect may
interfere with effect of the other. In such situation it’s difficult to determine
the difference(true difference) between the treatments

The solution to this problem is to lengthen the duration between the

treatments. Alternatively randomly assigns subjects to two sub-groups and
administer the different treatment to the subgroups.