ASSIGNMENT No.

1
COURSE CODE - 8602

CODE NAME - EDUCATIONAL ASSESSMENT

AND EVALUATION

NAME - MUHAMMAD AFZAL

STUDENT ID - 0000758015

ADMISSION - SPRING 2024

 --------------------------------------------
QUESTION NO. 1
--------------------------------------------
Explain the principles of classroom Assessment in detail?
--------------------------------------------
ANSWER
--------------------------------------------

Definition of Assessment
Kizlik (2011) defines assessment as the process by which information is obtained regarding some
known goal or objective. Assessment is a broad term that includes testing. For example, the teacher
can assess the knowledge of the English language through a test and the language ability of the
students through any other tool, such as an oral quiz or a presentation. Based on this view, we can
say that every test is an assessment, but every assessment is not a test. The term "assessment" is
derived from the Latin word "assidere" which means "to sit beside". Unlike testing, the tone of the
assessment term is non-threatening, suggesting a partnership based on mutual trust and
understanding. This emphasizes that there should be a positive rather than a negative relationship
between assessment and the teaching and learning process in schools. In the broadest sense,
assessment refers to children's progress and achievement. In a comprehensive and specific way,
classroom assessment can be defined as: the process of collecting, recording, interpreting, using
and communicating information about a child's progress and achievements during the
development of knowledge, concepts, skills and attitudes. (NCCA, 2004)

In short, assessment involves much more than testing. It is an ongoing process that involves many
formal and informal activities designed to monitor and improve teaching and learning.
Principles of Classroom Assessment
ACCORDING TO HAMIDI
Hamidi (2010) described following principles of classroom assessment.
1. Assessment should be formative.
Classroom assessment should be conducted regularly to provide information on ongoing teaching
and learning. It should be formative because it refers to the shaping of a concept or process. To be
formative, assessment refers to how the learner develops or forms. So it should be for learning. In
other words, it has a key role in "informing the teacher about the extent to which students as a
group and the extent to which individuals within that group have understood what they have
learned or still need to learn, as well as the appropriateness of their class. Teachers use it to
determine how well students have mastered what they were supposed to learn, so classroom
assessment must reach its full formative potential if a teacher is to be truly effective in teaching.
2. Should determine planning.
Classroom assessment should help teachers plan future work. First, teachers should determine the
purposes of assessment—that is, specify the kinds of decisions they want teachers to make as a
result of the assessment. Second, they should collect information related to the decisions they have
made. Next, they interpret the information gathered – that is, it must be put into context before it
is meaningful. Ultimately, they should make final or professional decisions. Plans are a vehicle for
implementing learning objectives that are put into practice as classroom assessments to achieve
real outcomes.
3. Assessment should serve teaching.
Classroom assessment serves instruction by providing feedback on student learning that would
make the next instructional activity more effective, in a positive, upward direction. Therefore,
assessment must be an integral part of teaching. Assessment appears to drive instruction by forcing
teachers to teach what will be assessed. Teaching includes assessment; that is, whenever a student
answers a question, offers a comment, or tries a new word or structure, the teacher subconsciously
evaluates the student's performance. So when they teach, they also assess. A good teacher never
stops evaluating students, whether those evaluations are random or intentional.
4. Assessment should serve learning.
Classroom assessment is also an integral part of the learning process. The ways in which students
are assessed and assessed strongly influence the way they study and learn. It is the process of
finding out who students are, what their abilities are, what they need to know and how learning
will affect them. In the assessment, the student is simply informed of how well or poorly they have
performed. It can encourage students to set goals for themselves. Assessment and learning are seen
as inextricably linked rather than separate processes as they interact. Learning itself is meaningless
without assessment and vice versa.
5. Assessment should be curriculum-driven.
Classroom assessment should be the servant, not the master, of the curriculum.
Assessment experts consider it an integral part of the entire curriculum cycle.
Therefore, decisions about how to assess students must be considered from the very beginning of
curriculum development or course planning.
6. Evaluation should be interactive.
Students should be active in selecting content for assessment. It provides a context for learning as
the meaning and purpose of learning and engages students in social interaction to develop oral and
written language and social skills. Assessment and learning are inextricably linked and are not
separate processes. Effective assessment is not a process carried out by one person, eg a teacher,
another, a pupil, it is understood as a two-way process involving interaction between both parties.
Thus, assessment should be viewed as an interactive process that involves both teacher and student
in monitoring student performance.
7. Assessment should be student-centered.
Because learner-centered teaching methods are fundamentally concerned with the learner's needs,
students are encouraged to take more responsibility for their own learning and to choose their own
learning goals and projects. Therefore, in student-centered assessment, they are actively involved
in the assessment process. Engaging students in aspects of classroom assessment minimizes
learning anxiety and leads to greater student motivation.
8. Assessment should be diagnostic.
Classroom assessment is diagnostic because teachers use it to identify students' strengths and
weaknesses during instruction. They also identify learning difficulties. If the purpose of
assessment is to provide diagnostic feedback, this feedback must be provided in a form – either
verbal or written – that is understandable and usable by [Link] of assessment is to
provide diagnostic feedback, then this feedback needs to be provided in a form – either verbal or
written – that is for learners to understand and use.
9. Assessment should be exposed to students.
Teachers are supposed to enlighten students about accurate assessment information.
In other words, it should be transparent to students. They need to know when the
assessments take place, what they cover in terms of skills and materials, what the
assessments are worth and when they can get their results and when the results will
be used. They must also be aware of why they are being evaluated because they are
part of the evaluation process. Because assessment is part of the learning process, it
should be done with the students, not at them. It is also important to provide an
assessment schedule before classes begin.
10. Evaluation should not be judgmental.
In classroom assessment, everything focuses on learning, which results from a
number of factors such as student needs, student motivation, teaching style, time on
task, intensity of study, background knowledge, course objectives, etc. So there is
no praise or blame for a particular outcome learning. Teachers should not take any
position to determine who did better and who failed. Assessment should allow
students to have reasonable opportunities to demonstrate their expertise without
facing barriers.
11. Evaluation should develop mutual understanding.
Mutual understanding occurs when two people come to a similar sense of reality. In
second language learning, this understanding requires a linguistic environment in
which the teacher and students interact with each other based on assessment
objectives. Therefore, assessment has the ability to create a new picture of the world
by having individuals share their thoughts useful in the learning process. When
learning occurs, it is certainly the result of a shared understanding between teacher
and students.
12. Assessment should lead to learner autonomy.
Autonomy is the principle whereby students are placed in a state where they make
their own decisions when learning languages. They take maximum responsibility for
what they learn and how they learn it. Autonomous learning occurs when students
move from teacher assessment to self-assessment. This requires teachers to
encourage students to reflect on their own learning, to assess their own strengths and
weaknesses, and to identify their own learning goals. Teachers also need to help
students develop their self-regulatory and accomplished cognitive strategies.
Autonomy is a construct to be fostered in students, not taught by teachers.
13. Assessment should include reflective teaching.
Reflective teaching is a teaching approach in which teachers should develop their
understanding of teaching (quality) based on data/information obtained and gathered
through critical reflection on their teaching experiences. This information can be
collected through formative assessment (i.e. using various methods and tools such
as class quizzes, questionnaires, surveys, field notes, peer feedback, classroom
ethnography, observation notes, etc.) and summative assessment (i.e. various types
of achievements tests at the end of the semester).
According to AERA
According to Standards for Educational and Psychological Testing (AERA, 2014) [1],
Validity, reliability, and fairness are critical attributes of evaluation quality. Mathematical
measures of these attributes have been proposed and effectively used in large-scale assessment
(LSA) quality estimation. These measurements are mostly statistical in nature, so the availability
of a large data set is essential, as is the case with LSA. Unfortunately, statistical measures are not
applicable to classroom assessments because the population size is limited. In this article, the
importance of Validity, Reliability and Fairness for the classroom context is discussed and guiding
principles for qualitative assessment are proposed. These principles, while rough, can nevertheless
provide useful insights for classroom teachers willing to assess the quality of the assessments they
design. The importance of validity, reliability, and equity in a classroom context.
1. Validity
AERA [1] defines Validity as the extent to which evidence and theory support interpretations of
test scores for proposed test uses. This definition emphasizes the use of scores to denote
assessments that are primarily conducted to assign scores to students based on their level of
knowledge of learning outcomes. However, the purpose of classroom assessment, which is mostly
formative in nature, is to provide feedback to the entire teaching and learning process. Thus,
validity here is related to the accuracy and quality of inferences about student learning outcomes.
Valid assessment should be aligned with the learning outcome. The cognitive ability for the content
domain assessed by the assessment item should be consistent with the cognitive level and content
domain as indicated in the learning outcome to ensure validity. For example, for the learning
outcome - presents collected information in tables and bar graphs and draws conclusions, example
1 is more valid than example 2 because the assessment item assesses both the cognitive ability of
analysis (draws conclusions) and content domains. (tables and bar graphs) as stated in the
statement of learning outcomes.
Validity is also compromised when irrelevant material is used in the item stem. For example: items
containing long and complex sentences; items that contain difficult vocabulary; unnecessarily
complex illustrations and diagrams; items in a math or science assessment that require a lot of
reading, etc. These factors should be avoided to make items more or less difficult, as they do not
contribute to the assessment of students' skills and knowledge to achieve the desired learning
outcome.
AERA reliability [1] defines reliability in terms of consistency over replications of a test
procedure. Classroom assessment consists of a series of informal and formal assessments including
observations, projects, portfolios, debate and science exhibition. Reliability in the classroom
context is understood in terms of the properties of consistency and uniformity. Consistency is
ensured when student performance is consistent across different assessment methods. Also,
sharing clearly defined assessment expectations in the form of rubrics ensures consistency in
student performance and uniformity in teacher evaluation.
Fairness AERA [1] defines Fairness as the ability to respond to individual characteristics and test
contexts so that test results provide valid interpretations for the intended use. In the context of
classroom assessment, fair assessment should firstly provide students with the opportunity to best
demonstrate how they have achieved learning outcomes through a variety of assessment methods
without explicit or implicit teacher bias. Second, for assessment to be fair, students should be
aware of the learning outcomes, the content of the assessment, and its use. Third, topics or texts
that are not accessible to students; rating items that favor a particular gender, group, community,
religion or state should not be used. In practice, indicators of validity, reliability and fairness
overlap. Ensuring one attribute also guarantees another attribute and vice versa. For example,
while multiple assessment methods need to be used to ensure reliability and fairness, they also
provide accurate information about student learning for learning outcomes that cannot be measured
with just one assessment method. Also, there are no absolute values for validity, reliability, and
fairness. In qualitative assessment, the values range from less to more instead of 0 to 1. Meeting
the four important guiding principles questions will help the classroom teacher to ensure the
validity, reliability and fairness of the classroom assessment and thus be able to make better use
of the assessment results. effectively.
The key to quality classroom assessment
Why am I rating? Classroom assessment is a process used to collect, analyze, and use
evidence of student learning for a variety of purposes, including diagnosing student strengths and
weaknesses, tracking student progress toward required proficiency levels, assigning grades, and
providing feedback to parents. Outside the classroom, accountability assessments are conducted
to help the education system introspect on its own health and effectiveness. Stiggins [2] states that
inept assessment can lead to ineffective decisions about student learning.
What am I assessing?
Learning outcomes are like a navigation tool for both students and teachers. He leads both
pedagogy and assessment. They indicate what a student will be able to do at the end of a unit of
instruction by breaking down broader learning goals such as problem solving and critical thinking
into more measurable and observable behaviors for each class. For example, critical thinking in
language for a 4th grader is the ability to ask questions about texts read, while for a 5th grader it
is the ability to draw conclusions from the text read. A good understanding of these outcomes helps
the teacher design accurate assessments to assess the skills students need to achieve. These results
should also be communicated to students to ensure fair assessment.

How will I rate? High-quality assessments need to be designed to assess the achievement of
learning outcomes. The correct assessment method should be determined based on the nature of
the learning outcomes. For example, for learning outcomes - Construct a Newton's Color Disc
using materials from the environment and explain their work, project assessment will provide the
most valid, reliable and faithful information on how students have achieved learning outcomes.
High-quality assessment items should be designed that are substantively and conceptually correct
and free of sensitivity issues. Comprehensive rubrics should be developed in accordance with the
cognitive level of learning outcomes and shared with students to ensure fair assessment.

How will I communicate the results of the assessment? The purpose of communicating
assessment results that are used in classrooms is not only to report but also to support learning.
Students need descriptive feedback focused on strengths and weaknesses regarding specific
misconceptions and learning gaps that reveal how they can improve next time and what scaffolding
the teacher needs to provide. Results can be communicated through words, pictures, illustrations,
examples, and many other means that provide a description of student performance.

Summary

The importance of quality attributes and the main principles will help the classroom teacher to
enable better learning for students by
• providing information about what they should be able to do at the end of the learning unit (the
learning outcomes they should achieve), their responsibilities and the criteria used to assess their
performance.

• evaluating them using valid, reliable and fair assessments.

• facilitate their learning outcomes and engage and challenge them at the right level.

• providing constructive feedback and the necessary scaffolding to enable them to perform better.

--------------------------------------------
QUESTION NO. 2
--------------------------------------------
Critically analyze the role of Bloom's taxonomy of educational objectives in
preparing tests.
--------------------------------------------
ANSWER
--------------------------------------------
Bloom's taxonomy helps teachers understand the learning objectives in the classroom. It
guides them to change the complexity of the questions and helps students reach higher
levels of the hierarchy. It also helps develop critical thinking among teachers.

The 6 levels of Bloom's taxonomy and how to use them when creating quizzes

To effectively check students' knowledge, Bloom suggests creating six types of questions.
Each type corresponds to a certain level of taxonomy. For example, you can first check
how well the person learned the material, then find out what they understood, find out
what knowledge they can apply, and so on.

If the student answers the first type of questions correctly, he gets access to the questions
of the next level. If the student makes a mistake, he must revise the material and take the
test again.

Example. When answering the first group of questions, James gave 4 correct answers
out of 10. The main purpose of these questions was to check how well he had learned
the course material. Since Jack failed more than half of the questions, there is no point in
testing him further and testing his practical skills. It will be best if the test automatically
sends him to the beginning of the course for retraining or provides additional information
on problematic questions.

Quick tip. You can set up branching for your courses and quizzes using the iSpring Suite
authoring toolset. If the student answers incorrectly, he can be automatically redirected
to the theory block.

Read this article on branching scenarios to learn how you can easily create non-linear
courses and tests with iSpring.

Bloom's Taxonomy Level 1: Test Knowledge

What does Bloom's theory say? In the "Knowledge" phase, it is important to check how
well the student has learned the new information: specific facts, dates and terms. If they
know the answers, you can test them further.

How to write test questions. Use verbs like 'define', 'describe', 'name', 'select', 'show',
'give a definition' or 'choose the correct answer' because they are specific and the student
clearly understands what is expected of them. in this matter.

Bloom's Taxonomy Level 2: Check Comprehension

What does Bloom's theory say? At the "comprehension" level, the test helps verify that
the student can go beyond basic memorization and understand the meaning and
correlation of key concepts.

How to write test questions. Use verbs like 'explain', 'find', 'define', 'compare',
'paraphrase' or 'generalise'. To answer such questions, a student must not only know the
terms but also understand the concept.

Bloom's Taxonomy Level 3: Test Knowledge in Practice

What does Bloom's theory say? The third type of question focuses on Applications.
They help to test the student's ability to apply the acquired knowledge in practice.
How to write test questions. These questions should begin with verbs such as "use,"
"decide," "calculate," "apply," "transform," "change," or "create." They help simulate a
real-life situation and nudge students to show they can apply the information they've
learned.

Bloom's Taxonomy Levels 4 and 5: Determine if the student can improvise

What does Bloom's theory say? In some cases, it is impossible to work strictly
according to a script or instructions. To answer the question, the student must analyze
the situation. This skill is tested at the "analysis" and "synthesis" levels of Bloom's
taxonomy.

How to write test questions. Use the following action verbs: "compare," "contrast,"
"highlight," "contrast," "sort," "find," "summarize," and "group." Such questions force the
student to come up with his own solution instead of looking for ready-made answers.

Bloom's Taxonomy Level 6: Check that the student can make decisions

What does Bloom's theory say? At the "assessment" level, you need to check
whether the student is sufficiently immersed in the topic and can make decisions based
on new information. If you want to assess management skills, this type of question is a
must.

How to write test questions. These questions should start with the words "conclude",
"prove", "justify", "assess", "check", "evaluate" and recommend. At this level, it is essential
to use "essay" type questions. This way, the student will have no guide to look at - he will
only be able to rely on his own knowledge and skills.

To take advantage of Bloom's Taxonomy, make sure the questions in your test cover all
6 levels, from knowledge, understanding and application to analysis, synthesis and
evaluation. In this way, you will be able to effectively check the knowledge and skills of
the student.
Use specific action verbs in the questions at each level. They will help you assess the
depth of knowledge of your students. If a student can't deal with a certain type of question,
ask them to review the material and take the test again.

--------------------------------------------
QUESTION NO. 3

--------------------------------------------

What is standardized testing? Explain the conditions of standardized testing

with appropriate examples.

--------------------------------------------
ANSWER
--------------------------------------------

STANDARDIZED TESTING

Standardized tests are instruments designed to measure student performance relative to everyone
else taking the same test. A standardized test is a test that is administered and scored in a consistent
or "standard" way. Standardized tests are designed so that questions, administration conditions,
scoring procedures, and interpretation are consistent and are administered and scored in a
predetermined standard manner. Any test in which the same test is given the same way to all test
takers is a standardized test. Standardized tests need not be high-stakes, timed, or multiple-choice
tests. The opposite of a standardized test is a non-standardized test. Nonstandardized testing gives
significantly different tests to different test takers, or gives the same test under significantly
different conditions (e.g., one group is given much less time to complete the test than another
group), or is scored differently (e.g., the same answer counts as correct for one student , but for
another student as bad). Standardized testing has been criticized by psychologists, educators and
parents. Criticisms of academic testing often focus on linguistic bias against minorities, testing
methods that may not work for all types of students, and negative reinforcement of lower-achieving
students.
(a) Types of standardized testing

There are two types of standardized tests: norm-referenced and criterion-referenced. Norm-
referenced testing measures performance relative to all other students taking the same test. It lets
you know how well the student did compared to the rest of the test population. For example, if a
student is ranked in the 86th percentile, that means they did better than 86 percent of the other test
takers. This type of testing is the most common among standardized testing. Criterion-based testing
measures factual knowledge about a defined set of material. Examples of this type of testing are
multiple-choice tests that people take to get a license or a fractions test. In addition to the two main
categories of standardized tests, these tests can be further divided into performance tests or aptitude
tests. Achievement tests are assessments of what has already been learned in a certain area, while
aptitude tests are assessments of abilities or skills that are considered important for future success
in school. Intelligence tests are also standardized tests designed to determine how well a person
can handle problem solving using higher level cognitive thinking. A typical IQ test, often called
just an IQ test for common usage, asks questions about pattern recognition and logical reasoning.
It then takes into account the time needed and the number of questions the person gets right, with
a penalty for guessing. Specific tests and how the results are

uses district-by-district variation, but intelligence testing is common during the early years of
schooling.

(b) Benefits

• It is easily obtained and available to the researcher.

• Can be quickly adopted and implemented.

• Reduces or eliminates faculty time-consuming development and classification of tools.

• It helps to score objectively.

• Can provide external validity of the test.

• Helps to provide measurement of reference groups.

• Can make longitudinal comparisons.

• Can test a large number of students.

(c) Disadvantages

• Measures relatively superficial knowledge or learning.

• Norm-referenced data may be less useful than criterion-referenced data.

• May be financially prohibitive to administer as a pre- and post-test.

• Is summative rather than formative (can be difficult to isolate what

changes are required).

• It may be difficult to get results in a timely manner.

(d) Recommendations

• Must be carefully selected based on faculty evaluation and determination of fit between test
content and curriculum content.

• Ask the publisher for a technical manual and information on reliability and validity.

• Get information from other users.

• If possible, purchase a data disk to create customized reports.

• If possible, select tests that also provide results according to the criteria.

• Compare the results with those obtained by other assessment methods.

• Embedding a test as part of course requirements can improve student motivation.

Conditions and examples of standardized testing

Standardized tests require all test-takers to answer the same questions in the same way and are
scored in a consistent manner, allowing the relative performance of students or groups of students
to be compared. Standardized tests are a common practice in modern academia, with students
taking them as early as preschool. Read about three examples of common standardized tests: the
SAT (Scholastic Aptitude Test), the Stanford-Binet Intelligence Scale, and the GED (General
Educational Development).
 --------------------------------------------
QUESTION NO. 4
--------------------------------------------
Compare the characteristics of essay type test and objective type test with
appropriate examples?
--------------------------------------------
ANSWER
--------------------------------------------

ESSAY TYPE TEST

An essay item is an item in which the examinee relies on his memory and past associations to
answer questions in just a few words. Since such items can be answered in any way you like and
these items are also known as free response items.

Essay items are best suited to measure higher mental processes, which include the process of
synthesizing, analyzing, evaluating, organizing, and critiquing past events. Thus, essay tests are
suitable for measuring qualities such as critical thinking, originality, and the ability to integrate a
synthesis or analyze different events.

Types of Essays

Essay items are of two types

• Types of short answers

• Long Answer Type / Extended Answer Essay Type

A short answer essay item is one where the examinee provides a response in one or two lines,
usually dealing with one central concept.

A long-answer essay item is one where the examinee's answer consists of several sentences. Such
an item usually covers more than one central concept.

Suggestions for writing good essays

1 – An essay item must contain explicitly defined problems, usually essay items are intended to
measure a higher mental process as such, it is important that they contain the problems in clear
and explicit terms so that each examinee interprets them more or less the same. Therefore, an essay
item is set as invalid if its interpretation varies between examinees

2 – It must contain problems whose answers are not too broad. In case the student is asked to
answer a problem with more content. He can start writing anything he knows without any
discrimination, in such a situation he must not write about the facts or information that the article
needs, thus reducing the validity of the essay.

3 – Essay items must have clear directions or instructions for the examinee, the instructions should
indicate the total time to spend on the particular test item. What type of information is required
and the likely age range to be assigned to each item so that the examinee can appreciate the relative
importance of the essay questions and adjust the length of the answer accordingly.

4 – Sufficient time should be allowed in the construction of essay items, such items measure higher
mental processes, to actually measure what they purport to measure. It is essential that essay items
are carefully worded and organized so that all items can be broken in the same way.

Characteristics of Essay Type Test and Objective Type Test

The difference between essay tests and objective tests

1 – For essay items, the examinee writes the answer in his own words, while for the objective type
of tests, the examinee chooses the correct answer from several given alternatives.

2 – Thinking and writing are important in essay tests, while reading and thinking are important in
objective type tests. In essay tests, the examinee answers questions in several lines. Thinks
critically about the issues the questions raise, organizes the thought in order and expresses it in
writing. In the objective type, the examinee does not have to write in many cases. He is simply
asked to check/mark. However, to make the right choice, it is necessary for him to read both the
stem and the alternative answers very carefully and then think critically and decide.

3 – Essay tests are difficult to score objectively and accurately, while objective tests are easy to
score objectively and accurately.

4 – Essay tests are difficult to score objectively and partly because the answers are not fixed like
the answers of the objective items due to the variability of the scorer's judgment about the content
of the answers in the objective types of tests, either choice or prompt. scoring can be done
accurately because the answers are fixed in them. Scoring will also be objective because when
answers are fixed, there will obviously be complete interpersonal agreement between students.

5 – In objective type tests, the quality of the subject depends on the skill of the test designer, but
in the essay test, the quality of the subject depends on the skill of the evaluator. Writing an
objective type test item is quite a challenging task. Only an experienced test designer can write
good objective items. The quality of tested items must suffer. If the test designer lacks item writing
skills as well as limited knowledge of essay items, it is easy to construct. A test constructor can
prepare relatively high-quality essay items even with minimal knowledge of item writing.

6 – Objective test items, no matter how well constructed, allow and encourage examinee guessing,
while essay test items, no matter how well constructed, allow and encourage examinee bluffing.
For objective type test items, the probability of guessing cannot be completely zeroed out. The
effect of guessing is to inflate the actual score obtained on the test. Guessing is most evident when
the test length is short and a two-alternative target form is used, or when difficult alternative
answers are included in multiple-choice or matching items and the test length is short.

7 - The assignment of numerical scores on essay test items is entirely in the hands of the evaluator,
while the assignment of numerical scores on objective type test items is entirely determined by the
scoring key in the manual.

Common Points Between Essay Tests and Objective Tests

Despite all these differences, the following are the common points or main similarities that lie in
the essay test or the objective test.

1. There is an element of subjectivity in both objective-type tests and essay tests. In objective tests,
subjectivity refers to the writing of test items when choosing a specific criterion for test validation.
In essay tests, subjectivity refers to the writing and selection of items. The most obvious influence
of subjectivity in the essay test is seen in the scoring of the essay items.

2. Both essay tests and objective-type tests emphasize objectivity in interpreting test scores. By
objectivity is meant that the scores must be nearly the same for all observers or raters who assigned
them. If this is not the case, it means that the scoring lacks objectivity, thus reducing the usefulness
of the score.

3. Any educational achievement, such as the ability to spell English words, knowledge of grammar,
and performance in history, geography, and educational psychology, can be measured by both an
essay test and an objective type test.

If the intention is to measure critical thinking, tests of originality and organizational ability are
preferred, but if the intention is to measure partial knowledge in any subject, objective type tests
are preferred.

However, this line of demarcation is now rapidly disappearing as objective items have been
effectively used to measure test takers' representations of achievement, critical thinking and
originality. Similarly, essay items, particularly short-answer essay items, have been used
successfully in measuring achievement representing partial knowledge of any subject.

--------------------------------------------
QUESTION NO. 5
--------------------------------------------

Write a detailed note on the types of reliability.

--------------------------------------------
ANSWER
--------------------------------------------
Reliability
What does reliability mean? Reliability means trustworthy. A test score is called reliable when we
have reason to believe that the test score is stable and objective. For example, if the same test is
given to two classes and marked by different teachers, even though it produced similar results, it
can be considered reliable. Stability and trustworthiness depend on how free the score is from
random errors. First, we need to create a conceptual bridge between the question asked by the
individual (ie, are my results reliable?) and how reliability is measured scientifically. This bridge
is not as simple as it seems at first glance. When one thinks about reliability, many things may
come to mind - my friend is very reliable, my car is very reliable, my online bill payment process
is very reliable, my client's performance is very reliable, and so on. The characteristics we are
concerned with are concepts such as consistency, reliability, predictability, variability, etc. Note
that the implied reliability statement is that behavior, machine performance, data processes, and
work performance may sometimes not be reliable. The question is, “How much do test scores vary
across observations?
Definition of Reliability:
According to the Merriam Webster Dictionary
"Reliability is the extent to which an experiment, test, or measurement procedure gives the same
results on repeated trials."
According to Hopkins & Antes (2000).
"Reliability is the consistency of observations obtained over repeated recordings either for a single
subject or for a set of subjects."
Joppe (2000) defines reliability as.
“…The degree to which the results are consistent over time and accurately represent the total study
population is referred to as reliability, and if the results of the study can be reproduced with similar
methodology, then the research instrument is considered reliable. .” (p. 1). A more general
definition of reliability is: The degree to which a score is stable and consistent when measured at
different times (test-retest reliability), in different ways (parallel forms and alternate forms), or
with different items within the same scale (internal consistency).
Types of reliability
Reliability is one of the most important elements of test quality. It is related to the consistency or
reproducibility of the performance tested in the test. Reliability cannot be precisely calculated.
Instead, we have to estimate reliability, and this is always an imperfect attempt. Here we present
the main reliability estimates and discuss their strengths and weaknesses.
There are six general classes of reliability estimators, each of which estimates reliability in a
different way. They are:
i) Inter-rater or inter-observer reliability
To assess the extent to which different raters/observers provide consistent estimates of the same
phenomenon. This means that if two teachers mark the same test and the results are similar, this
indicates inter-rater or observer reliability.
ii) Test-retest reliability:
To assess the consistency of a measurement from one point to another, when the same test is
administered twice and the results of both administrations are similar, it represents test-retest
reliability. Students may remember and mature after the first administration creating a problem for
test-retest reliability.
iii) Reliability of parallel form:
To assess the consistency of the results of two tests created in the same way from the same content
domain. Here the test designer tries to develop two tests of similar kind and after administration
the results are similar then it shows the reliability of parallel form.
iv) Internal consistency reliability:
To assess the consistency of results across items within a test, it is the correlation of individual
item scores with the entire test.
v) Reliability in half:
To assess the consistency of results by comparing two halves of a single test, these halves can be
odd-even items of a single test.
vi) Kuder-Richardson Reliability:
Assess consistency of results using all possible split halves of the test. Let's discuss each of them
in turn.
Inter-rater or inter-observer reliability
Whenever we observe or do people, we need to think about a process for reliable and consistent
results. For this, two or more observers are assigned to watch students or teachers. So how do we
determine if two observers are consistent in their observations? We should probably establish
interrater reliability by considering the similarity of scores given by two observers. After all, if we
use data to determine reliability, and we find that reliability is low. We should focus on the criteria
set for observation. And if you try it in a real situation first, then it can help to develop reasonable
criteria for observation and can be more objective.
There are two main ways to actually estimate interrater reliability. If your measurement consists
of categories—raters check which category each observation falls into—you can calculate the
percentage of agreement between raters. For example, say you had 100 observations that were
rated by two raters. For each observation, the rater could tick one of three categories. Imagine that
for 86 out of 100 observations, raters ticked the same category. In this case, the percentage of
agreement would be 86%. OK, it's a rough measure, but it gives an idea of how much agreement
there is, and it works regardless of how many categories are used for each observation. Another
major method of estimating interrater reliability is appropriate if the measurement is continuous.
There, it is sufficient to calculate the correlation between the ratings of two observers. For example,
they could rate the overall level of activity in the classroom on a scale of 1 to 7. You could have
them rate at regular intervals (eg every 30 seconds). The correlation between these ratings would
give you an estimate of inter-rater reliability or consistency. One might think of this type of
reliability as the "calibration" of observers. There are other things that can be done to promote
interobserver reliability, even without estimation. For example, in a psychiatric unit where every
morning a nurse had to do a ten-point assessment of every patient in the unit. Of course, it is
difficult to count on the same nurse being present every day, so a way needs to be found to ensure
that any of the nurses give comparable assessments. The way we did this was to hold weekly
"calibration" meetings where we had all the nurses rate multiple patients and discuss why they
chose the particular values they chose. If there were disagreements, the nurses would discuss them
and try to come up with rules for deciding when to give a "3" or a "4" for evaluation of a specific
item. Although this was not an estimate of reliability, it probably advanced to improve interrater
reliability.
Test-retest reliability
Test-retest is a statistical method used to determine the reliability of a test. The test is performed
twice; in the case of a questionnaire, this would mean giving the same questionnaire to a group of
participants on two separate occasions. This form of reliability is used to assess the consistency of
results between items on the same test. Essentially, you are comparing test items that measure the
same construct to determine the internal consistency of the tests. When you see a question that
seems very similar to another test question, it may mean that the two questions are being used to
measure reliability. Because the two questions are similar and designed to measure the same thing,
the test taker should answer both questions the same way, which would mean that the test has
internal consistency.
We estimate test-retest reliability when we administer the same test to the same sample on two
separate occasions. This approach assumes that there is no substantial change in the measured
construct between the two cases. The time between measures is critical. We know that if we
measure the same thing twice, the correlation between the two observations will depend in part on
how much time passes between the two measurement occasions. The shorter the time interval, the
higher the correlation; the longer the time interval, the lower the correlation. This is because the
two observations are related in time—the closer in time we get, the more similar the factors that
contribute to the error are. Since this correlation is an estimate of test-retest reliability, you can get
quite different estimates depending on the interval.
Reliability split in half
Suppose you need to create a 30-item test and want to know how reliable the test is? What you
need to do is enter the test, mark it and divide it into two parts by placing all the even items
(2,4,6……) in one half and the odd ones. items (1,3,5…………..) in the second. Calculate the
reliability using the Spearman-Brown prediction formula below. In fact, in split-half reliability,
we randomly divide all items that claim to measure the same content into two sets. We administer
the entire instrument to a sample of students and calculate a total score for each randomly divided
half. The reliability estimate at the half is simply the correlation between the two total scores.
Usually one test is used to create two shorter alternative forms. This method has the advantage that
only one test administration is required and therefore does not involve memory and the effect of
practice and maturation. In addition, it does not require two tests. Therefore, it has many
advantages over the parallel form and test-retest methods, which is why it is the most frequently
used method of determining the internal consistency of class tests. The formula used for the
reliability of the entire test is the Spearman-Brown prediction formula
as shown below.
2 (half test reliability)
Full Test Reliability = _______________________
1+ (half test reliability)
Reliability of parallel shape
With parallel form reliability, we must create two different tests of the same content to measure
the same learning outcomes. The easiest way to do this is to write a large set of questions that deal
with the same content and then randomly divide the questions into two sets. Now is the time to
administer both tools to the same students at the same time. The correlation between two parallel
forms is an estimate of reliability. One of the main problems with this approach is that you need
to be able to write many items that reflect the same content. This is often not easy to do.
Furthermore, this approach assumes that the randomly distributed halves are parallel or equivalent.
By no means will it ever be so. The parallel forms approach is very similar to the split-half
reliability described earlier. The main difference is that parallel forms are constructed so that the
two forms can be used independently
each other and considered equivalent measures. For example, we may worry about testing threats
to internal validity. If we use form A for the pretest and form B for the posttest, we minimize this
problem. It would be even better if we randomly assigned individuals to receive Form A or B in
the pretest and then switched them to the posttest. With split-half reliability, we have a tool we
want to use as one
measuring instrument and develop only randomly distributed halves for reliability estimation
purposes.
Internal consistency Reliability
We use our single test to estimate internal consistency reliability. The test is administered to a
group of students on one occasion to estimate reliability. In effect, we assess the reliability of an
instrument by estimating how well items that reflect the same content produce similar results. We
track how consistent the results are for different items for the same construct within a measure.
There are a wide variety of measures of internal consistency that can be used.
Kuder Richardson Reliability
Estimates of the internal consistency of a test are commonly calculated using Kuder-Richardson
methods. This measures the extent to which items within one form of a test have as much in
common with each other as items in that one form have in common with corresponding items in
an equivalent form. The strength of this reliability estimate depends on the context in which the
entire test represents a single, fairly consistent measure of the concept.
Normally, these estimates are lower than split-half estimates, but higher than test-retest and
parallel-form estimates. These techniques are also called item-total correlations. There are various
techniques for estimating the internal consistency of a test using K-R procedures, but two of them
are more commonly used by measurement professionals. The first KR-20 is difficult to calculate
because it is based on information about the percentage of students who pass each item on the test.
However, it gives more accurate results (Kubiszyn and Borich, 2003). The KR-20 formula is
shown below.
Formula KR20
Where "pq" gives the variance of test errors for the "average" person, we know that the people in
the sample are different, i.e. the variance of their raw scores is greater than zero. High or low
scorers have less variance in score error than individuals with close to fifty percent correct scores,
where the variance in score error is maximal. Because the "average" person variance used in the
KR20 formula is always greater than the error variance of lower scorers for extreme scorers, it
must always overestimate their score error variances.
The second formula, which is easier to calculate but slightly less accurate, is called KR21. It only
requires information about the number of items, the mean of the test scores, and the standard
deviation. The formula of KR21 is as below.

Studies have shown that this formula provides good results even when item difficulty is not
consistent.

Factors affecting reliability

Test reliability is an important characteristic because we use test results to make future decisions
about students' educational progress and for job selection and many others. Methods for ensuring
test reliability were discussed. Many examples have been provided to provide a deeper
understanding of the concepts. Here we will focus on the various factors that can affect the
reliability of the test. The degree of influence of each factor varies from situation to situation.
Factor manipulation can improve reliability and otherwise reduce the consistency of score
production. Some of the factors that directly or indirectly affect the reliability of the test are listed
below.
1. Duration of the test:
As a general rule, adding more homogenous questions to a test will increase the reliability of the
test. The more observations of a particular feature there are, the more accurate the measurement is
likely to be. Adding more questions to a psychological test is akin to adding finer distinctions to a
tape measure.
2. Method used to estimate reliability:
The reliability coefficient is an estimate that can vary depending on the method used to calculate
it. The method chosen to estimate reliability should be appropriate for the way the test will be used.
3. Score heterogeneity
Heterogeneity is referred to as differences between scores obtained from a class. You can say that
some students have high scores and some students who have low scores or intelligent students who
have high scores and others have low scores or the difference can be due to any reason, it can be
income level, intelligence of students, qualifications parents, etc. Whatever the reason for score
variability, the greater the variability (range) of test scores, the higher the reliability. Increasing
the heterogeneity of the sample under investigation increases variability (individual differences)
and thus increases reliability.
4. Difficulty
A test that is too difficult or too easy reduces reliability (eg, fewer test-takers get the answers right
or vice versa). A moderate level of difficulty increases the reliability of the test.
5. Errors that can increase or decrease an individual score:
Test developers can make certain mistakes that also affect the reliability of teacher-created tests.
These errors initially affect students' scores, represent a deviation from students' actual abilities,
and therefore affect reliability. Careful consideration of these factors can help gauge students' true
abilities.
• The test itself: the overall appearance of the test can affect student scores. Usually the test is
written in an easily readable font size and style, the language of the test should be simple and
understandable.
• Test Administration: After the test is created, the test developer can prepare a test administration
manual, time, environment, guarding and anxiety also affect students' performance when
attempting the test. Therefore, the uniform administration of the test leads to an increase in
reliability.
• Test scoring: Test scoring is another factor influencing variation in student scores. There are
usually many raters who rate the student's answers/responses in the test. Objective test-type items
and a grading rubric for essay-type/delivery-type test items help obtain consistent scores.
Ensuring test reliability:
The most direct ways to improve test reliability are "
First, calculate item-test correlations and rewrite or reject those that are too low. Any item that
does not correlate with the total test at least (point-biserial) r = 0.25 should be reconsidered.
Second, look at the items that correlated well and write more like them. The longer the test, the
higher the reliability.