Measurement and Survey Data Collection Methods

Learning Objectives
To understand measurement as it relates to Marketing Research.
To be able to distinguish between Reliability and Validity

To learn about advantages and disadvantages of surveys.

To become knowledgeable of the details of different types of survey data collection methods such as personal interviews, telephone interviews, and computer-assisted interviews. To comprehend factors researchers consider when choosing a particular survey method.

Measurement
Assignment of numbers to characteristics of objects, persons, states or events, according to rules

Keys to Measurement
You

do not measure the object, person, state or event, but characteristics of the object Numbers are used to represent the observable/unobservable characteristics Rules specify how the numbers are to be assigned to the characteristics

Measurement Definitions
Concepts:

Invented name for the property of an object, person, state or event Construct: A concept, having been deliberately and consciously invented or adapted for a special scientific purpose Constitutive Defn Operational Defn

Components of Measurement

M=C+E
M = Measurement C = Characteristic being measured E = Errors

Source: McDaniels & Gates (2002, p. 171)

Content Error

Response Error

Processing or Researcher Error

Revisiting something we discussed earlier!

Reliability and Validity

Reliability:

Refers to the degree of variable error represented in a measurement Validity: Refers to the extend to which the measurement is free from systematic error Perfect validiy suggests M=C; no E.

Approaches to Assess Reliability

Test-Retest
Split

Sample Alternative Forms Internal Comparison (Cronbachs Coefficient Alpha) Inter-Scorer Reliability

Approaches to Assess Validity

Content Validity (Face) - Representativeness of the

sampling adequacy of the items contained in the measurement instrument.

Criterion-Related Validity
Concurrent Validity: Two measures of the same construct are highly related. (Done in order to use a shortened version of the scale. Predictive Validity: Ability of the scale to predict some outside criterion - generally some outcome, a current or future state of affairs

Approaches to Assess Validity

Construct Validity
Trait Validity
Reliability Convergent Validity: the extent to which one measure correlates
highly with other methods designed to measure the same construct

Discriminate Validity: extent to which the measure is indeed

novel and is simply not a reflection of some other variable. Quite simply, the measure should not correlate too highly with measures of different constructs.

Nomological Validity: Does the measure behave as expected.

Are the relationships between the measure of interest with respect to sign and magnitude consistent with theory.

Your Team Project

What is the main purpose for the study you are conducting for your client? What are the major components you want to measure? What is the Measurement Model Provide Constitutive Definition Provide Operational Definition Provide the Measurement Scale

ISU Green Team Project

What was the main purpose for the study conducted for the ISU Green Team Project?

ISU Green Team Project

What were the major components measured? (The Research Questions)

Awareness of Recycling Programs on Campus Attitudes toward
Recycling Energy Conservation

Likelihood (Behavioral Intention) of Recycling/Energy Conservation in the future Perceptions of ISU Green Team Programs Differences among Administrators, Faculty, Staff, Students Differences among On-Campus versus Off-Campus Students ???

Understanding the Relationships Among the Constructs: Model of Goal Directed Behavior Generalized to Recycling Behavior
Utilitarian Attitudeact Toward Recycling Hedonic Attitudeact Toward Recycling Positive Anticipated Emotions from Recycling Negative Anticipated Emotions from Recycling Subjective Norms Concerning Recycling Perceived Difficulty of Recycling Perceived Anticipated Regret Toward Recycling Frequency of Past Recycling Recency of Recycling Behavior

Desire to Recycle

Intention To Recycle
Perceived Behavioral Control of Recycling Self Efficacy (Confidence)

Recycling Behavior

Control

ISU Green Team Project

Provide

Constitutive Definition for Key Constructs (What are the key constructs?)

ISU Green Team Project

Provide Operational Definition

ISU Green Team Project

Provide the Measurement Scale (what appears on the survey instrument)

 SoWhere

are you with your research project? What model have you developed to help guide your research? What are the key constructs that you need to measure?

Now that we know what we need to measure, we can now proceed with how to measure it, and how do we gather that data.

Basic Question-Response Format

Considerations in Choosing a Question-Response Format

Nature of the Construct: The properties of the construct being measured often determine the appropriate response format. Previous Studies: Researchers try to use question formats that are tried and true. Data Collection Mode: The method of data collection determines the appropriate response format. Ability of Respondent: Some respondents may relate better to one type of response format than another. Scale Level Desired: Your preferred statistical methods will dictate minimum scale levels of captured data.

Types of Measurement Scales

Nominal

Label Categorical Mixed Ordinal Interval Ratio

Types of Measurement Scales

Nominal Categorical Scale example What is your gender? ___ Male ___Female How many hours have you completed toward your degree? ___under 30 hours ___30-59 hours ___60-89 hours ___90 or more hours

Types of Measurement Scales

Ordinal Scale Example Please rank order the following as to how often you recycle each item where 1=item you most often recycle, 7=item you recycle the least. ___Cardboard ___Glass ___Newspaper ___Plastic ___Other Paper Products ___Aluminum ___Other, please specify________________

Common Types of Interval Scales

Likert

Semantic

Differential

Rating Noncomparative Comparative

Itemized

Likert
Circle the number that best represents your agreement or disagreement with this statement I always recycle paper, plastic, glass items.
Neither Strongly Agree or Strongly Agree Agree Disagree Disagree Disagree

Semantic Differential
Please check the blank that best describes the phrase below. The closer you believe the word or word phrase relates, the you would place your check nearer to the word/word phrase. Recycling is ...
Boring Exciting

My responsibility Necessary for the preservation of earth Does little to help the environment

Not my responsibility
Not necessary for the preservation of earth Does a great deal to help the environment

Noncomparative Rating Scales

Overall, how would you rate your level of recycling? Please check the appropriate response. Exceptionally Poor Exceptionally Good

Noncomparative Rating Scales

Overall, how would your level of recycling. Please place a check mark along the line that best represents your rating. Exceptionally Poor Exceptionally Good

Non Comparative Rating Scales

Overall, how would you rate your level of recycling? Please circle the appropriate response.
Exceptionally Exceptionally

Poor

1 2

3 4

Good 6 7

Comparative Rating Scales

Overall, how would you rate your level of recycling compared to other college students? Please circle the appropriate response.
Exceptionally Poor Exceptionally Good

Types of Measurement Scales

Ratio Scale Over the past week, how many plastic bottles have you placed into a recycling bin? _____ Over the past week, what percentage of plastic bottles that you used, did you placed into a recycling bin? _____%

Issues Concerning Rating Scales

Nature

& Degree of Verbal Descriptor Number of Categories Balanced vs Unbalanced Odd vs Even No. of Categories Forced vs Nonforced Choice

Types of Surveys
Personal

Interviews Telephone Interviews Mail Surveys On-Line/Internet-Based Surveys Fax

Mail Survey Issues

Method

of Notification Preliminary Concurrent Follow-up

Mail Survey Issues

Envelopes Return Postage Method of Addressing

Mail Survey Issues - Cover Letter

Sponsorship

Personalization
Type

of Appeal Deadline Date Incentive Signature

Mail Survey Issues - Questionnaire

Length Content Layout Color

Size
Reproduction

Format
Anonymity

Confidentiality

On-Line/Internet-Based Surveys

Tailored Design: Writing Questions

1. 2. 3. 4.

5.
6. 7.

8.
9.

Ensure that questions are answerable by 99% of audience (check for excessive use of NA category) Ready made answers are easy, thoughtful answers are not Make sure respondent is motivated to answer the question Watch out for memory effects in reports of past events Order response categories logically and conventionally Use definite quantifiers rather than vague quantifiers Limit the length of lists of unordered response choices (e.g., colors, political candidates, models of products) Avoid the appearance of bias in questions and responses Invest in high quality sentence construction

Tailored Design: Internet Instrument Construction

1. 2. 3.

4.
5. 6. 7. 8. 9. 10. 11. 12.

Provide a welcome screen that is motivational Include access controls Make initial experience highly engaging Draw parallels to experience w/ paper surveys Restrain use of bold colors and designs Maintain figure ground consistency Design for minimum bandwidth requirements Build for cross-platform compatibility Design controls for lowest skill population Allow for skips and backtracking where appropriate Balance scrolling and paging, indicate progress Make sure stems and response options stay visible

Tailored Design: Web Study Deployment

1. 2.

3.

4.
5.

Use an advance email contact or pre-screening procedure Second contact: Email a link to the questionnaire with a carefully constructed recruitment message; include pre-study incentive Third contact: Reminder message targeted to non-respondents Subsequent contacts: Thank you messages, or second reminders Provide any post-study incentives for participation

Summary for Use of Web-based Surveys

Encourage high response rates through researchproven techniques Online research deploys rapidly: Use this to your advantage by reducing instrument size Think of the response process as social exchange: Your job as a researcher is to make the study as enjoyable and beneficial as possible for the respondents

13 Guidelines For Online Research

1. 2. 3.

4.
5. 6. 7.

8.
9. 10. 11.

12.
13.

Thou shalt not SPAM! Begin instrument with an explanation of study, expected risks, expected Obtain informed consent through active behavior: checkmark, initials, or access code Allow missing data fields and page skips Avoid coercive monetary incentives Use anonymous research whenever possible Discard identifiers at the earliest possible time; This includes IP addresses! Avoid using cookies if possible, otherwise use expiring cookies with no personal info Warn participants that their data submission is not encrypted and could be intercepted Archive and purge all online databases at regular intervals to avoid hacking Avoid deception whenever possible Quiz comprehension of risks prior to providing access to instrument Pilot and modify research materials to soften psychological impact (e.g., asking about layoffs) Integrate debriefing into required sequence of pages, use multimedia, provide contact info

Go Forth and Build Your Survey