Scientific Research in

Information Systems: A
Beginner’s Guide (2 edition)
nd

Teaching Materials

Created by Professor Jan Recker

Teaching Notes: Scientific Research in Information Systems (2nd edition) ~ © Copyright 2021 Jan Recker. All Rights Reserved. ~ 1
Copyright Notice & Citation
© Copyright 2021 Jan Recker. All Rights Reserved.

Recker, J. (2021): Scientific Research in Information Systems: A Beginner’s Guide.

2nd edition, Springer.

Available at Springer, Amazon, and other booksellers

Teaching Notes: Scientific Research in Information Systems (2nd edition) ~ © Copyright 2021 Jan Recker. All Rights Reserved. ~ 2
 Overview

Content
Part 1: Basic Principles of Research
Part 2: Conducting Research
Part 3: Publishing Research

Teaching Notes: Scientific Research in Information Systems (2nd edition) ~ © Copyright 2021 Jan Recker. All Rights Reserved. ~ 3
Chapter 5:
Research Methods

4
Quantitative Methods

5
 Cornerstones of Quantitative Methods (1)

 Procedures that feature research methods such as experiments or surveys and

which are characterized by an emphasis on quantitative data (think of these
procedures as having a focus on “numbers”).
 Quantitative data are types of data whose value is measured in the form of
numbers, with a unique numerical value associated with each data record.
 Quantitative methods emphasize state-of-the-art analysis of such data to create
valid and reliable general claims.
 Cornerstones of Quantitative Methods (2)

 Quantitative methods emphasize (post-) positivist philosophy.

 Positivist researchers generally assume that reality is objectively given and can be
discovered by a researcher and described by measurable properties independent of the
observer (researcher) and his or her instruments.
 Interpretive researchers, on the other hand, start out with the assumption that access to
reality (given or socially constructed) is only through social constructions such as language,
consciousness, and shared meanings
 Ontologically, quantitative research is based on the idea that scientific theories can be
proposed that can be falsified by comparing theory to carefully collected empirical data.
The world has an objective reality that can be captured and translated into testable
hypotheses, usually in the form of statistical or other numerical analyses
 Example: Einstein’s theory of relativity really became trusted when in 1919, Eddington’s
eclipse observation showed that Einstein’s predictions were correct and Newton’s
predictions incorrect.
 The hypothetico-deductive model to science

1. Researchers posit a new theory in the form of one or more hypotheses (e.g., an alternative
hypothesis that people with small hands type faster), expressed in contrast to a null
hypothesis of no effect (e.g., people with small hands do not type faster).

2. They design an empirical study to obtain data (e.g., measures of typing speed and hand Report findings
size). Develop hypotheses

3. They collect the data from a sample (e.g., a group of students or knowledge workers).

4. They test their hypotheses, by analyzing the gathered data and calculating one or another
test statistic (e.g., a t-test comparing typing speed of those with large hands to those with Hypothetico-
small hands). They calculate a probability, the p-value, under the assumptions of a deductive
Interpret results
specified statistical model, that a particular test statistic (e.g., the average typing speed) scientific Design study
would be equal to or more extreme than its observed value. Through this test, they process
examine in the data whether the null hypothesis holds true in the population (e.g., people
with small and large hands type at the same speed). This prediction is called a null
hypothesis because it typically assumes the absence of an effect (i.e., no difference in
typing speed). The p-value—the probability of finding a difference in typing speed in our Analyze data to evaluate
sample, assuming that there is no difference in the population—is then usually compared hypotheses
to certain thresholds (typically 0.05 or 0.01) known as the alpha protection level. Collect data

5. They interpret the results from the statistical tests. If the null hypothesis is rejected,
researchers typically construe this result as denoting “accept” or “support” for their stated
alternative hypothesis (that people with small hands indeed type faster).
Mertens, W., & Recker, J. (2020). New Guidelines for Null Hypothesis Significance Testing in Hypothetico-
Deductive IS Research. Journal of the Association for Information Systems, 21(4), 1072-1102. 8
 Fundamentals of Quantitative Methods

 The Importance of Measurement

 Quantitative methods depend heavily on exact measurement. measurement

provides the fundamental connection between empirical observation and the
theoretical and mathematical expression of quantitative relationships
Measures and measurement

Theory Instrumentation Reality

Measure Operationalization Measurement

Conceptualization Real-world domain
and data

Primary measure issue: Primary measurement issue:

shared meaning accuracy

Burton-Jones, A., and Lee, A. S. 2017. "Thinking About Measures and

Measurement in Positivist Research: A Proposal for Refocusing on
Fundamentals," Information Systems Research (28:3), pp. 451-467.
 Two requirements of measures and
measurement

 The variables that are chosen as operationalizations must also guarantee that
data can be collected from the selected empirical referents accurately (i.e.,
consistently and precisely). This step concerns the reliability of measurement.

 The variables that are chosen as operationalizations to measure a theoretical

construct must share its meaning (in all its complexity if possible). This step
concerns the validity of the measures.

Burton-Jones, A., and Lee, A. S. 2017. "Thinking About Measures and

Measurement in Positivist Research: A Proposal for Refocusing on
Fundamentals," Information Systems Research (28:3), pp. 451-467.
Reliability and Validity
 Validity

 Validity describes whether the operationalizations and the collected data share
the true meaning of the constructs researchers set out to measure.
 Can be assess theoretically and/or empirically.
 Types of Validity

 Content validity
 refers to the extent to which a researcher’s conceptualization of a
construct is reflected in her operationalization of it, that is, how well a
set of measures match with and capture the relevant content domain of
a theoretical construct.
 Content validity is important because researchers have many choices
in creating ways to measure a construct. Did they choose wisely so
that the measures they use capture the essence of the construct? They
could, of course, err on the side of inclusion or exclusion.
 Assessments may include an expert panel that peruse a rating scheme
and/or a qualitative assessment technique such as the Q-sort method.
 Types of Validity

 Construct validity
 is an issue of operationalization and measurement between constructs. The
baseline issue here is whether different theoretical constructs are separable from
each other.
 Convergent validity: Items or phrases in the instrumentation are not related in the
way they should be or they are not related in the ways they should not be.
 Discriminant validity: Items or phrases in the instrumentation do not segregate or
differ from each other as they should.
 Nomological validity: assesses whether measurements and data about different
constructs correlate in a way that matches how previous literature predicted the
causal (or nomological) relationships of the underlying theoretical constructs.
 Construct validity is typically assessed empirically through statistical, correlational
logic.
 Types of Validity

 Internal validity
 assesses whether alternative explanations of the dependent variable(s) exist that
have not been ruled out.
 Factors:
 temporal precedence of IVs before DVs
 Covariation
 ruling out rival hypotheses
 Typically assessed through the inclusion of statistical control variables such as
firm size, experience, gender, etc.
 Types of Validity

 Other types of validity

 Manipulation validity
 used in experiments to assess whether an experimental group (but not the control group) is faithfully
manipulated and we can thus trust that any observed group differences are in fact attributable to the
experimental manipulation.
 Statistical conclusion validity
 assesses the appropriate use of statistics to infer whether the presumed independent and dependent
variables co-vary as predicted
 Predictive validity
 assesses the extent to which a measure successfully predicts a future outcome that is theoretically
expected
 Ecological validity
 assesses the ability to generalize study findings to real-world settings
 Reliability

 describes the extent to which a measurement variable or set of variables is

consistent in what it is intended to measure.
 important to the scientific principles of replicability because reliability implies that
the operations of a study can be repeated in equal settings with the same
results.
 Sources of reliability problems often stem from a reliance on overly subjective
observations and data collections.
 Types of Reliability

 Internal consistency
 measures whether several measurement items that propose to measure the
same general construct produce similar scores.
 The most common test is through Cronbach’s (1951) alpha.
 Types of Reliability

 Interrater reliability
 is important when several subjects, researchers, raters, or judges code the same data
 When a range of individuals (multiple study subjects or multiple researchers, for
example) all rate the same observation and we look to get consistent, consensual
results
 Cohen’s (1960) coefficient Kappa is the most commonly used test.
 Types of Reliability

 Other types of reliability

 unidimensional reliability, composite reliability, split-half reliability, or test-retest
reliability
 See optional reading

Straub, D. W., Boudreau, M.-C., & Gefen, D. (2004). Validation Guidelines for IS
Positivist Research. Communications of the Association for Information
Systems, 13(24), 380-427.
 Developing and Assessing Measures and
Measurements

 Establishing reliability and validity of measures and measurement is a

demanding and resource-intensive task.
 The first rule should always be to identify and re-use where possible, existing
measures and measurements that have already been developed and assessed.
 Examples:
 the Handbook of Management Scales
([Link]
 the Inter-Nomological Network ([Link]

22
 Procedure for Developing and Assessing New
Measures and Measurements

1. Define the conceptual domain of a construct

2. generate pools of candidate measurement
items are for each construct
3. Purify the list of candidate items through
testing and revisions
4. obtain statistical evidence for reliability and
validity of the measures and measurements
 Types of Quantitative Methods

14.02.2022 24
 Descriptive/correlational quantiative research:
Survey

 A non-experimental, observational research method that does not involve controlling or manipulating independent variables.

 to gather information about the characteristics, actions, perceptions, attitudes, or opinions of a large group of units of observations (such as individuals, groups, or
organizations), referred to as a “population”.

 involve collecting data about a population from a random sample of that population through questionnaire-type instruments that can be distributed and completed
via mail, online, telephone, or, less frequently, through structured interviews.

 Traditionally, the dominant technique for data collection in IS.

 Are preferable when

 the central questions of interest about a phenomenon are “what is happening?” and “how and why is it happening?” and
 when control of the independent and dependent variables is not required or not possible.

 Can be used for at least three purposes:

 Exploration: to identify factors that appear relevant (e.g., success/failure factors)
 Description: to ascertain facts about the situations, events, attitudes, opinions, processes, or behaviors that are occurring in a population. (e.g., typical
political polls)
 Explanation: to test theory and hypothetical causal relationships between theoretical constructs (the most common form)

25
Example: Survey research
 Hypotheses

 Three determinants of continuance

 whether users form a positive belief about the
Perceived
Ease of Use

actual use of a technique, viz., whether they

find it useful and easy to use in actual
process modelling practice, and
 whether users are able to confirm (or Perceived
Usefulness
Intention to
Continue to

disconfirm) initial expectations from the pre-

Use

usage phase about a technique.

Satisfaction

Confirmation

Recker, J. (2010). Explaining Usage of Process Modeling Grammars:

Comparing Three Theoretical Models in the Study of Two Grammars.
Information & Management, 47(5-6), 316-324.
 Measurement development

14.02.2022 28
 The Instrument

14.02.2022 29
 Survey administration

 Population: all process and systems analysts

 Sample: convenience sample (all I could get)
 Administration: Web-based survey
 Sample size: about 600 if I remember correctly.

14.02.2022 30
 Results: Demographics

14.02.2022
Statistical results about the hypotheses

Confirmation Key

*** p < 0.001

0.47*** ** p < 0.01
Perceived * p < 0.05
Usefulness 0.42*** ns
non significant
R2 = 0.42
0.26***

Satisfaction
0.26***
R2 = 0.58

0.25***
0.18***
Perceived Ease 0.49***
of Use
0.21***

Intention to
Continue to
Use
R2 = 0.40
 Interlude – shared variance

 R2 describes how much variance in the levels of the dependent variable is

explained through the variance in the levels of the independent variable(s).

Satisfaction
PAY 49
%
 The Problem is…

 Reality is much more complex than our sets

of 1,2,3,... explanatory variables.

 Remember:
Our measures
always, invariably,
contain error.

Satisfaction

PAY
 Guidelines for survey research

1. Carry out careful development and assessment of measures and measurements.

2. Pre- and pilot-test your survey instrument.

3. Disclose your sampling strategy.

4. Report a profile of the sample framework.

5. Include your instruments in your reports.

6. Report your response rate.

7. Establish validity and reliability.

8. Follow the latest guidelines for data analysis.

35
Example: experimental research
 Experimental research

 Quantitative methods specifically intended to examine cause-and-effect relationships.

 used to examine such relationships by imposing a treatment on one group of respondents (the treatment group) but not
on another group (the control group) while maintaining control over potential confounding factors.

 Treatment: a manipulation that an experimenter administers to the treatment group so the experimenter can observe a
response.

 Primary advantage: Internal validity

 Primary disadvantage: ecological validity

 Often considered “the gold standard”.

 Can take place in the laboratory (lab experiment) or in reality (field experiment)
 Example: Skinner‘s Box

14.02.2022 38
 Basic experimental concepts

 Treatment manipulation
 the control for the cause in cause-effect relationships by identifying the type and
number of stimulus levels (provision versus non-provision, low/medium/high levels
of stimulus, and so forth).
 Experimental designs typically involve a phase prior to treatment manipulation called
pre-test measures, and usually a phase after treatment manipulation called post-
test measures.

14.02.2022 39
 Basic experimental concepts

 Experimental controls
 mechanisms employed to ensure that the responses observed are due to the
treatments and not because of confounding factors (e.g., placebo effect)
 also used in experiments to rule out rival theories, that is, alternative explanations.

14.02.2022 40
 Basic experimental concepts

 Randomization
 the process of selecting a sample from a population in such a way that personal
characteristics and predispositions do not interfere with the treatment or the
response to the treatment.
 Through matched allocation (expensive and difficult) or random assignment (key for
true experiments)

 Quasi-experiments lack random assignment of subjects to groups and hence

are experiments with non-equivalent groups (e.g., males versus females)

14.02.2022 41
True-Experimental Designs
Quasi-Experimental Designs
 Example: The impact of modeling grammar

Driver has
registered

Identify Goods receipt

delivery officer

XOR

Delivery
Delivery with
without
Purchase
Purchase
Ordder
Ordder

Contact Goods receipt

booking clerk officer

Decide upon
Booking clerk
acceptance

Notify goods
Booking clerk
receipt officer
Booking Clerk

XOR

Decide upon Notify goods

acceptance receipt officer
XOR

Delivery is
accepted
Organization

Contact
Goods receipt officer

booking clerk Reject goods XOR

Delivery Note

Delivery without PO
Determine Goods receipt
delivery ramp officer
Identify Determine Inspect goods Place goods
delivery delivery ramp quality in stock
Delivery with PO

Inspect goods Goods receipt

quality officer

XOR

XOR
Goods are
okay

Goods
delivery is
rejected
Place goods Goods receipt
into stock officer

Recker, J., & Dreiling, A. (2011). The Effects of Content Presentation Format and User Characteristics on
Novice Developers’ Understanding of Process Models. Communications of the Association for
Goods placed
into stock

Information Systems, 28(6), 65-84.

 A Better Example

Mendling, J., Strembeck, M., & Recker, J. (2012). Factors of Process

Model Comprehension — Findings from a Series of Experiments.
Decision Support Systems, 53(1), 195-206.
 Guidelines for experimental research

1. Carry out experiments only in the presence of strong theory.

2. Design your treatments carefully.
3. Perform manipulation checks.
4. Rule out alternative hypotheses..
5. Ensure ecological validity.
6. Check for the latest guidelines on experiments in the literature.

46
Qualitative Methods

47
 Qualitative Research
Overview

 are strategies of empirical inquiry that investigate phenomena within a real-life context.

 are helpful especially when the boundaries between phenomena and context are not
apparent, or when you want to study a particular phenomenon in depth.

 are well suited for exploratory research where a phenomenon is not yet fully understood,
not well researched, or still emerging.

 are also ideal for studying social, cultural, or political aspects of a phenomenon.

 stresses on the “why" and “how" of things rather than the “what," “where" and “when" of
things. It involves detailed study of a small sample or group.
 Examples

 Alan Peshkin’s 1986 book God's Choice: The Total World of a Fundamentalist
Christian School published by the University of Chicago Press
 Peshkin studies the culture of Bethany Baptist Academy by interviewing the students,
parents, teachers, and members of the community, and spending eighteen months
observing, to provide a comprehensive and in depth analysis of Christian schooling as an
alternative to public education.
 Paskin's work represents qualitative research as it is an in-depth study using tools such as
observations and unstructured interviews, without any assumptions or hypothesis, and aimed
at securing descriptive or non-quantifiable data on Bethany Baptist Academy specifically,
without attempting to generalize the findings to other schools.
 Other examples

 Victor of Aveyron
 [Link]
 broke new ground in the education of the developmentally delayed.

 Piaget's Theory of Cognitive Development

 [Link]
 Developed a constructivist theory of learning and instruction about the process of
“coming to know” and the stages we move through as we gradually acquire this ability.
 Basic tenets

 Scientific studies with procedures that feature research methods such as case
study, ethnography or phenomenology and which are characterized by an
emphasis on qualitative data.
 (Think of these procedures as having a focus on “words”)
 They emphasize understanding of phenomena through direct observation,
communication with participants, or analysis of texts, and may stress contextual
subjective accuracy over generality.
 Basic principles

 Natural setting: performed in the field, studying a phenomenon in the context in

which it occurs.
 Researchers as a key instrument: researchers collect data and information
themselves, often through face-to-face interactions, observing behaviours,
studying documents, or interviewing participants.
 Multiple sources of data: researchers typically gather a variety of data of
different sorts, from interviews to documents to observations and so forth.
 Basic principles

 Inductive analysis: emphasise bottom-up analysis of data and the build-up of

patterns, themes, and concepts into increasingly abstract units from the data.
 Evolutionary design: follow an evolutionary research process in which a
research plan, a theory, data collection, or analysis can unfold and change over
time as the research progresses.
 Often used terms and methods

 Interviews: Conversations with key informants

 Observations: observing phenomena/behaviors directly
 Documentation: studying documents, plans, schemes etc
 Triangulation: using multiple sources of data
 Coding: assigning tags or labels as units of meaning to pieces or chunks of data
collected
 Memoing: a subjective commentary or reflection about what was happening at
the time or place of the data collection
 Popular Qualitative Research Approaches

 Ethnographic Research
 Example: the study of a particular culture and their understanding of the role of a particular disease in their cultural
framework.

 Grounded Theory
 an inductive type of research, based or "grounded" in the observations or data from which it was developed

 Phenomenology
 describes the "subjective reality" of an event, as perceived by the study population; it is the study of a phenomenon.

 Critical Social Research

 used by a researcher to understand how people communicate and develop symbolic meanings.

 Ethical Inquiry
 an intellectual analysis of ethical problems. It includes the study of ethics as related to obligation, rights, duty, right
and wrong, choice etc.
 Popular Qualitative Research Approaches

 Activist Research
 aims to raise the views of the underprivileged or "underdogs" to prominence to the elite or master
classes, the latter who often control the public view or positions.

 Historical Research
 to discuss past and present events in the context of the present condition, and allows one to reflect and
provide possible answers to current issues and problems.

 Visual Ethnography
 uses visual methods of data collection, including photo, voice, photo elicitation, collaging, drawing, and
mapping.

 Autoethnography
 the study of self: a method in which the researcher uses their personal experience to address an issue.
 Qualitative versus Quantitative Methods

Quantitative Qualitative
to explain & predict; to test, confirm and validate
Purpose theory to describe & explain; to explore and interpret; to generate theory
focused; deals with known variables; uses
established guidelines; static designs; context free; holistic approach; unknown variables; flexible guidelines;
Research Process objective 'emergent' design; context bound; subjective
deductive - from general case (theory) to specific
Form of Reasoning situations inductive - from specific situation to general case
narrative description; words and quotes; personal voice; literary
Nature of Findings numerical data; statistics; formal and 'scientific' style
there is at least some objective reality that can be there are multiple, constructed realities that defy easy
Researcher Beliefs measured measurement or categorization

Research Literature relatively large relatively limited

Research Question confirmatory or predictive exploratory or interpretive

Research statistics and deductive reasoning, and able to write inductive reasoning, attentiveness to detail, and able to write in a
Skills Recker, J. (2022). Scientific more
in a technical and scientific style Research in Information
literary, Systems: A Beginner's Guide (2nd ed.). Springer.
narrative style
 Why and when we choose qualitative research

Research Design Decisions

Spectrum One end of Continuum Other End of Continuum
Method Qualitative vs. Quantitative
Aim Exploratory vs. Explanatory
Boundary Case vs. Statistical
Setting Field vs. Laboratory
Timing Longitudinal vs. Cross-sectional
Outcome Descriptive vs. Causal
Ambition Understanding vs. Predicting
Genres of Qualitative Research
Sarker, S., Xiao, X., Beaulieu, T., and Lee, A. S. 2018. "Learning from First-Generation Qualitative
Approaches in the Is Discipline: An Evolutionary View and Some Implications for Authors and Evaluators
(Part 1/2)," Journal of the Association for Information Systems (19:8), pp. 752-774.
 Inductive vs Deductive: Using Theory – or not

 A priori theory  No a priori theory

 Informs an understanding of possible  You enter data collection with a “blank
answers to RQ before you do your study slate”
 Can be used to develop interview  Avoids bias towards a certain
protocol perspective, idea or concept
 Identifies relevant concepts and  No guidance on interview protocol focus
relationships that you can develop  Needs to be broad, open and generative
questions about so you don’t “miss anything important”
 Allows you to evaluate the theory based  Can lead to the generation of entirely
on the results novel theory
 Do your interview findings resonate with
the theory, confirm or disconfirm it?
 May lead to findings that have already
been explained by existing theory
 Two Primary Uses of Qualitative Methods

 In exploratory research:  In explanatory research:

to discover to test, explain or compare

 Example  Example
 Dutta, S., Zbaracki, M.J., and  Markus, M.L. "Power, Politics,
Bergen, M. "Pricing Process as a and MIS Implementation,"
Capability: a Resource-based Communications of the ACM
Perspective," Strategic (26:6) 1983, pp 430-444.
Management Journal (24:7) 2003,
pp 615-630.
 Dutta, S., Zbaracki, M.J., and Bergen, M. "Pricing Process as a
Capability: a Resource-based Perspective," Strategic Management
Journal (24:7) 2003, pp 615-630.

• A primarily inductive application of the case study method

• Analysed data from a study of the pricing process of a large manufacturing firm
• Compared the data to existing theories.
• Then developed a new theory.
• Then returned to the data to see how the emergent theory matched the data.
• Finally returned to the theory for yet another revision
 Markus, M.L. "Power, Politics, and MIS Implementation,"
Communications of the ACM (26:6) 1983, pp 430-444.

 Compares three theories of resistance with the implementation of a computer

system, using an in-depth case study to test the predictions of each theory
 Theory 1: people resist change – people are the case of resistance
 Theory 2: resistance is determined by the environment of the technology
– technology is the case of resistance
 Theory 3: interaction between characteristics of people and technology –
both are the case of resistance
 The case data is used to contrast the explanatory and predictive power of the
theories
 Case study

 Case study research uses empirical evidence from one or more organizations
where an attempt is made to study the subject matter in context. Multiple
sources of evidence are used.
Myers, M.D. Qualitative Research in Business and Management Sage, Thousand Oaks, California, 2009.

 Three important points:

 In business, the case is very often a firm or organization even if the subject
matter is not.
 The difference to ethnography is that case study normally does not involve
participant observation or fieldwork.
 Case study research is per se philosophically neutral.
 Qualitative Research

 Case Study Designs Single-case designs Multiple-case designs

Case Case Case

Holistic

(single unit
of analysis)
Case Case

Case Case Case

Unit of Unit of
analysis 1 analysis 1
Unit of Unit of
analysis 2 analysis 2
Embedded
Unit of analysis 1
(multiple
units of
analysis)
Case Case
Unit of analysis 2 Unit of Unit of
analysis 1 analysis 1
Unit of Unit of
analysis 2 analysis 2
 Single-case Designs

 Single cases are often argued to be idiosyncratic – not affording great potential
for development of abstract, generalizable theory
 However, they are still useful in many situations for purposes of knowledge
contributions
 Often a particular rationale is needed for single-case designs
 Critical case: case meets all conditions for testing a theory.
 Unique case: case is extreme or rare.
 Representative case: case is typical for everyday/commonplace situations.
 Revelatory case: case presents a previously inaccessible opportunity.
 Longitudinal case: case reflects the change of a subject matter over two more different
points in time.
 Example Single Case Study

Seidel, S., Recker, J., & vom Brocke, J. (2013). Sensemaking and Sustainable Practicing: Functional
Affordances of Information Systems in Green Transformations. MIS Quarterly, 37(4), 1275-1299.
 Multiple-case Designs

 Often presents more completing evidence  research appears more robust

 Rationales for single-cases often cannot be satisfied by multiple cases
 Require more extensive resourcing and time
 Require a replication logic (a heuristic to select the additional cases):
 Literal: a case where similar results are predicted
 Theoretical: a case where contrasting results for anticipatable reasons are predicted
 Example Multiple Case Study

Lehmann, J., Recker, J., Yoo, Y., & Rosenkranz, C. (2022). Designing Digital Market Offerings: How Digital Ventures Navigate the Tension
Between Generative Digital Technology and the Existing Environment. MIS Quarterly, (46:3), DOI: 10.25300/MISQ/2022/16026.
 Embedded and Holistic Case Designs

 Embedded design means that there is more than one unit of analysis in a study
of one or several cases related to the same object of investigation.
 allows a researcher to define an appropriate set of subunits and thereby add to
the sensitivity of the investigation

 Holistic designs characterize case studies that investigate a phenomenon on a

more global level.
 advantageous either when no logical subunits can be identified or when the
theory itself is of a holistic nature.
 Qualitative Research Genres
- Case Study Procedures

 Planning refers to identifying the research questions and

other rationale for doing a case study.

Prepare  Designing refers to defining the unit of analysis, the

number and types of cases to be studied, and the potential
use of theory or propositions to guide the study.

 Preparing involves taking the necessary steps to conduct

high-quality data collection

Plan Design Collect  Collecting means executing the case study protocol(s)
and gathering data, preferably from multiple sources.

 Analysing consists of examining, categorising, coding,

tabulating, testing or otherwise combining and studying the
evidence collected to draw empirically based inferences
and other conclusions.

Share Analyse  Sharing refers to bringing case study results and findings
to a close by identifying and addressing relevant audiences
and providing them with the findings through appropriate
reporting or presentation.
Example: Positive Deviance
Planning:
Case Study protocols
Designing:
Case selection based on theoretical sampling
Positive Deviants

Sales

Customer penetration
Designing:
Case selection based on theoretical sampling
Collecting: Being In the field
 Qualitative Research Genres
- Action Research

 an interactive method of inquiry

DIAGNOSING
Identifying or defining the
problem

 builds upon the idea of introducing

SPECIFYING LEARNING
ACTION PLANNING
Considering alternative changes or other sorts of
interventions into a context and
Identifying general
courses of action to
findings
resolve the problem
studying the effects of those actions

 distinctive feature of action research

EVALUATING
Studying the
ACTION TAKING is the deliberate introduction of
interventions by the researcher
Selecting a course of
consequences of the
action
action
 Qualitative Research Genres
- Action Research

 Advantages
 the opportunity to contribute to both academic knowledge and to solving a real-world
problem.
 combines relevance and rigor in research.
 Disadvantages
 doing action and research together is a challenging act for anyone, let alone an
inexperienced scholar.
 assuming a position of a value-neutral, independent observer to the extent that it
allows for critical reflection and analysis, while at the same time maintaining a role
as an influencer and intervener .
 Access to participating organizations that put control to the researcher is hard to
organize.
 Qualitative Research Genres
- Grounded Theory

 a type of qualitative research that relies on inductive generation (building) of

theory based on (“grounded in”) qualitative data systematically collected and
analysed about a phenomenon.
 The main purpose of the grounded theory method is theory building, not testing.
 Prior domain knowledge should not lead to pre-conceived hypotheses or conjectures about
the research that the research then seeks to falsify or verify.
 The research process involves the constant endeavour to jointly collect and compare data,
and to constantly contrast new data to any emerging concepts and constructs of the theory
being built.
 All kinds of data are applicable, and are selected by the researcher through theoretical
sampling.
 Qualitative Research Genres
- Grounded Theory

 Advantages
 tight and early immersion into data analysis – unlike, say, quantitative research where data analysis is
typically conducted at a much later stage of the research process.
 encourages systematic and detailed data analysis and the literature provides ample guidelines for
conducting these steps.

 Disadvantages
 detailed and systematic bottom-up analysis of data: It is very easy to get bogged down in data analysis
on a very low level of detail
 it is difficult to integrate data to higher levels of abstraction.
 dependent on both excellent and rich data – collected typically before knowing what to look for
 dependent on creative and critical thinking ability – a skill not easily learned or taught.
 particularly challenging method especially for early career researchers.
 Doing Qualitative Research

Traditional secondary data  Traditional primary data collection

collection methods methods
 Archival analysis (review of  Interviews
documents or other media)  Focus groups
 Content from the Internet  Observations
 Open ended surveys
 Interviews for Data Collection

 A method for data collection that is Format Advantages Disadvantages

 Targeted: the focus is directly on a selected topic, Structured Features

consistency
Cannot follow
emergent new
 Insightful: can provide causal inferences as perceived and reliability lines of inquiry

by interviewees. Semi-structured Combines

strengths and
minimizes risk
 allowing some level of control: the interviewee can use Unstructured Allows free Requires
follow-up and probing questions to steer the talk by
interviewees
interviewee to
be in a free-
conversation into certain areas of interest. about what
they find
flowing,
talkative mode.
 flexible and responsive: can accommodate a range of important If too talkative,
white noise data
research problems and can be used to explore is generated.

additional research questions if they arise.

 allow the collection of rich and descriptive data
 a familiar method: a conversation with a purpose

14.02.2022 82
 Interview Protocol & Questions

 Items to consider:
 Do you have predetermined topics/concepts you wish to ask questions about?
 Topic guide format (semi-structured) or actual questions (fully structured)?
 How many questions will/can you ask?
 Less is more
 What question will you ask?
 Type (what/how/when/why/etc.)
 Ending (open/closed)
 Structure (unstructured/semi-structured/structured)
 Probes (pre-determined/emergent)
 Rough Procedure for Interviewing

 Entry: opening question

 Introduction: foster conversation
 Transition: move to key questions
 Core: 2-5 key questions
 Closure: ending the conversation
 Summative statement (“all things considered…”)
 Validation question (“so what I think you said was …”)
 Summary question: (“anything else you want to add?”)
 Qualitative Data Analysis Techniques

 Coding
 organizes raw data into conceptual categories, where each code is effectively a
category or ‘bin’ into which a piece of data is placed

 Memoing
 a subjective commentary or reflection on what was happening at the time or place of
the data collection

 Critical incident analysis

 identifying series of ‘events’ or ‘states’ that occur (e.g., in chronological order) and the
transitions between them

14.02.2022 85
Example Coding
Example Coding in Practice
Example Memoing
 Qualitative Data Analysis

 Data reduction: organize and

reduce massive qualitative data into
key insights
Data collection
 Data display: present rich data in Data display

accessible and varied form

 Conclusion-drawing: develop and
verify conclusions based on data Data reduction
Conclusion
and notes drawing/verifying

14.02.2022 89
 Example Data reduction

Bandara, W., G. G. Gable, and M. Rosemann (2005)

"Factors and Measures of Business Process Modelling:
Model Building Through a Multiple Case Study",
European Journal of Information Systems (14)4, pp.
347-360

14.02.2022 90
 Examples Data Display

Orlikowski, W.J., and Scott, S.V.

"Sociomateriality: Challenging the Separation
of Technology, Work and Organization," The
Academy of Management Annals (2:1) 2008,
Sarker, S., Sarker, S., Sahaym, A., and Bjørn-Anderson, N. "Exploring Value Cocreation in pp 433-474.
Relationships Between an ERP Vendor and its Partners: A Revelatory Case Study," MIS Quarterly
14.02.2022 (36:1) 2012, pp 317-338. 91
 Example: Conclusion-drawing

Lehmann, J., Recker, J., Yoo, Y., & Rosenkranz, C. (2022).

Designing Digital Market Offerings: How Digital Ventures
Navigate the Tension Between Generative Digital
Technology and the Existing Environment. MIS Quarterly,
(46:3), DOI: 10.25300/MISQ/2022/16026.
92
Design Methods

93
 Design Methods

 Procedures that feature methods to build and evaluate novel and innovative
artefacts (such as new models, methods or systems) as outcomes and which
are characterized by an emphasis on the construction of the artefact and the
demonstration of its utility.
 You can think of these procedures as having a focus on “artefacts”.
 Example: Corona Dashboards

 “We built this dashboard because we think it is important

for the public to have an understanding of the outbreak
situation as it unfolds with transparent data sources.”

• Used by billions of citizens

• Source of decision-making by policymakers and health 70,000,000 350,000

professionals 60,000,000 300,000

50,000,000 250,000

40,000,000 200,000

• How good are the dashboards? Can they be made 30,000,000 150,000

better?
20,000,000 100,000

10,000,000 50,000

,,0 ,0
Jan-20 Feb-20 Mar-20 Apr-20 May-20 Jun-20 Jul-20 Aug-20 Sep-20 Oct-20

Visits to [Link] web page (left-hand y-axis)

New monthly Covid-19 infections in Germany (right-hand y-axis)

14.02.2022 95
 Infections, deaths, and
estimated recoveries,

Example: Corona Dashboards

in total and by day

Infections by age group

(bars) and gender
(color)

Infections and deaths, Date of infection (blue)

by German states and date of record
generation (yellow)

Infections relative to
population by German
states (color coded)

14.02.2022 96
 Analysis of the RKI Dashboard

Dashboard element What is represented? What information is conveyed? What is the representation useful for?
Numerical attributes convey the values of
Total infections, deaths, For representation: they summarize the state of a collection of
Top element on right properties in general of the population of
and recoveries (as people (Germany’s population) at some point in time (the time of
hand side of Figure 1 Germany (i.e., a system of things that share
numbers). visit).
non-binding mutual properties).
Total infections (as For representation: they summarize the state of a subset of
Numerical and visual attributes convey the
Middle element on right numbers), by age group the collection of people (Germany’s population) that are of
values of properties in general of some
hand side of Figure 1 and gender (categories particular interest, because of presumed risk of infection, at
subsets of the population of Germany.
color coded). some point in time (the time of visit).
Sum of daily infections
and daily reported data, Numerical and visual attributes convey the
Bottom element on For state-tracking: The inclusion of temporal event data
by date (as numbers, the values of one property in general
right hand side of (successive dates) allows following the change in infection or
two types of data are (infections) of the population of Germany by
Figure 1 reporting data over time.
separated visually event (dates).
through color cording).
Numerical attributes convey the values of
For representation: they summarize the state of the collection
Left hand side element one property in general (infections) in
Total infections, by state. of people (Germany’s population) decomposed into sub-sets (by
in Figure 1 different subsets (states) of the population
state), at some point in time (the time of visit).
of Germany.
Total infections relative to Visual attributes convey the values of one For representation: they summarize the state of the collection
Middle element in population (categorized property in general (infections) in two of people (Germany’s population) decomposed into sub-sets (by
Figure 1 through color coding), by different subsets of the population (state state and population density), at some point in time (the time of
state (visual). and population density) of Germany. visit).

14.02.2022 Recker, J. (2021). Improving the State-Tracking Ability of Corona Dashboards. European Journal of Information Systems, 30(5), 476-495. 97
 Recommendations to Esri and RKI

1. It must at all times be possible to observe relevant

status indicators for particular collectives of people. Legend DP3 (External events)

high
(I: high, A: low, E: medium-high)
Actability
2. It should be possible to project relevant future states (low, medium, high)
based on extant transformation laws.

3. It must at all times be possible to track external

events (e.g., political interventions such as lockdowns,

Effectiveness
DP4 (Sequencing)
release of new technologies such as vaccines, or (I: high, A: medium, E: low-medium)

change in season) and map changes in state DP1 (Mapping)

variables (e.g., infection rate, death rate, etc.) to the (I: medium-high, A: low-medium, E: low-medium)
occurrence of those events.

4. It must be possible to track the sequence of relevant

events that occur.
DP2 (Tracking)

low
(I: medium, A: medium, E: low)

low high
Importance

14.02.2022 98
 What is design science

“A research paradigm in which a designer answers questions relevant to human

problems via the creation of innovative artefacts, thereby contributing new
knowledge to the body of scientific evidence. The designed artefacts are both
useful and fundamental in understanding that problem.“
Hevner, A. R., & Chatterjee, S. (2010). Design Research in Information Systems: Theory
and Practice (Vol. 22). Springer, p. 5.

“The research paradigm is about problem solving; it is about presenting

solutions through systems and IT artifacts, broadly defined as constructs,
models, methods, and instantiations. […] Design science is at the center of
innovation creation and solution building.

Paulo Goes, 2014: MISQ Editorial Comments.

14.02.2022 99
 Categorization Design Science Research

 Design Science Research is a pragmatic, problem-solving paradigm that

seeks to contribute to human knowledge via the creation of innovative artifacts.
 DSR is a prominent form of Engaged Scholarship, in which multiple key
stakeholders (researchers, users, clients, sponsors, practitioners) collaborate to
understand and address an important, complex problem/opportunity.
 Engaged scholarship: teaching and research that connect the resources of
the university to our most pressing social, civic, and ethical problems.
Van de Ven, A. H. (2007). Engaged Scholarship: A Guide for Organizational and Social Research. Oxford University Press.

100
 Design Science Research

 Design Science is a creative, engaged research paradigm that informs multiple

audiences:
 Researchers: Design principles and mid-range design theories
 Practitioners: Artifact (e.g., product and/or process) instantiations
 Managers: Work and application system controls
 Government: Economic and social welfare

101
 History and Origins

 Simon’s Nobel-prize winning work:

 Our world is full with artefacts that are

man-made, not naturally occurring.
 Artefacts as empirical phenomena
are “artificial” rather than “natural.”

 Because the artificial artefacts are

human-created, the science of artefacts
involves the study of the designs used
to perform tasks or fulfill goals and
functions with the artefact.

Sciences of the Artificial, 3rd Ed. – Simon 1996

 Historical Background

 Design has been prevalent throughout history

 Engineering, Education, Anthropology, Architecture, Art, …

 Design received scientific legitimacy through Simon’s work

 Design in Information Systems

 Always prevalent in many European countries such as
Germany and Scandinavia

 Received global attention in Information Systems in the 1990s

and 2000s
 Walls, J. G., Widmeyer, G. R., & El Sawy, O. A. (1992). Building an
Information Systems Design Theory for Vigilant EIS. Information
Systems Research, 3(1), 36-59.
 March, S. T., & Smith, G. F. (1995). Design and Natural Science
Research on Information Technology. Decision Support Systems,
15(4), 251-266.
 Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design
Science in Information Systems Research. MIS Quarterly, 28(1),
75-105.
 Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S.
(2007). A Design Science Research Methodology for Information
Systems Research. Journal of Management Information Systems,
24(3), 45-77.

103
 Main principle:
Bridging SCIENCE  TECHNOLOGY

• Technology Evolution (TE)

• Very Rapid, marked by continuous improvements
• Process driven by human and economic utilities
• See (Arthur, The Nature of Technology, 2009)

• Science Evolution (SE)

• Slow, marked by paradigm shifts
• Process driven by evaluation, gathering of empirical
evidence, and hypothesis testing
• See (Kuhn, The Structure of Scientific Revolutions, 1996)

• Technology Evolutions precede and drive Science Evolutions

• Science Evolutions ground and direct Technology Evolutions

104
 Focus of Design Methods: The artefact

 the research interest is on creating or changing such artefacts with the aim of improving on
existing solutions to problems or perhaps providing a first solution to a problem.

 Different types of artefacts are conceivable:

 Constructs (vocabulary and symbols)
 Models (abstractions and representations)
 Methods (algorithms and practices)
 Instantiations (implemented and prototype systems)
 Design theories (improved models of design or design processes)

105
 Key evaluation criterion: demonstrated utility

 The designed artefact created through design research must provide improved
utility beyond the current state of utility.
 Three implications:
a) the artefact’s demonstrated utility is novel.
b) the utility of an artefact in comparison to existing work makes a positive
difference.
c) a thorough evaluation provides decisive evidence of the artefact’s superior
utility.
 Definition of utility can vary (e.g., performance, effectiveness, efficiency).

106
Design research framework (overview)

Hevner, A.R. "A Three Cycle View of Design Science Research", Scandinavian Journal of Information Systems
(19:2) 2007, pp. 87-92.
Design research framework (detailed)

Environment Relevance IS Research Rigor Knowledge Base

People Foundations
•Roles •Theories
•Capabilities Build •Frameworks
•Characteristics •Design Theories •Experimental
•Experience •Artifacts Instruments
Organizations •Constructs
•Strategies Business Applicable •Models
•Structure •Methods
•Culture
Needs Knowledge •Instantiations
•Processes Assess Refine Methodologies
Technology •Experimentation
•Infrastructure •Data Analysis Techniques
•Applications •Formalisms
•Communications •Measures
Evaluate •Validation Criteria
Architecture
•Development Capabilities •Analytical •Optimization
•Case Study
•Experimental
•Field Study
•Simulation

Contributions to the Contributions to the

Application Environment Knowledge Base
108
 Components of the framework

 Build (constructing the artefact) and evaluate (testing the artefact) are the core research processes in
design research.

 Environment
 the problem space in which the phenomena of interest reside.
 E.g., people, organizational structures, and existing digital information or communication technologies and
infrastructures.
 Ensures relevance of the artefact

 Knowledge base
 provides the materials from and through which design science research is accomplished;
 I.e., prior research and results from reference disciplines provide foundational theories, frameworks, instruments,
constructs, models, methods, and instantiations that are available for use in the design phase.
 Ensures rigor of design science

109
 Components of the framework

 The relevance cycle bridges the research project’s contextual environment and
the design science activities.
 The rigor cycle connects the design science activities with the knowledge base
of scientific foundations, experience, and expertise that inform the research
project.
 The design cycle iterates between the core activities of building and evaluating
the design artefact and the research processes.
 All three cycles must be present and clearly identifiable in a design science
research project.

110
 Guidelines in design science

Guideline Description
Guideline 1: Design as an Artifact Design-science research must produce a viable artifact in the form of a construct, a model, a method, or an instantiation.

Guideline 2: Problem Relevance The objective of design-science research is to develop technology-based solutions to important and relevant business problems.

Guideline 3: Design Evaluation The utility, quality, and efficacy of a design artifact must be rigorously demonstrated via well-executed evaluation methods.

Guideline 4: Research Contributions Effective design-science research must provide clear and verifiable contributions in the areas of the design artifact, design foundations, and/or design
methodologies.
Guideline 5: Research Rigor Design-science research relies upon the application of rigorous methods in both the construction and evaluation of the design artifact.

Guideline 6: Design as a Search The search for an effective artifact requires utilizing available means to reach desired ends while satisfying laws in the problem environment.
Process
Guideline 7: Communication of Design-science research must be presented effectively both to technology-oriented as well as management-oriented audiences.
Research

Hevner, A.R., March, S.T., Park, J., and Ram, S. "Design Science in Information Systems Research", MIS Quarterly (28:1) 2004, pp. 75-105.
 The Artifact as Knowledge

Contribution type Examples

More abstract, complete, and Level 3. Well-developed Design theories (mid-
mature knowledge design theory about range and grand
embedded phenomena theories)
Level 2. Nascent design Constructs, methods,
theory – knowledge as models, design
↕↕↕↕ operational
principles/architecture
principles,
technological rules.
Level 1. Situated Instantiations
implementation of (software products or
artifact implemented methods)
More specific, limited, and less
mature knowledge

Gregor, S. and Hevner, A.R. "Positioning and Presenting Design Science Research for Maximum Impact", MIS Quarterly (37:2) 2013, pp. 337-355. 112
 Example level-1 contribution

Hilpert, H., M. Schumann, and J. Kranz (2013) "Leveraging Green IS in Logistics:

Developing an Artifact for Greenhouse Gas Emission Tracking", Business &
Information Systems Engineering (5)5, pp. 315-325
14.02.2022 113
 Example level-2 contribution

Seidel, S., Chandra Kruse, L., Székely, N., Gau, M., & Stieger, D. (2018). Design principles for sensemaking support systems in environmental
sustainability transformations. European Journal of Information Systems, 27(2), 221-247.
14.02.2022 114
 Example level-2 contribution

DP 1: Provide novel information in the form of environmental facts, observations, or general behavior , so that the system affords users disruptive
ambiguity and surprise in environmental sustainability transformations.

DP 2: Provide functions of storing and simple and unambiguous categorization of ideas, so that the system affords noticing and bracketing to users in
environmental sustainability transformations.

DP 3a: Provide features for interactive communication, so that the system affords users to engage in an open and inclusive discussion in environmental
sustainability transformations.

DP 3b: Provide users with an overview of all other users along with features for direct communication between users, so that the system affords users to
engage in an open and inclusive discussion in environmental sustainability transformations.

DP 3c: Provide features to relate comments to other comments so that the system affords users to comprehend circumstances and turning them into
words and categories on a social ground in environmental sustainability transformations.

DP 3d: Provide features to assign roles to users so that the system affords user specific actions, such as moderation of discussions in environmental
sustainability transformations.

DP 4a: Provide features for categorization of action possibilities to distinguish presumptions from actual planed actions, so that the system affords users
presumption and action planning in environmental sustainability transformations.

DP 4b: Provide features for dedicated feedback about the implementation and consequences of the implementation of actions in environmental
sustainability transformations.

Seidel, S., Chandra Kruse, L., Székely, N., Gau, M., & Stieger, D. (2018). Design principles for sensemaking support systems in environmental
14.02.2022 sustainability transformations. European Journal of Information Systems, 27(2), 221-247. 115
 Example level-3 contribution

 Belief formation captures how beliefs, desires, orientations etc. Function

Form: Required Environmentally Exemplary system
about the natural environment are formed. Relevant Affordances designs
Macro level
 Macro-level: the ways an organization coordinates and divides labor Review
environmental Selected material Suitable symbolic Systems with reflective
and how the organization defines environmental expectations of its disclosure

Outcome assessment
sustainability of properties expressions
affordances*
agents. decisions
Data collection Dashboard
Micro level
 Micro-level: how an individual forms beliefs about the natural Assess Data analytics Traffic light system

environment environmental
Sustainability of Data presentation Navigation systems
Systems with output
monitoring
affordances*
work practices

 Action formation describes how psychic states about the natural Macro level
environment translate into actions. Select
environmentally Selected material Suitable symbolic Systems with live

Action formation
sustainable properties expressions decision review
 Macro-level: actions taken by an organization to affect the actions decisions
affordances

taken by its agents. Impact assessment Decision tree

Micro level
Instant response
Perform Workflow engine
 Micro-level: actions taken by agents environmentally
system
Systems with work
virtualization
sustainable work Decision support Action wizard
affordances
practices

 Outcomes describe what the consequences of the actions are. Macro level
Sensemaking of
 Macro-level: the measure of the environmentally functioning of environmental Selected material Suitable symbolic
Systems with
information
organizations.

Belief formation
sustainabIlity properties expressions democratization
decisions affordances*
Recommendation
 Micro-level: the measure of the environmental behavior of agents Social networking
agent
Micro level
Recommender
Create attitude for functionality
Newsfeed
environmentally Systems with attitude
sustainable work Knowledge sharing Avatar representations
Meta-requirement 1: A Green IS must perform one or more of
reflection affordances
 practices

these functions.

Recker, J. (2016). Toward A Design Theory for Green Information Systems 49th Hawaiian International Conference on Systems Sciences, Kuaui, Hawaii.
14.02.2022 116
 Propositions of the Design Theory

 Scope of operation
1. Any Green IS instantiation needs to operate at the level of belief formation, action
formation, or outcome measurement.
2. Green IS instantiations will be more effective if they operate at the level of belief
formation, action formation, and outcome measurement rather than one of the
levels only.

 Level of operation
3. Any Green IS instantiation needs to operate at least at the macro or the micro
level of organizations.
4. Any Green IS instantiation will be more effective if they operate at both the macro
and micro level rather than one level only.
 Propositions of the Design Theory

 Effective utility
5. To provide effective utility, any Green IS instantiation requires the provision of
actualizable environmentally relevant functional affordances at either the macro or
micro level.
6. In any Green IS instantiation, environmentally relevant functional affordances
need to be designed such that required material properties are accompanied by
suitable symbolic expressions appropriate for the intended user groups.
 How powerful are current “Green Systems”?

 We searched all publications on “green/sustainability information systems”,

across all sorts of publication databases
 We found 416 in total since 2010; 74 that presented some sort of digital system
artefact.
 36 of these were able to measure and report environmental data.
 Only 6 of these can be used to change people’s attitudes and beliefs.
 Only 11 of these allowed the users to actually make green actions or
decisions.
 What are the knowledge contributions that are
possible?

Improvement: Develop Invention: Invent new

solutions for new

Low
new solutions for known
problems problems
Research Opportunity Research Opportunity

Solution Maturity

Routine Design: Apply Exaptation: Extend known

High
known solutions to known solutions to new problems
problems (e.g. Adopt solutions from
other fields)
Research Opportunity

High Low
Application Domain (Problem) Maturity

Gregor, S. and Hevner, A.R. "Positioning and Presenting Design Science Research for Maximum Impact", MIS Quarterly (37:2) 2013, pp. 337-355. 120
 Invention Quadrant

 An invention is a radical breakthrough; a departure from accepted ways of thinking and

doing

 DSR projects in which little understanding of the problem context exists and no effective
artifacts are available as solutions

 Research contributions are usually novel artifacts or inventions, i.e., level 1 artifacts

 The newness of artifact makes this research difficult

 Insufficiently grounded in theory
 Design is incomplete and not fully evaluated
 Understanding is insufficient to provide new contribution to theory via the design

121
 Invention Example

 Agrawal, R., Imielinski, T. and Swami, A. (1993). “Mining Association Rules between
Sets of Items in Large Databases”, Proceedings of the 1993 ACM SIGMOD
Conference, Washington DC, May.
 Aim: produce an algorithm that generates all significant association rules
between items in the database
 Practical importance: Allows organizations to find interesting relationships (e.g.
shopping patterns)
 Theoretical significance (newness): Shows (Sect 5) that no other work has done
same thing
 Description of new method: Shows requirements (Sect 1), new concepts
(association rule, support, confidence), Formal Model (pseudocode) (Sects 2-3)
 Proof: Experiments (Sect 4)
122
 Improvement Quadrant

 An improvement is a better artifact solution in the form of more efficient and

effective products, processes, services, technologies, or ideas.
 DSR projects in which the problem context is mature but there is a great need
for more effective artifacts as solutions
 Improvement DSR is judged by:
 Clearly grounding, representing, and communicating the new artifact design
 Convincing evaluation providing evidence of improvements over current solutions

 All levels of artifact knowledge contribution can be made

123
 Improvement Examples

 Many DSR projects in IS are in the Improvement Quadrant, for example:

 Better data mining algorithms for knowledge discovery (extending the
initial ideas invented by Agrawal et al. (1993)); for example, (Fayyad et
al. 1996; Zhang et al. 2004; Witten et al. 2011)
 Improved recommendation systems for use in e-commerce; for example
(Herlocker et al. 2004; Adomavicius and Tuzhilin 2005)
 Better technologies and use strategies for saving energy in IT
applications; for example (Donnellan et al. 2011; Watson and Boudreau
2011)
 Improved routing algorithms for business supply chains; for example (van
der Aalst and Hee 2004; Liu et al. 2005)
124
 Exaptation Quadrant

 An exaptation is the expropriation of an artifact in one field to solve problems in

another field
 DSR projects in which the problem context is not well understood but there exist
mature artifacts in other fields that can be exapted as effective solutions
 Exaptation DSR is judged by:
 Clearly grounding, representing, and communicating the exapted artifact design
 Convincing evaluation providing evidence of how well the new artifact solves the
given problem

 All levels of artifact knowledge contribution can be made

125
 Exaptation Examples

 Exaptation DSR is employed when new technologies provide opportunities to solve new
and/or different IS problems; for example:
 Codd’s exaptation of relational mathematics to the problem of database systems design
leading to relational database concepts, models, methods, and instantiations (Codd,
1970; Codd, 1982)
 Berners-Lee original concept of the World Wide Web was one of simply sharing research
documents in a hypertext form among multiple computers. In short time, however, many
individuals saw the potential of this rapidly expanding interconnection environment to
exapt applications from old platforms to the WWW platforms. These new Internet
applications were very different from previous versions adding many new artifacts.
 Research by Berndt et al. (2003) on the CATCH data warehouse for health care
information. Well-known methods of data warehouse development (e.g. Inmon, 1992)
were exapted to new and interesting areas of health care systems and decision-making
applications.

126
 Routine Design Quadrant

 Professional design or system building to be distinguished from DSR

 However, evolving or best practices may be observed and documented in
“extractive case study” work (Van Aken)
 Study of best practices in routine design may lead to empirical generalization
 Example – Davenport’s observation of BPR (Davenport & Short SMR 1990)

127
 MISQ Papers mapped to Framework
Knowledge Article Knowledge Contribution Claims
Contribution
Improvement A Multilevel Model for Measuring Fit Between a Firm’s Competitive Strategies and There is a need for a more fine-grained model for diagnosing the individual IS capabilities that contribute to the overall fit or
Information Systems Capabilities (McLaren et al., 2011) misfit between a firm’s competitive strategies and IS capabilities (p.2) (See also Table 4).

Improvement Guidelines for Designing Visual Ontologies to Support Knowledge Identification There could be several ways to address OWL’s inability to show state changes… We have taken a different path, taking the
(Bera et al., 2011) view that we can keep the existing OWL syntax and improve the extent to which it support s knowledge identification (pp.
885-886).
Exaptation Co-creation in Virtual Worlds: The Design of the User Experience (Kohler et al., While Nambisan and his colleagues provide a useful framework for the online environment in general, little is known about
2011) designing co-creation experiences in virtual worlds (p. 774).

Exaptation Design Principles for Virtual Worlds ABVWs comprise a new class of information systems… Thus, they require an extension of the corresponding information
(Chaturvedi et al., 2011) system design principles (p. 675)

Improvement Correlated Failures, Diversification, and Information Security Risk Management While our model to estimate security loss due to unavailable (i.e., system downtime) is based on well-established queuing
(Chen et al., 2011) models, one innovation of our model is that the distribution from which the number of requests sent to the queue is drawn
is endogeneous to system variables (p. 399).
Exaptation The Effects of Tree-View Based Presentation Adaptation on Mobile Web Browsing. Presentation adaptation has been studied in the desktop environment and has been proven beneficial … However, research
(Adipat et al., 2011) on adaptation of Web content presentation for mobile handheld devices is still rare (p. 100).

Improvement Improving Employees’ Compliance Through Information Systems Security Training: There is a need for IS security training approaches that are theory-based and empirically evaluated. … (p. 757). To address
An Action Research Study. (Puhakainen and Sipponen 2010) this deficiency … this paper developed a theory-based training program … This paper then tested the practical workability
through an action research intervention (p. 776).

Improvement Detecting Fake Websites: The Contribution of Statistical Learning Theory. (Abbasi et Systems grounded in SLT can more accurately detect various categories of fake web sites (p. 435).
al., 2010)

Improvement The Design Theory Nexus. ( Pries-Heje and Baskerville, 2008) The work suggests that the design theory nexus approach is more universal than previous approaches to contingency
theory, because it can operate in both symmetrical and asymmetrical settings (p. 748).
Improvement Process Grammar as a Tool for Business Process Design. (Lee et al., 2008) The method improves on existing approaches by offering the generative power of grammar-based methods while
addressing the principal challenge to using such approaches … (p. 757).
Improvement Making Sense of Technology Trends in the Information Technology Landscape: A Our approach may complement existing technology forecasting methods … by providing structured input and formal
Design Science Approach. (Adomavicius, et al., 2008) analysis of the past and current states of the IT landscape (p. 802).
Improvement CyberGate: A Design Framework and System for Text Analysis of Computer- The results revealed that the CyberGate system and its underlying design framework can dramatically improve CMC text
Mediated Communication. (Abbasi and Chen 2008) analysis capabilities over those provided by existing systems (p. 811).
Improvement Using Cognitive Principles to Guide Classification in Information Systems Modeling. Despite the importance of classification, no well-grounded methods exist .. (p. 840). We provide empirical evidence…that
( Parsons and Wand 2008) the rules can guide the construction of semantically clearer and more useful models (p. 858).
128
 Challenges in doing design research

 Design research projects are usually:

 Team-based
 Longitudinal
 Goal-driven

 All of these are difficult.

129
 Challenges in doing design research

 Attempting to do good design research is an audacious venture

 It is not for those that value optimal and repeatable results
 Relying on existing theories often does not produce predictable results
 Multiple, rapid cycles of build and evaluate produce emergent and satisfactory
results
 Even the most useful results might become eclipsed by rapid changes in the
problem and solution spaces

130
© Alan Hevner, Presentation at CIIS Dec 2019
 Some Challenges to Doing Good DSR

1. Complexity
2. Confidence
3. Contribution

131
 Complexity

 IS Research studies Complex Socio-Technical Systems (Sarker et al. MISQ 2019)

 Information systems are complex artefacts :
 Diverse
 Interdependent
 Connected
 Adaptive
 We attempt to Manage Complexity by Capturing/Representing the DSR Problem
Space:
 Context (Domain, Stakeholders, Time, Space)
 Goodness Criteria (Goals, Evaluation Measures)
 Dancing Landscape (Emergent Behaviors, Self-Organization, Change)

December 2019 132

 Manging complexity: Representing the problem
space
 The first step of any DSR project is understanding and representing the Problem.

 “Every problem-solving effort must begin with creating a representation for the problem – a
problem space in which the search for the solution can take place. … Occasionally, however,
we encounter a situation that doesn’t seem to fit any of the problem spaces we have
encountered before, even with some stretching and shaping. Then we are faced with a task
of discovery that may be as formidable as finding a new natural law.” Simon 1996, p. 108
 “This view can be extended to all of problem solving – solving a problem simply means
representing it so as to make the solution transparent.” Simon 1996, p. 132

 But: Problem of “The Dancing Landscape” – The introduction of a design solution into a
problem space changes the problem space. The next DSR cycle faces a new problem.

133
 DSR Goals and Evaluation Measures

 The identification of DSR goals and their transformation into well-defined evaluation
measures are understudied topics.
 What are the important goals of the different stakeholders and how do we reconcile
conflicting goals?
 How can we prioritize and weigh goals in a problem ‘utility’ function?
 How do we measure the achievement of these goals?
 How do we evaluate the “goodness of fit” of a designed artifact as a solution in an application
environment?
 How do we rank potential design candidates so as to select the best one for implementation
as a solution?

 Such questions require the DSR project team to define the goals and evaluation
measures for the project and design rigorous methods for evaluating the design
artifacts under these criteria. The rigor and credibility of a DSR project is determined by
these evaluation decisions.
134
 Design Evaluation Methods – The Toolbox

1. Observational Case Study – Study artifact in depth in business environment

Field Study – Monitor use of artifact in multiple projects
2. Analytical Static Analysis – Examine structure of artifact for static qualities (e.g., complexity)

Architecture Analysis – Study fit of artifact into technical IS architecture

Optimization – Demonstrate inherent optimal properties of artifact or provide optimality bounds on artifact behavior

Dynamic Analysis – Study artifact in use for dynamic qualities (e.g., performance)

3. Experimental Controlled Experiment – Study artifact in controlled environment for qualities (e.g., usability)

Simulation – Execute artifact with artificial data

4. Testing Functional (Black Box) Testing – Execute artifact interfaces to discover failures and identify defects

Structural (White Box) Testing – Perform coverage testing of some metric (e.g., execution paths) in the artifact implementation

5. Descriptive Informed Argument – Use information from the knowledge base (e.g., relevant research) to build a convincing argument for the artifact’s utility

Scenarios – Construct detailed scenarios around the artifact to demonstrate its utility

135
 Selection of Evaluation Methods

 Match Evaluation Methods to:

 Research Question
 Goals and Evaluation Measures
 Hypotheses and Dependent Variables
 Application Context
 Controls in the Application Context
 Availability of Data Sources
 Qualitative
 Quantitative
 Primary vs. Secondary
 Research Team Evaluation Skills
 Research Evaluation Tools
 Evaluation Methods will be different for Evaluation in Lab (Formative) vs. Evaluate in Context
(Summative)
136
 Contributions

 DSR must make contributions to the

application environment and to the
knowledge base (Hevner et al.
2004).
 The design artifact embodies new
knowledge at varying levels of
abstraction (Gregor and Hevner
2013).
 Design is also a verb, so we learn
something about designing, as well.

137
 Producing and Consuming Knowledge

A. Drechsler and A. Hevner, (2018) Utilizing, Producing, and Contributing Design Knowledge in DSR Projects, DESRIST 2018, India.
138
Computational Methods

139
 Computational Methods

 An umbrella term that describes a variety of software tools that assist with such
research processes as data generation or discovery, data processing or
cleansing, and data analysis or interpretation.
 You can think of these procedures as having a focus on “digital trace data”.
 Digital trace data

 Evidence of activities and events that are logged and stored digitally.
 Many things people do these days involves or is mediated by digital
technologies.
 Historically, text data such as emails, transaction data from enterprise systems,
and posts and comments on social media and networking platforms are all forms
of digital trace data.
 Today, bio health data recorded by wearables, logs produced by digital objects
such as toothbrushes and energy meters, and traces generated by digital
objects such as electric vehicles are also forms of digital trace data.

141
 Examples: research with digital traces

 Vaast, E., Safadi, H., Lapointe, L., & Negoita, B.

(2017). Social Media Affordances for Connective
Action: An Examination of Microblogging Use During
the Gulf of Mexico Oil Spill. MIS Quarterly, 41(4),
1179-1205.
 analyzed more than 23,000 tweets that carried the
hashtags #oilspill or #bpoilspill
 First study that showed that individuals engaged in
collective action by coproducing and circulating social
media content based on an issue of mutual interest.

142
 Examples: research with digital traces

Stop the drill Boycott BP Hair and Fur

Encouraging efforts to Encouraging people not to Encouraging the collection

stop offshore drilling, remain customers of the of hair and fur to create
often via the signature of company widely perceived boons that absorb oil
online petitions. as at the origin of the oil spreading in the sea.
spill.
“#oilspill knows no mercy “Declare your Oil “Fight the #oilspill: cut
for anyone or anything – Independence on your hair
national parks threatened Independence Day [Link] RT”
by #oil [Link] Weekend. Boycott #BP. Oil “GREAT to SEE! People R
#stopthedrill” “Yes! to a Spill #boycottbp #oilspill” clicking the link to donate
permanent FL drilling ban “If only the seafood in the hair or fur to save Gulf
- Video: Gulf/Keys were basting in Coast wetlands
[Link] olive & not crude oil. But [Link]
_wvs #oilspill #sayfie” nooooo. #oilspill #FAIL #oilspill”
#BoycottBP”
701 tweets, 500 tweeters 897 tweets, 602 tweeters 284 tweets, 258 tweeters
14.02.2022 143
 One day at the U. of Rochester dermatology
clinic
Pentland, B. T., Recker, J., Ryan Wolf, J., & Wyner, G. (2020). Bringing Context Inside Process Research With Digital Trace Data. Journal of the Association for Information Systems, 21(5), 1214-1236.

Clock time

Event time
 Figuring out what‘s going on.

Social Network Narrative Network / Process model

(relationships between involved actors) (relationships between involved actions)
 Figuring out what‘s going on.

Affordance Network
(actions carried out by actors using technologies)
Adding context unravels the graph.

(a) Affordance network (b) Affordance network plus “location”

 Digital trace data is difficult research data

 Digital trace data is organic, not designed: a byproduct of activities, not data
generated for the purpose of research.
 They appear organically and researchers “find” and collect them.
 Researchers have less control over the validity of organic data than they do
over designed research data because the data-generation process is opaque
(or even unknown), and we have little to no control over that process.
 For example, we do not know why, how, or in what context some Twitter
posts in a conversation were made.

148
 Digital trace data is difficult research data

 Digital trace can be both heterogeneous and unstructured: they often include
text, images, video, or sound.
 The richness of such data can be a strength because it expands the number of
perspectives of a phenomenon
 But it also makes the data more difficult to analyze.

149
 Digital trace data is difficult research data

 Digital trace can be enormous in volume: it typically records fine-grained

events and actions such as individual clicks, posts, and comments.
 provides a more precise view of behaviors and occurrences than traditional
modes of collection
 But the sheer size of the data can also quickly become overwhelming for
scholars.
 Example: it is not feasible to manually code comments made by the 257 million
followers of Christiano Ronaldo on Instagram.

150
 Digital trace data is difficult research data

 Digital trace is inherently event-based: it connects actions and behaviors that

are enabled or mediated by digital technologies as they unfold at various points
in time.
 Time-stamps are great because they allow analysis of temporal aspects
(dynamics, change, transformation, exceptions, etc.)
 But it is also notoriously difficult to analyse with standard scientific analysis
tools.

151
 Computational Data Processing and Analysis

 Techniques and tools that have in common that certain steps during data
generation, processing, or analysis are carried out through—or with the help
of—algorithms that either augment manual work or fully automate an otherwise
manual activity.

152
Overview

153
 Computational support tools

 Assist researchers in carrying out their work through dedicated research

computer software.
 The software tools do not carry out the steps involved in the research but
support scholars in carrying them out in the sense that they assist manual
activities and make them easier or faster to complete.
 Examples for data generation: Skype and Zoom can record video and audio
streams digitally
 Examples for data analysis: R, SPSS, LISREL, Nvivo, [Link]

14.02.2022 154
 Computational augmentation tools

 Software that is used to complement and amplify human activity, rather than to
supplant it.
 Example data analysis:
 statistical software packages such as LISREL automatically make suggestions for (re-) specifying a
hypothesized model based on shared correlations between the latent constructs it discovers in the
covariance matrix of observed data.
 Researchers may implement these suggestions or not; because the suggestions are empirically
based, they may or may not be conceptually logical or plausible.

 Example data generation:

 researchers often write scripts that help them process a web document and extract information from
it (scraping) or assist them in iteratively finding and fetching web links beginning with a list of seed
web domains (crawling).

155
 Computational augmentation tool: Example
LitSonar
 [Link]
 A literature search algorithm that can scan millions of documents for the
presence of keywords
 Helps researchers to identify related literature, but they must still read and
assess the papers for relevance.

156
 Computational augmentation tool: Example
Leximancer
 [Link]
 A text analysis for in-depth analysis of the text.

 Produces concepts mads that allows researchers to explore and gain insight from the key
concepts in texts.

 Researchers must still interpret the outcomes.

157
 Computational augmentation tool: Example
Leximancer

 [Link]
 A text analysis for in-depth analysis of the text.

 Produces concepts mads that allows researchers to explore and gain insight from the key
concepts in texts.

 Researchers must still interpret the outcomes.

158
 Automated Concept Analysis:
Example Leximancer

What do business journals publish?

Fields and Journals Years No of Abstracts
Information Systems
Information Systems Research [ISR] 1990-2001 258
MIS Quarterly [MISQ] 1977-2001 607
Management
Academy of Management Journal [AMJ] 1977-2001 1938
Academy of Management Review [AMR] 1977-2001 1236
Administrative Science Quarterly [ASQ] 1977-2001 736
Strategic Management Journal [SMJ] 1980-2001 1167
Accounting
Accounting Review [AR] 1977-2001 1200
Journal of Accounting & Economics [JAE] 1977-2001 442
Journal of Accounting Research [JAR] 1977-2001 960
Indulska, M., Hovorka, D.S., and Recker, J. "Quantitative Approaches to Content Analysis: Identifying
Total Conceptual Drift Across Publication Outlets," European
1977-2001 8544
Journal of Information Systems (21:1) 2012, pp 49-69.
Movement of themes over time
In IS journals 1977-1981
Movement of themes over time
In IS journals 1997-2001
Identifying concept drift

75 70
Accounting Accounting
(R² = 0.57) (R² = 0.71)
65
70
60

65
55

Degrees

Degrees
60 Management
50 IS
(R² = 0.25)
(R² = 0.25)
45

55
40

IS Management
50 35
1977 1981 1985 1989 1993 1997 2001 1977 1981 1985 1989 1993 1997 2001
Time Time

Figure 3a: Information Systems vs. Accounting and Figure 3b: Management vs. Information Systems and
Management Journal Abstract Samples Accounting Journal Abstract Samples

75

IS )
(R² = 0.57)
70

65
Degrees

60
Management
(R² = 0.71)

55

50
Accounting
1977 1981 1985 1989 1993 1997 2001

Time

Figure 3c: Accounting vs. Information Systems and

Management Journal Abstract Samples
 Computational automation tools

 Software tools that carry out algorithmic data generation, processing, or

analysis with little to no human intervention or oversight.
 Examples:
 Text mining automatically extracts information from text through algorithms that extract
and parse text, classify text, derive patterns in text, and evaluate and display text using
statistics, graphs, and/or visual diagrams.
 Social network analysis uses algorithms that automatically parse person-relational
data, categorize them based on statistical properties, and display the information
statistically and/or graphically.
 Supervised or unsupervised machine learning algorithms can automatically find
patterns and relationships in the data that would be unlikely to find manually.

163
 Advantages and Challenges of Computational
Methods

 Advantages  Challenges
 can substantially expand the reach  few clear and robust methodological
and scope of research guidelines available
 can take substantially less time than  make it challenging to focus on and
manual execution of research tasks account for the context(s) in which
 can increase the reproducibility of digital trace data were generated
data processing and analysis and  Data validity threats from errors in
help reduce human biases in these algorithmic outputs, benign errors
tasks from relying on probabilistic
algorithms such as random search,
and lack of generalizability and
replicability

164
Mixed Methods

165
 Mixed Methods

 A type of inquiry that features the sequential or concurrent combination of

methods for data collection and analysis.

 Historically: mixing of methods from quantitative and qualitative research traditions

 Nowadays: increasingly also mixing of methods from other traditions, such as
design plus quantitative methods or computational plus qualitative methods.
 Aims of Mixing Methods

1. Strengthening inferences,
2. Providing a greater diversity of views, and
3. Enabling researchers to answer confirmatory and exploratory questions
simultaneously (verifying and generating theory at the same time)
 In other words: Mixing methods tries to
 leverage the complementary strengths of research methods and mitigate
their weaknesses
 offer deeper insights into a phenomenon than each of the methods alone
could provide
 Five Main Different Purposes of Mixing
Methods
 Triangulation
 establish convergence of, and corroborate results from, multiple methods and designs used to study the same
phenomenon

 Complementarity
 elaboration, enhancement, illustration, and clarification of the results from one method with results from another
method

 Initiation
 finds paradoxes and contradictions in one study that lead to a re-framing of the research questions using a different
method

 Development
 uses the findings from one method to help inform the other method.

 Expansion
 used to expand the breadth and range of research by using different methods for different components of an inquiry
Venkatesh, V., Brown, S.A., and Bala, H. "Bridging the Qualitative-Quantitative Divide: Guidelines for Conducting Mixed Methods Research in Information Systems," MIS
Quarterly (37:1) 2013, pp 21-54.
 Mixed Method Designs

 Key design component: Timing

 the temporal ordering of the phases
in which the methods are carried out:
 sequential (one after another),
 parallel (both separately but
concurrently),
 conversion (data from one method is
transformed to be used with another
method),
 or fully integrated (all at once).

14.02.2022 169
Remember: Positive Deviance
 Mixed Method Design

 Study of Positive Deviance, Management and Leadership

 Involved qualitative exploration of 19 stores across Australia
 Measurement development and theorizing through engagement with literature
 Cross-sectional multi-level survey (managers and dept. managers)

Data Data
Collection I Analysis I Inference I
Research Meta-
QL QL QL
questions I inference
Conversion

QN

Research
QN QN QN
questions II
Data Data
Collection II Analysis II Inference II
Sampling

Positive Deviants!

Sales

Customer penetration
Conduct
 Mixed Method Findings

Mertens, W., Recker, J., Kummer, T.-F., Kohlborn, T., Viaene, S. (2016): Constructive Deviance as a Driver for Performance in Retail.
Journal of Retailing and Consumer Services, Vol. 30, pp. 193-203.
 Mixed Method Designs

 Other important design decisions:

 Weighing (deciding whether to give the quantitative and qualitative

components of a mixed study equal status or to give one paradigm the
dominant status);
 Mixing, which can form a continuum from mono-method to fully mixed
methods; and
 Placing, that is, deciding where mixing should occur (in the research
questions, data collection, data analysis, or data interpretation).
 Mixed Method Design Particularities

 Data transformation
 Data must be transformed between data formats to suit the differing analysis
techniques
 Typically needed in concurrent, conversion, and integration mixed method
research
 Examples:
 qualitative data (e.g., codes) may have to be quantified (e.g., by counting the
frequency of occurrence in text)
 quantitative data may have to be qualified (e.g., annotated with text)
 Mixed Method Design Particularities

 Data correlation
 Data about the same phenomenon collected using multiple methods must be
compared with a view to identifying triangulation outliers that may require
further analysis.
 often achieved using a data matrix that combines a quantitative axis and a
qualitative axis to identify similarities and differences.
 Mixed Method Design Particularities

 Legitimation
 the description of the steps undertaken to ensure the validity, accuracy,
and/or plausibility of meta-inferences.
 The quality of meta-inferences depends on the strength of inferences that
emerge from the study’s individual methods.
 The individual inferences can be divergent, convergent, or
complementary, each of which require legitimation.
 Strengths and Weaknesses of Mixed Method
Research

Strengths Weaknesses
Words, pictures, and narrative can be used to add meaning to It can be difficult for a single researcher to carry out both qualitative
and quantitative research, especially if two or more approaches are to
numbers. be used concurrently.
Numbers can be used to add precision to words, pictures, artefacts, The researcher has to learn about multiple methods and approaches
and learn how to mix them appropriately.
and narrative.
The research can benefit from the individual strengths of different Methodological purists contend that one should always work within
either a qualitative or a quantitative paradigm.
research methods.
The researcher can more easily generate and rigorously test a theory. Mixed method research is typically more resource-intensive than
research that uses a single method and may require a larger research
team.
Mixed method research can answer a broader and more complete Mixed method research is typically more time consuming than mono
method research.
range of research questions.
Mixed method research can be used to provide stronger evidence for a Some of the details of mixed research remain to be worked out fully by
research methodologists.
conclusion.
Mixed method research can be used to increase the generalisability of Mixed method research can be difficult to publish
(e.g., because it requires more space).
results.
 Examples of mixed method research

 Wunderlich, P., Veit, D. J., & Sarker, S. (2019). Adoption of Sustainable Technologies: A Mixed-
Methods Study of German Households. MIS Quarterly, 43(2), 673-691.

 Spiegel, O., Abbassi, P., Zylka, M. P., Schlagwein, D., Fischbach, K., & Schoder, D. (2016). Business
Model Development, Founders’ Social Capital and the Success of Early Stage Internet Start-Ups: A
Mixed-Method Study. Information Systems Journal, 26(5), 412-449.

 Califf, C. B., Sarker, S., & Sarker, S. (2020). The Bright and Dark Sides of Technostress: A Mixed-
Methods Study Involving Healthcare IT. MIS Quarterly, 44(2), 809-856.

 Mertens, W., Recker, J., Kummer, T.-F., Kohlborn, T., & Viaene, S. (2016). Constructive Deviance as a
Driver for Performance in Retail. Journal of Retailing and Consumer Services, 30, 193-203.

14.02.2022 180
 End of Chapter 5
© Copyright 2021 Jan Recker. All Rights Reserved.

14.02.2022 Teaching Notes: Scientific Research in Information Systems ~ © Copyright 2021 Jan Recker. All Rights Reserved. ~ 181