D ESIGNING E XPLAINABLE P REDICTIVE M ACHINE L EARNING

A RTIFACTS : M ETHODOLOGY AND P RACTICAL D EMONSTRATION

Giacomo Welsch, Peter Kowalczyk

Chair of Information Systems Engineering
University of Würzburg
Würzburg
arXiv:2306.11771v1 [[Link]] 20 Jun 2023

{[Link], [Link]}@[Link]

A BSTRACT
Prediction-oriented machine learning is becoming increasingly valuable to organizations, as it may
drive applications in crucial business areas. However, decision-makers from companies across
various industries are still largely reluctant to employ applications based on modern machine learning
algorithms. We ascribe this issue to the widely held view on advanced machine learning algorithms
as "black boxes" whose complexity does not allow for uncovering the factors that drive the output of
a corresponding system. To contribute to overcome this adoption barrier, we argue that research in
information systems should devote more attention to the design of prototypical prediction-oriented
machine learning applications (i.e., artifacts) whose predictions can be explained to human decision-
makers. However, despite the recent emergence of a variety of tools that facilitate the development of
such artifacts, there has so far been little research on their development. We attribute this research gap
to the lack of methodological guidance to support the creation of these artifacts. For this reason, we
develop a methodology which unifies methodological knowledge from design science research and
predictive analytics with state-of-the-art approaches to explainable artificial intelligence. Moreover,
we showcase the methodology using the example of price prediction in the sharing economy (i.e., on
Airbnb).

Keywords Design Science · Machine Learning · Explainable AI · Predictive Analytics

1 Introduction
Machine learning (ML) is a focal element of digitization that affects many areas of modern society: besides driving
a plethora of physical and virtual products already woven into our daily lives, such as smartphones and social media
platforms, ML techniques can be leveraged to power a wide range of business applications [1, 2]. Although ML as
an umbrella term comprises various techniques, some of which are aimed at different purposes, most ML algorithms
are designed to calculate empirical predictions based on given data [2]. This prediction-oriented approach to ML is
widely referred to as supervised learning, predictive analytics, or predictive modeling, and initially requires at least two
data sets: one for model training and one for testing [2, 3]. While the former allows a given ML algorithm to "learn"
patterns that connect the model input and output, the latter serves to evaluate the predictive accuracy of a trained model.
In practice, if a corresponding ML model is attributed to possess a sufficient degree of predictive power, it may be
deployed in a productive environment to compute real-world predictions, e.g., to support managerial decision making.
The application of supervised learning in business contexts is highly relevant as it may drive applications in the fields of
predictive maintenance, financial fraud detection, personalized product recommendation, and more. Consequently, the
global ML market size was valued at US$ 34.56 billion in 2021 and is expected to grow to US$ 74.99 billion by 2028 at
a compound annual growth rate of 25.7% [4].
Given the enormous business potential of ML, a considerable number of companies have already begun to launch data
analytics initiatives to automate their processes or support their decision making over the last years. Yet, Chui et al.
(2018) [1] report that most of these companies could (i) increase their performance or (ii) generate additional insights if
they were employing applications based on more advanced ML algorithms, such as artificial neural networks. Moreover,
Designing Explainable Predictive Machine Learning Artifacts

for many decision makers from companies across various industries, the application of modern ML techniques is still
perceived more as a hype than reality; just as concerning is that the majority of managers in leading positions are
currently not feeling prepared for the adoption of ML [5]. We ascribe these issues particularly to one widely held view
on advanced ML algorithms, namely that models generated by such algorithms are broadly regarded as “black boxes”
whose complexity does not allow for uncovering associations between the model input and output [6]. Even beyond
business cases that require such model interpretability (e.g., if predictions need to be explained to prudential regulators),
and regardless of its predictive accuracy, human decision-makers tend to feel uncertain relying on a decision support
system if they do not understand the factors that drive the system output [7]. Thus, we contend that the sustained
perception of ML models as black boxes by decision makers prevents ML from diffusion, which bears the risk of
entailing severe competitive consequences for the companies concerned.
We argue that academic research can and should contribute to overcome this barrier to adoption by providing (i) tools
to support ML model interpretation and (ii) prototypical applications (i.e., artifacts) incorporating such tools. As for
the former, we recognize that research in computer science has by now developed a range of such tools. Among these
tools, one is particularly compelling as it is capable of efficiently and reliably calculating and visualizing the impact
of independent input variables on an ML model output: SHAP (“SHapley Additive exPlanations”). The tool was
recently presented within the scope of the seminal Nature Machine Intelligence article by Lundberg et al. (2020) [8].
Moreover, for the development of corresponding artifacts, we consider one paradigm in information systems (IS) to
be especially suitable, as its research constantly strives to extend the boundaries of organizational capabilities: design
science research (DSR) [9]. Artifacts developed in this field commonly seek to provide a novel degree of utility with
respect to a given problem space and, ideally, yield insights enabled through their application. However, as we show
in the literature review of this paper, there has been little research on the development of prediction-oriented artifacts
leveraging approaches to black box model interpretation in DSR so far.
We attribute this issue to the lack of methodological guidance on how to develop these artifacts; although research
efforts in DSR led to several methodologies that seek to guide artifact development, none of them were deliberately
designed to create artifacts powered by ML [10]. Hence, current DSR methodologies do not provide a solution to the
“black box problem”, i.e., they do not include a step that explicitly considers ML model interpretation. Aside from the
DSR methodologies, IS research also resulted in valuable data analytics guidelines that are applicable to conducting
prediction-oriented studies [11, 12, 3]. Although these guidelines may be transferable to developing artifacts in the
sense of DSR, the corresponding authors neither bridge the gap between predictive modeling and DSR nor do they
provide distinct guidance on how to efficiently uncover reliable associations underlying the output of complex ML
models in general.
In the present paper we address this research gap by developing and showcasing a methodology tailored to build and
interpret prediction-oriented ML artifacts. For this purpose, we build upon suitable methodologies from DSR and
predictive analytics. Moreover, we consider the use of the above-mentioned SHAP. Thus, the contribution of our study is
threefold. First, we contribute to the body of methodologies that guide IS research efforts by combining methodological
knowledge from the complementary fields of DSR and predictive analytics with techniques that support ML model
interpretation. Second, by providing a methodology to develop interpretable predictive ML artifacts, we contribute to
overcome the adoption barrier of ML in organizations, as such artifacts may be explained to human decision makers.
Third, from a scientific viewpoint, insights gained through prediction-oriented ML model interpretation can be used to
stimulate theorizing, which may lead to new or refined theory [3]. Artifacts developed by the use of our methodology
are thus highly suited for providing additional scientific knowledge through their application, which is, according to
Hevner (2007) [9], one focal purpose of DSR.

2 Theoretical Background and Preliminaries

2.1 Design Science Research

2.1.1 About the Paradigm

Apart from behavioral research, DSR is frequently referred to as one of the two pillars of IS research [9]. While the
former is particularly concerned with gaining knowledge about human-technology interaction, the latter is focused on
developing useful artifacts—often in business contexts. Since artifacts developed in the course of DSR studies are—as
opposed to theoretical insights gained from behavioral studies—traditionally more practice-oriented, scholars in this
field first struggled to position their artifacts as scientific outcomes [13]. To overcome this hurdle to justification, research
efforts led to seminal contributions which illustrate the importance of DSR as a scientific discipline [14, 15, 9, 16]. One
oft-cited framework in this context is the three cycle view of DSR put forward by Hevner (2007) [17]. As depicted in
Figure 1, the corresponding three cycles refer to the interactions between (i) a given environment and a DSR project

2
Designing Explainable Predictive Machine Learning Artifacts

(relevance cycle), (ii) the IS knowledge base and the DSR project (rigor cycle), and, finally, (iii) the iterative process of
building and evaluating a respective artifact (design cycle). Apart from the obvious purpose of the latter, the first cycle
is aimed at assuring that the artifact under construction possesses a sufficient degree of practical utility, whereas the
second cycle is concerned with the artifact building upon existing knowledge to ensure its innovation. Moreover, the
rigor cycle seeks a DSR project to result in valuable additions to the knowledge base, which are gained throughout
artifact development and evaluation, such as theoretical insights, methodological improvements, or new meta-artifacts
[17]. Notably, the three cycle view has recently been extended by a fourth cycle (change and impact cycle) that accounts
for dynamic changes in the environment into which a DSR project is embedded [18]. Although this extension provides
great value to the conceptualization of DSR, we refrain from describing it in detail, as this would go beyond the scope
of our paper. However, we encourage practitioners and researchers to consider the change and impact cycle when
executing a DSR project.
Besides artifact development, DSR studies often focus on the creation of design theory, i.e., prescriptive knowledge
about why and how to build artifacts [19, 14, 15]. As the boundaries between these outcome-types tend to blur, some
DSR contributions ought to position themselves as hybrids providing some degree of prescriptive knowledge and
innovative practical utility. Therefore, in order to shed light on the spectrum between artifact and theory and, thus, DSR
contributions, Baskerville, Baiyere, Gregor, Hevner, & Rossi (2018) [20] propose five proper DSR objectives. Those
are (i) technology and science evolutions, (ii) design artifacts, (iii) design theories, (iv) DSR processes, and (v) DSR
impacts. While the (i) first objective ensures that DSR efforts clarify their contribution to science-technology evolution,
(ii) artifacts are most often designed before the development of corresponding theories and must be evaluated in a DSR
project. (iii) Design theories, on the other hand, represent design knowledge and should be recognized as inputs and
outputs of a DSR endeavor. Furthermore, (iv) DSR processes serve to conduct DSR studies properly, whereas (v) DSR
impacts ensure that respective research efforts impact both practice and research, which is in line with the three cycle
view.

Figure 1: Three cycle view of DSR (Hevner, 2007)

Against the backdrop of the DSR objectives, we seek to develop a DSR process (i.e., a methodology) within the scope
of this article. This process aims to provide guidance on how to develop and interpret ML-driven artifacts.

2.1.2 Design Science Research Methodologies

To provide guidance on how to conduct DSR studies, scholars developed several methodologies over the years. Venable
et al. (2017) [10] summarize the six most commonly used methodologies in their article. These are (i) systems
development research methodology (SDRM) [21]¸ (ii) DSR process model (DSRPM) [22, 23, 24], (iii) design science
research methodology (DSRM) [25], (iv) action design research (ADR) [26], (v) soft design science methodology
(SDSM) [27], and (vi) participatory action design research (PADR) [28]. They are depicted in Figure 2. All of them
are particularly suitable for artifact development but are also applicable for creating design theories.
After examining the methodologies, we were able to derive six generic consecutive phases of a DSR project, as shown
in the figure. All the steps of each methodology are assignable to these phases, as they reflect the progress of a DSR

3
Designing Explainable Predictive Machine Learning Artifacts

endeavor. The phases are (i) problem identification, (ii) conceptualization, (iii) implementation, (iv) evaluation, (v)
understanding, and, finally, (vi) publication. While the first five steps constitute the execution of a DSR project, the
final step is concerned with making the study available to an audience that consists of researchers and practitioners,
typically in the form of a scholarly publication. Although Figure 2 suggests the conceptualization phase to be longer
than the other phases, the extent of the different phases is not by any means determined and may vary across projects.
The actual reason of the conceptualization phase being illustrated by a longer arrow is that particularly SDSM, but also
SDRM, put a stronger emphasis on conceptualization, i.e., these methodologies describe conceptualization extensively
by using numerous steps. Moreover, the width of the single steps in the figure does not inherently imply the extent of
each step—it is merely due to the correct assignment of steps and phases.
Observing Figure 2 further, we recognize that not all methodologies consider all the phases derived. However, this
does not mean that the corresponding authors consider missing phases as unimportant. Instead, this means that the
authors do not provide distinct guidance on how to perform these phases because their respective methodologies put
more emphasis on the phases considered.
In this paper, we focus on project execution. Therefore, we only consider the first five phases of a DSR endeavor, thus
neglecting the publication phase. We substantiate this decision by arguing that providing guidance on how to publish a
research project would be beyond the scope of our research agenda. However, our methodology is well-suited to present
a DSR project throughout a publication.

Figure 2: DSR methodologies

2.2 Predictive Analytics

2.2.1 About the Paradigm

Predictive analytics refers to a type of statistical modeling that aims to make empirical rather than theoretical predictions
[29, 3]. While the latter deductively arise from theoretical models—typically in form of a priori formulated hypotheses
by a researcher—prediction-oriented models and, thus, their predictions arise inductively from data. Shmueli & Koppius
(2011) [3] refer to the explanation-oriented, theory-driven approach to statistical modeling as explanatory modeling and
to the prediction-oriented, data-driven approach as predictive modeling. “[W]hereas explanatory modeling seeks to
minimize model bias (i.e., specification error) to obtain the most accurate representation of the underlying theoretical
model, predictive modeling seeks to minimize the combination of model bias and sampling variance” [3]. As a
consequence, explanatory models are typically validated by means of goodness-of-fit tests, the R2 metric, and p-values
that support hypotheses rejection and confirmation. In contrast, predictive models are typically built by applying
appropriate methods (e.g., ML algorithms) to a training data set and evaluated by calculating out-of-sample predictions

4
Designing Explainable Predictive Machine Learning Artifacts

using a second test data set. Here, evaluation metrics such as the mean absolute error (MAE), mean squared error
(MSE), or mean absolute percentage error (MAPE) may be utilized in case the trained model serves to solve a regression
problem (i.e., if the predicted target is a continuous variable). Moreover, if the model serves to solve a classification
problem (i.e., if the predicted target is a discrete variable), metrics such as the accuracy, precision, recall, or Fβ -score
may be used. While IS has a rich history in empirical explanatory research, prediction-oriented studies are traditionally
rather underrepresented [3].
Against this background, academic researchers increasingly started to emphasize the scientific value of predictive
modeling, often with reference to the rich data availability that has emerged in recent years [30, 29, 12]. Aside from
holding the potential to complement explanatory research efforts in behavioral research, as outlined by Shmueli &
Koppius (2011) [3], predictive modeling is also well-suited to drive artifacts that rely on empirical predictions, such as
various types of decision support systems [29]. For the training of models capable of calculating such predictions, a
wide range of different methods exists. Technically, even more traditional statistical methods, such as linear or logistic
regression, may be trained to predict new data in the sense of predictive modeling. The approach of using such “white
box” methods to train predictive models has the obvious advantage of producing models that are easy to interpret.
However, predictive modeling typically unfolds its full potential (i.e., it typically yields the most accurate predictions)
in cases where complex black box ML algorithms, such as tree ensembles or neural networks, are trained by the use of
large data sets [31, 12]. Therefore, in this paper, we focus on the development and interpretation of artifacts driven by
sophisticated ML algorithms that are broadly considered black boxes.

2.2.2 Predictive Analytics Methodologies

In order to guide IS research concerned with predictive modeling, several methodologies have been developed. We now
present three methodologies that are oft-cited, published in leading outlets, and particularly suitable for our study.
The first is the process for knowledge discovery in databases (KDD) put forward by Fayyad et al. (1996) [11]. Although
the process is not exclusively designed for predictive modeling, it emphasizes the broader topic of data-driven analyses
(i.e., data mining) and incorporates a final step for pattern interpretation that is aimed at gaining knowledge. Therefore,
its fundamental idea and, moreover, its generic steps are highly transferable to our study. However, because of recent
advances in data availability and ML, the descriptions of KDDs steps—due to the maturity of the process—neglect
aspects which are crucial for contemporary predictive analytics efforts, such as data partitioning or black box model
interpretation.
The second methodology we consider are the steps in building an empirical model (SBEM) proposed by Shmueli
and Koppius (2011) [3]. As opposed to KDD, SBEM is focused on building predictive empirical models. However,
the authors—in their seminal article—stress the roles of predictive modeling in behavioral research rather than in
DSR—which becomes particularly evident in their exemplary showcase considering a popular model from technology
acceptance research. Moreover, even though the final step of the methodology (i.e., model use & reporting) may
technically be applied to model interpretation, SBEM does not provide guidance on how to interpret predictive models.

Figure 3: Predictive modeling methodologies

Finally, we consider the big data analytics guidelines (BDAG) presented by Müller et al. (2016) [12]. BDAG is
predominantly focused on analyzing vast data sets (“big data”) without the compelling necessity of employing predictive

5
Designing Explainable Predictive Machine Learning Artifacts

modeling. However, the authors repeatedly emphasize the application of ML algorithms for prediction—therefore,
we constitute BDAG to be well-suited to guide predictive modeling efforts. As opposed to SBEM, BDAG explicitly
incorporates a step that considers ML model interpretation. Unfortunately, at the time the article by Müller et al. (2016)
[12] was written, a sophisticated tool to support the efficient and reliable interpretation of any black box ML model,
such as SHAP [8, 31], was not yet available. Hence, BDAG does not provide distinct guidance on how to interpret
black boxes using such a tool. Thus, our methodology does not only bridge the gap between BDAG and DSR, but also
aims to extend the methodology’s interpretation step by considering the use of SHAP.
In Figure 3, we assigned the steps of the methodologies for conducting predictive studies to the phases of a DSR
project, as derived from the DSR methodologies. Examining the figure, we find that only SBEM roughly incorporates
conceptualization (in form of the step goal definition), which is—according to all of the DSR methodologies—a
crucial step within the scope of a DSR endeavor. Yet, the predictive analytics methodologies put more emphasis on
implementation than on the conceptualization phase. After examining corresponding articles about the methodologies,
we were able to derive three consistent sub-phases the implementation phase can be divided into if the goal is to build a
predictive model. These are (i) data collection, (ii) data exploration and preparation, and (iii) model training. The
corresponding steps of each methodology can be assigned to these sub-phases, as shown in Figure 3. As for the widths
of the steps in the figure we—again—note that these are due to the correct assignment of steps and phases, i.e., the
width does not necessarily correspond to the extent of a step. Moreover, in line with our decision to neglect phase 6 in
our methodology, none of the methodologies provide guidance on how to write and publish a research article.
In the remainder of this paper, we build upon the insights gained from examining the DSR and predictive modeling
methodologies.

2.3 Prediction-Oriented ML Artifacts and Black Box Interpretation

2.3.1 Value for IS Research

In this section, we shed light on the value of artifacts driven by black box ML models and approaches that enable the
interpretation of such artifacts for IS research. Notably, despite the debates about the differences between ML and
artificial intelligence (AI) on the part of scholars as well as practitioners [32], such interpretation approaches are broadly
referred to as explainable AI approaches (and not explainable ML approaches). For the sake of clarity, and in line with
the deliberations of Iriondo (2018) [32], in this paper, we refer to ML as a branch of AI that focuses on algorithms,
which aim to learn from experience represented by data. In the upcoming subsection, we briefly present a selection of
explainable AI approaches suitable for ML model interpretation; for now, we note that the before-mentioned SHAP
is currently the most advanced of these approaches. Therefore, we utilize SHAP as an instance of explainable AI
throughout this paper. However, our methodology does not necessarily rely on using SHAP, as it is conceivable that, in
the future, even more sophisticated tools emerge which have advances over SHAP. Moreover, some research projects
may consider the use of other existing tools, e.g., due to project-specific requirements.
To highlight the value of DSR studies concerned with predictive ML modeling and explainable AI for IS research,
we draw upon the before-mentioned three cycle view of DSR as shown in Figure 4. Here, we assigned the steps of a
DSR project, which we derived from the methodologies, to the corresponding cycles. We now briefly elaborate on this
assignment.
First, we consider phase 1—as being the initiator of a DSR project—to be part of the relevance cycle. We substantiate
this view by pointing out that DSR has its roots in solving problems that arise from the environment rather than the
knowledge base. In the words of Hevner et al. (2004) [9]:
The environment defines the problem space [33] in which reside the phenomena of interest.
As for a prediction-oriented ML artifact, a corresponding problem might be the emergence of high costs due to
repeatedly failing machines that are crucial within the manufacturing process of a company. In this subsection, we
continue to consider this as an exemplary problem that serves to illustrate the purposes of the phases within the three
cycle view.

6
Designing Explainable Predictive Machine Learning Artifacts

Figure 4: Value of explainable AI in DSR in case an artifact is powered by prediction-oriented ML

Next, we assigned the phases 2, 3, and 4 to the design cycle, as these are obviously concerned with the iterative process
of artifact development and evaluation as described by Hevner (2007) [17]. Despite being assigned to the design cycle,
these three phases are also interwoven into the relevance and the rigor cycle, at least for problems that may be addressed
by means of prediction-oriented ML artifacts. This means that these phases must consider the environment (e.g., data
availability, data quality, use case) as well as the knowledge base (e.g., methods, guidelines, evaluation metrics). With
regard to the exemplary problem of a manufacturing company described above, a corresponding artifact might utilize
data gathered by sensors that aim to monitor the conditions of the machines (i.e., data from the environment) in order
to find patterns that might predict machine failure leveraging state-of-the-art ML algorithms (i.e., methods from the
knowledge base). Such artifact would need to possess a sufficient degree of predictive power, which would be assessed
using techniques and metrics from the knowledge base. If such artifact would yield sound predictions, it would be
capable of being deployed to avoid machine failure, e.g., by initiating preventive maintenance jobs. This would provide
utility to the environment and at least experience to the knowledge base (maybe more, depending on the findings of the
project).
Finally, and most importantly for our research agenda, we state that the employment of an explainable AI tool, such
as SHAP, at the end of a design cycle (i.e., when the artifact is built and evaluated), enables analysts to gain insights
into the factors that drive the artifact’s predictions. Should the artifact possess a sufficient degree of predictive power,
such insights are especially valuable to inform (i) decision makers in organizations and (ii) researchers. Therefore,
we consider phase 5 to affect both the environment and the knowledge base, and, thus, to be part of the relevance
and the rigor cycle. With respect to the exemplary manufacturing company, such insights gained from ML artifact
interpretation might be that the temperature of a given machine component is crucial for predicting machine failure
during production. If this were the case, decision makers—aside from being more likely to adopt the artifact due to
its interpretability—could consider taking actions either to prevent the machine component from reaching a critical
temperature or to replace the component with a more robust substitute. Moreover, as for researchers, such insight might
help to develop theories for resilient shop-floor design or to extend the set of antecedents that explain machine failure in
a theoretical model in the field of production research.
However, we note that associations uncovered by using explainable AI tools are not to be confused with causal
explanations, as such tools are not capable of providing the latter. Yet, combined with proper logical reasoning and/or
further research, such associations are well-suited to deliver indications that point towards causality. Beyond that,
even in cases where the examination of these associations does not result in uncovering novel causal explanations, the
associations may still serve to enhance a given understanding of a phenomenon, e.g., by serving as specifications of the
surrounding of this phenomenon.

2.3.2 Explainable AI
The research field of explainable AI has its roots in the oft-cited tension between accuracy and interpretability of
predictive models [6, 31, 12]: Models that yield particularly accurate predictions typically tend to be complex black
boxes that are hard to interpret, whereas interpretable white box models, such as linear models, typically tend to

7
Designing Explainable Predictive Machine Learning Artifacts

lack predictive power. Apparently, this tension often prompts users to prefer white box models—due to their ease of
interpretation—over the better performing black boxes [31]. In order to remedy the tradeoff between accuracy and
interpretability, scholars in the field of explainable AI recently proposed approaches that enable the interpretation
of models, which have long been considered black boxes, such as ensemble or deep learning models. One of these
approaches is SHAP, which utilizes metrics (i.e., SHAP values) from cooperative game theory as a reliable measure
of feature importance. As mentioned above, we currently identify SHAP to be the most advanced approach to black
box model interpretation—notwithstanding, we recognize that popular approaches exist that are comparable to SHAP.
Therefore, following the deliberations of Lundberg & Lee (2017) [31], we briefly provide an introduction to the rationale
behind these and SHAP and explain why SHAP is currently preferable over other approaches.
Lundberg & Lee (2017) [31] point out that the two reasons why white box models are capable of providing explanations
of the associations between the features (i.e., its input variables) and the target (i.e., its output variable) are that the
model itself (i) represents these associations and (ii) is easy to understand. Although black box models also associate
features with the target, they are not easy to understand, which is why they do not provide convenient explanations.
Therefore, explainable AI approaches typically seek to find a relatively simple, interpretable explanation model g
that approximates a given black box model f . For this purpose, SHAP, as well as other approaches, build upon local
explanations. Local explanations aim to explain each prediction f (x) based on a single input x, i.e., features from one
observation. In this context, explanation models typically map x to simplified inputs x′ by using a mapping function
x = hx (x′ ). Such explanation models further seek to ensure g(z ′ ) ≈ f (hx (z ′ )) in cases where z ′ ≈ x′ . Here, one
frequently used explanation model is the following [31]:

M
X
g(z ′ ) = ϕ0 + ϕi zi′ , (1)
i=1

where z ′ ∈ 0, 1M , M is the numer of simplified input features, and ϕi ∈ R.

Lundberg & Lee (2017) [31] refer to methods that build upon Equation 1 as additive feature attribution methods,
which “attribute an effect ϕi to each feature, and summing the effects of all feature attributions approximates the
output f (x) of the original model.” Since additive feature attribution methods (i) commonly serve to drive explainable
AI approaches [34, 31] and (ii) can be utilized to explain predictions both on a local (i.e., feature importance of an
individual prediction) and global level (i.e., feature importance of an entire model) [8], we focus on such methods.
Besides SHAP, popular examples of such approaches are [31]:

• LIME [35]: The local interpretable model-agnostic explanation method (LIME) explains model predictions
using local approximations of the model around the prediction at hand. It may be used to explain the predictions
of any black box model.
• DeepLIFT [36]: This recursive additive feature attribution method particularly serves to explain the predictions
of deep learning models.
• Layer-Wise Relevance Proposition [37]: This method is a special case of DeepLIFT and also serves to
explain the predictions of deep learning models.
• Classic Shapley Value Estimation: There are several methods that employ Shapley values from cooperative
game theory to explain model predictions: Shapley regression values [38], Shapley sampling values [39], and
quantitative input influence [40]. While the first assigns effects of feature attributions to linear models, the
remaining two methods are technically applicable to any model.

In contrast to each of these approaches, SHAP unifies the whole class of additive feature attribution methods in order
to propose SHAP values as a unified measure of feature importance, which are applicable to any black box model
(i.e., they are model-agnostic). As a consequence, SHAP values have several desirable properties compared to the
other approaches, e.g., better alignment with human intuition, faster calculation of optimal explanations, being the only
feature attribution method to provide consistent feature attributions for tree ensembles, and more [8, 41, 31]. SHAP
uses the so-called Shapley values (or SHAP values) for the calculation of feature attributes ϕi as a reliable measure of
the importance of features with respect to a prediction. These are now explained according to the paper by Lundberg
and Lee [31].
SHAP values originate in cooperative game theory and are a measure of the importance of features within statistical
models in the presence of multicollinearity. Here, S is a choice of features for which S ⊆ F holds, where F represents
all features. The objective is to assign an importance value to each feature with respect to a prediction of a model
under consideration. For this purpose, for a given feature, a model fSU {i} with this feature and a model fS without
this feature are trained. Then, the predictions of the two trained models are compared using the features of relevance:

8
Designing Explainable Predictive Machine Learning Artifacts

fSU {i} (xSU {i} ) − fS (xS ), where xS are the feature values of the set S. Since the effect of withholding a feature
depends on the values of the other features used in the model, the differences described are calculated for all possible
subsets S ⊆ F \ {i}. Now the SHAP values can be calculated as a weighted average of all combinations of the possible
differences:

X |S|!(|F | − |S| − 1)!

ϕi = [fSU {i} (xSU {i} ) − fS (xS )] (2)
|F |!
S⊆F \{i}

For these SHAP values, the previously mentioned mapping function hx assigns either the value one or zero to the
features, where one means that a feature is considered in the model (i.e. it is important for the corresponding prediction).
Zero, on the other hand, means that it is not considered (i.e., it is not important for the prediction). Moreover, ϕi = f∅ (∅)
must hold for SHAP values to correspond to additive feature attributes. As it would go beyond the scope of this paper
to mention all the merits of SHAP, we hereby refer to the referenced articles by Lundberg et al. and to the GitHub page
of the corresponding Python package1 for a deeper understanding about the tool.
In their study, Tritscher et al. (2020) [42] show that SHAP currently provides the most reliable explanations for
black box models and their predictions compared to other prominent XAI approaches, closely followed by LIME.
Furthermore, SHAP values have several advantages over other feature attributes. These include that (i) they can be
interpreted more intuitively by humans, (ii) they can be used to compute optimal explanations more quickly, and (iii)
they are the only feature attributes that provide consistent explanations for decision tree-based models (e.g., Random
Forests or XGBoost) [41, 8, 31]. For these reasons, SHAP is used as a demonstrative XAI approach in the course of
this paper.

3 Literature Review

In this section, we present the results of a literature review we conducted. These results provide an overview of research
endeavors that are concerned with (i) the design of a prediction-oriented black box artifact and (ii) the explanation of its
predictions. In order to gather a representative sample of research articles, we searched five databases (i.e., Google
Scholar, AIS electronic Library, IEEE Xplore, ACM Digital Library and EBSCOhost) using a query that consisted
of the following five building blocks, which had to appear in the full texts of the articles: (i) “machine learning” or
“artificial intelligence”; (ii) “predict”, “predictive”, or “prediction”; (iii) “black box”; (iv) "interpretation", "explanation",
"explainable", or "interpretable"; and (v) “design science”. This search resulted in 1501 articles (1040 in Google
Scholar, 387 in the AIS electronic library, 25 in IEEE Xplore, 10 in the ACM Digital Library, and 39 in EBSCOhost)2 .
We then manually searched these articles for (i) peer-reviewed studies employing (ii) predictive modeling and (iii)
explainable AI to build prototypical applications (i.e., artifacts). Although the vast majority of the articles either present
prediction-oriented studies that refrain from model interpretation or non-quantitative studies, numerous authors stress
the importance of explainable AI. Yet, we only identified 35 articles that met our search criteria. Among these articles,
twelve build their research upon existing methodologies while the remaining 23 do not organize their research according
to a standard process (see Table 1).

1
[Link]
2
We used the following search query for Google Scholar: ("machine learning" OR "artificial intelligence") AND ("predict" OR
"predictive" OR "prediction") AND "black box" AND ("interpretation" OR "explanation" OR " interpretable" OR "explainable ")
AND "design science". For the remaining databases, we adapted the query’s syntax according to their respective requirements. We
conducted the search on 17th of April 2022.

9
 Designing Explainable Predictive Machine Learning Artifacts

Table 1: Methodologies used in studies that build artifacts employing predictive modeling and explainable AI
Publication DSR methodology Other methodology No methodology
Müller et al. (2016) [12] x
Bohanec, Robnik-Šikonja, & Borštnar (2017a) [43] x x
Bohanec, Robnik-Šikonja, & Borštnar (2017b) [44] x x
Ming, Qu, & Bertini (2018) [45] x
Ventura, Cerquitelli, & Giacalone (2018) [46] x
Colace et al. (2019) [47] x
Eitle & Buxmann (2019) [48] x
Yeganejou, Dick, & Miller (2019) [49] x
Alvanpour, Das, Robinson, Nasraoui, & Popa (2020) [50] x
Bramhall, Horn, Tieu, & Lohia (2020) [51] x
Dolk, Kridel, Dineen, & Castillo (2020) [52] x
Harl, Weinzierl, Stierle, & Matzner (2020) [53] x
Kim, Srinivasan, & Ram (2020) [54] x
Mehdiyev & Fettke (2020) [55] x
Yadam, Moharir, & Srivastava (2020) [56] x
Zhang, Du, & Zhang (2020) [57]
Asmussen, Jørgensen, & Møller (2021) [58] x x
Barfar, Padmanabhan, & Hevner (2021) [59] x
Garriga, Aarns, Tsigkanos, Tamburri, & Heuvel (2021) [60] x
Han et al. (2021) [61] x
Herm, Wanner, Seubert, & Janiesch (2021) [62] x
Johnson, Albizri, Harfouche, & Tutun (2021) [63] x
Liu, Du, Hong, & Fan (2021) [64] x
Mehdiyev & Fettke (2021) [65] x
Mombini et al. (2021) [66] x
Pereira et al. (2021) [67] x
Velichety & Ram (2021) [68] x
Wang et al. (2021) [69] x
Wastensteiner, Michael Weiss, Haag, & Hopf (2021) [70] x
Zhou, Wang, Ren, & Chen (2021) [71] x
Bodendorf, Xie, Merkl, & Franke (2022) [72] x
Johnson, Albizri, Harfouche, & Fosso-Wamba (2022) [73] x
Kowalczyk, Röder, Dürr, & Thiesse (2022) [74] x
Tao, Zhou, & Hickey (2022) [75] x
Yang, Yuan, & Lau (2022) [76] x

These articles present studies in the areas of business process management [58, 72, 60, 53, 73, 55, 65], healthcare
[61, 63, 77, 45, 66], human resource management [51, 47], image recognition [50, 46, 49], sales [44, 43, 48], security
[74, 75, 69, 76, 57, 71] and others [59, 52, 62, 64, 12, 67, 68, 70, 56]. Moreover, all of these articles were published in
recent years, with 27 of them being published since 2020. Thus, we consider their authors pioneers in the development
of interpretable predictive ML artifacts and further point out that their recent publication highlights the topicality of
explainable AI in predictive modeling. On the downside, the examination of the articles unveils the lack of a consistent
methodology that guides their underlying research efforts. This becomes particularly evident in the articles by Bohanec
et al. (2017) [44, 43] and Asmussen et al. (2021) [58]. In these articles, the authors are compelled to combine a
DSR methodology (i.e., ADR) with a non-academic methodology for data mining (i.e., CRISP-DM) [78] to train
and interpret ML models. Moreover, the six studies of Eitle and Buxmann (2019) [48], Mehdiyev and Fettke (2020,
2021) [55, 65], Garriga et al. (2021) [60], Wastensteiner et al. (2021) [70] and Bodendorf et al. (2022) [72] adapted
methodologies from DSR to their settings, even though these methodologies do not natively account for all of the steps
required to build predictive models. Although the authors—due to their experience—broadly managed to evade the
pitfalls that may come with such adaption, we argue that the presence of a suitable methodology for the development of
interpretable predictive black box artifacts may not only contribute to prevent methodological mistakes in the future
but may even encourage more scholars to conduct research in this area. Despite the described articles, our literature
review also yielded the article by Müller et al. (2016) [12], which we already discussed in section 2.2.2, as it presents
and showcases BDAG. The remaining studies do not follow a standard methodology. As a consequence, the studies

10
Designing Explainable Predictive Machine Learning Artifacts

are highly heterogeneous, which hampers comparing their results. Although we recognize that, due to the variety
across studies, some of them are hardly comparable, we note that the widespread use of a standard methodology would
particularly foster the comparability of similar studies.

4 Methodology
To create a methodology that guides DSR efforts in building and interpreting prediction-oriented ML artifacts, we
unify the strengths of the methodologies from DSR and predictive modeling. In doing so, we build upon the phases
of a DSR project as derived from the DSR methodologies. As shown in Figure 3, these phases are also applicable to
the methodologies from predictive modeling. Yet, the DSR methodologies put more emphasis on the execution of
the conceptualization phase, whereas the methodologies from predictive modeling focus more on the implementation
phase. We not only consider this in our methodology, but also put strong emphasis on black box model interpretation by
providing guidance on how to understand a black box model using SHAP. Figure 5 depicts the steps of our proposed
methodology.

Figure 5: Steps in developing and understanding prediction-oriented black box ML artifacts

4.1 Phase 1: Problem Identification

In the first phase of a DSR project that aims to develop an interpretable prediction-oriented ML artifact we follow
the recommendations of all the DSR methodologies as well as BDAG by proposing to motivate and define a research
problem. As shown in Figure 4, such problem typically arises from the environment and forms the basis of a DSR
project. However, theoretical knowledge might also serve to substantiate and highlight the relevance of a problem.
Problem definition is crucial for project execution, as it guides the development of an effective artifact by providing
valuable grounding for conceptualization and evaluation. Moreover, this phase has a strong justificatory purpose and
serves to motivate the pursuit of a solution towards an audience [25]. After the problem definition, researchers may also
formulate one or more research questions that may be answered throughout artifact evaluation and/or interpretation.
The formulation of such research question(s) might explicitly take into account that theoretical contributions may arise
inductively from data and not compellingly from mere deductive reasoning [12].

4.2 Phase 2: Conceptualization

In order to facilitate conceptualization, we divide this phase into two steps which we synthesize from the DSR
methodologies and SBEM, namely (i) objective definition and (ii) designing a conceptual approach. Although these
two steps do not equally apply to all of the DSR methodologies, they are crucial within the methodologies in which they
appear, as they account for the definition of (i) the solution space and (ii) the solution itself (i.e., the artifact), respectively.
For this reason, we constitute that dividing conceptualization into these two steps highly supports developing, structuring,
and presenting a concept.

4.2.1 Phase 2.1: Objective Definition

In the first step, objectives have to be inferred from the problem definition with respect to the feasibility of a possible
solution [25]. As for interpretable prediction-oriented ML artifacts, objective definition should commonly take two
aspects into account: (i) one that applies to the practical utility of the prediction itself with respect to the given problem
and (ii) one that pertains to the interpretation of a black box model and, thus, to the understanding of the phenomenon
at hand. To this end, the artifact needs careful specification of what needs to be predicted, as this accounts for the
measurement of the phenomenon of interest and impacts the choice of methods to be used throughout implementation
[3]. This specification includes determining whether the artifact seeks to solve a classification or regression problem,
that is, the specification as to whether the target variable at hand is categorical or numerical. As for categorical outcomes,

11
Designing Explainable Predictive Machine Learning Artifacts

an alternative approach is to rank new observations according to their belonging to a certain class, which is referred to
as ranking [3].

4.2.2 Phase 2.2: Conceptual Approach

In the second step of conceptualization, the defined objectives serve to build a conceptual approach in the sense of a
solution’s system architecture, as proposed by Nunamaker et al. (1990) [21]. Such an approach may provide a summary
of the functionalities of the artifacts by putting its components, their relationships, and their interactions into perspective.
Apart from system components, the conceptual approach may also include the interaction between the artifact and other
entities, such as data sources, human stakeholders, or other artifacts. Thus, by clearly defining how the artifact aims to
realize the objectives in the light of the given problem, such a conceptual approach particularly serves as a road map
for implementation and evaluation. Moreover, the conceptual approach is well-suited to highlight the innovation of a
solution. For this purpose, we recommend to build upon the DSR knowledge contribution framework put forward by
Gregor and Hevner (2013) [14]. The framework divides knowledge contributions (i.e., innovation) essentially into three
types: (i) improvements (i.e., new solutions for known problems), (ii) exaptation (i.e., extension of known solutions to
new problems), and (iii) inventions (i.e., new solutions for new problems). It is illustrative to visualize the conceptual
approach by using a corresponding figure.

4.3 Phase 3: Implementation

In line with all the predictive modeling methodologies, we divide implementation into three sub-phases: (i) data
collection, (ii) data exploration & preparation, and (iii) model training.

4.3.1 Phase 3.1: Data Collection

Data Sources. Although it is desirable in the course of a DSR project, which aims to develop a data-driven artifact
for computing empirical predictions, to draw upon rich and relevant real-world data, many research scenarios do
not have access to such data and therefore depend on the use of alternative data sources. Beyond that, even in cases
where data can be gathered from practice, predictive modeling often benefits from enriching the available data with
data from complementary sources. For example, Hanke, Hauser, Dürr, & Thiesse (2018) [79] develop a stationary
product recommendation system for smart fashion retail environments using practitioner data (i.e., customer purchase
history) enriched with publicly available contextual information (i.e., weather data from weather stations close to the
specific stores under examination). Their results show that the incorporation of such open data leads to considerable
improvements in the predictive accuracy of their models. Another project in the context of stationary fashion retailing,
which even entirely refrains from using real-world data to develop a data-driven artifact, is presented by Hauser, Günther,
Flath, & Thiesse (2019) [80]. The authors develop an RFID-based automated checkout system which is capable of
predicting product purchases of individual customers within fashion stores. Their artifact builds upon data (i.e., walking
paths) gathered from a laboratory experiment, which was designed to mimic real world scenarios. Their results provide
valuable insights into the opportunities and challenges that may come with the implementation of such system in the
real world. The studies of Hanke et al. (2018) [79] and Hauser et al. (2019) [80] are only two striking examples to
highlight that data-driven DSR projects may consider the use of various data sources, depending on availability and
suitability. However, we argue that particularly suited data sources to drive a prediction-oriented ML artifact may be,
but are not limited to:

• Practitioner data (e.g., sales data, financial data, business process data, sensor data)
• Publicly available data sets (e.g., from Google dataset search)
• Publicly available data from APIs or websites (e.g., gathered via web scraping)
• Data from laboratory experiments
• Data from crowdsourcing (e.g., Amazon Mturk or [Link])
• Simulated data (e.g., from Monte Carlo simulations)

Sample Size. Another important aspect of data collection is sample size. Most authors agree that sample sizes should be
larger in predictive modeling compared to explanatory modeling [30, 12, 3]. Shmueli and Koppius (2011) [3] point out
that this is due to four major reasons: (i) as opposed to estimating population-level parameters in explanatory modeling,
predicting individual observations has a higher degree of uncertainty and therefore requires more observations; (ii)
data-driven algorithms seek to build the structure of an empirical model by learning from data—the more data there is,
the more there is to be learned; (iii) as predictive modeling often aims to capture complex relationships, larger samples
may help to reduce model bias and sampling variance; (iv) more data are required for the creation of a test set that

12
Designing Explainable Predictive Machine Learning Artifacts

serves to evaluate the predictive accuracy of a model. However, in some situations—depending on the nature of the
data, the problem at hand, and the methods used—even smaller samples may serve to train a model that possesses a
sufficient degree of predictive power. Dürr, Griebel, Welsch, & Thiesse (2020) [81], for example, conducted a study that
utilizes texts from whitepapers to predict whether corresponding initial coin offerings are fraudulent or not. Although
their final training sample—due to their under-sampling strategy—only consists of 166 observations, the authors are
able to train an ML model that predicts fraud with a precision of 80% on their test set. Thus, we conclude that, although
larger sample sizes are preferable when it comes to training a prediction-oriented ML artifact, the utility of such artifact
is determined by the accuracy of its out-of-sample predictions and not by the size of the training sample. Hence, smaller
samples may be used to train predictive ML artifacts whenever effective.
Dimensionality. As opposed to the widely held belief that high-dimensional data sets are a curse in statistical
modeling, Breiman (2001) [82]—in his prominent Statistical Science article—points out that “dimensionality can be a
blessing” in predictive analytics. He substantiates this statement by arguing that each additional feature may comprise
additional information that might be valuable for prediction—or, in his words: “The more predictor variables, the more
information.” Moreover, in line with Müller et al. (2016) [12] and Shmueli and Koppius (2011) [3], we recognize high
data dimensionality to be characteristic for many predictive analytics studies that employ advanced ML algorithms,
as such algorithms are designed to find patterns within high-dimensional data. In contrast to explanatory modeling,
Shmueli and Koppius (2011) [3] emphasize that in predictive modeling not every feature within a data set needs to
be justified by theory. We argue that such insistence on theory would even lower the chance of uncovering novel
patterns. However, as opposed to research efforts that have no ambition on gaining insights from predictive black box
ML modeling—e.g., because their focus is on producing particularly accurate predictions, even at the expense of model
interpretability—we recommend to be careful when it comes to incorporating features whose meaning is unclear. Such
features are prone to providing opaque explanations to a given prediction.
Data Types. The proliferation of deep learning led to algorithms, such as convolutional neural networks, that are
capable of processing various data types. Thus, prediction-oriented ML artifacts are not restricted to build upon
traditional tabular data. Other possible data types may be texts, images, audio spectrograms, or videos [2]. However,
although modern approaches to explainable AI support the interpretation of models that use other than tabular data, we
recommend to verify the level of interpretability of the data under consideration before developing the artifact.
Synthetic Data. According to Gartner (2020) [83], “[s]ynthetic data is generated by applying a sampling technique
to real-world data or by creating simulation scenarios where models and processes interact to create completely new
data not directly taken from the real world.” Thus, algorithms that generate synthetic data are a promising means to (i)
increase sample size or (ii) simulate whole data sets in cases where data is sparse. By now, these algorithms are already
capable of generating various types of data, such as images or tabular data. Besides more traditional approaches to
creating synthetic data, such as Monte Carlo simulations, modern examples of such algorithms are data augmentation
techniques or generative adversarial networks [84, 85, 86, 87].

4.3.2 Phase 3.2: Data Exploration & Preparation

This phase aims to (i) gain a comprehensive understanding about the data at hand and (ii) preprocess the raw data
into convenient data sets. As opposed to SBEM, where data preparation comes before data exploration and variable
selection, we argue that these three steps are difficult to separate from each other since they are heavily interwoven.
Typically, this phase comprises an iterative procedure of exhaustively browsing, visualizing, understanding, selecting,
and altering data. Therefore, in practice, this step is usually particularly error-prone and often time-consuming [78].
Moreover, this phase, in some cases, may result in more than one plausible final feature set. In such cases, the predictive
performances of models trained using alternative feature sets may be compared in the evaluation phase.
Data Partitioning. According to Shmueli and Koppius (2011) [3], the available data need to be partitioned into three
parts: (i) a training set, (ii) a validation set, and, finally, (iii) a test set. While the first serves to fit ML models, the second
is used to facilitate algorithmic parameter adjustment (i.e., hyper-parameter tuning) and, thus, model selection. The test
set, however, is only used in the evaluation phase to assess model generalizability, that is, predictive out-of-sample
performance of the selected model(s). Alternatively, instead of using a validation set, cross-validation may be employed
during model training to enable hyper-parameter tuning and model selection. Even in such situations, it remains
necessary to create a test set for evaluation. Typically, the test set comprises around 5-30% the size of all the available
data.
Visualization. According to Fayyad, Grinstein, & Wirse (2002) [88], descriptive data visualization can support analysts
in finding patterns that facilitate further analyses. In their words (p. 21):
A visualization can provide a qualitative overview of large and complex data sets, can summarize data,
and can assist in identifying regions of interest and appropriate parameters for more focused quantitative

13
Designing Explainable Predictive Machine Learning Artifacts

analysis. In an ideal system, visualization harnesses the perceptual capabilities of the human visual
system.
However, as data visualization comprises a vast number of techniques and depends highly on the nature of the underlying
data, neither scholars nor practitioners have yet developed a standard process to data visualization (and probably never
will). Thus, according to Shmueli and Koppius (2011) [3], exploratory data visualization should rather be understood as
a “free-form fashion” to pattern recognition. Hence, the quality, suitability, and, ultimately, utility of data visualization
heavily depend on the analytical and creative skills of an analyst. Exemplary methods that support data visualization
are line graphs, scatter plots, isosurfaces, rubber sheets, volume visualization, and scalar glyphs [88].
Data Cleansing. Unclean data sets can be a serious issue when it comes to building a predictive ML artifact, as these
may lead to (i) a biased model (in case the training set is unclean), (ii) a biased evaluation (in case the test set is
unclean), or (iii) even both. Here, unclean data may consist of missing values, duplicate entries, measurement noise,
typographical errors, or systematic errors [89]. Before an analyst can cope with erroneous data, the errors should be
unveiled. For this purpose, data visualization may serve as a means to detect anomalies within the data at hand. As
opposed to duplicate entries, typographical errors, and systematic errors—which are typically relatively easy to fix once
detected, i.e., through performing basic operations, such as deleting or adjusting respective entries—, missing values
and measurement noise are sometimes challenging to handle. As for missing data, it should first be clarified if the data is
informative of the to-be-predicted target [90]. Thereupon, methods such as removing observations, removing variables,
using proxy variables, creating dummy variables that indicate missing entries, and algorithms for imputation may be
applied to handle missing values [3]. With regard to measurement noise—although it is desirable in most scenarios to
have as little noise as possible—, Witten et al. (2011) [89] point out that it may be helpful to add artificial noise to
features within the training data in cases where the corresponding features within the test set are noisy; however, this
does not apply to target noise—here, it is best to train on noise-free instances.
Feature Engineering. Feature engineering is the process of transforming raw data into formats that are suitable for ML
model training [91]. It differs from data cleansing in that it aims to process clean data instead of cleaning erroneous
data. Similar to data visualization, feature engineering comprises an enormous number of methods, such as text mining,
normalization, standardization, one hot encoding, dimension reduction, and much more [92, 91]. Moreover, feature
engineering may consider combining available data with contextual information (e.g., from open data sources) to
build new features. For instance, Teubner, Hawlitschek, & Dann (2017) [93], explain prices for homes on Airbnb by
incorporating the independent variable distance to city center, which they calculate using (i) the homes geo location
coordinates from the Airbnb website and (ii) the respective coordinates from the corresponding city centers. As a result,
the authors unveil the feature distance to city center to have a significant impact on Airbnb prices. Thus, the utility of
feature engineering—again—, similar to data visualization, highly depends on the analytical and creative capabilities of
an analyst. Apart from that, domain and theoretical knowledge may also serve to facilitate feature engineering. As
opposed to predictive modeling that refrains from model interpretation, we recommend building interpretable features
that can be explained to an audience. As for dimension reduction, for example, this means that the utilization of
techniques which produce rather uninterpretable features, such as Principal Component Analysis (PCA), should be
avoided. Following the guidelines of the oft-cited article by Costello & Osborne (2005) [94], we recommend using
other forms of Exploratory Factor Analysis (EFA) instead of PCA in cases where features are to be compressed into
fewer features. This means, we particularly recommend the use of EFA methods that employ oblique rotation over
traditional PCA that utilizes orthogonal rotation, even though the former are prone to result in correlating features.
However, such features from EFA have the advantages of (i) being more interpretable and, often, (ii) better capturing the
initial features, particularly in cases where features reflect human behavior, as, in such cases, “we generally expect some
correlation among [features], since behavior is rarely partitioned into neatly packaged units that function independently
of one another” [94]. Beyond that, advanced ML algorithms are capable of handling multi-collinearity. Subsequently,
in order to calculate factor scores (i.e., feature values) based on EFA results, we recommend to follow the guidelines of
DiStefano, Zhu, & Mîndrilǎ (2008) [95]. The authors describe several methods that serve the purpose of computing
factors scores, i.e., (i) several types of sum scores, (ii) regression scores, (iii) Bartlett scores, and (iv) Anderson-Rubin
scores. Although EFA-based factor scores are usually used in explanatory modeling and—in some situations—may
lead to models that possess lower predictive accuracy compared to models trained on PCA scores, we point out that
the development of interpretable prediction-oriented artifacts sits at the nexus of explanatory and predictive modeling.
In such cases, “predictive analytics can focus on predictors [. . . ] that produce a relatively transparent model, while
perhaps sacrificing some predictive power” [3].
Feature Selection. Feature selection (from a clean training data set) can roughly be divided into (i) theory-driven and
(ii) data-driven approaches. Whereas the former focus on applying theoretical and domain knowledge to select features
(as is common in explanatory modeling), the latter seek to select features by performing data analyses. Many modern
ML algorithms do not require such a priori feature selection, as they are designed to find important predictors within a
set of features autonomously. However, in some situations, model training can benefit from feature selection, e.g., in

14
Designing Explainable Predictive Machine Learning Artifacts

cases where the goal is to develop a particularly parsimonious model or if limited computational resources lead to slow
training. Data-driven approaches to facilitate feature selection include correlation analyses [3], lasso regression (i.e., L1
regularization) [96, 97], and tests that employ feature selection metrics, such as chi-square, information gain, or odds
ratios [98].

4.3.3 Phase 3.3: Model Training

In the last step of implementation, one or more ML models are fitted to the training data. To this end, appropriate ML
algorithms need to be selected. As for regression or classification scenarios that employ tabular data a wide range of
popular black box algorithms exist. Among these are support vector machines [99], deep neural networks [100], and
ensemble methods, such as random forests [6], or boosted trees [101]. Thus, it is common in predictive modeling based
on tabular data to train various alternative algorithms on a given training set to compare their predictive performances
in the evaluation step. Here, aside from black box ML algorithms, simpler white box algorithms, such as linear or
logistic regression may be trained to serve as predictive benchmarks. With regard to other than tabular data, the choice
of algorithms is typically more limited to deep learning techniques [100, 2] and may often result in only one or few
suitable algorithms to address a given problem.
However, selecting a suitable algorithm does not automatically result in a well-performing model, even if a convenient
training set is available. That is because even state-of-the-art ML algorithms—although usually less prone to over-fitting
compared to more traditional statistical methods—bear the risk of over-fitting their respective training data. Over-fitting
occurs whenever a model exhibits a particularly good fit with the training data, which, on the downside, comes
at the expense of its generalizability, i.e., the accuracy of its out-of-sample predictions. To assess over-fitting, the
predictive performance on the training set may be compared to the performance on the validation set. If the model
lacks performance on the latter, the given algorithm’s hyper-parameters may be adjusted to improve out-of-sample
predictions, if possible. For this purpose, strategies such as grid search or random search may be employed [102].
Alternatively, in projects that refrain from creating and using a validation set, hyper-parameter-search may be combined
with k-fold cross-validation to verify the predictive accuracy of a model:
K-fold cross-validation is an out-of-sample approach: a sample is split into k parts, where k-1 parts are
used as the training sample (for model estimation) and the k-th sample is designated out-of-sample. This
process is repeated k times, each time designating a different subsample as the out-of-sample part [103].
For each algorithm under consideration, the process of adjusting parameters and validating predictive out-of-sample
performance of the respective model is repeated until, ideally, a relatively generalizable model is found. Ultimately, this
model is selected to be tested on the withheld test data set in the evaluation phase.

4.4 Phase 4: Evaluation

In order to provide guidance on how to conduct evaluations in DSR projects, Venable, Pries-Heje, & Baskerville (2016)
[104] propose a framework for evaluation in design science (FEDS). In their framework, the authors suggest specifying
DSR evaluation efforts along two dimensions: (i) functional purpose of the evaluation and (ii) its paradigm. We now
elaborate on these two dimensions and their transferability to our methodology.
Functional Purpose. According to FEDS, the functional purpose of an evaluation specifies why to evaluate and may
either be (i) formative or (ii) summative. According to Lee and Hubona (2009) [105], formative validity is “an attribute
of the process by which a theory [or an artifact] is [. . . ] built” and summative validity is “an attribute of the sum result
or product of the process”. Accordingly, in FEDS, formative evaluations may help to improve the process of artifact
development and, thus, the artifact itself, whereas summative evaluations seek to judge the extent to which an artifact
matches expectations (i.e., the utility of the final artifact).
When it comes to building and evaluating prediction-oriented ML artifacts, however, the alternating process of hyper-
parameter-tuning and validating the predictive performance of a model during training comprises multiple formative
evaluation steps. This means that each assessment of the predictive performance of a model on a corresponding
validation set (or in k-fold cross validation) is—according to FEDS—equal to a formative evaluation step, as it serves
as “feedback” to improve the process of artifact development (i.e., model training). Thus, as such formative evaluation
is already considered in phase 3.3 of our methodology, and in line with all of the methodologies presented in section 2,
we propose to perform a summative evaluation in the actual evaluation phase, i.e., one evaluation per considered model
using the test set.
Paradigm. Aside from the functional purpose of an evaluation, FEDS distinguishes between two paradigms: (i)
artificial and (ii) naturalistic evaluation. The chosen paradigm specifies how to evaluate. In essence, artificial evaluation
refers to evaluation efforts that are conducted beyond the real environment, such as laboratory experiments, simulations,

15
Designing Explainable Predictive Machine Learning Artifacts

and mathematical proofs. In contrast, naturalistic evaluation aims to capture the performance of an artifact in its real
environment (e.g., an organization) and typically utilizes methods such as case studies, field experiments, and surveys.
Consequently, as for prediction-oriented ML artifacts, the extent to which an evaluation is rather artificial or naturalistic
depends on the source of the holdout data (i.e., the validation set for formative evaluations and the test set for summative
evaluations). In case the holdout set is simulated or comes from a laboratory experiment, the corresponding evaluation
is more artificial; in contrast, if the holdout set comes from the real environment, the corresponding evaluation is
more naturalistic. In both cases, the evaluation phase aims to assess model generalizability, that is, the accuracy of
out-of-sample predictions using suitable metrics, such as those mentioned in section 2.2. Moreover, in this phase, the
predictive performance of at least one ML model may be compared with competing models or at least one suitable
benchmark.
However, we contend that all evaluations that solely strive to assess the predictive accuracy of an ML model are rather
artificial. We justify this view by arguing that, in the real world, more aspects are important for evaluation than mere
predictive power, such as the actual—or at least intended—use of an artifact by relevant practitioners. That means
that, even in cases where an artifact may possess substantial predictive power, it cannot exploit its utility if it fails
adoption in the real world. By now, technology adoption research has repeatedly shown that the use of an artifact
is not compellingly caused by its objective usefulness, but rather its perceived usefulness, its perceived ease of use,
social influence, habit, trust, and other antecedents [106, 107, 108]. Thus, a comprehensive naturalistic evaluation
would consider (i) more aspects that are aimed at investigating adoption and (ii) respective human behavior and/or
opinions. However, in this paper, we refrain from exhaustively examining naturalistic evaluations, as these are neither
required for proving the predictions of an ML artifact accurate nor to interpret a black box ML model. Still, studies that
aim to develop predictive ML artifacts may consider conducting optional naturalistic evaluations—particularly once
artificial evaluations and model interpretations can be presented to an audience (i.e., after performing all the steps of our
methodology), as these most likely affect artifact adoption. Here, we especially expect model interpretability to foster
adoption. Figure 6 summarizes our elaborations on ML artifact evaluation.

Figure 6: FEDS for prediction-oriented artifacts

4.5 Phase 5: Understanding

After evaluation, at least one model needs to be selected for interpretation. We recommend opting for the model with
the highest predictive accuracy that still builds upon interpretable features. Furthermore, we deliberately discourage
analysts from interpreting models with low predictive power, as this will certainly result in unreliable explanations.

16
Designing Explainable Predictive Machine Learning Artifacts

Moreover, it is conceivable that, in this phase, interpretations from competing models are compared to verify consistency
of feature importance across models. As outlined before, to understand a black box model, we utilize SHAP as an
instance of explainable AI tools.
Once a model is chosen, explainable AI tools commonly allow for uncovering (i) local feature effects for single
observations, (ii) global feature effects for an entire model, (iii) feature interaction, and (iv) data type specific patterns.
For these purposes, SHAP offers respective visualizations, which we describe in the following according to their
documentations3 . Typically, to calculate SHAP values, the training data set is used [8]. However, in some exceptional
cases, it may be reasonable to calculate SHAP values based on other data sets, such as the test set, e.g., if local
predictions in the test data need to be explained. Yet, apart from such cases, we recommend drawing upon the training
data when it comes to model interpretation.
Force Plots. Force plots are designed to explain local predictions. To this end, force plots for single observations—and
their similar waterfall plots, to which we also refer as force plots—visualize the impact of each feature on the model
output, that is, the extent to which each feature contributes to push the corresponding prediction from the base value
(i.e., the average model output). In this context, every feature may either exert (i) a positive, (ii) a negative, or (iii) no
impact on a corresponding prediction. Force plots further order features by their respective importance for a given
prediction. Consequently, such plots are particularly suited to explain how and why individual predictions occur. In
addition, SHAP allows for visualizing multiple stacked force plots in one compound plot. Thereby, explanations for an
entire data set can be visualized. This approach is particularly suitable for exploring clusters of similar observations
within a respective data set (i.e., observations with similar model inputs and outputs) [31].
Summary Plots. As opposed to force plots, summary plots are rather suited to gain a global understanding of model-
wide feature importance. For this purpose, SHAP supports the visualization of average SHAP value magnitudes per
feature over an entire data set in the form of a summarized bar plot, which sorts features in descending order of global
importance. Such type of summary plot is especially valuable in situations that require an easily comprehensible
overview of important factors that are commonly associated with the phenomenon of interest. Aside from bar plots,
SHAP offers another type of summary plot (i.e., bee swarm plots) that visualizes the SHAP values of each feature
for every observation in the underlying data set. Although this plot also sorts features by their global importance, it
provides deeper insights into the impact of each feature on the model output, that is, the distribution of low/negative and
high/positive SHAP values across all observations. Therefore, this plot may particularly serve to investigate whether the
relationship between a feature and a target is positive/negative or linear/non-linear.
Dependence Plots. These plots are a valuable means to (i) understand the effect of a single feature on the model
output and (ii) to uncover interaction effects between features. Dependence plots achieve the former by plotting the
actual values of a given features against their corresponding SHAP values in a scatter plot. Moreover, to investigate
interactions, the points within the plot can be colored according to the values of a second feature. This allows for
uncovering bivariate interactions, i.e., the extent to which the model output is sensitive to changes in the values of two
interacting features.
Decision Plots. Decision plots are a powerful tool with which to support various types of analyses. They allow to
visualize the path of decisions, which is undergone when the feature values of an observation pass through a given model
when computing a respective prediction, starting from the base value. Decision plots support (i) visualizing the paths of
single and multiple observations from one or more models and (ii) processing SHAP as well as interaction values. As a
consequence, according to the documentation, such plots may serve to (i) show a large number of feature effects clearly;
(ii) visualize multi-output predictions; (iii) display the cumulative effect of interactions; (iv) explore feature effects for a
range of feature values; (v) identify outliers; (vi) identify typical predictions paths; (vii) and compare predictions for
several methods.
Heatmaps. Heatmaps combine several aspects of force plots for entire models and summary plots. Besides showing
feature importance across an entire data set, heatmaps visualize (i) model outputs and (ii) SHAP values for each feature
and observation. Moreover, heatmaps use coloring to highlight feature effects according to their corresponding SHAP
values. In doing so, SHAP orders observations in a way that visualizes aggregations of similar SHAP values as visible
clusters. Apart from supporting the uncovering of clusters, such heatmaps serve to explain both local and global feature
importance.
Data Type Specific Plots. These plots represent a special case of visualization supported by SHAP, as they are restricted
to visualize SHAP values for specific data types. On the upside, these types of plots are tailored for uncovering data
type specific patterns. Currently supported data types for data type specific plots are texts and images. Image plots, on
the one hand, serve to assign SHAP values to the pixels of a given image and, ultimately, color these pixels according to
3
The visualization types are documented at [Link] and [Link]
en/latest/[Link], which we accessed on the 23rd of September, 2020.

17
Designing Explainable Predictive Machine Learning Artifacts

these values. This allows to highlight regions on an image that are important for prediction. These plots are applicable
in cases where image data serves to train deep learning models. On the other hand, text plots allow for highlighting
words and text regions that are important in prediction-oriented text mining.
As mentioned in section 2.3.1, we hereby—again—point out that associations between a feature and the output of a
model do not inherently represent causal relationships, even in cases where a feature exerts a strong impact on the
target. Still, the investigation of such association-based explanations is highly valuable for gaining insights into the
nature of a given phenomenon and, thus, to inform both theory and practice, particularly because these associations—as
being the result of predictive modeling—already proved to possess a certain degree of generalizability. Therefore, such
associations should be understood as generalizable patterns that hold the potential to indicate causality.

5 Illustrative Use Case

After the development and description of a methodology that is suitable for guiding DSR efforts in building and
interpreting predictive ML-driven artifacts, we now showcase its application. For this purpose, we consider an
exemplary use case from the sharing economy. More precisely, we pass through each step of our methodology in order
to build and interpret an artifact that computes price recommendations for Airbnb listings. These recommendations are
intended to support property owners in finding a suitable rental price per night when first entering the market. Before
artifact development, in line with our methodology, we start by motivating and defining the corresponding research
problem and formulate proper research questions.

5.1 Problem Identification

Airbnb is a showpiece in the sharing economy comprising over 7 million listings in over 100,000 cities and more than
150 million users worldwide [109]. Consequently, Airbnb’s value exceeded $113 billion in 2021 with the company’s
yearly revenue being around $5.992 billion [110]. Since the company’s business model builds upon providing a
platform that brings together (i) property owners who want to rent out their homes for short time periods (hosts) and (ii)
guests, it is not only Airbnb that makes profit from their platform, but also the property owners themselves. This led
to an ever-increasing number of hosts who made a business from participating in Airbnb (currently around 650,000
worldwide) [109]. But, despite the great opportunity to profit from renting, participating in Airbnb can be challenging
for hosts, especially when it comes to finding the right rental price per night. This difficulty becomes particularly
evident for hosts who are new to the market for two reasons: (i) hosts may be unfamiliar with suitable pricing ranges
and (ii) their user profiles lack important price determinants that come with increasing membership duration, such as an
average rating score, number of ratings, and whether they are superhosts4 or not [93]. With reference to the literature
on recommender systems, we refer to this initial lack of information as cold-start problem. As a consequence of this
problem, a considerable number of tutorials in corresponding blogs and forums emerged which support hosts in finding
the right prices on Airbnb [111, 112, 113, 114]. Although these tutorials may have a certain value, most of them are
intended for hosts with some experience, provide instructions that are based on excessive manual research, or give
rather vague price recommendations. Moreover, from a scientific viewpoint, there has been some research on Airbnb
pricing—however, we are not aware of studies that particularly address the cold-start problem we described. Thus, we
argue that a tool which helps hosts on Airbnb to find a reasonable starting price when they first enter the market would
be both useful and innovative. Therefore, in the present exemplary study, we seek to address this issue by developing
an ML-driven artifact, which is able to predict appropriate starting prices per night and to provide corresponding
explanations. For this purpose, proper research questions are:
RQ1: What might be a suitable design of an ML artifact that assists hosts in finding starting prices on
Airbnb?
RQ2: Which insights can be drawn from the interpretation of the black box ML model driving the
artifact?

5.2 Conceptualization

5.2.1 Objective Definition

As per our formulated research questions, this exemplary study comprises two distinct objectives: (i) building an artifact
capable of predicting accurate prices for homes on Airbnb and (ii) gaining insights into factors driving Airbnb pricing
4
Hosts are typically awarded with the superhost badge in case they meet a range of pre-determined criteria, such as accommodating
at least 10 guests within a year, receiving at least a share 80% five-star ratings, and more.

18
Designing Explainable Predictive Machine Learning Artifacts

from the interpretation of the artifact. In order to meet the first objective, we train suitable ML models using actual data
from Airbnb. In doing so, we identify the model that can predict average prices per night most accurately. We calculate
these using a selection of prices from a one-year period. Thus, as the considered target variable is numerical, we solve a
regression problem. Moreover, as the artifact we develop is intended to support hosts that are new to the market, we
refrain from utilizing features associated with membership duration, such as the number of ratings, since the profiles of
new members lack these kinds of information. Besides the exclusion of such features, we do not exclude other features
from modeling a priori. With regard to our second objective, we employ SHAP to get insights into the ML model that
proves to possess the greatest predictive accuracy. These insights may assist (i) hosts in finding settings that can be
adjusted by them in order to charge higher prices per night and (ii) researchers in broadening their knowledge about
price finding in the sharing economy, particularly on Airbnb.

5.2.2 Conceptual Approach

In order to meet the objectives, we propose the conceptual approach depicted in Figure 7. The approach is divided
into two parts: (i) the concept for artifact development, evaluation, and global model interpretation and (ii) the concept
for the practical use of the artifact. The former illustrates how we make use of a data sample from Airbnb to develop
the artifact. In essence, we split the available data into a test and a training set to build and evaluate multiple black
box and benchmark models. Here, we opted for employing k-fold cross validation during model training instead of
using a validation set. Once the models are trained, we select the model with the highest predictive accuracy for global
interpretation. The insights gained from interpreting the entire model are particularly interesting for researchers who
seek to understand price finding on Airbnb in general. Furthermore, the right part of the conceptual approach depicts
how hosts may use the artifact once it is developed: A host may put the data of a home into the artifact to obtain a price
recommendation and a local explanation. Ideally, the latter supports the host in making useful adjustments in the listing
(e.g., upload more pictures) that contribute to justify an increase in price. As the proposed artifact is aimed at solving a
known problem (i.e., the cold-start problem on Airbnb) in a new way (i.e., by calculating price recommendations and
corresponding explanations automatically), it is an improvement in the sense of Gregor and Hevner’s (2013) [14] DSR
knowledge contribution framework.

Figure 7: Conceptual approach

19
Designing Explainable Predictive Machine Learning Artifacts

5.3 Implementation

5.3.1 Data Collection

To collect Airbnb data, we used a suitable API, which is available on GitHub5 . The API supports the collection of
various data on homes in a given city, such as locations, amenities, home types, and more. We collected 50 homes from
each of the 50 most populated cities in the USA, resulting in a total of 2,500 homes. Moreover, for each home, the API
allows to collect prices per night for a period of one year in advance. We collected all available prices per night for each
home. The data was gathered on April 30, 2020.

5.3.2 Data Exploration & Preparation

After data collection, we first reviewed the data for (i) missing values and (ii) duplicate entries6 . In doing so, we
deleted homes with at least one missing feature or target value to ensure data reliability. Moreover, we deleted duplicate
homes. This procedure resulted in the exclusion of 388 homes. Subsequently, to get representative prices per night, we
calculated average prices for each home. The left plot in Figure 8 depicts the corresponding price distribution. However,
the prices comprised 16 homes ( 0.76%) with average prices over 500$ per night, which we considered outliers. The
right plot in the figure shows the price distribution without outliers—note that both distribution plots are illustrated
using two different scales on both axes. Our further analysis builds upon the outlier-free data, which encompasses 2096
homes.

Figure 8: Price distribution

In a next step, we randomly split the available data into a training and a test set. While the former consists of
approximately 90% of the available data (i.e., 1886 homes), the latter comprises the remaining 10% (i.e., 210 homes).
As is common in predictive analytics, we conducted further analyses based on the training set. Thus, the test set solely
serves to assess the predictive power of the trained models in the evaluation step.
After splitting the data, we performed various feature engineering steps, in particular, clustering amenities and home
types; calculating distances to corresponding city centers; counting the number of foreign languages spoken by the host;
computing the ratios (i) beds per bedroom, (ii) persons per bedroom, and (iii) persons per bathroom; and leveraging
deep learning (i.e., convolutional neural networks) not only to classify preview pictures of each home as (i) a bathroom,
(ii) a bedroom, (iii) a dining room, (iv) an exterior area, (v) an interior area, (vi) a kitchen, or (vii) a living room
automatically [115], but also to assign an MOS (mean opinion score) as a measure of picture quality to each of these
preview pictures [116]. Moreover, we incorporated external data on the cities, where these homes are located, i.e., the
population and population density. Subsequently, we utilized the L1 regularization which is implemented in extreme
gradient boosting (XGBoost) [101], to select suitable features for model training. The final feature set consists of 47
features—we provide an overview of these in Appendix.
5
[Link]
6
The data exhibited duplicate homes because we considered the boroughs of New York as distinct cities when collecting the data
using the API. However, some homes were assigned to more than one borough on Airbnb, which is why some of these appeared
more than once in our sample.

20
Designing Explainable Predictive Machine Learning Artifacts

5.3.3 Model Training

For model training, we opted for using (i) two naïve dummy models that serve as predictive benchmarks, (ii) two white
box methods, and (iii) three popular state-of-the-art black box algorithms. The first are strategies that use arithmetic
mean and median values of the training prices to predict prices in the test set; the second are linear regression and a
decision tree; the third are an artificial neural network as implemented in FastAI (with two layers), a random forest, and
XGBoost. Moreover, we employed 5-fold cross validation combined with grid search to select hyper-parameters during
the training of the white box and black box models.

5.3.4 Evaluation
After training, we utilized every model to predict prices in the test set. To evaluate the predictive power of each model,
we calculated the mean average error (MAE), median absolute deviation (MAD), mean squared error (MSE), and the
prominent R2 metric. The results are shown in Table 2.

Table 2: Evaluation results

Model MAE MAD MSE Out-of-sample R2
Dummy (mean) 39.77 32.07 2859.39 -0.04
Dummy (median) 37.83 27.17 2957.64 -0.01
LR 31.54 22.81 1996.25 0.29
DT 31.99 21.5 2338.39 0.18
ANN 30.11 21.07 2085.01 0.27
RF 28.98 20.81 1819.85 0.36
XGB 27.35 18.91 1741.47 0.39
Models:
Dummy (mean) predicts the test data using the target mean from the training data.
Dummy (median) predicts the test data using the target median from the training data.
LR := Linear regression
DT := Decision tree
ANN := Artificial neural network
RF := Random forest
XGB := Extreme gradient boosting

Metrics:
MAE := Mean absolute error (lower values=better)
MAD := Median absolute deviation (lower values=better)
MSE := Mean squared error (lower values=better)
Out-of-sample R2 := Coefficient of determination calculated using predictions/observations
from the test set (higher values=better)

The evaluation unveils the two dummy models to be the two worst choices to predict prices on Airbnb. However, due to
their simplicity, this finding is rather unsurprising. More interesting is the considerable predictive power of the linear
regression model which not only outperforms the dummy benchmarks, but also surpasses the decision tree in all metrics
except for the MAD and even slightly beats the artificial neural networks MSE and R2 . Here, we particularly assume the
lack in predictive accuracy of the neural networks to result from the size of the training set and argue that the algorithm
would produce a superior model in a “big data” setting. However, both tree ensemble approaches provide predictions
superior to those of all other models, with the XGBoost model outperforming the random forest, thus being the most
accurate model.
As for the first of the research questions we formulated in the problem identification step, we can conclude that our
conceptual approach, combined with the features we used, our employed training strategy, and the XGBoost algorithm
constitutes a fairly suitable design for an artifact to predict starting prices on Airbnb. However, when recreating our
artifact, developers might consider using a larger training data set, which could take listings from other countries or
regions into account, depending on the corresponding field of application. Moreover, the creation of additional or
refined features is conceivable. By using a larger training sample or more features, we would generally expect the
XGBoost model to increase in predictive accuracy.
To answer the second of our research questions, we interpret the XGBoost model in the next step.

21
Designing Explainable Predictive Machine Learning Artifacts

5.3.5 Understanding

As per the conceptual approach we put forward, we are interested in (i) gaining a global understanding of the XGBoost
model and (ii) explanations of individual predictions that may serve to inform hosts.
We hereby start with global understanding. For this purpose, we employ SHAPs summary plots, as described in section
4.5. The left plot in Figure 9 depicts a bar summary plot, whereas the plot on the right side of the figure is a bee
swarm plot. Both plots show the 23 most important features for prediction in descending order. The remaining features
are comparatively negligible for price prediction. While the bars in the bar plot illustrate the average SHAP value
magnitude of each feature, the bee swarm plot additionally shows the distribution of individual feature values across
their corresponding SHAP values.

Figure 9: Global understanding

Figure 9 unveils the distance of a home to its corresponding city center (“distance_to_center”) as the most important
feature to predict prices per night on Airbnb. Here, despite very few exceptions, medium and high distances exert a
negative impact on the model output, whereas low distances tend to predict higher prices. The next most important
features are the population density of a city (“population_density”), a home’s capacity in terms of people that can stay
overnight (“person_capacity”), and a city’s total population (“city_population”). In essence, all of these features tend to
push the XGB models output towards lower prices if feature values are low and vice versa. Therefore, until here, we can
conclude that large homes, which are close to the centers of big cities, are typically more expensive than other homes.
The remaining features may be examined in a similar fashion to gain a more complete understanding of the model.
However, instead of investigating every effect, we focus on the most interesting ones:

• Number of uploaded pictures (“picture_count”): An increasing number of pictures is associated with higher
prices. This effect is not only the sixth most important feature that predicts the target in the XGBoost model,
but it is also especially interesting as it is not determined by the inherent characteristics of a home or its
location. Although we cannot reliably confirm a direct causal effect between the number of pictures and the
price (i.e., both variables may also be the result of other causes), we argue that, in practice, it might help to
upload more pictures to justify an increase in price—even though we cannot confirm causality, we cannot
preclude it either.

22
Designing Explainable Predictive Machine Learning Artifacts

• Number of persons per bedroom/bathroom (“persons_per_bedroom”/”persons_per_bathroom”): As much as it

would be conceivable that a lower number of persons per bedroom or bathroom was associated with higher
prices (due to more space per person), the opposite (i.e., more persons per room leading to higher prices)
would likewise be reasonable (due to a greater person capacity). According to the bee swarm plot, the latter is
the case. Yet, these effects are heavily influenced by the corresponding home type (i.e., whether the home is an
entire place or a private room). We provide an overview of these and other feature interactions in Appendix.
• Quality of the preview picture (“MOS”): The quality of the preview picture, which we measured using the
MOS as proposed by Hosu et al. (2020) [116], is also considerably predictive of our target. Whereas lower
quality pictures tend to predict lower prices, higher MOSs broadly push the model output towards higher
prices. Here, we draw the same conclusions as for the number of pictures—although we cannot confirm a
causal effect, we recommend hosts to use high quality preview pictures if they want to raise prices.
• Flexible canceling (“CANCEL_FLEXIBLE”): This effect is particularly interesting as the presence of a
flexible canceling policy clearly exerts a negative influence on the model output, whereas other, stricter
canceling policies contribute to increase prices. We argue that this most likely is not a direct causal effect, as it
is meaningless—we would decisively expect flexible canceling to be perceived as convenient by guests. We
rather find it conceivable that other factors prompt hosts to select flexible canceling for their low-price homes
(e.g., to compensate for other disadvantages). Yet, this effect may have the potential to constitute a worthwhile
avenue for future research.
• The preview picture shows a bedroom ("picture_bedroom"): Interestingly, having a bedroom as a preview
picture exerts a negative impact on the price while other preview pictures (e.g., living room, kitchen, or
exterior) have a positive or no impact on the target. Although this effect might be sample-specific, with our
current state of knowledge we would recommend hosts to refrain from using bedroom pictures as preview
pictures. Yet, this effect might be investigated further in the future.
• Maximum/minimum number of overnight stays (“max_nights”/”min_nights”): While a high number of
minimum nights required to rent a home are distinctly associated with lower prices, low values can either have
a negative, positive, or no impact on the model output. Moreover, some medium values even push the model
output towards high prices. Even more ambiguous is the effect of maximum nights—while high values tend
to have less impact on the model output than low values, the latter either have (i) a negative or (ii) a strong
positive influence on prices. Without further analyses, this effect is hardly explainable.
• Number of foreign languages (“sum_foreign_languages”): Along with “max_nights”, interpreting this effect
would also require further analyses. It seems like a medium number of foreign languages have a negative
impact on the target variable, while high values either have a strong positive or negative influence. Low values
have no or a weak positive influence on prices.

While some of the global interpretations are highly informative (e.g., that more pictures are associated with higher
prices), some are rather vague (e.g., that flexible canceling is associated with lower prices). However, besides the value
of the former to serve as cornerstones for price finding on Airbnb, the latter may serve to draw researchers’ attention
and, thus, prompt them to investigate these unclear effects in their future efforts. Moreover, our findings are well-suited
to be incorporated in explanatory models that serve to explain prices on Airbnb or even the sharing economy as a whole.
In addition to a global understanding, we next show how local predictions and explanations may be used to support
individual hosts. For this purpose, we use SHAP’s force plots and consider two exemplary homes from our data:
one with a lower price (prediction 1) and one with a higher price (prediction 2). Both predictions—along with their
corresponding force plots—are depicted in Figure 10. For each prediction, the figure shows a preview of the input
a host would need to provide (i.e., the preview picture of a home and additional data required to build our features)
and the composition of the corresponding prediction as per the force plot. Each force plot illustrates the XGBoost
model’s (i) base value (i.e., the price the model would predict if no feature values were available), (ii) important feature
values that contribute to push the actual model output from the base value (orange contributions increase the price/green
contributions decrease the price), and the actual model output (i.e., the prediction) shown in bold between the orange
and green contributions. We note that the prices are consistently given in US$ and that both predictions are fairly
accurate—while prediction 1 (27.33$) exhibits an absolute error of -8.63$ (i.e., the actual average price is 18.70$),
prediction 2 (215.56$) has an error of 7.86$ (i.e., the actual average price is 223.42$).
As opposed to prediction 1, the listing of prediction 2 broadly exploits the range of features that constitute a relatively
high price. Here, besides features that are determined by the characteristics of a home, particularly (i) the canceling
policy, (ii) the high quality preview picture, and (iii) the host’s verified profile contribute to an increased price. On
the downside, the relatively high distance to the city center slightly contributes to a decrease in price. The force
plot of prediction 1, on the other hand, exhibits the listing’s potential for improvement with respect to a higher price

23
Designing Explainable Predictive Machine Learning Artifacts

recommendation. This refers in particular to (i) using a high-quality preview picture (an MOS of 2.99 is rather low in
our data) of (ii) another room (i.e., not the bedroom) and (iii) uploading more pictures.

Figure 10: Local explanations

6 Conclusion
In the present paper we developed and showcased a methodology that is tailored to guide the development and
interpretation of ML-driven artifacts. To this end, we first positioned our paper according to existing research in DSR
and predictive analytics in IS and presented available methodologies in these fields. Moreover, we elaborated on
research in explainable AI and highlighted how the integration of corresponding knowledge into DSR may add value to
IS research. We further uncovered the current lack of interpretable predictive modeling efforts in DSR and the need for a
consistent methodology in this area by means of a respective literature review. Subsequently, we built our methodology
upon the most important aspects of methodologies from DSR, predictive analytics, and explainable AI and provided
valuable descriptions and execution guidelines. Finally, we showcased the methodology using the example of Airbnb
price prediction and explanation for hosts that are new to the platform.
Throughout the paper, we were able to provide several contributions to research in IS. Among these are particularly the
following: first, since our methodology is composed of knowledge from DSR and predictive analytics, we bridged the
boundaries between these two fields. This is especially valuable for scholars who seek to develop prediction-oriented
artifacts, as they are no longer compelled to decide whether to follow the DSR or predictive analytics paradigm—with
the use of our methodology, they can do both at once. Moreover, our methodology is even applicable in prediction-
oriented DSR studies that deliberately refrain from black box interpretation, as such efforts could simply follow the
process through to evaluation and then disregard the understanding phase. Second, by considering the use of SHAP
within the methodology, we introduced the proper use of a state-of-the-art approach to explainable AI from research in
computer science to DSR. With this contribution, we particularly intend to pave the way for more studies in DSR that

24
Designing Explainable Predictive Machine Learning Artifacts

develop interpretable black box ML artifacts for prediction tasks. Third, in our use case, we showed how to extract
insights from a black box ML model by the use of global explanations. Such extraction of insights may serve to
stimulate theorizing and complement more traditional quantitative research in IS (i.e., explanatory modeling) in that it
makes use of knowledge induction instead of deduction. In IS research, such inductive generation of knowledge is
currently almost exclusively reserved for qualitative research in the sense of grounded theory [117, 12]. However, most
of the principles of grounded theory are transferrable to quantitative data and are well-suited to be applied along with
predictive analytics [3]. Hence, the use of our methodology may help to inform explanatory research efforts by enabling
knowledge induction from quantitative data.
In addition to the contributions to research, our paper has certain practical implications. These are particularly the
following: first, as our paper aims to encourage scholars to develop interpretable prediction-oriented ML artifacts, we
contribute to the diffusion of ML in organizations, as such artifacts may serve as blueprints for real-world applications
whose predictions can be explained to human decision-makers. The value of this contribution is twofold: (i) with the
incorporation of explainable AI, ML applications can be employed in business cases that require system explainability,
e.g., if decisions based on system predictions need to be justified legally or vis-à-vis prudential regulators; (ii) human
decision-makers are more likely to adopt a system if they can comprehend the factors that constitute a prediction. Second,
apart from researchers, practitioners may also use our methodology to develop artifacts—this might be particularly
interesting within R&D departments of organizations. Here, one promising avenue might be the incorporation of
explainable AI within automated ML services, such as Google Cloud AutoML. Such services help consumers to develop
ML-based applications—the incorporation of explainable AI might additionally support consumers in understanding an
underlying model. Third, by showcasing the use of local explanations in our use case, we showed how explainable AI
may assist practitioners in their decision-making. Thereby, we hope to inspire developers to build similar applications
that may add value to practice.
Despite our contributions, our paper naturally comprises certain limitations, particularly with regard to our exemplary
use case, which we deliberately kept compact to avoid exceeding an appropriate scope of this paper. One limitation arises
from (i) the size of the Airbnb sample and (ii) its scope, that is, the sample could have been more comprehensive. This
means that we might have trained a more generalizable model if we had used a sample with more observations, ideally
from more regions in the USA and perhaps even other countries. Another limitation is our feature selection—although
we already considered the use of numerous features, it would have been conceivable to incorporate more variables, such
as points of interest near the individual homes (e.g., restaurants, gas stations, or supermarkets), additional information
about neighborhoods (e.g., crime rates or weather data), or textual information from the descriptions of the homes.
Moreover, by predicting average prices for each home, we did not take seasonal price fluctuations into account, which
might have provided an even deeper understanding about pricing on Airbnb. However, since the focus of our use
case was in showcasing our methodology, we consider its scope appropriate. In addition, the limitations may serve to
encourage scholars to investigate the corresponding effects in future studies. Beyond the use case, we recognize that our
paper puts a particularly strong emphasis on the use of SHAP and, thereby, neglects the application of other explainable
AI approaches. Yet, as we pointed out throughout the paper, we currently find SHAP to be the most advanced approach
to explainable AI. Therefore, we consider it well-suited to demonstrate the capabilities of explainable AI (i.e., in
particular, the use of global/local explanations and feature interactions). Still, authors may use other approaches in the
future, depending on the suitability with their projects or advances in explainable AI.
With our paper, we particularly intend to actively contribute to the development, acceptance, and use of ML-based
applications, as we believe these hold great potential to facilitate knowledge induction in academia, increase productivity
in organizations, and may assist us in our daily lives.

References

[1] Michael Chui, James Manyika, Mehdi Miremadi, Nicolaus Henke, Rita Chung, Pieter Nel, and Sankalp Malhotra.
Notes from the AI frontier: Applications and value of deep learning. Technical report, 2018.
[2] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436–444, 2015.
[3] Galit Shmueli and Otto R. Koppius. Predictive analytics in information systems research. MIS Quarterly,
35(3):553–572, 2011.
[4] Grand View Research Inc. Advanced Analytics Market Size Report, 2022-2030. Technical report, 2022.
[5] Sreekar Krishna, Bharat Rao, Bob Parr, and Martin Sokalski. Living in an AI world. Technical report, 2020.
[6] Leo Breiman. Statistical modeling: The two cultures. Statistical Science, 16(3):199–231, 2001.

25
Designing Explainable Predictive Machine Learning Artifacts

[7] Ujwal Kayande, Arnaud De Bruyn, Gary L. Lilien, Arvind Rangaswamy, and Gerrit H. van Bruggen. How
incorporating feedback mechanisms in a DSS affects DSS evaluations. Information Systems Research, 20(4):527–
546, 2009.
[8] Scott M. Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M. Prutkin, Bala Nair, Ronit Katz,
Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. From local explanations to global understanding with
explainable AI for trees. Nature Machine Intelligence, 2(1):56–67, 2020.
[9] Alan R Hevner, Salvatore T March, Jinsoo Park, Sudha Ram, and Sudha Ram. Research Essay Design Science
in Information. MIS Quarterly, 28(1):75–105, 2004.
[10] John Venable, Jan Pries-heje, and Richard Baskerville. Choosing a design science research methodology. In
Australasian Conference on Information Systems, pages 1–11, 2017.
[11] Usama Fayyad, Gregory Piatetsky-Shapiro, and Padhraic Smyth. The KDD process for extracting useful
knowledge from volumes of data. Communications of the ACM, 39(11):27–34, 1996.
[12] Oliver Müller, Iris Junglas, Jan vom Brocke, and Stefan Debortoli. Utilizing big data analytics for information
systems research: challenges, promises and guidelines. European Journal of Information Systems, 25(4):289–302,
2016.
[13] Ken Peffers, Tuure Tuunanen, and Björn Niehaves. Design science research genres: introduction to the special
issue on exemplars and criteria for applicable design science research. European Journal of Information Systems,
27(2):129–139, 2018.
[14] Shirley Gregor and Alan R. Hevner. Positioning and presenting design science research for maximum impact.
MIS Quarterly, 37(2):337–355, 2013.
[15] Shirley Gregor and David Jones. The anatomy of a design theory. Journal of the Association for Information
Systems, 8(5):312–335, 2007.
[16] Salvatore T. March and Gerald F. Smith. Design and natural science research on information technology.
Decision Support Systems, 15(4):251–266, 1995.
[17] Alan R. Hevner. A three cycle view of design science research. Scandinavian Journal of Information Systems,
19(2):87–92, 2007.
[18] Andreas Drechsler and Alan Hevner. A four-cycle model of IS design science research: Capturing the dynamic
nature of IS artifact design. university College Cork, Ireland, 2016.
[19] Richard Baskerville and Jan Pries-Heje. Explanatory design theory. Business & Information Systems Engineering,
2(5):271–282, 2010.
[20] Richard Baskerville, Abayomi Baiyere, Shirley Gregor, Alan Hevner, and Matti Rossi. Design science research
contributions: Finding a balance between artifact and theory. Journal of the Association for Information Systems,
19(5):358–376, 2018.
[21] Jay F. Nunamaker, Minder Chen, and Titus D.M. Purdin. Systems development in information systems research.
Journal of Management Information Systems, 7(3):89–106, 1990.
[22] Vijay Vaishnavi and William Kuechler. Design science research methods and patterns: Innovating information
and communication technology. CRC Press, Inc., USA, 2015.
[23] Vijay Vaishnavi and William Kuechler. Design research in information systems. Association for Information
Systems, 2004.
[24] Vijay Vaishnavi and Bill Kuechler. On theory development in design science research: Anatomy of a research
project. European Journal of Information Systems, 17(5):489–504, 2008.
[25] Ken Peffers, Tuure Tuunanen, Marcus A. Rothenberger, and Samir Chatterjee. A design science research
methodology for information systems research. Journal of Management Information Systems, 24(3):45–77,
2007.
[26] Sein, Henfridsson, Purao, Rossi, and Lindgren. Action design research. MIS Quarterly, 35(1):37, 2011.
[27] Richard Baskerville, Jan Pries-Heje, and John Venable. Soft design science methodology. In Proceedings of the
International Conference on Design Science Research in Information Systems and Technology, 2009.
[28] Mark Bilandzic and John Venable. Towards participatory action design research: Adapting action research and
design science research methods for urban informatics. The Journal of Community Informatics, 7(3):1–17, 2011.
[29] Dursun Delen and Hamed M. Zolbanin. The analytics paradigm in business research. Journal of Business
Research, 90:186–195, 2018.

26
Designing Explainable Predictive Machine Learning Artifacts

[30] Ritu Agarwal and Vasant Dhar. Big data, data science, and analytics: The opportunity and challenge for IS
research. Information Systems Research, 25(3):443–448, 2014.
[31] Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In International
Conference on Neural Information Processing Systems, pages 4768–4777, 2017.
[32] Roberto Iriondo. Machine learning (ML) vs. AI and their important differences, 2018.
[33] Herbert Alexander Simon. The sciences of the artificial. MIT Press, Cambridge, Massachusetts; London,
England, 1996.
[34] Giuseppe Casalicchio, Christoph Molnar, and Bernd Bischl. Visualizing the feature importance for black box
models. In Michele Berlingerio, Francesco Bonchi, Thomas Gärtner, Neil Hurley, and Georgiana Ifrim, editors,
Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2018. Lecture Notes in Computer
Science, pages 655–670. Springer, Cham, 2019.
[35] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Model-Agnostic Interpretability of Machine Learning.
(Whi), 2016.
[36] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating
activation differences. International Conference on Machine Learning, 7:4844–4866, 2017.
[37] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus Robert Müller, and Wojciech
Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS
ONE, 10(7):1–46, 2015.
[38] Stan Lipovetsky and Michael Conklin. Analysis of regression in game theory approach. Applied Stochastic
Models in Business and Industry, 17(4):319–330, 2001.
[39] Erik Štrumbelj and Igor Kononenko. Explaining prediction models and individual predictions with feature
contributions. Knowledge and Information Systems, 41(3):647–665, 2014.
[40] Anupam Datta, Shayak Sen, and Yair Zick. Algorithmic transparency via quantitative input influence: Theory
and experiments with learning systems. Proceedings of the Symposium on Security and Privacy, pages 598–617,
2016.
[41] Scott M. Lundberg and Su-In Lee. Consistent feature attribution for tree ensembles. 2017.
[42] Julian Tritscher, Markus Ring, Daniel Schlör, Lena Hettinger, and Andreas Hotho. Evaluation of post-hoc xai
approaches through synthetic tabular data. In International Syposium on Methodologies for Intelligent Systems,
2020.
[43] Marko Bohanec, Marko Robnik-Šikonja, and Mirjana Borštnar. Organizational learning supported by machine
learning models coupled with general explanation methods: A case of B2B sales forecasting. Organizacija,
50(3):217–233, 2017.
[44] Marko Bohanec, Marko Robnik-Šikonja, and Mirjana Borštnar. Decision-making framework with double-loop
learning through interpretable black-box machine learning models. Industrial Management and Data Systems,
117(7):1389–1406, 2017.
[45] Yao Ming, Huamin Qu, and Enrico Bertini. Rulematrix: Visualizing and understanding classifiers with rules.
IEEE transactions on visualization and computer graphics, 25(1):342–352, 2018.
[46] Francesco Ventura, Tania Cerquitelli, and Francesco Giacalone. Black-box model explained through an assess-
ment of its interpretable features. In Communications in Computer and Information Science, 2018.
[47] Francesco Colace, Massimo De Santo, Marco Lombardi, Fabio Mercorio, Mario Mezzanzanica, and Francesco
Pascale. Towards labour market intelligence through topic modelling. In Proceedings of the Hawaii International
Conference on System Sciences, 2019.
[48] Verena Eitle and Peter Buxmann. Business analytics for sales pipeline management in the software industry: A
machine learning perspective. In Proceedings of the Hawaii International Conference on System Sciences, pages
1013–1022, 2019.
[49] Mojtaba Yeganejou, Scott Dick, and James Miller. Interpretable deep convolutional fuzzy classifier. Transactions
on Fuzzy Systems, 2019.
[50] Aneseh Alvanpour, Sumit Kumar Das, Christopher Kevin Robinson, Olfa Nasraoui, and Dan Popa. Robot Failure
Mode Prediction with Explainable Machine Learning. In IEEE International Conference on Automation Science
and Engineering, volume 2020-August, 2020.
[51] Steven Bramhall, Hayley Horn, Michael Tieu, and Nibhrat Lohia. QLIME—A quadratic local interpretable
model-agnostic explanation approach. SMU Data Science Review, 3(1):73–88, 2020.

27
Designing Explainable Predictive Machine Learning Artifacts

[52] Daniel Dolk, Donald Kridel, Jacob Dineen, and David Castillo. Model interpretation and explainability towards
creating transparency in prediction models. In Proceedings of the Hawaii International Conference on System
Sciences, pages 964–973, 2020.
[53] Maximilian Harl, Sven Weinzierl, Mathias Stierle, and Martin Matzner. Explainable predictive business process
monitoring using gated graph neural networks. Journal of Decision Systems, pages 1–16, 2020.
[54] Buomsoo Kim, Karthik Srinivasan, and Sudha Ram. Robust local explanations for healthcare predictive analytics:
An application to fragility fracture risk modeling. In International Conference on Information Systems, 2020.
[55] Nijat Mehdiyev and Peter Fettke. Prescriptive process analytics with deep learning and explainable artificial
intelligence. In European Conference on Information Systems, 2020.
[56] Gopikrishna Yadam, Akshata Kishore Moharir, and Ishita Srivastava. Explainable and visually interpretable
machine learning for flight sciences. In International Conference on Electronics, Computing and Communication
Technologies, pages 1–6, 2020.
[57] Xiaohui Zhang, Qianzhou Du, and Zhongju Zhang. An explainable machine learning framework for fake
financial news detection. In International Conference on Information Systems, ICIS 2020 - Making Digital
Inclusive: Blending the Local and the Global, 2020.
[58] Claus Boye Asmussen, Steffen Lundgaard Jørgensen, and Charles Møller. Design and deployment of an analytic
artefact–investigating mechanisms for integrating analytics and manufacturing execution system. Enterprise
Information Systems, 2021.
[59] Arash Barfar, Balaji Padmanabhan, and Alan Hevner. Peak cubes in service operations: Bringing multidimen-
sionality into decision support systems. Decision Support Systems, 140:113442, 2021.
[60] Martin Garriga, Koen Aarns, Christos Tsigkanos, Damian A. Tamburri, and Wjan Van Den Heuvel. DataOps
for Cyber-Physical Systems Governance: The Airport Passenger Flow Case. ACM Transactions on Internet
Technology, 21(2), 2021.
[61] Xiao Han, Leye Wang, and Weiguo Fan. Is hidden safe? Location protection against machine-learning prediction
attacks in social networks, 2021.
[62] Lukas-Valentin Herm, Jonas Wanner, Franz Seubert, and Christian Janiesch. I DON’T GET IT, BUT IT SEEMS
VALID! THE CONNECTION BETWEEN EXPLAINABILITY AND COMPREHENSIBILITY IN (X)AI
RESEARCH. In European Conference on Information Systems, pages 6–14, 2021.
[63] Marina Johnson, Abdullah Albizri, Antoine Harfouche, and Salih Tutun. Digital transformation to mitigate
emergency situations: increasing opioid overdose survival rates through explainable artificial intelligence.
Industrial Management and Data Systems, 2021.
[64] Angela Xia Liu, Yilin Li, and Sean Xin Xu. Assessing the unacquainted: Inferred reviewer personality and
review helpfulness. MIS Quarterly: Management Information Systems, 45(3), 2021.
[65] Nijat Mehdiyev and Peter Fettke. LOCAL POST-HOC EXPLANATIONS FOR PREDICTIVE PROCESS
MONITORING IN MANUFACTURING. In European Conference on Information Systems, pages 6–14, 2021.
[66] Haadi Mombini, Bengisu Tulu, Diane Strong, Emmanuel O. Agu, Clifford Lindsay, Lorraine Loretz, Peder C.
Pedersen, and Raymond Dunn. An explainable machine learning model for chronic wound management decisions.
In 27th Annual Americas Conference on Information Systems, AMCIS 2021, 2021.
[67] Filipe Dwan Pereira, Samuel C. Fonseca, Elaine H.T. Oliveira, Alexandra I. Cristea, Henrik Bellhauser, Luiz
Rodrigues, David B.F. Oliveira, Seiji Isotani, and Leandro S.G. Carvalho. Explaining individual and collective
programming students’ behaviour by interpreting a black-box predictive model. IEEE Access, 2021.
[68] Srikar Velichety and Sudha Ram. Finding a needle in the haystack: Recommending online communities on
social media platforms using network and design science. Journal of the Association for Information Systems,
22(5), 2021.
[69] Tong Wang, Cheng He, Fujie Jin, and Yu Jeffrey Hu. Evaluating the Effectiveness of Marketing Campaigns for
Malls Using a Novel Interpretable Machine Learning Model. Information Systems Research, 2021.
[70] Jacqueline Wastensteiner, Tobias Michael Weiss, Felix Haag, and Konstantin Hopf. Explainable AI for tailored
electricity consumption feedback–An experimental evaluation of visualizations experimental evaluation of
visualizations. ECIS 2021 Research Papers, (Ecis), 2021.
[71] Jilei Zhou, Cong Wang, Fei Ren, and Guoqing Chen. Inferring multi-stage risk for online consumer credit
services: An integrated scheme using data augmentation and model enhancement. Decision Support Systems,
149, 2021.

28
Designing Explainable Predictive Machine Learning Artifacts

[72] Frank Bodendorf, Qiao Xie, Philipp Merkl, and Jörg Franke. A multi-perspective approach to support collabora-
tive cost management in supplier-buyer dyads. International Journal of Production Economics, 245:108380,
2022.
[73] Marina Johnson, Abdullah Albizri, Antoine Harfouche, and Samuel Fosso-Wamba. Integrating human knowledge
into artificial intelligence for complex and ill-structured problems: Informed artificial intelligence. International
Journal of Information Management, 64(102479), 2022.
[74] Peter Kowalczyk, Marco Röder, Alexander Dürr, and Frédéric Thiesse. Detecting and Understanding Textual
Deepfakes in Online Reviews. In Hawaii International Conference on System Sciences, 2022.
[75] Jie Tao, Lina Zhou, and Kevin Hickey. Making sense of the black-boxes: Toward interpretable text classification
using deep learning models. Journal of the Association for Information Science and Technology (JASIST), pages
1–16, 2022.
[76] Kai Yang, Hui Yuan, and Raymond Y K Lau. PsyCredit: An interpretable deep learning-based credit assessment
approach facilitated by psychometric natural language processing. Expert Systems with Applications, 198:116847,
2022.
[77] Been Kim, Rajiv Khanna, and Oluwasanmi Koyejo. Examples are not enough, learn to criticize! Criticism for
interpretability. In Advances in Neural Information Processing Systems, 2016.
[78] Rüdiger Wirth and Jochen Hipp. CRISP-DM: Towards a standard process model for data mining. In International
Conference on the Practical Application of Knowledge Discovery and Data Mining, 2000.
[79] Jannis Hanke, Matthias Hauser, Alexander Dürr, and Frédéric Thiesse. Redefining the offline retail experience:
Designing product recommendation systems for fashion stores. In European Conference on Information Systems,
pages 1–14, 2018.
[80] Matthias Hauser, Sebastian A. Günther, Christoph M. Flath, and Frédéric Thiesse. Towards digital transformation
in fashion retailing: A design-oriented IS research study of automated checkout systems. Business & Information
Systems Engineering, 61(1):51–66, 2019.
[81] Alexander Dürr, Matthias Griebel, Giacomo Welsch, and Frédéric Thiesse. Predicting fraudulent initial coin
offerings using information extracted from whitepapers. In European Conference on Information Systems, 2020.
[82] Leo Breiman. Random forests. Machine Learning, 45:5–32, 2001.
[83] Gartner Inc. Synthetic Data, 2020.
[84] Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image
synthesis. 2018.
[85] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville,
and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages
2672–2680, 2014.
[86] Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. Modeling tabular data using
conditional GAN. In Advances in Neural Information Processing Systems, pages 7335–7345, 2019.
[87] Lei Xu and Kalyan Veeramachaneni. Synthesizing tabular data using generative adversarial networks. 2018.
[88] Usama Fayyad, G. Georges Grinstein, and Andreas Wirse. Information visualization in data mining and
knowledge discovery. Morgan Kaufmann, San Francisco, CA, United States, 2002.
[89] Ian H. Witten, Frank Eibe, and Mark A. Hall. Data mining: Practical machine learning tools and techniques.
Morgan Kaufmann, Burlington, MA, USA, 2011.
[90] Yufeng Ding and Jeffrey S. Simonoff. An investigation of missing data methods for classification trees applied
to binary response data. Journal of Machine Learning Research, 11:131–170, 2010.
[91] Alice Zheng and Amanda Casari. Feature engineering for machine learning: Principles and techniques for data
scientists. O’Reilly Media, Inc., Sebastopol, CA, 2018.
[92] Charu C. Aggarwal and ChengXiang Zhai. Mining text data. Springer US, Boston, MA, 2012.
[93] Timm Teubner, Florian Hawlitschek, and David Dann. Price determinants on Airbnb: How reputation pays off in
the sharing economy. Journal of Self-Governance and Management Economics, 5(4):53, 2017.
[94] Anna B. Costello and Jason W. Osborne. Best practices in exploratory factor analysis: Four recommendations
for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(7):1–9, 2005.
[95] Christine DiStefano, Min Zhu, and Diana Mîndrilǎ. Understanding and using factor scores: Considerations for
the applied researcher. Practical Assessment, Research, and Evaluation, 14(20):1–11, 2008.

29
Designing Explainable Predictive Machine Learning Artifacts

[96] Anuja Nagpal. L1 and L2 regularization methods, 2017.

[97] Andrew Y Ng. Feature selection, L1 vs. L2 regularization, and rotational invariance. In International Conference
on Machine Learning, 2004.
[98] Zhaohui Zheng, Xiaoyun Wu, and Rohini Srihari. Feature selection for text categorization on imbalanced data.
ACM SIGKDD Explorations Newsletter, 6(1):80–89, 2004.
[99] Vladimir Vapnik. The nature of statistical learning theory. Springer, New York, NY, USA, 1995.
[100] Jeremy Howard and Sylvain Gugger. Fastai: A layered API for deep learning. 2020.
[101] Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. In International Conference on
Knowledge Discovery, 2016.
[102] James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. Journal of Machine
Learning Research, 13(10):281–305, 2012.
[103] Galit Shmueli, Soumya Ray, Juan Manuel Velasquez Estrada, and Suneel Babu Chatla. The elephant in the room:
Predictive performance of PLS models. Journal of Business Research, 69(10):4552–4564, 2016.
[104] John Venable, Jan Pries-Heje, and Richard Baskerville. FEDS: A framework for evaluation in design science
research. European Journal of Information Systems, 25:77–89, 2016.
[105] Allen S. Lee and Geoffrey S. Hubona. A scientific basis for rigor in information systems research. MIS Quarterly:
Management Information Systems, 2009.
[106] Fred D. Davis, Richard P. Bagozzi, and Paul R. Warshaw. User acceptance of computer technology: A comparison
of two theoretical models. Management Science, 35(8):982–1003, 1989.
[107] David Gefen, Elena Karahanna, and Detmar W. Straub. Trust and tam in online shopping: An integrated model.
MIS Quarterly: Management Information Systems, 27(1):51–90, 2003.
[108] Viswanath Venkatesh, James Y.L. Thong, and Xin Xu. Consumer acceptance and use of information technology:
Extending the unified theory of acceptance and use of technology. MIS Quarterly: Management Information
Systems, 36(1):157–178, 2012.
[109] IPropertyManagement. Airbnb statistics, 2020.
[110] Macrotrends LLC. Airbnb Revenue 2018-2022. [Link]
2022.
[111] Airbnb Community. How to start an Airbnb, 2016.
[112] Airbnb Inc. Set a price strategy, 2020.
[113] Guesty Inc. 2 Steps To Skyrocket Your Airbnb Pricing Strategy, 2018.
[114] Renting Your Place. Airbnb Pricing Strategy, 2020.
[115] Omid Poursaeed, Tomáš Matera, and Serge Belongie. Vision-based real estate price estimation. Machine Vision
and Applications, 29:667–676, 2018.
[116] Vlad Hosu, Hanhe Lin, Tamas Sziranyi, and Dietmar Saupe. KonIQ-10k: An ecologically valid database for
deep learning of blind image quality assessment. Transactions on Image Processing, 9:4041–4056, 2020.
[117] Barney G. Glaser and Anselm L. Strauss. The discovery of grounded theory: Strategies for qualitative research.
Aldine Publishing, Chicago, USA, 1967.

30
Designing Explainable Predictive Machine Learning Artifacts

Appendix A: Feature Set

Feature Name Feature Description

The mean opinion score is a measure of perceived image quality.
MOS We applied it to homes’ preview images. MOS values are between
1 (low quality) and 5 (high quality).
bathrooms Number of bathrooms.
bedrooms Number of bedrooms.
beds Number of beds.
person_capacity Maximum number of guests.
picture_count Number of uploaded pictures of a home.
min_nights Minimum number of overnight stays required for booking.
max_nights Maximum number of overnight stays.
can_instant_book Boolean value that signalizes whether a home can be booked instantly.
verified_card Boolean value that signalizes whether the host has a verified profile.
en Boolean value that signalizes whether the host speaks English.
preview_wifi Boolean value that signalizes whether WiFi is a preview amenity.
preview_kitchen Boolean value that signalizes whether a kitchen is a preview amenity.
preview_heating Boolean value that signalizes whether heating is a preview amenity.
preview_air_conditioning Boolean value that signalizes whether air conditioning is a preview amenity.
preview_free_parking Boolean value that signalizes whether free parking is a preview amenity.
preview_washer Boolean value that signalizes whether a washer is a preview amenity.
preview_pool Boolean value that signalizes whether a pool is a preview amenity.
preview_hot_tub Boolean value that signalizes whether a hot tub is a preview amenity.
CANCEL_FLEXIBLE Boolean value that signalizes whether canceling is “flexible”.
CANCEL_MODERATE Boolean value that signalizes whether canceling is “moderate”.
CANCEL_STRICT_14 Boolean value that signalizes whether canceling is allowed within 14 days.
CANCEL_SUPER_STRICT_30 Boolean value that signalizes whether canceling is allowed within 30 days.
CANCEL_SUPER_STRICT_60 Boolean value that signalizes whether canceling is allowed within 60 days.
picture_bathroom Boolean value that signalizes whether the preview picture shows a bathroom.
picture_bedroom Boolean value that signalizes whether the preview picture shows a bedroom.
picture_dining_room Boolean value that signalizes whether the preview picture shows a dining room.
picture_exterior Boolean value that signalizes whether the preview picture shows an exterior.
picture_interior Boolean value that signalizes whether the preview picture shows an interior.
picture_kitchen Boolean value that signalizes whether the preview picture shows a kitchen.
picture_living_room Boolean value that signalizes whether the preview picture shows a living room.
Boolean value that signalizes whether a home is equipped with amenities for families
family_amenities (e.g., baby bath, babysitter recommendations, children’s dinnerware).
This feature results from hierarchical clustering of amenities.
Boolean value that signalizes whether a home only comprises essential amenities
basic_amenities (e.g., WiFi, heating, smoke alarm). This feature results from hierarchical
clustering of amenities.
Boolean value that signalizes whether a home comprises more than only
normal_amenities
essential amenities. This feature results from hierarchical clustering of amenities.
Boolean value that signalizes whether a home is equipped with amenities that indicate
accessible_amenities accessibility (e.g., no stairs and steps, wide entrance, extra space around bed).
This feature results from hierarchical clustering of amenities.
entire_home Boolean value that indicates whether a listing is an entire home.
private_room Boolean value that indicates whether a listing is a private room.
hotel_room Boolean value that indicates whether a listing is a hotel room.
shared_room Boolean value that indicates whether a listing is a shared room.
Boolean value that indicates whether a listing is an alternative home
other_home
(e.g., houseboat, tree house, tiny house).
sum_foreign_languages Number of foreign languages supported by the host.
distance_to_center Distance from home to the city center, measured in kilometers.
city_population Population of the city that corresponds to a home.

31
Designing Explainable Predictive Machine Learning Artifacts

population_density Population density per square kilometer of the city that corresponds to a home.
beds_per_bedroom Ratio of beds per bedroom.
persons_per_bedroom Ratio of persons per bedroom.
persons_per_bathroom Ratio of persons per bathroom.

Appendix B: Interaction Effects

Here, we show a selection of conspicuous interaction effects from the XGBoost model using dependence plots. These
are scatter plots that show values of a feature on the x-axis and the corresponding SHAP values on the y-axis. Moreover,
the plotted dots are colored according to the values of an interacting feature.
Figure B1 depicts two plots, each showing the SHAP values of corresponding numbers of persons per bathroom. While
the left plot colors dots according to the values of the binary feature “entire_home”, the right colors these according
to the values of “private_room”. The figure unveils that negative SHAP values of “persons_per_bathroom” highly
interact with private rooms, whereas entire homes heavily interact with SHAP values that are either positive or close
to zero. This indicates that entire homes exert a positive impact on prices per night, while private rooms tend to be
less expensive. Although this effect might seem trivial, it is highly suited to demonstrate the appearance of feature
interaction in dependence plots.

Figure B1: Interaction effects between persons per bathroom and home types

Figure B2 depicts interaction effects between “population_density” and (i) whether a home’s listing uses the amenity
“heating” as a preview as well as (ii) whether the listing uses the amenity “free parking” as a preview. The figure’s
left plot unveils homes in cities with a high population density to use heating more often as a preview compared to
less densely populated cities. However, this does not mean that homes with heating as a preview amenity are more
expensive because of heating—it is also conceivable that the increased SHAP values are due to rental prices that tend to
be higher in bigger cities. It is also unclear if the value of “preview_heating” is actually a result of higher population
density or vice versa, as it might likewise be the case that other characteristics of the corresponding cities cause the
values of both features. Moreover, the right plot of Figure B2 shows that free parking is not used as a preview amenity
in cities with a particularly high population density. This effect is rather unsurprising, as we expect parking spots in
highly populated cities to be rarer. Yet, having only the information from the figure’s right plot, we cannot confirm free
parking to increase the SHAP values of low “population_density” values considerably.

32
Designing Explainable Predictive Machine Learning Artifacts

Figure B2: Interaction effects between population density and preview amenities

Figure B3 depicts interaction effects of the binary variables “picture_exterior” (i.e., whether the preview picture shows
an exterior) and “picture_bedroom” (i.e., whether the preview picture shows a bedroom). Here, the left plot unveils
that for homes with an exterior picture, higher distances to city centers tend to contribute to an increase in a home’s
SHAP values and vice versa. This might be due to the fact that hosts use attractive surroundings of secluded homes
as preview pictures to compensate for their high distances to city centers—or, these homes are deliberately far from
cities, because they aim to address guests who seek to stay in a place less urban. The plot in the middle of the figure
further shows that, for these homes with an exterior picture, high population tends to increase prices. This leads to the
following assumption: Homes with an attractive surrounding at the edge of a particularly large city seem to be more
expensive. Finally, the right plot shows that, for homes with a bedroom as preview picture, higher population values
interact with lower SHAP values and vice versa.

Figure B3: Interaction effects between preview pictures and location-based features

Figure B4 depicts interactions of canceling policies. Interestingly, for homes with a flexible canceling strategy, more
bedrooms tend to decrease SHAP values. Moreover, while the use of a pool as a preview amenity seems to increase
SHAP values of homes with a moderate canceling strategy, we observe the opposite effect for homes with a strict
canceling policy of 14 days.

Figure B4: Interaction effects of canceling policies

33