HackerRank Real-Time Evaluation Test
Uploaded by
Aamir JadoonHackerRank Real-Time Evaluation Test
Uploaded by
Aamir Jadoon14/10/2021, 22:13 HackerRank
Data Scientist - Intact Financial Corporation Aamir Khan
57 mins Answered: 1 / 14
5. A/B testing II
ALL
We perform an online experiment to test a new feature: half of our clients receive the
new feature, while the other half keeps the same feature and we measure how much
time they spend on our website.
1 We use p < 0.05 to decide to implement our feature. Our backend tool monitors the
experiments and sends an email every day to the experimenter with the p-value, and
2 the ability to continue or conclude the experiment. Most of our experimenters stop
the experiment when they reach significant difference between features (p < 0.05).
What would you conclude from our process?
3
Pick ONE OR MORE options
4
You should change your p-value threshold.
5
You could conclude the experiment automatically as
6 soon as the p-value goes below 0.05.
7 The p-values from the experiments are probably higher
than estimated.
8
The proportion of clients exposed to new features
should be increased.
9
The proportion of clients exposed to new features
should be decreased.
10
Clear Selection
11
12
[Link] 1/1