Scribd
Upload
7 views2 pages

Data Analysis and Experiment Design Guide

The document outlines an exercise for a Data Science course focusing on data analysis and experimental workflows using the MovieLens 100K Dataset. Students are required to perform univariate and bivariate analyses, address ethical issues, and formulate testable hypotheses for controlled machine learning experiments. The exercise examination is scheduled for November 8th, 2024, with a retake option on November 22nd, and students can bring a one-page document summarizing their analyses to the exam.

Uploaded by

raquel.panapalen
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
7 views2 pages

Data Analysis and Experiment Design Guide

The document outlines an exercise for a Data Science course focusing on data analysis and experimental workflows using the MovieLens 100K Dataset. Students are required to perform univariate and bivariate analyses, address ethical issues, and formulate testable hypotheses for controlled machine learning experiments. The exercise examination is scheduled for November 8th, 2024, with a retake option on November 22nd, and students can bring a one-page document summarizing their analyses to the exam.

Uploaded by

raquel.panapalen
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Experiment Design for Data Science – WS 2024/25

Exercise 1: Data analysis and experimental workflows

Exercise examination on Fri, November 8th, 2024 (alternative/retake: Fri, Nov 22nd)

I. Data analysis

Explore data collected by the MovieLens project, namely the MovieLens 100K Dataset
(ml-100k). Download the dataset at [Link] from
section “older datasets” and familiarize yourself with the contents. (Hint: the files
[Link], [Link], and [Link] contain the particularly relevant data.)

Explore the data with a tool of your choice (R, Python, Julia, Matlab, Excel, …,
whichever you are most comfortable with).

Univariate Analysis
• For numerical variables, choose appropriate statistical moments to compute
(e.g., mean, median, standard deviation. Also, visualise the variables – how are
the values distributed?
• For categorical variables, tabulate or visualise the frequency of the different
categories. Are there imbalances?

Bivariate Analysis
• For numerical variables, determine correlations between variables.
• For categorical variables, create contingency tables.
• Choose combinations of a numerical and a categorical variable and visualise the
distributions of numerical values in different groups. Choose appropriate
statistical moments to describe them.

Ethical issues
• Which ethical issues do you see with the dataset?

Make sure you understand the dataset and its variables well. You will be asked multiple
choice questions on the dataset and the above-mentioned insights at the exercise
examination. You may bring a one-page A4 document (double sided) containing your
analyses, visualizations and any other information you deem relevant for understanding
and describing the data with you to the exercise exam. Make sure this document
contains your name and student ID number as you will have to hand in this document
at the end of the exercise exam.

II. Hypotheses and Experimental Workflows

From the insights you gained, formulate hypotheses that can be tested given the data
in controlled machine learning experiments and outline experimental designs that are
suitable to investigate these hypotheses. Specifically, for each hypothesis, be very
clear about the following:
• Clearly indicate what the independent and dependent variable is
• Describe which control conditions you foresee
• Which performance indicator (or which type of performance criteria) you deem
effective to compare the different conditions
• Consider the scale type of the dependent variable and identify a suitable
strategy to build a prediction model, i.e., what type of method/algorithm is
appropriate to model data of this type
• How would you design the experiment to simulate real-world performance
conditions? (Data preparation, data splitting strategy, etc.)

Consider diverse hypotheses that exhibit different performance criteria, different


strategies for building prediction models, and different training/testing scenarios,
choosing alternative target variables and/or subsets of attributes. Consider how a
regression task could be represented, how a classification task could be represented,
and also consider which hypotheses could not be investigated given the data at hand.

Your hypotheses can be discussed at the beginning of the next lectures.

You might also like