Data Analytics - 2022-23
Dr. Vivek Kumar and Dr. Hitesh Singh
1. Important Dates
• Release date: 21/02/2023
• Submission deadline: 24/2/2023
2. Project overview and deliverables
The submission involves two files: a data analytic report (see Deliverable 1) and a
Knime/Jupyter notebook(see Deliverable 2).
• You should address a data-related problem in your professional field or a field you are
interested in (e.g., healthcare, sports, bioinformatics, gaming, finance, etc). If you are
motivated by the subject matter, the project will be more fun for you, and you will
likely produce a better report.
• Once you determine the area that interests you the most, you should search for a suitable
data set or collate the data set yourself (see Section 5 for possible data sources).
• You should apply a minimum of TWO data analytic techniques (i.e. machine learning
algorithms) of your choice to your data, from those covered in this course up to and
including. The aim is to learn two models and contrast their performance on your
input data. You are allowed to test more than TWO data analytic techniques if you
wish (e.g., using multiple techniques to learn a model, or learning more than two
models), but this is not a requirement and will not necessarily improve your mark.
Remember to use the page limit wisely against the marking criteria (see below). The
algorithms you can choose from are:
▪ Linear, non-linear and logistic regression,
▪ Support vector classification or
▪ regression,
▪ Decision trees,
▪ KNN,
▪ k-means, GMMs.
▪ ANN
▪ PCA
▪ LDA
▪ etc
4. Roadmap
The Project lasts for almost 3 Days. To ensure the Project runs smoothly, be careful not to
deviate much from the timetable below:
Step 1 : Determine your project area and data set.
What is the question you hope to answer? What data are you planning to use to answer
that question? What do you know about the data so far? Why did you choose this topic?
You may discover during your data exploration that you do not have the data
necessary to answer your project’s question. You may decide to change the research
question to address in the project. You should aim to finalise any changes as soon as
possible.
Our advice is to spend your time during the first wisely doing some research
on the data sources and the data analytic methods covered in the labs, depending on
the problem you are trying to address. Researching appropriate data sets and
determining what data analytic method to use is part of the Project. Various data
sources are provided for reference at the end of this document.
Step 2: Data processing.
What data have you gathered, and how did you gather it? What steps have you taken to
explore the data? Which areas of the data have you ‘cleaned’ (if any)? If your data may
not need cleaning, explain why. What insights have you gained from your exploration
(visualisations are optional here – consider page limit)? Will you be able to answer your
question with these data, or do you need to gather more data (or perhaps adjust the project
aims)? How might you use modelling to answer your question?
Step 3: Apply two data analytic methods and analyse results.
Which are the two data analytic methods you have selected and why? Is this a
supervised or an unsupervised learning problem? Is this a classification or a
regression problem? What do you understand about the two data analytic methods and
why are they appropriate in answering the project question/s? Apply them to your
data and start exploring the results. Generate plots that help explain the results.
Step 4: Have produced your first draft.
At a minimum, this should include a) literature review and background information
on your selected topic, b) narrative of what you have done so far, c) visualisations of
the results.
Step 5: Finalise draft.
Remember we have no lectures or labs on Day 7, so use that Day to complete the
Project.
5. Data sources
Using public data is the most common choice. If you have access to private data, that is also
an option, though you will have to be careful about what results you can release to us. Some
sources of publicly available data are listed below (you don`t have to use these sources).
• Kaggle
[Link]
Over 50,000 public data sets for machine learning.
• UK Covid Data
[Link]
Official UK COVID data
• [Link]
[Link]
This is the resource for most government-related data.
• Socrata
[Link]
Socrata is a good place to explore government-related data. Furthermore, it provides
some visualization tools for exploring data.
• UN3ta
[Link]
UN data is an Internet-based data service which brings UN statistical databases.
• European Union Open Data Portal
[Link]
This site provides a lot of data from European Union institutions.
• [Link]
[Link]
This site of the UK Government includes the British National Bibliography: metadata
on all UK books and publications since 1950.
• The CIA World Factbook
[Link]
This site of the Central Intelligence Agency provides a lot of information on history,
population, economy, government, infrastructure, and military of 267 countries.
• US Census Bureau
[Link]
This site provides information about US citizens covering population data, geographic
data, and education.
• Health Data
[Link]
[Link]
This site provides medical data about epidemiology and population statistics.
• NHS Health and Social Care Information Centre
[Link]
Health datasets from the UK National Health Service.
• Social Data Facebook Graph
[Link]
Facebook provides this API which allows you to query the huge amount of
information that users are sharing with the world.
• Topsy
[Link]
Topsy provides a searchable database of public tweets going back to 2006 as well as
several tools to analyze the conversations.
• Google Trends
[Link]
Statistics on search volume (as a proportion of total search) for any given term, since
2004.
• Likebutton
[Link]
Mines Facebook's public data--globally and from your own network--to give an
overview of what people "Like" at the moment.
• Amazon Web Services public datasets
[Link]
The public data sets on Amazon Web Services provide a centralized repository of
public data sets. An interesting dataset is the 1000 Genome Project, an attempt to
build the most comprehensive database of human genetic information. Also a NASA
database of satellite imagery of Earth is available.
• DBPedia
[Link]
Wikipedia contains millions of pieces of data, structured and unstructured, on every
subject. DBPedia is an ambitious project to catalogue and create a public, freely
distributable database allowing anyone to analyze this data.
• Freebase
[Link]
This community database provides information about several topics, with over 45
million entries.
• Gapminder
[Link]
This site provides data coming from the World Health Organization and World Bank
covering economic, medical, and social statistics from around the world.
• Google Finance
[Link]
Forty years' worth of stock market data, updated in real time.
• National Climatic Data Center
[Link]
Huge collection of environmental, meteorological, and climate data sets from the US
National Climatic Data Center. The world's largest archive of weather data.
• WeatherBase
[Link]
This site provides climate averages, forecasts, and current conditions for over 40,000
cities worldwide.
• Wunderground
[Link]
This site provides climatic data from satellites and weather stations, allowing you to
get all information about the temperature, wind, and other climatic measurements.
• Football datasets
[Link]
This site provides historical data for football matches around the world.
• Pro-Football-Reference
[Link]
This site provides data about football and several other sports.
• New York Times
[Link]
Searchable, indexed archive of news articles going back to 1851.
• Google Books Ngrams
[Link]
This source searches and analyses the full text of any of the millions of books
digitized as part of the Google Books project.
• Million Song Data Set
[Link]
Metadata on over a million songs and pieces of music. Part of Amazon Web Services.