Play Video
Welcome to Data Crunchers. We're so glad that you accepted our internship position. You'll
find this opportunity fast-paced and exciting. We are a freelance consulting firm that provides
data analytics services to many different types of organizations. As an intern, you will get to
work with our team of data analysts, engineers, and scientists, as we help our clients to make
sense of their data, find trends, and solve problems. Our team is working on some challenging
projects right now in the fields of retail, climate science, and agriculture. So let's get you started
working with our data analysts. Data analysts must be familiar with what data is, how it is
collected, and how to use visualizations to present the data in an understandable and useful
form.
WHAT IS DATA AND WHERE DOES IT COME FROM?
Through your daily activities, you generate a great amount of data that can be volunteered,
observed, or inferred. You volunteer data by filling out forms, posting photos and videos on
social media, or consenting to website policies. Organizations also collect data about you
through observations such as your location, stores you frequent, and websites you visit. Your
personal style, likes and dislikes, and behavioral habits can even be inferred from your actions.
By itself all this data can be rather meaningless; however, that changes immediately once you
apply data analytics to your data. The process of taking data from its raw state and using it to
inform useful actions is the realm of data analytics. Through the power of analytics,
organizations can use your volunteered, observe, or inferred data to provide you with a
personalized experience.
DATA USAGE IN DAILY LIFE
The collection and analysis of data is making information readily available for many different
purposes. We are more connected than ever. In our home, schools, work, and even the areas in
which we play, advancements in technologies are generating large quantities of data.
Everywhere you go and everything you do in this digital world becomes a new source of data.
Data is being generated from sensors, devices, video, audio, networks, log files, transactional
applications, the web, and social media. It is more commonly streaming over the networks.
THE IMPORTANCE OF VISUALIZATION
The data analysts at Data Crunchers work hard to create stories and reports from their clients’
data. Analysts rely heavily on visualization techniques to make the information relevant and
actionable. Visualizations are a way to display data in a manner that is easily understood. The
most common ways to represent data visually are charts and graphs.
There are many types of charts used to present information graphically. It is crucial to ensure
that the type of visualization used best illustrates the pattern, problem, or trend that needs
highlighting; not all types of visualizations are appropriate for all kinds of data.
Factors to consider when choosing a visualization:
The number of variables, which are the characteristics measured, that need are shown
The number of data points, or units of information, in each variable
Whether the data illustrates changes over time (hourly, daily, weekly)
The need to make a comparison or correlation between different data points
Often, analysts try a few different types of charts to choose the one that provides the most
effective view of the information.
WAYS TO VISUALIZE DATA
Line Charts
Line charts are one of the most commonly used charts for comparing two data sets. Use line
charts when the number of data points is high, and you want to show a trend in the data over
time.
Use cases for line charts:
A company’s quarterly sales for the past five years.
The number of customers per week in the first year of a new retail shop.
Changes in a stock’s price from opening to the closing bell.
Best practices for line charts:
Label the axes and the reference lines used to measure the graph coordinates. It is
common to plot time on the x-axis (horizontal) and the data values on the y-axis
(vertical).
Use a solid line to connect the data points to illustrate trends.
Keep the number of plotted lines to a minimum, typically no more than 5, so the chart
does not become cluttered and difficult to read.
Add a legend, a small visual representation of the chart’s data, that tells what each line
represents to help your audience understand what they are viewing.
Always add a title.
Column Charts
Column charts are positioned vertically, as shown in the figure. They are probably the most
common chart type used to display the numerical value of a specific data point and compare
that value across similar categories. They allow for easy comparison among several data
points.
Use cases for column charts:
Revenue by country, as shown in the chart example.
Last year’s sales for the top four car companies in the US.
Average student test scores for each of six math classes.
Best practices for column charts:
Label the axes.
If the chart shows changes over time, plot the time increments on the x-axis.
If time is not part of the data, consider ordering the column heights to ascend or
descend to demonstrate changes or trends.
Keep the number of columns low, typically no more than 7, so the viewer can see the
value for each column.
Start the value of the y-axis at zero to accurately reflect the column's total value.
The spacing between columns should be roughly half the width of a column.
Bar Charts
Bar charts are similar to column charts, except the data is horizontally displayed. Bar charts
also allow for easy comparison between several data points. The data point labels on the
horizontal bar chart are on the left side and are more readable when the label contains text rather
than values.
Use cases for bar charts:
Gross domestic product (GDP) of the 25 highest-grossing nations.
The number of cars at a dealership sold by each sales representative.
Exam scores for each student in a math class.
Best practices for bar charts:
Label the axes.
Consider ordering the bars so that the lengths go from longest to shortest. The data
type will most likely determine whether the longest bar should be on the bottom or the
top to best illustrate the intended pattern or trend.
Start the value of the x-axis at zero to accurately reflect the total value of the bars.
The spacing between bars should be roughly half the width of a bar.
Pie Charts
Pie charts show parts of a whole. Each slice, or segment, of the “pie”, represents a percentage
of the total number. The total sum of the segments must equal 100%. A pie chart displays the
different values of a given variable. Some use cases that illustrate comparing the information
with a pie chart include:
Annual expenses categories for a corporation (e.g., rent, administrative, utilities,
production)
A country’s energy sources (e.g., oil, coal, gas, solar, wind)
Survey results for favorite type of movie (e.g., action, romance, comedy, drama,
science fiction)
Some best practices for pie charts include:
Keep the number of categories minimal so the viewer can differentiate between
segments. Beyond ten segments, the slices begin to lose meaning and impact. If
necessary, consolidate smaller segments into one segment with a label such as “Other”
or “Miscellaneous”.
Use a different color or darkness of grayscale for each segment.
Order the segments according to size.
Make sure the value of all segments equals 100%.
Scatter Plots
Scatter plots are very popular for correlation visualizations or when you want to show the
distribution, or all possible values, of a large number of data points. Scatter plots are also useful
for demonstrating clustering or identifying outliers in the data. Some use cases that illustrate
visualizing the distribution of many data points with a scatter plot include:
Comparing countries’ life expectancies to their GDPs (Gross Domestic Product).
Comparing the daily sales of ice cream to the average outside temperature across
multiple days.
Comparing the weight to the height of each person in a large group.
Some best practices for scatter plots include:
Label your axes.
Make sure the data set is large enough to provide visualization for clustering or
outliers.
Start the value of the y-axis at zero to represent the data accurately. The value of the x-
axis will depend on the data. For example, age ranges might be labeled on the x-axis.
Consider adding a trend line if a scatter plot shows a correlation between x- and y-
axes.
Do not use more than two trend lines.
DATA IS ALL AROUND US
Discrete vs Continuous Data
Data analysis begins with understanding the types of data you're working with. Data can first
be defined as quantitative or qualitative. Qualitative data is descriptive. It is typically
represented as text, is non-numerical in nature, and is collected through surveys, questionnaires,
interviews, or observations. An example of qualitative data is the reviews that are collected
about various products on a shopping site. Quantitative data is numerical in nature. You are
trying to quantify information.
Quantitative data can further be broken down into discrete and continuous data.
Discrete data can only take on certain values, typically whole numbers. Examples of discrete
data include account of transactions processed in a day or the number of people who visit a
website each week.
Continuous data is measured and can be represented in an infinite number of ways. An
example of continuous data could be the average distance traveled by a moving car, which is
constantly being updated as the car moves forward.
Understanding quantitative versus qualitative data is an initial step, but strong familiarity with
all types of data is necessary if you're going to be a versatile analyst.
DATA TYPES
All pieces of data have a feature called data type that tells applications how to treat
it. Operations performed are defined by the data type. Identifying data types is helpful in
analytics because we may need to group the data, sort the data, or run calculations on the data.
Data must be grouped by their given type to perform required operations. For example, consider
the following Data Crunchers Employee Information table.
Each column (or field) in the Employee Information table has data that is all the same type.
The next table describes the various data types. Read the descriptions for each data type.
Compare them to the data displayed in the Employee Information table and think about which
type each field is (For example, Job Title contains string data).
VARIETY OF DATA
Data is all around us, and it is coming at ever-increasing rates and in many different formats.
For example, when mobile phones first became popular, they had only a few megabytes of data
storage and were limited to the variety of data they could store. These early mobile phones,
used primarily for phone calls, could keep text messages, a couple of MP3 music files, and a
few low-resolution photos. Modern-day mobile phones can come with up to a terabyte (1000
gigabytes) of storage. They can store a growing variety of data types, including high-definition
video files, photographs, audio files, documents, apps, books, calendars, and emails, to name a
few.
Structured vs Unstructured Data
In order to process, store, and analyze all of these different types of data, it is important to think
about whether they are structured data or unstructured data.
Structured Data
Structured data makes up about 10%-20% of generated data and has clearly defined data types
and patterns that make them easily stored and organized into columns and rows. This
organization makes structured data easy to search and analyze. Sources of structured data
include sales records, airline reservation systems, and inventory control. Structured data is
usually stored in relational databases such as Structured Query Language (SQL) databases or
in spreadsheets such as Microsoft Excel.
Unstructured Data
Unstructured data makes up most data that is generated, about 80%, and cannot be organized
into row and columns. This makes unstructured data difficult to search, manage, and analyze.
Sources of unstructured data include images, PDFs, sensor data, and social media posts.
Unstructured data is usually stored in a non-relational database also known as NoSQL Database.
Selecting Relevant Data
Selecting relevant data starts with defining the question that you want to answer. For example,
suppose you are researching the factors that contribute to the overall wellness of the workforce.
You can start by looking at the available data and determining which fields or measurements
are relevant to your question. In the Employee Information table, repeated
below, Employment Date has no impact on employee wellness and therefore is irrelevant to the
analysis. In contrast, overwork could impact employee wellness, and therefore, Average
Weekly Hours could be relevant to the research.
Video - Data Analytics for Business Insights
Our connected world is complex. As you learned, this complexity generates an ever-increasing
amount of data, which is now available at our fingertips.
Data science enables businesses to better understand the impact of their products and services,
adjust their methods and goals and provide the customers with better products faster. The
insights gained from data analysis can make a difference to the success of the business, as well
as to the lives of their employees. There are multiple types of analysis that provide businesses
with information that can drive innovation, improve efficiency and mitigate risk. The type of
analytics to implement will depend on the problem that needs to be solve or questions that need
to be answered. Trend analysis is one way to gain insights into key performance indicators,
KPI, over time. One of our clients at Data Crunchers is a bicycle store that is having difficulty
with attracting new customers. We can use analysis of the types of bicycles or services that their
previous new customers purchased and during what time of the year they made their purchases.
This analysis identifies trends in these purchases, such as many new customers are buying
certain types of bikes at a specific time of the year. The store uses this information to purchase
advertising during those time periods to target new buyers. Afterwards, store management can
analyze sales data to determine the effectiveness of the advertising. Analytics can also be used
to predict customer behaviors, gauge user satisfaction and even to identify patterns that can be
used to train machines to perform complex functions