CISCO
Chapter 3: Everything
Generates Data
Introduction to the Internet of Things v2.0
• 3.1 Big Data
• Explain the concept of Big Data.
• Describe the sources of Big Data.
• Explain the challenges and solutions to Big Data storage.
• Explain how Big Data analytics are used to support Business.
• 'Data is information that comes from a variety of
sources, such as people, pictures, text, sensors, web
sites and technology devices.
• Three characteristics that indicate an organization may
be dealing with Big Data:
• Volume - A large amount of data that increasingly
requires more storage space.
• Velocity - An amount of data that is growing
exponentially fast.
• Variety - Data that is generated in different formats.
• Examples of data amounts collected by sensors:
• One autonomous car can generate 4,000
• gigabits (Gb) of data per day.
• One smart connected home can produce as much as 1
gigabyte (GB) of information a week.
• Companies do not necessarily have to
generate their own Big Data.
• "There are sources of free data sets
available, ready to be used and
analyzed.
• IBM's Big Data estimates conclude that “each
day we create 2.5 quintillion bytes of data"
• Five major storage problems with Big Data:
• Management
• Security
• Redundancy
• Analytics
• Access
• Big data is typically stored on multiple servers, in
data centers.
• Fog computing utilizes end-user clients or "edge"
devices to do a substantial amount of the pre-
processing and storage.
• Data from that pre-processed analysis can be fed
back into the companies' systems to modify
processes if required.
• Communications to and from the servers and
devices is quicker and requires less bandwidth
than constantly going out to the cloud.
• The cloud is a collection of data centers or groups of connected servers.
• Cloud services for individuals include:
• Storage of data, such as pictures, music, movies, and emails.
• Access many applications instead of downloading onto local device.
• Access data and applications anywhere, anytime, and on any device.
• Cloud Services for an Enterprise include:
• Access to organizational data anywhere and at any time.
• Streamlines the IT operations of an organization.
• Eliminates or reduces the need for onsite IT equipment, maintenance, and management.
• Reduces cost for equipment, energy, physical plant requirements, and personnel training
needs.
• Distributed data processing takes the large volume of data and
breaks it into smaller pieces.
• These smaller pieces are distributed in many locations to be
processed by many computers.
• Each computer in the distributed architecture analyzes its part of the
Big Data picture (horizontal scaling).
• Hadoop was created to deal with these Big Data volumes. It has two
main features that has made it the industry standard:
• Scalability- Larger cluster sizes improve performance and
provide higher data processing capabilities.
• Fault tolerance - Hadoop automatically replicates data across
clusters.
• Data analytics allows businesses to better understand the
impact of their products and services, adjust their methods
and goals, and provide their customers with better products
faster.
• " Value comes from two primary types of processed data,
transactional and analytical.
• Transactional information is captured and processed as events
happen.
• Used to analyze daily sales reports and production
schedules to determine how much inventory to carry.
• Analytical information supports managerial analysis tasks like
determining whether the organization should build a new
manufacturing plant.
• Data originates from sensors and anything that has been
scanned, entered, and released to the Internet.
• Collected data can be categorized as structured or
unstructured.
• Structured data is created by applications that use "fixed"
format input such as spreadsheets. May need to be manipulated
into a common format such as CSV.
• Unstructured data is generated in a "freeform" style such as
audio, video, web pages, and tweets.
• " Examples of tools to prepare unstructured data for processing
are:
• "Web scraping" tools automatically extract data from HTML
pages.
• RESTful application program interfaces (APls).
• Data mining is the process of tuming raw data
into meaningful information.
• The mined data must be analyzed and presented
to managers and decision makers.
• Determining the best visualization tools to use
will vary based on the following:
• Number of variables
• Number of data points in each variable
• Is the data representing a timeline
• Items require comparisons
• Popular charts include line, column, bar, pie, and
scatter.
• Data analysis is the process of inspecting,
cleaning, transforming, and mode ling data to
uncover useful information.
• Having a strategy helps a business determine the
type of analysis required and the besttool to do
the analysis.
• Tools and applications range from using an Excel
spreadsheet or Google Analytics for small to
medium data samples, to the applications
dedicated to manipulating and analyzing really big
datasets.
• Examples include Knime, OpenRefine, Orange, and
RapidMiner.