Data analysis processes:
Ask -> Prepare -> Process-> Analyze -> Share -> Act
Data - collection of facts and informations. through analysis it can be used to draw conclusions,
make decisions and predictions.
Data analysis - the collection, transformation, and organization of data in order to draw
conclusions, make predictions, and drive informed decision-making.
The Role of Data Analysts in Business:
-Companies need analysts to manage large volumes of data and extract useful information from
it.
-Analysts help transform data into insights that can lead to new products or improved customer
service.
The Importance of Data Analytics:
Insights from data can change the direction of a business and lead to significant improvements.
Data ecosystems - the various elements that interact with one another in order to produce,
manage, store, organize, analyze, and share data.
Cloud - a place to keep data online, rather than a computer hard drive.
Data science - creating new ways of modeling and understanding the unknown by using raw
data.
Data analytics is the science of data.
Data-driven decision-making - using facts to guide business strategy.
Dataset - a collection of data that can be manipulated or analyzed as one unit
Analytical skills - qualities and characteristics associated with solving problems using facts.
5 essential points of analytical skills:
1) curiosity (desire to learn something)
2) understanding context (Context - the condition in which something exists or happens)
3) having a technical mindset (the ability to break things down into smaller steps or pieces and
work with them in an orderly and logical way)
4) data design (how you organize information)
5) data strategy (the management of the people, processes and tools used in data analysis)
Analytical thinking - identifying and defining a problem and them solving it by using data in an
organized step-by-step manner.
5 key aspects to analytical thinking:
1. Visualization (the graphical representation of information)
2. Strategy
3. Problem-orientation
4. Correlation
5. Big-picture and detail-oriented thinking
Root cause - the reason why a problem occurs.
Gap analysis - a method for examining and evaluating how a process works currently in order
to get where you want to be in the future.
6 stages of the data life cycle:
1. plan (decide what kind of data is needed, how it will be managed, and who will be
responsible for it)
2. capture (collect or bring in data from a variety of different sources)
3. manage (care for and maintain the data. This includes determining how and where it is
stored and the tools used to do so)
4. analyze (use the data to solve problems, make decisions, and support business goals)
5. archive (keep relevant data stored for long-term and future reference)
6. destroy (remove data from storage and delete any shared copies of the data)
Stakeholders - people who have invested time and resources into a project and are interested in
the outcome.
At the start of any successful data analysis, the data analyst:
Takes the time to fully understand stakeholder expectations
Defines the problem to be solved
Decides which questions to answer in order to solve the problem
The process phase:
Cleaning data
Transforming data into a more useful format
Combining two or more datasets to make information more complete
Removing outliers (data points that could skew the information)
Function - a preset command that automatically performs a specific process or task using the
data in a spreadsheet.
Formula - a set of instructions used to perform a calculation using the data in a spreadsheet
Query language - a computer programming language that allows you to retrieve and manipulate
data from a database.
Spreadsheets
Data analysts rely on spreadsheets to collect and organize data.
Spreadsheets structure data in a meaningful way by letting you
Collect, store, organize, and sort information
Identify patterns and piece the data together in a way that works for each specific data project
Create excellent data visualizations, like graphs and charts.
Databases and query languages
A database is a collection of structured data stored in a computer system. Some popular
Structured Query Language (SQL) programs include MySQL, Microsoft SQL Server, and
BigQuery.
Database - a collection of data stored in a computer system.
Query languages
Allow analysts to isolate specific information from a database(s)
Make it easier for you to learn and understand the requests made to databases
Allow analysts to select, create, add, or download data from a database for analysis
Visualization tools
Data analysts use a number of visualization tools, like graphs, maps, tables, charts, and more.
Two popular visualization tools are Tableau and Looker.
These tools
Turn complex numbers into a story that people can understand
Help stakeholders come up with conclusions that lead to informed decisions and effective
business strategies
Have multiple features
- Tableau's simple drag-and-drop feature lets users create interactive graphs in dashboards and
worksheets
- Looker communicates directly with a database, allowing you to connect your data right to
the visual tool you choose
Attribute - a characteristic or quality of data used to label a column in a table.
Observation - all of the attributes for something contained in a row of a data table.
The syntax of every SQL query is the same:
Use SELECT to choose the columns you want to return (specifies the columns from which to
retrieve data)
Use FROM to choose the tables where the columns you want are located (specifies the table
from which to retrieve data)
Use WHERE to filter for certain information (Specifies criteria that the data must meet)
!!!The dataset name is always followed by a dot, and then the table name!!!
You can conclude that the LIKE clause is very powerful because it allows you to tell the database
to look for a certain pattern! The percent sign % is used as a wildcard to match one or more
characters. In the example above, both Chavez and Chen would be returned. Note that in some
databases an asterisk * is used as the wildcard instead of a percent sign %.
Comments
Some tables aren’t designed with descriptive enough naming conventions. In the example, field1
was the column for a customer’s last name, but you wouldn’t know it by the name. A better name
would have been something such as last_name. In these cases, you can place comments
alongside your SQL to help you remember what the name represents. Comments are text placed
between certain characters, /* and */, or after two dashes --) as shown below.
SELECT
field1 /* this is the last name column */
FROM
table -- this is the customer data table
WHERE
field1 LIKE 'Ch%';
-- This is an important query used later to join with the accounts table
SELECT
rowkey, -- key used to join with account_id
[Link], -- date is in string format YYYY-MM-DD HH:MM:SS
[Link] -- e.g., 'pub-###'
FROM Publishers
-- Pull basic information from the customer table
SELECT
customer_id, --main ID used to join with customer_addresss
first_name, --customer's first name from loyalty program
last_name --customer's last name
FROM
customer_data.customer_name
Example of a query with aliases
SELECT
my_table_alias.actual_column_name AS my_column_alias
FROM
actual_table_name AS my_table_alias
Imagine you are a data analyst for a small business and your manager asks you for some
employee data. You decide to write a query with SQL to get what you need from the database.
You want to pull all the columns: empID, firstName, lastName, jobCode, and salary. Because
you know the database isn’t that big, instead of entering each column name in the SELECT
clause, you use SELECT *. This will select all the columns from the Employee table in the
FROM clause.
SELECT
*
FROM
Employee
Suppose you notice a large salary range for the 'SFI' job code. You might like to flag all
employees in all departments with lower salaries for your manager. Because interns are also
included in the table and they have salaries less than $30,000, you want to make sure your results
give you only the full time employees with salaries that are $30,000 or less. In other words, you
want to exclude interns with the 'INT' job code who also earn less than $30,000. The AND clause
enables you to test for both conditions.
You create a SQL query similar to below, where <> means "does not equal":
SELECT
*
FROM
Employee
WHERE
jobCode <> 'INT'
AND salary <= 30000;
Business task - the question or problem data analysis answers for a business.
Fairness - ensuring that your analysis doesn't create or reinforce bias
Following are some strategies that support fair analysis:
Consider all of the available data
Identify surrounding factors
Include self-reported data
Use oversampling effectively
Think about fairness from beginning to end
Self-reporting is a data collection technique where participants provide information about
themselves
Oversampling is the process of increasing the sample size of nondominant groups in a
population.