NPTEL NOTES(Data Science)
WEEK1
Week 1: Introduction to R & Programming Fundamentals
This week establishes the foundational knowledge of R, the primary
programming language for the course. The topics progress from setting up
the environment to understanding basic data structures and control flow.
1. The R Environment: R and RStudio
R as a Language: R is an open-source programming language
widely used as a statistical software and data analysis tool. It
typically uses a command-line interface.
RStudio as an IDE: For this course, you will use RStudio, an
Integrated Development Environment (IDE) that provides a user-
friendly graphical interface (GUI). RStudio's interface is divided into
several panes:
o Console Panel: This is where you can type commands and
see the output.
o Environment/History Pane: The Environment tab shows the
variables created in your temporary workspace. The History
tab lists all the commands you have used.
o Files/Plots/Packages/Help Pane: This pane contains tabs
for navigating files, viewing plots, managing installed
packages, and accessing R documentation for functions.
Setting Up and Running Code:
o Working Directory: You can set the working directory from
the GUI using the "More" button in the Files tab or by using the
setwd() command in the console.
o Creating Files: An R script file can be created by navigating
to File > New File > R Script or by clicking the + button below
the File tab.
o Executing Code:
The Run button or Ctrl + Enter executes a single line of
code or a selected block.
The Source command executes the entire R file and
prints only the final output.
The Source with Echo command executes the entire file
and prints both the commands and their outputs, which
can be useful for debugging.
2. Variables and Data Types
Variable Naming Rules:
o Must be alphanumeric characters.
o Can include the special characters underscore (_) or period (.).
o Must always start with an alphabet.
o Invalid Variable Names: The assignments test this rule.
Names like 1_variable, _variable, or variable@ are invalid
because they either start with a number, a special character,
or contain an illegal special character (@).
Basic Data Types: R has five primary data types :
o Logical: TRUE or FALSE
o Integer: Whole numbers
o Numeric: Real numbers
o Complex: Complex numbers
o Character: Alphabets and special characters
Data Type Functions:
o typeof(): Returns the data type of an object.
o is.data_type(): Verifies if an object is of a specific data type
(e.g., [Link](a)).
o as.data_type(): Coerces an object from one data type to
another, though not all coercions are possible.
3. R Objects: Vectors, Lists, and Data Frames
Vectors: An ordered collection of elements that must all be of the
same data type. You create a vector using the concatenation
function
c().
Lists: An ordered collection of objects that can be of different
data types. Lists can contain other lists, vectors, matrices, or
functions.
o Accessing Elements:
By index: The double slicing operator [[...]] is used to
access top-level components of a list (e.g., list[[1]]). To
access an element within a component, you combine
operators like list[[component_index]]
[inner_element_index].
By name: Use the $ operator followed by the
component's name (e.g., [Link]$names).
Data Frames: The most common data structure for data science,
used for tabular data where each column can have a different data
type.
o Accessing and Editing: Use the format df[row, column].
Rows and columns can be specified by number or an array of
values, such as 1:2 or c(1,3).
o Adding/Deleting Elements: New rows can be added with
rbind(), and new columns with cbind(). Rows or columns can
be deleted by specifying their negative index (e.g.,
df[-3, ] excludes the third row).
o Importing Data: You can create a data frame by importing
data from a file using [Link](), specifying the file path and
a separator if needed.
o Recasting and Joining: The course materials introduce
recasting for reshaping data and joining for combining data
frames, using functions like left_join and right_join.
Matrix Operations: Assignments may test basic matrix operations
like subsetting and finding diagonal elements. The command a[-3, ]
excludes the third row of a matrix a, and diag(a) extracts the
diagonal elements.
4. Control Structures and Advanced Concepts
Control Structures: Week 1 covers the syntax and logic of if-else
and loop constructs. Assignments often test the ability to trace the
output of a code snippet that uses these structures.
o if-else logic: Conditions are checked sequentially. If an if or
else if condition is met, the corresponding code block is
executed, and the rest of the chain is ignored.
o for loops: The structure is for (iter in sequence)
{ statements }.
sequence can be a vector or a list.
Comments and Workspace Management:
o Comments: Use # to add comments to a single line or Ctrl +
Shift + C to comment/uncomment multiple lines.
o Clearing Console/Environment: Use Ctrl + L to clear the
console without deleting variables. Use the
rm() command to remove variables from the environment (rm(variable)
for a single variable, rm(list=ls()) for all variables).
Saving/Loading: The workspace is temporary. You can save it manually
o with save() or [Link]() and load it later with the load()
command.
Sources and related
Week 2 :-
Week 2 introduces the mathematical foundation of linear algebra, which is
crucial for understanding later data science algorithms. The course divides
this topic into two main parts: an algebraic perspective and a geometric
perspective.
Key Concepts
The lectures for this week aim to provide a conceptual understanding of
linear algebra as it applies to data science, rather than focusing on
complex proofs. The recommended reference book for this week is
INTRODUCTION TO LINEAR ALGEBRA - BY GILBERT STRANG.
1. Algebraic View
This section covers the core computational concepts of linear algebra.
Solving Linear Equations: The course frames linear equations in
the general form Ax = b.
Rank and Null Space: The concepts of a matrix's rank and null
space are introduced as important for solving systems of linear
equations.
Pseudo-inverse: A key takeaway is the concept of a pseudo-
inverse, which is a unifying method to find a solution for all types of
linear equation systems, including those that are rectangular (non-
square), over-determined (m > n), or under-determined (m < n). The
pseudo-inverse is a critical concept in machine learning because it
allows for the inversion of non-square matrices to solve for unknown
variables.
2. Geometric View
This part of the course provides an intuitive, spatial understanding of
linear algebra concepts.
Vectors: A vector X with variables x1 and x2 can be interpreted as
either a point in a 2D space or a vector drawn from the origin to that
point. In higher dimensions (e.g., 3D), a vector represents a point in
that dimensional space.
Distance and Magnitude: The magnitude of a vector is its
Euclidean distance from the origin. The distance between two points
(vectors x1 and x2) can be calculated using the formula sqrt((x2 -
x1)^T * (x2 - x1)).
Unit Vectors: A unit vector is a vector with a magnitude of 1. Any
vector can be represented as its magnitude multiplied by its unit
vector.
Orthogonality: Two vectors A and B are orthogonal to each other if
their dot product A^T * B is equal to zero. The dot product A^T * B
is equivalent to |A| |B| cos(θ), where θ is the angle between the
vectors. Therefore, if the dot product is zero, the vectors are
perpendicular.
Hyperplanes and Half-spaces: A single linear equation n^T * X +
b = 0 represents a hyperplane in an N-dimensional space.
o In 2D, this equation represents a line.
o In 3D, it represents a plane.
o The vector n is the normal vector to the hyperplane,
meaning it is perpendicular to it.
o The value of the expression n^T * X + b can be used to
determine which "half-space" a given point X lies in.
If n^T * X + b > 0, the point lies on one side of the
hyperplane.
If n^T * X + b < 0, the point lies on the other side.
If n^T * X + b = 0, the point lies directly on the
hyperplane.
Projections: The course introduces the concept of projecting a
vector onto a surface (like a plane) as a way to find the best
approximation of the vector using a smaller set of basis vectors.
This idea is important for data reduction and noise reduction in data
science.
Eigenvalues and Eigenvectors: This topic is central to linear
algebra and is used heavily in many data science algorithms,
including Principal Component Analysis (PCA).
o The core equation is Ax = λx, where A is a matrix, x is an
eigenvector, and λ is an eigenvalue.
o For symmetric matrices (the primary focus in the lectures),
the eigenvectors corresponding to zero eigenvalues span the
null space of the matrix A.
o Eigenvectors corresponding to non-zero eigenvalues span the
column space of the matrix A.
Important Formulae
Euclidean Distance (Magnitude of a Vector X from the
origin): |X| = sqrt(x1^2 + x2^2 +... + xn^2)
Distance between Two Vectors x1 and x2: d = sqrt((x2 - x1)^T
* (x2 - x1))
Dot Product of Two Vectors A and B: A^T * B = sum(a_i * b_i)
Hyperplane Equation: n^T * X + b = 0
Eigenvalue Equation: Ax = λx
WEEK 3:-