Unit-1|Introduction to Data Science, Python and NumPy

UNIT 1
Introduction to Data Science, Python and NumPy
1. Introduction to Data Science
Data Science is an interdisciplinary field that uses scientific methods, algorithms, processes, and
systems to extract knowledge and actionable insights from structured and unstructured data. It
combines concepts from statistics, mathematics, computer science, and domain expertise to
analyze data and solve real-world problems.
The data science process typically involves several stages that take raw data and convert it into
meaningful insight that supports decision-making.

1.1 Key Stages of Data Science Workflow

• Data Collection: Gathering raw data from various sources such as databases, sensors, APIs,
web scraping, and surveys.
• Data Cleaning (Data Wrangling): Handling missing values, removing duplicates, correcting
inconsistent formats, and preparing data for analysis.
• Exploratory Data Analysis (EDA): Understanding data through summary statistics and
visualizations to discover patterns, trends, and anomalies.
• Modeling: Applying statistical or machine learning models to the cleaned data to make
predictions or classifications.
• Evaluation: Assessing model performance using appropriate metrics (accuracy, precision,
recall, RMSE, etc.).
• Deployment: Integrating the model into production systems so it can be used for real-world
decision-making.
• Communication: Presenting insights through reports, dashboards, and visualizations to
stakeholders.
1.2 Why Data Science is Important
Organizations today generate massive volumes of data every day. Data science enables businesses
to make data-driven decisions, predict future trends, automate processes, personalize customer
experience, detect fraud, and optimize operations. It is applied across domains such as healthcare,
finance, e-commerce, sports, government, and social media.

1.3 Roles Related to Data Science

• Data Scientist: Builds models and extracts insights using statistics and machine learning.
• Data Analyst: Focuses on interpreting data and creating reports/dashboards.

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

• Data Engineer: Builds and maintains the infrastructure and pipelines for data flow.
• Machine Learning Engineer: Focuses on building and deploying ML models at scale.

2. Why Python?
Python has become the most popular programming language for data science, machine learning,
and scientific computing. It was created by Guido van Rossum and first released in 1991. Python's
simplicity and readability make it an excellent choice for both beginners and expert programmers.

2.1 Reasons for Python's Popularity in Data Science

• Easy to Learn and Read: Python has a simple, English-like syntax that reduces the learning
curve and improves code readability.
• Rich Ecosystem of Libraries: Libraries such as NumPy, pandas, Matplotlib, scikit-learn,
TensorFlow, and PyTorch provide ready-to-use tools for numerical computing, data
analysis, visualization, and machine learning.
• Open Source and Free: Python and most of its data science libraries are free and open source,
encouraging community contribution and continuous improvement.
• Cross-Platform Compatibility: Python code runs on Windows, macOS, and Linux without
modification.
• Strong Community Support: A large global community contributes tutorials, documentation,
packages, and forums (Stack Overflow, GitHub) that make problem-solving easier.
• Integration Capability: Python integrates easily with other languages (C, C++, Java) and
technologies (SQL databases, Hadoop, Spark, web frameworks).
• Versatility: Python is used not only for data science but also for web development,
automation, scripting, game development, and more, making it a general-purpose language.
• Support for Multiple Programming Paradigms: Python supports procedural, object-oriented,
and functional programming styles.
2.2 Python vs Other Languages for Data Science
Compared to R, Python is more general-purpose and better suited for building production systems.
Compared to Java or C++, Python offers faster development time due to its simpler syntax and
dynamic typing, though it may be slower in raw execution speed. This trade-off is usually
acceptable because performance-critical operations in libraries like NumPy are implemented in
optimized C code under the hood.

3. Essential Python Libraries

The Python data science ecosystem consists of several core libraries, each serving a specific
purpose in the data analysis pipeline.
[Link] ,DEPT OF CSE ,ANU
 Unit-1|Introduction to Data Science, Python and NumPy

3.1 NumPy (Numerical Python)

NumPy is the foundational package for numerical computation in Python. It provides support for
large multi-dimensional arrays and matrices, along with a large collection of high-level
mathematical functions to operate on these arrays efficiently.

3.2 pandas

pandas provides fast, flexible, and expressive data structures such as Series (1D) and DataFrame
(2D) designed to make working with structured (tabular) data simple and intuitive. It offers
powerful tools for reading/writing data, handling missing data, merging, reshaping, and grouping.

3.3 Matplotlib

Matplotlib is the primary library used for creating static, animated, and interactive visualizations
in Python, such as line plots, bar charts, histograms, and scatter plots.

3.4 Seaborn

Built on top of Matplotlib, Seaborn provides a high-level interface for drawing attractive and
informative statistical graphics with less code.

3.5 SciPy

SciPy builds on NumPy and provides additional modules for optimization, linear algebra,
integration, interpolation, and statistics.

3.6 scikit-learn

scikit-learn is a widely used library for machine learning that provides simple and efficient tools
for classification, regression, clustering, dimensionality reduction, and model evaluation.

3.7 Other Notable Libraries

• Statsmodels: For statistical modeling and hypothesis testing.

• TensorFlow / PyTorch: For deep learning and neural networks.
• Jupyter: An interactive notebook environment widely used for exploratory data analysis and
sharing code with visualizations and narrative text.
• Requests / BeautifulSoup: For web scraping and handling HTTP requests.
4. Installation and Setup
Before starting data science work in Python, the environment must be properly installed and
configured.
[Link] ,DEPT OF CSE ,ANU
 Unit-1|Introduction to Data Science, Python and NumPy

4.1 Installing Python

Python can be downloaded from the official website [Link]. It is recommended to install the
latest stable version (Python 3.x) since Python 2 has reached end-of-life and is no longer supported.

4.2 Using Anaconda Distribution

Anaconda is a popular distribution that bundles Python along with most commonly used data
science libraries (NumPy, pandas, Matplotlib, scikit-learn, Jupyter) and a package/environment
manager called conda. It greatly simplifies setup, especially for beginners, since it avoids manual
installation of dependencies.

4.3 Package Management

Python packages can be installed using pip (Python's default package installer) or conda
(Anaconda's package manager).
pip install numpy pandas matplotlib scikit-learn

conda install numpy pandas matplotlib scikit-learn

4.4 Virtual Environments

Virtual environments allow developers to create isolated environments with their own set of
dependencies, preventing version conflicts between different projects.
# Using venv

python -m venv myenv

myenv\Scripts\activate # Windows

source myenv/bin/activate # macOS/Linux

# Using conda

conda create -n myenv python=3.11

conda activate myenv

4.5 Integrated Development Environments (IDEs) and Editors

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

• Jupyter Notebook / JupyterLab: Interactive, cell-based coding environment ideal for data
exploration and visualization.
• PyCharm: A full-featured IDE with debugging and project management tools.
• VS Code: A lightweight, highly extensible editor with excellent Python and Jupyter support.
• Google Colab: A free, cloud-based Jupyter notebook environment that requires no local
setup and provides free GPU/TPU access.
4.6 Verifying the Installation
python --version

pip list

import numpy as np

print(np.__version__)

5. Python Language Basics

5.1 The Python Interpreter

Python is an interpreted language, meaning code is executed line by line by the Python interpreter
rather than compiled beforehand into machine code. This allows for interactive experimentation
using the interactive shell or notebooks.

5.2 Indentation
Unlike many languages that use braces {} to define code blocks, Python uses whitespace
(indentation) to define the scope of loops, functions, classes, and conditional statements.
Consistent indentation is mandatory and improves readability.

if x > 0:

print("Positive")

else:

print("Non-positive")

5.3 Comments

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

Comments are used to annotate code and are ignored by the interpreter. Single-line comments
begin with #, while multi-line comments/docstrings use triple quotes.
# This is a single-line comment

"""

This is a multi-line comment

or docstring

"""

5.4 Variables and Object References

In Python, variables are references (names) bound to objects in memory rather than containers
holding values directly. Variables do not need explicit type declarations since Python uses dynamic
typing.
a = 10

b = a # b now refers to the same object as a

a = [1, 2, 3]

print(type(a)) # <class 'list'>

5.5 Data Types

Python has several built-in scalar data types:
• int – integer numbers, e.g. 10, -5
• float – floating point (decimal) numbers, e.g. 3.14
• str – text strings, e.g. "hello"
• bool – Boolean values, True or False
• complex – complex numbers, e.g. 2+3j
• NoneType – represents the absence of a value (None)
5.6 Operators
• Arithmetic operators: +, -, *, /, // (floor division), % (modulus), ** (exponentiation)
• Comparison operators: ==, !=, >, <, >=, <=
• Logical operators: and, or, not
• Assignment operators: =, +=, -=, *=, /=
• Membership operators: in, not in

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

• Identity operators: is, is not

5.7 Control Flow

Python supports conditional statements (if, elif, else) and loops (for, while) to control the flow of
program execution.
for i in range(5):

if i % 2 == 0:

print(i, "is even")

else:

print(i, "is odd")

n = 0

while n < 3:

print(n)

n += 1

5.8 Everything is an Object

In Python, every piece of data — numbers, strings, functions, and even classes — is an object with
an associated type. This uniform object model gives Python great flexibility, such as passing
functions as arguments to other functions.

6. Built-in Data Structures

Python provides several built-in data structures that are essential for organizing and manipulating
collections of data.

6.1 Lists
A list is an ordered, mutable (changeable) collection that can hold items of different data types.
Lists are defined using square brackets.

my_list = [1, 2, 3, "four", 5.0]

my_list.append(6) # add element at the end

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

my_list.insert(0, "start") # insert at index 0

my_list.remove("four") # remove by value

my_list.pop() # remove and return last element

print(my_list[1:3]) # slicing

my_list.sort() # sort in place (elements must be comparable)

6.2 Tuples
A tuple is an ordered, immutable (unchangeable) collection defined using parentheses. Because
tuples cannot be modified after creation, they are often used to represent fixed collections of
values, and can be used as dictionary keys.

my_tuple = (1, 2, 3)

a, b, c = my_tuple # tuple unpacking

print(my_tuple.count(2))

print(my_tuple.index(3))

6.3 Dictionaries
A dictionary stores data as key-value pairs and is defined using curly braces. Dictionaries are
mutable and unordered prior to Python 3.7, but preserve insertion order from Python 3.7 onward.
Keys must be unique and hashable (immutable).
student = {"name": "Alice", "age": 21, "course": "Data Science"}

print(student["name"])

student["grade"] = "A" # add new key-value pair

[Link]({"age": 22}) # update existing value

for key, value in [Link]():

print(key, ":", value)

6.4 Sets
[Link] ,DEPT OF CSE ,ANU
 Unit-1|Introduction to Data Science, Python and NumPy

A set is an unordered collection of unique elements defined using curly braces or the set() function.
Sets are useful for removing duplicates and performing mathematical set operations.
set_a = {1, 2, 3}

set_b = {2, 3, 4}

print(set_a.union(set_b)) # {1, 2, 3, 4}

print(set_a.intersection(set_b)) # {2, 3}

print(set_a.difference(set_b)) # {1}

6.5 Strings
Strings are immutable sequences of characters. Python provides a rich set of built-in string methods
for manipulation.
s = "Data Science"

print([Link]()) # DATA SCIENCE

print([Link]()) # data science

print([Link]()) # ['Data', 'Science']

print([Link]("Data", "Applied"))

print(s[0:4]) # slicing -> 'Data'

print(f"Course: {s}") # f-string formatting

6.6 Choosing the Right Data Structure

• Use a list when you need an ordered, changeable collection with possible duplicates.
• Use a tuple when the collection should remain constant/unchangeable.
• Use a dictionary when data needs to be accessed via a descriptive key rather than a numeric
index.
• Use a set when you need to store unique items and perform set operations quickly.
7. Functions
Functions are reusable blocks of code designed to perform a specific task. They help organize
code, avoid repetition, and improve readability and maintainability.

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

7.1 Defining and Calling Functions

def greet(name):

"""Return a greeting message for the given name."""

return f"Hello, {name}!"

print(greet("Alice"))

7.2 Default and Keyword Arguments

def power(base, exponent=2):

return base ** exponent

print(power(5)) # uses default exponent=2 -> 25

print(power(5, exponent=3)) # keyword argument -> 125

7.3 Variable-Length Arguments (*args and **kwargs)

def total(*args, **kwargs):

print("Positional:", args)

print("Keyword:", kwargs)

total(1, 2, 3, x=10, y=20)

7.4 Return Values

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

A function can return a single value, multiple values (as a tuple), or nothing (in which case it
returns None by default).
def min_max(values):

return min(values), max(values)

low, high = min_max([4, 2, 9, 1])

7.5 Lambda (Anonymous) Functions

Lambda functions are small, single-expression functions defined without a name using the lambda
keyword. They are often used as short throwaway functions passed to higher-order functions like
map(), filter(), and sorted().
square = lambda x: x ** 2

print(square(5)) # 25

nums = [1, 2, 3, 4]

squared = list(map(lambda x: x ** 2, nums))

evens = list(filter(lambda x: x % 2 == 0, nums))

7.6 Scope: Local vs Global Variables

Variables defined inside a function have local scope and are not accessible outside it, while
variables defined outside functions have global scope. The global keyword can be used inside a
function to modify a global variable.

7.7 Generators
Generators are functions that use the yield keyword to produce a sequence of values lazily (one at
a time), which is memory-efficient for large datasets.
def count_up_to(n):

i = 1

while i <= n:

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

yield i

i += 1

for num in count_up_to(5):

print(num)

8. Files
Working with files is essential in data science for reading input data and writing output results.
Python provides built-in functions for file handling.

8.1 Opening and Closing Files

The open() function is used to open a file, and the close() method releases the file resource. It is
best practice to use the "with" statement, which automatically closes the file even if an error occurs.
with open("[Link]", "r") as f:

content = [Link]()

print(content)

8.2 File Modes

• "r" – Read (default mode); raises an error if the file does not exist.
• "w" – Write; creates a new file or overwrites an existing file.
• "a" – Append; adds data to the end of the file without removing existing content.
• "r+" – Read and write.
• "b" – Binary mode, used for non-text files (e.g., "rb", "wb").
8.3 Reading Files
with open("[Link]", "r") as f:

line = [Link]() # read a single line

all_lines = [Link]() # read all lines into a list

with open("[Link]", "r") as f:

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

for line in f: # iterate line by line (memory efficient)

print([Link]())

8.4 Writing Files

with open("[Link]", "w") as f:

[Link]("Hello, Data Science!\n")

[Link](["Line 1\n", "Line 2\n"])

8.5 Working with CSV Files

CSV (Comma-Separated Values) is a common tabular data format. Python's built-in csv module
or pandas library can be used to read and write CSV files.
import csv

with open("[Link]", "r") as f:

reader = [Link](f)

for row in reader:

print(row)

# Using pandas (more common in data science)

import pandas as pd

df = pd.read_csv("[Link]")

8.6 Exception Handling with Files

It is good practice to handle exceptions such as FileNotFoundError when working with files, to
prevent the program from crashing unexpectedly.
try:

with open("[Link]", "r") as f:

data = [Link]()

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

except FileNotFoundError:

print("The file does not exist.")

9. Introduction to NumPy
NumPy (Numerical Python) is the fundamental package for scientific and numerical computing in
Python. It provides a powerful N-dimensional array object (ndarray) and a collection of tools for
performing fast mathematical operations on arrays without writing explicit loops.

9.1 Why NumPy?

• Performance: NumPy arrays are stored in contiguous memory blocks and operations are
implemented in optimized, pre-compiled C code, making them significantly faster than
native Python lists for numerical operations.
• Vectorization: NumPy allows operations on entire arrays without explicit for-loops,
resulting in concise and efficient code.
• Memory Efficiency: NumPy arrays consume less memory compared to Python lists because
they store elements of a single, fixed data type.
• Foundation for the Data Science Stack: Libraries such as pandas, scikit-learn, and Matplotlib
are built on top of NumPy arrays.
• Broadcasting: NumPy can perform arithmetic operations on arrays of different shapes in an
intuitive and efficient manner.
9.2 Importing NumPy
import numpy as np

10. The Basics of NumPy Arrays

10.1 Creating Arrays

The core data structure in NumPy is the ndarray (N-dimensional array). Arrays can be created from
Python lists or using built-in NumPy functions.
a = [Link]([1, 2, 3, 4]) # 1D array

b = [Link]([[1, 2, 3], [4, 5, 6]]) # 2D array

zeros = [Link]((3, 4)) # array of zeros, shape (3,4)

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

ones = [Link]((2, 3)) # array of ones

full = [Link]((2, 2), 7) # array filled with a constant value

identity = [Link](3) # 3x3 identity matrix

seq = [Link](0, 10, 2) # [0 2 4 6 8], like range()

lin = [Link](0, 1, 5) # 5 evenly spaced values between 0 and 1

rand = [Link]((2, 2)) # random floats in [0,1)

randn = [Link](0, 1, (3,3)) # normal distribution samples

randint = [Link](0, 10, (2,3)) # random integers

10.2 Array Attributes

Every ndarray has attributes describing its structure:
• ndim – number of dimensions (axes) of the array.
• shape – a tuple indicating the size of the array along each dimension.
• size – the total number of elements in the array.
• dtype – the data type of the array's elements (e.g., int64, float64).
• itemsize – size in bytes of each element.
• nbytes – total size in bytes of the array (size × itemsize).
arr = [Link]([[1, 2, 3], [4, 5, 6]])

print([Link]) # 2

print([Link]) # (2, 3)

print([Link]) # 6

print([Link]) # int64

print([Link]) # 8

print([Link]) # 48

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

10.3 Array Data Types

Unlike Python lists, NumPy arrays are homogeneous — all elements must share the same data
type. Common dtypes include int8/16/32/64, float16/32/64, bool, and complex64/128. The data
type can be explicitly specified during array creation using the dtype parameter.
arr = [Link]([1, 2, 3], dtype=np.float32)

arr2 = [Link](np.int32) # convert (cast) to another dtype

10.4 Array Indexing (Basics)

Elements in a NumPy array can be accessed using square bracket notation, similar to Python lists,
but extended to multiple dimensions using comma-separated indices.
arr = [Link]([[1, 2, 3], [4, 5, 6]])

print(arr[0, 0]) # 1 (row 0, column 0)

print(arr[1, 2]) # 6 (row 1, column 2)

arr[0, 0] = 100 # modify an element

10.5 Array Slicing

Slicing allows the extraction of sub-arrays using the syntax start:stop:step, applied independently
to each dimension. Note that array slices in NumPy return views (not copies) of the original array
by default, meaning modifications to a slice affect the original array.
arr = [Link](10)

print(arr[2:7]) # [2 3 4 5 6]

print(arr[::2]) # [0 2 4 6 8]

print(arr[::-1]) # reversed array

grid = [Link]([[1,2,3],[4,5,6],[7,8,9]])

print(grid[:2, 1:]) # first two rows, columns from index 1

print(grid[:, 0]) # first column of every row

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

sub = grid[:2, :2]

sub[0, 0] = 99 # modifies grid too, since sub is a view

sub_copy = grid[:2, :2].copy() # explicit copy, independent of grid

10.6 Reshaping Arrays

The reshape() method changes the shape of an array without changing its data, as long as the total
number of elements remains the same.
grid = [Link](1, 10).reshape((3, 3))

flat = [Link](-1) # flatten to 1D; -1 means "infer this dimension"

flat2 = [Link]() # returns a flattened copy

flat3 = [Link]() # returns a flattened view when possible

10.7 Array Concatenation and Splitting

x = [Link]([1, 2, 3])

y = [Link]([4, 5, 6])

print([Link]([x, y])) # [1 2 3 4 5 6]

grid = [Link]([[1,2],[3,4]])

print([Link]([grid, [5, 6]])) # stack vertically (add row)

print([Link]([grid, [[7],[8]]])) # stack horizontally (add column)

a, b, c = [Link]([Link](9), [3, 6]) # split at indices 3 and 6

upper, lower = [Link](grid, [1])

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

left, right = [Link](grid, [1])

11. Computation on NumPy Arrays

11.1 Universal Functions (ufuncs) and Vectorization

NumPy performs fast element-wise operations through universal functions (ufuncs), which are
vectorized wrappers around simple functions. Vectorized operations avoid slow Python-level for-
loops by pushing the loop into compiled C code.
arr = [Link]([1, 2, 3, 4])

print(arr + 5) # [6 7 8 9] (broadcasting a scalar)

print(arr * 2) # [2 4 6 8]

print(arr ** 2) # [1 4 9 16]

print([Link](arr)) # element-wise square root

print([Link](arr)) # element-wise exponential

print([Link](arr)) # element-wise natural logarithm

print([Link](arr)) # element-wise trigonometric sine

11.2 Arithmetic Between Arrays

When two arrays of the same shape are combined with an arithmetic operator, the operation is
applied element-wise.
a = [Link]([1, 2, 3])

b = [Link]([10, 20, 30])

print(a + b) # [11 22 33]

print(a * b) # [10 40 90]

print(b / a) # [10. 10. 10.]

11.3 Broadcasting

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

Broadcasting is a set of rules that allows NumPy to perform arithmetic operations on arrays with
different (but compatible) shapes, without making unnecessary copies of data. Two dimensions
are compatible when they are equal or one of them is 1.
a = [Link]([[1, 2, 3], [4, 5, 6]]) # shape (2,3)

b = [Link]([10, 20, 30]) # shape (3,)

print(a + b)

# b is "broadcast" across each row of a:

# [[11 22 33]

# [14 25 36]]

11.4 Common Mathematical Functions

• [Link], [Link], [Link], [Link] – basic arithmetic ufuncs.
• [Link], [Link] – exponentiation and modulus.
• [Link] – absolute value.
• [Link], [Link], [Link] – rounding functions.
• [Link], [Link], [Link], np.log2, np.log10 – roots, exponentials and logarithms.
• [Link], [Link], [Link] – trigonometric functions.
11.5 Performance: Vectorized Operations vs Loops
A simple example illustrates the performance benefit of vectorization: computing the reciprocal of
every element in a large array using a Python for-loop can be orders of magnitude slower than
using NumPy's vectorized division.
# Slow: explicit Python loop

def compute_reciprocals(values):

output = [Link](len(values))

for i in range(len(values)):

output[i] = 1.0 / values[i]

return output

# Fast: vectorized ufunc

output = 1.0 / values

[Link] ,DEPT OF CSE ,ANU
 Unit-1|Introduction to Data Science, Python and NumPy

12. Aggregations
Aggregation functions summarize data by computing a single value (or a reduced set of values)
from an array, such as sum, mean, or maximum.

12.1 Common Aggregation Functions

arr = [Link]([[1, 2, 3], [4, 5, 6]])

print([Link](arr)) # 21 (sum of all elements)

print([Link](arr)) # 1

print([Link](arr)) # 6

print([Link](arr)) # 3.5

print([Link](arr)) # 3.5

print([Link](arr)) # standard deviation

print([Link](arr)) # variance

print([Link](arr)) # product of all elements

print([Link](arr > 5)) # True if any element satisfies condition

print([Link](arr > 0)) # True if all elements satisfy condition

12.2 Aggregating Along an Axis

Multi-dimensional arrays can be aggregated along a specific axis using the axis parameter. axis=0
aggregates down each column, while axis=1 aggregates across each row.
arr = [Link]([[1, 2, 3], [4, 5, 6]])

print([Link](axis=0)) # [5 7 9] -> column-wise sum

print([Link](axis=1)) # [6 15] -> row-wise sum

print([Link](axis=0)) # [4 5 6]

print([Link](axis=1)) # [2. 5.]

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

12.3 NaN-Safe Aggregations

When a dataset contains missing values represented as NaN (Not a Number), regular aggregation
functions will return NaN as well. NumPy provides NaN-safe versions of these functions that
ignore missing values.
arr = [Link]([1, [Link], 3, 4])

print([Link](arr)) # nan

print([Link](arr)) # 8.0

print([Link](arr)) # 2.666...

print([Link](arr)) # 4.0

12.4 Practical Use Case

Aggregations are especially useful for exploratory data analysis, such as quickly computing
summary statistics (mean height, minimum temperature, maximum sales) across large datasets
without writing manual loops.

13. Comparisons, Masks, and Boolean Logic

13.1 Comparison Operators as ufuncs

Comparison operators (<, >, <=, >=, ==, !=) can be applied directly to NumPy arrays, producing
a new Boolean array of the same shape where each element indicates the result of the comparison
for the corresponding element.
arr = [Link]([1, 2, 3, 4, 5])

print(arr < 3) # [ True True False False False]

print(arr == 3) # [False False True False False]

print(arr != 3) # [ True True False True True]

13.2 Working with Boolean Arrays

arr = [Link]([1, 2, 3, 4, 5, 6])

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

print(np.count_nonzero(arr > 3)) # 3, count of True values

print([Link](arr > 3)) # 3, True treated as 1, False as 0

print([Link](arr > 5)) # True

print([Link](arr > 0)) # True

13.3 Boolean Logic Operators

To combine multiple conditions on arrays, Python's keywords and/or/not cannot be used directly
(since they operate on single Boolean values). Instead, NumPy provides the bitwise logic operators
& (and), | (or), ~ (not), and ^ (xor), which work element-wise on Boolean arrays. Each condition
must be wrapped in parentheses due to operator precedence.
arr = [Link](10)

print((arr > 2) & (arr < 8)) # element-wise AND

print((arr < 2) | (arr > 8)) # element-wise OR

print(~(arr > 5)) # element-wise NOT

13.4 Masking (Boolean Array Indexing)

A Boolean array can be used as a "mask" to select only the elements of an array that satisfy a given
condition. This is one of the most powerful and frequently used features of NumPy for data
filtering.
arr = [Link]([10, 15, 20, 25, 30])

mask = arr > 18

print(mask) # [False False True True True]

print(arr[mask]) # [20 25 30] -> filtered array

print(arr[arr > 18]) # same result written directly

arr[arr < 15] = 0 # conditional assignment using a mask

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

13.5 Practical Example

Masking is commonly used for tasks such as filtering out outliers, selecting rows meeting a
business rule (e.g., sales above a threshold), or replacing invalid entries with a default value, all
without writing explicit loops.

14. Indexing
Beyond basic (simple) indexing and slicing, NumPy supports advanced indexing techniques that
provide flexible and powerful ways to access and modify array elements.

14.1 Basic Indexing (Recap)

arr = [Link]([10, 20, 30, 40, 50])

print(arr[0]) # 10

print(arr[-1]) # 50 (last element)

print(arr[1:4]) # [20 30 40]

14.2 Fancy Indexing

Fancy indexing involves passing an array (or list) of indices to access multiple, potentially non-
contiguous elements at once. Unlike simple slicing, fancy indexing always returns a copy of the
data, not a view.
arr = [Link]([10, 20, 30, 40, 50])

indices = [0, 2, 4]

print(arr[indices]) # [10 30 50]

grid = [Link](12).reshape(3, 4)

row_idx = [Link]([0, 1, 2])

col_idx = [Link]([2, 1, 3])

print(grid[row_idx, col_idx]) # elements at (0,2), (1,1), (2,3)

14.3 Combined Indexing

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

Fancy indexing can be combined with simple indices, slices, and Boolean masks for highly flexible
data selection.

grid = [Link](12).reshape(3, 4)

print(grid[1, [0, 2]]) # row 1, columns 0 and 2

print(grid[1:, [0, 2]]) # rows from index 1, columns 0 and 2

print(grid[grid > 5]) # combined with Boolean masking

14.4 Modifying Values with Fancy Indexing

arr = [Link](10)

arr[[1, 3, 5]] = 0 # set multiple elements at once

print(arr)

arr = [Link](10)

[Link](arr, [0, 0, 2], 1) # correctly handles repeated indices

print(arr) # [2. 0. 1. 0. ...]

14.5 [Link]()
The [Link]() function returns the indices where a condition is true, or can be used to choose
between two arrays element-wise based on a condition.
arr = [Link]([1, -2, 3, -4, 5])

print([Link](arr > 0)) # (array([0, 2, 4]),) - indices

print([Link](arr > 0, arr, 0)) # [1 0 3 0 5] - replace negatives with 0

15. Sorting
NumPy provides efficient algorithms for sorting arrays, which are essential for data analysis tasks
such as ranking, finding percentiles, and preparing data for search algorithms.

15.1 [Link]()

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

[Link]() returns a new, sorted copy of an array without modifying the original array. By default,
NumPy uses an efficient quicksort algorithm (O(N log N) complexity).
arr = [Link]([5, 2, 8, 1, 9])

sorted_arr = [Link](arr)

print(sorted_arr) # [1 2 5 8 9]

print(arr) # original array remains unchanged

15.2 The sort() Method (In-Place Sorting)

Calling the sort() method directly on an array sorts it in-place, modifying the original array and
returning None.
arr = [Link]([5, 2, 8, 1, 9])

[Link]()

print(arr) # [1 2 5 8 9], original array modified

15.3 argsort()

[Link]() returns the indices that would sort the array, which is useful when you need to sort
one array based on the order of another (e.g., sorting names by corresponding scores).
arr = [Link]([50, 20, 80, 10])

indices = [Link](arr)

print(indices) # [3 1 0 2]

print(arr[indices]) # [10 20 50 80], sorted using the indices

15.4 Sorting Along an Axis

For multi-dimensional arrays, sorting can be performed independently along rows or columns
using the axis parameter.

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

grid = [Link]([[3, 7, 1], [9, 2, 5]])

print([Link](grid, axis=0)) # sort each column independently

print([Link](grid, axis=1)) # sort each row independently

15.5 Partial Sorts: Partitioning

Sometimes we only need the k smallest (or largest) values in an array, not a fully sorted array.
[Link]() is more efficient than a full sort for this purpose: it places the k smallest values first
(in arbitrary order) followed by the remaining values.
arr = [Link]([7, 2, 3, 1, 6, 5, 4])

print([Link](arr, 3)) # smallest 3 values appear first (any order)

print([Link](arr, 3)) # indices version of partition

15.6 Sorting with Keys: [Link]

[Link]() performs an indirect stable sort using multiple keys, sorting primarily by the last key
provided, useful for sorting on multiple columns (e.g., sort by department, then by salary).
salary = [Link]([50000, 60000, 50000])

dept = [Link]([2, 1, 1])

order = [Link]((salary, dept)) # sort by dept first, then salary

print(order)

15.7 Summary Table

• [Link](arr) – returns a sorted copy.
• [Link]() – sorts in-place, returns None.
• [Link](arr) – returns indices that would sort the array.
• [Link](arr, k) – partial sort, k smallest values first.

[Link] ,DEPT OF CSE ,ANU

 Unit-1|Introduction to Data Science, Python and NumPy

• axis parameter – controls whether sorting happens row-wise or column-wise for 2D arrays.

[Link] ,DEPT OF CSE ,ANU