Computing (Python) Bootcamp
Python for Data Analysis
Ch4: Numpy Basics
Nat Butler; natbutler@[Link]
Nat Butler Python Programming,
SES 350 1/e 1
11 August 2014
Numpy
Short for “Numerical Python”
High performance scientific computing
and data analysis.
All you ever need! (?)
-MATLAB:
[Link]
-IDL:
[Link]
Nat Butler Python
SESProgramming,
350 1/e 2 11 August 2014
Numpy
ndarray: fast and memory-efficient
array for “vectorized” computation
Math functions on ndarrays (no
looping!)
Input/Output tools for files, linear
algebra, etc.
Nat Butler Python
SESProgramming,
350 1/e 3 11 August 2014
ndarray
In [8]: data
Out[8]:
array([[ 0.9325, -1.0739, -1.335 , -0.4788, -0.3803],
[-1.4074, -0.9413, -2.0035, 0.8486, -1.2619],
[ 2.1333, 0.1417, -1.8533, 0.188 , -0.2685],
[ 0.6461, 0.7336, -1.7335, 0.2672, -0.1442]])
In [9]: data * 10
Out[9]:
array([[ 9.3248, -10.7393, -13.3504, -4.7884, -3.8027],
[-14.0743, -9.4125, -20.0349, 8.4858, -12.6193],
[ 21.3334, 1.4174, -18.5326, 1.8796, -2.6849],
[ 6.4613, 7.3358, -17.3352, 2.6722, -1.4416]])
In [10]: [Link]
Out[10]: (4,5)
In [11]: [Link]
Out[11]: dtype('float64')
Nat Butler Python
SESProgramming,
350 1/e 4 11 August 2014
ndarray
In [8]: data
Out[8]:
array([[ 0.9325, -1.0739, -1.335 , -0.4788, -0.3803],
[-1.4074, -0.9413, -2.0035, 0.8486, -1.2619],
[ 2.1333, 0.1417, -1.8533, 0.188 , -0.2685],
[ 0.6461, 0.7336, -1.7335, 0.2672, -0.1442]])
In [9]: data * 10
Out[9]:
array([[ 9.3248, -10.7393, -13.3504, -4.7884, -3.8027],
[-14.0743, -9.4125, -20.0349, 8.4858, -12.6193],
[ 21.3334, 1.4174, -18.5326, 1.8796, -2.6849],
[ 6.4613, 7.3358, -17.3352, 2.6722, -1.4416]])
In [10]: [Link]
Out[10]: (4,5)
In [11]: [Link]
Out[11]: dtype('float64')
Nat Butler Python
SESProgramming,
350 1/e 5 11 August 2014
Creating ndarray's
In [13]: data1 = [6, 7.5, 8, 0, 1]
In [14]: arr1 = [Link](data1)
In [15]: arr1
Out[15]: array([ 6. , 7.5, 8. , 0. , 1. ])
In [16]: data2 = [[1, 2, 3, 4], [5, 6, 7, 8]]
In [17]: arr2 = [Link](data2)
In [18]: arr2
Out[18]:
array([[1, 2, 3, 4],
[5, 6, 7, 8]])
In [19]: [Link]
Out[19]: 2
Nat Butler Python
SESProgramming,
350 1/e 6 11 August 2014
Creating ndarray's
In [22]: [Link](10)
Out[22]: array([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
In [23]: [Link]((3, 7))
Out[23]:
array([[ 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0.]])
In [24]: [Link]((2, 3, 2))
Out[24]:
array([[[ 0., 0.],
[ 0., 0.],
[ 0., 0.]],
[[ 0., 0.],
[ 0., 0.],
[ 0., 0.]]])
In [25]: [Link](15)
Out[25]: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14])
Nat Butler Python
SESProgramming,
350 1/e 7 11 August 2014
Creating ndarray's
array Convert input data (list, tuple, array, or other sequence type)
to an ndarray either by inferring a dtype or explicitly specifying
a dtype. Always copies the input data by default.
asarray Convert input to ndarray, but do not copy if the input is already
an ndarray
arange Like the built-in range but returns an ndarray
ones, ones_like Produce an array of all 1's with the given shape and dtype.
ones_like takes another array and produces a ones array of the
same shape and dtype.
zeros, zeros_like Like ones and ones_like but producing arrays of 0's instead.
empty, empty_like Create new arrays by allocating new memory, but do not
populate with any values like ones and zeros
eye, identity Create a square N x N identity matrix (1's on the diagonal and
0's elsewhere)
Nat Butler Python
SESProgramming,
350 1/e 8 11 August 2014
Data Types
In [26]: arr1 = [Link]([1, 2, 3], dtype=np.float64)
In [27]: arr2 = [Link]([1, 2, 3], dtype=np.int32)
In [28]: [Link]
Out[28]: dtype('float64')
In [29]: [Link]
Out[29]: dtype('int32')
The default is typically a double precision float
(float64).
We set these when we create the arrays. Changing
the precision, copies the data and takes more
memory.
In [30]: arr3 = [Link](np.int32)
Nat Butler Python
SESProgramming,
350 1/e 9 11 August 2014
Basic Indexing and Slicing
In [50]: arr = [Link](10)
In [51]: arr
Out[51]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
In [52]: arr[5]
Out[52]: 5
In [53]: arr[5:8]
Out[53]: array([5, 6, 7])
In [54]: arr[5:8] = 12 ← “Broadcasting”!!
In [55]: arr
Out[55]: array([ 0, 1, 2, 3, 4, 12, 12, 12, 8, 9])
Nat Butler Python
SESProgramming,
350 1/e 10 11 August 2014
Basic Indexing and Slicing
In [56]: arr_slice = arr[5:8]
In [57]: arr_slice[1] = 12345
In [58]: arr
Out[58]: array([ 0, 1, 2, 3, 4, 12, 12345, 12, 8, 9])
In [59]: arr_slice[:] = 64
In [60]: arr
Out[60]: array([ 0, 1, 2, 3, 4, 64, 64, 64, 8, 9])
These slices point to the original data!
They do not copy it!
Nat Butler Python
SESProgramming,
350 1/e 11 11 August 2014
Boolean Selection
In [82]: names = [Link](['Bob', 'Joe', 'Will', 'Bob', 'Will', 'Joe', 'Joe'])
In [83]: data = randn(7, 4)
In [84]: names
In [86]: names == 'Bob'
Out[84]:
Out[86]: array([ True, False, False, True,
array(['Bob', 'Joe', 'Will', 'Bob', 'Will', 'Joe', 'Joe'], False, False, False], dtype=bool)
dtype='|S4')
In [85]: data In [87]: data[names == 'Bob']
Out[85]: Out[87]:
array([[-2.084 , 0.0879, -0.7845, -1.0453], array([[-2.084 , 0.0879, -0.7845, -1.0453],
[-0.3906, 0.0568, -0.7603, 0.7858], [-0.0321, -0.2718, 0.2614, 1.1471]])
[-0.5682, 0.5355, -1.7193, -1.1767],
[-0.0321, -0.2718, 0.2614, 1.1471],
[-0.2561, 1.4774, 0.2625, -0.798 ],
[ 0.5256, -0.0144, -1.0517, -1.0416],
[-1.5989, 0.9365, -0.2866, 2.2322]])
Nat Butler Python
SESProgramming,
350 1/e 12 11 August 2014
Code Breakout
Write a python script add_arrays.py to define a
function add_array.
The function should take as input a numpy array,
create 4 different arrays of the same length using
numpy array building utilities (e.g., ones, zeros,
arange).
The function should return the sum of the original
array and these 4 newly created arrays.
(In your script, use at top “import numpy as np”)
Nat Butler Python
SESProgramming,
350 1/e 13 11 August 2014
Universal Functions
ufunc: functions that perform element-wise
operations on ndarrays (no loops!)
abs, fabs
sqrt
square
cos, cosh, sin, sinh,
exp
tan, tanh
log, log10, log2, log1p
arccos, arccosh, arcsin,arcsinh, arctan, arctanh
sign
logical_not
ceil
floor
rint
modf
isnan
isfinite, isinf
Nat Butler Python
SESProgramming,
350 1/e 14 11 August 2014
Universal Functions
Also have binary ufuncs (two arguments).
add
subtract
multiply
divide, floor_divide
power
maximum, fmax
minimum, fmin
mod
copysign
greater, greater_equal, less, less_equal, equal, not_equal
Nat Butler Python
SESProgramming,
350 1/e 15 11 August 2014
Math and Stats
Mean, sum, cumsum, cumprod, std, var,
min, max, argmin, argmax...
In [150]: arr = [Link](5, 4) # normally-distributed data
In [151]: [Link]()
Out[151]: 0.097586955881090537
In [152]: [Link](arr)
Out[152]: 0.097586955881090537
In [153]: [Link]()
Out[153]: 1.9517391176218106
Nat Butler Python
SESProgramming,
350 1/e 16 11 August 2014
Sorting Operations
Unary: sort, argsort, unique
Binary: intersect1d, union1d, in1d, setdiff1d,
setxor1d
In [175]: names = [Link](['Bob', 'Joe', 'Will', 'Bob', 'Will', 'Joe', 'Joe'])
In [176]: [Link](names)
Out[176]:
array(['Bob', 'Joe', 'Will'], dtype='|S4')
In [179]: values = [Link]([6, 0, 0, 3, 2, 5, 6])
In [180]: np.in1d(values, [2, 3, 6])
Out[180]: array([ True, False, False, True, True, False, True],dtype=bool)
Nat Butler Python
SESProgramming,
350 1/e 17 11 August 2014
Code Breakout II
Take your code add_arrays.py, and create a
modified version tweak_arrays.py.
Apply a different python ufunc to each of the
4 new arrays created.
Calculate the sum of all the arrays, and have
your code return a single number: the sum
(over elements) of that final array.
Nat Butler Python
SESProgramming,
350 1/e 18 11 August 2014
File I/O
Binary and text file support.
In [181]: arr = [Link](10)
In [182]: [Link]('some_array', arr)
In [183]: [Link]('some_array.npy')
Out[183]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
Multiple arrays:
In [184]: [Link]('array_archive.npz', a=arr, b=arr)
In [185]: arch = [Link]('array_archive.npz')
In [186]: arch['b']
Out[186]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
Nat Butler Python
SESProgramming,
350 1/e 19 11 August 2014
File I/O
Binary and text file support.
In [189]: !cat array_ex.txt
0.580052,0.186730,1.040717,1.134411
0.194163,-0.636917,-0.938659,0.124094
-0.126410,0.268607,-0.695724,0.047428
-1.484413,0.004176,-0.744203,0.005487
2.302869,0.200131,1.670238,-1.881090
-0.193230,1.047233,0.482803,0.960334
This can be loaded into a 2D array like so:
In [190]: arr = [Link]('array_ex.txt', delimiter=',')
In [191]: arr
Out[191]:
Array([[ 0.5801, 0.1867, 1.0407, 1.1344], ...
Nat Butler Python
SESProgramming,
350 1/e 20 11 August 2014