Data Analytics for Exploration Geologists

Why Python
1. Easy to read and simple syntatx
2. Extensive community of
developers and external
languages
3. Cross platform
4. Excellent for rapid development
5. Strong Data Science and
Machine learning capabilities
6. Can be used for scripting AND
production quality code
Course outline
1. Introduction to python (today) 3. Supervised classification
Python data types Train a model to predict labels on an
Import and manipulate tablulated data unseen dataset

QAQC geochemistry data

2. Unsupervised clustering
4. Modal mineralogy
Exploratory data analysis
Making plots Calculate mineralogy from assay data

Automatically domain geochemical

data
Using external dependencies
Python has a significant ecosystem full of 3rd party specialised tools for performing
tasks quickly and easily:

pandas for dealing with tablulate data

matplotlib and seaborn for creating plots

pyvista for 3D visualisation

scikit-learn/scikit-image for machine learning and image analysis

geopandas for dealing with spatial datasets

When choosing 3rd party libraries be vigilant and try to use trusted sources e.g.
numfocus, subsurface
Installing

pip
The basic python package manage pip is great for installing basic libraries however it
can have trouble with libraries requiring external dependencies e.g. geospatial data
using gdal.

python -m pip install numpy pandas

anaconda python
Anaconda python is a popular alternative to pip and allows you to use the conda
installation process which is better at managing all of the required dependencies

conda install -c conda-forge numpy pandas

Importing
To import an external library there are different syntax that can be used:

1. Import the entire package

import numpy
a = [Link]([0])

2. Import the package using an alias (most popular)

import numpy as np
a = [Link]([0])

3. Import what we need

from numpy import array

a = array([0])
Tabular data using Pandas

What is a DataFrame ?
DataFrame : A two-dimensional, size-mutable, and potentially heterogeneous tabular
data structure.
Structure: Organized in rows and columns (like a spreadsheet or SQL table).
Key Features:
Allows for heterogeneous data types (e.g., strings, integers, floats).
Each column is a Series , and the index labels the rows.
Loading/Creating a csv

CSV XLS PARQUET

CSV XLS PARQUET

read_* to_*
HTML HDF5 JSON HTML HDF5 JSON
<> {} <> {}
GBQ SQL GBQ SQL
... ...

import pandas as pd

data = pd.read_csv(path)
Accessing Data in a DataFrame
Accessing Columns:
Use the column name as a key: df['Mineral'] .
Accessing Rows:
Use the .iloc[] for position-based indexing: [Link][0] .
Use .loc[] for label-based indexing: [Link][0] (assuming 0 is the row label).

# Accessing a column by name

print("Mineral column:\n", df['Mineral'])

# Accessing a row by position

print("First row:\n", [Link][0])

# Accessing a row by label (if index is custom)

df_custom_index = df.set_index('Mineral')
print("Row for Feldspar:\n", df_custom_index.loc['Feldspar'])
Simple calculations
We can use mathematical operators on the DataFrame to perform row operations.

data['a_plus_b'] = data['a']+data['b'] # adding a and b

data['a_div_b'] = data['a'] / data['b'] #a / b
data['a_times_b'] = data['a'] * data['b'] #a * b
Working with the Index
Indexing: DataFrames have an index, which can be set and accessed.
Default Index: Automatically generated (0, 1, 2,...).
Custom Index: Set custom index like a mineral name or geological property.

# Custom index using the 'Mineral' column

df_custom_index = df.set_index('Mineral')
print("Custom Index DataFrame:\n", df_custom_index)

# Accessing row by custom index label

print("Row for Mica:\n", df_custom_index.loc['Mica'])
Slicing and Filtering DataFrames
Column Slicing: Access multiple columns at once.
Example: df[['Mineral', 'Density']]
Row Filtering: Use conditions to filter rows.
Example: df[df['Density'] > 2.6] → All rows where density > 2.6.

# Slicing multiple columns

print("Mineral and Color columns:\n", df[['Mineral', 'Color']])

# Filtering rows based on condition

print("Minerals with density > 2.6:\n", df[df['Density'] > 2.6])
Modifying DataFrames
Adding a Column: Add new columns with calculated or constant values.
Updating a Column: Assign new values to an existing column.
Dropping a Column: Remove a column with drop() .

# Adding a new column

df['Hardness'] = [7, 6, 3]
print("DataFrame with Hardness column:\n", df)

# Updating a column
df['Density'] = df['Density'] * 1.1
print("Updated Density column:\n", df)

# Dropping a column
df = [Link](columns='Color')
print("DataFrame without Color column:\n", df)
Example: DataFrame Operations
# DataFrame with mineral data
df = [Link]({
'Mineral': ['Quartz', 'Feldspar', 'Mica'],
'Density': [2.65, 2.56, 2.78],
'Color': ['Clear', 'White', 'Grey']
})

# Accessing a column by name

print("Mineral column:\n", df['Mineral'])

# Filtering rows
filtered_df = df[df['Density'] > 2.6]
print("Minerals with density > 2.6:\n", filtered_df)

# Adding a new column

df['Hardness'] = [7, 6, 3]
print("DataFrame with Hardness column:\n", df)
import pandas as pd

# Creating a DataFrame
data = {
'Mineral': ['Quartz', 'Feldspar', 'Mica'],
'Density': [2.65, 2.56, 2.78],
'Color': ['Clear', 'White', 'Grey']
}

df = [Link](data)
print("DataFrame:\n", df)
Variables in Python
Definition: Variables are containers for storing data values.
Naming Rules:
Must start with a letter or underscore (_)
Can contain letters, numbers, and underscores (e.g., rock_density , _depth )
Case-sensitive (e.g., RockType and rocktype are different)
Assigning Variables
Assignment: Use the = operator to assign a value.
Example:
rock_type = "Sandstone" # String
depth_m = 50 # Integer
density_kg_m3 = 2.65 # Float

Dynamic Typing: Python infers the variable's type from the value assigned to it.
Updating and Using Variables
Variables can be updated, and the new value replaces the old one:
depth_m = 60

Variables allow for reuse throughout code, simplifying changes.

Introduction to Data Types
Definition: Data types determine the kind of values stored and manipulated in a
program.
Importance: Knowing data types is foundational for writing effective, error-free
code.
Common Python Data Types
Integers ( int ): Whole numbers, positive or negative, without decimals.
Example: 5 , -3 , 42
Floats ( float ): Numbers with decimal points.
Example: 3.14 , -2.5 , 0.0
Strings ( str ): A sequence of characters, used for text.
Example: "Hello" , "Python123" , "Geology"
Booleans ( bool ): True/False values, useful for conditional logic.
Example: True , False
Lists ( list ): Ordered, mutable collections of items, can hold mixed types.
Example: [1, 2, 3] , ["rock", "mineral", "fossil"]
Dictionaries ( dict ): Unordered collections of key-value pairs, useful for data
mapping.
Example: {"type": "sandstone", "age": "Jurassic"}
Example of Basic Data Types in Action
# Basic Data Types Example
rock_density = 2.65 # Float: density of a rock in g/cm³
mineral_name = "Quartz" # String: name of a mineral
age_of_sample = 120 # Integer: age of sample in millions of years
is_metamorphic = False # Boolean: metamorphic status of rock

# List of mineral samples

mineral_samples = ["Quartz", "Feldspar", "Mica"]

# Dictionary to store rock sample data

rock_sample = {
"density": rock_density,
"name": mineral_name,
"age": age_of_sample,
"metamorphic": is_metamorphic
}

# Printing to show types and values

print("Mineral Name:", mineral_name)
print("Rock Density:", rock_density)
print("Sample Age:", age_of_sample)
print("Is Metamorphic:", is_metamorphic)
print("Mineral Samples List:", mineral_samples)
print("Rock Sample Dictionary:", rock_sample)
Immutable vs. Mutable Data Types
Immutable: Cannot be changed after creation
Mutable: Can be modified in place
Immutable Data Types

Once created, these values cannot be changed; any "change" results in a new
object.

Examples:

Integers: int_var = 42
Floats: pi = 3.14159
Strings: rock_type = "Sandstone"
Tuples: coordinates = (34.2, -118.5)
# Example: Immutable string
rock_type = "Sandstone"
rock_type[0] = "B"
Mutable Data Types

Can be changed in place without creating a new object.

Examples:

Lists: samples = [10, 20, 30]

Dictionaries: properties = {"density": 2.65, "porosity": 0.15}
Sets: minerals = {"Quartz", "Feldspar", "Mica"}
# Example: Mutable list
samples = [10, 20, 30]
samples[0] = 15
References vs. New Objects in Python
References: Variables are references (pointers) to objects in memory.
Assigning a variable to another does not create a new object; it creates a new
reference to the same object.
Example: References with Mutable Types

# Mutable Example: Lists

list_a = [1, 2, 3]
list_b = list_a # Both reference the same list

list_b[0] = 100 # Modifying list_b also changes list_a

print(list_a) # Output: [100, 2, 3]

list_a and list_b point to the same object in memory.

Changing one affects the other.

Creating New Objects
Use copying methods to create a new object, not a reference.

# Create a new list using slicing or copy()

list_c = list_a[:] # Shallow copy using slicing
list_d = list_a.copy() # Using copy method

list_c[0] = 50 # Does not affect list_a

print(list_a) # Output: [100, 2, 3]
print(list_c) # Output: [50, 2, 3]

New Objects: Changes to list_c do not impact list_a or list_b .

Why It Matters
Avoid Unintended Side-Effects: Understand references when working with
mutable objects (lists, dictionaries).
Copy Carefully: Use copy() for lists or deepcopy() for nested structures if a truly
independent object is needed.
Operators in Python
Operators are symbols or keywords that are used to instruct Python to perform specific
operations on variables and values. These are how different algorithms are
implemented.
Basic Arithmetic Operators
Addition ( + )
Subtraction ( - )
Multiplication ( * )
Division ( / )

These operators work across multiple data types with specific behaviors.
Using Operators with Different Data Types

Numeric Types (int, float)

Addition ( + ): 3 + 2 → 5
Subtraction ( - ): 5 - 2 → 3
Multiplication ( * ): 4 * 2 → 8
Division ( / ): 10 / 2 → 5.0
Strings (str)
Concatenation ( + ): Joins two strings together.
Example: "Geo" + "logy" → "Geology"
Repetition ( * ): Repeats the string.
Example: "Rock" * 3 → "RockRockRock"

Lists (list)
Concatenation ( + ): Joins two lists.
Example: [1, 2] + [3, 4] → [1, 2, 3, 4]
Repetition ( * ): Repeats elements in the list.
Example: [1, 2] * 2 → [1, 2, 1, 2]
Example: Operators in Action
# Working with numbers
result1 = 5 + 3 # Addition
result2 = 10 - 2 # Subtraction
result3 = 7 * 4 # Multiplication
result4 = 8 / 2 # Division

# Working with strings

word1 = "Geo"
word2 = "logy"
result5 = word1 + word2 # Concatenation

# Working with lists

minerals1 = ["Quartz", "Feldspar"]
minerals2 = ["Mica"]
result6 = minerals1 + minerals2 # Concatenate lists

# Printing results
print("Addition:", result1)
print("Subtraction:", result2)
print("Multiplication:", result3)
print("Division:", result4)
print("String Concatenation:", result5)
print("List Concatenation:", result6)
String Manipulation in Python
A string is the datatype used to store text in Python and is something quiet commonly
found in geological datasets. Python has many powerful tools for manipulating strings
that can be used to process text datasets prior to further analysis, querying data or
automating QAQC workflows.
Common String Operations
Concatenation ( + ): Combining two or more strings.
Example: "Geo" + "logy" → "Geology"
Repetition ( * ): Repeating a string a specified number of times.
Example: "Rock" * 3 → "RockRockRock"
Length ( len() ): Getting the number of characters in a string.
Example: len("Geology") → 7
Indexing: Accessing individual characters in a string by position.
Example: "Geology"[2] → "o"
Slicing: Extracting a substring from a string.
Example: "Geology"[1:4] → "eol"
f-string: Allow you to evaluate python expressions or variables and include them in
the string.
Example: f"Geology_{4*10}" → "Geology_40"
String Methods
upper() : Converts all characters to uppercase.
Example: "geology".upper() → "GEOLOGY"
lower() : Converts all characters to lowercase.
Example: "GEOLOGY".lower() → "geology"
replace(old, new) : Replaces a substring with a new substring.
Example: "Geology".replace("Geo", "Rock") → "Rocklogy"
split() : Splits a string into a list at a specified delimiter (default is whitespace).
Example: "Quartz Feldspar Mica".split() → ["Quartz", "Feldspar", "Mica"]
strip() : Removes leading and trailing whitespace from a string.
Example: " Geology ".strip() → "Geology"
Example: String Manipulation in Action
# Concatenation
rock_type = "Geo" + "logy"
print("Concatenation:", rock_type)

# Repetition
minerals = "Quartz " * 3
print("Repetition:", minerals)

# Length
sample_name = "Feldspar"
print("Length of Sample:", len(sample_name))

# Indexing
first_char = "Mica"[0]
print("First character:", first_char)

# Slicing
mineral_name = "Quartz"
slice_name = mineral_name[1:4]
print("Sliced Name:", slice_name)

# String Methods
capitalized = "feldspar".upper()
print("Uppercase:", capitalized)

replaced_name = "Geology".replace("Geo", "Rock")

print("Replaced String:", replaced_name)

split_name = "Quartz Feldspar Mica".split()

print("Split String:", split_name)

stripped_name = " Mica ".strip()

print("Stripped String:", stripped_name)
Key Takeaways
String operations like concatenation and repetition are simple but powerful for text
manipulation.
Methods like upper() , split() , and replace() are essential for transforming
strings.
Geological Applications: Combine mineral names, manipulate sample labels, or
clean data for processing.
Control Flow in Python
Control flow is the order in which code statements are executed. We can use different
statements to change how the code is run e.g. repeating a process or skipping parts of
the code for come cases.
What is Control Flow?
Definition:
Types of Control Flow:
Conditional statements (if-else)
Loops (for, while)
Loop control (break, continue)
Conditional Statements
if : Executes code block if the condition is true.

elif : Checks another condition if the previous conditions are false.

else : Executes code block if all previous conditions are false.

rock_type = "igneous"
if rock_type == "sedimentary":
print("Rock type is sedimentary.")
elif rock_type == "igneous":
print("Rock type is igneous.")
else:
print("Rock type is metamorphic.")
Loops

for Loop
Iterates over a sequence like a list or range.

minerals = ["Quartz", "Feldspar", "Mica"]

for mineral in minerals:
print(mineral)
While
Continues the statement in the block as long as the conidtion is true

count = 0
while count < 3:
print("Counting:", count)
count += 1
Loop Control Statements
break : Exits the loop.

continue : Skips to the next iteration.

for sample in ["Quartz", "Unknown", "Feldspar"]:

if sample == "Unknown":
continue # Skip unknown samples
print("Sample:", sample)
Indexing and Slicing Lists
Indexing: Access a specific element by its position.
Example: minerals[0] → First element of the list.
Slicing: Extract a subset of elements.
Example: minerals[1:4] → Elements from index 1 to 3.
Negative Indexing: Access elements from the end of the list.
Example: minerals[-1] → Last element of the list.

minerals = ["Quartz", "Feldspar", "Mica", "Calcite"]

print("First mineral:", minerals[0]) # Quartz
print("Last mineral:", minerals[-1]) # Calcite
print("Slice of minerals:", minerals[1:3]) # ['Feldspar', 'Mica']
Indexing with a List of Indices
You can select multiple elements by passing a list of indices.
Example: [minerals[i] for i in [0, 2, 3]] → Selects elements at indices 0,
2, and 3.

indices = [0, 2, 3]
selected_minerals = [minerals[i] for i in indices]
print("Selected minerals:", selected_minerals) # ['Quartz', 'Mica', 'Calcite']
Accessing and Updating Dictionaries
Accessing: Use a key to retrieve a value.
Example: rock_data["Quartz"] → Retrieves the value for "Quartz" .
Updating: Assign a new value to an existing key.
Example: rock_data["Quartz"] = "Silicate"

rock_data = {
"Quartz": "SiO2",
"Feldspar": "KAlSi3O8",
"Mica": "KAl2(AlSi3O10)(F,OH)2"
}
print("Quartz composition:", rock_data["Quartz"])

# Update a value
rock_data["Quartz"] = "Silicate"
print("Updated Quartz composition:", rock_data["Quartz"])
Slicing and Looping Over Dictionaries
Keys: Access all keys using [Link]() .
Values: Access all values using [Link]() .
Items: Access key-value pairs using [Link]() .

# Loop over keys

for mineral in rock_data.keys():
print("Mineral:", mineral)

# Loop over values

for composition in rock_data.values():
print("Composition:", composition)

# Loop over key-value pairs

for mineral, composition in rock_data.items():
print(f"{mineral}: {composition}")
Accessing the File System in Python
Reading and writing files will be one of the fundamental tools you will need to use when
applying Python to your own workflows.
Absolute vs. Relative Paths
Absolute Path: Specifies the complete path from the root directory or C drive.
Example: "/Users/user_name/Documents/[Link]"
Relative Path: Specifies the path relative to the current working directory.
Example: "../data/[Link]"
Use Case:
Absolute Paths are useful for fixed directories.
Relative Paths are flexible and ideal for projects that may change locations,
for example when sharing a project with colleagues.
Working with Paths in Python
Avoid spaces in directory names
Paths on Windows, Linux and MacOSX can be different.

[Link] and pathlib modules help with cross-platform file paths.

Path Operations:

[Link]() : Gets the current working directory.

[Link](path) : Changes the current working directory to path .

[Link] : Offers easy-to-use methods for paths.

import os
from pathlib import Path

# Get current directory

current_dir = [Link]()
print("Current Directory:", current_dir)

# Change directory
[Link]("/path/to/directory")
print("New Directory:", [Link]())

# Using pathlib for paths

data_path = Path("data/[Link]")
print("Path Exists?", data_path.exists())
Finding Files with glob
glob : Searches for files matching a pattern.

Wildcard Patterns:
"*.txt" : Matches all .txt files.

"data/*.csv" : Matches all .csv files in the data folder.

from glob import glob

# Find all text files in current directory

text_files = glob("*.txt")
print("Text Files:", text_files)

# Find all CSV files in data folder

csv_files = glob("data/*.csv")
print("CSV Files:", csv_files)
Example: File System Operations
from pathlib import Path
import os

# Set up directory path

project_root = Path("project")
data_dir = project_root / "data"

# Check if directory exists and list files

if data_dir.exists():
print("Data Directory:", data_dir)
for file in data_dir.glob("*.txt"):
print("Found text file:", [Link])

# Change working directory to project root

[Link](project_root)
print("Working in:", [Link]())
NumPy Arrays
What is an ndarray ?
ndarray : The main data structure in NumPy for handling multi-dimensional arrays.

Benefits:
Efficient handling of large datasets.
Supports mathematical operations across entire arrays.
Creating Arrays:
From a list: [Link]([1, 2, 3])
Ranges: [Link](0, 10, 2) (similar to range() but returns an array)

import numpy as np

# Creating arrays
array_1d = [Link]([1, 2, 3, 4])
array_2d = [Link]([[1, 2], [3, 4]])
print("1D Array:", array_1d)
print("2D Array:\n", array_2d)
Array Indexing and Slicing
Indexing: Access elements like lists: array[index]
Slicing: Extract parts of arrays: array[start:end]
Multidimensional Slicing: Slice across multiple dimensions

array = [Link]([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# Accessing single element

print("Element at (0, 1):", array[0, 1]) # Output: 2

# Slicing rows and columns

print("First row:", array[0, :]) # [1, 2, 3]
print("First two columns:\n", array[:, :2])
Boolean Masking

Boolean Masks: Filter arrays based on conditions.

Example: Select values greater than 4 with array > 4
Using Masks: Pass the mask directly to filter the array.

mask = array > 4

print("Boolean Mask:\n", mask)
print("Values > 4:", array[mask]) # Output: [5, 6, 7, 8, 9]
Reshaping Arrays
Reshape: Change the shape of an array without changing its data.
Example: Convert 1D array to 2D: [Link](rows, cols)
Important: Total elements must stay the same.

array_1d = [Link](1, 10) # Array: [1, 2, ..., 9]

array_reshaped = array_1d.reshape(3, 3)
print("Reshaped Array:\n", array_reshaped)
Example: Array Operations
# Create a 1D array and reshape it
data = [Link](1, 10)
matrix = [Link](3, 3)

# Boolean masking
greater_than_five = matrix[matrix > 5]
print("Elements > 5:", greater_than_five)

# Slicing specific rows and columns

first_two_rows = matrix[:2, :]
print("First two rows:\n", first_two_rows)