Image Processing Lab Scripts

Python Scripts + Easy Explanation

This PDF explains each lab script. For every script, you will find the script code, what the script does, its input and output,
and the important functions/instructions.
Important note: code inside triple quotes (""" ... """) is commented, so it does not run unless the quotes are removed. Some
emoji characters in print messages were changed to plain text to keep the PDF readable.

Scripts included
1. LabML17_CNN.py
2. LabML18_CNN2.py
3. LabML19_CNN3real.py
4. LabML20_CNN_aug_real.py
5. OpCV12_pointDet.py
6. OpCV13_siftdetandmatch.py
7. OpCV14_FaceAndPesondetcam.py
8. OpCV15_calibrate.py
9. OpCV16_depthestimation.py
10. OpCV16_depth_face_person.py
11. OpCV17_point_track.py
12. OPCV18_object_tracking.py
13. [Link]

Image Processing Lab Scripts - Explanation Page 1

 1. LabML17_CNN.py
Script code
001:
002:
003: from [Link] import cifar10,cifar100,mnist,fashion_mnist
004: import tensorflow as tf
005: import numpy as np
006: import keras
007: from [Link] import Sequential,Model
008: from [Link] import Dense, Dropout, Flatten,Conv2D, MaxPooling2D
009: import [Link] as plt
010: import pandas as pd
011: from [Link] import load_model
012:
013: #Data base 1: read database 100 classes cifar100
014:
015: class_names = [
016: "apple",
017: "aquarium_fish",
018: "baby",
019: "bear",
020: "beaver",
021: "bed",
022: "bee",
023: "beetle",
024: "bicycle",
025: "bottle",
026: "bowl",
027: "boy",
028: "bridge",
029: "bus",
030: "butterfly",
031: "camel",
032: "can",
033: "castle",
034: "caterpillar",
035: "cattle",
036: "chair",
037: "chimpanzee",
038: "clock",
039: "cloud",
040: "cockroach",
041: "couch",
042: "crab",
043: "crocodile",
044: "cup",
045: "dinosaur",
046: "dolphin",
047: "elephant",
048: "flatfish",
049: "forest",
050: "fox",
051: "girl",
052: "hamster",
053: "house",
054: "kangaroo",
055: "keyboard",
056: "lamp",
057: "lawn_mower",
058: "leopard",
059: "lion",
060: "lizard",
061: "lobster",
062: "man",
063: "maple_tree",
064: "motorcycle",
065: "mountain",
066: "mouse",
067: "mushroom",
068: "oak_tree",
069: "orange",
070: "orchid",
071: "otter",
072: "palm_tree",
073: "pear",
074: "pickup_truck",
075: "pine_tree",
076: "plain",
077: "plate",
078: "poppy",
079: "porcupine",
080: "possum",
081: "rabbit",
082: "raccoon",
083: "ray",
084: "road",
085: "rocket",
086: "rose",
087: "sea",
088: "seal",
089: "shark",
090: "shrew",
091: "skunk",
092: "skyscraper",
093: "snail",
094: "snake",
095: "spider",
096: "squirrel",
097: "streetcar",
098: "sunflower",
099: "sweet_pepper",

Image Processing Lab Scripts - Explanation Page 2

 100: "table",
101: "tank",
102: "telephone",
103: "television",
104: "tiger",
105: "tractor",
106: "train",
107: "trout",
108: "tulip",
109: "turtle",
110: "wardrobe",
111: "whale",
112: "willow_tree",
113: "wolf",
114: "woman",
115: "worm"]
116:
117: (train_images, train_labels), (test_images, test_labels) = cifar100.load_data()
118: print(train_images.shape)
119: print(test_images.shape)
120: print(train_labels)
121:
122: # Normalize pixel values to be between 0 and 1
123: train_images, test_images = train_images / 255.0, test_images / 255.0
124:
125: [Link](figsize=(10, 10))
126:
127: #print first 25 images from training data
128: for i in range(25):
129: [Link](5, 5, i + 1)
130: [Link]([])
131: [Link]([])
132: [Link](False)
133: [Link](train_images[i])
134: # The CIFAR labels happen to be arrays,
135: # which is why you need the extra index
136: [Link](class_names[train_labels[i][0]])
137: [Link]()
138:
139: """
140:
141: #Data base 2: read database 10 classes cifar
142: (train_images, train_labels), (test_images, test_labels) = cifar10.load_data()
143: print(train_images.shape)
144: print(test_images.shape)
145: print(train_labels)
146:
147: # Normalize pixel values to be between 0 and 1
148: train_images, test_images = train_images / 255.0, test_images / 255.0
149: class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',
150: 'dog', 'frog', 'horse', 'ship', 'truck']
151: [Link](figsize=(10, 10))
152:
153: #print first 25 images from training data
154: for i in range(25):
155: [Link](5, 5, i + 1)
156: [Link]([])
157: [Link]([])
158: [Link](False)
159: [Link](train_images[i])
160: # The CIFAR labels happen to be arrays,
161: # which is why you need the extra index
162: [Link](class_names[train_labels[i][0]])
163: [Link]()
164:
165: #print second 25 from training data
166: [Link](figsize=(10, 10))
167: j=0
168: for i in range(25,50,1):
169: j=j+1
170: [Link](5, 5, j)
171: [Link]([])
172: [Link]([])
173: [Link](False)
174: [Link](train_images[i])
175: [Link](class_names[train_labels[i][0]])
176: [Link]()
177: """
178: """
179: #Database 3 read database Minst digits
180: (train_images, train_labels), (test_images, test_labels) = mnist.load_data()
181: print(train_images.shape)
182: print(test_images.shape)
183: print(train_labels.shape)
184:
185: # Normalize pixel values to be between 0 and 1
186: train_images, test_images = train_images / 255.0, test_images / 255.0
187: class_names = ['0', '1', '2', '3', '4','5', '6', '7', '8', '9']
188: [Link](figsize=(7, 7))
189:
190: #print first 25 images from training data
191: for i in range(25):
192: [Link](5, 5, i + 1)
193: [Link]([])
194: [Link]([])
195: [Link](False)
196: [Link](train_images[i],'gray')
197: # The CIFAR labels happen to be arrays,
198: # which is why you need the extra index
199: [Link](class_names[train_labels[i]])
200: [Link]()
201:
202: #print second 25 from training data
203: [Link](figsize=(7, 7))
204: j=0

Image Processing Lab Scripts - Explanation Page 3

 205: for i in range(25,50,1):
206: j=j+1
207: [Link](5, 5, j)
208: [Link]([])
209: [Link]([])
210: [Link](False)
211: [Link](train_images[i],'gray')
212: [Link](class_names[train_labels[i]])
213: [Link]()
214: """
215:
216: #define the CNN
217: batch_size=32
218: NUM_CLASSES = len(class_names)
219:
220: model = Sequential()
221: [Link](Conv2D(30, (3, 3), activation='relu', input_shape=(32, 32, 3),padding='same')) #(3*3*1)*30+30=300
222: #put input_shape=(28, 28,1) for Mnist digit : #[Link](Conv2D(10, (3, 3), activation='relu', input_shape=(32,
32, 3),padding='same'))
223: #put input_shape=(32, 32,3) for color :
224: [Link](MaxPooling2D((2, 2))) #30*14*14
225: [Link](Conv2D(20, (3, 3), activation='relu',padding='same',strides=1)) #(3*3*30)*20+20=5420
226: [Link](MaxPooling2D((2, 2))) #20*7*7
227: [Link](Conv2D(30, (3, 3), activation='relu',padding='same'))#(3*3*20*30)+30=5430
228: [Link](MaxPooling2D((2, 2))) #30*3*3=
229: [Link](Flatten()) #30*3*3=270
230: [Link](Dense(20, activation='relu')) #270*20+20=5420
231: [Link](Dense(30, activation='relu'))#30*20+30=630
232: [Link](Dense(10, activation='relu'))#30*10+10=310
233: [Link](Dense(NUM_CLASSES, activation='softmax'))#10*10+10=110
234:
235: [Link]()
236:
237: [Link](optimizer='adam',
238: loss=[Link](from_logits=True),
239: metrics=['accuracy'])
240:
241: outclf = [Link](train_images, train_labels,
242: epochs=10,
243: validation_data=(test_images, test_labels),
244: batch_size = batch_size,
245: verbose = 1)
246:
247: [Link]([Link]['accuracy'], label='accuracy')
248: [Link]('Epoch')
249: [Link]('Accuracy')
250: [Link]([0.5, 1])
251: [Link](loc='lower right')
252:
253: test_loss, test_acc = [Link](test_images, test_labels, verbose=2)
254: print(test_loss)
255: print(test_acc)
256:
257: metrics_df = [Link]([Link])
258: metrics_df[["loss","val_loss"]].plot();
259: metrics_df[["accuracy","val_accuracy"]].plot();
260: [Link]()
261:
262: #to save the model in the comouter
263: [Link]("cnn_model.h5")
264:
265: #to read the model and try it:
266: model = load_model("cnn_model.h5")
267:

What this script does

- Loads an image dataset, mainly CIFAR-100, and prepares it for image classification.
- Displays example training images with their class names.
- Builds a simple CNN with convolution, max-pooling, flatten, and dense layers.
- Trains the CNN, evaluates it on the test set, plots loss/accuracy, and saves the model.
- There are commented parts for CIFAR-10 and MNIST. These parts do not run unless the triple quotes are removed.

Input and output

- Input: CIFAR-100 images from Keras. Optional commented inputs: CIFAR-10 and MNIST.
- Output: printed dataset shapes, displayed sample images, training/validation curves, test loss, test accuracy, and saved
model cnn_model.h5.

Important functions and instructions

- cifar100.load_data(): downloads/loads CIFAR-100 and returns train and test images with labels.
- train_images / 255.0: normalizes pixel values from 0-255 to 0-1 so the CNN trains better.
- [Link](), [Link](), [Link](): display many example images in a grid with labels.
- Sequential(): creates a neural network layer by layer.
- Conv2D(): applies convolution filters to learn image features such as edges and textures.

Image Processing Lab Scripts - Explanation Page 4

 - MaxPooling2D(): reduces feature map size and keeps the strongest information.
- Flatten(): changes 2D/3D feature maps into a 1D vector before dense layers.
- Dense(): fully connected layer used for final classification.
- softmax: converts final outputs into class probabilities.
- [Link](): chooses optimizer, loss function, and metric.
- [Link](): trains the CNN on the training images.
- [Link](): computes loss and accuracy on test images.
- [Link]([Link]): converts training history to a table for easy plotting.
- [Link]() and load_model(): save and reload the trained CNN.
- Important note: the model uses softmax, so from_logits=True is not ideal. Usually from_logits should be False when the last
layer is softmax.

Image Processing Lab Scripts - Explanation Page 5

 2. LabML18_CNN2.py
Script code
001: from [Link] import fashion_mnist
002: import tensorflow as tf
003: import numpy as np
004: from [Link] import to_categorical
005: import [Link] as plt
006: import keras
007: from [Link] import Sequential,Model
008: from [Link] import Dense, Dropout, Flatten
009: from [Link] import Conv2D, MaxPooling2D
010: from sklearn.model_selection import train_test_split
011: from [Link] import classification_report,confusion_matrix, ConfusionMatrixDisplay
012:
013: (train_X,train_Y), (test_X,test_Y) = fashion_mnist.load_data()
014:
015: #matplotlib inline
016:
017: print('Training data shape : ', train_X.shape, train_Y.shape)
018: print('Testing data shape : ', test_X.shape, test_Y.shape)
019: # Find the unique numbers from the train labels
020: classes = [Link](train_Y)
021: nClasses = len(classes)
022: print('Total number of outputs : ', nClasses)
023: print('Output classes : ', classes)
024: #('Total number of outputs : ', 10)
025: #('Output classes : ', array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=uint8))
026: [Link](figsize=[5,5])
027:
028: # Display the first image in training data
029: [Link](121)
030: [Link](train_X[6,:,:], cmap='gray')
031: [Link]("Ground Truth : {}".format(train_Y[7]))
032:
033: # Display the second image in testing data
034: [Link](122)
035: [Link](test_X[10,:,:], cmap='gray')
036: [Link]("Ground Truth : {}".format(test_Y[15]))
037: #Text(0.5,1,u'Ground Truth : 9')
038: [Link]()
039:
040: train_X,valid_X,train_label,valid_label = train_test_split(train_X, train_Y, test_size=0.2, random_state=13)
041:
042: print(train_X.shape,valid_X.shape,train_label.shape,valid_label.shape)
043:
044:
045: #from [Link] import BatchNormalization
046: #from [Link].advanced_activations import LeakyReLU
047:
048: batch_size = 64
049: epochs = 5
050: num_classes = nClasses
051: clf = Sequential()
052: [Link](Conv2D(32, kernel_size=(3, 3),activation='relu',input_shape=(28,28,1),padding='same'))
053: [Link](MaxPooling2D((2, 2),padding='same'))
054: [Link](Conv2D(64, (3, 3), activation='relu',padding='same'))
055: [Link](MaxPooling2D(pool_size=(2, 2),padding='same'))
056: [Link](Conv2D(128, (3, 3), activation='relu',padding='same'))
057: [Link](MaxPooling2D(pool_size=(2, 2),padding='same'))
058: [Link](Flatten())
059: [Link](Dense(128, activation='relu'))
060: #[Link](Dropout(0.3))
061: [Link](Dense(num_classes, activation='softmax'))
062:
063: [Link]()
064: #for this we must use to catagorical way:0100000 100000...
065: #[Link](loss=[Link].categorical_crossentropy,
066: # optimizer=[Link](),
067: # metrics=['accuracy'])
068:
069: [Link](optimizer='adam',
070: loss=[Link](from_logits=True),
071: metrics=['accuracy'])
072:
073: clf_trained = [Link](train_X, train_label,
074: batch_size=batch_size,
075: epochs=epochs,
076: verbose=1,
077: validation_data=(valid_X, valid_label))
078:
079: #By setting verbose 0, 1 or 2 you just say how do you want to 'see' the training progress for each epoch.
080: #verbose=0 will show you nothing (silent)
081: #verbose=1 will show you an animated progress bar like this:
082: #progres_bar
083: #verbose=2 will just mention the number of epoch like this:
084: #enter image description here
085:
086: #The batch size is a number of samples processed before the model is updated.
087: # The number of epochs is the number of complete passes through the training dataset.
088: # The size of a batch must be more than or equal to one and less than
089: # or equal to the number of samples in the training dataset
090:
091: #you can save then open
092: [Link]('model.h5')
093:
094: #load_saved_model = [Link].load_model("model.h5")
095: #load_saved_model.summary()
096:
097: test_eval = [Link](test_X, test_Y, verbose=0)
098: print('Test loss:', test_eval[0])
099: print('Test accuracy:', test_eval[1])

Image Processing Lab Scripts - Explanation Page 6

 100:
101: accuracy = clf_trained.history['accuracy']
102: val_accuracy = clf_trained.history['val_accuracy']
103: loss = clf_trained.history['loss']
104: val_loss = clf_trained.history['val_loss']
105: epochs = range(len(accuracy))
106: [Link](epochs, accuracy, 'bo', label='Training accuracy')
107: [Link](epochs, val_accuracy, 'b', label='Validation accuracy')
108: [Link]('Training and validation accuracy')
109: [Link]()
110: [Link]()
111: [Link](epochs, loss, 'bo', label='Training loss')
112: [Link](epochs, val_loss, 'b', label='Validation loss')
113: [Link]('Training and validation loss')
114: [Link]()
115: [Link]()
116:
117: #The lower the loss, the better a model (unless the model has over-fitted to the training data).
118: # The loss is calculated on training(loss) and validation(val_loss) and its interperation is how well the model
119: # is doing for these two sets. Unlike accuracy, loss is not a percentage.
120: # It is a summation of the errors made for each example in training or validation sets.
121:
122: #Put the test data at the input then calculate the corresponding classes and classification report
123: predicted_classes =[Link](test_X)
124: predicted_classes = [Link]([Link](predicted_classes),axis=1)
125:
126:
127: target_names = ["Class {}".format(i) for i in range(num_classes)]
128: print(classification_report(test_Y, predicted_classes, target_names=target_names))
129:
130: #print confusion matrix
131: cm = confusion_matrix(test_Y, predicted_classes)
132: print(cm)
133: cmd = ConfusionMatrixDisplay(confusion_matrix=cm)
134: [Link](include_values=True, cmap='viridis', ax=None, xticks_rotation='horizontal')
135: [Link]()

What this script does

- Loads the Fashion-MNIST dataset and trains a CNN to classify clothes into 10 classes.
- Shows example grayscale images from the train and test data.
- Splits the training data into training and validation sets.
- Builds, trains, evaluates, and saves a CNN model.
- Prints a classification report and displays a confusion matrix.

Input and output

- Input: Fashion-MNIST images, 28x28 grayscale, with labels from 0 to 9.
- Output: trained model model.h5, test loss/accuracy, accuracy/loss plots, classification report, and confusion matrix.

Important functions and instructions

- fashion_mnist.load_data(): loads the clothing image dataset.
- [Link](): finds all different labels/classes in the dataset.
- train_test_split(): separates part of the training data for validation.
- Conv2D(), MaxPooling2D(), Flatten(), Dense(): build the CNN architecture.
- [Link](): prints the layer structure and number of parameters.
- SparseCategoricalCrossentropy(): loss used when labels are integers like 0,1,2,...,9.
- [Link](): trains the model.
- [Link](): saves the trained model.
- [Link](): tests the model on test images.
- [Link](): gives prediction probabilities for test images.
- [Link](): converts probability vectors into predicted class numbers.
- classification_report(): gives precision, recall, and f1-score for each class.
- confusion_matrix(): counts correct and incorrect predictions between classes.
- Important note: Conv2D expects images with a channel dimension. It is safer to reshape train_X and test_X to (N, 28, 28, 1).
- Important note: the last layer uses softmax, so from_logits=True should usually be False.

Image Processing Lab Scripts - Explanation Page 7

 3. LabML19_CNN3real.py
Script code
001: import os
002: import pathlib
003: import numpy as np
004: import pandas as pd
005: import [Link] as plt
006: import tensorflow as tf
007: from matplotlib import image as mpimg
008: from [Link] import RMSprop
009: import tensorflow as tf
010: from [Link] import ImageDataGenerator
011: from [Link] import layers, models
012: """
013: # Part 1: Classification if we have the classes and their train and test folders:
014: # Folder cats_and_dogs_filtered: Train and Test folders: Dog and cats folders
015:
016: #- Read pathes of train and Validation
017: base_dir = 'cats_and_dogs_filtered'
018: train_dir = [Link](base_dir, 'train')
019: validation_dir = [Link](base_dir, 'validation')
020:
021: # Directory with our training cat pictures
022: train_cats_dir = [Link](train_dir, 'cats')
023:
024: # Directory with our training dog pictures
025: train_dogs_dir = [Link](train_dir, 'dogs')
026:
027: # Directory with our validation cat pictures
028: validation_cats_dir = [Link](validation_dir, 'cats')
029:
030: # Directory with our validation dog pictures
031: validation_dogs_dir = [Link](validation_dir, 'dogs')
032:
033: #Plot the first 4 cats in train cats and first 4 dogs in dog train directory
034: # Grab the filenames of the first 4 cats and 4 dogs
035: cat_fnames = [Link](train_cats_dir)[:4]
036: dog_fnames = [Link](train_dogs_dir)[:4]
037:
038: # Set up a 2-row, 4-column figure
039: fig = [Link](figsize=(12, 6))
040:
041: # Plot the 4 cats on the top row
042: for i, fname in enumerate(cat_fnames):
043: img_path = [Link](train_cats_dir, fname)
044: img = [Link](img_path)
045: ax = fig.add_subplot(2, 4, i + 1) # 2 rows, 4 columns, position i+1
046: [Link](img)
047: [Link]('off')
048: ax.set_title('Original Cat')
049:
050: # Plot the 4 dogs on the bottom row
051: for i, fname in enumerate(dog_fnames):
052: img_path = [Link](train_dogs_dir, fname)
053: img = [Link](img_path)
054: ax = fig.add_subplot(2, 4, i + 5) # Start at position 5 for the bottom row
055: [Link](img)
056: [Link]('off')
057: ax.set_title('Original Dog')
058:
059: plt.tight_layout()
060: [Link]()
061:
062:
063: # All images will be rescaled by 1./255
064: train_datagen = ImageDataGenerator(rescale=1./255)
065: test_datagen = ImageDataGenerator(rescale=1./255)
066:
067: # Flow training images in batches of 20 using train_datagen generator
068: train_generator = train_datagen.flow_from_directory(
069: train_dir, # This is the source directory for training images
070: target_size=(150, 150), # All images will be resized to 150x150
071: batch_size=20,
072: # Since we use binary_crossentropy loss, we need binary labels
073: class_mode='binary')
074:
075: # Flow validation images in batches of 20 using test_datagen generator
076: validation_generator = test_datagen.flow_from_directory(
077: validation_dir,
078: target_size=(150, 150),
079: batch_size=20,
080: class_mode='binary')
081:
082: #Create the CNN Model
083: model = [Link]([
084: # Add the explicit Input layer here
085: [Link](shape=(150, 150, 3)),
086:
087: # Remove input_shape from the Conv2D layer
088: [Link].Conv2D(32, (3, 3), activation='relu'),
089: [Link].MaxPooling2D(2, 2),
090: [Link].Conv2D(64, (3, 3), activation='relu'),
091: [Link].MaxPooling2D(2, 2),
092: [Link].Conv2D(128, (3, 3), activation='relu'),
093: [Link].MaxPooling2D(2, 2),
094: [Link].Conv2D(128, (3, 3), activation='relu'),
095: [Link].MaxPooling2D(2, 2),
096: [Link](),
097: [Link](512, activation='relu'),
098: [Link](1, activation='sigmoid') #one output-->use loss=binarycrossentropy
099: ])

Image Processing Lab Scripts - Explanation Page 8

 100: [Link](loss='binary_crossentropy',
101: optimizer=RMSprop(learning_rate=1e-4),
102: metrics=['acc'])
103:
104: history = [Link](
105: train_generator,
106: steps_per_epoch=100, # 2000 images = batch_size * steps
107: epochs=8,
108: validation_data=validation_generator,
109: validation_steps=50, # 1000 images = batch_size * steps
110: verbose=1
111: )
112:
113: #visualise the results
114:
115: import [Link] as plt
116:
117: acc = [Link]['acc']
118: val_acc = [Link]['val_acc']
119: loss = [Link]['loss']
120: val_loss = [Link]['val_loss']
121:
122: epochs = range(len(acc))
123:
124: [Link](epochs, acc, 'bo', label='Training accuracy')
125: [Link](epochs, val_acc, 'b', label='Validation accuracy')
126: [Link]('Training and validation accuracy')
127:
128: [Link]()
129:
130: [Link](epochs, loss, 'bo', label='Training Loss')
131: [Link](epochs, val_loss, 'b', label='Validation Loss')
132: [Link]('Training and validation loss')
133: [Link]()
134:
135: [Link]()
136:
137: #save the model so we can use it later
138: [Link]("cats_and_dogs_filtered/cat_dog_model.keras")
139: """
140: """
141: #load the model from the directory and use it
142: import tensorflow as tf
143: model = [Link].load_model("cats_and_dogs_filtered/cat_dog_model.keras")
144: [Link]() # optional
145:
146: #Test the model on one image:
147: import numpy as np
148: import [Link] as plt
149: from [Link] import load_img, img_to_array
150:
151: img_height = 150
152: img_width = 150
153:
154: def predict_image(img_path):
155: img = load_img(img_path, target_size=(img_height, img_width))
156:
157: [Link](img)
158: [Link]("off")
159: [Link]()
160:
161: img_array = img_to_array(img) / 255.0 # IMPORTANT normalization
162: img_array = np.expand_dims(img_array, axis=0)
163:
164: prediction = [Link](img_array)
165: print (prediction)
166:
167: class_names = ["Cat", "Dog"]
168: imclass=int((prediction>=0.5))
169: print(imclass)
170:
171: predicted_class = class_names[imclass]
172: confidence = prediction * 100
173:
174: print("Prediction:", predicted_class)
175: print("Confidence:", round(confidence, 2), "%")
176:
177:
178: # Test
179: predict_image("cats_and_dogs_filtered/train/cats/[Link]")
180:
181: """
182:
183: #Part 2: CNN on folder containing only classes Cat and dogs.
184: # =========================
185: # 1. Dataset path
186: # =========================
187: base_dir = "Petimages" # contains Cat and Dog folders
188:
189: img_height = 100
190: img_width = 100
191: batch_size = 32
192:
193: # =========================
194: # 2. Remove corrupted images
195: # =========================
196: data_dir = [Link](base_dir)
197:
198: for file_path in data_dir.glob("*/*"):
199: try:
200: img = [Link].read_file(str(file_path))
201: [Link].decode_image(img, channels=3)
202: except:
203: print("Removing corrupted image:", file_path)
204: [Link](file_path)

Image Processing Lab Scripts - Explanation Page 9

 205:
206: # =========================
207: # 3. Load dataset
208: # =========================
209: train_ds = [Link].image_dataset_from_directory(
210: base_dir,
211: validation_split=0.1,
212: subset="training",
213: seed=123,
214: image_size=(img_height, img_width),
215: batch_size=batch_size,
216: color_mode="rgb"
217: )
218:
219: val_ds = [Link].image_dataset_from_directory(
220: base_dir,
221: validation_split=0.1,
222: subset="validation",
223: seed=123,
224: image_size=(img_height, img_width),
225: batch_size=batch_size,
226: color_mode="rgb"
227: )
228:
229: class_names = train_ds.class_names
230: print("Classes:", class_names)
231:
232: # Improve performance
233: AUTOTUNE = [Link]
234:
235: train_ds = train_ds.cache().shuffle(1000).prefetch(buffer_size=AUTOTUNE)
236: val_ds = val_ds.cache().prefetch(buffer_size=AUTOTUNE)
237:
238: # =========================
239: # 4. CNN model
240: # =========================
241: model = [Link]([
242: [Link](1./255, input_shape=(img_height, img_width, 3)),
243:
244: layers.Conv2D(10, (3, 3), activation="relu"),
245: layers.MaxPooling2D(),
246:
247: layers.Conv2D(10, (3, 3), activation="relu"),
248: layers.MaxPooling2D(),
249:
250: layers.Conv2D(10, (3, 3), activation="relu"),
251: layers.MaxPooling2D(),
252:
253: [Link](),
254:
255: [Link](20, activation="relu"),
256: [Link](0.3),
257:
258: [Link](10, activation="relu"),
259: [Link](0.2),
260:
261: [Link](2, activation="softmax")
262: ])
263:
264: [Link]()
265:
266: # =========================
267: # 5. Compile
268: # =========================
269: [Link](
270: optimizer="adam",
271: loss=[Link](),
272: metrics=["accuracy"]
273: )
274:
275: # =========================
276: # 6. Train
277: # =========================
278: history = [Link](
279: train_ds,
280: validation_data=val_ds,
281: epochs=10
282: )
283:
284: # =========================
285: # 7. Plot training curves
286: # =========================
287: history_df = [Link]([Link])
288:
289: [Link]()
290: history_df[["loss", "val_loss"]].plot()
291: [Link]("Loss Curve")
292: [Link]("Epoch")
293: [Link]("Loss")
294: [Link]()
295:
296: [Link]()
297: history_df[["accuracy", "val_accuracy"]].plot()
298: [Link]("Accuracy Curve")
299: [Link]("Epoch")
300: [Link]("Accuracy")
301: [Link]()
302:
303: # =========================
304: # 8. Save model
305: # =========================
306: [Link]("cat_dog_classifier.keras")
307:
308: # =========================
309: # 9. Predict one image

Image Processing Lab Scripts - Explanation Page 10

 310: # =========================
311: def predict_image(img_path):
312: img = [Link].load_img(
313: img_path,
314: target_size=(img_height, img_width)
315: )
316:
317: [Link](img)
318: [Link]("off")
319: [Link]()
320:
321: img_array = [Link].img_to_array(img)
322: img_array = np.expand_dims(img_array, axis=0)
323:
324: prediction = [Link](img_array)
325: predicted_index = [Link](prediction[0])
326: predicted_class = class_names[predicted_index]
327: confidence = prediction[0][predicted_index] * 100
328:
329: print("Prediction probabilities:", prediction)
330: print("Predicted class:", predicted_class)
331: print("Confidence:", round(confidence, 2), "%")
332:
333:
334: # Example predictions
335: predict_image("Petimages/Cat/[Link]")
336: predict_image("PetImages/Dog/[Link]")

What this script does

- Builds a real image classifier for cats and dogs using folders of images.
- The first large part is commented and shows another way using cats_and_dogs_filtered train/validation folders.
- The active part uses a Petimages folder that contains Cat and Dog subfolders.
- It removes corrupted images, creates training and validation datasets, trains a CNN, plots curves, saves the model, and
predicts single images.

Input and output

- Input: image folder Petimages with class subfolders such as Cat and Dog.
- Output: cleaned dataset, trained model cat_dog_classifier.keras, loss/accuracy plots, and printed predictions with
confidence.

Important functions and instructions

- [Link](base_dir).glob("*/*"): loops over all image files in class folders.
- [Link].read_file() and [Link].decode_image(): test whether an image can be read correctly.
- [Link](): deletes corrupted images.
- image_dataset_from_directory(): creates a TensorFlow dataset from class folders automatically.
- validation_split and subset: split the folder data into training and validation parts.
- cache(), shuffle(), prefetch(): improve loading speed during training.
- [Link](1./255): normalizes pixels inside the model.
- Conv2D and MaxPooling2D: learn visual features and reduce size.
- Dropout(): randomly disables some neurons during training to reduce overfitting.
- SparseCategoricalCrossentropy(): used because the labels are integer class indices.
- [Link](): trains the network.
- [Link]([Link]): stores loss and accuracy values for plotting.
- [Link](): saves the trained model in Keras format.
- load_img() and img_to_array(): load a single image and convert it to a numerical array.
- np.expand_dims(): adds the batch dimension, so one image becomes shape (1, height, width, channels).
- [Link](): selects the class with the highest probability.
- Important note: the script uses Petimages and PetImages in different lines. On some systems, folder names are
case-sensitive.

Image Processing Lab Scripts - Explanation Page 11

 4. LabML20_CNN_aug_real.py
Script code
001: import os
002: import pathlib
003: import numpy as np
004: import pandas as pd
005: import [Link] as plt
006: import tensorflow as tf
007: from matplotlib import image as mpimg
008: from [Link] import RMSprop
009: import tensorflow as tf
010: from [Link] import ImageDataGenerator
011: from [Link] import layers, models
012:
013:
014: # Part 1: Classification if we have the classes and their train and test folders:
015: # Folder cats_and_dogs_filtered: Train and Test folders: Dog and cats folders
016:
017: #- Read pathes of train and Validation
018: base_dir = 'cats_and_dogs_filtered'
019: train_dir = [Link](base_dir, 'train')
020: validation_dir = [Link](base_dir, 'validation')
021:
022: # Directory with our training cat pictures
023: train_cats_dir = [Link](train_dir, 'cats')
024:
025: # Directory with our training dog pictures
026: train_dogs_dir = [Link](train_dir, 'dogs')
027:
028: # Directory with our validation cat pictures
029: validation_cats_dir = [Link](validation_dir, 'cats')
030:
031: # Directory with our validation dog pictures
032: validation_dogs_dir = [Link](validation_dir, 'dogs')
033:
034:
035: """
036: #Part 1: Design of CNN for data augmentation
037: model_data_aug = [Link]([
038: # Add the explicit Input layer here
039: [Link](shape=(150, 150, 3)),
040: [Link].Conv2D(32, (3, 3), activation='relu'),
041: [Link].MaxPooling2D(2, 2),
042: [Link].Conv2D(64, (3, 3), activation='relu'),
043: [Link].MaxPooling2D(2, 2),
044: [Link].Conv2D(128, (3, 3), activation='relu'),
045: [Link].MaxPooling2D(2, 2),
046: [Link].Conv2D(128, (3, 3), activation='relu'),
047: [Link].MaxPooling2D(2, 2),
048: [Link](),
049: [Link](512, activation='relu'),
050: [Link](1, activation='sigmoid')
051: ])
052:
053: #Using ImageDataGenerator for data augmentation
054:
055: # This code has changed. Now instead of the ImageGenerator just rescaling
056: # the image, we also rotate and do other operations
057: # Updated to do image augmentation
058: train_datagen = ImageDataGenerator(
059: rescale=1. / 255,
060: rotation_range=40,
061: width_shift_range=0.2,
062: height_shift_range=0.2,
063: shear_range=0.2,
064: zoom_range=0.2,
065: horizontal_flip=True,
066: fill_mode='nearest')
067:
068: test_datagen = ImageDataGenerator(rescale=1. / 255)
069:
070: # Flow training images in batches of 20 using train_datagen generator
071: train_generator = train_datagen.flow_from_directory(
072: train_dir, # This is the source directory for training images
073: target_size=(150, 150), # All images will be resized to 150x150
074: batch_size=20,
075: # Since we use binary_crossentropy loss, we need binary labels
076: class_mode='binary')
077:
078: # Flow validation images in batches of 20 using test_datagen generator
079: validation_generator = test_datagen.flow_from_directory(
080: validation_dir,
081: target_size=(150, 150),
082: batch_size=20,
083: class_mode='binary')
084:
085: import [Link] as plt
086:
087: # Grab one batch (20 images and labels) from your new training generator
088: augmented_images, augmented_labels = next(train_generator)
089:
090: # Set up a 3x3 grid to plot 9 of the images
091: [Link](figsize=(15, 15))
092: for i in range(20):
093: [Link](5, 4, i + 1)
094:
095: # Display the image (generator already rescaled pixels to 0-1, which Matplotlib loves)
096: [Link](augmented_images[i])
097: [Link]('off')
098:
099: plt.tight_layout()

Image Processing Lab Scripts - Explanation Page 12

 100: [Link]()
101:
102:
103: model_data_aug.compile(loss='binary_crossentropy',
104: optimizer=RMSprop(learning_rate=1e-4),
105: metrics=['acc'])
106:
107: history = model_data_aug.fit(
108: train_generator,
109: steps_per_epoch=100, # 2000 images = batch_size * steps
110: epochs=3,
111: validation_data=validation_generator,
112: validation_steps=50, # 1000 images = batch_size * steps
113: verbose=1)
114:
115: import [Link] as plt
116: acc = [Link]['acc']
117: val_acc = [Link]['val_acc']
118: loss = [Link]['loss']
119: val_loss = [Link]['val_loss']
120:
121: epochs = range(len(acc))
122:
123: [Link](epochs, acc, 'bo', label='Training accuracy')
124: [Link](epochs, val_acc, 'b', label='Validation accuracy')
125: [Link]('Training and validation accuracy')
126:
127: [Link]()
128:
129: [Link](epochs, loss, 'bo', label='Training Loss')
130: [Link](epochs, val_loss, 'b', label='Validation Loss')
131: [Link]('Training and validation loss')
132: [Link]()
133:
134: [Link]()
135:
136: """
137:
138:
139: # Part 2: Using drop outs and data augmentation
140: model_drop_out = [Link]([
141: # Add the explicit Input layer here
142: [Link](shape=(150, 150, 3)),
143: [Link].Conv2D(32, (3,3), activation='relu'),
144: [Link].MaxPooling2D(2, 2),
145: [Link].Conv2D(64, (3,3), activation='relu'),
146: [Link].MaxPooling2D(2,2),
147: [Link].Conv2D(128, (3,3), activation='relu'),
148: [Link].MaxPooling2D(2,2),
149: [Link].Conv2D(128, (3,3), activation='relu'),
150: [Link].MaxPooling2D(2,2),
151: [Link](0.5), #Adding Dropout
152: [Link](),
153: [Link](512, activation='relu'),
154: [Link](1, activation='sigmoid')
155: ])
156:
157: model_drop_out.compile(loss='binary_crossentropy',
158: optimizer=RMSprop(learning_rate=1e-4),
159: metrics=['acc'])
160:
161: # # This code has changed. Now instead of the ImageGenerator just rescaling
162: # # the image, we also rotate and do other operations
163: # # Updated to do image augmentation
164: train_datagen = ImageDataGenerator(
165: rescale=1./255,
166: rotation_range=40,
167: width_shift_range=0.2,
168: height_shift_range=0.2,
169: shear_range=0.2,
170: zoom_range=0.2,
171: horizontal_flip=True,
172: fill_mode='nearest')
173:
174: test_datagen = ImageDataGenerator(rescale=1./255)
175:
176: # Flow training images in batches of 20 using train_datagen generator
177: train_generator = train_datagen.flow_from_directory(
178: train_dir, # This is the source directory for training images
179: target_size=(150, 150), # All images will be resized to 150x150
180: batch_size=20,
181: # Since we use binary_crossentropy loss, we need binary labels
182: class_mode='binary')
183:
184: # Flow validation images in batches of 20 using test_datagen generator
185: validation_generator = test_datagen.flow_from_directory(
186: validation_dir,
187: target_size=(150, 150),
188: batch_size=20,
189: class_mode='binary')
190:
191: # Start training
192: history = model_drop_out.fit(
193: train_generator,
194: steps_per_epoch=100, # 2000 images = batch_size * steps
195: epochs=2,
196: validation_data=validation_generator,
197: validation_steps=50, # 1000 images = batch_size * steps
198: verbose=2)
199:
200:
201: import [Link] as plt
202: acc = [Link]['acc']
203: val_acc = [Link]['val_acc']
204: loss = [Link]['loss']

Image Processing Lab Scripts - Explanation Page 13

 205: val_loss = [Link]['val_loss']
206:
207: epochs = range(len(acc))
208:
209: [Link](epochs, acc, 'bo', label='Training accuracy')
210: [Link](epochs, val_acc, 'b', label='Validation accuracy')
211: [Link]('Training and validation accuracy')
212:
213: [Link]()
214:
215: [Link](epochs, loss, 'bo', label='Training Loss')
216: [Link](epochs, val_loss, 'b', label='Validation Loss')
217: [Link]('Training and validation loss')
218: [Link]()
219:
220: [Link]()
221:

What this script does

- Trains a cats/dogs CNN using image data augmentation.
- The first model, inside a commented block, shows augmentation without dropout.
- The active model uses data augmentation plus dropout to reduce overfitting.
- It reads images from train and validation folders, trains the CNN, and plots training curves.

Input and output

- Input: folder cats_and_dogs_filtered with train/cats, train/dogs, validation/cats, and validation/dogs.
- Output: live augmented images if the commented part is activated, training history, accuracy/loss plots, and a trained binary
classifier in memory.

Important functions and instructions

- [Link](): builds folder paths safely.
- ImageDataGenerator(): prepares image loading, rescaling, and augmentation.
- rescale=1./255: converts pixel values to 0-1.
- rotation_range, width_shift_range, height_shift_range, shear_range, zoom_range, horizontal_flip: create random changed
versions of images.
- flow_from_directory(): reads images from folders and assigns labels automatically.
- class_mode="binary": gives labels 0 or 1 for two classes.
- Dropout(0.5): reduces overfitting by turning off 50 percent of units during training.
- Dense(1, activation="sigmoid"): produces one probability for binary classification.
- binary_crossentropy: correct loss for a two-class sigmoid output.
- RMSprop: optimizer used to update the weights.
- steps_per_epoch and validation_steps: number of generator batches used in each epoch.
- [Link]: dictionary containing loss and accuracy values for plotting.

Image Processing Lab Scripts - Explanation Page 14

 5. OpCV12_pointDet.py
Script code
001: """
002: import cv2
003: import numpy as np
004: # Part1 1: Corner points
005: # Load image then grayscale
006: #image = [Link]('images/[Link]')
007: #image = [Link]('images/[Link]')
008: image = [Link]('images/[Link]')
009: gray = [Link](image, cv2.COLOR_BGR2GRAY)
010: [Link]('image originale ', image)
011:
012: # The cornerHarris function requires the array datatype to be float32
013: gray = np.float32(gray)
014:
015: harris_corners = [Link](gray, 9, 3, 0.05)
016: #print(harris_corners.shape)
017:
018: #[Link]('Harris Corners image', np.uint8(harris_corners))
019: #[Link](0)
020:
021: #We use dilation of the corner points to enlarge them\
022: kernel = [Link]((7,7),np.uint8)
023: harris_corners = [Link](harris_corners, kernel, iterations = 1)
024:
025: # Threshold for an optimal value, it may vary depending on the image.
026: image[harris_corners > 0.005 * harris_corners.max() ] = [255, 0, 0]
027:
028: [Link]('Harris Corners', image)
029: [Link](0)
030: [Link]()
031: """
032:
033: #Part 2 : SIFT Points
034: import cv2
035: import numpy as np
036:
037: #image = [Link]('images/[Link]')
038:
039: image = [Link]('images/[Link]')
040: #image = [Link]('images/[Link]')
041: #image = [Link]('images/[Link]')
042:
043: gray = [Link](image, cv2.COLOR_BGR2GRAY)
044:
045: #Create SIFT Feature Detector object
046: sift = cv2.SIFT_create()
047:
048: #Detect key points
049: keypoints = [Link](gray, None)
050: print("Number of keypoints Detected: ", len(keypoints))
051: keypoints, descriptors = [Link](gray, keypoints)
052: print(keypoints)
053: print([Link])
054: # drawKeypoints function is used to draw keypoints
055: output_image1 = [Link](gray, keypoints, 0, (0, 0, 255),
056: flags=cv2.DRAW_MATCHES_FLAGS_NOT_DRAW_SINGLE_POINTS)
057: [Link]('Feature Method - SIFT DRAW_MATCHES_FLAGS_NOT_DRAW_SINGLE_POINTS', output_image1)
058: [Link](0)
059: [Link]()
060:
061: # drawKeypoints function is used to draw keypoints
062: output_image2 = [Link](gray, keypoints, 0, (255, 0, 0),
063: flags=cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
064: [Link]('Feature Method - SIFT DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS', output_image2)
065: [Link](0)
066: [Link]()
067: print(descriptors[1,:])
068: #output_image3 = [Link](output_image2 , keypoints, 0, (0, 255, 0),
069: # flags=cv2.DRAW_MATCHES_FLAGS_DEFAULT)
070:
071: #[Link]('Feature Method - SIFT RAW_MATCHES_FLAGS_DEFAULT', output_image3)
072: #[Link](0)
073: #[Link]()
074:
075:
076: """
077: #part 3: Surf points
078: import cv2
079: import numpy as np
080:
081: image = [Link]('images/[Link]')
082: gray = [Link](image, cv2.COLOR_BGR2GRAY)
083:
084: #Create SURF Feature Detector object
085: surf = [Link]()
086: # Only features, whose hessian is larger than hessianThreshold are retained by the detector
087:
088: keypoints, descriptors = [Link](gray, None)
089: print("Number of keypoints Detected: ", len(keypoints))
090:
091: # Draw rich key points on input image
092: image = [Link](image, keypoints, flags=cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
093:
094: [Link]('Feature Method - SURF', image)
095: [Link]()
096: [Link]()
097: """
098:
099: """

Image Processing Lab Scripts - Explanation Page 15

 100: import cv2
101: import numpy as np
102:
103: image = [Link]('images/[Link]')
104: image = [Link]('images/[Link]')
105: gray = [Link](image, cv2.COLOR_BGR2GRAY)
106:
107: # Create ORB object, we can specify the number of key points we desire
108: #orb = [Link]()
109:
110: orb=cv2.ORB_create()
111: # Determine key points
112: keypoints = [Link](gray, None)
113: # Obtain the descriptors
114: keypoints, descriptors = [Link](gray, keypoints)
115: print("Number of keypoints Detected: ", len(keypoints))
116: print(keypoints)
117: print(descriptors)
118: # Draw rich keypoints on input image
119: image = [Link](image,
keypoints,outImage=1,color=[255,0,0],flags=cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
120:
121: [Link]('Feature Method - ORB', image)
122: [Link]()
123: [Link]()
124:
125: """

What this script does

- Detects interest points in an image.
- The active part detects SIFT keypoints and computes SIFT descriptors.
- Other parts are commented: Harris corners, SURF points, and ORB points.
- It draws the detected keypoints on the image and prints descriptor information.

Input and output

- Input: one image, for example images/[Link].
- Output: OpenCV windows showing keypoints, number of keypoints, descriptor matrix shape, and one descriptor vector.

Important functions and instructions

- [Link](): loads the image from disk.
- [Link](..., COLOR_BGR2GRAY): converts the image to grayscale for feature detection.
- cv2.SIFT_create(): creates a SIFT detector.
- [Link](): finds interest points/keypoints.
- [Link](): computes a descriptor vector around each keypoint.
- [Link](): draws keypoints on an output image.
- DRAW_MATCHES_FLAGS_NOT_DRAW_SINGLE_POINTS: draws only detected points in a simple way.
- DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS: draws richer keypoints with size and orientation.
- [Link](), [Link](), [Link](): display and close image windows.
- Important concept: a descriptor is a numerical vector that describes the local region around a keypoint.

Image Processing Lab Scripts - Explanation Page 16

 6. OpCV13_siftdetandmatch.py
Script code
001: import cv2
002: import numpy as np
003: import [Link] as plt
004: #[Link](False)
005: import warnings
006: [Link]('ignore')
007: """
008: #part 1: Sift detect and match
009: import cv2
010: import [Link] as plt
011: #matplotlib inline
012:
013: # read images
014: #img1 = [Link]('images/[Link]')
015: #img2 = [Link]('images/[Link]')
016:
017: img1 = [Link]('images/[Link]') # queryImage
018: img2 = [Link]('images/[Link]') # trainImage
019: [Link]('box',img1)
020: [Link]('scene',img2)
021: [Link]()
022: [Link]()
023: img1 = [Link](img1, cv2.COLOR_BGR2GRAY)
024: img2 = [Link](img2, cv2.COLOR_BGR2GRAY)
025:
026: #sift
027: sift = cv2.SIFT_create()
028:
029: keypoints_1, descriptors_1 = [Link](img1,None)
030: keypoints_2, descriptors_2 = [Link](img2,None)
031: print(descriptors_1.shape)
032: print(descriptors_2.shape)
033:
034: #feature matching
035: bf = [Link](cv2.NORM_L2, crossCheck=True) #method SSD
036: matches = [Link](descriptors_1,descriptors_2)
037: matches = sorted(matches, key = lambda x:[Link])
038:
039: img3 = [Link](img1, keypoints_1, img2, keypoints_2, matches[:100], img2, flags=2)
040: [Link](img3),[Link]()
041:
042: """
043: """
044: #part 2 Sift detect and mach with threshold
045: # import required libraries
046: import numpy as np
047: import cv2
048: import [Link] as plt
049:
050: # read two input images as grayscale
051: #img1 = [Link]('images/[Link]',0) # queryImage
052: #img2 = [Link]('images/bmw_rot.jpg',0) # trainImage
053:
054: img1 = [Link]('images/[Link]',0) # queryImage
055: img2 = [Link]('images/[Link]',0) # trainImage
056:
057: # Initiate SIFT detector
058: sift = cv2.SIFT_create()
059:
060: # detect and compute the keypoints and descriptors with SIFT
061: kp1, des1 = [Link](img1,None)
062: kp2, des2 = [Link](img2,None)
063: print([Link])
064:
065: # create BFMatcher object
066: bf = [Link]()
067: matches = [Link](des1,des2, k=2)
068:
069: # Apply ratio test
070: goodmatches = []
071: for m,n in matches:
072: if [Link] < 0.9*[Link]:
073: [Link]([m])
074:
075: # [Link] expects a list of lists as matches.
076: img3 = [Link](img1,kp1,img2,kp2,goodmatches[:100],None,flags=0)
077: [Link](img3),
078: [Link]()
079:
080:
081: """
082: #part3 : Detect object in a scene
083: # Load images
084:
085: method_match='bf'
086: #method_match='knn'
087:
088: template = [Link]('images/[Link]')
089: #template = [Link]('images/[Link]')
090: #template = [Link]('images/[Link]',0)
091: template = [Link](template, cv2.COLOR_BGR2RGB)
092: template_crop=template
093: templateg= [Link](template_crop, cv2.COLOR_RGB2GRAY)
094:
095: scene = [Link]('images/[Link]')
096: #scene = [Link]('images/[Link]',0)
097: #scene = [Link]('images/[Link]',0)
098: scene = [Link](scene, cv2.COLOR_BGR2RGB)
099: scene_crop=scene

Image Processing Lab Scripts - Explanation Page 17

 100: sceneg = [Link](scene_crop, cv2.COLOR_RGB2GRAY)
101:
102: fig, (ax1, ax2) = [Link](nrows=1, ncols=2, figsize=(20, 8), constrained_layout=False)
103: [Link](templateg,'gray')
104: [Link](sceneg,'gray')
105: [Link]()
106:
107: # Initialize SIFT detector
108: sift = cv2.SIFT_create()
109:
110: # Find keypoints and descriptors for the template and scene images
111: kp1, des1 = [Link](templateg, None)
112: kp2, des2 = [Link](sceneg, None)
113: print(kp1)
114:
115: fig, (ax1, ax2) = [Link](nrows=1, ncols=2, figsize=(20, 8), constrained_layout=False)
116: [Link]([Link](templateg, kp1, None, color=(0, 255, 0)))
117: ax1.set_xlabel("(a)", fontsize=14)
118: [Link]([Link](sceneg, kp2, None, color=(0, 255, 0)))
119: ax2.set_xlabel("(b)", fontsize=14)
120: [Link]()
121: # Initialize Brute-Force Matcher
122: #bf = [Link]()
123:
124: if method_match=='bf':
125: bf = [Link](cv2.NORM_L2, crossCheck=True)
126: bmatches = [Link](des1, des2)
127: b1matches = sorted(bmatches, key=lambda x: [Link])
128: matchimage=[Link](template_crop, kp1,
scene_crop,kp2,b1matches[0:30],outImg=None,flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS,matchesThickness=2)
129: [Link](matchimage)
130: [Link]()
131:
132: if method_match=='knn':
133: bf = [Link]()
134: bmatches = [Link](des1, des2,k=2)
135: b1matches = []
136: for m, n in bmatches:
137: if [Link] < 0.7 * [Link]:
138: [Link](m)
139: matchimage = [Link](template_crop, kp1, scene_crop, kp2, b1matches, outImg=None,
140: flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS, matchesThickness=2)
141: [Link](matchimage)
142: [Link]('off')
143: [Link]()
144:
145: # If enough good matches are found, draw the bounding box around the object
146: if len(b1matches) > 10:
147: src_pts = np.float32([kp1[[Link]].pt for m in b1matches]).reshape(-1, 1, 2)
148: dst_pts = np.float32([kp2[[Link]].pt for m in b1matches]).reshape(-1, 1, 2)
149: print('srcpoints=',src_pts)
150:
151:
152: # Calculate homography
153: M, mask = [Link](src_pts, dst_pts, [Link], 0.1)
154:
155: #plot only good match aftre transformation
156: matchesMask = [Link]().tolist()
157: draw_params = dict(matchColor=(0, 255, 0), # draw matches in green color
158: singlePointColor=None,
159: matchesMask=matchesMask, # draw only inliers
160: flags=2)
161: matchimagefinal = [Link](template_crop, kp1, scene_crop, kp2, b1matches, None,**draw_params)
162: [Link](matchimagefinal)
163: [Link]('match after homography')
164: [Link]('off')
165: [Link]()
166:
167:
168: # Get coordinates of the corners of the template image
169: h, w = [Link]
170: corners = np.float32([[0, 0], [0, h - 1], [w - 1, h - 1], [w - 1, 0]]).reshape(-1, 1, 2)
171: # Transform coordinates of the corners to the scene image
172: transformed_corners = [Link](corners, M)
173: # Draw the bounding box around the detected object
174: scene_with_box = [Link](sceneg, [np.int32(transformed_corners)], True, (0,0,255), 3, cv2.LINE_AA)
175: [Link]('output',scene_with_box)
176: [Link](0)
177: [Link]()
178:
179: #projection of hole template image
180: width = [Link][1]
181: height =[Link][0]
182: result = [Link](templateg, M, (width, height))
183: [Link](figsize=(20, 10))
184: [Link]('off')
185: [Link](result,'gray')
186: [Link]()
187: else:
188: print('No corresponding points')
189:

What this script does

- Detects an object inside a scene using SIFT features and feature matching.
- The active part reads a template image and a scene image, detects SIFT points, matches them, estimates homography, and
draws the object boundary.
- The commented parts show simpler SIFT matching and KNN ratio-test matching.

Image Processing Lab Scripts - Explanation Page 18

 Input and output
- Input: a template image such as images/[Link] and a scene image such as images/[Link].
- Output: displayed keypoints, displayed matches, final inlier matches after homography, detected object polygon, and
warped template projection.

Important functions and instructions

- [Link](): reads template and scene images.
- [Link](): converts BGR to RGB for Matplotlib and to grayscale for SIFT.
- cv2.SIFT_create(): creates the SIFT detector.
- [Link](): detects keypoints and computes descriptors in one step.
- [Link](): brute-force matcher that compares descriptors.
- [Link](): finds one best match for each descriptor.
- [Link](k=2): finds the two closest descriptors, used for Lowe ratio test.
- sorted(..., key=lambda x: [Link]): sorts matches from best to worst.
- [Link](): draws lines between matched points.
- [Link](..., [Link]): estimates the transformation between template and scene while rejecting bad
matches.
- [Link]().tolist(): converts the RANSAC inlier mask to a list for drawing only good matches.
- [Link](): transforms template corner coordinates into the scene.
- [Link](): draws the detected object boundary.
- [Link](): projects the template using the homography.
- Important concept: homography is useful when the same flat object appears with perspective change.

Image Processing Lab Scripts - Explanation Page 19

 7. OpCV14_FaceAndPesondetcam.py
Script code
001:
002: #Part 1:
003: #detecter un visage dans une image
004: import cv2, sys, numpy, os
005:
006: #import la fonction prete qui contient les instructions de detection de visage
007: face_detector=[Link]('haarcascade_frontalface_default.xml')
008:
009: #preparer la camera de PC : mettre 0 dans [Link]
010: cap = [Link](0)
011: #cap = [Link]('./Project track/video.mp4')
012: #cap = [Link](
'[Link]
013: # The program loops until it has 300 images of the face.
014:
015: count = 1
016: while(1): # count < 300:
017: (_, im) = [Link]()
018: gray = [Link](im, cv2.COLOR_BGR2GRAY)
019: faces = face_detector.detectMultiScale(gray, 1.1, 4)
020: print(faces)
021: for (x, y, w, h) in faces:
022: [Link](im, (x, y), (x + w, y + h), (0, 255, 0), 2)
023: face = gray[y:y + h, x:x + w]
024: [Link]('face',face)
025: count += 1
026: [Link]('Image avec faces', im)
027: key = [Link](10)
028: if key == 27:
029: break
030: if [Link](1) & 0xFF == ord('q'):
031: break
032:
033:
034: """
035: #Part 2: person detection
036: import cv2
037: import imutils
038:
039: # Initializing the HOG person
040: # detector
041: hog = [Link]()
042: [Link](cv2.HOGDescriptor_getDefaultPeopleDetector())
043:
044: cap = [Link]('./project track/video2.mp4')
045:
046: while [Link]():
047: # Reading the video stream
048: ret, image = [Link]()
049: if ret:
050: image = [Link](image,
051: width=min(400, [Link][1]))
052:
053: # Detecting all the regions
054: # in the Image that has a
055: # pedestrians inside it
056: (regions, _) = [Link](image,
057: winStride=(4, 4),
058: padding=(4, 4),
059: scale=1.05)
060:
061: # Drawing the regions in the
062: # Image
063: for (x, y, w, h) in regions:
064: [Link](image, (x, y),
065: (x + w, y + h),
066: (0, 0, 255), 2)
067:
068: # Showing the output Image
069: [Link]("Image", image)
070: if [Link](25) & 0xFF == ord('q'):
071: break
072: else:
073: break
074:
075: [Link]()
076: [Link]()
077: """

What this script does

- Detects faces from a webcam using a Haar cascade classifier.
- For each detected face, it draws a rectangle and also shows the cropped face region.
- A second part for person detection using HOG is present but commented.

Input and output

- Input: live webcam video, or a video file if VideoCapture path is changed.
- Output: live window with face rectangles and a window showing the cropped grayscale face.

Image Processing Lab Scripts - Explanation Page 20

 Important functions and instructions
- [Link](): loads the trained Haar face detector XML file.
- [Link](0): opens the computer webcam.
- [Link](): reads one frame from the camera.
- [Link](..., COLOR_BGR2GRAY): makes grayscale image because Haar detection works on intensity.
- detectMultiScale(): detects faces at different sizes in the image.
- [Link](): draws bounding boxes around detected faces.
- gray[y:y+h, x:x+w]: crops the face region from the grayscale frame.
- [Link](): displays the frame and face crop.
- [Link](): listens for ESC or q to stop the loop.
- Important note: the XML file path must be correct. Often [Link] + filename is safer.

Image Processing Lab Scripts - Explanation Page 21

 8. OpCV15_calibrate.py
Script code
001: #Calibrating a camera
002:
003:
004: #Part 1: Calibrating camera using Chess Examples
005: import os
006: import cv2
007: import numpy as np
008: import glob
009:
010: # Define chessboard dimensions
011: chessboard_size = (9, 6) # (columns, rows) of inner corners
012: square_size = 10.0 # Set to real-world square size (e.g., in cm or mm)
013:
014: # Prepare object points: (0,0,0), (1,0,0), (2,0,0), ..., (8,5,0)
015: objp = [Link]((chessboard_size[0]*chessboard_size[1], 3), np.float32)
016: objp[:, :2] = [Link][0:chessboard_size[0], 0:chessboard_size[1]].[Link](-1, 2)
017: objp *= square_size
018: print(objp)
019: # Arrays to store object points and image points
020: objpoints = [] # 3D points in real world
021: imgpoints = [] # 2D points in image plane
022:
023: # Load calibration images
024: images = [Link]('./calibration/Mono/*.jpg') # Make sure this folder contains your chessboard images
025: print(images[0])
026: for fname in images:
027: img = [Link](fname)
028: gray = [Link](img, cv2.COLOR_BGR2GRAY)
029:
030: # Find chessboard corners
031: ret, corners = [Link](gray, chessboard_size, None)
032:
033: if ret:
034: [Link](objp)
035: [Link](corners)
036:
037: # Draw and display the corners
038: [Link](img, chessboard_size, corners, ret)
039: [Link]('Corners', img)
040: [Link](1000)
041:
042: [Link]()
043:
044: # Perform camera calibration
045: ret, camera_matrix, dist_coeffs, rvecs, tvecs = [Link](
046: objpoints, imgpoints, [Link][::-1], None, None)
047:
048: # Save calibration results
049: [Link]("camera_calibration_data.npz",
050: camera_matrix=camera_matrix,
051: dist_coeffs=dist_coeffs,
052: rvecs=rvecs,
053: tvecs=tvecs)
054:
055: # Display results
056: print("Camera matrix:\n", camera_matrix)
057: print("Distortion coefficients:\n", dist_coeffs)
058: rotation_matrix, _ = [Link](rvecs[8])
059: print(rotation_matrix)
060: print(tvecs[8])
061: # Compute and display reprojection error
062: total_error = 0
063: for i in range(len(objpoints)):
064: imgpoints2, _ = [Link](objpoints[i], rvecs[i], tvecs[i], camera_matrix, dist_coeffs)
065: error = [Link](imgpoints[i], imgpoints2, cv2.NORM_L2) / len(imgpoints2)
066: total_error += error
067:
068: print("Mean reprojection error: ", total_error / len(objpoints))
069:
070:
071: #Part 2: calibrating stereo images
072: import numpy as np
073: import cv2
074: import glob
075: #import open3d as o3d
076:
077: # Chessboard size
078: chessboard_size = (9, 6) # (columns, rows) of inner corners
079: square_size = 20.0 # Real-world size of a square (e.g., in cm or mm)
080:
081: # Prepare object points
082: objp = [Link](([Link](chessboard_size), 3), np.float32)
083: objp[:, :2] = [Link](chessboard_size).[Link](-1, 2)
084: objp *= square_size
085:
086: objpoints = [] # 3D points in real-world space
087: imgpoints_left = [] # 2D points in left image plane
088: imgpoints_right = [] # 2D points in right image plane
089:
090: # Path to images
091: left_images = sorted([Link]('./calibration/stereo/left/left*.jpg'))
092: right_images = sorted([Link]('./calibration/stereo/right/right*.jpg'))
093:
094: print(left_images)
095: print(right_images)
096:
097: # Check image pairs
098: assert len(left_images) == len(right_images), "Mismatched number of left and right images!"
099:

Image Processing Lab Scripts - Explanation Page 22

 100: # Detect corners
101: for left_path, right_path in zip(left_images, right_images):
102: img_left = [Link](left_path)
103: img_right = [Link](right_path)
104: gray_left = [Link](img_left, cv2.COLOR_BGR2GRAY)
105: gray_right = [Link](img_right, cv2.COLOR_BGR2GRAY)
106:
107: ret_left, corners_left = [Link](gray_left, chessboard_size, None)
108: ret_right, corners_right = [Link](gray_right, chessboard_size, None)
109:
110: if ret_left and ret_right:
111: [Link](objp)
112: corners_left = [Link](gray_left, corners_left, (11, 11), (-1, -1),
113: criteria=(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001))
114: corners_right = [Link](gray_right, corners_right, (11, 11), (-1, -1),
115: criteria=(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30,
0.001))
116: imgpoints_left.append(corners_left)
117: imgpoints_right.append(corners_right)
118:
119: # Image size
120: img_shape = gray_left.shape[::-1]
121:
122: # Calibrate each camera individually
123: ret_l, mtx_l, dist_l, _, _ = [Link](objpoints, imgpoints_left, img_shape, None, None)
124: ret_r, mtx_r, dist_r, _, _ = [Link](objpoints, imgpoints_right, img_shape, None, None)
125:
126: # Stereo calibration
127: flags = 0
128: flags |= cv2.CALIB_FIX_INTRINSIC
129:
130: criteria = (cv2.TERM_CRITERIA_MAX_ITER + cv2.TERM_CRITERIA_EPS, 100, 1e-5)
131:
132: ret_stereo, mtx_l, dist_l, mtx_r, dist_r, R, T, E, F = [Link](
133: objpoints, imgpoints_left, imgpoints_right,
134: mtx_l, dist_l, mtx_r, dist_r,
135: img_shape, criteria=criteria, flags=flags)
136:
137: print("Stereo Calibration RMS error:", ret_stereo)
138: print("Rotation Matrix:\n", R)
139: print("Translation Vector:\n", T)
140:
141: # Save calibration results
142: [Link]('stereo_calibration.npz',
143: mtx_l=mtx_l, dist_l=dist_l,
144: mtx_r=mtx_r, dist_r=dist_r,
145: R=R, T=T, E=E, F=F)
146:
147: print("Calibration saved to stereo_calibration.npz")
148:
149: """
150: #part 3: Open calibrated file and estimating 3D using one right and one left image
151: import open3d as o3d
152: import numpy as np
153: import cv2
154:
155: # Load stereo calibration parameters
156: data = [Link]('stereo_calibration.npz')
157: mtx_l = data['mtx_l']
158: dist_l = data['dist_l']
159: mtx_r = data['mtx_r']
160: dist_r = data['dist_r']
161: R = data['R']
162: T = data['T']
163:
164: # Load a pair of stereo images
165: img_left = [Link]('./calibration/stereo/left/[Link]')
166: img_right = [Link]('./calibration/stereo/right/[Link]')
167: gray_left = [Link](img_left, cv2.COLOR_BGR2GRAY)
168: gray_right = [Link](img_right, cv2.COLOR_BGR2GRAY)
169: img_size = gray_left.shape[::-1]
170:
171: # Stereo rectification
172: R1, R2, P1, P2, Q, _, _ = [Link](
173: mtx_l, dist_l, mtx_r, dist_r, img_size, R, T, alpha=0)
174:
175: # Save it correctly
176: fs = [Link]("Q_matrix.yml", cv2.FILE_STORAGE_WRITE)
177: [Link]("Q", Q)
178: [Link]()
179:
180: print("[OK] Q matrix saved to Q_matrix.yml")
181:
182: # Undistort and rectify images
183: map1_l, map2_l = [Link](mtx_l, dist_l, R1, P1, img_size, cv2.CV_16SC2)
184: map1_r, map2_r = [Link](mtx_r, dist_r, R2, P2, img_size, cv2.CV_16SC2)
185:
186: rectified_left = [Link](gray_left, map1_l, map2_l, cv2.INTER_LINEAR)
187: rectified_right = [Link](gray_right, map1_r, map2_r, cv2.INTER_LINEAR)
188:
189: # Compute disparity map
190: stereo = cv2.StereoBM_create(numDisparities=64, blockSize=15)
191: disparity = [Link](rectified_left, rectified_right).astype(np.float32) / 16.0
192:
193: # Reproject to 3D space
194: points_3D = cv2.reprojectImageTo3D(disparity, Q)
195:
196: # Create a mask of valid disparity points
197: mask = disparity > [Link]()
198:
199: # Extract only valid 3D points
200: output_points = points_3D[mask]
201: output_colors = img_left[mask]
202:
203: # Show images

Image Processing Lab Scripts - Explanation Page 23

 204: [Link]("Rectified Left", rectified_left)
205: [Link]("Rectified Right", rectified_right)
206: [Link]("Disparity Map", (disparity - [Link]()) / ([Link]() - [Link]()))
207: [Link](0)
208: [Link]()
209:
210:
211: def click_event(event, x, y, flags, param):
212: if event == cv2.EVENT_LBUTTONDOWN:
213: disparity_value = disparity[y, x]
214: point_3D = points_3D[y, x]
215:
216: print(f"Clicked pixel: ({x}, {y})")
217: print(f"Disparity: {disparity_value:.2f}")
218:
219: if [Link](point_3D[2]) or disparity_value <= 0:
220: print("[WARNING] Invalid depth at this point.")
221: else:
222: X, Y, Z = point_3D
223: print(f"Estimated 3D coordinates: X={X:.2f}, Y={Y:.2f}, Z (Depth)={Z:.2f}")
224: print('distance is: ', [Link](X**2+Y**2+Z**2))
225:
226: # Display on image
227: img = img_left.copy()
228: [Link](img, (x, y), 5, (0, 0, 255), -1)
229: [Link](img, f"Z={Z:.2f}", (x+10, y-10), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2)
230: [Link]("Left Image - Click to get depth", img)
231:
232: # Show window and wait for click
233: [Link]("Left Image - Click to get depth", img_left)
234: [Link]("Left Image - Click to get depth", click_event)
235: [Link](0)
236: [Link]()
237:
238:
239:
240: #view 3D
241: import open3d as o3d
242: import numpy as np
243:
244: # Filter valid points (non-zero disparity and finite 3D)
245: mask = (disparity > 0) & [Link](points_3D[:, :, 2])
246: output_points = points_3D[mask]
247: output_colors = img_left[mask]
248:
249: # Optional: remove too far/near points (bad depth values)
250: z_min, z_max = 0.1, 1000 # in same units as square_size (e.g. cm or mm)
251: depths = output_points[:, 2]
252: valid_depth_mask = (depths > z_min) & (depths < z_max)
253:
254: output_points = output_points[valid_depth_mask]
255: output_colors = output_colors[valid_depth_mask]
256:
257: # Visualize using open3d
258: pcd = [Link]()
259: [Link] = [Link].Vector3dVector(output_points)
260: [Link] = [Link].Vector3dVector(output_colors.astype(np.float32) / 255.0)
261:
262: [Link].draw_geometries([pcd])
263: """
264:

What this script does

- Calibrates one camera using chessboard images.
- Calibrates a stereo camera pair using left and right chessboard image pairs.
- Saves calibration matrices and prints reprojection error.
- A third commented part shows how to use stereo calibration to rectify images, compute disparity, estimate 3D points, and
show a point cloud.

Input and output

- Input: mono chessboard images in ./calibration/Mono/*.jpg and stereo pairs in ./calibration/stereo/left and right.
- Output: camera_calibration_data.npz, stereo_calibration.npz, printed camera matrices, distortion coefficients,
rotation/translation, and reprojection error.
- Optional output in commented part: Q_matrix.yml, disparity map, clicked 3D coordinates, and Open3D point cloud.

Important functions and instructions

- [Link]() and [Link]()/[Link](): create ideal 3D chessboard corner coordinates.
- square_size: converts chessboard grid coordinates to real-world units.
- [Link](): finds all calibration images in a folder.
- [Link](): detects internal chessboard corners in each image.
- [Link](): refines detected corner locations for more accurate calibration.
- objpoints and imgpoints: store matched 3D real points and 2D image points.

Image Processing Lab Scripts - Explanation Page 24

 - [Link](): estimates intrinsic matrix, distortion coefficients, rotation vectors, and translation vectors.
- [Link](): converts a rotation vector to a rotation matrix.
- [Link](): projects known 3D points back to the image to measure calibration quality.
- [Link](): computes reprojection error.
- [Link](): estimates rotation and translation between left and right cameras.
- cv2.CALIB_FIX_INTRINSIC: keeps individual camera intrinsics fixed during stereo calibration.
- [Link](): saves several calibration arrays in one file.
- Important concept: low reprojection error means the calibration is more reliable.

Image Processing Lab Scripts - Explanation Page 25

 9. OpCV16_depthestimation.py
Script code
001: import [Link]
002: import numpy as np
003: import cv2
004: import mediapipe as mp
005: # Load the MATLAB .mat file
006: mat = [Link]('./calibration/stereo/stereoParams_struct.mat',struct_as_record=False, squeeze_me=True)
007:
008: params = mat['paramsStruct']
009:
010: # Access camera structs
011: cam1 = params.CameraParameters1
012: cam2 = params.CameraParameters2
013:
014: # Use 'K' instead of 'IntrinsicMatrix'
015: mtx_l = cam1.K
016: dist_l = [Link]([[Link], [Link]])
017:
018: mtx_r = cam2.K
019: dist_r = [Link]([[Link], [Link]])
020:
021: # Extrinsics
022: R = params.RotationOfCamera2
023: T = params.TranslationOfCamera2
024:
025:
026: cx = mtx_l[0, 2]
027: cy = mtx_l[1, 2]
028: image_size = (int(cx), int(cy))
029:
030: # Now stereoRectify will work
031: R1, R2, P1, P2, Q, _, _ = [Link](mtx_l, dist_l, mtx_r, dist_r, image_size, R, T)
032:
033: # Save to .npz
034: output_file = "./calibration/stereo/stereo_calibration.npz"
035: [Link](output_file,
036: mtx_l=mtx_l, dist_l=dist_l,
037: mtx_r=mtx_r, dist_r=dist_r,
038: R=R, T=T, Q=Q)
039:
040: print(mtx_l)
041: print("[OK] Converted and saved as stereo_calibration_from_struct.npz")
042:
043:
044:
045: #Part 2: estimate depth from two images
046: import cv2
047: import numpy as np
048:
049: # === Load calibration ===
050: data = [Link]("./calibration/stereo/stereo_calibration.npz")
051: Q = data['Q']
052:
053: cap_l = [Link]("./calibration/stereo/left/handshake_left.avi")
054: cap_r = [Link]("./calibration/stereo/right/handshake_right.avi")
055:
056: ret_l, frame_l = cap_l.read()
057: ret_r, frame_r = cap_r.read()
058:
059: gray_l = [Link](frame_l, cv2.COLOR_BGR2GRAY)
060: gray_r = [Link](frame_r, cv2.COLOR_BGR2GRAY)
061:
062: # Resize to match calibration if needed
063: #frame_l = [Link](frame_l, (int(cx*2), int(cy*2)))
064: #frame_r = [Link](frame_r, (int(cx*2), int(cy*2)))
065:
066: #stereo = cv2.StereoBM_create(numDisparities=128, blockSize=5)
067:
068:
069: # === Stereo matcher (SGBM) ===
070: stereo = cv2.StereoSGBM_create(
071: minDisparity=0,
072: numDisparities=64,
073: blockSize=5,
074: P1=8 * 3 * 5 ** 2,
075: P2=32 * 3 * 5 ** 2,
076: disp12MaxDiff=1,
077: uniquenessRatio=1,
078: speckleWindowSize=100,
079: speckleRange=32
080: )
081:
082: # === Compute disparity and reproject to 3D ===
083: disparity = [Link](gray_l, gray_r).astype(np.float32)/16
084:
085: points_3D = cv2.reprojectImageTo3D(disparity, Q)/1000
086:
087: # === Mouse callback to get 3D point ===
088: def click_event(event, x, y, flags, param):
089: if event == cv2.EVENT_LBUTTONDOWN:
090: point = points_3D[y, x]
091: X,Y,Z = point
092:
093: if [Link](Z):
094: print(f"[POINT] Clicked at ({x}, {y}) -> X: {point[0]:.2f}, Y: {point[1]:.2f}, Z: {Z:.2f} units")
095: img_disp = frame_l.copy()
096: [Link](img_disp, (x, y), 5, (0, 0, 255), -1)
097: [Link](img_disp, f"dist: {[Link](X**2+Y**2+Z**2):.2f}", (x+10, y-10),
098: cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
099: [Link]("Left Image", img_disp)

Image Processing Lab Scripts - Explanation Page 26

 100: else:
101: print(f"[WARNING] Invalid depth at ({x}, {y})")
102:
103: # === Display the image and wait for click ===
104: [Link]("Left Image", frame_l)
105: [Link]("Left Image", click_event)
106:
107: [Link](0)
108: [Link]()
109:
110:
111:
112:
113:
114:
115: #part3: estimate faces
116: # === Load stereo calibration data ===
117: data = [Link]("./calibration/stereo/stereo_calibration.npz")
118: mtx_l = data['mtx_l']
119: dist_l = data['dist_l']
120: mtx_r = data['mtx_r']
121: dist_r = data['dist_r']
122: R = data['R']
123: T = data['T']
124: Q = data['Q']
125:
126: # === Open stereo videos ===
127: # === Open left and right videos ===
128: cap_l = [Link]("./calibration/stereo/left/handshake_left.avi")
129: cap_r = [Link]("./calibration/stereo/right/handshake_right.avi")
130: # === Load face detector ===
131: face_cascade = [Link]([Link] + "haarcascade_frontalface_default.xml")
132:
133: # === Stereo matcher ===
134: stereo = cv2.StereoSGBM_create(
135: minDisparity=0,
136: numDisparities=64,
137: blockSize=5,
138: P1=8 * 3 * 5 ** 2,
139: P2=32 * 3 * 5 ** 2,
140: disp12MaxDiff=1,
141: uniquenessRatio=10,
142: speckleWindowSize=100,
143: speckleRange=32
144: )
145:
146: while cap_l.isOpened() and cap_r.isOpened():
147: ret_l, frame_l = cap_l.read()
148: ret_r, frame_r = cap_r.read()
149: if not ret_l or not ret_r:
150: break
151:
152: #frame_l = [Link](frame_l, (640, 480))
153: #frame_r = [Link](frame_r, (640, 480))
154:
155: gray_l = [Link](frame_l, cv2.COLOR_BGR2GRAY)
156: gray_r = [Link](frame_r, cv2.COLOR_BGR2GRAY)
157:
158:
159: mp_face = [Link].face_detection
160: mp_draw = [Link].drawing_utils
161:
162: # Setup MediaPipe face detector
163: face_detection = mp_face.FaceDetection(model_selection=1, min_detection_confidence=0.9)
164:
165: # === Compute disparity and 3D reconstruction ===
166: disparity = [Link](gray_l, gray_r).astype(np.float32) / 16.0
167: points_3D = cv2.reprojectImageTo3D(disparity, Q)
168: # Use MediaPipe to detect faces in left frame
169: results = face_detection.process([Link](frame_l, cv2.COLOR_BGR2RGB))
170:
171: if [Link]:
172: for det in [Link]:
173: bbox = det.location_data.relative_bounding_box
174: ih, iw, _ = frame_l.shape
175: x = int([Link] * iw)
176: y = int([Link] * ih)
177: w = int([Link] * iw)
178: h = int([Link] * ih)
179:
180: cx = x + w // 2
181: cy = y + h // 2
182:
183: point = points_3D[cy, cx]
184: depth = point[2]
185:
186: [Link](frame_l, (x, y), (x + w, y + h), (255, 0, 0), 2)
187: [Link](frame_l, (cx, cy), 4, (0, 0, 255), -1)
188: [Link](frame_l, f"Z: {depth:.2f} units", (x, y - 10),
189: cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2)
190:
191: # === Show result ===
192: [Link]("Left Video + Face Depth", frame_l)
193:
194: if [Link](30) & 0xFF == 27:
195: break
196:
197: cap_l.release()
198: cap_r.release()
199: [Link]()
200:
201:
202:
203: #person detection
204:

Image Processing Lab Scripts - Explanation Page 27

 205: import cv2
206: import numpy as np
207:
208: # === Load stereo calibration ===
209: data = [Link]("./calibration/stereo/stereo_calibration.npz")
210: mtx_l = data['mtx_l']
211: dist_l = data['dist_l']
212: mtx_r = data['mtx_r']
213: dist_r = data['dist_r']
214: R = data['R']
215: T = data['T']
216: Q = data['Q']
217:
218: # === Open stereo videos ===
219: cap_l = [Link]("./calibration/stereo/left/handshake_left.avi")
220: cap_r = [Link]("./calibration/stereo/right/handshake_right.avi")
221:
222: # === Stereo matcher (SGBM) ===
223: stereo = cv2.StereoSGBM_create(
224: minDisparity=0,
225: numDisparities=64,
226: blockSize=5,
227: P1=8 * 3 * 5 ** 2,
228: P2=32 * 3 * 5 ** 2,
229: disp12MaxDiff=1,
230: uniquenessRatio=10,
231: speckleWindowSize=100,
232: speckleRange=32
233: )
234:
235: # === Person detector: OpenCV HOG + SVM ===
236: hog = [Link]()
237: [Link](cv2.HOGDescriptor_getDefaultPeopleDetector())
238:
239: while cap_l.isOpened() and cap_r.isOpened():
240: ret_l, frame_l = cap_l.read()
241: ret_r, frame_r = cap_r.read()
242: if not ret_l or not ret_r:
243: break
244:
245: # Resize to match calibration resolution
246: frame_l = [Link](frame_l, (640, 480))
247: frame_r = [Link](frame_r, (640, 480))
248:
249: gray_l = [Link](frame_l, cv2.COLOR_BGR2GRAY)
250: gray_r = [Link](frame_r, cv2.COLOR_BGR2GRAY)
251:
252: # === Disparity and 3D reconstruction ===
253: disparity = [Link](gray_l, gray_r).astype(np.float32) / 16.0
254: points_3D = cv2.reprojectImageTo3D(disparity, Q)/1000
255:
256: # === Detect people ===
257: boxes, _ = [Link](frame_l, winStride=(8, 8), padding=(16, 16), scale=1.05)
258:
259: for (x, y, w, h) in boxes:
260: cx = x + w // 2
261: cy = y + h // 2
262:
263: # Get 3D coordinates at the center of the person
264: if 0 <= cx < 640 and 0 <= cy < 480:
265: point = points_3D[cy, cx]
266: X,Y,Z=point
267: depth = point[2]
268:
269: if [Link](depth):
270: [Link](frame_l, f"Z: {[Link](X**2+Y**2+Z**2):.2f} units", (x, y - 10),
271: cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2)
272: else:
273: [Link](frame_l, f"Z: ???", (x, y - 10),
274: cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
275:
276: [Link](frame_l, (x, y), (x + w, y + h), (255, 0, 0), 2)
277: [Link](frame_l, (cx, cy), 4, (0, 0, 255), -1)
278:
279: # === Show result ===
280: [Link]("Left View + Person Depth", frame_l)
281: if [Link](30) & 0xFF == 27:
282: break
283:
284: cap_l.release()
285: cap_r.release()
286: [Link]()
287:
288:
289:
290:

What this script does

- Converts MATLAB stereo calibration parameters into a NumPy calibration file.
- Computes a disparity map from a stereo video pair and estimates 3D coordinates/depth.
- Lets the user click a point to read its 3D position and distance.
- Then estimates depth of detected faces and detected persons in stereo video.

Input and output

Image Processing Lab Scripts - Explanation Page 28

 - Input: MATLAB file ./calibration/stereo/stereoParams_struct.mat and left/right stereo videos handshake_left.avi and
handshake_right.avi.
- Output: stereo_calibration.npz, printed matrices, depth values on click, and video windows with face/person depth labels.

Important functions and instructions

- [Link](): loads stereo parameters exported from MATLAB.
- cam1.K and cam2.K: intrinsic matrices of the left and right cameras.
- [Link](): joins radial and tangential distortion coefficients.
- [Link](): computes rectification and the Q matrix for 3D reconstruction.
- [Link](): saves calibration values for later use.
- [Link](): opens left and right stereo videos.
- cv2.StereoSGBM_create(): creates a semi-global block matching stereo matcher.
- [Link](): computes disparity between left and right grayscale frames.
- cv2.reprojectImageTo3D(): converts disparity to 3D coordinates using Q.
- [Link](): calls a function when the user clicks on the image.
- [Link](X**2+Y**2+Z**2): computes distance from the camera to the 3D point.
- mediapipe FaceDetection: detects faces in the left video frame.
- cv2.HOGDescriptor_getDefaultPeopleDetector(): loads a pre-trained person detector.
- Important note: accurate depth normally needs rectified left/right frames. This script computes Q but does not remap frames
in all active parts.

Image Processing Lab Scripts - Explanation Page 29

 10. OpCV16_depth_face_person.py
Script code
001: import cv2
002: import numpy as np
003: import [Link]
004: import mediapipe as mp
005:
006: # =====================================================
007: # PART 1 - Convert MATLAB calibration and prepare maps
008: # =====================================================
009:
010: mat = [Link](
011: "./calibration/stereo/stereoParams_struct.mat",
012: struct_as_record=False,
013: squeeze_me=True
014: )
015:
016: params = mat["paramsStruct"]
017:
018: cam1 = params.CameraParameters1
019: cam2 = params.CameraParameters2
020:
021: mtx_l = cam1.K
022: mtx_r = cam2.K
023:
024: dist_l = [Link]([[Link], [Link]])
025: dist_r = [Link]([[Link], [Link]])
026:
027: R = params.RotationOfCamera2
028: T = params.TranslationOfCamera2
029:
030: # Read one frame to get the REAL image size
031: cap_test = [Link]("./calibration/stereo/left/handshake_left.avi")
032: ret, test_frame = cap_test.read()
033: cap_test.release()
034:
035: if not ret:
036: raise RuntimeError("Cannot read left video.")
037:
038: h, w = test_frame.shape[:2]
039: image_size = (w, h)
040:
041: R1, R2, P1, P2, Q, roi1, roi2 = [Link](
042: mtx_l, dist_l,
043: mtx_r, dist_r,
044: image_size,
045: R, T,
046: flags=cv2.CALIB_ZERO_DISPARITY,
047: alpha=0
048: )
049:
050: map1_l, map2_l = [Link](
051: mtx_l, dist_l, R1, P1, image_size, cv2.CV_32FC1
052: )
053:
054: map1_r, map2_r = [Link](
055: mtx_r, dist_r, R2, P2, image_size, cv2.CV_32FC1
056: )
057:
058: [Link](
059: "./calibration/stereo/stereo_calibration_corrected.npz",
060: mtx_l=mtx_l,
061: dist_l=dist_l,
062: mtx_r=mtx_r,
063: dist_r=dist_r,
064: R=R,
065: T=T,
066: Q=Q,
067: map1_l=map1_l,
068: map2_l=map2_l,
069: map1_r=map1_r,
070: map2_r=map2_r,
071: image_width=w,
072: image_height=h
073: )
074:
075: print("Calibration corrected and saved.")
076:
077:
078: # =====================================================
079: # Common functions
080: # =====================================================
081:
082: data = [Link]("./calibration/stereo/stereo_calibration_corrected.npz")
083:
084: Q = data["Q"]
085: map1_l = data["map1_l"]
086: map2_l = data["map2_l"]
087: map1_r = data["map1_r"]
088: map2_r = data["map2_r"]
089:
090: stereo = cv2.StereoSGBM_create(
091: minDisparity=0,
092: numDisparities=16 * 8,
093: blockSize=7,
094: P1=8 * 3 * 7 ** 2,
095: P2=32 * 3 * 7 ** 2,
096: disp12MaxDiff=1,
097: uniquenessRatio=10,
098: speckleWindowSize=150,
099: speckleRange=2,

Image Processing Lab Scripts - Explanation Page 30

 100: preFilterCap=63,
101: mode=cv2.STEREO_SGBM_MODE_SGBM_3WAY
102: )
103:
104:
105: def compute_depth(frame_l, frame_r):
106: rect_l = [Link](frame_l, map1_l, map2_l, cv2.INTER_LINEAR)
107: rect_r = [Link](frame_r, map1_r, map2_r, cv2.INTER_LINEAR)
108:
109: gray_l = [Link](rect_l, cv2.COLOR_BGR2GRAY)
110: gray_r = [Link](rect_r, cv2.COLOR_BGR2GRAY)
111:
112: disparity = [Link](gray_l, gray_r).astype(np.float32) / 16.0
113: disparity[disparity <= 0] = [Link]
114:
115: points_3D = cv2.reprojectImageTo3D(disparity, Q)
116:
117: return rect_l, rect_r, disparity, points_3D
118:
119:
120: def median_depth(points_3D, x, y, radius=5):
121: h, w = points_3D.shape[:2]
122:
123: x1 = max(0, x - radius)
124: x2 = min(w, x + radius)
125: y1 = max(0, y - radius)
126: y2 = min(h, y + radius)
127:
128: region = points_3D[y1:y2, x1:x2, 2]
129: region = region[[Link](region)]
130:
131: if len(region) == 0:
132: return None
133:
134: return [Link](region) / 1000.0
135:
136:
137: # =====================================================
138: # PART 2 - Depth after mouse click
139: # =====================================================
140:
141: cap_l = [Link]("./calibration/stereo/left/handshake_left.avi")
142: cap_r = [Link]("./calibration/stereo/right/handshake_right.avi")
143:
144: ret_l, frame_l = cap_l.read()
145: ret_r, frame_r = cap_r.read()
146:
147: if not ret_l or not ret_r:
148: raise RuntimeError("Cannot read stereo videos.")
149:
150: rect_l, rect_r, disparity, points_3D = compute_depth(frame_l, frame_r)
151:
152: def click_event(event, x, y, flags, param):
153: if event == cv2.EVENT_LBUTTONDOWN:
154: z = median_depth(points_3D, x, y, radius=7)
155:
156: display = rect_l.copy()
157:
158: if z is not None:
159: [Link](display, (x, y), 5, (0, 0, 255), -1)
160: [Link](
161: display,
162: f"Depth = {z:.2f} m",
163: (x + 10, y - 10),
164: cv2.FONT_HERSHEY_SIMPLEX,
165: 0.7,
166: (0, 0, 255),
167: 2
168: )
169: print(f"Clicked point: x={x}, y={y}, depth={z:.2f} m")
170: else:
171: print("Invalid depth.")
172:
173: [Link]("Click Depth", display)
174:
175: [Link]("Click Depth", rect_l)
176: [Link]("Click Depth", click_event)
177:
178: [Link](0)
179: [Link]()
180:
181: cap_l.release()
182: cap_r.release()
183:
184:
185: # =====================================================
186: # PART 3 - Face detection + depth
187: # =====================================================
188:
189: cap_l = [Link]("./calibration/stereo/left/handshake_left.avi")
190: cap_r = [Link]("./calibration/stereo/right/handshake_right.avi")
191:
192: mp_face = [Link].face_detection
193: face_detector = mp_face.FaceDetection(
194: model_selection=1,
195: min_detection_confidence=0.1
196: )
197:
198: while True:
199: ret_l, frame_l = cap_l.read()
200: ret_r, frame_r = cap_r.read()
201:
202: if not ret_l or not ret_r:
203: break
204:

Image Processing Lab Scripts - Explanation Page 31

 205: rect_l, rect_r, disparity, points_3D = compute_depth(frame_l, frame_r)
206:
207: rgb = [Link](rect_l, cv2.COLOR_BGR2RGB)
208: results = face_detector.process(rgb)
209:
210: if [Link]:
211: ih, iw = rect_l.shape[:2]
212:
213: for det in [Link]:
214: bbox = det.location_data.relative_bounding_box
215:
216: x = int([Link] * iw)
217: y = int([Link] * ih)
218: bw = int([Link] * iw)
219: bh = int([Link] * ih)
220:
221: x = max(0, x)
222: y = max(0, y)
223: bw = min(bw, iw - x)
224: bh = min(bh, ih - y)
225:
226: cx = x + bw // 2
227: cy = y + bh // 2
228:
229: z = median_depth(points_3D, cx, cy, radius=10)
230:
231: [Link](rect_l, (x, y), (x + bw, y + bh), (255, 0, 0), 2)
232: [Link](rect_l, (cx, cy), 4, (0, 0, 255), -1)
233:
234: if z is not None:
235: text = f"Face depth: {z:.2f} m"
236: else:
237: text = "Face depth: unknown"
238:
239: [Link](
240: rect_l,
241: text,
242: (x, y - 10),
243: cv2.FONT_HERSHEY_SIMPLEX,
244: 0.7,
245: (0, 255, 0),
246: 2
247: )
248:
249: [Link]("Face Depth", rect_l)
250:
251: if [Link](30) & 0xFF == 27:
252: break
253:
254: cap_l.release()
255: cap_r.release()
256: [Link]()
257:
258:
259: # =====================================================
260: # PART 4 - Person detection + depth
261: # =====================================================
262:
263: cap_l = [Link]("./calibration/stereo/left/handshake_left.avi")
264: cap_r = [Link]("./calibration/stereo/right/handshake_right.avi")
265:
266: hog = [Link]()
267: [Link](cv2.HOGDescriptor_getDefaultPeopleDetector())
268:
269: while True:
270: ret_l, frame_l = cap_l.read()
271: ret_r, frame_r = cap_r.read()
272:
273: if not ret_l or not ret_r:
274: break
275:
276: rect_l, rect_r, disparity, points_3D = compute_depth(frame_l, frame_r)
277:
278: boxes, weights = [Link](
279: rect_l,
280: winStride=(8, 8),
281: padding=(16, 16),
282: scale=1.05
283: )
284:
285: for (x, y, bw, bh) in boxes:
286: cx = x + bw // 2
287: cy = y + int(0.55 * bh)
288:
289: z = median_depth(points_3D, cx, cy, radius=12)
290:
291: [Link](rect_l, (x, y), (x + bw, y + bh), (255, 0, 0), 2)
292: [Link](rect_l, (cx, cy), 4, (0, 0, 255), -1)
293:
294: if z is not None:
295: text = f"Person depth: {z:.2f} m"
296: else:
297: text = "Person depth: unknown"
298:
299: [Link](
300: rect_l,
301: text,
302: (x, y - 10),
303: cv2.FONT_HERSHEY_SIMPLEX,
304: 0.7,
305: (0, 255, 0),
306: 2
307: )
308:
309: [Link]("Person Depth", rect_l)

Image Processing Lab Scripts - Explanation Page 32

 310:
311: if [Link](30) & 0xFF == 27:
312: break
313:
314: cap_l.release()
315: cap_r.release()
316: [Link]()

What this script does

- This is a cleaner stereo depth script for click depth, face depth, and person depth.
- It loads MATLAB calibration, gets the real video frame size, rectifies the stereo cameras, and saves corrected maps.
- It defines reusable functions to compute depth and median depth.
- It displays depth when the user clicks, then tracks face depth and person depth in the stereo video.

Input and output

- Input: stereoParams_struct.mat and the left/right handshake stereo videos.
- Output: stereo_calibration_corrected.npz, click depth in meters, face depth window, and person depth window.

Important functions and instructions

- [Link](): reads one frame to know the real image size and later reads videos.
- [Link](..., CALIB_ZERO_DISPARITY): aligns left and right images so matching is easier.
- [Link](): creates pixel maps for undistortion and rectification.
- [Link](): applies those maps to each frame.
- compute_depth(frame_l, frame_r): rectifies frames, converts them to grayscale, computes disparity, and reprojects to 3D.
- disparity[disparity <= 0] = [Link]: marks invalid disparity values as not-a-number.
- median_depth(points_3D, x, y, radius): uses a small neighborhood around a point to get a more stable depth.
- [Link](): removes invalid depth values.
- [Link](): lets the user click a point and get depth.
- MediaPipe FaceDetection: detects face bounding boxes.
- HOGDescriptor and detectMultiScale(): detect people in the image.
- [Link](), [Link](), [Link](): draw depth text, boxes, and center points.
- Important concept: median depth is more robust than using only one noisy pixel.

Image Processing Lab Scripts - Explanation Page 33

 11. OpCV17_point_track.py
Script code
001: """
002:
003: """
004: """
005: #part 1: Track 2 images coming from video
006: import cv2
007: import numpy as np
008: video_path = './Project track/video.mp4'
009: cap = [Link](video_path)
010: #cap = [Link](0)
011: # detect the face region by Haar Cascade Classifier
012: face_cascade = [Link]([Link] + 'haarcascade_frontalface_default.xml')
013:
014: # reading frame
015: ret, first_frame = [Link]()
016: gray_first_frame = [Link](first_frame, cv2.COLOR_BGR2GRAY)
017:
018: # Detect the face in the initial frame
019: faces = face_cascade.detectMultiScale(gray_first_frame, scaleFactor=1.1, minNeighbors=5, minSize=(30, 30))
020:
021: # use the first detected face for tracking
022: (x, y, w, h) = faces[0]
023: face_region = (x, y, w, h)
024: [Link](first_frame, (x, y), (x + w, y + h), (0, 0, 255), 2)
025: [Link]("Initial Frame with Face Detection", first_frame)
026: [Link](0)
027:
028: #definir un region d'interet
029: roi_gray = gray_first_frame[y:y+h, x:x+w]
030: points = [Link](roi_gray, maxCorners=100, qualityLevel=0.01, minDistance=10)
031:
032: # add points
033: points = [Link](-1, 2)
034: points += [x, y]
035: # display initial points on the face
036: for point in points:
037: [Link](first_frame, (int(point[0]), int(point[1])), 3, (255, 255, 255), -1)
038: [Link]("Interest Points on Face", first_frame)
039: [Link]()
040:
041: # I use Lucas-Kanade optical flow for tracking
042: lk_params = dict(winSize=(15, 15), maxLevel=2,
043: criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))
044:
045: prev_gray = gray_first_frame.copy()
046: prev_points = [Link](-1, 1, 2)
047: color = [Link](0, 255, (100, 3))
048: mask = np.zeros_like(first_frame)
049:
050: #read new image after 10 frames in the video
051: frame_number = 10
052:
053: [Link](cv2.CAP_PROP_POS_FRAMES, frame_number)
054: ret, frame = [Link]()
055:
056: gray_frame = [Link](frame, cv2.COLOR_BGR2GRAY)
057: new_points, status, _ = [Link](prev_gray, gray_frame, prev_points, None, **lk_params)
058: good_new_points = new_points[status == 1]
059: good_prev_points = prev_points[status == 1]
060:
061: for i, (new, old) in enumerate(zip(good_new_points, good_prev_points)):
062: a, b = [Link]()
063: [Link](frame, (int(a), int(b)), 3, (0, 255, 0), -1)
064: a, b = ([Link]()).astype(int)
065: c, d = ([Link]()).astype(int)
066: print(a,b,c,d)
067: mask = [Link](mask, (a, b), (c, d),
068: color[i].tolist(), 2)
069: frame=[Link](frame,mask)
070: [Link]("mask", mask)
071: [Link]("Face Tracking", frame)
072: [Link]()
073: [Link]()
074:
075: """
076: """
077: #part 2: Detect points in the whole frame and follow the points in real time
078: import numpy as np
079: import cv2
080:
081: cap = [Link]('./Project track/video.mp4')
082: #cap = [Link](
'[Link]
083: #cap = [Link](0)
084:
085: # Create some random colors
086: color = [Link](0, 255, (100, 3))
087:
088: # Take first frame and find corners in it
089: ret, old_frame = [Link]()
090: old_gray = [Link](old_frame, cv2.COLOR_BGR2GRAY)
091:
092: # corner detection
093: p0 = [Link](old_gray,
094: mask=None,
095: maxCorners=100,
096: qualityLevel=0.01,
097: minDistance=10,
098: blockSize=7)

Image Processing Lab Scripts - Explanation Page 34

 099:
100: # Create a mask image for drawing purposes
101: mask = np.zeros_like(old_frame)
102:
103: while (1):
104: ret, frame = [Link]()
105: frame_gray = [Link](frame, cv2.COLOR_BGR2GRAY)
106:
107: # calculate optical flow- lucas kanade optical flow
108: p1, st, err = [Link](old_gray,
109: frame_gray,
110: p0, None,
111: winSize=(15, 15),
112: maxLevel=2,
113: criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT,
114: 10, 0.03))
115: # Select good points
116: good_new = p1[st == 1]
117: good_old = p0[st == 1]
118:
119: # draw the tracks
120: for i, (new, old) in enumerate(zip(good_new,
121: good_old)):
122: a, b = [Link]().astype(int)
123: c, d = [Link]().astype(int)
124: mask = [Link](mask, (a, b), (c, d),#(0,0,255),2)
125: color[i].tolist(), 2)
126:
127: frame = [Link](frame, (a, b), 5,(0,0,255),-1)
128: # color[i].tolist(), -1)
129:
130: img = [Link](frame, mask)
131: [Link]('mask', mask)
132: [Link]('frame', img)
133:
134: k = [Link](25)
135: if k == 27:
136: break
137:
138: # Updating Previous frame and points
139: old_gray = frame_gray.copy()
140: p0 = good_new.reshape(-1, 1, 2)
141:
142: [Link]()
143: [Link]()
144:
145:
146: """
147: ##part 3: Detect points in the face or ROI and follow the points in real time
148: import cv2
149: import numpy as np
150: #video_path = './Project track/video.mp4'
151: #cap = [Link](
'[Link]
152: #cap = [Link](video_path)
153: cap = [Link](0)
154: # detect the face region by Haar Cascade Classifier
155: #face_cascade = [Link]([Link] + 'haarcascade_frontalface_default.xml')
156:
157: # reading frame
158: ret, first_frame = [Link]()
159: gray_first_frame = [Link](first_frame, cv2.COLOR_BGR2GRAY)
160:
161: ### Select part you want
162: r = [Link](gray_first_frame)# r(0), r(1), r(2),(r3): (x,y,w,l)
163: roi_gray = gray_first_frame[int(r[1]):int(r[1] + r[3]), int(r[0]):int(r[0] + r[2])]
164: x, y=r[0],r[1]
165:
166: points = [Link](roi_gray, maxCorners=100, qualityLevel=0.1, minDistance=30)
167:
168: # add points
169: points = [Link](-1, 2)
170: points += [x, y]
171: # display initial points on the face
172: for point in points:
173: [Link](first_frame, (int(point[0]), int(point[1])), 3, (255, 255, 255), -1)
174: [Link]("Interest Points on Face", first_frame)
175: [Link](500)
176:
177: # I use Lucas-Kanade optical flow for tracking
178: lk_params = dict(winSize=(15, 15), maxLevel=2,
179: criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))
180:
181: prev_gray = gray_first_frame.copy()
182: prev_points = [Link](-1, 1, 2)
183: color = [Link](0, 255, (100, 3))
184: mask = np.zeros_like(first_frame)
185: while True:
186: ret, frame = [Link]()
187: gray_frame = [Link](frame, cv2.COLOR_BGR2GRAY)
188: new_points, status, _ = [Link](prev_gray, gray_frame, prev_points, None, **lk_params)
189: good_new_points = new_points[status == 1]
190: good_prev_points = prev_points[status == 1]
191:
192: for i, (new, old) in enumerate(zip(good_new_points, good_prev_points)):
193: a, b = [Link]()
194: [Link](frame, (int(a), int(b)), 3, (0, 255, 0), -1)
195: a, b = ([Link]()).astype(int)
196: c, d = ([Link]()).astype(int)
197: print(a,b,c,d)
198: mask = [Link](mask, (a, b), (c, d),
199: color[i].tolist(), 2)
200: frame=[Link](frame,mask)
201:
202: [Link]("Face Tracking", frame)

Image Processing Lab Scripts - Explanation Page 35

 203: prev_gray = gray_frame.copy()
204: prev_points = good_new_points.reshape(-1, 1, 2)
205:
206: if [Link](1) & 0xFF == ord('q'):
207: break
208: [Link]()
209: [Link]()
210:
211:
212: """
213: #part 4: trak video with comment and error of reading
214: import cv2
215:
216: #video_path = './valentin/[Link]'
217: video_path = './Project track/video2.mp4'
218: #video_path = './Project track/sample.mp4'
219:
220: cap = [Link](video_path)
221: # detect the face region by Haar Cascade Classifier
222: face_cascade = [Link]([Link] + 'haarcascade_frontalface_default.xml')
223:
224: # Check if video opened successfully
225: if not [Link]():
226: print("Error: Could not open video.")
227: exit()
228:
229: output_path = 'output1_myvideo.mp4'
230: fourcc = cv2.VideoWriter_fourcc(*'XVID')
231: fps = [Link](cv2.CAP_PROP_FPS)
232: frame_width = int([Link](cv2.CAP_PROP_FRAME_WIDTH))
233: frame_height = int([Link](cv2.CAP_PROP_FRAME_HEIGHT))
234: out = [Link](output_path, fourcc, fps, (frame_width, frame_height))
235:
236: # reading frame
237: ret, first_frame = [Link]()
238: if not ret:
239: print("Error: Could not read the first frame.")
240: exit()
241:
242: gray_first_frame = [Link](first_frame, cv2.COLOR_BGR2GRAY)
243:
244: # Detect the face in the initial frame
245: faces = face_cascade.detectMultiScale(gray_first_frame, scaleFactor=1.1, minNeighbors=5, minSize=(30, 30))
246: if len(faces) == 0:
247: print("Error: No face detected in the first frame.")
248: exit()
249:
250: # use the first detected face for tracking
251: (x, y, w, h) = faces[0]
252: face_region = (x, y, w, h)
253:
254: [Link](first_frame, (x, y), (x + w, y + h), (0, 0, 255), 2)
255: [Link]("Initial Frame with Face Detection", first_frame)
256: [Link](500)
257:
258: roi_gray = gray_first_frame[y:y+h, x:x+w]
259: points = [Link](roi_gray, maxCorners=100, qualityLevel=0.01, minDistance=10)
260: if points is None:
261: print("Error: No features found on the face.")
262: exit()
263: print(points)
264: # add points
265: points = [Link](-1, 2)
266: print(points)
267: points += [x, y]
268: # display initial points on the face
269: for point in points:
270: [Link](first_frame, (int(point[0]), int(point[1])), 3, (255, 255, 255), -1)
271: [Link]("Interest Points on Face", first_frame)
272: [Link](500)
273:
274: # I use Lucas-Kanade optical flow for tracking
275: lk_params = dict(winSize=(15, 15), maxLevel=2,
276: criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))
277:
278: prev_gray = gray_first_frame.copy()
279: prev_points = [Link](-1, 1, 2)
280:
281: while True:
282: ret, frame = [Link]()
283: if not ret:
284: break
285:
286: gray_frame = [Link](frame, cv2.COLOR_BGR2GRAY)
287: new_points, status, _ = [Link](prev_gray, gray_frame, prev_points, None, **lk_params)
288: if new_points is None or status is None:
289:
290: print("Tracking lost; attempting to re-detect features.")
291: faces = face_cascade.detectMultiScale(gray_frame, scaleFactor=1.1, minNeighbors=5, minSize=(30, 30))
292: if len(faces) == 0:
293: print("Error: No face found. Exiting.")
294: break
295:
296: (x, y, w, h) = faces[0]
297: face_region = (x, y, w, h)
298: roi_gray = gray_frame[y:y+h, x:x+w]
299: points = [Link](roi_gray, maxCorners=100, qualityLevel=0.01, minDistance=10)
300: if points is not None:
301: points = [Link](-1, 2)
302: points += [x, y]
303: prev_points = [Link](-1, 1, 2)
304: continue
305:
306:
307: good_new_points = new_points[status == 1]

Image Processing Lab Scripts - Explanation Page 36

 308: good_prev_points = prev_points[status == 1]
309:
310: for i, (new, old) in enumerate(zip(good_new_points, good_prev_points)):
311: a, b = [Link]()
312: [Link](frame, (int(a), int(b)), 3, (0, 255, 0), -1)
313:
314: [Link]("Face Tracking", frame)
315: [Link](frame)
316:
317: prev_gray = gray_frame.copy()
318: prev_points = good_new_points.reshape(-1, 1, 2)
319:
320: if [Link](1) & 0xFF == ord('q'):
321: break
322:
323: [Link]()
324: [Link]()
325: [Link]()
326: """

What this script does

- Tracks interest points over time using Lucas-Kanade optical flow.
- The active part opens the webcam, lets the user select a ROI, detects good corner points inside it, and tracks them frame by
frame.
- Commented parts show face-region tracking, whole-frame tracking, and a version that saves output video.

Input and output

- Input: webcam stream or video file, depending on VideoCapture line.
- Output: live video showing tracked points and colored motion trails. In the commented part, output1_myvideo.mp4 can be
saved.

Important functions and instructions

- [Link](): opens webcam or video.
- [Link](): lets the user choose the region to track.
- [Link](): detects strong corner points to track.
- [Link](-1, 2): changes the point array shape for easier coordinate operations.
- points += [x, y]: converts ROI-local coordinates to full-image coordinates.
- [Link](): tracks points from the previous grayscale frame to the current one using Lucas-Kanade optical
flow.
- status == 1: keeps only points that were tracked successfully.
- np.zeros_like(first_frame): creates a black mask used to draw motion trails.
- [Link](): draws the path of each moving point.
- [Link](frame, mask): combines the current frame with the trail mask.
- prev_gray = gray_frame.copy() and prev_points = good_new_points.reshape(...): update the reference frame and points for
the next iteration.
- Important concept: optical flow estimates apparent motion of image points between frames.

Image Processing Lab Scripts - Explanation Page 37

 12. OPCV18_object_tracking.py
Script code
001: """
002: This script demonstrates object tracking in a video using the meanshift algorithm.
003: It reads a video, processes it frame by frame to track a specified region of interest (ROI),
004: and saves the output video with the tracking rectangle drawn on it.
005:
006: Steps:
007: 1. Import necessary libraries.
008: 2. Define the path to the input video and the output video.
009: 3. Open the video file for reading.
010: 4. Check if the video opened successfully.
011: 5. Get the width and height of the video frames.
012: 6. Initialize VideoWriter to save the output video.
013: 7. Read the first frame to set up the tracking window.
014: 8. Define the Region of Interest (ROI) for tracking and calculate its histogram.
015: 9. Set up termination criteria for the meanshift algorithm.
016: 10. Loop through each frame of the video:
017: a. Convert the current frame to HSV color space.
018: b. Calculate the back projection of the histogram onto the current frame.
019: c. Apply meanshift to find the new position of the ROI.
020: d. Draw the tracking rectangle on the frame.
021: e. Write the processed frame to the output video.
022: f. Display the current frame with the tracking rectangle.
023: 11. Release video resources when done.
024: """
025: """
026: import cv2
027: import numpy as np
028:
029: # Step 1: Load the video
030: #video_path = ('./Project track/video2.mp4')
031: #cap = [Link]( '[Link]
032: #cap = [Link](
'[Link]
033: cap = [Link](0)
034: #cap=cap = [Link](video_path)
035: # Step 2: Read the first frame
036: ret, frame = [Link]()
037: if not ret:
038: print("Unable to read video.")
039: exit()
040:
041: # Step 3: Select the ROI (Region of Interest) for tracking (initial bounding box)
042: r = [Link](frame)# r(0), r(1), r(2),(r3): (x,y,w,l)
043: roi = frame[int(r[1]):int(r[1] + r[3]), int(r[0]):int(r[0] + r[2])]
044:
045:
046: # Step 4: Convert the ROI to HSV and calculate the histogram
047: hsv_roi = [Link](roi, cv2.COLOR_BGR2HSV)
048: roi_hist = [Link]([hsv_roi], [0], None, [16], [0, 180])
049:
050: # Step 5: Normalize the histogram
051: [Link](roi_hist, roi_hist, 0, 255, cv2.NORM_MINMAX)
052:
053: # Step 6: Setup the termination criteria
054: # 10 iterations or move by at least 1 pixel
055: criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.1)
056:
057: while True:
058: # Step 7: Read the next frame
059: ret, frame = [Link]()
060: if not ret:
061: break
062:
063: # Step 8: Convert the frame to HSV color space
064: hsv = [Link](frame, cv2.COLOR_BGR2HSV)
065:
066: # Step 9: Back projection of the histogram to the current frame
067: probImage = [Link]([hsv], [0], roi_hist, [0, 255], 1)
068: [Link]('Projection des histogrames', probImage)
069: # Step 10: Apply meanShift to find the new location of the object
070: ret, window = [Link](probImage, tuple(map(int, r)), criteria)
071:
072: # Step 11: Draw the new tracking window
073: x, y, w, h = window
074: [Link](frame, (x, y), (x + w, y + h), (0, 255, 0), 2)
075:
076: # Step 12: Display the frame with the tracking window
077: [Link]('Tracking', frame)
078: [Link](100)
079:
080: # Break the loop if the user presses the 'ESC' key
081: if [Link](1) & 0xFF == ord('q'):
082: break
083:
084: # Release the video capture and close any OpenCV windows
085: [Link]()
086: [Link]()
087:
088: """
089:
090: import numpy as np
091: import cv2 as cv
092:
093: # path to the video file
094: #video_path = ('./Project track/[Link]')
095:
096: # Opening the video file
097: #cap = [Link](video_path)
098: cap = [Link](0)

Image Processing Lab Scripts - Explanation Page 38

 099: # Taking the first frame of the video
100:
101:
102: #acap = [Link](
'[Link]
103: #cap = [Link](0)
104:
105: ret, frame = [Link]()
106:
107: # Setting up initial location of the tracking window (ROI)
108: x, y, w, h = [Link]("Select ROI", frame, fromCenter=False, showCrosshair=True)
109: track_window = (x, y, w, h)
110:
111: # Setting up the ROI for tracking
112: roi = frame[y:y+h, x:x+w] # Extracting the ROI from the first frame
113: hsv_roi = [Link](roi, cv.COLOR_BGR2HSV) # Converting ROI to HSV color space
114: mask = [Link](hsv_roi, [Link]((0., 60., 32.)), [Link]((180., 255., 255.))) # Creating mask for the ROI
115: roi_hist = [Link]([hsv_roi], [0], mask, [180], [0, 180]) # Calculating histogram for the ROI
116: [Link](roi_hist, roi_hist, 0, 255, cv.NORM_MINMAX) # Normalizing the histogram
117:
118: # Setting up the termination criteria: either 10 iterations or move by at least 1 pt
119: term_crit = (cv.TERM_CRITERIA_EPS | cv.TERM_CRITERIA_COUNT, 10, 1)
120:
121: while True:
122: ret, frame = [Link]() # Reading the next frame
123:
124: if ret:
125: hsv = [Link](frame, cv.COLOR_BGR2HSV) # Converting the frame to HSV
126: dst = [Link]([hsv], [0], roi_hist, [0, 180], 1) # Back projection
127: [Link]('fdst', dst)
128: # Applying meanshift to get the new location of the tracking window
129: ret, track_window = [Link](dst, track_window, term_crit)
130:
131: # Drawing the rectangle on the image
132: x, y, w, h = track_window
133: img2 = [Link](frame, (x, y), (x + w, y + h), 255, 2)
134:
135: # Displaying the frame with the tracking rectangle
136: [Link]('Tracked Video', img2)
137:
138: k = [Link](30) & 0xff # Waiting for 30 ms
139: if k == 27: # Exiting if 'ESC' is pressed
140: break
141: else:
142: break # Exiting the loop if there are no more frames
143:
144: # Releasing everything if job is finished
145: [Link]() # Releasing the video capture object
146: [Link]() # Closing all OpenCV windows
147:
148: """
149: #part2: with output and error detection
150: import numpy as np
151: import cv2 as cv
152:
153: # path to the video file
154: video_path = './Project track/video.mp4'
155: output_path = './Project track/tracked_output.mp4' # Specifying the output file name here
156:
157: # Opening the video file
158: cap = [Link](video_path)
159:
160: # Checking if the video opened successfully
161: if not [Link]():
162: print("Error: Could not open video.")
163: exit()
164:
165: # Getting the width and height of the frames
166: frame_width = int([Link](cv.CAP_PROP_FRAME_WIDTH))
167: frame_height = int([Link](cv.CAP_PROP_FRAME_HEIGHT))
168:
169: # Defining the codec and create a VideoWriter object
170: fourcc = cv.VideoWriter_fourcc(*'mp4v') # Codec for mp4
171: out = [Link](output_path, fourcc, 30, (frame_width, frame_height))
172:
173: # Taking the first frame of the video
174: ret, frame = [Link]()
175:
176: # Checking if the frame was read successfully
177: if not ret:
178: print("Failed to read the video.")
179: [Link]()
180: [Link]()
181: [Link]()
182: exit()
183:
184: # Setting up initial location of the tracking window (ROI)
185: x, y, w, h = [Link]("Select ROI", frame, fromCenter=False, showCrosshair=True)
186: track_window = (x, y, w, h)
187:
188: # Setting up the ROI for tracking
189: roi = frame[y:y+h, x:x+w] # Extracting the ROI from the first frame
190: hsv_roi = [Link](roi, cv.COLOR_BGR2HSV) # Converting ROI to HSV color space
191: mask = [Link](hsv_roi, [Link]((0., 60., 32.)), [Link]((180., 255., 255.))) # Creating mask for the ROI
192: roi_hist = [Link]([hsv_roi], [0], mask, [180], [0, 180]) # Calculating histogram for the ROI
193: [Link](roi_hist, roi_hist, 0, 255, cv.NORM_MINMAX) # Normalizing the histogram
194:
195: # Setting up the termination criteria: either 10 iterations or move by at least 1 pt
196: term_crit = (cv.TERM_CRITERIA_EPS | cv.TERM_CRITERIA_COUNT, 10, 1)
197:
198: while True:
199: ret, frame = [Link]() # Reading the next frame
200:
201: if ret:
202: hsv = [Link](frame, cv.COLOR_BGR2HSV) # Converting the frame to HSV

Image Processing Lab Scripts - Explanation Page 39

 203: dst = [Link]([hsv], [0], roi_hist, [0, 180], 1) # Back projection
204:
205: # Applying meanshift to get the new location of the tracking window
206: ret, track_window = [Link](dst, track_window, term_crit)
207:
208: # Drawing the rectangle on the image
209: x, y, w, h = track_window
210: img2 = [Link](frame, (x, y), (x + w, y + h), 255, 2)
211:
212: # Writing the frame with the tracking rectangle to the output video
213: [Link](img2)
214:
215: # Displaying the frame with the tracking rectangle
216: [Link]('Tracked Video', img2)
217:
218: k = [Link](30) & 0xff # Waiting for 30 ms
219: if k == 27: # Exiting if 'ESC' is pressed
220: break
221: else:
222: break # Exiting the loop if there are no more frames
223:
224: # Releasing everything if job is finished
225: [Link]() # Releasing the video capture object
226: [Link]() # Releasing the video writer object
227: [Link]() # Closing all OpenCV windows
228:
229: """

What this script does

- Tracks a selected object using the Mean Shift algorithm.
- The active part uses the webcam and lets the user select the object ROI in the first frame.
- It builds an HSV color histogram of the ROI, then follows similar colors in the next frames.
- A commented part also shows a version that saves the tracked video.

Input and output

- Input: webcam stream or a video file.
- Output: live video with a tracking rectangle and a back-projection probability image.

Important functions and instructions

- [Link](): manually selects the object to track.
- [Link](..., COLOR_BGR2HSV): converts image to HSV color space, which is useful for color tracking.
- [Link](): creates a mask to keep valid color values and remove weak pixels.
- [Link](): computes the hue histogram of the selected ROI.
- [Link](): normalizes the histogram to a fixed range.
- [Link](): creates a probability image showing where the ROI colors appear in the new frame.
- [Link](): moves the tracking window toward the highest probability region.
- [Link](): draws the updated tracking window.
- term_crit: stopping condition for the mean shift iterations.
- Important concept: mean shift tracking works well when the object color is distinctive from the background.

Image Processing Lab Scripts - Explanation Page 40

 13. [Link]
Script code
001: import cv2 # Bibliotheque OpenCV pour le traitement d'images
002: import sys
003:
004: # ---------------------------------------------------------
005: # Ce script cree un panorama a partir de 3 images JPEG.
006: # ---------------------------------------------------------
007:
008: # Noms des fichiers image (JPEG)
009: image_files = ["IMG_1.jpeg", "IMG_2.jpeg", "IMG_3.jpeg"]
010:
011: # Liste pour stocker les images chargees
012: images = []
013:
014: for path in image_files:
015: # Lecture de l'image depuis le disque
016: img = [Link](path)
017:
018: # Verification que l'image est bien chargee
019: if img is None:
020: print(f"Erreur : impossible de lire le fichier {path}")
021: [Link](1)
022:
023: [Link](img)
024:
025: # Creation de l'objet " stitcher " (assembleur de panorama)
026: # Cette classe est fournie par le module de stitching d'OpenCV.
027: stitcher = cv2.Stitcher_create() # OpenCV 4.x
028:
029: # Lancement de l'assemblage des images en panorama
030: # La methode stitch() retourne :
031: # - un code de status
032: # - l'image resultante (si succes)
033: (status, panorama) = [Link](images)
034:
035: # Verification du statut
036: if status != cv2.Stitcher_OK:
037: print(f"Erreur lors de la creation du panorama, code = {status}")
038: [Link](1)
039:
040: # Affichage du resultat dans une fenetre
041: [Link]("Panorama", panorama)
042:
043: # Sauvegarde du panorama dans un fichier JPEG
044: [Link]("panorama_resultat.jpg", panorama)
045: print("Panorama cree et sauvegarde sous 'panorama_resultat.jpg'.")
046:
047: # Attente d'une touche, puis fermeture des fenetres
048: [Link](0)
049: [Link]()
050:

What this script does

- Creates a panorama by stitching three input images together.
- It loads the images, checks that they were loaded correctly, uses OpenCV Stitcher, displays the result, and saves it.

Input and output

- Input: IMG_1.jpeg, IMG_2.jpeg, and IMG_3.jpeg in the same folder.
- Output: displayed panorama window and saved file panorama_resultat.jpg.

Important functions and instructions

- [Link](): loads each image from disk.
- if img is None: checks if image loading failed.
- [Link](1): stops the program if an input image is missing or stitching fails.
- cv2.Stitcher_create(): creates the OpenCV panorama stitcher object.
- [Link](images): estimates the panorama transformation and combines the images.
- cv2.Stitcher_OK: status code meaning stitching succeeded.
- [Link](): displays the panorama.
- [Link](): saves the panorama to a JPEG file.
- [Link]() and [Link](): wait for a key press and close windows.

Image Processing Lab Scripts - Explanation Page 41