Trimble Business Center

Training Custom Classification Models

[Link]

© 2023, Trimble Inc. All rights reserved. Trimble and the Globe & Triangle logo are trademarks of Trimble Inc. registered in the United States and in other
countries. All other trademarks are the property of their respective owners.
 About this tutorial

About this tutorial

The Training tab in the Extract Classified Point Cloud Regions command pane enables
you to create (train) a custom point cloud classification model you can use to
automatically extract new point cloud regions and/or adjust existing automatically
extracted point cloud regions to your specific data. You do this by providing manually
corrected classified region examples to train and validate the model yourself using
deep-learning technology, resulting in a model with the highest accuracy possible based
on the input sample data.
In this tutorial, you will import five training point cloud files, each containing a number
of stockpile features that have been carefully extracted and classified. You will then
import two validation point cloud files, whose stockpiles have also been carefully
extracted and classified.
During the training process, the new model first "learns" from the training files what to
look for when considering features for classification. It then applies what it learned by
testing itself on the validation files to see how accurately it is able to match the
temporarily applied extraction/region boundaries with the existing boundaries in the
file. Based on the results of those tests, it continues the test/validation process through
multiple iterations (epochs) intended to achieve the highest rate of accuracy when using
the model to perform feature extraction/classification.

Important notes!

- An NVIDIA GPU with a minimum of 6 GB of VRAM is required to complete this tutorial.

- Depending on your computer, the training process could take several hours
(unattended) to complete.

Step 1. Create a project and import a point cloud

For this tutorial, you will create a new project into which you will import LAS (.las) point
cloud data files to be used for training and validation of the new classification model.

Note: The downloaded TrainingClassificationModels folder contains this PDF file and a
Data folder containing LAS (.las) point cloud data files you will use for the training
process.

1. In the TBC ribbon, select File > New. Then, select to use the <Blank Template> for
your new project.
2. In the TBC ribbon, select Home > View > 3D View to display the 3D View tab.

Training Custom Classification Models 2

 Step 1. Create a project and import a point cloud

Next, you will import into your new project one of the data files that you will use to
train the new classification model. This step is not necessary to train the new model,
but it will allow you to view a sample of the data to be used with classification
already performed. It will also provide a means to preview a selection voxel later in
this procedure.
3. Open the ..\TrainingClassificationModels\Data folder and drag-and-drop the
[Link] file onto the 3D View tab.
4. In the Point Cloud Scale for [Link] dialog, leave the Non-georeferenced ground-
scale point cloud option selected and click OK.
Georeferencing of the data is not necessary for this procedure.

5. When the import is complete, click OK in the Projection Definition dialog.

The imported point cloud displays in the 3D View.

Training Custom Classification Models 3

 Step 1. Create a project and import a point cloud

6. In the Project Explorer, double-click the Point Cloud Regions > Transmission Tower
node to display the Properties pane and select the Transmission Tower classified
point cloud regions in the project.

Training Custom Classification Models 4

 Step 2. Analyze the training and validation point cloud files

Although the classified point cloud regions are stockpile features, when they were
classified in the various point cloud data files you will use for training and validation,
they were assigned an LAS classification code of 15, which is typically used for
transmission towers (no stockpile classification code exists). So Transmission Tower
was the name assigned to the class. You will rename the class when you create the
new model.

Step 2. Analyze the training and validation point cloud files

In this step, you will select the LAS (.las) point cloud data files you want to use for
training and validation, select a voxel size to be used during training to help the new
model delineate the classified objects, and initiate the analysis of all of the selected
training and validation files to determine the classified regions they contain.
1. In the TBC ribbon, select Point Clouds > Extraction > Extract Classified Point Cloud
Regions. Then, click the Training tab.

Training Custom Classification Models 5

 Step 2. Analyze the training and validation point cloud files

2. In the Model name field, enter Stockpile.

Next, you will select five point cloud files in which stockpile features have already
been classified. These files will be used as examples of what a stockpile "looks like"
in the real world in able to train the new Stockpile model.
Although only one training file is required for training a new model, the more
training files you include (each showing variations on the classified point cloud
regions), the more accurate the resulting model will be.
3. Click the button located to the right of the Training file selection field to display the
Open window. Then, browse to the downloaded
..\TrainingClassificationModels\Data folder.
4. Use Shift+click to select files [Link] through [Link] and click Open.

Training Custom Classification Models 6

 Step 2. Analyze the training and validation point cloud files

The selected files are displayed in the Training file selection field.
Now, you will select two more point cloud files in which stockpile features have
already been accurately classified. These files will be used as examples against
which the extraction results of training will be compared to determine how
accurately the model's "learned" extraction/region boundaries match the existing
boundaries in the file. This continues for a specified number of iterations (epochs)
intended to achieve maximum accuracy.
As with training files, although only one validation file is required, more validation
files will help ensure a more accurate model.

Note: All training and validation files must be stored in the same folder.

5. Click the button located to the right of the Training file selection field to display the
Open window. Then, use Shift+click to select files [Link] and [Link] and click
Open.

Training Custom Classification Models 7

 Step 2. Analyze the training and validation point cloud files

Next, you will check the Voxel size, a parameter that helps delineate the classified
objects and appropriate surrounding context using a simple graphic view.
The voxel must be small enough to delineate the object of interest but large enough
to include some context around the object. Context information (correct voxel size)
can be an important factor in training the model where to find the object in the
point cloud. For example, knowing that the object of interest, say an insulator, is
always located on a tower with power lines around it, it would help to include some
of the tower and power lines in the voxel. But too much context (too large of a
voxel) can make it more difficult for the model to delineate the object of interest.
For this tutorial, you would like the voxel to include some of the flat ground
surrounding a medium-sized stockpile.

Training Custom Classification Models 8

 Step 2. Analyze the training and validation point cloud files

6. With the Voxel size set to the default 10 cm, click in the the View voxel effect field
and then click on an average sized stockpile in the 3D View, as shown here.

In this example, the voxel does not cover the entire stockpile.
7. Change the Voxel size to 20 cm and press the Tab key.

Again, the voxel appears to be just a bit too small to adequately cover the stockpile.
8. Change the Voxel size to 23 cm and press the Tab key.

Training Custom Classification Models 9

 Step 2. Analyze the training and validation point cloud files

The voxel now appears to adequately cover the entire chosen stockpile and some
flat ground around it, so there is no need for additional changes. Although other
stockpiles are of different sizes, this voxel size should work with them all.
You are now ready to have TBC analyze all of the selected training and validation
files to determine the classified regions they contain.
9. Click the Analyze Files button.
10. The analysis progress is displayed in the Status bar at the bottom of the TBC
window.

Training Custom Classification Models 10

 Step 3. Train the custom classification model

The analysis detects the classes (identified by class numbers) contained within the
selected training and validation files and the number of sample objects per class
type. It also determines the optimal number of epochs to perform during training,
which you can change using Advanced Settings, as discussed later in this procedure.
(Note that the number of classes found in your project may differ from those shown
here.)

Note: To train the new classification model to its highest accuracy, it is

recommended that your training files include four times the number of samples that
are included in your validation files. That is, training samples should represent 80%
of the total samples used for training and validating the model.

Step 3. Train the custom classification model

In this step, you will select the region class found in the various specified files you want
to use to train the new classification model, name the new model, select parameters to
be applied during training, initiate the automatic training process, and view the results.

Important notes!

- An NVIDIA GPU with a minimum of 6 GB of VRAM is required to complete this tutorial.

- Depending on your computer, the training process could take several hours
(unattended) to complete.

1. Click the Add Class to Train button.

A new row is displayed in the Define classes table showing the found class, which is
labeled with the LAS (ASPRS) classification number and class name: 15 (Power
Towers)

Training Custom Classification Models 11

 Step 3. Train the custom classification model

Now you will name the new custom class you are going to train.
2. Click in the Target class name field and enter Stockpile.

Before starting the training process, you will review the settings to be applied to the
process.
3. Click to expand the Advanced Settings section to review and select parameters to
be applied to the automatic training process:

 Extra point properties for training - For training, the model uses the position of
each point by default, but you can select to include the RGB and/or Intensity
value if it helps identify objects of interest. RGB can be useful for consistently
colored objects (for example, red fire hydrants).
For this tutorial, keep the Intensity check box checked.
 Max epochs/Batches per epoch/Samples per batch - The recommended
number of epochs and batches is determined automatically when the training
and validation files are initially analyzed. However, you can use these controls to
edit these values depending on the desired results. Higher values require
stronger GPU hardware.
For this tutorial, enter the following values:
 Max epochs: 20
 Batches per epoch: 8
 Samples per batch: 5

Training Custom Classification Models 12

 Step 3. Train the custom classification model

 Save only the best model - Check this check box if you want to save only the
most accurate version of the model. This is the typical workflow, but there are
times when saving every model might be important. For example, if you are
training a model on several classes at once, the model accuracy is based on the
average of the sum of the accuracies for each new class. Consider a situation
wherein:
 Model 1: Class A has accuracy 95 %, Class B 45 %. Average 75%
 Model 2: Class A has accuracy 70 %, Class B 70 %. Average 70%
If you opted to save the best model, it would be Model 1. However, you might
find Model 2 a more reasonable choice as both classes have fairly good
accuracy.
For this tutorial, keep the Save only the best model check box checked.

You are now ready to start the training process. Depending on your comptuter and
VRAM, it could take up to several hours to complete.
4. Click the Train button to start the training process.
Training progress is displayed in the Status Bar, epoch by epoch.
Training a new classification model using deep learning is an iterative process,
much like studying for an exam. First the model "studies" the class samples in the
training files. Then, it tries to apply what it learned to extract the same types of
objects from the validation files. It then compares the objects it found in the
validation file with the objects already correctly classified in the validation file to see
how accurately it performed. It then takes what it learned from the validation
process, re-studies the training file, and again extracts the same types of objects
from the validation file to see how it has improved. This cycle, referred to as an
epoch, repeats as necessary until the maximum number of epochs has been
performed.
After the first train/validate epoch is complete, a Training and Validation Accuracy
graph is displayed during the rest of the training process and after. The red and
green lines indicate the accuracy of training and validation of the model at the end
of each epoch.
After one epoch:

Training Custom Classification Models 13

 Step 3. Train the custom classification model

You can hover your cursor over the graph to see more precise accuracy values.
After seven epochs:

5. Optionally, do either or both of the following:

 Click the Show TensorBoard button to display a TensorBoard in a web browser
showing quality measurements and visualization during and after the training
process.

Training Custom Classification Models 14

 Step 4. Use the new custom classification model

 Click the Open Log File button to open a log file showing progress and accuracy
information during and after training.

Note: Regardless of the value you entered for the Max epochs setting, you can stop
the training process at any time, especially if you are observing optimal results in
the Training and Validation Accuracy graph. In this case, the model resulting from
the most accurate epoch will be saved.

After the specified 20 training epochs are complete, the new custom classification
model is saved as a new folder in C:\Users\__\Documents\Trimble Business
Center\Custom DL models. (You can change this default location using Options >
Project Management).
6. Click the Model Folder button to open the folder's location.

Note that the file name includes the MIoU% validation accuracy rating.

Step 4. Use the new custom classification model

In this step you will use the new Stockpile custom classification model to extract and
classify stockpiles in a point cloud that includes stockpile features that have not yet
been classified.
1. In the TBC ribbon, select File > New. Then select to use the <Blank Template> for
your new project.
2. When prompted to save the current project, click No.
3. Open the ..\TrainingClassificationModels\Data folder and drag-and-drop the
[Link] file onto the 3D View tab to import a new point cloud that has not
been classified.
4. In the Point Cloud Scale for [Link] dialog, leave the Non-georeferenced
ground-scale point cloud option selected and click OK.
5. In the Projection Definition dialog, click OK.
You can see that the new point cloud includes several stockpiles.

Training Custom Classification Models 15

 Step 4. Use the new custom classification model

6. In the Extract Classified Point Cloud Region command pane, click the Classification
tab.

Training Custom Classification Models 16

 Step 4. Use the new custom classification model

7. In the Classification Type drop-down list, select Custom.

If more than one model folder is saved in C:\Users\__\Documents\Trimble Business
Center\Custom DL models, a Model drop-down list displays in the command pane
enabling you to select the appropriate model. If there is only one model in the
folder, it is automatically selected and the Model drop-down list does not display.

8. Click in the Point cloud region selection field and then, in the Project Explorer,
select the Point Cloud Regions > Never Classified node.
In the Classes to extract list, Stockpile is checked by default.

Training Custom Classification Models 17

 Step 4. Use the new custom classification model

9. Click the Classify button

Classification progress is indicated the TBC window Status Bar.

When classification is complete, the stockpile features in the point cloud have been
successfully extracted and classified as Stockpile regions.

This completes the tutorial.

Training Custom Classification Models 18