Artificial Intelligence for Beginners (1): What Is AI?
Artificial Intelligence for Beginners (1): What Is AI?
The use of artificial intelligence is on the increase in many areas, and it is finding its way into many types of device and application. This year it is estimated that
worldwide business applications in the field of artificial intelligence will generate revenues of some $4.8bn, and that figure is predicted to rise to over $30bn by
2025. Anyone with knowledge of the subject area can look forward to enhanced employment prospects as expertise in artificial intelligence is urgently sought after.
It is therefore worthwhile to take a little time to get to grips with the topic and the new worlds of possible applications that it opens up. And last but not least, it’s a
lot of fun!
The term ‘artificial intelligence’ is generally considered to cover the creation of systems exhibiting intelligent behaviour. One part of this is ‘machine learning’,
whereby instead of having an application function specified in all its detail by the developer, a general-purpose programming framework in the form of a neural
network (NN) is used, which learns the required function by itself using a large corpus of training data. One class of machine learning is called ‘deep learning’,
where more complex and optimized program structures are used to improve results further. We will be looking at these ideas in more detail later in this short series
of articles.
However, artificial intelligence is not a new subject: its beginnings can be traced back to the 1950s. The breakthrough that occurred a few years ago was the result
of considerable increases in the amount of computational power available. Graphics cards with hundreds of processor cores operating in parallel and highly-
specialized artificial intelligence chips enabled artificial intelligence systems to be implemented and trained. That opened the door to automatic speech recognition,
as used in many personal assistants, and image recognition of objects of all kinds. artificial intelligence also plays an important role in autonomous vehicles. In
some areas AI has already surpassed human abilities: artificial intelligence systems can beat world champions in the games of chess and go, and can detect
tumours more reliably than human medical specialists. But do not worry: the systems are only superior in the particular areas in which they have been trained, and
the areas in which the systems are trained will continue to be chosen by humans.
A low-cost introduction
A good low-cost introduction to this world is the Maixduino, which in the form of the ‘MAix BiT’ kit including the board, camera and small LCD panel can be
obtained from Elektor: see the text box. The board has the same format as an Arduino Uno (from which it gets its name), but the hardware it includes is
considerably more comprehensive. The board is manufactured by the Chinese company Sipeed. Alternatives to the Maixduino include the Nvidia Jetson Nano, the
ROCK PI N10 Model A, the Intel Neural Compute Stick 2 and others, but these all cost considerably more.
In this first instalment of this series we will look at the rich hardware environment offered by the Maixduino and how it can be programmed with the help of the
Arduino IDE. As well as a couple of typical Arduino-esque applications we will look at how to use the camera and the screen. And the final highlight will be a
demonstration of how to use the board to recognize objects.
In the second and third instalments we will delve deeper into the topic of artificial intelligence, describe the structure of neural networks, install MicroPython and its
accompanying IDE and demonstrate how facial recognition works. We will also look at how you can program your own AI applications and how to communicate
with the Internet.
A deep dive into the ‘super Arduino’
The specifications of the Maixduino will have hardware enthusiasts licking their lips! The overall size and construction of the board is broadly aligned with that of
the Arduino Uno, but you will immediately notice the higher component density and that many additional connections are available. At the heart of the board there
are two large modules. The first is the Sipeed M1 AI processor module based on the Kendryte K210 device, whose interior workings we will look at later, and the
second is an ESP32 module for communicating over WLAN and Bluetooth and for acquiring analogue signals. The ESP32 contains two processor cores clocked
at 240 MHz and so by itself offers a considerable amount of processing power, and it can be used to offload communication functions from the main processors.
The ESP32 has already featured in many articles in this magazine, and so we will not go into further detail about it here.
The header connectors correspond in number and arrangement to the original Arduino, and the pin assignments are mostly identical. But beware that the inputs
and outputs are designed for 3.3 V or even 1.8 V operation, and the input circuitry will be destroyed if 5 V is applied.
Further details are shown in Figure 1. Two 24-pin sockets are provided for interfacing to the camera and to the LCD panel. A slot is provided for a microSD card
which can be used to make a large amount of storage available to the Maixduino. The USB connector is the modern type-C kind and is used for programming and
monitoring. And, to allow the processing of audio data, the board includes a digital microphone and an audio amplifier with a 3 W output driven by a digital-to-
analogue converter. So there is plenty of hardware on the board to let you get started on a range of possible projects without having to add extra boards.
Figure 1: The hardware facilities offered by the Maixduino.
The pinout of the board (see Figure 2) closely resembles the original. The supply voltage can be provided over the DC barrel jack or using the VIN pin at 6 V to
12 V; alternatively, 5 V can be supplied over the USB connector. Six of the ESP32’s GPIOs are brought out; these are the ones that can alternatively be used as
analogue inputs A0 to A5. On the opposite edge of the board are the inputs and outputs of the K210 module. These can be controlled using the usual Arduino
commands, with the ‘Arduino pin number’ being the number of the input or output bit of the K210.
Figure 2: Maixduino pinout.
#define LED 12 // K210 IO12, Maixduino pin 10
pinMode(LED, OUTPUT); // configure port as output
digitalWrite(LED, HIGH); // port to high (3.3 V)
The RST pin is designed to operate at 1.8 V and external circuitry should not apply a higher voltage than that to it. Outputs IO36 to IO47 are also designed for low-
voltage operation: these are not brought out to the headers, but are used internally, for example to drive the LCD panel. The serial RX and TX ports and the I2C
interface are provided with suitable pull-up resistors fitted on the board.
Unfortunately the K210 datasheet does not specify the maximum output current available at the outputs. However, they should be capable of driving an LED at up
to say 10 mA or so; for higher currents a driver IC should be added.
The big enchilada: Kendryte K210
The K210 SoC (system on a chip) made by the Chinese company Kendryte is at the heart of the Maixduino. It is made using a low-power silicon technology on a
28 nm process and has been available on the market since September 2018. For carrying out ‘normal’ work it offers two 64-bit processor cores that are clocked at
400 MHz, with overclocking up to 800 MHz possible. These cores are based on the RISC-V specification: this saves the manufacturer from having to pay licensing
fees to Arm and so helps to contribute to the low total cost of the device. Both processors come with an FPU (floating-point unit) that operates in both single and
double precision. Figure 3 shows in more detail what is inside this chip.
Figure 3: Block diagram of the Kendryte K210.
The special feature of the K210, however, is its KPU (knowledge processing unit) that can be used to construct and execute neural networks. The total compute
power available is astonishing at this price point: 0.46 Tops, or 460 billion operations per second. With overclocking this figure can even be doubled, allowing, for
example, up to 60 objects per second to be recognized. The high processing performance is achieved using 64 arithmetic units operating in parallel and a bus
width of 576 bits. Also, the power dissipation of just 0.3 W is very low in comparison to other artificial intelligence systems. For example, Nvidia recommends using
a 5 V 4 A (hence 20 W) power supply for its Jetson Nano, which at 0.4 Tops offers comparable computing power.
The KPU can implement advanced neural network architectures, including convolutional networks. These have a particularly efficient filter structure that makes
them especially well suited to image processing applications: more on this in the second instalment of this series. The main memory included in the SoC has a
capacity of 8 MB, divided into 2 MB for the main processors and 6 MB for the KPU. That means that up to 5.9 MB is available for storing the neural network
configuration, which is enough to implement a medium-sized network.
However, the K210 has more hardware tricks up its sleeve. These include an audio processing unit (APU) which is particularly helpful for pre-processing in speech
recognition applications. The unit can handle up to 8 channels (or 4 stereo channels) at input sample rates of up to 192 kHz. An FFT (fast Fourier transform) unit is
available to carry out analysis of the frequency spectrum of a signal.
Also noteworthy are the AES and SHA-256 accelerators provided to speed up cryptographic functions.
And of course the device includes the usual complement of peripherals including UART, I 2C, SPI, I2S, timer, RTC and PWM.
Sipeed has added 16 MB of flash memory on the board alongside power supply circuitry, the microphone and 3 W power amplifier, and the excellent ESP32
module.
What more could you wish for? More information on the operation of the board, the circuit diagram and other details can be found here, here and here.
Software development
It is not just the shape of the board that is similar to the original Arduino: the board uses the same Arduino IDE, with the Maixduino core being integrated into the
environment in the same way as the ESP8266 or ESP32. Under File -> Preferences it is necessary to add a new board manager URL: if you right-click on the
button to the right of the text entry box a small window will open to make entering the URL easier (see Figure 4). If you only want to program the Maixduino, then
only the ‘sipeed’ line needs to be added; then the on-board ESP32 will also be programmed using it.
Figure 4: Adding the Maixduino core configuration.
To install the Maixduino board tools select the menu item Tools -> Board -> Boards Manager. Enter the search term ‘Maix’ (as shown in Figure 5) and then
proceed to install the tools.
Figure 5: Installation of the Maixduino core tools.
Now we can start to do some programming. Rather than the conventional ‘hello world’ program, we will jump straight into to testing the camera and screen.
Connect the Maixduino over USB and in the Tools menu set the following parameters.
Board: Sipeed Maixduino
CPU Clock Frequency: 400 MHz
Burn Tool Firmware: open-ec
Burn Baud Rate: 1.5 Mbps
Tool Install Location: Standard
Port: <COM port that you are using>
Programmer: k-flash
A demonstration program that captures an image and displays it on the LCD is already available within the IDE. Call it up using File Examples Sipeed_OV2640
selfie and once it is uploaded to the board (you may need to press the reset button at this point) you can run it. The code is shown in Figure 6.
Figure 6: Arduino IDE with the ‘selfie’ program.
The program starts by bringing in the functions from the Sipeed libraries for driving the camera and LCD, which are connected to the board using an SPI bus. The
image format chosen is QVGA resolution (320 by 240 pixels) with RGB565 colour. The setup routine initializes both devices, and in the main infinite loop the
captured images are transferred directly to the LCD: it could hardly be simpler.
The image is not particularly contrasty, but it is sharp and updates are smooth. Figure 7 shows the results.
Figure 7: Captured image of an adaptor plug.
As you can see, the Maixduino is as easy to use as an Arduino Uno. However it offers many more possibilities and we can immediately start to build more
sophisticated applications.
Our first AI model
We will be looking in more detail at deep learning aspects of artificial intelligence in the second article in this series, but here we can demonstrate a simple
application. We will be using ‘MobileNet’: nothing to do with mobile phones, but rather an image classifier that can recognize and identify 1000 types of everyday
objects. This uses a neural network, which is a software structure built from nodes organized in layers trained using a lengthy process in which it is presented with
thousands of images.
Since the file containing the training image set runs to some 200 GB and the training process itself is very time-consuming, it is not really practical to carry it out on
the Maixduino. However, it is possible to install a ready-trained model and immediately set it to work recognizing images. The steps to achieve this are as follows.
The required software can be found here. It consists of the following components.
mobilnet_v1.ino: main C++ program for running the demonstration
MBNet_1000.h: header file for the demonstration routines
MBNet_1000.cpp: C++ routines for acquiring, recognizing and displaying objects
names.h: header file with object descriptions
[Link]: C++ routine for describing the recognized objects
Download these files and place them all in the same directory. The pre-trained artificial intelligence model mobilenet_0x300000.kfpkg can be downloaded from
link [5]: compressed, it is just a few megabytes long. After decompressing it with 7zip you should find a folder mobilenet_0x300000 in which there are two files: the
model is the one called ‘m’. Copy this file onto a microSD card in the root (top-level) directory and insert the card into the slot on the Maixduino.
A glance at the program code in Figure 8 shows that it is possible to program even very complex applications with little effort. That is of course all down to the
powerful libraries: in the field of artificial intelligence there are very many highly efficient libraries available. The first part of the program declares the camera, LCD
and the KPU along with their parameters. The camera resolution is configured to the same format as the training images, 224 by 224 pixels. Finally, the
object mbnet marshals together the KPU, camera and LCD resources.
Figure 8: MobileNet demonstration program.
The setup routine initializes the demonstration code, and then in the main loop we have the classification of the objects in the images acquired by the camera and
their display on the LCD panel. Again, it is hard to imagine that an object recognition program could be any simpler.
As a test we can put a photograph of a cat in front of the camera (as shown in Figure 9). The system immediately recognizes it as a tabby or Egyptian cat
(Figure 10). The application is capable of classifying about five images per second and is therefore capable of producing useful results even if the camera is
moving. The most important things are to have good lighting and not too busy a background to the image.
Figure 9: Acquisition of an image of an object.
Figure 10: The object is identified and classified.
As can be seen from Figure 10 the acquired image uses only a blue and white colour palette, in the interests of increasing the recognition rate. This is a
commonly-used method: instead of using all three colour channels in an image we use only one to reduce the amount of pixel data and hence the amount of
processing power required. The technique of using a ready-trained network, or in other words of carrying out training and classification on different platforms, is
also widespread in this type of application. Many users take advantage of the large amounts of processing power available on AWS, Microsoft Azure or Google
Cloud to train their neural networks, while running them (which requires considerably less in the way of resources) on smaller-scale platforms. The semiconductor
industry is already responding to this demand: Intel recently announced their Nervana NNP-T and NNP-I neural network processors, where the ‘T’ version offers
higher processing power for training, and the ‘I’ version is a lower-performance device aimed at inference and classification applications.
Coming up...
The demonstration application we have looked at in this article can only scratch the surface of the topic of artificial intelligence, but with luck your interest in the
possibilities of machines that learn has been piqued.
In the next instalment of this series we will look at the structure and function of a neural network in more detail. That will come with a new development
environment: we will be introduced to the temptations of Linux and the Python programming language, as that is where the most powerful libraries and frameworks
are available. These frameworks make the creation of a neural network as easy as plugging together Lego bricks. We will also look at a face recognition
application and at how to develop your own neural network structures. If you are too impatient or curious to wait for that, the book Make Your Own Neural
Network by Tariq Rashid is recommended.
Artificial Intelligence for Beginners (2): Deep Learning
The first article of this small series introduced the hardware of the Maixduino and showed how to program it in C++ with the Arduino IDE. The performance of the
processor was demonstrated using an artificial intelligence (AI) model for object recognition. How artificial intelligence works in deep learning will be discussed in
this installment. If you want to seriously deal with AI, Linux and Python are essential. But with the appropriate tools there is no magic wand.
Please do not expect to have fully grasped Deep Learning after reading the following sections. To do so, I had to study a few books and tutorials as well as
numerous programming attempts (and I still don't know much). But I would like to give you at least an overview which structures and methods are used to provide
a universally applicable program in the form of a Neural Network (NN) with specialized intelligence.
Structure of a neural network
A neural network (NN) consists of several layers, each of which has several nodes. In Figure1 the nodes of a layer are arranged vertically. Theoretically, there can
be any number of nodes per layer and any number of layers in the network. The architecture is determined by the respective task and the size is of course also
dependent on the resources of the computer platform used. For example, if the NN is to classify objects that it receives by picture, the first layer takes over the
data for solving the task and provides a corresponding number of input nodes for acquisition. If a low-resolution image consists only of 28 * 28 pixels, then 784
nodes are required for a display in grey values. For the acquisition of colour images, the number would triple. During image acquisition, each input node receives
the gray value of "its" pixel.
Figure 1: Structure of a neural network.
The result of the image analysis is presented in the output layer, which has a separate node for each result. If the NN were to recognize 1000 objects, this layer
would have the same number of output nodes. Each node shows a probability between 0 and 1.0 and thus indicates how reliable NN is in the result it has found.
For example, when a house cat is taken, the associated node could have a value of 0.85 and the "Tiger" node a value of 0.1. All other nodes would then have
even lower probabilities and the result would be unambiguous.
The actual analysis work is done by the hidden layers during deep learning. Depending on the task, any number of hidden layers can be used; in practice, there
are certainly 100 to 200 such layers in use. In addition to the architecture shown above, more complex structures with feedback or intermediate filters to increase
accuracy can be found. The number of nodes in each hidden layer is also arbitrary. In the above-mentioned example for object detection, three hidden layers with
200 nodes each could already provide useful results.
If in the human brain intelligence is achieved by linking neurons by means of synapses, the nodes in the NN are also networked with each other. Each node of a
layer is connected with all nodes of the following layers. Each connection (shown as an arrow in the picture) contains a value called weight. These weights
between the layers are stored in matrices. Therefore, in the field of deep learning, programming languages are preferred where matrix operations can be
performed easily and quickly.
So how does an NN achieve the desired results? Each node receives its input signals from all nodes of the layer in front of it, marked x in Figure 2.
Figure 2: Calculations within a node ([Link]
These values are multiplied by the weight values w and added, so the net input[Link]
net = x1*w1 + x2*w2 + X3*w3 ...
The calculated value of net is now multiplied by an activation function and sent to the output (and thus to all nodes of the subsequent layer). A node-specific
threshold value also determines whether this node "fires".
The activation functions shown in Figure 3 are assigned to the layers and ensure that the output values remain within the desired range. For example, the Relu
function suppresses all negative values and Sigmoid has a limitation between 0 and 1. These precautions ensure that NNs operate in a clear numerical range and
cannot be dominated by " runaways".
Figure 3: Frequently used activation functions.
Training
So after the input data has been recorded, NN performs numerous calculations in each shift and presents the results at the output. This process is called
inference. And how does the intelligence get there? This is done like a small child through education and training. With an untrained NN, the weights are usually
filled with random numbers in the value range -1 to +1, so the network is dumb and therefore only provides random results. For training, data and known target
output values are fed to the network. After calculation with these values, the result is compared with the setpoint and the difference is documented by a loss
function. Now follows the learning process, also called backpropagation. Starting in the last layer, the weights are adjusted in small steps to minimize the loss.
After many (often millions of times) of training rounds with different input data, all weights have the values appropriate to the task and NN can now analyze new
data unfamiliar to it.
A particularly suitable architecture for image and audio processing is the Convolutional Neural Network, which is also supported by Maixduino. In such a CNN, the
neurons (at least in some of the layers) are arranged two-dimensionally, which fits to two-dimensional input data such as an image. Compared to the network
indicated in Figure 1, where the activity of a neuron depends on all neurons of the previous layer (via different weighting factors), the dependency is simplified and
locally limited in the case of CNN. Here, the activity of a neuron depends only on the values of (for example) 3 x 3 neurons, which are located in the layer before it
— the weighting factors are the same. With such a network, small-scale structures such as lines, curves, points and other patterns can be detected particularly
well. In the subsequent layers, more complex details and finally whole faces are detected.
In terms of programming, an NN is similar to a spreadsheet, an arrangement of cells with predefined calculation instructions, which access multidimensional
matrices with the weights during execution. When classifying new input data, all layers from input to output are passed through and the outputs reflect the
respective probability of the result assigned to the node. Well-trained NNs can reach values around 0.9. During training, after the interference, backpropagation
takes place with a run from the back to the input to adjust the weights.
Sounds complicated at first, but there are numerous tools and libraries available for the implementation, which I will introduce in the following paragraphs.
Linux and Python
Already in the first article of this miniseries I pointed out that you have to be prepared for a learning effort when dealing with the topic of artificial intelligence. The
easiest way to do it in this world is on the Linux platform, because here most of the tools are available for free and of good quality. Furthermore, Linux offers the
same comfort as Windows, but is packaged differently (usually better). My preference is Ubuntu, which is also available as LTS version (Long Term Support) and
is supported for at least 4 years. But also the other derivatives like Debian, Mint etc. are just as useful, the choice is a matter of taste. Linux can be installed in a
virtual machine besides Windows, for example, so you don't need an extra computer.
And why Python? Python is an interpretive programming language, some will say, so potentially slow. This disadvantage is offset by numerous advantages: Firstly,
Python does away with the frills of brackets and semicolons and performs block building by indenting the lines. It has powerful data structures such as lists, tuples,
sets and dictionaries; furthermore, Python already has fully integrated matrix calculation. Other major advantages are the available artificial intelligence
frameworks and libraries, which can be fully or partially integrated due to their high modularity. These are mostly written in C++ and therefore have the required
performance. And all this can be installed easily with just a few instructions.
If you want to get started, please install your favourite Linux as well as pip3 and Python 3, instructions for this are abundantly available on the net.
Maixduino speaks MicroPython
To enable Python to run on systems with less memory, a lightweight version called MicroPython is available; it can be installed on platforms like Maixduino, ESP32
and others. MicroPython contains a balanced repertoire of commands and 55 additional modules for numerous mathematical and system functions. To add new
versions or artificial intelligence models, the flashing tool Kflash is required.
Installation of Kflash under Linux:
Download version 1.5.3 or higher from web link as an archived file kflash_gui_v1.5.3_linux.[Link].
Transfer to a folder of your choice.
Unpack by command tar xvf kflash_gui_v1.5.3_linux.[Link].
Change to the newly created folder /kflash_gui_v1.5.2_linux/kflash_gui.
Start with ./kflash_gui (in case of startup problems, check the box Run file as program in the properties of this file).
This starts the graphical user interface of Kflash (Figure 4) and you can now load firmware or artificial intelligence models into the Maixduino.
Figure 4: Kflash shell.
Installing firmware on Maixduino
Even though the Maixduino is already equipped with MicroPython at delivery, you should always download the latest version. When writing these lines, the
corresponding firmware is named v0.5.0 and can be downloaded via the web link. Choose maixpy_v0.5.0_8_g9c3b97f or higher and in the next picture choose the
variant maixpy_v0.5.0_8_g9c3b97f_minimum_with_ide_support.bin or higher, a file of about 700 kB, which also contains support for the MaixPy-IDE.
With the help of Kflash the new firmware is quickly installed. The Open File button is used to select it and after setting the board, port, baud rate and speed
mode as in Figure 5, a click on Download starts the loading process.
Figure 5: User interface of the MaixPy-IDE.
After that you can already execute the first Python commands with a terminal emulator (e.g. Putty) via the port /dev/ttyUSB0 on the Maixduino. Here is a small
example with array commands:
>>> # Python-Prompt
>>> import array as arr # Import Array Module
>>> a = [Link]('i',[1,2,3]) #
>>> b = [Link]('i',[1,1,1]) #
>>> c = sum(a + b) # Create sum of all array values
>>> print(a,b,c) # and output it
array('i', [1, 2, 3]) array('i', [1, 1, 1]) 9 # output
>>>
With libraries like numpy (under Linux) or umatlib even more functions are available.
Installation of MaixPy-IDE
Development and tests are much more comfortable with the development environment called MaixPy-IDE. Python programs can be developed and tested as well
as loaded and executed on the Maixduino. Additionally, as shown in Figure 5, tools for image analysis are available. The installation is performed as follows:
Download the version maixpy-ide-linux-x86_64-0.2.4-installer-archive.7z or higher from web link.
Transfer to a folder of your choice.
Unpack by the command tar maixpy-ide-linux-x86_64-0.2.4-installer-archive.7z.
Change to the new folder maixpy-ide-linux-x86_64-0.2.4-installer-archive and enter the following commands:
./[Link]
./bin/[Link]
After that the IDE starts. For subsequent starts, only the latter command is of course required.
Thus, all tools for implementing artificial intelligence models are available. In the following a face recognition is tested.
Face recognized!
An trained artificial intelligence model is used for face recognition, which has already analyzed several thousand faces for characteristic features - the weights of
the NN used are adjusted accordingly. The model is available for download under the web link under the name face_model_at_0x300000.kfpkg. The basis of the
development is the AI framework Yolo2 (You Only Look Once), which divides the image objects into several zones, analyzes them separately and thereby
achieves high recognition rates (I will talk about the artificial intelligence frameworks later in this series). The artificial intelligence model for the AI processor is
packed in kfpkg format and must be flashed to address 0x300000 in the Maixduino. This can also be done with Kflash; just find the file using Open File and load it
onto the board using the parameters from Figure 4.
The MaixPy-IDE is used for comfortable handling of the Python script. With this IDE you can develop and test programs and transfer them to
Maixduino. Figure 5 shows the interface divided into three windows:
Editor, top left: In this area the program input is done with syntax highlighting.
Terminal, below: Display of the program output.
Image Analysis, right: This is where the display of images and their spectral division into the colours red, green and blue is performed.
Among other available buttons and menu items, the two buttons on the lower left are important: The "paperclip" is used to establish (colour green) or disconnect
(red) the connection to the Maixduino via the port "ttyUSB0". A green triangle below it starts the script; then this button changes to a red dot with "x" and serves to
stop the program.
To perform the test, I printed the faces of two publicly known people (Albert Einstein and Rudi Völler) and pinned them to a wall. The selection was purely
coincidental, but both faces are said to resemble somewhat, which is not confirmed by me. When taking these pictures, the faces were immediately recognized
and marked with a frame. It is important to make sure that the images are displayed in landscape format as shown, otherwise the recognition rate drops noticeably.
The program [Link] can be found in the download folder on the Elektor website. Its brevity again reflects the performance of the libraries used. To start,
the required libraries for camera, LCD and KPU are integrated and initialized. After that, the NN is loaded into the KPU starting at address 0x300000. When the AI
model is initialized by the command kpu.init_yolo2, additional constants are transferred for setting the accuracy and optimization. Now the image classification is
carried out in an endless while loop, an image is taken and fed to the NN. If faces have been detected, the variable i receives for each face the coordinates and
size of a marker frame, which is then drawn into the image. Finally, the image (on the LCD panel) and the marking data (on the serial terminal) are output. More
details about the KPU commands can be found under the link.
In the MaixPy-IDE the picture is also shown in the upper right corner and the corresponding colour spectrum is displayed below. If you do not need this
information, the right image window can be switched off with the deactivate button.
For better handling I mounted Maixduino and LCD on a small board and aligned the camera to the front (see Figure 6). This makes it easy to capture and analyze
real faces, printed images or screen contents.
Figure 6: Test setup for face recognition.
You can see the display on the LCD panel in Figure 7, similarities of the persons are not visible.
Figure 7: Representation on the Maixduino LCD.
But what is behind this Yolo2 model? The neural network has 24 convolutional layers and two fully connected output layers (see Figure 8). In between there are
some maxpool layers as filters to remove complexity and to reduce the tendency to "memorize". It is noticeable that a window size of 3x3 is largely used for detail
recognition. Exactly this is supported by the KPU hardware, which ensures that the Maixduino is highly efficient for such tasks.
Figure 8: Network architecture for facial recognition (source: [Link]
Other known NN structures even have several hundred layers, have feedback paths or other extras. There are no limits to creativity in this area, much depends on
the budget, i.e. computing power.
And on we go!
The powerful hardware and the already available software environment show that the Maixduino is well suited for the entry into Artificial Intelligence. Due to its low
power consumption it is well suited for use in mobile devices with already trained neural networks. In a third part of the series I will show you how to develop, train
and execute your own neural network. The interface to the developer is the AI framework Keras, which is also known as a comfortable "Lego construction kit for
AI". You will also get hints on how to program ESP32 on the board - for example to acquire analogue values.
Stay curious!