Nom : Owusu Manu

Prénom : Mandy-Ann

Practical Session - Proxmox 2 v2

Part 1: Using LXC containers

Creating a container
In this part, you won’t need the virtual Proxomx serveur that you created in the first session. You
can create your LXC container directly in the computer attributed to you (706vdi-iot-xxx).
Creating a container is done from the web interface, you thus have to open a web browser and
connect to [Link]
To create a new container, you need a template. You will use “debian-10-standard”, a template
directly provided by the company that distribute Proxmox.
You start by choosing the storage space where you want to put the template (remember that Promox
allows to specify which type of data can be written in a given storage).

In which storage space are you going to put the template ? : local storage

To download a template from the repository provided by Proxmox, you have to go the desired
storage, and go to the “content” menu. You should see a “template” tab and from here you should
be able to download the correct template. Note that the computer having already been used, the
template may have already been downloaded.
After downloading the template, you have to create the container itself (once again, it’s not a
container, but a template that you just downloaded).
Do it the simple way:
 general
◦ name : testCT1
◦ Be sure the « unprivileged container » is checked
◦ You have to provide a password for the “root” account in this container (2 times). Use
“tpiot” as the password.
 Template
◦ source : the file you just downloaded
 root disk :
◦ Only ont storage space should be available. Use it.
◦ Size : 2GiB will be more than enough.
  CPU
◦ 1 core
 memory
◦ 512MiB
 network
◦ select the vmbr1 bridge, it’s the one that allows to connect to the internet
◦ IPv4 address: [Link]/24 (where xxx is neither 1 nor any other number you
already used).
◦ Gateway : [Link]
◦ no IPv6 configuration
 DNS
◦ host parameters

Once you acknowledge, the container creation from the template should not take more than 30
secondes.

Bridges and network configuration

You can now start your container and open a console. You have to use the “root” account and the
password (tpiot) that you specified before.
Note have fast the container starts (a few seconds at max!).
Verify that your container can connect to the internet:

root@testCT:~# ping [Link]

Note that as you use the vmbr0 bridge, you can also communicate with the virtual machine that
your created in the first session. You have to start it to be able to communicate with it !
So start the Proxmox virtual machine from the last session (proxmox1) and connect to its web
interface. Inside, create a new container, similar to the one you created before. Use the same
configuration as your first container, expect for:
 name: testCT2
 IPv4 address: an unused address in the range [Link]/24
Note that all you machines can communicate with each others (proxmox1, testCT1 and testCT2).
Test it by using the ping command.
Complete the provided scheme with the IPv4 addresses of the various interfaces. You can use the
help of the “ifconfig” and “route” commands.
Those commands may not be available by default, you can install them ! (the are in the net-tools
package).
#apt-get update
#apt-get install net-tools

Processes ion non priviledged containers.

Inside testCT_yourName1, you can start a new software. :

root@testCT1:~# nano [Link]

This open a text editor.
Come back to the command line without closing the editor. You can do that by pressing CRTL-z.
You should be back to the command line. nano should still by loaded in memory but in a paused
state.
You can verify it by listing all the processes currently existing inside the conatiner :

root@testCT1:~# ps aux

Note that the “aux” option command tell the “ps” command that you want yto display all the
processes, even those of the operting system or belonging to other users.
You should see the “nanp” process. Write down its PID (Process Identifier) on a piece of paper.
You should also be able to observe around 20 other processes that are part of the operating system.
For exemple, the “init” process that always has the PID 1, along with others daemons that handle
the login screen and the command line interpreter (bash) among others.
You can also display the amount of available memory:

root@testCT1:~# free -h
The amount of used memory should be low, in the order of 40 or 50MiB. This is to be expected,
considering the low number of processes running inside this container.
Now open a console (sheel) on you physical server (706vdi-iot-xxx). From this shell, once again
display the list of all the processes: :

root@dom-proxmox1-c:~#
root@706vdi-iot-20:~# ps aux
ps aux
The list should be much longer.

root@706vdi-iot-20:~# free -h
Displaying the free memory shoul also confirm that you have much more (15GiB) but also that you
used more of it (a few GiB).
Carefully observe the list of the processes running on 706vdi-iot-xxx. In particular, look for the
“init” and “nano” processes (use “ps” in conjunction with “grep” to find them with ease).

Explain what you observe and give your interpretation:

On the physical server, the list of process is a lot more longer so the memory used also compared to
the memory used by the container. The physical server contain itself, the virtual machines and the
total processes that are running.

Precesses inside privileded containers

Do not close your first container and let “nano” run inside.
Now create a new container on 706vdi-iot-xxx. Do it the same way as before (except for the name:
testCT3 and the IP address that has to be unique). But do not check the “unprivileged container”
box at the beginning of the creation process
Start this container and open a shell on it. Start a new process inside:

root@testCT3:~# nano [Link] &

Come back to the shell on «706vdi-iot-xxx » and list again all running processes. Use “grep” to
filter only the “nano” processes.

What do you observe ? Look at the name or user id (it’s the first column displayed by “pas aux”).
How do you interpret this difference ?
What are the differences between a privileged and unprivileged container ? (you also in the
documentation !)

Unprivileged containers are when the container is created and run by a user. They are more secure
than privileged containers which are run by the root.
 Part 2 : Deploying a redundant network storage with
glusterfs

Context
As we have seen during the previous lecture, you have to ensure that your data are always available
to your hypervisors and to the Virtual Machines running user’s applications.
You can (and usually should) use redundant disks, as in RAID Systems. But this is not sufficient, as
your storage server itself can fail. You have to make your storage system even more resilient.
In this session, we are going to build a redundant storage system using GlusterFS. We are going to
use multiple storage servers, that replicate data. One server will be the primary one, and the second
one will be in standby. In case of failure of the primary one, the second one will take over in a
matter of seconds, seamlessly letting users access their data.

Network topology
The topology we are looking to
implement is as follow (only virtual
servers are represented) :
 proxmox1 is our first hypervisor
 gluster1 and gluster2 are the
storage servers
 all computers can access to the
internet through the blue network
(and blue interface cards)
 a gateway separates us from the
internet (it’s in fact 706vdi-iot-
xxx)
 a second (pink) network is dedicated to data transfer between the servers.

Creating the gluster servers

Our gluster servers will also be virtual machines (in a real deployment, they would be physical
servers, of course). To improve the performances, we will be using containers instead of KVM
Virtual Machines.
Creating those containers is similat to what we have done before:
  general
◦ name : gluster1 et gluster2
◦ be sure the « unprivileged container » is not checked. Gluster will not work inside if it
is.
◦ You have to procide the password for the “root” account (use “tpiot”) two times.
 Template
◦ source : debian-10-standard
 root disk :
◦ Only ont storage space should be available. Use it.
◦ Size : 8GiB will be more than enough.
 CPU
◦ 1 core
 memory
◦ 512MiB
 network
◦ select the vmbr0 bridge, it’s the one that allows to connect to the internet
◦ IPv4 address: [Link]/24 (where xxx is neither 1 nor any other number you
already used).
◦ Gateway : [Link]
◦ no IPv6 configuration
 DNS
◦ host parameters
Start those containers and verify they can access the internet.

Configuring the data network

On the previous drawing, you have seen that we intend to use a dedicated network for Data. For
now, our servers have only one network interface (blue), that is intended for internet connection.
We need to add a second network interface to each of our Gluster servers. To allow
communications through those new interfaces, they need to be connected to each other. For this, we
will bind them all to the vmbr1 virtual bridge from 706vdi-iot-xxx. The network topology should
be the following:
Adding new network interfaces to the Gluster servers is particularly easy. In the “network”
properties of your container, click the “add” button.
You have to give a name to the new interface (the new interface will be know by this name INSIDE
the container). Here, “eth1” will do.
You have to select to correct bridge. Select “vmbr2”
Insert the IP address for this interface. Note that we don’t use the same network as before. The blue
and pink networks are two sub-networks, independent from each other.
Adding a new network interface to the KVM proxmox1 server requires a little more work.
You have first to go to the “hardware” properties of your KVM, and select “network device” from
the “add” button.
As before, select the vmbr1 bridge on 706vdi-iot-xxx (we want this interface to be able to
communicate with its counterparts in the gluster servers).
You can also select the VirtIO paravirtualized model, as it is supported by the proxmox system
running inside the VM.
But contrary to a container, you can’t configure the IP address or other parameters from here, as
they are managed by the operating system running inside the VM.
You have to connect to your proxmox1 VM (through its web interface) and go to the
“system/network » properties.
You should see that a new ens19 interface appeared, so you will have create a new vmbr1 bridge
and bind it to ens19. This vmbr1 bridge will hold the IP address on the pink network. Choose one
that is not already in use !

To activate this new interface it may be safer to restart “proxmox1”.

One the new interfaces have been added and configured on your three virtual machines, verify they
work as expected by pinging their addresses on th [Link]/24 network.
Also verify that the IP routes are correctly configured. Check with the “route” command on all three
virtual machines.
One again, if it is not available, install it by using:

:~# apt-get update

 root@dom-gluster1:~# apt-get update
Get:1 [Link] buster/updates InRelease [65.4 kB]
Hit:2 [Link] buster InRelease
Get:3 [Link] buster-updates InRelease [49.3 kB]
Fetched 115 kB in 1s (197 kB/s)
Reading package lists... Done
root@dom-gluster1:~#

:~# apt-get install net-tools

This command should produce the following output:

root@dom-gluster1:/etc# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default [Link] [Link] UG 0 0 0 eth0
[Link] [Link] [Link] U 0 0 0 eth1
[Link] [Link] [Link] U 0 0 0 eth0

What is the meaning of the output of the “route” command ? (what do the last three lines mean ?)

The route command helps to display the current routing table.

The last 3 lines display the network that should be used to send packets. Default mean that if he
did’nt find the destination address, he will send it through the default address.

Installing GlusterFS on your servers

You will have to install more packages on your gluster servers. We need the “glusterfs-server”
package.

root@glusterDom1:~# apt-get install glusterfs-server

[...]

It should not take very long to install.

Before starting the service, you have to make sure that the gluster servers can talk to each other over
the correct interface (eth1).
We are not using a DNS server, but we can modify the “/etc/hosts” file to allow each gluster server
to recognize the other by its name.

root@glusterDom1:~# nano /etc/hosts

You have to do the symmetric operation on the other gluster server. Note that the name
“glusterDom1” and “glusterDom2” are not on the same network as “[Link]-
[Link]”.
Be careful to not have the same name associated with multiple addresses ! In the example,
“glusterDom2” and “[Link]” are two different names.
After saving the files, try the configuration by pinging the other server by name:

root@glusterDom1:~# ping glusterDom2

PING glusterDom2 ([Link]) 56(84) bytes of data.
64 bytes from glusterDom2 ([Link]): icmp_seq=1 ttl=64 time=0.090 ms

root@glusterDom2:/etc# ping glusterDom1

PING glusterDom1 ([Link]) 56(84) bytes of data.
64 bytes from glusterDom1 ([Link]): icmp_seq=1 ttl=64 time=0.074 ms

You can now start the gluster service on both servers:

root@glusterDom1:~# systemctl start [Link]

root@glusterDom1:~# systemctl start [Link]

You can verify everything went well:

root@glusterDom1:~# systemctl status [Link]

We just stared the service manually. But it will not start automatically at the next reboot. To allow
this, you need to do the following on both servers:

root@glusterDom1:~# systemctl enable [Link]

root@glusterDom2:~# systemctl enable [Link]
Now that the service is running, you have to make sure each server is aware and willing to work
with the other. For this, you have to use the command:

root@glusterDom1:~# gluster peer probe glusterDom2

You have to run this command on only one of your gluster servers. You then can verify it worked :

root@glusterDom1:~# gluster peer status

Number of Peers: 1

Hostname: glusterDom2
Uuid: 4acac522-7ff0-4899-8d32-2e399f3f5fa8
State: Peer in Cluster (Connected)

The service (daemon) is running, the servers know each other, but nothing is shared yet. We have to
add some storage (disk) to our gluster servers, then configure them to replicate it and share it to the
network.
The gluster server will share storage space to other computers through the network. Normally they
never share space from the disk that contain their operating system. We always install additional
disks, dedicated to data.
We will do the same here. Except that, as we are using LXC instead of a real server, we will add a
new mount point to a virtual disk instead of a real disk.
You have to go to the « ressources » properties of your gluster servers and add a new « mount
point » from the « add menu ».

Set the size to 16GiB and select the largest storage available.
Set the path to where this new volume will be mounted inside the container. Use /data
In the console of your container, you can verify that everything went well by using the « df »
command.
 root@glusterDom1:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/pve-vm--107--disk--0 7.9G 934M 6.5G 13% /
none 492K 0 492K 0% /dev
tmpfs 7.9G 0 7.9G 0% /dev/shm
tmpfs 7.9G 8.1M 7.9G 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup
/dev/mapper/pve-vm--107--disk--1 16G 45M 15G 1% /data

Here, the last line shows the 16GiB, mostly free, volume mounted under /data
It is necessary to repeat the same steps on the two gluster servers.
Moreover, the shared space has to be the same size on both servers, as one will be used to replicate
the other.
You now have configure the GlusterFS srevice so that it put this space into use. For that, you have
to create a gluster volume. We will use « gv0 » as volume name.
We want our system to have 2 replica of the data.
We want the data to be put into the /data directory we previously configured.
What command should you use to create the gluster volume (look in the glusterFS documentation) ?

gluster volume create gv0 replica 2 transport tcp gluster1:/data gluster2:/data

After creating your volume, you have to start it

What command should you use to start the gluster volume (look in the glusterFS documentation) ?

gluster volume start gv0

Your gluster servers are now up and running, they are sharing a 16GiB space.
To test it, you have to mount it and write files on it. For this, we will use your proxmox1 virtual
server.
Connect to the console of your proxmox1 virtual server and create a new directory under /mnt

:~# mkdir /mnt/glusterTest

This client computer has to be able to connect to the gluster servers by their name, so we will edit
/etc/hosts similarly to what we have already done.
Verify that you can ping each gluster server by name from proxmox1.
Finally you can mount the shared volume :

root@proxmox-demo2:/mnt# mount -t glusterfs glusterDom1:/gv0 glusterTest

Verify that the volume mounted correctly with “df” command :

root@proxmox-demo2:/mnt# df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.9G 0 1.9G 0% /dev
tmpfs 394M 5.7M 388M 2% /run
/dev/mapper/pve-root 7.6G 2.1G 5.2G 29% /
tmpfs 2.0G 43M 1.9G 3% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/fuse 30M 16K 30M 1% /etc/pve
tmpfs 394M 0 394M 0% /run/user/0
glusterDom1:/gv0 16G 205M 15G 2% /mnt/glusterTest

We want to check whether the replication works or not. We will write files in the shared directory
and see what will happend.
On the client (proxmox1), create a new file in /mnt/glusterTest

root@proxmox-demo2:/mnt/glusterTest# touch toto

Then go to your gluster servers and observe the content of /data/gv0

root@glusterDom1:/data/gv0# ls

root@glusterDom2:/data/gv0# ls

What do you observe in those directories ?

The same file is created on both glusters.

 We will now terminate abruptly the first gluster server, using the « stop » entry in the container
menu.

After having stooped the first Gluster server, continue to work with the shared directory on the
proxmox1 VM :

root@proxmox-demo2:/mnt/glusterTest# ls
toto
root@proxmox-demo2:/mnt/glusterTest# touch titi
root@proxmox-demo2:/mnt/glusterTest# ls
titi toto
root@proxmox-demo2:/mnt/glusterTest#

You should be able to continue reading and writing file.

Go back to the console of the still running gluster server and list the content of /data/gv0
The new files should be there.
Finally, restart your first gluster server and go look into /data/gv0
After a few seconds, the files you created while this server was down should appear here.
Explain what happened :

when server 1 was turned off, server 2 took over. So when server 1 starts up again, since they are
replica servers, server 1 will update its data according to the actions that took place on server 2. In
other words, it will copy the difference between itself and Server 2.