Lab 10 – Raft Consensus Algorithm

---------------------------------------------------------------------------
In this week’s experiment you will learn:
1. Raft and installing raftos

2. Run a raftos cluster and demonstrate leader election.

3. Verify that data is being logged/stored by all nodes involved

What is Consensus?
To quote a very wise man: “Sometimes, when two (or more) computers love each other very much,
they talk to each other over a network; and 1-RTT later, a very cute distributed system is born”.

A fundamental problem in distributed computing is to achieve overall system reliability in

the presence of a number of faulty processes. This is a natural consequence of life, because
machines fail. Power cords disconnect. Processes error out. Tsunamis happen. A vengeful
employee burns down a datacenter.

This often requires coordinating processes to reach consensus, or agree on some data value
that is needed during computation. Example applications of consensus include agreeing on
what transactions to commit to a database in which order, state machine replication, and
atomic broadcasts.

Consider a system of five servers, each of which is responsible for performing some computation.

1. A client sends out a message that says

Set COOLEST_HACKATHON = ‘somerandomhackathon’; this is registered by all five.

2. Later, the client sends another message saying

Set COOLEST_HACKATHON ‘HashCode’

Except, before the client sent this, one of the machines failed;

Later it came back online, after the message was received by everyone else. Now if that
specific machine is queried for ‘COOLEST_HACKATHON’, it would get a reply saying
‘somerandomhackathon’, which is just obviously not true.

Real-world applications often requiring consensus include cloud computing, clock

synchronization, PageRank, opinion formation, smart power grids, state estimation, control of
UAVs (and multiple robots/agents in general), load balancing, blockchain, and others.
 What is Raft?
You’ve learnt about Paxos in class. Raft is another consensus algorithm, just like Paxos, except
simpler to understand and implement. Here’s what the Raft website has to say: “Raft is a
consensus algorithm that is designed to be easy to understand. It's equivalent to Paxos in fault-
tolerance and performance. The difference is that it's decomposed into relatively independent
subproblems, and it cleanly addresses all major pieces needed for practical systems. We hope
Raft will make consensus available to a wider audience, and that this wider audience will be
able to develop a variety of higher quality consensus-based systems than are available today.”

Kubernetes’ etcd is implemented using Raft!

Prerequisites:
1. Pyenv/Conda

2. Tmux

3. Linux/Linux-VM/macOS system. If you HAVE to use windows, use WSL2.

A. The following screenshots are to be submitted in a PDF file:

1a: Pyenv/conda installed and local Python version changed to 3.6.8. tmux 6 panes

2a: Initial leader election has occurred, and first value has replicated.

2b: Logs and state_machine after initial leader election

3a: Leader crashed, new leader elected, and second value replicated.

3b: Logs and state_machine after initial leader crashed and new leader elected

4a: Old leader comes back online, but becomes a follower instead and updates its old

state to its new state.

4b: Logs and state_machine after old leader comes back online

Instructions:
Please read ALL the instructions carefully before proceeding.

Task 1: Raft and installing Raftos

1. 2. Go through this visualization ([Link] to understand
how raft works. After completing the above visualization, you should be familiar with

leader election and log replication, and how they work in Raft. You will be quizzed on

basic Raft concepts! If you’re interested in actually interacting and playing around with

a raft- setup, [Link] is your friend.

Navigate into the raftos directory that you downloaded in the pre-install section, and

activate the local python version as 3.6.8 using pyenv. If you don’t know what this

means, go through the preinstall guide.

Your terminal should then look something like this:

Linux Users:

Mac Users:

If you’re using Conda, you’ll need to do conda activate py3.8.11.

Navigate into the raftos directory and edit the [Link] file by changing the
version of the cryptography library from 1.5.3 to 3.3, because stuff has changed.

4. Now run pip3 install setuptools and then run pip3 install –r [Link]

5. Now, run python3 [Link] install in raftos after you’re done correcting the version

in [Link]. This should install raftos on your system. You may have to sudo

the command if it errors out.

Task 2: Seeing Raft in Action

1. We’re going to be writing a Python script that simulates the running of a basic
server. Each instance of the script corresponds to one node, i.e., one server that is
part of the cluster. The only thing differentiating them will be their node-id’s.

Each node maintains three descriptors; a .log file that corresponds to command logs;
a .storage file that corresponds to the node’s persistent storage; and a .state_machine
file that corresponds to the node’s state. Simply put (and this is a bit of an
oversimplification), the ‘command log’ contains the set of instructions sent by the
client; on applying that instruction, the state of the node changes.

For example, a command in the command log might be “Set X to 4”

And, that, on application, will change the state of X to 4.

2. Why do we log the commands too, instead of just applying? One intuitive reason (of
many) is for failure recovery. If a node fails and comes back, the state of the node may
end up being wiped; Applying these logs sequentially will bring the node back to its
former state. Since the main goal of this experiment is to understand Raft, the code
has been provided for you already; Place the [Link] file provided to you in the raftos
directory. Also place the start_terminals.sh file provided to you in this directory.
Please change the SRN in the start_terminals.sh file accordingly.

3. Replace it in [Link] with your SRN too. This is important for evaluation. Before
executing any runs of the experiment, make sure to run start_terminals.sh. You may
need to provide it with executable permissions, with sudo chmod +x, then
execute ./start_terminals.sh

if tmux panes are not displaying then try to copy paste the whole code to
terminal from the start_terminals and run
5. This should open up 6 tmux panes in that terminal. Don’t fret (I don't ever wanna see you
And I never wanna miss you again), you can easily navigate between each pane by first
pressing Ctrl+b (or if that doesn’t work, control+b or control + \ for Mac users) and
using the arrow keys to change panes. Each time you need to change panes, you’ll
need to ctrl+b, unless you do it in quick succession. Do not press ctrl+b+arrow key
together, instead press the arrow key a second after the ctrl+b. Pressing them together
resizes the window. If you’re using Conda, you’ll need to do conda in every tmux pane
in the first column. (yes, again) (Take the screenshot 1a)

6. Make your tmux session full screen.

7. Run tail -f node1_CUSTOMLOG.log, tail -f node2_CUSTOMLOG.log and tail -f
node3_CUSTOMLOG.log in three panes in the right-hand side column, in that
order from top to bottom. 8. Run python [Link] --node 1, python [Link] --
node 2 and python [Link] --node 3 in the three panes in the first columns, in
that order from top to bottom. Now, your tmux session should look something
like this:
8. In my case, Node 2, i.e [Link]:8001, was elected leader, and hence
this happens: (Take a screenshot, 2a)
 9. We’ll do as suggested and check the .log and .state_machine files: It’s alright
if your term varies! That’s just an artifact of the election process itself, which
you’ll hopefully have understood via the visualization. (Take a screenshot, 2b)

10. We see now that the initial value has been replicated across all the nodes. Now it’s
time to demonstrate leader election. Go back to your tmux session, and stop the leader process
with ctrl+c. This simulates leader failure. Wait, and you should see a new leader elected, and
the second value replicated across the cluster! In my case, the old leader ’s node 2 (but yours
may be different), so it’ll look something like this: (Take a screenshot, 3a)

11. As before, check your .state_machine file and your .log file; Ta da! New values
have been replicated across the nodes that are still up. (Take a screenshot,
3b)

The best part is, if you were to revive the old leader (for me, node 2) now, by starting the
process again, it’ll notice that its term is lesser than the current leader’s term (for me, node
1), become a follower, and also update its state to reflect the new value! (Take a
screenshot, 4a)

Logs and state_machine (Take a Screenshot 4b)

 FAQ and Common Issues
1. .storage or .log files regenerate by themselves:

You closed the tmux session without ending the processes. You’ll need to end

them manually. For that, do:

ps -ef | grep python

Note down the process ids. Then, for each process number,

do: kill -9 <process_id>

2. Python version is not switching to 3.6.8:

You likely forgot to add certain lines to the .bashrc or .zshrc file; check the

pre- install guide again.

3. Other installation errors:

PES wi-fi may have blocked certain PPAs and repos; try switching to hotspot.

4. Screenshots 2b, 3b, and 4b :

In screenshots 2b, 3b, and 4b, all six files have been displayed simultaneously within a
single VS Code window using the split view feature. This arrangement has been
achieved by distributing the files across multiple horizontal and vertical panes within
the editor. You can do this by drag and dropping each file across the vertical or
horizontal panes. You can display each log individually too.

5. Tmux panes:

If somehow more than 6 panes are generated, you can close the tmux pane using Ctrl
+ b then type "X” followed by typing “y” and clicking enter. ( Please do not type
the “ “ , it's just written here to make the command more conspicuous )
Ctrl + B + “ to add a horizontal pane if somehow you spawned with less than 6
panes.

Mac users can use Control + b or Control + \ depending on what is configured in your
system. After that follow the same step as the one above.
And that’s your Raft Lab done and congratulations on making it through.

Credits: CC Faculty, CC TA group ‘25, CC TA group ‘24