How to message in the cloud

Introduction to Message oriented middleware

+
microservices rabbitMQ Kafka [Link]
Objectives
- Learn basic concepts in distributed
systems and messaging
- Learn in high level about Kafka,RabbitMQ
and [Link] and understand when to use
each
 Presentation overview

- Overview of distributed systems and their characteristics

- Overview of different messaging patterns in distributed systems
- RPC vs shared resources
- Synchronous vs asynchronous
- Orchestration vs choreography
- Overview of message oriented middleware(MOM)
- What are they
- Messaging schemas
- Overview of popular message oriented middleware
- Kafka
- RabbitMQ
- NATS
- Evaluate key difference between the popular middlewares
 Distributed system

- A collection of components located on different networked computers that

communicate and coordinate their task by passing messaging to one
another via a network to achieve a common goal
 Distributed system examples

- Process running on a single machine

for example database management systems

- Components can be physical machines

spread across a network for example kubernetes
 Characteristics of Distributed Systems:
Heterogeneity

- Distributed systems components differ by (Networks,Computer hardware,Operating systems,Programming languages). Distributed system
should mask those differences by common interfaces(middleware).

Openness

- Distributed systems should be easy to extend and improve by using detailed interfaces

Transparency

- Distributed systems should be perceived by users and application programmers as a whole rather than as a collection of cooperating
components.

Scalabilty

- Distributed systems should be designed to not be changed when the scale of the system changes,i.e the horizontal scale(increase the
number of components) and Vertical scale( increase the resources of the system)

Concurrency

- Distributed systems components execute in a concurrent manner to complete a task. Components access and update shared resources
(e.g. variables, databases, device drivers) which can produce unexpected behaviour when concurrent updates are not coordinated.
 Characteristics of Distributed Systems:
Failure handling

- Distributed systems components can [Link] systems must maintain availability even at low levels of
hardware/software/network [Link] systems must be fault tolerant(recoverable,redundant

Security

- Distributed systems components communicate using messaging channels. Security is required for concealing messaging(security
and privacy) , authentication and more
 Messaging patterns used in distributed systems

- Distributed system have multiple communication patterns between them.

- The following patterns will be explained:
- RPC versus shared resources
- Synchronous versus asynchronous
- Orchestration versus choreography
RPC versus shared resources
Synchronous versus asynchronous
Orchestration versus choreography
 Distributed systems networking architectures

- There are many different architectures for distributed system each with its
own use case and advantage
 Message oriented middleware(MOM)

- Distributed systems contain many components written in different

programing languages running on different hardware and operating
systems(heterogeneous).
- Middleware is a logical abstraction between heterogeneous apps that
allows apps to communicate in spite of the differences described above
- Middleware simplify network topology in a distributed systems(service
bus)
 General architecture of message-oriented middleware

- Producers: processes that send data

- Consumers: processes that recieve data
- Broker: containing routing logic and queues
- Messages: the data entries that is being exchanged between
the applications are called messages.
 Message-oriented middleware delivery modes

- Point-to-point (Queue): Point-to-point delivery

is when a producer sends a message and only one
consumer receives it e.g. one-to-one
communication.
- Publish/Subscribe: In this delivery mode
consumers subscribe to a [Link] a producer
sends a message to the topic, a copy of the message gets
forwarded to all consumers that have subscribed to that
topic e.g. one-to-many communication.
 Message-oriented middleware message characteristics:
performance
- Performance in MOM is exclusively defined by latency and throughput
- Latency: Latency is the time it takes for a message to travel from the
producer to the consumer.
- Throughput: Throughput is generally defined as the amount of data that are transferred in a given
time.

Latency:?
Latency:?
Throughput:?
Throughput:?
 Message-oriented middleware message characteristics:
Delivery schematics
- At most once : when a MOM is using at-most-once delivery schematics it does messages<=1
not guarantee that all messages will arrive at the consumer, messages can get
lost during failures. Lowest latency
- At least once: When a MOM is using exactly-once delivery schematics all
messages are guaranteed to be delivered to the consumer without any duplicates. 1<=messages
This requires a complicated acknowledgement and transaction scheme from the
producer to the [Link] latency
- Exactly once: when a MOM is using at-least-once delivery schematic all the
1<=messages<=1
messages are guaranteed to be delivered to the consumer. This is achieved by a
two-step acknowledgement scheme. The broker acknowledges all messages it
receives from the producer and the consumer acknowledges all messages it
receives from the [Link] latency
 Message-oriented middleware message characteristics:
Message ordering
- Some MOM guarantee message arrival
- Some MOM do not guarantee message arrival order
 Message-oriented middleware message characteristics:
Persistence
- Persistent storage of the queue and its messages
- Some MOM can store the entire queue in persistent storage
- Some MOM can only store the entire queue in main memory
- Some message can store the in both memory
- It is important to know the relationship between message guarantees and performance.
- Message guarantees often come at the cost of performance because it entails additional
overhead. For example, going from at-most-once to at-least-once
- Message ordering is another guarantee that consumes resources. Ordering requires
additional processing steps to sort messages in the broker
 Message-oriented middleware message characteristics:
Scalability
- Sometimes distributed system need to increase their workload
- MOM are built to support both both horizontal and vertical scaling
- Therefore,some MOM allow clustering modes where replicas
coordinate as one message broker to support increase in workload
 Review question

- GetTaxi is a company that allows people to match with tax drivers on demand,right away

The business wants the following capabilities:

- The user should match with a close by driver

- The pricing should “surge” if the number of drivers are low or the number of users is
high
- All the position data before and during the ride should be stored in an analytics store
so that the cost can be computed accurately
- How would you implement this using message broker?
 Review question solution

-
 Kafka

- [Link]
- Kafka was written in scala.
- Open sourced by linkedin in 2011 and as of today is maintained by
Confluent,IBM,Cloudera
- Licensed under the Apache 2
- The project had been stared :23.4k times and forked 12k times
- Used by companies such as: IBM,Paypal,Netflix,airbnb
Kafka architecture
 Kafka architecture

Topics

- Messages in kafka between producers and consumers are sent using data streams.
- Kafka can have multiple data streams.
- Data streams are not bound by a particular format
- A particular data stream in kafka is called topic
- Topics behave like queues
 Kafka architecture

- Topics are split into partitions

- Messages within each partition are ordered
- Each message within a partition gets an incremental id,called offset
- When a message is sent to to a partition it is assigned an offset.
- Offsets allow to keep track of
message ordering within the partition .
- Offset are used by consumers to
keep tract which message it has
consumed
- Once data is written to a partition it
cannot be changed
- Data is kept for limited time
 Kafka architecture

Brokers

- Kafka runs as a cluster of brokers

- Each broker contains certain topic partitions
- Each brokers manages data replication, topic/partition management, offset management.
- Brokers can span multiple data centers
 Kafka architecture

Producers

- They write data to topics

- Producers decide which partition to write to
- Producers can send a batch of messages
 Kafka architecture: Consumers

Consumers

- Consumers read data from a topic

- Consumers control what offset to read from
- Consumers can receive batched or compressed data\
 Kafka messages

- Messages are created by producers

- Messages are sent as binary over tcp
- Messages can be compressed
- Messages can be at most 1 MB
 Kafka message delivery models

- There are 3 delivery semantics in kafka :

- At least once
- Offsets are committed after message is processed
- If the processing goes wrong,message will be read again
- Messages might have duplicates
- At most once:
- Offsets are committed as message is received
- Messages might be lost
- Exactly once:
- Messages are committed twice(after message is
received and after message is processed)
 Kafka cluster mode

- Kafka is natively built with cluster mode

- Cluster mode consistent of a master broker(control) and slave brokers
- Master is chosen per topic

-
 Kafka Multi clustering

- Kafka does not natively supports multi cluster and requires plugins
- Kafka allows only a mode called cross-cluster mirroring
- Cross cluster mirroring is just copying of data from one cluster to another

-
 Why is Kafka fast ?

Reason 1: sequential access pattern

- Kafka utilises sequential data access and not random access

- Sequential data is faster in orders of magnitudes compares to random access
 Why is Kafka fast ?

Reason 2: zero copy principle

- Kafka utilises zero copy principle to improve performance

- Using the transferto() function avaliable by later versions of java file contents can be
copied into a kernel buffer by the DMA engine.
- Sequential data is faster in orders of magnitudes compared to random access
 Review question

- GetTaxi is a company that allows people to match with tax drivers on demand,right away

The business wants the following capabilities:

- The user should match with a close by driver

- The pricing should “surge” if the number of drivers are low or the number of users is
high
- All the position data before and during the ride should be stored in an analytics store
so that the cost can be computed accurately
- How would you implement this using Kafka?
 Review question solution

-
 RabbitMQ

- [Link]
- RabbitMQ was written in Erlang
- Open sourced by Rabbit Technologies Ltd (now vmware) in 2007 and as of today is
maintained by vmware
- Licensed under the Mozilla public license
- The project had been stared :10k times and forked 3.7k times
- RabbitMQ is used by AT&T ,Cisco, Reddit ,American Airlines and more
 RabbitMQ Architecture

-
 RabbitMQ Architecture

Producers

- They write data to the RabbitMQ broker

- They can attach key to the sent messages

Consumers

- Consumers read data from the RabbitMQ broker

- They read data from specific queues
 RabbitMQ Architecture

Broker

- Receives messages from producers and sends them to consumers

- Brokers are composed of the following:
- Exchange
- Queues
 RabbitMQ Exchange & queues

Exchange

- Part of the broker

- Exchanges receiver messages directly from
producers
- Exchanges send messages to queues
- Exchanges can be “bound” to queues

Queue

- Part of the broker

- Exchanges receiver messages from ezhanges
- Consumers read data from queues
 RabbitMQ Exchange distribution
models
● Exchanges can send messages to queues using different strategies
○ Fanout: Simply routes a received message to all queues that are bound to it.
Ignores routing key
○ Direct: Routes a received message to the queue that match “rounting-key”.
Nameless exchange is a “direct” exchange.
○ Topic: Routes a received message to queues, where binding key (defined as a
pattern) matches to the routing key. Example, binding-key: “*.[Link]”, routing-key:
“[Link]”
 RabbitMQ Messages

● RabbitMQ natively works with the AMQP protocol

● RabbitMQ also supports other protocols such as
AMQT,STOMP
● Message limit is 2GB
 RabbitMQ cluster mode

- RabbitMQ is natively built with cluster mode

- There are couple different types of cluster modes:
- Classic Queue Mirroring(deprecated):
- Quorum Queues
 RabbitMQ Multi clustering

- RabbitMQ allows communication between clusters using federation mode

- This mode is basically upstreaming and downstream of messages across data centers
 RabbitMQ Persistence and Durability

Durability

● AMQP property for queues and exchanges. Messages in durable entities can survive
server restarts, by
● being automatically recreated when server gets up.

Persistence

● Messages property. Stored on disk in special persistency log file, allowing them to be
restored once
● server gets up. Persistence has no effect a non-durable queues.
● Persistent messages are removed from a durable queue once they are consumed (and
acknowledged).
 RabbitMQ message delivery models

- There are 2 delivery semantics in RabbitMQ:

- At most once
- Messages are sent from producers to consumers
- Message arrival is not verified
- Message arrival is guaranteed to not have duplicates
- At least once
- Messages arrival is verified with ack
- Messages might have duplicates
 RabbitMQ advanced features

- RabbitMq supports “smart” queues options

- Priority queues:
- Messages can be sent with priority
- Values range from 0 to 10
- Dead Letter Queues(DLQ)
- Used for storing poison messages
- Delayed queues:
- Not an actual type but can be implemented using TTL for messages with DLQ
-
 Review question

- GetTaxi is a company that allows people to match with tax drivers on demand,right away

The business wants the following capabilities:

- The user should match with a close by driver

- The pricing should “surge” if the number of drivers are low or the number of users is
high
- All the position data before and during the ride should be stored in an analytics store
so that the cost can be computed accurately
- How would you implement this using RabbitMQ?
 Review question solution

-
 [Link]

- [Link]
- Nats was written in Go
- Nats was Open sourced by Synadia in 2011 and as of today is maintained by
Synadia(also me)
- It is part of the CNCF
- Licensed under the Apache 2.0
- Nats comes in two versions: core and jetstream
- The project had been stared :11.7k times and forked 1.1k times
- Nats is a simple and lightweight alternative to RabbitMQ and Kafka
- Nats is used by companies such as Alibaba, Ericcson, Tinder and Tesla
[Link] Architecture
 [Link] Architecture

Producers

- They write data to the [Link] broker

- Producers write data to a specific subject or token

Consumers

- Consumers subscribe to subjects

- Consumers read data from the broker

Broker

- Connects to producers and consumers using native golang channels

- Receive messages from producers and sends them to consumers
 [Link] Architecture: subjects

- Messages in [Link] between producers and consumers are sent to a nats-broker

- Each message must have a subject
- At its simplest, a subject is just a string of characters that form a name which the
publisher and subscriber can use to find each other.
It helps scope messages into streams or topics
- Messages are sent as binary over tcp
- Messages order of sending and receiving is guaranteed
- Each message in of the following format:
 [Link] Architecture: subjects

- For example a publish message:

- For example a subscription message:

 [Link] Architecture: subjects

- Subjects can be either of the following:

- Exact names: such as PUB [Link]

- Wilds cards matching a single token

-
-
-
- Wilds cards matching multiple tokens
-
 [Link] message paradigms

- Using its messaging format and smart routing capabilities [Link] is capable of using the
following message sending paradigms:
- Publish-Subscribe:

- Request-Reply:

- Queue groups
 [Link] message delivery models

- There are 1 delivery semantics in [Link] core:

- At most once
- Messages are sent from producers to consumers
- Message arrival is not verified
- Message arrival is guaranteed to not have duplicates
- If Jetsteam mode is enabled them [Link] also has the following semantic::
- At least once
- Messages arrival is verified with ack
- Messages might have duplicates
 [Link] Clustering

Cluster mode

- [Link] supports clustering mode

- Nodes can be connected inside a
network to create a cluster
- Each node can forward messages to
one adjacent node
- Clustering in quorum based
 [Link] Clustering

Super Cluster mode

- [Link] also support a super cluster mode(cluster of clusters)

- Super cluster has different node types:
- Broker
- Gateway
- Leaf Node
 Review question

- GetTaxi is a company that allows people to match with tax drivers on demand,right away

The business wants the following capabilities:

- The user should match with a close by driver

- The pricing should “surge” if the number of drivers are low or the number of users is
high
- All the position data before and during the ride should be stored in an analytics store
so that the cost can be computed accurately
- How would you implement this using [Link]?
Review question solution
Kafka vs RabbitMQ vs [Link] Features
Features Kafka RabbitMQ [Link]

Language developed in Scala Erlang Go

Started In 2011 2007 2011

Popularity 23.4k stars 10.1k stars 11.7k starts

Open source license Apache 2.0 Mozilla public license Apache 2.0

Messaging paradigms supported pub/sub pub/sub pub/sub

Message queue Message queue Message queue
Request- reply

Message protocols Binary over TCP/IP AMQP, STOMP, MQTT TCP/IP,MQTT,Websocket

Message size max 1 MiB 2 GiB 1 MiB

Message Delivery At most once At most once At most once

At least once At least once At least once
Exactly once
Kafka vs RabbitMQ vs [Link]
Features Kafka RabbitMQ [Link]

Language support 17 30 40

Message Storage Disk Disk Disk

In memory In memory

Distribution units Topics Quese Channels

Clustering Supported Supported Supported

Super clustering(clusters of clusters) No-Supported Not-Supported Supported

Quality of documentation High High Medium

Ease of use Hard easiest Easy

Ease of installing and maintaining Low High High

Docker image size 443MB 231MB 12.3MB

 Kafka vs RabbitMQ vs [Link] Performance

- Calculating performance for each broker might is not straight-forward

- Each broker container many modes which can impact performance
- In this section we will show the performance results of kafka vs RabbitMQ vs [Link] under different scenarios
- In the article “Performance evaluation of message-oriented middleware” the performance is measured
- The testing was done on the following hardware:

- We will explain use the results of this article

Kafka vs RabbitMQ vs [Link] Performance:
throughput graphs
Kafka vs RabbitMQ vs [Link] Performance:
latency graphs
Kafka vs RabbitMQ vs [Link] Performance:
summary
Scenario Kafka RabbitMQ [Link]

64B Throughput Latency Throughput Latency Throughput Latency(when no

persistent
otherwise it is
worst)

8B Best Worst Worst Medium Medium Best

64B Best Worst Worst Medium Medium Best

512B Best Worst Worst Medium Medium Best

4096 Worst Worst Best Medium Medium Best

32768B Medium Worst Best Medium Worst Best

1048000B Worst Worst Best Medium Medium Best

Conclusion-what to use?

- Is persistence important?
- What size of message will the system use?
- Should the system focus on throughput or
latency
- Is being lightweight important?
- Can messages be lost?
Bonus questions