https:/sunshinelearning.

in/

AWS architect certification training

• Course outline
Module 06 https:/[Link]/

Introduction to AWS
Security management in AWS
Object storage options
Amazon EC2
Load blanching , Auto-scaling and route 53
Database service and analytic
Networking and monitoring service
Application service and AWS lambda
Configuration management and automation
AWS Architectural designs -1
Topic
https:/[Link]/

Following are the topics covered in this module :

• Database and types of AWS database service
• Need of managed database service • Different between SQL and NoSQL databases
• Amazon RDS and its benefits • Amazon DynamoDB : data structure and DynamoDB API
• Amazon RDS database engines • Read consistency and calculation of throughput capacity
• Database schemas within DB engines • Benefits of DynamoDB, use case and working
• Amazon RDS – DB instance • How to create tables and load data in DynamoDB
• Read replica • Amazon ElastiCache : Memcached and Redis
• DB instance – regions, Availability zones and failover scenarious • Memcached v/s Redis
• Backup and restore • Data Warehouse
• How to store replication data in MySQL DB via AWS RDS • Amazon redshift and its feature
• Aurora database • Amazon redshift Architecture
• Storage and I/O traffic in Aurora • Streaming Data
• Crash recovery in AWS aurora • kinesis
• How to visualize the web traffic using kinesis data stream
Objectives
https:/[Link]/

After completing this module, you should be able to:

1. Understand the working of Amazon RDS
2. Describe Amazon Aurora
3. Perform CRUD operation on Amazon DynamoDB
4. Implement Amazon ElasiCache
5. Understand Amazon redshift
6. Work with AWS kinesis
 https:/[Link]/

Let us first understand what is a

Database
What is a database?
https:/[Link]/

A collection of individual data items which is stored in a highly structured manner is called as database

1. Its provides the ability to store large amount of information

2. Allows to quick access to information

3. Lets user to share information at different location

4. It ensure security data

5. Database is classified as relational database and Non-relational database

 https:/[Link]/

Types of AWS database services

 https:/[Link]/

Types of databases services By AWS

RDS Dynamo
Elasticache
• Manages relational database • NoSQL database
• In-memory cache
• Handles structured and tabular data • Document and key-value store
• Improve the performance of any web application
• Handles structured data

Redshift
• Data warehouse Aurora
• It is used for data analysis and reporting • MySQL compatible relational database with 5x performance
 https:/[Link]/

Type of AWS Database services :

Amazon RDS
 https:/[Link]/

Why managed database services?

App optimization
Scaling
High Availability
Database backups
When you choose DB software patches
on-premise DB software installs
database, you have
to consider all OS patches
these aspects OS installation
Server maintenance
Rack and Stack
Power, HVAC, net
 https:/[Link]/

Why managed database services?

App optimization
Scaling
High Availability
Database backups
When you deploy
database on EC2 DB software patches
instance some DB software installs
portion of OS patches
management is
done by AWS OS installation
Server maintenance
Rack and Stack
Power, HVAC, net
 https:/[Link]/

Why managed database services?

App optimization
Scaling
High Availability
Database backups
When you choose DB software patches
managed database DB software installs
services 90% of
management is OS patches
done by AWS OS installation
Server maintenance
Rack and Stack
Power, HVAC, net
Amazon RDS
https:/[Link]/

Amazon RDS is a relational database management service which manage relational database for users

 On buying a server you get CPU, Memory, IOPS and storage all packed together, but RDS provides them separately so
that they can be scaled individually

 The role of RDS is to look after the software patching , update, backup, recovery and automatic failure detection

 You can either manually create a backup via snapshot or can have an automated backup performed

 It includes a primary instance and a simultaneous secondary instance in order to provide high availability and avoid
failure

 It is mainly used to manage data for e-commerce, gamming, apps, websites and many more
Amazon RDS benefits
https:/[Link]/

 Simple and fast deploy

 Fast and predictable performance

 MySQL, PostgreSQL, Oracle, SQL servers are available under RDS

 Pay only when you use the service

 Patching , Backups and replication are handled by AWS

 Simple and fast to scale

 https:/[Link]/

RDS database engines

Amazon RDS database engines
https:/[Link]/

Database engine is an
underlying software
component used by a database
to create, read, update and
delete data from it
 https:/[Link]/

Database schemas within DB Engines

Here schema refers to organization of data to construct a database

Database Engines Number of Schemas

Amazon aurora No limit imposed by software
MySQL No limits imposed by software
Maria DB No limits imposed by software
Oracle 1 database per instance no limit on number of
schemas per database imposed by software
SQL server 30 database per instances
PostgreSQL No limits imposed by software
 https:/[Link]/

Database Engine Versions supported

Database Engine Version
Amazon Aurora Compatible with MySQL and PostgreSQL engines
MySQL 5.5, 5.6, 5.7
Maria DB 10.0, 10.1, 10.2
Oracle 11g, 12c
SQL Server 2008 R2 SP3 GDR 10.50.6560.0
2012 SP4 GDR 11.00.7462.6
2014 SP2 CU10 12.00.5571.0
2016 SP1 CU7 13.00.4466.4
2017 RTM CU3 14.00.3015.40
PostgreSQL 9.3, 9.4, 9.5, 9.6, 10.1, 10.3 11 Beta 1
 https:/[Link]/

RDS database instances

Amazon RDS – DB instance
https:/[Link]/

Database instance is set of memory structures that manages the database

1. It is basic building block of RDS which runs on a DB Engine

2. The computation and memory capacity of a DB instance is determined by its

DB instance class (selected as per the need)

3. Each DB instance can host multiple user created databases for a single oracle
database with multiple schemas

4. Each DB instances runs on DB engines

5. By default a customer can have 40 RDS instances

 https:/[Link]/

Amazon RDS – database instance class

1. The computation and memory capacity of a DB instances is decided by the DB instance class

2. The necessity of DB instance class depends on users processing power and memory requirement

3. Its supports mainly three types of storages : Magnetic, general purpose and provisioned IOPS

4. Instance classes offered by AWS RDS are -

Standard Memory Optimized Burstable performance

db.m4, db.m3, db.m1 Db.r4 and db.r3 Db.t2

 https:/[Link]/

Amazon RDS : Read Replica

What is read replica?
https:/[Link]/

Read Replica database is the copy of master database which reflects changes done to the master database instance

 Read Replica reduce the load on source DB instance

 By taking the snapshot of source DB instance (master

database) Amazon RDS creates a read only instance from the
snapshots

 Whenever , the source DB instance is updated the

corresponding read replica is also update via asynchronous
replication method

 The read replica allows only read connections

 https:/[Link]/

DB Instance Region and Availability

zones
 https:/[Link]/

DB Instance – Region availability zones

 Any Amazon RDS activity initiated (creating DB instances or listing
available DB instances) runs only in your current default region

 The default region can be changed in the console, by setting the EC2
region environment variable

 Amazon RDS provides high availability and failover support for DB

instances by maintaining asynchronous standby replica in multiple-
availability zone deployments

 This standby replica is provided in different availability zone

 https:/[Link]/

DB Instance – Failover Scenarios

Whenever following conditions occurs AWS RDS automatically switches from primary DB instances to standby replica
present in another availability zone-

If primary DB instance fails

If OS of DB instance is undergoing

On availability zone blackout

Software patching

A DB instance server
Type is changed

Note : Normal failover times are 60-120 seconds (this may be exceeded in case of heavy recovery process
 https:/[Link]/

RDS Backup and Restore

 https:/[Link]/

Amazon RDS backup and restore

 AWS RDS carries the automated backups of DB instances as per the specified backup retention period

 The backup retention period can be set between 1-35 days

 Even manually backups can be created via snapshot

 When a DB instance is deleted the automated backups too get deleted with it but manual snapshots remain the same
Billing of Amazon RDS
https:/[Link]/

Parameters Billing procedure

DB instance hours Based on the class, full hour will be considered if DB

instance is consumed for partial hour

Storage (per GB per month) Scaling your provisioned storage capacity within the
month, will be billed pro-rated

I/O request per month Total no of storage I/O request

Data transfer Data transfer in and out your DB instance on internet

 https:/[Link]/

Demo – Storing Application Data In

MySQL DB via AWS RDS
Demo – Steps
https:/[Link]/

1. Go to AWS management console and click on “RDS”

2. Select the data base engine

3. Fill the required details

4. Click on “launch DB instance”

5. Install WAMP 64 and give path of its location in command prompt

6. Enter the endpoint, username, port and password to connect AWS RDS and the WAMP server

7. Once the connect is done then perform CRUD operation on it

Note : please refer to Demo 1 in module 6 demo document

 https:/[Link]/

Types of AWS Database services:

Amazon Aurora
Aurora Database
https:/[Link]/

Amazon Aurora is a relational database which is managed by RDS

• It is MySQL-Compatible and delivers up to 5 times the

performances of MySQL

• It supports cross region read replica

• Offers simple monthly charge for each Amazon Aurora

database instance usage

• High performance and cost-effective

 https:/[Link]/

Storage and I/O Traffic In Aurora

 https:/[Link]/

Amazon Aurora Storage Engine

 Amazon Aurora consist a storage volume of 10 GB logical
blocks (it can scale Up to 64 TB when required

 The data is replication in 6 storage nodes allocated in 3

availability zones in parallel

 S3 manages the backup periodically

 Continuous monitoring of these nodes is done to overcome

repairs

Note : The reason why Amazon Aurora runs faster then MySQL is it sends and writes only the log record to storage
node
I/O Traffic In Aurora
https:/[Link]/

• In each of the storage node the record first enters an in-memory queue
• Persisted record is carried to update queue and primary instance is
acknowledged
• The records are organized and sort group identifies gaps in log. Hot log
carries the record that is to be stored
• If log sequence number of the storage node is lost then they are retrieved
from other nodes via gossip protocol
• Log record are coalesced and used to create data page
• The log records are backed up asynchronously to Amazon S3
• Periodically garbage collects old versions and current records are stored in
data books
• When all writers are considered successful an acknowledgement is returned
to client application
 https:/[Link]/

Crash Recovery In AWS Aurora

 https:/[Link]/

Crash Recovery – Traditional v/s Aurora Database

AWS Aurora
Traditional database
• As part of disk read, redo of records is done
• Replay logs since the last checkpoint
on-demand
• Generally takes 5 minutes between
• Operations are parallel, distributed,
checkpoints
Asynchronous
• MySQL works with single-thread ; number
• Does not replay on start-up of server
of disk access are very high
 https:/[Link]/

Types of AWS Database Services :

Amazon DynamoDB
 https:/[Link]/

Different Between SQL and NoSQL Databases

Characteristics SQL NoSQL

Workloads Adhoc queries, data warehousing, OLAP Web-Scale applications

Data Model Well-defined schema where data is Schema-less with a primary key and
normalized into tables, row and columns manages structured and semi-
structured data

Data Access SQL (structure query language) AWS management console or AWS
CLI and perform adhoc tasks

Performance Optimized for storage Optimized for compute

Scaling Vertical scaling Horizontal scaling

Amazon DynamoDB
https:/[Link]/

DynamoDB is a fully managed NoSQL database service offered by AWS

 It is used by system which require milli second read latency

Example: gaming websites where you have to display quick updates

 The record in every row is known as item

 Here you can set TTL (Time To Leave) to delete items automatically in the table once they expire

 Operations such as create, insert, update, query, scan and delete are performed in table via

appropriate API

 For faster performance and data durability the table data is stored in SSD disk and spread across many

servers in different available zone

 Data structure in DynamoDB
https:/[Link]/

Primary key
Attributes

Partition
key
Sort key Schema is different per item

Product The
Type Jane Eyre See Me
ID Notebook
Book ID
1 HP20111
Kamikaze Bloom

Album ID Invasion
2 crg123
Items

of Privacy
Album ID
2 crg999 Sybil Lucy Perfume
Movie ID
3 TRBlkmr
 https:/[Link]/

DynamoDB API
 DynamoDB API https:/[Link]/

Control plane lets you to create and manage DynamoDB table

• Create table: Used to create a table which includes table name, primary key, throughput settings
Control plane
• Describe Table: Used to view the details of the table

• Update Table: Used to modify settings of table or index and DynamoDB stream setting for table

• Delete Table: Used to remove an unused table (it is an unrecoverable process)

• List table: It returns the name of DynamoDB tables for the current AWS account and region

• Describe limits: It return the current read and write capacity limits for the current AWS account
and region
 DynamoDB API
https:/[Link]/

Data plane lets you to perform CRUD actions data in a table

• Creating data:
Put Item – to create an item
Batch write item – to create up to 25 items in one or more tables
Data Plane
• Reading data:
Get Item – To read an Item
Batch get Item – To read up to 100 items from one or more tables
• Uploading Data:
Update Item – To update an Item
• Deleting Data:
Delete Item – To delete an item
Batch write Item – delete up to 25 items in one or more tables
DynamoDB API
https:/[Link]/

A DynamoDB streams are used to replicate the data from a table to table in another region

Here the API’s used for data transfer are –

• List stream – Retrieves a list of stream descriptors for the current account and endpoint
DynamoDB
streams • Describe stream – Retrieves detailed information about a given stream. The output
includes a list of shards associate with the stream, including the shards IDs

• Get shard iterator – Retrieves a shared iterator, which describe a location within shard

• Get Records – Retrieves the stream records within a given shard

 https:/[Link]/

Read Consistency In DynamoDB

 https:/[Link]/

Read Consistency in DynamoDB

DynamoDB supports both Eventually consistent Reads and strongly consistent reads

Eventually Consistent Read

It provides the stale data instead of recently added
data in the DynamoDB ( if you repeat your read
request after a short time, the response should return
the latest data)
Strongly Consistent Read
Here, DynamoDB returns a response with the most up-
to-date data, reflecting the updates from all prior write
operation that were successful
A strongly consistent read might not be available if
there is a network delay or outage
Throughput Capacity
https:/[Link]/

Example :
A read capacity unit If a table is created with 6 write capacity
On creating a table in
represents one strongly
DynamoDB you are supposed units and 6 read capacity units. Then with
consistent read per seconds,
to specify the capacity
or two eventually consistent these settings, an application could :
requirements for read and
reads per seconds, for an item  Perform strongly consistent read of up to
write activity
up to 4 KB in size
24 kb per second ( 4 kb x 6 read capacity
units)
 Perform eventually consistent read of up
Throughput capacity limits is One write capacity unit to 48 kb per second ( twice as much read
specified in terms of read represents one write per throughput)
capacity unit and write second for an item up to 1 KB
capacity unit in size  Write up to 6 kb per second ( 1 kb x 6
write capacity units)
 https:/[Link]/

Amazon DynamoDB – benefits

Amazon DynamoDB – use case
https:/[Link]/

Users login into Duolingo for learning

Users can learn their preferable language
For their preferable languages

For high scalable database and high performance

Store 31 billion items in support of an online site delivering lessons for 80 languages
How DynamoDB work?
https:/[Link]/

Create table Add query Monitor &

Items Manage Table
 https:/[Link]/

Demo: Creating Tables, Loading Sample

Data and Running queries
Demo – DynamoDB
https:/[Link]/

1. Go to AWS management console and select the DynamoDB service

2. Click on “Create table” and enter the table name and primary keys

3. Now select Items and click on “ Create item” to insert data into the table

4. If the data is inserted successfully then you can read the inserted data from dashboard

5. If you want to remove an item from table then click on “Remove”

6. If you want to delete the table then click on “ Delete Table”

Note : For more detailed steps refer AWS Architect Module 6 demo 2 on LMS
 https:/[Link]/

Types of AWS Database Services :

Amazon ElastiCache
Amazon ElastiCache
https:/[Link]/

ElastiCache is a cache-in environment used to cache results in order to reduce overhead and latency on database

 ElastiCache is a web service that helps to improve the

performance of web applications by allowing the user to
retrieve information from fast, managed in-memory
caches

 It helps to set up , manage and scale a distributed in-

memory cache environment in the cloud

 ElastiCache supports two open source in-memory

engines: Redis and Memcached
 https:/[Link]/

Amazon Elasticache – Memcached

 https:/[Link]/

Amazon Elasticache – Memcached

Memcached is General purpose distributed memory caching system used to speed Up the Dynamic data driven websites

1. Memcached is simple to use and multithreaded

2. Memcached cluster can have a maximum 100 nodes in a region

3. Its support both horizontal and vertical scaling

4. It is instantly fast and is well established

 https:/[Link]/

Amazon Elasticache – Redis

Amazon Elasticache - Redis
https:/[Link]/

Redis is an in-memory data structure store, used as database, cache and message broker

 It is single threaded and its read replicas are synced asynchronously

 Collection of 1 to 6 Redis nodes is called as shard

 When cluster mode is enabled 1 to 15 shards can be used and when it is disabled
you can have only one shard

 Backups are stored in S3 with a retention period of 0 to 35 days

 https:/[Link]/

Demo – Redis cache

Demo –Redis cache
https:/[Link]/

1. Go to AWS management console and click on “ ElastiCache”

2. Select Redis cache

3. Launch an EC2 instance

4. Download and install GNU compiler collection

5. Download and install Redis CLI

6. Connect to your Redis cache

7. Run your Redis commands

Note : for more detailed steps refer AWS Architect module – 6 demo 3 on LMS
 https:/[Link]/

Memcached v/s Redis

Memcached v/s Redis
https:/[Link]/

Characteristics Memcached Redis

Description In-memory key-value, Originally In memory data structure store, use as
intended for caching database, cache and message broker

Replication Does not support replication Supports master-slave replication

Storage type Stores variables in memory and Redis is like a database that resides in
retrieves information directly from memory
server instead of DB

Read/Write speed Good to handle high traffic websites Neither can handle high traffic on read
nor heavy writes

Key-Length Memcached’s key length has a Redis has a maximum of 2 GB

maximum of 250 bytes

Ideal for Caching relatively small and static data, Session cache, full page cache (FPC),
such as HTML code fragments queues, Leader boards/ counting and
more
 https:/[Link]/

Popular use cases of Elasticache

Ad Tech Internet of things Gaming Mobile and web

• Ad serving • Tracking state • Recording game • Storing user
• ID looking • Metadata and details profile
• User profile reading from • Leaderboards • Session details
management million of • Session • Personalization
• Session tracking devices information settings
• Real time • Real time • Usage history • Entity-specific
bidding notification • Logs metadata
 https:/[Link]/

Types of AWS Database Services :

Amazon Redshift
What is a Data warehouse?
https:/[Link]/

Data warehouse is a repository where data generated by different sources is collected, transformed and stored for
organizations decision making process

 It reduces total turnaround time required for analysis and

reporting

 It is also used to store historical data, which helps user while

analysing the data

 It provides different time periods and trends to make future

predictions

 It is used in industries like Airline, Banking, Healthcare, Insurance,

Retail etc
AWS Redshift
https:/[Link]/

Redshift is fully managed peta-byte scale data warehouse service by AWS used to do analysis on data

1. Redshift is a combination of nodes and cluster

2. The total storage capacity of a cluster is determined via node size and number of
nodes

3. Here, in the same availability zone of a region all cluster nodes are created

4. To execute queries faster Redshift makes use of massively parallel processing,

columnar storage, data compression and zone maping
Why Amazon Redshift?
https:/[Link]/
Amazon Redshift – features
https:/[Link]/

 Optimized for data warehousing

 No Up-front costs

 Fully managed

 Fault tolerant

 Automated backup

 Fast restore encryption

 Network isolation
AWS Redshift Architecture
https:/[Link]/

• AWS Redshift is a collection of computing resources known as nodes, Client Application

where these nodes are organized to form of cluster
JDBC / ODBC
• Client application are the tools to interact with redshift, example: SQL,
Workbench, Dbeaver and ETL tools Leader Node
• Client Application connect to Redshift cluster via JDBC or ODBC drivers

• The leader node communicates with client application and compute

nodes

• It even develops query execution plans and distributes the parts of

query among the compute nodes
AWS Redshift Architect
https:/[Link]/

• Each compute node has its own CPU, memory and attached storage Client application

• There are two types of node JDBC / ODBC

Leader node
Dense compute(DC): SSD based

Dense storage (DS) :HDD based

• The role of compute note is to execute the compiled code and sends
Compute Compute Compute
the intermediate result to the leader node node 1 node 2 node n

• All the user data gets stored in compute node

Amazon S3 /DynamoDB / SSH
• It performs the functions such as load, backup, and restore via
Amazon S3, DynamoDB and SSH
AWS Redshift Architecture
https:/[Link]/

Client application
• Node slice is mainly used for the distribution of data within

the node

Leader Node
• Each slice is allotted a portion of node memory and disk space,

where it process the portion of worked assigned to its node

Node Node Node
• Once the leader node assigns the operations slice then these slices slices slices

slices work in parallel to complete the operation

Amazon S3/DynamoDB/SSH
 https:/[Link]/

Analytic – AWS kinesis

What is streaming data
https:/[Link]/

Streaming data is the data which is generated continuously from thousand of sources

• The sources data sends data records simultaneously in

small sizes i.e kilobytes
• This data needs to proceed sequentially and
incrementally on record basis and used for wide range of
analytics
• Such analytics provide the company with the visibility of
service usage, server activity, website clicks, Geo-
location of emerging situation
• Online service providing companies like Flipkart, Uber
cabs and many more deal with streaming data
What is kinesis?
https:/[Link]/

Kinesis is an data analysis service by Amazon which provide an easy way to collect process and analyse real-time streaming
data to get one time insights so as to provide quick response to the information

It mainly includes video stream, data stream, data firehose and data analytic

Kinesis video stream is used to capture, process and Analyse video stream for machine

learning and analytics

Kinesis data stream is used to build custom application to Analyse data stream using third

party stream processing system

kinesis data firehose is used to load data into AWS data stores

Kinesis data analytic is an easy way to process data stream with SQL
Features of Kinesis
https:/[Link]/

 Ensure high availability and durability by replicating data into multiple availability zones

 Can store and process thousands of terabytes of data per hour from different sources

 Streams are made up of Shards

 Each shard ingest data up to 1 MB/Sec and up to 1000TPS

 Each shard emits up 2 MB/sec

 All data stored for 24 hours

 Kinesis streams can be scaled by adding or removing shards

Benefits of Kinesis Stream
https:/[Link]/

 Easy to provision, deploy, and manage

 Elastically scalable and serverless

 Reduce latencies

 Pay as you go, no upfront costs

 Right service for your specific use cases

 https:/[Link]/

Demo – Visualize The Web Traffic

Using Kinesis Data Stream
 https:/[Link]/

Demo – Visualize The Web Traffic Using Kinesis

1. Create sample application through cloud Formation to generate data stream
2. Create a stack
3. Specify the details of stack and click on “Next”
4. Review the details and click on “create”
5. To check the output for data stream click on the public IP of the instance
6. Monitor the data produced by the producer and customer through the kinesis dashboard

Note: please refer to module 6 demo 4 in demo document for more detailed steps
https:/[Link]/