Azure Compute
Memi Lavi
[Link]
Compute
• Set of cloud services for hosting and running applications
• Allows uploading your code and then running it
• Offers various levels of control and flexibility
Source: [Link]
Compute
• We’ll talk about 4 types of Compute services:
Virtual Machines
App Services
AKS
Azure Functions
Virtual Machines
• A virtual (=not real) server running on a physical (=real) server
• Allows creating new servers extremely quick
• Based on existing resources of the physical server
• From the user’s point of view – a regular server, nothing new
• Called an Unmanaged Service
Virtual Machines Architecture
VM Density = No. of VMs per Host
In this case - 3
Code Code Code
Virtual Bins / Libs Bins / Libs Bins / Libs
Guest OS Guest OS Guest OS
Hypervisor
Physical
Host OS
Virtual Machines In Azure
Managed
Code Code Code
by us,
full
control Bins / Libs Bins / Libs Bins / Libs
and
access
Guest OS Guest OS Guest OS
Managed
by Azure, Hypervisor
no control
or access
Host OS
Virtual Machines in Azure
• Steps for creating VM in Azure:
• Select the location
Don’t
• Select the image forget to checksoftware)
(OS + Pre-Installed the price!
• Select the size
• That’s it, basically….
The Real Cost of VM
• Cost of VM includes:
• VM
• Disk
• IP
• Storage
The Real Cost of VM
Resource Type Monthly Cost ($)
VM D2v3 154.76
Disk P10 21.68
Public IP Dynamic 2.92
Storage LRS <1$
Total: ~180$
Reducing the Cost of VM
• Most effective techniques to reduce costs of VM:
Auto Shutdown
Reserved Instances
Spot Instances
Disk Optimization
Auto Shutdown
• As simple as it sounds…
• Automatically shuts down the machine when not needed
• Relevant mainly for test / dev machines
• Storage and IP (if static) costs still incurred
• Can save >50% of VM cost
Auto Shutdown
Reserved Instances
• Allow upfront payment with substantial discount
• Usually offered for 1 or 3 years
• Great for production machine which run continuously
• Offers great discounts (up to 62%)
• Can be divided to monthly payments
• Cannot be stopped / refunded
• Unless…
Reserved Instances
Spot Instances
• Machines that run on unused capacity in Azure
• Can be evicted any moment when needed by Azure
• Offers up to 90% discount, price fluctuates according to demand
• Great for non-critical, non-continuous tasks
• ie. Batch processes, long running calculations
Spot Instances
Disk Optimization
• Make sure to select the right disk for the machine
• Default is Premium SSD – the most expensive option
• Non IO-intensive machines can do with Standard SSD
• ie. App servers, in-memory cache
• Note: Disk type affects the SLA
Disk Optimization
More Cost Saving Techniques
• Select the right size for your machine
• CPU shouldn’t rest, you pay for it☺
• Select Linux over Windows when possible
• Check price in nearby regions
Availability of a VM
SLA (%) Yearly Downtime Allowed
95 18d 6h 17m 27s
99.5 1d 19h 49m 44s
99.9 8h 45m 56s
99.95 4h 22m 44s
99.99 52m 35s
Source: [Link]
Availability Concepts in Azure
Fault Domain
Update Domain
Availability Set
Availability Zone
Fault Domain
• Logical group of physical hardware that share a common power
source and network switch
• Similar to rack in a traditional data center
Fault Domain
If there’s a problem with the power or networking
in the domain (=rack) – all servers in it shut down
You want to make sure your servers are spread
across more than one fault domain (=rack)
Update Domain
• Logical group of physical hardware that can undergo maintenance
and be rebooted at the same time
• Maintenance is done by Azure at its own discretion
Update Domain
If all your servers are in the same update domain –
they’ll reboot at the same time during maintenance
You want to make sure your servers are spread
across more than one update domain
Availability Set
• A collection of Fault Domains and Update Domains your VMs will be
spread across
• Can contain up to 3 Fault Domains and up to 20 Update Domains
• All domains (Fault & Update) are in the same Zone (=datacenter)
Availability Set Example
Availability Set Example
Fault Domain #1 Fault Domain #2
VM #2 Update Domain #1
VM #1 Update Domain #2
Update Domain #3
Update Domain #2 Reboots
Fault Domain #1 Fault Domain #2
VM #2 Update Domain #1
VM #1 Update Domain #2
Update Domain #3
Fault Domain #1 Fails
Fault Domain #1 Fault Domain #2
VM #2 Update Domain #1
VM #1 Update Domain #2
Update Domain #3
Without Availability Set…
Fault Domain #1 Fault Domain #2
Update Domain #1
VM #1 Update Domain #2
VM #2 Update Domain #3
Taking Advantage of Availability Set
• Deploy identical VMs into the same Availability Set
• Ensures they won’t be shut down simultaneously when a single
fault domain shuts down or an update domain reboots
• If needed – deploy load balancer to route between the VMs
• Availability Set is free, you pay only for the additional VMs
Availability Zone
• A physically separate zone within an Azure region
• Technically – a building containing an autonomous data center
• Each zone functions as a fault & update domain
• Provides protection against a complete zone shutdown
• Hence the better SLA
Taking Advantage of Availability Zone
• Deploy identical VMs into separate Availability Zones in the same
Region
• Ensures they won’t be shut down simultaneously when the zone
shuts down
• If needed – deploy load balancer to route between the VMs
• Availability Zone is free, you pay only for the additional VMs
ARM Template
• Azure Resource Manager Template
• A JSON file describing the resource(s) to be created
• Used by Azure in (almost) all deployments
• Can be exported, modified, uploaded, deployed
• Can also be created from scratch
ARM Template
• ARM Template is a declarative way of deploying resources
Declarative Imperative
• Describes the end result • Sends instructions to run
• Allows “What-If” operation • Error prone
• Can deploy multiple resources at once • Can’t be verified
• Can be integrated in CI/CD processes • Can’t be source controlled
• Can be source controlled • Suited for quick and dirty operations
Used by: ARM Template Used by: Azure CLI, PowerShell
(Although they can run ARM Template too)
Virtual Machine Scale Set
• A group of separate VMs sharing the same image
• Managed as a group
• Can be scaled out or in manually or according to predefined
conditions
• Great for handling unpredictable load
Virtual Machine Scale Set
• Once set up, the machines should NOT be modified
• Change files, install apps etc.
• New machines created by the scale set will be based on the original
image
• For web apps, a load balancer should be put in front of the scale set
Scale Set Architecture
Load Balancer
Virtual Machine Scale Set
(VMSS)
Scale Set Pricing
• Scale Set is free
• You pay for the VMs deployed in it
Azure Instance Metadata Services
• A little known feature of Azure VMs
• A REST API accessible from the VM
• Providing a lot of info about the machine
• Info includes:
• SKU, storage, networking, scheduled events
• Accessible ONLY from the VM
Azure Instance Metadata Services
• With Scaleset –
• Get notification about upcoming eviction
• Can be polled every ~1 min to get enough time to close things up
Azure Architecture Diagram
• When designing architecture for Azure apps it’s a good idea to use
Azure symbols in the diagram
• There are hundreds of them…
Download Azure Icons
• [Link]
Cloud Architecture
VM VM
Weather API Catalog App
Cloud Architecture
A Word of Caution:
NEVER VM
Weather API
VM
Catalog App
leave a VM open to the
internet this way - Directly accessible from the internet
- Can be RDPed from anywhere
We will learn later on what should be done
App Services
• A fully managed web hosting for websites
• Publish your code – and it just runs
• No access to the underlying servers
• Secured and compliant
• Integrates with many source controls and DevOps engines:
• GitHub, BitBucket, Azure DevOps, DockerHub and more
App Services
• Supported platforms:
• .NET
• .NET Core
• [Link]
• Java
• Python
• PHP
• Supports containers
App Services
• App Types:
• Web Apps
• Web API
• Web Jobs (batch processes)
App Services
• Extremely easy to deploy:
1. Develop your app
2. Create Web App (can be done from the IDE)
3. Publish your code
4. Viola!
App Services Tiers
Lowest price,
goes up with
CPU & RAM
Source: [Link]
App Service Auto Scaling
• App Service can be autoscaled to support spikes in load
• Auto scale is based on various metrics
• Extremely flexible
App Service Auto Scaling
Cloud Architecture
App Service VM VM
Inventory App Weather API Catalog App
AKS
• Azure Kubernetes Services
• Managed Kubernetes on Azure
• Allows deploying containers and managing them using Kubernetes
on Azure
• Paying only on the instances (=VMs) used
Containers
• Traditional deployment:
• Code was copied and built on the production server
• Problems were found on the servers that weren’t found in the
dev machines
Containers to the Rescue!
[Link]
Containers
• Thin packaging model
• Packages software, its dependencies, and configuration files
• Can be copied between machines
• Uses the underlying operating system
Container vs VM
Why Containers?
The same package is deployed from the dev
Predictability machine to the test to production
Performance Container goes up in seconds vs minutes in VM
One server can run thousands of containers vs
Density dozens of VMs
Why Not Containers?
Containers share the same OS, so isolation is
Isolation lighter than VM
Docker
• The most popular container environment
• De-facto standard for containers
• Released in 2013
Docker Architecture
The Docker
Server
CLI to
manage the
images and
containers
Images of the
Repositories of
containers,
Instances of images. Can be
containing the
Images public or private
software to run
[Link]
dockerfile
• Contains instructions for building custom images
[Link]
Support for Docker
• Supported by all major operating systems (Windows, Linux, OSX)
• Supported by major cloud providers
Azure
ACR
Containers Management
• Containers are a great deployment mechanism
• Gain popularity
• What happens when there are too many of them?
Containers Management
- Deployment
Frontend Frontend
- Scalability
- Monitoring
- Routing Batch
- High-Availability Processes
Backend Backend
Database
Kubernetes
• The most popular container management
platform
• De-facto standard for container
management
• Released by Google in 2014
Kubernetes
• Provides all aspects of management:
• Routing
• Scaling
• High-Availability
• Automated Deployment
• Configuration Management
• And more…
Kubernetes Architecture
[Link]
Cloud Architecture
AKS App Service VM VM
Cart App Inventory App Weather API Catalog App
ACR
Cart Docker
Azure Functions
• Small, focused functions running as a result of an event
• Great for Event Driven systems
• Automatically managed by Azure
• Start, stop, autoscale
• Flexible pricing plans
• Serverless
Serverless
• Cloud resource that is completely managed by the cloud
• Users do not need to think about:
• VMs
• CPU
• Memory
• etc.
• It just works
Serverless
Remember:
Azure Function Example
Triggers and Bindings
Triggers Bindings
• The event that made the function run • Declarative connection to other
• Quite a few resource(s)
• Deeply integrated into other Azure • Input, output, or both
services • Provided as parameter to the function
• Technically not mandatory, but… • Makes connecting to other resources
extremely easy
• Not mandatory
Azure Function Example
Trigger (HTTP)
Binding (EventGrid)
Trigger Types
• Blob Storage • Queue Storage
• Cosmos DB • RabbitMQ
• Dapr • Service Bus
• Event Grid • Timer
• Event Hubs
• HTTP Requests
• IOT Hub
• Kafka
Binding Types (Input or Output)
• Blob Storage • Mobile Apps
• Cosmos DB • Notification Hub
• Dapr • Queue Storage
• Event Grid • RabbitMQ
• Event Hubs • SendGrid
• HTTP Requests • Service Bus
• IOT Hub • SignalR
• Kafka • Table Storage
Triggers and Bindings
• Example scenarios:
• Run every 5 minutes (Timer Trigger) and calculate the sum of a
column in a DB. If it’s above 115, send an event in EventGrid
(Binding)
Triggers and Bindings
• Example scenarios:
• When a message arrives in the Orders Queue (Queue Trigger)
save it in Cosmos DB (Binding) for future handling
Triggers and Bindings
• Example scenarios:
• Receive HTTP Request (HTTP Trigger) with 4 numbers, and
return the smallest one of them (no binding)
Supported Languages
• C#
• JavaScript (nodeJS)
• Java
• Python
• PowerShell
• F#
Cold Start
• Azure Functions are completely managed by Azure
• After some time of inactivity Azure might take down the Function’s
host
• The next activation of the Function will take time
• 2-3 seconds before the code runs
• A problem mainly for HTTP-Triggered functions
Cold Start
Source: [Link]
Cold Start
• How to avoid cold start?
• Select the right hosting plan
Azure Functions Hosting Plans
Consumption
Premium
Dedicated
Consumption Plan
• Pay only for what you actually use
• Note: In consumption plan there’s a limit of 1.5GB RAM
Consumption Plan
• Calculation example:
• Executions / month: 9m
• Avg. memory consumed / execution: 800MB
• Avg. execution duration: 1.5s
• Total seconds: 9m * 1.5s = 13.5m secs
• Total GB / sec = 13.5m * 0.8 = 10.8m – 400K free grant = 10.4m GB/sec
• Payment for execution time: 10.4m * 0.000016$ = 166.4$
• Payment for executions: 9m-1m free grant = 8m * 0.2$ / m = 1.6$
Total Payment: 168$
Consumption Plan
• Downsides:
• 1.5GB RAM limit
• Cold Start
Premium Plan
• Pay for pre-warmed instances (hosts)
• Pay for scale-out instances
Premium Plan
• What you get:
• No cold starts
• No memory limit (up to host RAM)
• Better performance
• VNet integration
• Predictable price
Premium Plan
• Calculation example:
• 1 pre-warmed instance
• 2 vCpus, 7GB RAM
• No scale out
• vCPU cost: 123.37 X 2 = 246.74$
• Memory cost: 8.833 X 7 = 61.83$
Total Payment: 308.57$
Premium Plan
• Downsides:
• More expensive
Dedicated Plan
• The Functions run on an existing App Service
• Great if server is under-utilized
• No additional costs
Dedicated Plan
• Make sure Always On setting is activated to avoid disabling
functions:
Dedicated Plan
• Downsides:
• No Auto-Scale
Durable Functions
• Stateful Functions that interact with external resources and keep
track of flow
• Offer very simple syntax, hide complexities of managing state,
retries, etc.
Durable Functions
• For example:
• Function Chaining – call various Functions sequentially, and
apply the output of each function to the next one:
Durable Functions
Cloud Architecture
Function App AKS App Service VM VM
Order Processing Cart App Inventory App Weather API Catalog App
ACR
Cart Docker
How to Choose Compute Type?
More Compute Options
• Logic Apps
• ACI – Azure Container Instance
• App Service Container – Deploy docker to App Service