Interleap Recommended Approach

Objective
Identify current gaps in the infrastructure and deployment process, CI/CD pipeline, azure
setup.

Efforts it will take to move to AWS from Azure, Github actions CI/CD from Jenkins.

Includes a roadmap to achieve the target state and ensure smooth transition.

Current State
→ Team Size:

4 Devs, 1 QA, total headcount less than 10.

→ Total repositories: 4 (Github)

2 - ReactJS

1 - NodeJS

1 - Python AI app

→ Environments:

Dev

Staging

Production

→ Tech/services in use:

1. FE/BE/App → ReactJS, NodeJS, Nginx, Python

2. VCS, CI/CD → Github, Jenkins, R-shiny (analytics)

3. Azure services/ AI → ALB, Blob storage, Azure whisper, OneCompiler, OpenAI GPT
4o,

4. Cloudflare DNS

→ Branching Strategy:

Feature Branching

Once the task is completed, merge the branch to the dev.

Dev is merged into staging after tests are performed

Interleap Recommended Approach 1

 Once staging is cleared, staging branch is merged into production

→ Velocity:

Development Environment:

Not in use

Staging Environment:

Frequent manual deployments.

Production Environment:

Deployment every Tuesday.

Manual deployment of ReactJS and NodeJS , jenkins pipeline is used for python AI

→ Current Infra setup

Azure Cloud (Prod Infra)

Prod VMs are running ReactJS, Nginx and NodeJS as service on a single vm

Staging environment has 1 VM for FE/BE , 1 VM for python AI app

Interleap Recommended Approach 2

 One2N has provided with a split VM setup for FE and BE

Dev - managed under dev-interleap resource group

Staging - managed under staging-interleap resource group

Production - managed under prod-interleap resource group

Above resources are managed by Github actions and ready to go live.

→ Current CI/CD setup:

Current steps in the CI/CD process

1. Environment variables configuration

2. Code checkout on Jenkins server

3. Manual approval for deployment

4. Deploy either using Ansible or SSH to server and git-pull

Jenkins:

Repo Name/ env Interleap-ai Interleap-backend Interleap-student Interleap-admin

Staging Not functional Not functional Not functional Does not exist

Prod Functional Not functional Not functional Does not exist

Github Actions: (new deployment by One2N)

Repo Name/ env Interleap-ai Interleap-backend Interleap-student Interleap-admin

Dev No environment Functional Functional Does not exist

Staging WIP Functional Functional Does not exist

Prod WIP Functional Functional Does not exist

*P.S Interleap-admin is hosted on netlify , there was a discussion to move it to azure. Once
that is done, we can create the ReactJS based pipeline in actions

Challenges
Dysfunctional Jenkins pipeline, missing code versioning, alerts/notifications

Multiple ALBs setup

Every instance has a public IP which is not required because the machines will be behind
a load balancer

Interleap Recommended Approach 3

 Infrastructure is deployed without proper planning leading to complications

1. Infra in current Azure cloud is not streamlined. A lot of over-provisioning is seen

causing confusion.

2. As DNS (Cloudflare) is outside of Azure, (backend services) are utilising public IP

(fqdn) to communicate with python AI application.

3. Azure way of deployment (resource groups, backend pools, VM scale sets) make it
difficult to manage the infrastructure.

4. Entire deployment is via clickOps. No IaC utilised to codify infrastructure.

5. There is no versioning strategy in place even for manual build and deploy of code.
(process is to ssh into vms from bastion, git pull, npm restart)

Limited Observability and Alerts

1. Limited number of system level metrics and no service level metrics.

2. Alerts/Notifications are not configured properly.

Target State
New CI/CD steps:

Stages/steps are similar from the Jenkins pipeline

Main reason of moving away from Jenkins is the maintenance of server and failing
dependencies resulting in breaking pipelines.

1. Improve CI/CD

a. Use, Github Actions with self-hosted github runner .

b. Application versioning setup → triggered pipelines should use commit sha values in
code bundles to tie them with merge commits.

c. Pipeline notifications for build, deploy, success, failure, waiting for approval etc

d. Linting, uncovered tests to be integrated in the pipeline.

2. Improve infrastructure

a. Reduce the infrastructure footprint

b. Improve security - do away with public IPs

c. Manage infra with Infrastructure as Code (IaC) like Terraform.

d. Manage DNS within Azure

e. We can move the test environment in Private Network or move behind VPN.

Interleap Recommended Approach 4

 f. Reduce number of Application Load Balancers (ALBs)

3. Improve observability and alerts

a. Configure system level alerts in Grafana and send notifications to slack/email.

b. Utilise slack for pipeline notifications for success, failure, approvals.

Improvement
1. Pipelines

a. Identified Pipelines as the most important task to be fixed first and steps have been
taken to build Github actions pipelines

Pros

1. Github managed → move to self managed runner

2. Easy to implement and config, readymade workflows, secrets management at

actions level

b. Current state of github actions setup :

a. Total 6 pipelines for Dev, Staging and Prod has been deployed which manages
the newly provisioned VMs.

b. Waiting for approval to move staging and production infra to the actions pipeline.

2. Infrastructure management

a. Comb through the current infrastructure to remove the over provisioned resources.

b. Use a tool like Terraformer [Link]

tab=readme-ov-file#demo-gcp which can help reverse map the infra to code.

3. DNS setup

a. Cloudflare DNS setup, all the fqdns are mapped to public IPs.

b. Leverage ALB. Azure DNS can allow mapping of fqns to private IPs for internal traffic.

c. If Cloudflare to continue, leverage zero trust, tunnelling and private dns features to
manage DNS securely.

Timelines
Tasks No. of Days Status comments

Interleap Recommended Approach 5

 Pipeline improvement with Almost Only python AI pipelines are WIP.
8 days
GH actions (self hosted) completed Rest completed

Pipeline alerts/notification 2 days not started

Infrastructure Remove over provisioned instances,

improvement (cleaning 8 days not started VNETs, security groups, resource
azure) groups, load balancers

Move DNS to Azure 1 days not started Either migrate or look into vpn

Validation of setup 2 days not started Buffer time

Plan of action
Gradual implementation of changes to dev and staging, verify deployments before
moving to production

Phase 1: Move Jenkins to Github actions.

1. Work is in progress to improve the pipelines.

2. Demo completed, official cutoff from Jenkins to Github actions pending for 2 out
of 3 repos

3. Target completion date: 29th Nov

Phase 2: Streamline infrastructure.

1. Work in progress to improve infrastructure simultaneously.

2. New instances added to be managed by Github actions pipeline. This will enable
us to remove old instance, un-utilised VNETs, Security Groups.

3. Consolidate Dev, Staging and Production environments under their respective

security groups.

4. Map Azure code to Terraform.

5. Target completion date: 10th Dec

Phase 3: Call of action on DNS setup.

1. Decision to move DNS from Cloudflare to Azure should be done by 6th Dec

2. Moving DNS will coincide with Phase 2

3. If decision is made to stay with Cloudflare (CF) then leverage tunnels and private
DNS setup to map public FQDNs to private IPs.

4. Target completion date: 10th Dec

Phase 4: Improve observability and alerts notifications

Interleap Recommended Approach 6

 1. Send alerts to slack from pipeline will be covered during Phase 1.

2. Set alerts in Grafana for system level resources and send them to slack and
email.

3. Target completion date: 13th Dec

Conclusion
These steps will take care of current limitations, improve security and infrastructure
management, streamline deployment process and enhance alerts/notifications.

AWS Migration
Cost
AWS compute + Egress cost: $500-$700/ month

Current Azure compute cost: $1200/ month

Efforts
Above section of the document identifies the pain point of the current setup within Azure
and the fixes around it.

If we have to do this from scratch in AWS then we need to divide this project in 5 parts

Part 1: Networking and Compute setup

1. Account setup

2. Identity Access Management (IAM) setup

3. Virtual Private Cloud (VPC), security group

4. Golden image build

5. EC2 setup - identify the types of instances required for dev, staging and prod

6. Route53, Loadbalancer - If moving from CF to R53

Part 2: Connection to supabase (staging and prod)

Interleap Recommended Approach 7

 1. Connect to supabase db using API same as current setup with Azure in staging first
and then prod

2. Links: [Link]

Part 3: Connection from python AI app OneCompiler, Open AI, Azure whisper

1. Connect over API keys and service endpoint with OneCompiler, Open AI, Azure
whisper in staging and prod

2. Backend Integrations

a. OneCompiler: 1M/month allowed, 600-700k reached

This is typically used for runtime code execution.

If using OneCompiler APIs, ensure API access and integrate using their SDK or
HTTP requests.

b. Azure AI:

Obtain an API key from OpenAI.

Install the openai Python SDK and set up API calls in your app.

c. Azure OpenAI Whisper, GPT 4 (2 API calls), GPT 4o - Total 4 APIs

Set up an Azure Speech resource in the Azure portal.

Retrieve the endpoint URL and API key.

Use the Azure SDK (e.g., azure-ai-speech ).

Part 4: Move data from Azure blob storage to AWS S3

1. Interleap customer data is stored on blob storage

2. Identify the type of bucket required, set retention policies, lifecycle etc.

3. Steps to migrate data: [Link]

guidance/latest/patterns/migrate-data-from-microsoft-azure-blob-to-amazon-s3-by-
[Link]

Part 5: Observability

1. When moving to AWS, we have 3 options

a. Self host GLP stack - More efforts.

b. Grafana Cloud - Use Grafana cloud hosted service, less effort no vendor lock in.

i. [Link]
cloud-provider/aws/?pg=solutions-cloud-monitoring-aws&plcmt=hero-btn-2

Interleap Recommended Approach 8

 ii. Cost: [Link]

c. Grafana managed by AWS - Hosted and Managed by AWS

i. [Link]

ii. Cost: Pay as you go, First 90 days free for upto 5 users

1. More info on pricing: [Link]

nc=sn&loc=3

2. Can use Cloudwatch for centralised logging.

a. This document explains how Cloudwatch can also be leveraged for monitoring
metrics
→[Link]
[Link]

b. [Link]

Can we move from Supabase pg to AWS RDS

AWS Relational
DB name Supabase Resources
Database (RDS)

The Amazon
RDS for
PostgreSQL
Free Tier
includes (T4g
and T3)
750 hours on
Current Plan:
select Single-
PRO plan
AZ instance
$25/month with [Link]
databases,
CPU compute
Cost 20 GB of
plan to [Link]
General
increase cpu. pg=pr&loc=3
Purpose SSD
(last month bill
(gp2) storage,
$25)
20 GB of
storage for
automated
database
backups each
month for one
year.

Interleap Recommended Approach 9

 Free tier does
[Link]
Offers multi- not offer multi-
Setup insight managed-services-aws-azure-gcp-and-
region support. zone, that cost
supabase
will be extra

Migrating from
SupaBase to
RDS will require
following
1. Export User
→
pg_dumpall
2. Export
Schema →
pg_dump

* Connecting to 3. Setup for

Supabase from logical
[Link]
Efforts EC2 will be replication
supabase-to-rds-user-guide/
easy. 4. Validation of
* Easy Setup data migration
5. Migrating
sequence
6. Cutover

* Efforts to
setup and
migrate will be
more compared
to using current
setup

Scale out strategy

1. AWS Autoscaling allows for dynamic scaling using target tracking scaling policies .

Target tracking supports the following predefined metrics:

ASGAverageCPUUtilization —Average CPU utilisation of the Auto Scaling group.

ASGAverageNetworkIn —Average number of bytes received by a single instance on all

network interfaces.

ASGAverageNetworkOut —Average number of bytes sent out from a single instance on all
network interfaces.

Interleap Recommended Approach 10

 ALBRequestCountPerTarget —Average Application Load Balancer request count per target.

Observations:
1. From the current metrics for ReactJS, NodeJS app and also for Python app we do not see
consistent CPU utilisation and current utilisation is not more than 5% of allocated
CPU.

2. Looking at the Prod-alb in Azure, connection count (SNAT connection) for the last 30
days is around 2200 connections/day.

3. We can setup autoscaling based on ALBRequestCountPerTarget where the count value can
be modified based on the inflow of traffic.

4. The threshold can only be decided once we see the incoming connection count and
its co-relation with the resource utilization.

Timeline
task no of days comments

AWS account setup, network, IAM, storage, compute, Route 53,

3-4 days
ALB

Supabase, Open AI Connect, Azure Whisper, OneCompiler

2-3 days
connection and validation

Pipeline integration 2-3 days

Observability, alerts, notifications 2-3 days

Overall validation 1-2 days

Plan of Action
Start with:

1. Phase 1: AWS setup, all the resources can be build for (Dev, Staging and Prod)

2. Phase 2: External service connections (Supabase and AI)

3. Phase 3: Github pipeline integration (4 pipelines/staging and prod)

4. Phase 4: Observability, Alerts, Notifications (most of this will be taken care of while Infra
setup)

Conclusion

Interleap Recommended Approach 11

 1. To do the entire end to end deployment in AWS will require 2 weeks of time. (8hr/day)

Interleap Recommended Approach 12