Course Name: DevOps
Course Code: TE7951
Faculty Mentors: Dr. Aditi Sharma, Prof. Arijit Dutta
Learning Objectives:
1. Understand the benefits of DevOps over other software development processes,
gain insights into the DevOps environment, get an overview of different DevOps
Tools and Get a picture of the working of the DevOps Delivery Pipeline.
2. Understand Version Control, perform management of files for small as well as
large projects, perform various Git commands such as git add, git fetch, git commit,
git init, etc. Work with remote repositories
3. Learn and install Selenium, Create Test Cases in Selenium WebDriver, Utilize X-
Path and TestNG to locate elements, execute code on several browsers using
Selenium suite of tools, and Integrate Selenium with Jenkins.
4. Managing authorization in Jenkins, Jenkins notification management, Master-
slave architecture in Jenkins, add a slave node to Jenkins master, Build and deploy
codes using Jenkins, build pipeline plugin in Jenkins, and Use Declarative pipeline
in Jenkins.
5. Get insight of deployment. Learn characteristics of deployment tools, and use
any deployment tools to deploy container.
Pre-requisites:
Python / Powershell / Perl / Java / .net, scripting, cloud technology, software
engineering.
Unit-1
▪ Introduction to DevOps :Introduction, Architecture, Lifecycle,
▪ Workflow Principles, DevOps Tools, Concept of Automation,
▪ Engineering, Pipeline Methodology, DevOps Vs Agile.
Current Industry Buzzwords – Which one you are not familiar?
1. Machine Learning/AI 7. Virtualization
2. Natural Language Processing 8. Cloud Computing
3. Voice Recognition, Prompt 9. Neural Networks
Engineering 10. Internet of Things (IOT)
4. BigData 11. DevOps – Containerization,
5. Blockchain (Web 3.0) Docker, CI/CD
6. Mobile Apps 12. Metaverse
13. ChatGPT
Natural Language Processing
Natural language processing (NLP) is an
area of computer science and artificial
intelligence concerned with the
interactions between computers and
human (natural) languages, in
particular how to program computers
to process and analyze large amounts
of natural language data.
Use cases: Any application where you need end user to
interact with your support staff…
BIG DATA
• Big data is data sets that are so big and
complex that traditional data-
processing application software are
inadequate to deal with them.
• Big data challenges include capturing
data, data storage, data analysis,
search, sharing, transfer, visualization, quer
ying, updating, information privacy and data
source.
• There are a number of concepts associated
withcases:
Use big data:
Anyoriginally there were
solution where 3 to
you need
concepts
query volume,
large database variety, velocity.
(trillion+ Other
records)…
concepts later attributed with big data
are veracity (i.e., how much noise is in the
Voice Recognition, Prompt Engineering
Voice recognition can refer to:
• Speaker recognition, determining who is
speaking
• Speech recognition, determining what is being
said.
Prompt engineering or in-context learning is
an emergent ability of a large language model's
size, that allows, from a user's standpoint,
crafting, priming, refining, or probing (series of)
prompts within the bounded scope of a single
conversation.
From a machine learning perspective, it is about learning
from user's interventions in order to adapt to them.
Use cases: Security system, Process Automation,
need help on an issue?
Blockchain
• A blockchain,is a growing list of records,
called blocks, which are linked using
cryptography.
• Blockchains which are readable by the
public are widely used by
cryptocurrencies.
• Private blockchains have been proposed
for business use.
Use cases: Healthcare, Financial Services, Cryptocurrency
transactions…
Mobile Apps
• A mobile app is a computer
program designed to run on a mobile
device such as a phone/tablet or watch.
• Mobile applications often stand in contrast
to desktop applications which run
on desktop computers, and with web
applications which run in mobile web
browsers rather than directly on the mobile
device.
Use cases: “Mobile First” application development
approach…
Virtualization
• In computing,
virtualization refers to the
act of creating a virtual
(rather than actual)
version of something,
including virtual computer
hardware platforms,
storage devices, and
computer network
resources.
Use cases: Efficient usages of physical resources…
ChatGPT
• ChatGPT is an AI-powered
language model developed by
OpenAI, capable of generating
human-like text based on
context and past conversations.
Use cases: CoPilot interface to any application including Operating system
10
Cloud Computing
• Cloud computing is an information
technology (IT) paradigm that enables
access to shared pools of configurable
system resources and higher-level services
that can be rapidly provisioned with
minimal management effort, often over
the Internet.
• Cloud computing relies on sharing of
resources to achieve coherence and
economies of scale, similar to a public
utility.
Use cases: Shared usage of physical resources…
Neural Networks
• Neural network refers to interconnected
populations of neurons or neuron simulations
that form the structure and architecture of
nervous systems, in animals, humans, and
computing systems:
• Artificial neural network, a computing system
inspired by the biological neural networks found
in animal brains
• Biological neural network or neural circuit, a
collection of physically interconnected neurons
that carry out a specific function together
Use cases: Intelligent and automated decision
making…
Internet of Things
• The Internet of Things (IoT) is the network of physical
devices, vehicles, home appliances, and other items
embedded with electronics, software, sensors,
actuators, and connectivity which enables these things
to connect and exchange data.
• Creating opportunities for more direct integration of
the physical world into computer-based systems,
resulting in efficiency improvements, economic
benefits, and reduced human exertions.
Use cases: Data collection, Monitoring, Remote Operations and
Reporting…
What is Metaverse in simple terms?
• In the metaverse, users traverse a
virtual world that mimics aspects of
the physical world using such
technologies such as virtual reality
(VR), augmented reality (AR), AI,
social media and digital currency.
• The internet is something that
people "browse." But, to a degree,
people can "live" in the metaverse…
Use cases: Gaming Industry, Improve Efficiency of people…
DevOps – Our Course…
• DevOps (a clipped compound of "development" and
"operations") is a software engineering culture and
practice that aims at unifying software development
(Dev) and software operation (Ops).
• The main characteristic of the DevOps movement is to
strongly advocate automation and monitoring at all
steps of software construction, from integration,
testing, releasing to deployment and infrastructure
management.
• DevOps aims at shorter development cycles, increased
deployment frequency, and more dependable releases,
in close alignment with business objectives.
Use cases: Improving productivity of developers and Operations team…
Devops- Development phases-
defining DevOps as "a set of practices intended to reduce the time between committing a change to a system and the change being placed into
normal production, while ensuring high quality".
Len Bass, Ingo Weber, and Liming Zhu—three computer science researchers from the CSIRO and the Software Engineering Institute
Commonwealth Scientific and Industrial Research Organisation (CSIRO) is an Australian Government agency
2009, the first conference named DevOps Days was held in Ghent, Belgium.
Founded by Belgian consultant, project manager and agile practitioner Patrick Debois.
the adoption of DevOps was accelerating.
2012, a report called "State of DevOps" was first published by Alanna Brown at Puppet Labs.
2014, the annual State of DevOps report was published by Nicole Forsgren, Gene Kim, Jez Humble and others
2014, Lisa Crispin and Janet Gregory wrote the book More Agile Testing, containing a chapter on testing and DevOps.
2016, the DORA DevOps Research and Assessment metrics for throughput (deployment frequency, lead time for changes), and stability (mean
time to recover, change failure rate) were published in the State of DevOps report.
DevOps- Primary Development Activities
1) A developer connects to github, clone and build application with Visual Studio Code IDE.
2) Building the application produces a Docker container image.
3) The image is pushed to a namespace in GCP Cloud Container Registry.
4) The application is deployed to a GCP Kubernetes cluster.
5) Users access the application.
Let us start our DevOps Course
What, Why and How of DevOps?
What is DevOps?
DevOps is a software development strategy which bridge the
gap between Dev and Ops side of the company
▪ DevOps bring together development (Dev) and operations (Ops).
DevOps is the union of people, process, and technology to continually
provide value to customers.
▪ It involves the practices, tools and philosophies used to create better
communication channels between different teams.
▪ Usually, the developers are judged based on how many features they
add to the code while operations engineers are judged based on the
uptime of the service
▪ Developers have very little systems knowledge and do not consider how
their code runs on the server, while operations engineers are expected
to deploy the code and keep the servers running
▪ DevOps aims at reducing this communication barrier by making small
incremental changes to the codebase.
Why DevOps?
To bridge the gap between Developers and Operators
▪ DevOps—the combination of development
(Dev) and operations (Ops) teams—is an
organizational approach that enables faster
development of applications and easier
maintenance of existing deployments.
▪ By enabling organizations to create stronger
bonds between Dev, Ops and other
stakeholders in the company.
▪ DevOps promotes shorter, more controllable
iterations through the adoption of best
practices, automation and new tools.
▪ DevOps is not a technology per se, but it
covers everything from the organization to
culture, processes and tooling.
Lifecycle, Workflow & Principles - Dev and Ops Views & Gaps
Due to a clash between the views of developers and operations engineers in the software development process,
there arises a need for an efficient communication channel between the two.
Developer View
▪ Mostly delivers features after testing in development systems
▪ Dev systems may not be the same as production system
▪ Developers will have faster turnaround time with respect to
features
▪ Not much concerned about the infrastructural as well as
deployment impact because of the code changes
Operations engineer view
▪ Worries more about PSR (Performance, Stability and
Reliability)
▪ Rewarded mainly for uptime
▪ Lesser turn around time around feature deployment and
testing due to large number of dev builds coming their way
▪ Very much concerned about the infrastructural as well as
deployment impact because of the code changes
DevOps Benefits
DevOps Phases and Respective Tools
DevOps - Different Phases
Planning : In planning phase the business owners and software
development team discuss project goals and create a plan
Code: Programmers then design and code the application and use
tools like git to store application code
Build: Build tools like MS Build, .NET CLI, Maven, Gradle take code
from different repositories and combine them to build the
complete application
Testing : Application is tested using automation testing tools like
XUnit, Selenium and JUnit to ensure software quality
Release: When testing is complete, new features are integrated
automatically to the already existing codebase
Deploy: Application is packaged after release and deployed from
development server to production server
Operate : Once the software is deployed, operations team
perform activities such as configuring servers and provisioning
them with the required resources
Monitoring: Monitoring allows IT organization to identify the
impact of individual releases on the end user
DevOps tools
Source Control Management
▪ SCM tools help the developers to stay on top of the changes made to the codebase
▪ They do so by maintaining different copies of the codebase called branches, allowing
developers to make changes to their own branch and then merge it with the main
branch
▪ They keep track of every modification to the code in a special kind of database. If a
mistake is made, developers can turn back the clock and compare earlier versions of
the code to help fix the mistake while minimizing disruption to all team members.
▪ Git, Subversion, TFS, Mercurial are some of the most famous tools in this space
▪ As far as GIT is concerned, it is known to have the smallest footprint possible and has
lightning-fast performance.
DevOps tools
Planning and coding phase
▪ IDEs are the text editors in which the developers write code
▪ They have excellent features like Syntax Highlighting, Autocomplete, Building Executables,
Debugging, etc to make writing code easier
▪ Some IDEs such as VS Code, sublime-text provide plugins to integrate with your SCM to
commit changes to your SCM on the go
▪ Without IDE, developers would need to spend time configuring multiple development
tools. With an IDE, developers have the same set of capabilities in one place, without the
need for constantly switching tools.
▪ VS Code, Sublime text, Eclipse, Notepad++ are the most famous in this space
▪ The choice of an IDE depends on the programming language and frameworks used to
build the software
DevOps tools
Build and Test
▪ Build tools automate the process of building executable code from source code
▪ The process of code compilation is essential to the creation of software when
high-level programming languages are used. Part of the function of the build tool
is to cope with errors in the compilation process of complex software systems.
▪ Modern build tools go further by optimizing complex build processes using
distributed build technologies, which involves running the build process in a
coherent, synchronized manner across several machines.
▪ Usual steps involved are downloading the dependencies, compiling source code
to the form of binary code, packaging that binary code, running the tests,
emailing the build logs to concerned teams
▪ MS Build, Maven, gradle, cruise control are some prominent names in these
space.
DevOps Tools
Test, Release and Deploy
▪ This step involves releasing the software to production environment.
▪ Code goes through different environments like Development, Staging, SIT and
UAT environments for the further integrations and testing.
▪ XL release, Clarive, Jenkins are most prominent names in this space
▪ Azure DevOps provide solutions for releasing and deploying software apart
from version control, project management and other services.
▪ Jenkins is a java based deployment automation tools that comes with a lot of
plugins making it compatible with almost any kind of build process.
▪ Clarive is ideal for Ops teams looking to centralize all delivery processes,
coordinate silos and resolve application dependencies.
DevOps tools
Monitor
▪ Application Platform Monitoring and Infrastructure monitoring are two of the most
important tasks required to be executed in this phase.
▪ This step is crucial as it forms the backbone of the feedback that goes to the
developer team based on which the code is optimized to run on real time servers
▪ Nagios and Zabbix are ‘traditional tools’ - widely used, downloadable and open
source
▪ IaaS Vendors like Amazon and Google offer AWS CloudWatch and StackDriver,
respectively, enabling operations to consume usage and performance information
through their APIs
▪ AWS Cloud Watch offers metric data on data transfer, disk usage, and CPU
utilization, which trigger email notifications when thresholds are exceeded
Engineering, Pipeline & Methodology –
Stages involved in a DevOps Cycle
Development environment: This is where the developers use
scm to build the application to be promoted to another
environment. The changes in this environment are fast paced.
QA environment : This environment mimics the production
environment more closely than the developer environment.
Because of the fast paced nature of the dev environment it
becomes difficult for QA teams to ensure proper standards. This
environment provides them with the necessary stage to be able
to do so.
Testing Environment Involves creating a space that lets you to
perform test executions of your component parts. It tests only
certain functions in predefined ways. Every app or component
you develop needs to be put to the test. Hence, it’s your
component under test that dictates the environment. And not
the other way around.
Staging Environment : Your staging environment is a setup that
replicates your production environment. In short, it’s an identical
replica of your product. It is the closest to production. It is a
space where you can throw everything together and find out how
it works. It is the ultimate Quality assurance test.
DevOps Vs Agile - Traditional Integration Vs Continuous Integration
Hands-On: Implement Ecommerce App and understand need for DevOps…
Assignment – Understand DevOps requirement of modern App?
• How will you design DevOps process for an Ecommerce App? DevOps Requirements -
• Requirements (User): Login, Show items to buy, Transactions, • How many times, items to buy
View Cart, Payment, Personalization, Chat Bot
changes?
- Mobile Application – Client Facing App
- Web Application – Client Facing Website
• How many times price of items or
• Requirements (Staff): Login, Ordered Items, Transactions,
Reporting, Support, Call Centre Operations discount for each item changes?
- Mobile Application – Internal Ecommerce Staff facing
App • How many times users are not able
- Web Application – Internal Ecommerce Staff facing
Website to proceed till payments?
- Dashboard Screens
- Transaction Screens • How many times personalization
- Notifications Window requirement changes?
- Reporting Screens
- Chat bot Interactions… • How many times new reports need
to be generated.
We will refer this diagram throughout the course…
Understand DevOps needs of the Modern Application
7 Layer Modern Application – Each Layer of the application changes frequently…..
UX (Portals, EDGE Apps, Mobility) Experience layer changes – New personalization requests…
7 EXPERIENCE
(Persona centric UX, actions, digital interactions)
BUSINESS BUSINESS PROCESS ORCHESTRATION (BPM , BOTS) Need Autonomous Business Process Automation – New business process to handle regional specific rules
6 PROCESS (Flexible Business Process Orchestration across systems) and regulations…
MANAGEMENT
MIDDLEWARE (API, Integration, B2B) Need Seamless collaboration via flexible Integration – Integration with new systems including IoT devices..
5 INTEGRATION (Partner Collaboration & Application Integration)
CUSTOM
APPLICATIONS (COTS / CUSTOM APPS) Need to use Cloud Native services for speed, modern architecture, save cost – SaaS ready solution..
4 APPLICATIONS
(Apps – augmenting the core business platforms)
COTS
ERP & CORE BUSINESS PLATFORM (ERPs, Core transaction Apps) Standardize the core – Use new platform for deployments
3 CORE (Core System of Records representing business functions)
APPLICATIONS
DATA (Data Lakes, Data Stores, Databases, Warehouses, AI) Update data– New prices, discounts, Bundles, VAT..
2 DATA (Data foundation for unified sources of truth across systems)
CORE INFRASTRUCTURE (AWS / GCP / AZURE) Agile, Secure Cloud way of development – Need to use Multi-Cloud, automated self service functionality is
1 PLATFORMS (Hyperscalers for Scale, flexibility, performance & eco-system) needed to provision new environment or backup process
What is pipeline concept in CI/CD?
Source Code
File
How Jenkins help implement pipeline?
Key Terminologies used in Jenkins
▪ Controller- It is a central, coordinating process which stores configuration, loads
plugins, and renders the various user interfaces for Jenkins.
▪ Agent- It is typically a machine or container which connects to a controller and
executes tasks when directed by the controller.
▪ Job- It is a given set of tasks that runs sequentially as defined by the user.
▪ Pipeline- It is a collection of jobs which are interlinked with one another in a
sequence
▪ Stage- It defines a conceptually distinct subset of tasks performed through the
entire Pipeline
▪ Step- A single task
▪ Jenkinsfile- It is a text file that contains the definition of a Pipeline
Hands-On Overview : First Version Control and then deploy code using Pipeline.. Docker…
As part of hands-on activity in next unit; we will go through an overview of setting up a CI/CD
pipeline in Jenkins that deploy python application.
1. Installation Pre-Requisites
▪ Install Python and libraries flask, pytest
▪ Download the Jenkins, Docker
2. Jenkins Setup
▪ Download required plugins — Docker pipeline (from dashboard Manage Jenkins -> Manage
plugins)
▪ Create GitHub account
▪ Set PATH variable to run Python application from command line
▪ Properties tick Environment variables and add name (PATH) , value)
3. Download Code - Part 1 CMD Code/[Link]
▪ [Link]
Path to Download Ecommerce Application Code:
[Link]
[Link]
Summary of Unit 1