Sushil
DevOps Notes
Why is DevOps in needed?
● Before DevOps the development and operation team worked in complete
isolation.
● Testing and Deployment were is isolated activities done after design-built hence
they consumed more time than actual build cycles.
● Without using DevOps, team members are spending a large amount of their time
in testing, deploying, and designing instead of building the project.
● Manual code deployment leads to human errors in production.
● Coding and operation teams have their separate timelines and are in synch
causing further delays.
There is a demand to increase the rate of software delivery by business stakeholders.
How is DevOps different from traditional IT.
● Let's compare traditional software waterfall model with DevOps to understand
the changes DevOps bring.
2
We assume the application is scheduled to go live in 2 weeks and coding is 80% done.
We assume the application is a fresh launch and the process of buying servers to ship
the code has just begun.
Old Process DevOps
After placing an order for new servers, the After placing an order for new servers
Development team works on testing, the Operation Development and Operations team work
team works on extensive paperworks as required together on the paperwork to set-up the new
in enterprises to deploy the infrastructure. servers. This results in better visibility of
infrastructure requirement.
Projection about failover, redundancy, data center Projection about failover, redundancy,
locations, and storage requirements are skewed as disaster recovery, data center locations, and
no inputs are available from developers who have storage requirements are pretty accurate due
deep knowledge of the application. to the inputs from the developers.
Operation team has no clue on the progress of the In DevOps the operation team completely
development team. Operation team develop a aware of the progress the developers are
monitoring plan as per their understanding. making. Operations team interact with
developers and jointly develop a monitoring
plan that caters to the IT and business needs.
They also use advance Application
Performance Monitoring (APM) Tools.
Before go-live the load testing crashes the Before go-live the load testing makes the
application. The release is delayed. application a bit slow. The development team
quickly fixes the bottlenecks. The application
is released on time.
Why is DevOps Used?
● DevOps allows agile Development Teams to implement continuous Integration
and continuous Delivery. This helps them to launch products faster into the
market.
● Other important reasons are:
3
○ Predictability: D
evOps offers significantly lower failure rate of new
release.
○ Reproducibility: version everything so that earlier version can be restored
any time.
○ Maintainability: E
ffortless process of recovery in the event of a new
release crashing or disabling the current system.
○ Time to Market: D
evOps reduces the time to market up to 50% through
streamlined software delivery. This is particularly the case for digital and
mobile application.
○ Grater Quality: DevOps helps the team to provide improved quality of
application development as it incorporates infrastructure issues.
○ Reduced Risk: DevOps incorporates security aspects in the software
delivery lifecycle. It helps in reduction of defects across the lifecycle.
○ Resiliency: The operational state of the software system is more stable,
secure and changes are auditable.
○ Cost Efficiency: DevOps offers cost efficiency in the software
development process which is always an aspiration of IT companies
management.
○ Breaks large code base into small pieces: DevOps is based on the agile
programming method. therefore , it allows breaking large code bases into
smaller and manageable chunks.
When to adopt DevOps?
● DevOps should be used for large distributed applications such as E-commerce
sites or applications hosted on a cloud platform.
When not to adopt DevOps?
● It should not be used in a mission-critical application like bank, power and other
sensitive data sites, such applications need strict access controls on the
production environment, a detailed change management policy, access control
policy to the data centers.
4
DevOps Lifecycle
● DevOps is deep integration between Development and Operations
● Understanding DevOps is not possible without knowing DevOps LifeCycle.
Here is a brief information about the continuous
( DevOps LifeCycle )
● Development
○ In this DevOps stage the development of software takes place constantly,
In this phase, the entire development process is separated into small
development cycles. This benefits DevOps team to speed up software
development and delivery process.
● Testing
○ QA team use tools like selenium to identify and fix bugs in the new piece
of code.
● Integration
○ In this stage new functionality is integrated with the prevailing code, and
testing takes place. Continuous development is only possible due to
continuous integration and testing.
● Deployment:
5
○ In this phase, the deployment process takes place continuously,. It is
performed in such a manner that only changes made any time in the code
should not affect the functioning of high traffic website.
● Monitoring:
○ In this phase, Operation team will take care of the inappropriate system
behavior or bugs which are found in production.
● DevOps workflow:
○ Workflow provide a visual overview of the sequence in which input is
provided. It also tells about actions are performed and output is generated
for an operations process
How is DevOps different from Agile?
● Stakeholders and communication chain a typical IT process
● Agile addresses gaps in Customer and Developer communication.
● DevOps addresses gaps in Developers and IT Operations communication.
Agile DevOps
Emphasize breaking down barriers DevOps is about software development
between developers and management. and Operations teams.
6
Addresses gap between customers Addresses the gap between development
requirements and development teams. and Operation team.
Focuses more on functional and It focuses operational and business
non-functional readiness. readiness
Agile development pertains mainly to the DevOps emphases on deploying software
way development is thought out by the in the most reliable and safest ways
company. which aren’t necessarily always the
fastest.
Agile developments puts a huge DevOps likes to divide and conquer
emphasis on training all team members spreading the skill set between the
to have varieties of similar and equal development and operation teams. It also
skills. So that when something goes maintains consistent communication.
wrong any team member can get
assistance from any member in the
absence of the team leader.
Agile development manages on sprints. It DevOps strives for consolidated
means that the time table is much shorter deadlines and benchmarks with major
(less than a month) and several features releases, rather than smaller and more
are to be produced and released in the frequent ones.
period.
DevOps Principles:
● Here, are Six Principles which are essential when adopting DevOps.
● Customer Centric Action:
○ DevOps team must take customer-centric action for that they should
constantly invest in products and services.
● End-to-End Responsibility:
○ The DevOps team need to provide performance support until they become
end-of life. This enhances the level of responsibility and the quality of the
products engineered.
● Continuous Improvement:
○ DevOps culture focuses on continuous improvement to minimize waste. It
continuously speeds up the improvement of product or services offered.
● Automate Everything:
7
○ Automation is vital principle of DevOps process. This is not only for the
software development but also for the entire infrastructure landscape.
● Work as one Team:
○ In the DevOps culture role of the designer, developer and tester are already
defined. All they needed to do is work as one team with complete
collaboration.
● Monitor and Test everything:
○ It is very important for DevOps team to have a robust monitoring and
testing procedures.
Who is DevOps Engineer?
● A DevOps Engineer is an IT Professional who works with software developers
system operators and other production IT staff to administer code releases.
DevOps should have hard as well as soft skills to communicate and collaborate
with development, testing and operations teams.
● DevOps approach need frequent, incremental changes to code versions, which
means frequent deployment and testing regimens. Although DevOps engineers
need to code occasionally from scratch. It is important that they should have the
basics of software development languages.
● A DevOps engineer will work with development team staff to tackle the coding
and scripting needed to connect elements of code, like libraries or software
development kits.
Roles, Responsibilities and skills of a DevOps
Engineer?
● DevOps engineers work full-time. They are responsible for the production and
ongoing maintenance of a software applications platform.
● Following are some expected roles, responsibilities and skills that is expected
from DevOps Engineer
○ Able to perform system troubleshooting and problem-solving across
platform and application domains.
○ Manage project effectively through open, standards-based platforms.
○ Increase project visibility thought traceability.
8
○ Improve quality and reduce development cost with collaboration.
○ Analyse, design and evaluate automation scripts and systems.
○ Ensuring critical resolution of system issues by using the best cloud
security solutions services.
○ DevOps Engineer should have the soft skill of problem-solver and quick
learner.
DevOps Automation Tools:
● It is vital to automate all the testing processes and configure them to achieve
speed and agility. This process is known as DevOps automation.
● This difficulty faced in large DevOps team that maintain large huge IT
infrastructure can be classified briefly into six different categories.
● Infrastructure Automation:
○ AWS: being cloud service you do not need to be physically present in the
data center. Also they are easy to scale on-demand there are no up front
hardware costs. It can be configured to provision more servers based on
traffic automatically.
● Configuration Management:
○ CHEF: It is useful DevOps tool for achieving speed, scale and consistency.
It can be used to easy out complex tasks and perform configuration
management with this tool, DevOps team can avoid making changes
across ten thousand servers. Instead they need to make changes in one
place which is automatically reflected in other servers.
● Deployment Automation:
○ Jenkins: This tool facilitates continuous integration and testing. It helps to
integrate project changes more easily by quickly finding issues as soon as
a built is deployed.
● Log Management:
○ Splunk: This is a tool solves the issues like aggregating, storing, and
analyzing all logs in one place.
● Performance Management:
○ App Dynamic: It is DevOps tool which offers real-time performance
monitoring. The data collected by this tool helps developers to debug
when issues occur.
9
● Monitoring:
○ Nagios: It is also important to make people are notified when
infrastructure and related services go down. Nagios is one such tool for
this purpose which helps DevOps teams to find and correct problems.
Introduction of git
Before understanding git understand Version Control System.
● Version Control System:
○ Version Control System (VCS) is a system that help software developers
to work together and maintain a complete history of their work.
● Listed below are the functions of a VCS
○ Allows developers to work simultaneously.
○ Does not allow overwriting each others changes.
○ Maintains a history every version.
● Following are the types of VCS
○ Local Version Control Systems
○ Centralized Version Control System (CVCS)
○ Distributed / Decentralized Version Control System (DCCS)
● Version Control System Tools:
○ Git
○ Concurrent Version System
○ Subversion
○ Mercurial
What is Git?
● Git is a distributed version controls tool that supports distributed non-linear
workflows by providing data assurance for developing quality software.
Features of Git
● Distributed:
○ Allows distributed development of code.
10
○ Every developer has a local copy of the entire development history and
changes are copied from one repository to another.
● Compatible:
○ Compatible with existing systems and protocols.
○ SVN repositories can be directly accessed using Git-SVN
● Non-Linear
○ Supports non-linear development of software.
○ Includes various techniques to navigate and visualize non-linear
development history.
● Branching
○ It takes only a few seconds to create and merge branches.
○ Master branch always contains production quality code.
● Lightweight:
○ Uses lossless compression technique to compress data on the client's
side.
● Speed:
○ Fetching data from local repository is 100 times faster than remote
repository.
○ Git is one order of magnitude faster than other VCS tools.
● Open Source:
○ You can modify its source code according to your needs.
● Reliable:
○ On events of system crash, the lost data can be easily recovered from any
of the local repositories of the collaborators.
● Secure
○ Uses SHA1 to name and identify objects.
○ Every file and commit is checksummed and is retrieved by its checksum at
time of check out.
● Economical
○ Released under GPL License. It is for free.
○ All heavy lifting is done on client-side, hence a lot of money can be saved
on costly servers.
11
What is a Repository?
● A directory or storage space where your project can live. It can be local to a
folder on your computer, or it can be a storage space on github or another online
host. You can keep code files, text files, image files, your name as a repository.
● There are two types of repositories:
● Central Repository
○ Typically Located on Remote Server
○ Exclusively consists of .git repository folder
○ Meant for team to share and exchange data.
● Local Repository
○ Typically located on local machine
○ Reside as a .git folder inside your projects root.
○ Only admin of the machine can work with this repository.
12
13
Git Operations and Commands
● Continuous Integration
● Problems before Continuous Integration
○ Developers have to wait till the complete software is developed for the
test results.
○ If the test fails then locating and fixing bugs is very difficult. Developers
have to check the entire source code of the software.
○ Software delivery process was slow.
○ Continuous feedback pertaining to things like coding or architectural
issues, build failures, test status etc.
● Continuous Integration
○ Since after every commit to the source code an auto build is triggered and
then it is automatically deployed on the test server.
○ If the test results shows that there is a bug in the code then the develops
only have to check the last commit made to the source code.
○ This also increases the frequency of new software releases.
○ The concerned teams are always provided with the relevant feedback.
14
Before Continuous Integration After Continuous Integration
The entire source code was built and then Every commit made in the source code is
tested. built and tested.
Developer have to wait for test results. Developers know the test result of every
commit made in the source code on the
run.
No feedback Feedback is present
What is Continuous Integration
● Continuous Integration is a development practice in which the developers are
required to commit changes so the source code in a shared repository several
times a day or more frequently.
● Every commit made it the repository is then built. This allow the teams to detect
the problems early.
15
Continuous Integration Tools
● Jenkins
● Buildbot
● Travis CI
● Bamboo
16
What is Jenkins
● Jenkins is an open source automation tool written in java with plugins built for
continuous integration purpose. Plugins allows integration of various DevOps
stages.
Docker
● Docker is the world's leading software container platform.
● We can use docker in
○ Design
○ Development
○ Deployment
○ Testing / Release
● Docker is present in the entire workflow, but its main use is in deployment.
17
● It works in Deployment
● Docker makes the process of application deployment very easy and efficient and
resolves a lot of issues related to deploying applications
Understand Dockers in Easy Way
●